List of Figures

(1)

Institutionen för kommunikation och information Examensarbete i datavetenskap 30hp

Avancerad nivå Vårterminen 2010

Measuring Performance

of

IT departments in Swedish municipalities

Michael Winkler

(2)

Measuring Performance

Submitted by Michael Winkler to the University of Skövde as a dissertation towards the degree of M.Sc. by examination and dissertation in the School of Humanities and Informatics.

Date

I hereby certify that all material in this dissertation which is not my own work has been identified and that no work is included for which a degree has already been conferred on me.

Signature: _______________________________________________

(3)

Measuring Performance Michael Winkler

Abstract

Every modern organization relies on information systems and has a need for measuring the performance of those systems and the IT department. It is furthermore wanted to be able to compare oneself to others. Municipalities struggle with the absence of an instrument designed for their specific needs and circumstances because most instruments are designed for profit organizations, focus heavily on technical and economical criteria and do not involve criteria that are easy to compare. In this project the success factor analysis, a performance measurement instrument, is selected, modified and finally applied in two Swedish municipalities to investigate the significance of this instrument and its ability to identify significant factors of success.

With interpreting the data gathered using on-line surveys, similarities and comparable criteria as well as eight significant factors could be identified that are generic throughout all surveys done.

Keywords: performance measurement, success factor analysis, IT department, municipality, success factors

(4)

List of Figures

7.1 Survey: Asking for performance . . . 30 8.1 Skövde: Graph showing classification for employee survey . . . 34 8.2 Skövde: Graph showing classification for IT employee survey . . . 35 8.3 Skövde: Graph showing success for every success factor of both surveys 36 8.4 Skövde: Graph showing the performance difference . . . 37 8.5 Tibro: Graph showing classification for employee survey . . . 39 8.6 Tibro: Graph showing classification for IT employee survey . . . 40 8.7 Tibro: Graph showing success for every success factor of both surveys 41 8.8 Tibro: Graph showing the performance difference . . . 42 9.1 Similarities throughout all four surveys . . . 45

(7)

List of Tables

9.1 Important factors . . . 45 13.1 Sk¨ovde: Survey’s importance and success . . . 63 13.2 Tibro: Survey’s importance and success . . . 64

(8)

1 Introduction

Information systems are used in every organization and due to the high dissemi- nation several different performance measurement methods and instruments were developed. At the beginning those performance measurements mostly focused on financial and monetary values (Buchta, et al., 2007). Over time managers and researchers realized that non-financial measures are also important and stepped away from only using monetary values (Chan, 2004). Especially for measuring the performance of an IT department it is not enough to solely rely on monetary measures. In the field of strategic information management several different method - categories and their respective tools have been developed over the years.

These available performance measurement instruments are mostly designed for profit organization and do not take the special circumstances encountered when applied in a Swedish municipality into account. This leads to the report’s aim of investigating the use of a performance measurement method and instrument for IT departments in Swedish municipalities and the search for important factors within these Swedish municipalities in general. This report includes new information for information system researchers as well as municipalities and IT managers wanting to measure their IT department and achieve knowledge about significant success factors as well as obtaining comparable criteria.

The thesis report is split into several different parts. First a theoretical background is given which is followed by the problem description and the development of the aim and research question as well as the identification of objectives relevant for completing this project. Subsequently an overview over scientific research methods is given and later on a specific research method is selected for every single objective. The first objective is to select the municipalities. This has been done with the help of Skaraborg Kommunalf¨orbund which led to the participating municipalities Sk¨ovde and Tibro. At the same time the most applicable performance measurement method and instrument was chosen (success factor analysis). Together with the municipality’s IT manager the chosen instrument has been prepared, modified and formulated. After preparing and adapting the instrument the data was collected within the two municipalities using four on-line surveys. The collected data was then evaluated and applied to the instrument’s parameters, identifying significant generic factors and proving the utility of the success factor analysis. The results have then been analyzed and the thesis closes with the conclusion chapter including suggestions for future work.

For further information the appendix includes an interview for identifying the success factors and some selected survey information and data (survey and success factors in Swedish, the comments collected and data calculated for the success factor analysis).

(9)

2 Background

This chapter offers the theoretical background to important concepts related to this project. Because the categorizations and definitions can differ depending on the author’s opinion and point of view (Heinrich and Lehner, 2005; Heinrich and Stelzer 2009; Krcmar and Schwertsik, 2009) the main reference and terminology used for this chapter and the whole project is from Heinrich and Stelzer (2009).

2.1 Information management

The model of information management (abbreviated as IM) builds the ground for developing a company wide concept to plan, oversee and steer IT infrastructure as well as it’s implementation and consists of system thinking (Heinrich and Stelzer, 2009). According to Heinrich and Stelzer (2009) IT infrastructure is a “system of information systems” used to obtain, distribute and use information and therefore enable communication. It consists of the following components:

• Computer infrastructure (hardware and software)

• Communication infrastructure (standards and devices used for digital communication)

• Data infrastructure (databases, definitions and transactions)

• Application infrastructure (Application software and systems)

• organizational infrastructure (rules for development, introduction and usage of infrastructure)

• institutional infrastructure (IT departments, Help-desk)

• personal infrastructure (knowledge and know-how) and other infrastructures (e.g. rooms, energy supply)

Heinrich and Stelzer (2009) write that the tasks of information management can be categorized into three layers: strategic, administrative and operational. If the task involves severe properties of the infrastructure it’s considered to be a strategic task (e.g. outsourcing, integration, distribution as well as properties like safety and quality). If the task involves separate components of the infrastructure (e.g. data system) it is an administrative task. Operational tasks involve service or use of components of the infrastructure (e.g. user service, problem management). Tiemeyer (2009) on the other hand only talks about strategic and operational tasks when it

(10)

comes to IT management, while Sch¨afer (2003) mentions realization goals as step between strategic goals and operational goals.

2.2 Strategic information management

The importance of formulating explicit strategic tasks is the possibility and necessity of information and communication as strategic success factor. Deriving from that emerges a system thinking for planning, overseeing and steering. This includes, according to Heinrich and Stelzer (2009), the following strategic tasks:

• Strategic situation analysis

The first task for strategic information management is to analyze the current situation to determine the role of the information function. The role can either be: support, factory, breakthrough or weapon depending on the current and future power of the information function. Building on this evaluation it is possible to further work out specific factors of success. For example the role weapon needs a high interaction between company strategy and IT strategy as well as highly educated IT personnel, while the role support has opposite demands.

• Strategic goal setting

The second task for strategic information management is, by using the outcomes of the strategic situation analysis and / or controlling information, to set the strategic goals accordingly.

• Strategy development

The third task for strategic information management is about developing an IT strategy according to the outcomes of the first two tasks to enable strategic action planning. Strategy development is the bridge between strategic goal setting and strategic action planning. The strategy can have the following characteristics: momentum, aggressive, moderate and defensive.

• Strategic action planning

Strategic action planning involves the company wide, long term approach to positively influence the competition position of the company using its provided budget. It is the fourth step for a successful strategic information management.

• Structure management

The aim of structure management is to create and coordinate tasks, compe- tences and resources of information function and infrastructure in a company.

This involves e.g.: centralization and decentralization.

• Quality management

Quality management involves everyone from top management to the operational level as well as from customers to distributors. One standard used for quality management is the ISO 9000 standard.

(11)

• Technology management

The main tasks of technology management contain keeping track of technology changes, determining the demand of technology, evaluating the deployment of technology and administer the technological inventory.

• Security management

Many companies rely on fail safe IT infrastructure. To maintain the security IT security management is needed. It is about decreasing (if possible avoiding) damage through human, natural, technical or environmental impacts.

• Catastrophe management

Catastrophe management is about preventing / surviving risks labeled as catastrophe. A catastrophe has a low occurrence probability and a high amount of damage to the IT infrastructure.

• Controlling

IT controlling delivers information as well as principles, methods and instruments for the management to plan, oversee and steer the company’s information technology. It’s developed in the strategic level and is implemented top-down through the administrative to the operational level resulting in a closed circuit.

• Revision

The core principle of IT revision is the comparison of actual- and target- values of an test item. Revision does not only have a correction-function but also a preventive-function. The difference to controlling is that controlling is embedded into work processes while revision is handled as separate process.

• Outsourcing

Outsourcing is the transition from in-house production to out-house production of IT activities and results in a lesser production depth for the company.

The intentions for this step can be economical, strategic, resource oriented and organizational benefits.

According to Heinrich and Stelzer (2009) the first four tasks mentioned (strategic situation analysis, strategic goal setting, strategy development and strategic action planning) are subtasks of the main task IT planning and are basic for a successful, conscious strategic information management.

2.3 Instruments for strategic IM

For clarifying the terms the instruments will be described using only Heinrich and Stelzer (2009) in this section. A more project related view and analysis of the instruments is given in the objective Choose measurement method (chapter 6 on page19). The review in chapter6focuses on the measurement part and applicability rather than the whole strategic approach of the respective methods and instruments while in this chapter the focus lies on explaining them in general and giving an overview and categorization.

(12)

2.3.1 Evaluation methods

Evaluating is about judging an object by a set of criteria and especially useful for projects with no routine job characteristics (Heinrich and Stelzer, 2009). Further- more Heinrich and Stelzer (2009) mention that evaluation criteria of specific objects and situations are sometimes part of action models.

2.3.2 Process models

According to Heinrich and Stelzer (2009) a process model (sometimes called action model) is, ideally, a standardized process that has been adjusted to fit the specific needs of the current task (e.g. construction of information systems) and can be part of the companies quality management. Well known action models are: Waterfall Model, Spiral Model, Extreme Programming (XP), V-Model and V-Model XT. This instrument is mostly used for architecture management, life cycle management and strategy development (Heinrich and Stelzer, 2009).

2.3.3 Profitability analysis

Heinrich and Stelzer (2009) categorize the instrument profitability analysis to be usable for measuring the formal goal cost-effectiveness / profitability of systems or system designs, products and services and development or deployment of methods and tools. In other words every kind of IT investment can be measured for profitability. Profitability is always the relationship between relative costs and outcomes.

2.3.4 Ratio systems

A ratio system uses systematic comparison of ratios for explaining specific phenom- ena and consists of planned values (desired value) and actual values for detecting possible needs of changing IT (Heinrich and Stelzer, 2009). One of the best known ratio systems is the Balanced Scorecard (BSC) and it’s modifications. Heinrich and Stelzer (2009) mention that ratio systems are mostly used for controlling, strategic goal setting and strategic action planning.

2.3.5 Success factor analysis

Success factor analysis consists of four key-areas (service, communication, human resources and repositioning) and is used for measuring the success of IT, detect strengths and weaknesses of the specific IT and to educe steps to ensure success and disposal of weaknesses (Heinrich and Stelzer, 2009). Heinrich and Stelzer (2009) identify it’s main use for strategic situation analysis, strategic goal setting and strategic action planning.

(13)

2.3.6 Value of benefit analysis

This instrument is a model for a systematic and comprehensible information gathering used for supporting decisions and is not used, like a cost-effectiveness analysis, for a decision calculation (Heinrich and Stelzer, 2009).

(14)

3 Problem

As information systems are relied upon more every day, the demand of evaluating and measuring the performance of an IT department grows. At the beginning those performance measurements mostly focused on financial and monetary values (Buchta, et al., 2007). Over time managers and researchers realized that non- financial measures are also important and stepped away from only using monetary values (Chan, 2004). Especially for measuring the performance of an IT department it is not enough to solely rely on monetary measures. In the field of strategic information management several different method - categories and their respective tools have been developed over the years. Every category has its own aim but still the categories can be overlapping (chapter Instruments for strategic IM on page4).

Most of these methods include the development of business strategies as well as the usage of control and change instruments. According to Heinrich and Stelzer (2009) the strategic information management duties are to plan, monitor and control the IT infrastructure for the organization as whole. Therefore the methods use a long term approach for being used as a management tool (Heinrich and Lehner (2005), Askim (2004)). However those methods and instruments do not initially consider the special circumstances in a non-profit and public organization like a Swedish municipality. One of the specialties about a municipality is the lack of competition.

Furthermore Swedish municipalities have to deal with constraints that affect the IT department and their possibilities severely in a way that makes many economic criteria redundant to be measured. In an IT department every decision for new software and hardware is channeled through governmental offers. Because of this a Swedish municipality can’t use additional synergy effects of buying together with another municipality. The absence of those, mostly used, economic criteria makes it difficult for IT departments of Swedish municipalities to compare each others performance and improvement opportunities.

Considering the available literature there are quite a few implementations of performance measurements in municipalities, non-profit organizations and the public sector in general. One method applied in a lot of cases (and mostly for the whole municipality) is the balanced scorecard (Chan, 2004; Askim, 2004). Chan (2004) executed a survey with municipal governments in USA (451 municipalities) and Canada (467 municipalities) about the implementation of the balanced scorecard.

In his survey (with a response rate from twenty percent) over 40 percent of the respondent administrators have heard of the balanced scorecard however only 14 municipalities (7,5 percent) implemented the balanced scorecard. Those 14 municipalities cited for example top management commitment, training and education, leadership buy-in, link of balanced scorecard to incentive and resources to implement system as factors for succeeding. All those factors need a high amount of money and / or time. The survey furthermore shows that still over 80 percent of

(15)

the respondent municipalities work with financial performance measures although the respondents administrators would like to include more non-financial measures.

Askim (2004) presents the implementation of the balanced scorecard in a Norwegian municipality over the time of five years. He focuses on the technical department, the culture and sports department, Byskogen school and the work and employment development. Information about the implementation in a service department like the IT department is not given. Foltin (1999) as well as Poister and Streib (1999) state that the implementation of a balanced scorecard system can be especially important for municipalities and their issue of concern about performance measurement and management. Again the emphasis lies on long term implementation consuming time and money to enable change and improvement in processes.

3.1 Aim and research question

There is currently no specific measurement instrument mentioned in the literature (queries have been conducted in the databases listed in chapter 4.2 on page 13) which takes the special circumstances of IT departments in Swedish municipalities (Service department of a non-profit, public sector governmental municipality) into account. However there are reports of performance measurement methods applied to municipalities in other countries but not with the focus on service departments.

Those implementations always build on a long term approach and a lot of top management involvement. This argues for the scientific importance and relevance of this project, focusing on performance measurement under the circumstances of a small amount of available time, to investigate the topic at hand. Therefore the aim for this thesis project is to investigate the usability of a measurement method and instrument for IT departments in Swedish municipalities.

The research question derived from this aim is: Can significant factors be named and discovered in a performance measurement method for an IT department of Swedish municipalities?

(16)

3.2 Objectives

To achieve the aim of investigating the usability of a measurement method and instrument for IT departments in Swedish municipalities several objectives have been identified. Those objectives include:

3.2.1 Selecting municipalities

In order to check and verify the measurement instrument a case study has to be conducted. For the case study different municipalities are being selected. In cooperation with their IT departments the instrument will be applied and used to obtain and test comparable factors. Through a cooperation with Skaraborg’s municipalities we can choose from differently sized municipalities.

3.2.2 Choosing measurement method

As mentioned earlier (chapter Background on page 2) several methods exist for strategic information engineering and measurement. Those methods can contain different instruments. For a full implementation most of these methods start with working out a business strategy as well as a strategy for the IT department and several other steps normally needed to really work with a method. Since the time alloted is short and developing strategies needs a lot of top management involvement the focus for the selected method / instrument lies on the main problem the IT departments of Swedish municipalities have, namely measuring and comparing their departments to others.

3.2.3 Preparing instrument

It is important to prepare the instrument as outlined in the literature. Further- more it is necessary to adapt the instrument to fit the specific needs of Swedish municipalities and their IT departments. One of the reasons for adapting the instrument is because they are normally designed for profit oriented organizations and not for public & non-profit municipalities. Another reason is the absence of several economic decision alternatives (chapter 7).

3.2.4 Collecting data

The instrument will be used to measure the criteria of IT departments in the selected municipalities. It is important that this data is collected carefully using scientific methods to assure the correctness. This allows discussions and conclusions about the used instrument as well as the data.

(17)

4 Method

For scientific research several methods are applicable depending on the tasks to be accomplished. In this chapter methods are first described in theory and afterwards how they can be applied to complete an objective. Furthermore arguments are given to support the decision for a certain method.

For clarification it is needed to mention the different use of terms in literature and the establishing of common terms for this project. When talking about unstructured / open methods it means, that the questions may not be given or asked in a specific order (structured / closed for the opposite). Open-ended questions are questions with no given answer (closed-ended for given answers). Those terms are in dependence on Kumar (2005) and Berndtsson, et al. (2008).

4.1 Methods in theory

This chapter gives a theoretical background and information about application re- garding relevant research methods.

4.1.1 Case study

The case study method helps to study specific criteria within a person, group, episode, process, community society or any other unit of social life (Kumar, 2005).

Furthermore Kumar (2005) recommends the use of case studies for typical cases of a certain type, thus allowing generalization and application to cases of a similar type. The advantage of this method is it’s advantage of analyzing specific details that other methods often overlook (Kumar, 2005).

4.1.2 Interview

Interviews are mostly classified by their degree of flexibility. In unstructured interviews the degree of flexibility is high because the interviewer has almost complete freedom in terms of content and structure and enables to gather in-depth information while in structured interviews where the degree of flexibility is low the comparabil- ity of the gathered data is ensured (Kumar, 2005). Kumar (2005) also mentions that unstructured interviews are conducted via face-to-face communication while structured interviews can be conducted using face-to-face, telephone or any other

(18)

electronic media. For both types of interviews open-ended and closed-ended questions can be used. Berndtsson, et al. (2008) mention several risks when using an interview as a method of gathering information. For open interviews (unstructured, issues to be covered with the interview are not planned in advance) it is important to ask in a way that enables answers to disclose issues. Closed interviews (structured) on the other hand have a fixed set of questions and have an advantage because of the repeatability they offer. Selected advantages of an interview applicable to this project are (Kumar, 2005):

• Can be used in complex situations

• Can be used for gathering in-depth information

• Information can be supplemented

• Questions can be explained

While its selected disadvantages are (Kumar, 2005):

• Time-consuming and expensive

• Quality of data depends on quality of interaction

• Quality of data depends on quality of interviewer / researcher

• Interviewer / researcher may introduce bias

4.1.3 Literature analysis

Literature analysis or literature review is an important part throughout a whole research process from the initial stages of research (for theoretical roots), the later stages of research (for enhancing and consolidating the knowledge base) to the integration of own findings into the context of existing literature (Kumar, 2005).

According to Kumar (2005) the main functions of a literature analysis or literature review are:

• Providing theoretical background

• Refining research methodology

• Enabling to show the own contribution

• Contextualizing findings

For conducting a successful literature review Kumar (2005) suggests the following four steps:

• Search for existing literature in the specific area

• Review the selected literature

• Development of theoretical framework

(19)

• Development of conceptual framework

Since the last two steps (development of theoretical and conceptual framework) are considered for a literature review (whereas in this project the literature analysis is used as one applied method out of many), they are not that important for this project and can be left aside.

4.1.4 Survey

Surveys are very useful for gathering information from a large base of respondents.

According to Berndtsson, et al. (2008) it is difficult to investigate complicated issues and impossible to clarify questions. When talking about questionnaires (a specific type of survey) Kumar (2005) states that open-ended and closed-ended questions may be used and identifies the following important information that should be included:

• Introduction to researcher and institution

• Description of the main objectives (2-3 sentences)

• Explanation of relevance

• Convey any general instructions

• Indication that participation is voluntarily

• Anonymity is assured

• Providing of contact number for any questions

• Return address and deadline

• “Thank you” for participating

Selected advantages of a questionnaire applicable to this project are (Kumar, 2005):

• Less expensive

• Offers great anonymity

• No spontaneous responses (the respondents have more time to think)

• Can be applied to large base of respondents (Berndtsson, et al., 2008) While its selected disadvantages are (Kumar, 2005):

• Low response-rate

• Self-selecting bias (someone who answers the questionnaire might have biased motivations)

• No clarification of issues

(20)

When using closed-ended questions a decision can be forced with using a four- point scale while a five-point scale offers the choice of neutrality (Goddard and Melville, 2007). Further Goddard and Melville (2007) identify several factors that can determine if a questionnaire will be successful:

• Completeness (all needed data is gathered)

• Shortness

• Includes only relevant questions

• Gives clear instructions

• Has precise, understandable and unambiguous questions

• Has objective questions that do not suggest answers

• Starts with general questions

• Has appropriate questions

• Sensitive questions are put at the end

• Mostly closed-ended questions are used (four-point scale)

4.2 Methods applied to objectives

This thesis project relies on literature analysis and case study. Building upon the theoretical background of methods (chapter 4.1 on page 10), the project objectives will be put into context with applicable methods.

Every time literature analysis as method is proposed, the following additional information is important and shall be considered. For literature analysis queries in the following databases have been conducted (in alphabetical order):

• ACM Digital Library

• EBSCO Host

• Emerald

• Google Scholar

• IEEE Xplore

• Inspec

• JStor

• ScienceDirect

• SpringerLink

(21)

• Wiley InterScience

After identifying relevant articles, forward and backward search on references and authors were performed. The queries have been done using bibliographic databases as well as journals and conference proceedings to ensure all relevant sources have been contemplated (Berndtsson, et al., 2008).

4.2.1 Selecting municipalities

In order to test and verify the usability of the measurement instrument a case study has to be conducted. For the case study different municipalities are being selected.

In cooperation with their IT departments the instrument will be applied and used to obtain comparable factors.

In order to select those municipalities there is no scientific method needed. The only constraint for a municipality is the willingness to participate in the project.

4.2.2 Choosing measurement method

As mentioned earlier (chapter Background on page 2) several methods exist for strategic information engineering and measurement. Those methods can contain different instruments. For a full implementation most of these methods start with working out a business strategy as well as a strategy for the IT department and several other steps normally needed to really implement them as a management instrument. Since the time alloted is short and developing strategies needs a lot of top management involvement the focus for the selected method / instrument lies on the main problem the IT departments of Swedish municipalities have, namely measuring and comparing their departments to others. For choosing the most suitable measurement method the following scientific research method has been found the most applicable one.

Literature analysis

There are several different methods for measuring IT departments mentioned in the literature. Furthermore there is a high number of available literature explaining, using and rating those methods. For describing the problem and giving a background about this project a literature analysis on measurement methods had to be conducted. The outcomes of this literature analysis can be used as initial input and help for this literature analysis for choosing the most suitable method.

4.2.3 Preparing instrument

It is important to choose a method and adapt one of the contained instrument.

Since the methods themselves sometimes overlap with other methods an adaption

(22)

of the selected instrument can inherit characteristics of other methods and instruments. One of the reasons for adapting the instrument is because they are normally designed for profit oriented organizations and not for public & non-profit municipalities. Another reason is the absence of several economic decision alternatives (chapter 7). Different kinds of applicable scientific research methods are listed below.

Literature analysis

Using the outcomes of the preceding literature analysis for refinement and input for ongoing literature searches enables the adaption of the selected instrument.

Interview

For adapting the instrument the involved IT managers can be interviewed using face-to-face, mail or phone communication. The IT managers of the involved municipalities most probably have some important insight to what they think is important or unnecessary for the instrument. On the basis of chapter 4.1.2 the first interview will be an unstructured one (face-to-face) with an IT-manager located near the University of Sk¨ovde to gather in-depth information and ideas in general while the subsequent interviews will be structured and can therefore be done using e-mail.

Survey

Surveys are very useful for gathering information from a large base of respondents.

At this specific time of the project we only need the IT-managers of the participating municipalities as respondents. Therefore the disadvantages mentioned in chapter 4.1.4 outweigh the advantages.

4.2.4 Collecting data

The adapted instrument will be used to measure the criteria for IT departments in the selected municipalities. It is important that this data is collected carefully using scientific methods to assure the correctness. This allows discussions and conclusions about the used instrument. For collecting the date the following scientific research methods are applicable.

Interview

For collecting information from a wide base of respondents interviews need a high amount of time. Interviewing the technicians and the users would be very time intensive. Because the time allotted is short this method can’t be used although

(23)

the gathered information could be meaningful. Thus the disadvantages mentioned in chapter4.1.2 outweigh the advantages.

Survey

A survey at this stage of the project is the best way to get a lot of generalized information from the two involved groups (technicians and users). Building upon the theory of the method (chapter 4.1.4) the questions will be mostly closed-ended and developed using different literature for questionnaires and surveys.

4.2.5 Overall approach

The overall approach for this thesis project is to carry out a case study on selected IT-departments of Swedish municipalities supported by findings through literature analysis. After selecting municipalities to work with, the base measurement method is chosen through literature analysis. This is followed with the measurement tool being adapted to fit the specific needs of a Swedish municipality using literature analysis for an initial input and refining the tool using one unstructured interview followed by structured interviews with the respective IT managers. Afterwards data is collected through two different surveys, one for the users and one for the technicians, using mostly closed-ended questions. This data is then analyzed and evaluated revealing significant factors in the performance measurement method.

(24)

5 Selecting municipalities

In order to ensure that the participating municipalities can be taken as representa- tives of different sizes, the chosen municipalities had to comply to two main aspects:

The municipality has to be Swedish and the municipalities have to have a different amount of inhabitants. These differences can help in determining if the chosen performance measurement method is applicable to Swedish municipalities in general, or if the method is depending on a municipality’s size. The main contact to those cities was established with the help of Skaraborgs Kommunalf¨orbund who sent a Swedish summary of my (English) proposal to the municipalities. Skaraborgs Kom- munalf¨orbund offered to work together, while the idea of measuring performance of IT departments was an own initiative for this thesis project.

5.1 Proposal for municipalities

The following project description and proposal has been sent to contacted municipalities:

Your contribution to this project: If you participate in this project I would like to ask you for a few different contributions like giving me some general information (size, organizational structure, what kind of cooperation there is between the partaking municipalities, the structure of the IT department, and so on) as well as some specific information from the IT department (how many help desk requests, the response time for a help desk ticket, etc.). Other data will be collected through (online-)surveys and face-to-face or phone interviews. I understand that your departments have a huge work load and therefore I will do my best to keep my time needed as low as possible.

Focus of the project: The aim of this project is to investigate the occurrence of performance indicators for comparing and measuring IT departments from alike municipalities in Sweden. In order to achieve that, several IT departments of comparable (in demands of size etc.) municipalities will be measured using scientific methods (for example balanced scorecards or other methods for performance management).

Which performance indicators to be used at the beginning will be de- termined through studying other performance management studies for municipalities and the tertiary sector in general. After measuring the different municipalities and comparing them to each other the differences and how they relate to each other will be discussed,

(25)

revealing performance indicators.

Why I would like to work with you: My interest in this topic is the ability to work with Swedish municipalities (I am an exchange student from Austria) and learn more about Sweden. Furthermore I would like to specialize in performance and quality management after my studies here in Sweden and Austria. Also this project as ground for my master thesis helps me getting closer to my goals very much.

What you can expect from this project: For the participating municipalities and their IT departments it is interesting to see where they are at when compared to other municipalities and how they can measure their performance. With the performance indicators at hand the municipalities will be able to further work with them and have a common ground if they wish to compare with the other partaking municipalities again.

5.2 Information about the participating municipalities

In this section additional information about the two participating municipalities is given. This includes the number of inhabitants, employees (both IT department and in general) as well as the location of the municipality.

5.2.1 Sk ¨ovde

Skövde municipality is located in Västra Götaland, Sweden. The number of inhabitants is 50.610 (Skövde, 2009) while the municipality has around 4500 employees and 30 IT employees.

5.2.2 Tibro

Tibro municipality is located in V¨astra G¨otaland, Sweden. The number of inhabitants is 10.600 (Tibro, 2010) while the municipality has 1053 employees and 5 IT employees.

(26)

6 Choosing measurement method

For choosing the most suitable measurement instrument and method the several instruments that have been introduced and categorized in chapter 2.3 Instruments for strategic IM (page 4) will be put into context to this project, listing arguments for approving or declining the specific instrument. The main focus of the instrument has to be the measurement of the IT departments and the applicability on Swedish municipalities.

The following characteristics are needed from the instrument:

• easy to apply

• applicable in a short time

• repeatable

• easy to modify

• easy to compare outcomes with other municipalities

• not focusing solely on economical or technical criteria

These characteristics are critical and have to be fulfilled by the instrument as much as possible.

6.1 Recap of instruments and arguments

In this section the introduced instruments from chapter2.3Instruments for strategic IM will be discussed in a listing.

• Evaluation methods

This instrument is mostly used for non routine jobs where judging of specific criteria is needed (Bludau, et al. 2003; Carlsson, 2003; Heinrich and Stelzer, 2009) which makes it usable for this project. However the development of an evaluation model for the IT departments of Swedish municipalities could turn out to be focused too much on technical implications due to the usage of benchmarks, experiments and other methods (Heinrich and Stelzer, 2009).

(27)

• Process models

This instrument is mostly used for software development and its parts for measurement and quality management are not prioritized (Acu˜na, 2005; Heinrich and Stelzer, 2009). Sommerville (2007) emphasizes the abstract presentation of a software process within the process models and their different perspec- tives. Since the IT departments of Swedish municipalities are mostly service departments not developing own software this kind of instrument is not suitable for this project and its purpose. Furthermore this instrument does not offer as many comparable measurements as other instruments.

• Profitability analysis

The measurement profitability is used for evaluating investments (Drury, 2008;

Heinrich and Stelzer, 2009) and is therefore, due to it’s decision calculation character, not suitable for this project. Tiemeyer (2009) enforces its economical focus and allocates the instrument to the activities of controlling. Due to its heavy focus on economical factors it is not useful for generating meaningful comparison measurements for this project purpose.

• Ratio systems

Ratio systems like Balanced Scorecard (BSC), IT-Balanced Scorecard (IT- BSC) and Work System Method always rely heavily on economic ratios be- sides non-economic ratios like customer satisfaction, internal processes and innovation (Van Grembergen, 2000). For designing a BSC it is important to first identify strategic goals. After identifying those goals the scorecard measures can be developed. If there is no agreement upon the strategic goals the scorecard measures can’t be developed (Chan, 2004). Further a BSC always involves a long-term approach and has to be deeply positioned within the company (Foltin, 1999; Poister and Streib, 1999; Askim, 2004). Those characteristics as well as the categorization into the IT controlling section speak against the use of a ratio system.

• Success factor analysis

Since the success factor analysis explicitly asks for developing company and department specific success factors (Heinrich and Stelzer, 2009) it suits the approach this project has very well. It is used for measuring the success of IT, detect strengths and weaknesses of the specific IT and to educe steps to ensure success and disposal of weaknesses (Heinrich and Stelzer, 2009) which makes it fit well for usage in a service department like the IT departments of public and non-profit organizations (in this specific case Swedish municipalities). It furthermore can be applied easily and in a short time as well as enabling the comparison of the performance of these factors (Kang and Bradley, 2002) between municipalities. Tiemeyer (2009) enforces the usability of this instrument especially for gathering information for strategic planning and action taking.

• Value of benefit analysis / benefit analysis

This instrument is used for supporting decisions for investment in hardware and software (Wall, 1999; Drury, 2008; Heinrich and Stelzer, 2009) and does furthermore not deliver as useful outcomes as the other instruments evaluated for this project. The focus here is mostly on economical criteria and is therefore

(28)

not of use for this project.

Swedish municipalities have the need for an easy applicable, changeable and comparable instrument for measuring their IT departments. Those requisites are met when using the success factor analysis and therefore it is the chosen instrument for this project. For identifying the success factors for the Swedish municipalities the factors used for ratio systems will also be considered.

6.2 Success factor analysis

This section explains the success factor analysis and how to apply it in detail followed by related work and the conclusions and comments of the respective authors.

6.2.1 Detailed instruction

This instrument is part of a method developed by Alloway for strategical IT-planning in the mid eighties and is used for measuring the success of IT, detect strengths and weaknesses of the specific IT and to educe steps to ensure success and disposal of weaknesses (Heinrich and Stelzer, 2009). It builds upon empirical studies conducted by Rockart. Rockart (1982) carried out interviews in nine companies trying to identify factors critical to the success of these companies. He (Rockart, 1982) comes to the following major conclusion: success factors differ from company to company but they can be summarized into four different categorizations. These four categorizations are as follows (Rockart, 1982):

• Service

This category refers not only to performance (efficiency and effectiveness) like on-time and response-times but also on how users perceive the services offered by the IT department.

• Communication

Communication concerns both directions of communication. One communication is from the IT department to the line and the other is from the IT department to the users. This category ensures that the IT department is not only technically oriented but oriented towards active communication and leadership.

• Human Resources

An IT department is a people-intensive field of work and quality of personnel and understanding of the users is a key to success. The quality of personnel refers to professional competence as well as social competence of employees of the IT department.

• Repositioning

This category is about comprehensive characteristics of IT (e.g. degree of out-

(29)

sourcing) and can be put into the four components technical, organizational, psychological and “managerial”.

In an early working paper Alloway (1980) already stated that a list of success factors has the tendency to include too many factors and therefore becomes hard to handle.

He suggests that, relying on the partitioning of strategic, managerial and operational activities developed by Anthony (1965), the list of success factors be shortened to include only 26 success factors. After developing those success factors they have to be judged by their performance and importance allowing the interpretation of the collected data.

The steps to take for working with a success factor analysis are as follows (Heinrich and Stelzer, 2009):

• Identification of success factors

Chapter 7.1 Identification of success factors on page 25. The chosen success factors should describe the characteristics of success for the particular organization and its IT. Gratuitous characteristics should not be represented in success factors. For describing the success factors it is important to use expla- nations understandable for everyone. Furthermore the success factors should not be more than 26.

• Choose participants for survey

Chapter 7.2Choose participants for survey on page 28. It is preferable to do a census with all IT employees and users. If the organization is too big a control sample with a maximum of 200 people has proven to be sufficient.

• Formulation of the survey

Chapter7.3Formulation of the survey on page28. The survey itself consists of the following three parts: importance, performance and overall performance.

• Collection and display of data

Chapter 8 Collect data on page 31. For collecting the data the participants need to know the purpose of the survey while a specific training is not needed.

After collecting the data it is needed to display the data using the following formulas (success (S), success factor (F = A...Z), importance (I), performance (P), participants (T = 1...t) and performance difference (D)):

Calculating the success S(F) for a specific factor F and all participants T:

S(F ) =

t

X

T =1

(I(F, T ) × P (F, T ))

t

X

T =1

I(F, T )

(30)

Calculating the success S(T) for a specific participant T and all success factors:

S(T ) =

Z

X

F =A

(I(F, T ) × P (F, T ))

Z

X

F =A

I(F, T )

The calculated S(T) should correlate with the performance rating given by the participant for the general performance. Is there no correlation and both values differ too much from each other, the particular participant is to be left out of the collected data.

Calculating the performance difference D(F) for a specific success factor F and all participants T (average rating for importance minus average rating for performance of success factor F):

D(F ) = 1 t

t

X

T =1

I(F, T ) − 1 t

t

X

T =1

P (F, T )

If the calculated performance difference D(F) is negative it is wisely to disin- vest (the higher the negative value the more urgent it is), if it is positive on the other hand it is wisely to invest more money to that particular success factor (the higher the value the higher the urgency for investment).

Furthermore it can be advisable to calculate statistical variables (e.g. standard deviation) to assure the data is correct and significant. The use of a spreadsheet program for calculating all variables is recommended.

• Interpretation of survey results

Chapter 8 Collect data on page 31 and chapter 9 Analysis on page 43. The interpretation of the collected data is supported by graphs and portfolios.

6.2.2 Related work and conclusions

In an information technology diagnosis done in a company Heinrich, et al. (1997) used the success factor analysis to gain insights about the user satisfaction. They concluded that the high degree of involvement (through participation of employees) can lead to a high acceptance for the results, making it easier to change strategic ways of action. Another finding was the ability of the instrument to measure standard values (importance) as well as the current values (performance). The only risk they mention when using the success factor analysis is the fact that it is not an objective measuring instrument but solely relies on subjective judgment of the participants.

For an easier understanding and better visual presentation, the success graph (vi- sualized using a profile diagram) used by Heinrich and Pomberger (2001) in their diagnosis done for a textile company, has shown to be more convenient than the original graph used by Alloway (1980). Heinrich and Pomberger (2001) further split

(31)

the participants for their diagnosis into smaller groups (five groups with together 250 participants, covering about 50% of all employees).

Rockart (1982) used success factors and the categorization (service, communication, human resources and repositioning) for interviews in nine companies (all from different domains like railroad, bank, manufacturing, airline and insurance) and reasons that, although success factors differ between companies, each industry still has a generic set of success factors.

In the related work mentioned the amount of participants of surveys and interviews differs between nine to 250 people, depending on the aim of the case study and the available possible participants.

(32)

7 Preparing instrument

In this chapter the approach taken for the success factor analysis will be introduced and explained. After identifying the success factors using literature and an extensive interview with the IT manager of Sk¨ovde the participants for the survey are chosen and afterwards the survey is formulated.

7.1 Identification of success factors

Using literature from Heinrich and Stelzer (2009), Heinrich and Lehner (2005), Alloway (1980), Alter (2006), Buchta and Schulte-Croonenberg (2007), Kang and Bradley (2002), Krcmar and Schwertsik (2009), Rockart (1982) as well as Van Grem- bergen (2000) several ideas and propositions for possible success factors were collected. In an interview with Fredrik Edholm, the IT manager from Sk¨ovde Kommun, the most critical and important success factors were discussed and identified (see chapter 13.1 Appendix: Interview with Sk¨ovde’s IT manager on page 53 for a summary of the interview and additional information). For the interview itself a list of several success factors was prepared using the mentioned literature. After discussing this initial list more success factors were presented and discussed which led to the naming and allocation of letters to the success factors in chronological order of their approval.

Service:

A Availability

This factor is a measure for the availability of equipment. Examples are hardware breakdown, software crash and network failure.

B Individual IT

This is about the possibility and support to use individual software and hardware (e.g. specific software for a user, supply of a notebook if needed).

C Response availability

Response availability measures the temporal aspect of responses given by the IT department (e.g. response time for the help desk).

D Usability

This factor is about simplicity of use, easy learnability and confident handling of information systems (e.g. are documentations easy to understand, is the graphical user interface clearly arranged).

List of Figures

Contents

List of Figures

List of Tables

1 Introduction

2 Background

3 Problem

4 Method

5 Selecting municipalities

6 Choosing measurement method

7 Preparing instrument