Självständigt arbete på avancerad nivå

(1)

1

Självständigt arbete på avancerad nivå

Independent degree project – second cycle

Master of Science in Engineering

Industrial Engineering and Management

The Subjective Perception of Risks by individual key stakeholders within SCRUM

A qualitative case study at an IT firm

(2)

MID SWEDEN UNIVERSITY

Department of Information Systems and Technology (IST) Examiner: Leif Olsson, leif.olsson@miun.se

Supervisor: Olof Nilsson, olof.nilsson@miun.se

Authors: Aydin Sergen Yesilkayali (seye1300@student.miun.se, Nils Patrik Sidén Eriksson (paer1202@student.miun.se)

Degree programme: Master of Science in Engineering: Industrial Engineering and Management, 300 ECTS

(3)

Abstract

The lack of knowledge regarding the perception of risks in agile project management is evident in research. With this knowledge, this study sets out to address this issue and close the gap in knowledge regarding this issue by identifying the individual perception of risk by a project leader, SCRUM master, and two system developers, and comparing the individual appetite matrices to a unified appetite matrix created by a Delphi panel. A case study at an IT consulting firm, working with several projects in parallel, in Sweden is conducted. A risk register is used to collect the data, with the principle of assisting in creation of the risk appetite matrices. The result shows that the individual risk appetites differ significantly from the group’s unified risk appetite. The group showed itself far more risk aggressive than the individual appetite which was risk aversive in relation to the group’s appetite. The lack of RM practices is evident in smaller IT firms such as these, as there are no clear standardized approaches to the mapping or classification of risks; a far more informal approach is taken.

(4)

2 Sammanfattning

(5)

3 Acknowledgements

(6)

1 Introduction

Complexity in software development has been an issue that most IT organizations have been struggling with for over 60 years. Common complexities in software development projects are e.g. delays within the partial and end deliveries of the project and misleading project goals. This also implies increasing costs and lead times within the project – in worst case even project cancellations. The usage of agile software development has been increasing frequently since the establishing of the Agile Manifesto, 2001. The nowadays called mainstream development methodology has been approved by large software providers as Microsoft, Adobe et al. (Schmidt, 2016)

The rate of success in software development projects is still an issue to deal with. Recent year’s rate of successful projects is still remarkable low, which implies a need for further research regarding potential issues. (Haas, 2007; Hastie, 2015).

Low rate of success in traditional projects has led to increasing popularity of the agile management project approach. Software development is built upon the adeptness of processes which are both unpredictable and complex, implying a low possibility of planning. Within the organizational management, concerning about managing risks within projects affects the actual insight of the organizations environment. In the organizations environmental basis, most difficultness lies within the management strategies of handling risks. These strategies are priority and resources which are directly related to the handling of the risks. (Ribeiro, 2008)

Identifying and handling these risks in agile projects may be vital to the success rate and reduce the cost in further projects. This can be done by applying risk management analysis upon a case study that makes use methodologies of agile software development.

(9)

3

1.1 Background and problem motivation

Risks in relation to agile development methods are often neglected or rarely intricately focused upon. Available literature in, e.g. Scrum, does not delve in detail into the risks that may be apparent in a project by an agile development team. (Smith & Pichler, 2018)

One parameter that is inherently fundamental in every organization is resource saving. These “resources” can be defined as liquidity. Every risk is likely to be tied to monetary value, e.g. risk that implies a time loss will eventually imply an increased cost of the project. A risk analysis is done in order to raise risk awareness and prepare for potential course of actions to take in order to either reduce the probability of the risk occurring or to mitigate it after its occurrence. (Hopkin, 2010)

There is disagreement among agile project owners whether explicit methods of risk management should be applied within agile projects (Walczak & Kuchta, 2013). To put the problem of IT projects and risks in context of project failure; organizations in the United States spends $250 billion a year on IT projects, distributed among 175 000 projects. 31.1% of these projects are cancelled before completion. 52.7% of the projects reach costs of over 189% of their original estimates. The success rate is 16.2% (i.e. completed on- time and on-budget) (Standish Group, 2014).

(10)

4

The main study approach is to look into how risk management can be applied within agile maintenance projects, and to give insight into how this can be done efficiently without compromising the efficient work ethic that is inherent within the agile approach. The main aim of agile methods is to be fluid and flexible without compromising on time or cost. Risk management adds another dimension to these two variables, which makes some agile stakeholders question the necessity of applying a risk methodology at all. By conducting a case study at an IT consulting firm with an ongoing maintenance project (conducted with an agile Scrum approach), it is of interest to expand the knowledge and provide new insight into the compatibility of risk management and agile maintenance projects.

1.2 Purpose of the study

This thesis aims to study what the general risks are within agile maintenance projects, and to see if and in which extent risk management methodology is compatible with such projects. Additionally, it is of interest to study how the subjective perception of risk is classified by the stakeholders within projects.

The research questions are formulated in order to answer the aim of the study:

• What are the general risks within agile maintenance projects? • In which ways is the used risk management methodology in this

study suitable respectively unsuitable for agile maintenance projects?

• How does the perception of the general risks within agile projects differ between key individuals?

The results of this study intend to add to the knowledge of the differing perception of risks by different relevant stakeholders within agile work methodologies. This study intends to add a wider awareness to if and how this differing perception may look like and why.

(11)

5

1.3 Delimitations of the study

This thesis will only dwell into the ongoing maintenance and development project ongoing at a consulting firm where the case study is done. The limitations of the research will be done in accordance to general risks, and inherent risks in agile software development maintenance projects. While this study is done relative to one case, the conclusions and discussions will nevertheless be general and applicable to other agile maintenance and development projects in general, and for projects where differing perception of risks by important stakeholders might cause issues or loss of trust for the project.

This study does not intend to look at intra-organizational risks, but rather at specific general and common risks found within agile maintenance and development projects.

Note that the case studied project is ongoing since 2014, and is

continuous until the firm’s customer decides to contract another firm

1.4 Overview

Chapter 2 describes the theoretical reference frame which the study is based on.

Chapter 3 describes the methodology and its components used in the study.

Chapter 4 describes the construction and alternative solutions of the study.

Chapter 5 describes the results of the study.

(12)

6

1.5 Th

e authors’ contributions

(13)

7

2 Theoretical background

Today’s organizations face efficiency goals mainly consisting of cost and resource efficiency. Close relationships between stakeholders and

customers have to be ensured through identification and tracking of those. Due to these restricted environments management of uncertainty has to work properly. Also, customers have to be satisfied with fewer resources. (Augustine, 2008)

2.1 Agile methodologies

The fundamentals of agile methodologies are based on the terms of agility. These implies delivering value to the customer while handling unpredictability and dynamisms within projects in the form of

recognizing and being adaptive to change. Direct related to agility is also to sustain flexibility and stability, dealing with chaos, performance planning and explorative optimization. Ability to adjust the speed of value added deliveries to the customer with respect to changes and uncertainty. Techniques related to agile methods are divided into partial workflow blocks to get feedback from customers and end users in an early stage to provide managing of complexities. Sprints and releases are commonly finished within a period of one-three months. The processes and tests are iterative and incremental carried out to incubate requirements, plans, code and design. Iterations are set up with a fixed length of usually two weeks. This is due to ensure information is achieved through feedback and maintain stability. Collocation in open work cell is performed to permeate the whole team with included onsite custom to facilitate face to face communication and sufficient

interactions between team members. (Augustine, 2008)

(14)

8

the release plan such as features of high level which are prioritized along and elaborated upon within tasks of implementation in the desired plan backlog. Developers set up prioritization through collaborative during an iteration planning game. Effort estimates are soon provided by the developers and business priority is set by the customer.

(Augustine, 2008)

Team members are self-organizing through collaborative, continuous finishing of tasks within the backlogs excluding a top to down control management. While iteration is active, tracing of tasks and features is carried out. Tracing is active until 100 percent is finished and there is no room for partial completions. All work related to Agile methodologies has to be of simple, lean character and adaptable to maintain value added processes to customer and adapting to change. (Augustine, 2008)

2.1.1 Agile software development

Since the early 2000’s Agile Software Development has seen major changes in relation to software development. This is a result of new ideas among developers through the global community which has been shared through the World Wide Web. This has also meant that the requirement for the ability to change and adapt to the customers wishes has changed. As the global market has emerged from the birth of

internet, the global community has given rise to more thoughts of abstract and different character; one project can during its lifetime require several different requirements and customer wishes to the functions that the end-product will contain. This means that the

functionality of the product must be flexible but also strictly follow the criteria set by the customer’s order. (Schmidt, 2016)

(15)

9

As collection concepts “agile methods” is formed by software

engineering methods introduced in the beginning of 2000’s and the core values within the engineering of ASD. As the agile method developed it can be derived that two integrated methods has become the most

popular ones: SCRUM and eXtreme Programming. (Maruping et al, 2009).

2.1.2 SCRUM

Scrum is an agile project management methodology produced as a framework for product development with adaptiveness to different complex projects. The framework structure is shown in figure 1

consisting the fundamentals of SCRUM and its contents including roles, artifacts and events. (Schwaber & Sutherland, 2013; Cervone, 2011; Misra et al, 2009)

Figure 1: The Agile SCRUM process. (Agileforall, 2015)

(16)

10

A scrum project is introduced with a kick-off, following with a sprint planning consisting a deposited time scale per sprint and month up to eight hours. Sprint planning includes the estimation of work duration and time needed to complete the highest prioritized items within the Product backlog by dint of planning poker. After estimation is done, the development team will select items to be done within the sprint,

initiated with the most prioritized; selecting items will be done from a list named Sprint backlog. Inherently the development team will iteratively query themselves how necessary the work is to reach the goal. This is done every time when items within the sprint backlog are considered, thus the goal must be defined. As soon as the sprint

planning is completed, the development team will start working on items. A sprint is defined to have a fixed time scale where the goal is to complete involved items within the deposited time. No changes will be made to the items within an active sprint except in special cases if the extent or value is changed. Items within the Sprint which are not finished will be restarted and replaced into the product backlog and taken care of in further coming sprint planning. (Schwaber &

Sutherland, 2013; Cervone, 2011; Misra et al, 2009)

Inherent to the sprint, daily scrum meetings consisting of 15 minutes per session are being performed every day. As prerequisite for the meetings is that all individuals attend at the meeting and being well prepared, answering three questions; What have I done since last meeting; What to do until next meeting; What obstacles prevents me to reach the goal? Responsible for the meetings is the Scrum Master which intends to that all three questions being answered and that all members are attended. After completing each sprint a sprint review is performed with a time scale of four hours per month and a sprint to review the performed sprint. (Schwaber & Sutherland, 2013) The latter presented by the framework is the Sprint retrospective meeting where used techniques, processes and all involved parties are being evaluated with a critical vision. This generates answers to how the work was done within the performed sprint in order to find improvements and suggestions to further sprints.

(17)

11

2.1.3 Phases within agile projects start-end

Agile software development and its inherent project phases build upon the agile project management. Within the agile project team, the

assessment of possibilities and the vital project phases are crucial to the value creating processes. Availability to be included or excluded in the team will permeate composting in the Organic Team implying

adaptability to changes in surrounding conditions. Due to construction of small teams, communication flows will function properly and

sustaining a low interaction penalty. In case of larger scale where larger teams are needed, the project will be organized into smaller organic teams which performing work in parallel to each other. (Augustine, 2008)

Adaptation and anticipation is directly related as a mechanism to the mental models of the team members involved. A shared mental model has the highest magnitude of the member’s behaviors’ when a project statement purpose is translated in accordance to the vision of the project. These principles build upon the U.S army’ commanders’ intent and the Army’s awareness of that the leader can’t be omnipresent. The manager of an agile project relates to these principles by constantly guidance of the team and supporting the team to consistently make right choices. A “good enough” vision is created by input change and assumptions instead of time consuming detailed plans. This implies that focus may be directed and applied to desirable outcomes and tasks associated to the plan will appear during time plan. Within

methodologies, rules appear as a constant underlying supporting process. It’s common that these rules being seen as burdensome and as an impact not being followed throughout the workflow. The

fundamentals of APM (Agile Project Management) are constructed so that the team members follow simple rules, over time complex

behaviors’ occurs within these. Information flow is the causal agent for enabling adaptation and change within agile projects. Most exchange of information occurs when members interact with each other and it appears that openness has the most impact of how rich the information is. (Augustine, 2008)

(18)

12

Adaptability of an agile team is directly related to dependency of a well-functioning information flow. To prevent disruptions within the

information flow caused by organization bottlenecks, these obstacles has to be constantly identified and removed. Due to inherent uncertainties in the world and its circumstances, control and stability is hard to achieve. Managers has nowadays realized and accepted how unrealistic it is to foreseen knowledge about future outcomes. This has resulted in a control methodology called “Light touch” which is built on the terms that increased control does not imply less uncertainty and higher

income. This method is moreover used in the APM where requisition of some control is made which leads to greater income. (Augustine, 2008)

Creativity and work that meets agile principles occurs when the very sensitive balance between interest and structure is reached. Optimal balance occurs when work is satisfyingly interesting due to non-predictableness and enough structured to not collapse. Principles of APM ensuring optimal workflow and outcome when managing a project as following: Guiding Vision; Organic Teams; Simple Rules; Open Information; Light Touch. Furthermore it’s not excluding the risk of teams working in wrong direction when constructing new ways of team interactions. Unintended outcomes are possibly to result in both positive and negative consequences due to non-linear behaviors’. Adaptive processes to leadership are carried out to imbue desired results through observations and assessment of practices. To enable the imputation of the results it requires the project manager to have an overseeing understanding of the entire project and its mutual interactions and being susceptible for learning and adaptability. (Augustine, 2008)

(19)

13

2.2 Risk management

2.2.1 Risk framework

There are many risk management standards to choose from. These standards illustrate and explain the process of managing risks. It is important to note that the authors of this study use the word risk

management as an umbrella term; it contains the identification,

assessment, and classification of risks. These frameworks take different approaches. Some are suitable for certain things; others are more

applicable in other general areas. (Hopkin, 2010)

There are several risk management standards, one of which are ISO 31000.

Figure 2: The process of risk management according to ISO 31000. (Hopkin, 2010)

(20)

14

The process begins with establishing the context of the risk process. This continues with the risk assessment part of the process, which contains identifying, analyzing and evaluating the risk. The next part is

generating recommendations on risk treatment. In parallel with these processes, there is communication and consultation with relevant stakeholders that are experts in the context (i.e. the process at risk). All the same while monitoring and reviewing every step of the way. (ISO, 2009)

The principles and framework of the ISO 31000 standard is illustrated below.

Figure 3: A more in-depth illustration of the ISO 31000 standard. (ISO, 2009)

(21)

15

2.2.1.1 Project risk management

There is no clear approach to risk management within agile projects. Many project stakeholders feel that working purely with an agile method is sufficient when it comes to handling risks. (Moran, 2012)

Some experts are of the opinion that a more individual approach to risk management is suitable in agile projects. For example, an agile project risk management methodology would allocate the ownership of risks to individuals, rather than having one risk manager who handles all of the risks accordingly. The traditional form of risk manager becomes to facilitate attention to risk management and work together with

individual risk owners in order to manage risks. This risk management role can be assumed by, e.g., the Scrum Master or the Project

Leader/Manager. (Moran, 2012)

Risk management applied to traditional projects often look at risks upfront and identifies issues that may affect the project as a whole. As traditional projects are very rigid (function is fixed, while cost and time are variables), this approach towards risk management suffices. This is not the case for agile projects. Agile, as previously mentioned, considers time and cost to be fixed, while the function (of the product) is a

variable. This makes the traditional risk management less suitable for agile projects. To solve this, agile risk management needs to look at potential risks throughout the lifecycle. Although this may seem like a major difference, it does not make general risk management less suitable to be used in agile projects. It is important to note that even though many projects state that they are purely Agile, this is rarely the case. There is no consensus regarding risk management in relation to agile methodologies. (Moran, 2012)

(22)

16

Project risks have three fundamental risks in them, all three tied to the failure of keeping or achieving the project parameters, i.e.: staying within cost or completion date parameter, or to achieve quality and/or functionality requirements. These risks are, suggested, to be influenced by several different associations, such as: the size and scope of the project, technological know-how of team members and the complexity of the projects structure. (Merna, 2010)

These general risks are portrayed in the table 1 below.

Table 1: General project risks. (Merna, 2010) Too many project members

Complex administration Complex management

Lack of communication amongst project participants Inaccurate forecasts

Late deliveries

Equipment/technology issues and break-down

2.2.2 Risk identification

Every aspect in an organization contains some proportion of risk. Independent of the process, risks are inherent and should be analyzed and classified in relation to cost and benefit. It is of interest when to balance risk management and when not to. Managing too much will cost a lot, managing too little will cost a lot. (Newton, 2015)

ISO 31000 defines “risk” as the “effect of uncertainty in objectives”. Risk management (RM) is the “coordination activity of directing and

controlling an organization with regard to risk”. There are many

different definitions, but the fundamental essence of every definition is more or less the same. (ISO, 2009)

(23)

17

Risks are often categorized and classified with the help of RM. This is done in order to handle these risks and work with them. These risks can be classified into three different types (Hopkin, 2010):

• Hazard risks • Control risks • Opportunity risks

Hazard risks are risks that can only have a negative consequence. Risks

such as these are considered insurable risks, i.e. the only way to avoid these risks is by applying mechanisms that, if the risk is apparent, there will be a buffer in order to minimize the impact of consequence.

Risk hazards are closely related to issues within health and safety work,

damage to property, and consequence of defective products. This category of risk is disruptive to the workings of an organization. Examples in relation to IT, these risks can be hacking attempts, destruction and interruption of software development, or malicious software. (Hopkin, 2010)

Another type of risk is Control risk. These are the risks which are apparent when the outcome of the situation has given rise to

uncertainty. These are often associated with the project methodology.

Control risks are risks that may affect the fixed parameters of a process.

Examples of these fixed parameters (which are often tied to resources) are time and cost. With respect to this uncertainty, a risk manager should strive to control these risk rather than insure against them (too costly), or avoid them completely (infeasible). (Hopkin, 2010)

The third category of risks is the risks that organizations strive to take because of their positive outcome. These are the Opportunity risks. The category of Opportunity risk is defined of having an expected

positive outcome (in a more or less state of degree). Examples of this risk could be the purchase of new software for a more effective value

(24)

18

How the categorization of these risks occurs, depends on the risk appetite of the organization.

These three risk categories can be broken down into two different subtypes. The authors of this paper have decided to categorize these risks into the two following sub-types

• Inherent risks • Artificial risks

Inherent (fundamental) risks are risks that are inherent in every aspect of a

specific process. These risks are equal in equal processes.

Artificial (overall) risks are risks that are artificially perceived, i.e. risks

that are not inherent in a process but rather are a result of actions by stakeholders. These risks are unique for equal processes.

If we are to properly work with these risks, we need to collect the correct information in order to be able to categorize these risks into one of three possible risk categories. This is done by describing these risks in a Risk

description. (Hopkin, 2010)

Table 2: Table of risk description (Hopkin, 2010).  Name of risk

 Statement of risk (scope, details of possible events and relations)

 Details of risk classification and potential impact  Internal and external stakeholders

 Risk tolerance for the risks  Consequence of risks

(25)

19

2.2.2.1 Risk assessment and techniques

In order to classify risks, they first need to be assessed, categorized and analyzed. There are several different methodologies and models that can be used - either individually or in tandem. We will present available models that are available in order to document recorded risks from gathered data. There are several techniques in place in order to collect data regarding risks. Respective data collection methodology has pros and cons, related to their techniques. The study will convey the most relevant data collection methods in relation to risks in this sub-chapter.

2.2.2.2 Questionnaires and checklists

This risk data collection methodology uses structured questionnaires and checklists in order to collect data regarding risks. This method helps the risk manager identify existing risks. (Merna, 2010) The risk manager can identify hazards, risks or control failures, with the help of data collected from relevant stakeholders. This method is at its strongest when applied as a data check (i.e. combined with another data collection method) in order to confirm collected data. (Valis & Koucky, 2009)

2.2.2.3 Workshops and brainstorming

Workshops and brainstorming combined consists of a free-flowing conversation among peers with knowledge regarding the risk

containing process. These peers discuss and identify potential problem areas that are associated with risks. Criteria regarding probabilities, consequences, and options for diminishing (or completely removing) likelihood or impact, are generated. (Valis & Koucky, 2009; Hopkin, 2010; Merna, 2010)

(26)

20

2.2.3 Risk analysis

Table 3: Qualitative and quantitative character of different risk data collection methodologies.

Qualitative Quantitative

Questionnaires and checklists

Workshops and brainstorming

Flowcharts and dependency analysis

Delphi technique

SWOT and PESTLE analysis

Monte Carlo simulations

Markov analysis

HAZOP and FMEA

Delphi technique

2.2.3.1 Delphi technique

The Delphi technique consists of brainstorming or interviewing relevant stakeholders (in this case, experts). This is especially useful in order to obtain intricate information regarding risks, such as in cases where one desires to reach a group consensus on risk classification. This procedure is done by obtaining a reliable consensus from stakeholders who are very invested and knowledgeable within the area of interested. This technique is used both in quantitative cases and qualitative cases; in order to generate actual tangible risks, stakeholders are interviewed - in the same way, in order to generate likelihoods (i.e. probabilities of a certain risk occurring), stakeholders (experts) are interviewed, and hence these numerical probabilities are generated.

(27)

21

2.2.3.2 Flowcharts and dependency analysis

This methodology makes use of flowcharts in order to identify

dependencies between sub-systems. With the help of this analysis, the risk manager can identify which areas are interdependent. (Hopkin, 2010)

2.2.3.3 HAZOP and FMEA approaches

HAZOP (Hazard and Operability) and FMEA (Failure Modes and Effects Analysis) is a combined approach that is very similar to each other. These methodologies identify how processes can cause failure in their designed system. It is of interest to identify potential failures, how these failures may affect the whole process (or system), the mechanism of failure, and how to avoid or mitigate the negative effects of a failure. (Valis & Koucky, 2009; Hopkin, 2010; Merna, 2010)

2.2.3.4 Monte-Carlo simulation

A Monte-Carlo simulation consists of randomly generating numbers within and interval and entering it into a defined function. The results of the function is summed up, thereon taking the mean of the total sum and extrapolating the most likely probability of a certain event happening. It is of quantitative character. (Merna, 2010)

2.2.3.5 Risk register

The purpose of a risk register is to record and document significant risks that have been identified. It serves as a record in order to control

(28)

22

Illustration 1: Illustration of a risk register. (Merna, 2010)

This risk register contains a unique ID for a specific risk, its risk author, the registered date, which organizational category the risk may affect, the description of the risk, its probability and impact, response category, its current binary status (active/inactive), and relevant stakeholders. (ISO, 2015; Schwalbe, 2009; Merna, 2010)

2.2.3.6 Risk score card

Risk score cards are done in order to summarize the main risks within a process. Most score cards have the same main parameters, while the secondary parameters are different. The vertical matrix consists of the main parameters, which is the same for every different framework. The horizontal matrix contains the sub-parameters, which is different depending upon the selected framework. There are several different frameworks. (Hopkin, 2010):

2.2.3.10 Risk matrix

A risk matrix consists of a two dimensional diagram where the x-axis denotes likelihood of risk, and the y-axis consisting of the magnitude of impact of said risk. The risks are plotted on the diagram accordingly to its x- and y-variables. Figure 4 below denotes an exemplified illustration of this.

(29)

23

likeliness of a certain risk to occur, while the impact denotes the impact to the organization (or project) - i.e. not the magnitude itself. (Merna, 2010; Hopkin, 2010; Wendestam, 2008)

Figure 4: Example of risk matrix.

With the help of this risk matrix in figure 4, we can categorically decide upon a suitable action to take in order to mitigate said risk. This

categorization is explained and illustrated in the next sub-chapter: Risk appetite.

2.2.3.12 Risk appetite

(30)

24

objectives”. Her Majesty’s Treasury (governmental body in the UK) recommends having a clear risk appetite laid out in order to sufficiently assist managers in taking the correct risk decisions, which will improve the organizations’ performance. (HM Treasury, 2006)

The illustrative matrix contains three different colors denoting the risk zone; green being comfortable, yellow being cautious, and red being concerned. The wording on the zones is decided upon by the authors of this paper. These three zones differ between individuals because of the difference in perception of the same risks. Figure 5 displays how such a risk appetite matrix might be portrayed.

Figure 5: Example of a risk appetite matrix (low risk appetite) (RIMS, 2012)

(31)

25

Figure 6: Example of a risk appetite matrix (high risk appetite) (RIMS, 2012)

2.2.4 Risk evaluation

2.2.4.1 SWOT and PESTLE analysis

SWOT (Strength, Weakness, Opportunity and Threat) and PESTLE (Political, Economic, Social, Technological, Legal, and Environmental) analysis are very similar in their approach. This methodology looks at specific categories and analyses information through a categorical approach. It is argued that respective method contains vital categories that the relevant stakeholders must have in mind when working with risks. (Hopkin, 2010)

2.2.4.2 BS 31100

British Standard 31100 looks at five different sub-parameters, these being: Strategic, Programmer, Project, Financial, and Operational. In contrast to SWOT/PESTLE, the sub-parameters are different, therefore suitable for different research areas. (Hopkin, 2010)

(32)

26

Table 4: Example of a risk score card for British Standard 31100. (Hopkin, 2010)

Strategic Project Financial Operational

Description Internal/external risk Quantifiable Measurement (performance indicator) Performance gap Control mechanisms

2.2.4.3 4T Risk mitigation

4T refers to the four different courses of actions that can be taken in order to mitigate risks “Transfer, Terminate, Tolerate and Treat”. (Hopkin, 2010) The categorization is done by first identifying, followed by classifying the risks. After plotting it on a diagram (such as the one found in figure 4), we can categorize the action. Depending on which quadrant the risk exists in, a consequent action can be decided upon. (Merna, 2010) This categorization is illustrated in both table 5 and figure 4.

(33)

27

Table 5: Risk matrix categorization with respect to the 4T’s of hazard management. (Hopkin, 2010) Risk Probability of occurrence Probability of consequence Response Risk 1

Medium High Treat

Risk 2

Medium Medium Treat

Risk 3

High Medium Terminate

Risk 4

Low High Transfer

In order to understand the thought process behind the nominal values on occurrence and consequence in table 5, these need to be

operationalized.

Table 6: Operationalizing nominal values for table 5. Nominal

value

Occurrence Adverse effects magnitude

LOW Once a year _{Low impact upon daily}

operations MEDIUM _{Once every financial}

quarter

Medium impact upon daily operations

(34)

28

The impact in table 6 denotes financial losses (most likely loss of liquidity, trust towards customer, and time loss).

The risks contained in table 5 are the risks gathered from the risk

identification process. The operationalization of nominal values in table 6 is done with respect to stakeholders (i.e. what the relevant

stakeholders perceive as low/medium/high occurrence respectively adverse effects of impact).

2.3 Previous research

The problem of integrating RM (Risk Management) practices with agile methodologies is well known, and has been attempted previously. The research done regarding the area looks at the application itself and not whether RM is applicable or not, and what the general risks are in the agile methodology. It is this gap in knowledge that this study attempts to fill. (Ribeiro, 2009) As RM inherently requires relevant key

individuals to hold intricate knowledge of specific projects, it is required, of them, to know the general risks (in order to go into more detailed risks), to be able to apply RM successfully. (Elbana & Sarker, 2016; Urvashi & Suprika, 2016)

A study conducted at the Princess Sumaya University of Technology (Albadarneh, 2015) attempts to research risk management within agile development projects. The paper discusses RM activities in agile

(35)

29

more analyzing details inherent to the capacity of risk management. Further on the practices of SCRUM provides a wider range of

combining with practices of risk management than eXtreme Programming.

One previous study attempts to apply an Agile Risk Management Process in multiple projects environments in order to focus on the

organizational environment in relation to project risks. The method used is combined with mPRIME process (similar to a PDCA cycle). The

conclusion of the study shows that successful implementation mitigated the main theorized risks that could have a negative impact on the

projects. The issues of the study, identified by the authors of the study, were that the used methodology did not hinder the occurrence of

(36)

30

3 Method

This chapter represents the methodology that will be used in order to fulfil the aim of the study and answer the research questions.

This study makes use of a qualitative approach. The reason for this choice is because of the exploratory approach to the problem of general risks within agile maintenance and development projects, as well as the difference in perception of risks by stakeholders within this project. As there has not been a study into this field of research previously, an exploratory approach is the most suitable.

3.1 Data collection

This study mainly looks at the agile methodology Scrum and the inherent risks within this methodology, stakeholders in the form of the Scrum master, project leader, and developers are the people with data that is required to answer the questions in this study. Collection of data from these stakeholders can be done in several ways. The approaches that have been chosen for this study contains of the Delphi technique (elaborated upon in the previous chapter) and interviews (semi-structured interviews). The main argument, when it comes to the chosen data collection method, is because the data is inherently subjective (e.g. the subjective perception of the impact or occurrence of a risk), therefore it is of importance to take into account that the wider the data collection is - in regards to this certain case -, the more reliable the data becomes.

3.2 Design

The data collected in the first phase will consist of interviews done with relevant key stakeholders – in this case, the project leader, the SCRUM master, and two system developers. The interviews will be transcribed and coded, in order to identify the relevant general risks within agile projects.

(37)

31

in the results chapter. The risk appetite matrices will act as a visual aid in order to give an easier overview of the individual appetite for risks.

The third phase consists of using a Delphi approach with the same stakeholders. This Delphi technique will be applied as a panel, where both the four different and same key stakeholders are interviewed together and asked the same questions as in the second phase. The panel will be asked to discuss the individual risks and decide upon its likelihood and magnitude of impact. The collected data will be portrayed as a summary of how the respective risks are classified, as well as a unified appetite matrix. This third phase is conducted in order to see if and/or how the unified appetite matrix differs from the individual appetite matrices.

Figure 7: The three phases of data collection in this study.

(38)

32

included, as this is outside the scope of the study. The study follows a clear theme of following this specific process throughout the paper.

3.2.1 Risk Management methodology

One inherent problematic area within the integration of RM into Agile processes is that there are no consensus on frameworks (in reality, there are not that many RM frameworks that has the sole purpose to be used with Agile). (Edzreena, 2018) Having the knowledge of this fact, the only frameworks which are available are those identified within the “traditional” RM school.1

Choosing a suitable framework is up to the risk manager; depending on which parameters and variables containing risks should be looked at. In the case of this study will be looking at an IT consulting firm that is using SCRUM to manage a large IT system. The chosen framework is ISO 31000. ISO 31000 is used by organizations in order to manage risks. The standard also has a sub-parameter where it includes risks tied to projects, while other frameworks do not. Even though ISO 31000 was not made with agile projects in mind, it is a suitable framework in assisting risk managers in handling risks. This choice can be seen as arbitrary, as there are no specific frameworks made specifically applicable with agile methodologies.

3.2.1.1 Establishing the context

Establishing the context is important in order for the risk manager to understand what kind of risks and within what area it is of interest to look at. This context generation is done by data collection with the relevant stakeholders.

The context, in our case, is pre-established in the form of several research questions.

(39)

33

3.2.2 Risk identification

The risk identification will be done by interviewing these key individuals. Their answers will be collected in a risk register. The reasoning for choosing a risk register is because of the amount of answers generated during these interviews, as a larger amount of answers (and identified general risks), makes the summarizing of these risks easier and more understandable with the help of a risk register.

3.2.2.1 Risk assessment and techniques

In order to classify risks, they first need to be assessed, categorized and analyzed. There are several different methodologies and models that can be used - either individually or in tandem. We will present available models that are available in order to document recorded risks from gathered data. There are several techniques in place in order to collect data regarding risks. Respective data collection methodology has pros and cons, related to their techniques. The study will convey the most relevant data collection methods in relation to risks in this sub-chapter.

3.2.2.2 Questionnaires and checklists

This risk data collection methodology uses structured questionnaires and checklists in order to collect data regarding risks. This method helps the risk manager identify existing risks. (Merna, 2010) The risk manager can identify hazards, risks or control failures, with the help of data collected from relevant stakeholders. This method is at its strongest when applied as a data check (i.e. combined with another data collection method) in order to confirm collected data. (Valis & Koucky, 2009)

3.2.2.3 Workshops and brainstorming

(40)

34

containing process. These peers discuss and identify potential problem areas that are associated with risks. Criteria regarding probabilities, consequences, and options for diminishing (or completely removing) likelihood or impact, are generated. (Valis & Koucky, 2009; Hopkin, 2010; Merna, 2010)

3.2.2.3 Interviewees

Four people have been determined of interest for the data collection. These being the four most relevant parties within this IT project: the SCRUM master, the project manager, two system developers (front- and back-end). The testing of the system (for Q&A reasons) are done by both system developers and the SCRUM master.

3.2.3 Risk analysis

After data has been collected from relevant stakeholders and presented in both a risk score card and a risk register, it will be portrayed in a risk matrix. This is done in order to relate respective risks magnitude of impact to their occurrence probability, and to give an easy-to-understand illustration of the impact-to-consequence relation. The risk matrix will be laid over a risk appetite matrix. The next step will consist of collecting data on how respective stakeholder perceives the individual risks; i.e. if they are within a comfort, cautious, or concerned zone. This will help the authors of this study to identify the subjective risk perception between different stakeholders within an organization.

3.2.4 Risk evaluation

(41)

35

3.2.5 Communication and consultation

The communication and consultation process is ongoing at all times parallel to every part of the frameworks process. (ISO 31000)

Communication between risk managers and project stakeholders is always ongoing, during consultation, information collecting, confirming assessments and identifications. This is done because of the wide

knowledge the stakeholders have regarding their processes. The risk manager is mostly focused on tying the knowledge of the experts to a risk process. This process is especially important in this study, as it is vital for the authors to gather the correct information and then confirm the collected and summarized information and data in the study. This communication and consultation mainly consists of the said relevant key stakeholders within the case study (in order to clarify or confirm collected data and the perception of it).

3.3 Qualitative methodology

A qualitative methodology has been chosen in order to answer the research questions. The combination of RM and agile methodologies is a relatively new area that has not had much insight into it. It is of interest to enlighten the research community as well as stakeholders such as other risk managers, project leaders, and scrum masters, regarding the general risks within agile methodologies such as SCRUM. In other words, an explorative approach has been taken - as it is a relatively non-researched area (which is evident from previous research).

3.3.1 Semi-structured interviews

(42)

36

summarized and analyzed, and assist in creating an individual risk appetite matrix. This risk appetite matrix is highly individual. With these facts in mind, a semi-structured interview form is most suiting; generating the area of interview (risk management) with the first questions, and later moving on to spontaneous questions in order to gain more in depth data.

The reason for semi-structured interviews is because of the fact that the interviewees might not make themselves comprehendible because of terminologies etc, and thus needs to be asked further questions in order to generate a more generalizable answer.

3.3.2 Coding of interviews

The coding of interviews will be done by respective individually listening to recordings of the interviews and summarizing different identified risk areas. After this is done, both researchers will, together, go through their individually identified risks areas and determine - by looking at keywords (e.g. “problem”, “risk”, “issue”, or any other keyword tied to such words) - if the certain risk is relevant and actually a risk (e.g. “work is hard” is equivalent to “there is too much [work] to do”)

After this is done, the researchers will put up their respective keywords up against another and discuss which risks are mentioned and in contradiction, thus only needs to be brought up once and which risks are brought up by two or more different stakeholders. This coding of data collected from interviews will be presented in an appendix.

3.3.3 Delphi technique

(43)

37

“current appetite” being the risk appetite after discussions with their peers).

The Delphi technique will be applied in a form of a panel. 3.3.4 Risk illustration tool

Because of the qualitative approach to this study, a fitting illustrative framework has been used in order to convey collected data and therefore been made easier to visualize for the reader. The used tool consists of EXCEL, where VBA coding has been used. The tool contributes to the majority of the tables and illustrative figures within this study.

3.3.5 Likelihood & Consequence

In order to do this study and answer the research questions, one needs to dwell more in-depth into the two fundamental terminologies within risk management: Likelihood and Consequence.

These two terminologies need to be quantified.2_{They need to be} operationalized if one is to relay and illustrate the collected data in matrices. The operationalization of these two variables has been chosen to be, Likelihood and Order of Impact.

Likelihood - The subjective perception by a stakeholder that event X will

occur with the probability of P.

Order of Impact - The degree of impact which the events X will affect the

organization.

In order to illustrate the placement of both these variables, we need nominal output functions. These are illustrated in table 7. It is important to note that these nominal values, in relation to their individual risks, are data gathered from interviewed stakeholders. This data is inherently subjective, which the authors will map in a risk appetite matrix. Furthermore, in order to make it easier and illustrative in a plot, the chosen numerical values for both variables were decided to an interval of 0 to 10.

(44)

38

Table 7: Operationalizing of numerical values in relation to likelihood and impact.

Numerical interval

Likelihood Impact

0-3 Low probability to occur under one fiscal year.

Low impact upon

individual projects during a fiscal year.

4-6 _{Medium probability to} occur under one fiscal year.

Medium impact upon individual projects during a fiscal year.

7-10 _{High probability to} occur under one fiscal year.

High impact upon

individual projects during a fiscal year.

3.3.6 Risk appetite matrix

(45)

39

interviewee is not worried about the risk and can take a course of action when and if it eventually arrives.

These zones are determined by the interviewee in the second and third phase of the data collection method. These color zones are determined by asking the interviewee to decide the nominal impact of a risk and its likelihood, as well as which comfort zone said risk would be perceived at. When the data points of risk (R1 through R22) have been plotted, a visual Green-Yellow-Red zone is created in order to assist the reader in reading out and understanding the risk appetite matrix. The reason why this methodology has been chosen is because it makes it easier for the authors of this study to generate a risk appetite matrix, as without it, there would be no way to determine whether a interviewee is risk aversive, neutral, or aggressive.

3.4 Case study

In order to generate the general risks within the SCRUM methodology and the subjective perception of risk by stakeholders, it is of interest to conduct a case study at an organization that conducts work with SCRUM. The case study is conducted at an IT consulting firm in Sundsvall, Sweden. Initially the data collected will help answer the questions regarding the general risks within SCRUM, the following data collections will portray, with the help of interviews, different stakeholders perception of risk. When this has been done, the same stakeholders will be involved in a Delphi panel and in tandem create a unified risk appetite matrix. When this has been done, the same stakeholders will then be interviewed once again, individually, in order to see if and how their risk appetites have changed. In order to limit the large amount of potential data that may be collected, only a chosen few stakeholders will be chosen. The most relevant ones being the SCRUM master, product owner, and a system developer.

(46)

40

3.5 Social and ethical principles

Due to the research being a case study applied to an organization it is of great importance to be consistent and keeping the ethical considerations in mind throughout the study. Initially the organization within the case study will be informed that all data collection will be secured and only used for research purposes. Furthermore, any data about employees or other sensitive information will be stored safely and anonymous unless otherwise specified. Respondents’ identity will be secured and their rights will be told, a form about information and consent will make it possible. The respondents admit their participation after signing this form. Each interview begins with the basics that respondent has opportunity to quit their participation and a query for accepting recording of the interview. When the results of the interviews are concluded the respondents will have the opportunity to investigate them in order to provide insight of what their answers will be used for. Finally the used language will exclude all types of victimization and discrimination and to show respect to all involved parts.

3.6 Validity and reliability

In order to strive for high validity and reliability it is important to ensure that the authors measures what is relevant in the context of the study and in a reliable manner. Validity is ensured throughout using the intended measurement tools at the right time, while reliability implies how trustworthy the tool is. Therefore a high reliability does not imply a high validity itself but a high validity requires a high reliability. (Eliasson, 2006) In purpose to vouch an arbitrary validity within this study, interviewed employees within the business case will sustain informed about schedule and definitions of concepts. This in accordance to what seems to be the indispensable for the study due to tight schedule and unforeseen meetings may affect presence and answers to interviews. Beyond interviews within the management, variation of included product developers may also affect the result due to subjective influences. Product developers within the business case have a few years of experience and are incorporated to the business and well conjunct to the others.

(47)

41

seems arbitrary due to the business case having about 15 employees including management, working with the project. Employees has different roles and purpose to the project; Product Owner; Scrum master; Product developers, and others. An information and consent form were used to increase the respondents understanding and insight about the purpose and aim of the study.

The methods risk register and risk appetite was used in the study due to the distance and complement of these methods seems important. Another way of identifying risks and their relevance could yield a different result, on the other hand they could not make considerations about the aspects of distance between the both methods.

(48)

42

4 Results

In this chapter the results of the study are presented. Firstly the three phases of interviews which fulfil the data collection and secondly the methodologies of risk management applied to the identified risks inherent to the SCRUM process of the case study.

Risk identification

4.1 First phase

The data collected points out several different risk areas mainly tied the failure to collect the ultimate objective of the project. The different stakeholders mention different risks tied to their respective areas of workspace

Person A, whom has the SCRUM master role (See Annex A),mentions the incapability of not collecting all the requirements needed for a sprint, this implies that the project group would need to make

assumptions on functions etc. Said person also mentions other risk areas such as developers and tester’s lack of competence, loss of competent workers, unclear end goal, vaguely commented code, loss of expert consultation, and the Snowball effect when changing around in the code.

Person B which has a system developer role conveyed the difficulty of estimating time for user cases and stories because of the unclear end goal. Other mentioned risks were: the lack of customer involvement in the SCRUM process, potentially bad team chemistry, contracts and agreements not being fully done, issues with prioritization (e.g.

(49)

43

Person C which has a system developer role, identifies risks such as contradictory requirements from customer, bad estimation of cost and time (which affects time efficiency), and security risks tied to treatment of sensitive personal information (tied to PUL/GDPR laws), lack of risk analysis, identification and management, as well as risk awareness. This person also identified risks which the previous persons have brought up, specifically: change of requirements (cause-effect from external stakeholders); external incidents or events that affects the planning; unclear, vague, or non-existent end goals.

Person D which has a product owner role, identified risks in the form of misunderstanding customer requirements, and issues related to

misunderstanding events in the testing phase. Other risks are the assumption by developers on how the acquis communautaire is to be applied in certain areas, lack of testing, lack of personnel.

(50)

44 Table 8: Risks identified within phase one

Risk No. Risk description R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 R13 R14 R15 R16 R17 R18 R19 R20 R21 R22

Not collecting all requirements from customers. Lack of knowledge.

Bugs in the system.

Unclear and vague end goals. Relying on expert consultation Loss of expert (acquis) consultation. Time estimation problems.

Customer not sufficiently involved in the development process. Agreements and contracts not realized.

Prioritization issues (stories and functions).

Customers lack of identification of necessary functions. Manual testing.

Parallel ongoing projects.

Contradictory demands from the customer. Handling of personal information.

Lack of risk identification, analysis, management, and awareness. Misunderstanding customer demands.

Misunderstanding testing events. Assumptions of acquis and functions. Lack of testing.

(51)

45

4.2 Second phase

4.2.1 Risk register

By presenting the compilation of various risks collected during phase 1 within a risk register. Each respondent could respond with their

perception of each risk and risk register questionnaires could be carried out to set values on likelihood of the risk and which impact it would yield if the risk itself occurs. Additionally, after receiving values on likelihood and impact per risk and individual, respondents were asked how comfortable they would be in case of exposure of each risk;

Comfortable; Cautious; Concerned. Understanding the likelihood and impact of each identified risk yields a deeper knowledge on the

frequency of occurrence and the projects vulnerability of the risk.

4.2.2 Interview answers

Table 9: Answers from the individual interviews with the four stakeholders.

(52)

46

The colours in table 9 denotes the risk perception of the individuals derived from the interviews. Red denotes the highest level of risk, yellow denotes a mediocre risk perception, while green denotes a comfortable approach to the risk. Together with these answers, we can create four individual risk appetite matrices.

4.2.5 Risk classification and appetite

In order to classify the risks and to understand the risk appetite of the respondents, risks identified per individual were mapped onto a risk appetite matrix with likelihood as x-axis and impact as y-axis. With a scale from 0-10 were 0 is no likelihood and 10 is very high likelihood on the x-axis. On the y-axis 0 is no impact and 10 is very high impact, were non-impact does not affect the project and very high impact will affect the project in such a high grade that it could be necessary to quit the project. Risks are given a number 1-22 in following order by the table 11. The appetite matrix helps us in understanding how different

stakeholders perceive risks. Some stakeholders are more risk aggressive towards higher likelihoods than impacts, and vise-versa. The way to read this fact out is by looking at the color coding of either the Y-axis or the X-axis. If we look at figure 8, the risk appetite matrix for person A, the green zone is around f_impact(6.5) and f_likelihood(8.5), which gives us a certain visual area that makes it easier to compare this persons’ risk appetite compared to its counterparts

Person A

Person A’s risk appetite matrix was created by plotting out the data points unto a program and giving these data points the color of the majority of that point (i.e. R20, R7, R16 are all the same point but since yellow is the majority of these three risks’ zones, yellow is the chosen color). The approximation of mapped zones can be likened to linear regression, where we try to approximate lines (instead we do it with a zone).

(53)

47 Figure 8: Risk appetite matrix for person A.

Person B

Person B is considered to be risk aversive, as the universe of risk is quite large while the comfortable zone (green area) is relatively smaller than its counterparts.

(54)

48

Person C

The universe of risk (the red zone) for person C is relatively small compared to the three other risk matrices, and hence could be

considered to be risk aggressive (more prone to take higher risks). This can be seen by looking at the largeness of the green area, where a risk such as R12 is considered comfortable, and R6 cautious.

Figure 10: Risk appetite matrix for person C.

Person D

Person D has a larger appetite for risk in relation to impact compared to likelihood (a risk with 0/10 [likelihood/impact] is considered

comfortable while a risk with 7/0 is considered comfortable as well). As person D has a very angled appetite matrix, it may be a bit hard to correctly classify whether it is risk neutral or aggressive, as there is no clear framework upon deciding this. In this case, we will consider it somewhat risk aggressive.

(55)

49 Figure 11: Risk appetite matrix for person D.

These four individuals have differing risk appetite matrices. From tables Person A to Person D, we can see that person D has the largest universe of risk, followed by person B, person C and lastly person A. In unclear cases of which individual has a larger or smaller universe matrix, Euclidean methods can be used in order to calculate and evaluate the area of risk universe.

Data points plotted out on each illustration shows the classification of individual risks by respective stakeholder. It is apparent that risks are very diversely classified. In two of these four cases, both have the same job role at the firm (person A and person B).

(56)

50

4.3 Third phase

The third phase of the interviews was conducted with all four

previously interviewed individuals in group. These individuals were instructed to discuss among themselves regarding respective risks and determine a consensus on how to classify them. The collected data is summarized in appendix C, and is visualized in table 12.

Table 12: Delphi panel results.

(57)

51

Table 13 shows the group’s result of the perception of the risks

identified in table 8. The group reached a consensus that 8 risks were considered cautious while 14 were deemed comfortable. It was noted during the interview that one of the interviewees had more to say than the other three (namely person D, the project leader), while person B had the least to say. Naturally, as person D is the project leader, he holds most of the responsibility and demands more respect in relation to the three other individuals. Person A (the first system developer) was the second most vocal individual and had differing opinions on some of the risk classifications compared to the rest of the group. Person B (second system developer) and C (the SCRUM master) mostly followed the project leaders lead and classifications on their probabilities and magnitudes of impact.

Another interesting point was brought up by the interviewees; the

(58)

52

Table 13: Comparison of group answers to individual answers.

(59)

53

Figure 12: The risk appetite of the four individuals together.

Självständigt arbete på avancerad nivå