Is it SAFe to use WSJF for
prioritisation in financial software development?
A case study of prioritisation needs at a Swedish bank
GUSTAV DAHLSTRÖM
JESPER ROBERTSSON LUND
KTH ROYAL INSTITUTE OF TECHNOLOGY
This page is intentionally left blank
Is it SAFe to use WSJF for prioritisation in financial software development?
A case study of prioritisation needs at a Swedish bank
by
Gustav Dahlström Jesper Robertsson Lund
Master of Science Thesis TRITA-ITM-EX 2018:XYZ KTH Industrial Engineering and Management
Industrial Management SE-100 44 STOCKHOLM
Är det SAFe att använda WSJF för prioritering inom finansiell
mjukvaruutveckling?
En fallstudie av prioriteringsbehov hos en svensk bank
av
Gustav Dahlström
Jesper Robertsson Lund
Examensarbete TRITA-ITM-EX 2018:XYZ KTH Industriell teknik och management
Is it SAFe to use WSJF for prioritisation in financial software development?
Gustav Dahlström Jesper Robertsson Lund
Approved
2019-06-15
Examiner
Pernilla Ulfvengren
Supervisor
Matti Kaulio
Commissioner Contact person
Christer Berg Abstract
As a response to digitalisation and new niche competitors, incumbent banks tries to increase their flexibility by adopting agile principles at their development departments. However, the flexibility of the agile departments is constrained by the flexibility of the rest of the organisation, which are still working according to the waterfall method. To address this problem, several incumbent banks are focusing towards scaling their agile work to the entire organisation. The case company for this study is a bank trying to scale their agile work by using the Scaled Agile Framework (SAFe). Nonetheless, there is scepticism within the bank whether Weighted Shortest Job First (WSJF), the prioritisation model used in SAFe, is applicable in financial software development. The available case studies are often conducted by consultancy firms implementing the framework and are focused towards the positive effects, excluding challenges or adaptions made for specific industrial contexts. This study contributes to this gap in the literature with empirical findings by assessing the adaptability of WSJF and its ability to satisfy the prioritisation needs of financial software development. Moreover, empirical data is used to formulate recommendations for how WSJF can be adapted and integrated with the current processes of financial software development.
To operationalise the study, the following research question was formulated: "How could WSJF be modified to better suit financial software development?". To answer upon this, a qualitative case study employing semi-structured interviews was conducted. The semi-structured interviews were used to identify the prioritisation needs at a selected case department. The needs were used to formulate prioritisation requirements that were structured according to an analytical framework. The framework was used to evaluate how many of the requirements WSJF were able to satisfy. Moreover, the analytical framework was used as a guideline in formulating the recommendations of modifications to better satisfy the prioritisation needs.
The results showed that no single set of needs is able to conclude the prioritisation needs of all organisational levels. Employees on an operational level have a more practical approach towards prioritisation and requires a model that is easy to use and time efficient. Meanwhile, the strategic level requires a model with high precision that could estimate the value of deliveries monetarily, allowing them to track the realisation of deliveries in the financial figures. WSJF are able to satisfy the needs of the operational level but fails to satisfy the needs of the strategic level. Derived from the obtained needs of both levels, the following two modifications are recommended: Establish a uniform value definition that are used by the entire organisation and Implement a prioritisation model that can combine monetarily CoD at a strategic level with relative CoD at an operational level. By implementing these changes the percentage of satisfied requirements increased from 50% to 89% on a strategic level and from 85% to 90% on an operational level. Resulting from this, the conclusion is that WSJF can be modified to better suit financial software development by implementing the recommended changes formulated in this report.
Key-words: Prioritisation, WSJF, CD3, SAFe, Agile, Cost of delay.
Är det SAFe att använda WSJF för prioritering inom finansiell mjukvaruutveckling?
Gustav Dahlström Jesper Robertsson Lund
Godkänt
2019-06-15
Examinator
Pernilla Ulfvengren
Handledare
Matti Kaulio
Uppdragsgivare Kontaktperson
Christer Berg Sammanfattning
De traditionella storbankerna försöker implementera agila arbetssätt inom deras utvecklingsavdelningar för att öka flexibiliteten. Detta är ett svar på en ökad digitalisering och nya nischade konkurrenter. Dock är flexibiliteten av de agila arbetssätten begränsad av flexibiliteten av den övriga organisationen, som fortfarande jobbar enligt vattenfallsmodellen. För att adressera denna begränsning fokuserar bankerna på att skala upp det agila arbetet så att det täcker hela organisationen. I denna studie studeras en av dessa banker, som har valt att använda sig av det agila skalningsramverket Scaled Agile Framework (SAFe).
Trots valet av SAFe finns en viss skepticism huruvida den medföljande prioriteringsmodellen Weighted Shortest Job First (WSJF) lämpar sig för finansiell mjukvaruutveckling. De studier som finns tillgängliga kring SAFe och WSJF är till stor del framtagna av konsultfirmor vars uppgift är att implementera SAFe.
Således är dessa rapporter fokuserade på de positiva effekter som uppnås, samtidigt som utmaningar och anpassningar för det specifika industriella kontexten utelämnas. Detta skapar ett gap inom litteraturen, till vilket denna studie ämnar att bidra med empiriska resultat från en undersökning av WSJFs förmåga att tillfredsställa och anpassas efter prioriteringsbehoven inom finansiell mjukvaruutveckling. Därutöver ligger de empiriska resultaten till grund för rekommendationer gällande hur WSJF bättre kan anpassas och integreras inom finansiell mjukvaruutveckling.
För att operationalisera studien formulerades följande forskningsfråga: "Kan WSJF modifieras för att lämpa sig bättre inom finansiell mjukvaruutveckling?". För att besvara detta genomfördes en kvalitativ fallstudie där semi-strukturerade intervjuer användes för att identifiera prioriteringsbehoven av en utvecklingsavdelning hos den undersökta banken. Behoven användes för att populera ett analytiskt ramverk, som sedan användes för att utvärdera hur många prioriteringskrav som WSJF lyckas tillfredsställa. Dessutom agerade ramverket som guide i framtagandet av rekommenderade modifikationer, för att öka antalet krav som WSJF tillfredsställer.
Resultatet indikerar att ingen enskild uppsättning av behov täcker samtliga prioriteringsbehov i organisationen. Anställda på en operationell nivå har en praktisk syn på prioritering och efterfrågar en modell som är tidseffektiv och enkel att använda. Anställda på en strategisk nivå kräver en prioriteringsmodell med hög precision som kan uppskatta värdet av en leverans monetärt, då detta möjliggör en uppföljning av hur leveransvärdet realiserats genom de finansiella siffrorna. WSJF tillfredsställer prioriteringskraven från den operationella nivån medan den misslyckas med att tillfredsställa hälften av kraven från den strategiska nivån. Utifrån de prioriteringskrav som samlats in rekommenderas följande två modifikationer: Fastställ en gemensam värdedefinition som används i hela organisationen och Implementera en prioriteringsmodell som kombinerar monetärt värde på en strategisk nivå med relativt värde på en operationell nivå. Genom att implementera dessa modifikationer ökar antalet tillfredsställda krav från 50% till 89% på en strategisk nivå och från 85% till 90% på en operationell nivå. Utifrån detta är slutsatsen att WSJF kan modifieras för att bättre lämpa sig för finansiell mjukvaruutveckling.
1 Introduction 1
1.1 Background . . . 1
1.2 Problematisation . . . 2
1.3 Purpose and Research Questions . . . 3
1.4 Scientific Contribution . . . 3
1.5 Industry Contribution . . . 4
1.6 Delimitations . . . 4
1.7 Limitations . . . 4
1.8 Thesis Outline . . . 4
2 Literature Review 6 2.1 Agile Principles . . . 6
2.2 Scaled Agile Framework . . . 7
2.3 Using Cost of Delay in Prioritisation . . . 9
2.3.1 Weighted Shortest Job First . . . 10
2.3.2 Cost of Delay Divided by Duration . . . 11
2.3.3 Qualitative Cost of Delay Using Urgency and Value . . . 12
2.3.4 Useful Tools for Calculating Cost of Delay . . . 15
2.3.4.1 Assigning Cost of Delay to Enablers . . . 16
2.3.4.2 Minimal Viable Product . . . 16
2.4 User-Centered Design . . . 16
2.5 Situational Analysis . . . 17
2.6 Summary of Literature . . . 18
3 Method 21 3.1 Model of Analysis . . . 21
3.1.1 Work Process . . . 21
3.1.2 Analytical Framework . . . 22
3.2 Research Design . . . 24
3.3 Pilot Study . . . 25
3.4 Main Study . . . 26
3.5 Data Collection . . . 28
3.5.1 Interviews . . . 28
3.5.2 Observation Methodology . . . 30
3.5.3 Literature Search and Review . . . 31
3.5.4 Document Study . . . 32
3.6 Data Analysis . . . 32
3.7 Reliability, Validity and Research Ethics . . . 34
4 Empircal Findings 36 4.1 Introduction of the case company . . . 36
4.2 Needs and Requirements . . . 36
4.2.1 Value-based Prioritisation . . . 38
4.2.2 Difficult Valuation Cases . . . 40
4.2.2.1 Regulations and Compliance Risk . . . 41
4.2.2.2 Operational Risk . . . 41
4.2.2.3 Enablers . . . 42
4.2.3 Monitoring Delivered Value . . . 42
4.2.4 Roles of Prioritisation . . . 43
4.2.5 Easy to Use . . . 44
5 Analysis and Discussion 46 5.1 Prioritisation Needs of Financial Software Development . . . 46
5.1.1 Functional Perspective . . . 48
5.1.2 Environmental Perspective . . . 49
5.1.3 Organisational Perspective . . . 50
5.1.4 Metrics . . . 51
5.1.5 Types of Projects . . . 51
5.2 Assessment of WSJF in Financial Software Development . . . 51
5.2.1 Functional Perspective . . . 53
5.2.2 Environmental Perspective . . . 54
5.2.3 Organisational Perspective . . . 54
5.2.4 Metrics . . . 55
5.2.5 Types of Projects . . . 56
5.2.6 Summary of WSJF Evaluation . . . 56
5.3 Recommended Prioritisation Approach for Financial Software Devel- opment . . . 57
5.3.1 Uniform Value Definition . . . 57
5.3.2 Combining Monetary CoD with WSJF . . . 59
5.3.2.1 Integration Between WSJF and CD3 . . . 59
5.3.2.2 Recommendations for getting to Monetary Value . . 61
5.4 Assessment of WSJF with Modifications for Strategic Prioritisation . 63 6 Conclusion 67 6.1 Implications . . . 69
6.1.1 Academic . . . 69
6.1.2 Industrial . . . 69
6.2 Limitations and Future Research . . . 70
APPENDICES I
A Interview Guide I
2.1 Illustration of SAFe (Scaled Agile 2018). . . 9 2.2 The WSJF formula (Scaled Agile 2018). . . 10 2.3 Components of CoD used in WSJF (Scaled Agile 2018). . . 10 2.4 The iterative valuation process using 5-Whys (Arnold & Yüce 2013). 12 2.5 Illustration of Cost of Delay x Urgency (Arnold 2017). . . 13 2.6 Urgency profile for a short benefit horizon and with a reduced peak
due to late delivery (Arnold & Yüce 2013). . . 14 2.7 Urgency profile for projects with a long-life and reduced peak due to
late delivery (Arnold & Yüce 2013). . . 15 2.8 Urgency profile for projects with a long-life and a peak unaffected by
delay (Arnold & Yüce 2013). . . 15 2.9 Illustration of the UCD workflow. . . 17 3.1 Illustration of how the UCD approach is used to answer the research
questions. . . 22 3.2 Workflow of the study. . . 25 3.3 Illustration of the funnel model. . . 30 3.4 Illustration of the procedure for analysing qualitative data by Miles
& Huberman (1994). . . 33 4.1 Value distribution of a development team at Maersk, investigated by
Black Swan Farming (Arnold & Yüce 2013). . . 39 5.1 Illustration and formula for calculating enabling value. (Based on
theories from Wright (2018)). . . 58 5.2 Illustration of how monetary CoD interacts with WSJF. . . 60 5.3 Illustration of combination of CD3 and WSJF. . . 61 5.4 Urgency profile for ideas with a long-life and a peak unaffected by
delay (Arnold & Yüce 2013). . . 62 5.5 Urgency profile for for regulatory projects with a deadline. . . 63
3.1 Analytical framework. . . 23
3.2 Interviews and the observation conducted in the pilot study. . . 26
3.3 Interviews of the main study. . . 28
3.4 Summary of documents used to conduct this study. . . 32
5.1 Populated evaluation framework on an operational level. . . 47
5.2 Populated evaluation framework on a strategic level. . . 48
5.3 Assessment of WSJF on an operational level. . . 52
5.4 Assessment of WSJF on a strategic level. . . 53
5.5 Distribution of satisfied, unsatisfied and partly satisfied needs on an operational and a strategic level. . . 56
5.6 Unsatisfied needs with recommended solutions. . . 57
5.7 Assessment of modified WSJF from a strategic perspective. The re- quirements affected by the recommendations are highlighted in bold. . 64
5.8 Distribution of satisfied, unsatisfied and partly satisfied needs on a strategic and operational level before and after modifications. . . 65
Introduction
This chapter aims to introduce the reader to the study by presenting its background as well as presenting the problem. This is followed by the purpose and the research questions which forms the foundation for this study. Lastly, the contribution, delim- itations and limitations of the study are presented.
1.1 Background
The software development industry becomes more complex for each day, the indus- try is fast changing and the customers demand a higher level of customisation, at the same time as the products become more complex and harder to specify. This creates a business climate that challenges the industry to be efficient and flexible, forcing its actors to change and adapt their processes to stay competitive (Stober &
Hansmann 2010). Many attempts to address these challenges have been made but it was not until 2001, when seventeen software developers met at the Snowbird ski re- sort in Utah and formulated the agile manifesto, a wide-spread change was initiated.
The agile manifesto promotes: Individuals and interactions over processes and tools;
Working software over comprehensive documentation; Customer collaboration over contract negotiation; Responding to change over following a plan (Beck et al. 2001).
The manifesto led to the development of a handful of frameworks, including famous frameworks such as Scrum and Extreme Programming.
The agile principles were quickly embraced by software development companies and have since then spread to other industries where software development partly con- stitutes to the main business. The Swedish financial sector is one example were the business climate has moved in the same direction as the software development industry. With its niche competitors utilising digitalisation to gain competitive ad- vantages, the incumbent banks realise the risk of having a technical debt and falling behind in technology (Vives 2017). This has led to a search for methodologies that replace the traditional waterfall model, that is used for technical development within the incumbent banks today. With a replacement of the waterfall model, the banks aim to increase their efficiency and ability to respond to changes in customer be- haviour. With the search for alternative methodologies, many of the incumbent
multiple divisions, making the flexibility of the agile frameworks constrained by the flexibility of the rest of the organisation. To address this problem, many banks search beyond traditional agile frameworks and attempt to scale their agile work through- out the organisation. The banks’ interest in scaling the agile work becomes clear when browsing the website of two of the larger frameworks for scaled agile work, the Scaled Agile Framework (SAFe) and Large Scale Scrum (LeSS). The SAFe website boasts with case studies from Standard Bank, Nordea, Deutsche Bank and Capital One and the LeSS website does the same with Bank of America and JP Morgan Chase (Scaled Agile 2019, The LeSS Company 2019).
1.2 Problematisation
The scaling of agile work has been successful in many organisations, leading to re- sults such as shorter lead-times, reduced costs and higher productivity (Scaled Agile 2019, The LeSS Company 2019). The success stories from SAFe implementations have drawn the attention of many banks, such as the case company for this study, towards SAFe. The case company is one of the larger full-service banks in Sweden.
The bank has already begun with a small scale implementation of SAFe, where a few departments have been working according to SAFe for a few months.
Even though the implementation of SAFe at the case company has already started, there is some scepticism among its employees of how well the SAFe, and especially the integrated prioritisation model Weighted Shortest Job First (WSJF), is suited for the financial industry. WSJF prioritises the backlog based on the Cost of Delay (CoD) of each feature. Simply, this means that the prioritisation is based upon an assessment of what feature leads to the highest loss in potential value if not being developed immediately. The scepticism arise from the fact that the financial indus- try is highly regulated and the incumbent banks often have a large technical debt compared to its niche competitors. This makes regulatory projects and technical enablers, such as updates of old systems, two common types of projects in financial software development. Today, most of the regulatory projects are imperative and the system updates are seen as necessary development costs and does not have a value connected to them. The lack of a clear business value makes it challenging to estimate the CoD of these types of projects. Consequently, several employees at the case company question if WSJF could be used in this context or if it is developed with more traditional software development in mind, where a larger share of the projects are connected to business development. Other concerns among the employ- ees regarding WSJF are the usage of relative numbers with unclear scales and the combination of different units, such as time and business value. Resulting from this, the case company wants to investigate if WSJF satisfies the prioritisation needs of financial software development or if the model needs customisation.
Even though there is a significant number of reports from case studies where SAFe and WSJF have been implemented, no report presents the challenges faced or adap- tions being made for the specific industry context. A possible reason for this might be that the case studies are conducted by the founders of SAFe or by consultancy
firms implementing it. Hence, one can question the bias of the authors and the objec- tivity of the reports. This creates a lack of independent studies, creating problems for companies to assess the applicability of using WSJF in their specific context.
Thus, there is a need for an independent study investigating how well WSJF, in its original form, adapts to industry specific challenges. This study attempts to address the gap in the literature by investigating the industry-specific prioritisation needs in financial software development and assess the adaptability of WSJF.
1.3 Purpose and Research Questions
The purpose of this study is to assess the adaptability of WSJF by investigating its ability to satisfy the prioritisation needs of financial software development. More- over, empirical data will be used to formulate recommendations for how WSJF could be adapted and integrated with the current processes of financial software develop- ment.
From a sustainability perspective, the purpose of this study is to improve the eco- nomic sustainability of the financial industry by enlighten the value of regulatory projects and technical enablers. This would allow the industry to focus its re- sources to maximise its value efficiency. Additionally, the study aims to achieve an improved social sustainability by increasing the transparency of value in the organ- isation. This would facilitate the involvement of the employees in the prioritisation process and thereby give them increased influence in the planning of their own work.
To fulfil the purpose of the study, the following research questions (RQs) have to be addressed:
Main RQ How could WSJF be modified to better suit financial software develop- ment?
Sub RQ1 What are the prioritisation needs of financial software development?
Sub RQ2 How does WSJF satisfy the needs of prioritisation in financial software development?
1.4 Scientific Contribution
This study will contribute to the scientific field of prioritisation by conducting an independent study of the prioritisation model WSJF, investigating its adaptability.
Moreover, the study will contribute with empirical findings of the prioritisation needs of the financial software development industry.
1.5 Industry Contribution
This study will contribute to the financial industry by collecting empirical data of the prioritisation needs of financial software development. This information will be used to formulate recommendations of how WSJF can be adapted in such an industry.
1.6 Delimitations
This study is delimited to focused on prioritisation in financial software development with an underlying focus of prioritisation in a scaled agile environment. Moreover, the study covers prioritisation on a team level and higher, excluding prioritisation made on an individual level.
Due to the geographical location of the case company, the data collection is geo- graphically limited to Stockholm, Sweden. Additionally, the selection of case com- pany restricts the coverage of the study to full-service banks.
1.7 Limitations
Due to the competitive nature of the software development industry, companies may be unwilling to share their experience of prioritisation in a scaled agile environment, especially where any modifications to the original model have been made. Conse- quently, the sourcing of interviewees may be limited to the employees of the case company. Furthermore, the study is limited by the previous strategic decisions of the case company to scale their agile work by using SAFe. Consequently, the study investigates prioritisation in an organisation with the target of scaling their agile work, limiting the study to only investigate prioritisation where a scaling decision has already been taken.
As the study is conducted as a Master thesis, the study is obligated to follow a time requirement determined by the supervising institution. The time for the study is determined to five months, limiting the possibility to test the results of the study in a practical environment. Resulting from this, the testing of the recommendations formulated in this report will be limited to theoretical testing and discussions with employees.
1.8 Thesis Outline
Chapter 1: Introduction. This chapter aims to introduce the reader to the study by presenting its background as well as presenting the problem. This is followed by the purpose and the research questions which forms the foundation for this study.
Lastly, the contribution, delimitations and limitations of the study are presented.
Chapter 2: Literature Review. This chapter introduces the literature that forms the theoretical foundation of this study. Initially, information regarding ag- ile principles and SAFe are presented. This is followed by a review of the current prioritisation literature. Finally, the chapter presents frameworks that are used to operationalise the study.
Chapter 3: Method. This chapter begins with an introduction of the work pro- cess and the analytical framework used to structure the study. This is followed by the research design and work procedures of the pilot and the main study. Lastly, the data collection and analysis methods are presented followed by a discussion of the reliability, validity and research ethics of the study.
Chapter 4: Empirical Findings. This chapter presents the context in which the data has been collected followed by the empirical findings from the interviews and the observation. The empirical findings are grouped into common themes to obtain a better overview of the main topics.
Chapter 5: Analysis and Discussion. The following chapter is structured based on the research questions. Initially, an analysis of the empirical data is used to an- swer upon Sub RQ1. The answers are then used to assess the suitability of WSJF and thereby answer upon Sub RQ2. The findings from the two Sub RQs are then used to answer the main RQ.
Chapter 6: Conclusions. This final chapter concludes the thesis and provides a concrete answer to each of the research questions. This is followed by an explanation of the implications of the thesis from an academic and industrial perspective. At last, the limitations of the study are presented in combination with recommendations for further research.
Literature Review
This chapter introduces the literature that forms the theoretical foundation of this study. Initially, information regarding agile principles and SAFe are presented. This is followed by a review of the current prioritisation literature. Finally, the chapter presents frameworks that are used to operationalise the study.
2.1 Agile Principles
To better grasp the current prioritisation model and prioritisation needs at the case department, their current work procedures must be understood. The case depart- ment works according to the agile methodology Scrum. Hence, literature regarding the agile mindset and Scrum play an important role in mapping processes at the case department.
As described in the background for this study, the agile Manifesto was formulated by seventeen people at The Lodge at Snowbird ski resort in 2001. The manifesto includes 12 principles, handling subjects such as how to treat the firm’s customers, employees, methods, etc. These principles support the values of the agile mindset, which are compiled into the following four values:
• Individuals and interactions over processes and tools.
• Working software over comprehensive documentation.
• Customer collaboration over contract negotiation.
• Responding to change over following a plan (Beck et al. 2001).
Agility permeates the four values, where the values focus on collaborations and trust rather than comprehensive plans. The agile principles should not be seen as a set of tools, instead it should be seen from a broader perspective where the tools act as support in a comprehensive mindset. To have the right mindset is equally, if not more, important than implementing the right tools (Measey 2015).
Besides being adopted by the traditional software developing companies, agile method- ologies have been adopted by the financial sector. One example of the agile adaption in the financial sector is the adoption by Capital One, they began their agile im- plementation in 2004. The implementation was driven by the desire to reduce the
time to market of their IT projects. Other arguments for an Agile transformation are an increasing competitive IT landscape, increased efficiency and to sustain the revenue growth in a more digital market (Silva & Doss 2007). However, there are some scepticism in taking the leap towards working agile. Agile software develop- ment promotes working software over comprehensive documentation, which is the opposite of the requirements in the strictly regulated financial sector. Furthermore, the industry tends to stick to the traditional waterfall method due to the large amount of regulation within the industry. This is a result of the beliefs that the waterfall method is more predictable than the agile mindset (Ihme 2013). Despite these doubts, the trend indicates an increased adaption of the agile mindset in the financial sector.
Many banks work according to the agile framework Scrum. Scrum is together with Extreme Programming, of which Scrum overtook the dominant role in 2004, the two most used agile frameworks. Scrum is an incremental framework building upon three pillars of control: Transparency, Inspection and Adaption. The work is di- vided into shorter increments, sprints, that often have a duration between one week and one month. Scrum is built around three roles, which are briefly describes below:
• ScrumMaster
The ScrumMaster have a servant-leadership approach for the development team. The role of the ScrumMaster is to support the development team while protecting it from external distractions.
• Product owner
The product owner prioritises and defines the product backlog. The Product owner determines what needs to be done and how to maximise the value de- livery.
• Development team
The development teams are self-organised and cross-functional with a mix of competences. The teams determine how to best realise the requirements of the current sprint. (Measey 2015)
2.2 Scaled Agile Framework
The agile principles are clearly defining how agile teams should work. However, it has been criticised for not describing how multiple teams could collaborate in large projects or how to involve product portfolio management (Brenner & Wun- der 2015). In many organisations, the development teams are working according to Scrum while the rest of the organisation often use the traditional waterfall model for project planning and budgeting (Dingsøyr et al. 2014). This creates a friction between the agile teams and the waterfall organisation. To address the challenges of large scale projects, frameworks like LeSS and SAFe have emerged, trying to apply
study to understand its main principles.
Dean Leffingwell, the creator of SAFe, attempts to describe how the agile method- ologies can be adopted in an entire organisation by combining existing lean and agile principles. In SAFe, large projects are divided into epics, capabilities, features and stories. Epics are used on the highest level and are a container for large solutions that requires financial approval before implementation. Epics are further broken down to capabilities which often cuts across more than one value stream. A capabil- ity is sized to fit multiple features that contains a benefit hypothesis and acceptance criteria to fulfil stakeholder needs. Lastly, stories are used at the lowest level and contains a short description of desired functionality. (Knaster & Leffingwell 2017) SAFe consists of processes for team, program, large solution and portfolio level. The team level uses the traditional principles of Scrum, working in 2-week sprints. At the program level, SAFe extends the ideas of Scrum to a higher level, but instead of working in 2-week sprints, the program level works in Agile Release Trains (ART), composed of 5 sprints. Each ART consists of multiple teams making deliveries into the ART. The five sprints make up a 10-week period, called a Program Increment (PI). Before each PI, there is a two-day planning event where all teams, together with the business owners, are planning what features to include in the upcoming 5 sprints. After each PI, there is another event including all teams and program stakeholders, called Inspect and Adapt (I&A). At the I&A, the teams have demos of the current state of the solution, a quantitative follow up of actual delivered value compared to planned value and a retrospective of the completed work. The large solution level includes economical frameworks and processes to coordinate multiple ARTs. At the highest level, the portfolio level, SAFe has a focus on optimising value streams to help management prioritising the epics and features to be distributed to the ARTs. (Knaster & Leffingwell 2017) Figure 2.1 shows an illustration of SAFe.
Figure 2.1: Illustration of SAFe (Scaled Agile 2018).
2.3 Using Cost of Delay in Prioritisation
To better understand WSJF, one must understand its foundational mindset. WSJF builds upon the concept of Cost of Delay (CoD). Hence, literature regarding CoD are needed to better evaluate the SAFe prioritisation model in a financial software development context.
CoD is an efficient mindset in communicating how the outcome of a project is effected by time. In simple terms, CoD indicate how much value leaks over a predefined pe- riod of time. This could for example be in terms of missed revenue or higher costs due to a delay. CoD is expressed in the unit $/time, where time is a predefined interval, often selected to one week, month or year. Arnold & Yüce (2013) describes the concept of CoD as following:
"The Cost of Delay of a feature is the value that could be generated over a given period of time, if it was available immediately."
When calculating the CoD, one must assess how much value being missed by not having the feature ready when needed (Arnold & Yüce 2013). An example of this could be a feature that increases efficiency by automatising administration tasks.
The feature makes it possible to automatise the work of two full-time employees
cost of the two full-time employees per month.
In the book "The Principles of Product Development Flow: Second Generation Lean Product Development", Reinertsen (2009) describes CoD as "The one thing to quantify" and explains that only 15% of the product managers are aware of CoD.
Reinertsen continues by explaining that the value estimations can vary by a factor of 50 between members in the same team. Hence, value estimations by intuition are a poor way to estimate CoD and instead he argues that it should be calculated.
2.3.1 Weighted Shortest Job First
SAFe use WSJF for prioritisation, making use of the concept of CoD. WSJF is used to valuate and prioritise features before every PI planning, to ensure that the backlog is updated and the most valuable features are included in the upcoming sprints. The WSJF formula, shown in Figure 2.2, is composed by dividing the CoD with the job duration (Knaster & Leffingwell 2017).
Figure 2.2: The WSJF formula (Scaled Agile 2018).
In WSJF, the CoD consists of three parameters: User-Business Value, Time Critical- ity and Risk Reduction/Opportunity Enablement, see Figure 2.3. The user-business value corresponds to what the user wants and the revenue impact on the business.
The Time criticality represents how the value decay over time, is there a potential penalty or other consequences if we delay? The third element, risk reduction/oppor- tunity enablement corresponds to what else the feature might do for the business.
Does it reduce a risk, or will it open up new business opportunities? (Knaster &
Leffingwell 2017)
Figure 2.3: Components of CoD used in WSJF (Scaled Agile 2018).
The valuation of each component is not made in absolute numbers, instead it is based on a relative comparison of other items in the backlog, using a Fibonacci
scale (Scaled Agile 2018). The feature with the highest perceived business value is assigned the highest number in the business value component and so on. This makes the model simple and effective to use for backlog prioritisation.
The job duration can be difficult to determine, especially early on in the project planning phase when the knowledge regarding who is going to perform the job and the capacity allocations of the teams are limited. WSJF suggests that story points are a good proxy for job duration (Knaster & Leffingwell 2017). Story points are a relative measure that describes how much functionality a team is able to deliver.
The definition of a story point is unique to the specific team and is based on the complexity of work, the team’s skill and level of experience (Moreira 2013).
Even though WSJF has received positive feedback for bringing a value focus into the prioritisation process, the model has been criticised by parts of the agile community for:
• Having unclear definitions of its CoD components.
• Combining monetary value measures with other unit measures, such as time criticality and risk.
• Having relative numbers which makes it hard to quantify the value outcome (Arnold 2014).
The SAFe framework sees the relative numbers as an advantage, since the features are only compared against each other rather than given their own value. This keeps the valuation process relatively quick and simple. However, Arnold & Yüce (2013) argues that putting an absolute monetary value on each component in a project backlog can have even higher effects on value outcome.
2.3.2 Cost of Delay Divided by Duration
Cost of Delay Divided by Duration, also known as CD3, is a prioritisation method that makes use of monetary CoD in the prioritisation process. In CD3, the feature with the highest CD3-value will deliver the highest value per time unit. The CD3- value is calculated by dividing the CoD with the duration of the feature. The result indicates what feature to be developed to reduce the total CoD as fast as possible.
This method is effective when the development capacity is limited and the value of the features needs to be compared (Arnold & Yüce 2013).
In CD3, the CoD is often broken down into four types of benefits. These are: Increase Revenue, Protect Revenue, Reduce Costs and Avoid Costs. To get to these values, Lean methods such as 5-Whys are commonly used. The method simply encourages its user to ask "why?" until the benefits are identified and categorised (Arnold &
Yüce 2013). The iterative process is illustrated in Figure 2.4.
Figure 2.4: The iterative valuation process using 5-Whys (Arnold & Yüce 2013).
When the benefits are identified and categorised, Arnold & Yüce (2013) suggest a two-stage approach for estimating the value of the feature. The first step is to understand what effects being connected with the feature. For example, if old data can be migrated to a modern system, the dependency of the old system is removed and the licence can be terminated. Hence, the CoD of the project is the cost of the licence for the older system. If the effects are hard to estimate, the second step is to investigate the cost of alternatives. An example of this is when a feature enables automation of a manual process. The CoD for this process can be estimated to the alternative cost for the feature, which in this case could be equal to the cost of the current manual work.
When the CoD is estimated, the development team is estimating the development time of the feature. The duration could either be estimated in absolute numbers such as weeks or in relative numbers such as story points. The CD3 method is independent on what unit being selected as long as the duration of all the compared features are estimated using the same unit. Once the CoD and the duration of the feature is estimated, the feature is assigned a CD3-score and could be compared to other features in the backlog.
2.3.3 Qualitative Cost of Delay Using Urgency and Value
An alternative solution to WSJF, handling the critique of mixing units, was devel- oped by Arnold (2017), making use of CoD and urgency. He is clear that these two factors are not additive but that they are dependent on each other. The simplest way to illustrate this dependency is by a 3x3 matrix with urgency on the x-axis and value on the y-axis, see Figure 2.5. This is a qualitative approach and the scales for value and urgency are suggested as follows:
Urgency
• ASAP — The highest level of urgency. If we do not deliver this now, the value will decrease or someone else will get there before us.
• Soon — The middle level of urgency. If we do not deliver soon, the value will start to decrease.
• Whenever — The lowest level of urgency. The total value is not really af- fected by delay.
Value
• Killer — The highest level of value. If we do the project, we make a "killing"
and if we do not, we will be "killed". Very few items should reach this level.
• Bonus — The middle level of value. Could be things that we want to tell the customers or even make a press release about.
• "Meh" — The lowest level of value. This is the work that are valuable and worth doing but not the things that get customers to excited.
Figure 2.5: Illustration of Cost of Delay x Urgency (Arnold 2017).
Derived from the matrix, an item in the upper right corner has a high CoD while an item in the lower left corner has a low CoD. Naturally, you begin with tasks from the upper right corner and work your way down to the lower left. However, one important thing to remember is that this scale is dynamic, meaning the perception of value and urgency can change over time. Hence, the matrix should be updated regularly to ensure the right tasks are included in the upcoming sprint. According to Arnold (2017), the method helps to improve prioritisation, enables better decision- making and creates discussions about value.
To get a deeper understanding of how urgency affects the cost of delay, Arnold &
Yüce (2013) have developed three different urgency profiles. These profiles are help- ful to understand the life-cycle of benefits and the effects of late deliveries, which are the two main things to consider when prioritising.
The first urgency profile is for projects with a short life-cycle and where the peak is affected by delay, see Figure 2.6. This is a common case in consumer electronics where the demand quickly ramps up and after a while it becomes standard for everyone and hence the value of the project decreases (Arnold & Yüce 2013).
Figure 2.6: Urgency profile for a short benefit horizon and with a reduced peak due to late delivery (Arnold & Yüce 2013).
The second urgency profile is for projects with a long life-cycle and where the peak is affected by delay, see Figure 2.7. This is often referred to as first mover advantage.
If you are too late to market, it will be difficult to gain market shares. An example of this is PC operating systems, in which the market consists of a few main players (Arnold & Yüce 2013).
Figure 2.7: Urgency profile for projects with a long-life and reduced peak due to late delivery (Arnold & Yüce 2013).
The third urgency profile is for projects with a long life-cycle and where the peak is unaffected by delay, see Figure 2.8. This is the easiest profile to calculate cost of delay. This could be an automation of a process or cost-cutting where the benefit of doing so is the same regardless of when it is done (Arnold & Yüce 2013).
Figure 2.8: Urgency profile for projects with a long-life and a peak unaffected by delay (Arnold & Yüce 2013).
2.3.4 Useful Tools for Calculating Cost of Delay
Besides the established theories regarding CoD, complimentary theories that often are mentioned in the CoD context are included in the literature review. The compli- mentary theories of interest for this study handled enablers and project break down.
These theories are presented in Section 2.3.4.1 and 2.3.4.2.
2.3.4.1 Assigning Cost of Delay to Enablers
Some projects do not deliver direct value to the business, instead they enable for future value capturing (Scaled Agile 2018). These projects are called enabler projects and are often quite difficult to valuate in terms of CoD. This could be projects like transferring an old system written in COBOL into a new platform written in Java or Python, enabling for development of future features. The transfer itself do not add any business value but the system upgrade has enabled for future value capturing. Wright (2018) has developed an approach for assigning value to enablers.
His approach is to assign the enabler a portion of the value it has enabled. The size of the portion is based on the enablers size in comparison to the enabled work. For example, if an enabler takes two months to develop and it has enabled value for a project that takes six months to develop, then the enabler would get 25% of the total value since the total developing time is eight months.
2.3.4.2 Minimal Viable Product
Minimum Viable Product (MVP) is not a prioritisation tool itself. However, the concept of MVP is used in the agile methodology and is an efficient mindset to break down projects into working prototypes. This concept is utilised when break- ing down projects into components in Scrum and SAFe. Consequently, theories regarding MVP are important to increase the understanding of how agile depart- ments break down and prioritise their projects.
MVP was defined by Frank Robinson in 2001 as a concept to obtain a high amount of customer validated learning to a minimal effort (Lenarduzzi & Taibi 2016). The MVP is a product, developed with minimal possible effort, that offers a working so- lution in accordance with the customer needs. This enables the customer to interact with the product and the actual user behaviour can be tracked. The MVP bridge knowledge between the developers and customers to minimise the risk of developing a faulty product. Hence, the risk and cost of a project could be reduced by breaking down projects into MVPs, as new or changed requirements are discovered in an early phase of the project development (Duc & Abrahamsson 2016).
2.4 User-Centered Design
As one part of this study consists of formulating recommendations for how the pri- oritisation model could be improved, it requires continuous interactions with the employees of the case department. Hence, a User-Centered Design approach (UCD) is used to guide this iterative process. Jokela et al. (2003) has identified four general principles characterising a UCD:
• The active involvement of users and a clear understanding of user and task requirements.
• An appropriate allocation of functions between users and technology.
• Iteration of design solutions.
• Multi-disciplinary design.
The UCD approach is based on an iterative mindset, in which the interaction be- tween the researcher and user steers the direction of each iteration (Jokela et al.
2003). The activities of the UCD approach are illustrated in Figure 2.9.
Figure 2.9: Illustration of the UCD workflow.
The iterative part of the model contains of four steps. The first step is to know the user, specify how the product will be used and in which context it will be used. In the second step, the information gathered in the first step is used to formulate user requirements. Step three is to design the actual product based on the requirements from the previous step. Then the solution needs evaluation to test if it meets the user requirements, this is done in the fourth and last step. If the solution does not meet the requirements, the process starts all over again from step one. This iterative cycle is performed together with the users until the designed solution meets the user requirements. (Jokela et al. 2003)
2.5 Situational Analysis
with the UCD approach to understand how a product interact with the user in a real world scenario. The situational analysis helps to assess the user needs and how they are affected by their surroundings. This model is useful for this study as it guides the work to systematically categorise the prioritisation needs of financial software development. The situational analysis helps to understand the context and what needs that are related. The analysis takes a holistic approach and includes both the perspective of the users and other influencing factors. The different perspectives of the analysis are defined by Still & Crane (2017) as following:
• Functional analysis: What is the product expected to do?
• Environmental analysis: In what context will the product be used and how could it be integrated in the user environment?
• Organisational analysis: How is the product design affected by the specifics of the organisation, its budget and target?
• Competitor analysis: What is the unique selling point of the product? Does it exist products competing with yours?
• Materials analysis: How is your product affected by the choice of material?
• Content analysis: What content does the product have to process to satisfy the user needs?
By investigating each perspective of the situation, the researcher is able to under- stand the needs from all the client perspectives. Hence, the researcher is able to complete an assessment of all the clients needs and process them into product re- quirements that satisfies the client (Still & Crane 2017).
2.6 Summary of Literature
As the study intend to understand the prioritisation needs at a case department that is working according to the agile framework Scrum, it is crucial for this study to include the fundamentals of it. Scrum is built around the three roles Scrum Mas- ter, Product Owner and Development Team. These people work together in 2-week sprints and deliver value continuously. The product owner has a central role for this study, as its tasks include backlog prioritisation and maximisation of the value delivery.
Besides understanding the current work method, one must understand in what direc- tion the case department want to change. As the case department aims to scale their agile work with guidance of SAFe, literature about SAFe helps to better understand potential future prioritisation needs. SAFe consists of processes for team, program, large solution and portfolio level. The team level uses the traditional principles of Scrum, working in 2-week sprints. At the program level, SAFe extends the ideas of
Scrum to a higher level, but instead of working in 2-week sprints, the program level works in agile release trains (ART), composed of 5 sprints. Each ART consists of multiple teams making deliveries into the ART.
WSJF builds upon the concept of CoD. CoD indicate how much value leak over a predefined period of time. In other terms, CoD attempts to assess how much value being missed by not having a feature ready when needed. In SAFe, CoD is utilised in prioritisation through the prioritisation model WSJF. However, WSJF have a simpler approach towards CoD and defines it by three components: User-Business Value, Time Criticality and Risk Reduction/Opportunity Enablement. These com- ponents are set by using a Fibonacci scale and the values are relative the other items in the backlog. To calculate the WSJF score these parameters are summarised and then divided by job duration. However, WSJF has received some critique for having relative numbers, having vague definitions of the components of CoD and mixing different units, such as time criticality and risk.
Another method, also built around the concept of CoD, is CD3. This prioritisation approach is an absolute prioritisation approach, meaning that the effects of the fea- ture are estimated to its actual realisable value instead of putting it in relation to other features in the backlog. The CD3-value is calculated by dividing a monetary CoD with the duration of the feature.
Within the theme of CoD, Arnold & Yüce (2013) developed the method CoD x Urgency. This is a relative prioritisation approach built on the correlation between CoD and urgency. The concept is to categorise the value and the urgency of each item and fit them in a 3x3 matrix with urgency on the x-axis and value on the y-axis. Consequently, the items with high priority are placed in the upper right corner and the items with low priority are placed in the lower left corner. To get a better understanding of how urgency affects CoD and to simplify the urgency categorisation, (Arnold & Yüce 2013) developed three types of urgency profiles for the most common types of projects. These urgency profiles illustrate how the CoD of a feature change over time and are useful to have in mind when valuating and prioritising tasks.
The literature study stretched beyond the well-established prioritisation models and included practices that could enhance the performance of a prioritisation model in the financial context. The identified practises handle the assignment of value to enabler features and how work can be broken down into MVP’s. When assigning value to enablers, Wright (2018) suggests a practise that compare the size of the enabler with the size of the project(s) it enables. This ratio is then used to assign the enabler with a portion of the total value equal to its share of the total size.
The MVP mindset challenges the development team to deliver a working product that satisfies the customer needs with minimal effort. By quickly develop a working product, the customer is able to interact with it and changes in the requirements could be identified in an early stage, reducing the risk of extra work due to the
As the investigation of prioritisation needs at the case department is similar to the type of investigation conducted in product development and design, literature in that field of study contributed to the work procedure of the study. The UCD ap- proach is divided into four general principles, where the two initial steps focus on identifying the context and needs of the user. In the third step, the design solution is produced. The design solution is then evaluated against the requirements of the user in the fourth step. This process is iterated until a solution satisfies all the user requirements. Extending from the UCD approach, literature regarding the usage of situational analysis, to evaluate the needs of the users and their surroundings, was investigated. The situational analysis investigates the requirements from the follow- ing perspectives: Functional, Environmental, Organisational, Competitor, Materials and Content. By using the situational analysis, the researcher are able to obtain a broad perspective of the needs and formulate them into requirements of the client (Still & Crane 2017).
Method
This chapter begins with an introduction of the work process and the analytical frame- work used to structure the study. This is followed by the research design and work procedures of the pilot and the main study. Lastly, the data collection and analysis methods are presented followed by a discussion of the reliability, validity and research ethics of the study.
3.1 Model of Analysis
The model of analysis had the purpose to structure the work process and to stan- dardise the analysis. Moreover, the model of analysis included a framework that helped to categorise the prioritisation needs, standardised the evaluation process and enabled a fair comparison of how well different prioritisation models satisfied the needs.
3.1.1 Work Process
The study could be compared with a product development project, where the user needs are identified, converted into requirements and used to develop a product.
Similar to product development, the result of the study was highly dependent on the users´ involvement to identify their needs. Hence, the study was structured similar to a product development process, where a UCD approach was used as a guideline.
The analysis was structured in four phases: analysis of the needs, specification of requirements, development of the model and evaluation of the results. As indicated in Figure 3.1, the research questions were answered in different phases of the UCD approach. Sub RQ1 was answered in the second step of the UCD cycle. In this step the needs of the case department were collected and specified. In first iteration after all needs were collected, WSJF was assessed. This was done by using WSJF as input instead of designing a solution in the third step. This made it possible to evaluate how well WSJF satisfied the requirements in the fourth step, which answered Sub RQ2. As WSJF showed to be insufficient, the UCD iterations continued with rec-
main RQ could be answered upon. By following the UCD structure, the quality of the findings were improved, as an active involvement of the users enhanced their knowledge regarding the research and thereby their ability to contribute with more detailed input.
Figure 3.1: Illustration of how the UCD approach is used to answer the research questions.
3.1.2 Analytical Framework
To structure the analysis of how well WSJF and the recommended modifications served the requirements of prioritisation in financial software development, the per- spectives presented by Still & Crane (2017) were used as a foundation in the struc- turing of an analytical framework. The framework divides the analysis into six different perspectives, which together gives a holistic perspective of the needs of the user and its surrounding. For this study, the analytical framework was tweaked to suit the evaluation of a prioritisation model. The resulting framework is presented in Table 3.1.
Table 3.1: Analytical framework.
Analytical Method Evaluated Prioritisation Model
Functional Yes/No/Partly
Requirement 1 :
Requirement 2 :
Requirement 3 :
Environmental Yes/No/Partly
Requirement 1 :
Requirement 2 :
Requirement 3 :
Organisational Yes/No/Partly
Requirement 1 :
Requirement 2 :
Requirement 3 :
Metrics Yes/No/Partly
Requirement 1 :
Requirement 2 :
Requirement 3 :
Types of projects Yes/No/Partly
Requirement 1 :
Requirement 2 :
Requirement 3 :
For the model to better suit an evaluation of prioritisation models, three modifi- cations were made to the framework. As prioritisation models are often developed and used internally, it is not exposed to the same type of external competition as a many other products might be. This reduces the need of analysing the prioritisa- tion model’s competitive landscape. Hence, as a first modification, the competitor analysis was excluded from the study. The second modification was a changed focus for the material analysis. The material analysis in the original framework focus on the need of tangible material, which is small or non-existent in prioritisation model.
Instead, the material analysis focused on the need of intangible material, which for this study was the need of prioritisation metrics. As the content of a prioritisation model consists of the projects being prioritised in it, it is important to analyse to what extent it can be generalised to suit different types of projects. Consequently, as a third modification, the content analysis focused on analysing what type of projects that can be prioritised in the model.
The requirements were based upon the user needs obtained from the analysis in the first step of the UCD process. The requirements were included in the analytical framework under its corresponding perspective. The requirements were determined continuously throughout study and conclusions were drawn based upon the final version of the analytical framework.
3.2 Research Design
Due to the novel nature of the research area and lack of a precise problem definition, the study had an exploratory research approach. This approach was suitable as it gave flexibility and adaptability to revise the researchable problem throughout the study (Denscombe 2010). By having an exploratory approach, significant findings influenced the direction of the study and increase its scientific relevance (Blomkvist
& Hallin 2015). To allow for a broad perspective of the researched area, an inductive approach was selected where no predefined hypothesis was tested(Denscombe 2010).
By having an inductive approach, the study intended to contribute with a deeper understanding of the research area, rather than resulting in a single defined truth (Neuman 2011).
As the study was conducted at a company with an interest of solving their firm specific problem, the research had an action science approach. Action science have a two-sided research purpose: solving the problem for the client and contributing to science (Collis & Hussey 2013). Hence, the study was conducted in collaboration with the case company to share competences as well as presenting the firm spe- cific findings in a way that is simple to understand for its employees (Gummesson 2000). As an effect, the practices and models are presented in a simple manner for a maximum chance of a successful implementation. To achieve understandable results, the study had a participative inquiry approach when formulating suggested improvements for the resulting model. The participative inquiry approach involves the participants in the research and thus enables them, in cooperation with the re- searchers, to steer the research in a direction where it becomes beneficial both for the firm and science (Traylen 1994).
The study was divided into a pilot and a main study. The purpose of the pilot study was to clarify the research problem as well as increase the understanding of the current prioritisation process and strategic direction of the case department.
The pilot study had a qualitative approach and sourced information from unstruc- tured interviews, an observation and literature. Empirical data provided insights specific for the case company, while the literature provided a general understanding of scientific field of prioritisation. The purpose of the main study was to understand the prioritisation needs of the department, assess the suitability of WSJF and for- mulate recommendations of how the model could be adapted to better satisfy the needs. The main study employed a focused literature review and semi-structured interviews, in which the findings from the pilot study helped to formulate themes.
The data analysis was structured according to a data analysis procedure developed by Miles & Huberman (1994). This procedure guided the analysis through the three steps of data reduction, data displays and conclusion and verification. The proce- dure was suitable for this study as it allowed for data analysis of multiple types of qualitative data, as the study utilised data collected from interviews, literature and an observation. To ensure the validity of the collected data, it was validated by triangulation where data from the different sources were compared.
The study was conducted from the beginning of January 2019 to early June 2019 as a Master thesis for KTH, Royal Institute of Technology, and a major Swedish bank.
The workflow of the study is illustrated in Figure 3.2.
Figure 3.2: Workflow of the study.
3.3 Pilot Study
As the prioritisation model originally used by the case company was developed in- house, the experience of using it was tied to a few key stakeholders. Hence, the initial phase of the study consisted of mapping these stakeholders and understand how the prioritisation model was put to practise. Furthermore, the target for the prioritisation work was vague, hence a pilot study was needed to define the prioriti- sation needs of the department and understand the future target for prioritisation.
The initial phase of the pilot study consisted of interviews with key stakeholders and a literature study, including literature regarding different prioritisation approaches and agile methodologies. Additionally, documentation of internal processes provided by the case company was examined. The stakeholders and internal documentation helped to explain the internal prioritisation processes and their experience of using them. The literature provided general knowledge in the scientific field and func- tioned as a foundation for the empirical part of the pilot study. Furthermore, the pilot study provided useful information to narrow down the research problem and functioned as a foundation from where the research questions and purpose could be constructed.
The interviews with the stakeholders of the case department revealed that the case company is in a transition process towards implementing SAFe and some depart- ments of the company were already working in accordance with SAFe. The case department wanted to investigate if WSJF was suitable for their needs. To adapt the study for this finding, SAFe literature was included in the literature review and an observation of one department using WSJF was conducted. The observation was
department.
The initial interviewees for the pilot study had the positions of Head of Business De- velopment and Head of Quantitative Development. They were selected due to their knowledge within the existing prioritisation processes and the banks overall strategy.
Additionally, they had a wide network of contacts and could therefore recommend other persons with relevant knowledge. To understand the strategic direction of the bank, the Program Manager for developing the future operational strategies was interviewed. The Agile Coach and Metric Responsible are working at a department that is using WSJF for prioritisation and could therefore contribute with experience and knowledge from using WSJF. To better understand the prioritisation needs of the case department, interviewees with a practical experience of prioritising work were needed. Hence, a Product Owner and a Chief Backlog Owner from the case department were interviewed. Information about the interviews are summarised in Table 3.2. In addition to contributing with information, the interviews from the pilot study were a powerful tool in the selection of interviewees for the main study, as the interviews generated recommendations of other knowledgeable people that could be interviewed.
Table 3.2: Interviews and the observation conducted in the pilot study.
Title Code Date Duration
Head of Quantitative Development A 2019-01-18 2 h
Agile Coach B 2019-01-25 45 min
Metric Responsible C 2019-01-31 50 min
Program Manager, Central Operational Coordination D 2019-02-06 1 h 20 min
Chief Backlog Owner E 2019-02-06 45 min
Product Owner F 2019-02-07 45 min
Head of Business Development G 2019-02-12 45 min
Observation of PI Planning Event H 2019-01-23 2 full days
3.4 Main Study
With the pilot study as a foundation, the main study had a focused approach on investigating the expressed needs of prioritisation at the case department. The main study was divided into four phases: identify and understand the prioritisa- tion needs, formulate prioritisation requirements, compare and evaluate how well WSJF satisfies the needs and requirements and if phase three indicates a mismatch, make recommendations for how the prioritisation model could better satisfy the requirements. The purpose of the first phase was to identify what prioritisation measures, roles and practises that satisfied the needs of the case department. With the needs identified, the analytical framework could be populated with requirements and the suitability of WSJF at the case department could be assessed. This was
done by theoretically assessing how many of the requirements that could be satisfied by using WSJF. The theoretical assessment utilised a combination of theories from literature and empirical data to compare the capabilities of the investigated model with the requirements defined in the analytical framework. As WSJF showed to be deficient in satisfying the requirements of the case department, recommendations of modifications were formulated. The recommendations were formulated through a qualitative analysis where the unsatisfied requirements formed a lens, from which the literature and empirical data were viewed. With the unsatisfied requirements, empirical data and literature as input, the recommendations for how to modify the prioritisation approach to better satisfy the requirements were developed through discussions between the researchers and iterative prototyping with input from the employees at the case department.
The information sources for the main study were a focused literature review and semi-structured interviews at the case department. As a general literature study was conducted in the pilot study, specific literature related to the research ques- tions were examined in the main study. This included literature focusing on the concept of CoD, prioritisation models utilising CoD and prioritisation models used with SAFe. Moreover, the literature review sourced information and tools used to structure the study, including analytical frameworks and work procedures. However, there are only a few publications investigating prioritisation methods in scaled agile organisations and none of them were conducted in a financial software development context. Thus, the study had to largely rely on primary empirical data gathered from the interviews.
From the pilot study, the employees affected by the prioritisation process could be divided into three groups. These groups included employees: working based on prioritisation outcome, working with prioritisation or deciding the strategic target of prioritisation. Besides the three groups, employees working with regulations or technical enablers were highlighted as important for the study as they experienced problems to estimate the value of their deliveries. To get a coverage from multiple stakeholders, semi-structured interviews were conducted with a mix of employees from different parts of the organisation.
To get insight of the prioritisation need from the employees working based on the prioritisation outcome, a Scrum Master for a development team at the case depart- ment was interviewed. The Scrum Master is working close to the team and therefore has knowledge of the prioritisation needs of the daily operations. To obtain the pri- oritisation needs of the employees working with prioritisation, a Backlog Owner and a Product Owner were interviewed. These interviewees are working with prioriti- sation on a daily basis and have knowledge of the current process. Furthermore, they contributed with requirements that were favourable but not satisfied by the current prioritisation model. The Head of Quantitative Development, Head of Busi- ness Development and the Program Manager were interviewed to gather information regarding the strategic target of prioritisation at the case company. The Program
