Built-in software quality in Agile development
Mehdi Saleh
Subject: Information Systems Corresponds to: 30 hp
Presented: VT 2018
Supervisor: Görkem Pacaci
Department of Informatics and Media
1
Abstract
During the past decade, the amount of software code in automotive industry has been increased annually by millions lines of codes.
(Businessinsider.com, 2018) This has forced many automotive OEMs to perform an organizational transition, being more software focused rather than experts within hardware development. Thus, software development methodologies and practices become essential to maintain a successful organizational switch. Waterfall and Agile are the two most popular software development methodologies. Waterfall as the traditional one, is a progressive method. The progress flows in one direction, and upon the completion of previous process. For example, user requirement must be frozen before development, and software test only happens after compete development. Waterfall can be referred as long-plan-driven methodology. Agile is another software development methodology with more iterative approach and possible change of requirements; with incremental delivery. Volvo Cars has experienced software development transformation, from traditional waterfall to agile development.
Agile introduces freedom to requirement change and iterative delivery.
However, such a liberty should not disrate software quality. Especially in automotive industry that deals with human safety and security. In such an iterative environment, there is a higher risk to compromise on quality checks before each delivery, due to short intense lead times.
There will not be enough time for intensive quality assurance activities after each iteration. However, the OEM still requires high quality software and cannot sacrifice quality for more frequent deliveries, as
2
this could result in unexpected penalty costs and failure to satisfy legal requirements. The solution to downsize the intensive quality check after each iteration, is to improve development quality during the development and building quality into development processes. This is what we refer as “Built-in quality”; the quality that is built during software artefacts development on a continuous basis.
Built-in quality can be a huge challenge for organizations with developers from different background and quality mind-set, who are benefiting from different quality methods and tools.
This study conducted at Volvo Cars during Agile transformation time and the main objective was to connect and emphasize the importance of built-in quality in agile software development. In this study we look at existing challenges that decrease quality of software artifacts during the development. Thus, by prevailing those challenges we can improve the software quality during iteration delivery. Such an improvement decreases the amount of intensive quality check after each iteration. Additionally we look at guidelines and tools that used by different development teams to improve software artifacts’ quality.
We also investigate how quality assurance engineer can support built- in quality during agile transformation.
Keywords
Built-in quality, Agile development, SAFe, Just in time reporting, Building quality in code, building quality in requirement specification
3
Table of content
1. Introduction ... 4
1.1. Background ... 4
1.1.1. Traditional quality assurance activities ... 9
1.1.2. Agile development and SAFe framework ... 9
1.2. Problem statement ... 11
1.3. Research questions ... 11
1.4. Delimitation ... 12
1.5. Outline ... 12
2. Theoretical framework ... 14
2.1. Quality, Internal Quality and Built-in software quality ... 14
2.2. Agile way of working ... 15
2.3. Built-in quality method and tools ... 17
2.3.1. Built-in quality into requirement ... 17
2.3.2. Built-in quality into code ... 20
3. Study design ... 23
3.1. Research method ... 23
4. Data collection ... 24
4.1. Strategy ... 24
4.2. Questionnaire ... 24
4.2.1. Demographics ... 24
4.2.2. Main questions ... 24
4.3. Collected Data ... 26
5. Findings and conclusions ... 32
5.1. Summary of the findings and conclusion ... 32
5.2. Future work ... 35
References ... 36
Appendix ... 39
A Appendix: Software Development Standards ... 39
B Appendix : Quality assurance documents ... 40
4
1. Introduction
This chapter discusses the background and the purpose of the study.
It describes the importance and the relation between agile development and built-in quality. It defines the problem area along with the related research questions. The delimitations of the thesis are also be presented and an outline is listed at the end.
1.1. Background
At the beginning let’s look at the definition of “Software” in automotive industry and how it works within different hardware components.
An Electronic Control Unit (ECU) is an embedded system in automotive electronics that controls one or more electrical systems or subsystems in the vehicle; Like Engine Control Module (ECM), Brake Control Module (BCM) or Central System Display (CSD). An ECU includes both hardware and software.
“Software” in automotive, is the embedded application that runs on every ECU in the vehicle.
But why does the OEM demand on agile software development methodology? The answer to this question is directly connected to highly competitive market and customers’ expectation.
Customers frequently request for better features and functions.
Customers’ needs may also change during the development period.
Apart from these, technology also changes frequently with much faster pace than old days. To meet customers’ expectations and being the brand of choice in a highly competitive automotive market, a software
5
development methodology is needed with shorter release and delivery time for new functions and features.
Previously, Waterfall methodology was broadly used for software development and has been used at Volvo Cars. Figure 1 shows waterfall software process model (Ajmer Rajasthan, 2013) “Its name is derived from structural specification. Every phase comes after a phase is completed and tasks can be divided according to phases. The output of one phase becomes input of next phase but we have the option to revisit phases in the next cycle.” (Ajmer Rajasthan, 2013)
In waterfall, quality assurance activities are performed after completion of each phase. Such as joint reviews between software supplier and Volvo Cars quality assurance team.
Figure 1 - Iterative waterfall software process model (Ajmer R, 2013)
6
In past decade, Agile as incremental software development methodology has been introduced and commonly used in different industries. Agile is based on iterative deliveries with focus on customer’s need and freedom to change. In such a development, customer may change or modify the requirements. (Figure 2)
This methodology enables the OEM to deliver new features and functions more frequently. However, in Agile, it will not be possible to fully assess the product quality in every fixed interval with lengthy and time-consuming legacy quality assessment activities. In that case
Figure 2 - Incremental software process model ((Ajmer R, 2013)
7
the quality activities will lag behind the deliveries and become ineffective. (Figure 3) Activities like joint reviews, comprehensive documentation and complete risk assessment prior to development.
To overcome with this challenge in agile development, the majority of quality assurance activities should be integrated (built-in) with the development activities.
Development team is responsible for the quality and building quality into the product from early stage of development and quality must be a non-segregated element from development activities and assured within software artefact continuously. These artefacts may refer as requirement specification, architect, design, code and test.
When discussing built-in quality, one need to emphasize the importance of automated feedback system. These systems are used to measure the quality of the software artefact and provide with instant
Figure 3 - QA activities; Waterfall vs Agile
8
reporting during the development. The developer can instantly update his/her work product to achieve the desire quality of work.
For example, deep nesting is one of the characteristics of code complexity. The automated reporting system measures code nesting and provides with instant reporting. If nesting was above that maximum threshold (based on company quality policy), the developer should refactor and improve his/her software code.
Volvo Cars has already started the agile transformation journey and adaptation to agile software development methodology.
9
1.1.1. Traditional quality assurance activities
At Volvo Cars, majority of software development are outsourced to external suppliers. To ensure supplier quality of work, Volvo Cars requested certain standards certification from the supplier, such as ISO/IEC 12207 (Appendix A.1) and Automotive SPICE (Appendix A.2).
Volvo Cars also requested the supplier to prepare and present several documents during four software joint reviews. In these software joint reviews, the supplier represents documentation related to design, requirement traceability, test plan and quality assurance plan. The primary intention was to qualify supplier’s development process.
Appendix B is the list of these documents and the purpose of them.
1.1.2. Agile development and SAFe framework
Volvo Cars has introduced Scalable Agile Framework (SAFe) for software development.
Full Safe in figure 4 (SAFe for Lean Enterprise, 2018) demonstrates different levels in SAFe framework. Developers (suppliers or in-house development team) are primarily working on team level. They can use different agile methodologies such as Scrum or XP. In a large enterprise environment like Volvo Cars, often the work of several teams can be integrated to an Agile Release Train (ART). ART is part of a large solution that is ideally delivers a specific function to large solution. “The Agile Release Train (ART) is a long-lived team of agile teams which, along with other stakeholders, develops and delivers solutions incrementally, using a series of fixed-length Iterations within a Program Increment (PI) time box. The ART aligns teams to a common business and technology mission. Each ART is a virtual
10
organization (50 – 125 people) that plans, commits, and executes together. ARTs are organized around the enterprise’s significant Value Streams and exist solely to realize the promise of that value by building Solutions that deliver benefit to the end user.”(Agile Release Train - SAFe f for Lean Enterprise, 2018)
For large solution with several product streams, SAFe framework supports organization to prevail challenges like short lead time and continues integration with sustainable deliveries on team, ARTs (Agile Release Train) or program level.
Figure 4 – Full SAfe (www.scaledagileframework.com)
11
1.2. Problem statement
Volvo Cars is moving toward Agile and significant increase of in-house software development, while maintaining supplier deliveries. Volvo Cars decided to use SAFe development framework. Traditionally, software quality assurance had risk of degrading processes and late design modifications, considering that majority of software in automotive industry are safety-critical. In order to mitigate this risk, quality assurance is desired to be an integrated activity, where every engineer is responsible for their product quality with the help of quality assurance engineer. Such a mindset allows building quality into the software product all the time thus mitigating the aforementioned risk. However, relevant methods are needed to support this mind set, e. g. automated feedback systems, systematic educational practices, etc.
There are different methods and automated reporting tools available to support built-in quality. The aim of this study is to investigate the existing challenges to build software quality into the product on continues basis and what methods and tools there are for such a support and to determine how quality assurance engineer can support to improve it.
1.3.
Research questions
Below are the research questions in order to serve the purpose of this study.
12
What are the existing challenges and activities related to building software quality into the product on a continuous basis within Volvo Cars electronic and software department?
What are the existing guidelines and automated software quality measurement systems used in Volvo Cars electronic and software department?
How quality assurance engineer can support and improve built-in quality in large enterprise agile environment?
1.4. Delimitation
This is a case study with focus on company built-in quality methods and tools for following software artefacts.
1- Requirement specification 2- Code development
This report will not cover architect, design, test and documentation.
Apart from that, the study will not suggest the best practise for Volvo Cars.
1.5. Outline
This dissertation contains a total of five chapters, and it has been organised in the following way. The first chapter provides a general view of research study and problem statement among with research questions. Chapter two is a brief explanation of software built-in quality and agile transformation in a theoretical framework. Chapter three describes the research methodology. Data collection and
13
questionnaire displayed in chapter four and at the end; chapter five is the place for discussions and conclusions.
14
2. Theoretical framework
This chapter explains the concept of built-in quality. In addition, we will look at some methods and tools for automated feedback system for improving software artefact quality.
2.1. Quality, Internal Quality and Built-in software quality
There are several definitions for Quality. Here we look at few of them. “The degree to which a set of inherent characteristics fulfils requirements.” (SS-EN ISO 9000:2000)
“Fitness for use” (Joseph Juran, 1953)
“Conformance to requirements (requirements meaning both the product and the customer's requirements)” (Philips Crosby, 1979)
“Quality of a product (part of service) is the ability to satisfy and preferably exceed the customer needs and expectation” (Bergman Klefsjö, 2004)
Quality means improving services, products, processes and systems, to assure that the organization products or services are fault free and satisfy customers’ need.
Internal Quality in software development is the degree of the design excellence of software artefacts that comprise the software product.
Built-In Quality refers to the practice when the internal quality of software is built-in the software development artefacts on a continuous basis with help of automated measurement systems and guidelines.
Crucially, internal quality of software (figure 5) has a decisive impact on product quality. The benefits of built-in quality practice are: (V.
Antinyan, etal. ,2017)
15
Significantly reduced number of defects
Minimized risk of late design modification
250-500 % reduced maintenance time
2.2. Agile way of working
Agile Software development methodology focuses on iterative and incremental deliveries. Unlike long-plan-driven development, requirements are not fixed and frozen before the development.
Customer may come with new requirement(s) during the development.
Customer is one of the main stakeholders and customer collaboration is one of the essential elements of agile manifesto. Agile manifesto emphasizes on following statements. (The Agile Manifesto, 2001)
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
Figure 5 – Internal quality ISO/IEC 9126-1:2000
16
Agile methodology emphasizes on Empowered team. A self-organized knowledgeable team that is responsible and capable of high quality product delivery.
As stated earlier, Agile proved to be a successful methodology for software development. Faster delivery in shorter iteration, delivering customer value on every
iteration, with the possibility to change of requirement; increase customer satisfaction.
Figure 6 is a comparison between agile and waterfall methodologies.
In Agile, developers are within an empowered, self-organized team
without traditional management and fixed deadlines. Agile team is responsible for software quality and there will not be a dedicated separate team looking after quality and quality assurance activities.
Many software companies found agile methodology as a suitable way of working with better performance and moving toward this change.
To build a quality product, every software product artefact, including requirement specification, architect, model, code, test and documentation, must be well qualified during the development. For example writing bad requirement may lead to wrong architect and wrong code. Necessary methods and tools are needed to support this success.
Figure 6 – Agile vs Waterfall
17
2.3. Built-in quality method and tools
This section refers to previous studies and findings around built-in quality into requirements writing and code development. And then looks at existing tools for measuring these two development artefacts.
2.3.1. Built-in quality in requirement
Unclear requirement specification can be misinterpreted and thus will be the cause of incorrect design and late design modification (Kamata and Tamai, 2007). But what is a good or bad requirement? And is there any measurement to measure the quality of software requirement specification?
Detail study to answer those questions is not the objective of this thesis. However, the answer is yes; there are differences between good and bad requirement specification. There are several studies and papers for building quality into software requirement specification and there are several frameworks to measure it.
Below is the list of important factors for building quality into requirement specification.
Purpose
Is the purpose for the requirement clearly stated?
Does the requirement add value?
Complete
Does the requirement contain all relevant information?
Including use cases, modes etc.
Are all requirements simple and one at a time (atomic)?
18
Are all prerequisites clearly defined?
Clear
Does the requirement contain enough information? (Boolean expressions like AND, OR, should be kept as simple as possible) It is recommended that all statements written in positive terms.
Does each requirement state WHAT the system SHALL do and under which conditions?
Interpretation of the requirement must be free of contradictions.
Verifiable
It should be possible to test all requirements through system demonstration, inspection and analysis.
Does the requirement define what quantity shall be measured and in what unit?
Traceable
Can the requirement be traced back to the requirement it originates from?
Requirement should have a unique ID.
Requirement vs Implementation
Is the requirement as free of solutions as possible?
(Focus on WHAT the system shall do. Avoid describing HOW the system will do something)
Is the requirement feasible / realistic to achieve in terms of Cost, implementation & added value?
The question is how to measure textual requirements quality. Briand et al. (1996) proposed a set of internal quality properties for software artefacts. An overview presented in Figure 7
19
In the left side, there are measure aspects including complexity, coupling and size, and the right side is the list of measures to quantify those aspects. For example large number of conjunctions and/or vague phrases indicates the high complexity of the requirement.
Below table (Table 1) presents internal properties of textual requirement and definitions and measures with method for the measuring of given measure (V Antinyan et al., 2016)
Figure 7 – Requirements internal quality measurement model
20
Some companies like Volvo Cars developed their own internal system to provide with just-in-time feedback to requirement writer, based on above or similar methods. These kinds of methods empowers the systems responsible to get instant feedback while specifying the requirement.
2.3.2. Built-in quality in code
There are intensive studies around code complexity in past decade as it was the main characteristic of code internal quality. Code complexity is the major aspect of internal quality with direct effect on code maintainability (Banker et al. 1993)
Table 1 – Measures and measurement method
21
Nesting depth, control statement, misleading comments, multi developers, multi-tasking, frequent changes and complex
requirements, are among of important code characteristics that influence code complexity. There are several tools and methods to measure the complexity and provide with just-in-time reporting.
Figure 8 is the result of a code complexity study with 100 participants from seven companies and two universities. (Antinyan V et al., 2017) The figure represents the influence of code characteristics on complexity.
The order from left to right shows the important of influence factor in code complexity in this study.
Figure 8 – influence of code characteristics on complexity. (Antinyan V et al., 2017)
22
There are several metrics and measurement systems that can give automated feedback on software complexity; like SonarQube, Pylint and Pycodestyle.
23
3. Study design
This chapter is rather brief and is to explain the approach to conduct the research at Volvo Cars, to be able answering the research questions, presented in chapter 1.3
3.1. Research method
The research is a case study and quantitative approach. The objective was to find the challenges and existing methods within the organization. This case study is around existing quality process and prediction cannot be made. The interviews conducted were centered on built-in quality to get a detailed understanding of the participants’
sentiments. The questions asked were open ended allowing for the possibility to explore and dig further into the interviewee’s responses.
“Open-ended questions allow the respondent to express an opinion without being influenced by the researcher.” (Foddy, W, 1992)
Interviews conducted with eight key engineers. Thematic analysis was used to generate patterns from individual ideas and interviewees practices. Those patterns were prioritized and marked based on number of interviewees with similar idea and then used for analyses.
In the end of this report the findings from Volvo Cars will be discussed and whether the research questions were answered or not.
24
4. Data collection
This chapter represents the strategy and questioners that I used during interviews for data collection.
4.1. Strategy
They strategy is to conduct total of eight interviews for requirements writing and source code development. It is a semi-structured interview with open-ended questions.
4.2. Questionnaire
The interview consists of three sections and planned for 30-45 minutes.
4.2.1. Demographics
The first part of the interview investigated the participant demographics. Three fields were given for demographics information.
1- What is your role?
2- Can you briefly state the main activities of your role?
3- How many years of experience do you have in your job?
4.2.2. Main questions
The second and main part the interview structured in a way to achieve following objectives that are directly link to research questions.
Main challenges and activities to increase built-in quality of source code or requirement.
Existing standard and guidelines to improve quality.
25
Metrics and measures for built-in quality.
To investigate above mentioned objective, following interview questionnaire was designed and used.
1- Can you please describe the main challenges that decrease the quality of Software artefact?
2- What activities do you have that intend to increase the quality of Software artefact?
3- How often do you perform them?
4- How much do you think it helps to increase code quality?
5- What guidelines and standards do you use for improving Software artefact quality?
6- What metrics or measurement systems do you have that can give automated feedback on Software artefact quality?
7- How much do you thinks it helps?
8- What decision criteria are defined based on measurements to trigger corrective actions?
9- What do you think is missing from the current state of measures, guidelines, and tools?
Above nine questions are directly link to research problem and allow us to draw patterns to answer the research questions.
4.2.3. Invitees and data collection
The interviewees have been selected by departments’ managers and the interviews conducted in person and one to one meeting. Data collected by interviews during interview and compiled after completion of all interviews.
26
4.3. Collected Data
Total eight interviews conducted from eight participants including five developers that were responsible for software source code development and three system responsible who were in charge of software requirements specification. As mentioned in delamination chapters, the study was only around two software product artefacts.
Requirement specification
Source code development
The demographic result in table 2 shows that the majority of the interviewees had more than 6 years’ experiences in their field of working. Due to nature of the business, 3 out of 5 developers were model developers rather than source code programmer. They generate models based on requirements specification and benefit from auto generation code tool.
Role Main activity Experience
Software developer (5) Code developer (2) <3 years (1) System responsible (3) Model developer (3) 3-5 years (1)
Requirement specification
(3) 6-10 years (4)
>10 years (2)
Table 2 – Demographic information
27
Below Table 3 is the list of main challenges that decrease the quality of software artefact. (Second question of questionnaire)
Main Challenges Description
Uncertain, complex and unclear
requirement (3)
Lack of communication to know how the function and
interfaces work with other requirements and providing crisp information to next level. Next level can be requirement realization or software architect.
If the requirement is very high level and not broken, supplier may understand it differently.
Lack of good test platform (2)
Matlab is not adequate for test development and has lack of effectiveness to find defects. Hard to mock in Matlab to find problems.
Lack of Continues Integration (CI) machinery.
Delivery vs qualified delivery (2)
It is more about delivery than quality with fix deadline, budget and content. Quality is the only thing that you can sacrifice. In general, focus is on faster deliveries rather than qualified deliveries. It is a development cultural issue.
Unclear systems solution and interfaces(1)
Unclear system solution and interfaces between signals (in signal database). High complexity and coupling between ECUs because of the old functions. This is more an architect issue that may change in future platforms. However,
currently unclear requirement and complex ECUs coupling is a challenge for good quality.
Personal rotation and staff turn over (1)
Occasional team member recruitments that requires large effort for training and communication. Every time a new person joins the team, time and effort is needed to adopt to the new environment, company culture and processes..
Tools dependencies (1)
Dependencies of tools create a complex development tool chain. This complexity can result in certain tools not working when a new operating system or other tools are introduced. Bugs and problems can be introduced due to updates of these tools or operating systems.
Lack of software development knowledge (1)
Many testers are expected to write code for automating their tests. But these testers traditionally were not coders.
This is a car manufacturer company and many developers came from mechanical background with limited experience of software development.
Hardware performance (1)
Windows 7 workstation performance are not fit for Software development. This problem make developers to take
shortcuts. To avoid writing complete test and even to think how to write a test that take shorter run time and skip pushing suite.
Table 3 – Main challenges to decrease the quality of software artefact
28
Activities that intend to increase the quality of software artefact is shown in Table 4. The second column is the total number of positive responses and the third column is the total number of interviewees that the activity was applicable to their working software artefact.
They perform those activities whenever there is new requirement or software release. Whenever new code is going to be merged. At least once in every sprint. A sprint is the development iteration and usually two weeks long.
All the interviewees believed those activities helps in different ways. They mentioned following statements.
It will be much easier for developers to find issues early before integration to complete software.
It is good but there is a lot of room to make it better.
It helps a lot in a long run, keeps the high maintainability.
Learning from each other and correcting each other mistakes
Activities Number Total
number of interviewee
Peer review - design review 8 8
Statistic code analyser 2 4
Unit test 3 4
Visualizing testing chain 1 4
Integration test 2 4
Test coverage 3 4
Knowledge transfer within the team 1 4 Table 4 – Activities that intend to increase the quality of software
artefact
29
Next question was about guideline and standards that they use for improving software artefact quality. Below table 5 is the list of interviewees’ feedbacks. In this table the first column is the guidelines and standards that one or more interviewees mentioned during the interviews and second column is the number of interviewees who benefit from this guideline and standard.
Guideline and Standards Number of interviewees Volvo Cars guidelines for creating simulating
model 3
Internal coding guidelines 2
Awesome framework internal guidelines. This is
an internal build guideline 1
Internal Wiki modelling guidelines 1
Review template for filling review records.
Spelling error, logical error and et 1 Table 5 –Guideline and Standards
30
Table 6 is the list of metrics and measurement systems that can give automated feedback on software quality artefact. The column two is the number of interviewees that use related tool.
Tool name Number of
interviewees
Cyclomatic complexity for code 3
SonarQube for code static analysis 1
Pylint and Pycodestyle for static analysis that gives automatic fast feedback for code violations, writing
styles, and nesting depth. 1
Code coverage tool for unit test. Pass/fail for unit test 1
In-house developed tool to indicate which models have
been reviewed or when are they going to be reviewed. 1 Table 6 – Metrics and measurement systems
31
Table 7 shows the list of items that are missing from the current state of measures, guidelines, and tools. Second column indicates number of interviewees who mentioned the missing item during interviews.
Items Number of
interviewees
Measures and guidelines 4
Continues integration complete platform 2
User guide documentation for tools and benefits 1
Developers training 1
Table 7 – What is missing from current state of measures, guidelines and tools
32
5. Findings and conclusions
This chapter is the summery of finding and conclusions. At the end it presents the future work.
5.1. Summary of the findings and conclusion
All interviews were open discussions for open equations and findings, to allow the interviewees to discuss any uncovered area. All of interviewees were well experienced in their field of expertise. Built-in quality was a very interesting subject for all, especially during agile transformation at Volvo Cars.
The first and important observation was the finding in main challenges. The expectation was to find more about code complexity characteristics and technical subjects. Although some of the main challenges were technical challenges, but majority of them were around communication and business non-technical concerns. Some of the points like “delivery vs qualified delivery” was more personal and team dependent.
However, this is an improving opportunity for Volvo Cars during agile transformation. Because communication and self-organized team is one of the agile essentials. Such conclusion proves that agile transformation was the right decision for Volvo Cars to overcome with these challenges. It was interesting to see how hardware performance can challenge software product quality. It shows that quality engineers must look at bigger picture to be able to help more efficiently. It should not be only about code and requirement, it is about overall software quality life cycle. More communication with
33
other stakeholders including higher management can resolve some business-related challenges like personal rotation.
Based on answers to second question, almost non-of the suggested activities that intend to increase the quality of the software artifact are directly related to exiting challenges in fist question. Table 8 shows the relation between challenges and activities suggested by each interviews.
Once again, agile is an opportunity to elaborate on sharing best practices within the different units and teams.
Main Challenges Activities to increase the quality of the software Uncertain, complex and
unclear requirement Peer review -
design review Unit test Lack of good test
platform Peer review -
design review
Knowledge transfer within
the team
Delivery vs qualitied
delivery Peer review -
design review Statistic code
analyser Unit test Unclear systems solution
and interfaces Peer review -
design review Unit test Personal rotation and
staff turn over
Tools dependencies Peer review -
design review Visualizing
testing chain Lack of software
development knowledge
Hardware performance
Unit test isolated from main
development machine
Integration test Statistic code analyser Table 8 – Relation between challenges and activities
34
Another finding from this study is that the quality assurance engineer can support and possibly improve built-in quality at Volvo Cars. This can be achieved by meeting development team and understanding their challenges and activities. It may not be the same in different organizations. Hence, quality assurance engineer should not assume that all companies operate in same way with similar challenges.
For guidelines and standards, the study indicates that large enterprise solution with large number of development teams, requires centralized guidelines and standards. This action can save lot of time and money, and to some extent overcomes with staff turnover challenge. New or transferred developer can be trained easier and best practices can be discussed between teams.
When we use different metrics and measurements systems for automated feedback on software artefact quality, time and energy are needed for mastering these tools. It has been observed that lack of awareness on guidelines and systems, leads to poor quality of software artefact. In case of centralized quality guidelines, measurement systems and standards, quality assurance engineer can master the tools with focus of selected guideline and standard. In this case training and empowering with best practices can be more effective and easier.
35
5.2. Future work
The first step was to study limited number of developers. However, this study can be much more efficient if we increase number of interviewees and decode more findings. We can then eliminate top challenges by providing related activities. More interviews are needed to understand existing guidelines and measurement systems. Then providing with best practices for everyone in development team.
Secondly, we need to expand the study to other software artefacts, including architecture, design, test and documentation.
36
References
Ajmer, Rajasthan (2013) A Comparative Study between Iterative Waterfall and Incremental Software Development Life Cycle Model for Optimizing the Resources Using Computer Simulation, 978-81- 920249-7-4/13/ 2013 IEEE, pp. 189
Antinyan V, et al. (2017) Empirical Software Engineering, https://link.springer.com/article/10.1007/s10664-017-9508-2
[Accessed 15 May 2018].
Antinyan V et al. (2016) A method for automated reviews of textual requirements, Elsevier
Antinyan V et al. (2017) Evaluating code complexity triggers, use of complexity measures and the influence of code complexity on maintenance time. DOI 10.1007/s10664-017-9508-2
Banker RD et al. (1993) Software complexity and maintenance costs.
Commun ACM 36(11):81– 95
Bergman Klefsjö (2004) Quality from Customer Needs to Customer Satisfaction, Studentlitteratur AB, ISBN-10: 9144041667
Briand, L.C. , Morasca, S. , Basili, V.R. , 1996. Property-based software engineering measurement. IEEE Trans. Softw. Eng. 22, 68–86 .
Businessinsider.com (2018),
37
<https://www.businessinsider.com/how-many-lines-of-code-it-takes- to-run-different-software-2017-2?r=US&IR=T&IR=T> [Accessed 25 August 2018].
Foddy, W, 1992, Constructing Questions for Interviews and
Questionnaires: Theory and Practice in Social Research. Cambridge:
Cambridge University Press.
IEC-15504 (2014),
<https://en.wikipedia.org/wiki/ISO/IEC_15504#cite_note-1> [Accessed 23 March 2018].
ISO/IEC/IEEE 12207:2017. Standards catalogue. International Organization for Standardization. November 2017. Retrieved 21 June 2018.
Kamata, M.I. , Tamai, T. , 2007. How does requirements quality relate to project success or failure? In: Requirements Engineering Conference, 2007. RE’07. 15th IEEE International, pp. 69–78 .
Philips Crosby (1979) Quality is Free, New York: McGraw- Hill. ISBN 0-07-014512-1.
R. Lawrence (2006) Three-way cultural change: introducing agile within two non-agile companies and a non-agile methodology, IEEE 0- 7695-2562-8
38
SAFe f for Lean Enterprise (2018)
<https://www.scaledagileframework.com/#> [Accessed 25 April 2018].
Agile Release Train - SAFe f for Lean Enterprise (2018)
<https://www.scaledagileframework.com/agile-release-train/>
[Accessed 15 August 2018].
The Agile Manifesto (2001) < http://agilemanifesto.org/>[Accessed 15 August 2018].
39
Appendix
A Appendix: Software Development Standards A.1 ISO/IEC 12207
ISO/IEC 12207 “establishes a set of processes for managing the lifecycle of software, including processes and activities applied during the acquisition, supply, configuration, operation, maintenance, and disposal of the software system, with each process having an associated set of outcomes. The standard, however, does not prescribe a specific software life cycle model, development methodology, method, modelling approach, or technique." (ISO/IEC/IEEE 12207:2017)
A.2 SO/IEC 15504 - SPICE
“SO/IEC 15504 Information technology – Process assessment, also termed Software Process Improvement and Capability Determination (SPICE), is a set of technical standards documents for the computer software development process and related business management functions. It is one of the joint International Organization for Standardization (ISO) and International Electrotechnical Commission (IEC) standards, which was developed by the ISO and IEC joint subcommittee, ISO/IEC JTC 1/SC 7.” (IEC-15504, 2014)
40
B Appendix : Quality assurance documents
1- Software Development Plan (SWDP) to demonstrate scope, resources aligned with management and how team looks like. If supplier used new/reused modules, software lifecycle and it activities, company processes in how to work with Software and how to maintain Software that is in production. First doc to go to when introducing new team members
2- Software Risk Management. How the supplier collect risk information? How often review them to mitigate them? And which ones are shared with Volvo Cars
3- Software Development Schedule (SWDS) Overall planning for the activities mentioned in SWDP
4- Software Progress Report (SWPR) What was done and where are we heading. Milestones.
5- Software Configuration Management Plan (SWCP) Where to find work products belonging to this project. Configuration item list.
Who reaches them? Baselines, change management and problem resolution
6- Software Quality Plan (SWQP) Quality goals for this project.
Reviews, percentage of unit testing? QA organization?
7- Software Requirements Traceability Matrix (SWRT). To
understand which requirement are implemented and tested with one/many test cases. The relation traceability between test and code and requirement.
8- Software Design Description (SWDD) High level architecture and modules and low level for how the modules works. Very
41
important to understand construction when adding/removing functionality or bug fixing.
9- Hardware/Software Interface Specification. Hardware and Software interface for memory mapping.
10- Software Product Specification (SWPS) How to recreate a certain and configure files. To make sure that knowledge will be transferred in team.
11- Software Version Description (SWVD). What was done in this version compare to last one? Problems, what works what doesn't.
12- Software Test Plan (SWTP). Test organization. Test environment, test levels, automatic tests / manual tests.
13- Software Test Description (SWTD). How a requirement is tested, which test steps. If test case verifies requirement
14- Software Test Report (SWTR) Evidence that SW will have all desired requirements. Correctly respond to all inputs. use it to as part of decision of SW is good enough to release at Volvo
15- Software Worst-Case Analysis Report (SWWCA) In real-time computing (in vehicle), the worst-case execution time is often of particular concern since it is important to know how much time might be needed in the worst case to guarantee that the algorithm will always finish on time.
16- SPICE assessment, Process maturity is only one part that contributes to product quality. The more structured one works, the lower is the likelihood to introduce systematic faults.
17- SPICE assessment action plan. Who drives this to make sure that improvements will be done? (effective if independent quality organization)
