Qualification of Tool for Static Code Analysis: Processes and Requirements for Approval of Static Code Analysis in the Aviation Industry

(1)

Qualification of Tool for Static Code Analysis

Processes and Requirements for Approval of Static Code Analysis in the Aviation Industry

CHRISTOPHER GUSTAFSON SAM FLORIN

KTH ROYAL INSTITUTE OF TECHNOLOGY

SCHOOL OF ELECTRICAL ENGINEERING AND COMPUTER SCIENCE

(2)

Code Analysis

Processes and Requirements for Approval of Static Code Analysis in the Aviation Industry

SAM FLORIN

CHRISTOPHER GUSTAFSON

Degree Programme in Computer Engineering Date: June 5, 2020

Supervisor: Johan Montelius Examiner: Anders Sjögren

School of Electrical Engineering and Computer Science Host company: Saab AB Avionics Systems

Swedish title: Kvalificering av Statiskt Kodanalysverktyg

Swedish subtitle: Process och Krav för Godkännandet av Statisk Kodanalys i Flygindustrin

(3)

2020 Sam Florin and Christopher Gustafson

(4)

Abstract

In the aviation industry, the use of software development tools is not as easily adopted as in other industries. Due to the catastrophic consequences of software errors in airborne systems, software development processes has rigorous requirements. One of these requirements is that a code standard must be followed. Code standards are used to exclude code constructions which could result in unwanted behaviours. The process of manually ensuring a specific code standard can be costly. This process could be automated by a tool for static code analysis, however, this requires a formal qualification.

This thesis evaluates the process of qualifying a tool for static code analysis in accordance with the requirements of the major aviation authorities EASA and FAA. To describe the qualification process, a literature study was conducted. To further explain how an existing tool could be put through the qualification process, a case study of the existing tool Parasoft C/C++ test was conducted.

The results of the literature study show what processes must be completed in order to qualify a static code analysis tool. Importantly, the study shows that no requirements are put on the development process of the tool. This was an important takeaway as it meant that an existing tool could be qualified without any additional data from the developer of the tool.

The case study of Parasoft C/C++ test showed how the tool could be configured and verified to analyze code in accordance with a small set of code rules. Furthermore, three documents including qualification data were produced showing how the qualification process should be documented in order to communicate the process to an authority.

The results of the thesis do not provide the full picture of how a tool could be qualified as the software, in which the tool is used, is considerations the are specific to the software the tool is used to develop still need to be taken into consideration. The thesis does, however, provide guidance on the majority of the applicable requirements. Future research could be done to provide the complete picture of the qualification process, as well as how the process would look like for other types of tools.

Keywords

Static code analysis, Tool qualification, Aviation industry, Code standard, Parasoft C/C++ Test

(5)

(6)

Sammanfattning

Inom flygindustrin är användandet av olika programmeringsverktyg inte lika självklart som inom andra industrier. På grund av de katastrofala konsekvenser som fel i mjukvaran i ett flygplan kan resultera i finns det rigorösa krav på mjukvaruutvecklingsprocessen. Ett av dessa krav är att en viss kodstandard måste upprätthållas. Kodstandarder används för att exkludera vissa strukturer i kod som kan leda till oönskat beteende. Upprätthållandet av en viss kodstandard är en långdragen process att genomföra manuellt, och kan därför automatiseras med hjälp av ett statiskt kodanalysverktyg. För att kunna använda ett sådant verktyg behövs däremot en formell verktygskvalificering.

I denna uppsats kommer kvalificeringsprocessen av ett verktyg för statisk kodanalys att evalueras enligt de krav som de två stora flygmyndigheterna EASA och FAA ställer. För att förklara processen av att kvalificera ett sådant verktyg gjordes en litteraturstudie följt av en fallstudie av det existerande verktyget Parasoft C/C++ test.

Resultaten av litteraturstudien beskriver de olika processerna som måste genomföras för att kvalificera ett statiskt kodanalysverktyg. Noterbart är att resultaten visar att inga krav ställs på utvecklingsprocessen av verktyget själv.

Detta betyder att ett existerande kommersiellt verktyg kan kvalificeras utan att verktygsutvecklarna själva behöver bidra med extra information.

Fallstudien visade hur verktyget Parasoft C/C++ test kan konfigureras och verifieras att följa en viss kodstandard. Vidare resulterade fallstudien i utkast av de nödvändiga dokumenten som behöver produceras för att kommunicera kvalificeringsprocessen till en myndighet.

De resultat som presenteras i denna uppsats är i sig inte tillräckliga för beskriva hela kvalificeringsprocessen. Ytterligare överväganden som är specifika till den mjukvaran som verktyget ska användas till att utveckla måste göras för att en komplett kvalificering ska kunna genomföras. Uppsatsen bidrar däremot med riktlinjer och vägledning av majoriteten av de processerna som behöver genomföras. Ytterligare forskning kan göras för att bidra med den kompletta bilden av verktygskvalificering av ett statiskt kodanalysverktyg, samt hur kvalificering kan göras av andra typer av verktyg.

Nyckelord

Statisk kodanalys, Verktygskvalificering, Flygindustrin, Kodstandard, Parasoft C/C++ Test

(7)

(8)

Acknowledgments

We would like to express our appreciation to our supervisors, at Saab Avionics, for the opportunity of pursuing this work, making time, and solving workplace and communication problems despite the current pandemic. We would also like to thank our examiner Anders Sjögren for valuable discussions and feedback for academic writing. Special thanks to our academic supervisor Johan Montelius for productive group meetings, and valuable discussions and guidance in both work-related problems and academic writing.

Jönköping and Stockholm, June 2020 Sam Florin and Christopher Gustafson

(9)

(10)

Chapter 1 Introduction

The use of software development tools in the aviation industry is not as easily adopted due to the rigorous requirements of the industry. Errors in software for airborne systems could have catastrophic consequences, and the developed software thus have to be developed with great care. The use of a tool for static code analysis is not easily adopted. This thesis seeks to describe how this sort of tool could be proven sufficient for use in the aviation industry.

1.1 Background

The process of software development today heavily relies on different types of tools. Tools like version control, build tools, debugging tools and testing tools are musts in order to build software in the current industry. Tools speed up the development process by auto-generating code and suggesting what code should be written, but they also help the developer in identifying errors and faults once the code is written. A worldwide survey from 2019 shows that only 3% of software developers are not using any type of development tool [1].

One common type of tool is static code analysis tools (SCA-tools). These tools look for errors in the source code that is not recognised by the compiler.

As most errors that are made by software developers are similar, these SCA- tools works by recognising common error pattern in code. The word static, in this regards, means that the tool looks at the code before it is executed.

Dynamic analysis, on the other hand, would be made during execution, for example by different testing tools [2].

In the aviation industry, development of software needs to take additional measures to ensure the correctness of the code that is produced. For code that is executed in the computer of an aircraft, errors simply cannot occur under

(15)

any circumstances, as they could have catastrophic consequences. Therefore, all the steps in developing software in the aviation industry require measures to ensure the correctness of this software. The use of software development tools is thus not as easily adopted into the process of creating software for an aircraft. In order to certify software to be used in airborne systems, not only the software but also the tools used to produce the software needs to meet certain quality and safety criteria [3].

Certification can be achieved by compliance with an aviation authority.

Aviation authorities work to ensure safe and secure airborne operations in the aviation industry. There are several aviation authorities in the world, both nationally and internationally. Two of the largest authorities are the Federal Aviation Administration (FAA) that is part of the U.S. Department of transportation [4] and the European Union Aviation Safety Agency (EASA) that operates for the European Union [5].

While the certification process is done through aviation authorities such as EASA and FAA, the basis for the certification is often reliant on guidelines by other organizations such as the Radio Technical Commission for Aeronautics (RTCA) [6]. Both FAA and EASA recognises guidelines provided by RTCA as means to comply with the requirements for software certification [7] [8].

1.2 Problem

Software code developed for the aircraft industry has rigorous requirements due to the catastrophic consequences of an error. To reduce the chances of such an error, any code construction that could potentially result in unwanted behaviour must be excluded by performing manual or automatic analysis to ensure that a certain code standard is followed. The effort required to manually analyse code for larger projects can be time-consuming and costly. In order to reduce the budget and time spent on analysis, the process could be automated using an SCA-tool. To be able to trust a tool of this purpose, a formal status through qualification is needed from aviation authorities such as EASA and FAA.

1.3 Purpose

The purpose of this thesis is to explore what the requirements are to get an SCA-tool qualified by the aviation authorities EASA and FAA and to propose how this could be accomplished. The qualification of an SCA-tool would

(16)

save time and resources for the software development process in the aviation industry, as the tool would be able to replace the manual code analysis that is necessary without an SCA-tool, which is a costly and time-consuming process.

1.4 Goal

The goal of this thesis is to provide guidance on the qualification process of an SCA-tool in accordance with the requirements from authorities EASA and FAA. By doing so, the following central research questions will be answered:

What is required to qualify a tool for static code analysis according to authorities EASA and FAA?

1.5 Research Methodology

The process of proving confidence in a software tool within an airborne system is complex. Therefore, this research is falling back on already composed material within the field including scientific reports and guidelines by the RTCA commission. A literature study of the two documents "Software Considerations in Airborne Systems and Equipment Certification" (RTCA DO-178C [9]) and "Software Tool Qualification Considerations" (RTCA DO- 330 [10]) will be carried out in order to explain the sought realities for the qualification of an SCA-tool.

Furthermore, a case study will be conducted to provide guidance on what efforts are required to qualify an SCA-tool. The case study will be conducted on an existing tool which is identified based on the findings of the literature study. The tool will then be configured to follow a subset of a given code standard that is commonly used in safety-critical systems. To verify that the tool does comply with the code standard, the tool will be run on code with known errors, to see if it finds them. Finally, a draft of the required documents will be made for the tool qualification of the identified tool.

1.6 Delimitations

As the qualification of a tool is done in the context of the specific software it is used to develop, any aspects of qualification that are specific for the software it is applied to is in this thesis disregarded. However, efforts will be made to present a general qualification process of an SCA-tool where possible. This

(17)

report will focus on the use of tools for static code analysis and apply all the parts of tool qualification that is relevant for this purpose.

The case study described in chapter 5 will due to the time and budget constraints of this project not cover all of the processes that would be necessary to perform for certification to be achieved. All of the necessary processes will be explained in chapter4, but only the Tool Operational Requirements Process and the Tool Verification and Validation process will be regarded in the case study. Furthermore, only drafts of the required documents for qualification will be created, these will cover as much as possible while considering these delimitations.

The code standard that would be necessary to ensure the correctness of safety-critical software would have to include a large number of rules and requirements. However, the case study of the proposed SCA-tool will not cover all the specific code rules that an SCA-tool would have had to cover in order to be used in the development of software for airborne systems. A subset of the required rules will instead be considered in this report. Considering only a few rules still allows for the concept of tool qualification to be explained, while not spending unnecessary time repeating the same procedures.

1.7 Ethics

Qualifying a tool for static code analysis results in parts of a software developers work to be done by a computer. Here, an ethical aspect of trusting a computer can be discussed. In software systems which can cause the death of hundreds of people if fatal errors occur, one could ask themselves whether trusting a tool is the correct decision. The qualification process should prove that the tool does reach the quality of the manual code analysis and therefore, should be trusted as much as a software developer performing manual analysis.

Thus, we do not see any ethical aspect that talks against the qualification of an SCA-tool.

(18)

1.8 Document Outline

Chapter 2- Background

Background information on the problem statement, key concepts and relations are explained.

Chapter 3- Method

Chosen underlying methods and processes for the project are described and motivated.

Chapter 4- Literature Study

Results from the conducted literature study are presented.

Chapter 5- Case Study

Results from a case study of an existing SCA-tool is presented.

Chapter 6- Evaluation

Evaluation and discussion of the results of the project is presented.

Chapter 7- Conclusion and Future Work

Drawn conclusions are presented and future work is suggested.

(19)

Chapter 2 Background

The following chapter describes key concepts that are necessary to know in order to follow the rest of the work presented in this thesis.

2.1 Static Code Analysis Tool

One common type of tool to use for software where safety and security are particularly important is static code analysis tools. These tools look for errors in the source code that is not recognised by the compiler. As most errors that are made by software developers are similar, these static analysis tools work by recognising common error patterns in code. The word static, in this regards, means that the tool looks at the code before it is compiled and executed.

Dynamic analysis, on the other hand, would be made during execution, for example by different testing tools [2].

Static code analysis tools can also be used to follow a coding standard. The purpose of a coding standard is to for instance both increase the readability and maintainability of the code. This is done by ensuring that similar coding patterns are embraced among all the different parts of a software. Different rules of how software should be written are adhered to, with the goal of for example ensuring that only code constructs that guarantee the safety and security of the code are used [11].

The C and C++ coding guidelines by MISRA (Motor Industry Software Reliability Association) is an example that is widely used for software projects where safety and security are important considerations [12]. The guidelines consist of directives and rules that should be followed in order to ensure safe and secure software [13]. Examples of directives and rules that are a part of the MISRA guidelines are found in table2.1.

(20)

Directive 4.4 Sections of code should not be

“commented out”

Directive 4.12 Dynamic memory allocation shall not be used

Rule 15.5 A function should have a single point of exit at the end

Rule 21.7 The atof, atoi, atol and atoll functions of

<stdlib.h> shall not be used

Table 2.1: Examples of directives and rules in the MISRA C:2020 guidelines [13]

Static analysis is effective in finding errors in program source code, but commonly generates false positives, code sections flagged by the analyzer without any errors [14, p. 398-400]. If a static analysis results in a high number of false positives, the capability and suitability are decreased since every false positive must be viewed as a potential violation.

2.2 Aviation Authorities

Aviation authorities work to ensure safety and security. The two authorities that are considered in this thesis are FAA and EASA.

2.2.1 FAA

Federal Aviation Administration (FAA) is an aviation authority part of the U.S Department of transportation with the safety responsibility of civil aviation.

FAA holds many major roles such as regulating civil aviation to provide safety, encouraging and developing civil aeronautics, including new aviation technology [4].

2.2.2 EASA

European Union Aviation Safety Agency (EASA) is an organization with representatives from all EU members and external members within the region.

Their major roles include providing a single regulatory and certification process among the member states and work with other international aviation organisations and regulators [5].

(21)

2.3 Aviation Standardization Organizations

Aviation standardization organizations develop standardization for the aviation industry. The two organizations considered in this thesis are EUROCAE (European Organization for Civil Aviation Equipment) and RTCA (Radio Technical Commission of Aeronautics).

2.3.1 EUROCAE

European Organization for Civil Aviation Equipment (EUROCAE) is a not- for-profit organization dealing exclusively with aviation standardization. They focus on electronic equipment for air transportation but deals with both airborne and ground systems and equipment [15].

2.3.2 RTCA

Radio Technical Commission of Aeronautics (RTCA) is an organization formed to advance the art and science of aviation for the benefit of the public. The organization functions as a Federal Advisory Committee and develops consensus-based recommendations and guidelines on aviation issues [10, FOREWORD].

History of RTCA DO-178C

During the early 1980s, the use of software in airborne systems increased rapidly. This resulted in a need for industry-accepted common guidance satisfying airworthiness requirements. This lead RTCA in concert with EUROCAE to create and publish RTCA DO-178 and EUROCAE’s document ED-12, which is the their corresponding document. Due to the advances in software development together with misinterpreted sections, these documents were over the years passed through three major revisions and resulted in

"Software Considerations in Airborne Systems and Equipment Certification"

(RTCA DO-178C) and shortly after also ED-12C by EUROCAE [9, p. 1] [9, Appendix 1].

History of RTCA DO-330

The guidance to qualify a tool was removed from RTCA DO-178C and ED- 12C in the last revision but was introduced in the independent, external

"Software Tool Qualification Considerations" (RTCA DO-330 or ED-215)

(22)

including guidance, objectives, activities and life cycle data required for each TQL (Tool Qualification Level) [9, p.85]. The joint work between RTCA and EUROCAE lead to the different publications DO-330 (RTCA) and ED-215 (EUROCAE) with the same technical content. The documents were developed to provide guidance for airborne and ground-based software, introduce guidance for the tool domain for both tool developers and tool users and finally helping tool development teams to avoid confusion and misinterpretation [10, p.1].

2.4 Tool Qualification

Tool Qualification is in the book Certifiable Software Applications 2 by Jean- Louis Boulanger defined in the following way; "Tool qualification is a process that allows us to demonstrate that a tool can be used as part of the realization of a software application with a determined safety goal." [16, p. 172]. Tool Qualification is thus a way of ensuring that a tool follows certain security criteria, with the ultimate goal of ensuring that the software, of which the tool is used to develop, is meeting certain requirements. As tools in software development are often used to reduce, eliminate or replace an existing process in software development, tool qualification is used to ensure that the tool produces results that are of equal or better quality as the process it is replacing [9].

2.4.1 Tool Qualification in accordance with EASA and FAA

Looking at the requirements for software tool qualification by EASA and FAA, both refer to the use of RTCA DO-178C and RTCA DO-330. According to EASA, the level of complexity in the aviation industry makes it impossible to have one specific set of regulations. Instead, EASA divides their regulations into three levels of regulatory material; the basic regulation, implementation of the basic regulations and Certification Specifications (CS), Acceptable Means of Compliance and Guidance Material (GM) [17]. The AMC describes what means that can be adopted in order to meet the basic regulations and their implementations. In AMC-20 / Amendment 14, EASA describes the use of RTCA DO-178C and RTCA DO-330 as a means to compliance. They state that "The applicant should submit to EASA the life cycle data specified in Section 9.3 of ED-12C/DO-178C, and Section 9.0 a. of ED-215/DO-330, as applicable to tool qualification." [8, p. 6].

(23)

Similarly, FAA describes in order 8110.49A how their staff can use RTCA DO-178C for approving software for airborne systems, thus implying that following RTCA DO-178C would be a way to align with their requirements [7].

2.4.2 RTCA DO-330

Software and Tool Life Cycle

When software is decided to be created, the software life cycle is initiated.

There are many well-known software life cycle models such as Agile [18] and Spiral [19]. Common for all of these is that they contain a set of processes used to develop and evolve the software. The software life cycle is active until the phase-out and retirement of the software. For critical systems, especially airborne systems, RTCA DO-178C "Software Considerations in Airborne Systems and Equipment Certification", provide descriptions for these processes. Similar to the software life cycle, where the tool is used, a set of processes is used to develop and evolve the tool itself, this is called the tool life cycle [10, p. 5-6]. The tool life cycle processes are the necessary activities that should be carried out in order for the tool to comply with regulations by aviation authorities described in RTCA DO-330 "Software Tool Qualification Considerations", and referenced to from the RTCA DO-178C. RTCA DO-330 focus on guidance for tools which eliminates, reduces or automates any of the processes in the software life cycle but also provides additional considerations [9, p. 85-86]. The tool life cycle processes are described in table2.2 below.

The main documents for managing certification of software are the Plan for Software Aspects of Certification (PSAC), Software life-cycle Environment Configuration Index (SECI) and Software Accomplishment Summary (SAS).

The Relation between Software and Tool Life Cycle

For airborne software development, a tool is always qualified within the context of its software. This means that the tool qualification liaison process in the tool life cycle is performed alongside with the certification liaison process in its corresponding software life cycle [10, p. 37]. This requires coordination between the software life cycle and the tool life cycle which is obtained by providing tool-specific information or references in the three software life cycle data documents [10, p. 2]:

1. In PSAC: Identification of the tool and its qualification need.

(24)

Process Description Tool Qualification Planning

Process

The tool is identified and the impact on the software it is used for is assessed. The rest of the tool life cycle processes are described.

Tool Development Process Defines the requirements of and implements the tool software.

Tool Verification Process Detect and remove any errors that may have resulted in the software from the tool development process

Tool Configuration

Management Process

Defines the configuration of the tool as well as ways to control, review and assess the tool.

Tool Quality Assurance Process

Ensures that the tool objectives are satisfied, that errors are found and acted on and that the tool follows the guidelines of DO-330.

Tool Qualification Liaison Process

The process of agreeing upon the qualification with the aviation authority for the specified tool, in the context of the software it will be used for.

Table 2.2: Tool life cycle processes in RTCA DO-330 [10, p. 5,9,17,27,33,37]

(25)

2. In SECI: Description of the software environment and identification of the tool.

3. In SAS: Status and summary of all the activities performed together with the status of the tool.

Table 2.3: Overview Example of Software and Tool Life Cycle in a Certification

The minimum data that has to be submitted to the aviation authorities along with the PSAC, SAS and SECI are the tool life cycle data documents [10, p. 37]:

1. Tool Qualification Plan (TQP): A tool specific plan describing the tool qualification process. Referenced from the PSAC.

2. Tool Configuration Index (TCI): Description of the tool environment, identification of the tool and reference to the tool qualification data.

Referenced from the SECI.

3. Tool Accomplishment Summary (TAS): A status of the tool and a summary and status of all the activities performed showing compliance with the TQP. Referenced from the SAS.

An example of the context and relation between the tool life cycle and the software life cycle is shown in figure2.3.

(26)

Software Level

The system safety assessment process determines the critically of different functionalities of the aircraft. Depending on the critically of the system functionality, the rigor of the development processes used for the software development shall be adapted. In RTCA 178C, the level of rigor is determined by the software level [9, p. 11]. The process identifies and analyses relations between potential software errors and failure conditions and is performed in the system life cycle processes [9, p. 11-12]. The software level definitions in RTCA DO-178C relates to the failure condition categorization of the system safety assessment process where software level A has catastrophic, level B has hazardous, level C has major, level D has minor and level E has no safety effects [9, p. 13-14]. The software level is used in table2.4below to map an appropriate tool qualification level described in section2.4.2.

Impact Criteria

The processes in the software life cycle which could be eliminated, reduced or automated by a tool is described in RTCA DO-178C. By identifying and analysing these processes, the impact of the tool is determined and matched to one of the three defined criteria. Criteria 1 is used in cases where tool result could insert an error. Criteria 2 is used in cases where the tool result is used to compromise either a verification process or a development process which could fail to detect an error. Criteria 3 is used where the tool could fail to detect an error within its intended domain [9, p. 85]. The criteria are shown in table2.4below to map an appropriate TQL described in section2.4.2.

Tool Qualification Level

Tools for software in airborne systems are categorised into five different levels depending on the impact of the tool in the software development process, as well as the assurance level that is required for the software that will be developed. These levels are numbered TQL-1 to TQL-5 (TQL stands for Tool Qualification Level) where TQL-1 requires the most meticulous qualification, and TQL-5 the least meticulous qualification [10, p. 7]. A specific TQL is based on the usage (software level) of the tool and its potential impact on the system safety (impact criteria). Table2.4shows the appropriate TQL mapped for all combination of software levels and criteria. In RTCA DO-330 and DO- 178C, the TQL is used to determine and navigate through objectives, activities and outputs for different setup of software and environments [10, p. Annex A].

(27)

For each TQL, different set of objectives needs to be satisfied, and for different TQLs the objectives might need to be satisfied with independence or not [10, Appendix B-1, 2].

Table 2.4: Tool Qualification Level Determination Table [9, p. 85]

Orientation and Usage

RTCA DO-330 defines processes of the tool life cycle containing objectives in order to comply with aviation authorities. In order to fulfil those, activities are specified and described for each objective. The number of objectives and activities that must be fulfilled respectively performed is dependant of the proposed TQL and could thereby be different from one qualification case to another. To be able to meet the objectives for the desired TQL, reference tables are used for navigation in the document. The table points to the objectives, its corresponding activities and the output for that desired TQL, which becomes part of the deliverable to the aviation authorities further described in the tool qualification liaison process. In order to understand the guidance and reference tables, the full body of the document should be considered [10, p. 65 ANNEX A].

Qualification of COTS tool

For qualification of a COTS (Commercial off-the-shelf) tool, that is a tool which is developed without a specific software in mind, the objectives of the tool qualification are divided between the tool developer and the tool user.

While the final responsibility of the qualification lies on the tool user, a lot of the previously mentioned processes are performed and data is produced by the tool developer [10, p. 57].

With the use of COTS tools, the tool developer is responsible for the tool development and verification process. The tool developer is also responsible

(28)

for parts of the tool planning process, which includes providing a developer- specific tool operational requirements (TOR). This TOR should be produced in order for potential tool users to be able to determine whether the tool can meet their requirements. During the qualification process for the tool user, the developer-TOR is used to develop their own TOR, which should specify how the tool is to be used in their software development process.

Furthermore, the tool configuration management process and quality assurance process are performed by both the tool developer and tool user, but in the scope of their respective responsibilities. That is for the tool developer to produce the data for their general tool, and the tool user to produce the data for the use of the tool in their intended situation. Finally, the tool user is responsible for the tool qualification liaison process.

Qualification of Multi-Function Tools

A multi-functional tool is a tool which can provide multiple functions, and where the different functionalities are provided through one or more executable files [10, p. 53]. During qualification of a multi-functional tool, each used functionality should be given an individual TQL. RTCA DO-330 presents additional guidance in section 11.1, "Multi-Function Tools", which must be used when qualifying a tool with only certain functions enabled, or when functions have been given different TQLs. Another possible approach is to qualify the entire tool to the highest TQL given to a function [10, p. 53-54].

Similar to a normal tool, all multi-functional tools should be described in the appropriate tool plans. Except for the effective information for a normal tool, a multi-functional tool must include a description of all functionality used or not used, and an explanation of how the additional guidance provided in section 11.1 "Multi-Function Tools" will be satisfied [10, p. 53].

2.5 Related Work Area

2.5.1 DO-330 COTS Tool Qualification Pitfalls

Although the dedicated guidance on tool qualification in DO-330 including guidance on commercial-off-the-shelf (COTS) tools, there is still room for misinterpretation. Common pitfalls are identified and a brief overview of the qualification process of a COTS tool using DO-330 is given in [20]. Pitfall 1 states that COTS tool users are heavily reliant in the Tool Qualification Kit (TQKs). Furthermore, tool vendor TQKs can be incomplete and vary in

(29)

quality. This requires the tool user to review the TORs and potentially improve and complement them, see Marques and Cunha [20].

(30)

Chapter 3 Method

The following chapter describes the research methodologies and the different phases that outline the work conducted for this thesis.

3.1 Research Methodologies

The choice of research methods and methodologies is an important part of a research project. The choices steer the way the work is conducted in order to achieve correct and well-grounded results. To choose correct methodologies is a necessary step in order to formulate a well-structured research [21].

3.1.1 Qualitative & Quantitative Methods

There are two types of basic categories for research methods, one using qualitative data the other using quantitative data. What data there is and how data is processed determines the character of a project. The choice of whether the project is of quantitative or qualitative character is the first scientific standpoint taken into consideration and will affect what other methods to be used. While the quantitative method requires large data sets using statistic as a certainty, the qualitative method uses smaller but sufficient data sets forming certainties by understanding it [21, p. 3]. Given the material stated in chapter 1.5, a smaller data set is to be used. The problem statement in chapter1.2will be answered and the artefact stated in chapter1.4will be formed by analysing and understanding the material. Thus this project is of the qualitative character.

(31)

3.1.2 Analytical & Applied Research

Two research methods were considered, analytical- and applied research. Both methods are often built on existing data but applied research uses real work data to solve problems. Analytical research has an advantage in decision- making areas performing a critical evaluation of the data through analysis, while applied research is conducted with a particular application in mind applying results and data to solve known problems [21, p. 4]. Due to the set of circumstances described in chapter1.1and the fact that the problem statement in chapter 1.2 is a direct question rather than a hypothesis, this project is conducting the applied research method.

3.1.3 Deductive, Inductive & Abductive Approaches

There are three possible approaches to take in research. The deductive approach draws conclusions from known premises. This is usually done by using quantitative methods with large data sets. The result is a generalization based on the collected data. The opposite, the inductive approach, processes observations to formulate propositions with alternative explanations. The third one is a hybrid approach using both deductive and inductive approach to draw conclusions [21]. In this thesis, the inductive approach will be used.

The literature will be analysed to gain an understanding of how to achieve a formal qualification from the guidance documents and a case study will provide additional experience and establishment of what the requirements in section1.2are, in line with the inductive approach.

3.1.4 Bunge’s Method

The book "Epistemology and Methodology I: Exploring the world" by Mario Bunge describes a general sequence which a scientific method is comprised of.

This general sequence can be tailored for technological research and follow the steps below [22, p. 253-]:

1. How can the current problem statement be solved?

2. How can a technique or a product be developed to efficiently solve the problem?

3. What material and information are provided to be able to develop the technique or product?

(32)

4. Develop the technique or product from the findings in step 3. If the technique or product turn out satisfactory, advance to step 6.

5. Try with a new technology or product.

6. Create a model or simulation of the proposed technique or product.

7. What is achieved by the model or simulation, what are the consequences?

8. Test the implementation of the model or simulation. If the model or simulation turns out satisfactory, advance to step 10. Otherwise, advance to step 9.

9. Identify and adjust deficiencies in the model or simulation.

10. Evaluate the result in relation to current knowledge, identify new problem areas for additional research

Bunge’s method will be adopted in this project, with the different project phases corresponding to one or more steps in the method, as described in chapter3.3.

3.1.5 Case Study

A case study is conducted using empirical studies, which investigates a certain phenomenon in a real-life environment. It draws conclusions from the observations made on the real-life environment, comparing it to the theory behind it [21]. In this project, a case study will be conducted in order to further investigate the process of qualifying an SCA-tool. The findings of the case study will thus be used to complement those of the literature study, which provides a theoretical basis.

3.2 Research Method

The research question that the research conducted in this thesis will answer is:

What is required to qualify a tool for static code analysis according to authorities EASA and FAA?

To ensure that the research question together with the delimitations in chapter 1.6 is answered a set of sub-questions are defined which are considered sufficient to answer the research question, the sub-questions are as followed:

(33)

1. What processes need to be carried out for an SCA-tool to be qualified?

2. What properties does an SCA-tool need to possess in order to be qualified?

Using an SCA-tool with the properties identified from question 2. The following sub-questions will be answered:

3. How could an SCA-tool be configured in order to meet the qualification requirements?

4. How could an SCA-tool be verified to ensure correct functionality?

5. How could the qualification process of the tool be documented?

To define and answer the following questions, the processes shown in figure 3.1are carried out. These processes reflect the different steps that Mario Bunge defines for technological research.

Figure 3.1: Processes of the research method

3.2.1 Pre Study

During the pre study, the topics of tool qualification and static code analysis tools were explored. The pre study resulted in a good understanding of the topic at hand, which helped in formulating an accurate problem statement.

This process corresponds to step 1 in Bunge’s method. Information was gathered from online resources like scientific articles, recorded interviews and e-books.

3.2.2 Problem Analysis

Using the acquired knowledge of the pre study, chapter 1 was formulated, where the problem, purpose, research question, goals and delimitations of this thesis were defined. The data that have to be collected to fulfil the goals of the

(34)

thesis was formulated as the previously described sub-questions. This data will be collected during the literature study and the case study described below.

3.2.3 Literature Study

To answer questions 1 and 2, a literature study will be conducted. The goal of the literature study is to provide the necessary data to describe what authorities EASA and FAA requires for the qualification of an SCA-tool.

The details of the literature Study is further described in chapter 3.3.1. This process corresponds to steps 2 and 3 of Bunge’s method where the process of qualification is first described, and existing SCA-tools that can be used will be identified.

The data that will be collected during the literature study will be:

• Descriptions of the different processes that are required in order to qualify an SCA-tool

• The different properties that an existing SCA-tool would have to possess in order to be qualified

This data can be found in chapter4.

3.2.4 Case Study

To answer questions 3, 4 and 5, a tool selected in accordance with the identified tool properties in the literature study will be used to conduct a case study. The case study is described further in chapter 3.3.2. The case study corresponds to steps 4 to 9 of Bunge’s method.

The data that will be collected during the case study will be:

• A description of the tool that will be used to conduct the case study

• A code standard with a set of code rules that the tool will be configured according to.

• Instructions and examples of how the tool could be configured to follow the code standard.

• Instructions and examples of how to verify that the tool does point out violations of the code standard

• Drafts of the required documents that would have to be produced in order to qualify the SCA-tool

This data can be found in chapter5.

(35)

3.2.5 Evaluation

In the last step, the results of the literature study and case study are evaluated and the collected data is used to answer the research question defined in chapter 1.4. The evaluation step is equivalent to the final step in Bunge’s method. The evaluation is found in chapter6.

3.3 Project Phases

The project to be conducted for success of this research encapsulates the literature study and the case study in the research method. Success in this regard means meeting the previously described requirements of answering the research question, as well as keeping the project within the given time and budget limits. The project is the part of the research where data will be collected in order to answer the research question of this thesis. The different phases of the project are shown in figure3.2.

Figure 3.2: Project Phases

3.3.1 Literature Study of RTCA documents

The reports DO-178C and DO-330 by RTCA is the basis of the literature study.

Due to the general descriptions in the DO-178C and DO-330 and the fact that they are guidance documents, it is of high importance for this project to be elaborate and accurate in interpreting the guidelines and critically evaluate and complement the qualification data that will be produced.

(36)

The literature study will explain the different processes that should be carried out in order to qualify an SCA-tool. It will also describe the properties that a tool should possess in order to fulfil the requirements of these processes.

3.3.2 Case Study of SCA-tool

With the knowledge provided by the literature study, a case study will be conducted to provide guidance on how an SCA-tool could be qualified for use in software development for airborne systems. The case study will result in a draft of the necessary documents and data that will be required to get the sought certification. Due to the time limit of this project the case study will be based on an existing, so-called commercial-off-the-shelf (COTS) tool, the tool itself will thus not be developed in the scope of this project. The COTS tool that is used will be picked to fit the requirements identified in the literature study.

Configuration

Configuration of the tool will be made in order to meet the requirements that are identified during the literature study. The configuration will also be made in order to ensure that the tool follows the given code standard. As COTS tools are developed without a specific software application in mind, the identified tool will have to be configured to look for errors with regard to the code standard defined for the case study. The configuration phase will be performed to look for which of the code standard rules that are already considered, and in case of missing rules, add these to the configuration of the tool. Figure3.3 shows an iterative process of configuring a new rule based on predefined test cases. This process was based on fundamental trial and error.

Verification

The verification phase of the case study will be performed in order to provide the necessary tool qualification data that verifies that the tool complies with its defined requirements. For every code rule, test cases will be designed, test data will be prepared, the tool will be run with the test data and test results will be compared to the expected test results. This process is illustrated in figure 3.3.

For each test case, preparations, test steps and expected result will be defined. Test data will be created in the form of short source code files. The test data should correspond to different scenarios with violations to the current rule

(37)

to see that the tool does point out code standard violations. Test data will also be created with code patterns where the tool could report a violation without it actually being a violation, in order to ensure that tool is not reporting false positives.

With the test data created and prepared, the data will be run on the tool and the result will be compared to the expected result of each corresponding test case. If the test passes, a test case for a new rule will be designed. However, if the test fails, an iterative trial and error process of configuring the rule will be initiated.

Figure 3.3: Configuration and Verification Process

The results of the verification will not be enough to justify that the tool works perfectly and that it under no circumstance would fail in detecting a violation of the code standard. To achieve this, one would have to test all the possible code constructions that exist in the C language, which is simply impossible. The verification step will, however, show that it does point out the violation to the largest possible extent. Showing that the tool makes the correct decision in multiple scenarios, both with and without errors, gives the basis for drawing the conclusion that the tool makes mostly correct decisions. This is in this regard enough as it shows that the tool corresponds to the quality of human manual code analysis, which also has the risk of missing a rule violation.

Draft Qualification Documents

The final part of the case study will be to draft the necessary documents and data items needed in the qualification process. These documents and data are the primary means in which you communicate and decide upon the qualification with the aviation authority. These documents will be produced in accordance with the findings of the literature study and using the data that

(38)

is produced during the literature study.

As described in the delimitations in chapter 1.6, not all qualification processes will be carried out. Parts of the documents and data that address these processes will thus not be included in the produced document drafts.

Thus, the documents are only drafts and not complete. To qualify a tool, these documents would have to be complemented with the parts that are not addressed.

(39)

Chapter 4 Literature Study

The following chapter presents the results of the conducted literature study of documents RTCA DO-178C "Software Considerations in Airborne Systems"

[9] and RTCA DO-330 "Software Tool Qualification Considerations" [10].

The data to be collected during the literature study is descriptions of the necessary qualification processes and properties that an SCA-tool have to possess, as described in chapter3.2.3.

4.1 Establishing Tool Qualification Need

The first thing that is done when a tool is considered to be used in the process of developing software for airborne systems is that the need for tool qualification is established. As earlier described, tool qualification is in RTCA DO-178C stated to be needed when "processes of this document are eliminated, reduced or automated by the use of a software tool without its output being verified"

[9, p B-1]. An SCA-tool used to confirm that source code complies with a given coding standard, will in development of airborne systems automate the process "Reviews and Analysis of Source Code, Conformance to standards”

that is described in section 6.3.4.d in RTCA DO-178C [9]. The output of the tool will not be further verified, and therefore a need for tool qualification exists.

Following the established need for tool qualification, the impact of the tool and its suitable TQL is determined, this process is described in chapter2.4.2.

Since the tool is used in the purpose of verification, it could fail to detect an error. An SCA-tool would therefore not fall under criteria 1. It will not fall under criteria 2 either since the tool will not be used to eliminate, reduce or automate any other process except that of code analysis. The proper criteria is

(40)

thus criteria 3 since an SCA-tool only could fail to detect an error in the scope of the code analysis. No other processes will be automated by the use of this tool. A tool that falls under criteria 3, is always categorised as TQL-5, which is the appropriate TQL for an SCA-tool.

4.2 Qualification of a TQL-5 tool

Agreeing upon the qualification of a tool with an aviation authority is done during the "Tool Qualification Liaison Process". It should be noted that this process is never performed alone, but rather in the context of the software the tool is used to develop [10, p. 39]. However, for an SCA-tool with the purpose of conforming with a given coding standard, the produced data could be used for different airborne software as long as the same coding standard is used.

The tool-specific documents to be produced for submission to aviation authorities is a Tool Qualification Plan (TQP), a Tool Configuration Index (TCI) and a Tool Accomplishment Summary (TAS), the contents of these are described in table4.1. It is stated in RTCA DO-330 that these documents are not required for TQL-5 [10] and that its information could be provided in their corresponding documents for the specific software (That is the PSAC, SECI and SAS respectively). For the purpose of generalization, these documents could still be provided, and referenced in their corresponding software-specific documents.

In short, the TQP describes how the qualification process will be done, the TCI describes the configuration and verification of the tool and the TAS showcases that the TQP has been followed. These documents all rely on data produced during the different qualification processes (Further described in chapters4.2.1-4.2.5). This data is specified in RTCA DO-330 [10] for TQL-5 by the outcomes of the applicable TQL-5 objectives and is further referred to as the Tool Qualification Data, these are summarized in table4.2.

The objectives for qualification at TQL-5 differs from the rest of the TQLs in that some of the processes are not necessary at all. The tool planning, development and verification processes are not required in order to qualify a tool at TQL-5. These processes are connected to the development of the tool itself and because of this, tool qualification at TQL-5 can be made without any data provided by the developer of the tool. For TQL-5 the objectives are mainly focused on the operational aspects of the tool. These are the aspects concerning the tool-usage in the context of the software it is used to develop.

Figure4.1 shows the processes required and the data produced for a TQL-5 tool described below.

(41)

Document Reference Content

TQP RTCA

DO-330:

10.1.1 10.1.2

• Identification of the tool and eventual configuration

• Proposed TQL and the sought certification

• The functionality of the tool

• Tool operational environment

• Description of how the qualification process will be shown to the aviation authority

• Activities to be performed during the qualification process

• Data to be produced during these activities, that is the tool qualification data

TCI RTCA

DO-330:

10.1.11 10.1.17

Identification of:

• The tool

• Tool executable object code

• Tool source code

• Instructions for building the tool executable object code

References to:

• The tool operational requirements

• The tool verification and validation cases and procedures

• The tool verification and validation results

TAS RTCA

DO-330:

10.1.15 10.1.16

This document should correspond to the TQP, and explain any deviations or changes made from this document. It should include:

• Identification and functional overview of the tool

• Details of the sought certification, that is the process that the tool is automating

• References to the tool qualification data

Table 4.1: Documents submitted to AA

(42)

Data Objects RTCA DO-330 [10] Reference Tool Operational Requirements Section 10.3.1

Tool Executable Object Code Section 10.2.4 Tool Installation Report Section 10.3.2 Tool Verification and Validation Cases

and Procedures

Section 10.3.3 Tool Verification and Validation Results Section 10.3.4 Tool Configuration Management

Records

Section 10.1.13 Tool Quality Assurance Records Section 10.1.14

Table 4.2: Tool Qualification Data

Figure 4.1: Processes for qualification of TQL-5

4.2.1 Tool Configuration Management Process

The Tool Configuration Management Process is carried out in order to ensure that all the produced qualification data is produced, stored and retrieved in a way that ensures the integrity of the data. The process includes identifying and labelling configuration items. Configuration items are software components or tool life cycle data that is treated as a unit, for the purpose of this process.

After identification, the following needs to be ensured and documented in Tool Configuration Management Records for a tool of TQL-5, that is the output of this process:

• Configuration items need to be retrievable

• Measures to ensure the integrity of the configuration items are taking.

For example, making sure that no unauthorized changes are made, that data is not lost or corrupted or that data is copyable.

(43)

4.2.2 Tool Operational Requirements Process

For the tool to be qualified in accordance with RTCA DO-330, tool operational requirements (TORs) need to be defined. These requirements specify how the tool is to be used within the software life cycle. In order to achieve compliance, the TORs must be verifiable, consistent and must include enough detail to demonstrate that the functionality and the resulting output from the tool correspond to the activities that the tool is replacing. For an SCA-tool, the requirements of the tool should correspond to the requirements that are otherwise fulfilled by manual code analysis. The output of this process is the TORs which should include the following:

• Description of the context of the tool, and how the output of the tool is integrated into the software development life cycle.

• Description of where the tool is installed, defined as the Tool Operational Environment

• Description of input and output files to the tool

• Requirements for all the functionality the tool provides to the software life cycle

• Relevant tool user information, for example a user manual

• Description of the operational use of the tool.

• Performance requirements and its impact on the output

4.2.3 Tool Operational Integration Process

During this process, the SCA-tool is installed in the previously defined tool operational environment, and the resulting Tool Executable Object Code, including all associated files, are produced as output. A Tool Installation Reportis also produced including the following:

• Identification of the tool operational environment

• Identification of the version of the tool executable object code.

• Instructions of how to execute the code

(44)

4.2.4 Tool Operational Verification and Validation Process

This process is done in order to provide confidence that the tool meets the TORs, and the needs of the software life cycle itself, that is the assurance of a given code standard. To do so requires verification that the tool meets the TORs, as well as validation that the TORs are sufficient enough to justify that the manual code review of coding standards can be automated by the SCA- tool.

In order to verify and validate an SCA-tool, test cases that cover all the rules of the coding standard need to be defined and carried out. These test cases should showcase that for any given rule in the coding standard, inputted source code that does not comply with that rule is identified. For every test case, specific inputs, parameters and configuration need to be documented, as well as step-by-step instructions of how to execute the test.

All the instructions for the test cases and procedures are documented in what is defined as the Tool Operational Verification and Validation Cases and Procedures in RTCA DO-330 and their results are documented in the Tool Operational Verification and Validation Results[10, p 51].

4.2.5 Tool Quality Assurance Process

The tool quality assurance process assesses the previously described processes and their respectively produced data in order to assure that their objectives are satisfied, that eventual deficiencies are considered and that the tool qualification is aligned with the guidelines of RTCA DO-330 and other eventual regulations. Outputs of this process are called Tool Quality Assurance Records, and should include the following information:

1. Assurance that the produced data is in accordance with the specifications described in the tool configuration management process

2. A tool conformity review (described below)

The purpose of the tool conformity review is to obtain assurance that the processes and produced data are complete and that the tool executable object code is controlled and can be regenerated. This is done by providing evidence that the produced data and performed activities follow these plans, and that eventual problems have been reported and taken care of. The reputability is proven by showing that the tool executable object code can be regenerated using the tool source code.

(45)

4.2.6 Other Considerations

The definition of a multi-function tool along with the guidelines of qualifying one is provided in section2.4.2. For TQL-5, section 11.1.d in RTCA DO-330 is the only section which applies out of the additional guidance [10, p. 54]:

If a multi-functional tool both produces an output and verifies this same output, it should be demonstrated that the functions of the tool have been developed using protection techniques to avoid an error that might affect both functions [...].

However, for an SCA-tool, this is not applicable since it does not manipulate the source code and produces thus no output. Given the appropriate TQL-5 to a multi-functional tool, the same guidelines as for a normal tool applies with the exception of describing all the functions.

4.3 Prerequisites for a COTS-tool

As previously described in chapter2.4.2, qualification of a COTS tool requires cooperation between the tool developer and the tool user. When choosing an existing tool to qualify in general, this has to be taken into consideration. The results of the literature study show however, that for an SCA-tool that falls under TQL-5, objectives connected to the tool planning, development and verification processes are not required. Since these are the main objectives of the tool developer, no data from the tool developer is needed when the qualification of a TQL-5 tool is done. When identifying a tool, there is thus no requirements on the data that the tool developer has to provide. Prerequisites for the tool will therefore only consist of the required functionality of the tool.

The required functionality of the SCA-tool is that it should be able to automate the process of manual code analysis with the same or higher quality.

For the SCA-tool to meet that requirement, a predefined code standard should be ensured after the tool have been used. The requirements of the tool could therefore be divided into two parts:

• Given a set of rules, the tool should be able to ensure that the code inputted to the tool is compliant with these rules.

• The set of rules considered by the tool should correspond to a given code standard

(46)

When identifying a tool, the first requirement was fairly fundamental, since this essentially describes the functionality of an SCA-tool in general. Instead, the second requirement became the main requirement that had to be fulfilled in order to be possible to qualify. For the rule-set to correspond to a given code standard, functionality that allows the user to configure or add the rules considered by the tool needed to be available.

(47)

Chapter 5 Case Study

The following chapter will present a case study of Parasoft C/C++ test. Firstly, the tool and its capabilities are explained. Secondly, the tool will be configured and verified to meet a certain code standard. Lastly, drafts of the required documents for qualification will be presented. The data to be collected during the case study is described in chapter3.2.4.

5.1 Parasoft C/C++ test

Parasoft C/C++ test [23] was chosen as the tool for the case study. The tool offers a lot of different functionality, including static analysis which will be the focus of this chapter. The static analysis functionality offers support for a plethora of different predefined code standards, as well as the ability to create own code rules. It gives the user the ability to define their own code standard by combining predefined and self-defined rules, and thus configure the tool for the user-specific needs.

The requirements described in the literature study was that an SCA-tool had to be able to adapt to a given set of rules, and thus have the ability to use self-defined rules. C/C++ test by Parasoft is, therefore, a tool that meets the requirements and is able to be qualified to replace manual analysis of C code.

The tool selection was motivated with the establishment of the tool provider and the acquisition of a free license. However, any SCA tool on the market with the property to add a new rule could be used.

(48)

MISRA Rule Rule Title

Rule 8.8 An external object or function shall be declared in one and only one file.

Rule 12.2 The value of an expression shall be the same under any order of evaluation that the standard permits.

Rule 12.10 The comma operator shall not be used.

Rule 12.13 The increment and decrement operators should not be mixed with other operators in an expression.

Table 5.1: Code Standard used in the case study, rules taken from MISRA- C:2004 [24].

5.1.1 Code Standard

As described in chapter 1.6, the tool will only be configured and verified to ensure a subset of the necessary code rules. The rules considered can be found in table5.1, and are a small subset of MISRA-C:2004: Guidelines for the use of the C language in critical systems [24], which is commonly considered when discussing safety-critical software.

5.1.2 Tool Operational Requirements

As described in chapter4.2.2, tool operational requirements have to be defined for the tool in order to showcase that the tool corresponds to the manual code analysis it replaces. These requirements are then verified during the verification step of the qualification. Once the requirements are verified to be fulfilled, the tool can be considered to represent the quality of the manual code analysis.

In total 14 tool operational requirements were defined for the use of Parasoft C/C++ test and with the four code rules shown in table5.1 in mind.

The first four of the requirements are found in table5.2which reflect the four code rules. These requirements will be under focus for verification. The rest of the requirements can be found in appendixDand focus more on the specific circumstances that the tool should operate in.

(49)

TOR ID Requirement

TOR-1 The tool shall point out every occurrence (with the line number and filename) of an external object or function that is declared in more than one file. (Rule 8.8)

TOR-2 The tool shall point out every occurrence (with the line number and filename) of an if-else statement which are not followed by a compound statement. (Rule 14.9)

TOR-3 The tool shall point out every occurrence (with the line number and filename) of the comma operator being used. (Rule 12.10) TOR-4 The tool shall point out every occurrence (with the line number

and filename) of the increment or decrement operator being mixed with other operators in a single expression. (Rule 12.13)

Table 5.2: The first four Tool Operational Requirements

5.2 Configuration

Parasoft C/C++ test provides a lot of functionality out of the box, however, in order for the tool to point out violations to a predefined code standard, it had to be customized accordingly.

5.2.1 Configuring the rules

Configuration of Parasoft C/C++ test is done through the creation of test configurations. Test configurations consist of a set of rules that are applied on the inputted source code. To ensure that a certain code standard is followed, these rules should, therefore, reflect the rules of that standard. Parasoft C/C++

Test provides a plethora of different premade rules corresponding to some of the industry standards that are commonly used. For example, premade rules are provided for the likes of MISRA-C [13], AUTOSAR C++[25], SEI CERT C [26] and many more. Some of the available premade code standards and categories of other premade rules can be found in figure5.1.

For the code standard that is used for this case study, all of the rules are already implemented in Parasoft C/C++ test. Configuring the tool to apply the rules in the MISRA standard was thus made by simply selecting the wanted rules, and saving them in a test configuration. To actually run the tests, one

Qualification of Tool for Static Code Analysis: Processes and Requirements for Approval of Static Code Analysis in the Aviation Industry