Development and Evaluation of an Artefact Model to Support Security Compliance for DevSecOps

(1)

Master of Science in Software Engineering February 2021

Development and Evaluation of an Artefact Model to Support Security

Compliance for DevSecOps

Pranavi Bitra

Chandra Srilekha Achanta

Faculty of Computing, Blekinge Institute of Technology, 371 79 Karlskrona, Sweden

(2)

This thesis is submitted to the Faculty of Computing at Blekinge Institute of Technology in partial fulﬁlment of the requirements for the degree of Master of Science in Software Engineering.

The thesis is equivalent to 20 weeks of full time studies.

The authors declare that they are the sole authors of this thesis and that they have not used any sources other than those listed in the bibliography and identiﬁed as references. They further declare that they have not submitted this thesis at any other institution to obtain a degree.

Contact Information:

Author(s):

Pranavi Bitra

E-mail: prbi18@student.bth.se Chandra Srilekha Achanta E-mail: chac18@student.bth.se

University Supervisor:

Prof. Daniel Mendez Fernandez Department of Software Engineering Advisor:

Fabiola Moyon

Technical University of Munich

Faculty of Computing Internet : www.bth.se

Blekinge Institute of Technology Phone : +46 455 38 50 00 SE–371 79 Karlskrona, Sweden Fax : +46 455 38 50 57

(3)

Abstract

Background. DevOps represents a set of principles and practices of the software development (Dev) and information technology operations (Ops) of the product life- cycle requirements. DevOps has become a buzzword in organizations because it is an agile software development offspring. Now-a-days, there is a shift in organizations from DevOps to DevSecOps, which is bringing in a higher level of security built into software delivery pipelines. DevSecOps ensures security is a core component in the workflow to implement secure development and operations processes of automating every aspect. Security inevitably includes issues like compliance in terms of security standards that are concerning with looming cybersecurity threats. There is little known about different concepts of assessing security compliance in terms of security standards in DevOps pipelines. Understanding the artefacts and their dependencies requirements in the software workflow are fundamental to demonstrate compliance.

The thesis study proposes to ensure the IEC 62443-4-1 standard for secure product development in industrial systems is incorporated into the artefact model to capture the information related to security compliance.

Objectives. The thesis aims to investigate the artefacts and identify its dependencies to develop and design an artefact model for DevSecOps. This artefact model has the possibility to measure security compliance with the IEC 62443-4-1 standard to ensure traceability in DevOps pipeline and evaluate the usability of it.

Methods. In this qualitative research, we have conducted a literature review with snowballing to gather information on artefacts that undergo synthesis to develop and design the artefact model. We have conducted interviews with practitioners to collect the data on the usability of the artefact model.

Results. The literature review with snowballing is done to identify ten papers in the final data set. We have identified 100 artefacts from the papers. The artefacts are categorized and matched according to practices and activities descriptions. The synthesis of the literature review artefacts provides the basis for designing the artefact model and its dependencies for DevSecOps workflow. The interview results are thematically coded and we have obtained a list of benefits, challenges, and security compliance barriers with DevOps pipelines. This process evaluates the practitioners’

understanding of the designed artefact model usability in the industry to assess the standard’s security compliance.

Conclusions. The research study identiﬁes the artefacts that help with developing the artefact model. It provides the practitioners’ understanding of the usability of the artefact model in the industry to meet the secure software development product life-cycle requirements according to the IEC 62443-4-1 standard. The results demon- strated the evidence of assessing the security compliance for DevSecOps workﬂow in DevOps pipeline.

Keywords: DevOps, DevSecOps, Security Compliance, Standard, Artefacts

(4)

(5)

Acknowledgments

We wish to express our sincere gratitude to those who have helped us in the successful completion of this thesis. We want to thank our thesis supervisor, Professor Daniel Mendez of the Department of Software Engineering at Blekinge Institute of Technology. He had many insightful conversations during the development of this thesis ideas, and valuable suggestions on the topic.

We would also like to thank our advisor, Fabiola Moyon, who focuses on security in software engineering at the Technical University of Munich. We are very grateful for her continuous support to provide the necessary information and keen observa- tion while preparing the thesis. Both of them have steered us in the right direction through the process of researching and writing the thesis. We are thankful to both of you for the guidance, support, and encouragement which has been invaluable throughout this research.

We also express our profound gratitude to one and all who, directly or indirectly, have lent their unfailing support and continuous motivation throughout this study.

iii

(6)

(7)

List of Figures

2.1 DevSecOps . . . 7

2.2 The IEC 62443 Standard series: Security practices for industrial automation and control systems [1]. . . 11

3.1 Research Design . . . 17

3.2 Literature Review with Snowballing Steps . . . 19

3.3 Snowballing Procedure . . . 22

3.4 Interview Steps . . . 25

3.5 Interview Process . . . 27

4.1 Paper Publications throughout the years . . . 29

4.2 Concept Centric Matrix . . . 31

4.3 IEC 62443-4-1 Standard artefacts . . . 34

4.4 Legend of Artefact Model . . . 40

4.5 Artefact Model . . . 41

5.1 Theme Colors Example . . . 54

A.1 Start Set . . . 73

A.2 Mendeley Start Set . . . 73

A.3 Final Data Set . . . 74

A.4 Google Sheet Data 1 . . . 75

A.5 Google Sheet Data 2 . . . 75

B.1 Google Sheet Data Analysis 1 . . . 81

B.2 Google Sheet Data Analysis 2 . . . 81

C.1 Interview Invitation . . . 83

C.2 Interview Protocol 1 . . . 84

C.3 Interview Protocol 2 . . . 85

C.4 Interview Questionnaire 1 . . . 86

C.5 Interview Questionnaire 2 . . . 87

C.6 Mural Workspace . . . 87

C.7 Codes for Themes Page 1 . . . 88

C.8 Codes for Themes Page 2 . . . 88

vii

(10)

(11)

List of Tables

3.1 Search Strings . . . 20

4.1 Final Data Set . . . 30

4.2 Research Artefacts . . . 33

4.3 Artefacts Analysis . . . 46

5.1 Demographics of interviewees . . . 49

5.2 Interviewees level of knowledge on selected topics of the research . . . 50

5.3 Theme 1: Beneﬁts . . . 54

5.4 Theme 2: Challenges . . . 56

5.5 Theme 3: Security Compliance in DevOps Barriers . . . 57

A.1 Practice 1 Security Management(SM) Artefacts . . . 76

A.2 Practice 2 Security Requirement Speciﬁcation(SR) Artefacts . . . 77

A.3 Practice 3 Secure Design(SD) Artefacts . . . 77

A.4 Practice 4 Secure Implementation(SI) Artefacts . . . 78

A.5 Practice 5 Secure Veriﬁcation and Validation Testing(SVV) Artefacts 78 A.6 Practice 6 Management of Security-Related Issues(DM) Artefacts . . 78

A.7 Practice 7 Security Update Management(SUM) Artefacts . . . 79

A.8 Practice 8 Security Guidelines (SG) Artefacts . . . 79

ix

(12)

(13)

List of Abbreviations

DevOps Development and Operations

DevSecOps Development, Security and Operations IACS Industrial Automation and Control Systems UML Uniﬁed Modeling Language

IEC International Electrotechnical Commission CI/CD Continuous Integration/Continuous Delivery SDLC Secure Development Life-Cycle

BPMN Business Process Modeling Notation SLR Systematic Literature Review

RA Research Artefact

4-1 IEC 62443-4-1 Standard 4-2 IEC 62443-4-2 Standard

SM Security Management

SR Security requirements

SD Secure Design

SI Secure Implementation

SVV Security Veriﬁcation and Validation Testing DM Management of Security-Related Issues SUM Security Update Management

SG Security Guidelines

xi

(14)

(15)

Chapter 1 Introduction

DevOps is a combination of cultural philosophies, practices, and tools that increase an organization’s ability to deliver products faster. DevOps became popular, and many organizations are adopting a set of practices that combines software development (Dev) and technology operations (Ops) [2]. DevOps is bridging the gap between traditionally separate development and operations departments to work more closely at every product’s life-cycle stage. A DevOps pipeline represents the product’s life- cycle fundamental stages that include develop, build, test, and deploy. There are concerns raised in the produced software product security aspects during the stages, and there is a need for a secure development process. DevOps is a powerful paradigm shift, and organizations often do not understand how security ﬁts in the pipeline.

DevSecOps provides security at every stage or cycle of the pipeline to increases the security level.

DevSecOps has become a new trend to attempt and create modern security practices into the fast and agile world of DevOps [3]. It promotes collaboration between the development teams (Dev), security teams (Sec), operational teams (Ops) to have shared responsibility to incorporate security into the product from the start. DevSecOps builds on the best practices of general DevOps and advocates that security should be built into the product. It requires DevOps practices to make them an integral part of the information ﬂow for the possibility of implementing security [2]. It follows a secure development process to ensure assessments against industry-recognised security compliance standards.

A security requirement is a secure feature that ensures security properties of software product is being satisﬁed when derived from industry standards. Any organization demands to meet the security requirements as a set of standards against the organization’s activities and processes that can be compared as either compliant or non-compliant. DevSecOps entails cybersecurity as the main cause for vulnerabilities in industrial organizations which inﬂuences security in the pipeline. Organizations that focus on large industrial automation process need DevSecOps with cybersecurity compliance standards to strengthen security. Regulators demand that industrial companies follow standard best practices so software products are less vulnerable to attacks. Cybersecurity is obtained through systematic development, and the practitioners should be aware of the vulnerabilities and security issues [4]. The IEC 62443 standard series aids in applying the necessary mitigations in the organization.

1

(16)

2 Chapter 1. Introduction The IEC 62443 family of standard series contains product development life-cycle requirements related to cybersecurity for products intended for use in the industrial automation and control systems (IACS) environment. The security standard recommended in particular is IEC 62443-4-1 and IEC 62443-4-2. When it comes to the standard background, IEC 62443 consists of a series of standards for network and system security published by the International Electrotechnical Commission (IEC).

This research particularly focuses on the Group 4 requirements for industrial automation and control systems where the 4-1 describes the process requirements for secure product development [5] and the 4-2 discusses further into the technical requirements for IACS components. The 4-1 also provides guidance on how to meet the requirements described for each element for practitioners of any automation and control products where security is a concern [1]. Hence, we chose to utilize the IEC 62443-4-1 standard requirements instead of IEC 62443-4-2 requirements in this thesis. IACS applicable to all industry sectors has a signiﬁcant need of integrating DevSecOps with the standard for secure development activities or artefacts.

Artefacts play an important role in software development processes, and they can appear in different representations and structural properties. Artefacts represent outputs and deliverables of workflow, having a sequence of tasks needed to prove compliance [6]. Software development has artefacts like security tool reports, code snippets, log files and other documentation [5]. They are the objects created as part of the software product development in the DevOps pipeline to describe the objects of security requirements that provide traceability and demonstrate compliance [7].

Artefact model for DevSecOps is developed to capture all relevant artefacts and their dependencies integrated with 4-1 standard security requirements. It provides a basis to ensure traceability in the result structures between the activities generated by the practices and the requirements, using functional specifications relating to the respective artefact’s content through the pipeline [8]. Artefact model is used to guide certain security aspects through DevOps automation and security compliance assessment [3]. So, artefact model is a significant part concentrating on the artefacts in DevOps pipeline. It creates a DevSecOps workflow that captures the results and reveals traces of practices related to assessing security compliance. Practitioners in an organization can use the artefact model to assess security compliance according to the standard. They can show with artefact model that they meet the secure software development product life-cycle requirements.

1.1 Problem

Due to organizations facing cybersecurity threats in large industrial automation sectors, the current process requirements are not sufficient enough. These organizations require security compliance with security standards in the pipelines to improve security and help with workflow of the software product life-cycle. To deliver the requirements faster, companies utilize DevOps, but compliance with security standards is yet a remarkable challenge. They need to prove while working with pipelines that compliance is achieved. This thesis fills the research gap of assessing security

(17)

1.2. Aim and Objectives 3 compliance in accordance with the security standard in DevOps pipeline by designing and evaluating an artefact model. The designed artefact model contains artefacts and its dependencies information to measure security compliance for the desired security standard in secure software product development. The traceability is in between the artefacts through the pipeline. Hence, the thesis’s scope is to develop and evaluate an artefact model that shows artefacts, contents, and dependencies for security compliant pipelines. The motivation behind the selected direction of research is the need to understand these relations between artefacts and design a vision of formalizing them using artefact model.

There are various compliance and security driven approaches, but there is no consis- tent proposal through which artefacts are versatile to support security compliance in DevOps. The security standards deﬁne mandatory artefacts needed for DevSecOps.

There is a need to make artefacts explicit and their relation with standard artefacts through a DevOps pipeline. So, with the IEC 62443-4-1 standard, the artefacts identified to effectively design artefact model and remediate security findings to ensure required security is pushed to DevOps pipelines. The purpose is to increase aware- ness for security compliant pipelines to provide an important basis for understanding security compliance via an artefact model. It can help the practitioners in industry in fulfilling the continuous security compliance to design an artefact model. It also helps them integrate with the IEC 62443-4-1 security standard that addresses cybersecurity from the point of view of secure software product development into DevSecOps.

1.2 Aim and Objectives

Aim: This research aims to identify artefacts from literature to develop and evaluate an artefact model for DevSecOps with the IEC 62443-4-1 standard that is compliant to DevOps pipeline. To achieve this aim, we formulated the following objectives:

• O1: To investigate which artefacts are proposed currently in literature and how they are related to the security of a software product in a DevOps pipeline.

• O2: To perform a synthesis of the literature to develop and design a model of artefacts and their dependencies as recommended by security standard for DevSecOps workﬂow.

• O3: To evaluate the usability of the artefact model with practitioners in industry by assessing the developed model with its content and information ﬂows related to security compliance from the practitioners’ perspective.

1.3 Scope

The thesis aims to develop, design, and evaluate an artefact model that captures a blueprint of signiﬁcant artefacts and their inter-dependencies. This artefact model serves as a basis to deﬁne security-compliant DevOps pipelines for security compliance in an everyday industrial project or product settings of the organization. This

(18)

4 Chapter 1. Introduction thesis research can support a variety of future work as it provides, for instance, a conceptual basis for consistency and traceability between the artefacts through the DevOps pipeline creating DevSecOps workﬂow that focuses on assessing the security compliance in the industry.

1.4 Outline

The thesis structure as follows, Chapter 1 consists of introduction overview, the research problem, aim and objectives, and the research scope. Chapter 2 provides the background and related work of the topics in this study. Chapter 3 presents research questions and the research methods used to implement this thesis. Chapter 4 provides the literature review with snowballing and synthesis results obtained and described in the analysis. Chapter 5 presents the evaluation of the artefact model through interview results and analysis. Chapter 6 discusses the research questions and threats to validity. Chapter 7 provides the conclusions made in this thesis, and suggestions are discussed in future work.

(19)

Chapter 2 Background and Related Work

2.1 Background

2.1.1 Secure Software Product Development

Secure software product development is important to ensure that industrial companies utilize the products in a secure manner. It integrates security practices into the software development life-cycle to capture the industry standard security activities or artefacts in DevOps pipelines. Necessary secure requirements are must for the artefacts of a secure software development activity in the workflow [9]. It specifies a set of best practices and represents how practitioners can apply various artefacts produced during software development. The initiatives are aligned to an assessment of security compliance assure product development artefacts meet the standard requirements. Some effective methods and tools support the development of secure products for practitioners in the industry.

2.1.2 DevOps

DevOps is a convergence of emerging concerns from the software development community and operations community. From the development perspective, Agile also comes into the picture to play a major role in the software life-cycle. From the operations perspective, production software focuses on automating processes and standardization [10]. DevOps is a word to explain the collaboration between the development (Dev) and operations (Ops) teams. Agile and lean practices in a product life-cycle perspective are adopted to represent a change in organizational culture and focus on rapid service delivery. DevOps transforms software product development life-cycle to focus on using automation to manage many tasks like building, testing, and deploying software products which also oﬀer security.

DevOps is applicable and relevant to any industrial company that increases the workﬂow through technology for maintaining quality, reliability, and security [11].

There are three ways of presenting the underpinning principles from which the ob- served DevOps behaviour and patterns are derived in some aspects. The first way enables fast workflow in one direction from development to operations. The second way enables fast and constant feedback flow from right to left at all value stream stages. The third way enables the creation of a trust culture of continuous learning and experimentation [11]. With these three ways, the approach can be redefined with

5

(20)

6 Chapter 2. Background and Related Work security work by establishing security workﬂow, ensuring instant security feedback, and encouraging security culture.

DevOps practices aﬀect developers throughout secure software product development life-cycle. DevOps is a set of practices intended to reduce the time between coding for software system and the system’s change being placed into normal pipeline production while ensuring high quality [12]. The practices that help organizations comply with innovating faster through automation and streamlining with proper tooling. The practices are comprised of activities which consist of smaller actions or processes. The coordination and execution of all the activities are required during software’s development and operations transition.

Several teams explored the industrial companies which are making progress in tran- sitioning from Agile methods to DevOps practices. DevOps artefacts are embedded to build the deployment pipeline using automation. To support it, industrial companies use a combination of tools to construct a secure software product and technical aspect of the development environment to a test environment and staging environment to the production environment. The practices come from industry standards wherein requirements include all artefacts of the secure development life-cycle for security integrated into the product. It should be easy for practitioners to integrate practices with the support of appropriate tools smoothly into the DevOps pipeline.

Many practices strive to achieve compliance as code by building codified compliance policies into development and operation, enabling these guardrails to become an integral part of teams [13]. At times, security is missed out in part of the workflow and eventually leads to exploitation. Security is a crucial element in DevOps implementation because it influences any organizations process.

2.1.3 DevSecOps

DevSecOps is a new trend where the needs of security in DevOps shall be met.

It includes artefacts, practitioners experiences with practices they perform in security, compliance, process automation and tools for DevSecOps [2]. It creates a collaboration between the development (Dev), security (Sec), operation (Ops) teams incorporating the security standards [3]. The integration of standards requirements in DevOps pipelines and processes can help the teams work and highlight existing tools to strengthen security. When DevSecOps is used to create security in pipelines, it can help combat the future cyber threats.

DevSecOps practices to encourage secure coding practices and improve cybersecurity hygiene that can impact the advancements in planning, tools, and training. It builds automated security practices into the software product development life-cycle more often and much earlier in process. DevSecOps spans the entire life-cycle integrating security seamlessly from planning and design to coding, building, testing, release, with real-time deployment, operations and maintenance, as seen in Figure 2.1 [14].

The changes to compliance as code policies or workﬂows should be approved and documented before instrumenting in code. It is being driven by software quality and security practices, compliance requirements and avoiding risk in cybersecurity [15].

(21)

2.1. Background 7 There is a constant flow of code, data and artefacts that needs to be involved in the securing process of the pipeline. DevSecOps integrates various challenges into a coherent and effective approach to identify security issues and vulnerabilities early in the secure software development process rather than after the product are released. It also associates with fixing security compliance by building security into every stage, gathering the standard requirements. DevSecOps as, a practice and a concept have several industrial organizations implement it as a solution to their security issues [16].

Figure 2.1: DevSecOps

2.1.4 Pipelines

A pipeline in secure software product development consists of a tools, workflows, and automated processes, enabling the teams to build and deploy software. Pipeline execution generates many artefacts such as build production logs, bug tracks, param- eter configurations, and temporary files. The automating tools and practices mean security, regulatory and compliance requirements can be embedded as code into the software product delivery pipeline to ensure that any code deployed is secure and compliant [16]. The core of the DevOps pipeline consists of continuous integration and continuous delivery (CI/CD). Continuous integration is a software development practice in which developers of the team frequently integrate their workflow several times per day [17]. Continuous delivery is a software development process to build a deployment pipeline so that software can be delivered at any time [17].

The ideal process of hardening the pipeline’s security is to identify the security standard requirements and vulnerabilities in build and deploy activities or artefacts to know which can be trustworthy or untrustworthy components [18]. The important consideration of building a DevOps pipeline is integrating the code base’s automated security to deliver quality secure code in a timely manner. With DevSecOps, security compliance requirements and tools are integrated into the development and CI/CD pipeline to assess workﬂow traceability. There should be strong collaboration among

(22)

8 Chapter 2. Background and Related Work the development, operations, testing, and security teams, and automate as many security processes as possible.

2.1.4.1 Stages

A DevOps pipeline is a set of practices that keep the secure software product development process organized and focused. The structure of the pipeline representation is to divide it into stages, as seen in Figure 2.1. The development stages are plan, code, build and test. The operation stages are release, deploy, operate and monitor [19].

• Plan: This stage involves designing a road-map of the entire workﬂow requirements that will guide the team along the process.

• Code: This stage involves developers to create software code in the repository.

• Build: This stage is where the software product is built using an integrated code in the repository.

• Test: This stage is where various tests are executed, and if issues are found, it is sent back for resolution.

• Release: This stage authorizes the software product release into a targeted environment.

• Deploy: This stage is where the ﬁnal version of the software product is conﬁg- ured and deployed.

• Operate: This stage makes sure everything is running smoothly and gathers feedback on the software product.

• Monitor: This stage monitors the software product to identify and collect issues.

2.1.5 Artefacts

An artefact is a by-product of software development process. It is something that is created, so a piece of software can be developed. Some artefacts explain how a piece of software is supposed to work, while others allow that software program to run. In the reviewed literature about DevOps and DevSecOps, most artefacts tend to enforce association among the development, security, and operation teams to integrate the security principles into DevOps. A DevOps artefact focuses on achieving a goal that has a tangible result in the pipeline. A security practice is a collection of artefacts that can be combined based on the similarities within those artefacts.

The automation and conﬁguration management helps to implement traceability of code thus makes it easier to identify the vulnerabilities of immutable artefacts [16].

Automation concept is based on the type of security tools in the pipeline approaches, which may diﬀer in terms of push or pull mechanisms of artefacts from continuous integration to continuous delivery [17].

(23)

2.1. Background 9

The artefact’s role in the security process and activities used to create or modify those that will inﬂuence the security techniques. Artefacts are deliverable that can be produced, modiﬁed, or used by the sequence of tasks that value the role. It also supports requirement engineering processes in the industry [6]. DevOps considers an artefact that can be adapted to match the industrial company’s unique environment to assess security compliance with the standard. The practitioners can apply various practices to software artefacts produced during the development, which means software security will be cycled through as software product evolves.

2.1.5.1 Artefact model

Relationship between the artefacts in the DevOps environment can inﬂuence the use of security artefacts, activities and practices for collaboration in this research to develop the artefact model for DevSecOps. The artefact model provides a basic structure for the artefacts and the relation between the artefacts. It constitutes and supports traceability concepts for a relationship to provide a basis for linking and tracing between the artefacts [6].

2.1.6 Security Compliance

Security and compliance are top of mind in an industrial company that follows secure software product development [16]. By coding the compliance requirements and integrating into the security policy automation makes the pipeline task simpler.

Compliance is achieved when using automated security conﬁguration assessment to reduce the vulnerabilities and maintain continuous security compliance [16]. De- vOps focuses on the automation, which enables continuous compliance in industrial organizations to integrate compliance as code. It minimizes the compliance burden where compliance requirements are deﬁned in a human and machine readable format, enabling automation and rigidity. It implements secure software product development life-cycle where compliance would not be a documentation centric endeavour of paperwork but automated reports, checks, balances from the tools in the delivery pipeline. A reliable DevSecOps solution brings all the key elements of compliance best practices, policies, and tools into each stage, ensuring the security.

2.1.7 Security Standards

Standards are the best way to give economic benefits in the present world [20]. Pro- tecting personal records is important in any field, i.e., medical, politics, information technology and many more. Security standards provide necessary tools, policies and procedures to protect sensitive information. The main goal of using standards is to reduce risks in the software products and prevent future risks. Security says standards are everyone’s responsibility in implementing secure software product development. The coding standards should be checked constantly against the security recommendations, and changes need to be verified in the process. A cybersecurity standard specifies functional and assurance requirements within the product, system, or process to enable consistency among the teams [21]. This thesis environment is

(24)

10 Chapter 2. Background and Related Work based on a large company where security practices are still challenging along the product life-cycle. This thesis is mainly focusing on IEC 62443 family security standards and further concentrating on IEC 62443 4-1 standard in particular. These standards refer to the organization during secure software product development to map appropriately to one or more related compliance rules.

2.1.8 The IEC 62443 Series

The IEC 62443 is a series of standards that provides a ﬂexible framework to address and mitigate current and future security vulnerabilities in industrial automation and control systems (IACS). International Electrotechnical Commission (IEC) maintains the standards which can be seen in Figure 2.2 .

To fully articulate the systems and components the IEC 62443 series address, the range of coverage may be deﬁned and understood from several perspectives, including the following [22]:

• Range of included functionality.

• Speciﬁc systems and interfaces.

• Criteria for selecting included activities.

• Criteria for selecting included assets.

2.1.8.1 IEC 62443 4-1 and IEC 62443 4-2 Standards

IEC 62443 4-1 and IEC 62443 4-2 standards are branches for the IEC 62443 family that provide requirements supporting the IACS security [1]. This research is focused on IEC 62443 4-1 standard which released in 2018 as it already has lot of information and solely focuses on the secure product development life-cycle requirements. The IEC 62443 4-2 standard which released in 2019 focuses on the technical requirements in IACS. Therefore, only the 4-1 standard is considered in this research. A brief description of 4-2 standard is given because it is part of IEC 62443 Group 4 series.

IEC 62443 4-1

This standard focuses on process requirements for the secure development of products used in IACS. The main benefiter for using this standard is the product development life-cycle. This standard defines the secure development life-cycle (SDL) to develop and maintain with security requirements definition, secure design, secure implementation, security verification and validation, secure defect management and product end-of-life. Such requirements can be used to develop, maintain and retire hardware, software and firmware for any product [1].

The primary goal for these requirements in the standards is to provide a framework for several aspects in IACS. This framework’s application provides a secure system or

(25)

2.1. Background 11

Figure 2.2: The IEC 62443 Standard series: Security practices for industrial automation and control systems [1].

product from any expected level of threat in software development lifestyle. The sec- ondary goal includes requirements to align the process with elevated security needs of the product to IACS security users. It means the process needs to generate documents of diﬀerent aspects uncovered in the product. A key concept is defense in depth strategy philosophy of the secure product life-cycle [1].

The IEC 62443 4-1 standard requirement have eight practices that are listed ac- cordingly:

1. Security management (SM): This practice ensures that security related activities are adequately planned, documented and executed through the product’s life-cycle. These activities related to security organizational processes can be executed and documented for the eﬀectiveness of secure product development life-cycle.

2. Security requirements (SR): This practice describes the process that should be employed to ensure secure capabilities that are required for a product security context. These capabilities consist of both physical and cybersecurity such as encryption, auditing, authentication, authorisation, and network security.

3. Secure Design (SD): This practice contains processes that ensure a product is secure during design. It applies to the whole architecture to design the indi-

(26)

12 Chapter 2. Background and Related Work vidual components. It is employed to develop and document a secure by design that identiﬁes and characterizes the product’s physical and logical interfaces.

4. Secure Implementation (SI): This practice has processes to ensure that the product features are implemented securely. The features include requirements of the product from the supplier for hardware and software development.

5. Security Veriﬁcation and Validation Testing (SVV): This practice explains that the security testing has been properly performed and documented.

Security testing conﬁrms whether all the security requirements have been met and the product is maintained. It also ensures that the defense in depth strategy is in place when there are faults in the process.

6. Management of Security-Related Issues (DM): This practice works on the processes to handle the product security-related issues that attack the defense in the depth strategy in the security product context. The supplier will release a process to receive and track the security-related issues reported by internal and external sources such as developers, testers, users and third-party suppliers.

7. Security Update Management (SUM): This practice describes the processes on ensuring that the security updates associated with the product are tested for regression and provided for the users promptly. The product supplier shall employ a process to verify that the security update by the internal team solves the vulnerabilities, and the security update does not contradict other operational, safety or legal constraints.

8. Security Guidelines (SG): This practice describes the processes that provide documentation on how to integrate, conﬁgure, and maintain the defense in depth strategy of the product in the security product context. These processes support the product installation, operation and maintenance.

IEC 62443 4-2

This standard provides a detailed technical control system component requirements linked with the seven foundational requirements in IEC 62443 1-1 and the requirements for control system capability security levels and its components [23].

The seven foundational requirements (FRs) are:

• identiﬁcation and authentication control (IAC),

• use control (UC),

• system integrity (SI),

• data conﬁdentiality (DC),

• restricted data ﬂow (RDF),

• timely response to events (TRE), and

(27)

2.2. Related Work 13

• resource availability (RA).

The seven foundational requirements are base for deﬁning the control system security capability levels. Deﬁning security capability levels for the control system component are the goals for this standard [23]. Each foundational requirement has four security levels based on IEC 62443 3-3, so this standard considers of both IEC 62443 1-1 and 3-3.

2.2 Related Work

The thesis topic is a relatively new concept in the industry, and there is not much existing relevant published work at present. Hence, the research study for fulﬁlling the aim uses the existing publications and standards as a part of the thesis. A set of documents were given at the start of the thesis to get an idea on the topic and the contributions to this area are very low in real-world industry scenarios. The contributions mentioned from this thesis would ﬁll the existing gap to achieve the aim and objectives. Few published research works we reviewed are presented here.

Crowley et al. [24] discussed DevOps capability and its adoption. They defined DevOps in its origin and its information technology capabilities. They focused on understanding DevOps basic concepts about their benefits and challenges based on how the DevOps was developed in capability. Several DevOps adoption practices, according to the technology perspective, were discussed. Based on DevOps practitioners’ interviews, it was concluded in the paper that the adoption of DevOps was a positive move for software development. They recommended ascertaining the empirical evidence to prove that the benefits obtained were from DevOps alone. This approach gives scope to how far the DevOps teams can bring security and compliance as an approach.

Myrbakken et al. [2] reviewed to understand DevSecOps fully and concluded there should be a focus on DevOps. They gave an overview of what DevSecOps means for an organization regarding principles, practices, and standards to keep up with security methods. Aspects related to the definition, security best practices, compliance, process automation, and tools for DevSecOps were discussed. They also identified the challenges the organizations faced and the benefits that were gained when implementing DevSecOps.

Mohan et al. [3] described DevOps security aspects from academia and industry to demonstrate DevSecOps was not a buzzword but an essential subject for further studies. They identified the activities that impact the security software the organizations use to integrate security into DevOps. The aspects discussed were definition, security best practices, compliance, process automation, tools for DevSecOps, software configuration, team collaboration, availability of activity data, and information secrecy. These aspects give a basis of what DevSecOps means and the wide variety of knowledge for the organizations to adopt DevSecOps that helps understand and assess the security compliance with the standards.

(28)

14 Chapter 2. Background and Related Work Mendez et al. [6] discussed the term artefact in software engineering. It empha- sized an underlying basis for defining the notion of related, equivalent and refined artefacts. They stated the clear responsibilities that capture the workflow of established concepts of a particular domain through artefact models. This knowledge helps in understanding the development of artefact models in a broader context of software engineering practices. It contributes a meta-model with artefacts at three levels of perception with processing steps, forming a basis for tracing between artefacts. This paper gives details on designing a model which is helpful for this thesis to achieve the workflow.

Dännart et al. [5] provided a foundation for secure product development in industrial control systems combined with security compliance tailored to the security standard IEC 62443-4-1. They precisely described the artefacts that were introduced and improved to build the Security-standard Compliance Assessment Model that delivered compliance in the Agile software engineering environment. It led to a gap in knowledge to contribute to the area that supports DevOps capabilities. This paper solely focuses on agile with the standard and with this concept information this thesis achieves DevOps with the standard.

Soares et al. [25] proposed integrating security activities into DevOps pipelines that described secure compliant product development of IEC 62443-4-1 standard. This concept was the 4-1 activities obey the regulations and standard in particular cybersecurity requirements in industrial control systems. The high level 4-1 compliant pipeline speciﬁcation is a visual design representation of automated tool support that stores and generates artefacts. This research is the core understanding of standards compliance to DevSecOps. This research led to a gap in knowledge to further built to support 4-1 standard to identify artefacts and develop an artefact model. This paper focuses on 4-1 compliant pipeline speciﬁcations of the 4-1 standard practices which is helpful for this thesis to achieve the categorization of the artefacts into DevOps pipeline.

(29)

Chapter 3 Research Methodology

The thesis’s environment is a large industrial automation company implementing security practices and activities while developing and integrating software products.

DevSecOps entails cybersecurity as the main cause for vulnerabilities in industrial organizations which inﬂuences security in the DevOps pipeline. A link between the artefacts and the IEC 62443-4-1 standard is necessary to provide traceability in DevOps pipeline and demonstrate by assessing the security compliance related to each other. It is essential to design an artefact model that contains information to measure security compliance in secure software product development and evaluate the usability of it. The purpose of developing and designing the artefact model is to support the practitioners’ process requirements for the secure development of products used in an IACS organization.

3.1 Research Questions

According to the aim and objectives described, we formulated the following research questions.

• RQ1: What artefacts are proposed in literature in order to demonstrate security compliance when applying DevOps pipelines?

Motivation: The motivation for selecting this research question is to understand the basic concept of artefacts in DevOps, know the current process of the industry compliance to standards, security requirements and practices that practitioners currently follow. The artefacts can be identiﬁed from present literature used for the security compliance in the DevOps pipelines. This question provides a base for the RQ2 question on developing and designing the model.

• RQ2: How are these artefacts related to each other and how can they be used for the DevSecOps workﬂow?

Motivation: The motivation for selecting this question is to construct an artefact model from the identiﬁed artefacts from the above question. The artefacts are put together to understand the dependencies for DevSecOps workﬂow that can be used in the industry. This question provides a base for RQ3 on assessing the compliance of the standard with the artefact model.

• RQ3: What are the beneﬁts, challenges and barriers faced by practitioners considering the perspective of setting up a security compliant DevOps pipeline and assessing its compliance?

15

(30)

16 Chapter 3. Research Methodology Motivation: The motivation for selecting this research question is to evaluate the beneﬁts, challenges and security compliance barriers of usability of the artefact model by the practitioners. It also perceives to understand the potential usage of the model and assess security compliance with IEC 62443 4-1 standard.

3.2 Research Methods

Qualitative research is the suitable research methodology for this software engineering research. We found it useful when the purpose is to design a model to integrate the security standard into DevOps pipelines. The study’s nature is explorative research that is concerned with findings from observations, collecting and analyzing the data. The selection of the research method is crucial as it defines how to conduct the research with a flexible design. Many elements considered in this study include research questions, design, scope, and many research methods like survey, experiment, action research, case study, ethnography, and grounded theory [26]. The applicabil- ity of each of these methods was scrutinized alongside quantitative methods. The research design of the work done is seen in Figure 3.1.

• RQ1: A literature review based on snowballing is conducted for this research question. The publication landscape is still scarce, so snowballing is followed rather than conducting a systematic literature review. This snowballing procedure allowed to gather the existing literature based on initially given key publications about the thesis topic to increase knowledge and understanding.

The documents given beforehand consist of a set of papers to know the methods, artefacts, processes, and the standards implemented into secure product development. Systematic mapping study only identifies and classifies evidence covering a broad field that is not suitable because this thesis concerns a particular standard.

• RQ2: The method used for this research question is a narrative synthesis of qualitative research that involves interpreting the data from the snowballing method to construct an artefact model. The research searches on artefacts in DevOps pipelines where new conceptual understandings can emerge, leading to a design that explains how they are related to each other for DevSecOps.

The synthesis is chosen over a case study because it is diﬃcult to generalize and harder to interpret the dependencies when security compliance comes into action in a company.

• RQ3: The method used for this research question is the interview study of practitioners in the industry to evaluate the artefact model. The practitioners’ data is further analyzed based on how the artefact model aﬀects the organization’s security compliance issues. The interviews are chosen over survey because of security and conﬁdentiality aspects to gain explicit information from the practitioners. As some companies have security restrictions for practitioners to participate in a survey, interviews are chosen.

(31)

3.2. Research Methods 17

Figure 3.1: Research Design

Alternative Methods

A case study is used as initial in-depth investigations of some phenomena to derive new hypotheses and build theories to reveal the mechanisms for which cause-eﬀect relationships occur [27]. It is used to show how or why certain phenomena occur, whereas, this thesis focuses on designing a model that represents the insights of researchers through narrative synthesis not on building theories, and selecting cases.

Hence, it is not suitable.

Action research attempts to solve real-world problems while simultaneously studying the changes inﬂuencing certain aspects of solving the problem [27]. Since the thesis topic is still new, this method requires real-time data in long term commitment since the eﬀects may take years to emerge. Hence, it is not suitable.

A survey is used to ﬁnd the characteristics of a broad population of individuals.

The deﬁning characteristics are selecting a sample from a well-deﬁned population to answer the questionnaires for data collection [27]. The thesis topic focuses on individuals who work with IEC 62443-4-1 standard to get more accurate results. However, the population of individuals to select in the required area is a lot less. Hence, the concentration is on interviews instead of a survey.

An experiment is deﬁned as “an empirical enquiry that manipulates one factor or variable of the studied setting” [26]. It focuses on an existed study and studies the deep relationships between the entities. In the present study, some modules are still yet to be discovered and studied. There is no control over the entire study, but, there is control over the data presented in this study. The manipulation cannot be

(32)

18 Chapter 3. Research Methodology measured in this study and the statistical analysis is only based on the data gathered.

Hence, this method is not adopted for this study.

Hence, we ﬁnally decided that this thesis consists of a research design that involves a literature review, synthesis, and interview study. The research questions proposed have a scope of methods that lay a foundation for a successful evaluation that makes it easier to decide what data to collect, analyze the collected data and report it.

3.2.1 Literature Review

A literature review constructively informs the reader about accredited scholars and researchers published work on a topic [28]. The first step in this study is to do a literature review using the snowballing method to introduce the research context and identify the topic’s relevant papers. Literature review process identified the relevant scientific data related to the thesis topics which helps understand the security compliance measures in DevOps pipelines according to the standard and the artefacts produced from practices and activities in the pipelines. Systematic Literature Review (SLR) is an other alternative to conduct this research to identify precise pieces of information, analyze and interpret all available evidence related to a specific research question [26]. However, SLR is not considered for this research because of verification of each paper, narrow and precise research area to analyze making it restrictive.

Hence, we have chosen Literature review that followed similar principles due to time and resource constraints.

3.2.1.1 Snowballing

In detail, the snowballing method was used to identify the relevant papers that give information on artefacts of security status in DevOps and DevSecOps along with compliance with the given security standard. In this research area, there were a limited set of papers, so it was important to decide on the inclusion of a paper for snowballing to build knowledge and look into the useful research [29]. Snowball sampling is a non-probability and non-random sampling method that has been used when the characteristics of a particular sample are rare and diﬃcult to ﬁnd [30].

The case is to ﬁnd more literature about the topic in this speciﬁc focus area having terminology related to DevOps and DevSecOps area. The snowballing technique can be used for conducting qualitative research where the potential data sources are hard to locate.

A starting set of research papers were considered from the available content in Google Scholar search engine and some documents provided beforehand from the industrial research conducted which are stored in the Mendeley. The reason to choose Google Scholar as a database search engine because it provides all kinds of research material including, journal papers, technical reports, books and other materials. It includes all the other databases like Scopus, IEEE Xplore, SpringerLink and ACM to use the search strings to obtain relevant articles. Google Scholar search engine focused on ﬁnding established literature. The reason for choosing Mendeley was because the documents provided beforehand were shared in that platform to help with the thesis

(33)

3.2. Research Methods 19 research. Some tools were used, such as Mendeley, Google Sheets, Google Docs to organize the searching process and the analysis of the published literature. Essential planning for the literature review with snowballing is presented in steps shown in Figure 3.2.

Figure 3.2: Literature Review with Snowballing Steps

3.2.1.2 Search Strings

In the database search, it was crucial to identify the keywords, formulate the search strings, devise inclusion/exclusion criteria, conduct the database search and then snowballing technique was performed to ensure all relevant studies were covered. A

(34)

20 Chapter 3. Research Methodology good start set can be identified using Google Scholar, which has different literature from other databases. The keywords were identified from the research questions and objectives. Search strings were generated by combining the keywords using Boolean operators "AND" and "OR" with synonyms and terms related to keywords.

The keywords identiﬁed from related studies were: DevOps, Pipelines, DevSecOps, Artefacts, Security Compliance, Standards, Regulations

Five search strings were formulated since the thesis topic is new and ability to find the research material increases. Different search strings were used to gain knowledge about the topics and sub-topics that could be useful and gather relevant research material. Various search strings were used to find the papers. It resulted in producing many references based on the topic when the snowballing procedure was performed.

The search strings used in Google Scholar are listed in Table 3.1. We established the search strategy and supervisors reviewed it to make sure no papers were missed.

The reason for not finding many results was because of the less literature published on DevSecOps. However, while exploring different search strings, many references were found based on different keywords of the thesis. The reason for considering search strings for only Google scholar instead for individual electronic databases was because the research method was not a Systematic Literature Review (SLR).

Database Search string

Google Scholar: IEEE Xplore, ACM, Scopus, SpringerLink

SS-1: ("DevOps" OR "DevSecOps" AND

"pipelines") AND "security compliance"

AND "standards" AND "artefacts"

SS-2: ("DevOps" OR "DevSecOps" OR

"pipelines") AND ("security") AND ("compliance" OR "standards" OR "regulation") AND "artefacts"

SS-3: ("DevOps" OR "DevSecOps" and

"continuous") AND ("security") AND ("compliance" OR "standards" OR "regulation") AND "artefacts"

"pipelines") AND ("security") AND ("compliance" OR "standards" OR "regulation") AND "artefacts"

"pipelines") AND "security compliance"

AND "artefacts" OR "standards" OR "regulation"

Table 3.1: Search Strings

(35)

3.2. Research Methods 21 3.2.1.3 Selection Criteria

Inclusion/Exclusion criteria were deﬁned to remove the irrelevant and duplicate studies from the database search results. Research quality is often used as a criterion for making decisions about including or excluding particular research papers. We deﬁned the criteria and supervisors checked whether the criteria were correctly ap- plied. A set of predetermined conditions provided a basis for including or excluding certain studies.

Inclusion Criteria:

• Literature from Google Scholar search engine having diﬀerent databases like IEEE Xplore, ACM, Scopus, SpringerLink.

• Papers from the timeline 2000 to 2020.

• Subject area in the ﬁelds of Software Engineering and Computer Science.

• Search space having title, abstract, keyword.

• Quality of peer-reviewed publications.

• Papers that are written in English.

Exclusion Criteria:

• Papers not related to topic or research area.

• Papers that are not written in English.

• Papers that are not available in full text.

• Grey literature and unpublished papers.

• Papers that are duplicated.

3.2.1.4 Iterations

Snowballing is a way of conducting searches to identify research papers that are more relevant to the topic. In this approach, the new research papers were identified by looking into references and citations of the papers. The reason for using the snowballing technique along with search strings was because sometimes the terminology formulated may give irrelevant and unnecessary papers. We made sure the searching techniques were not confined to only a particular method. Using the snowballing method along with database search will increase the chance of finding relevant literature and the threat of losing important literature can be mitigated [29].

Backward snowballing and forward snowballing were conducted to decide a set of papers included in the ﬁnal analysis. Backward snowballing means using the reference list to identify new papers and forward snowballing means identifying new papers based on those papers citing the paper being examined using Google Scholar.

(36)

22 Chapter 3. Research Methodology The process of backward snowballing and forward snowballing was carried out in iterations. The newly identiﬁed research papers were examined to extract as much information as possible to be added to the start set [29]. The snowballing procedure was implemented in next iterations until no papers were found because the papers that mentioned the keywords were scarce. Hence, we have only performed the itera- tion once to make the information on artefacts and standard explicit.

Backward snowballing and forward snowballing was conducted in one step. We checked each paper independently to reduce the bias. The disagreements when searching and choosing papers were solved in joint discussions. If the opinions differed and could not be solved were escalated to supervisors. After filtering and removing duplication two papers were identified from the search strings. Five new studies were identified from those two papers after using the snowballing technique. After filtering and removing duplication three documents were identified from the Mendeley and included in the snowballing procedure, and no new papers were identified. The grey colored boxes represented the papers that were selected for the final data set according to the snowballing procedure. The final data set consisted of 10 research papers represented in the Figure 3.3 that depict snowballing procedure.

Figure 3.3: Snowballing Procedure

3.2.1.5 Data Extraction

The papers were thoroughly scanned, and relevant data were extracted at the end of snowballing. The research questions and the inclusion/exclusion criteria determined which research papers were included. Google Sheets were used to interpret the ﬁndings of the parameters like relevance to title and abstract of the paper. The

(37)

3.2. Research Methods 23 spreadsheet was designed to use for the data extraction to make the content simple without deviating from the research’s aim. We screened the title and abstract of the papers, and reference context was examined. The filtered references of the start set were determined as Yes/No decision by checking the title and abstract are related (for example see Appendix Figure A.1 and A.2). We performed data extraction where each other’s decisions were compared, and unlikely causes for selecting or removing papers were discussed. Then, the final decision whether the paper to be included or not was taken and added in the final data set.

An author-centric or a concept-centric approach is used to extract the literature review data. Author-centric approach presented a summary of relevant papers, and concept-centric approach determined the organizing framework. The concept-centric was found better than the author-centric because that method failed to synthesize the literature. Tables and figures are an effective means of communicating major findings and insights of the papers into concepts [31]. The concept-centric approach for the data extraction process facilitated with concept matrix that lists the key concepts of a topic along one axis of the matrix and the literature papers in which they were addressed along the other axis [32]. An initial understanding and description of artefacts focusing on DevOps and DevSecOps can be stated according to the practices and activities. The evidence of artefacts paved the way for the artefact model for more analytical and interpretive synthesis processes to follow in the next step.

3.2.2 Synthesis

Narrative synthesis has multiple studies related to the objective of research and further explains the findings [33]. The second step is the synthesis which allowed to focus on the findings for the thesis. This approach’s main objective was that this synthesis process would be like "telling a story" of the findings from the research papers. In summary, it can be told in four main elements: making a model that can explain how the findings work, developing an initial synthesis, searching relationships within the data, and then evaluating the robustness of the outcomes [34]. We have chosen narrative synthesis because it includes a relatively smaller number of research papers than other synthesis types. The number of research papers found for this thesis is small, so it is convenient to summarize findings from the papers.

Narrative synthesis integrates the existing knowledge and research findings emerging from the publications under consideration in the literature review, i.e. it searches the research papers to draw the findings from individual papers [35]. It provides guidance to structure the designing of a model, develop an initial description of the findings, information about the aspects of relationships, and assess the trustworthiness of synthesis. The obtained information on the artefacts proposed in the various publications helps conclude the qualitative findings to design a holistic artefact model.

The artefact model consists of the artefacts from the research papers and describes the relationships among them in DevOps pipelines leading to DevSecOps workﬂow.

It is an interpretive method with more interest in qualitative ﬁnding details for better understanding the synthesis evidence. Textual descriptions, tabulating the data,

(38)

24 Chapter 3. Research Methodology and visual representation of the relationships are common formats used to present the details while using narrative synthesis [33]. The artefacts were found during the literature review, and textual descriptions of relationships among the artefacts were described to get a better understanding to form a visual representation of the model.

The artefacts were tabulated, and respective columns were added to examine the characteristics further to explore the relationships. Identifying and analyzing the relationships between the research artefacts and the 4-1 standard artefacts was crit- ical to the quality of the process. In this step, the feedback was helpful from expert advisors as they have thorough knowledge about the topic and guided the way to match among the artefacts as accurate as possible.

Once the matching was done, the initial outline of the model was drawn according to the DevOps stages. The artefacts were inserted into respective stages, and relationships between them were drawn. The connections were based on the similarities between the respective research artefacts and respective 4-1 standard artefacts.

The comparison was made to see which research artefacts have common 4-1 standard artefacts and the connections were made between them along with a general description of rationale. Based on the literature study the artefact model was drawn and designed in an intricately detailed manner. Some of the elementary relationships were linked together to get an overall understanding of the model’s ﬂow, which does not necessarily change the model, nor does it contradict the papers. It enhanced the level of understanding for evaluation in the next step to ﬁnding the usability of artefact model.

3.2.3 Interview Study

The third step in the thesis is an interview study. The motivation for selecting this approach was to perform research to get feedback from practitioners about security compliance based on the artefact model. We adopted this method because it involves development and operations which are related to artefacts in the area of software engineering. This interview study methodology is a ﬂexible design strategy.

The process copes up with real-world phenomena where conclusions are derived con- sistently, which adds to existing knowledge about the DevSecOps [26]. Interviews used in empirical method studies often collect data to provide insights into qualitative research goals [36]. The evaluation is the purpose and systematic assessment of the security compliance within an organizational context with some real-world eﬀect. The selection of the variables that represent the situation, i.e., evaluating the usability of the artefact model and conducting interviews with the practitioners that provided relevant information on security compliance with standards. Conducting interviews was the main motive for this study to extract meaningful information to provide valuable feedback on the artefact model for DevSecOps. The practitioners gave high-quality information about the artefact model and shared their experiences within the ﬁeld.

(39)

3.2. Research Methods 25 3.2.3.1 Data Collection

Interviews were used as data collection method to evaluate the artefact model with practitioners from the industrial company. This study was used to ﬁgure out the artefacts usability in artefact model related to DevSecOps. Data collection procedure was direct contact with the practitioners in the industry to collect data in real-time.

The gathered data can analyze current artefacts that refer to the security status of a software product in DevOps pipeline and security standard artefacts that were recommended. For the qualitative research method, an interview is commonly used for collecting data [37].

We decided on semi-structured interviews where the interviewees were ensured con- fidentiality regarding the questions that contain sensitive information. It provided more flexibility to gather information than structured interviews. This type of semi- structured interviews allowed interviewers to ask additional follow-up questions for more clarification, which enriched the quality of the interview and the research. It was conducted face to face manner where at least one researcher talks to at least one interviewee. The questionnaire consisted of both open-ended and close-ended questions in the single round of semi-structured interview. The practical understanding of the artefact model usability can be evaluated because of conducting interviews with the practitioners. The steps followed in this interview study is described in Figure 3.4.

Figure 3.4: Interview Steps

3.2.3.2 Interview Guide

Interviewers using the semi-structured interview approach generally follow a document called an interview guide [36].

• Preparation: The interviewees’ characteristics are practitioners and experts knowledgeable in the areas of standard, DevOps, DevSecOps, artefacts and related topics of the thesis as the target population. They were contacted through supervisors and invitation letters (Figure C.1) by email that addresses

Development and Evaluation of an Artefact Model to Support Security Compliance for DevSecOps