Evaluating the Compliance Re-Certification Efficiency Enabled by the AMASS Platform for Medical Devices

(1)

Mälardalen University

School of Innovation, Design and Engineering

Västerås, Sweden

DVA423

Thesis for the Degree of Master of Science (60 credits) in Computer

Science with specialization in Software Engineering

Evaluating the Compliance Re-Certification

Efficiency Enabled by the AMASS Platform

for Medical Devices

Aleksandër Pulla

asa19008@student.mdh.se

Antonela Bregu

abu19002@student.mdh.se

Examiner: Jan Gustafsson

Mälardalen University, Västerås, Sweden

Supervisors: Barbara Gallina, Julieth Castellanos

Mälardalen University, Västerås, Sweden

(2)

Abstract

The certification of systems in the medical domain aims to ensure that a system is acceptably safe in order to bear the CE mark. Such process is exhaustive, expensive, time-consuming and safety-critical. Medical devices shall be re-certified under Medical Device Regulations. The first de-facto platform for re-certification is delivered by AMASS project. This thesis is expected to fill the specific gap: evaluate the compliance re-certification efficiency of the platform in the medical domain during the re-certification effort required as a consequence of a change in the normative space. Due to the lack of demonstrations in this safety-critical domain, the standard for medical devices, ISO 14971 with its versions and the Notified Bodies Recommendation Group (NBRG) Consensus paper are considered. There are several differences among them, in terms of the normative part and the fact whether they are international or only applicable in Europe. The evaluation will be conducted on a case study and the research has followed best practicing in case study design/execution. The focus is on two changes. The first change in the normative space is represented by the introduction of the EU directives (EU Medical Device Directives (MDDs): 90/385/EEC, 93/42/EEC, and 98/79/EC.) in relation to ISO 14971:2007, which required the introduction of ISO 14971:2012 (which applies only to manufacturers placing devices on the market in Europe). The second change is represented by the introduction of ISO 14971:2019, an international standard. Through the tool-chain (EPF Composer-BVR Tool), the families of standards and processes are modeled. The reuse of compo-nents is assessed through the application of selected metrics creating the measurement framework. The aim is to increase evidence according to the usefulness of the tool-chain in other domains. This master thesis will contribute with a case study evaluation of the tool-chain (a subset of the platform), considering cross-jurisdictional challenges. This work could represent the starting point for an evaluation where not only reference-processes are considered, but also the processes actually modelled in industrial settings.

Keywords: AMASS project · certification · re-certification · efficiency · medical devices · ISO 14971 · NBRG Consensus paper · reuse · standard · tool-chain · metrics · directives · regulations

(3)

Acknowledgements

We are grateful for having the opportunity to develop a master thesis under the supervision of Barbara Gallina, who has frequently assisted us, sharing knowledge and providing support to work focused and motivated. Her dedication and passion for what she does has encouraged us to bring out the best of us.

We would like to truly thank our other supervisor, Julieth Patricia Castellanos, for helping us, providing practical knowledge and advice.

We are thankful to our families for always trusting in our abilities, inspiring us to see the bright side of things and boosting our confidence.

Finally, we would like to express gratitude for each other, for the unconditional love and care shown during all this difficult year, for the joy and excitement that helped us go through the challenging moments.

(4)

List of Figures

1 Overview of risk management activities as applied to medical devices [1] [2] [3] . . 5

2 Key terminology defined in this specification mapped to Method Content vs Process [4] . . . 9

3 An example of SPEM 2.0 compliant model [5] . . . 11

4 Icon customization of Requirement practice, adapted from [5] . . . 12

5 An example of requirements definition, adapted from [5] . . . 12

6 EN 50126 (focus on Phase 6) standard requirements modeling [6] . . . 13

7 EN 50126 (focus on Phase 6) lifecycle process modeling [6] . . . 14

8 EN 50126 (focus on Phase 6) partial compliant model [6] . . . 15

9 An example of VSpec model [7] . . . 16

10 An example of Resolution model, adapted from [7] . . . 17

11 Overview of the seamless integration between EPF Composer/BVR tool-chain [8] . 18 12 ISO 14971 standards and NBRG Consensus libraries in EPF Composer . . . 24

13 ISO 14971:2007 standard requirements modeling . . . 25

14 EN ISO 14971:2012 standard requirements . . . 26

15 ISO 14971:2019 standard requirements . . . 27

16 NBRG Consensus requirements (industry recommendations) . . . 28

17 ISO 14971:2007 lifecycle process . . . 29

18 Activity Diagrams of (a) Risk Management process, (b) Risk Analysis phase and (c) Risk Evaluation phase, with respect to ISO 14971:2007 . . . 30

19 EN ISO 14971:2012 lifecycle process . . . 31

20 Activity Diagrams of (a) Risk Management process, (b) Risk Analysis phase and (c) Risk Evaluation phase, with respect to EN ISO 14971:2012 . . . 32

21 ISO 14971:2019 lifecycle process . . . 33

22 Activity Diagrams of (a) Risk Management process, (b) Risk Analysis phase and (c) Risk Evaluation phase, with respect to ISO 14971:2019 . . . 34

23 NBRG Consensus lifecycle process . . . 35

24 Compliance of Risk Analysis phase with respect to ISO 14971:2007 . . . 37

25 Compliance of Risk Evaluation phase with respect to ISO 14971:2007 . . . 38

26 Compliance of Content Deviations (Z Annexes) with respect to EN ISO 14971:2012 38 27 Compliance of Risk Analysis phase with respect to ISO 14971:2019 (part 1) . . . . 40

28 Compliance of Risk Analysis phase with respect to ISO 14971:2019 (part 2) . . . . 42

29 Compliance of Risk Evaluation phase with respect to ISO 14971:2019 . . . 43

30 Compliance of industry recommendations (interpretation of EN ISO 14971:2012 Z Annexes) with respect to NBRG Consensus paper . . . 43

31 High-level overview of ISO 14971 variability modelling . . . 44

32 Expansion of Standard family . . . 44

33 Expansion of Risk_Analysis choice . . . 45

34 Expansion of Risk_Analysis "Clause 4" . . . 45

35 Expansion of Risk_Analysis_Process . . . 46 36 Expansion of Intended_Use_and_Identification_of_Characteristics_related_to_the _Safety . . . 46 37 Expansion of Identification_of_Hazards . . . 46 38 Expansion of Risk_Estimation_for_each_Hazardous_Situation . . . 47 39 Expansion of Treatment_of_Negligible_Risks . . . 47

40 Expansion of Risk_Analysis_2019 "Clause 5" . . . 48

41 Expansion of Risk_Analysis_Process "5.1" . . . 48 42 Expansion of Intended_Use_and_Reasonably_Foreseeable_Misuse . . . 49 43 Expansion of Identification_of_Characteristics_related_to_Safety . . . 49 44 Expansion of Identification_of_Hazards_and_Hazardous_Situations . . . 50 45 Expansion of Risk_Estimation . . . 51 46 Expansion of Risk_Evaluation . . . 51 47 Expansion of Risk_Evaluation . . . 52 48 Expansion of Discretionary_Power_regarding_Risk_Acceptability_Policy . . . . 52

(7)

49 Expansion of RAP . . . 53

50 Expansion of Risk_Evaluation_2019 . . . 53

51 Expansion of Determine_Risk_regarding_Risk_Acceptability_Policy . . . 54

52 Expansion of Process family . . . 54

53 Expansion of Risk_Analysis "Phase" into optional choices . . . 55

54 Expansion of Risk_Analysis "Phase" into mandatory choices . . . 55

55 Expansion of Risk_Analysis "Phase" into constraints . . . 56

56 Expansion of Identification_of_Hazards "Task" . . . 56

57 Expansion of Estimation_Risks_for_each_Hazardous_situation "Task" . . . 56

58 Expansion of Identification_of_Hazards_and_Hazardous_Situations "Task" . . . 57

59 Expansion of Risk_Estimation task . . . 57

60 Expansion of Risk_Evaluation "Task" into mandatory choices . . . 57

61 Expansion of Risk_Evaluation "Task" into optional choices . . . 58

62 Expansion of Risk_Evaluation "Task" into constraints . . . 58

63 Resolution model of ISO 14971:2007 (part 1) . . . 59

64 Resolution model of EN ISO 14971:2012 (part 1) . . . 59

66 Resolution model of NBRG Consensus paper (part 1) . . . 60

(8)

104 Resolution model validation of: ISO 14971:2007 (a); EN ISO 14971:2012 (b); ISO 14971:2019 (c); NBRG Consensus paper (d) . . . 79

List of Tables

1 Comparison among ISO 14971 versions and NBRG Consensus paper . . . 24 2 Required and Optional components of the "Process" family . . . 80 3 Required and Optional components of the "Standard" family . . . 81 4 Results of metrics for the "Process" family, with respect to ISO 14971 standard

versions and NBRG Consensus paper variability modeling . . . 82 5 Results of metrics for the "Standard" family, with respect to ISO 14971 standard

versions and NBRG Consensus paper variability modeling . . . 83 6 Results of metrics for the whole family ("Standard" + "Process"), with respect to

(9)

Acronyms

AFAP As Far As Possible.

ALARP As Low As Reasonably Practicable.

AMASS Architecture-driven, Multi-concern and Seamless Assurance and Certication of Cyber-Physical Systems.

BVR Base Variability Resolution. CCL Common Certication Language. CVL Common Variability Language. EPF Eclipse Process Framework. EU European Union.

FMEA Failure Mode and Eects Analysis. FTA Fault Tree Analysis.

HAZOP HAZard and OPerability study.

ISO International Organization for Standardization. IVD in-vitro diagnostic.

MD Medical Device.

MDD Medical Device Directive. MMA Mobile Medical Applications.

NBRG Notied Bodies Recommendation Group. OMG Object Management Group.

OO object-oriented.

OPENCOSS Open Platform for EvolutioNary Certication of Safety-critical Systems. PHA Preliminary Hazard Analysis.

PL Product Line.

PLA Product Line Architecture. PLE Product Line Engineering. PrL Process Line.

PrRi Product-related Reusability. QMS Quality Management System. RAP Risk Acceptability Policy. RPG Role Playing Games.

(10)

SD Software Development.

SiSoPLE Security-informed Safety-oriented Process Line Engineering. SoC Size of Commonality.

SoPL Safety-oriented Process Line.

SoPLE Safety-oriented Process Line Engineering.

SPEM Software & Systems Process Engineering Metamodel Specication. SPICE Software Process Improvement and Capability Determination. SPL Software Product Line.

UMA Unied Method Architecture. WP Work Product.

(11)

1 Introduction

This chapter presents an introduction to the thesis, to help readers get a general idea of what is going to be treated. It is organized as follows: Section 1.1 presents the context in which this thesis is set; Section 1.2 states the motivation for conducting this thesis; Section 1.3 presents the contribution of this thesis in the field; and Section 1.4 consists of the outline of this master thesis work.

1.1 Context

Medical devices shall be certified in order to reach the market, but certification1 _{is an activity}

that requires time, money and quality. The AMASS project [9] [10] [11] has delivered the first de-facto platform for re-certification2_. _{This platform consists of an ecosystem of tools, which}

seamlessly interoperate to support engineers during the assurance and re-certification activities, including efficient compliance management via reuse of certification artifacts. The compliance re-certification efficiency of the platform has been illustrated and partially demonstrated in some safety-critical domains taking into consideration re-certification needs in case of different types of changes, e.g. criticality level, concern (safety/security). However, the re-certification efficiency of the platform has not yet been evaluated.

Within AMASS, compliance is supported in various ways. In the context of this thesis, the focus is on compliance via EPF Composer [15], which is the tool supporting the reference im-plementation of SPEM 2.0 (Systems and Software Process Engineering Meta-model [4]) included in the AMASS tools platform. SPEM 2.0 is a conceptual framework that provides the essential concepts for modeling, documenting, presenting, and managing method content and processes. In particular, SPEM 2.0 permits the modeling of: 1) the requirements of the standards, 2) the pro-cesses mandated by the standards, and 3) the mapping between the requirements and the process to show the satisfaction of the requirements via the modeled process elements. When new ver-sions of the standards are released, requirements need to be updated as well as the corresponding processes. In some cases, there is even the co-existence of different versions for a while. Thus, ap-propriate reconfiguration mechanisms are needed to guarantee efficient compliance management. Within the AMASS platform, efficient compliance management is supported via the integration of EPF Composer and BVR Tool [16]. The seamless integration between the tool-chain supports Safety-oriented Process Line Engineering (SoPLE) [8]. SoPLE is a methodological framework to systematically model commonalities (the term that stands for the similarities found among the processes) and variabilities (the term that stands for the differences found among the processes) between highly-related processes to facilitate reuse and flexible process derivation. The family of standards modeled by BVR Tool will serve as a process line. The derived communal and variable artifacts will serve as inputs for the metrics to evaluate the re-certification efficiency.

ISO 14971 [17] [1] [2] [3] is the standard that was developed specifically for medical device/sys-tem manufacturers using established principles of risk management. In this standard, IVD (in-vitro diagnostic) medical devices are considered as well. ISO 14971 deals with processes for managing risks, primarily to the patient, but also to the operator, other persons, other equipment and the environment. ISO 14971 specifies a process through which the manufacturer of a medical device can identify hazards associated with a medical device, estimate and evaluate the risks associated with these hazards, control these risks, and monitor the effectiveness of the controls throughout the life cycle of the medical device. With respect to its scope, the standard for medical devices is not applicable in clinical decision making and does not specify acceptable risk levels. Furthermore, it does not require the manufacturer to have a quality management in place. The first release of this standard was published in 2000 and it is now obsolete. Its content has been evolving over the years, constituting three additional versions of ISO 14971 (respectively ISO 14971:2007, EN ISO 14971:2012 and ISO 14971:2019), consisting of modifications and refinement. ISO 14971:2007 was internationally used, EN ISO 14971:2012 was only endorsed for Europe and ISO 14971:2019 brought changes, making it internationally endorsed again. The detailed changes existing among

1_{The process that certifies the medical device with the CE mark from MDD certification, showing that it is}

acceptably safe

(12)

these versions of the standard for medical devices are represented in Subsection 2.2.2

1.2 Motivation

In the context of AMASS, as recalled in Section 1.1, EPF Composer and BVR Tool have been integrated to facilitate process-related variability management. SPEM 2.0 and CCL (Common Certification Language) [9] do offer support for variability management, but this does not satisfy AMASS purposes to manage the variability in three dimensions (process, products and quality assurance) and orthogonally. Hence, the seamless integration of EPF Composer/BVR tool-chain was defined as a solution. The tool-chain serves as modeling means for systematizing commonalities and variabilities and offers support for variability management of information related to families of processes and standards. In particular, EPF Composer provides support according to authoring, tailoring and deploying systems development processes, while BVR tool enables feature modeling (through VSpec editor), resolution (through Resolution editor), realization (through Realization editor) and derivation of specific family members, as well as their testing and analysis. BVR allows managing the variability orthogonally (in an abstract way). The output of EPF Composer will serve as an input for BVR Tool through representing the concepts manually.

Evaluating the re-certification efficiency by AMASS platform is of high interest in the context of medical devices because of the need to increase evidence regarding the usefulness of the tool-chain in other domains (i.e. automotive and avionics). However, no demonstration was conducted to evaluate the compliance efficiency of the platform during the re-certification effort required as a consequence of a change in the normative space. Through studying the available literature, it is evident that there is a need to do so, due to the lack of demonstrations being done by other authors. Hence testifying the usefulness of the tool-chain, via the integration of standards (the tailoring approach) is aimed to satisfy the need.

Besides, providing additional examples for the training material, as well as extending the com-munity to the medical devices manufacturers are of relevance.

1.3 Contribution

The outcome of this master thesis will consist of:

• a case study-based evaluation of the tool-chain, within AMASS platform on the medical domain, considering cross-jurisdiction challenges (inside/outside EU);

• a paper, which was submitted to the 13th International Conference on the Quality of In-formation and Communications Technology (QUATIC), notification expected by the end of June 2020.

1.4 Document Outline

Chapter 2 sets the background of the thesis, addressing the concept of medical devices, the evolution of ISO 14971:2000 and the identification of similarities and differences among the standard versions (focus on Risk Analysis and Risk Evaluation), an overview of process modeling and variability management tool-chain and their integration for SoPLE as well as the metrics for efficiency. Chapter 3 provides the related work for this thesis with respect to the compliance in medical domain and the reuse of product/process lines.

Chapter 4 presents the problem formulation and analysis. Chapter 5 presents the method used to elaborate this thesis.

Chapter 6 presents the description of the work done with respect to the elaboration of this thesis. Chapter 7 addresses the results that derive from the evaluation of the applicability (reuse) of the artefacts, enabled by the commonalities among the processes mandated by the standard versions, despite the existing variabilities.

Chapter 8 presents the discussions in accordance to the research questions and threats to validity. Chapter 9 presents the ethical and societal considerations that are raised from this work.

(13)

2 Background

This chapter introduces the main concepts, stipulating the understanding of this thesis work. It is structured as follows: Section 2.1 introduces the definition of ’medical devices’, ’in vitro diagnostic medical devices’ and ’active implantable medical devices’ with respect to the medical device di-rectives; Section 2.2 presents ISO 14971:2000 and its evolution, and addresses the directives that shall be satisfied by the standard; Section 2.3 presents the concept of process lines and provides ex-planations regarding Safety-oriented Process Lines, Safety-oriented Process Line Engineering and Security-informed Safety-oriented Process Line Engineering approaches; Section 2.4 provides an introduction to SPEM 2.0; Section 2.5 provides knowledge regarding the standard modeling and process compliance; Section 2.6 presents the variability management supported by BVR Tool; Sec-tion 2.7 provides the tool-chain integraSec-tion for SoPLE and SecSec-tion 2.8 presents the measurement framework via the metrics for efficiency.

2.1 Medical Devices

The following text is an excerpt from [18] [19] [20]: "The term ’Medical Device’ is defined as any instrument, apparatus, appliance, material or other article, whether used alone or in combination, together with any accessories or software for its proper functioning, intended by the manufacturer to be used for human beings in the:

• diagnosis, prevention, monitoring, treatment or alleviation of disease or injury;

• investigation, replacement or modification of the anatomy or of a physiological process; • control of conception,

and which does not achieve its principal intended action by pharmacological, chemical, im-munological or metabolic means, but which may be assisted in its function by such means." According to [18], "Active implantable medical device stands for any active medical device which is intended to be totally or partially introduced, surgically or medically, into the human body or by medical intervention into a natural orifice, and which is intended to remain after the procedure."

The following text is an excerpt from [20]: "‘in vitro diagnostic medical device’ means any medical device which is a reagent, reagent product, calibrator, control material, kit, instrument, apparatus, equipment, or system, whether used alone or in combination, intended by the manufac-turer to be used in vitro for the examination of specimens, including blood and tissue donations, derived from the human body, solely or principally for the purpose of providing information:

• concerning a physiological or pathological state, or • concerning a congenital abnormality, or

• to determine the safety and compatibility with potential recipients, or • to monitor therapeutic measures.

Products for general laboratory use are not in vitro diagnostic medical devices unless such products, in view of their characteristics, are specifically intended by their manufacturer to be used for in vitro diagnostic examination."

2.2 ISO 14971:2000 and its evolution

ISO 14971 [1] [2] [3] is the standard that addresses Risk Management process in medical devices. It provides manufacturers with a framework within which experience, insight and judgment are applied systematically to manage the risks associated with the use of medical devices. There are three adopted directives containing essential requirements that should be satisfied by the medical devices: 90/385/EEC [18] for the active implantable medical devices, 93/42/EEC [19] for medical devices and 98/79/EC [20] for IVD medical devices. In compliance with the essential requirements

(14)

provided by the directives, ISO 14971, the standard for medical devices, is endorsed and applied to all stages of the life-cycle of a medical device. The first edition of ISO 14971 was published in 2000 and it is now obsolete. Within the last two decades, several versions of the standard have been released. This thesis focuses on three editions: ISO 14971:2007 [1], status: withdrawn, EN ISO 14971:2012 [2] recently withdrawn, and ISO 14971:2019 [3]. The focus is on Risk Assessment, which prescribes the Risk Analysis and Risk Evaluation required for medical devices. In particular, Subsection 2.2.1 summarizes the similarities of this part in the three versions of the standard, while Subsection 2.2.2 summarizes their differences.

2.2.1 Risk Analysis and Risk Evaluation: Similarities

The following text is an excerpt from ISO 14971 [1] [2] [3]: "The manufacturer shall establish, doc-ument and maintain throughout the life-cycle an ongoing process for identifying hazards associated with a medical device, estimating and evaluating the associated risks, controlling these risks, and monitoring the effectiveness of the controls. This process shall include the following elements:

• risk analysis; • risk evaluation; • risk control;

• production and post-production information."

As it is denoted in this thesis scope, only the first two phases are considered, because the goal of this thesis is to evaluate the tool-chain by selecting the right portion of the standards where the most differences are reflected. Figure 1 illustrates the application of Risk Analysis and Risk Evaluation in medical devices, highlighting the portion in focus of this thesis.

ISO 14971 implies that the Risk Analysis phase shall be performed for the particular medical device and its implementation and derived results shall be recorded in the risk management file including: a) a description and identification of the medical device that was analyzed; b) identifi-cation of the person(s) and organization who carried out the risk analysis; c) scope and date of the risk analysis [1] [2] [3]. Regarding the particular medical device that is taken into consideration, the manufacturer is responsible for documenting the intended use and reasonably foreseeable misuse, as well as for identifying and documenting the qualitative and quantitative characteristics that are likely to affect the safety of this particular medical device. The hazard identification shall be accomplished by the manufacturer in both normal and fault condition. Risk evaluation requires that all events, which may degenerate into hazardous situations shall be taken into consideration. Therefore, the recording of hazardous situations is of importance, since for each of them, the related risk shall be estimated.

Risk Evaluation phase consists of deciding whether the risk associated to the identified haz-ardous situation shall be reduced. In case the risk reduction is not necessary, Risk control option analysis and Risks arising from risk control measures will not be performed. The documentation in the risk management file is required.

(15)

Figure 1: Overview of risk management activities as applied to medical devices [1] [2] [3] (focus on Risk Analysis and Risk Evaluation)

(16)

2.2.2 Risk Analysis and Risk Evaluation: Differences

The normative parts of ISO 14971:2017 and EN ISO14971:2012 have two main differences: First, ISO 14971:2007 is the international version of the standard for medical devices. In contrast, EN ISO 14971:2012 is applicable only for EU. Second, Z Annexes of EN ISO 14971:2012 (Annex ZA, Annex ZB and Annex ZC) provide seven content deviations of ISO 14971 [2] from the essential requirements of the Directives (only three of them are inside our scope), while Notified Bodies Recommendation Group (NBRG) consensus paper [21] provides industry recommendation for those content deviations that consist of:

1. Treatment of negligible risks In ISO 14971:2007 it is stated that manufacturer shall discard the negligible risk, while in EN ISO 14971:2012 requires that all risks, despite their dimension, shall be considered and reduced as far as possible.

2. Discretionary power of manufacturers to the acceptability of risks ISO 14971:2007 affirms that the manufacturer is free to determine the risk acceptability level and the integrate non-acceptable risk into the risk-benefit analysis. In contrast to this, EN ISO 14971:2012 based on the directive’s essential requirements, states that the manufacturer shall not define any risk acceptability criteria and that all risks have to be reduced as far as possible and balanced against the benefit of the device.

3. Risk Acceptability Policy respectively ALARP ("As Low As Reasonably Practicable") for ISO 14971:2007 and AFAP ("As Far As Possible") for EN ISO 14971:2012.

Considering the last version of ISO 14971 that corresponds to 2019, there are some parts that differ in comparison to the former versions.

- Risk Management Plan

ISO 14971:2007 refers Annex F for guidance on developing a risk management plan, while ISO 14971:2019 provides ISO/TR24971 for this purpose and for establishing the risk acceptability criteria as well.

- Risk Analysis

An obvious difference between ISO 14971:2007 and ISO 14971:2019 with respect to risk analysis is the number of phases; it is organized in 4 phases in the 2007’s version and 5 phases in the 2019’ver-sion, where Intended use and identification of characteristics related to the safety of the medical de-vice (2007) is divided in two phases, respectively: Intended use and reasonably foreseeable misuse and Identification of characteristics related to safety (2019). In addition, every extended infor-mation regarding the phases and guidance related to the tasks is provided in: Annexes for ISO 14971:2007 and ISO/TR 24971 for ISO 14971:2019.

In the Risk Analysis process (the first phase), ISO 14971:2007 includes guidance in risk analysis technique for toxicological. Unlike that, ISO 14971:2019 does not include this guidance. According to the Identification of hazards (third phase), ISO 14971:2019 identifies the hazardous situations and considers the intended use, reasonably foreseeable misuse and the characteristics related to safety of medical devices, which ISO 14971:2007 does not mention in this part. As for the Risk Estimation (the last phase) ISO 14971:2007 admits that for each hazardous situation for which the probability of harm cannot be estimated, the possible consequences shall be listed for use in risk evaluation and risk control [1]. In ISO 14971:2019 it is said that in case of minimal harm, an initial hazard and consequence analysis could be sufficient, while when the information or data are insufficient, a conservative estimate of probability of occurrence of harm can give indication on the risk.

- Risk Evaluation

In ISO 14971:2007, it is stated that manufacturer shall decide for each hazardous situation if the risk reduction is required and in case it is not required, the risk control requirements are not applied. While in ISO 14971:2019, the manufacturer is obliged to evaluate the estimated risk and determine whether the risk is acceptable or not. In case of an acceptable risk, it is not required to apply the risk control requirements given and the risk is treated as residual risk. If the risk is

(17)

unacceptable, the manufacturer shall perform risk control activities as described in clauses 7.1 to 7.6 of the standard.

Regarding the risk acceptability criteria, the version of 2007 states that the manufacturer shall choose to either:

- indicate in a matrix which combinations of probability of occurrence and severity of harm are un/acceptable, or

- further separate the matrix, requiring that risk become ALARP before being acceptable. Whichever option is selected, it shall be compliant to the manufacturer’s policy for determining the risk acceptability criteria, the national/regional regulations and International Standard, as well as consider the state of the art and the stakeholders’ concerns. While ISO 14971:2019 takes care of evaluating the overall residual risk and the criteria for acceptability of the overall residual risk. The evaluation consists of gathering and reviewing literature and data regarding the medical devices considered (and similar to them on the market) and/or involve judgement by a cross-functional team of experts with application knowledge and clinical expertise [3].

2.3 Process Lines, SoPL, SoPLE and SiSoPLE

A Process Line (PrL) is a set of processes that capture commonalities and controlled variabilities. Each of these processes is developed from a common set of core assets (features) in a prescribed way [22]. Hence, a software process line (SPrL) represents an instrument to systematically construct and manage variable software processes, by combining pre-defined and standardized process assets that can be reused, modified, and extended using a well-defined customization approach. Process engineers can ground context-specific process variants in a standardized or domain-specific reference model that can be adapted to the respective context [23]. A beneficial process line that over-weigh the inceptive cost is achieved by pursuing three steps [24]:

1. performing a sound scoping to define the boundaries for ensuring the commonalities maxi-mization and the ad-hoc variabilities minimaxi-mization;

2. determining the commonalities and variabilities; 3. creating the architecture of the process line;

A Safety-oriented Process Line (SoPL) represents a set of safety-oriented processes that may exhibit: full commonalities (equal process elements), partial commonalities (structured process elements that are partially equal), and variabilities (elements that present the ability to change) [25]. The identification of common core assets is important because those imply the reusable assets/process elements. Eventually, the benefits of reuse possibility in SoPL are perceived in terms of time and cost reduction as well as quality increment. The questions: "What varies?" and "How does it vary?" are essential when working with SoPL, since those determine the variability subject (a real-world variable item or property of an item, known as variation point) and the variability object (a certain instance of variability subject, known as variant). The determination of variabilities is of relevance since those assets imply process evolution. Five crucial elements are used to model processes, consisting of: tasks, roles, work products, guidance and tools.

Integration of the tool-chain is required for Safety-oriented Process Line Engineering (SoPLE), introduced in [26] [24]. The following text is an excerpt from [26]: "SoPLE consists of a two-phase method. The first phase is aimed at engineering the domain from a process perspective, i.e., iden-tifying and systematising process-related commonalities and variabilities in order to concurrently engineer a set of (SoPL). The second phase is aimed at deriving single safety-oriented processes via selection and composition of commonalities and variabilities."

As Gallina et al. [27] recalls from previous work, whenever prescriptive processes mandated by the standards exhibit evident similarities they can be treated as a safety-oriented process line. SoPLE deals with the identification and systematization of commonalities and variabilities to concurrently engineer a set of processes [8]. Eventually, the achievement of single processes is based on the selection and composition of commonalities and variabilities. Implementation of SoPLE needs to be planned rigorously. It starts in the initial phases of software development, such as software design and continues towards the whole process until verification and validation phase. Organizations considering adoption of SoPLE are faced with the upfront questions regarding the

(18)

selection of the right processes for conversion to derive the maximum benefits in the shortest time frame [28].

The following text is an excerpt from [26]: "Security-informed Safety-oriented Process Line Engineering (SiSoPLE) has been proposed as a sound solution to systematise common and variable process elements in the context of security-informed safety-oriented processes described within security, as well as safety-related standard." This methodological framework represents an extension of SoPLE.

SoPLE and SiSoPLE exhibit some benefits [25], in terms of:

• integrity: due to the possibility to model the commonalities and variabilities as well as reuse the communal assets, SoPLE and SiSoPLE are considered generally sound;

• scalability: since the family of processes can be structured by using components, due to the modeling language that is exploited (extension of SPEM 2.0), SoPLE and SiSoPLE are considered scalable and the attribute of scalability is satisfied via the "divide and conquer" principle;

• effectiveness: since SiSoPLE provides an early solution to the alignment conflict between security and safety through a framework, it avoids re-work later on in the development lifecycle and ensures a safe system being engineered;

There are demonstrations of SoPLE and SiSoPLE implementation in some safety-critical (au-tomotive and avionics) domains, thus these methodological frameworks are tool-supported.

2.4 SPEM 2.0

This section is organized as follows: Subsection 2.4.1 provides an overview of SPEM 2.0, the language used by EPF Composer and the modeling elements that are going to be used in this thesis work; and Subsection 2.4.2 provides the brief description of the process modeling tool, EPF Composer and its connection to SPEM 2.0.

2.4.1 Overview

The Software and Systems Process Engineering Meta-model (SPEM) [4] is a process engineering meta-model as well as conceptual framework, which can provide the necessary concepts for mod-eling, documenting, presenting, managing, interchanging, and enacting development methods and processes. It provides support regarding the reusable process content definition and variability modeling, as well. SPEM is responsible for enabling the specification of (safety-oriented) process lines and provides process engineers with several process definition structures that give information for the way in which a process shall be validated. Figure 2 illustrates the key terminology of SPEM 2.0 specification, with respect to Method Content and Process.

(19)

Figure 2: Key terminology defined in this specification mapped to Method Content vs Process [4] The modeling elements that are going to be used for process modeling are as follows [4]: 1. Method Content

• Role – a set of related skills, competencies, and responsibilities, used by Task Defini-tions to define who performs them

• Task – the work being performed by Roles Definition instances

• Work Product (WP) – is used, modified, and produced by Task Definitions (only artifact and deliverable elements will be used)

- Artifact , which consist of a tangible work product that is consumed, produced, or modified by one or more tasks.

- Deliverable , which is a collection of work products, usually artefacts, used to define typical or recommended content in the form of work products packaged for delivery. • Guidance – supplementary free-form documentation (only guidelines, checklists and

practices will be used)

- Guidelines , which provide additional detail about how to perform a particular task or grouping of tasks or that provides additional details, rules, and recommendations about work products and their properties. Among others, a guideline can include de-tails about best practices and different approaches for doing work, about how to use particular types of work products, information about different subtypes and variants of the work product and how they evolve throughout a lifecycle, discussions about skills the performing roles should acquire or improve upon, and measurements for progress and maturity.

- Checklist , which identifies a series of items that need to be completed or verified. - Practice , which represents a proven way or strategy of doing work to achieve a goal that has a positive impact on the work product or process quality. Practices are defined orthogonally to methods and processes. They could summarize aspects that impact may different parts of a method or specific process.

(20)

2. Process

• Delivery Process is the process that covers a whole development lifecycle from be-ginning to end, including predefined phases, iterations, and activities.

• Task Descriptor describes the assignable unit of work that generates one or more Work Product Descriptor.

2.4.2 EPF Composer

EPF Composer is a tool that implements UMA (Unified Method Architecture) metamodel, which covers most of the elements that are present in SPEM 2.0. In other words, EPF Composer comprises instances of all major SPEM 2.0 concepts [4]. It is worth to note that, in 2018, EPF Composer was brought back to the future by MDH-AMASS-team [29]. EPF Composer provides an extensible framework and exemplary tools based on SPEM 2.0 concepts for defining and managing software development processes. EPF Composer implements the method plugin package, which defines capabilities for modularization and extensibility. Such plugins, which can contain libraries of method content and processes, are reusable. The importance and usage of plugins with respect to standard modeling are explained in Subsections 2.5.1, 2.5.2 and 2.5.3. The rudimentary principle of EPF Composer tool is the separation of reusable core method content from its application in processes. Typically, two key problems need to be addressed for such process implementation [15]: 1. Developers need to understand the methods and key practices of software development (e.g.

elements that are described in Subsection 2.4.1).

2. Development teams also need to define how to apply their development methods and best practices throughout a project lifecycle. They need to define or select a development process, as well. A clear understanding of how the different tasks within the methods relate to each other is also needed (e.g. in this thesis it is important to understand the impact that "Hazard Identification" task has over "Risk Analysis" phase, during Risk Management process). In the context of the tool qualification process line, Gallina et al. [27], refers to Eclipse Pro-cess Framework Composer as a SPEM 2.0-compatible open-source tool for authoring development method content and publishing processes. Concerning the collected information that has been doc-umented in a spreadsheet, EPF Composer/SPEM 2.0 is used to model the cross-domain process line regarding a proposed methodological framework. The basic idea supported is enabling reuse and decreasing the cost and time by identifying commonalities and variabilities, when it comes to going from one domain to another. Through modeling the family of tool qualification processes via a safety-oriented process line, it is possible to identify reusable process elements and thus speed up the re-certification process when tools are expected to be used in different domains. In [30], it is stated that through Eclipse and Eclipse Modeling Framework, engineers are enabled to utilize the standard techniques and patterns with the aim of upgrading the components’ quality, scalability and re-usability.

Figure 3 illustrates an example of a compliant model realized through EPF Composer by us-ing SPEM 2.0 modelus-ing elements. The delivery process ’Software Design and Implementation’ is compounded by several activities (such as ’Coding and Testing ’, ’Integration’ etc.) and ca-pability patterns (which are also compounded by several activities, phases like ’Integration of software’ and task descriptor like ’Collect Requirement’). It is modeled in the ’ecss-e-st-40c_lifecycle’ plugin, which is important for compliance management.

(21)

Figure 3: An example of SPEM 2.0 compliant model [5]

2.5 Standard modeling and compliance in EPF Composer

The main part of this thesis consists of modeling the Risk Management process mandated by ISO 14971 standard versions and NBRG Consensus paper (focus on Risk Analysis and Risk Evaluation phases). This section presents the fundamental knowledge for satisfying the goal. It is organ-ised as follows: Subsection 2.5.1 presents the procedure of modeling the standard; Subsection 2.5.2 presents the procedure of modeling the lifecycle process; and Subsection 2.5.3 addresses the compliance managements achieved via mapping the requirements.

2.5.1 Standard modeling

In EPF Composer, standard modeling is achieved through requirements determination. Although there is not such element in EPF Composer to define requirements, a practical approach from previous works is followed to satisfy the goal.

Figure 4 illustrates the establishment of Requirement modeling element, achieved by the icon customization of Practice (a "Guidance" modeling element). First of all, a plugin called "cus-tomized_icon" is created, which will contain a customized Practice called "Requirement". This practice will be extended to define requirements that are provided in each standard. After defining the requirement, in the "Icon" section, a target icon is provided to ensure that it differs from regular Practices and the other elements of Guidance.

(22)

Figure 4: Icon customization of Requirement practice, adapted from [5]

Figure 5 illustrates an example of the requirements definition. A second plugin is created for defining the requirements of that standard version. In this plugin, a regular Practice is added, but in the "Content Variability" section, "Extends" type and "Requirement" element provided in the "customized_icon" plugin, are selected. The selected "Content Variability" type inherits the same properties as the base element, although the values can be overrided). Further, the requirement is customized and is provided with the exact name as the corresponding requirement from the standard. This process is repeated until all standard requirements are defined. A benefit of modeling through EPF Composer is that it allows the defined requirements to be nested, which may resemble to the structure of the standard.

Figure 5: An example of requirements definition, adapted from [5]

Figure 6 illustrates ’en50126_requirements’ plugin, which contains the requirements of EN 50126 (phase 6) standard [31]. A Method Content called ’requirements_of_en50126_phase_6’ is created and inside the Guidance element, customized practices with the variability relationship ’Ex-tends’ are created (as explained above) to represent nested requirements (such as ’en50126_system_ lifecycle_requirements’, ’staffing).

(23)

Figure 6: EN 50126 (focus on Phase 6) standard requirements modeling [6]

2.5.2 Lifecycle process modeling

The EPF Composer fully supports the re-use capacities of the SPEM 2.0, which makes it possible to define a library of process elements that can be re-used to assemble different processes [5]. In order to model contents related to a process, the existence of a plugin labeled after the name of lifecycle process is crucial. Inside the Method Content and Process of the created plugin, every SPEM 2.0 modeling element (i.e., tasks, roles, work products, delivery process etc. - as described in Subsection 2.4.1) is created.

Figure 7 illustrates lifecycle process modeling of EN 50126 (focus on Phase 6) standard. In-side the ’design and implementation’ Method Content of ’en50126_process_lifecycle’ plugin several SPEM 2.0 modeling elements are created. For example, inside the Roles folder, no individual with a set of related skills, responsibilities and knowledge, that will perform the tasks is defined. Regard-ing Tasks, EN 50126 (focus on Phase 6) process consists of two units of work to be done (specifically ’design_subsystem_and_components’ and ’realize_the_design_of_subsystem_and_components). Furthermore, these tasks either use, modify or produce Work Products (such as ’subsystem _and_components_design’ and ’subsystem_and_components_implementation’).

(24)

Figure 7: EN 50126 (focus on Phase 6) lifecycle process modeling [6]

2.5.3 Compliance Management

Standard-Process compliance makes possible to reuse standards and processes in different mappings for compliance. In order to achieve this procedure, three plugins should be available in the EPF Composer Method Library. The first and second plugins are used to model the standard and lifecycle process (see Subsections 2.5.1 and 2.5.2), while the third plugin (that will be explained in this subsection), serves as the mapping between standard and process plugins. Each requirement that is defined in the standard plugin should be copied in the mapping plugin.

There are different situations of compliance that can be modeled, such as full compliance, partial compliance and no compliance. Full compliance is achieved when there is evidence (in this case task linked to the corresponding requirement) for each requirement and its sub-requirement(s). Partial compliance is achieved when there is evidence for requirements but not their sub-requirements. No compliance is achieved when there is no evidence provided for requirements at all.

Figure 8 illustrates the mapped requirements of EN 50126 (focus on Phase 6) standard. Every requirement (that was modeled in Subsection 2.5.1 - as Figure 6 illustrates) is referenced to its corresponding lifecycle process element (that was modeled in Subsection 2.5.2 - as Figure 7 illus-trates). Regarding ’Requirements of Design and Implementation Phase 6’ requirement that is referenced to the ’Design and Implementation System’ phase, it is compounded by several nested requirements. One of these requirements (specifically ’Design Subsystem and Components Req 1’) is referenced to ’Subsystem and Components Design’ activity and also is compounded by an ad-ditional requirement ’Design Subsystem and Components’ that is referenced to ’Design Subsystem and Components’ task. Since every requirement is not referenced to a particular lifecycle process element (i.e., ’Staffing Plan’ is not referenced to any activity or task), EN 50126 (focus on Phase 6) model is partially compliant.

(25)

Figure 8: EN 50126 (focus on Phase 6) partial compliant model [6]

2.6 Variability Management supported by BVR and BVR Tool

This section is organized as follows: Subsection 2.6.1 provides a brief description of BVR language; and Subsection 2.6.2 presents BVR Tool altogether with its editors.

2.6.1 BVR language

BVR (Base Variability Resolution) is a language developed to fulfill the industrial needs in the safety domain for variability modeling [32]. It is built on the OMG Revised Submission of CVL (Common Variability Language)[16], but is simplified and enhanced relative to that language. BVR is a modern language to build software product lines (SPL) by incorporating advanced concepts for feature modeling, reuse and realization of components of these SPL. Via this language, variability is defined orthogonally to avoid cluttering of a model. BVR language represents the effort to improve variability modeling, dedicating a special attention to the variability realization and the enhancement of feature modeling in terms of multiplicities, types and additional beneficial concepts. This language supports the determination of references and variables. According to the BVR specifications, the VSpec models, Resolution models and Realization models are created via those three BVR editors.

As Vasilevskiy et al. [30] affirm in their paper, BVR language provides concepts to model variability abstractions, map these abstractions to a model, and specifies operations to derive a new product.

2.6.2 BVR Tool

The BVR bundle [33] implements and supports the BVR language [16]. This tool covers design, implementation and quality assurance to close the development cycle. The BVR bundle is built upon tools and editors which support standard variability modeling activities such as: feature modeling, resolution, realization, mapping abstract features to implementation artifacts, testing product lines, analysis and derivation of new products. The tool consists of Eclipse plug-ins, that can either seamlessly co-work and operate integrated into the BVR tool chain to support these activities, or proceed as stand-alone components. Recalling 2.6.1, three editors permit users to model the VSpec models, the Resolution models and the Realization models:

(26)

• VSpec is the metaclass for Variability Specifications. Variability abstraction provides con-structs for specifying and resolving variability on an abstract level, i.e., without specifying the exact nature of the variability with respect to the base model. A base model is any model of a target language where variability is defined using BVR. A substitution fragment outlines a set of elements to modify in a base model to derive a new product [30]. The central concept is that of VSpec, which is technically similar to feature in feature modeling, but also perceived as abstract variability specifiers (in the context of this thesis). VSpecs allow users to model the variability in a feature diagram-like fashion, but those may as well be arranged as trees, where the parent-child relationship organizes the resolution space by imposing structure and logic on permissible resolutions [34]. In order to model the family of standards in the VSpec, the following elements of this BVR editor [16] are used:

– Choice is a VSpec that represents a yes/no decision. When a VariationPoint is bound to a choice, it is dependent upon whether the resolution is a PosResolution or a NegResolution to determine what VariationPoint to execute. PosResolution means that one needs to resolve the eventual CompoundNode referred by this PosResolution. A NegResolution means that the decision is negative and there is no need to continue down the eventual VNode tree referred by this NegResolution.

– Constraint specifies restrictions on permissible resolution models. A constraint can refer to any entity in the closest namespace in which it is contained. Any entity referred by a constraint must be uniquely determined.

– Group is a VNode attribute, consisting of a multiplicity interval, which dictates the number of Choice resolutions.

Relationships between a parent feature and its child features – 0.. * - refers to ’none’;

– 1..1 which refers to ’alternative/xor’: one of the child features must be selected. – 1...* which refers to ’or’: at least one of the child features must be selected. – Mandatory: child feature is required

– Optional : child feature is optional

Figure 9 illustrates an example of car modeling, which illustrates the usage of the VSpec elements.

Figure 9: An example of VSpec model [7]

The Car (represented by choice element) is compounded by Engine (which is either HP140 or HP110, defined by the grouping element), SteeringWheel (of the type ’Material’, defined by the VType element), Wheel (with a specific diameter of the rims, defined by the Vari-able element), Parking_Assist (which is present only in the cars with the type of engine HP140, as the Constraint element denotes) and Seats (in a number no more than 5 and no less than 1, regarding the Multiplicity element; of the type ’Material’ and with an extra Adjustable_backrest choice).

(27)

• Resolution is an abstract representation or configuration of a desired product [32]. It allows users to perform configuration of their process, based on the specified inclusion/exclusion choices. Resolution models are automatically generated with a command, after the VSpec models are designed. The editor contains a mechanism that suggests a default resolution, which can be altered regarding the needs or requirements. There are no additional modeling elements to be used in this editor. Still, the entire idea is to evaluate the choices to "true", indicating that those features are included in the current model, or "false" to denote that the choices representing those features are excluded and do not correspond to the current model. The editor provides error checking and validation capabilities. To conform if the resolution is compliant to VSpec model, the validation process is executed. Figure 10 illustrates the same example provided in Figure 9 but generated from the Resolution editor. Because of the constraint that was put in the VSpec model (as Figure 9 illustrates), the choice of HP110 is evaluated as ’false’, while the Parking_Assist choice is evaluated as ’true’.

Figure 10: An example of Resolution model, adapted from [7]

• Realization [30] allows users to hold the conceptual representation of the variable elements together with the concrete elements in the base model. The substitution fragment is bound to the base model through placements and replacements. Eventually, establishing the relation-ship between VSpec features and fragment substitutions is required to create a traceability link between the fragment substitution and its implementation. This editor is out of focus regarding this thesis.

2.7 EPF Composer/BVR tool-chain seamless integration for SoPLE

The integration between EPF Composer for process engineering and BVR tool for variability management is required to tailor of safety-oriented processes with variabilities to individual projects in a similar way to the product lines [35].

As illustrated in Figure 11, where the seamless integration between the tool-chain is achieved, the ‘Seamless Integrator’ is expected to import all the libraries from EPF Composer and enable the communication with the BVR tool. When the variabilities are modeled (through VSpec editor), resolved (through Resolution editor), and realized (through Realization editor) inside the BVR tool (important for managing standard families), then the outcome is exported back again to the EPF Composer.

SoPLE establishment by the integration of the above tool-chain might be done in different ways. In this thesis, in the context of the AMASS project, it is decided to achieve the seamless integration between EPF Composer and BVR Tool, as illustrated in Figure 11. This choice is motivated by the fact that it provides advanced support for establishing SoPL.

(28)

Figure 11: Overview of the seamless integration between EPF Composer/BVR tool-chain [8]

2.8 Measurement framework and metrics for efficiency

For measuring the ability to form a product line from existing products, a set of metrics is used by Berger et al. [36]. As for the process of evaluating a set of similar products, each of those is decomposed into i = 1...n reasonable atomic pieces, for a concrete product, which is self-contained and reusable. Identifying sufficient commonalities, as well as variabilities, is an essential part of evaluating the product line-ability, via the proposed metrics.

After reviewing the literature, due to the absence of metrics applicable in process lines, it was decided to use the metrics of product lines, motivated by the fact that i.e., a role, a task, or a work product may be considered as a component. Eventually, the rationale behind the computations will remain the same, as in product lines. Two of these metrics that will be used in this thesis are Size of Commonality (SoC) and Product-related Reusability (PrRi). Product-related Reusability is translated into process-related context, and it is called Process-related Reusability. SoC measures the number of reusable/identical components in a product/process line. It is computed as follows:

SoC = |

n

\

i=i

Cpi| (1)

such that, Cpi represents the set of components of the process i, where pirepresents the

product-s/processes of the product/process line, and i ranges from 1 to n. To perform decomposition, all components must be identified and formally specified. By comparing the component signatures, SoC is derived. If the signatures are identical, eventually, the components are identical.

The second metric measures the extent of the common components’ reusability for a specific pro-duct/process. It is based on the result of the first metric, and the corresponding formula is:

P rRi=

SoC | Cpi|

(2)

(29)

3 Related Work

This chapter addresses the work done by other authors, with regard to the topic of this master thesis. It is organized as follows: Section 3.1 presents the previous attempts towards reaching compliance in the medical domain; Section 3.2 outlines the reuse of product/process lines for re-certification purposes in safety-critical systems.

3.1 Compliance in the medical domain

Compliance in the medical domain embodies a research topic that requires attention. Mobile medical apps shall comply with Medical Device Regulations. In [37], an approach for tailoring the MDevSPICE framework to support mobile medical application development is presented,R

providing compliance with the required medical device regulations. The MDevSPICE frameworkR

is introduced as a solution that supports the development of MMA (mobile medical applications) software in a more flexible and timely manner, compared to the traditional MD (medical device) software, via agile practices. Plan-driven software development approaches are usually adopted to achieve regulatory compliance, but this is not precisely the method used to develop mobile applications. Thus, the authors in this paper discuss the Reduced V-model approach’s adoption as the most suitable agile approach in terms of light-weight documentation for satisfying the regulatory requirements, seeing that the regulatory compliance is achieved seamlessly during the development. Since, MMAs are considered as MDs, to reduce the effort and costly overhead associated with the preparation for regulatory audits, MDevSPICE framework was developed.R

It integrates the requirements form the medical device standards and guidance documents into a single reference source. This procedure is similar to the re-certification process that relies on the core of this master thesis.

A large number of commercial solutions in the market can automatically provide compliance to the medical device regulators. Such regulatory compliance software tools are mostly evolved from the regulatory perspective. The more affordable those tools are, the faster medical devices reach the market, and the better the margins.

An investigation regarding the standards that companies shall implement, to satisfy the regu-lators and the challenges they face in the attempt to be compliant to such regulations is provided by Bujok et al. [38]. The authors conduct a comparison among the QMS (Quality Management System) standards and related SD (Software Development) standards as well as their mapping. By doing so, the common and the specific requirements that those standards hold are denoted. The mapping aims to inspect the areas which are common among the standards and those that differ, as well. This approach is pretty much similar to the one that is applied in this thesis. The essence of this paper is the investigation of how the combination of the Quality management sys-tem standards, Software development, and Risk Management standards can assist in implementing practices, which will permit developers to comply with specific regulations. Yet, there is no imple-mentation of such practices applied in their paper so far. Thus it is addressed by the authors as future work.

3.2 Reuse of product/process lines

The concept of reusability represents an essential attribute in safety-critical systems engineering. A systematic approach to effectively reuse safety certification artifacts is demonstrated by Ruiz et al. [39], based on the mapping of the two standards, which is validated in a case study on reuse from the railway and avionics domain, concretely. Authors reclaim from previous works that building safety-critical systems from scratch is very time-consuming and not cost-effective. Thus systems reuse improves productivity and reliability of development projects, reduces the time of throwing the systems into the market, and lowers their overall cost. This reuse approach is developed through OPENCOSS (Open Platform for EvolutioNary Certification of Safety-critical Systems) project, dedicated to the railway, avionics, and automotive markets. The case study has concluded in the reuse of all railway domain artifacts, effort reduction by above 50%, and cost reduction by above 25% - sufficient evidence that demonstrates a promising way of making cross-domain reuse more cost-effective in the industry. Contrasting this paper that focuses on

(30)

cross-domain and considers the OPENCOSS project, this thesis focuses on the intra-domain and considers the AMASS platform. Moreover, the metrics used for measuring the reuse among the above-mentioned articles are different (recalling Section 2.8). However, the line of reasoning within both of them remains similar.

The reuse concept is also discussed by Gallina et al. [27], where authors reason that for re-certification purposes, compliance with tool qualification processes is necessary. A new cost-effective, and less time-consuming tool-certification process approach is proposed. According to them, reuse is enabled when manufacturers recognize what is common and what varies with cross-domain. In this paper, authors do not only focus on modeling a cross-domain tool-qualification process line to enable reuse, but process lines are related to the family of process-based arguments, with respect to process compliance. Hence, such relationship both facilitates the reuse of artifacts and provides relevant knowledge, as far as certification strategies are concerned. To conclude, the re-certification process is accelerated through modeling the family of tool-qualification processes using SoPL, when transiting from one domain to another. However, authors in this paper do not provide a measurement framework to quantitatively evaluate the artifact reuse and compliance (re-)certification. The reasoning within this thesis is similar to the one in this paper, concerning the SoPL approach, but the additional step that this thesis takes is the application of metrics for measuring the compliance re-certification efficiency.

In [40], a practical approach for formally modeling a software product line (SPL) is addressed. Such approach enables the computation of product line common parts appropriately. A formal dependency model afterward extends this work. Calculation laws are drawn by authors, making this approach practical and applicable in real-world examples. Eventually, this paves the way for establishing metrics that support the computation of commonalities in an SPL. Their presence is relevant in product line engineering, in terms of assessing the product lines’ success or either predicting the benefit of introducing an SPL according to a determined set of assets. To put it simply, what this paper misses is the development of the above-mentioned metrics, which is of relevance in the context of reuse and re-certification process.

According to Her et al. [41], product line engineering (PLE) is an effective approach to software reuse, and a core asset is their most crucial element. Authors have proposed a measurement framework for evaluating the reusability of those core assets in product line engineering, consisting of five core quality attributes and their corresponding metrics. Those metrics are defined to measure the quality attributes by referring them to the artifacts of PLE. However, according to the authors, it is still necessary to extend this work in the future, to include different quality characteristics, the respective metrics, and guidelines for measuring each metric.

PLA (product line architecture) is considered to be the most essential asset of SPL. In [42], a project with the applied product-line approach is established, supporting the implementation of RPG (Role Playing Games) in mobiles. By considering the four RPGs’ families, the commonalities and variabilities among them are elicited, and further, an RPG-PLA is constructed through the application of a reuse technique. The authors presented technical details for the solution and OO (object-oriented) metrics as provided to support the results. Authors of [43] have proposed some metrics that are based on the interfaces of software components. Their work is focused on providing an incremental set of metrics at PLA-level, which allows an architect to make decisions about the usage indicators of architectural components and the validity of product family architectures. However, those proposed metrics are useful only for object-oriented architectures. Similar to the previous paper, Zhang et al. [44] have provided an approach for assessing the quality of product line architecture by analyzing its formal vADL specification (architecture description language). To achieve that, some metrics that assess similarity, reusability, complexity, and variability of PLA have been used. Such metrics can be computed automatically through the analysis of vADL specification. Contrasting paper [42],[43] and [44], this thesis considers the risk management process of three versions of ISO 14971 standard and NBRG Consensus paper, which are treated as a process line. The concept of PLA is not present within this thesis.

Evaluating the Compliance Re-Certification Efficiency Enabled by the AMASS Platform for Medical Devices

Mälardalen University

School of Innovation, Design and Engineering

Västerås, Sweden

DVA423

Thesis for the Degree of Master of Science (60 credits) in Computer

Science with specialization in Software Engineering

Evaluating the Compliance Re-Certification

Efficiency Enabled by the AMASS Platform

for Medical Devices

Aleksandër Pulla

asa19008@student.mdh.se

Antonela Bregu

abu19002@student.mdh.se

Examiner: Jan Gustafsson

Mälardalen University, Västerås, Sweden

Supervisors: Barbara Gallina, Julieth Castellanos

Mälardalen University, Västerås, Sweden

Table of Contents

List of Figures

List of Tables

Acronyms

1

Introduction

1.1

Context

1.2

Motivation

1.3

Contribution

1.4

Document Outline

2

Background

2.1

Medical Devices

2.2

ISO 14971:2000 and its evolution

2.3

Process Lines, SoPL, SoPLE and SiSoPLE

2.4

SPEM 2.0

2.5

Standard modeling and compliance in EPF Composer

2.6

Variability Management supported by BVR and BVR Tool

2.7

EPF Composer/BVR tool-chain seamless integration for SoPLE

2.8

Measurement framework and metrics for efficiency

3

Related Work

3.1

Compliance in the medical domain

3.2

Reuse of product/process lines