Ontology-based Analysis and Scalable Model Checking of Embedded Systems Models

Mälardalen University Press Licentiate Theses No. 262

ONTOLOGY-BASED ANALYSIS AND SCALABLE

MODEL CHECKING OF EMBEDDED SYSTEMS MODELS

Nesredin Mahmud

2017

Copyright © Nesredin Mahmud, 2017 ISBN 978-91-7485-337-7

ISSN 1651-9256

M¨alardalen University Licentiate Thesis

No.262

Ontology-based Analysis and

Scalable Model Checking of

Embedded Systems Models

Nesredin Mahmud

June 2017

Department of Computer Science and Engineering

M¨alardalen University

Copyright © Nesredin Mahmud, 2017 ISSN 1651-9256

ISBN 978-91-7485-337-7

Popul¨arvetenskaplig

sammanfattning

F¨or n¨arvarande saknas effektiva och skalbara metoder f¨or att specificera och analysera krav, och formellt analysera beteendemodeller, f¨or inbyggda sys-tem. De flesta systemkrav f¨or inbyggda system uttrycks i naturligt spr˚ak, vilket ¨ar flexibelt och intuitivt men ofta tvetydigt, vagt och sv˚arbegripligt. F¨orutom naturligt spr˚ak anv¨ands mallbaserade kravspecifikationsmetoder f¨or att speci-ficera krav (speciellt i s¨akerhetskritiska till¨ampningar), vilket minskar tvety-digheten och f¨orb¨attrar begripligheten av specifikationerna. De mallbaserade metoderna ¨ar emellertid vanligtvis oflexibla p˚a grund av mallarnas fasta struk-turer. De saknar ocks˚a meta-modeller f¨or f¨or¨andring, och val av mallar ¨ar utmanande.

I denna avhandling f¨oresl˚ar vi ett dom¨anspecifikt spr˚ak f¨or inbyggda sys-tem, ReSA, som ¨ar ett begr¨ansat naturligt spr˚ak men tillr¨ackligt flexibelt f¨or att ge ingenj¨orer m¨ojlighet att anv¨anda olika konstruktioner f¨or att specificera krav. Spr˚aket har en formell semantik i satslogik, och ¨aven i beskrivningslogik, vilket m¨ojligg¨or en icke-trivial och rigor¨os analys av kravspecifikationer, t.ex. konsekvenskontroll, fullst¨andighet av specifikationer etc. Vi visar hur man utf¨or konsekvenskontroll av krav formaliserade i boolesk logik, men ocks˚a p˚a djupare semantiska niv˚aer inom ReSA-ontologin.

Dessutom f¨oresl˚ar vi en metod f¨or skalbar formell analys av Simulink-modeller, eftersom Simulink ¨ar den mest anv¨anda modellerings- och simuler-ingsmilj¨on f¨or inbyggda system inom industrin. Simulink-modeller anv¨ands ocks˚a f¨or att generera kod automatiskt f¨or olika h˚ardvaruplattformar. D¨arf¨or ¨ar det viktigt att kunna formellt analysera Simulink-modeller f¨or att f˚a insikt i beteendet hos det inbyggda systemet och f¨or att f¨orhindra potentiella fel fr˚an att sprida sig till implementeringen.

Vi validerar v˚ara tillv¨agag˚angss¨att genom fallstudier fr˚an fordonsindustrin, n¨amligen Justerbar hastighetsbegr¨ansare (ASL) och BrakebyWire (BBW) -systemen fr˚an Volvo Group Trucks Technology, Sverige.

Abstract

Currently, there is a lack of effective and scalable methods to specify and an-alyze requirements, and formally anan-alyze the behavioral models of embedded systems. Most embedded systems requirements are expressed in natural lan-guage, which is flexible and intuitive but frequently ambiguous, vague and in-comprehensive. Besides natural language, template-based requirements spec-ification methods are used to specify requirements (esp. in safety-critical ap-plications), which reduce ambiguity and improve the comprehensibility of the specifications. However, the template-based methods are usually rigid due to the fixed structures of the templates. They also lack meta-models for extensi-bility, and template selection is challenging.

In this thesis, we propose a domain specific language for embedded sys-tems, ReSA, which is a constrained natural language but flexible enough to allow engineers to use different constructs to specify requirements. The lan-guage has formal semantics in propositional logic, and also in description logic, which enables non-trivial and rigorous analysis of requirements specifications, e.g., consistency checking, completeness of specifications, etc. We show how to carry out consistency checking of requirements formalized in boolean logic, but also at deeper semantic levels, within the ReSA ontology.

Moreover, we propose a method for the scalable formal analysis of Simulink models, as Simulink is the de facto modeling and simulation environment of embedded systems in industry. Simulink models are also used to generate code automatically for various hardware platforms. Therefore, it is essential to be able to formally analyze Simulink models, to get insight into the behavior of the embedded system, and also prevent potential errors from propagating into the implementation.

We validate our approaches on industrial use cases from the automotive industry, that is, the Adjustable Speed Limiter (ASL) and the Brake-By-Wire (BBW) systems from Volvo Group Trucks Technology, Sweden.

Acknowledgement

I would like to express my deepest gratitude to my main supervisor Asso-ciate Professor Cristina Seceleanu. Your passion for research is inspiring, and your guidance, encouragements, determination, and patience have cre-ated momentum in me. Thank you. I would like to thank my co-supervisor Oscar Ljungkrantz for the lively reception at Volvo Group Trucks Technology (VGTT), Gothenburg and the wonderful researchers he introduced to me. It is my pleasure also to thank my co-supervisor Guillermo Rodriguez-Navas, for the great team work, valuable feedback, and encouragements. I am grateful to have you as my supervisors. Your effort has made this work worthwhile.

Moreover, I would like to thank my grading committee Associate Professor Raffaela Mirandola, Professor Bj¨orn Lisper, especially my examiner Associate Professor Riccardo Scandariato for accepting our call. Thank you for review-ing my licentiate thesis. I am honored to have you as my committee members. I would also like to extend my appreciation to Damir Isovic, Thomas Nolte, Hans Hansson, Moris Behnam and Carola Ryttersson who have facilitated my licentiate defense and have influenced my academic career.

I wish to acknowledge the help provided by Henrik L¨onn, Henrik Kaijser, Oscar Ljungkrantz, Daniel Karlsson and Mattias Nyberg, on various industrial challenges. Your willingness to share and support me, is highly admired. It’s my pleasure working with you. For the list is too long, I briefly express my respect and love to my colleagues and friends here at MDH and outside. Your friendly support, jokes, and cultural exchanges will always be memorable.

I thank Allah for cherishing me the well-being and strength to do my studies. Thank you my parents for you have given me unconditional love and have sup-ported me throughout my life. Elu, my dear love, thank you for being under-standing and be patient, even in times of your difficulties. Our sweet baby Mohammed,“the little alarm clock”, welcome to this amazing world.

I would like to thank VINNOVA for sponsoring this research via the VeriSpec project, in which the financial support has made this work possible.

Nesredin Mahmud V¨aster˚as, June 16, 2017

List of Publications

Papers Included in the Licentiate Thesis

Paper A ReSA: An Ontology-based Requirement Specification Language

Tai-lored to Automotive systems. Nesredin Mahmud, Cristina Seceleanu,

Oscar Ljungkrantz. In Proceedings of the 10th IEEE International Sym-posium on Industrial Embedded Systems (SIES 2015), pp. 1-10, IEEE, 2015.

Paper B ReSA Tool: Structured Requirements Specification and SAT-based

Consistency-checking. Nesredin Mahmud, Cristina Seceleanu, Oscar

Ljungkrantz. In Proceedings of the Federated Conference on Computer Science and Information System (FedCSIS 2016), pp. 1737-1746, IEEE, 2016.

Paper C Semantic Analysis of Embedded System Requirements Specifications. Nesredin Mahmud, Cristina Seceleanu, Oscar Ljungkrantz. Technical Report, M¨alardalen Real-Time Research Center, Sweden, May 3, 2017 (Accepted as full paper in the 15th International Conference on Software Engineering and Formal Methods (SEFM 2017)).

Paper D Simulink to UPPAAL Statistical Model Checker: Analyzing

Automo-tive Industrial Systems. Predrag Filipovikj, Nesredin Mahmud, Raluca

Marinescu, Cristina Seceleanu, Oscar Ljungkrantz, Henrik L¨onn. In Pro-ceedings of the 21st International Symposium on Formal Methods (FM 2016), pp. 748-756, Springer International Publishing, 2016.

Additional Papers not Included in the Licentiate

Thesis

Evaluating Industrial Applicability of Virtualization on a Distributed Multicore Platform. Nesredin Mahmud, Kristian Sandstr¨om, and Aneta

Vulgarakis. In Proceedings of the IEEE Emerging Technology and Fac-tory Automation (ETFA 2014), pp. 1-8, IEEE, 2014.

The Multi-resource Server for Predictable Execution on Multi-core Plat-forms. Rafia Inam, Nesredin Mahmud, Moris Behnam, Thomas Nolte,

and Mikael Sj¨odin. In Proceedings of the 19th Real-Time and Embed-ded Technology and Applications Symposium (RTAS 2014), pp. 1-12, IEEE, 2014.

Contents

I

Thesis

1

2.2 Boolean Satisfiability Problem . . . 12

2.3 Description Logic . . . 13

2.4 Timed Computation Tree Logic . . . 15

2.5 Simulink . . . 15

2.6 Stochastic Timed Automata . . . 17

3 Research Goals 21 3.1 Problem Description . . . 21

3.2 Research Goals Formulation . . . 22

4 Research Method 25 5 Thesis Contributions 29 5.1 ReSA - An Ontology-based Requirements Specification Language . . . 29

5.1.1 Syntax of ReSA . . . 30

5.1.2 Semantics of ReSA . . . 33

5.2 Requirements Specification Analysis . . . 34

5.2.1 SAT-based Analysis . . . 34

5.2.2 Ontology-based Analysis . . . 36

5.2.3 ReSA Tool . . . 39

Contents

5.3.1 Stochastic Timed Automata Patterns . . . 41

5.3.2 Validation on the Brake-By-Wire System . . . 42

5.3.3 Research Goals Revisited . . . 43

6 Related Work 45 6.1 Requirements Specification based on Constrained Natural Language . . . 45

6.2 Analysis of Requirements Specifications . . . 46

6.3 Verification of Simulink Models . . . 47

7 Conclusions and Future Work 49 7.1 Future Work . . . 50

Bibliography 51

II

Included Papers

59

8.2 Requirements Specification Methods . . . 3

8.2.1 Semi-formal Specification Methods . . . 3

8.2.2 Formal Specification Methods . . . 4

8.3 Use-case: Adjustable Speed Limiter (ASL) . . . 4

8.3.1 ASL System Description . . . 5

8.3.2 EAST-ADL Model of ASL . . . 5

8.3.3 Describing ASL Requirements in Natural Language: Issues . . . 7

8.4 ReSA: A Requirements Specification Language . . . 8

8.4.1 Requirements Specification Ontology . . . 8

8.4.2 ReSA Context-Free Grammar . . . 14

8.4.3 Sentential forms (Requirements Boilerplates) . . . 14

8.4.4 Implementation . . . 20

8.5 Applying ReSA to ASL Requirements . . . 20

8.6 Requirements Analysis . . . 21

8.7 Related Work . . . 23

8.8 Conclusions and Future work . . . 24

Bibliography . . . 25 xii

9 Paper B:

ReSA Tool: Structured Requirements Specification and SAT-based

Consistency-checking 31

9.1 Introduction . . . 2

9.2 Preliminaries . . . 3

9.2.1 Overview of ReSA . . . 3

9.2.2 EAST-ADL Levels of Abstraction . . . 4

9.2.3 XText Grammar . . . 6

9.2.4 The Boolean Satisfiability Problem (SAT) . . . 7

9.3 The ReSA Toolchain . . . 8

9.3.1 The Toolchain Architecture . . . 8

9.3.2 Implementation . . . 10

9.4 Automated Consistency checking . . . 12

9.4.1 Consistency Checking Approach . . . 13

9.4.2 ReSA-to-boolean Transformation . . . 13

9.4.3 Consistency Checking . . . 15

9.5 Industrial Use Case: Adjustable Speed Limiter . . . 16

9.5.1 ASL Requirements Expressed in ReSA . . . 16

9.5.2 Evaluation of the ReSA Toolchain with Practitioners . 19 9.5.3 Consistency Checking on the Use Case . . . 20

9.6 Related work . . . 22

9.6.1 Template-based Specification Tools . . . 22

9.6.2 Computer-processable CNL Tools . . . 24

9.7 Conclusion . . . 25

Bibliography . . . 26

10 Paper C: Specification and Semantic Analysis of Embedded Systems Require-ments: From Description Logic to Temporal Logic 31 10.1 Introduction . . . 33

10.2 Preliminaries . . . 34

10.2.1 ReSA . . . 34

10.2.2 Description Logic (DL) . . . 35

10.2.3 Timed Computation Tree Logic (TCTL) . . . 36

10.3 Defining ReSA Semantics in DL . . . 36

10.3.1 Semantic Analysis Framework . . . 37

10.3.2 Semantic Representation via the Event-based Approach . . . 38

Contents

10.3.3 The ReSA Ontology . . . 39

10.3.4 Semantics of Clauses and Statements in ReSA . . . . 41

10.4 Proving Semantic Equivalence of ReSA Specifications . . . 43

10.5 Automated Analysis and Transformation to TCTL Properties . 45 10.5.1 Consistency checking . . . 45 10.5.2 Completeness . . . 45 10.5.3 Transformation to TCTL . . . 46 10.6 Related Work . . . 47 10.7 Conclusion . . . 48 Bibliography . . . 49 11 Paper D: Simulink to UPPAAL Statistical Model Checker: Analyzing Auto-motive Industrial Systems 55 11.1 Introduction . . . 57

11.2 Preliminaries . . . 58

11.3 Simulink to UPPAAL SMC: Transformation Approach . . . . 59

11.4 Application on Industrial Use Cases: Results . . . 62

11.5 Discussion and Conclusions . . . 65

Bibliography . . . 66

I

Thesis

Chapter 1

Introduction

Embedded systems are ubiquitous in various industrial domains, e.g., automo-tive, avionics, industrial automation, chemical processes domains. Most of the systems are real time and safety critical, hence assuring their correctness is a must. The correctness of an embedded system assumes both the functional correctness of the system’s outputs as well as the time at which the outputs are produced by the system. For instance, in the automotive domain, an Adjustable Speed Limiter (ASL) system automatically adjusts the vehicle speed to not ex-ceed a predefined speed, in real time. The correct functionality of the system depends both on computing the correct output torque that limits the vehicle speed, as well as the time at which the torque request is issued to the brake system, to slow down the vehicle. If the brake is applied late, a traffic accident is possible; otherwise, early brake could mean bad driving experience, which is not desirable.

Due to the increasing number of functional requirements and safety mea-sures that safety-critical systems implement, their development has become complex and difficult to manage. A viable approach to manage the complex-ity is to employ model-based development techniques, and formal methods that enable the systematic analysis of system models at early stages of development, before implementation. Such analysis methods include requirements analysis and verification of behavioral models.

In the development of embedded systems, requirements specification [1] entails documenting the functional and extra-functional (e.g. timing, reliability etc.) requirements of a system in a comprehensive way. In practice, natural lan-guage is the de facto standard for requirements specification, and quite seldom

Chapter 1. Introduction

notations such as use cases, scenarios, UML diagrams, state machines. How-ever, requirements expressed in natural language can sometimes be ambiguous, vague, inconsistent, while intuitive to use. In order to mitigate the aforemen-tioned problems, several approaches that involve semi-formal and formal spec-ification methods have been investigated. Template-based approaches [2], such as requirements boilerplates [3] and property specification patterns (SPS), are semi-formal methods that use frequently reoccurring patterns, whereas other semi-formal methods rely on requirements meta-model [4][5]. The benefits of the template-based approaches are syntactic and semantic similarities to natu-ral language with reduced ambiguity and a high degree of reuse.

However, the methods are not scalable and template selection requires a lot of effort. In contrast, formal specification methods, e.g., temporal logics [6], Z language [7], etc., enable rigorous analysis but demand mathematical knowledge, which makes them not appealing to the average software engineers. Given the need for a structured yet flexible requirements specification frame-work, in this thesis we propose ReSA as our first contribution. ReSA is a structured specification language in which we extend the requirements boil-erplate approach with the domain knowledge of embedded systems, which consists of concepts (vocabularies) and axioms of specification. Further, the domain knowledge contains equivalence axioms that aim to improve flexibility of specification, in an attempt of making it more usable by engineers. In or-der to improve the quality of the specifications, we check their consistency by transforming them into boolean expressions that are analyzed for consistency via the Boolean Satisfiability technique (SAT) [8].

Requirements specification methods should support advanced analysis in order to facilitate detection of hidden specification errors, e.g., logical incon-sistencies, incompleteness, etc. However, most specification methods fail to consider the lexical semantics and semantic relations of words in the analy-sis [9][10][11]. For instance, the fact that requirements“ASL is activated”and “ASL is deactivated”are opposite statements, is determined only if we estab-lish a semantic relation between the words“activated”and“deactivated”words as Antonyms. In the case of Specification Pattern System (SPS) [2][12], the atomic propositions abstract substantial semantic knowledge, making the anal-ysis shallow.

As a second contribution of this thesis, we introduce a linguistic approach to the semantic representation of the requirements specifications semantics, using ontology encoded in Description Logic (DL) [13]. The ontology is ac-companied by associated reasoning services that allow consistency checking, completeness and logical entailment using the reasoner engine Hermit [14].

The benefit of the ontology extends to the generation of optimal (i.e., consistent and minimal) temporal-logic properties, frequently used in model checking.

Besides the analysis of requirements specifications, the analysis of fquently used design models is crucial in order to ensure that the specified re-quirements are fulfilled, prior to implementation. Model checking [15] is an au-tomated formal verification technique that has gained popularity in academia, and slowly in industry, enabling checking hardware and software finite-state system models against requirements formalized in temporal logics. Model checking verifies if a model satisfies a property by exhaustively exploring the state-space of the model. The properties are usually expressed in Computation Tree Logic (CTL), Timed CTL (TCTL), etc. [6]. The limited applicability of exhaustive model checking in industry is mainly due to the lack of support for legacy systems, and the poor scalability to large models. The latter issue is due to the well-known state-space explosion problem during verification.

In order to provide formal analysis to design models, while addressing the scalability issue, we chose to apply a formal method called Statistical model

Checking, which is a compromise between formal verification and testing.

Sta-tistical model checking is found to scale better than exact model checking on large system models [16]. It is applied to analyze Simulink models as Simulink is the de facto modeling framework in industry. Therefore, as a third contri-bution, we propose a pattern-based transformation of Simulink models to net-works of stochastic timed automata, and apply the statistical model checking using UPPAAL SMC [17].

Our methods of specifying and analyzing requirements and behavioral mod-els such as Simulink modmod-els, by employing rigorous methods, enable practi-tioners to increase the quality of the design artifacts used during embedded systems development. The ultimate goal is to be able to transfer such technol-ogy into industrial practice.

1.1

Thesis Overview

The thesis is divided into two parts. The first part is a summary of our research. It is organized as follows: in Chapter 2, we give the background information on requirements boilerplates, description logic, boolean satisfiability problem, Simulink, network of Stochastic Timed Automata. In Chapter 4, we describe the research problems and outline the research goals. In Chapter 3, we describe the research method employed to conduct the research. The thesis contribu-tions are described in Chapter 5, followed by the related work in Chapter 6.

Chapter 1. Introduction

Finally, in Chapter 7, we conclude the thesis and outline possible directions for future work.

The second part is a collection of papers included in this thesis, listed as follows:

Paper A. ReSA: An Ontology-based Requirement Specification Language Tailored to Automotive systems. Nesredin Mahmud, Cristina Seceleanu,

Os-car Ljungkrantz. In Proceedings of the 10th IEEE International Symposium on Industrial Embedded Systems (SIES 2015), pp. 1-10, IEEE, 2015.

Abstract:Automotive systems are developed using multi-leveled architectural

abstractions in an attempt to manage the increasing complexity and critical-ity of automotive functions. Consequently, well-structured and unambiguously specified requirements are needed on all levels of abstraction, in order to enable early detection of possible design errors. However, automotive industry often relies on requirements specified in ambiguous natural language, sometimes in large and incomprehensible documents. Semi-formal requirements specifica-tion approaches (e.g., requirement boilerplates, pattern-based specificaspecifica-tions, etc.) aim to reduce requirements ambiguity, without altering their readabil-ity and expressiveness. Nevertheless, such approaches do not offer support for specifying requirements in terms of multi-leveled architectural concepts, nor do they provide means for early-stage rigorous analysis of the specified requirements.

In this paper, we propose a language, called ReSA, which allows require-ments specification at various levels of abstraction, modeled in the architec-tural language of EAST-ADL. ReSA uses an automotive systems’ ontology that offers typing and syntactic axioms for the specification. Besides enforc-ing structure and more rigor in specifyenforc-ing requirements, our approach enables checking refinement as well as consistency of requirements, by proving ordi-nary boolean implications. To illustrate ReSA’s applicability, we show how to specify some requirements of the Adjustable Speed Limiter, which is a com-plex, safety-critical Volvo Trucks user function.

Contributions: I was the main driver of the paper. I developed the ReSA

lan-guage and specialized it to the different EAST-ADL levels of abstractions. I evaluated the language on a toy example from the Adjustable Speed Limiter (ASL) use case. I also wrote most of the paper, and Cristina Seceleanu gave useful comments on the design of the language, whereas Oscar Ljungkrantz provided useful materials from the automotive industry and arranged meetings

with practitioners from VGTT.

Paper B ReSA Tool: Structured Requirements Specification and SAT-based Consistency-checking. Nesredin Mahmud, Cristina Seceleanu, Oscar

Ljungkrantz. In Proceedings of the Federated Conference on Computer Sci-ence and Information System (FedCSIS 2016), pp. 1737-1746, IEEE, 2016.

Abstract: Most industrial embedded systems requirements are specified in

nat-ural language, hence they can sometimes be ambiguous and error-prone. More-over, employing an early-stage model-based incremental system development using multiple levels of abstraction, for instance via architectural languages such as EAST-ADL, calls for different granularity requirements specifications described with abstraction-specific concepts that reflect the respective abstrac-tion level effectively.

In this paper, we propose a toolchain for structured requirements specifi-cation in the ReSA language, which scales to multiple EAST-ADL levels of abstraction. Furthermore, we introduce a consistency function that is seam-lessly integrated into the specification toolchain, for the automatic analysis of requirements logical consistency prior to their temporal logic formalization for full formal verification. The consistency check subsumes two parts: (i) transforming ReSA requirements specification into boolean expressions, and (ii) checking the consistency of the resulting boolean expressions by solving the satisfiability of their conjunction with the Z3 SMT solver. For validation, we apply the ReSA toolchain on an industrial vehicle speed control system, namely the Adjustable Speed Limiter.

Contributions: I was the main driver of the paper. I implemented the ReSA

tool and evaluated it on industrial use case. I conducted the case study at Volvo Group Trucks Technology (VGTT) through demo, interviews and question-naire preparation. I also wrote most of the paper, whereas Cristina Seceleanu contributed the SAT-based consistency checking, and Oscar Ljungkrantz pro-vided useful ideas and comments on the tool and the consistency-checking approach.

Paper C Semantic Analysis of Embedded System Requirements Specifica-tions. Nesredin Mahmud, Cristina Seceleanu, Oscar Ljungkrantz. Technical

Report, M¨alardalen Real-Time Research Center, Sweden, May 3, 2017 (Ac-cepted as full paper in the 15th International Conference on Software Engi-neering and Formal Methods (SEFM 2017)).

Chapter 1. Introduction

Abstract: Due to the increasing complexity of embedded systems, early

de-tection of software/hardware errors has become desirable. In this context, ef-fective yet flexible specification methods that support rigorous analysis of em-bedded system requirements are needed. Current specification methods such as pattern-based, boilerplates normally lack meta-models for extensibility and flexibility. In contrast, formal specification languages, e.g., temporal logic, Z, etc. are too mathematical to be used by the average software engineer in in-dustry. In this paper, we propose a specification representation that considers thematic roles and domain knowledge that enable a deep semantic analysis of requirements. The specification is complemented by our constrained natural language specification framework, ReSA, which acts as interface to the repre-sentation. The representation that we propose is encoded in the logic-based, usually decidable ontology language called Description Logic. With support from the ontology reasoner, Hermit, we check for consistency and complete-ness of requirements. Moreover, we transform the ontology-based specifica-tion into Timed Computaspecifica-tion Tree Logic formulas, to be used further in model checking system behavioral models.

Contributions: I was the main driver of the paper. I developed the ReSA

se-mantics using a linguistic approach. I evaluated it on a simplified version of the Adjustable Speed Limiter (ASL). I also wrote most of the paper, and Cristina Seceleanu and Oscar Ljungkrantz provided with useful ideas and comments. Paper D Simulink to UPPAAL Statistical Model Checker: Analyzing Au-tomotive Industrial Systems. Predrag Filipovikj, Nesredin Mahmud, Raluca

Marinescu, Cristina Seceleanu, Oscar Ljungkrantz, Henrik L¨onn. In Proceed-ings of the 21st International Symposium on Formal Methods (FM 2016), pp. 748-756, Springer International Publishing, 2016.

Abstract: The advanced technology used for developing modern automotive

systems increases their complexity, making their correctness assurance very tedious. To enable analysis by simulation, but also enhance understanding and communication, engineers use MATLAB/Simulink modeling during sys-tem development. In this paper, we provide further analysis means to indus-trial Simulink models by proposing a pattern-based, execution-order preserv-ing transformation of Simulink blocks into the input language of UPPAAL Statistical Model checker, that is, timed (or hybrid) automata with stochastic semantics. The approach leads to being able to analyze complex Simulink

models of automotive systems, and we report our experience with two vehicu-lar systems, the Brake-by-Wire and the Adjustable Speed Limiter.

Contributions: Together with Predrag, Raluca and Cristina, I contributed to the

design of the STA transformation patterns. Using the patterns, I transformed the whole BBW Simulik model manually into a network of STA and imple-mented the execution order of the automata using inter-arrival times. I also wrote a subsection pertaining to the ASL use case. Predrag wrote the trans-formation section, developed the flattening algorithm and applied verification using Dafny, Cristina wrote the introduction and discussion sections and con-tributed with valuable ideas on the formalization of Simulink models, Raluca wrote the preliminaries section and the section for the BBW use case. Raluca also validated our approach on the BBW use case. Oscar Ljungkrantz, Henrik L¨onn provided with useful ideas and comments on the industrial relevance of our research.

Chapter 2

Preliminaries

In this section, we overview the knowledge needed to comprehend the con-tributions of this thesis, that is, requirements boilerplates, boolean satisfiabil-ity problem, description logic, timed computation tree logic, Simulink, and stochastic timed automata.

2.1

Requirements Boilerplates

Requirements boilerplates are frequently reoccurring patterns of statements to

express software requirements. They have been first introduced to software engineering by Jeremy et al. [3] in order to improve requirements specifications by reducing ambiguities and enabling the reuse of specification patterns. A requirement boilerplate consists of static and dynamic syntactic parts.

Table 2.1 shows examples of boilerplates proposed by Farfeleder [18]. The syntactic elements within the angle brackets are the dynamic parts, otherwise syntactic parts. The engineer has to select the right boilerplates from a repos-itory that suits the target requirements and fill in the dynamic parts in order to complete the specification. For instance, in order to state the requirement for the activation of the ASL system, the engineer might consider usingBP1, BP2 and BP3, and fill in the dynamic parts, stating as:

if <ASLBtn is pressed>, <ASL:system> shall be able to <limit the vehicle speed> within 500ms.

The notion of requirement boilerplate is attractive due to its syntax and semantics being similar to natural language. However, it lacks precise

(math-Chapter 2. Preliminaries

Boilerplate

Id. T ype Requirements Specification

BP1 System shall be able to Action BP2 ... withinQuantityUnit

BP3 ifEvent ...

BP4 System shall have Quality Factor of at least Quantity

Unit

Table 2.1: Examples of Requirements Boilerplates [18]

ematical) semantics, hence less support for rigorous analysis such as consis-tency checking. In this thesis, we use requirements boilerplates as a base on which our domain-specific requirements specification language ReSA is built The specifications in ReSA are automatically transformed into boolean ex-pressions, and checked for logical consistency using the Boolean Satisfiability (SAT) approach, which we overview in the following subsection.

2.2

Boolean Satisfiability Problem

The study of a boolean formula is generally concerned with the set of truth assignments (assignments of 0 or 1 to each of the variables) that make the for-mula true. Finding such assignments by answering the simpler question: “does there exist a truth assignment that satisfies the boolean formula?” is called the boolean satisfiability problem (SAT), which is proven to be NP complete. However, heuristics and approximation algorithms do exist to return the result. A SAT problem is formally defined as follows [19]:

Definition 1 (SAT Problem). Given a propositional formulaΦ = (p1, p2, p3,

..., pn) over the logical connectives ¬, ∨, ∧, →, where: p1, ..., pn are atomic propositions, the SAT problem equates to checking if there exists an interpre-tation (or a model) that satisfiesΦ, that is, a TRUE or FALSE assignment to the variables of p1, ..., pn, such that Φ evaluates to true, in which case Φ is called satisfiable. In the opposite case, if no such assignment exists, thenΦ is

unsatisfiable [19].

Finding the answer to the SAT problem for large boolean formulas require tool support, which we introduce below.

Z3 Theorem Prover/ SMT Solver. Satisfiability Modulo Theories (SMT) [20] is a decision problem expressed in first-order logic which integrates a col-lection of background theories, e.g., on real numbers, bit vector, array, etc. SAT problems are usually described in terms of satisfiability, which is checked using decision procedures, known as SAT solvers. Z3 is a well-known SMT solver as well as a theorem prover developed by Microsoft Research which integrates several background theories and decision procedures using a DPLL-based SAT solver [21]. It has been used in the formal verification and reasoning of software and hardware systems. Further, it has a strong support for integra-tion to verificaintegra-tion tools using its well-known APIs, written in Java, Phyton, C++, C and .Net.

The Z3’s input language is an extension of the SMT-LIB2 standard [22] which is used to assert formulas (expressions) as commands. The formulas can be checked for satisfiability using the command (check-sat). If the formulas are satisfiable, Z3 returns sat; if the formulas are not satisfiable, Z3 returns unsat, or if it cannot decide it returns unknown. In case of unsat, if the unsat-core option is enabled, Z3 returns the unsatisfiable subsets of assertions using the

(get-unsat-core) command. In this thesis, we apply Z3 to check the consistency

of requirements specifications, expressed in ReSA [23] (Chapter 9).

A propositional formula is in standard form if it is expressed in Conjunc-tive Normal Form (CNF). A CNF formula is constructed from the conjunctions (∧) of clauses. A clause is a disjunction (∨) of literals, which are propositional variables p or¬p. Although the SAT approach can handle the analysis of re-quirements specifications with of hundreds of propositional variables [24], in reasonable time, substantial information is abstracted by the propositional vari-ables, e.g., lexical semantics and adjuncts, such as timing information. Conse-quently, a deeper analysis is not possible1. In order to circumvent the problem, we complement the SAT-based requirements analysis approach with one that uses a knowledge base, where requirements specifications are represented in description logic that enables a deep reasoning based on lexical semantics (see Section 5.2). Hence, in the following subsection, we briefly overview descrip-tion logic.

2.3

Description Logic

Description Logic (DL) [13] has its origins in the 1970s, and is intended to address the need for logical support to the representation of knowledge bases,

Chapter 2. Preliminaries

mostly designed by using frames and semantic networks. Currently, the lan-guage is predominantly used in web semantics (e.g., OWL) [25], artificial intel-ligence [26], bioinformatics [27], software engineering, natural language pro-cessing. The language is designed with practicality in mind; consequently, it is usually backed by solid reasoning services that terminate and deliver results in reasonable time. The language uses Concepts (unary predicates), Roles (binary predicates) and Instances (logical symbols) that are recursively constructed us-ing Constructors. The most common constructor Subconcept (or concept inclu-sion), is used to define concept hierarchies, and the equivalence constructor

≡ defines the equivalence of concepts, through the bidirectional concept

inclu-sions. Other basic constructors of the language include the conjunction (inter-section), disjunction (union) , the existential quantifier ∃, and the universal quantifier∀. Currently, there are many variants of the language that differ in their expressivity. ALC (Attribute Language Complement) is considered the basis of the many Description Logic variants, for instance,SROIQ which is implemented in OWL 2 extendsALC (abbr. S) with the role inclusion (R), nominal (O), inverse role (I), qualified cardinality restrictions (Q). A well-formedALC expression is obtained recursively from the following syntax:

(1) ::= A| C | C  D | C  D | ∃R.C | ∀R.C (2) ::= a:C| (a, b):R

where: A atomic concept, C, D complex concepts, R a role, a and b -instances

Line (1) shows the syntax to construct the terminological axioms (also known as TBox) that assert the general problem of a domain, and Line (2) shows the syntax to construct assertion axioms that assert the concrete facts of a domain (also known as ABox). In DL, a knowledge baseK = (T, A) contains the TBox (T) and ABox (A) and the interpretationI of the knowledge-base K is via set theory. Consequently, the interpretationI = (ΔI, FI) contains a

non-empty domainΔI, and a function FI that maps every concept, role, in-stance of the knowledge base to the the elements of the domain.

In this thesis, we use DL to represent the semantics of requirements spec-ifications using the notion of ontology. An ontology is a shared and reusable resource for knowledge-based systems. The requirements specifications are initially expressed in the constrained natural language ReSA. The benefit of this approach is the fact that it allows a deep analysis of requirements, not just at the syntactic level, but also on the semantics of the requirements statements that are expressed in a language that closely renders the syntax and semantics of natural language.

2.4

Timed Computation Tree Logic

TCTL is the timed extension of Computation Tree Logic, and it enables rea-soning about timing behavior of finite-state models besides the temporal oc-currence of events [28]. The semantics of TCTL specifications (or properties) are based on branching time and states. A well-formed TCTL property is in-ductively generated based on the following production rule:

φ:= true | p | ¬p | φ | φ ∨ φ | E[φU_tφ] | A[φU_tφ] (2.1) where: p ranges over a set of atomic formulas, A and E are the universal and existential path quantifiers, respectively, U (Until) is a temporal operator and,  represents one of the relational operators:{=, <, ≤}

Based on the minimal set of operators given in 2.1, we can define prop-erties involving the “Always” (), or “Eventually” (♦) temporal operators, as follows:

• Invariance:A  p - the property states that for all paths, p always holds. • Time-bounded Response:A  (p ⇒ A ♦_≤tq). The property states that,

for all paths, it is always the case that once p holds, q eventually holds within t time units (♦_≤tis the “eventually within t” operator).

Given a finite-state model M and an initial state s₀, and a (T)CTL formula

φ, the fact that M satisfies φ can be formally expressed as follows: M, s₀|= φ.

2.5

Simulink

Simulink is a graphical development environment for the modeling, simulation

and analysis of embedded systems. It is widely used in industry to model and observe the dynamics of systems before implementation. It supports the mod-eling of hybrid systems, which exhibit both discrete and continuous behaviour, and multirate systems, which have multiple sample times. Furthermore, it is common that code is generated from Simulink models, especially in industry, hence the need to analyze Simulink models for errors.

A Simulink model is built from communicating function blocks, known as Simulink blocks. The blocks implement simple to complex functions over various datatypes (integer, floating, boolean, etc.) and input/output dimensions (scalar, vector, matrix). Simulink blocks are classified into virtual and

Chapter 2. Preliminaries

organize the models graphically and do not play active roles in the simula-tion of the models. In contrast, the non-virtual blocks, e.g., Gain, Integrator, Atomic SubSystem, etc., are computational blocks, hence determine the sim-ulations’ behavior. In addition, Simulink blocks can be classified into atomic and composite blocks. Atomic blocks, such as Gain, Integrator, etc., are prim-itive blocks, whereas composite blocks, e.g., SubSystem and Model blocks, integrate atomic and other composite blocks to create hierarchical models.

(a) Simulink Diagram (b) Simulation Output

Figure 2.1: Discrete vs. Continuous Block Simulations

The atomic blocks that are non-virtual (computational) are categorized into

continuous and discrete blocks based on the semantics of the blocks’

execu-tions. Figure 2.1b shows the simulation traces of both types, which are graph-ically modelled as shown in Figure 2.1a. A discrete block executes period-ically with sample time ts, whereas a continuous block executes over small

time intervals given by a minor sample time, where the successive states of a model are computed using a solver. Since the Simulink blocks libraries are not usually sufficient to model practical embedded system systems, Simulink has provided extensions mechanisms, that engineers can use to fulfill their require-ments. The mechanisms include S-function, Custom Block and Masking [29]. S-function is a computer language of Simulink blocks which allows advanced implementations of block routines, written in MATLAB, C, C++, or Fortran.

2.6

Stochastic Timed Automata

In the 1990s Alur and Dill [28] proposed the timed automata theory in re-sponse to the need of formally verifying real-time systems. A real-time model described in terms of timed automata can be used to verify, by model checking, if it satisfies the given requirements, usually expressed in temporal logics such as TCTL.

In the following we give the formal definitions of timed automata and stochastic (priced) timed automata that we use in Paper D (Chapter 11), for the formal analysis of Simulink models [30].

Timed Automata (TA). Timed automata formalism extends finite-state au-tomata with real-valued variables called clocks, which measure the passage of time and can be reset. Assume that X = {x1, x2, ..., xn} is a finite set of

clocks. Then, a clock valuation is a mapping ν : X → RX_{, and B(X)}

repre-sents the set of formulas called clock constraints of the following form: x  c, or x− z  c, where x, z ∈ X, c ∈ N and ∈ {<, ≤, =, >, ≥} . A clock constraint is downwards closed if ∈ {<, ≤, =}. A timed automaton is a tuple:

T A= L, l₀, X,Σ, E, I (2.2)

where:

• L is a finite set of locations, • l₀∈ L is the initial location,

• X is a finite set of dense-time clock variables,

• Σ = Σi Σois a finite set of actions, whereΣiis a set of input actions,

andΣois a set of output actions,

• E ⊆ L × B(X) × Σ × 2X _{× L is a finite set of edges denoted by}

(l, g, a, r, l_{), where l and l} _{are the source and destination locations of}

the edge, respectively, g is a predicate onRX_{called guard, a∈ Σ is an}

action, r is the set of clocks that are reset,

• I: L → B(X) assigns invariants (downwards closed clock constraints) to locations; invariants bound the time allowed in a particular location. The semantics of a timed automaton is defined over a timed transition

system S, −→ where S is the set of states s of pairs (l, ν) ∈ L × RX_,

where l is a location and ν is a clock valuation, and−→ is the transition re-lation that defines: (i) discrete transitions,(l, ν) −→ (la , ν), occurs if edge

Chapter 2. Preliminaries

e= (l, g, a, r, l), such that ν |= g, ν = [r → 0]ν, where [r → 0]ν denotes

the clock assignment that maps all clocks in r to0 and agree with ν for the other clocks in X\r, and (ii) delay transitions, (l, ν)−→ (l, ν + d), occurs ifd

ν|= I(l), and (ν + d) |= I(l), d ∈ R_≥0.

Stochastic Timed Automata (STA). Most real-time systems are physical processes that operate in a naturally stochastic environment. The behavior of such real-time systems requires timely operation, and involves system and environment variables that change randomly over time. Stochastic Timed Au-tomata (STA) [31][32] is a theory proposed to model and verify real-time sys-tems with a stochastic nature. Accordingly, the discrete and delay nondeter-ministic choices are replaced by probabilistic ones, as follows:

STA = TA, μ, γ (2.3)

where:

• TA is a timed automaton,

• μ is the delay density function, which is the set of delay density functions (μs), s∈ S,

• γ is the output probability function, which is the set of output probability functions (γ_s) over the output edges of STA.

Network of STA. Under the assumption of input-enabledness, disjointness of clock sets and output action sets, a network of STA (denoted by NSTA) is parallel composition(A₁A₂...An) where A1, A2...An are STA. The state

of a networked TA is a tuples1, s2, ..., sn where si∈ Aiof the form(l, ν) ∈

L_i× X_i, and i∈ N. In NSTA, the automata communicate through broadcast channels and globally shared variables. Moreover, the semantics of the NSTA relies on the independent computation of the individual STA: each component automaton, based on the delay density function and the output probability func-tion repeatedly decides on which output to generate and at which point in time. In this race, the output will be determined by the component automaton that has chosen to produce output after the minimum delay [31].

UPPAAL Statistical Model Checker. UPPAAL Statistical Model Checker (SMC) is a major upgrade of the UPPAAL model checker [33] with support for model checking stochastic timed automata, by using a statistical approach [34], such as hypothesis testing and Monte Carlo simulation.

In UPPAAL SMC, the delay probability density function μ_soverR_≥0takes a uniform or an exponential distribution depending on the delay in location l being bound by an invariant, or not, respectively. Let El denote the

disjunc-tion of(l, g, o, −, −) ∈ E for some output o. The infimum delay before the output edge El is denoted by d(l, ν) = inf{d ∈ R≥0 : ν + d |= El},

whereas D(l, ν) = sup{d ∈ R≥0 : ν + d |= I(l)} is the supremum delay.

If d(l, ν) < ∞ the delay probability density function μsis a uniform

distri-bution over the interval[d(l, ν), D(l, ν)], for every state s, otherwise (when the delay is unbounded) it is an exponential distribution with the rate P(l), where P : L → R_≥0 is the distribution rate function applied to all locations not bounded by an invariant. The output probability function γ_soverΣ_ois a uniform distribution for every s, and(l, g, o, −, −) ∈ E_l∧ ν |= g whenever

E_l= ∅.

L1

x=0

x>6

L2

x
2

x4

1000

guards

invariant

rate

locations

clock reset

Figure 2.2: Example: STA

An example of STA in UPPAAL SMC is shown in Figure 2.2, and a clock observance of x on the automaton is shown in Figure 2.3. The delay in the location L1 is distributed uniformly in the time-bounded interval[0, 2], which is enforced using the invariant x≤ 4 and the guard x ≥ 2 as shown in Figure 2.3a. In the location L2, the automaton makes delay transitions according to the exponential distribution with the rate λ= 1000 since D(L2, ν) is unbounded, as shown in Figure 2.3b. The probability of leaving L2 after t time units is given by P r(leaving after x) = 1 − e−λx. The higher the exponential rate is, the sooner the automaton leaves location L2.

Chapter 2. Preliminaries

0

2

4

18

12

6

(a) Bounded Delay, L1

0

6

12

6

(b) Unbounded Delay, L2

Figure 2.3: Clock Observance over Different Instances of x

Properties Specification. In UPPAAL SMC, properties are expressed in the probabilistic extension of the weighted metric temporal logic (WMTL), de-noted by PWMTL [35]. The WMTL properties are recursively constructed using the following production rule:

φ::= true | p | ¬p | φ ∧ φ | φU_c≤tφ (2.4) where: p is an atomic proposition, t ∈ N, c ∈ X, and U is the “until” path-specific operator. Operators equivalences: “eventually within time t” ♦c≤tφ= trueUc≤tφ, and “always within time t”c≤t= ¬♦c≤t¬φ.

A valid PWMTL formula is defined as follows:

ψ::= P (♦_c≤tφ)  p | P (_c≤tφ)  p (2.5) where: c is the observer clock of the automaton under analysis, φ is a state property, ∈ {<, ≤, =, ≥, >}, and p ∈ [0, 1].

The following probabilistic properties can be checked using UPPAAL SMC: • Hypothesis testing (P r([∗]c≤tφ) ≥ p), checks the if the probability to

satisfy a time-bounded property is greater than or equal to p, where: [∗] is either♦ or , t ∈ N.

• Probability comparison (P r([∗]_c1≤Cφ₁)  P r([∗]_c2≤Cφ₂)) - compares

the probabilities of satisfying the time-bounded properties, where: ∈

{<, >, =, ≤, ≥}.

• Probability evaluation P r([∗]c≤Cφ)) - evaluates the probability of

satis-fying the specified property by a random run of some NSTA.

Chapter 3

Research Goals

In this section, we present the research problem that is dealt with in this thesis. The research problem is approached via the formulation of research goals that raise critical research challenges and refine the problem into manageable goals.

3.1

Problem Description

Requirements specification and analysis, as well as verification and validation of embedded software models are critical stages of the system development, as they allow detection and removal of software bugs at early stages, preventing their propagation to the implementation stage. Early treatment of software bugs reduces maintenance cost, being also time efficient. In practice, embedded sys-tem requirements are specified in natural language, which is intuitive, expres-sive and flexible to use. However, natural language can also lead to frequently ambiguous, vague, incomprehensible and inconsistent specifications. Conse-quently, finding a way to specify requirements in a structured and reusable manner, to reduce possible ambiguity, while still allowing flexibility in speci-fication is a challenging research problem. An equally challenging problem is analyzing requirements for logical inconsistencies, in a fast and effective way. Verifying embedded system models is a crucial activity in order to detect de-sign errors that prevent the models from meeting their functional and timing requirements. However, currently there are no formal approaches that can han-dle large models described in languages such as Simulink, such that they can be successfully applied to their verification. Defining a rigorous approach for the scalable analysis of industrial Simulink models is a challenging problem.

Chapter 3. Research Goals

3.2

Research Goals Formulation

In order to address the research problems described above, we define the over-all research goal as follows:

Overall Research Goal.

Improve software quality of embedded systems by marrying effective and scalable formal techniques with engineering friendliness.

Since improving the quality of embedded systems software relates closely to improving the quality of the artifacts used during the development lifecycle, one of the main challenges of addressing the overall goal is to find appropriate formal techniques able to help in this endeavor. Another challenge is to hide such formal techniques from the engineers, while providing the latter with user-friendly interfaces.

Since the overall goal is broad, we split it in six narrower research subgoals, defined below. We have argued previously that requirements specification is one of the crucial artifacts of embedded systems development, whose compre-hensibility and quality contribute to the end quality of the software. Based on this, we formulate our first research subgoal, as follows:

RS 1. Improve the structure, comprehensibility and flexibility of embedded

systems requirements specifications via a domain-specific language.

On the one hand, the language should allow comprehensible requirements specifications that can easily be understood by the technical and non-technical stakeholders. Further, it should be engineer friendly, e.g., it should use similar syntax and semantics to natural language. On the other hand, the domain-specific language should prevent one from specifying ambiguous requirements. Therefore, an appropriate tradeoff between flexibility and formality should be found.

Requirements specifications should also be supported by analysis in order to detect nontrivial specification errors, e.g., logical inconsistencies or incom-pleteness of specifications. Therefore, the second subgoal is:

RS 2. Enable the semantic analysis and transformation in temporal logics of

requirements specifications expressed in constrained natural language.

The challenges of achieving research subgoal 2 are twofold: i) first, the se-mantics representation should capture the intended meaning (sese-mantics) of the

requirement specifications as accurately as possible, ii) second, the represen-tations should consider the tradeoff between the represenrepresen-tations’ expressivity and the reasoning complexity. The more expressive the language of the se-mantic representations is, the less decidable and computationally tractable the reasoning becomes.

The benefit of the semantic representation of requirements specifications is justified by defining operations on the specifications that enable engineers to do useful and rigorous analysis in order to detect and resolve specification errors. Therefore, the third subgoal is to:

RS 3. Analyze formally the consistency of requirements specifications

expressed in constrained natural language.

A requirement specification is a statement constructed from constituents (i.e., words and phrases) that have lexical semantics. Therefore, it is crucial that we consider the lexical semantic relations in the analysis of requirements specifications. Examples of lexical relations consist of synonymous, antonyms, thematic roles, etc. In this context, the challenge is to develop axioms that consider the various lexical relations in the target analysis, e.g., consistency checking, which are efficient and effective to handle large sets of requirements specifications.

In order to fully support specifying and analyzing artifacts’ quality, tool automation is crucial. In this context, the fifth subgoal is to:

RS 4. Automate the specification and analysis of embedded systems

re-quirements using an effective and engineer-friendly approach.

The automated framework should support the specification and analysis of requirements. Besides, it should be engineer friendly, and easy to integrate into existing toolchains.

Simulink is the “de facto” framework for modeling, analysis and simulation of embedded systems models in industry. Further, Simulink supports automatic code generation from models, and it is expected that the generated code has minimal or no software bugs. A practical solution to facilitate meeting such an expectation is to use formal analysis at design time, hence sanitizing the models before code generation. Therefore, the fifth subgoal is to:

Chapter 3. Research Goals

The challenges of subgoal 5 are stemming from three problems: i) the Simulink language is expressive enough to describe large and complex prac-tical modeling problems through various constructs and configurations, which could overload the formal analysis; ii) Simulink is a proprietary framework as, therefore its working principles are not necessarily publicly available, espe-cially in terms of execution semantics; iii) many industrial models are large, therefore, the formal analysis method chosen to address subgoal 5 should be able to scale, which is a usually difficult goal to achieve.

In order to validate the research results, the proposed methods and tools should be evaluated on industrial use cases, or in the industrial context. There-fore, the last subgoal is to:

RS 6. Evaluate the industrial applicability of the proposed software

improving techniques and their automated tool support.

Relevant industrial use cases should serve as the validation basis, and to the extent possible the evaluation activities should also include collecting feedback from engineers and experts in industry.

Chapter 4

Research Method

A research method is broadly defined as “an approach, procedure, and guide-lines that are used in conducting a research” [36], e.g., through experiments, case studies, interviews, observation, etc. Depending on the research topic, several research methods can be incorporated to conduct research and solve the research questions. The research methods usually involve cyclic and recur-sive activities of data collection and analysis, known as research process. In contrast to research methods, a research methodology, according to Merriam-Webster is the “body of methods, rules, and postulates employed by a disci-pline” [37], which allows researchers to investigate and choose the right re-search methods that could be used in the rere-search processes in order to conduct research effectively.

In this thesis, we pursue the research method framework proposed by Hi-lary et al. [38], also known as Computing Research Methods (CRM) frame-work. The framework is appealing because it provides a generic method of conducting research that can be leveraged in various research contexts. Further, the framework provides a taxonomy to identify the levels of research-method competencies required to conduct research, hence allowing the researcher to self-regulate his/her research-method competence. We apply the CRM frame-work in the context of industry-academia collaboration to conduct research on the application of formal methods in order to improve the quality of embedded systems software.

Our research problem originates from the challenges gathered from devel-oping automotive systems in the Scania and Volvo Group Trucks Technology (VGTT) automotive companies. In order to address some of the issues

con-Chapter 4. Research Method

nected to requirements and other artifacts’ specification and analysis, the two automotive companies jointed the VeriSpec project led by M¨alardalen Univer-sity, which aims to improve the quality of automotive systems through the ap-plication of formal methods. Figure 4.1 illustrates the research process that we pursue in order to address the research problem. The research problem is ex-plained through empirical study at VGTT, which has involved interviews, dis-cussions with practitioners as well as investigating existing automotive system development tools and development guidelines. Consequently, the research problem is formulated into an overall research goal.

Identify Research Goals Propose Solution Validation State-of-the-Art State-of-the-Practice Publication Prototype Problems in Industry Use Case/

Toy example Case Study Industrial-related activities

Academic-related activities

Figure 4.1: Research Process

The overall goal is refined into manageable research subgoals in an itera-tive way. For each subgoal identified, we have developed a solution through proof-of-concept [39], which has involved conceptual models, a prototype tool, mathematical proofs, etc. We have developed a requirements specification lan-guage tailored to embedded systems and analysis techniques based on SAT and ontology. Moreover, we have provided a formal verification of Simulink models that scales to industrial applications. When developing the solutions, there have been frequent discussions and participation in workshops, which has allowed us to connect and share challenges and the state-of-the-practice. Consequently, there have been frequent modifications to the research goals and the proposed solutions. We have also contributed with scientific knowledge through publications and presentations on international venues.

We validate our solutions through industrial uses cases, toy examples, and 26

case studies [40] at the companies. So far, we have validated the proposed requirements specification language and its associated tool support on the Ad-justable Speed Limiter (ASL) use case and through a case study at VGTT. The case study has targeted identification of potential practitioners who are in-volved in the requirements engineering, software engineering, verification and research. Besides, we have demonstrated the language and the tool to the prac-titioners through video demo and exemplified scenarios. Next, we have asked them to apply our methods on a selected requirements set, we have interviewed them and provided them with a questionnaire to get feedback for further im-provements. Moreover, we have validated our proposed formal analysis of Simulink models on the Brake by Wire (BBW) and Adjustable Speed Limiter Engine Manager (ASL-EM).

Chapter 5

Thesis Contributions

In this chapter, we summarize the contributions of this thesis, which address the research goals discussed in Section 3.2. The main contributions consist of: (i) a domain-aware requirements specification language, described in Section 5.1, (ii) requirements analysis using SAT-based and ontology-based techniques, presented in Section 5.2, and (iii) scalable statistical analysis of Simulink mod-els, overviewed in Section 5.3. The use cases employed to demonstrate the contributions are the Adjustable Speed Limiter (ASL) and the Brake-By-Wire (BBW) industrial systems. The ASL system is an operational system from Volvo Group Trucks Technology (VGTT), whereas BBW is a prototype sys-tem from the same company. For each use case, we have been provided with system development artifacts, including the requirements specification docu-ments, software and hardware architectural models, and behavioral models de-scribed in Simulink.

5.1

ReSA - An Ontology-based Requirements

Specification Language

Our first contribution is ReSA [41], a domain-specific requirements specifica-tion language designed to specify funcspecifica-tional and timing requirements of auto-motive systems in constrained natural language, as proposed in Paper A (Chap-ter 8), which addresses Subgoal 1. ReSA is based on an ontology that uses em-bedded systems concepts and syntactic axioms required to specify well-formed requirements specifications with reduced ambiguity and improved

comprehen-Chapter 5. Thesis Contributions

sibility. According to Borst, “an ontology is a formal specification of a shared conceptualization” [42]. The main atomic concepts of the requirements specifi-cation ontology are System,InP arameter, OutP arameter, InDevice (e.g., sensor devices), OutDevice (e.g., actuator devices), State, M ode and Event. The concepts are classified into Entity, Attribute and U ser complex con-cepts according to semantic similarity. User-defined concon-cepts can be created and existing ones can be specialized to suit the abstraction that the language is describing through the SubTyping principle, which allows a subclass instance to be used safely in places where a superclass instance is expected.

The development of the language is inspired partly by the lack of a meta-model of requirements boilerplates [43]. As a result, the valid strings of the ReSA language comprise boilerplates (or sentential-forms, i.e., contain vari-ables). The valid classes of boilerplates supported by the language are shown in Table 5.1. Moreover, the language supports the specification of requirements at various levels of abstraction through the use of words, phrases and syntactic axioms that are distinct for each abstraction level, which is exemplified in this thesis to describe requirements specifications at the various EAST-ADL levels of abstractions.

5.1.1

Syntax of ReSA

The ReSA syntax is carefully designed to render the syntax of English in order to promote its usability by engineers. Furthermore, ReSA allows typesetting of syntactic elements with the construct term : T ype for inline instantia-tions of model elements during specification, where term is a value, and T ype is one of the embedded system concepts. Moreover, ReSA supports condi-tional statements using delimiters, e.g., if. . . endif, in order to structure and make the specifications clearer and easier to comprehend. The ReSA grammar

G is context-free and is defined by the tuple:

G::= (N, N_s, T_a, T, S, P) (5.1) where: N is a set of non-terminal strings, Ns ⊆ N is a set of system-level concepts, T_a ⊆ N_s|Ta ∩ Ts = ∅is a set of abstraction-level concepts,T |T ∩ N = ∅

is an infinite set of terminal strings,S ⊆ Nis a set of initial strings andP is a set of production rules.

The ReSA language L(G) supports simple, complex and compound state-ments, and thus can express a wide range of requirements classes, e.g., con-ditional, occurrence of actions, timing (e.g., periodicity, duration, time points

Boilerpalte Description

Simple Instantiates a simple statement and contains a modal verb, such as, shall, e,g., system shall be activated.

Proposition Similar to Simple except it is a proposition (or an assertive state-ment) [44, p.435], e.g., button is pressed.

Complex Instantiates a complex statement and is constructed from Sim-ple, Proposition boilerplates and an adverbial conjunctive (such as while, when, until, etc.). For example, the error shall be reported while the fault is present Compound Instantiates a compound statement, and is composed of two or

more Simple or Proposition boilerplates and the logical opera-tors AND/OR. For example system shall be activated

and driver shall be notified.

Conditional Instantiates a conditional statement. The boilerplate can be in-stantiated to a different variant of conditional statements, i.e., if,

if-else, if-elseif, or if-elseif-else, and conditional nesting. Prepositional

Phrase

Instantiates a prepositional phrase, and can be used to describe timing properties, occurrence of events, other complements to the subject of a main phrase. For example, within 5ms, by the driver

Table 5.1: Boilerplate Types

i.e., at t, time interval), etc. Some examples of requirements of the Adjustable Speed Limiter use case, specified in ReSA, are shown in Table 5.2.

We have evaluated the language on 304 ASL requirements out of which 62 are functional, 104 are responsibility requirements1_{, 40 are performance}

requirements, e.g., timing, and the distribution of the requirements types is shown in Figure 5.1. After rewriting the requirements in ReSA, the distribution of the boilerplates used in the specifications is seen in Figure 5.2. The number of functional block responsibility requirements is higher than the functional re-quirements; however, in the latter figure, we see that the number of boilerplates used for specifying functional requirements is much higher than for the other types of requirements, which indicates encoding of more information in the former. The language has been evaluated at a small scale, via questionnaires, 1_{concise descriptions of roles, which are refined into functional and extra-functional} require-ments.

Chapter 5. Thesis Contributions

ReqId. T ype Requirements Specification

R1 TimingSimple : ASL:system shall send "the driver":user notification:status every 200ms. R2 ifconditionalConditional:

if (ASL:system is activated and

(“the driver”:user presses PAUSEBtn:inDevice or vehicle:user is not in running:mode))

then

ASL:system shall stop to limit “vehicle speed”:parameter and “ASL status”:status shall be set to “enabled” and

endif

R3 OccurenceComplex:

After ASL:system is enabled, if IncButton:inDevice is pressed, ASL:system shall be activated.

Table 5.2: Examples of ASL Requirements Expressed in ReSA

HMI Requirements 10% High-level Requirements 8% Configurability Requirements 10% Functional requirements 21% Functional-block Responsibility Requirement 34% Performance Requirements 13% Safety Requirement…

Figure 5.1: Requirements Types Distribution

interviews, and hands-on experience with specifying requirements in ReSA, with engineers from VGTT and based on their feedback, we have modified some constructs of the language to increase its appeal.