The effects of test process automation on fault frequency for a spreader system

(1)

(2)

.

Examensarbete MMK2016:82 MDA531

Inverkan p˚a felfrekvensen av testprocessautomation f¨or lyftok Henrik Hamber

Linus Skjutar Apell

Godk¨ant: Examinator: Handledare:

31-05-2016 Martin Edin Grinmheden Jinzhi Lu

Uppdragsgivare: Kontaktperson:

Bromma Conquip Anders Br¨annstr¨om .

Sammanfattning

.

Det här examensarbetet utfördes p˚a R&D avdelningen p˚a Bromma Conquip, den tekniskt ledande tillverkaren av kontainerlyftok. I linje med den allmänna utvecklingen har oken kommit av inkludera mer avancerad elektronik och mjukvara. Detta har gjort dem sv˚arare att utveckla med nuvarande metoder och ökat vikten av mjukvarutestning. I dagsläget används modeller av systemet men dem representerar inte dess dynamik. Testprocessen för mjukvaran är fullständigt manuell vilket gör den tidkrävande och känslig för mänskliga fel. Uppgiften best˚ar i att undersöka hur komplex modellen behöver vara för att vara användbar och hur mjukvarutestningen bäst ska automatiseras. Dessutom skulle ytterligare eventuell nytta av modellen och automationen undersökas.

En litteraturstudie genomfördes för att bilda den teoretiska grunden för arbetet. Denna komplimenterades med intervjuer med personal p˚a företaget för att samla specifik information om deras produkter, utvecklingsprocess och företagskultur. Ett systemtekniskt tillvägag˚angssätt användes genomg˚aende i arbetet. För att hitta den lämpliga niv˚an av komplexitet tillämpades en modellbaserad utvecklingsprocess p˚a det teleskopiska lyftoket Bromma STS45. Processen bestod utav en systemanalys, kravhantering, modellering och simulering vilket följdes av verifiering och validering. För att undersöka hur mjukvarutestningen bäst ska automatiseras användes en trestegsprocess. Först analyserades den nuvarande testprocessen. Sedan definierades best practice inom industrin baserat p˚a standarder för säkerhetskritiska system. Slutligen relaterades denna best practice till företagets behov för att bestämma vilka delar av denna som var relevanta.

F¨or att hitta eventuell ytterligare nytta av modellen och automationen analyserades resultaten och komplimenterades med en genomg˚ang med f¨oretaget.

Resultatet för denna examensarbete visade att det var tillräckligt för modellen att represen- tera mekaniken hos den teleskopiska balken för att uppn˚a en passande niv˚a av komplexitet. En samling av föreslagna practices för testautomation baserat p˚a de viktigaste best practices pre- senterades. Dessa föreslagna practices kan minska frekvensen av mjukvarufel för systemet. Flera andra avdelningar p˚a företaget kunde dra nytta av modellen och automationen. Den främsta av dessa var möjligheten att överg˚a till en modellbaserad utvecklingsprocess för R&D avdelningen.

(3)

.

Master of Science MMK2016:82 MDA531

The effects of test process automation on fault frequency for a spreader system

Henrik Hamber Linus Skjutar Apell

Approved: Examiner: Supervisor:

31-05-2016 Martin Edin Grimheden Jinzhi Lu

Commissioner: Contact person:

Bromma Conquip Anders Br¨annstr¨om .

Abstract

.

This master thesis was performed at the R&D department of Bromma Conquip, the technical leader in the business of cargo container spreaders. In line with the general turn in the industry the design of the spreaders has come to include more advanced electronics and embedded software.

This has made the system more difficult to develop and increased the importance of software testing. Currently, models of the system are used in the testing but they do not represent its dynamics. The test process for the embedded software is currently completely manual, making it time consuming and susceptible to human error. The task consisted of finding how complex a simulation model needed to be in order to be useful and to investigate how to best automate the test process. A third task consisted of finding additional benefits of the model and the automation.

A literature survey was conduced to form the theoretical foundation of the thesis. This was complemented by interviews with personnel at the company to gather specific information about their products, development process and company culture. A Systems Engineering approach was used throughout the thesis work. To find the suitable level of dynamics a Model-Based Development process was applied to the Bromma STS45 telescoping spreader. The process consisted of a system analysis, requirements engineering, modelling and simulation followed by verification and validation. To find how to best automate the testing a three stage process was used. First, the current manual process was analysed. Second, best practice in the industry as defined by standards for safety-critical systems was defined. Third, best practice was related to the needs of the company to find which of them that was relevant. To find the additional benefits of the model and the automation the results was analysed and complemented with a review wit the company.

The results of the this master thesis is that it is sufficient for the model of represent the mechanics of the telescopic beam of the spreader to achieve a suitable level of dynamics. A set of purposed practices for test automation based on the most important best practices was presented. These purposed practices will reduce the frequency of software related faults in the system. Several additional benefits for other departments was also presented. The main benefit consists of the ability to transition to a Model-Based Development process for the R&D department.

(4)

Foreword

This master thesis in mechatronics was performed at Bromma Conquip AB, a Cargotech company. The master thesis is the final part of the Master’s programme in Engineering Design with the mechatronics track at the Royal Institute of Technology (KTH) in Stockholm, Sweden. This thesis is a collaborative effort by Henrik Hamber and Linus Skjutar Apell. Since the thesis is a part of the examination for the degree the work was divided so that individual assessment was possible.

Henrik Hamber focused on the process and requirements aspects of the thesis. Some of these are included in the Sections 3.1 Systems Engineering, 3.2 Model-Based Systems Engineering, 3.2.1 Model-Based Development, 3.4 The development process used by Bromma Conquip, 4.2 Requirements analysis and 6.6 Feedback from the investigation of Model-Based Development.

Henrik Hamber had the overall responsibility for Section 5 Test Automation.

Linus Skjutar Apell focused on the modelling and analysis aspects. Some of the sections where this is included is Section 3.3 Bromma STS45 telescoping spreader system description, 4.3 System design, 4.4 Subsystem design, 4.5 Component design, 4.6 and 4.7 Validation. Linus Skjutar Apell had the overall responsibility for Section 4 Modelling & Simulation of the Bromma STS45 telescoping spreader.

(5)

Acknowledgements

There are several people that has made contributions to this master thesis and helped it towards its completion and deserves to be acknowledged. Fredrik Asplund for his help with the research questions and advice for the methodology during the early phases of the work. Jinzhi Lu for providing support and valuable feedback from an academic viewpoint as supervisor at KTH.

Anders Brännström for providing insight into the tasks at hand and feedback from an industrial viewpoint as company supervisor. Fredrik Löwnhielm for providing invaluable insight into the workings of the company and the cargo industry. The personnel at the R&D department at Bromma Conquip in Stockholm for their helpful and welcoming attitude.

(6)

List of abbreviations

SE - Systems Engineering

MBSE - Model-Based Systems engineering MBD - Model-Based Development

DOF - Degrees Of Freedom BC - Bromma Conquip AB TBU - Telescopic Beam Unit AC - Alternating Current

ISO - International Organization for Standardization

SI - Syst`eme International d’Unit´es / International System of Units V - Volt, SI unit

VM - Virtual Machine

ECU - Electronic Control Unit CAN - Controller Area Network SiL - Software in the loop

PLC - Programmable Logic Controller

(9)

1 Introduction

This Master Thesis work is performed at Bromma Conquip by two student from KTH, Kungliga Tekniska H¨ogskolan, in Stockholm. In this chapter the introduction to the thesis is presented.

First the background to the problem which the thesis have focused on is described in section 1.1. Then the actual problem description is presented in section 1.2 after which the research questions that the thesis aims to unravel are listed in section 1.3. The chapter ends with section 1.4 where a description of the limitations which have affected the thesis work are presented.

1.1 Background

Today almost all technical products include embedded software to control or monitor their behaviour during operation. This is the result of the last years intensive technical development which have led to smaller and faster microcomputers which in turn have made automation more feasible and widespread. However, these new features make successful development increasingly complex which have given the industry new challenges. Challenges which include Systems En- gineering practices which with the use of models works towards realizing Model Based Systems Engineering. One of the industries in which smart technology and digitalization have led to many improvements both in performance and safety is the cargo freight industry. Improvements such as automation of cranes, spreaders and whole freight terminals have been more and more common during the last decade.

Bromma Conquip (BC) is one of the technical leaders in the industry of cargo container spreaders and produced the first telescoping spreader in 1965 [1][2]. The development of spreaders at BC have transitioned from technology driven to market driven during the last 20 years. Opened up possibilities by improved computer technology led to the creation of the first smart spreader in 1995 and the first all electric spreader in 2001. As the computer technology development sped up the transition became more apparent. Creation of the height indication system in 2002, third generation spreader control system SCS3 in 2007 and load sensing system 2010 are all implementations focusing on safety and performance due an increasing customer demand. The latest version of the spreader control system SCS4 was released in 2013 [3]. A photo of a STS45 Bromma spreader can be seen in Figure 1.

Figure 1: Bromma STS45 spreader lifting a 45ft container.

(10)

The main customers are crane manufacturers and ports which have high demands on safety and reliability. In order to meet the customers’ requirements on safety and reliability the spreaders have been developed into not only a mechanical construction but also more and more integrated components such as embedded computers and sensors which makes the development of the spreaders an increasingly complex task.

To ensure high reliability testing is integrated in the development process and during operation uptime is ensured by diagnostics and support. However, some faults atill found in late stages of the development process such as deployment or operation which cause more problems and cost compared with those found earlier in the process. The embedded systems software testing, called functional testing using simulation model, is currently done for developed and customized software templates. These templates are developed in Sweden and tested manually in Malaysia with a simulation model as reference, a so called Software in the loop test (SiL).

State of the art research in cargo container freight industry includes some research on the dynamics of container cranes and spreaders which consists of movement of the container e.g.

swinging, lifting speeds and stability. Simulations of the movement of the spreader body and crane in several degrees of freedom are presented in [4]. The dynamic functions i.e. the combined operation of electrical, hydraulic and mechanical systems of the spreader have not been researched, although these areas are well researched in other contexts such as automotive applications. To increase the overall performance and reliability further research on electrical, embedded, hydraulic and mechanical functions of the spreader are needed.

1.2 Problem description

To ensure high reliability and safe operation Bromma Conquip use software testing in the pre- production stages to verify the embedded software in the spreaders. To find possible faults in the software the software templates are tested against a physical system model during the software development phase. Currently, however, the software testing is done manually which results in that some faults are not found until the production or deployment stage of the development process. Faults in these stages of the development process are a possible source for increased downtime which leads to economic losses for both Bromma Conquip and their customers.

The best practice for the development of safety-critical systems requires that the software goes through regression testing after changes are applied to the code [5]. For regression testing to be feasible it requires the software testing to be performed fast and with reliable evaluation methods.

The model used today represents the embedded system with user interface and physical components of the spreader, implements the systems dynamic properties in a simple manner which decrease the accuracy of tests. Today the software testing is done manually which is time consuming and prone to human errors for each set of tests. This improvement would be a great benefit for both the safety and quality of the STS45 spreader with less downtime and lower costs as a result.

The problems that the authors want to present solutions for in this thesis consists of: (A) increasing the model reliability by enhancing the dynamics in the model in a suitable way for Bromma Conquip, and (B) propose an optimal way of automating the test process of the embedded software. These two problems are focused on Model Based Systems Engineering practices and together they open up a third question: (C) how other departments of Bromma Conquip can gain benefit from the results of (A) and (B) in a transition towards full scale use of Model Based Systems Engineering (MBSE).

Increased complexity calls for increasing system and subsystem testing to ensure safe operation in all situations. A more detailed simulation model which has been verified and validated is needed for more specific test cases to be implemented into the test process. This problem relates to Research Question A & B presented in section 1.3

Test automation is one possible way to improve the feasibility of regression testing. Such a solution is effective when it can both save time and perform just as good as the human tester or so called Human Oracle does in the current test process [6]. This problem relates to Research Question B presented in section 1.3.

(11)

To investigate the possibility to expand the use of model-based thinking throughout the development process at Bromma the eventual benefit of the findings regarding model complexity and SW test automation will be analysed. This problem relates to Research Question C presented in section 1.3.

1.3 Research Question

In this Master Thesis report we have made an attempt to answer the following research questions:

A) Which level of dynamics (complexity) are suitable given the requirements from all parts of the life-cycle (testing, production and deployment)?

B) Which best practice test automation is most important to the personal affected (developers, testers, management) by the results of the testing? How much can automation reduce the frequency of software related faults of the system and how is this best achieved given a system engineering perspective of the product?

C) Could other departments at Bromma Conquip, e.g. mechanical engineering, benefit from access to the model and the related automation? What are the possible limitations that prohibits this, and how can they be overcome?

1.4 Limitations

There were a number of limitations to the thesis work. Some were the consequence of conscious decisions to narrow the scope to a suitable level while others were limiting factors to the ability to answer the research questions. The limitations follows:

• The telescoping function of the STS45 spreader was selected as the case for the modelling and simulation.

• The intended life-cycle of the Simulation Model are only used in the functional embedded systems testing, meaning that no requirements from other parts of the life-cycle of the product existed.

• The design of the STS45 spreader or any equivalent product does not have many sensors, there is only a single encoder. This made the availability of experimental data very scarce.

• Since the type of modelling and simulation described in this thesis was not previously done by Bromma Conquip, the experimental data required to verify the Simulation Model was not available initially. Special experiments had to be performed to get the data, the experiments were delayed which impeded the modelling process.

• While the thesis includes studies of the industry, the results only directly concern Bromma Conquip and their products.

• The implementation of Test Automation desired by Bromma Conquip only concerned the automation of the Functional Testing. The test cases and methods from the manual process would be reused.

(12)

2 Methodology

This chapter described the methodology used to answer the research questions of the thesis.

The general research methodology is described in Section 2.1 and the methodology used for the literature survey and interview is described in Sections 2.2 and 2.3 respectively.

2.1 General research methodology

The purpose of this section is to provide motivation for the decisions made regarding the an- swering of the research questions described in section 1.3.

The purpose of this thesis is to present findings related to the research questions and in order to achieve this goal a general methodology was defined. The general methodology that is presented in figure 2 was inspired by the Micro-Level cycle described in the VDI 2206 Guidelines [7].

Figure 2: Visualization of the general process of the thesis work.

• Situation analysis: Problem description and definition.

• Target definition: Stakeholder requirements elicitation and definition of research questions.

• Synthesis/analysis cycle: Literature survey, requirements elicitation, design, implementation, testing and/or analysis.

• Evaluation: Validation and summary of results and the usefulness of the same in relevant context.

• Future work: Suggestions for which projects should be pursued next and learnings from the current project which will be useful in the upcoming projects.

For the three research questions the Synthesis/analysis cycle and Evaluation have been performed with different methods for each research question. These methodologies are described in detail in Sections 4.1 and 5.1. The execution of the remaining process steps have been used for all the research questions.

Figure 3: Visualization of relations between the research questions, MBD, MBSE and SE.

(13)

A common theme for the three research questions is their importance in relation to the broad areas Model-Based Systems Engineering (MBSE) and Systems Engineering (SE). The relation between these areas and the research questions are visualized in Figure 3.

To answer the research questions the concepts of MBSE and SE must also be studied to provide sufficient knowledge to support the findings in the right context.

2.2 Literature survey

This section describes the literature survey undertaken to find background information used to answer the research questions stated in Section 1.3. The purpose is to describe how the survey was undertaken and to describe and motivate the chosen methodology.

2.2.1 Survey methodology

The literature survey consisted of three parts carried out in sequence. The surveys were iterative, with subsequent surveys being dependent on the result of the previous. The surveys were the following:

1. Initial survey, a wide preparatory search with the task of finding resources to use for the following surveys. The resources consisted of databases, journals, books and websites etc.

2. Exploratory survey, a wide search with the task of finding existing academic research related to the research questions of the thesis. Related background topics should also be identified for subsequent surveys.

3. Focused survey, a narrow search to find information on specific topics identified by the previous surveys.

How the initial, exploratory and focused survey was conducted and their findings will be described in detail in Section 2.2.2, 2.2.3 and 2.2.4 respectively.

To rank the findings of the surveys the following metric were used:

1. The level of relevance to the research question.

2. The year of publication.

3. The cite number of the publication.

The first criteria was mostly based on subjective judgement of how useful the article or papers appeared to be. This was decided by reading the abstract to see if further reading was needed to determine its usefulness. The second criteria was needed to ensure that the result of the publications were still valid. Older publications are more likely to have either been disproved by or replaced by more recent research. The third criteria was less important but still kept in mind to ensure that the publication was valid.

2.2.2 Initial survey

The areas of research for the thesis were modelling and simulation of dynamics systems, automated software testing and model-based development practices. Inspec, IEEE Xplore, Primo and Google Scholar was selected as resources because they all covered the relevant research domains.

Inspec is a bibliographic research database with a extensive coverage of science and engineering domains, published by the Institution of Engineering and technology (IET). As such it contains a very large amount of papers and articles. It also features a thesaurus which is useful tool for expanding searches. Its main issue is that it is bibliographic, meaning that not all publications is available. This makes it more likely to find more relevant results per search but that they may not be available.

IEEE Xplore is another research database with a focus on computer science, electrical engineering and electronics, published by the Institute of Electrical and Electronics Engineers (IEEE).

Being a full-text database it provides full access to all its papers and articles at the cost of having

(14)

a smaller library than Inspec. While this means that it may give fewer relevant results per search, they will be more useful. IEEE Xplore together with Inspec as the main search resources.

Primo is the search service of the KTH Library. It provides access to the publications, including books, available in the library in electronic form. The database is bibliographic with a broad scope. The broader scope and the access to books in electronic from sets Primo apart from Inspec and IEEE Xplore. Google Scholar is free access bibliographic databased published by Google. Primo and Google Scholar was used as complementary resources to Inspec and IEEE Xplore.

Best practice as defined by industry and academia for the research topics of the thesis would be used as a measure in the analysis. To help define the best practices industrial standards for the relevant topics were used. The standards were access via e-nav, the on line library for standards published by the Swedish Standards Institute (SIS).

2.2.3 Exploratory survey

The search topics were directly related to the technologies associated with the research questions.

The topics were: modelling and simulation for cargo container spreaders for research question A, automated software testing for research question B and Model-Based Systems Engineering for research question C.

The survey into modelling and simulation of spreaders revealed that prior research was lim- ited. The existing work mainly considered the environment surrounding the spreader, the interactions between crane and spreader or the spatial dynamics of the spreader. No relevant work concerning the modelling and simulation of the inner workings of spreaders was found. This means that either no research had been done in this area or the survey was flawed in some way.

Whilst possible it seems unlikely that the initial literature review was faulty, too narrow, used the wrong search terms or used the wrong databases. If no relevant publications was found in these databases that covers the relevant domains, it seems unlikely that they exist. The lack of research in the area is corroborated by the fact that Bromma Conquip, the technologically leading spreader manufacturer, does not use or have models of the inner workings of their products.

Based on this result, the scope of the subsequent survey was expanded to look at systems with similar characteristics. These characteristics included being hydraulically actuated, electrically controlled and having power transmissions with gearboxes and chain-drives. The purpose was to find models that could form an applicable basis for the Simulation Model.

The survey into automated software testing proved that the topic had been the subject of extensive research. Because of the large number of articles and papers related to the topic, further searches was needed to narrow down the results. It also became clear that automated testing could either refer to the automation of the test procedure or to the automatic generation of test cases based on the code to be tested, or sometimes both combined. Since the former is relevant to the research question, it was identified as the focus for subsequent surveys. Systems Engineering and software testing was also identified as topics of interest.

The survey into Model-Based Systems Engineering showed it to be a topic of ongoing research and as such lacking an agreed upon definition. Because of this subsequent surveys was needed to find a definition that suited the thesis. Model-Based Development was identified as a topic for further survey because it linked the three technologies of the thesis together.

2.2.4 Focused survey

The focused survey consisted of carrying out specific searches into the topics that had been identified as relevant by the previous searches. The goal was to find useful references for the thesis. The findings of the focused survey provided the background information for the Frame- of-Reference.

The search for work related to the systems modelling which concern hydraulically actuated, electrically controlled and mechanical systems. The mechanical systems focused on systems including power transmissions with gearboxes and chain-drives in their design. This provided several useful findings. These findings is the basis for the design of the Simulation Model described in Section 4. The search for work related to practices for Test Automation also provided

(15)

several useful findings. Together with the findings related to Systems Engineering and software testing, these findings served as basis for investigation into the relevant best practices for Test Automation presented in Section 5. Several useful standards was also found.

A statistical summary of the focused survey results are presented in Figure 4. As can be seen in the figure there is a fairly even balance between the quantity of references for each topics.

The vast majority of the references were also recent works being published after the year 2000.

The origin of the references, the type of publication, was also considered. The majority of the references were academic publications from journals and conferences.

Figure 4: Summary of the focused survey in terms of Topic, Year of publication and Origin of the references.

2.2.5 Literature criticism

The publications that were used in the frame-of-reference fares well over all. The majority have been published during the last five years, reducing the likelihood of them being aged. There is also little dependency between the sources, coming from different publications, conferences and authors. They should be authentic, having been published which should imply that they have passed some kind of review. While not being a guarantee it should at least reduce the likelihood. The sources does not seem to be tendentious, except those published employees of MathWorks, dSpace and other companies. While it does not seem to be a problem in this case, it is important to consider that they are a modelling tool vendors or exist in the industry in some manner. This implies that they may have a vested interest in how practices related to their products are presented.

2.3 Interview methodology

Interviews were conducted to acquire information that was not available in documented form for this thesis. Since the culture of Bromma Conquip were largely informal this were an important source for information. The primary goal of the interviews were to learn the details of how the system to be modelled was built and functioned, which its most important dynamic characteristics were and how the product development process was structured. This information were

(16)

needed to answer the research questions. The secondary goal was to learn about the culture at Bromma Conquip, how they worked and were organized, their strengths and weaknesses and the problems they were facing. This information was not strictly needed to answer the research question, but would rather serve to provide context to the answers based on industry practices.

The interview were largely informal and exploratory in nature. They were conducted with product development personnel and management at the R&D department at the office in Akalla, Stockholm continuously during the work process. While lacking a formal structure in the form of a questionnaire, the questions were adapted to suit the current information need. This provided flexibility and made the process more exploratory since it was not constrained by a fixed format that a fixed set of questions would imply.

The interviews were structured so that they was started with a specific question that had been formulated beforehand. The questions originated from two sources. Either in a need for information about a certain topic, such as who was supposed use the Simulation Model, how it would fit into the current development process or which was its most important function. They could also originate for a need to know how a specific topic defined by the literature survey was related to the company and their business.

To summarize, the interviews were iterative and the process began with interviews based on open ended questions, proceeding to interviews with structured questions based on an information need and finally finished with reviews to verify the previously collected information.

The goal of gathering information regarding industry practices warrants a comment. Since only a single company was part of the study it is difficult to draw general conclusion from the results with regards to the whole industry. This is not to say that it provides valuable insight, but that limitation should be kept in mind.

To obtain further information from the stakeholders the interviews were complemented with design reviews. The purpose of the activity was receive feedback during the development process.

This was an important source of information due to the initial lack of data from the system to be modelled, which meant that the system developers was the only reference. The reviews consisted of a short presentation of the current work progress, status and results for the stakeholders at the company and allow them to provide feedback.

(17)

3 Frame-of-reference

In this section the background knowledge about the topics relevant to this thesis is presented.

The purpose is to define how the topics is interpreted in the context of the thesis. The section has a top-down approach and starts with a review of Systems Engineering in Section 3.1, followed by Model-Based Systems Engineering in Section 3.2 and a description of the system to be modelled in Section 3.3. The section ends with a review of the development and testing process used by Bromma Conquip in Section 3.4.

3.1 Systems Engineering

Systems Engineering is an interdisciplinary field of engineering that focus on the successful design, management and realization of complex engineering systems. Its purpose is to develop systems with high quality in their entire life cycles [8].

A definition of a system is provided by [9] as following: ”A combination of interacting elements organized to achieve one or more stated purposes”. These system elements can themselves be considered as systems , creating a system of systems. An important implication of the systems of systems concept is that it is possible to break any system, regardless of its complexity, into smaller subsets and thus reducing the level of complexity. Another central concept is that of emergent properties. This refers to the fact that a system often is greater than the sum of its components. In this case the functionality of the system is dependent on the interactions between its components to achieve it.

The concept of the system life cycle is a key concept within Systems Engineering. The system life cycle is defined by ISO/IEC/IEEE 15288 as a series of sequential processes through which the system progresses during its lifespan [9].

Figure 5: System life cycle processes.

(18)

These processes range from the first conception of the system based on a stakeholder need, through the planning and development stages on to the eventual decommissioning of the system.

The full list of life cycle processes defined by the standard can be seen in Figure 5. Although most of the traditional engineering work is carried out during the technical stage, the importance of the organizational and managerial stages should not be overlooked as they play an important role in creating the conditions for their success.

Since the life cycle of a complex system consists of a wide range of different processes it is critical to have holistic perspective to be able to successfully manage its development. Hence the holistic view of a system is perhaps the most important concept of systems engineering. This way of thinking is commonly referred to as system thinking.

It is not uncommon for the system functionality to depend on synergy that stem from the successful integration of technologies from different technical fields. Because of this engineers and specialists from these fields need to cooperate closely, making Systems Engineering an interdisciplinary field. Since each field usually has their own tools and methods it may be difficult for engineers with different domains to work together effectively. This combined with interdisciplinary design with different stakeholders and views makes the development process more complex.

In response to this growing complexity several different development process models has been developed to facilitate the development of engineering systems. One process model used in the industry is the V-model, whose general structure can be seen in Figure 6.

Figure 6: A conceptual overview of the V-model cycle.

The starting point of the general process is a requirement analysis for the system being developed for its stakeholders. The requirements are the input to the process, define the goal of the process and used to validate the final result. During the system design phase the overall system functionality is laid out according to the requirements. The design starts from a general system level, usually with a high level of abstraction, to more detailed and specific as it progresses through the design phase for the subsystems and modules of the system, from top- down perspective. At the lowest level the design is split into the domain specific parts and implemented.

When the design phase is completed, the system integration phase begins. The integration is executed from a bottom-up perspective so that the modules are integrated into subsystems which in turn are integrated into the system. The result of the integration is continuously checked against the corresponding design for verification and validation which is needed in order to confirm that the system properties can satisfy the requirements from the stakeholders.

A common way to use the V-model is to break down the full development process of a system into several cycles, where the outcome of each cycle is a system of increasing level of maturity and complexity. For example, the goal of the first cycle can be to produce a prototype system which serves as a proof-of-concept. The following cycles will bring the system closer to a production version which is the ultimate end goal of the macro process.

(19)

3.1.1 Requirements Engineering

Requirements Engineering is an engineering discipline that focus on the elicitation and management of the requirements placed upon a system by its stakeholders. As such it is an important subset of Systems Engineering. An introduction to the field is given by [10, Chapter 1]. The stakeholders can be the customer that will use the system such as company staff, governmental bodies or anyone else who has a relevant interest in the system. Because of this the perceived success of the system depends how well it matches the expectations and desires of the stakeholders.

The requirements life cycle can be compared to the process of the V-model, described in Section 3.1, where the top level with high abstraction and low detail levels is said to exist in the problem domain. As the process progresses to the lower levels the requirements become less abstract and more detailed as they move into the solution domain. Thus one of the challenges of Requirements Engineering process is to transform the needs of the stakeholders into formal specifications or a more detailed technical system specification depending on system level. The requirements at a certain level is used to validate the corresponding level of the implementation against its design; for example the implementation of a system is checked against the system level requirements. An example of the multiple requirement level approach can be seen in Figure 7.

Figure 7: Requirements in the V-model [10, Chapter 1].

With this approach the concept of traceability becomes relevant, which means that it should be possible to trace each formal specification at the lowest level back to a top level requirement.

The benefit of this is two-fold. First, it ensures that each formal specification contributes to the fulfilment of the stakeholder requirements, thus reducing waste; and secondly it makes it possible to follow the design to prove its conformance to a standard.

Requirements can be divided into two categories: functional and non-functional requirements. The functional requirements describes the system behaviour of the system while the non-functional requirements describes the performance of the desired behaviour and how it is achieved.

3.1.2 Engineering standards and references

The International Organization for Standardization, ISO, define a standard as a document that provides requirements, specifications, guidelines or characteristics that can be used consistently to ensure that materials, products, processes and services are fit for their purpose [11].

As stated by the definition above, standards is a useful tool to ensure that a specified level of quality is achieved and maintained over a system’s life cycle. While claiming conformance to a standard can be seen as a general mark of quality for a system, their use convey other benefits as well [12]. These benefits are both economic and social gains that arise from simplified

(20)

cooperation and trade at organizational, national and global levels. As such standards play an important role in the modern society.

The standards and engineering references used in the thesis along with a short description of their use can be found below. The standards are the following:

• ISO/IEEE/IEC 15288 Systems and software engineering - System life cycle processes [9], defines a set of processes to facilitate communication among acquirers, suppliers and other stakeholders in the life cycle of a system.

• ISO/IEEE/IEC 26262 Road Vehicles - Functional Safety [13], defines processes and methods for the development of safety critical software for automotive applications.

• RTCA DO-178C Software Considerations in Airborne Systems and Equipment Certifica- tion, provides guidance for the software development of safety critical aviation systems.

• RTCA DO-330 Software Tool Qualification Considerations [14], provides guidance for tool qualification for use with safety critical system. A supplement to RTCA Do-178C.

• SWEBOK V3.0 Guide to the Software Engineering Body of Knowledge [15], describes, organizes and provides topical access to the generally accepted portion of the software engineering Body of Knowledge.

3.2 Model-Based Systems Engineering

Model-Based Systems Engineering (MBSE) is an approach that provides a framework based on computer tools to facilitate the Systems Engineering process [16]. According to [17], the framework is based on the two fundamental notions:

1. raising the level of abstraction by raising the level of specifications further away from the technological implementation, and

2. raising the degree of computer-based automation to bridge the gap between design specification and implementation.

In an MBSE process these notions are realized with the aid of computer tools which are used to create models. In this context, a model denotes a specification expressed with a higher-level formalism. This can either imply an abstraction that omits details or a complete implementation specification from which code can be automatically generated. Different types of models are used at different stages in the process, in the early stages models with high abstraction is common while during the later stages more detailed models become predominant. The models makes it possible to raise the level of abstraction while code generators and test automation tools raise the degree of computer-based automation in the development process.

The development is centred around a system model that is the heart of an executable specification which is used and elaborated throughout the process [18]. It starts from a model of the requirement specification and progresses through the design, implementation and testing. One important role of the models is to replace documentation, which means that all the information regarding the system should be contained in the models.

Two important concepts of MBSE are the tool-chain and tool-integration. The tool-chain refers to the sequence of software tools that are used in the development process, it defines their behaviour and interaction. Tool-integration refers to the process of integrating the software tools in the tool-chain into a unified tool. This reduces the time needed to transfer and convert models between tools which increase the efficiency of the design and reduce the risk of faults. This also provides a way to the automation of the development process.

A framework for MBSE is presented by [19] which is referred to as the SPIT-framework, an overview of it can be seen in Table 1. The framework consists of several layers that specify models which are used for different purposes. The social layer contains the models which defines things such as role assignment and communication structure for the development organization.

The process layer defines the process model used for the system development, such as the V- model or the waterfall-model. The information layer defines how information is managed in the

(21)

development process, how it is stored and distributed. The technical layer defines the domain- specific models used for the design of the subsystems, these models consists of simulation models used to analyse the design.

Table 1: Overview of the SPIT-framework

Layer Models Concern

Social

Behavioural, organizational, managerial and cultural views to describe engineering activities throughout the whole life cycle.

People and organisation.

Process

Business process for product design and development, based on industrial standards.

Design process.

Information Information about the

requirements and system design.

Requirements, features, functions, tool configuration,

verification & validation, data linking, structure model and interface design.

Technical Domain specific design

of the subsystems. Simulation and analysis.

3.2.1 Model-Based Development

The increasing occurrence of embedded computers has made the increasing amount of functionality in modern products. This has made the development process of embedded systems increasingly complex and challenging as this trend continues, [20]. Model-Based Development is an approach to the development of embedded systems relies on models to meet these new challenges. Models can be defined as explicit and operational descriptions of relevant entities that occur during the development. The models give the developers the ability to obtain information about the behaviour of a system before it is completed. The information can be used to evaluate the feasibility of solutions before they are implemented as without physical prototypes [21].

Different models are used depending on the design task. The models role has been extended from traditional purposes like verification and validation to information description. Such kinds of models can connect the requirements to the mathematical descriptions of system behaviour. The examples of types range from simple flowcharts that map user behaviour to complex mathematical models that represents dynamic behaviour. A specified example of model usage in embedded systems development is to use a model to represent the physical system under control while designing embedded control systems. Such a model is commonly referred to as a plant model.

The plant model typically consists of a mathematical representation of the physical system. This way, both the open- and close-loop behaviour of the control system can be evaluated without needing physical hardware, allowing quick design iterations. The conceptual layout of the closed- loop system can be seen in Figure 8, featuring the controller, sensor and plant model. This design process is referred to as Rapid Control Prototyping.

Figure 8: Conceptual view of the control system with plant model

The models can generally be divided into process and product models [22]. The process models describe the development activities. Being explicit descriptions, the activities are re- peatable, undo able and traceable. This means that it is possible obtain insight into what the

(22)

development activities actually accomplished, allowing for easier reviews of the process. The activities consists of both low- and high-level tasks, such as code implementation and abstract allocation of functions on the hardware platform. The product models consists of the entities that describe the artefact under development and the necessary parts of its environment, as well as the relationships between these entities. To relate the product models to the process models, all the activities in the process models are defined in terms of the entities in the product models.

Two important concepts to Model-Based Development are abstraction and code generation, which are related to each other. Abstraction is provided by the models when system details are disregarded, either by choice or necessity. The level of abstraction in a model depends on the stages of the development. Early in the process, during the system design stage, the level of abstraction in the models is high, meaning a lot of details is omitted. As the process proceeds the level of abstraction is decreased as more detail is added to the models.

Code generation refers to the practice of automatically generating code based on the models representing the system, rather than writing the code by hand. This practice takes advantage of the abstraction offered by the models to allow the developers to focus on the system design rather than writing code by hand.

3.2.2 Modelling & Simulation

Modelling and simulation are important in the embedded systems development used for verification in the early phases. The model is used to represent the physical system under control of the embedded system. This makes it possible to simulate the system and verify it before developing the physical prototype. An introductory overview of modelling and simulation in this context is given by [23].

Modelling is defined as the process of developing a model; a model is defined as the representation of the construction and workings of the system being modelled. The purpose of a system model is to provide the developer with the ability to predict the effects of changes to the system. While a model should be a close approximation of the real system, which means it should include as many of its features as possible. It should not be so complex that it is impossible to understand and work with. This means that a good model has a balance between realism and simplicity.

There are several different approaches to modelling which results in models with different structures. Which approach that is suitable depends on the purpose of the model and how much prior information that is available. These model structures are commonly colour coded in shades of grey [24]. A black-box model only models the input-output behaviour of a system but provides no insight of its inner workings. This approach is necessary when little or no prior knowledge is available. This approach may also be necessary if the system is very complex. When some prior knowledge is available but with unknown parameters a grey-box model is suitable. The grey-box structure can be based on physical grounds or be semi-physical, which means suggesting non-linear combinations of measured data treated as a black-box model. If all the needed prior knowledge is available then the model is a white-box model, providing full physical insight into the system. Which modelling approach that is suitable for a given task also depends of the effort required to develop the model. Developing a white-box model can be an arduous process which may not be feasible if the system is too complex. In this case it may be necessary to construct a black- or grey-box model.

Simulation is defined as the operation of a model. A simulation model is often built manually and the theoretical knowledge of the developer is used to verify and validate its performance.

The system behaviour can be predicted by the simulation model based on the simulation result.

Different configurations of the simulation model can help the engineers to optimize system properties and find the best solution for the system. Because of this, simulation is an important tool for the testing of embedded systems.

If the model will be used for simulation it needs to be implemented in a modelling and simulation environment. Examples of such environment are MATLAB and Simulink from MathWorks and LabVIEW from National Instruments, both offering several ways to implement mathematical models. The models can either be built as block diagrams, transfer functions or as state-space representations.

(23)

3.2.3 Embedded Systems Testing

To ensure that the embedded system and its components work as they were expected, they have to be verified and validated in line with the chosen process, such as in the V-model described in Section 3.1. This is achieved through test activities in the different steps of the development process. Verification and validation are two activities which both individually are critical to any successful Model-Based Development process [18]. Verification is defined by [15] as the set of activities which ensure that a product meets its specifications whilst validation is described as activities which ensure the fulfilment of stakeholders expectations and intended purposes for the product. To reduce ambiguity formal requirements are defined based on stakeholder needs and then used in verification and validation activities to ensure that the intended system was created and that it works in the desired way. There are a large number of different approaches to the design of test cases and methods for executing tests. Testing of embedded systems include both software testing and testing of how the software control a specific set of hardware components.

Software testing is a very broad area in the software domain with many different parts and aspects such as test case design, oracle design, test evaluation and test platforms. Software testing is like other types of testing critical to the development process e.g. because improper test case design or implementation can result in bugs which if left unattended cause errors in the resulting software, which in turn can lead to delays, failures and large costs. In its broadest sense testing can be defined as an activity where a system is stimulated with a stimuli and the system response is observed [6].

The ISO Technical Report 19759:2015 [15], which can be seen as a best practice for software engineering, defines software testing more explicitly as:

Software testing consists of the dynamic verification that a program provides expected behaviours on a finite set of test cases, suitably selected from the usually infinite execution domain.

ISO Technical Report 19759:2015, Chapter 4.

The challenges highlighted by this definition are to decide what is the objective of testing, what comprises a desired test result and which test cases that will be able to provide this information. The objective of testing is very individual but commonly testing is done to verify that functionality of the software is in line with existing specifications or to validate whether the software fulfils the expectations of the stakeholders. Requirements on the system under test become the base for the expected results of testing. Desired results fulfil functional or safety requirements in a clear way which should make it an easy task to evaluate the success of the test at hand. Sadly enough this is not always the case. The choice of test method depends on the type, level and purpose of the test and the choice can be a combination of two or more methods.

Methods are often based on either knowledge and experience about the system, the proposed usage of the system, the program code or the nature of the system.

In embedded systems testing the definition provided by ISO Technical Report 19759:2015 can be translated into that the objective of testing is to verify that the system (model, software, processor or hardware) fulfils functional and safety requirements. Depending on what behaviour the tests are supposed to investigate they can be defined as so called black box tests where the input and output behaviour of the system are evaluated or as a glass box test where the archi- tectural design of the test level is of main interest. The desired results of such tests are derived from stakeholder requirements into formal requirements, work flow models or state machines[15].

3.2.4 Model-Based Testing

The term Model-Based Testing is commonly used to describe all test activities in a Model-Based Development process. Redesign during the initial design phase incur a much lower cost than during later phases. This can be avoided by means modelling and simulation activities which make it possible to test the system before it have been physically implemented and thus discover design flaws earlier.

A methodology for Model-Based Testing of embedded systems is given by [25] that consists of so-called in-the-loop tests. This test methodology defines tests at different integration levels

(24)

to verify and validate the design and implementation of the embedded system. The integration levels of the tests are [26]:

Model-in-the-loop (MiL): The model is executed within a simulation environment without the target hardware. The test is performed on a Windows- or Linux-based desktop machine.

The purpose of this step is to verify and validate the model.

Software-in-the-loop (SiL): The code of the embedded software is compiled and executed within a simulated environment, without the target hardware. The test is performed on a Windows- or Linux-based desktop machine. The purpose of this level is to verify and validate the code.

Processor-in-the-loop (PiL): The code of the embedded software is compiled and down- loaded to the target hardware platform (ECU) with the target processor or a target processor emulator. The code is executed on a ECU that connected to a Windows- or Linux-based desktop machine via. Communication is done via the host-to-target communication of the Integrated Development Environment. The purpose of this step is to verify and validate that the code is compatible with the target compiler and the target hardware architecture.

Hardware-in-the-loop (HiL): The code of the embedded software is compiled and executed on the target hardware platform (ECU), which is interacting with a simulated environment via its digital and analogue electrical connectors. The purpose of the step is to test the communication functions and real-time behaviour of the embedded system.

The purpose of the embedded system is to interact with and control some kind of physical system. This physical system often has complex, usually non-linear, dynamics that needs to be taken into account during design and testing activities. With an accurate model it is possible to design test cases that takes the dynamics of the physical system into account. This makes it possible to perform tests such as in-the-loop tests, before the system has been built.

3.2.5 Test Automation

Test Automation refers to the practice of introducing automation into the test process. Its main characteristic is that in contrast to manual testing, at least part of the test process is performed by a machine, without human interaction. According to [6] there are no fully automated test methods available today, instead the processes are automated with more or less human interaction.

Automated testing is a subject which have become more and more popular, in both the software and hardware domains, during the last years. In line with the emerging technologies and solutions found in the SE domain test automation is an activity which greatly can improve a development process, if implemented correctly. The objective of test automation is commonly to make testing more effective in terms of either time, money or quality by either doing the tests faster or avoid repeating tests manually[6] [27]. However it is not always clear how these objectives are best achieved, in other words how the automation of testing is best implemented in each situation.

It should also be noted that Test Automation does not necessarily have the objective to take away the jobs of testers. As described by Barr et al.:

The point of automation is not to eliminate testers but to make better use of their time [6].

In addition to the definitions of testing adopted in section 3.2.3 the more specific scope of automated software and embedded test automation are given the description presented by [28]:

Software test automation represents the use of software to control the execution of tests, the comparison of actual outcomes to predicted outcomes, the setting up of test pre-conditions, and other test control and test reporting functions [28].

(25)

Automated tests must have a different set of requirements for their design of the corresponding test suite and evaluation criteria applied by the test oracle. There is also need for a dedicated software tool that can perform the automated testing in collaboration with the human tester whose efforts the automation aims to reduce. All these changes propagate into that changes in management of testing might be required to ideally withhold the reliability of the test process at hand.

Amaricai and Constantinescu [28] continues to suggest that a test automation framework should be used to structure the automated testing. They define such a framework with the loaned description:

An automated test framework may be loosely defined as a set of abstract concepts, processes, procedures and environment in which automated tests will be designed, created and implemented. In addition, it includes the physical structures used for test creation and implementation, as well as the logical interactions among those components [28].

In other words such a framework represent the highest level of abstraction for automated software testing where a test can be executed just the push of a button [29]. More specifically the framework can consist of the formal requirements used to define test cases, the test suites which consist of test cases, the test oracle which evaluate the test results, the automated testing tool chain which execute the tests and create test reports, the physical environment consisting of hardware on which the tool chain can run and embedded components if the tests are PiL or HiL based. According to [29] there are three generations of frameworks which all can be either data-driven, key-word driven or hybrid which is a combination of the two previous approaches.

The implementation consists of a software tool which performs the tests automatically, a test suite which defines the set of test cases, an oracle which interprets the results of the test cases.

A more detailed description of such a framework is presented in section 5.

3.3 Bromma STS45 telescoping spreader system description

This section describes the STS45 spreader system produced by BC. Emphasis is on the telescoping functionality of the system as it is the focus of the thesis.

The purpose of the STS45 system is to lift containers safely and reliably. The spreader system consists of a physical part and an embedded part. The physical system includes the electric system, hydraulic system and mechanical system and the embedded system includes the software, CAN and control system.

An overview of the STS45 Bromma spreader can be seen in Figure 9.

Figure 9: Overview of the STS45 Bromma Spreader; 1) Twistlocks, 2) Flippers, 3) Telescopic Beam Units, 4) Retractable Twistlock Boxes (for Twin Separation), 5) Powerpack, 6) Hydraulic Pump, 7) Steel Main Frame. Source: Bromma Conquip - Sales and Marketing

(26)

The three main functions of the STS45 system are Telescoping,Twistlocks and Flipper Arms.

The purpose of telescoping is to position the Telescopic Beam Units (TBU) at a desired distance from each other in order to fit a certain container sizes: 20, 40, 45 feet[30] or 2x20 feet [31].

Twistlocks are cylinders placed on the underside of the spreader in the end of the TBU. Twistlocks are fitted into holes on the roof of a container and turned to interlock spreader and container in order to enable safe lifting. Flipper Arms are used to help the operator of the crane to guide the spreader to the container properly so that the Twistlocks find their holes.

One of the main goals of this thesis is to investigate and model the TBU of the STS45 Bromma Spreader and how it is actuated. This part will therefore be the focus of the physical system description.

3.3.1 Physical System

The main purpose of the physical system is to provide and transfer power to the different mechanical functions of the spreader system at any time. This is realized by the electric, hydraulic and mechanical subsystems. In Figure 10, telescoping functionality of the physical system is shown.

Figure 10: The telescoping functionality of the STS45 Spreader

The electrical system is powered by a 400V AC power supply on the cargo crane. An induction motor with a rating of 7.5 kW provides power to the hydraulic system.

The hydraulic system is a valve-controlled which means that in order to actuate linear and rotary actuators, valves are opened and closed to control the hydraulic power [32, Chapter 2.3].

The hydraulic system get power from the induction motor which cause a variable displacement pump to transfer the power from torque and angular acceleration into pressure and flow. For the Telescoping there is a directional proportional valve, controlled by the embedded system, which supplies the power to the hydraulic motor. The hydraulic motor provides the torque that actuates the mechanical system. Hydraulic oil is not only the medium transferring the power in a hydraulic system, but also lubricates and helps cool the system and its components.

The output shaft of the hydraulic motor is connected to a gearbox with 1 gear ratio to provide torque able to move the heavy mechanical load. To synchronize the positioning of the TBUs a Chain Drive System with an endless chain, depicted in Figure 11, transfers the motion from rotational to linear.

(27)

Figure 11: Detailed view of the Chain Drive System. 1) Driving wheel; 2) Encoder wheel; 3) Endless chain; 4) Attachments between chain and tension rod; 5) Tension rod; 6) Gearbox.

The two TBUs account for about half of the spreaders total weight, 2500 kg each, which means the friction has a noticeable effect on their movement. Each TBU glides on two glide plates and are supported against the upper part of the structure by rollers as described by Figure 12.

Figure 12: Side view of a Telescopic Beam Unit

3.3.2 Embedded System

The embedded system of STS45 controls the spreader and provides functionality and monitoring for the crane operator and service personnel. The control system of the STS45 is called SCS4 (Spreader Control System 4) running on a PLC (Programmable Logic Controller) system. The SCS4s purpose is to control the spreader. This is realized through a distributed network of ECUs which communicates using the CANOpen protocol, defined by the OSI-model [33], on a Controller Area Network (CAN). Figure 13 shows an overview of the embedded system architecture.

The SCS4 controls the spreader by manipulating the directional valves of the hydraulic system based on encoder data sent via CAN.

(28)

Figure 13: Overview of the Embedded System

3.4 The development process used by Bromma Conquip

The Development Process at Bromma Conquip begin with a selection process before a new project is started. This selection process is described in Figure 14. The R&D department has a backlog of projects ideas that are possible candidates for further development. Each year the backlog is reviewed to select the project that are to be performed during the year. The first selection is made by a designated Product Owner with market knowledge who selects a number of projects that is deemed to be the most beneficial and contributes to the overall business strategy. The final selection is done in another review which decides whether projects fits the time frame and budget. These are the projects that the R&D department will work on until the following year [34].

Figure 14: Overview of the Project Selection Process

The Development Process is similar to a general stage-gate process [35]. The development process is divided into a number of phases in which the various development activities, from conceptualization to product launch, are conducted sequentially. An overview the process layout can be seen in Figure 15. Each new phase is started by a review to decide if the results of the completed phase fulfil the requirements. If the requirements are fulfilled then the process proceeds to the next phase and if not it will either return to the previous phase to revise the faults or the project will be cancelled, [34].

The effects of test process automation on fault frequency for a spreader system

Sammanfattning

Abstract

Foreword

Acknowledgements

Contents

List of abbreviations

1 Introduction

1.1 Background

1.2 Problem description

1.3 Research Question

1.4 Limitations

2 Methodology

2.1 General research methodology

2.2 Literature survey

2.3 Interview methodology

3 Frame-of-reference

3.1 Systems Engineering

3.2 Model-Based Systems Engineering

3.3 Bromma STS45 telescoping spreader system description

3.4 The development process used by Bromma Conquip