TRACEABILITY IN MODEL BASED TESTING

by

Beaulah Vineela Pasupulati

A THESIS

Submitted in partial fulfillment of the requirements for the degree

MASTER OF SOFTWARE ENGINEERING

School of Innovation, Design and Engineering

Mälardalen University

Västerås, SWEDEN

June 2009

Supervisor

Antonio Cicchetti,

Ph.D in Computer Science,

Copyright

By

TRACEABILITY IN MODEL BASED TESTING

APPROVED BY

SUPERVISING COMMITTEE

Supervisor: Antonio Cicchetti

ABSTRACT

SAMMANFATTNING

Från början dags för modellering spelat en avgörande roll inom mjukvaruutveckling, i synnerhet för att i detalj förstå det system som håller på att utvecklas. På grund av vikten och nyttan av modellering, Model Driven Development är kraftigt växande i den senaste tiden. Utnyttja fördelarna med modellen tillvägagångssätt i utformning och genomförande faser av

mjukvaruutveckling livscykel är det att utvidgas till andra faser av mjukvaruutveckling också. I test så väl, vilket är en viktig fas i utvecklingen, för att leverera kvalitet mjukvara till kunder, detta koncept av modeller och modellering tekniker håller på att verkställas. Användning av moduleringstekniker i mjukvara testning har gjort tester effektivare. Modell baserad Testing är en black-box testning strategi där testfall genereras automatiskt från beteendevetenskaplig modell av systemet underenheten. För att känna till betydelsen av dessa automatiskt genererade testfall på systemet underenheten, spårbarhet av dessa testfall till krav är viktig faktor. Underhålla spårbarhet med testfall skulle underlätta mjukvaruteknik verksamheter som utför

TABLE OF CONTENTS

ABSTRACT...iv

SAMMANFATTNING... v

TABLE OF CONTENTS...i

LIST OF FIGURES ...iii

LIST OF TABLES...iv

ACKNOWLEDGEMENTS... v

1 Introduction... 1

1.1 Importance of Models and Modeling...1

1.2 Background... 2

1.3 Scope and Objectives... 3

1.4 Document Overview... 3

2 Literature Review ... 4

2.1 Test Automation using Models... 4

2.2 Model Based Testing (MBT)... 5

2.3 Different phases in MBT ... 5

2.3.1 Creation of the model ... 5

2.3.2 Generation of the automatic test cases and test selection... 6

2.3.3 Translation of abstract test suites... 7

2.3.4 Test scripts execution ... 7

2.3.5 Validation of results ... 7

2.4 Advantages of Model Based Testing... 7

2.4.1 Improve Quality of the product: ... 7

2.4.2 Test case management in case of requirements is changed:... 8

2.4.3 Effective test coverage : ... 8

2.5 Disadvantages of MBT... 8

3 MBT Tools... 9

3.1 Tool Survey ... 9

3.1.1 SPEC EXPLORER [6] ... 9

3.1.2 ModelJUnit [7] ... 10

3.2 Comparison and analysis of the tools ... 11

4 Traceability Approaches and Prototype... 14

4.2 Traceability in Model Driven Development... 15

4.2.1 Model to Model Transformation ... 16

4.2.2 Model to Text Transformation ... 16

4.2.3 Importance of Traceability in Model Transformations – A case study... 16

4.4 Requirements Traceability in MBT ... 19

4.5 Different Traceability techniques ... 20

4.6 Prototype... 22

5 Results... 26

6 Conclusions and Future Work ... 27

6.1 Conclusions ... 27

6.2 Future Work... 27

LIST OF FIGURES

Figure 1 Spec Explorer Functionalities [6] ... 10

Figure 2 Domain Package Diagram of Microwave oven system... 17

Figure 3 Class Diagram of Microwave oven system ... 18

Figure 4 Requirements model... 24

Figure 5: State-chart diagram for Chat system ... 24

LIST OF TABLES

Table 1: General properties of MBT tools... 11 Table 2 : Quality attributes of MBT tools... 12 Table 3: Traceability techniques... 21

ACKNOWLEDGEMENTS

I would like to express my gratitude to all the people who have given their heart whelming support in my master thesis.

I thank God Almighty for His strength, which kept me standing and for the hope, which He gave me in believing that this discretion is possible and would be interesting.

I would like to thank my supervisor Antonio Cicchetti who gave constant support and

suggestions from through out this discretion period and made it happen. I also want to thank my family and friends who inspired, encouraged and fully supported both morally as well as spiritually.

1 Introduction

Model Driven Development is a new emerging software development approach, where system is developed from the model. A model is a pictorial representation of the application system, which is presented at higher level of abstraction so that developers understand the system easily. Due to increasing use of MDD in current scenario, importance for models and modeling is also increasing. This chapter describes the importance of models and concepts related to MDD and research work carried out in this area.

1.1 Importance of Models and Modeling

good model predicts the behavioural qualities of the system. Also, whenever there is a change in requirement or specification that can be viewed and understood easily by the model. By having a look at the model different stakeholders of the project – developer, designer, customer and tester comes to know the key characteristics of the system.

1.2 Background

Model driven development offers the potential for automatic transformation of high level abstract application models into running systems. A model expressed in a modelling language will ignore the realization of associations, leaving those decisions to a model compiler or human designer. The most recent innovations in the field of MDD have focused on notations and tools that allow developers to model system at higher abstraction level and later which can be transformed into the programming language code that can be compiled for a particular operating system platform. In 2002, Object Management Group (OMG) introduced Model Driven Architecture (MDA) to organize and manage enterprise architecture for specification of models and their transformation into complete systems[2]. This implies models are given as input to the code generation engines and an executable code is produced from these engines. MDA is one of the approaches in the field of MDD. It is the most known realization of model driven software development approach and basic idea of the MDA is to separate business functions from its technological implementations, enabling code generation and reuse of components. The overall objective is to improve the interoperability and reduce overall cost and time in software development.

all these properties in testing is by using the models defined in software construction to drive the testing process, in particular to automatically generate the test cases[3].

1.3 Scope and Objectives

The scope of this thesis report is limited to Model based testing, survey containing Model- based testing tools and requirements traceability in Model based testing. There are many tools available in industry for MBT, but due to time constraint and this thesis being an academic one, only two open source that are used in the survey. In addition, there are many traceability approaches to track requirements. Only those approaches, which are relevant to model based testing process, are considered in this report.

1.4 Document Overview

2 Literature Review

Considering rapid market changes that occur day by day, it has become mandatory that those changes will be implemented in your application as early as possible for the success and to with stand in the competition. Modeling is one technique that helps in adopting the changes as fast as possible. As the name infers Model based testing is about how testing is done using system models.

2.1 Test Automation using Models

2.2 Model Based Testing (MBT)

Model-Based Testing is a black box testing methodology where test cases are generated automatically using models created from system requirements. Model based testing relies on the execution traces of behavior models. The behavioral model is build from the requirements and specifications of the system. Thus, the generated model represents the behavior of the system under test for the given conditions/specifications. A behavior model is an abstraction of the actual system. The level of abstraction is raised so that the SUT is viewed in simple terms by developers or testers. Typically, the behavior model interprets test cases in a model based testing. Each input to the model and the expected output constitute a test case. The outputs of the implementation code having same inputs fed to the model are captured as well. These expected outputs from the model and actual outputs of the implementation code are compared. If the expected and actual output matches then, the test is passed, or else the test has failed. When there is a failure, the developer can go to that part of the code where the actual output is deviated from the expected behavior of the system and make necessary corrections in the code.

2.3 Different phases in MBT

Model-based testing processes contains a series of five phases which includes (a) creating a test model, (b) automatically generating test cases and selecting the test cases, (c) translating selected abstract test scripts to executable test scripts, (d) executing the test scripts and finally (e) comparing results with those of expected results from the model. All these different phases together form the MBT process. Below is the brief description of each of these phases.

2.3.1 Creation of the model

must be taken for building the right model. On the other hand, modeling can begin early in the development cycle. This may lead to discovering inconsistencies in the specification, thus leading to correct code from the very beginning[15]. A model of the system under test is build using modeling languages. There are many modelling languages which support the creation of models. A model based testing tool consists of a model editor which supports creation of the model by using some modeling languages. When a particular input is given to the SUT what are possible states that system can exhibit is represented in the model. This model can be yet times consider as a state machine diagram where different states of the system upon giving an input is specified.

2.3.2 Generation of the automatic test cases and test selection

selection is a difficult task. A simple solution is to make a random selection. By making such a selection exhaustiveness is lost, but soundness is preserved. A selection strategy should aim at detecting as many erroneous implementations as possible within a restricted period and it should maximize the chance of detecting an error while minimizing the cost of executing the test suite.

2.3.3 Translation of abstract test suites

Test cases generated in the previous phase are abstract test cases as the model which is given as input to test generation process is the abstract representation of the SUT. To perform testing, these test cases need to be converted into executable test scripts. In this phase of MBT abstract tests are translated to executable test scripts. For this conversion, an adapter is used that bridges the gap between the abstract test cases and the SUT. An adapter coverts abstract specific names from the model to the concrete names that are understood by the implementation system.

2.3.4 Test scripts execution

Generated test scripts are executed on the SUT and results are used for evaluating whether the test is a pass or fail.

2.3.5 Validation of results

An important phase of MBT is to validate the results which are generated by executing the test cases of the model that is created. This process is called conformance testing. By comparing actual output of the SUT with the expected output from the behavioral model, it can be decided if the test is passed or failed. Conformance testing is a unique feature of MBT, which makes the MBT process more efficient and effective.

2.4 Advantages of Model Based Testing

Model based testing has many advantages. Some are listed below.

2.4.1 Improve product quality:

2.4.2 Test case management in case of requirements change:

Test case management becomes easy by using MBT. In traditional testing techniques, whenever there is a change in requirement there are changed test plans, test designs and test cases. MBT allows tests to be linked directly to requirements through a model[13]. Therefore, it is sufficient that only one model is updated as per the new requirements. Then new set of test cases are automatically generated from the updated model.

2.4.3 Effective test coverage:

Testing a software product should assure that all the functionalities that are mentioned in the specification document are achieved. The goal of the tester is to develop test scripts that have full coverage of all specifications. Constructing the model based on specification documents ensure that all the functionalities of the system are covered. Automatically generated test scripts from such a model provide the full functional coverage[14].

2.5 Disadvantages of MBT

3 MBT Tools

A MBT approach may automate the phases described in the previous section by algorithms or guidelines. But it requires a tool to make the execution of these phases feasible. It is important to observe what phases of MBT process a tool can support.

3.1 Tool Survey

A survey is conducted on some of the available open source MBT tools. The survey includes gaining brief knowledge about the functionality of the tool, what phases of MBT the tool supports and the possibility to interchange the models between different tools. Based on the experience gained working with the tools, traceability in model based testing is analysed. With an emerging technology, there came up many tools that support a model based testing. But, by analyzing based on certain crucial factors two model based tools are picked for carrying the study further. These crucial factors include – if the tool is an open source tool or not, and weather all phases that an MBT tool support matches the actual MBT process or not. Analysis on these factors shortened the list of MBT tools available in the market and resulted in the selection of two MBT tools. This section describes in detail these two model based testing tools SPEC EXPLORER and MODEL J UNIT, which are an open source MBT tools.

3.1.1 SPEC EXPLORER

Figure 1 Spec Explorer Functionalities [6]

From the model program, a state machine diagram is generated. By exploring all the states of a state machine, the test cases are generated. There are options available to choose different model explorations. Generated test cases are executed against the SUT. This tool provides the conformance testing, in which expected results from the model and actual results from the SUT are compared and an automatic verdict if the test case is passed or failed is given. Spec Explorer provides the option to export states-transitions of the state machine and generated test suites into XML file.

3.1.2 ModelJUnit

ModelJUnit [7] is a Java library that extends JUnit to support model-based testing. ModelJUnit is an open source tool. In this tool, the Finite State Machine (FSM) or Extended FSM (EFSM) models are written in Java. From these FSM models, test cases are generated. It has three test generation algorithms. They are Random walk, Greedy walk and Look ahead walk. The generated test cases measure various model coverage metrics.

An FSM Model class is written in Java. Modlejunit.jar and junit.jar needs to be included in the class path. Test cases are generated when this main method is executed in the class model. Generated test cases can be saved and used for later execution, which is called offline testing. To do online testing, the test generation code is usually written within the TestXYZ () methods of JUnit test classes, so that each time JUnit test suite is executed, it will generate a suite of tests from the FSM model. In addition, the @Action... methods in the model class will include code to call the methods of the SUT, check their return value, and check the status of the SUT. In this

Generate Model Explored scenarios User-written wrapper Test suites Explored by Spec Explorer

Provides actual results Are run by Invokes Graph views Visualized by Log of conformance-checking test run

way, every time JUnit tests are executed, the model is used to generate a sequence of @Action... calls and test the SUT.

3.2 Comparison and analysis of the tools

Based on the usage of MBT tools, comparison tables are made of these tools. Table 1 specifies the general properties of the MBT tools like which modeling language is used, if the tool supports all the phases of MBT etc.

Table 1: General properties of MBT tools

Tool Name Type Modeling Language Usability Automatic generation of test cases Export models to other formats Model Simulation Platform Spec Explorer Research SPEC # and ASML

Yes Yes Can export to XML file

No Windows

Model Junit

Academic Java No Yes Can export to XML file

No Windows

Table 2 : Quality attributes of MBT tools

Tool Name

Functionality Reliability Usability Efficiency Maintainability Portability Licensing and pricing Spec Explorer Interoperability : Databases parts of states, transitions and test suites can be exported to an XML file but not the State machine diagram as whole. So interoperability is not achieved in this tool Fault Tolerance: Tool does not support this. Recoverabili ty: If the application crashes, it can be recovered as data is stored locally Has a good GUI, which makes application to be operated easily. However, application does not support lower versions Microsoft word 2007 An efficient MBT tool supporting all phases of MBT– modeling to result comparison The application behavior can be analyzed easily using this tool Installability: Simple and easy to inst-all instructions are provided Open Source

4 Traceability Approaches and Prototype

This chapter describes the requirements traceability in detail in relation with Model driven development and Model based testing. Based on analysis of various techniques and approaches a prototype for traceability in model based testing is described in last section of this chapter.

4.1 Requirements Traceability

The Institute of Electrical and Electronics Engineers (IEEE) Standard Glossary of Software Engineering Terminology defines traceability as “the degree to which a relationship can be established between two or more products of the development process, especially products having a predecessor-successor or master-subordinate relationship to one another” [11]. The Capability Maturity Model Integration (CMMI) Model requires the use of requirements traceability for CMMI Maturity Level 2 and has adopted the IEEE definition for traceability as the degree to which a relationship can be established between two or more work products of the development process. Traceability means tracking relationship from initial requirements gathering phase to the final software product. Traceability as well means tracking relationship from final product to requirements. The first definition of traceability is called forward traceability while the later one is backward traceability. Tracking the requirements in forward as well as backward direction is called bi-directional traceability [17]. Traceability would provide details about different software artefacts like requirements, design, code and test cases and how these artefacts are associated with each other. Various traceability techniques (forward as well as backward traceability) enable to identify the need to update an artefact when a change is introduced [33]. (The change may be during the development process or even in later stages during software maintenance phase). Traceability further assures developers that correct directions are followed in the product evolution. Tracking to requirements from the product ensures that requirements have been kept through out product evolution. Which implies no functionality of the developed product is out of scope of the specifications. In the rest of this report terms ‘traceability’ and ‘requirement traceability’ are used inter–changeably.

The major challenge for performing traceability of software artefacts is the cost involved. The cost of maintaining traceability information is high because of following reasons:

 Software artefacts are handled by different people at different locations due to the distributed development environment. Having a centralised traceability mechanism among different artefacts and among different groups requires more resources and involves more cost.

 Availability of tools with correct semantic support for traceability is low. If traceability is mandatory in any project, some additional traceability measures related to semantics need to be manually taken by project team to make the tool compatible with project needs. The major benefit of traceability is to improve the quality of the software. When a comparison of cost involved in maintaining traceability and the benefits of the traceability is done, immediate or visual benefits of traceability are less [18]. Due to which traceability is not given much importance. But because of the emphasis on software quality in the recent years, traceability has gained significant importance in software development.

4.2 Traceability in Model Driven Development

Initially traceability relationships are manually maintained. Some of the techniques that are used for maintaining traceability relationships include a traceability matrix using cross referencing [19], graph based approaches [21] and formal methods [22]. Later semi-automated traceability approaches like hyperlinks that are manually added are used [20]. Realising the power of automation techniques like deep traceability and surface traceability which derive traceability information automatically are developed [23]. In most of these traceability techniques ability to link the artefacts is mentioned. But for the traceability to be effective and qualitative (so that customer could be easily track the requirements from the given artefacts) it is important to have correct semantics between these links.

transformation. One is model to model transformation and the other is model to text transformation.

4.2.1 Model to Model Transformation

MDD recognizes the need of many different intermediate models to represent a system. Source models are built at much higher level of abstraction for the developer to easily understand the system. But to generate code, it is necessary that these abstract models are converted to concrete models with target language. Abstract models are transformed into more specific ones while preserving the behaviour of the specified software system. When models are represented in the source platform initially it gives the scope to transform this model to a target platform as required by different customers. This further assists in standardizing source model platforms. Initially these model transformations are done manually. But with recent development in model driven engineering concepts, transformation engines are used to transform from one model to another. In a model to model transformation trace information needs to be kept up-to-date whenever models change.

4.2.2 Model to Text Transformation

A model to text transformation includes the ability of the transformation engine to produce output in a file format, which contains source code, deployment specifications, reports and documents from the model, ability to interact with system libraries and ability to support parameterized transformation (where parameters are provided during transformation) [24]. From the details provided in concrete model, source code artifacts are generated. This generated source code is a text-based artifact. Any changes made in models needs to be effectively reflected in the text files that are generated. To check if changes done in model are effectively implemented in code it is necessary to maintain traceability links between models and text artifacts.

4.2.3 Importance of Traceability in Model Transformations – A case study

lead by these sets of transformation rules [25]. These relationships carry the indented behaviour of the application system through out the transformation process.

Case Study

Importance of traceability is explained in detail by the following case study. Consider developing a system called Microwave Oven. Microwave Oven is a system which performs normal microwave oven operations. All the elements involved in developing this system are built under a package diagram. At an abstract system level a package diagram is used to represent the system. Each element in the package is further represented in a class diagram. One of the requirements of user of the Microwave Oven is to successfully ‘open’ it. So in this example we consider how this requirement is traced among different model diagrams that are used to represent the Microwave system.

Figure 2 Domain Package Diagram of Microwave oven system

system. This Datatype package may include built-in data types, user-defined data types and enumerations.

Each Microwave subsystem is represented as a static class diagram and it includes associations of different objects in that class diagram. The class diagram represents a set of objects and each object has the properties such as attributes, operations, associations and current state of the object. In this example, the class diagram has six objects. Each individual object has a ‘unique_id’. This ‘unique_id’ is a built-in data type taken from Datatype package.

Figure 3 Class Diagram of Microwave Oven system

another is guided by set of transformation rules. Traceability from the package diagram to the class diagram is achieved by parameters passed to the function and by the association relationship that exists between two objects which are included as part of the transformation rules. Therefore, transformation rules play an important role in achieving traceability between artifacts. Each class in the Class model is further transformed into a State Machine diagram where the behavior of each class is represented. By using parameters and associations, traceability is maintained between these transformations.

4.4 Requirements Traceability in MBT

Traceability between different software artifacts in development life cycle includes having traceability links between the test cases and requirements as well. Taking into consideration the importance and role of traceability in Model Driven Development, it is extended to model based testing as well. This section describes the importance the traceability from the perspective of model based testing.

Traceability of requirements would enable

 To Perform Impact analysis: executing a test case on an updated code may some times result in a failed test case due to code changes. Such a test failure leads to perform a proper ‘impact analysis’

 To identify the criticality of the requirement: criticality of a requirement in the application system can be known whenever there is test failure

 To analyze the functionality of test case: by tracking requirement from test case, actual functionality of the test case can be obtained

 To enhance project management: project management is effectively done by tracking test coverage of each requirement

 To improve the test coverage and quality: traceability would help in viewing untested requirements. This would help in having full test coverage and thereby increase the quality of the application system

We consider a model-based testing approach, where Unified Modelling Language (UML) models are used represent the behavior of the system under test. These models are created based upon system requirements. Various UML diagrams, which represent SUT behavior includes, use case diagrams, class diagrams, activity diagrams, sequence diagrams, and state chart diagrams. The customer gives the specification of the system in textual format. Use cases represent the textual requirements effectively in a model format. Each textual use case acts as a high-level informal test objective. By using a template structure, textual requirements are transformed into a use case diagram. These use-case needs to be transformed into sequence diagrams, describing behavior of the system. These sequence diagrams represent an abstract test case. From sequence diagrams, state chart diagrams are built. States and transitions of State chart diagrams exhibit the dynamic behavior of the system under test [35]. A most common model that is given as input to MBT tools for test case generation is state chart diagrams. Therefore, from textual requirements to generation of state chart diagrams there is a series of model-to-model transformations takes place. Traceability in model-to-model transformation can be maintained using internal links between the models and guided by transformation rules. Later from a state chart diagram, which is an input to a MBT tool, test scripts would be generated. A test generation tool generates numerous test cases. However, it is quite important that each of these generated test cases is linked to a requirement.

4.5 Different Traceability techniques

Table 3: Traceability techniques

Name of the

Traceability

Technique

Process

_Advantages

_{Disadvantages}

Requirements Engineering Through Hypertext (RTHE)

This method is based on ‘hypertext’ technology. Traceability is achieved by maintaining hypertext links between different software artifacts involved in the project [26].

1. Bi directional links results in an all around traceability both forward and backward.

2. Too is not only for managing existing requirements but also for creating new requirements [27].

This technique is implemented manually, which in turn requires additional resources for maintaining traceability.

Information Retrieval (IR)

Traceability links are created on the basis of the similarity between text present in different software artifacts.

Due to the incremental traceability between different artifacts using IR techniques, quality of the artifacts is also improved [28].

Is suitable for those projects in which documents are important artifacts and not suitable for agile

development work Annotations with

Identifier tags

Annotations are comments, notes, explanations, or other external remarks that are specified in an artifact. During modeling phase, models are annotated with requirement identifier tags, which are read out automatically by modeling tool [29]

Though it is effective traceability, technique annotations are added manually.

T-Vec Tabular Model

This is derived from Software Cost Reduction (SCR) where requirements are specified in a table format.

Predefined modeling structures in table format are present where requirements are specified. The requirements identifier (ID) in table is used for traceability of requirements.

This method facilitates in writing and understanding the functions and aids in detecting the

specification errors.

An annotation with identifier tags is the most common traceability techniques used in many of the traceability tools. For example, the LRDA tool suite uses this approach for attaining the traceability. T-Vec Tabular Model (TTM) is one component of RAVE, which is a requirement-based automated testing method. TTM approach needs to be integrated with an external RM tool such as DOOR’s for tracing the requirements.

4.6 Prototype

‘Change’ is common and frequently used term during software development and maintenance phases. This change may be in requirements, design or in code. Entire software development and maintenance phases should be flexible enough to adapt these changes. Traceability helps developers understand relations between artifacts such as requirements, design and code. Whenever there is a change in a software artifact, this traceability would enable in analyzing impact of that change on other artifacts[36]. As specified earlier ‘impact analysis’ is one of the important reasons for maintaining traceability. Tracing techniques should enable the developer or tester in achieving a proper impact analysis. For example, if there are any design changes made towards the final stages of the software development, an efficient traceability technique should enable to track the effect of this design change in both directions. That is to the requirements as well as to the code and functionality of the application. Traceability is one way through which change in different software artifacts are managed. This section describes the proposed method that could be used to track the requirements in a model based testing scenario.

In [31], a requirement meta-model is proposed which is based on UML. This includes a class diagram that represents the abstract view of the SUT. Similarly, for addressing the traceability of the requirements, we propose a technique of modeling requirements. In our approach, we have modeled the requirements in a semi formal class diagram. This requirements model includes the two basic stake elements. One is a class element named requirements containing requirements represented as identifiers. The second-class element includes a group of classes, which are the main actors of the application system. These classes contain all functions an application system performs and these functions are represented as operations of a class diagram.

Test cases in the MBT process constitute a pair of input and output behavior of a model [30]. In a state chart diagram, dynamic behavior of the system is represented. When a state chart diagram is given as input to the MBT tool, each path in a state chart is considered as a test case. Initially the application system would be in a particular state and we suppose it to be the initial state of the system. When an input is fed to the system, a transition is made to a state, which would be an output state. This input, output combination form a unique test case. Input given to the application system would be represented as a function or method in the state chart diagram. Output state represents the expected behavior of the application system. The traceability issue could be addressed by maintaining a relationship between the state chart diagram and the requirement model. The relationship between the state chart and requirement model would enable in tracking the requirement identifier. Some times a change in code or requirements or design would result in a change in the behavior of the application system. This changed behavior would be different from the actual or expected behavior of the system that is modeled in a state chart diagram. When a test case that is generated from a state chart diagram is executed on such an updated code of the SUT it would result in failure. In case of a test-failure being reported, the tester can track the actual expected behavior of the system. The function or input method, which is provided to the state chart diagram due to which test case is failed, could be tracked. Based on relation between state chart and requirement model, and the corresponding requirement identifier that is responsible for this test case failure could be obtained.

Figure 4 Requirements model

The initial system would be in an ideal state. Once an action is performed, a transition takes place to another state. To make it simple , state chart diagram with only one state Message send state is considered. Figure 6 shows the state chart diagram of the chat system.

This state chart diagram can be extended for different clients and different states. By maintaining relationships between the requirements model and state chart diagram, the requirements identifier link (‘ID1’ in this case) is passed to the test case generation process. The operations represented in the requirements models –‘c1.Sending Msg()’ is tagged with the corresponding requirement identifier. This is implemented by internal traceability links in the model. Input in the form of a function ‘Sending Msg()’ is given to the initial state. Then the system moves to a state – ‘Message sent’. A test case for a chat system constitutes of

Input state: Initial Sate; Input action: Sending Msg; Output State: Message sent.

5 Results

This chapter describes the results of the proposed approach and advantages of this approach over other traceability techniques. This approach is mainly to simplify the traceability process in context of model based testing. Having transparent tracking techniques would ease the maintainability of the software. Having requirements represented in a model as identifiers would make the tracking process transparent. In general for attaining traceability, the model based testing tools is integrated with a third party requirements management tool. For integrating the MBT tool and requirement management (RM) tool, an additional application program interface (API) or a connector that is suitable for both tool environments needs to be developed. This would result in additional work. Figure 6 depicts how the MBT tool and the RM tool needs to be connected.

Figure 6 Integration between the RM tool and the MBT tool

6 Conclusions and Future Work

6.1 Conclusions

This report describes a traceability method, which could be used within the MBT tool. The method specifically focuses on design issues, addressing traceability of requirements in an automated test generation process. A major aim of the research was to analyze various traceability approaches and techniques, and to find the most suitable approach for Model based testing environment. The strength of this approach is that traceability is attained within the MBT tool. Thereby the need for an external requirements management tool to maintain the traceability is eliminated. A major challenge during implementation would be representing requirements in a requirement model. In a practical scenario, managing hundreds of requirements in a model diagram would be a difficult task. Since the major focus of model based testing is to test behavior of the SUT, the requirement model may be constrained to representing only functional requirements.

6.2 Future Work

REFERENCES

[1] Robert France and Bernhard Rumpe, 2007, Model-driven Development of Complex Software: A Research Roadmap, IEEE computer society

[2] Alan Brown, 2004, An Introduction to Model Driven Architecture, http://www.ibm.com/developerworks/rational/library/3100.html , last accessed on 26th

December 2008

[3] Antonia Bertolino, 2007, Software Testing Research: Achievements, Challenges, Dreams,

IEEE computer society

[4] Harry Robinson, 2000, Intelligent Test Automation – A model based method for generating tests from a description of an application behavior, Software Testing and Quality Engineering Magazine

[5] Douglas Hoffman, 1999, Test Automation Architectures: Planning for Test Automation,

Software Quality Methods, LLC

[6] Spec Explorer Reference Manual, Microsoft Corporation

[7] http://www.cs.waikato.ac.nz/~marku/mbt/modeljunit/, last accessed on 15th_{March 15, 2009.}

[8] F. Bouquet, E. Jaffuel, B. Legeard, F. Peureux and M. Utting, 2005, The University of Franche-Comté - France, LEIRIOS Technologies –France, The University of Waikato – New-Zeland, Requirements Traceability in Automated Test Generation - Application to Smart Card Software Validation, ACM Digital Library

[9] Orlena C. Z. Gotel and Anthony C. W. Finkelstein , 1994, Imperial College of Science, Technology & Medicine Department of Computing, 180 Queen's Gate, London, An Analysis of the Requirements Traceability Problem, IEEE

[10] https://www.goldpractices.com/practices/mbt/index.php, last accessed on 16 March 2009 [11] IEEE Standards Software Engineering, IEEE Standard Glossary of Software Engineering

Terminology, IEEE Std. 610-1990, The Institute of Electrical and Electronics Engineers, 1999, ISBN 0-7381-1559-2

[12] Axel Belinfante, Ed Brinksma, Jan Feenstra,Jan Tretmans and Ren´e de Vries, 2001,

University of Twente, The Net

herlands, Automatic model-based testing of communication protocols -http://fmt.cs.utwente.nl/publications/files/125_ctit-workshop2001.pdf

[13] Mark Utting, The University of Waikato, New Zealand, Position Paper: Model-Based Testing -http://www.cs.waikato.ac.nz/~marku/papers/utting_mbt_position.pdf

[14] Peter Becker ,Eliminating Functional Defects Through ModelBased Testing -http://www.stickyminds.com/sitewide.asp?Function=edetail&ObjectType=ART&ObjectId=5 995, last accessed on April 7, 2009

[15]

Harry Robinson and Steven Rosaria, Intelligent Search Test Group, Microsoft

Corporation,

Applying Models in your Testing Process

[16] Criteria for Software Testing Tool Evaluation – A Task Oriented View -http://www.softwareforschung.de/fileadmin/_primium/downloads/publikationen/IHPR2005.p df

[18] Perdita Stevens, School of Informatics University of Edinburgh, Traceability in (bidirectional) model transformations - http://www.nesc.ac.uk/talks/955/stevens.pdf, last accessed on 2nd_{May 2009}

[19] M. West, "Quality Function Deployment in Software Development," IEE Colloquium on Tools and Techniques for Maintaining Traceability During Design

[20] J. Jackson, "A Keyphrase Based Traceability Scheme," IEE Colloquium on Tools and Techniques for Maintaining Traceability During Design

[21] I. Alexander, "Toward Automatic Traceability in Industrial Practice," Proceedings of the 1st International Workshop on Traceability in Emerging Forms of Software Engineering, Edinburgh, Scotland

[22] F.A.C. Pinheiro and J. A. Goguen, "An Object-Oriented Tool for Tracing Requirements," IEEE Software

[23] Julian Richardson and Jeff Green, Automated Software Engineering Group NASA Ames Research Center, Automating Traceability for Generated Software Artifacts, IEEE Computer Society

[24] Jon Oldevik, Tor Neple and Jan Oyvind Aagedal, SINTEF Information and Communication Technology, Norway, “Model Abstraction versus Model to Text Transformation”

http://www.cs.kent.ac.uk/projects/kmf/mdaworkshop/submissions/Oldevik.pdf, last accessed May 4, 2009

[25] Netta Aizenbud Reshef , Brian T. Nolan, Julia Rubin and Yoel Shaham Gafni, IBM Journals, “Model Traceability” -http://www.entrepreneur.com/tradejournals/article/150989185_5.html, last accessed on May 5, 2009

[26] Hermann Kaindl, SIEMENS AG, Austria, 1993, “The Missing Link in Requirements Engineering”, ACM SIGSOFT, Software Engineering Notes vol 18 no 2 - page 30 -38

[27] Gerald Ebner and Hermann Kaindl, 2002,"Tracing All Around in Reengineering", IEEE

Xplore

[28] Rocco Oliveto, University of Salerno,Italy,2008," Traceability Management meets Information Retrieval Methods– Strengths and Limitations", IEEE Xplore

[29] Method for automatically creating a behavior pattern of a computer program for model-based testing techniques, July 2007, http://www.freshpatents.com/-dt20090205ptan20090037884.php?type=description, last accessed on May 9, 2009

[30] Mark Utting, Alexander Pretschner and Bruno Legeard, “A Taxonomy of Model-Based Testing", - , last accessed on 11 May 2009.

[31] Robert V Binder,1999, Testing Object-oriented Systems Models, Patterns, and Tools, Addison-Wesley Professional , 1119 p

[32] S.R. Dalal, A. Jain, N. Karunanithi, J.M. Leaton, C.M. Lott, G.C. Patton and B.M. Horowitz, New Jersey ,USA,1999, “Model-Based Testing in Practice”, ACM Digital Library [33] Jane Cleland-Huang, Carl K. Chang and Mark Christensen, IEEE , 2003, “Event-Based

Traceability for Managing Evolutionary Change”, IEEE Computer Society.

[35] Mark Utting, and Bruno Legeard, 2007, “Practical Model-Based Testing: A tools approach”, Elsevier Inc.

[36] Kannan Mohan, Peng Xu , Lan Cao and Balasubramaniam Ramesh, United States, 2008, "Improving change management in software development: Integrating traceability and software configuration management", Elsevier, Science Direct