Quality Assurance of PaaS Components Configurations

(1)

AKADEMIN FÖR TEKNIK OCH MILJÖ

Avdelningen för datavetenskap och samhällsbyggnad

Quality Assurance of PaaS Components

Configurations

- A Case Study at Sogeti

Omar Totangy

2020-06

Examensarbete, Grundnivå (kandidatexamen), 15 hp Datavetenskap

IT-systemutveckling - mot geografiska informationssystem

(2)

(3)

i

Acknowledgements

The research for this thesis has been conducted at the department of computer and geospatial sciences at University of Gävle. Throughout this thesis, I have received generous support from numerous people, who in different ways has contributed to the completion of this academic report.

I would like to thank my supervisor at Sogeti, Johan Burman. Without your support and guidance, this thesis would have never been accomplished. I would also like to thank my supervisor at University of Gävle, Dr. Julia Åhlen, who has supported and encouraged me throughout this work. I will always keep the knowledge and support you have contributed with.

Finally, i´d like to thank my family and friends for the support and knowledge you have contributed with. I am grateful for the hours you have spent on reading this thesis and the valuable comments that you have given me.

Omar Totangy

(4)

(5)

iii

Abstract

Summary: In a world where software systems are essential for our everyday life, a

vast amount of software errors have led to severe consequences, almost causing a nuclear war. Ensuring the quality of software systems has increasingly come into demand due to the rising complexity of technologies used in today´s solutions such as cloud computing, internet of things (IoT). Companies strive to ensure the quality of their softwares and have different approaches to do so. Some companies follow an industry standard such as ISO 9000, others develop their own standards and

implement a software to ensure conformance. Quality Assurance (QA) has traditionally been implemented in the later stages of the development process. Research suggests however that early or constant implementation of QA is more likely to provide better end results. Current studies explore the stages of implementation for QA, the effects of QA and approaches for QA. Few studies however investigate how QA is approached within private companies. This study investigates the implementation of QA within private companies, the use of QA and which stage of the development process QA should be implemented. The

investigation was done through suggesting a suitable approach to achieve QA for the case company Sogeti. The challenge that Sogeti faced was to ensure the quality of their PaaS components configurations. To further understand the problem, a QA plan was developed with the use of Robert P. Elliots ten steps of developing a QA specification. The QA plan indicated that a software would be a suitable approach due to the specific demands that Sogeti provided, the software would then implemented in the later stages of the QA development process before the components reach the end customer. The final prototype of the software helped discovering the amount of improper configurations that exists (38,46% components with improper configurations). Early QA might have been a better approach, assuming that the problems were discovered early. When the problems are

dicovered late, the stage of QA implementation might not matter as the approach for QA is reactive. In order to draw that conclusion however, further research is needed where multiple companies are analyzed to visualize the qualitative and quantitative effects of QA.

Keywords: Quality Assurance, PaaS, Azure, Software development

(6)

(7)

v

1 Glossary and Abbreviations

API – Application Programming Interface

ARM-Script – Azure Resource Manager Scripts Blob – Binary large object

Cloud Computing – Developing server hosted solutions. CMD – Command Line Interpreter

Commit – Change to the code, used in GIT. Component – Part of a system or application C# – Programming language

GIT – Version control system.

ISO – International Organization for Standardization IaaS – Infrastructure as a Service

IoT – Internet of Things

JSON – JavaScript Object Notation, a file format where information is stored in “keys”

such as, Animal: “Dog”, where the key is “Animal” and the value is “Dog”.

Microsoft Azure – Cloud computing platform PaaS – Platform as a Service

QA – Quality Assurance QC – Quality Control

QMS – Quality Management Systems SaaS – Software as a Service

Terminal – Application that provides test-based access to the operating system. Available

for MacOS.

(10)

1

2 Introduction

Quality Assurance (QA) is a way of preventing mistakes to end products or services. In software development, QA plays a vital role to ensure proper functionality of a software. The benefits that QA offers is for instance, detecting codes that are generally associated with bad programming practices and design or ensure conformance to programming or company guidelines. Technological advancements in recent years has made QA more integrated with cloud computing and has enabled the creation of new solutions on all parts of the cloud service model (hardware, platform and software). The platform (Platform as a Service, PaaS) is a bridge between hardware (servers) and the software (e.g. mailing services). This study investigates the importance of QA for PaaS components configurations by analyzing the PaaS configurations of the case company Sogeti and investigate if they meet common configuration demands and company guidelines.

(11)

2 2.1 Background

Quality Assurance (QA) is a way to ensure that an end product meets desired quality by preventing mistakes and defects based on a set of specifications. It plays a vital role in the software life cycle process where testing the QA specifications is necessary [1]. According to the ISO 9000 standard, “Quality Assurance is part of quality management focused on providing confidence that the quality requirements will be fulfilled” [2]. QA offers potential benefits when it comes to detecting e.g. “code smells” meaning code that is generally linked to unsatisfactory programming practices and design. Examples of “code smells” are variable names or functions that do not explain the variables. QA can also ensure conformance to programming or company guidelines which makes it easier for a large team of

programmers to develop, integrate and maintain a particular piece of software [3]. Cost of repairing a bug is lower when the bug is found early in the development cycle and earlier research suggest that QA should be implemented in every stage of the development process [4]. While some research suggests that QA should be implemented in all stages of the development process, other research suggests that QA should focus on early

implementation. However traditionally QA methods has been focused on the later

development stage such as implementation and related testing activities [5]. Advancements in technology such as cloud computing has made QA more in demand due to the

complexity of the technology. QA is becoming more integrated with cloud computing enabling the creation of new, more competitive and agile products and solutions [6]. Cloud computing has been described as one of the emerging technologies that will lead to the next generation of Internet and consist of three parts, IaaS (Infrastructure as a Service), PaaS (Platform as a Service) and SaaS (Software as a Service). IaaS components include hardware, connectivity, cloud infrastructure, servers etc. PaaS allows users to create applications with programming languages and their deployment onto the IaaS components. SaaS allows the use of applications that run on PaaS components and are accessible from various client devices[7]. One of the companies that host these cloud services is Microsoft Azure [8]. The user can through an API and a coding language be able to use the

(12)

3 2.2 Aim of the Study

This study investigates how QA is implemented within private companies by analyzing different approaches for QA and suggesting a method to achieve QA for case company Sogeti. The study is limited to PaaS components configurations which Sogeti provides and is also only limited to Sogeti.

2.3 Research Questions

- How can QA of PaaS components configurations help deliver common application demands?

- How can QA be implemented within private companies?

- In what stage of the development process should QA be implemented within private companies?

2.4 Delimitations

(13)

4

3 Theoretical Framework

This chapter present the theoretical framework, chapter begins with definitions of quality assurance and describes previous research within this area.

3.1 Why does Quality Assurance matter? How is it implemented?

Transport, communication networks and financial markets are systems which is essential for our everyday life, these systems are by a large extent controlled by software systems. Those systems has had a countless amount of instances where a software error has led to severe consequences [11]. A software error in 1980 almost caused a nuclear war because data was misrepresented by the software [12]. While the need for QA is rising, the

software’s is advancing too. The evolution of software’s brings new challenges for QA such as, programming paradigms, new complex process models and more. Some approaches to achieve QA, according to previous research [13], is to implement:

- Open source software development model

o The implementation details are exposed due to transparency, which is critical for software quality assurance.

- Software configuration management system such as GIT

o Changes to a code repository is logged, which is critical for software quality assurance.

- Object oriented design o Improves data locality o Easier refactoring

o Easier to change/add or delete parts of the software - Automated testing suites

o Unit tests in order to make sure that the code can´t break - Online documentation

o In order to help developers, understand each other’s code.

(14)

5

3.2 Quality Assurance: Specification Development and Implementation

QA can only be beneficial if the QA specifications are properly developed. Robert P. Elliot recommends a set of steps in order to develop the QA specifications. First step is to determine how important it is to develop a QA system, considering risk of failure, cost of the QA system and the risk of failure if the system developed is inadequate. The second step is to identify the controlling properties that control the items performance or acceptability, sometimes the property cannot be controlled but the related property can. The third step is selecting a method which tests both precision and accuracy, the definition of accuracy is correct but individual value might not be correct, the definition of precision is the same value in every test but the value might not be correct. Step four is to define what is acceptable or not, sometimes what is considered unacceptable might be acceptable and vice versa, it is also recommended to consider if it is realistic. Step five is to identify reasonable risks that might occur based on the importance of the item and the severity of the consequences of its failure. Step six is to select a sampling plan based on principles of small sample statistics. The objective is to balance the cost and effort of testing against the risks of an incorrect acceptable-unacceptable decisions. Step seven is to examine different ways of handling the methods for handling construction items that do not conform to the desired quality. Available options are to remove and replace the work, rework until the quality is met or accept the work. Step eight is to define the contractor’s quality control (QC) responsibility, different methods are presented such as submitting a QC plan for review and control or including the QC test results into its own acceptance program. Step nine is to formulate the QA plan in a clear manner that is only open to one interpretation. The final step is to review the QA plan to ensure that it is practical and enforceable [14].

3.3 Previous Code-Detecting QA Software

By understanding how previous research used a software to achieve QA, a similar approach can be taken for this thesis. In previous research by Emden and Moonen [3], a QA software was developed to detect “code smells”. The authors describe “code smells” as code that are generally associated with bad programming practices and design. In their approach they presented what type of code smells they were going to detect, how they were going to detect them and how the results would be visualized. The type of code smells that they decided to detect was based on a list presented in a book called “Refactoring: Improving the design of existing code” [15].

(15)

6

Emden and Moonen´s architecture is shown in figure 2. The author´s first use a parser that reads the source code and produces a parse tree which contains all structural information contained in the code. Then the authors use an analyzer that reads all the parse trees and traverses them according to the program structure. During the traversal, the analyzer visits all the program entities and stores the “smells”, program structure and the primitive properties. This object model covers software that need to parse items, extract program structure from those items and visualize the results, making it ideal for this projects software.

Figure 1. Architecture of code smell browser – Author´s figure based on [3]

3.4 Structure of the ARM-Scripts

The components that this thesis focuses on is the ARM-Scripts (Azure Resource Manager Scripts). The ARM-Scripts contain configurations for components and can be connected to other scripts and components and can trigger functions. These scripts show the name of each component and all its configurations such as error handling or logging activities. The ARM-Scripts contain a schema link, content version, parameters, variables and resources. The resources contain the configuration for each component, every component within the ARM-Script is in the resources array. The parameters contain the name of each component that exist in the script and also contains components that refer to other ARM-scripts, these components are called “external components” in this thesis. This is shown more clearly in figure 3, however due to classified information in the ARM-Scripts, the values are not shown.

(16)

7

The resources which contain the configurations for the components contain the properties, tags, dependencies, type, name, comments, location and api version. The log-message, context-log and retry policy configurations exist within the properties. The dependencies refer to other components within or outside the current ARM-Script. The types that exist for components are workflows and connections, the connections are usually connected to an api such as “azure-log-analytics”. The name shows which parameter the component is connected to, where in the parameters default value lies the true name for the component. Figure 4 shows the structure of the resources more clearly.

Figure 4. Resources mapped in tree structure – Authors Figure

3.4.1 Azure Service Bus

(17)

8

4 Methodology

This chapter presents the methods used in this thesis to answer the research questions. First, a description of how the data handled in this report is presented. Thereafter, how the literature for this report was collected. Furthermore, the definition of quality is presented and finally, how the software of this thesis was developed.

4.1 Data and Ethics

The ARM-scripts that Sogeti have provided are classified and therefore deleted after the completion of this thesis and is not shown in any way in this report. No data which may identify any person is presented in the thesis without permission from said person. The collected data is only used for scientific purposes and participants in the data collection are aware of the purpose and usage of the data.

4.2 Collecting Literature

The following key words was used to search for relevant studies where studies between 2016-2020 was favored, older studies however was not excluded:

- Quality Assurance

- Quality Assurance Program - Software Quality Assurance - Software Quality

- Cloud Computing - PaaS

- Microsoft Azure - ARM

- Quality Assurance Techniques - JIRA

- Windows Azure

(18)

9

4.3 Developing the QA Specifications

Prior to suggesting an approach, proper QA specifications need to be created because poorly developed QA specifications can lead to lost benefits [14]. Sogeti provided a list of functional and non-functional demands which the software must meet. These demands are as follows:

Demand 1: No underscores in the component names

Demand 2: All components with “endpoint” in their names should have a context log

implemented and a message log implemented.

Demand 3: All components with “endpoint-to” in their names should have the

“retryPolicy” set to none.

Demand 4: Components from the test environment should not communicate with

components from the production environment.

Demand 5: Show all connections between components

The specific demands indicates that a software might be the optimal approach where as an industry standard might be more optimal for more generic demands. The ten steps earlier described by Robert P. Elliott is used in order to develop the specifications of the software, with the exception of step six and step ten due to lack of statistics, samples and time. Step nine is the formulation of the QA plan, which is followed in this thesis. The steps are presented in Table 1.

Table 1: The Ten Steps.

Description: The table presents the number of the step, the name of that step and the

implementation of that step. The only step which is not included in the table is the final QA plan.

Step Name Implementation

1. Determine importance of the QA system considering cost and effort

Due to this being a thesis and no cost will be made, the QA system is optimal.

2. Identifying the controlling properties

Due to this being a thesis and no cost will be made, the QA system is optimal.

3. Selecting a method of testing The method of testing will be Unit tests if there is time. Testing should focus on achieving reliable results, where accuracy is preferred over precision since inaccurate results on average may present an issue in the future.

4. Defining acceptability limits Demand 1:

(19)

10

name is available, the name directly under the resources is not acceptable however.

Demand 2:

- The context log and message log should exist under the resource within “actions”.

Demand 3:

- The retry policy should exist under the resource within “actions” and should be set to “none”.

Demand 4:

- The ARM-Script should not refer to another ARM-Script nor component outside the test environment. Demand 5:

- Direct and loose connections are required; loose connections include servicebus.

5. Identifying reasonable risks Main risks is inaccurate results which may lead to time spent on troubleshooting and correcting the values. If the values are correct but present themselves as incorrect, it will be a waste of time for the developers.

7. Handle rejected lots The components that do not conform to the demands will be presented in a task list where development teams will rework the items.

8. Defining the contractor’s Quality Control responsibility

(20)

11 Step 9. Formulating the QA plan:

The QA system uses a software which detects the errors and will be implemented at the most practical stage of the development phase with consideration into performance and the simplicity to make changes to the software. The controlling properties are determined as the ARM-Scripts for the PaaS component configurations. The method of testing is based on accuracy to achieve correct results on average for all of the provided ARM-Scripts, unit tests will give the accuracy values. Unit tests however are not mandatory, it is only

required if there is time to implement it. The functional and non-functional demands are as follows:

- Demand 1: No underscores in the component names where the names are collected from the parameters which the component refers to.

- Demand 2: All components with “endpoint” in their names should have a context log and a message log existing under the components actions.

- Demand 3: All components with “endpoint-to” in their names should also ensure that the retry policy is set to “none”, which exists under the components actions. - Demand 4: No component nor ARM-Script should be able in any way

communicate outside the test environment. This includes both direct connections and loose connections such as references or service busses.

- Demand 5: The output should give an overview of the communications for each component and ARM-Script in order to see the effects of the errors.

Existing risks are inaccurate results due to errors beyond the contractor’s responsibility such as networking issues, updates in Azure etc. Components that does not conform to the demands 1-5 will be reworked by the development teams. The contractor shall test the software and produce accurate results on average based on the demands previously stated, exceptions for this includes only ARM-Scripts outside the test environment and external issues such as networking, updates etc.

4.4 Developing the QA Software

The software is designed with C# in VSCode and deployed onto Microsoft Azure, it is a script that runs through an http trigger and returns the results from all ARM-Scripts. VSCode was chosen due to the popularity and the simplicity of the IDE to install plugins. The language C# was recommended from Sogeti, which is why it was the chosen language. Sogeti uses Azure which is why it was the selected cloud service provider, since the

solution is aimed towards their PaaS component configurations.

The tasks of the development cycle are as follows:

- Integrate to Azure Storage account, Azure Functions - Implemention of JSON.Net

(21)

12 - Create a list of classes

- Create an object model in Draw.io for the software - Develope the methods and classes in VSCode

- Create an algorithm that checks whether the ARM-script is proper according to the QA specifications

- Providing a log that reports the existing errors

4.4.1 Azure Storage Account and Azure Functions

Microsoft Azure plugins are available in VSCode (the chosen Integrated Development Environment) and allows the user to sign in directly from VSCode. A storage account in Microsoft Azure will host all of the ARM-scripts. The storage account is accessible through a connection string and by configuring the settings of Azure to give user access to download and upload files. The download method is further explained in 4.4.6.1.

The other use of microsoft Azure in the software is Azure Functions. A function in Azure is created that is triggered by an http request which in turn triggers all of the code that is necessary to perform the task. “Azure function core tools” needs to be installed in order to get the functions to work. The HTTP trigger class has by default all the necessary

implementations. The HTTP trigger class calls on the rest of the classes and returns a response message in the form of a JSON to the user.

4.4.2 Implementation of JSON.Net

The ARM-Scripts that are provided from Sogeti are in JSON format. The data in the scripts need to be parsed and deserialized in order to be processed. To do so, the package

JSON.Net is used due to their popularity amongst users. JSON.Net allows deserialization of JSON data with a few lines of code.

4.4.3 Software Development Model

(22)

13 4.4.4 List of Classes

Table 2: Class List

Description: The table below reveals the amount of classes, their name, their inheritance

relationships, a short description of what that class does and the name of the methods for that class. Purpose of this is to help the reader understand the object model in section 4.4.5.

Class Name Parent

class

Description Methods

HttpTriggerCsharp2 None It is the http

trigger class which triggers all the other classes.

Run

ReadJSON None Collects all the

results of the ARM scripts.

ProcessJSON

ProcessExternalComponents

GetErrorARM GetAllArmJSON

ArmDAO None Fetches the ARM

scripts from the storage account in azure and returns the highlighted results for each ARM-script.

Download

ProcessARM None Iterates through

the ARM-script in order to check for the demands and stores the results for each ARM-script in HighlightedResults . GetHighlightedResults MatchComponentName FindToken ReadARMComponents GetTopicBus GetQueueBus

ParametersARM None Stores the values

of the parameters of each ARM-script.

No methods in this class

HighlightedResults None Stores all the

components within the ARM-script.

EnvironmentCheck GetErrorComponents

ExternalComponents None Components that

are linked through an external id, such as a link.

GetARMInfo

GetSpecificedComponent

Components None Stores the

attributes of the component, its relations with other components and which

ARM-SetBasedOnComponentParameters

(23)

14

script it is currently at.

Connections Components It is a connection

component. Usually azures built in

components which only need an api to connect to. The connection

components can be for instance

“Azure-log-analytics”.

BusParams None Contains the

parameters for servicebus component.

ServiceBus None Contains the

connection name, path and the bus parameters.

Topic ServiceBus Represents a

servicebus of the type “Topic”.

QueueBus ServiceBus Represents a

servicebus of the type “queue”.

ErrorARM None Contains all

ARM-Scripts that has errors.

SetErrorComponents

GlobalMethods None Contains static

methods for all classes in order to download and parse json files.

DownloadJSONFile GetJSONText

(24)

15 4.4.5 Software Object Model

The figure below represents the object model for the software. It is divided into three chuncks: data processing, output results and servicebus. When a line is drawn from one item to a chunk, it implies that the item has relations with most of the items inside the chunk. In this way, it would be more clear for the reader instead of having multiple lines drawn everywhere.

(25)

16 4.4.6 Class Structure and Setup

This chapter explains the methodology for each class. It covers the most significant parts of the code and explains briefly the less significant parts of the code.

4.4.6.1 Azure Storage Account Setup – Class: ArmDAO

The Azure Storage Account setup consist of two parts, the coding process and the “non-coding” process. The “non-“non-coding” process includes downloading and configuring third party tools. The coding process can start only when the “non-coding” process has been done.

Non-coding process:

1. Download following extensions in VSCode: 1.1 Azure Account

1.2 Azure Storage 1.3 Azure CLI Tools 2. Create an Azure account

3. Create a Storage account in Azure

4. Create a new container with access level set to “Blob” 5. Upload the files into the container

6. Get the connection string from access keys in storage account

6.1 Save the connection string as an environment variable in Terminal or CMD

Coding process:

1. Create a dictionary which consist of a string and a JSchema object. 2. Create a list of strings where the keys of each JSchema object is stored

3. Get the environment variable for the connection string and store it in a variable “a” 4. Create a storage account variable “b” by parsing “a”

5. Create a cloud blob client variable “c” by using the method “b”.CreateCloudBlobClient() 6. Create a cloud blob container variable “d” by referencing “c” towards the name of the container.

7. Iterate through all the blobs in the container

7.1. Create cloud block blob variable “e” which contains “d” 7.2. Download the data and name from each blob

7.3. Store the name in the list of strings in step 2

7.4. Deserialize the data using JSON.Net and store it in a variable “f” 7.5. Add the name and “f” to the dictionary in step 1

(26)

17

4.4.6.2 ARM-Script Processing Results – Class: HighlightedResults, Components, Connections, ExternalComponents and ParametersARM

The highlighted results class is where all the data for one ARM-Script is collected. It contains the other classes Components, Connections and ExternalComponents. Besides the other classes, it stores the name for the ARM-Script, number of errors, different types of errors and has a method to collect the error components within that ARM-Script. The method is called “GetErrorComponents” and it iterates through the components in that class and checks the demands 1-5 (mentioned in 4.3).

The Components class contains: - Name of that component - Name of the parent ARM-Script - True or false values for demands 1-5 - List of related components

- The parameter path for that component - List of dependencies

- List of Service busses

The Components class has two methods, the first method configures the name of the component, parent ARM-Script name, if the name has underscores and the parameter path for that component. The second method corrects the relations list as explained below:

Figure 6. CorrectTheRelations method – Authors Figure

The Connections class extends the Components class. In that way, components can be separated from connections but still be included in the components list in the

HighlightedResults class.

External component is a reference to a component in another ARM-Script. The data for the referenced components exists in the other ARM-Script, but the component is accessed in the ARM-Script which has that external component. The reference default value itself looks like a link which points at the connected ARM-Script and the component.

(27)

18

- Name of the connected ARM-Script, name of the current ARM-Script and the name of the connected component.

- HighlightedResults of the connected ARM-Script

- Boolean variable which determines if the component is of the type Connection or not - Boolean variable which determines if the component is within the test environment or not - The connected component

In order to get the name of the connected ARM-Script and the name of the component. The default value is split by slashes (“/”) and stored in a list, then the list is iterated and checks whether the item has the value “workflows”, “connections” or “resourceGroups”. The class also has a method which returns the connected component in that ARM-Script by comparing the components in the connected ARM-Script and the name of the component. 4.4.6.3 Obtaining Servicebus – Class: ServiceBus, QueueBus, Topic and BusParams The servicebus connections are available to collect in resources where the keywords “servicebus”, “topic”, “queue” or “send” is located in that resources triggers or actions. The servicebus data contains of the name of the connection and the path, servicebus also references to the parameters for that componenten. That is where the name and the id of the servicebus can be collected.

The BusParams class contains of “connectionId”, “connectionName” and “id”. The

ServiceBus class creates a BusParams component where the values are stored, besides those values, ServiceBus also stores the path and the referencename for the servicebus.

Topic and QueueBus classes extend the ServiceBus class and are made in order to

differenciate between a queue connection and a topic connection for future configurations and settings.

4.4.6.4 Processing the Individual ARM-Scripts – Class: ProcessARM

(28)

19

- ReadARMComponents: Divides the ARM-Script into different parts where one check for the parameters and the other checks the resources for the QA specifications. The parameters are iterated through in order to collect the

components name and to decide if it is an external component or not, if it contains “subscriptions” then it is an external component. The resources are checked to get the name of the component with the use of the method MatchComponentName. The depedencies are collected from the component and later matched with the correct connecting component, stored as a relation. The values for the context log, message log and the retry policy are collected with the method FindToken. When the data has been collected, the values in the component class are set and stored to a list of components which is then added to the HighlightedResults. In order to find the use of a servicebus within that component, the dependency list is used to reveal the number of times “servicebus” exists in that list. If the number is higher then one then the method GetQueueBus is called, if not then it is GetTopicBus that is called. - GetHighlightedResults: Simply returns the results of an ARM-Script in the C#

class HighlightedResults.

- MatchComponentName: Takes in a string “componentName” and a list of ParametersARM, it finds the ParametersARM where the componentName contains the ParametersARM path.

- FindToken: Takes in a JToken and a string which contains the path. It is a recusrive algorithm where as long as when using the built in method “SelectToken” returns null, it calls back on the method where it returns JTokens children. - GetQueueBus: The method first searches for the parameters for that servicebus

and collects the data in BusParams. When the data is collected, the other values are retreived where it goes through a numerous of conditions to ensure that there is no errors.

- GetTopicBus: The method is like GetQueueBus however the conditions differ since it searches for “Topic” instead of “Queue”.

4.4.6.5 Data Processing – Class: ReadJSON, GlobalMethods and ErrorARM

The class ReadJSON processes all of the ARM-Scripts at once in order to retreive correct connections between different ARM-Scripts and components. The class calls to ArmDAO where it retreives a dictionary of JSchema objects and keys for each object, it later loops through the scripts and uses ProcessARM in order to process the scripts individually. When all of the ARM-Scripts has been processed, the final stage is to make sure that the

connections are set up correctly. This is done with a method called

(29)

20

When that is done, the user have the option to download the output as a JSON file using the class GlobalMethods. The class has two methods, one to download JSON files and another to read downloaded JSON files. These methods are useful when the output JSON can´t be serialized due to an “Out.Of.Memory” exception for the JSON-Net library. If the user only is interested in the ARM-Scripts and components with errors, it is possible to call on the method GetErrorARM in ReadJSON. A new list is created and only the ARM-Scripts and components with errors are stored in that list. The new list is serialized and presented back to the user.

4.4.6.6 Azure Functions Http Trigger – Class: HttpTriggerCSharp2

The function in Azure is set up through VSCode, an http trigger is choosen where VSCode automatically produces a class which contain all of the needed code to run the http. In that class there needs a call to the ReadJSON class in order to trigger all of the other classes. When the software runs as normally, the output is the JSON file straight into the http as a response message. When the user only wants to see the results, the user types in

“?Name=<optionalString>” (the optionalString can be set up to any character) which triggers the method GetErrorARM in ReadJSON returns the output json straight into the http.

4.4.7 UML Diagram

The following figure is the UML diagram for the software. It visualizes what was previously stated. It shows all the classes, methods and items within the classes.

(30)

21

5 Results

This chapter presents the results of this study.

The findings of this study showed that the approaches for QA can either be through an industry standard or by developing own standards that is specifically aimed towards a company [3] [9]. Developing own standards can be done directly or by following a method such as Robert P Elliott´s ten steps [14]. This study used the ten steps to understand how to develop the software which caught all of the errors that was demanded. Other studies however have developed a QA software without the use of the Robert P Elliott´s ten steps and worked perfectly [19].

The developed software helped discover the amount of faulty ARM-Scripts and their components. Out of 102 ARM-Scripts, 93 of them had error configurations. Figure 8 presents the amount of error ARM-Scripts.

Figure 8. Amont of ARM-Scripts with errors – Authors Figure

The faulty ARM-Scripts had in total 1015 components where 402 components had errors. Figure 9 presents the amount of error/total components in the faulty ARM-Scripts.

Figure 9. Amount of Error/Correct Components – Authors figure

(31)

22

Figure 10. Loop to Calculate the Component Count – Authors Figure

Figure 11. The Count of the Total Components – Authors Figure

(32)

23

6 Discussion

In this chapter the chosen methods and the results are discussed and the strenghts and weaknesses of the work is brought up.

(33)

24

The error-configuration detecting QA software fullfills it´s purpose. The final results of the software presented that 38,46% of all the components was incorrectly configured, 91,17% of all the ARM-Scripts had incorrectly configured components. The software is aimed for late QA implementation but can also be adapted to early QA if it is merged with error-handling software in order to prevent wrong configurations. Early QA implementation would obviously have been more efficient to prevent the problem from occuring but the problem was not discovered early.

While this thesis was only limited to Sogeti, future research can compare the approaches with multiple case studies, visualizing the qualitative and quantitative effects of QA and the differences/similarities.

(34)

25

7 Conclusions

This chapter presents the major results of the thesis and aims to answer the research questions. Finally, suggestions for futrther research are presented.

7.1 How can QA be implemented within private companies?

The methods to implement QA can be to conform to an industry standard such as ISO 9000 or to develop own standards, which can be executed by a software. The most viable choice depends on favored outcome, if the QA specifications are very specific towards one company then a QA software might be the most suitable alternative. If the QA

specifications are more general, an industry standard might be more suitable. Once an approach has been selected, the development/implementation of the solution can start. In this thesis developing own standards was the chosen approach. The QA specifications was developed using Robert P Elliott´s ten steps (mentioned in 3.2) which was the foundation for the development of a software. The software detects error configurations within the PaaS components configurations of Sogeti. The software can be used to achieve QA when implemented as a firewall to prevent faulty components from reaching the end customer.

7.2 How can QA of PaaS components configurations help deliver common application demands?

The final prototype of the error-configuration detecting QA software, meets all the

established requirements. The software allows the user to ensure that faulty components do not reach the end customer when implemented correctly, thus meeting the common application demands. The software however is hardcoded with the exact specifications that Sogeti requires, thus limiting it´s scalability. The limitations can be fixed if the

(35)

26

7.3 In What Stage of the Development Process Should QA be Implemented Within Private Companies?

For this case study, the approach is late QA implementation. Though early QA might have been better to ensure that the faults does not appear in the beginning, the faults was not discovered in the beginning. Early QA might be hard to achieve because the faults and defects might not be clear in the beginning. New problems and defects might also arise with new functions, updates etc. which might not be considered until they present a problem. However, it is optimal to strive for early QA for known mistakes that might occur. QA in every step in the development phase would seem to be the best way to implement QA, but it can take time and resources. For problems that has not been presented yet, the stage of QA implementation might not matter since the problems are unknown.

7.4 Future Research

(36)

27 8

References

[1] H. M. Sneed and A. Merey, "Automated Software Quality Assurance," IEEE TRANSACTIONS

ON SOFTWARE ENGINEERING, Vols. SE-11, no. 9, September 1985.

[2] International Organization for Standardization, "Quality management systems — Fundamentals and vocabulary," September 2015. [Online]. Available:

https://www.iso.org/obp/ui/#iso:std:iso:9000:ed-4:v1:en. [Accessed 02 April 2020]. [3] E. V. Emden and L. Moonen, "Java Quality Assurance by Detecting Code Smells," IEEE, 29

January 2003.

[4] S. Lleshi, "The Effectiveness of QMS Implementation in Applying of Quality Health Care for Patients in Health Institutions of Kosovo," European Journal of Interdisciplinary Studies, vol. 2, 30 Augusti 2016.

[5] C. Denger and T. Olsson, "Quality Assurance in Requirements Engineering," in Engineering

and Managing Software Requirements, Springer, Berlin, Heidelberg, 2005, pp. 163-185.

[6] S. Veloudis, I. Paraskakis, A. Friesen, Y. Verginadis, I. Patiniotakis and A. Rossini,

"Continuous Quality Assurance and Optimisation in Cloud-Based Virtual Enterprises," IFIP

Advances in Information and Communication Technology, vol. 434, 2014.

[7] I. Bojanova and A. Samba, "Analysis of cloud computing delivery architecture models,"

Workshops of International Conference on Advanced Information Networking and Applications, 5 May

2011.

[8] D. Agarwal and S. K. Prasad, "AzureBench: Benchmarking the storage services of the Azure Cloud Platform," 26th International Parallel and Distributed Processing Symposium Workshops & PhD

Forum, 2012.

[9] C. M. Fuentes, F. B. Benavent, M. A. E. Moreno, T. G. Cruz and M. P. del Val, "Analysis of the implementation of ISO 9000 quality assurance systems," Work Study , vol. 49, 1 November 2000.

[10] M. Tuteja and G. Dubey, "A Research Study on importance of Testing and Quality Assurance in Software Development Life Cycle (SDLC) Models," International Journal of Soft Computing

and Engineering (IJSCE), vol. 2, no. 3, July 2012.

[11] N. Walkinshaw, "What Is Software Quality, and Why Does it Matter?," in Software Quality

Assurance. Undergraduate Topics in Computer Science, Springer International Publishing AG 2017,

2017, pp. 7-21.

[12] A. Borning, "COMPUTER SYSTEM RELIABILITY AND NUCLEAR WAR," Communications

of the ACM, vol. 30, p. 112–131, February 1987.

[13] J. M. Frederick and G. E. Hammond, "Maintaining quality assurance within software evolution: Lessons learned with PFLOTRAN," in SIAM Conference on Mathematical and

(37)

28

[14] R. P. Elliott, "Quality Assurance: Specification development and implementation,"

Transportation Research Record 1310, 1991.

[15] K. Beck and M. Fowler, Refactoring: Improving the Design of Existing Code, Boston, MA: Addison-Wesley, 1999.

[16] G. Carutasu, C. Botezatu, M. A. Botezatu and M. Pirnau, "Cloud computing and windows azure," in 2016 8th International Conference on Electronics, Computers and Artificial Intelligence

(ECAI), Ploiesti, 2016.

[17] G. Edward P, D. M. Ruiz and J. K. Ubaque, "Decision Model to Deploy IoT Solutions on Cloud Computing Based Platforms," in Proceedings of the International MultiConference of Engineers

and Computer Scientists 2017 Vol II,, Hong Kong, 2017.

[18] S. Venkatasubramanian, "serverless360," 28 January 2020. [Online]. Available: https://www.serverless360.com/blog/azure-service-bus-queues-vs-topics.

[19] R. M. Weed, "Development of M ulticharacteristic Acceptance Procedures for Rigid Pavement," Transportation Research Record 885, pp. 25-35, 1983.

[20] L. Zhao and S. Elbaum, "Quality Assurance under the open source development model," The

Journal of Systems and Software, 15 April 2002.

[21] L. Filion, N. Daviot, J.-P. Le Bel and M. Gagnon, "Using Atlassian tools for efficient requirements management: An industrial case study," in Annual IEEE International Systems

Quality Assurance of PaaS Components Configurations

AKADEMIN FÖR TEKNIK OCH MILJÖ