BACHELOR THESIS
Tools and Processes for Increased Quality in Software Development Projects
How does Automation in a Development Environment Affect the Quality of Software Developed?
Daniel Hambraeus 2014
Bachelor of Science in Engineering Technology
Computer Game Programming
Abstract
Automation is currently popular for companies as a time saving measure. Its less known how automation impacts quality. This report focuses on automation in a developer environment, and how it impacts the quality of the work performed.
The type of automation in focus is continuous integration and scripting.
Sammanfattning
Automatisering ¨ar ett popul¨art s¨att f¨or f¨oretag att spara utvecklingstid. Fr˚agan
¨
ar hur automatisering p˚averkar kvalit´en i programvaran. Den h¨ar rapporten fokuserar p˚a automatiseringen i en utvecklingsmilj¨o, och hur automatiseringen p˚averkar kvalit´en hos den programkod som produceras. Den typ av automatis- ering som kommer att behandlas ¨ar continuous integration och scripting.
Contents
1 Introduction 1
1.1 Research Question . . . 2
1.2 Goal and Purpose . . . 2
2 Background 3 2.1 What is quality? . . . 3
2.2 What is software automation? . . . 4
2.3 Continuous Integration . . . 4
2.4 Jenkins . . . 4
2.5 Unit Tests & Test Driven Development . . . 5
2.6 Scripting . . . 5
2.7 Trac & #define . . . 5
2.8 Python . . . 5
2.9 HTML & CSS . . . 5
2.10 Genshi . . . 6
3 Method 6 3.1 Semi-structured interviews . . . 6
3.2 Work on the plugin . . . 6
4 Result 7 4.1 Interview Results . . . 7
4.2 Practical Results . . . 8
5 Discussion 9 5.1 Social, Ethical and Environmental Considerations . . . 11
6 Conclusion 12 7 References 13 A Appendix - Interview Transcript 15 A.1 Interview with Anders Abrahamsson . . . 15
A.2 Interview with Christer Ekholm . . . 16
1 Introduction
CGI in Ume˚a have an internal development team that focuses on developing their own application and development management tools. The team uses open source technology, and their product #define is based on the open source prod- uct Trac. The service assists agile development.
Quality of work is important in software development, as is how much time is spent working and maintaining the product. Automation is an approach to minimize time spent working on maintenance and the number of bugs. The focus of #define is to maintain traceability and automate workflows as much as possible. Its goal is to integrate all the tools to save time, and log everything.
One type of automation is continuous integration. Continuous integration is used to schedule and automate builds, for example running daily builds, and presenting reports on whether the builds were successful. There are other possibilities as well, chaining builds together, running scripts before and after the build. Jenkins is a system for continuous integration, that lets the user schedule and automate builds, it is the system that has been used in the practical parts of this project. There are other approaches for automation, and one other that will be brought up in this report is scripting, using bash scripts, powershell script or similar. Scripting is to run several commands in a certain order, automating a process.
The practical part of this project focuses on developing a plugin for Trac.
The plugin integrates Jenkins with Trac. It would eliminate manual labor, log- ging Jenkins output directly in Trac and it would remove the need of managing a separate program. The #define team will later on adapt the plugin for the
#define environment.
The main part of the project will be theoretical, to study the effects of automation. How automation affects development, in terms of software quality.
1.1 Research Question
I will study how automation affects the development of software, in terms of quality of the end-product. The research question is formulated the following way:
”How does automation in a development environment affect the quality of soft- ware developed?”
The types of automation that will be examined are scripting and continuous integration. The software is the end result software, the one produced by the developers.
1.2 Goal and Purpose
The purpose of this project is to show how automation might affect the quality of the software in a development environment. The main focus is to find the link between automation and quality, if there even is one. The project also has a practical part, where an implementation of automation is tested in practice as part of the research on automation.
• Run a Jenkins server for continuous integration
• Script the build and deploy
• Integrate Jenkins with Trac, manage Jenkins from Trac
• The theoretical sides of automation
• Conduct semi-structured interviews for information about automation
2 Background
To answer the research question, quality and automation need to be defined since they are very wide concepts.
2.1 What is quality?
ISO 9126-1 Quality Model from 2001 defines an international standard for qual- ity, based on several different parameters. Botella et al(2004)[1] made a write up on the standard, and evaluated how it could be used in practice. Quality is evaluated from six different parameters. The parameters are the following:
Functionality; ”The capability of the software product to provide functions which meet stated and implied needs when the software is used under specified conditions.”[2]
Reliability; ”The capability of the software product to maintain a specified level of performance when used under specified conditions.”[2]
Usability; ”The capability of the software product to be understood, learned, used and attractive to the user, when used under specified conditions.”[2]
Efficiency; ”The capability of the software product to provide appropriate per- formance, relative to the amount of resources used, under stated conditions.”[2]
Maintainability; ”The capability of the software product to be modified. Modi- fications may include corrections, improvements or adaptation of the software to changes in environment, and in requirements and functional specifications.”[2]
Portability; ”The capability of the software product to be transferred from one environment to another.”[2]
Each of the six characteristics also have sub-characteristics, which are dif- ferent aspects of the main characteristics. The quality of software is evaluated using different sets of internal and external metrics.[2]
An example for how the choice of evaluation would be like;
”A maintainer could evaluate software product using metrics for maintainabil- ity.”[2]
And there are different types of demands on the metrics depending on the goal;
”To be objective, there shall be a written and agreed procedure for assigning the number or category to the attribute of the product.”[2]
Botella et al(2004) expands on the metrics, and uses a goal-question-metric(GQM) approach.[1] Goal is purpose, issue and viewpoint, which are different perspec- tives and define the metric. The metric itself is the answer, and what it is evaluated based on. Each question creates one metric.
Example from Botella et al(2004):[1]
Goal; Purpose: Have an appropriate. Issue: Category variety. Viewpoint: Ada
community.
Question A; Has the library the most frequently used categories of containers?
Question B; Has the library a robust proposal of categories of containers?
Metric A; % of basic categories.
Metric B; 100 - % of unnecessary categories.
2.2 What is software automation?
Georgakopoulos et al(1995) define automation as the following;
”With the introduction of information technology, processes in the workplace are partially or totally automated by information systems, i.e., computer programs performing tasks and enforcing rules which were previously implemented by hu- mans.”[3]
To support a workflow the computer environment has many requirements on it. For an overall improved workflow an automated infrastructure is required.
In an automated infrastructure manual work done by humans are instead done automatically by a computer. Human labour is still required before and after the automated process.[3]
In an office environment increased amount of automation leads to improved quality of the work and communication. Compared to a less automated office environment, the stress level of the workers is also lower. Many work processes have been shown to be possible to streamline and automate which limits the amount of human error. By automating the office system the productivity improves as well.[4]
2.3 Continuous Integration
Continuous integration (CI), is a development practice where developers check in and integrate their work up to several times per day. The idea is that each check-in is verified by an automated build. All developers of a product share a single repository and make the build self-testing. The build should be done on a build-server so everyone builds on the same place. A goal is to keep the build fast and automate deployment. An advantage of this is that anyone can see what is happening and it is easy to get the latest binaries.[8]
2.4 Jenkins
Jenkins is a piece of software for Continuous Integration. It is hosted on a server developers can connect to and each project adds its own job which is a set of configurations, scheduling, logs and data. Builds can be started manually, scheduled or triggered from another build. Each build is saved, and it is possible to examine the output of all past builds. Its also possible to run automated tests.
Jenkins also has a REST API. It is possible to access the Jenkins back-end through other means than its front-end. The rest API also has wrappers made for Ruby and Python.[9]
2.5 Unit Tests & Test Driven Development
Unit tests is when a small part of the code is tested, to control that piece of code works. Test driven development is a way of working when the unit tests are written before the code, and the code that is written is written to fullfill the tests. What code to write is defined by the tests. Unit tests can be used to determine if there are errors in the code or to ensure the code is clean.[10]
2.6 Scripting
The type of scripting referred to in this report is bash scripts, power scripts and similar. The scripts are used to run any number of terminal command and in a set order. This types of scripts are then run from automated services, or are manually executed.[11]
2.7 Trac & #define
Trac is an open source issue tracking system for software development projects, with support for agile development. Trac has support for tickets, user manage- ment and a wiki. Plugins are coded in Python, Genshi and HTML, and are added as Python Eggs to Trac.[12]
#define is based on Trac and is developed in-house at CGI. It is a commercial service sold by CGI but mainly used internally at CGI. #define have added levels of authorization and other enterprise features for Trac. It uses similar types of plugins as Trac, and plugins made for Trac can be converted to #define.[13]
2.8 Python
Python is an interpreted programming language, which means it is run on the spot as long as the machine has support for Python. Python is a fairly popular language, since it is interpreted it is naturally working cross-platform. Python has soft typing, which makes it easy and fast to code in, and overall has very short syntax.[14] Especially when used together with Pep8, which is a python coding standard to increase readability. Higher readability saves time when a programmer is to read the code, both when coding and debugging.[18] There are two branches of Python currently in use, the 2.X branch usually 2.7 and the 3.X branch. 2.X is the standard in Mac and Linux, and it is legacy. 2.X is the version being used by both Trac and #define, the differences between 2.X and 3.X makes an upgrade difficult, which is the reason it is still commonly used.[19]
2.9 HTML & CSS
HTML (hyper text markup language) is a language used to create static pages that are meant to be presented in a web browser. It provides the possibility to create links, tables, buttons and overall create a page.[15] CSS is a language for describing presentation of web pages. CSS is independent of HTML and can be used with any XML-based markup language. The separation of HTML and
CSS makes it easier to maintain sites, share style sheets across pages, and tailor pages to different environments. Almost everything on the web uses HTML at some level, and generally together with CSS.[16]
2.10 Genshi
HTML alone provides the same static page every time, genshi allows generating HTML dynamically from templates. Genshi templates uses a combination of HTML and Python. The python program generates HTML code, which allows for dynamic parts. Genshi is developed by the same developers that maintain Trac, and Trac uses Genshi.[17]
3 Method
3.1 Semi-structured interviews
The interviews are conducted in a semi-structured way, with premade questions asked during the interview, but the orders and which questions asked depend on the situation. Semi-structured interviews are well suited to explore attitudes, values and beliefs of the person being interviewed. It also ensures all questions are answered, and that it is only the person asked that answers the questions, without any assistance from others. Other gains from semi-structured interviews is that they are not completely strict, so the interviewers can choose to word the questions differently if need be, or probe deeper on a certain topic. [5]
Semi-structured interviews are commonly used with qualitative research.
The process is generally organized around several predetermined open-ended questions, and then if questions emerge during the dialogue those are asked as well. Semi-structured in-depth interviews are the most widely used interviewing format for qualitative research.[6]
Semi-structured interviews was chosen for the reasons mentioned above, and because at the beginning of the project a hypothesis was yet formulated. Be- cause of time constraints, it was not feasible to do a quantitative study, since much work would be required to create a hypothesis before a quantitative study could be conducted.
3.2 Work on the plugin
The plugin is written for Trac, while the company have their own system; #de- fine. The plugin is made for Trac of several reasons. Trac is open source, so it is much easier to share the plugin if the plugin is made for Trac. The two systems are similar enough so the company can use the plugin as a basis #define plugin.
#define is also more complex than Trac, so developing a plugin for Trac is faster and easier. #define have a complex login and authorization system which would have had to be taken into consideration.
The developing operating system for the plugin was Linux, since the entire
#define-team is working in Linux. Linux is free and open source, and the team is focused around developing in open source environment. It has all the relevant software, and it is free to use. The editor of choice for the project is Vim. Vim is an efficient editor for the experienced user, it has many plugins that can help increase productivity.[7]
The plugin is written in Python 2.7. Later version of Python like 3.X was not chosen, because Trac uses 2.X, and there would be compatibility issues if 3.X had been used.
To create HTML pages Genshi templates are used, they are a part of many Trac plugins since they allow dynamic content. It is standard to use Genshi in Trac plugins.
In the beginning of the project a Jenkins and a Trac server was installed on a computer and accessed through local host. This was the simplest and most efficient way of working. By setting up Trac and Jenkins I also got basic under- standing for how the two programs worked.
The whole plugin is compressed into a python egg, using Python setup tools.
It is the method which Trac uses for packaging and compression. The egg is installed on the Trac-server using a file browser. Every time the plugin was changed, the server had to be shut down, the plugin manually removed, the server started, and the plugin reinstalled.
To access and test the plugin, the browser Firefox was used. To speed the process up, it was scripted. Running a script, instead of inputting several commands is much faster and minimizes chance of errors.
4 Result
4.1 Interview Results
In the interviews high quality was described as what is wanted, what is useful and as everything that works well. If all works as intended, and all functionality is desired the software is of high quality.
High quality software was said to save time. Software of high quality can include functionality that educates the user. In such cases high quality means that the software helps the user grow and learn, and helps the user develop software of high quality.
According to the interviews low quality can affect time consumption, make communication worse and even introduce bugs and errors. Most of all it causes frustration. In the interviews it was implied frustration negatively affects qual- ity of work and productivity. Worst case scenario was described as leaving the company due to frustration over low software quality.
From the interviews it was gathered that the most important quality of soft- ware, was that the software at all times worked, that it was reliable and correct.
Second most important is functionality, what can actually be done with the program. Usability is third most important, a good program should be easy to use and easy to learn.
When it came to the interviewee’s experience with automation the impres- sion from one of the interviewees was that automation is important, but priori- tized too low. It was also felt that the competence required for automation to work smoothly had been lacking. According to the interviewee automation is not used enough because it is poorly understood. The interviewed people had mixed feelings about continuous integration. One felt it was inevitable that a build failed. He thought it could be improved with better control over what code is accepted. The other thought continuous integration was great, but required good unit tests that were maintained.
One of the interviewed explained his positive experience with scripted de- ploys. The deploys were done manually before, but a series of scripts was implemented that backed up the system both before and after the deploy. The scripts made the deploy run more smoothly. Overall this saved large amount of time, on average time went down from four hours to about half an hour, and in the most extreme case it went down from ten hours to about three hours. After the changes all customers used same binaries, the difference between customers is in unique configurations.
Overall there was higher reliability, with proper backups that let the worker roll the deploy back and forward in case of problems. It lead to better tracing, since you could compare the system before and after the deploy. It also leads to higher reusability, with customer specific configurations being the only difference between different installations.
The effect was that the number of bugs went down, the time consumption went down and overall the customer satisfaction was increased.
4.2 Practical Results
During the project Jenkins was integrated into Trac. The plugin lets the user access Jenkins directly from Trac and allows the user to run builds, remove jobs or builds, and read the logs for all builds. Overall the practical work was meant to be a step in the direction of deeper understanding of how automation worked, and part of the research on how automation impacts a project, and how automation works in practice. The outcome of this research was a deeper un- derstanding of automation, required for further research and better knowledge when diving into the theoretical area. The goal was to improve understanding and get experience of how continuous integration works.
Each time the project was built several manual tasks had to be performed.
Using easy install from python the plugin was made into a python egg. Installing the plugin using the installer in the Trac server. Every time a new plugin was built the server had to be shut down, the plugin manually removed, and then the server started again.
The egg-build is one command, the server shutdown one command, removing the plugin one command, restarting the server one command, installing the plugin requires clicking three different buttons, and using a file browser to add the plugin. Also many of the commands had to be run from different folders.
Everything related to the trac server was in one folder, with a subfolder for the plugin, the egg was built in the plugin folder. This means either there had to be separate terminals for each of those, or there had to be folder switching between all the commands. Overall the whole build process is estimated to have taken around one and half minute.
At first a script was implemented to start the server and remove the old plu- gin, this saved one folder switch, and one command to run. It is estimated to save roughly twenty to thirty seconds. Then a script to execute the build from same folder as the server was run from was made, and automatically moved the finished plugin into the right folder, this script was added to the server-start script.
Overall the time consumed by the build process went down from roughly one and a half minutes to about ten seconds, with most manual work eliminated.
Builds were performed on average a hundred times per day, the script was saving overall two hours of work per day, and lead to a less frustrating and less repetitive workflow.
5 Discussion
For the work on the plugin most choices came down to following the standard.
Plugins for Trac uses Python, Genshi and HTML. None of those choices really was made by me, as much as I followed and tried to fit in with the rest. There was one major choices for the plugin, the choice of using the REST API for Jenkins to connect to Jenkins. Other options would be to instead add a plugin to Jenkins to send the data to Trac, and make the plugins interact. Using the REST API felt like the simplest way of doing the job, with direct call-backs and no need to check over and over. It also didnt need any extra work done in Jenkins.
Based on the interviews Continuous Integration(CI) alone does not increase quality, it is only one piece in the bigger picture, without the other pieces CI does not help. CI require unit tests for the scheduled builds to be of value, to give feedback when errors are introduced.
CI is not a silver bullet that will fix all problems, it requires change of habits, and change of systems and overall workflow. In the interviews this was
brought to light, and while their personal views of CI was different, there was an agreement that control of what code was accepted was required. Unit testing was agreed to be a good way to control the code.
The downside of unit tests, are that they require to be constantly updated.
Outdated unit tests can give false positive errors or false negatives. Unit tests are not ”fire and forget”, they must constantly be maintained. Working with unit tests requires them to be incorporated into pipeline in a way that they are constantly revised, to ensure they stay up to date.
There are other forms of code control that also should be evaluated such as code review which was brought up during interviews as well. CI requires a strong tailored way of working, and you simply can not skip it. Without proper way of working with CI it leads to bugs, errors and frustration.
Does CI improve quality? It is hard to give a definitive answer, but it is likely in conjunction with other things that help improve quality. As mentioned before it can not stand alone, but is part of a bigger system. In the system it is part of it is harder to do wrong, and easier to detect what went wrong. If used incorrectly though; if the developers lack proper knowledge of it, it might cause more harm than good. Be prepared to do a proper introduction of it into the environment.
Scripting is the other big part of automation. Scripting seems to be much easier to do and there seems to be very few downsides to it. Biggest advantage is the time consumption, in my project I could save 90% of the time it took to do a build by scripting it, which saved me hours every day. In the project mentioned in the interviews, the time was brought down by roughly 85% during average deploys. There are also many other upsides to scripting solutions, it is much more reliable, with less causes for error. In the case of the interviewee’s project with a scripted solution they could add more logging and functionality.
Also with a shorter deploy time it meant less downtime for the customer. Based on the software quality standard there were quality improvements in reliability, maintainability and functionality. Based on the software quality standard there was improvement in quality from automation.
The chosen method of using qualitative interviews was highly educational and from my perspective very experimental. I had no clue what I was doing, but realized I couldnt fit in a proper quantitative study within the time-frame of the project. During the course of the project I learnt the value of qualitative studies, and what the purpose of them is. A good qualitative study is a great as a starting point for research, when the subject is not fully known.
Qualitative research is good because it doesnt matter as much how many instances you have of something, but the quality is what is important. You want to show there exist one instance of something happening, and find out as much as you can about it, and later do a deeper research and quantify to see how often it occurs.
Semi-structured interviews is a great way to obtain knowledge that is already there and write it together in a meaningful way. The reason they are great is
because the interviewer have very high level of freedom, and can adapt to the situation. I noticed halfway through one of my interviews that half my questions was impossible to utilize. Instead of asking them, I just winged it at that point and asked questions I made up on the spot. I tried to obtain as much meaningful information I could from the interviewee, on the things he did know about, even if I didnt have interview questions with me.
All that said I needed more interviews I felt, what I lacked was in the Contin- uous Integration-department. I wish I had the chance to interview a real master of continuous integration that had used it for years. During my interviews I got the feeling both the people I had interviewed had done one or two projects using it, and which is not overly deep experience of using it. Semi-structured inter- views quality is highly dependant on the quality of the people you interview.
And one interview can be enough to give you a good view of how it all works, but often several is needed to together give the complete picture.
Overall it feels like I have gained answers to the research question I set up.
Automation in a developer environment seems to have an affect on the quality of the end product. There have been cases where the quality have been improved.
How often and to which extent is out of scale for this project.
5.1 Social, Ethical and Environmental Considerations
Automation saves time, by saving time the computers can be used more effi- ciently which could possibly lead to lower energy consumption, which overall is a positive effect. This is not tested or proven, but its possible there might be positive environmental effects.
Ethical and social aspects are more relevant questions. Automation as a whole can have social impact. Automation means less menial work is done by humans, and this can have impact on society. The more automated the less work is left for humans, if enough things are automated there will not be enough work for humans. This would mean a reform of society would be required, since our current society is based on everyone working for most of their lives.
Is it right or not to let machines take over all human work, to the degree of humans not doing any relevant work any longer, is an interesting ethical question when it comes to automation. It is not within the scope of this project, and would require an entire project dedicated to it to give the question justice. This is not directly a relevant to my project, since it is only about one aspect of automation, but for automation as a whole, it is a highly relevant consideration to take.
Neither the ethical or social aspect of automation is brought up in this work though, since it is looking at how automation impacts quality in a software development environment, and only at that.
6 Conclusion
Automation in form of scripting has been shown to improve quality and reduce time consumption. Continuous integration can possibly give improvements in quality but requires changes in structure of how the work is conducted, and needs a workflow tailored to it. If it is used incorrectly it can have a negative impact on quality.
7 References
[1] P. Botella, X. Burgus, J-P Carvallo, X. Franch, G. Grau, J. Marco and C. Quer ISO/IEC 9126 in practice: what do we need to know?, Software Measurement European Forum 2004, p297-306, Rome, 2004.
[2] ISO/IEC 9126. Software engineering – Product quality, ISO/IEC, 2001.
[3] D. Georgakopoulos, M. Hornick and A. Sheth An overview of workflow management: From process modeling to workflow automation infrastructure, Distributed and Parallel Databases #2 p119-p153, April 1995
[4] M. Olson , H. Lucas, The impact of office automation on the orga- nization: some implications for research and practice, Communi- cations of the ACM #11 p119-p153, Nov 1982
[5] L.Barnball, A. While, Collecting data using a semi-structured in- terview: a discussion paper, Journal of Advanced Nursing 1994 #19 p328-335, 1994
[6] B. DiCicco-Bloom, B. Crabtree, The Qualitative research interview, Medical Education 2006 #40 p314-p321, 2006
[7] J. McPherson (2006) http://robertames.com/files/vim-editing.html Accessed: 2014-06-26
[8] M. Fowler (2006) http://martinfowler.com/articles/continuousIntegration.html Accessed: 2014-06-26
[9] R.Tyler Croy (2014) http://jenkins-ci.org/content/about-jenkins-ci Accessed: 2014-06-26
[10] S.Ambler (2013) http://www.agiledata.org/essays/tdd.html Ac- cessed: 2014-06-26
[11] P.Csaba (2013) http://code.tutsplus.com/tutorials/the- fundamentals-of-bash-scripting–net-32093 Accessed: 2014-06-26 [12] Edgewall Software(2013) http://trac.edgewall.org/ Accessed: 2014-06-
26
[13] CGI (2012) https://define.primeportal.com/define/pages/overview.aspx Accessed: 2014-06-26
[14] Python Software Foundation(2014) https://www.python.org/about/
Accessed: 2014-06-26
[15] R. Shannon (2014) http://www.yourhtmlsource.com/starthere/whatishtml.html Accessed: 2014-06-26
[16] K. Pearce(2014) http://www.diygenius.com/web-design- fundamentals-everything-you-need-to-learn-html-and-css/ Ac- cessed: 2014-06-26
[17] Edgewall Software(2012) http://genshi.edgewall.org/ Accessed: 2014- 06-26
[18] G.vanRossum, B.Warsaw, N.Coghlan (2013)
http://legacy.python.org/dev/peps/pep-0008/ Accessed: 2014- 06-26
[19] Python Software Foundation (2014) https://wiki.python.org/moin/Python2orPython3 Accessed: 2014-06-26
A Appendix - Interview Transcript
A.1 Interview with Anders Abrahamsson
Interview with Anders Abrahamsson, Software Architect, CGI, translated from Swedish.
What is quality?
To deliver to the customer what it wants, according to good design principles.
How does software quality affect your work?
Good software quality reduces the time spent, and the program might help you out with good design practices. ReSharper does that to example. This would improve the quality of your work.
Bad software quality leads to loss of time and frustration. Low functionality of the software does not affect the quality of your own work though.
Which are the most important properties of a program?
Functionality is a necessity; how well the program does what it should do, and what the program can do. Usability is really important; how easy the program is to use, and how fast you learn it. It should be intuitive and do what you expect it to do.
Talk about your experience with the introduction of automation.
Mainly worked with automation of build and deploy.
For build, we used continuous integration with daily scheduled builds. It worked poorly, often something went wrong and things broke. It was always just a matter of time before something broke. When something broke the trunk was broken, and you could not get binaries. Overall it worked badly. If continu- ous integration is to work well, some level of control on what is committed is required. The code needs to be tested continuously, or there needs to be code reviews. If that does not exist, then it all falls apart and it is risky to use continuous integration.
When it comes to deploy, the whole deploy sequence was scripted using powershell. It includes a back-up both before and after the deploy, which makes it easy to jump between the two, and reset things to how they was before. This worked extremely well, and saved loads of time.
What happened during the change?
Scripts with support for rollbacks make the deployment much easier, since you can roll back in case of mistakes. Time usage massively went down, from average of around four hours, down to about half an hour. The extremes when everything went wrong, went down from roughly ten hours, to around three hours.
Management of different customers is handled entirely with separate config- urations. All customers share binaries, but have their own unique configuration.
Overall higher quality and lower cost in time.
How was the quality affected?
Higher reliability with lower risks of anything going wrong, and you can roll the system back in case something goes wrong. There is also better tracing,
since you can see what it was like before and after the deploy. The reusability is also better, with configurations being the only difference between installations.
What was the effect on customer satisfaction, time consumption and number of bugs?
Number of config bugs went down, much lower time consumption. Overall this makes the customers more satisfied, and have higher level of trust. Less time spent means less money spent, and less bugs means more satisfied customers.
A.2 Interview with Christer Ekholm
Christer Ekholmc Test Manager, CGI, translated to english What is quality?
Quality differs from project to project. Quality is that everything works like it should in the program. It is what the receiver can work in, and find useful.
Which are the most important properties of a program?
Low quality can lead to frustration or even that you get tired of the company.
It can create small bugs in your own work. Low quality lowers your productivity, which leads to higher time cost.
For example, TFS has a burn down chart which lacks the functionality to re- move weekends from the chart, which makes it incorrect, since work only happen during the workweek. In communication within the team or with a customer, this thus gives a faulty picture of the project, which makes the communication worse.
Which are the most important properties of a program?
Correctness is the most important, the program should always work, and always do what it should. Usability is also very important, how easy you can learn the program and how easy it is to utilize all the functionality.
Talk about your experience with the introduction of automation.
Automation is important, but is prioritized too low in many projects. Com- petence with automation is also relatively low. The reason why automation is used too little is because it is poorly understood.
More unit tests are needed, but the knowledge about them is lacking. More unit tests would improve productivity. Continuous integration with builds every night is good with clear unit tests. It is important the unit tests are updated though, so they never get depreciated. If this is not done there will be false- positive answers from the unit tests, and the errors they raise are faulty.
