Research questions synthesis

study in Paper VI it can be concluded that the solutions are worth maintaining in RiskUse and further evaluate. The findings from Paper V show that usability testing can contribute to the software risk management process was further strengthened in this case study.

Concerning further work, it includes addressing the identified improvement proposals as well as further evaluation of the risk management process. The two last phases, the monitoring phase and the completion phase and need to be evaluated and require an evaluation over time and in addition the whole risk management method. To fully understand and evaluate the possibilities and potential benefits of RiskUse, the process has to be used over time and in a complete project from the start to the end. Broad generalisations of the results can therefore not be made since this is the first evaluation of the risk management process, only involving one organisation. However the case study shows that the risk management process is applicable and the positive results provides a strong argument to continue the evaluation and to promote the risk management process, RiskUse.

Research contribution

RQ2: What differences can be identified between the users of a system and developers of a system with respect to risk identification?

Two different experiments were launched, looking at differences with respect to risk identification. An experiment was conducted where physicians, developers and medical device developers were asked to identify risks (Paper II). According the number of identified risks, developers identified a larger amount of risk per person than physicians.

It was also shown that the physicians did not identify any risks that are specific medical risks. The other two groups could have identified these risks as well. Another difference identified between the groups where with respect to which risks they saw as important. In the experiment described in Paper III the results show a difference with respect to the perceived importance of the risks but it is not possible to state that any role is more risk seeking than any other role. To conclude from both experiments, there is a difference with respect to the perceived importance of the risks and Paper II indicates that different experiences, will affect the risk identification process.

RQ3: How can different people’s risk tendency be defined in an adequate way with respect to conception of risk, in order to support the risk management process?

Utility functions were used in a controlled experiment to investigate different people’s opinions about the importance of identified risks (Paper III). With the help of utility functions it is possible to discuss whether different individuals act as risk-averse or risk-seeking, and the experiment showed that there is a difference with respect to the perceived importance of the risks. It is possible to generalise this and conclude that different people in the software engineering process are more or less risk seeking and it can be concluded that a risk management process methods for assessing the level of risk tendency are available, but in most cases it is enough to be aware of the differences.

Based on the differences between different people with respect to the perceived importance of a risk it can be concluded that it is necessary to involve multiple persons in the risk assessment process in order to get a more accurate risk assessment. It is also shown that people are more or less risk seeking and by having a risk management group with multiple participants, preferable with different roles, the group will probably consist of both risk seeking and risk averse participants. If the risk

manager is aware of the differences and tries to balance the group setting it would lead to a more accurate risk assessment.

RQ4: How can users be integrated in the risk management process?

To comply with regulatory requirements, get an improved risk management process and incorporating usability engineering into the risk management process the user needs to be a part of the process. The users and their perspective have been incorporated in the risk management process in different ways. In this thesis users are integrated by the use of use cases as input in the risk identification process and users as participants at risk meetings (Paper IV and VI), and by performing usability testing as part of the risk management process.

The use cases (named user scenarios in Paper IV) are perceived as easy to work with and the use of them makes the risk meeting participants feel safe and secure in the discussions. The discussions are also focusing on the right things, saving effort and time.

Users attending the risk meeting also bring the user perspective into the process since users contribute to the discussions with their domain knowledge. Paper IV shows that the user representatives dominated the discussions whereas the developer representatives held a lower profile. To address this dominance, every one of the participants can explicitly be addressed, which would rule out the dominance during the discussions.

This was done in the case study presented in Paper VI. However the discussion may be affected by participants’ personality.

Usability testing focuses on the end users and that will bring in a new category of users into the process and new perspective on the use of the medical device. Usability tests can indicate risks that are not identified in the risk management process and give a possibility to verify if risks with high risk value actually cause the presumed problems.

RQ5: How can usability evaluation methods, especially usability testing contribute to the risk management process?

The results from the case studies in both paper V and VI show that usability tests can give valuable input to the risk management process.

The usability tests indicate risks that are not identified in the risk management processes and they also catch problems that are good risk candidates, where the functionality is unclear to the users and where the developers and the users have different mental models.

Research contribution

The usability tests can, for example, verify that a risk with a high risk-value actually is a problem for the users before any changes are made.

Risk values are assumptions, so if they can be identified in additional ways before any action is taken, effort and time can be saved by the development organisation, due to the avoidance of unnecessary changes RQ6: How can a software risk management process including user perspective be designed to be appropriate for a medical device development organisation?

The main goal with the risk management process RiskUse is to provide practitioners, mainly risk managers, with a software risk management process that has a well defined user perspective, is easy to apply and including hands-on recommendations on how to use the process. The goal is to integrate users and user perspectives in the software risk management process and to introduce usability testing, as an integrated part in the risk management process contributing to the goal of integrating users. Three case studies (Papers IV, V and VI) examined the risk management process and how it can be tailored to incorporate users and user perspectives. The three first steps, risk identification, risk analysis, and risk planning including use cases and user participation at risk meetings were studied in Paper IV. It was concluded that the used risk process was considered effective and easy for new personnel to adapt to. The results in Paper V and VI show that usability testing contributes in a positive way to the risk management process. RiskUse was evaluated in a case study (Paper VI) and in conclusion, RiskUse was found to support the practitioners within the medical device domain in their work with risks and risk management including users and user perspective.

RiskUse needs to be further evaluated and requires an evaluation over time to further identify possible improvement, fully understand and evaluate RiskUse.