Modernizing the User Interface of a Legacy System at the Swedish Police Authority: Collaborative Mental Model: A New Participatory Design Method

Full text

(1)DEGREE PROJECT IN COMPUTER SCIENCE AND ENGINEERING, SECOND CYCLE, 30 CREDITS STOCKHOLM, SWEDEN 2019. Modernizing the User Interface of a Legacy System at the Swedish Police Authority Collaborative Mental Model: A New Participatory Design Method TUNCAY DAGDELEN. KTH ROYAL INSTITUTE OF TECHNOLOGY SCHOOL OF ELECTRICAL ENGINEERING AND COMPUTER SCIENCE.

(2)

(3) Modernizing the User Interface of a Legacy System at the Swedish Police Authority Collaborative Mental Model: A New Participatory Design Method. TUNCAY DAGDELEN. Master in Computer Science Date: July 14, 2019 Supervisor at KTH: Jonas Moll Supervisor at the Police: Sabina Åberg Hamrén Examiner: Elena Troubitsyna School of Electrical Engineering and Computer Science Employer: Swedish Police Authority Swedish title: Modernisering av äldre gränssnitt hos den svenska polismyndigheten.

(4)

(5) iii. Abstract IT-projects are complex and require a good understanding of the users before development starts. This is especially true for projects within large organizations, where the communication with the users might be poor or even nonexisting. Involving the users in the design process has shown positive effects and is becoming a trend throughout different IT-projects. However, there is some critique questioning whether users should act as designer. This thesis proposes a new method, the Collaborative Mental Model, with the aim of involving users in the design process without requiring any design skills. A case study was conducted at the Swedish police authority to test this new method. The underlying design process began with contextual inquiry and Card Sorting sessions in order to understand the users and the usability issues. After the problem was defined, the collaborative mental model was used to find inspiration, with the help of the users, for a design solution. The prototype was then created with this inspiration in mind. Finally, the users evaluated the prototype with the Think-aloud method. This study showed that the Collaborative Mental Model gave valuable insights about the users. These insights were translated into different features of the prototype. The evaluation showed that all the features were appreciated by the users. The collaborative mental model had a positive impact on the design process and thus, the user experience..

(6) iv. Sammanfattning IT-projekt är komplexa och kräver en god förståelse för användarna innan utveckling av systemet. Det är särskilt viktigt för stora organisationer att ta hänsyn till sina användare. Detta eftersom kommunikationen mellan användarna och den övriga organisationen kan vara dålig, eller till och med obefintlig i större organisationer. Att inkludera användarna i designprocessen har visat sig vara en framgångfaktor för en del projekt men kritiker hävdar att användarna inte bör, eller till och med inte kan, medverka i designen av ett system. Den här studien presenterar en ny metod, Collaborative Mental Model, som har till syfte att involvera användarna i designprocessen utan krav på tidigare färdigheter eller kunskap inom design. En fallstudie utfördes hos polismyndigheten i Stockholm för att testa den nya metoden. Studien började med kontextuella intervjuer och card sorting-övningar för att få en förståelse för användarna och deras behov. Detta åtföljdes av Collaborative Mental Model, vars syfte var att förstå vad för interaktioner användarna var vana vid sedan innan. Denna metod gav inspiration som sedan användes i skapandet av prototypen. Användarna fick i det slutgiltiga skedet evaluera prototypen. Denna studie visade att den nya metoden, the Collaborative Mental Model, bidrog med inspiration som införlivades i prototypen genom ett antal funktionaliteter. Resultatet från användarnas evaluering visade att samtliga funktionaliteter uppskattades. Det innebär att Collaborative Mental Model bidrog positivt till designprocessen och användarupplevelsen..

(7) v. We have to design systems for people the way they are, not the way we want them to be - Don Norman.

(8) vi. Acknowledgements I would first like to thank my group manager Malin Isaksson and my supervisor Sabina Åberg Hamrén at the Swedish Police Authority for giving me the opportunity to do my master thesis at the Swedish Police Authority. Your trust and support gave me invaluable strength. Thanks to the PAP-team for always being there for me. Further, I would like to thank Jonas Moll, my supervisor at KTH, for the careful reading of my report but even more for showing interest in the new method and helping me to shape it. Thanks to my examiner at KTH, Elena Troubitsyna, for improving the report with your feedback. A big thank you to the administrators in Stockholm and Gothenburg that participated in this study. You all showed unforgettable kindness. Thanks to my friend Ezeddin Al Hakim for the long discussions we had inside and outside the school. Finally, thanks to my family for your love and support..

(9) Contents 1. 2. Introduction 1.1 The Case . . . . . . . . . 1.2 Motivation and Purpose . . 1.3 Expected Scientific Results 1.4 Research Question . . . . 1.5 Delimitation . . . . . . . . 1.6 Report Structure . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. Background 2.1 Cognitive Science and Human-Factors Engineering 2.2 Human-Computer Interaction . . . . . . . . . . . . 2.3 User Experience Design . . . . . . . . . . . . . . 2.4 User Centered Design . . . . . . . . . . . . . . . . 2.4.1 Contextual Inquiry . . . . . . . . . . . . . 2.4.2 Card Sorting . . . . . . . . . . . . . . . . 2.4.3 Prototyping . . . . . . . . . . . . . . . . . 2.4.4 Think-aloud . . . . . . . . . . . . . . . . . 2.5 The Challenge of Legacy Systems . . . . . . . . . 2.6 Participatory Design . . . . . . . . . . . . . . . .. . . . . . .. . . . . . . . . . .. . . . . . .. . . . . . . . . . .. . . . . . .. . . . . . . . . . .. . . . . . .. . . . . . . . . . .. . . . . . .. . . . . . . . . . .. . . . . . .. 1 2 2 2 3 3 3. . . . . . . . . . .. 5 5 6 7 8 8 9 9 10 10 11. 3. Preliminaries 13 3.1 Research Design . . . . . . . . . . . . . . . . . . . . . . . . 13 3.2 The Collaborative Mental Model . . . . . . . . . . . . . . . . 14. 4. The Design Process 4.1 Phase 1 - Discover . . . . . . . . . . . . . . . . . . . 4.1.1 Contextual Inquiry . . . . . . . . . . . . . . . 4.1.2 Findings from the Contextual Inquiry sessions . 4.2 Phase 2 - Define . . . . . . . . . . . . . . . . . . . . . 4.2.1 Card Sorting . . . . . . . . . . . . . . . . . .. vii. . . . . .. . . . . .. . . . . .. . . . . .. 17 17 17 18 19 20.

(10) viii. CONTENTS. 4.3. 4.4. 4.2.2 Findings from the Card Sorting sessions . . . . . . . . 4.2.3 Summary of Phase 1 and 2 . . . . . . . . . . . . . . . 4.2.4 Integrating Manuals to the IAM System . . . . . . . . Phase 3 - Development . . . . . . . . . . . . . . . . . . . . . 4.3.1 The Collaborative Mental Model . . . . . . . . . . . . 4.3.2 Findings from the Collaborative Mental Model sessions 4.3.3 Prototyping . . . . . . . . . . . . . . . . . . . . . . . Phase 4 - Evaluation . . . . . . . . . . . . . . . . . . . . . . 4.4.1 Think-aloud . . . . . . . . . . . . . . . . . . . . . . . 4.4.2 Findings from the Think-aloud sessions . . . . . . . .. 20 22 23 23 24 26 30 36 36 36. 5. Results 39 5.1 SRQ 1 - The Usability Issues . . . . . . . . . . . . . . . . . . 39 5.2 SRQ 2 - The Collaborative Mental Model . . . . . . . . . . . 39. 6. Discussion 6.1 Discussion of the Collaborative Mental Model 6.2 Discussion of the Other Methods . . . . . . . 6.3 The Design Process . . . . . . . . . . . . . . 6.4 Sustainability and Ethics . . . . . . . . . . . 6.5 Future Work . . . . . . . . . . . . . . . . . .. 7. Conclusion 7.1 Sub-Research Question 1 . . 7.2 Sub-Research Question 2 . . 7.3 The Main Research Question 7.4 Suggestions for the Police . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . . .. . . . .. . . . . .. . . . .. . . . . .. . . . .. . . . . .. . . . .. . . . . .. . . . .. . . . . .. . . . .. . . . . .. . . . .. . . . . .. 41 41 43 44 45 45. . . . .. 46 46 46 47 47. Bibliography. 48. A Card-Sorting-Elements. 52. B The Sketches. 54. C Think-aloud tasks. 59. D The Prototype. 63.

(11) Chapter 1 Introduction IT-projects are complex and require a diverse set of competences from computer science. From the perspective of Human-Computer Interaction (HCI), one can not stress enough the importance of understanding the users. There are different approaches to understand the needs and behavior of the users. User Experience Design tries to connect with the users on a more personal level. The methods offered by this approach are more of the qualitative nature [1]. On the other hand, User Centered Design has an approach that is based on quantitative methods. Further, Participatory Design is an approach which focuses on involving the users in the prototyping phase of a design project [2]. Often, different methods within these approaches are combined in IT-projects and new methods arise in order to tackle different problems. A concern especially among big cooperations and governments is to modernize legacy systems. This is a technical challenge but also a challenge in terms of user behavior. The users are trained to use these kinds of systems and they might use these systems for several years or even decades. They gain experience and develop routines. Redesigning these systems will affect their routines and how they interact with the system. A bad design will have a negative impact on the user experience. Therefore, it is important to understand the users and their needs in order to make the modernization process as painless as possible. This degree project proposes a new method, the Collaborative Mental Model, which has the purpose of increasing the understanding of the users and thus, increasing the chance of a successful modernization of such kind of systems. A case study has been conducted at the Swedish Police Authority where the new method has been tested.. 1.

(12) 2. 1.1. CHAPTER 1. INTRODUCTION. The Case. The case study is about improving the user experience for the users working with the Identity Access Management (IAM) system at the Police. The IAM system is being used to authorize access to other systems at the government. There are mainly two user groups, those who apply for authorization, i.e. police officers and civilian personnel within the Swedish Police, and those who manage and approve/disapprove those applications, i.e. the administrators, see figure 1.1. The focus of the thesis project was to improve the user experience for the administrators.. IAM Police officers/Civil Personnel. Administrators. Figure 1.1: Application process. 1.2. Motivation and Purpose. Users and developers might have different understandings, or mental models, about how certain features within a system should work. When it comes to modernizing legacy systems, there are often a lot of things that could be changed. By understanding what a specific group of users is comfortable with, one could design the new user interface with respect to that in order to make these changes as painless as possible. Therefore, the purpose of the Collaborative Mental Model method is to make the users and developers collaborate in order to create a common mental model, thus the idea for the name Collaborative Mental Model.. 1.3. Expected Scientific Results. The expected result from this thesis is the demonstration of a new method, the Collaborative Mental Model, which considers the gap between legacy user interfaces and modern user interfaces and tries to make the switch between them as painless as possible..

(13) CHAPTER 1. INTRODUCTION. 1.4. 3. Research Question. This thesis has one main research question and two sub-research questions. The purpose of the sub-research questions was to brake down the main research question into two parts that together can answer the main research question. The main research question addresses the new method and its impact on the redesign process. The first sub-research question addresses the usability issues. The second sub-research question addresses how well the new method works with established methods from HCI. Main Research Question: How does the Collaborative Mental Model method impact the redesign process of a legacy system at the Swedish Police Authority? Sub-Research Question 1: What usability issues are defined by the users regarding the current legacy system? Sub-Research Question 2: How can the Collaborative Mental Model method be combined with the Contextual Inquiry, Card Sorting and the Think-aloud methods in order to solve the identified usability issues?. 1.5. Delimitation. The term "modernization" is broad and has different meaning depending on the context. This study will focus on modernization in terms of user experience: how is the current user experience among the administrators and how can it be improved? The focus will be on using proven methodology in order to understand user needs and address them. This study does not focus on technical modernization methods such as restructuring the code base or wrapping the current system with a software layer to hide complexity.. 1.6. Report Structure. This thesis is divided into six chapters. The Introduction chapter covers the overall topic of this thesis and the problem statements. The Background chapter starts with history of the Human-Computer Interaction (HCI) field followed.

(14) 4. CHAPTER 1. INTRODUCTION. by different approaches that has emerged from that field, e.g. User Experience, and also the specific HCI methods that will be used in this thesis. It ends with describing legacy systems. The Methods chapter presents the Collaborative Mental Model. Further, it presents the design process and its phases, what methods that were used for each of the phases and how the Collaborative Mental Model method was conducted. The Results chapter presents the results from each of the phases from the design process. The Discussion chapter analyzes the different methods used, in particular the Collaborative Mental Model Method and its impact on the redesign process. The Conclusion chapter summarizes this thesis and the key findings in relation to the main research question as well as the two sub-research questions..

(15) Chapter 2 Background This chapter presents the history behind the emergence of the Human-Computer Interaction field. It proceeds with describing the methods that were used in this study, namely Contextual Inquiry, Card Sorting, Prototyping and Thinkaloud. Following that is a description of legacy systems. The chapter ends with describing Participatory Design, an approach that inspired the Collaborative Mental Model, described in the Methods section.. 2.1. Cognitive Science and Human-Factors Engineering. The advancement of Information Technology has led to an increasing need to understand the users and how they interact with computers. The early users of computers were mainly professionals but this changed in the late 1970s when softwares such as text editors and spreadsheets were introduced to the general public. Everyone in the world became a potential computer user. These computers had text-based graphical interfaces which required the user to use commands in order to interact with the computers. This way of interacting was considered hard for the non-professional users. Personal computing was generally considered to have challenges, especially with regards to usability and there was a need to understand the intersecting point where humans meet technology. Two different fields started to address these challenges: cognitive science and Human-Factors Engineering. Cognitive science is an interdisciplinary field incorporating sciences like psychology, philosophy and computer science [3]. The purpose is to combine these different fields in order to understand and model human intelligence [4]. Cognitive science began to. 5.

(16) 6. CHAPTER 2. BACKGROUND. influence Information Technology with concepts about the human skills and intelligence, and especially with the concept of measuring cognitive functions such as attention, perception and memory [3, 5]. Human-Factors Engineering is also an interdisciplinary field combining sciences such as psychology, physiology and industrial design. It was originally about designing war machines and equipments during the second world war but developed into studying human-machine systems in general [6]. Previously, engineering projects did not methodologically incorporate human behaviors in the design process, i.e. the user was not in focus. This changed with Human-Factors Engineering which had developed rigorous scientific methods in order to understand the human-machine interaction from the perspective of behavioral science [7]. Cognitive Science and Human-Factors Engineering laid the foundation for Human-Computer Interaction (HCI).. 2.2. Human-Computer Interaction. The ACM Special Interest Group on Computer-Human Interaction (SIGCHI) defines HCI as "a discipline concerned with the design, evaluation and implementation of interactive computing systems for human use and with the study of major phenomena surrounding them" [8]. HCI is a research field that studies both humanities (e.g. linguistics, social science) and computer science (e.g. computer graphics, operating systems) in order to understand and improve the interaction between humans and computers. It is about understanding people’s contemporary needs, preferences and also the activities in which we engage with different interactive artifacts. The interaction we have with these artifacts will then affect us and as a consequence, affect the design of the new artifacts. This is called the task-artifact cycle usability. It is illustrated in figure 2.1.. Adoption, appropriation, use. Tasks. Artifacts Requirements and design ideas. Figure 2.1: People affect how the artifacts are designed. The experience we have with these artifacts will then affect the design of new the artifacts. [9].

(17) CHAPTER 2. BACKGROUND. 7. A significant aspect of HCI is its ever-evolving nature. Throughout the years, it has expanded its interpretation of the concept of usability. It was originally about focusing on the "ease of use" for the user but is today involving qualities that were not considered before, such as fun, aesthetics, and well being. Also, HCI is not just about the interaction with computers anymore, rather, it is also about understanding the interaction with other interactive artifacts such as mobile phones, watches, glasses and home appliances. Further, HCI is both a theoretical and practical science. It uses its diverse background in e.g. computer science and humanities in order to develop new guidelines, or models. One of the first guidelines evolved from HCI was User Experience Design.. 2.3. User Experience Design. The ISO definition of User Experience (UX) is "A person’s perceptions and responses that result from the use and/or anticipated use of a product, system or service" [10]. UX can be mistaken for User Interface Design. However, UX is not just concerned with what you see on e.g. the screen of a phone. It is about understanding everything around it: the marketing of the phone and how it was delivered to you, how it looks and feels, and the services surrounding it e.g. cloud-storage which automatically can save your photos and other data. A UX designer first tries to understand why the users must use a specific product. It can be that the user must use the product in order to accomplish specific tasks but it could also be that the ownership of a specific product could bring some additional values to the consumer. This is important to understand in order to create a meaningful product. The functionalities are then designed based on the purpose of the product [11]. Thus, there are three critical questions that work as a guideline in the design of a product; • Why - What are the motivations for the user to use the product? • What - What tasks can the user accomplish with the product? • How - How should the features of the product be designed? The main approach in order to answer these questions is User Centered Design [11]..

(18) 8. CHAPTER 2. BACKGROUND. 2.4. User Centered Design. User Center Design (UCD) is an approach that is characterized by involving the user in each phase of a design project with the goal of understanding the problems and needs of the user. It is an iterative process in which the user can provide feedback at every phase [12]. UCD defines the following four phases: 1. Context - understand the context in which the system is used (or going to be used). 2. Requirements - gather and analyze the feedback from the previous phase and specify the requirements. 3. Design - build the design and incorporate the requirements from the previous phase. This part may be done iteratively in several phases, from concept to the final design. 4. Evaluate - show the design to the actual users and conduct usability tests. Get feedback and re-iterate any of the previous phases, depending on the type of the feedback [13]. There are usually several iterations between the phases, for instance, between phase three and four: the design is updated (phase 3) based on the feedback from user tests (phase 4). UCD does not decide which specific methods or tools to use for each of the phases. UCD outlines how a design project should be carried out based on current HCI research. The specific methods are chosen with consideration to the type of project. Some methods are more suitable for a certain type of project than others. However, there are proven methods within the HCI community for each of the steps. One typical example is to use the following methods in the respective phase: Contextual Inquiry, Card Sorting, Prototyping and Think-aloud. These methods were used in this thesis project.. 2.4.1. Contextual Inquiry. Contextual Inquiry is essentially about observing the users at their workplace while they are working as usual. The observer follows the user and when suitable, asks questions regarding the work. There are three principles for it to be successful: the observer must be at the same physical location as the user i.e. not via video call or similar, the observer and the user must collaborate to explore the system, and there must be a focus i.e. the inquiry should address.

(19) CHAPTER 2. BACKGROUND. 9. a predefined set of concerns [14]. As for UX in general, it is important that there is no hierarchy between the observer and the participant, rather, it should be a partnership in order for the observer to get closer to the user. This creates a dialog which then enables valuable communication that is not possible in traditional interviews [14]. The next step in the design process is to analyze the data that has been gathered. This could be conducted through e.g. the Card Sorting method.. 2.4.2. Card Sorting. Websites and applications contain different types of information and items which need to be structured in ways that make it easy for the user to navigate [15]. Card sorting is a method in which the user helps to categorize those items. The user is given cards of size 3x5, each with different items e.g. keywords, product labels or page titles and is then asked to group them together based on what they think would be ideal. There are two types of card sorting; closed card sorting and open card sorting. If the groups are named and predefined, it is called closed card sorting. If the groups have not been defined i.e. the users have to write the name of the groups themselves, it is called open card sorting [16]. It is not unusual that these are combined i.e. some groups are predefined and some groups are open for the user to define. The number of cards depends on the specific project but should be limited to somewhere between 50 and 60 [17]. Card sorting allows the design team to see how different items are related to each other. It helps to identify and distinguish between high-prioritized and low-prioritized problems [18]. In addition to that, it shows the thinking of the user during the categorization and how they decide to label the groups [19]. When the design team has a clear understanding of what functionalities to design, it is suitable to design it with the help of different prototyping techniques.. 2.4.3. Prototyping. Prototypes are a visualization of how a potential interface can look like and it makes it easier for the user and other stakeholders to see the potential interface at an early phase. This is important since it would be expensive to redesign an interface that is almost finished, both in terms of time and money. Prototyping is generally divided into two levels; low-fidelity prototypes and high-fidelity prototypes. Low-fidelity prototypes are made of sketches and.

(20) 10. CHAPTER 2. BACKGROUND. the focus is on the basic layout of an interface and how to structure different components. The focus is not on the color and aesthetics, rather the focus is on creating the underlying structure of an application and how to interact with it. The sketches can be done with pen and paper but also with physical objects that are more tangible such as paperboards. There are also softwares which can be used to make low-fidelity prototypes. The main advantage with lowfidelity prototypes is that the proposed design can be redesigned quickly and at low costs. High-fidelity prototypes show how the final product will look like. There is more focus on the aesthetics and interactions at this level [20]. A prototype is redesigned in several iterations in order to improve it. It is important to get feedback from the users in each iteration and consider the feedback for the new version of the prototype. This can be done by allowing the user to test the prototype with Think-aloud sessions.. 2.4.4. Think-aloud. Think-aloud sessions allow the design team to see and understand how the user thinks and interacts with a prototype i.e. the potential system. The user is instructed to perform a specific task in the prototype and simultaneously speak about every thought that goes through the mind i.e. think-aloud. Users are asked to say what they are looking at, what they are doing and what they feel. The observer must remind the user to speak if he/she becomes silent during the session [21]. There is usually a short interview afterwards where the user can give general feedback on the prototype. The design team gathers all of the feedback from the users and considers them when designing the next iteration of the prototype.. 2.5. The Challenge of Legacy Systems. A legacy system is considered as a critical system of an organization but with some difficulties to adapt to new technologies [22]. These systems were developed years ago and are being updated as needed but the frequent modifications might lead to unstructured code, which makes the system harder to maintain. Thus, there will eventually be a need for modernizing those kind of systems but this is a difficult process. Although legacy systems might be considered "old", there are some benefits from them. They are often at the core of an organization and any failure can have serious negative impact on the daily operations. Also, they are built on a proven technology and have been tested throughout.

(21) CHAPTER 2. BACKGROUND. 11. the years. Hence, there is a trust towards those kind of systems, i.e. they give a sense of reliability [22]. The modernization process must thus be carefully considering these aspects.. 2.6. Participatory Design. Participatory Design (PD) is an approach which emphasizes the importance of involving different stakeholders, including the end-users, in the designing of a system. It emerged from Scandinavia with the political motivation of making workplaces more democratic and brings different stakeholders together in order to design the new system [23]. The idea is to involve the user in the creation of the actual prototype [2]. The user gets to construct the prototype in collaboration with the designers. The prototyping can be done in different ways. It could be with paper mock-ups but also with computer programs. By using drag-and-drop and animations, the users can directly see how a potential system could look like. The goal with this approach is to give the user a say in the design process. This can have several positive impacts. It facilitates mutual learning between the users and developers which can make the design process easier. Maybe more importantly, the different stakeholders can easily agree on the same vision regarding the technical but also the social aspects a new system will bring to an organization [24]. The earliest projects that used participatory design were projects within manufacturing, mainly metals and chemicals. With time, it spread to other areas such as public administration and health care. One of the earliest projects applying participatory design was conducted in 1970s by the Norwegian Computing Center (NCC). The main focus was on working with three unions to educate them about how to use information technology and how it can benefit them in their work [25]. The best known PD project UTOPIA was conducted by researchers from Sweden and Denmark. It was found that existing technologies inhibited workers to influence technological and organizational change in the workplace. The designers worked with the union and the workers in order to develop tools for graphic workers. The use of PD can have several positive impacts. It increases the user knowledge about new technology and the social impacts it has for them [25]. It can also lead to more open discussions between the workers and the management. The workers get a chance to move away from their traditional role. They get to design and evaluate i.e. they get to express their opinions [26]. However, PD is criticized because it can be complex. The users can embrace.

(22) 12. CHAPTER 2. BACKGROUND. different types of roles during the design sessions e.g. informant, tester and design partner. Depending on the role, the type of feedback will be different and thus affect the design in different ways [27]. Further, the users need an interest for design [25]. It is also argued that users are not designers and thus not eligible to participate in the design [28]..

(23) Chapter 3 Preliminaries The following section describes how the case study was conducted. It finished with describing the Collaborative Mental Model and how it is supposed to be used.. 3.1. Research Design. An inductive research approach has been used in this study to address how to improve the user experience for the administrators using the Identity Access Management (IAM) system. This study is based on the four phases advocated by the UCD approach, namely understanding the context and needs of the user (discovery phase), specifying the requirements (defining phase), proposing a design (development phase) and evaluating that design (evaluation phase). Five methods were used. Four of them are established methods from the UCD community; Contextual Inquiry, Card Sorting, Prototyping and Thinkaloud. The fifth method, the Collaborative Mental Model, is the new method proposed by this study and used in the second phase. Figure 3.1 presents the different phases and the methods that were used in each phase.. 13.

(24) 14. CHAPTER 3. PRELIMINARIES. DISCOVER. DEFINE. DEVELOP. EVALUATE. Collaborative Contextual. Inquiry. Card Sorting. Mental Model. Think-aloud. +. Prototyping. Figure 3.1: The Design Process Each method yielded data that was used by the next method. All of the methods are consequently dependent on each other. The structure of this report is reflecting that. Therefore, all of the methods and how they performed are described in the next chapter "The Design Process".. 3.2. The Collaborative Mental Model. The Collaborative Mental Model (CMM) method is inspired by the Participatory Design approach in the sense that the user and the designer collaborate in order to create a prototype. The basic idea is the following: you have a blank sheet of paper that represents an empty website. Next to this sheet, you have cut-out pieces of UI-elements (e.g. search bars, input fields etc). The user is instructed to think of an UI that he or she is comfortable about and then to reconstruct it by placing the elements on the blank sheet, see figure 3.2. The users will be asked to think-aloud while doing that. This reveals the thoughts of the user. These thoughts are valuable information to be considered in the prototyping phase..

(25) CHAPTER 3. PRELIMINARIES. 15. Search. Figure 3.2: The UI-elements to the left. The blank website to the right. There are two requirements that must be considered before using this method. The first requirement is to decide what type of UI the user will reconstruct. This will be decided after analyzing the feedback from the previous phase i.e. the phase of discovering the usability issues. The first phase will hopefully give an idea of what the problem is and thus, what type of UI must be designed. The second requirement is the UI elements. These must also be related to the findings from the previous phase. The first phase could, for instance, reveal that the users are frequently using the search functionality of a certain system. The UI-elements will then be related to that functionality. The reason for these requirements is to narrow the user interfaces that the user must think of to those who are relevant. An important aspect that differentiates this method from the Participatory Design approach is that the user is not asked to design the actual prototype for the system to be. Instead, the user must recall of an UI that he or she is used to and reconstruct it. The reason for this is to make the user start thinking about interfaces and interactions in general, without having the responsibility to think about how to construct the actual prototype. One of the critiques against Participatory Design is that the users lack motivation, or even the skill, to design a prototype [28]. The CMM method does not require design skills, rather, it asks the user to create a rough reconstruction of the user interfaces.

(26) 16. CHAPTER 3. PRELIMINARIES. that the user has seen before. This creative process of building an UI can give valuable insights into the mental model of the users and their attitudes towards different types of interactions. These insights will be valuable for the designer when creating the prototype. The goal with this method is to give the designer insight into the mental model of the user i.e. to understand the interfaces and type of interactions the user recognizes and appreciates..

(27) Chapter 4 The Design Process The design process was divided into four phases. Phase 1 was about understanding the users and the context in which they work. Phase 2 narrowed the findings from phase 1 which helped to define the problem to focus on in this study. Phase 3 was the phase in which the Collaborative Mental Model was used. Also, the prototype was created in this phase. Finally, phase 4 was the phase in which the users evaluated the prototype. This chapter is divided into four sections, one section for each phase in the design process. Each section begins with presenting how the methods were used and ends with the findings from those methods.. 4.1. Phase 1 - Discover. The aim for this phase was to gain a general understanding of the administrators work and needs. Two methods were used for that, Contextual Inquiry and Card Sorting.. 4.1.1. Contextual Inquiry. The purpose with the contextual inquires was to understand how the administrators work and how they process the applications that comes in to the IAM system. The contextual inquires were conducted in Stockholm and Gothenburg. Each contextual inquiry was individual. Five administrators were interviewed in Stockholm and three administrators were interviewed in Gothenburg. The administrators were informed beforehand about the structure and purpose of the sessions. Each contextual inquiry began with a semi-structured interviews. The purpose of the interviews was to get to know the users and. 17.

(28) 18. CHAPTER 4. THE DESIGN PROCESS. their role. The interviews were immediately followed by the contextual inquiry. The interviews ran for 30 minutes. The contextual inquires ran between two and four hours. The traditional interviews and the contextual inquires were carried out at the same place i.e. the office in which the administrators work. The administrators sat at their work desk during the contextual inquires and were observed from a position next to them. The sessions were characterized by two types of actions. The first type was of the general nature where the administrators explained the IAM system, how a regular work day looks like and what tools they use to structure up their work e.g. calendars and post-it notes. The second type of action that took place was the main objective of the contextual inquiry which was to observe them handling applications through the IAM system.. 4.1.2. Findings from the Contextual Inquiry sessions. There were two main types of findings, one regarding the processes within the Police and the other one regarding specifically the IAM system. Processes There are many systems connected to the IAM system. The police staff, both uniformed and the civilian personnel, must apply for authorization to those systems in order to be able to use them. These applications are handled by the administrators. The administrators must have knowledge about those systems in order to judge whether the applicant has the right to use it. New systems are getting connected to the IAM system continuously as the Police organization grows. This implies increase in workload. Every new system works differently and the administrators must have knowledge about those systems. The information about the system is given in a manual which describes the system and how to give authorization to it. This information is preferably given before the system is connected to IAM but that is not the case every time. One participants mentioned that "Sometimes, we receive the manuals the same day that a new system is launched. This makes it difficult for us. We don’t know the how to give authorization to that system". Another participant had a similar citation. The format of some manuals can also be overwhelming and troublesome to search through. One participant expressed that "some manuals are big and can be up to 80 pages. It can be overwhelming to open them". Also, the administrators prefer to get information about the new system during its development. This would give the administrators a chance to keep up with the changes within the Police organization. This would also help them.

(29) CHAPTER 4. THE DESIGN PROCESS. 19. to be prepared before the new system is launched. The feedback from the participants reveal that there is room for improvement regarding the manuals and the communication between the different levels of the organization. The IAM system The administrators work is of the repetitive nature. The applications arrive to the IAM system. The administrator picks an application and makes a judgment about it by analyzing the applicant profile and the system or systems it has applied for. It is not uncommon that the administrator must do further investigation before approving or disapproving the application. This leads to many clicks for the administrator in the IAM system. They must sometimes go into other systems as well which leads to more clicks. This clicking becomes a problem when the IAM system is slow. The combination of a slow system and a lot of clicking is the main complaint regarding the IAM system and is clearly affecting the user experience negatively. Other observations The Contextual Inquiry method has the purpose of not just bring forth the issues but also give a general picture of the organization through observations. These are the main observations: • The administrators did occasionally call the applicant to get further information about the application and clear any misunderstandings. This way, the process did not need to start all over again i.e. the administrator disapproving the application on which the applicant must re-apply. This was appreciated by the applicants during the Contextual Inquriy. • The administrators consult each other in case they have questions. • The administrators had a variety of information sources e.g. e-mails, the telephone, internal chat, post-it notes, pdf-documents, paper documents and an internal document which comprised information about most of the systems. The administrators seemed experienced in using these different sources.. 4.2. Phase 2 - Define. The aim of this phase was to narrow down the findings from the previous phase and to define the problem. The Card Sorting method was used for this purpose..

(30) 20. 4.2.1. CHAPTER 4. THE DESIGN PROCESS. Card Sorting. Four card sorting session were conducted, each with one administrator at the time. Each session ran for 20 to 30 minutes and was audio recorded. A photo was taken on the notes after each session. The participants were given 36 items that were chosen from the Identity Access Management system, see Appendix A. The items were links and text. These are related to the items the administrators use when handling applications in the system. The items were written on post-it notes of green color before showing them to the participants. The participants were asked to group the items, i.e. the green post-it notes, according to what they felt was relevant. They were given blank red post-it notes to place on top of each group on which they wrote a name that they feel best describes the group. The participants were also given blank green post-it notes if they wanted to add an item that they thought was missing. They were asked to speak while grouping the items and explain the reason behind each placement of the items. The purpose with these sessions was to understand what items are related to each other from the perspective of the participants. It can be the case that the developers and the users (administrators) have different opinions of the information architecture i.e. the structure of the items and how they are related. By writing the items down on post-it notes, they were brought out of the context of which the participants were used to see them on the user interface. This forces the participants to structure the items as they feel is relevant, without getting biased from the how they are aligned on the interface.. 4.2.2. Findings from the Card Sorting sessions. The participants organized the items chronologically, that is, from when an application arrives to the IAM system until it has been processed. They were not told to order them chronologically before the sessions. It was the initiative of the participants to order the items in that fashion. In a sense, the participants were compelled to think of each step in the application process. There were three different steps common for all of the participants: filtering the applications, picking one application to be examined and finally approve or disapprove it. Thinking of each of these steps evoked self-reflection. The participants thought of how they usually structure their own way of working. Below is one citation regarding that: "I start by filtering on region and then the date of the application. If there are several applications from the same applicant, I open them all in new tabs. I.

(31) CHAPTER 4. THE DESIGN PROCESS. 21. examine them one by one and look if there is something missing. If it looks OK, I approve the application. Otherwise, I make further investigations. It can either be by consulting my colleagues or to contact the applicant directly". The grouping of each participant can be seen in figure 4.1. One finding was that it occasionally happens that one administrator starts to examine an application at the same time as another administrator. The system will accept the changes done by the administrator who finishes first. The other administrators work will then not be accepted. This leads to unnecessary time loss for that administrator. This situation could be prevented by the administrator using the functionality called "Lock". However, it happens that this functionality is not always used. Another finding was concerned with how some pages reset the filters when turning back to them after having visited other pages. Also, the topic of manuals was raised again during one of the session. The participant in that session expressed that "We have not yet received the manual for the new system", referring to a new system that just was launched..

(32) 22. CHAPTER 4. THE DESIGN PROCESS. Figure 4.1: The groupings from all of the four sessions. 4.2.3. Summary of Phase 1 and 2. These were the main findings from the Contextual Inquiry and Card Sorting sessions:.

(33) CHAPTER 4. THE DESIGN PROCESS. 23. • The administrators find the system they are using i.e. the IAM system, slow. • There are many clicks caused by using the IAM system as well as the other systems. • Information about the new systems to be launched are received to late. • The format of some manuals can be overwhelming. Also, the manuals do sometimes arrive late.. 4.2.4. Integrating Manuals to the IAM System. Phase 1 and 2 revealed that there were no major issues with the user interface of the current IAM system. Rather, the issues were mainly concerned with processes within the organization, mainly regarding the information of the new systems. The administrators would like to have the description of a new system in advance, which is not always the case today. This would give them time to understand how the system work and the purpose of it. The description of a system and its purpose can be found in the manual for that system. During the phase 1 and 2, there were complaints about the manuals arriving late. It was also revealed that the format of some manuals can be overwhelming. Clearly, manuals are a part of the processes in the organization; it transfers knowledge from the system owners to the administrators. An improvement to this area could increase the overall user experience for the administrators. This could be achieved by making the manuals digital in form of a web application. All of the manuals would have the same look and structure in order to make them less overwhelming. This web application could then be integrated with the current IAM system. Phase 3 and 4 of this study was focused on making the paper manuals digital by creating a digital prototype.. 4.3. Phase 3 - Development. From this phase and onward, the focus was on manuals. The Collaborative Mental Model was used to understand the administrators previous experiences with manuals in general. The insights yielded from that method was then used in creating the prototype..

(34) 24. 4.3.1. CHAPTER 4. THE DESIGN PROCESS. The Collaborative Mental Model. The requirements for the Collaborative Mental Model (CMM) was to decide what user interface (UI) the participants should focus on and the UI-elements to use. The previous two phases were supposed to help with this and they did. The UI to focus on is manuals. The UI-elements must therefore be related to manuals. A search for manuals was performed on Google in order get inspiration for what type of UI-elements to use. Two manuals from the systems at the Police was also used as source of inspiration. Four elements were created, presented in figure 4.2:. (a) Text. (b) Figure. (c) Gray box. (d) White box. Figure 4.2: The four UI-elements Image a represents a text field. Image b represents an image. The gray box is general and can represent whatever the participant wishes. The white box has the same purpose, the only difference is the color. Contrasting elements can help our eyes to navigate when looking at an interface. All of the.

(35) CHAPTER 4. THE DESIGN PROCESS. 25. three elements were created in three different sizes; 40x40mm, 70x70 mm and 90x150mm. The elements are general and do not represent detailed things such as home buttons, drop-down menus etc. The reason is to not overwhelm the participant with different elements. It is also a way to reduce the need for design skills and only focus on the broad aspects of a design. The grey and white boxes can be used if there is a need for a more detailed element. The participants were given pens to be able to sketch on any of the elements. The figure 4.3 was printed on an A4 paper and represented the UI on which the participants placed the elements on:. Figure 4.3: An A4 paper representing a blank website. CMM was conducted with four administrators individually. They ran for 30 minutes each and were audio recorded. The participants were instructed to think about the manuals they have experience from. It can be manuals that they liked or disliked. It can also be how they think the perfect manual should look like. It can be any manual except the manuals that are used at the police. This is important since the participants are not supposed to create the actual prototype (as in Participatory Design). They were then instructed to place the UI-elements on the blank website. While doing so, they were encouraged to think out loud..

(36) 26. 4.3.2. CHAPTER 4. THE DESIGN PROCESS. Findings from the Collaborative Mental Model sessions. The CMM method led to conversations about manuals with the participants. The conversations were about what kind of experiences the participants had when encountering manuals from the personal life. One popular manual that two of the participants brought up was the IKEA manual. It was appreciated because of its use of images in a simplistic way. One participant mentioned a manual for a vacuum cleaner. It had hand-drawn sketches of how to change the bag. The use of sketching instead of real photos was appreciated because it made the manual more fun and enjoyable. The Reconstructions Inspired from the discussions, the participants started to re-construct a manual that they would appreciate to use.. Figure 4.4: Manual reconstruction with participant 1 Figure 4.4 shows how Participant 1 would prefer a digital manual to look like. The participant preferred to see a table of content first (the leftmost paper). This page does also contain a short description (the white box) of the.

(37) CHAPTER 4. THE DESIGN PROCESS. 27. system which the manual is for. Clicking on any heading will take you to the next page (the paper in the middle). The grey areas on this page represents general information of any system. To get more detailed information, you can click on the grey area which then will take you the page on right. This page consists of detailed information (the grey boxes) with a small image next to it. Each image has descriptive text under them (the white boxes).. Figure 4.5: Manual reconstruction with participant 2 Figure 4.5 shows that Participant 2 preferred bigger images with descriptions next to them (the paper to the left). This participant also wanted a short description (the white box) of the system which the manual is for. Clicking any of the images takes you to the page to the right. This page shows detailed information about the image you just clicked. The page consists of bigger images with description to the right of them. All of the information is not shown at once, you have to click on the "more information" button (the white box at the bottom) to get more information..

(38) 28. CHAPTER 4. THE DESIGN PROCESS. Figure 4.6: Manual reconstruction with participant 3 Figure 4.6 shows that Participant 3 preferred to have an overview of the different systems first (the leftmost paper). Clicking on any of these systems brings you to the page in the middle. This page has a header on the top. The header consists of a short description of the system (the square white box) and a link path next to it. The link path shows the previous pages that the user has visited in order to reach the current page. Below the header is the main content. It contains images with descriptive text next to them. Clicking any of the images will take you to the page to the right. This page contains more detailed information about the image you just clicked..

(39) CHAPTER 4. THE DESIGN PROCESS. 29. Figure 4.7: Manual reconstruction with participant 4 Figure 4.7 shows that Participant 4 preferred a header on the top of the page (the paper to the left). This header contains general information (the grey box) about the system. Next to it is a table of content. Below the header is the main content. The main content is made of images with a descriptive text next to them. They also have a title. This participant also added a page for troubleshooting i.e. a FAQ page (the paper in the middle). This page lists the questions with a short title for each problem (the white stripes). Clicking on one title will make it expand and show more information (the paper to the right). This shows an image with explaining text next to it. Clicking on the title (the white box) will take you back to the previous view i.e. the paper in the middle. Summary of the CMM method The reconstructions as well as the discussions revealed that the participants prefer images with an explanatory text next to it. They also prefer to have an overview first with the option to click on the images to get further detail. Further, it was mentioned that manuals should have an overview of the current level you are at..

(40) 30. 4.3.3. CHAPTER 4. THE DESIGN PROCESS. Prototyping. This section describes how the prototype was made. The prototype was first drawn on paper and then with a design application called Figma. Paper sketches The first sketches of the prototype was done on paper. The focus of the sketches was to draw the layout and the main elements of the prototype and not on the aesthetics such as font style, colors and shapes. This made it easier to focus on the logic behind the interface and how different elements should be related to each other.. (a) Home page. (b) Introduction text of the system. (c) Table of content. (d) The requirements. Figure 4.8: The paper sketches Four paper sketches were made, as presented in figure 4.8. Image a illustrates the home page. Image b shows were the introduction text to each system was placed i.e. to the right. It also shows the placement (center) of the table of content for that manual. Image c illustrates a simple animation; when the user hovers the mouse over different headings of the table of content, an overview of that content is displayed to the right. Image d illustrates that the requirements were structured in table format..

(41) CHAPTER 4. THE DESIGN PROCESS. 31. These sketches can be found in Appendix B. The next step in the design process was to create the prototype, based on these sketches. The prototype Previous phases of the design process gave valuable insights about what type of layout and interactions that the administrators would prefer. They prefer 1. To minimize the amount of clicks where it is possible 2. To have an overview of the content first and detailed information one click away 3. To not have the interface cluttered with a lot of information at once 4. To see how "far" you are in a web page so that you do not lose track of where you are Figure 4.9 shows how the homepage of the prototype looks like: MANUALS. Sök. Sortera L. Senaste Granskning. SYSTEM 1. 2018-12-01. SYSTEM 2. 2019-02-14. SYSTEM 3. 2019-04-01. SYSTEM 4. 2019-06-10. Starta programmet Övrigt. 1. Figure 4.9: The home page All the manuals are listed such as SYSTEM 1, SYSTEM 2, SYSTEM 3 and SYSTEM 4. These are some of the actual systems used at the Police. Clicking any of these buttons takes you to the manual of that system. A short text about.

(42) 32. CHAPTER 4. THE DESIGN PROCESS. the system is displayed to the right of the button when hovering over it, as illustrated in figure 4.10.. SYSTEM 1. 2018-12-01. SYSTEM 2. 2019-02-14. SYSTEM 3. 2019-04-01. SYSTEM 4. 2019-06-10. Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.. Ut enim ad minima veniam, quis nostrum exercitationem ullam corporis suscipit laboriosam.. 1. Figure 4.10: Hovering over the SYSTEM 1 manual The red circle next to the button indicates that there has been an update to that manual, in this case System 4. The number within the circle represents the number of updates for that manual. The date within each button indicates when the manual was last updated. Further, there is an sort functionality i.e. "Sortera". This can sort the manuals according to their names or according to when they were last updated. The search field at the top allows the user to search for any content within all of the manuals. The sidebar to the left shows the recent pages that the administrator has visited. The purpose with this functionality is to reduce the number of clicks that the user has to do. Instead of clicking through several pages, the user can just revisit recent pages just by clicking here. This functionality addresses insight number 1, namely avoid redundant clicking. Clicking on the SYSTEM 1 button takes the user to the home page for the SYSTEM 1 manual, see figure 4.11:.

(43) CHAPTER 4. THE DESIGN PROCESS. MANUALS. 33. Sök. Manualer / System 1. Behörigheter. SYSTEM 1 Behörigheter. Granskning. Granskning Starta programmet Behörighetshantering. Starta programmet. Sök användare. Behörighetshantering. Övrigt. Registrera personuppgifter Tilldela behörighet. L. Senaste. Meddela inloggningsuppifter. Granskning Starta programmet Övrigt. Sök användare Övrigt. Figure 4.11: The first page for the SYSTEM 1 manual This page shows the table of content for the SYSTEM 1 manual e.g. Behörigheter (requirements), Granskning (Audit) and Starta programmet (Start the program). These are links that the user can follow. The sidebar does also show the table of content and will stay there even if the user clicks any of the links. The purpose of this is to enable the user to have an overview of the manual and make it easier to navigate through it. When the user hovers the mouse over any link, there will be an overview of what that page contains as illustrated figure 4.12..

(44) 34. CHAPTER 4. THE DESIGN PROCESS. MANUALS. Sök. Manualer / System 1. SYSTEM 1 Behörigheter Granskning Starta programmet Behörighetshantering. L. Behörigheter Granskning Starta programmet. Sök användare. Behörighetshantering. Övrigt. Registrera personuppgifter. Senaste Granskning. Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip.. Tilldela behörighet Meddela inloggningsuppifter. Starta programmet. Sök användare. Övrigt. Övrigt. Figure 4.12: Overview to the right. Clicking on a link, e.g. "Meddela inloggningsuppgifter", takes the user to a page with more details, as shown in figure 4.13. MANUALS. Sök. Manualer / System 1 / Behörighetshantering / Meddela inloggningsuppgifter. SYSTEM 1. Meddela inloggningsuppgifter. Behörigheter. Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.. Granskning Starta programmet Behörighetshantering Sök användare Övrigt L. Senaste Granskning Starta programmet Övrigt. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.. Figure 4.13: Details on demand This principle of having overview first and detail on demand was insight.

(45) CHAPTER 4. THE DESIGN PROCESS. 35. number 2. The main part of the manual describes to whom the administrators may grant access to that system, see figure 4.14. MANUALS. Sök. Manualer / System 1 / Nationell. SYSTEM 1 Regional. L. Nationell. Nationell. AUTE. OFFICIA. PROIDENT. NOSTRUM. Förvaltning. Sequi nesciunt. Sed ut perspiciatis unde omnis iste natus error sit voluptatem accusantium doloremque laudantium, totam rem aperiam, eaque ipsa quae ab illo inventore veritatis et quasi architecto beatae vitae dicta sunt explicabo.. Nemo enim ipsam voluptatem quia voluptas sit aspernatur aut odit aut fugit, sed quia consequuntur magni dolores eos qui ratione voluptatem sequi nesciunt. Neque porro quisquam est, qui dolorem ipsum quia dolor sit amet, consectetur, adipisci velit, sed quia non numquam eius modi tempora incidunt ut labore et dolore magnam aliquam quaerat voluptatem.. Ja. Voluptatum. Ut enim ad minima veniam, quis. At vero eos et accusamus et iusto odio dignissimos ducimus qui blanditiis praesentium voluptatum deleniti atque.. Nej. Asperiores delectus. Et harum quidem rerum facilis est et expedita.. Taque earum rerum hic tenetur a sapiente delectus, ut aut reiciendis voluptatibus maiores alias consequatur aut perferendis doloribus asperiores repellat.. Ja. Senaste Granskning Starta programmet Övrigt. nostrum exercitationem ullam corporis suscipit laboriosam, nisi ut aliquid ex ea commodi consequatur.. Figure 4.14: Requirements The table shows the requirements that an applicant must have in order to be able to use that system (SYSTEM 1 in this case). This specific system does also have several levels of security, shown under "Behörighetsnivå". There is a description for each level "Beskrivning" and those who are eligible for that level "Tilldelas". There are different types levels such as "Regionell", "Nationell" and "Förvaltning". This page shows the type "Nationell". In order to see the other types, the user must click on the links to the left. This addresses insight number 3, which was to not clutter the interface with a lot of information at once. Instead of showing every type on the same page, each type has its own page. Insight number 4 was to let the user know how far the user has navigated in the manual. This is addressed with the link path "Manualer/System 1/Nationell", below the search bar at the top. Every page shows the path that the user has taken in order to visit that page. The user can navigate to the previous pages in the path just by clicking on it..

(46) 36. 4.4. CHAPTER 4. THE DESIGN PROCESS. Phase 4 - Evaluation. This phase was about letting the users evaluate the prototype with the Thinkaloud method.. 4.4.1. Think-aloud. Four Think-aloud sessions were conducted with four administrators individually. The sessions ran for 20 minutes and was audio recorded. The administrators sat in front of a laptop. They could use a mouse to navigate if they wished so. They were told that they will be given tasks to perform and think out loud while performing them. There were totally seven tasks and they were about what they see on the current page and how they would navigate to the next page that they were asked to navigate to. These tasks can be found in Appendix C. After they have finished all of the tasks, they were asked about their impression of the prototype and whether they would like to make any changes to it.. 4.4.2. Findings from the Think-aloud sessions. All of the participants successfully finished the tasks that were given to them i.e. they navigated through all of the pages without any difficulty. There were two UI-elements that the participants interpreted differently than they were intended for, see figure 4.15. The first misconception was the functionality that shows the recently visited pages i.e. "Senaste". Two participants interpreted it as updates to the manuals. The second issue was the red circle that shows that there has been an update. One participant interpreted it as another user is active and currently watching that document. Another participant understood that it indicates an update but was not sure how to see the updated information..

(47) CHAPTER 4. THE DESIGN PROCESS. L. 37. Senaste Granskning Starta programmet Övrigt. 1. AGRESSO. 2019-06-10. Figure 4.15: The two functionalities that were interpreted differently Further, there were two suggestions from the participants. The text is too small on the page where the requirements are listed (figure 4.14). The user should be able to increase or decrease the size of the text. Further, the prototype should be able to print out the manual in a printer friendly format i.e. printing the main content and not specific web elements such as search bars and links. There might be administrators that prefers the manuals on paper. The overall impression of the prototype was, despite the issues mentioned above, positive. The functionality that shows the recent visited pages, "Senaste", was appreciated. One participant mentioned "it is perfect because it reduces the amount of clicks. You don’t have to navigate through all of the pages again". Even those who interpreted that functionality differently in the beginning appreciated it afterwards when the purpose of it was explained. The participants did also appreciate the structure of the interface. One participant said that "it is nice to see a short introduction text of the system when hovering the mouse over it", referring to figure 4.10. The same participant did also mentioned the table of content in figure 4.11 and how it gave a good overview of the manual. Another participant mentioned the same thoughts about the table of content and added "it makes the page more clear by not putting all of the information in your face at once". The participants did also mention that it is easy to navigate between the different pages with the help of the menu in the sidebar and the link path at the top, see figure 4.16..

(48) 38. CHAPTER 4. THE DESIGN PROCESS. MANUALS. Sök. Manualer / System 1. SYSTEM 1. Behörigheter. Behörigheter Granskning. Granskning. Starta programmet. Starta programmet. Behörighetshantering Sök användare. L. Behörighetshantering. Övrigt. Sök användare. Senaste. Övrigt. Granskning. MANUALS STORM. Sök. Manualer / System 1 / Behörighetshantering / Meddela inloggningsuppgifter. Om STORM Meddela inloggningsuppgifter SYSTEM 1 IT-stöd som används STORM är det vid polisens. Regionsledningscentral/Nationella ledningscentral Lorem ipsum dolor sit amet, consectetur adipiscing elit, Behörigheter (RLC/NLC) för dokumentation avsed inkommande do eiusmod tempor incididunt ut labore et dolore Granskning händelser. Det finns en instans av STORM per RLC samtad minim veniam, quis nostrud magna aliqua. Ut enim programmet en Starta för NLC.. exercitation ullamco laboris nisi ut aliquip. Behörighetshantering WEBSTORM är en webbaserad spegling av det som Sök användare bokförs i STORM och utvecklades för de som endast Övrigt inhämta information från STORM. behöver Senaste. Starta programmet Övrigt. Figure 4.16: Two ways of navigating.

(49) Chapter 5 Results This section presents the answers to the two sub-research questions. The first sub-research questions was to define the usability issues. The second subresearch question was to answer how the Collaborative Mental Model can be combined with the Contextual Inquiry, Card Sorting and Think-aloud methods in order to solve those usability issues.. 5.1. SRQ 1 - The Usability Issues. Phase 1 and Phase 2 of the design process revealed the following usability issues: • The administrators find the system they are using i.e. the IAM system, slow. • There are many clicks caused by using the IAM system as well as the other systems. • Information about the new systems to be launched are received to late. • The format of some manuals can be overwhelming. Also, the manuals do sometimes arrive late.. 5.2. SRQ 2 - The Collaborative Mental Model. The goal with the Collaborative Mental Model (CMM) was to give the designer insights on what type of interactions and layouts the user prefers regarding an interface. These insights would then be considered when creating. 39.

(50) 40. CHAPTER 5. RESULTS. the prototype. The type of interface to focus on was based on the findings from the Contextual Inquiry and Card Sorting methods, i.e. manuals. The CMM sessions yielded three insights. The user wants to • Have an overview of the content first and detailed information one click away • Not have the interface cluttered with a lot of information at once • See how "far" you are in a web page so that you do not lose track of where you are The evaluation phase reveals that the users appreciated all of these three points. Thus, the contributions of the CMM method had a positive impact on the prototype and in the end, the user experience. Figure 5.1 shows how the CMM method was combined with the other methods used in this study.. Contextual Inquiry. +. Card Sorting. Manuals. CMM. Insights. Prototyping. Figure 5.1: How the CMM worked together with the other methods.

(51) Chapter 6 Discussion This section is divided into four parts. The first part discusses why this study chose to focus on the manuals. It also discusses the performance of the Collaborative Mental Model. The second part discusses the other methods used in this study. The third discusses the design process. Finally, the last part suggests future work.. 6.1. Discussion of the Collaborative Mental Model. The Choice of Designing Digital Manuals The thing to design was not decided until phase two in the design process. It was important to first understand the users and their needs before deciding on what to design. The manuals were a recurring topic during the user research in phase 1 and phase 2. The manuals contain information that the administrators depend on when deciding whether to approve or disapprove an application. They are therefore an important part of the administrators work. Therefore, this study focused on the manuals and how they could be designed with consideration to the preferences of the administrators. Phase 3 and phase 4 did then suggest a prototype that the administrators would prefer. It must be mentioned that the biggest issue revealed during the user research was that the Identity Access Management system is generally working slow. However, solving this issue would be out of the scope of this thesis since the focus would shifted away from the of Human-Computer Interaction. Therefore, the design to focus on was manuals.. 41.

(52) 42. CHAPTER 6. DISCUSSION. The Performance of the Collaborative Mental Model The main focus of this thesis was to see how the Collaborative Mental Model (CMM) could contribute to the design process. The CMM method gave three important insights that were considered when creating the different features in the prototype. As revealed in the Results section, all of the participants expressed appreciation for those features. Therefore, one could state the CMM method was successful. However, there could be other ways to get those insights. In many projects, the prototype is created directly after interviewing the users. That prototype is then evaluated by the users. The designers would then consider the evaluation in the next iteration of the prototype. It could be the case that these iterations could yield the same insights as the CMM method did. This would make the CMM method redundant. But there is one certain aspect of the CMM method that is important and that is to incorporate the users in the design phase. This has proven democratic outcomes. The users feel more involved in the organization as they now have a voice in the design process. Often, there are different stakeholders in a design process from different levels within the organization. Thus, the distance between the users and the stakeholders gets closer which in turn improves the communication. This is an important aspect that should be prioritized especially by large organizations where there is a long distance between the users and rest of the organization. Many organizations are reluctant when it comes to investing in UX-design: it is sometimes portrayed as time consuming. This is an aspect that must be considered when evaluating new UX-methods. The CMM method might reduce the number of iterations needed in the prototyping phase by understanding the users even before creating the prototype. This could save time and money. Further, one must not forget that UX is not just about designing new systems. Depending on the findings from the user research, the solution might not always be a new product. Rather, the solution could be to do an organizational change. That could have more impact on the user experience than just an update to the aesthetics of the user interface. Also, it could be more economical and ecological sustainable to not invest in new hardware and systems when the solution lies somewhere else. One critique against involving the users in the design process, especially the prototyping phase, is that "users should not design", or even that "they do not posses the design skills required to create a prototype". The CMM method attempts to address this critique. The CMM method does not require the user to know how to handle a prototyping software. Instead, the user is asked to think about previous experiences with other software interfaces and.

No results found