Security vs. Usability: designing a secure and usable access control event log

(1)

Faculty of Technology and Society

Department of Computer Science and Media Technology

Bachelor’s Thesis

15 credits, undergraduate level

Security vs. Usability:

designing a secure and usable

access control event log

Säkerhet kontra Användbarhet:

att designa en säker och användbar

händelselogg för passagekontroll

Lykke Levin

Vedrana Zeba

Degree: Bachelor of Science, 180 credits Supervisor: Fredrik Rutz Field of study: Computer Science Examiner: Nancy Russo

(2)

Abstract

Security and usability are often thought of as being contradictive. In this thesis, we explore the possibility of incorporating both security and usability in an access control GUI. The research is concentrated towards the part of the access control that is referred to as the event log. The purpose of the log is to store and present information about events that occur at monitored entry points.

The intention of the research is to investigate to what extent it is possible to implement user requirements, while still complying with security and usability heuristics. A traditional interaction design process is conducted. Semi-structured interviews are held with respondents from two different target groups, to see if their needs differ. One of the groups consists of users who primarily do security related work, and the other one consists of users who have security as a secondary job assignment. The answers undergo a thematic analysis. The outcome of the analysis is four different themes, consisting of a total of 26 user requirements. The user requirements and the heuristics are taken into consideration when creating a prototype. The prototype is then subjected to a heuristic evaluation by experts.

The results of this research indicate that the gathering of user requirements does aid the compliance with heuristics. Moreover, the user needs between the two groups do differ on several accounts. The requirements that originate from the first group can be thought of as more dynamic and instantaneous, while the other group has requirements that are more static and occasional.

(3)

Sammanfattning

Säkerhet och användbarhet beskrivs ofta som motpoler. I detta examensarbete så undersöks möjligheterna till att inkorporera både säkerhet och användbarhet i ett passagekontrollsgränssnitt. Forskningen är fokuserad på den del av passagekontrollen som benämns som händelseloggen. Loggens ändamål är att lagra och presentera information om händelser som sker i övervakade entréer.

Syftet med forskningen är att undersöka i vilken utsträckning det är möjligt att implementera användarkrav och samtidigt uppfylla säkerhets- och användbarhetsheuristik. En klassisk interaktionsdesignsprocess utförs. Semi-strukturerade intervjuer genomförs med respondenter från två olika målgrupper, för att kontrollera om deras behov skiljer sig åt. Den ena gruppen består av användare som primärt jobbar med säkerhetsrelaterade arbetsuppgifter medan den andra gruppen har säkerhet som sekundär arbetsuppgift. Svaren analyseras genom en tematisk analys. Analysen resulterar i fyra olika teman innehållandes 26 stycken användarkrav. Användarkraven och heuristiken tas i beaktning när en prototyp skapas. Prototypen utvärderas sedan genom en heuristisk utvärdering av experter.

Resultatet av denna forskning tyder på att användarkrav bidrar till att uppfylla heuristik. Utöver detta, så visar det sig att de två målgrupperna, på flera punkter, har olika behov. Användarkrav som härstammar från den första gruppen anses vara mer dynamiska och omedelbara, medan den andra gruppen har krav som är desto mer statiska och sporadiska.

Nyckelord: säkerhet, användbarhet, händelselogg, passerkontroll, gränssnitt, heuristik,

(4)

1 Introduction

In the field of physical security, access control is used to make sure that only authorized people get access to a delimited zone that requires user authentication [1]. The market for access control systems has steadily increased during recent years, with a worth of 7.5 billion USD in 2018 and an expected worth of 12.1 billion USD in 2024 [2]. This expansion of the market is due to the growing demand for security systems from companies, in order to be able to protect both people, products, property and information [1].

The difficulty in balancing security and usability has been brought up by several researchers [3, 4, 5, 6, 7, 8, 9], and the general presumption is that these two non-functional requirements are unable to co-exist, as they tend to cancel each other out. Frankly, they are often thought of as opposites [3, 4, 5, 6, 7]. Security and usability are both important requirements when developing any type of system. Naturally, an access control system is no exception. This becomes even more evident when looking at an access control GUI (Graphical User Interface), which is used by professionals to install, configure and monitor the system. On one hand, the interface incorporates multiple security features. On the other hand, the interface comes in contact with many different types of users, which requires it to be usable.

Figure 1: Contradicting entities.

In this thesis, we explore the struggle of balancing the relationship between these two requirements in the specified domain. More specifically, we examine the possibility of implementing both requirements, while also satisfying the users' needs.

1.1 Background

The following sections will introduce:

− what access control is, how it is used and by whom. The belonging GUIs for these systems are also briefly described.

− what is meant by security, in the context of access control. − what is meant by usability, distinguishing different definitions.

− why security and usability are thought of as opposites, and how a balance between them would manifest itself.

(7)

6

1.1.1 Access control

Access control is used to verify authorized users who are attempting to get access to a physical location or a digital resource. As shown in Fig. 2, physical access control is used to make sure that only authorized people get access to a delimited zone that requires user authentication [1]. Authentication is done by using some form of authentication token, like a smartcard, PIN-number, fingerprint or other type of biometric identification. An entrance usually serves as the common entry-point managed by the system [1]. Access control enables us to allow, deny, limit and revoke access [10].

Figure 2: A user is granted access to a delimited zone.

A GUI is often provided by the company supplying the access control, or otherwise by a third-party supplier. Depending on the type of access control system, the GUI can include different functionalities. Its main task is to assist in the management of the access control, providing an interface that will aid the user when installing, configuring and monitoring the system. The GUI usually allows professional users to administer authentication tokens and their holders, as well as monitor the events taking place at the entry points, as shown in Fig. 3. For instance, a typical event that usually gets logged by the system, is the opening and closing of the monitored door. This information needs to be registered and visualized in an event log, so that professionals interacting with the interface are able to confirm that the system is functioning properly, and ensure that no suspicious activity is taking place. Note that the user in Fig. 2 and Fig. 3, are not supposed to be thought of as the same type of user. The user in Fig. 2 is not a professional working with the system. Their only objective is to gain access to a delimited zone. Their attempts at doing so, might get monitored by the professional user, shown in Fig. 3.

(8)

7

1.1.2 Security

In the context of information security, there are three major concepts that are in need of protection – confidentiality, integrity and availability [10]. Information security and physical security are closely linked – they both deal with the protection of assets.

It is in many ways easier to define what is insecure than what is secure, since “secure” is not a definitive state. Even though something is deemed to be secure, it still always has weaknesses that can be exploited by an attacker. The best we can do, is to mitigate the risk for potential threats by using different methods of control. There are three different types of control – physical, logical and administrative [10]. One could argue that in the context of access control, most of the time, all of these controls are needed. Locks for instance, are a part of physical control. The authorization of access is a logical type of control. Administrative controls deal with policies, procedures and guidelines – an access control system can help to enforce these, as they monitor and log events, which can be used to hold people accountable.

1.1.3 Usability

Before videogames had its breakthrough and computers still were restricted to computer professionals, programs and systems were developed by the same programmers that would interact with the system [11]. When the computer became an everyday item in peoples’ homes, computer systems and user interfaces were no longer restricted to professionals and developers. Programs and applications were now to be used by a large number of people with different backgrounds [11]. When discussing how to achieve usability for an interface today, the interface should be designed to attract and cater for all users that are intended to interact with it. According to the ISO (International Organization for Standardization) 9241-11 standard [12], usability is defined in the following way:

“Extent to which a product can be used by specified users to achieve specified goals

with effectiveness, efficiency and satisfaction in a specified context of use.”

This definition contains three key-concepts - effectiveness, efficiency and satisfaction. In ISO 9241-11, the effectiveness describes to which extent a user achieves a specified goal in terms of accuracy and completeness. Efficiency expresses the consumed resources regarding the achieved results, while satisfaction describes how well the application fulfill the user’s needs and expectations. While this definition is seen as the primary goals of usability [13, p. 76] today, extensions and other interpretations are common [14]. One recurrent example is the learnability of the system [14, 15, 16]. This can be defined as how easy it is for a user to reach a level of understanding for the system and how to use it. Another example is the memorability of the system, which describes the ease of remembering a system’s functions after being away from the system for a period of time [15, 16, 17]. Another closely linked attribute to usability is the system’s accessibility, that defines how available a system and its functions are to different users. These include people with disabilities or other difficulties [13, 15] and are essential for people needing to use the application or system [13]. The different interpretations and explanations of what usability truly is leads to the conclusion that usability is a hard term to define.

(9)

8

1.1.4 Security vs. usability

The complex relationship between the aspects of security and usability requirements has been greatly discussed. The usual perception is that these two non-functional requirements are opposites of each other, and therefore have a hard time co-existing [5, 6, 7].

User authentication is a typical example of this phenomenon; a secure password should be both non-guessable and long to minimize the risk for passwords being compromised, and therefore increase security. At the same time, a user has a hard time remembering long and complex passwords, increasing the likelihood of the user writing the passwords down - this leads to a considerable security risk [8, 18, 19]. When developing software, security and usability are rarely being incorporated throughout the development process, and therefore one of them tends to get added on as an afterthought [9]. This often leads to poorly executed security or usability in the software [9]. The lack of usability in an otherwise seemingly secure system, may in fact actually cause the system to be insecure instead [20].

It is not always easy to distinguish what exactly security and usability is or is not, nor what a sought-after symbiosis between them would look like. Yee [3] defines it in the following way:

“Security isn’t about making all operations difficult; it’s about restricting access to operations with undesirable effects. Usability isn’t about making all operations easy, either; it’s about improving access to operations with desirable effects.” [3]

He then proceeds to define the balance between the both by stating that:

“Security and usability come into harmony when a system correctly interprets

the user’s desires.” [3]

Research has shown that security and usability can be integrated and co-exist when these two non-functional requirements are being considered and tended to from the start of the development [9, 20, 21].

1.2 Related work

When designing a GUI, the aim is to create a user-friendly interface that meets the demands and expectations of the end-users. In addition to that, if the GUI has any safety features included, other problems arise.

Johnston et al. [8] studied the interface of a Microsoft firewall, in search for the visibility of its security features. They implied that the users of this firewall were not encouraged to trust it, since the firewall activated itself and ran in the background, and therefore lacked visible feedback. The users were in other words unaware of being protected in the first place. Even when the firewall prevented attempted hacks, it failed to inform the user of this in an adequate way [8]. Johnston proposed improvements to the interface and concluded that these changes made the system both easier to use and more secure [8].

(10)

9

Karp and Stiegler [22] on the other hand tried to make security “invisible”, in order to not disrupt the user's workflow. Their primary focus was to hide the decision-making of security policies. However, in the end they wondered whether or not the user would deem the system to not be secure, as one of the users asked “how to turn on security” [22]. This might suggest that Johnston [8] may be correct regarding visualizing security features.

Gujrati and Vasserman [23] did an evaluation of two interfaces - a pre-existing interface and a modified version - for a disk encryption software. Their focus was directed towards novice users, and their objective was to improve the usability of this particular security software. They concluded that usability could be improved without removing existing features, and that the most prominent problems they found were due to lack of intuitive design, poor terminology and illogical or inconsistent task flow [23].

Furnell [24] examined security features in Internet Explorer and Word back in 2007. The main interest was to investigate whether or not there had been any advancements in usable security in practice, given that the topic had gotten increased attention in research. Furnell found that Internet Explorer had made certain progress, while Word on the other hand had worsened. Some of the identified usability problems correlate to the ones found by Gujrati and Vasserman [23], which might indicate that these heuristics are either difficult to achieve or tend to get overlooked.

Nohlberg and Bäckström [25] adapted the security information presented on a management information interface to fit the needs of a specific user; the managers at a company. They found that managers had a different mental model of security compared to security specialists, and that the former were more focused on risk in a strategic and financial sense. In order to aid this visually on the interface, the information was presented by giving a quick overview of the situation (through the use of charts and other graphic elements), and also removing detailed security information from plain sight [25].

Muñoz-Artega et al. [26] created a design methodology that applied a combination of visual and auditory feedback to security notifications. They suggested that different threats (of different

magnitude) should be accompanied by appropriate sounds. The threats should also be color

coded accordingly. Their results indicated that the use of a dangerous sounds (e.g. a tiger's roar) combined with the right color made the users more aware of the caliber of the security notification [26].

1.2.1 Usability heuristics and security heuristics

Design heuristics in the UI/UX-domain (User Interface/User Experience) are most often described as “general rules of thumb” for creating interfaces [27]. Usually, one is looking to comply with proven set of design rules in order to achieve a certain non-functional requirement, such as usability.

When it comes to usability heuristics, there are plenty to choose from [28, 29, 30, 31]. However, many of them consist of similar attributes. The seemingly most established one was presented by Nielsen [28] in 1994. He is considered to be an expert in regard to usability, and there have been attempts at creating new heuristics using Nielsen’s work as a foundation [8, 32].

(11)

10

In contrast, security heuristics are somewhat less homogeneous. There have been a couple of noteworthy contributions [7, 8]. One of the more prominent security heuristics were introduced by Johnston et al. [8] and are referred to as HCI-S (Human Computer Interaction Security). Like the previously mentioned usability heuristics, these too were created with Nielsen’s heuristics serving as a foundation.

1.2.1.1 Nielsen’s usability heuristics

It was the collaborative heuristic research between Molich and Nielsen [33, 34] that would later help Nielsen to define 10 usability heuristics [35]. These heuristics, presented in [28, Tab. 1] are used for creating usable interfaces [28]. They exemplify what a system should incorporate in order for users to be able to perform desired tasks without complications.

Table 1:Nielsen’s usability heuristics.

No. Criterion Description

1 Visibility of system status The system should always keep users informed about what is going on, through appropriate feedback within reasonable time.

2 Match between system and the

real world The system should speak the users' language, with words, phrases and concepts familiar to the user, rather than system-oriented terms. Follow real-world conventions, making information appear in a natural and logical order. 3 User control and freedom Users often choose system functions by mistake and will need a clearly marked

"emergency exit" to leave the unwanted state without having to go through an extended dialogue. Support undo and redo.

4 Consistency and standards Users should not have to wonder whether different words, situations, or actions mean the same thing.

5 Error prevention Even better than good error messages is a careful design which prevents a problem from occurring in the first place. Either eliminate error-prone conditions or check for them and present users with a confirmation option before they commit to the action. 6 Recognition rather than recall Minimize the user's memory load by making objects, actions, and options visible. The

user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate.

7 Flexibility and efficiency of use Accelerators — unseen by the novice user — may often speed up the interaction for the expert user such that the system can cater to both inexperienced and

experienced users. Allow users to tailor frequent actions.

8 Aesthetic and minimalist design Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of

information and diminishes their relative visibility. 9 Help users recognize, diagnose,

and recover from errors Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution. 10 Help and documentation Even though it is better if the system can be used without documentation, it may be

necessary to provide help and documentation. Any such information should be easy to search, focused on the user's task, list concrete steps to be carried out, and not be too large.

Source: Nielsen [28].

1.2.1.2 HCI-S

Johnston et al. [8] introduced the term HCI-S in 2003. The objective was to broaden traditional HCI (Human Computer Interaction) design principles with security aspects.The result consisted of six heuristic criteria [8, Tab. 2] for implementing computer security in HCI: convey features,

(12)

11

visibility of system status, learnability, aesthetic and minimalist design, errors and finally satisfaction. If all of the six criteria are implemented, a seventh criterion is indirectly also

considered to have been met - trust. These criteria can be applied when incorporating security features in a GUI, while still keeping it user-friendly. By maintaining the ease of use, the user is less inclined to try to bypass the security features, which in turn helps to support the security of the system [8].

Table 2: HCI-S heuristic criteria.

No. Criterion Description

1 Convey features The interface needs to convey the available security features to the user. 2 Visibility of system status It is important for the user to be able to observe the security status of the internal

operations.

3 Learnability The interface needs to be as non-threatening and easy to learn as possible. 4 Aesthetic and minimalist design Only relevant security information should be displayed.

5 Errors It is important for the error message to be detailed and to state, if necessary, where to obtain help.

6 Satisfaction Does the interface aid the user in having a satisfactory experience with a system? Does the interface lead to trust being developed?

7 Trust It is essential for the user to trust the system. This is particularly important in a security environment.

(13)

12

2 Problem statement

An access control GUI usually has multiple types of users. Depending on what line of work these users are in, they may interact with the GUI in different ways, and therefore have different needs. Consequently, there is a strong need for usability, since all of these users need to feel comfortable navigating the interface, regardless of their educational background. Additionally, as the access control GUI is a part of a security system and incorporates security features, there is also a strong need for it to be secure. In other words, the problem that will be investigated arises from the need of both security and usability in this GUI, as these non-functional requirements usually tend to cancel each other out.

2.1 Objective

The objective of this thesis is to create an interface for an access control system, where both usability and security can coincide without dissatisfying users from different target groups. More specifically, we will be attempting to create an interface solution concerning the event log, where both user requirements and heuristic criteria are met.

2.2 Research question

This research is focused on two different types of users; those who do administrative related work (hereafter referred to as A.R.W) and those who do security related work (hereafter referred

to as S.R.W). The main distinction that is made between these two groups, is that S.R.W-users are

professionals that have security as their primary job assignment, whereas A.R.W-users are professionals that only deal with security to a certain extent.

Our research questions are the following:

1. Do user requirements (for an access control event log) differ depending on whether or not the respondent is from a target group that deals primarily with security issues?

1.1. What are some of the primary concerns for the A.R.W-users who use the event log? 1.2. What are some of the primary concerns for the S.R.W-users who use the event log?

2. To what extent is it possible to implement user requirements while complying with both Nielsen’s heuristics [28] and HCI-S heuristics [8]?

2.1. Can the gathering of requirements from A.R.W-users and S.R.W-users aid the compliance with Nielsen’s heuristics and HCI-S heuristics?

2.3 Scope and limitations

In order to answer these questions, we have chosen to include users of pre-existing access control GUIs. Some of these users have used the same GUI from the same supplier, while others have used different ones from other suppliers. We will not be attempting to reach a specific number of different GUIs, since both time and access to users are limited, but we are interested in a certain amount of diversity when possible.

(14)

13

As previously mentioned, the users for this research have been categorized into two groups. This means that other types of users, who do not fit into either one of these categories, have been excluded.

The research will be limited to the part of the GUI that displays the logging and filtering of events. In other words, we will not be creating a complete interface with several types of features, as features may vary depending on which type of market the product is trying to reach. The reason for choosing the event log as our primary focus lies in the fact that logging and filtering can be considered as must-have features in any access control GUI. It is a key feature for establishing security, as it enables a chain of evidence if the system is ever exposed to vulnerabilities. It is important to examining current usability issues and establish usability in event logs, as this feature is used by many different users for various reasons. A system that is perceived as secure, could in fact be the opposite if a user does not understand the security features inside the system, or in other cases tries to work around the security to increase usability [36].

(15)

14

3 Method

An interaction design process [15] has been chosen as the overall methodology for this research. This process balances structure and flexibility, and helps to drive a project forward by working through the designated steps [15]. It can be described as a lifecycle consisting of four phases:

1. Establishing requirements. Research is conducted to understand the user’s needs. 2. Designing alternatives. Design choices need to be made. This can be aided with the help

of heuristics [37].

3. Prototyping. In order for the design to be assessed, a prototype is made.

4. Evaluation. An evaluation of the prototype is performed to ensure that the product meets user requirements and/or heuristics [15].

This type of lifecycle has an iterative nature, which means that once the fourth phase has been completed, one can choose to go through all of the phases again. If the design is deemed good enough, one can chose to exit the cycle instead, and start implementing/deploying the design [37].

Our process has undergone an initial (single) iteration, as limited time was a factor. Our methods mirror the interaction design process in the following way:

1. Establishing requirements. In order to understand the user’s needs, semi-structured interviews have been conducted.

a. The interviews have been subjected to thematic analysis in order to convert these to user requirements.

2. Designing alternatives. This phase has taken the user requirements into account. Suitable heuristics have also been chosen (Nielsen’s & HCI-S).

3. Prototyping. A prototype of an event log has been made, corresponding to user requirements and heuristics.

4. Evaluation. A heuristic evaluation of the prototype has been performed by experts.

3.1 Semi-structured interviews

Semi-structured interviews served as the starting point of the data gathering, as they work well when trying to understand user goals, tasks and task flows [38]. The respondents consisted of users of pre-existing access control GUIs, which means that they had experience and special knowledge of working with these systems. In other words, respondents have been selected through purposive sampling [39]. The users were categorized into two groups - those who do A.R.W and those who do S.R.W.

Semi-structured interviews were conducted, and the predefined questions (appendix I & II) were the same for both types of respondents. These questions were all concentrated towards the event log. Because of the flexibility enabled by this interviewing technique, respondents were encouraged to speak their minds when touching on subjects close to security and usability. The semi-structured interview allowed the respondents to raise other issues not covered by our predefined questions [40], which is something that a structured interview would not have permitted. In a structured interview, all respondents are asked the same questions, in the same way, with no opportunity to talk freely about the subjects. Since the objective for conducting the

(16)

15

interviews was to gather and uncover the needs of all respondents, a structured interview technique was not applicable [40]. Neither was using an unstructured interviewing technique, were the respondents are given a topic to explore and talk about without us interfering [40], as we had specific questions about the event log, and we needed answers for a set of questions from all respondents.

All answers from the different respondents were first collected. The answers then underwent a thematic analysis [41].

3.2 Thematic analysis

Thematic analysis is a systematic process for identifying themes in data and coding these accordingly [41]. This analysis method was suitable for this research, as there already existed a determined general interest [41] - we were for instance interested if answers from A.R.W-users and S.R.W-users differed. Thematic analysis is also a preferable method in cases where all of the data has been collected [41], which was true in our case, as we conducted all of the interviews before we started analyzing the answers. The first stage of the analysis is referred to as “open

coding”, which can for instance involve making notes in the margins of transcriptions [41]. The

material should be thoroughly examined, with several read-throughs [40]. One can then proceed to compare these notes to each other, in search of an emerging theme. The next stage is called

“axial coding”, which refers the actual placing of the different codes into themes [41].

The results from the analysis partially answered our research questions, but also supported the formulation of user requirements.

3.3 Prototyping

Once the user requirements had been formulated, wireframes [42, pp.138-139] were sketched and designed. By creating wireframes for our first design iteration, we had the flexibility to alter, update and visualize features. A lo-fi (low-fidelity) paper-prototype [42, pp. 140-141] was created using the wireframes. The prototype was used to test alternative ways to interact with the interface together with the sketched design, while still being flexible for adjustments and improvements. This led to many different sketches, drawings and ideas, which gave us a good starting point for creating a hi-fi (high-fidelity) prototype [42, pp. 141-142].

The hi-fi prototype was created using Photoshop [43] and a web-based prototyping tool, MarvelApp [44]. The prototype aimed to reflect both the needs that the respondents had presented, as well as the heuristics provided by Nielsen [28] and Johnston et al. [8]. Therefore, both security and usability aspects were infused into the prototype. The prototype was later subjected to a heuristic evaluation, to see if the heuristic criteria were met. Since the evaluation required us not to engage with the evaluators, an interactive prototype was crucial, as it could give feedback independently without our assistance.

3.4 Heuristic evaluation

In order to conduct a heuristic evaluation of the hi-fi prototype, unbiased experts were asked to examine it. Nielsen [45] suggests that one should aim for about 3-5 evaluators. Larger numbers

(17)

16

than that are considered to only add minor contributions overall [45]. We aimed to follow Nielsen’s suggestion. Each of our chosen experts had a background in UI/UX design and possessed domain expertise within the field, which is a necessity when performing this type of evaluation [14].

The evaluation was performed by letting the experts inspect and explore the prototype unattended. They then assessed whether the heuristics had been met by filling in a form (appendix

III). Both Nielsen’s usability heuristics [28] and the HCI-S heuristics [8] were included in the form.

The experts were also encouraged to write down their comments, both positive and negative, as well as suggestions for possible solutions to a specific problem. By letting the experts evaluate the prototype using both sets of heuristics, the prototype was thoroughly examined regarding both security and usability.

When the evaluations were completed, the data was analyzed qualitatively. The number of experts who assessed that a criterion was met, was summarized and used as a validator regarding if the heuristics were met or not. The comments and suggested solutions that we gathered from the experts were analyzed in a qualitative way, finding themes, keywords and phrases. This contributed both to answering our research questions as well as adding valuable information about how a possible finalized interface would look like.

3.5 Alternative methods and discussion

As mentioned in the problem statement, the objective for this study was to create an interface for an access control system, that incorporated both security and usability without displeasing users from our two target groups. To achieve this objective, the user's knowledge, needs and expectations had to be identified and analyzed. This, in combination with the fact that a user’s experience of a system is highly individual, showed that the data we needed to collect from the users was of a qualitative type.

The advantage of conducting semi-structured interviews was that the different respondents were encouraged to speak freely about their experiences, which leads to a more profound understanding about their feelings and perceptions [28]. Another way to collect this type of qualitative data would have been by observing [38] the users, gathering both visual and verbal information while they were working with an access control system. The problem with observation in this case, is that it might have been too intrusive to observe, e.g. the head of security, while they were performing security related work - observations are typically not suitable when dealing with security and/or privacy matters [38]. Also, observations require a certain geographical closeness, which we did not have with all our respondents.

An alternative approach to the heuristic evaluation would have been to let the users evaluate the prototype regarding user requirements. This would have enabled participatory design - especially if the users would have been involved in the prototyping as well. However, this research was a bit more focused on heuristics than user requirements, and therefore, we have chosen to prioritize a heuristic evaluation performed by experts.

(18)

17

4 Establishing requirements

4.1 Results of semi-structured interviews

Each of the groups (A.R.W & S.R.W) consisted of three respondents, bringing the total to six respondents. Many of the respondents had encountered multiple different access control GUIs. Some of them had used the same systems. All in all, we estimate that the number of different systems used by the respondents were around 7+. All interviews were recorded, except for one

(consent to be recorded was not given). The recordings were transcribed. Extensive notes were

taken during the interviews, which aided not only the transcriptions, but also the interview which was not recorded.

The respondents from the S.R.W-group primarily had security as their main job assignment, and they generally came in contact with event logs on a regular basis. The S.R.W-interviews lasted approximately 1 hour (±10 min).

The A.R.W-group’s job assignments were generally more diverse. Typically, security related work was a portion of their daily work, while the rest of the workload consisted of other matters. They therefore came in contact with event logs less often than the S.R.W-respondents. The A.R.W-interviews lasted approximately 45 minutes (±5 min).

The S.R.W-respondents are referred to as #S1, #S2 and #S3, while the A.R.W-respondents are referred to as #A1, #A2 and #A3.

4.1.1 Using the event log

Our first set of questions were focused on the actual usage of the event log.

4.1.1.1 Question 3.1-3.3

The respondents were asked about the learnability of their access control event log.The answers were mixed. Respondent #S3 said that it varied a lot from system to system, claiming that certain systems probably could be comprehended within a couple of hours of use, while others could take months to learn. Respondent #S2 said that their system was pretty easy to learn but pointed out that they had an IT-background, and that this probably helped. Respondent #A3 said that the event log was confusing because of the information and data that was shown in the event log. The respondent stated that the information was very technical and not “human-readable”. Respondents #A2 found their system to be simple, while respondent #A1 stated that they did not find the system self-explanatory:

“The user should understand how to do things without an instruction. That doesn't apply here – it's almost like an instruction-manual should’ve been included, telling you what you may and may not do, what you can and cannot do.”

They were then questioned about what drives them to look at the event log, and in what types of situations they come in contact with it.They were asked to describe the situations in detail- how often did these situations occur, if there were some more frequent than others, and what they usually would do once they had gotten the information from the event log. For two of the

(19)

S.R.W-18

respondents, #S1 & #S3, alarms were a common reason for checking the event log; burglar-alarms and operational burglar-alarms were of interest. Respondent #S1 said this regarding operational alarms:

“If the power supply is cut, the up power turns on - an alarm is sent. If the back-up power fails, another alarm is sent. That log is extremely important. We want to know how often this occurs – sometimes you have to discuss the matter with your electric company.”

They would also use the event log to monitor possible faulty locks and see if the doors were closing properly. Respondents #S1 & #S3 also frequently checked how much pressure specific doors were under. However, one of the S.R.W-respondents, #S2, disliked using the event log, and often avoided it. This was enabled by the fact that their company also had video surveillance:

“No, I don’t use it at all - because we have a camera solution, and if I have an idea of something suspicious happening in the area, I will go to the camera first. They have sensors, with timestamps. That’s easier. If the event log would send me messages - then I would use it instead of the camera system. You always use the easiest way.”

The respondent elaborated:

“There’s no alarm that sounds if there is an unauthorized user. Could be nice if there was a text or something - if there was a user who didn't have the proper code or was using an illegal card or something like that. I would like to know for instance - if someone tried to enter at 4 o’clock in the morning, since our employees are only authorized between 6-23. I know that it’s in here [the information], but when it’s in here - you have a lot of other things you have to look at.”

The seemingly most recurring event was the prolonged opening of a door, which produces an alarm. Respondent #S3 explained that it was not uncommon to have 15-20 alarms (different

types) per day, but not all of them were acted on immediately. The same respondent stated that

work experience and knowledge of the usual environment flow determined which events were of higher priority.

The A.R.W-respondents had other motivations for checking the event log –respondent #A1 used it when a unit seemingly was broken, or to troubleshoot readers that were not responding to access cards as intended (problems with access rights). They sometimes checked which unit was which, by blipping their card at an entry point, and then going to the event log to see which unit had granted the access. All of the A.R.W-respondents explained that the event log was used to enforce accountability from the personnel, as these were required to check in at certain areas within certain timespans. One of the respondents, #A2, said that they did not really use the system to monitor the building, but rather as a tool for going back and fetching information, which could then be made into a report.

4.1.1.2 Question 3.4-3.8

We then proceeded to ask the respondents which features in the event log they most frequently used, and how these features supported them in their work. The respondents mentioned

(20)

19

functions that were a part of the system, but not necessarily were part of the event log. However, filtering the events was brought up. Another function, that was mentioned by respondent #A2, was the exportation of the event log (often as a CSV-file).

We asked if they found any information or functionalities to be unclear. One of the S.R.W-respondents, #S1, felt like the information presented was reasonably clear. Another one, #S2, pointed out that it was the amount of information that was the problem:

“On a busy day, we have 400 people using the card - it would be almost impossible to see a problem.”

The respondent was probed further on this matter and was asked whether they felt like the information got lost because of redundancy in the events, which the respondent answered yes to.

Respondent #A1 said that they did not feel like they knew what type of information was retrievable from the system, or what it all meant. There had been situations where they had been accompanied by security support, and the support had pointed out a few things quickly, but there had never actually occurred a proper education on the matter. Another A.R.W-respondent, #A3, said that the event log in their system showed a lot of different numbers that the respondent did not know what they meant or what they stood for.

The respondents were asked how they went about looking for specific event - which filters they frequently used, and which (if any) filters they did not use. All of the S.R.W-respondents mentioned time-filtering; being able to define the time-range was important. Additionally, respondent #S2 (who preferred video surveillance over event logs), pointed out that the cameras technically filtered out information for themselves, as they only captured footage when motion was detected, as well as each camera in itself being a filter regarding a certain area. The respondent suggested that there should be a link between these systems, where the combination of camera footage and event log information was presented.

Other filters mentioned by S.R.W-respondents included event topic, user, unit, and alarms. Respondent #S1 said that they usually chose a certain time-range (e.g. a month) that was always visible in the event log (the rest of the information could be retrieved from the database if needed). Respondent #S3 praised the fact that their system had graphic representations of each floor plan, and that the respondent could therefore easily choose a specific unit and see the unit’s log. This system seemingly also had different tabs for different types of events, which meant that the tabs served as pre-applied filters. The tab that was most often looked at by the respondent showed alarms.

None of the S.R.W-respondents could give an example of filters that they did not use. They acknowledged that these filers probably existed, but they had not noted them.

One of the A.R.W-respondents, #A1, did not use filters at all, as they were only in need of the most recent 2-3 events, when performing their job assignments. Respondent #A2 (who created reports

once monthly) was not aided much by the filtering when doing so, as the exports did not keep the

filtered structure, and had to be manipulated in the exported file later on. But occasionally, the respondent would look for access-denied-events or look at specific readers.

(21)

20

4.1.1.3 Question 3.9-3.10

We asked the respondents if they could describe a time where they could not perform what they wanted to while using the event log.Respondent #S1 said that it sometimes could be difficult to remember in which order a certain task flow was supposed to be done. The other two S.R.W-respondents said that they could perform what they wanted to, unless the system was down in some way. This quote is from respondent #S2:

“There can be situations where to cameras aren’t recording, when the alarm system is offline, the software is offline. It would be nice to have a warning,[Imitating the

system] “I’m offline, please do something”, if there’s something wrong with the

system.”

To conclude the first set of questions, we asked in what way the system assisted (or did not assist) the respondent whilst trying to accomplish their job tasks.All of the respondents’ answers were positive - they stated that the system made sure that only allowed people were allowed access, and that the system was very useful both in the prevention of and the following up on security breaches.

4.1.2 Design and functionality

Our second set of questions were focused on the design aspects and functionality of the event log.

4.1.2.1 Question 4.1-4.3

We began with a question about their opinions regarding the event log’s design and functionality. Respondent #S2 said the following when asked about the design of the event log:

“[Laughing] I don’t care about design, it really doesn’t matter.”

Respondent #S3 said that since they used multiple systems, the design of the event log differed. On the matter if the respondent thought there was any advantages or disadvantages in terms of design and functionality between the systems, the respondent pointed out that some systems included very detailed technical information in the event logs, and said:

“I think that only necessary information should be in an event log. So that you can quickly find what you are looking for.”

Respondent #A1 said that they experienced the design of the event log to be cluttered, containing many buttons. The respondent referred to the design as “old school”, making a reference to the operating system Windows 97. Another respondent, #A3, said:

“...you can see from how it’s laid out, it’s a technical person [that] should be the one using it, not a regular person.”

We then proceeded to ask which features they liked and disliked, and if there were any missing features that they would like to have. Respondent #S3 said that a graphical blueprint, showing different entry points and readers, was a good way to orientate oneself – especially for a user that

(22)

21

does not have a lot of knowledge about the system and the area that the system is covering. The respondent stated that a system that only had text-based content was not as nice to work with, as a system that included graphic visualization. The respondent also pointed out that in some systems, they could click on an event to open the graphical presentation, showing the event in the blueprint. The respondent found this feature helpful when wanting to find information quickly. Respondent #S1 said that their event log was designed using tabs, which was a good way to interact with the system.

Respondent A#1 said the following about less liked and missing functions:

“I’ve learnt something that I assume works, so therefore I do it like that. If it’s right

or wrong, I don’t know. But it feels like there should be a smoother way, like a more obvious way to do things. You start with one step and then there are tabs, and you can map down... Like it is now, there are buttons and then there’s suddenly a save button in the middle somewhere, which is not logical either.”

The respondent then made a comparison on good and bad feedback:

"It [the system] usually warns before removing stuff. [Imitating the system] “This will erase everything if you do this, do you really want to do it? Yes.” And that’s good, because it is an easy mistake to make. But when you start the system, it’s more like - okay, where do I start? It is not clear that you should start from the left, and that there are different things that you should go through."

Respondent #A2 said that they would have liked to be able to build reports in the event log by filtering, and then export them. They were also in need of read-only permission levels of the system interface.

We then asked more specifically about events - which events the respondent was in need of seeing

(and if there were any that they would rather not see), which events were of most importance to

them, and finally, how these events should be visualized and what type of information should be included in these visualizations. Respondent #S1 said that it could be useful to know how many people were in a specific room, and how people moved through the building, but also added that this type of feature required that the system demanded authentication both when entering and exiting the room, which few companies implement (often, doors may be opened from the inside

with the push of a button). Events showing denied access were also important, as well as alarms.

Respondent #A1 was mostly interested in seeing which card had been used on which unit, and when.

When asked about how these events should be visualized, both S.R.W-respondents and A.R.W-respondents mentioned blue prints, to clarify where an event was taking place. The groups also agreed that detailed information could come in handy and should always be retrievable – but, this did not mean that it should always be presented. Respondent #S2 explained when detailed information was nice to have:

“The more evidence I have for showing weaknesses, the more money I will likely get to spend on security.”

(23)

22

There were also suggestions of being notified, either by text or email from respondent #S2 & #S3, of certain events. If cameras were part of the company's security system, photos should be included together with the event. Some event logs used red text, or blinking read elements on the blue print, in order to notify the user, which helped, but demanded that the user was looking at the screen. Respondent #S3 mentioned sound as a notification in one of their systems:

“The sounds were different depending on the alarm – if glass had been broken, a sound of glass shattering was played. If it was a major alarm, a siren. It helped in situations where I was maybe doing something else at the time, and then I would hear a sound play - I could tell by the sound if it was important or not, and if I needed to act quickly.”

Respondent #A1 stated that seeing identification numbers of the units were not helpful and wished that some information was allowed to be renamed. Respondent #A2 said that they would like to be alarmed if a door stayed opened and also mentioned being alerted by sound in those instances. Respondent #A3 said that they would like the wording in the event log to be simplified, to make it more readable and less technical.

4.1.2.2 Question 4.3-4.5

The respondents were then questioned about filtering. They were asked to describe a time when the filters did not suit the respondent's specific purpose. They were also asked whether applying multiple filters worked in an adequate way.Pre-applied filters (different tabs, for different events) were mentioned by two of the S.R.W-respondents, #S1 & #S3. They preferred this in most situations, as it was faster. However, sometimes this entailed a lot of jumping back and forth between tabs, so it could be useful in some situations to have one long scroll list with all the events combined, and then including/excluding events by applying filters. This on the other hand, required more knowledge from the user, #S3 expressed.

Regarding possible missing filtering capabilities, respondent #S1 mentioned filtering by buildings – doors and floors were possible to filter by, but sometimes there was a need to know which units were in which buildings.

Respondent #A1 seldom used filters and therefore didn’t have much to say about the topic. Another found filtering to be easy.

To conclude the second set of questions, we asked in what way the system aided the respondent in finding/noticing events that required their attention. We also asked if the system assisted the respondent in solving a problem, once an error or problem had occurred. Respondent #S3 said that the system aided them in such a way, that they were given information about where, locally speaking, an event had occurred. However, the system failed to inform them about technical problems within the system – for instance, network connection problems were not logged. Seemingly, this was something they had to learn to detect through work experience – this sparked a question about newly employed personnel, and their lack of experience. The respondent answered that all new employees underwent an education on which events should be acted on quickly and so forth, but stated that it was not always easy for them to recall exactly what to do in certain situations.

(24)

23

Another issue that was brought up was the lack of being informed by the system at startup if something required the respondent's attention. Stated by respondent #S3:

“When I arrive to work in the morning, I would like the system to tell me – without having to go through the alarm log and filters and everything - what happened when I was gone... [Being notified] Either by mail, or presented to me directly after system startup; these are the events that happened from when you logged out yesterday, until you logged in today.”

Respondent #A2 spoke about a speaker system they had, which allowed them to configure sound, and wondered whether it would be possible to integrate it with their event log:

“They [the speakers] are so configurable... I like stuff like that, it’s fun to play around with.”

4.1.3 Open questions

We ended the interviews with a couple of final open questions, to see if there was anything else that the respondents have thought about regarding the event log.

4.1.3.1 Question 5.1-5.2

The ability to customize the event log to a certain extent was brought up by respondents #S1 & #S3 as a good feature – different users have different ways of looking for events, so being able to create a setup that worked for them was preferable. However, we were also informed that only certain users had customization-rights, in order to keep the interface familiar for everyone.

Respondent #S1 mentioned that it would be nice if one was able to save a certain applied filter, for future use. This respondent also suggested that it would be good if one could build their own log with events that they found interesting.

Respondent #A2 said that it could be nice if one was able to apply a certain color to a certain event, enabling a quick visual of events that were of significance. Another one of the respondents, #A1, wanted to see an improvement in user-friendliness, as well as in the use of color. The respondent stated:

“The design is kind of depressing... It’s not very fun to play around with, I don’t find it fun to use. But I'm not the most technically-advanced person either... I do what I have to do, and then I leave the rest up to those who know it [the system] better.”

Respondent #A3 said the following regarding the design of the event log:

(25)

24

4.2 Analysis of semi-structured interviews

We each, separately, performed open coding on all of the transcripts/notes. The transcripts and notes were read multiple times. Then, we discussed our findings together, and proceeded with the axial coding. The result of the axial coding generated multiple themes. Each theme consists of different categories, found in the codes during the analysis. The themes and categories we found in our data is presented in section 4.2.1 to 4.2.4. In section 4.2.5, the findings from the analysis are presented as user requirements. To give a better overview of how these requirements came to be, the requirement prefixes (letter.number) are referred to throughout the text in sections 4.2.1-4.2.4.

4.2.1 Customization

The needs of an event log differ depending on both a company's security strategy and on the personnel using it. This is where customization comes in to play. To a certain extent, it correlates with “flexibility and efficiency of use” by Nielsen [28], which suggests that the system should enable the ability to speed up frequent actions made by the user.

The two groups of respondents both use the event log to control that only authorized people are granted access to the closed off sections. Yet, there are some distinctions in assignments between the two groups. The A.R.W-users main reason to use the event log was to ensure accountability and see to that assignments and activities had been executed. The event log was also viewed if a reader or controller was suspected to be damaged or broken. The majority of the S.R.W-users however, used the event log to monitor real-time events, to find issues and illegitimate behavior. This course of action is by far more frequent from the S.R.W-users, as they need to respond to sudden events at the time when they occur. However, depending on what type of security work is included in the user’s job description, the need for monitoring live events can be less significant. This indicates that the users job assignments are an important factor when understanding the need for accessing the event log.

Respondents from both user groups explained their need for customization during the setup of the system and its interface. Answers demonstrated that different permission levels to the system was much needed and important, and that the system should be able to support this kind of customization of access right to the interface and event log (C.1, C.1.1). However, respondents also said that it is important for a user to recognize and be familiar with the interface they are working with, illustrating that a limit for customization of a system is also of importance (C.2).

The difference between the usage of the event log, including both the different types of users as well as the different types of job assignments the users have, demonstrates the need of a customizable interface. The result showed that S.R.W-users have a need of getting detailed information about ongoing issues in the event log, while an A.R.W-user is not interested of knowing them. The two different target groups also use the filter function of the log in different ways. S.R.W-users frequently use many different filters to find a specific event in the event log, whereas the A.R.W-users do not (C.3). The S.R.W-users differed amongst themselves when talking about the most useful filters, except for the time-range filter. This indicates that different companies and employees have different needs and should be able to save frequent filters (C.4) that they use. This is also supported by the respondent who suggested this type of functionality

(26)

25

and would also be compliant with Nielsen’s usability criteria for “flexibility and efficiency of use” [28].

Both target groups use the event log as a tool for accumulation of information, for example to be used as evidence in a possible criminal investigation. The ability to create and export reports from the event log is an important feature for A.R.W-users, but the content and level of details in the reports differ depending on the report's intention. Users expressed the need for a simple way to create reports using the different filters inside the event log, removing unnecessary steps to customize their reports (C.5).

4.2.2 Notification and feedback

Both Nielsen’s heuristics [28] and HCI-S heuristics [8] recognize the importance of proper visibility of system status and error handling. The answers from both groups of respondents suggest that these criteria are being violated in their access control systems. Respondents mention not being informed about technical problems within the system (N.1). Moreover, the handling of a problem was up to the user to figure out, which in turn demands work experience. For instance, experienced users knew that a lagging system meant network connection problems, and also knew that is was time to call the electrical company if the system showed multiple signs of cut power supply – this was not conveyed by the system (N.2). It was also noted that the system at startup failed to inform the user about possible things that they should go through (for instance,

some event happened during the night, investigate this) (N.3).

The way of being notified could seemingly also be improved upon - there were S.R.W-users who expressed a desire of getting a text or email when certain events occurred (N.4). Also, just as Muñoz-Artega et al. [26] suggested, multiple users felt like sound (N.5) and color (N.6) could aid them in their work, both when getting notified and when prioritizing events. This suggests that there is a need of being notified when the user does not have the interface directly in front of them – both texts, emails and sounds could potentially accomplish this.

There was also a statement from a user about good feedback in the form of confirmation boxes when performing something critical – this prevents the user from making an uninformed decision

(N.7).

4.2.3 Overview

Nohlberg & Bäckström [25] spoke about enabling quick overviews and using graphic elements to support this. Many of the respondents mentioned blue prints as an easy way to quickly determine where an event had occurred, and stated that in these contexts, graphic visualization was superior to text information (O.1). It was important that graphic visuals and text were closely linked (O.2).

Nohlberg & Bäckström [25] also suggested removing detailed security information from plain sight, which is consistent with the “aesthetic and minimalist design”-criterion described in both Nielsen’s heuristics [28] and HCI-S [8]. Some respondents said that their systems logged quite detailed technical information at times, and would have preferred to not have this present at all times (O.3). Moreover, the amount of information present was also a challenge, as the redundancy (O.4)of certain events watered down the more important events. Unless a timespan-filter was applied, the information was an endless scroll. This might be another reason, besides

(27)

26

being faster, to why pre-applied filters are preferred over applying filters oneself - the information amount became more manageable when events already were sorted according to topics (O.5). However, even though the information in divided into topics, one should always be able to combine events from different topics if needed (O.6).

Even though the respondents never specifically mentioned numbers, counting events seemed to be a reoccurring theme. Sometimes, respondents wanted to check the pressure on a specific door

(number of entries), other times they were interested in how often the electricity went down (number of power shortages). If not those, then number of alarms, number of people, number of

network connection problems. This makes us believe that numbers are meaningful when conveying quick and useful information, and might indirectly also help discover system weaknesses (O.7).

Another suggestion that might fit into this theme, is the possibility of giving the user the ability to build logs themselves, consisting of events that they found interesting (O.8). The user would then not be overwhelmed by other unimportant information, while investigating a chain of events.

4.2.4 Design

The respondents from the different groups seemed to have different mental models of what the word “design” meant.

The answers of the S.R.W-users indicated that they felt like design meant “pretty” or aesthetically pleasing. They expressed that they did not care too much of the design in the GUI, as long as the features they needed functioned as they were intended to. However, when touching on subjects regarding design elements, such as icons, layout and ways to interact with the interface, the respondents did express the need for good design in the GUI. The respondents did for example mention that different colors in the event log would be helpful for visualizing important events. They also mentioned that the interface should be as simple as possible(D.1) so that information

can be found and accessed quickly (D.2).

A.R.W-users thought of good design as something that allowed them to be more playful (D.3) when using and learning the system - the term “play” was actually used by two A.R.W-respondents. Good design meant giving the system a sense of modernity (D.4), which could intrigue them to explore the possibilities the system provided them. The respondents expressed that a system that is not intuitive is difficult to understand and learn. The fact that these systems often also used poor terminology when explaining events that had occurred, was probably not beneficial either (D.5). This indicates that known design conventions should be used when designing an interface as Johnston et al. [8] suggested, since it increases the learnability of the interface.

These differences might suggest that S.R.W-users need less motivation to use the system. After all, using the system is an important aspect of their job. A.R.W-users on the other hand have more diverse job assignments, and therefore prioritize using the system differently.