SECURITY AWARENESS THROUGH MICRO-TRAINING: An initial evaluation of a context based micro-training framework

(1)

SECURITY AWARENESS THROUGH MICRO-TRAINING:

An initial evaluation of a context based micro-training framework

Bachelor Degree Project in Computer Science G2F, 15 ECTS

Spring term 2014 Jonas Werme

(2)

Abstract

Phishing is a growing problem. Most current research about mitigating damages from or preventing such attacks is aimed at either preventing users from receiving and clicking links to fraudulent sites, or using indicators to show the user whether a visited site can be trusted. This thesis focuses on education of the user for situations where a link has already been clicked, raising awareness of available security indicators. This is done using an

implementation of a framework developed by Dr. Marcus Nohlberg to provide micro-training within and about the context of e-commerce. An experiment using two groups of students, and four webshops created for the experiment, show positive results for the group receiving micro-training from the framework implementation.

(3)

1 Introduction

Phishing attacks exploit human vulnerabilities, luring victims to fraudulent web sites through emails (Kumaraguru, Sheng, Acquisti, Cranor, & Hong, 2010) or Instant Message services (Parno, Kuo, & Perrig, 2006). A fraudulent web site allows an attacker to collect sensitive information for purposes such as identity theft, corporate espionage or financial fraud (Kumaraguru et al., 2010). Most research with the aim to combat these attacks focuses on either eliminating the threat or warning the user of possible attacks, rather than on educating the users in how to identify and avoid such frauds (Kumaraguru et al., 2010).

This thesis focuses on investigating the impact of using event driven situational or contextual learning by implementing a learning module based on a framework developed by Dr. Marcus Nohlberg. An interview with the creator of the framework, to iron out the details of what the framework is and what it is intended to do, was deemed a necessity for this thesis.

To evaluate the framework within the context of e-commerce an experimental approach was chosen. The experiment was designed with validity threats in mind and implementation was done by creating a set of e-commerce sites, half of which deliberately contains signs of untrustworthiness. Participants in the experiments were divided into two groups where one group were given micro-training in identifying the implemented indicators of

untrustworthiness. Both groups were asked to buy the same items and their efforts were recorded, both using third party software in the form of GhostRec and by storing data about the purchases in a MySQL database.

(6)

2

2 Background

There are many types of attacks aimed at making users enter sensitive information into websites controlled by entities with malicious intent. According to Dr. Marcus Nohlberg (see Appendix A) this can be achieved by a man-in-the-middle attack, where an attacker answers the users DNS requests with an address pointing to a malicious server instead of the intended server, e.g. when trying to use a third party payment service in a webshop. Another common way, as explained by Kumaraguru et al. (2010), to lure an unsuspecting user to end up at an attacker-controlled web site is to send e-mails containing links to a fraudulent site. Links can be masqueraded into displaying one thing to users but linking to something completely

different, e.g. a malicious web site. When masquerading links it is common to use the name of a real site, a company name or even a real URL to a legit site, as was the case in the study performed by West, Carronade, and Ferguson (2005).

2.1 Phishing

According to Kumaraguru et al. (2010), the phishing attacks are semantic attacks (sometimes referred to as cognitive attacks) that exploit human vulnerabilities rather than exploiting system vulnerabilities. Phishing attacks are relatively easy to perform and are often successful, much due to lack of computer knowledge on the user side (Dhamija, Tygar, & Berkeley, 2006). The attacks are commonly carried out by sending out large amounts of emails (Kumaraguru et al., 2010) or by contacting users on various Instant Message (IM) services (Parno et al., 2006), e.g. Yahoo Messenger, MSN Messenger, Skype, etc. Messages sent out during phishing attacks are aimed towards luring the victim towards visiting fake websites in order for the attacker to collect lucrative information.

Phishing attacks are a growing problem, inflicting considerable damages on victims

(Kumaraguru et al., 2010). Statistics from the Anti Phishing Work Group (APWG), as can be seen in Figure 1, show that this has been the trend for many years.

Figure 1. Unique phishing sites detected, assembled from data collected by APWG (2014). 0 10000 20000 30000 40000 50000 60000 70000 1 2 3 4 5 6 7 8 9 10 11 12

Unique Phishing Sites Detected

(7)

3 Kumaraguru et al. (2010) claims in their study that “Most anti-phishing research has focused on solving the problem by eliminating the threat or warning users” (p. 2), rather than on educating users about this type of attack. They state that such education is difficult due to a lack of motivation from the users, security itself being viewed as a secondary concern to the tasks the users are performing, and that the users’ tendency to misjudge non-threats as threats increase as a direct result of the education (Kumaraguru et al., 2010). Dhamija et al. (2006) saw similar difficulties and tendencies but also noticed that users whom were trained to look for green locked padlock icons only looked for the icon, irrespective of its position.

2.2 Security Indicators

There are many aids available for users when surfing the web that tries to indicate whether a visited site is secure or not (Kumaraguru et al., 2010; Sheng et al., 2007; Wu, Miller, & Garfinkel, 2006; Yue & Wang, 2010). Some of the indicators to whether a site could be fraudulent are discussed in this thesis. It should be noted however that phishing attacks and other attacks are not limited to only using the indicators discussed herein.

2.2.1 Third Party Browser Extensions

According to Wu, Miller, and Garfinkel (2006) some aids are built into the browsers, but there are also third party tools available for added security. Such tools usually consist of an extension, or plugin, that users install which depends on the browser. The browser extensions can be in the form of a toolbar, text, image or other types of visual indicators for site status to provide quick visual aid for users to help determine if the site is safe to use (Wu et al., 2006; Yue & Wang, 2010). The extensions examined in the study performed by Wu et al. (2006) were able to indicate whether a site actually belongs to eBay or PayPal. One extension gave users a better visual presentation when SSL was used, by adding the websites logo along with the logo of the certificate authority. Other extensions showed the popularity of the site based on statistics from other users with the same extension or toolbar. There was also one

extension that displayed information about the domain name such as the country it is registered in along with date of registration (Wu et al., 2006).

Yue and Wang (2010) developed an extension that they claim send a lot of bogus data into forms on sites deemed fraudulent, hence the extension name BogusBiter, in order to hide the vital information supplied by the end user. This defense technique can be viewed as a

Security-by-Obscurity approach for when users do fall for a phishing attack. BogusBiter does

not stop the user from providing the information and is intended, according to Yue and Wang (2010), to be a compliment to other anti-phishing solutions.

2.2.2 Third Party E-Commerce Certificate Authorities

There are third party certificate authorities that e-commerce vendors can use to make

customers feel safe when shopping from their store (Trygg E-Handel, n.d.-a). When moving the mouse cursor over a 3rd_{party certificate authority’s logo on a webshop, the user is}

(8)

4 Figure 2. Example of 3rd party E-commerce Certificate Authority certification information.

According to Trygg E-Handel (n.d.-b), four out of ten shoppers in a study performed by Svensk Distanshandel in 2012 recognized the Trygg E-handel logo. They also claim that the majority of the Swedish online shoppers in the study saw the logo as, loosely translated, “one of the most crucial parameters when deciding what store to buy from” (Trygg E-Handel, n.d.-b).

2.2.3 Built-in Browser Indicators

Herzberg and Jbara (2008) claims that some browsers include basic indicators to help users avoid phishing attacks. Such indicators include, but are not limited to, padlock icons and using the text “https://” in front of the URL in the address bar. Google (n.d.) claims their product Google Chrome uses a green padlock icon and changes the “https” text in the address bar to be presented in a green color when a site uses a valid certificate. Certificates are only valid and trusted if they are signed by a trusted Certificate Authority (CA). The CA can either be a root CA or a CA trusted through a chain of trust. A prerequisite for the Google Chrome browser to show the related indicators for a web site with valid certificate is that the public certificate of the CA or root CA is present on the local computer. Browsers can use their own certificate store or use the certificate store for the operating system. Major certificate vendors root certificate are often bundled in with the browsers or operating systems, and do not need to be added by the end user. There are several different padlock icons, each indicating

(9)

5 Figure 3. SSL enabled and normal site indicators in Google Chrome.

2.3 Situated Learning

According to Herrington and Oliver (1991) situated learning was first introduced as a concept by Brown, Collins, and Duguid (1989), although it was not an entirely new idea as it was inspired by the work of others in the field of cognition. Brown et al. (1989) claims that the culture, context and activity in which the learning takes place has an impact on the knowledge retained, something which is usually ignored in schooling of the conventional type. Lave & Wenger (1991) also state that situated learning focuses on the social situations in which the learning takes place, as well as the relationship between the situation and learning. They also suggest that the context of learning and the social engagement is more crucial to the learning process than the conceptual structures and cognitive processes involved. Herrington and Oliver (1991) talks about mimicking real-life scenarios with authentic activities and context, where the receiver of the knowledge is able to, through activities, interact with the learning environment in order to find or solve the problem.

Dr. Nohlbergs framework for contextual or situation based learning (as described during interview and transcribed in Appendix A, and discussed by Kävrestad (2014)) builds upon the same principles as situated learning. It uses event triggered micro-training to provide useful information and educate end users about computer science related security issues. According to Dr. Nohlberg the framework is aimed at educating computer users about information security risks, but may also prove useable for other applications outside this area.

2.4 Swedish Laws and Regulations

According to Konsumentverket (2007) the law “Distans- och hemförsäljningslagen

(2005:59)” apply when shopping from a geographical distance, e.g. shopping from web sites or by phone. The law regulates, among other things, what information must be provided to the customer before an agreement of purchase takes place; e.g. the selling entities or persons name and address, return policy, price for shipment and the price of the product

(Konsumentverket, 2007). Svensk författningssamling (2005) claims that this law will expire 2014-06-13 and be replaced with an updated law called “Lag (2014:05) om distansavtal och avtal utanför affärslokaler”. Svensk författningssamling (2014) show that the replacement law still requires the before mentioned examples from Konsumentverket (2007), but adds

(10)

6

2.5 Background in Related Work

West et al. (2005) investigated the likelihood of students and cadets at United States Military Academy (USMA) to click on links in fraudulent e-mails. Jackson, Ferguson, and Cobb (2005) performed another experiment, on the same target group, based on the West et al. (2005) findings, yielding similar results. Both studies targeted the use of links in e-mails to redirect the victims to fraudulent locations online. The studies showed that a high percentage of participants clicked links in e-mails that had indicators of being legit, e.g. look like they were sent from superiors or mentioning grades. West et al. (2005) saw that as much as 90% of the freshmen participating in the study at USMA clicked embedded links in e-mails, which led to fraudulent sites, even though they had been given four hours of computer security instruction at an earlier stage.

(11)

7

3 Problem

The aim of this thesis is to evaluate if a learning module, based on Dr. Nohlbergs framework designed to teach users about information security, can help raise awareness about security issues through small game-like learning sessions within the context of online shopping (e-commerce).

The problem statement is:

“Does contextual micro-training regarding security issues impact the way users perform e-commerce related tasks?”

3.1 Purpose

The purpose of this thesis is to evaluate if a practical implementation of a learning module, based on Dr. Nohlbergs framework, yields a different results for users participating in micro-training before performing common e-commerce related tasks, as opposed to users without micro-training.

The framework is designed for general use in teaching users about information security related issues, and one area proposed by Dr. Nohlberg during an interview is security in e-commerce sites (see Appendix A). Kävrestad (2014) also suggest a website implementation and proposes that an experiment is performed using two groups.

3.2 Motivation

Internet frauds and identity theft are common occurrences on the internet (Kumaraguru et al., 2010). This is supported by statistics about the number of reported and found fraudulent sites gathered by APWG (2014), and can be seen in Figure 1.

In a research note created for Gartner Inc, aimed at evaluating identity theft using phishing and what methods there are to mitigate such attacks, Litan (2004) stated:

“More than 1.4 million users have suffered from identity theft fraud, costing banks and card issuers $1.2 billion in direct losses in the past year.” (p. 1)

Dr. Nohlberg stated during an interview (see Appendix A for full interview in Swedish) that there is a growing interest in being able to educate people when and where needed. The educational events do not have to be long sessions of intense learning, but rather a form of micro-training within and about the context the user is shown; as opposed to the more traditional, planned educational events that are performed when and where deemed convenient.

3.3 Limitations

(12)

8 To avoid the results of the experiment being affected by cognitive aspects, as this thesis emphasizes on the computer science aspect of the learning process, test subjects will be equally exposed to design choices in both fraudulent and “real” scenarios, e.g. using the same layout on all sites, using the same amount of products, using the same color schemes for bad and good sites to avoid colors influencing the site picked, etc.

In this thesis Dr. Nohlbergs framework will only be tested in the context of e-commerce.The thesis is therefore limited to only examine if the framework is applicable for this context, and nothing else. If the experiments, research and tests were to show that the framework is not applicable for the context e-commerce, this would not mean it does not perform better in other contexts.

(13)

9

4 Method

The two methods used in this thesis is interview and experiment. The interview is performed in order to gather background information about the framework, which then was implemented and tested using an experimental approach with students at the University of Skövde as the sample group.

In a master’s thesis by Kävrestad (2014), with Dr. Nohlberg as supervisor, a survey method was used and the need for a more practical implementation was identified. One such

experiment could use an isolated environment to emulate real world scenarios. Kävrestad (2014) proposed that participants should be divided into two groups; where one group

receives education through a “defense mechanism” before the main tasks and the other group performs the same main tasks without this education. Kävrestad (2014) expected the users going through the education process to perform the main tasks better in regards to the issues being taught beforehand.

A case study with real implementation was considered as method, but a practical test with possible real world scenarios in a controlled environment was deemed the best choice; both ethical implications and possible negative impacts on sales for a real e-commerce site could thus be avoided.

A survey similar to the one performed by Downs et al. (2007), where participants performed role play when answering the multiple choice questions, could have been an interesting approach. It would, however, need more participants and data collection to be statistically significant than the timeframe for this thesis would allow.

4.1 Interview

To fully grasp the concept of the learning module, an open ended interview with Dr. Marcus Nohlberg, the author of the framework, was deemed a necessity. According to Berndtsson, Hansson, Olsson, & Lundell (2008), open ended interviews are performed using questions that require more than a simple yes or no as an answer from the interviewee. Due to the open nature of the questions, the interviewee has the ability to answer freely and may emphasize and elaborate on the areas he or she finds most important. It is therefore up to the interviewer to pre plan the flow and steer the conversation towards the areas relevant to the purpose of the interview, which may be difficult for inexperienced interviewers (Berndtsson et al., 2008). When performing an open ended interview, it may be hard to take notes during the session. In

(14)

10

4.2 Experiment

The scope of the experiment is to collect data and analyze whether the control group or the experiment group bought from the fraudulent sites or from the supposedly real counter parts. Fraudulent and real is in this case determined by the included security indicators, or lack thereof. The difference in execution between the two groups is that the experiment group goes through micro-training using the learning module before performing the e-commerce related tasks. This scope is put together using the goal template for experiments in software

engineering, as described by Wohlin et al. (2012, p. 85)

To make sure that the learning module is what is being tested, without interference of other factors, all aspects of the different configurations and websites were tested with each change. This use of good software development practice as well as testing, in order to ensure that the experiment is actually evaluating the intended parameters, is something that Berndtsson et al. (2008) claims is crucial to the validity of the project.

The context of the experiment is limited to online shopping and performed in an isolated environment to avoid ethical issues, such as the webshops accidentally being visited by the general public. Another ethical issue that was a contributing factor to isolating the experiment was the use of product images; the study is in no way intended to damage any brands,

companies or product lines, and was therefore not published publicly. For this thesis all product names, brands and identifying properties have been omitted or replaced with the following: “Brand name”, “Product name” and “Product property”. The completely made up products and brands are still left intact. Real names and real product images were used to elevate the authenticity of the situation the subjects were put in.

Threats against the validity of the conclusion in this experiment are the sample size and sampling method. To avoid this being an issue the sample size was discussed with Dr. Nohlberg, where a sample size of 15 people in each group, 30 participants in total, was deemed an adequate number for this initial study. The sampling method of choice for the experiment was accidental sampling instead of probabilistic and the accidental sampling took place at the University of Skövde. The sampling was however performed during 4 days and in different locations of the campus, to avoid scheduled events having too big of an impact on the selection process.

Validity problems may arise from the treatment of subjects during implementation according to Wohlin et al. (2012), who also offers advice on how to avoid such instances:

“There is a risk that the implementation is not similar between different persons applying the treatment or between different occasions. The implementation should hence be as standard as possible over different subjects and occasions.” (p. 106)

(15)

11 The “social threats to construct validity”, as discussed by Wohlin et al. (2012) may

be applicable in this type of experiment. Wohlin et al. (2012) claims that the

subjects deviate from their standard behavior due to the fact that they know they are taking part in a study, thus yielding false results. The previously mentioned written instructions asks the participants to try and act as if they were shopping for

themselves, from the comfort of their home, thus resulting in small role playing event where the participant are playing the role of themselves. Due to the design of the experiment, both groups are faced with the same issues and the potential deviations in behavior would therefore impact both groups equally, thus being a minor issue for the outcome of the experiment.

The sampling groups are, as previously mentioned, limited to people within the geographical proximity of where the sampling takes place. The participants were therefore expected to consist mainly, if not exclusively, of students at the

University of Skövde. This type of sampling reduces the external validity of the study according to Wohlin et al. (2012) and yields a results that is not

representative for the general public. Dr. Nohlberg (see Appendix A) did however say, loosely translated and summarized, that the study can be considered a proof of concept, providing an inclination as to whether there is a need for deeper research when it comes to session based micro-training in the context of e-commerce. To be able to properly analyze the subjects and the impact of the learning module, all visits to the web sites were recorded using a third party service called GhostRec, developed for web usability testing. To avoid ethical issues regarding integrity, this is done with the consent of the subjects who are informed of being recorded prior to participating in the experiment. Recording the actions of the subjects allow for deeper analysis about informed decisions, e.g. it is possible to see what page a user visited or how many indicators of fraudulency were identified before a purchase was aborted. Without such recordings, this type of data would have to be gathered using surveys after the subjects have finished their tasks, adding an additional process where misinterpretation of questions, memory based limitations and variations in the consistency of answers between subjects may result in errors. The following aspects have been identified as being key aspects to examine:

 Generate scores for each customer based on the items bought and from which shop they were bought.

 Identify number of perfect scores for the two groups.

(16)

12

5 Related Work

Current research focus on stopping users from ending up on fraudulent websites and very little research effort is put into improving protection when mitigation techniques fail. A common way for an attacker to get users to end up on a fraudulent site is to send e-mail to potential victims (Downs et al., 2007; Jackson et al., 2005; West et al., 2005). Another way to contact the potential victims is to use IM services (Parno et al., 2006).

Dhamija et al. (2006) performed a usability study with 22 participants that were shown 20 different websites with the aim to identify fraudulent sites. The study showed that 90% of the participants were fooled by the fraudulent web sites and as much as 23% either completely ignored or did not examine the security indicators, such as padlock symbols for SSL encryption, status bars, address bar, etc. Dhamija et al. (2006) used a method where they presented participants with warnings in the form of popup messages, which were ignored by 68.8% (15 out of 22) of the participants.

Sheng et al. (2007) describes how they created a learning based game in Flash 8 with the goal of teaching the players about phishing attacks. The game is called “Anti-Phishing Phil” and it prompts players with questions about common events and attacks while the players are performing their primary game tasks, answering correctly allows them to progress in the game. Sheng et al. (2007) evaluated their creation using three groups of users in a user study. They explain how all three groups were tested in their ability to identify fraudulent sites. One of the groups were then asked to read training material already available online about how to identify phishing sites, one group played the game and the last group read a tutorial

(17)

13

6 Execution

This chapter describes the steps taken to perform the experiment such as the creation of the learning module, creation of the e-commerce sites, configuration changes of the backend servers, client installation and data collection.

6.1 The Learning Module

A working learning module based on Dr. Marcus Nohlbergs framework for event driven situated learning was created. The module consisted of a simple Lightbox, which is a

JavaScript that creates a popup window within a webpage while at the same time dimming the area around, making it seem like it is floating on top of the page content. The Lightbox was used to show content through HTML and run JavaScript for dynamic updates, navigation and validation. Kävrestad (2014) shows an example of an implementation on a website, within a different context, that uses a similar approach as the learning module in this experiment. The learning module used a simplistic design and appeared when a user was about to enter an e-commerce site. This simulates functionality of a 3rd party service noticing that you are about to visit a certain type of site and steps in with useful information. The content of the HTML pages displayed within the Lightbox were simple questionnaires about common indicators useful for identifying fraudulent sites. The indicators that users were asked to look for were SSL indicators, third party certification authority notices and things that are required by Swedish law to be present on an e-commerce site. The visitors were asked to choose between two options, one correct and one wrong, before moving on to the next page. Each user

prompted with the learning module were asked a total of three questions, see Figure 4 for an example question from the learning module or see Appendix B for all questions and answers used.

Figure 4. First question prompted by the learning module

(18)

14 Figure 5. Grading, tips and information for the first question

6.2 The E-Commerce Sites

In the experiment phase a total of four e-commerce sites were created to act as real world implementations. The four sites were based on the same HTML template to avoid esthetical aspects of design such as color scheme or layout differences influencing the choice of site from which to buy. Two of the sites sold sunglasses and two sites sold electronics and gadgets, of which one of each type was set up with deliberate signs of fraudulency

corresponding to what is taught by the learning module described in chapter 6.1. The sites were given basic logic using PHP for all pages provided by the HTML template in order to mimic real e-commerce sites. The pages of the sites were a signup, login, front page with product display, single product information page, shopping cart and a checkout page where users could supply credit card information and delivery address.

(19)

15 The e-commerce sites were all given domain names which correlated to the products sold on the respective site, and were not registered by any registrar at the time of the experiment. A

DNS server was set up to answer to DNS-requests regarding the chosen domain names, but

only for requests which originated from the IP addresses used by the client computers in the experiment, thus avoiding anyone reaching the sites by mistake. Table 1 shows the domain names used in the experiment. To avoid ethical issues and copyright infringements on the domain names, regular checks on registrars were made during the 4 days the experiment was active.

Table 1. Domains used and errors implemented in the experiment

Domain Type SSL (HTTPS) Contract 3rd party

sunglasses.nu Real Yes Yes Yes

shades.nu Fradulent No No Failed

gadgetstore.nu Fradulent Yes Missing Failed

megagadet.nu Real Yes Yes Yes

6.3 Environment Setup

The following information explains how the respective servers and client computers were configured. The mentioned configuration changes are the only changes performed on the systems; all other services and configurations were left at default settings.

6.3.1 Backend servers

All servers used in the experiment ran the Debian 7.5 operating system and were installed to run as CLI-based environments using the predefined settings in the installer for everything except the addition of “Standard system utilities” and “SSH server”. All additional packages were installed from the default repository; see Table 2 for list of additional packages.

Table 2. Servers and installed packages

Server OS Installed packages/services

web1 Debian 7 Apache2, PHP + dependecies db1 Debian 7 MySQL

ca1 Debian 7 OpenSSL dns1 Debian 7 Bind9

6.3.2 Web server

Apache2 was installed on the web server along with PHP and the dependencies that are

(20)

16 A .htaccess-file was placed in each sites root directory in order to block any IP but the IPs of the client machines, thus isolating the environment and avoiding ethical implications that could arise if the pages were publicly accessible. The .htaccess-files contain the following:

order deny,allow deny from all

allow from <Public IPs omitted> 6.3.3 Database server

The database manager of choice in the experiment was MySQL for no reason other than it being the database manager used in courses of the Network and System Administration program at the University of Skövde and is available through the default repository. No changes were made to the default configuration for this server except for changing the

listening IP to the local IP instead of the loopback address, allowing the web server to connect over the network. A database called exjobb was created containing the following tables (for full SQL files see 0):

 webshop1  webshop2  webshop3  webshop4  purchase

The webshop tables contained the information about their respective products and the

purchase table contained purchases made during the data collection phase. Each purchase was stored with the product identifier, name and price along with the ID of both the webshop and customer.

6.3.4 Certificate Authority server

During their studies of real and active phishing sites, Chou, Ledesma, Teraguchi, Mitchell, and Ca (n.d.) found that most fraudulent web sites do not use HTTPS even if the honest site they are mimicking does. Setting up the site without HTTPS simplifies the creation and deployment process as no certificates are required (Chou et al., n.d.). Certificates for SSL were therefore implemented on one of the sunglasses sites and on both of the gadget sites. The CA server used the default OpenSSL installation that comes with the Debian operating system. A self-signed CA-certificate was created and used to sign the web server certificates, as mentioned in 6.3.2, by following the guide written by Searle (2008).

6.3.5 DNS server

The DNS server was configured to only allow requests from the IPs of the client computers, and was authoritative for the following zones (for zone declarations see Appendix E):

(21)

17  Megagadget.nu

Each of the zones used their own zone-file, containing only the minimum information required and two domain records. Both records, for all zones, points to the web servers IP address.

6.3.6 Client computers

All client computers in the experiment were laptops that had a clean Windows 7 operating system installed with the only extra software being the Google Chrome browser. This browser was chosen as it was the most commonly used browser at the time (StatCounter, 2014), as can be seen in Figure 6.

Figure 6. Browser statistics for April 2013-April 2014, assembled from data collected by StatCounter (2014)

The root certificate of the ca1 server was transferred to each client and added as a trusted root certificate in Windows. The Windows version of Google Chrome use the Windows

Certificate Store to know what CA to trust (Chromium, 2008).This was achieved by importing the root certificate using the certmgr.msc tool that ships with Windows 7. The client computers were connected to the eduroam wireless network at the University of Skövde, which covers the entire campus and allows for roaming. Eduroam was used to allow for mobility during the data collection phase and in order for the client computers to

communicate with the backend servers.

Except for the previously mentioned SSL certificates and installation of the browser, the only other change to the client computers were to set the DNS property of the network

configuration to point to the DNS server in the backend.

6.4 Collecting the Data

Data collection took place at the University of Skövde during a four day period. Due to validity concerns a number of different locations were used each day to avoid only using students from the same class, as discussed in chapter 4.2.

0 5 10 15 20 25 30 35 40 45 50

Jan-13 Mar-13 May-13 Jun-13 Aug-13 Oct-13 Nov-13 Jan-14 Mar-14 Apr-14

Browser Statistics

(22)

18 During the experiment two groups were used, as recommended by Kävrestad (2014) and discussed with Dr. Nohlberg (see Appendix A). The groups are referred to as the “experiment group” and the “control group” in this thesis.

The experiment group went through a micro-training event using the learning module (see chapter 6.1) before ending up at the start position of the control group. At this point all participants from both groups were asked to buy the same items and were given the exact same instructions.

Each participant in both groups were provided with a shopping list containing 5 items, that they had to buy, along with a fake credit card printed on regular paper. Some of the items were specified with name and brand; those properties are however omitted in this thesis to avoid ethical issues. The 5 items were:

 1x Sunglasses with black frame of a certain model  1x Sunglasses with a thick frame

 1x Streaming device for TV  1x Printer for computer  1x A Tablet with 10” display

The sunglasses allowed for a free interpretation of the items to buy since both sites containing that kind of product had several items that matched the description, whereas the gadget shops both had a single item matching respective item described on the list.

(23)

19

7 Results and Analysis

A total of 32 subjects participated in the experiment, whereas 16 went through the micro-training provided by the learning module and the remaining 16 performed the tasks without the learning module. The participants in the control group bought items as shown in Table 3: Table 3. Number of products purchased per store by the control group members.

Sunglasses.nu Shades.nu Megagadget.nu Gadgetshop.nu

Control User 1 2 0 3 0 Control User 2 0 2 1 2 Control User 3 2 0 2 1 Control User 4 2 0 0 3 Control User 5 2 0 3 0 Control User 6 1 1 1 2 Control User 7 1 1 1 2 Control User 8 1 1 1 2 Control User 9 1 1 1 2 Control User 10 0 2 2 1 Control User 11 2 0 0 3 Control User 12 0 2 3 0 Control User 13 2 0 1 2 Control User 14 2 0 0 3 Control User 15 1 1 1 2 Control User 16 1 1 0 3

Participants in the experiment group purchased items as shown in Table 4, along with the amount of time each participant spent with the learning module:

Table 4. Number of products purchased per store by the experiment group members and time spent in the learning module.

Sunglasses.nu Shades.nu Megagadget.nu Gadgetshop.nu Time (s)

(24)

20 A perfect score, as mentioned in chapter 4.2, indicate that a user bought all items from a non-fraudulent store. The perfect scores will be used to indicate whether participants successfully avoided buying from the fraudulent store within the category. In this chapter analysis was performed individually for the two categories of shops, gadgets and sunglasses, before looking at the complete picture. An analysis was also performed separately for the overall performance of each of the groups as can be seen in chapter 7.3 and 7.4.

To differentiate and visualize the results each shop is analyzed one by one with color representation. In the following chapters table elements highlighted in green color ( ) indicates that the subject bought all items from a non-fraudulent store for that category. Red color ( ) indicates that the subject bought at least one item from at least one fraudulent store for that category. Chapter 7.3 and chapter 7.4 look at the overall results per subject basis and perfect scores for overall performance is visualized by a darker green color with white numbers ( # ). The overall perfect score requires subjects to not have bought any items from fraudulent stores.

7.1 Sunglasses Shops

Table 5 and Table 6 show the scores for the shops selling sunglasses. The tables show that the experiment group had a total of 68.75% perfect scores, while the control group had 43.75% perfect scores.

(25)

21 Table 6. Tally of perfect scores in the experiment group during purchase of sunglasses.

Sunglasses.nu Shades.nu Experiment User 1 1 1 Experiment User 2 1 1 Experiment User 3 1 1 Experiment User 4 2 0 Experiment User 5 2 0 Experiment User 6 2 0 Experiment User 7 2 0 Experiment User 8 2 0 Experiment User 9 2 0 Experiment User 10 2 0 Experiment User 11 1 1 Experiment User 12 2 0 Experiment User 13 2 0 Experiment User 14 1 1 Experiment User 15 2 0 Experiment User 16 2 0

7.2 Gadget Shops

For the gadget stores the results indicated a smaller amount of perfect scores across the board for both groups, when compared to the sunglasses shops. Gadget shop results can be seen in Table 7 and Table 8. The tables show that the experiment group had a total of 50.00% perfect scores, while the control group had 18.75%.

(26)

22 Table 8. Tally of perfect scores in the experiment group during purchase of gadgets.

Megagadget.nu Gadgetshop.nu Experiment User 1 2 1 Experiment User 2 2 1 Experiment User 3 1 2 Experiment User 4 3 0 Experiment User 5 0 3 Experiment User 6 0 3 Experiment User 7 0 3 Experiment User 8 3 0 Experiment User 9 3 0 Experiment User 10 3 0 Experiment User 11 0 3 Experiment User 12 3 0 Experiment User 13 0 3 Experiment User 14 3 0 Experiment User 15 3 0 Experiment User 16 3 0

7.3 Overall Performance for the Control Group

The overall perfect score for both shop categories, counting them individually, when looking at the control group was 31.25%. When looking at both categories together, two participants in the control group managed a perfect score in both the sunglasses purchase and the gadget purchase, which amounts to 12.50%.

Table 9. Overall performance for the control group, both shop categories.

Sunglasses.nu Shades.nu Megagadget.nu Gadgetshop.nu

(27)

23

7.4 Overall Performance for the Experiment Group

The overall perfect score for both shop categories, counting them individually, when looking at the experiment group was 59.38%. A total of seven participants in the experiment group managed a perfect score in both the sunglasses purchase and the gadget purchase, amounting to 43.75%.

Table 10. Overall performance for the experiment group, both shop categories.

Sunglasses.nu Shades.nu Megagadget.nu Gadgetshop.nu

Experiment User 1 1 1 2 1 Experiment User 2 1 1 2 1 Experiment User 3 1 1 1 2 Experiment User 4 2 0 3 0 Experiment User 5 2 0 0 3 Experiment User 6 2 0 0 3 Experiment User 7 2 0 0 3 Experiment User 8 2 0 3 0 Experiment User 9 2 0 3 0 Experiment User 10 2 0 3 0 Experiment User 11 1 1 0 3 Experiment User 12 2 0 3 0 Experiment User 13 2 0 0 3 Experiment User 14 1 1 3 0 Experiment User 15 2 0 3 0 Experiment User 16 2 0 3 0

7.5 Analysis of recordings

Looking at the recordings from each users visit to the webshops, created by GhostRec, only a single user in the control group (Control User 1) used the 3rd party e-commerce certification authority indicator by moving the mouse over the logo on all sites. One single control group participant performed a mouse over event on the logo on two of the sites, but nonetheless bought from a fraudulent store were the participant neglected to examine the logo.

(28)

24

8 Comparison to Related Work

The results in before mentioned study performed by Dhamija et al. (2006), to test users awareness of security indicators, shows a lower percentage of participants being able to identify fraudulent websites than what is presented in the tables in chapter 7. Both studies do, however, use a relatively small sample group which cannot be seen as statistically significant or representative of the general public. Even though both studies experiment groups are comprised of university students and faculty members the studies test subjects could still vary in many aspects, such as age, distribution in terms of gender, computer habits, computer knowledge, etc. Dhamija et al. (2006) use all indicators used in this thesis, but have additional indicators as well. The experiments themselves differ too much in execution to make any conclusion or comparison of the final results in detected or undetected security indicators. Even though no comparison can be made between the two studies, both show that the topic of educating users about security indicators appear to have an impact and further research with larger experiment groups is necessary.

(29)

25

9 Discussion

The results and analysis clearly show a significant improvement for the group participating in the micro-training when compared to the group without micro-training. The four participants in the experiment group who bought from the correct sunglasses store, but also bought from the fraudulent gadget store, still indicate a change in behavior when compared to the control group. The reason why the four participants chose the wrong shop in the gadget category is unclear, and a clear pattern cannot be seen in the recordings, perhaps the formulation in the teaching process was a bit off and would need to be changed. In a real environment the questions and answers would be evaluated frequently in an iterative process to stay current and up to par. What can be seen in the recordings is that none of the four participants used the 3rd party e-commerce certificate indicator as intended, thus the certificate error never showing which required the user to hover the mouse over the logo.

Talking to participants after they performed the tasks, a few were surprised by that they apparently acted as if they were at home, using their own money. Some of the participants that fell victim to the fraudulent sites, from both groups, were shocked at how easily they gave up their credit card details to a fraudulent store. Two participants claimed to be frequent online shoppers but still bought items from the fraudulent stores, even though they both considered themselves experienced and went through the micro-training. One user from the control group quickly walked up and explained that there was something wrong with two of the sites, and asked if the experiment was broken. He had correctly identified that one site was missing a SSL certificate, the other didn’t have a sales agreement and both had broken 3rd party e-commerce certificates. Because of this he claimed that whatever evaluation was being performed could therefore not be valid, due to the sites not having the same basic

functionalities.

(30)

26

10 Conclusions

This chapter briefly summarizes the problem, experiment, results, contributions made and suggestions for future work. The future work section provides general ideas of further evaluation of the framework and other methods that might be useful to evaluate against.

10.1 Summary

The focus of the problem statement in this thesis aims to give a first evaluation of Dr. Nohlbergs framework, for situated learning through a micro-training sessions, by implementing a learning module in the context of e-commerce.

The experimental approach used for the evaluation has been suggested by both Kävrestad (2014) and Dr. Nohlberg (see Appendix A). In the experiment four webshops were created, two selling gadgets and two selling sunglasses. One webshop of each type was introduced with indicators of untrustworthiness, the remaining two showing no signs of fraud. The experiment used two groups of randomly selected student participants, who were tasked with buying a selection of items consisting of gadgets and glasses on a provided list. The

participants were free to use any of the four webshops and instructed to act as they would with their own money, in their own homes. One of the participant groups was put through the learning module before proceeding with their purchasing tasks.

The results from the experiment show a variation of randomness with an inclination towards good when the learning module was applied. The statistical significance is however not sufficient, due to the low amount of participants and the limited selection process of subjects, to make any conclusions to whether the result fully reflects the outcome in a real world scenario. This is however acceptable since the aim was to give an indication to whether the framework could prove useful in a larger implementation within the context of e-commerce.

10.2 Contributions

The thesis serves as a first attempt at implementing Dr. Nohlbergs framework for situational learning using an experiment. Earlier evaluations of the framework have been performed using survey and focus groups.

The majority of current research in the area of phishing prevention from a user perspective focuses on stopping the user from ending up on a fraudulent website. This thesis aims to aid in extending the protection by adding another layer, namely education of the user for when prevention is not successful. There is existing research on education of end users regarding phishing, most of which is focused on fraudulent e-mail detection. The current research about web security indicators lacks in the educational approaches and this is one step in filling that gap.

10.3 Future Work

(31)

27 provide validation to the result or identify weaknesses to consider during future

implementations.

There are many contexts Dr. Nohlbergs framework could be applied to. One such context is online auction sites and retail sites for used products, e.g. Tradera and Blocket. The

framework could be used to provide context aware information about online frauds regarding the shipment method they have chosen, the product they are searching for or the region they are in. The framework could be embedded within the site as service provided by the owners of the site or as third party browser plugin.

An implementation of the framework could also be tested for web based e-mail clients, social media sites or community sites to teach users about how to identify spam, links to fraudulent sites and identity theft attempts.

A more general approach of implementation could be as a browser plugin aimed at providing information when passwords are entered or chosen on various sites.

There are other areas than computer science related security issues to consider evaluating the framework for, such as education within companies, e.g. when a new expansion or update is introduced to an application or policies for how certain things are supposed to be performed change.

(32)

28

References

APWG. (2014). APWG Reports. Retrieved May 26, 2014, from http://www.antiphishing.org/resources/apwg-reports/

Berndtsson, M., Hansson, J., Olsson, B., & Lundell, B. (2008). Thesis Projects: A Guide for

Students in Computer Science and Information Systems (Second Edi.). London:

Springer-Verlag London Limited.

Brown, J. S., Collins, A., & Duguid, P. (1989). Situated Cognition and the Culture of Learning. Educational Researcher, 18(1), 32–42. doi:10.3102/0013189X018001032 Chou, N., Ledesma, R., Teraguchi, Y., Mitchell, J. C., & Ca, S. (n.d.). Client-side defense

against web-based identity theft.

Chromium. (2008). Issue 2170 - chromium - Add AddTrust External CA Root to net/base/ev_root_ca_metadata.cc. Retrieved May 25, 2014, from

https://code.google.com/p/chromium/issues/detail?id=2170

Dhamija, R., Tygar, J. D., & Berkeley, U. C. (2006). Why Phishing Works, (November 2005), 581–590.

Downs, J. S., Holbrook, M., & Cranor, L. F. (2007). Behavioral response to phishing risk. In

Proceedings of the anti-phishing working groups 2nd annual eCrime researchers summit on - eCrime ’07 (pp. 37–44). New York, New York, USA: ACM Press.

doi:10.1145/1299015.1299019

EEM Technologies. (2014). Usability Testing for your website. Retrieved May 25, 2014, from http://www.ghostrec.com/

Google. (n.d.). Website settings - See if the site is using a secure connection (SSL). Retrieved May 30, 2014, from https://support.google.com/chrome/answer/95617?hl=en

Herrington, J., & Oliver, R. (1991). Critical Characteristics of Situated Learning : Implications for the Instructional Design of Multimedia.

Herzberg, A., & Jbara, A. (2008). Security and identification indicators for browsers against spoofing and phishing attacks. ACM Transactions on Internet Technology, 8(4), 1–36. doi:10.1145/1391949.1391950

Jackson, J. W., Ferguson, A. J., & Cobb, M. J. (2005). Building a University-wide Automated Information Assurance Awareness Exercise. In Proceedings Frontiers in Education 35th

Annual Conference (pp. T2E–7–T2E–11). IEEE. doi:10.1109/FIE.2005.1611900

Konsumentverket. (2007). Distans- och hemförsäljningslagen, (april 2007).

Kumaraguru, P., Sheng, S., Acquisti, A., Cranor, L. F., & Hong, J. (2010). Teaching Johnny not to fall for phish. ACM Transactions on Internet Technology, 10(2), 1–31.

(33)

29 Kävrestad, J. (2014). Defining, categorizing and defending against online fraud. University of

Skövde.

Lave, J., & Wenger, E. (1991). Situated Learning: Legitimate Peripheral Participation. Retrieved from http://www.google.se/books?hl=sv&lr=&id=CAVIOrW3vYAC&pgis=1 Litan, A. (2004). Research Note 14. Gartner Inc. Retrieved from

ftp://rayvenproductions.com/activcard/marketresearch_docs/gartner_phishing_victim.pdf Parno, B., Kuo, C., & Perrig, A. (2006). Phoolproof Phishing Prevention, 1–19.

Searle, C. (2008). Certificate Authority (CA) with OpenSSL. Retrieved May 25, 2014, from

http://www.debian-administration.org/article/618/Certificate_Authority_CA_with_OpenSSL

Sheng, S., Magnien, B., Kumaraguru, P., Acquisti, A., Cranor, L. F., Hong, J., & Nunge, E. (2007). Anti-Phishing Phil : The Design and Evaluation of a Game That Teaches People Not to Fall for Phish.

StatCounter. (2014). Top 5 Desktop, Tablet & Console Browsers from Apr 2013 to Apr 2014. Retrieved May 25, 2014, from http://gs.statcounter.com/

Svensk författningssamling. Distans- och hemförsäljningslag (2005:59) (2005). Sweden. Retrieved from

http://www.riksdagen.se/sv/Dokument-Lagar/Lagar/Svenskforfattningssamling/Distans--och-hemforsaljningsla_sfs-2005-59/ Svensk författningssamling. (2011). Lag (2002:562) om elektronisk handel och andra

informationssamhällets tjänster. Retrieved May 28, 2014, from

http://www.riksdagen.se/sv/Dokument-Lagar/Lagar/Svenskforfattningssamling/Lag-2002562-om-elektronisk-_sfs-2002-562/

Svensk författningssamling. Lag SFS 2014:14 - om ändring i distans- och

hemförsäljningslagen (2005:59); (2014). Justitiedepartementet. Retrieved from

http://www.lagboken.se/Views/Pages/GetFile.ashx?portalId=56&cat=202462&docId=18 97951&propId=5

Trygg E-Handel. (n.d.-a). E-handlarnas egna ord. Retrieved June 07, 2014, from http://www.tryggehandel.se/?sida=egnaord

Trygg E-Handel. (n.d.-b). För e-handlaren. Retrieved June 07, 2014, from https://www.tryggehandel.se/?sida=ehandlaren

West, T., Carronade, P., & Ferguson, B. A. J. (2005). Fostering E-Mail Security Awareness :, (1), 54–57.

Wohlin, C., Runeson, P., Höst, M., Ohlsson, M. C., Regnell, B., & Wesslén, A. (2012).

Experimentation in Software Engineering. Berlin, Heidelberg: Springer Berlin

(34)

30 Wu, M., Miller, R. C., & Garfinkel, S. L. (2006). Do security toolbars actually prevent

phishing attacks? In Proceedings of the SIGCHI conference on Human Factors in

computing systems - CHI ’06 (p. 601). New York, New York, USA: ACM Press.

doi:10.1145/1124772.1124863

(35)

1

- Transcript from interview

Transcript from interview with Dr. Marcus Nohlberg

Deltagare: Jonas Werme (ref. Jonas), Dr. Marcus Nohlberg (ref. Marcus)

<Intervju startar> Jonas:

Vad är syftet med lärmodulen/försvarsmekanismen? Marcus:

Det finns en ganska tydlig forskningsbakgrund, med att utbildning i sig är väldigt svårt att veta om det fungerar. Väldigt mycket av de utbildningsinsatserna som vi gör inom informationssäkerhet, där vi har en väldigt stor tilltro till att de fungerar, vet vi faktiskt väldigt lite om. Ett av de generella problemen som finns, är lite av att man tror det finns en överkopplingseffekt. T.ex. dag 1 har vi en informationssäkerhetsutbildning, dag 287 hamnar du i en situation där någonting dumt händer. Man tror då att det ska finnas en överkoppling där emellan. Det finns ett växande intresse för att man lägger utbildningsinsatsen när den behövs, istället för när det är bekvämt att planera in det, och att man då satsar på någon form av mikroutbildning riktad till exakt det caset du håller på och jobbar med för stunden. Genom detta började jag titta på den här sortens utbildningsinsats, så att när du gör någonting som är potentiellt sett riskabelt så får du en genomgång av vad du behöver tänka på där. Jonas:

Så med andra ord är det situationsbaserat? Marcus:

Ja, och situationsbaserat innebär ju att du ska få den utbildning du behöver. I vårt sätt att se det får du den utbildning du behöver i den stunden du behöver den. Eller du erbjuds en möjlighet till det i vart fall. De preliminära resultaten indikerar att det här är åtminstone användbart, det är chockerande sällan använt dock. Jonas:

Hur är den lilla mekanismen och lärsituationen som uppstår tänkt att fungera? Marcus:

Jag kan ju ge ett generellt svar på den frågan, men ett exakt svar på varje punkt är svårt att säga. Men det generella svaret är det att; när du gör någonting där det finns en risk för dig som är onödigt stor. Allting från att du ska köpa en specifik produkt till att du ska installera en programvara, till att du ska byta ditt lösenord. I den stunden får du en väldigt enkel introduktion, en väldigt enkel utbildning, som gärna är scenariobaserad och handlar om just den saken du ska göra. Idén är att det kanske bara ska ta max en minut, det ska vara en väldigt enkel förklaring. Syftet är att man kommer åt det jag brukar tjata om, som är de lågt hängande frukterna. Det vill säga att vi inte kommer kunna ge en utbildning som är så kort och ändå täcker de mest sofistikerade angreppen, men vi kan göra det betydligt svårare att utföra [en lyckad] attack. En ganska stor del utav angreppen skulle kunna förhindras med en gemensam utbildningsinsats. Det viktiga med de verktygen vi har tittat på här är inte bara det faktum att de är väldigt snabba och väldigt effektiva i utbildningsstunden, utan ska också vara snabba på andra sidan. Du ska kunna anpassa den utbildningsinsats som du har eller skapa en ny på så kort tid som möjligt. Så idén är inte att det ska ta dig flera timmar utan om, som exempelvis i det här fallet, en e-handels plats inser att det skett en ny sorts angrepp här, då kan man på någon minut uppdatera utbildningsinsatsen så att den täcker det aktuella angreppet. Det är ju det som är specifikt, att du ska kunna leverera eftersom verktyget är lätt att jobba med på båda sidorna, så ska du kunna leverera en oerhört riktad utbildning på en väldigt kort tid eller en väldigt, väldigt snabb tid.

Jonas:

(36)

2

<paus i 30 sekunder för utrustningskontroll> Jonas:

Så där det verkar fungera. Skulle du kunna ge ett exempel på hur en implementation skulle kunna ske? Marcus:

Ett typiskt exempel som finns skulle kunna vara… Låt oss slänga till med en man-in-the-middle attack. Någon hackar betalningstjänsten, de hackar din DNS så att den pekar mot en annan betalningstjänst, så att du skriver in ditt kontonummer på en falsk site. Det är en ganska basic attack att tänka sig, och är fortfarande en ganska datalogisk attack om vi nu ska ha på oss den hatten. Då skulle ju du som e-handelsbutik kunna erbjuda, i samband med… Jag vet inte hur mycket du handlar på nätet, men väldigt ofta när jag handlar på nätet klickar man och blir vidarebefordrat till en betalningshanterare, DIBS eller liknande. Under den väntetiden som är, skulle man kanske kunna förlänga den väntetiden fem sekunder och visa en animation som förklarar; ”titta på det här för att se att du kommit till en säker site”. Som, hypotetiskt nu, låt oss säga att den känner av att du surfar med Chrome, den vet hur symbolen ser ut för en säker site och du visar den animationen varje gång; ”glöm inte att kontrollera så att det här stämmer”, ”kolla så att du alltid har det här ikryssat” eller ”klicka gärna här och se att det här stämmer för det är såhär det ska vara”. Den delen äger du ju fortfarande som internetbutik och det innebär ju också att om du skulle ha en trend med en ökad mängd angrepp på det här, om det skulle bli ”inne” med man-in-the-middle attacker. Då kommer vi dessutom kunna; dels tvinga ut det till alla så att alla får se det hela tiden, vi kan också göra det extra tydligt beroende på HUR sker det här angreppet. Istället för att säga; ”glöm inte kolla att du alltid har ett hänglås”, så kan vi vara extremt effektiva med att säga; ”kolla så att du har ett hänglås, att det är grönt och att det inte står den här serveradressen”. Vi kan i DEN stunden ge information, och här behöver det inte vara en två minuters film eller en tio minuters film om; ”så här fungerar SSL”, för det bryr sig ingen om. Utan det kan vara titta här just nu för det här är viktigt. Så det är ju ett sätt där man kan in och jobba, och informera folk. Sen behöver man kanske inte visa det varje gång, man kan ju sätta en cookie på att du har sett den här filmen två gånger så behöver man inte hela tiden bli upprepad. Det kan man reglera på lite olika sätt. Idén är ju inte att störa folk utan att informera där det behövs, att stärka.

En rimlig motfråga är kanske att jag frågar, hur har du tänkt genomföra experimentet? Jonas:

Så som experimentet är upplagd nu så har jag två testgrupper, en testgrupp och en baseline. Baseline får fyra länkar till fyra olika webbshoppar där de får handla fritt utifrån en inköpslista de fått. De får även ett kreditkort de ska använda för att handla med. Den andra gruppen kommer först hamna i lärmodulen som säger att du är på väg att besöka en webbshop med frågor att svara på, t.ex. två bilder på URL:er med hänglåssymbolen på ena där de sedan får veta om det är rätt eller fel och får lite mer information om varför det är så. Efter utbildningen fortsätter de från den punkt baseline gruppen startade. Är det i linje med hur du tänk att det skulle kunna fungera?

Marcus:

Ja, det är helt rimligt, så jobbar vi. Frågebiten finns också med, med rätt eller fel så att du är engagerad.

Exemplet jag berättade om är bara ett extremt minimalistiskt exempel. Det är helt korrekt, så rimligtvis hade man i den här stunden kunnat lägga till; ”vilken är ett godkänt hänglås”, för att i stunden efter gå vidare. Så det man kan reflektera över i exemplet du har, är i strukturen. Huruvida det är gynnsamt att ge den informationen innan du börjar med en bred uppgift eller om det skulle vara att ge det precis där incidenten kommer. Exemplet jag hade är väl ett av de mest optimala fallen, när du kan ge det precis där risken sker. Verkligheten är ju att det är svårt att göra det. Det finns ju några sådana case, t.ex. när det är dags att välja ett nytt lösenord så får du en utbildning, ett litet sådant minitest, med så här väljer du ett bra lösenord. Men generellt så ser jag inget problem med det sättet du strukturerar upp det eftersom det ändå blir inom en ändlig session, jag kan tänka mig att sessionen tar ca 15 minuter för dem eller?

Jonas:

(37)

3

Marcus:

Ja, precis. Då är det ju fortfarande i närminnet, jag ser inga metodproblem med det här. Det är bara ett sätt för dig att strukturera upp en uppgift för att bli hanterbar, det är jättesvårt annars.

Jonas:

Ja, det är inte så jättelätt. Hur jämför det sig med andra skyddsmetoder och lärmetoder? Marcus:

Dilemmat är ju att vi inte vet hur någonting är. Vi hade nyligen en ganska stor studie tillsammans med MSB, där vi gick igenom; vad är sagt om utbildningsnytta i samband med informationssäkerhet. Vi gick igenom, bokstavligen, hundratals papers. I stort sett allting som publicerats, och slutsatsen är att vi har ingen aning. Vi vet helt enkelt inte. Det finns massa antaganden om vad som är bra, det finns massa truismer, det finns olika exempel på saker som fungerat bra och mindre bra. Men vi har helt enkelt inte kunna göra ett bra jämförande test med; hur utbildar vi folk med det här. Det i sig är ju ganska chockerande tycker jag, eftersom man egentligen inte har ifrågasatt vilken utbildningsinsats vi ska ha. Utan man kan möjligtvis se, fungerar den här formen utav utbildning. Men det mer övergripande då… ska vi utbilda folk vid ett tillfälle, ska vi ha webbutbildningar, ska vi ha hands-on, ska vi ha träningsscenarion, ska vi testa folk, ska vi göra penetrationstester, ska vi ha den här sortens mikroutbildning eller något annat. Den frågan har man inte riktigt ställt sig. Det är väldigt svårt att säga om den här är bättre eller sämre än någonting annat, eftersom vi vet ju inte om någonting annat ens är bra eller dåligt. Vad man möjligtvis kan se i den här sortens studie är om det fungerar bättre än inget. Det blir ju någon sorts, ja, är det bättre än att inte ha någonting alls jämförelse. Ja, det är det ju förhoppningsvis. Men en annan fråga som är viktig; är detta över huvud taget en effektiv väg att gå. Så till vida, kommer folk att bli irriterade, vilken funktion får det. Men det som är, nu spinner jag loss här som jag gärna gör… det som är viktigt att ha med är att det finns ju en specifik målgrupp med det här tänket. Målgruppen med det här tänket är ju människor som faktiskt inte har säkerhet som en primary task, som inte är engagerade primärt utav säkerhet. Så att, man kan inte riktigt jämföra äpplen och päron. För på ena sidan kan man få; hur utbildar vi folk för att bli en duktig säkerhetsspecialist. Ja, en mångårig utbildning, praktik och hela kittet bör ju vara det effektivaste sättet för att lära upp dig att bli en Senior Administratör på ett företag. Det här [konceptet] är ju riktat mot ointresserade som bara vill handla en koppvärmare via teknikmagasinet och plötsligt dyker det upp massa saker som de måste kunna om säkerhet. De vill handla sin koppvärmare och vara nöjda med det, då kan man utbilda dessa med en LITEN insats men ändå göra en stor nytta. Mycket av den forskning som ändå har skett har varit riktat med; hur får vi jätteduktiga användare i en stor organisation som kan allt möjligt om säkerhet. Där kan det här vara ett verktyg, men målgruppen med vårt tänk är framförallt allmänheten.

Jonas:

Så med andra ord är det första steget att få ett proof of concept? Marcus:

Ja, och här testar vi ju lite olika vinklar på det. Vi kollar webbshop biten via projektet du är med i, vi har tittat på begagnat siter och även mobilapp. Min magkänsla just nu utifrån de preliminära resultaten vi har är att det här är användbart. Det är framförallt underutnyttjat i förhållande till hur pass användbart det är. Jag tror att det är både kostnadseffektivt och att det är effektivt som en förhindringsegenskap. Sen får man ju vara försiktig, för det finns ju inom ekonomiska termer; ”the law of diminishing return”, vilket innebär att om alla siter använder detta hela tiden i allting du gör. [Överanvändande kommer resultera i att] det bara blir som en irriterande popup hela tiden, [vilket då gör det] oanvändbart. Om det kommer det upp en enminutsfilm med en fråga i slutändan om vad du ska tänka på varje gång du söker efter iPhone på blocket, så blir det ju inte effektivt. Men jag tror att om vi… för det är ju ändå som så att vid 99,5 % av vår internetanvändning, eller teknikanvändning över lag, gör vi inte saker som är riskabla. Att skriva ett mail till en kompis är inte riskabelt. Däremot finns det ju vissa tillfällen där vi blir exponerade för någonting som är uppenbart riskabelt; saker och ting som involverar lösenordsbyten,

(38)

4

Jonas:

Om man nu skulle sätta upp ett experiment för att få ett proof of concept, hur omfattande måste experimentet vara i så fall?

Marcus:

Tänker du på antalet subjekt eller tester? Jonas:

Ja, framför allt antalet subjekt för att få validiteten i experimentet. Marcus:

Många gånger handlar ju sådant här om en magkänsla, mer än om man har en enkät. Gör man en enkät kan man ju räkna ut exakt, med konfidensintervall och sådana saker, men det är ju inte riktigt vad det handlar om här. Men jag kan tänka mig att om man har som en absolut lägstanivå, två grupper om tio personer. Under det skulle jag nog ifrågasätta det. Sen kan man ju argumentera med för och nackdelar för det. Tjugo är bättre givetvis, det finns ju ingenting uppåt där som inte är bättre än den tidigare siffran. Men som sagt, skulle man hamna på två tester om tre subjekt så får man vara väldigt tydlig… men det är ju den andra delen. Tio är någon slags baseline-magkänsla. Men det viktigaste är; hur har urvalet skett. Om det är två gånger tio Nätverk- och

Systemadministrations studenter som läst informationssäkerhet, nätverkssäkerhet och riskhantering, så är det ju väldigt svårt att uttala sig generellt om någonting. Då är ju frågeställningen mer inriktad på hur man skulle kunna påverka NSA studenter inom bla. bla. bla. Vilket i sig också skulle kunna vara en helt valid frågeställning givetvis. Det handlar mer om att lyckas få till ett hyffsat jämnt urval eller en hyffsad representation. Tio är ju ungefär det man på någotvis, utan att behöva skämmas, kan argumentera för att man kan få någon sorts jämt flöde med olika åldrar, olika kön o.s.v. Tjugo är väl egentligen bättre, ska vi tala om en statistisk säkrad undersökning i större skala så talar vi om hundratals. Jag vet inte riktigt vad ambitionsnivån ligger på idag? Jonas:

Jag satsar på att få runt trettio deltagare, två grupper om femton personer. Marcus:

Ja, och för ett exjobb tycker jag absolut att det är indikativt om vart saker och ting är på väg. Men kika på hur det slumpmässiga urvalet ska ske.

Jonas:

Tanken är att vi ska stå på campus och fråga alla som passerar förbi och är studerande vid skolan, de som tackar ja får delta.

Marcus:

Ja, ok. Men då kan man beskriva det på det viset. Ofta är det lätt att be sina kompisar göra testet och ur kompisurvalet blir det som oftast en våldsam snehet.

Jonas:

Nej det blir de som passerar där vi står på campus, det kan ju hända att det passerar NSA studerande också dock. Marcus:

Ja, ja. De finns ju också så det är ju inget konstigt på det viset. Jonas:

Du nämnde blocket förut, hur skulle detta kunna implementeras för just blocket? Marcus:

Det visar sig att om man kollar på de här så kallade blocket bedrägerierna, eller

SECURITY AWARENESS THROUGH MICRO-TRAINING: An initial evaluation of a context based micro-training framework