INOM
EXAMENSARBETE
TEKNIK,
GRUNDNIVÅ, 15 HP
,
STOCKHOLM SVERIGE 2020
Why Phishing Works on
Smartphones
JOAKIM LOXDAL
MÅNS ANDERSSON
Why Phishing Works on Smartphones
Joakim Loxdal, M˚
ans Andersson
{loxdal, manande}@kth.se
BTh KTH Royal Institute of Technology
Swedish title: Varf¨
or smartphone-phishing fungerar
Supervisor: Robert Lagerstr¨
om (robertl@kth.se)
June 5, 2020
Abstract
Phishing is a form of internet fraud where an attacker attempts to acquire sensitive information from a target by posing as a trustworthy entity. One strategy to fool the target is to create a spoofed (illegiti-mate copy) of a legiti(illegiti-mate website. But why do people fall for spoofed sites in smartphone browsers, and what security indicators are utilized or not when a user decides the legitimacy of a website? Can smartphone browsers make it easier for users to identify phishing sites? In this study, 20 participants were observed when they analyzed and classified websites as legitimate or spoofed on their own smartphones. 17 websites (8 spoofed and 9 legitimate) were presented to the participants in random order and classified by the participants.
The best phishing site fooled 50%, and on average participants classi-fied 69% of the websites correctly, similar to previous studies results. The URL was used as an indicator by a majority of the participants (80%), a result that also matches previous similar studies. This indicates that user behaviour and ease of identifying spoofed and legitimate websites is not very different in a smartphone browser compared to a desktop computer browser. Those not evaluating the URL performed the worst. Almost all of the participants (>90%) used the content of the website (design, information, functionality) at least once when deciding if a website was spoofed or legitimate. Just one participant used Google to find the legiti-mate websites and compare to the one he was presented with in the study. He was the only participant with a success rate of 100%.
We suggest that browsers put more emphasis on the domain name, and that browser developers should even consider hiding sub domains in the smartphone address bar.
Sammanfattning
Phishing ¨ar en typ av internetbedr¨ageri d¨ar en bedragare f¨ors¨oker komma ˚at k¨anslig information fr˚an ett offer genom att utge sig f¨or att vara n˚agon annan. En strategi som ¨ar vanlig ¨ar att bedragaren skapar en falsk kopia av en verklig hemsida. Men varf¨or faller anv¨andare f¨or phishing i smartphone-webbl¨asare, och vilka s¨akerhetsindikatorer anv¨ands f¨or att avg¨ora om en sida ¨ar ¨akta eller falsk? Kan smartphone-webbl¨asare g¨ora det enklare att identifiera phishing-sidor? I denna studie observerades 20 deltagare n¨ar de analyserade och klassificerade hemsidor som legitima eller falska. 17 hemsidor (8 falska och 9 ¨akta) presenterades f¨or deltagarna i slumpad ordning och klassificerades.
Den b¨asta phishing-sidan lurade 50%, men i genomsnitt gissade delta-garna r¨att i 69% av fallen, ett resultat som ¨overensst¨ammer med tidigare liknande studier. En majoritet av deltagarna (80%) anv¨ande URL:en som en indikator, n˚agot som ocks˚a ¨overensst¨ammer med tidigare studier. Detta indikerar att anv¨andares beteende och f¨orm˚aga att identifiera ¨akta och falska hemsidor i en webbl¨asare inte skiljer sig s¨arskilt mycket mellan smartphone och dator. De som inte tittade p˚a URL:en hade l¨agst antal r¨att. N¨astan alla deltagare (>90%) anv¨ande sig av sidans inneh˚all (de-sign, information, funktionalitet) ˚atminstone en g˚ang n¨ar de skulle avg¨ora om en sida var ¨akta eller falsk. En enda deltagare anv¨ande Google f¨or att hitta de legitima sidorna och j¨amf¨ora med de han fick presenterade framf¨or sig i studien. Han var den enda deltagaren som identifierade alla sidor i studien korrekt.
Vi f¨oresl˚ar att webbl¨asare tydligg¨or dom¨annamnet i st¨orre utstr¨ackning och att utvecklare av webbl¨asare till och med b¨or ¨overv¨aga att helt d¨olja subdom¨aner i adressf¨altet p˚a smartphones.
Acknowledgements
A special thanks to Robert Lagerstr¨om and Simon Hacks for providing great support and feedback. We would also like to thank all participants who made the study possible.
Contents
1 Introduction 1 1.1 Problem statements . . . 2 1.1.1 Scope . . . 2 1.1.2 Expectations . . . 2 2 Background 3 2.1 Web technologies . . . 32.1.1 Domain names and URLs . . . 3
2.1.2 SSL/TLS/HTTPS . . . 3
2.1.3 Certificates . . . 4
2.2 Browser security indicators comparison . . . 4
2.3 Previous phishing research . . . 5
2.3.1 Smartphone phishing research . . . 6
2.4 Why Phishing Works & Why Phishing Still Works . . . 7
2.5 Countermeasures against phishing on smartphones . . . 8
3 Methodology 9 3.1 Overview of study . . . 9
3.2 Inviting participants to the study . . . 9
3.3 Information gathered about participants . . . 9
3.4 URL Spoofing techniques . . . 10
3.5 Website cloning techniques . . . 11
3.6 Server Setup . . . 11
3.7 The websites . . . 11
3.7.1 Spoofed websites . . . 12
3.7.2 Legitimate websites . . . 13
3.8 Information given to participants . . . 13
3.9 Pilot Study . . . 14 4 Results 15 4.1 Participants . . . 15 4.2 Correlation . . . 15 4.2.1 Age . . . 15 4.2.2 Technical Proficiency . . . 15 4.2.3 Sex . . . 15
4.2.4 Active smartphone use per day . . . 15
4.3 Strategies . . . 16
4.3.1 Strategy 1: Evaluating the sites design . . . 16
4.3.2 Strategy 2: Evaluating the sites functionality . . . 16
4.3.3 Strategy 3: Evaluating the sites information . . . 16
4.3.4 Strategy 4: Evaluating the sites URL . . . 17
4.3.5 Strategy 5: Using security indicators in the browser . . . 17
4.3.6 Strategy 6: Using a search engine to find the legitimate site 18 4.4 Combinations of strategies . . . 18
4.5 Website difficulty . . . 19 5 Discussion 20 5.1 Correlations . . . 20 5.2 Website evaluation . . . 20 5.2.1 Spoofed sites . . . 20 5.2.2 Legitimate sites . . . 21 5.3 Participants strategies . . . 22
5.4 Smartphone browser improvements . . . 24
5.5 Comparing with previous studies . . . 25
5.6 Future work . . . 25
5.7 Threats to validity . . . 26
5.8 Conclusion . . . 26
6 References 28
1
Introduction
Cyber crime is becoming more and more sophisticated and cyber criminals reached $3.5bn in profits in 2019, BBC reported in February 2020, citing the FBI1. Phishing and extortion are the most common ways of scamming
peo-ple1.
Phishing is a form of internet fraud where an attacker attempts to acquire sensitive information from a target by posing as a trustworthy entity. The goal of phishing is often monetary gain by getting access to the targets information or access privileges. What differs phishing from other forms of cyber crime is that it takes advantage of human nature by fooling the target into giving up the information voluntarily. For instance by posing as the targets bank or employer and requesting login details, often by linking the target to a spoofed website (a fraudulent copy) that is hard to distinguish from the original website. Phishing can also include sending an attachment or link containing malware.
Phishing is one of the largest security threats and contributes to 90% of all data breaches. In 2018 the Federal Bureau of Investigation estimated losses of $12 billion for companies worldwide due to phishing (Das et al. 2019).
A common phishing methodology using website/email spoofing (Oest et al. 2018):
1. The attacker spoofs a website
2. The attacker sends messages to the target with a link to the spoofed website, insisting action is required
3. If the target is successfully fooled, he or she visits the website and inputs credentials
4. Finally, the information is sent back to the phisher who uses it for mone-tary gain or to gain access to sensitive information.
Our study focuses on phishing using website spoofing techniques since it’s a common threat towards users and organisations. According to the Anti Phishing Working Group, the number of phishing websites in September 2019 were at the highest level since 2016 (APWG 2019a).
In our study we examine what makes a phishing website convincing on a smart-phone, and what methods users utilize to determine the legitimacy of a website when viewing it in a smartphone browser. This is examined through interviews where the participants are shown a random sample of websites (both legitimate and spoofed). The participants then decide if each website is spoofed or legiti-mate. Previous studies have examined this in a desktop computer environment, but there is a lack of studies examining phishing susceptibility among smart-phone users. Our methodology is based on the methods used in the studies 1 ”Cyber-crime profits reached $3.5bn in 2019, says FBI.”, BBC. 12 February 2020.
“Why Phishing Works” (Dhamija, Tygar, and Hearst 2006) and “Why Phish-ing Still Works” (Alsharnouby, Alaca, and Chiasson 2015), but on smartphones instead of desktop computers. Smartphone phishing is a relevant subject to examine, since a majority of website traffic in 2019 came from mobile devices according to the web analysis firm Statcounter2.
1.1
Problem statements
• What are indicators that users utilize or fail to utilize to decide whether a website is legitimate or spoofed on a smartphone?
• How can browsers make it easier for users to identify spoofed websites? • Is there any correlation between susceptibility of phishing and age,
tech-nical proficiency, sex or smartphone habits? • Does the results differ from previous studies? 1.1.1 Scope
Our study is limited to smartphones running Android and iOS, as they are the two most common smartphone operating systems. Our study focuses on the website spoofing part of phishing and does not examine phishing messages which is often the first part of a phishing attempt.
1.1.2 Expectations
The URL (Uniform Resource Locator) is expected to be one of the most com-monly used indicators when a user decides if a website is legit. In the study ”Why Phishing Works” 77% of the participants evaluated the address bar to make judgements on a websites legitimacy (Dhamija, Tygar, and Hearst 2006), 32% used the padlock icon while only 9% used certificates. Evaluating the sites overall design and functionality is probably the most common strategy (in combination with other strategies), which was observed in both ”Why Phishing Works” and ”Why Phishing Still Works”.
The fact that our study is performed on the participants own mobile devices might make it easier to identify fraudulent websites, compared to previous stud-ies, since some websites might open in apps and some might already be logged in. No significant correlation between phishing susceptibility and age, sex, tech-nical proficiency or computer use was found in ”Why Phishing Works” or ”Why Phishing Still Works”, and is therefore not expected.
2Statcounter. ”Desktop vs Mobile vs Tablet Market Share Worldwide”, Statcounter, April
2
Background
2.1
Web technologies
2.1.1 Domain names and URLsDomain names are used as aliases for IP addresses on the internet. When a user asks for example.com, a request is made to a DNS server that responds with the IP address of example.com.
There are different conventions for describing different parts of a URL. Below is an example to describe the terminology used in this report.
Example URL: https://login.example.com/path 1. Scheme: https
2. Host name: login.example.com 3. Domain name: example.com 4. Top level domain (TLD): .com 5. Sub domain: login
6. Path: /path
The scheme specifies the protocol used in the request to the host. The host name will lead the request to a specific IP address. ”.com” is a TLD, among more than a thousand other TLDs. example.com and example.net can lead to entirely different web sites and have different owners. A domain name can be bought by anyone from a domain name registrar. Different sub domains can lead to different IP addresses, but the owner of example.com is also in control of login.example.com and all other possible host names ending with example.com. The path is the specific resource that is requested from a website, for example a specific page on the site or an image.
2.1.2 SSL/TLS/HTTPS
TLS (Transport Layer Security) is a cryptographic protocol used for transfering data privately over a network. It is a part of the HTTPS protocol (Hyper-text Transfer Protocol Secure) to secure connections between web servers and browsers. TLS replaced SSL when it was deprecated in 20153. Correctly
im-plemented and configured HTTPS encrypts the communication between web servers and browsers and guarantee that the correct website is displayed for the requested domain. A guarantee that the client is communicating with the cor-rect server is made possible through the use of certificates and a process called a TLS-handshake. Non-HTTPS sites are susceptible to DNS spoofing where an attacker would poison a DNS cache in order to associate a host name with an 3Barnes, et al. ”Deprecating Secure Sockets Layer Version 3.0”, Internet Engineering Task
IP address under their control. HTTPS websites are displayed with a padlock in the address bar in all commonly used browsers.
According to the Anti Phishing Working Group, 74% of the phishing websites they examined during the 4th quarter 2019 were using SSL/TLS, so HTTPS gives no guarantee of a websites reliability (APWG 2019b).
2.1.3 Certificates
Certificates signed by special Certificate Authorities (CA) are used in when establishing a HTTPS connection to a website. If the web server provides a domain validated (DV) certificate and a HTTPS connection is successfully es-tablished, web browsers show this by displaying a padlock next to the URL in the address bar. Extended validation (EV) certificates provide a larger degree of security since they prove the legal identity of the certificates owner.
A DV certificate can verify that the website being visited server actually belongs to the owner of the domain name. But the DV certificate does not give you any information about who the owner is, unlike EV certificates. Some web browsers (e.g. Safari) color the URL and padlock green to visualize that the website has an EV certificate. Studies have observed that the introduction of EV-certificate indicators in browsers actually increase confusion rather than increasing trust among users (Sobey, Van Oorschot, and Patrick 2009). The referenced study was made in 2009 and since then multiple browsers have stopped visually indi-cating EV-certificates directly in the address bar. 4
2.2
Browser security indicators comparison
iOS Safari, Android Chrome and Samsung Internet are the three most common mobile browsers according to Statcounter5. All of them use common methods
to inform the user about insecure connections and known phishing websites. Below is a more detailed list of observed security indicators in the different browsers.
Safari (iOS version 13.3.1 running on an iPhone 7):
• Padlock icon displayed to the left of the URL indicating HTTPS
• The text ”Not secure” displayed to the left of the URL indicating HTTP • Green URL text color and padlock to indicate use of an EV-certificate • Shows the host name (including sub domains). The path and scheme is
hidden.
4 Keizer, Gregg. ”Chrome, Firefox to expunge Extended Validation cert signals”,
Com-puter World, August 15 2019. https://www.computerworld.com/article/3431667/chrome-firefox-to-expunge-extended-validation-cert-signals.html
5 ”Mobile Browser Market Share Worldwide”, Statcounter, April 2020.
• Displays a warning about sites that have been flagged as phishing or mal-ware
Chrome v.81.0.4044.111 (Android 9 running on a Huawei P20 Pro): • Padlock icon displayed to the left of the URL indicating HTTPS • ”i” icon displayed to the left of the URL indicating HTTP
• The ability to inspect TLS/SSL certificates (by clicking the padlock) • Highlights the host name (including sub domains) in a contrasting color,
and displays path in a less visible color. The scheme is hidden.
• Displays a warning about sites that have been flagged as phishing or mal-ware or if ”The sites URL is slightly different from a URL in your browsing history”6
Samsung Internet v.11.1.2.2 (Android 9 running on a Huawei P20 Pro): • Padlock icon displayed to the left of the URL indicating HTTPS • ”i” icon displayed to the left of the URL indicating HTTP
• The ability to inspect TLS/SSL certificates (by clicking the padlock) • Highlights the host name (including sub domains) in a contrasting color.
The scheme and path is hidden.
• Displays a warning about sites that have been flagged as phishing or mal-ware
A difference between the browsers is that Safari indicates the use of a HTTP-connection by displaying the text ”Not secure” to the left of the URL. Chrome and Samsung internet instead display an ”i”-icon. Clicking the icon displays a message stating that the connection is not secure.
Both Chrome and Samsung Internet allows you to look at the certificate which includes information about what CA signed it and more. This is not possible in Safari.
Another difference is that Chrome and Samsung internet completely hides the address bar when you scroll down on a site. Safari makes the address bar smaller but the host name is always visible.
2.3
Previous phishing research
In ”Investigating personal determinants of phishing and the effect of national culture” (Flores et al. 2015) cultural differences in phishing behaviour is ex-amined. The study concluded that national culture has some influence on the correlation between phishing determinants and employees’ observed phishing 6”Manage warnings about unsafe sites”, Google, 2020.
behavior. This means that different approaches should be considered when de-signing effective information security programs depending on the cultural con-text of the country. The results also indicate that it is hard to create security indicators that work for all users.
In ”Who falls for phish? A demographic analysis of phishing susceptibility and effectiveness of interventions” (Sheng et al. 2010) further analysis of factors linked to susceptibility of phishing is examined. The study results suggests that women are more susceptible to phishing than males and that the age group 18-25 is more susceptible to phishing than other age groups. Furthermore the study saw a 40% reduced risk of participants clicking phishing links, when users received educational material about phishing. But even after training, partici-pants fell for 28% of the phishing messages in the study.
According to the article ”Social phishing” (Jagatic et al. 2007), the way a mes-sage containing a malicious link is constructed and who seems to be the sender of the message have a big impact on how many will fall for the phishing attempt. In the study, 921 unknowing subjects received phishing attacks. The researchers used information found on social media accounts and spoofed email accounts of victims friends to make the phishing attempts more convincing. This proved successful. 72% of those being phished with the help of social media information followed a ’malicious’ link, while only 16% from the control group followed the link.
Worth noting is that all three studies mentioned in this section defined a user as being phished when they clicked a phishing link in an email.
2.3.1 Smartphone phishing research
In the study Phishing on Mobile Devices (Felt and Wagner 2011), the researches examine how vulnerable smartphones are to phishing. The study notes that security indicators are less visible for the user on smartphones, both when de-termining the identity of a website in a browser window and the identity of a running application. This is because of the smartphones limited screen size. Another risk factor on smartphones is that users often click on links in other applications (e.g. Social media apps) which then display the website in an embedded browser window. The embedded browser window show even less information about the website being visited, with smaller text.
The article ”Mobile phishing attacks and defence mechanisms: State of art and open research challenges” (Goel and Jain 2018) brings up the fact that smart-phones introduces new paths for phishing attacks. 87% of all phishing attempts on smartphones are not carried out using email. Through text messages (SMS), MMS and trusted mobile applications users are fooled to click links, get redi-rected, or share data.
According to the article, mobile devices are three times more vulnerable to phishing attacks than desktop computers because of screen size, lack of
aware-ness and inconvenience of user input. Therefore separate techniques are needed to avoid these attacks.
2.4
Why Phishing Works & Why Phishing Still Works
The methodology in this study is based on the studies Why Phishing Works (WPW) (Dhamija, Tygar, and Hearst 2006) and Why Phishing Still Works (WPSW) (Alsharnouby, Alaca, and Chiasson 2015). These studies use a very similar methodology except for the fact that WPSW utilizes eye tracking tech-nology to investigate where users look when they try to determine the legitimacy of a website. WPSW was performed with participants recruited from Carleton University in Ottawa, Canada. In WPW participants were also recruited from a university (both students and staff) but it is not specified which one. From the report it can be concluded that study was most likely performed in North America. The method used in both studies was that the users participating got to look at a collection of websites (24 and 19 websites respectively). For each website the user was asked to decide whether they believed the website was legitimate or not. The average success rate in WPSW for correctly identi-fying a website as either legitimate or not was 64% compared to 58% in WPW. However if you only look at the non-legitimate sites the succes rate was 53% and 54% respectively. Even though 9 years passed between the studies (2006 and 2015) it does not seem like users are getting better at identifying fraudulent websites. For legitimate sites only, the success rate was 79% in WPSW and 75% in WPW. Drawing conclusions from comparisons of the studies should be done with care since they used different websites for their studies even though the spoofing techniques used were similar. Additionally, the studies were limited to 22 and 21 participants respectively, meaning the results does not necessarily represent the average user.The different spoofing techniques to create fake websites used in WPSW were: • Incorrect URL. These sites were a perfect copy of the original and all
the links except the login link pointed to the legitimate version of the site. Sometimes they created a URL that was just a mistyped version of the original e.g. cartelon.ca instead of carleton.ca. In other cases they created URLs that were completely different from the original but could still come off as legitimate e.g. scotiabank.secure-encrypt05.com instead of just scotiabank.com. This method was used for 8 sites.
• IP address URL. The same technique as above was utilized but no domain name was used so the URL bar just showed the IP address of the server. This method was only used with one site.
• Fake chrome. The website showed a fake version of the web browser Google Chrome inside of the real browser together with the content of the website. The fake version of Google Chrome showed the legitimate URL whereas the real browser showed a fake URL. This was used for one site.
• Popups asking for credentials. The fake website presented a popup screen which asked the user to input login information. This was used for one site.
• Overlaid popup windows. A popup that covered all of the content of the website whereas the original browser window was redirected to the real website. In this case you could notice that there were actually two Google Chrome windows. This was used for one site.
• Fraudulent based on context. They included one website that was not trying to look like any other site. Instead it was a website that was meant to be obviously fraudulent. It asked for the users credit card information in order to check if the credit card had been stolen or not.
The sites described above were used in the study together with 10 legitimate sites. Some of the legitimate sites presented were non-HTTPS. For example the HTTP version of Netflix was used. These days https has become industry standard and most big organizations only allow https connections.
In WPW, a few different strategies used by phishing attackers are defined and organized in three dimensions: lack of knowledge, visual deception, and lack of attention. Lack of knowledge includes the users knowledge about security indicators, email, their operating system and domain names. For example a user may believe that paypal-fastpayments.com belongs to paypal.com. Security indicators like the HTTPS Padlock may give users a false sense of security, when they don’t know the meaning of it, or it might not be recognized at all. Visual deception includes visually deceptive text (e.g. paypall.com instead of paypal.com), fake links or images that look like browser UI elements. Lack of attention is the last dimension, that combined with the previous strategy may lead to the user being fooled even though they hold knowledge about security indicators.
2.5
Countermeasures against phishing on smartphones
In the article ”Mobile anti-phishing: Approaches and challenges” (Shahriar et al. 2019) several mitigation techniques are described as suggestions against phish-ing on mobile devices. One technique is to analyze IP packets with Machine learning. Another is to analyze if the layout of the visited website seems to be copying another websites layout. Another method to identify phishing websites is static analysis. The article mentions MobiFish, an anti phishing application that detects IP URLs and warns the user. It also identifies forms on the website and warns the user if the website requests login-details. If it does, MobiFish searches for the host name in a whitelist. If the host name is not whitelisted and not displaying its domain name (excluding TLD) in the websites content, the user is presented with a warning. Other techniques about user authentication is discussed. The authors of the article suggest to implement several of these techniques to combat phishing.In ”Learn to Spot Phishing URLs with the Android NoPhish App” (Canova et al. 2015) the authors discuss how education can help counter phishing on smartphones. The mobile education application NoPhish teach users to evaluate URLs and not to trust email addresses and links. The different URL spoofing techniques that the app educates users about are:
• IP address
• Random address, no brand name
• Random address, brand name in sub domain • Random address, brand name in path
• ’Derivated domain’ (e.g. facebook-login.com instead of facebook.com). • Domain name with a typo (e.g. facebok.com instead of facebook.com) • Replacing characters (e.g. arnazon.com instead of amazon.com)
3
Methodology
3.1
Overview of study
To answer our problem statements we interviewed 20 users about their expe-rience and reasoning while looking at legitimate and spoofed web sites. We spoofed 8 popular websites and their URLs using different techniques. 9 legiti-mate websites were also used in the study. The participants were directed to a navigation site (see figure 1) that contained links to all of the 17 web pages in a random order. We let users use their own smartphone when performing the study. All interviews were conducted through the video conference app Zoom where participants shared their screen to make it easier for us to observe their behaviour.
3.2
Inviting participants to the study
We wanted a diverse group in respect to age, sex, technical knowledge and smartphone use. We sent out invitations on our personal Facebook accounts and to two different Facebook groups. We also asked friends and family to recommend participants. All participants were native Swedish speakers and smartphone users. Participants were rewarded with a movie ticket.
3.3
Information gathered about participants
We asked about the participants age, sex and technical proficiency. We also asked about the participants smartphone operating system, browser as well as average active time spent on their smartphone each day. For every website shown we asked about the reasoning behind the participants decision and how
Figure 1: The navigation site.
confident they felt in their decision (on a scale 1-5). This is information gath-ered in previous similar studies and is therefore relevant when comparing our results.
In order to determine a users technical proficiency we utilized the same method used in WPSW. Five questions were asked which were considered as represen-tative of a users technical proficiency in a web context. Each yes-answer gave the user one point. As a result each user got a technical proficiency score from 0 to 5.
The questions were:
• Have you ever designed a website? • Have you ever changed firewall settings? • Have you ever installed an operating system? • Have you ever registered a domain?
• Have you ever used Telnet or SSH?
3.4
URL Spoofing techniques
Techniques that were used for spoofing website URLs:
U1. A mistyped URL close to the original (e.g. ablidris.com instead of adlib-ris.com)
U2. A URL that is quite different from the original but might come off as legitimate (e.g. swedbank-privat.se instead of swedbank.se)
U3. A different top-level domain than the original (e.g. elgiganten.online in-stead of elgiganten.se)
U4. A sub domain used for deception (e.g. outlook.com-secure.live instead of outlook.live.com/owa)
U5. A URL consisting of a regular IP address (e.g. 35.228.129.249)
3.5
Website cloning techniques
We used the tools HTTrack, wget and the Firefox plugin SingleFile for cloning our selected websites. For some clones manual tinkering was necessary in order to get the site working properly. For example there could be some Javascript on the page that was not executing properly in the cloned version. Not all pages had all of their functionality cloned perfectly. A more detailed description of flaws can be found in section 3.7.1. This was not considered a problem since it was found interesting to discover if users would be able to notice these flaws and raise suspicion.
3.6
Server Setup
We served our navigation page and spoofed sites on a Node.js express server running on the cloud hosting provider DigitalOcean. The domain names were purchased from GoDaddy.com and MissHosting.se and were all pointed towards the servers IP address. All SSL-certificates were signed by LetsEncrypt, a free Certificate Authority. We tried to make our setup as realistic as possible and made it possible for participants to connect to our server from any location on their own smartphone.
Our server identified which host name was used for connecting to the server and served the corresponding website. We also checked the Referer-header in every request to make sure no external user would visit our websites unintentionally. The server was turned off after each interview was conducted for the same reason.
3.7
The websites
We looked at the top websites in Sweden using alexa.com and similarweb.com, to make sure that most participants had visited the web page before. Our focus was on e-commerce, finance and social media sites, since all those pages require login details. According to the APWG (APWG 2019b) Saas/Webmail, Pay-ment/Financial, Social media and E-commerce were the most targeted sectors in the 4th quarter 2019. In the two sections below all websites used in the study are listed, (Table 1 and 2). E in the ”Type”-field stands for e-commerce, F for finance and SM for social media.
3.7.1 Spoofed websites
ID
Type
URL
Description
Method
S1
SM
https://fb.login.com.se
Facebook Login
U4
S2
F
IP ADDRESS (http)
Klarna front page
U5
S3
SM
http://twitter.loginsecurity.online
Twitter Login
U4
S4
E
https://ablidris.com/
Adlibris front page
U1
S5
F
https://swedbank-privat.se
Swedbank front page
U2
S6
F
https://skattesverket.se
Skatteverket front page
U1
S7
SM
https://outlook.com-secure.live
Outlook front page
U4
S8
E
https://elgiganten.online
Elgiganten front page
U3
Table 1: The spoofed sites used in the study.
The spoofed sites had all of their links pointing to the real version of the website. The website cloning techniques used provided perfect copies of some sites but there were others that were not cloned perfectly.
• S4 (Adlibris) lacked the functionality to add items to the cart. Clicking the add-to-cart button for a product started a loading animation that never stopped.
• S5 (Swedbank) had some replaced functionality. The real version had a text box to ask questions to a chat bot, which we did not manage to recreate. The functionality was changed so that typing in a question redirected you to a search page on the real swedbank.se website. There was also a difference in functionality when visiting the cloned version on an iOS device compared to an Android device. On Android a banner showed up with a link to download the Swedbank app. On iOS no such banner was shown for the App Store version even though the original site had this functionality.
• S6 (Skatteverket) had some passed dates.
• S8 (Elgiganten) had a button with the text ”Visa produkt” (show product) which did not work. However you were able to click the image or the name of the product which would redirect you to the real product page on elgiganten.se. It was also not possible to use the function which compared products to each other. The site also showed some dates which had already passed (e.g. ”Back in stock”).
3.7.2 Legitimate websites
ID Type URL Description
L1 SM https://outlook.live.com Outlook front page L2 F https://internetbanken.privat.nordea.se/nsp/login Nordea login page
L3 F https://klarna.com Klarna front page
L4 E https://cdon.se Cdon.com front page
L5 SM https://i.reddit.com Reddit mobile front page
L6 E https://adlibris.com Adlibris front page
L7 F https://paypal.com PayPal front page
L8 F https://handelsbanken.se Handelsbanken front page
L9 E https://tradera.com Tradera front page
Table 2: The legitimate websites used in the study.
Figure 2: ablidris.com (S5) on the left, and the legitimate adlibris.com (L6) on the right shown in Safari on iOS 13. Some content had changed on the legitimate site since the clone was made.
3.8
Information given to participants
A short introduction of phishing was given to each participant. We explained that phishing is often performed by creating a fake website mimicking a legiti-mate one to retrieve login or bank details from targets.
Before the study began we presented the participant with a scenario: ”Imagine that you receive a message from a trusted person or company that asks you to
click on one of the following links. Imagine that you decide to click on the link to see if it is a legitimate website or a fraudulent copy of that website.” (Quote partly taken from WPW).
The participants were then directed to the navigation site. It was explained to the user that the websites would show up in a random order and that the same website could appear more than once. The statement that the same website could appear more than once was actually not true. The reason for this lie was that we included both spoofed and legitimate versions of some websites. If the user thought that a website would only appear once they might be able to use that information when the page appeared the second time. The participants were then told to identify the websites as legitimate or spoofed. We made it clear that they were allowed to use the web sites as any other web site, but we emphasized that the first page they arrived at was the one they were supposed to identify as legitimate or spoofed. This was clarified since our spoofed websites contained links to the legitimate version of each website.
The full manuscript was in Swedish and can be found in appendix A.
3.9
Pilot Study
Before performing the real study, we performed a pilot study on three volun-teering PhD Students studying at KTH Division of Network and Systems En-gineering. This pilot study was performed to test our methodology and receive feedback. With feedback from the pilot study we added a progress indicator on the study website, fixed a few problems with the Spoofed websites and changed the link styling so that visited links wouldn’t turn purple (the standard color of visited links).
4
Results
4.1
Participants
20 persons participated in the study. Of the participants 50% (10) were female and 50% were male. 25% (5) used Android and the rest used iOS. Safari was the most commonly used browser, used by 70% (14), followed by Chrome 25% (5) and Samsung Internet 5% (1). Participants received 1 point for each correctly identified site and the total scores for each participant ranged from 7 to 17. The average number of points for all participants was 11.75 (69% success rate) and the median number of correct guesses was 12 (71% success rate) (variance = 9.25, s.d = 3.04).
4.2
Correlation
The correlation coefficient was calculated using Pearson Product-Moment Cor-relation (N = 20) when comparing two sets of values. When comparing two populations t-test was used. A p-value < 0.05 was considered statistically sig-nificant for all tests.
4.2.1 Age
No significant correlation was found between score and age. r = -0.243, p = .30
4.2.2 Technical Proficiency
No significant correlation was found between score and technical proficiency. r = 0.408, p = .074
4.2.3 Sex
A statistically significant correlation was found between score and sex. The mean score was 10.3 for females (variance = 5.7, s.d = 2.39) and 13.9 for males (variance = 8.4, s.d = 2.90).
Levene’s test was performed to ensure the homogeneity of the groups, (F = 0.100, p = .755). The result of the t-test was t(18) = 2.602, p = .018. The result shows that there is a difference between the male and female groups.
4.2.4 Active smartphone use per day
No significant correlation was found between score and active smartphone use per day.
4.3
Strategies
Several strategies and combinations of strategies were identified during the in-terviews. A participant was considered to be a user of a specific strategy if they mentioned something regarding the specific strategy at some point while evalu-ating the sites or if we observed the participant use the strategy. For example, if a user mentioned the padlock icon in the address bar at some point they were considered using strategy 5 (”Using security indicators in the browser”). 4.3.1 Strategy 1: Evaluating the sites design
Participants using this strategy considered the look and aesthetics of the website such as font, logos and layout. 90% (18) of the participants mentioned the sites design at least once in our study as a factor when making a decision. Those who used this strategy at least once, had an average success rate of 68% on all pages, compared to 76% for the two participants that did not mention the sites design.
L2 Nordea and L5 Reddit are two examples where evaluating design lead to the wrong conclusion for legitimate sites. 25% (5) and 35% (7) respectively mistakenly classified the sites as spoofs with the design as motivation.
4.3.2 Strategy 2: Evaluating the sites functionality
This strategy involves evaluating the sites behavior when interacting with dif-ferent elements on the website such as links, buttons, forms and interactive animations. 95% (19) of our users evaluated the sites functionality at least once when trying to decide a sites legitimacy. This included participants click-ing links on the site. Those usclick-ing this strategy had an average success rate of 68% compared to 82% for the one participant who did not use this strategy at all.
An example of where this strategy was successful was when one participant noticed that the ”Visa Produkt” (show product) button did not work on the spoofed site S8 (Elgiganten) which helped him identify the site as spoofed. 4.3.3 Strategy 3: Evaluating the sites information
Participants using Strategy 3 considered the information displayed on the web-site. This includes wording, spelling, type of information and language. The difference from Strategy 1 is that this strategy considers what information is displayed instead of how it is displayed. 90% (18) of the participants used this strategy at least once during the interview. Those who used it had an average success rate of 70%, compared to 74% for the two participants who did not use it.
In some special cases users got a personalized version of a legitimate website. For example one user was logged in to Outlook on their phone and the link to
the legitimate Outlook front page redirected them to their inbox. The user used this as a basis for their decision that the website was legitimate. Some users also used the reverse argument, claiming that they should already be logged in on a specific site. Reddit opened up in an app (instead of browser window) for 3 participants and Tradera for 4 participants.
The evaluating information-strategy had the most potential on the spoofed sites with outdated content (described in section 3.7.1). Only one user found out-dated information and used it as a basis for labeling a website illegitimate. This was on S8 (Elgiganten) which described a weekly offer which was valid until a date that had already passed.
4.3.4 Strategy 4: Evaluating the sites URL
This strategy was used by 80% (16) of the participants. Participants using the strategy had an average success rate of 75% whereas participants not using it had a 44% success rate. Some users were inconsistent and mentioned the URL at some point but did not mention it for most of the sites. Two users only men-tioned the sites URL on one of the sites they evaluated. The maximum number of times the URL was mentioned was 16 which was done by two users. The average times mentioned (for those who utilized the strategy) was 9.2 (variance = 28.3, s.d = 5.32). A statistically significant correlation was found between participant scores and number of times they mentioned a sites URL (r = 0.80, p = .000028).
On S1 (fb.login.com.se) 9 participants (45%) used the URL as motivation to correctly identify the site as spoofed. Even though the strategy was successful in general it did not guarantee correct identification of spoofed websites. For example one participant mentioned the URL of S5 (swedbank-privat.se) and still considered it legitimate. Two participants inspected or mentioned the URL of S7 (outlook.com-secure.live) but still classified it as legitimate. Some participants were suspicious about legitimate sites in Swedish having .com and not .se as their TLD. For example, one participant incorrectly classified the legitimate klarna website (L3) as a spoof since it used .com instead of .se. On the legitimate site cdon.se (L4) the domain name was expected to be cdon.com by several participants.
The way the URL was displayed in the mobile browser window differed. One participant thought that the URL for S8 (Elgiganten) was giganten.online in-stead of elgiganten.online since the address bar was cropped, caused by a smaller screen size (shown in figure 3).
4.3.5 Strategy 5: Using security indicators in the browser
This strategy involves looking at and interpreting the browsers security warnings and information displayed in the address bar. This differed between Safari, Chrome and Samsung Internet, which was explained in section 2.2.
Figure 3: elgiganten.online in Chrome on a participants smartphone. Observe the partially hidden domain name.
The strategy was used at least once by 35% (7) of the participants. Those using it had a 80% average success rate identifying all websites compared to 63% average success rate for those who never used this strategy. One participant inspected certificates.
There were 5 legitimate sites (L2 Nordea, L4 CDON, L7 Paypal, L8 Handels-banken and L9 Tradera) using EV-certificates. These sites had their host name and the padlock icon displayed in green in Safari, but not in Chrome or Samsung Internet. Only 2 out of 14 participants (14%) using Safari mentioned the green padlock or green URL. One of those two participants mentioned the green URL on L2 (Nordea) but incorrectly classified the site as a spoof.
4.3.6 Strategy 6: Using a search engine to find the legitimate site One participant googled sites he was unsure about in a separate browser tab to determine if URL:s and website content was identical. That participant had a success rate of 100% on all sites, compared to the average of 67% success rate for the other 19 participants.
4.4
Combinations of strategies
All participants used a combination of at least 3 strategies. Participants who only relied on a combination of the Website content-strategies (Strategy 1, 2 and 3), were the same group of people who did not utilize Strategy 4 (URL check). They had a low 44% average success rate. The three worst performing participants in the study, with a success rate of 47%, 47% and 41% respectively were all a part of this group.
Multiple participants showed that they did not understand that website con-tent can be cloned quite easily. For example one participant said that ”It would be too hard to make a fake site like this” when mistakenly identifying S7 (outlook.com-secure.live) as legitimate.
Every participant who used strategy 5 (security indicators) also used strategy 4 (URL) and these participants had an average success rate of 80%. Those who used strategy 4 but not strategy 5 had an average success rate of 71%.
The three best performing participants all checked the URL (strategy 4) but only one of them mentioned security indicators (strategy 5). These 3 had a success rate of 100%, 89% and 89% respectively.
4.5
Website difficulty
The tables below show the success rate of correctly identifying all of our web-sites. The number shown in parentheses is the average confidence level (1-5) that participants reported for their correct/incorrect decision. The first table show results of all the spoofed websites and the second of all legitimate web-sites.
ID URL %Correct (avg certainty) %Incorrect (avg certainty) S5 https://swedbank-privat.se 50% (3.7) 50% (3.89) S7 https://outlook.com-secure.live 60% (2.67) 40% (4) S8 https://elgiganten.online 60% (3.58) 40% (3.88) S3 http://twitter.loginsecurity.online 65% (3.54) 35% (3.58) S4 https://ablidris.com/ 70% (4.69) 30% (3.83) S6 https://skattesverket.se 80% (4.38) 20% (4.5) S2 IP ADDRESS 80% (3.63) 20% (3.5) S1 https://fb.login.com.se 90% (3.5) 10% (4.0)
- All spoofed sites 69% (3.71) 31% (3.90)
Table 3: Study results for all spoofed websites.
ID URL %Correct (avg certainty) %Incorrect (avg certainty) L2 https://internetbanken.privat.nordea.se/nsp/login 35% (3.71) 65% (3.00) L5 https://i.reddit.com 50% (3.2) 50% (3.5) L4 https://cdon.se 55% (4.18) 45% (3.32) L3 https://klarna.com/ 65% (3.54) 35% (3.57) L1 https://outlook.live.com 75% (3.73) 25% (2.4) L6 https://adlibris.com/se 75% (4.13) 25% (2.6) L7 https://paypal.com 80% (3.63) 20% (3.25) L8 https://handelsbanken.se/ 90% (4.1) 10% (4) L9 http://tradera.com/se 95% (3.95) 5% (3)
- All legitimate sites 69% (3.79) 31% (3.21)
5
Discussion
5.1
Correlations
A statistically significant difference between the male and female groups per-formance was found using a t-test. However it is important to note that even though there was a significant difference in this group of 20 people no conclu-sions can be drawn with regards to the entire population with such a small amount of participants. There is no indication of a trend when looking at pre-vious studies. WPW and WPSW (who also had around 20 participants each) found no correlation when looking at sex and performance.
When looking at age, technical proficiency and active smartphone use per day no statistically significant correlations or differences were found. This corresponds to WPW and WPSW who also found no correlation when looking at age and technical proficiency.
5.2
Website evaluation
5.2.1 Spoofed sitesThe three spoofing sites which fooled the largest amount of participants were S5 (swedbank-privat.se, shown in figure 4), S7 (outlook.com-secure.live) and S8 (elgiganten.online). All of these sites used different URL spoofing techniques so there are no indications that one technique would be superior. What they do have in common is that all three URLs contain the full name (correctly spelled) of the organization they are imposing as. This could fool someone who does not understand domain names, even when looking at the URL.
The spoofed sites that misspelled the organization name, S4 (ablidris.com) and S6 (skattesverket.se) fooled a relatively small portion of participants, 30% and 20% respectively. Maybe these sites would have been more successful in a real world context. The participants who actually cared about the URL probably read it more thoroughly than they would have otherwise due to the context of the study.
S2 (Klarna with IP address URL) and S1 (fb.login.com.se) were the spoofed sites that fooled the smallest amount of people. This indicates that the URL of a website is important for a user when deciding if a website is legitimate. Only one person thought the fake Facebook login site was legitimate even though the content of the site looked exactly like the real mobile Facebook login page. The usage of fb instead of facebook in the URL is probably a part of the explanation. The login.com.se domain name also felt suspicious for many participants. The fact that the Klarna site with an IP address URL did not fool a lot of people is not surprising. The site also used http instead of https which meant that Safari displayed the text ”Not secure” to the left of the URL. Four people thought that the site was legitimate, none of those mentioned the IP address, indicating they either didn’t understand URLs or that they did not pay attention
Figure 4: swedbank-privat.se (S5) shown in Chrome on Android 9
to the address bar. It is noteworthy that this site actually fooled more people than the spoofed Facebook login site despite having an IP address URL. One explanation for that could be that the Facebook site (S1) was a login page, while the Klarna site (S2) was a front page.
5.2.2 Legitimate sites
The three legitimate sites that was the hardest for our participants to identify correctly were L2 (Nordea), L5 (Reddit mobile) and L4 (CDON). This is not very surprising. On Reddit and Nordea (shown in figure 5), two sites with an old-looking design, many people mistakenly classified the sites as spoofs with the design as motivation. Nordea was not designed for mobile screens and also
had a long URL with two sub domains. Cdon.se had a success rate of just 55% even though the URL was not suspicious and they even had an EV-certificate. The reason for the low success rate was that many of the participants expected the domain for cdon.se to be cdon.com. Cdon.com was previously the brand name of cdon that was used in commercials and logos. Some participants also expected cdon to have a different product range.
Tradera (L9), Handelsbanken (L8) and Paypal (L7) were the easiest sites for participants to identify as legitimate. Tradera is a marketplace where people buy and sell almost anything. A reason that so many people identified it as legitimate may be that the site had a lot of content, displaying auctions with the time remaining. All of the three sites have a clear URL containing only the organization name and a TLD which probably contributed to the legitimate impression. A modern design was also a common denominator for the three sites. Paypal had some flaws in the language used on the site. For example there was an incomplete sentence that read ”Ta reda p˚a varf¨or ¨over 255 miljoner.” (”Find out why over 255 million”.) Most participants (80%) successfully identified the site as legitimate anyways. This indicates that the actual information on the site might not be very important when determining legitimacy.
An interesting find regarding security indicators was that most Safari users that could see the green URL and padlock in the address bar (representing EV-certificates) did not know how to use this information. This indicates that EV-certificates and indicators for it has not been understood by most users. This has been observed in previous studies and it is the reason why many browsers have moved away from visually indicating EV-certificates.
The legitimate login page for Nordea (L2) was classified as a spoof by 65% of participants while the spoofed Outlook page (S7) was identified as a spoof by 60% of participants. This result indicates that the strategies used are not very effective and that a site with older looking design can be more suspicious looking than one with a spoofed domain name.
5.3
Participants strategies
It was noticed that almost all participants depended on a sites information, design and functionality when deciding if it was legitimate or spoofed. This seems like an obvious thing to do when visiting a new site, but it is often not a successful strategy in itself. This is indicated by the results where participants with a completely content-based strategy had a 44% success rate compared to 75% for those who mentioned the URL at least once.
Since there was always a 50% chance to answer correctly some users applying only content-based strategies still got a decent result by discovering what they thought was errors in the spoofed websites design and information. For example one user correctly identified S1 (fake Facebook login) as a spoof stating that the font in the Facebook logo was incorrect even though it was the real logo.
Figure 5: internetbanken.privat.nordea.se (L2) shown in Chrome on Android 9
Looking at the security indicators and evaluating the URL proved to be a more successful than relying solely on the website content. One interesting find was that some users applied the URL-strategy once and then never used it again. A very significant and strong correlation was found between the success rate and the number of times a user mentioned the URL during the study. Further-more, the users who not only mentioned the URL but also the browsers security indicators had an even better success rate at 80%.
It is likely that the users that did not look at the URL has a bad understanding of how web technologies work. Therefore it is a bit surprising that no signif-icant correlation was found between technical proficiency and performance or between daily smartphone usage and performance. It is possible that the five
questions that were asked to determine a participants technical proficiency were not accurate enough.
The only user with a 100% successful strategy googled sites and compared them to the sites he was directed to in the interview. This is the most fool proof strategy identified, and includes both URL and website content as factors. This strategy would also work when visiting a new site that you have not visited before, assuming it is a spoof and not a unique fraudulent website that the search engine has indexed. It is possible that other participants thought that they would not be allowed to use external sources to investigate the legitimacy of a website but the instructions given did not limit the methods participants could use in any way.
5.4
Smartphone browser improvements
One of the main flaws made by participants when deciding if a spoofed website was legitimate or spoofed was to not look at the URL at all and focusing solely on content. Another flaw was misinterpreting the URL. This can be considered an error on the users side, but browsers can also affect how the users interpret the URL.
One way for browsers to make this easier for users could be to display the URL more visibly or even double check if the user really want to move from one domain to another. This could of course be intrusive and make the browsing experience less comfortable.
A less intrusive approach could be to highlight the domain name somehow, like displaying it in a different color or higher opacity. This would make it easier to interpret the URL as the domain name is the most important part of the URL when deciding a websites legitimacy. This is already done today in the desktop version of the browser Firefox. However, this might not help users not aware of what a domain name or URL is at all.
To expand on this idea you could take it one step further and only show the domain name in the address bar, excluding the sub domains completely. Cur-rently Safari on iOS only shows the host name but as was seen with the spoofed Outlook page, the host name can be deceptive. If only the domain name was shown, the address bar for spoofed Outlook would have read com-secure.live instead of outlook.com-secure.live which probably would have raised suspicion among more participants. One could claim that it would be more difficult to know what website you are currently on if you could only see the domain name but it would still be possible to click the address bar and see the full URL at any time. This slight inconvenience could be worth it if there is a security benefit.
Another feature that might make it more difficult to interpret the domain name is that the address bar becomes hidden on Chrome when the user scrolls down on a web page. The design choice makes sense, since smartphones have less
screen space and as much as possible is used for showing content. Safari on the other hand makes the address bar smaller when the user scrolls down on a page but never hides it completely. The Safari-approach probably has security benefits.
5.5
Comparing with previous studies
The participants in this study had a better average success rate than the users in WPW and WPSW. The average success rate was 58%, 64% and 69% in WPW, WPSW and this study respectively. For spoofed sites only the success rates were 54%, 53% and 69%. For legitimate sites only the rates were 75%, 79% and 69%. There could be numerous reasons for these differences. Different web sites, different participants, chance and so on. It is very hard to tell if it has anything to do with a difference between smartphones and desktop computers. But something that was an obvious difference between the studies was that we let participants use their own mobile devices. This led to the fact that some websites opened in mobile applications and that some were already logged in to a legitimate website. This was used by some participants to decide whether a website was spoofed or legitimate.
In previous studies participants were considerably better at identifying legiti-mate websites correctly than spoofed websites. This was not the case in this study where the success rate was equal for both legitimate and spoofed web-sites. The participants in this study were more suspicious and more likely to consider websites fake overall. This could be due to cultural differences which has been observed in a previous study (Flores et al. 2015) made on phishing emails. WPW and WPSW were both performed in North America whereas this study was performed in Sweden. However, no conclusions can be drawn. It is worth noting that the same strategies that were identified among partici-pants in this study were also found in WPW and WPSW.
5.6
Future work
Our study lacks a few perspectives that should be considered in future studies. Firstly, it lacks a big population which could lead to more conclusions. Secondly, it lacks a variety of spoofing techniques unrelated to the actual URL of the sites. One spoofing technique that would be interesting to test is the inception bar, a spoofed address bar suggested by James H. Fisher7. The inception bar takes advantage of the fact that chrome hides the address bar when a user scrolls down, and replaces it with a fake address bar that is hard to distinguish from the real address bar.
Just as in WPW and WPSW the results of this study show no correlation between performance and technical proficiency. This is something we found 7H. Fisher, James. ”The inception bar: a new phishing method”, jameshfisher.com, April
surprising and it would be interesting to research this further. In future studies a different way to measure technical proficiency and a more precise definition of what is meant by technical proficiency would be beneficial. For example, a number of proficiency levels could be defined, and each participant could be classified as one of those levels. Another way would be to let participants self-evaluate their technical proficiency in several categories.
A future study could also focus on comparing different browser security indi-cators to make a more thorough analysis about which are more efficient. A flaw in our study was the large amount of variables and the small population, which made it hard to draw conclusions. Another interesting subject would be to compare results in a similar phishing study between different countries and cultures. Lastly it would be interesting to study if websites opened in embedded browsers through social media apps are harder to identify as spoofed/legitimate than in a dedicated smartphone browser.
5.7
Threats to validity
The biggest sources of error in the study was that a couple of web sites behaved differently on Safari and Chrome, and that all pages functionality was not cloned perfectly. The spoofed Klarna page (S2) had ”Download our app”-banner that only showed up on Safari. This was by their design and the same behaviour was seen on the legitimate Klarna page (L3). The same difference was noticed on swedbank-privat.se (S5) after the study had been conducted, but in that case the same behaviour was not seen on the legitimate site swedbank.se. This, combined with many other variables makes it difficult to draw any conclusions about what URL spoofing technique was the most successful.
One other problem with our study when it comes to comparing it to a real world context is that all participants was aware that the study was conducted and that their task was to identify legitimate and spoofed websites. This probably made the participants more careful and more observant. The results of the study indicate an upper bound of peoples ability to verify the legitimacy of a website. Another difference from phishing in a real world context is that participants in our study did not receive a phishing message with the link, which could have given clues about the legitimacy of the link that they received.
The technical proficiency-questions in our interview might have been too arbi-trary. They were slightly more focused on the web in general but perhaps not good enough to indicate the technical knowledge of a user. For example, a user who had changed firewall settings was considered as proficient as a user who had designed a web site.
5.8
Conclusion
The results show that almost all (>90%) of the participants evaluated a sites information, design and functionality when deciding if a site is legitimate or
spoofed. This was not a successful strategy when used on its own, leading to a mere 44% success rate for the few users who never mentioned the URL or security indicators. However, most participants (80%) evaluated the URL of a web site at least once during the interview. These people performed a lot better and had a 75% success rate of classifying sites correctly. Those who not only mentioned the URL but also mentioned the browsers security indicators performed better still with an 80% success rate.
The results indicate that smartphone users are not more susceptible to phishing than users on computers, at least when using a browser. About the same share of participants evaluated the URL at least once during our study compared to previous similar studies made on desktop computers. No significant correlation was found between scores and age or active time spent on smartphone per day. This was also the case in the previous similar studies WPW and WPSW. Technical proficiency was not found to correlate with performance either. This is surprising since knowledge of how URLs and the web works in general seemed to be important during the interviews, but at the same time it matches previous studies results. More users correctly identified spoofed and legitimate websites in this study compared to previous studies. One reason is probably that our participants used their own devices and that some websites opened in an app or were already logged in to their accounts.
A significant correlation was found between sex and susceptibility of phishing, where male participants scored better. This was not found in WPW or WPSW, but in ”Who Falls for Phish” (Sheng et al. 2010) a correlation was found between sex and susceptibility to click malicious links. However, clicking a link and incorrectly identifying a site as legitimate are two different things. Overall the quantitative results of the study could show some indications when compared to the previous similar studies but larger studies would need to be done in the future to draw more definitive conclusions.
Many users were confused about the meaning of padlock icon (HTTPS) and green text/padlock in Safari (EV-certificates). The domain name was sometimes hidden/partly hidden on participants smartphone screens, which could make identifying a spoofed website more difficult compared to a browser on a computer screen. Even when participants looked at the URL they were sometimes fooled by the spoofed websites. To combat this browsers could display the URL in a more clear way. One suggestion is for browsers to only show the domain name in the address bar. This could help in the cases where the sub domain is deceptive.
With regards to user behavior it was clear that the participants that put too much emphasis on the website content performed very poorly in the study. Those who mentioned security indicators performed a lot better even though many of them did not fully understand their meaning.
6
References
Alsharnouby, Mohamed, Furkan Alaca, and Sonia Chiasson (2015). “Why phish-ing still works: User strategies for combatphish-ing phishphish-ing attacks”. In: Interna-tional Journal of Human-Computer Studies 82, pp. 69–82.
APWG (2019a). Phishing Activity Trends Repor 3rd Quarter 2019. Tech. rep. Anti Phishing Working Group.
— (2019b). Phishing Activity Trends Repor 4th Quarter 2019. Tech. rep. Anti Phishing Working Group.
Canova, Gamze et al. (2015). “Learn to Spot Phishing URLs with the Android NoPhish App”. In: Information Security Education Across the Curriculum. Ed. by Matt Bishop, Natalia Miloslavskaya, and Marianthi Theocharidou. Cham: Springer International Publishing, pp. 87–100. isbn: 978-3-319-18500-2.
Das, Sanchari et al. (2019). “All About Phishing: Exploring User Research through a Systematic Literature Review”. In: arXiv preprint arXiv:1908.05897. Dhamija, Rachna, J Doug Tygar, and Marti Hearst (2006). “Why phishing
works”. In: Proceedings of the SIGCHI conference on Human Factors in computing systems, pp. 581–590.
Felt, Adrienne Porter and David Wagner (2011). Phishing on mobile devices. na.
Flores, Waldo Rocha et al. (2015). “Investigating personal determinants of phishing and the effect of national culture”. In: Information & Computer Security.
Goel, Diksha and Ankit Kumar Jain (2018). “Mobile phishing attacks and de-fence mechanisms: State of art and open research challenges”. In: Computers & Security 73, pp. 519–544.
Jagatic, Tom N et al. (2007). “Social phishing”. In: Communications of the ACM 50.10, pp. 94–100.
Oest, Adam et al. (2018). “Inside a phisher’s mind: Understanding the anti-phishing ecosystem through anti-phishing kit analysis”. In: 2018 APWG Sympo-sium on Electronic Crime Research (eCrime). IEEE, pp. 1–12.
Shahriar, Hossain et al. (2019). “Mobile anti-phishing: Approaches and chal-lenges”. In: Information Security Journal: A Global Perspective 28.6, pp. 178– 193.
Sheng, Steve et al. (2010). “Who falls for phish? A demographic analysis of phishing susceptibility and effectiveness of interventions”. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 373– 382.
Sobey, Jennifer, PC Van Oorschot, and Andrew S Patrick (2009). “Browser In-terfaces and EV-SSL Certificates: Confusion, Inconsistencies and HCI Chal-lenges”. In: Carleton University School of Computer Science, Canada, Tech-nical Report TR-09-02 15.
A
Manuscript (Swedish)
Fyll i enk¨aten
Inledning: Phishing ¨ar en typ av bedr¨ageri som anv¨ands f¨or att komma ¨over k¨ansliga uppgifter, som inloggningsuppgifter och kreditkortsinformation. Phish-ing kan g˚a ut p˚a att lura anv¨andaren att den bes¨oker en p˚alitlig hemsida, som egentligen ¨ar en kopia (spoof) som beg¨ar k¨ansliga uppgifter.
Var god s¨att mobilen p˚a St¨or Ej om du vill. Dela sedan sk¨arm med oss i Zoom. Kopiera l¨anken vi skickar i Zoom-chatten. ¨Oppna sedan ett Inkognito-f¨onster i din webbl¨asare och g˚a in p˚a l¨anken du kopierade. Klicka inte p˚a n˚agot ¨
annu.
Kom ih˚ag att inte ladda om eller st¨anga ned denna sida under studiens g˚ang. Vi vill att du t¨anker dig att du tar emot ett meddelande fr˚an en person/f¨oretag du litar p˚a som ber dig klicka p˚a en av l¨ankarna som kommer dyka upp. T¨ank dig att du best¨amt dig f¨or att klicka p˚a l¨anken f¨or att se om sidan ¨ar legitim eller ett bedr¨agerif¨ors¨ok. L¨anken kommer ¨oppnas i en ny flik som du kan st¨anga n¨ar du ska ¨oppna n¨asta.
17 l¨ankar kommer att presenteras i slumpm¨assig ordning p˚a sidan du ser framf¨or dig. Samma l¨ank kan f¨orekomma mer ¨an en g˚ang.
N¨ar du ¨ar inne p˚a en sida f˚ar du klicka runt p˚a sidan och p˚a l¨ankar p˚a samma s¨att som du skulle anv¨ant en hemsida i vanliga fall, men det ¨ar den f¨orsta sidan du ska avg¨ora om den ¨ar legitim eller ej.
T¨ank g¨arna h¨ogt s˚a att vi vet hur du t¨anker n¨ar du ska avg¨ora sidans legitimitet. N¨ar du har best¨amt dig, s¨ag vad du tror (legitim eller ej) och s¨ag hur s¨aker du ¨
ar p˚a ditt beslut p˚a en skala 1-5. St¨ang sedan ned sidan och g˚a tillbaka till navigationssidan f¨or att forts¨atta till n¨asta l¨ank.
Vi kommer att s¨aga till om vi vill att du ska utveckla n˚agot men i ¨ovrigt kommer vi att vara ganska tysta.
¨
Ar det okej om vi spelar in Zoom-m¨otet? Det ¨ar inte ett krav men underl¨attar f¨or oss om vi beh¨over g˚a tillbaka och dubbelkolla n˚agot.
