User Experience Design
on Online Newspapers
A study of how the user experience design of
self-financed Swedish online newspaper is
PAPER WITHIN Informatics
AUTHOR: Mihir Chouhan, Frida Jönsson TUTOR: He Tan
JÖNKÖPING 02 2019
This exam work has been carried out at the School of Engineering in Jönköping in the subject area informatics. The work is a part of the three-year Bachelor of Science in Engineering programme. The authors take full responsibility for opinions, conclusions and findings presented.
Examiner: Bruce Ferwerda Supervisor: He Tan Scope: 15 credits Date: 01-04-2019
Abstract
In this paper the study has audited the user experience design of the two self-financed Swedish online newspaper Aftonbladet and Jnytt. The eleven participants were in the ages of eighteen and twenty-six and had no earlier expertise in web development, due to one exception where a participant had taken a short web design course during a couple of weeks. The importance of having participants without earlier knowledge in web development was to make sure the user experience tested on the online newspaper were applicable to people who might use the site regularly without knowing the basics of building a website. The purpose of the study were to investigate if online newspaper had a good structured and whether or not they create a good user experience. Earlier studies of user experience design on online newspaper is limited and therefore this study will contribute with more research within the field. It is not accurate to generalize this study for all self-financed Swedish online newspapers since only two have been compared due to the lack of time, which is a limitation in this study. The study was executed with a usability test where participants were asked to fulfill tasks on the two sites and say what they were thinking out loud, supported with a short questionnaire and a semi-structured interview. The questionnaire was to make sure the participants were potential end-users and the semi-structured interview gave more clarity to the test as well as gave an insight of their first impression and preferences. The study gave a result showing that neither the sites had fully potential user experience design in focus and could be improved. However, Jnytt was the preferred online newspaper over Aftonbladet in terms of being less cluttered, more thoughtful placements of ads, a working search function and better color scheme. Some new findings were made on Aftonbladet where the search function did not work as planned, which annoyed most participants who came across the issue. When asked if the participant would consider visiting the online newspaper more often if there would be major changes to online newspaper over all, some of them said they thought it would as the experience would be more pleasant while some participant had other reasons to not visit the page, such as their lack of interests for news, and therefore do not believe that the web design and a better experience would affect it.
Sammanfattning
I den här uppsatsen har studien granskat användarupplevelse av de två självfinansierade svenska
nättidningarna Aftonbladet och Jnytt. De elva deltagarna var i åldrarna från arton till tjugosex och hade inga tidigare erfarenheter inom webbutveckling, förutom ett undantag där en deltagare hade tagit en kort webbdesignkurs på några veckor. Betydelsen av att ha deltagare utan tidigare kunskaper i webbutveckling är för att vara säker så användarupplevelsen testad på nättidningarna var applicerad för personer som kanske använder webbplatsen regelbundet utan att veta grunderna för att bygga en webbplats. Syftet med studien var att undersöka om online nättidningarna hade en bra struktur och huruvida de skapade en bra
användarupplevelse eller inte. Tidigare studier av användarupplevelse på nättidningar är begränsade och därför kommer denna studie att bidra med mer forskning inom området. Det är inte korrekt att generalisera denna studie för alla självfinansierade svenska nättidningar eftersom endast två har jämförts på grund av brist på tid, vilket är en begränsning i denna uppsats. Studien genomfördes av ett användbarhetstest där deltagarna ombads att genomföra vissa uppgifterna på de två hemsidorna och säga vad de tänkte högt. Ett frågeformulär och en semistrukturerad intervju gjordes tillsammans med användbarhetstest, där
frågeformuläret var att se till att deltagaren var potentiella slutanvändare och den semistrukturerade intervjun gav mer klarhet i användbarhetstest samt gav insikt om deltagarnas första intryck och preferenser. Studien gav resultat som visade att nättidningarna inte hade fullt potentiell användarupplevelse i fokus och kan förbättras. Jnytt var den föredragna nättidningen framför Aftonbladet i form av mindre rörig, mer vältänkt placeringar av annonser, en bättre sökfunktion och ett bättre färgtema. Några nya resultat gjordes på Aftonbladet där sökfunktionen inte fungerade som planerat vilket irriterade de flesta deltagare som stötte på problemet. När deltagaren blev tillfrågade om de skulle kunna överväga att läsa nättidningar oftare i samband med att större förändringar gällande design och upplägg för nättidningar skulle ske, sa några att de trodde att det skulle läsa nättidningar mer eftersom upplevelsen skulle vara trevligare. Några andra deltagare hade andra skäl att inte läsa tidningar som till exempel deras brist på intresse för nyheter, och tror därför inte att webbdesignen och en bättre upplevelse skulle påverka.
Keywords
Online newspaper, UX-design, User experience design, Web design, Usability test.
Table of Content
Abstract 2 Sammanfattning 3 Keywords 4 Table of Content 5 1. Introduction 7 1.1 Background 71.2 Purpose and research questions 8
1.3 Delimitations 8
1.4 Outline 8
2. Theoretical background 10
2.1 The History of Newspapers 10
2.2 The Internet Revolution 10
2.3 Defining a Good Web Design 11
2.4 Trends in Web Design 11
2.5 Definition of UX-design 12
2.6 The Online Newspapers in the Study 12
2.7 Related Work 14
3. Method and implementation 16
3.1 Data Gathering 17
3.1.1 Questionnaire 17
3.1.2 Usability test 17
3.1.3 Interview 18
4. Findings and analysis 19
4.1 Findings of collected data 19
4.1.1 Participants 19
4.1.2 Aftonbladet 20
4.1.3 Jnytt 23
4.1.4 Interview Findings 24
4.2 Analysis of findings 24
5. Discussion and conclusions 27
5.1. Discussion of method 27 5.2. Discussion of findings 28 5.3. Conclusions 29 5.4 Further studies 30 6. References 31 8. Appendices 33 Appendix 1 34
Appendix 3 37 Appendix 4 38 Appendix 5 41 Appendix 6 43 Appendix 7 44 Appendix 8 45 Appendix 9 46 Appendix 10 49 Appendix 11 50 Appendix 12 51 Appendix 13 52 Appendix 14 53
1.
Introduction
Online newspapers are today quick to become the preferred medium of reading the news as opposed to the printed newspaper. According to Limbäck & Lundqvist (2011), the participants in their study experience the readability of online newspapers decreasing due to all the ads surrounding most sites and they also feel distracted or annoyed. If online newspapers are designed with the user in focus will be investigated in this report through use of previous research, literature, usability tests and interviews. The work has been carried out at Jönköping University, School of Engineering as part of a bachelor’s degree.
1.1
Background
It was in the beginning of the seventeenth century when the newspaper started to get published around Europe. People could now get information about events going on around the world on a regular basis in a printed publication. As the newspaper industry started to grow, the development of printing techniques improved and while the printing techniques got more effective and higher in quality along with a better choice of news, the attractiveness of newspaper among the community grew as well (Weber, 2006).
When the internet came to be, it revolutionized how people received news as it used to be obtained through printed newspapers. As platforms with internet access has increased, in the sense of smartphones, tablets and computers, the manner in which the information is presented has changed. News are today presented across multiple platforms, ranging from social media channels to televised programs to websites. As the digital revolution progresses, the need to adapt to this new environment online is necessary in order to stay relevant according to Nasir (2012). In their study it was concluded that information technology’s role in different print and advertising media is essential to the survival of media. The digital revolution also contributed in making online newspapers the preferred medium to the printed news, especially among the younger audiences (Kit & Teng, 2015).
The web design of online newspapers differs from the design of other websites in the sense that they follow the flow and look of a printed newspaper, where headings compete for attention, ads are taking up space and pop-up windows hide content. While a lot of websites with a modern web design make it easy for users to understand and read the content, most online newspapers have a web design that does not follow this modern design. This is partly because interesting and timely content are more at focus on online newspaper, as reported by Aranyi & van Schaik (2015). In their research it was also concluded that a high-quality experience of an online newspaper was correlated to how well the aesthetics of respective online newspaper. In order to have the information presented on webpages organized and structured properly, user experience design, also called UX-design needs to be incorporated. To date there is not much research on how online newspapers are designed from a user centered perspective.
UX-design is a abbreviation of user experience design, where the user is in focus of the design process, in this case of a website’s development. The goal of UX-design is to create a site or product that is user-friendly with a good accessibility and high usability, which in turn will lead to a greater experience for the user and higher satisfaction (Hassenzahl & Tractinsky, 2006).
Not only does UX-design touch on the user experience but also on the interaction design. Interaction design is about making sure it is clear for the user how, in this case, the website is working and what options the user have. An example is how the user navigates throughout a website (Buxton, 2007).
In this report, online newspaper is the term that will be used throughout to describe a website that focuses on publishing recent and relevant news. Although online newspaper website as well as news website and news
site are synonyms with the same meaning.
1.2
Purpose and research questions
The purpose of this study is to gain knowledge of how different Swedish online newspaper are structured and how the user experience is. This will be done by studying people’s interactions with different online newspapers and analyze their interactions, paths, thoughts and feelings. The decision to focus on Swedish online newspapers was made due to the lack of existing research on this area in the country.
The study will aim to answer the following question:
How is the user experience design of self-financed Swedish online newspapers?
The findings of this research will add to the little research on the user experience of online newspapers, in particular Swedish ones. The study may end up useful for further development of existing or new online newspapers in hope to increase people’s interest to visit online newspapers more frequently.
1.3
Delimitations
Due to the limited time frame of this research, the study will focus on two online newspapers, Aftonbladet and Jnytt. The first mention is one of Sweden's most popular online newspapers and Jnytt is a local one in the city of Jönköping. The reason to not compare the two most popular online newspaper in Sweden was to see if the smaller company might do something different compared to the bigger company. In order for the results to be unbiased, both are online newspapers financed by the company and need to make a profit. The choice to study two online newspapers will result in that the findings of the two online newspapers will be generalized to almost all other Swedish online newspapers. For further studies there is an option to analyze more websites for a more specific result and might be of interest to investigate in different areas for example online newspapers financed by the government.
The study will be tied to a limited group of people due to lack of time. This study will be using eleven participants in the age between eighteen and twenty-six. The choice of age is relevant as they are believed to be the generation obtaining news online. The usability test as well as the interview has been conducted in Swedish and translated by us in the study.
1.4
Outline
Theoretical background
This part will include information about what areas this report will cover such as user experience design, web design, and the digitization of newspapers.
Method and Implementation
In this part the chosen methods will be explained and described. Previous work and literature that has aided in the making of the method will be mentioned as well. How the usability tests and interviews will be carried out and how the data has been collected and processed will be presented here.
Findings and analysis
The results from the method will be reported and analyzed in this part.
Discussions and Conclusions
In this part the findings and method will be discussed in regard to their strengths and weaknesses and will finally present a conclusion of the study.
2.
Theoretical background
2.1 The History of Newspapers
Around the seventh century, it is believed that China and Japan had invented one of the first printing techniques. The two countries were carving ideograms into wooden blocks and made it possible to print a whole page, multiple times. This technique is called block printing (Briggs & Burke, 2009).
It was not until 1450 that printing took off in Europe. It was probably Johannes Gutenberg of Mainz who invented the printing press. He used movable metal type which made the printing technique more flexible than block printing. With movable metal type, there were no need to carve out a whole page on the same block but could instead re-use the letters for another page by rearranging them. Although, China had some struggle with the printing press since they had many more ideograms than Europe had letters, so the shift in China went slowly. However, people in Europe who had been doing everything by hand was now amazed over the amount of copies that could be created in the short amount of time. A revolutionary invention which gave birth to the process of publishing (Briggs & Burke, 2009).
As the time passes, printing techniques got more advanced and it spread rapidly across Europe. By the seventeenth century, people were able to publish the first modern newspapers and people could now regularly get information of events around the world in a printed publication. It made it possible for ordinary people to stay updated about the news and made communication in Europe easier. (Briggs & Burke, 2009)
A result of the growing newspaper industry was the further development of printing techniques. The quality of newspapers grew as there were better choices of news which in turn increased the attractiveness of the newspaper (Weber, 2006).
2.2 The Internet Revolution
The Internet grew its roots early in the twentieth century, but it was not until 1995 that the internet grew into becoming what it is today. Although, the internet was created much earlier, dated far back to the 1960s. Back then, the internet was shared between a few universities and research departments where the users were primarily engineers and physicists. In 1989, the term World Wide Web was coined by Tim Berners-Lee, who suggested the idea of information from different computers being shared between one another (Briggs & Burke, 2009).
In 1993 the World Wide Web browser Mosaic was launched which displayed images and text together. Created by Marc Andreessen, it was one of the first web browsers with a graphic user interface that allowed users to click on icons, menus and windows (Graphical User Interface, n.d.). This resulted in an exponential growth in traffic on the web. The Internet would allow people across the world to inform and educate each other on a common global platform (Briggs & Burke 2009, p. 266). News could spread faster, as the time for an event to occur to having it printed and delivered to people was shortened.
Between 1993 and 1994 the design of websites was in the hands of those with knowledge of the programming language HTML (Hypertext Markup Language). Although the process of creating and
editing web pages included an excessive amount steps. When the program AOLPress was released this process was reduced to less steps as the program allowed editing of the web pages while browsing (Moschovitis, Poole, Schuyler & Senft, 1999).
In the beginning of the twenty first century, newspapers started to implement their content onto the Web. Smaller newspapers faced a lot of competition already from newspapers that were for free and relied on advertisement (Briggs & Burke 2009, p. 299). The added competition of news published online forced these businesses to either adapt to the new environment or go bankrupt.
2.3 Defining a Good Web Design
In the process of designing a website, numerous guidelines exist in order to aid in creating a successful product. The process usually starts with user research, where the intended target audience of the website is studied and analyzed for the purpose of creating a website that best fits its users’ needs and goals. The two aspects used to determine whether a web design is good or bad usually comes down to usability and aesthetics. These two aspects need to be combined in harmony as one without the other will result in a bad web design. A website that looks good but lacks functionality might attracts users but will not keep them there as their goals cannot be completed. On the flipside, a website that works as intended but lacks a pleasing design will not keep their users there for long as they will not enjoy their visit (Beaird & George, 2014).
Another important aspect of a web design is the theme. Consistency is key to avoid confusing the users. Elements such as colors, navigation menus and text should be consistent throughout all pages of the website, so users recognize the pages they are on belong to the same website. The design of websites usually also resembles their real life counterparts. Newspapers for example, make great use of hierarchy as they group together titles and images in different sizes depending on their relevance. This structure can be seen online as well (Krug, 2014). Furthermore, clickable links on a website should clearly convey that they are such. This is one of many rules to follow to create proper web design.
The navigation of the website should be easy to find but not take attention away from the content. The type of navigation may vary but a recent trend has shown that a horizontal navigation with a drop-down menu has become more occurring as opposed to the vertical left navigation, also known as the sidebar navigation (Comeaux, 2016). The content itself should be visible to the users at a quick glance as people’s patience and attention span online is minimal. Though it is not necessary, many websites have a footer with contact information and copyrights. There are different types of footers, but the common denominator is that it is always located at the bottom of the page. Another important element of a web design is white space, or negative space. This is when there is a significant amount of space between images, sections and text in order to create a pleasant environment. A website where all content has no white space would give users a feeling of being in a crowded room (Beaird & George, 2014).
2.4 Trends in Web Design
As mentioned before, a new trend is the use of horizontal navigation bars. Other trends in recent years focus on the “above the fold” part of a website. Above the fold is a term that refers to all the content displayed on a newspaper when it is folded in half. In website lingo, this would refer to all content that is visible at the top of the page without scrolling. Many websites have come to use this space to show full screen backgrounds
and in some cases video backgrounds. Another trend is the use of masonry layout, where the content is vertically arranged in piles next to each other (Beaird & George, 2014). This layout can be seen on Pinterest’s homepage.
2.5 Definition of UX-design
UX-design is an abbreviation for User Experience Design and can be applied to any product or service. In this report, the focus is on UX-design for websites. (Nichols & Chesnut, 2014). Further on, the word website will be used when talking about UX-design, not to forget that UX-design, as mentioned above, can be applied to any product or service.
UX-design is about creating a good experience for the user in terms of the website being useful, usable and desirable. It is necessary that the website, as well as all of its functions, are being useful for the user. The user should not have to question why something is there or why the user are able to do something that is not relatable to the website’s goal. The website also needs to be easy to use and clear enough, so the users know what they are supposed to do and not have to figuring out how everything functions. The website also needs to engage the user and give the user a feeling of satisfaction or enjoyment. If any of those three categories, useful, usable, and desirable, is missing then the user is likely to choose another website with the same goal (Nichols & Chesnut, 2014).
A good UX-design can lead to multiple positive things such as building up a trust for the user. It can lead to a stronger relationship between the user and the brand which in turn can make the user use the website more and recommend it to others. When a website feels clear and consistent, the user know that the system works fast and efficient and will have a positive attitude next time the user needs the site. A good UX-design can also keep the user at the website for a longer time as the user are enjoying the experience. Then it is more likely the user will visit the site again with a good attitude and impression towards the brand (Nichols & Chesnut, 2014).
2.6 The Online Newspapers in the Study
Jnytt is a local Swedish online newspaper, mainly focusing on news in the city of Jönköping. The online newspaper averages 60 000 visitors every day. Jnytt exists solely on the web and has been there since 2006. (Jnytt, n.d.) Jnytt has an online presence on social media and are also visible on the local buses (Twitter, 2019). They link news from their website to their Facebook and Twitter while their Instagram is less informatic (Instagram, 2019). See Figure 1 for an overlook of the online newspaper.
Figure 1. Print screen of the online newspaper Jnytt first page. (Jnytt, 2019)
Aftonbladet is one of Sweden’s most visited online newspaper (Westlund, 2018). It started out as a newspaper and launched its website in the mid-1990s during the expansion of the internet. Shortly thereafter the newspaper became the biggest as well (Aftonbladet, 2007). Aftonbladet post news on their website as well as their social media. Their content can be seen on Instagram, Facebook and Twitter (Facebook & Instagram & Twitter, 2019). See figure 2 for an overlook of the online newspaper.
Figure 2. Print screen of the online newspaper Aftonbladet first page. (Aftonbladet, 2019).
The results of this study will give further knowledge of how the UX design is for online newspapers, in particular self-financed ones in Sweden. The input given from users in the semi-structured interviews will be used to determine if and how one’s experience on online newspapers can be improved.
2.7 Related Work
Aranyi & van Schaik did a study in 2016 to test a model of the UX for certain online newspapers where users’ satisfaction and trust were in focus. In their study two online newspapers where used, British Broadcasting Corporation (BBC) and New Zealand Herald (NZH). All participants were familiar with BBC however none of the participants knew of the NZH website. One of their conclusions were that satisfaction with a website could be correlated to the reputation of the news provider. Another one of the findings was that a hands-on experience would provide more information about the appearance, and less
about the usability, of an online newspaper. This conclusion was based on that the users browse the websites without specific tasks (Aranyi & van Schaik, 2016).
Limbäck & Lundquist (2011) were studying ads on online newspaper for different platforms. In their study they concluded that ads annoyed participants and decreased the readability on the online newspaper. Ads also made first pages harder to scan, which can be an important function when a user’s goal is to scan for news they want to read more about. The participant in their study preferred to read news on the online newspaper which looked more clean and felt less frustrating, in terms of less ads, rather than the site with more ads claiming for attention. It was also preferred to read the news on an iPad rather than on a PC, since the ads were less visible on an iPad. Limbäck and Lundquist concluded the less ads, the more preferred by the user and the higher readability and scalability gets.
3.
Method and implementation
A case study has been conducted in order to answer the research question stated in the chapter Purpose earlier in this report. One quantitative method and two qualitative methods were used to analyze the UX-design of self-financed Swedish online newspapers. It was preferred to implement qualitative methods in order to gather a rich amount of data and possibly encounter new findings, rather than a quantitative method that would give a less detailed result. A possible drawback of a qualitative method is the limited amount of participants used in the methods due to the time and effort it takes to implement the qualitative method, which makes it difficult to generalize the findings. The chosen methods were a short questionnaire (See appendix 1), a usability test where participants were asked to fulfill a certain number of tasks on the online newspaper Aftonbladet and Jnytt, followed up with a short semi-structured interview (See appendix 14) to get out the most of the research.
The choice to investigate self-financed online newspaper was made in order to make the study more accurate. A company financed by the government does not need to deal with advertisement and commercials which can result in a website looking clearer and structure compare to self-financed online newspapers which rely on advertisement in order to make a profit. The study is investigating one national online newspaper and one local online newspaper. There is an interest to compare those two types of online newspapers to possible find different structures of the websites and in turn to get an understanding of what users prefer and why. Aftonbladet is Sweden's most popular online newspaper while Jnytt is the most popular local one in Jönköping, where the study has been conducted.
The participants in the study were males and females between the ages of eighteen and twenty-six with no expertise within web development (see appendix 2). This group of participants are believed to be the generation which are most likely to read news online. Since the participants was meant to represent the majority of the population, the participants were also chosen based on their general knowledge of web design and development. It was preferred to have people with little or no expertise within this field as the usability test should determine if the websites’ design is apparent for all audiences, as they should be even for those who lack knowledge in how websites are structured.
In order to make sure the participants were within the target group of the study, they were asked to fill out a short questionnaire about their expertise in using the internet (Unger & Chandler 2012, 303), usage of online newspapers and their knowledge of the two websites in study. The usability test consisted of five tasks for respective online newspaper, ten in total (see appendix 3). The first task was to browse the website for a minute and talk out loud what their first impressions were. In the second task the participants were asked to find a specific news that had occurred. There were numerous ways of doing this including scrolling the page or using the search bar. The third task was to find all news related to sports. The purpose of these tasks was to investigate how easily the participants could find specific information that they were looking for (Rubin & Chisnell 2008, 70). The fourth task would be to return to the homepage, while standing in the sports section. This purpose of this task, “to find out how easily users can return to the homepage”, was also mentioned in Rubin & Chisnell’s book, Handbook of usability testing: how to plan, design and conduct effective tests. There was more than one way to achieve this. The final task on each website was to find all news related to one specific category, which could only be done through the closed menu. For Aftonbladet the category was technology while for Jnytt it was finance. This task would answer the question “How well do users understand symbols and icons?” (Rubin & Chisnell, 2008). Half of the participants performed all tasks on Aftonbladet first and then proceeded with all tasks on Jnytt while the
other participants started with all tasks on Jnytt before proceeding with all tasks on Aftonbladet. Six participants started the usability test at the online newspaper Aftonbladet, while the other five participants started out with Jnytt. Throughout this test the participants were asked to think out loud about everything that came to mind when completing the tasks. What they said as well as their reactions, if there were any that stood out, would be noted both for data and in order to possibly ask questions in the semi-structured interview that took place after the usability test.
The semi-structured interview consisted of four open-ended questions (see appendix 14) with room for follow ups, depending on what the participants answered or said during the usability test. In the interview, participants would answer which of the two online newspapers they preferred and why. This would give a general idea of the participants’ thoughts and attitude towards the online newspaper they liked more. The second and third question would then ask the participants what they thought was positive and negative about respective online newspaper. Depending on their answers and on what they said during the usability test, follow-up questions would be asked to gain understanding of why they thought something was positive or negative. The last question would ask the participants if a redesigned online newspaper would make them more prone to visit online newspapers on a regular basis. Their answer was followed up by a why or why not question in order to get an elaborated response. This would give an idea if the UX-design on an online newspaper could affect users attitude and interest to read news online.
3.1 Data Gathering
The data gathered from the questionnaire, usability tests and interviews were collected and structured in different tables. The participants were informed that all recordings would be deleted after the material being analyzed and how the data would be presented anonymously in the study. The usability test as well as the interview has been conducted in Swedish and translated by the researchers in the study.
3.1.1 Questionnaire
The data from the questionnaire were first written down by hand on paper, by the participant, and later on transferred into a digital document in form of a table. By having all the participants’ information gathered in one place made it easier to analyze the relation between the participants’ pre-knowledge and their results from the usability test.
3.1.2 Usability test
A screen-recording software program called QuickTime was used to record the screen along with the audio, while the participants were interacting with the different online newspapers in the usability test. The recordings was beneficial when analyzing the material, it made it possible to back and follow the
participants’ path through the tasks as well their outspoken comments and thoughts. The data gathering method also eliminated the risk of missing important information and decisions made by the participant during the usability test.
One of the researchers conducting this study took notes and observed the participant going through the test while the other researcher, also conducting this study, was guiding the participant throughout the test and presenting the tasks. The notes were mainly used to ask follow-up questions after the test were done, in order to understand if something the participant said or did was unclear. It also eases the process of analysing data as the observing researcher jotted down interesting moments for further analysis. After analyzing the data from a participant, the participants outspoken comments as well as the result of each
tasks they performed, were put in a table in order to get a greater overview and analyze relations between tasks and participants.
3.1.3 Interview
At the end of the session, the participants were asked some follow-up questions in a semi-structured interview. These interviews were recorded and then transcribed. The first question goes as follows “Which online newspaper do you prefer?” The second question asked is “Was there something positive with Aftonbladet?” The third question will be “Was there something negative with Aftonbladet?” The fourth question is “Was there something positive with Jnytt?” The fifth question asked is “Was there something negative with Jnytt?” The final question goes “If the design was to be changed, would you visit online
4.
Findings and analysis
4.1 Findings of collected data
4.1.1 Participants
The following findings (Figure 3) were obtained through the questionnaires, filled in by the participants themselves. The eleven participants in the study were between the ages of eighteen and twenty-six.
Figure 3. A diagram showing the age of the participants in the study.
All the participants used the Internet at home at least once a day. None of them had any previous
knowledge of web development with the exception of one participant who had taken a short course in the programming language JavaScript a few years earlier. Out of all participants, five reported to visiting online newspapers a few times a month, another five reported to visiting online newspapers several times a week, and one participant reported to not visiting online newspapers on a regular basis.
Figure 4. A diagram showing how often the participants visited online newspapers.
All participants were familiar with Aftonbladet’s website whereas nine out of eleven were familiar with Jnytt’s website. However, one of the participants who reported to be familiar with Aftonbladet’s website mentioned only being familiar with the smartphone app of the online newspaper (See appendix 2).
Figure 5. A diagram showing how many participants that were or were not familiar with the online newspapers in study.
4.1.2 Aftonbladet
The first task given to the participant on the online newspaper Aftonbladet, were to scroll down the site and say out loud what they were thinking. Each participant was given approximately sixty seconds each. The
task would show the participants first impression and attitude towards the online newspapers. Six out of eleven participant scrolled down the page slowly in a steady pace. Four of those six participant started with spending some time on the top part of the site without scrolling, and later on sometimes scrolled up a bit in order to see what they had passed by too fast. The other five participants scrolled down really fast and three participants of those five scrolled up a bit too see if they had missed something. Two participants stopped at the bigger news before continuing scrolling downwards (see appendix 4).
While executing the first task, nine out of eleven participants said their first impression of the site was cluttered with a lot going on, while two other participants did not comment the amount of content. Seven participants also comment on the commercials and ads being too much and two of those felt they had too much options to click on. Three of the participants thought the titles were big and written in a manner to attract more people rather than describing the article. Two participants liked the placement of “Senaste nytt” (Recent news) while two others did not feel the site’s content to be organized well enough. One participant mentioned a feeling of the site being chaotic while another participant thought the text were clear. One participant said they noticed how colors matter for a website in terms of dividing up parts such as the sports-part had a pink background in the feed rather than a white as for the other articles. One
participant said the structure of the website reminds of a printed newspaper with its ads and layout (see appendix 4).
For the second task, the participants were asked to find a specific news on the online newspaper. The news different for some of the participants as the test were conducted during several days and the online newspaper is constantly updating their content. To be as accurate as possible, the news were appearing on the same place at the site, a bit down in the right column. The study will show how successful the users are in finding the information they are looking for. Eight participants cancelled the tasks and three found the article. Six participants out of eleven started with the search function where two participants found the news immediately. The other four of the six participants could not find it after several attempts with the search function and went to one of the categories placed in the navigation bar and started scrolling down the chosen page. Only one participant found the article and the other four canceled the task. Two out of eleven participants started with scrolling down the homepage where one passed by the article without noticed and canceled the tasks and the other one continued by searching in the URL which took the participant to Google where they opened the wrong article. One of the eleven participants started with scanning the navigation bar for a suiting category and click it. The participant scrolled down and then tried five other categories without any success before cancelling the task. The last participant out of the eleven used the CTRL+F function, which gives the user a search function from the web browser to search for any chosen words on respective page. The participant did not find it due to a misspell and tried the sites own search function and scrolling the homepage before cancelling the task (see appendix 5).
During the test four participants said they thought the article would be under a specific category but when they clicked it, it was not what they expected to end up. Three participants said the first thing they would do when looking for a specific new would be to search, and the search function did not work the way they thought which felt strange and bad. One participant said it takes too long to find the article they were looking for while another participant said it felt like it would never been found. One participant said it was annoying to try to find it and one said it was easy to find the search button (see appendix 5).
articles. All of the participants found the category in the navigation bar immediately of which one participant chose to first click the logo and then click the category in the navigation bar. Two of the participants who started out with the tasks on Jnytt, said it was in the same location and looked the same as Jnytt (see appendix 6).
Now that the participants were standing in the sports page, they were asked to go to the homepage. The reason for this task was a change in the layout when visiting the sport page on Aftonbladet. The logo you could press to go back to the homepage was now replaced with another logo connected to the sports page, meaning if it was clicked, the user would not been taken back to the homepage. The task would give insight if this would cause a problem or not, and how easy it is for the user to return to the homepage. One of the eleven participants went to the URL, took away the last part (/sportbladet) and hit enter which took the participant to the homepage. Another one out of eleven participants clicked the back button in the web browser and ended up on the first page. Nine out of the eleven clicked on the category “Nyheter” (News) in the navigation bar of which three had first clicked the replaced logo and ended up at the sport page again. Three other participants hovered the new logo before clicking “Nyheter” and three went directly to the homepage. Two of the participants showed confusion of the logo being replaced and one participant was not sure if she or he had ended up on the first page (see appendix 7).
The final task for the online newspaper was to find the hidden category “Teknik” (Technology) in the dropdown menu. The menu button and search function were placed at the right side in the navigation bar with some distance from the categories together. This task was made to see if the symbol for the dropdown menu was understandable and recognizable for the participants as well as the placement of the menu icon. Seven out of eleven participants were looking among the categories several times in the navigation bar until they found the dropdown menu. Of those seven participants, three of them only scanned the categories while the other four started to click and visit different categories. Of the four participants clicking around, three participants first clicked the category TV which took them to a site with film recorded news rather than articles. Four out of eleven participants found the dropdown menu at once. One participant
mentioned they really thought it was under the category TV, while another participant sighed frustratingly when clicking the TV category and two participants said it should be in the red line, referring to the navigation bar (see appendix 8).
After the usability test for the online newspaper Aftonbladet was conducted, the participants were asked two questions. The first question asked was what the participant found positive with the online newspaper. The drop-down menu and its symbol were mentioned in a positive manner by five participants. They liked how they had the choice to choose a specific category and how the menu was structured. The amount of news and constant updates were positive mentions by five participants who said they believed if they hear a news, they can go to Aftonbladet and find an article related to the news at the top. The way the online newspaper organized the content with its big news at the top and “Mest Lästa” (Most Viewed) was
commented on by two participants and another participant said it was easy to navigate through the site. The search function was easier compared to Jnytt according to two participants and one participant liked the color scheme. Another participant said there was nothing positive with Aftonbladet at all (see appendix 14).
The other question was if they found something negative on the online newspaper. Ten out of eleven said there is too much going on at the site where seven of them specifically were talking about ads. According to seven participants it was difficult to find what they were looking for and four of those did not like how the
search function did not fulfill their task. One participant out of eleven said there was nothing negative with the online newspaper (see appendix 14).
4.1.3 Jnytt
The first task given to participants on Jnytt’s website was to scroll the website for a minute and think out loud what came to mind. During this task seven participants were not only scrolling down but sometimes went up a bit before continuing scrolling down. Out of all participants, four of them scrolled down and never up again, two of which stopped at the articles with big titles and pictures before continuing scrolling. The footer of Jnytt’s website was visited by three participants during this task (see appendix 9).
Some topics mentioned by the participants during this task were the colors on the website, ads, and the “Mest Lästa”. The ads on Jnytt’s website was mentioned by six participants of which three participants mentioned that there were less ads than on Aftonbladet. The ads were perceived to take up more space by two participants however one participant perceived the ads to be structured in a more tasteful way. The most viewed articles on Jnytt are presented on top of the page and was pointed out by five participants out of which one explicitly perceived it as not so relevant and thought news should be at the top, one perceived it to be good to have it there and the other three gave no opinion. Four participants mentioned the colors on the website where 3 of which perceived it to be positive. One of these participants specifically mentioned the colors in the navigation bar. Other comments mentioned that were considered of relevance included the use of better pictures than in Aftonbladet, the repetitiveness of articles, the more basic design making it easier to see content, and the comment that the participant’s attention was directed more towards the left side of the website while the right side is ignored (see appendix 9).
The second task in the experiment was to search for a specific article on Jnytt’s website. All participants but one was able to find the given article. The one who did not find it was scrolling the page in order to find it and scrolled passed the article. In total five participants attempted to find the article by scrolling however two succeeded in finding the article by only scrolling. Six participants used the search function in the hamburger menu and managed to find the article while two participants used the categories in the hamburger menu to find the article. One of the participants who searched for the article found it and as they were about to click on it, an ad popped up and the participant ended up clicking on the ad instead, redirecting them to another site. The participant closed the window and clicked the article. The comments during this task included mentioning that the search function and the categories were hard to find (see appendix 10).
The third task was to find all sport news on the website. Nine participants quickly found “Sport” link in the navigation bar at the top of the page without any issues. The other two participants started scrolling
through the sport section on the homepage for a while first before eventually finding the “Sport” link in the navigation bar. One of them scrolled to the top of the page to find it while the other found it after clicking into another article. The comments during this task mentioned no difficulties in completing the task, similar to that of the fourth task (see appendix 11).
The fourth task was returning to the homepage from the Sports page. Eight participants completed the task by clicking the Jnytt logo in the top left corner. Two participants changed the web address in the URL to return to the homepage while one participant used the back button on the web browser. All participants completed the task with one click with the exception of one participant who clicked “Nyheter” in the navigation bar and ended up on a different section of the site, before clicking the logo. Another participant
almost had the same experience as they were hovering over the “Nyheter” tab before clicking on the logo (see appendix 12).
The fifth and final task on Jnytt was to find the page called “Näringsliv” with all news about finance and business. Ten participants found the page of which seven found it instantly of which four had done the tasks on Aftonbladet earlier. The one who did not find the page started scanning the navigation bar and hovered over “Jobb” (Work) before they started scrolling the page all the way to the bottom and then cancelling the task. Another participant also scanned the navigation bar and scrolled to the footer before clicking on “Näringsliv” in the hamburger menu. Two participants accidentally ended up on another page before clicking on the right link. One of them clicked on another category while the other clicked on a link that was supposed to only be visible on the homepage, but a glitch made so that it appeared in front of the “Näringsliv” link. This was commented on by two of the participants during the task (see appendix 13).
The semi-structured interview that followed the tasks included questions about the participants’ thoughts about Jnytt as a website. When asked what they found positive about Jnytt, the navigation bar was mentioned by eight participants of which four thought the colors of the different categories in the navigation bar were good. The other positive comment given on Jnytt was the use of most viewed articles, the section of the page including the sections and headlines of the page, well placements of ads, the search function and the ease in taking in the content on the page. When asked about the negative aspects of Jnytt, six people thought a glitch in the hamburger menu was bad. The glitch was perceived to be an article or and ad that covered the categories in the hamburger menu. Apart from this, other criticism included a too simple design, too many ads according to one participant and too little content on the page (see appendix 14).
4.1.4 Interview Findings
When asked which of the two websites the participants preferred, nine said that they preferred Jnytt while the other two said they preferred Aftonbladet. One of those two participants preferred Aftonbladet because how the page was color coded while the other participant preferred Aftonbladet due to the easy search functionality. Out of the nine participants that preferred Jnytt’s website, four of them did so due to Aftonbladet being too messy. Two participants preferred Jnytt because Aftonbladet had too many ads, including ads with motion. These two participants also preferred Jnytt because the design was better, the search was easier to use or because the color theme in the navigation bar was better than that of Aftonbladet. Two participants preferred Jnytt although they thought it did not have as much content as Aftonbladet while one participant preferred Jnytt despite having used Aftonbladet considerably more (see appendix 14).
A final question was asked to all participants if they would visit online newspapers more often if the web design would be improved. Five participants said that they think they would of which one thought that a better design would result in more visits in general and another participant mentioned that if Aftonbladet was to be redesigned she would prefer a smaller number of columns with news. Four participants thought that a better web design would not make them visit online newspapers more often while the two last participants were unsure and said that it depends, referring to needing to see the redesign for themselves before knowing (see appendix 14).
4.2 Analysis of findings
The data gathered from the usability tests, semi-structured interviews and questionnaires were analyzed through different methods. The voice recordings were transcribed word by word in separate document.
Information that was of relevance to the purpose of the study and respective task or question in the
transcribed interviews were extracted and inserted into tables. The actions seen in the screen recordings were noted down as well in the same tables. Observations made during each task, that were not mentioned in either the notes from the screen recordings or voice recordings, were also includes in the tables. Each table would include all these notes for one task. A separate table was made for all the information gathered from the questionnaires. Once all these tables were formed, comparative analysis were made to draw conclusions.
The result of the study shows Jnytt being the preferred online newspaper over Aftonbladet according to the participants when talking about the design and structure of the site. The majority of the participants said the online newspaper Aftonbladet gave an impression of being cluttered and too much going on. A
comment of the site being chaotic was also made. They also thought Aftonbladet had too many ads on their site. All of the participants who began the usability test on Aftonbladet, noticed when they visited Jnytt that there were less ads and one participant thought the ads were structured in a more tasteful way. There were other comments of the ads on Jnytt being placed in a better way.
The participant had divided opinions about Jnytt’s layout where some thought the headline ”Mest Lästa” followed by a list of most viewed articles at the top of the page, were smart to have while most of the participant thought it was not relevant and could therefore be removed or placed somewhere else. Some participants liked that Aftonbladet had the title ”Senaste Nyheterna” (Recent News) available while the some said they did not like how the site organized its content, with news being too randomly placed. It was mentioned that Aftonbladet’s web design resembled the design of a printed newspaper in terms of
placement of ads, articles and things screaming for attention. They could however not decide if it was a good or a bad thing.
The colors had been pointed out on both websites, showing its importance to the participants. One participant noticed how the colors for the sports section on the first page on Aftonbladet had a pink
background instead of matching the other articles, which made it clearer that this section had its own theme. Participants also mentioned the colors on Jnytt in a positive manner and liked the variations of colors in the navigation bar in comparison to Aftonbladet’s navigation bar.
Roughly half of the participants said they liked the constant update of news on Aftonbladet since the online newspaper are believed to have more to offer than the local one. Jnytt also got an opinion of the articles being repetitive and showing up at many spots on the same page. Aftonbladet was more preferred by the participants when it came to the amount of content, although some mentioned there were a better use of pictures at Jnytt.
The search functionality of the two online newspapers was tested in the second task. There were numerous ways to find the given article and all of them were used in some manner during the experiments. The most common method for finding an article was to use the websites’ search function. This functionality was proven more useful on Jnytt than on Aftonbladet. On Aftonbladet the title of the articles seen on the homepage were not the same as the title on the page for respective article therefore the title participants used to search for the article did not exist anywhere other than on the homepage. Another common method for finding the article was to simply scroll the homepage and scan through all articles, looking for the right one. The success of this method was only seen on Jnytt’s website. However, there were participants who scrolled to find the news only to scroll past the article when it was visible on the homepage. This was seen on both
CTRL + F function, a short command on the keyboard that opens up a search function in the web browser, would have been the quickest method to find the article and one participant did use it on Aftonbladet without success as the functionality requires that the spelling is accurate. The last search method used was using Google search to find the article. One participant attempted this for the article on Aftonbladet however failed in finding the article. The results from the tasks given show that participants had more success finding an article on Jnytt as opposed to Aftonbladet.
The third and fourth task tested the navigation of the two online newspapers. Both navigation bars on the online newspapers were praised by the participants who all found the Sports page with relative ease on Aftonbladet. Participants found the Sports page on Jnytt without real issue here as well however the process did not go as smoothly as on Aftonbladet. When returning to the homepage, clicking the logo was the most popular method for participants on Jnytt while on Aftonbladet, most participants clicked “Nyheter” in the navigation bar. Participants had no considerable issue when returning to the home on either website. The final task proved to be more difficult to complete on both online newspapers as participants spent considerable time scanning and searching for an icon or label that would guide them to the page they were looking for. The majority of the participants found the page on Jnytt instantly however many of them had a difficult time finding the hamburger menu. Those who found it with ease had already completed the tasks on Aftonbladet and were familiar with the icon for the dropdown menu there. Finding the hamburger menu was not the only issue participants encountered during this task as there was a glitch when opening the side menu. This was confusing to most participants. In comparison, the difficulties participants had with this task on Aftonbladet was actually finding the icon for the dropdown menu. Even though all participants completed the task successfully, only a minority found it at once while the majority spent much time scanning the navigation bar before identifying the icon. Several participants considered the TV
5.
Discussion and conclusions
The analysis of the result has been discussed and conclusions has been drawn. There is suggestion for further research in order to contribute to more research within the field.
5.1. Discussion of method
In order to better generalize the findings of this study, the participants chosen were narrowed down to be between the age of eighteen and twenty-six. A strength of this choice was that not as many participants were required in order to establish patterns made in the usability test. The experience with navigating the Internet varies across different ages and if the findings were to be generalized to more people, the study would have to be extended for a longer period in order to conduct the usability tests and semi-structured interviews. The downside to using a selected age group in the study is that the findings cannot be generalized to audiences outside of the participants’ age.
Similar to the choice of participants, the choice to do usability tests on Aftonbladet and Jnytt was made since Aftonbladet is one of the most popular online newspapers in Sweden. In order to avoid the bias participants would have towards a site they were familiar with, Jnytt was chosen as the second online newspaper since all participants had heard of it. The downside to using two online newspapers is that it would be hard to generalize the results to other Swedish online newspapers, as their web design nor their functionalities, although similar, has not been tested. This in turn, decreases the validity of the research. An upside however in limiting the research to two online newspapers is in depth research in one of the most visited online newspapers in Sweden. What Aftonbladet does well and not so well in creating a good user experience was thoroughly researched.
The choice of giving tasks to participants was to capture the process users goes through when browsing the websites in their own time. Different users have different purposes for using online newspapers and the tasks were meant to resemble what users might do. If not given tasks, another option would have been to simply allow the participants to browse the websites with no limitations and no instructions. This would have given more information in how users behave on the online newspapers however there would be rich amount of data not desirable to the purpose of this study. The participants were also asked to think out loud while completing the tasks, something they probably would not do if browsing the websites in their own time. Also, there might be some participant bias as some of the things they said may be what they think the researchers would want to hear.
A minor inconsistency in the method of the usability test was the second task for both online newspapers. Every day the online newspapers in study upload new articles to their website thus the location of the article in task two may be different the next day, if it even is visible on the homepage. Therefore, every day a usability test was conducted, the article chosen to find for task two was chosen based on its location on the homepage. The upside of using this method was seeing if participants paid attention to articles that did not take up most of the space on the screen while scrolling.
The technology used to analyze their behavior was screen recording which made sure that their actions were possible to analyze. However, when participants were scanning and searching the website it was not possible to pinpoint where their eyes were looking. Tracking the movement of the mouse was possible but many participants were looking around the site before moving the mouse. During the usability test this was made
up by observing the participants’ emotions and facial expressions. Nevertheless, their actual attention could not be tracked which would be something for further studies with the proper technology.
The participants of the usability test performed all tasks on a fourteen-inch MacBook Pro and used the trackpad in order to scroll and move the mouse. This could have affected the behavior of the participants to a certain extent as some of the participants may not be used to using a MacBook Pro or the trackpad.
In hindsight, the method used during this study gave findings that could be used to answer the research question to a certain extent and to fulfill the purpose. Certain adjustments could have been made for more accurate results, for example including a question in the semi-structured interview if the participants thought the website was good in general. With certain technology, such as eye tracking detection, the findings could also have been more accurate.
5.2. Discussion of findings
From analyzing the findings, it was seen that Aftonbladet does not use white space which is necessary to create balance on a website. The reason for this is quite possibly because Aftonbladet covers news from all over the world and have many employees. In order to run their business, they need advertisement and as much of it as possible in order to generate income. This would also explain the reason for having more ads than Jnytt since it only covers news in the city of Jönköping. This makes Jnytt’s website look much cleaner and pleasing to the eye in comparison. The absence of white space and the abundance of ads could be why some people missed the article they were looking for when scrolling the homepage. Many participants started out scrolling the page for a while before deciding to search for it.
Aftonbladet’s poor search functionality on the other hand cannot be blamed on either ads or the amount of content displayed. Only three of the eleven participants found the article in task two on Aftonbladet, even when given the exact title of the article seen on the homepage. The reason Aftonbladet has different titles for the article on the homepage and the page for the article, might be because the articles on the homepage are meant to attract users, as one participant reported, with clickbait titles. Nevertheless, this new finding was a surprise as it would be expected of a major website as Aftonbladet to have a proper search
functionality, considering how much content the website has. In contrast, the search functionality on Jnytt proved efficient for users and better than Aftonbladet. Users can find desired articles without issue by either searching in the hamburger menu or using the categories in the hamburger menu and navigation bar.
The navigation bar on Jnytt was praised, the colors in particular which was a surprise. The color scheme om Jnytt seem to follow no particular theme. Categories with little to nothing in common share the same color on their labels and the colors themselves are bright and demand attention. This might be why users would like the navigation bar as it is easier to find what they are looking for. A downside to using bright colors is taking attention away from that which is not bright on the website. This was seen when participants had difficulties finding the hamburger button. Most of those who did not have an issue finding the hamburger button had previously done the tasks on Aftonbladet and might have recognized the button do to its resemblance to Aftonbladet’s icon for the dropdown menu. Aftonbladet’s icon for the dropdown menu was also difficult to find, probably because the icon is located on the right side of the navigation menu with considerable space in between the icon and the rest of the labels in the navigation.
Both online newspapers showed little to no issues for users in returning to the homepage. The change in logo on Aftonbladet’s sports page shows the importance of consistency as many participants clicked or considered clicking on the new logo on the sports page. By contrast, all but one participant returned to the homepage with one click on Jnytt. The one who did not had done all tasks on Aftonbladet first, where returning to the homepage was done by clicking “Nyheter” and did the same on Jnytt.
In general users would prefer an online newspaper like Jnytt. Only two people preferred Aftonbladet to Jnytt and those two participants had completed all their tasks on Aftonbladet without issue, including task two with most amount of fails among the participants. Many also preferred Jnytt mainly because
Aftonbladet was too messy, not because they thought Jnytt is good. Whether or not the design of these online newspapers or any other ones can increase visits is unclear as the participants’ answers to the question were divided. The participants reported to not know what they thought of the idea before seeing the new web design.
The findings answers the research question to a certain extent in the sense that the user experience design of the online newspapers Aftonbladet and Jnytt were thoroughly evaluated. The research question is however not fully answered since it refers to all Swedish online newspapers and only two were tested. However, when comparing the findings to the findings of previous studies, the results do not differ by much. In Limbäck & Lundqvist’s study, the Swedish self-financed online newspapers Expressen and Svenska Dagbladet were investigated. The participants in their study reported to find the advertisement on the online newspapers disturbing as well, more so on a PC than on an iPad.
5.3. Conclusions
The research question “How is the user experience design of self-financed Swedish online newspapers?” has been answered in the study by having eleven participants without any pre-knowledge of web development to do a usability test and with a support of a short questionnaire and a semi-structured interview.
The designs of the online newspapers are well thought out and work but do not have the UX-design fully in mind. The first online newspaper, Aftonbladet, were not so scannable, meaning the site being too cluttered and messy to have a clear overview of the site. It was difficult for the participants to search the article they were looking for on the first page and some were not even willing to do so and went directly to the search function. The scannability decreased as both online newspapers had too many ads, however the participants found Jnytt more acceptable than Aftonbladet which had too many ads with motion calling for attention everywhere, which made participants lose focus and get annoyed.
As Limbäck & Lundqvist (2011) concluded in their study, the ads placement and amount play an important role for online newspaper and decrease the readability as it increases the frustration for most users. Almost every participant in this study mentioned something negative about the ads. Some thought it took the focus away, some said they could not always know when there wasn’t an ad or when there was an article and all participant preferred the placement of the ads on Jnytt instead of Aftonbladet since they were more static and spread out rather than filling the sites with ads and motions.
The placement and use of the navigation bar made the navigation on the sites easy. The participants also knew they would go to the navigation bar to look for the search button even though Aftonbladet’s search function did not work as expected and frustrated the participants. The dropdown menu at Aftonbladet as
