DEGREE PROJECT IN TECHNOLOGY, SECOND CYCLE, 30 CREDITS
STOCKHOLM, SWEDEN 2020
Investigating the
impact of interactivity on the credibility of digital news media
KTH Master Thesis
CarolinaItsaso Diess
KTH ROYAL INSTITUTE OF TECHNOLOGY
Author
CarolinaItsaso Diess diess@kth.se HumanComputer Interaction and Design KTH Royal Institute of Technology
Place for Project
Stockholm, Sweden Berlin, Germany
Examiner
Sandra Pauletto Stockholm, Sweden
KTH Royal Institute of Technology
Supervisor
Adrian Benigno Latupeirissa Stockholm, Sweden
KTH Royal Institute of Technology
Abstract
Consuming news and defining its credibility play a large role in our everyday lives. The digitalisation of news has enabled new interactions with the medium, that have yet to be analysed in their impact on credibility. This study aims to investigate the effects interactivity has on perceived credibility and how user interactions can be applied to the digital news medium.
The analysis is done through a usercentric approach using both qualitative and quantitative methods based on design thinking. The methods used include a digital questionnaire, user interviews and prototype testing.
Using these methods we find that no strong association can be made between the frequency of use of specific digital interactions such as sharing, liking and commenting and perceived credibility. While most users see an added benefit in having more interactive elements on a digital platform, it cannot be concluded that overall higher levels of interactivity lead to higher credibility. However, if interactivity is used to enable people to voice their opinion an increase in trust can be built, which subsequently increases credibility.
Keywords
news credibility, interactivity, news consumption, digital news
Abstract
Att konsumera nyheter och definiera deras trovärdighet spelar en stor roll i vår vardag. Digitaliseringen av nyheter har möjliggjort nya interaktioner med mediet, som ännu inte har analyserats i deras inverkan på trovärdigheten. Denna studie syftar till att undersöka effekterna som interaktivitet har på upplevd trovärdighet och hur användarinteraktioner kan tillämpas på det digitala nyhetsmediet.
Analysen görs genom en användarcentrerad metod med både kvalitativa och kvantitativa metoder baserade på designtänkande. Metoderna som används inkluderar ett digitalt frågeformulär, användarintervjuer och prototyptestning.
Med hjälp av dessa metoder finner vi att ingen stark koppling kan göras mellan frekvensen för användning av specifika digitala interaktioner som att dela, gilla och kommentera och upplevd trovärdighet. Medan de flesta användare ser en extra fördel med att ha fler interaktiva element på en digital plattform, kan man inte dra slutsatsen att övergripande högre nivåer av interaktivitet leder till högre trovärdighet. Men om interaktivitet används för att göra det möjligt för människor att uttala sig, kan man öka förtroendet, vilket därefter ökar trovärdigheten.
Nyckelord
nyheter trovärdighet, interaktivitet, nyheter konsumtion, digitala nyheter
Acknowledgements
This thesis concludes my Master in HumanComputer Interaction and Design at KTH and Aalto University as part of the EIT Program in ICT Innovation. Throughout my master, I have met and learned from a brilliant variety of people whom I would like to thank.
Firstly, I would like to thank my supervisor Adrian Benigno Latupeirissa for guiding me throughout this thesis process. Alongside him, I would like to thank my supervision group for sharing their knowledge.
Secondly, I would like to thank my examiner Sandra Pauletto for providing valuable feedback to enhance the quality of this work.
Finally, I would like to thank my friends and family, who have encouraged me
throughout my studies and supported me wherever they could. My family, for making
such an amazing international opportunity possible for me. My friends, for always
listening and giving advice when needed.
Contents
1 Introduction 1
1.1 Motivation . . . . 1
1.1.1 Problem Definition . . . . 2
1.1.2 Outline of Thesis . . . . 3
2 Background 4 2.1 Digital News Media . . . . 4
2.1.1 Definition . . . . 4
2.1.2 Media Consumption . . . . 5
2.1.3 Shift to Digital Media . . . . 6
2.2 Credibility . . . . 7
2.3 User Interaction . . . . 8
2.3.1 Interactivity . . . . 8
2.3.2 Common User Interactions . . . . 9
2.3.3 Changes in Interaction with Digital Media . . . . 12
3 Research Methods 13 3.1 Research Design with Design Thinking . . . . 13
3.2 Quantitative methods . . . . 15
3.2.1 Questionnaire . . . . 15
3.2.2 Statistical Analysis Fisher’s Exact Test . . . . 16
3.3 Prototype . . . . 17
3.4 Qualitative Methods . . . . 19
3.4.1 User Testing Setup . . . . 19
3.4.2 Interviews . . . . 21
3.4.3 Thematic Analysis . . . . 22
CONTENTS
4 Results 24
4.1 Questionnaire . . . . 24
4.1.1 Demographics . . . . 24
4.1.2 Basic News Consumption . . . . 25
4.1.3 Credibility . . . . 26
4.1.4 Interaction . . . . 29
4.1.5 Associations between Credibility and News Consumption Habits 30 4.2 Prototype Testing . . . . 35
4.3 Interviews Thematic Analysis . . . . 36
4.3.1 Information . . . . 37
4.3.2 Entertainment . . . . 37
4.3.3 Communication . . . . 39
4.3.4 Trust . . . . 41
4.3.5 Neutrality . . . . 42
4.3.6 Transparency . . . . 43
5 Discussion 45 5.1 Results . . . . 45
5.2 Limitations and Method Critique . . . . 47
6 Conclusions 48
References 50
Chapter 1 Introduction
1.1 Motivation
According to a recent Statista
1study, 67% of people worldwide have said that they are following news coverage more closely than before, due to the Coronavirus outbreak.
In times of heightened media consumption and worry over a global pandemic, investigating news credibility and human interaction with news media seems more relevant than ever. Not only is there more news media being produced than ever before, but it is also being spread more rapidly and on a global scale due to the rise of social media. Past research on the topic of news credibility is a foundation to build upon, however new research is required to analyse media credibility in the circumstances we are facing today.
Previous research has often compared newspaper and television news as a form of media [26]. In this day and age, a whole digital world has been added to this that also needs to be taken into account [16]. Not only is there a new platform through which news is being shared but the type of media and how we consume and interact with it has evolved. We cannot simply take a classical print newspaper, digitalise the text, and believe it to have the same impact on credibility. This is why research considering modern channels of media consumption and interaction is necessary.
The impact of interactivity in itself has also been assessed on digital platforms and more specifically related to digital news media. Enhanced interactivity has been found
1https://www.statista.com/statistics/1106498/homemediaconsumptioncoronavirusworldwide
bycountry/
CHAPTER 1. INTRODUCTION
to have a positive impact on satisfaction and adaption [5], [30]. Chung et al. [4] find that traditional credibility factors still play a more important role in building credibility than technological factors. Interactivity is one of three technological factors analysed alongside multimediality and hypertextuality. However, the focus of their study lies on multimedialety and requires further research into details of interactivity. The following research focuses specifically on the factor of interactivity. It aims to provide an in
depth understanding of the correlation between interactivity and credibility by using a usercentric approach to gain profound insights into user motivations.
Media and assessing news credibility is something that affects us all. News media is something we interact with on a daily basis. Investigating a medium that some, such as Lewis [20], have argued has an impact on democracy can have a large impact on society. Not only is there a benefit in understanding credibility and user interaction with news media for individuals but also for publishing houses which have been struggling to stay relevant in a time of media abundance. Publishing houses and other platforms could benefit from setting themselves apart from competitors through new types of interactions. Interesting findings from this research could be further developed to build guidelines for interactions with digital news media.
1.1.1 Problem Definition
In a world of abundant news media, it is vital to filter out what is relevant and credible information, and what is not. People have always depended on a variety of heuristics such as reputation and selfconfirmation, for evaluating credibility [23]. Whether it be evaluating the credibility of a story somebody is telling or assessing the believability of an established newspaper. However, digital media is evolving at a rapid pace and the interaction people have with a digital medium is changing. Therefore we need to understand whether how people evaluate credibility is also evolving. This research investigates typical social media interactions such as commenting, liking and sharing to assess their effect on credibility. It is within the scope of this work to dig deeper to understand not only how people interact with news but also why they do so.
Research Questions
This research aims to answer the following main research question and sub
question.
CHAPTER 1. INTRODUCTION
1. What impact does user interactivity have on digital news media credibility?
2. How are different user interactions applicable in the context of news media?
Goals
The degree project will be fulfilled when all research questions can be answered. A general understanding of how users interact with news media will have been obtained from the qualitative survey. Also, it will have been tested whether a relationship can be made between frequency in specific user interactions and credibility. Different user interactions will have been investigated and applied to a news media platform in form of a prototype. The testing and interview will give more indepth insights into user interactions and how they are perceived by the user. Finally, future recommendations on how user interactions can be used on digital news media platforms will be made.
1.1.2 Outline of Thesis
This research is structured in six main parts. Starting with an introduction of the topic,
explaining the research goals and why they are of importance. The second section is
background research, in which key terms are defined. News consumption and how it
has transformed over the years is looked into. Next, previous works on the believability
and credibility of news media are reviewed. Looking at seminal works going back to
McCroskeys Scales for the measurement of ethos [22] and Hovland, Janis and Kelley’s
work on Communication and persuasion [13]. The background section is concluded
by looking at user interactions. First, defining different types of user interactions
and then looking more closely at three user interactions that are common in social
media. Section three explains the methods used throughout this research. It explains
the research design based on the design thinking methodology. Both the qualitative
and quantitative methods used are defined.Further, the making and testing of the
prototype are illustrated. This section is followed by the results section. Here the
gathered data from the various research methods used are presented. Analysis and
interpretation of these results is done in the discussion section. The thesis is finished
with a conclusion answering the research questions and giving a brief overview of key
findings, key learnings and next steps.
Chapter 2 Background
2.1 Digital News Media
2.1.1 Definition
A common understanding of the term news media is vital to make this research consistent and comparable to other studies. Although news is a word used daily, the understanding of the term may vary. In predigital ages, news media was commonly defined as an article found in a newspaper or something being reported about on news television or radio. In a digital age, such distinctions have become more difficult to make. Not only are there digital versions of classical print media such as newspapers, but we also find news being spread and created through social media platforms.
Formerly, big publishing houses and journalists selected what news was published, now anybody can go online and create a blog post or video. The issue here is that these posts can reach from individuals sharing personal photos to established newspapers sharing breaking news all in one place with little to no distinction. However, studies have found that users have a consensus when defining what is social content versus what is news content on social media [38]. Therefore, we can use the Oxford Dictionary definition of news being reports of recent events that appear in newspapers or on television, radio or the internet.
During the prototype testing section of this research, digital news articles in form of
image and headline were used as these fall into a common understanding of what news
is. Throughout the interviews news media was looked at in a broader sense not focusing
CHAPTER 2. BACKGROUND
only on one news format, but trying to get an understanding of what interviewees define as news. This approach includes news as reports of recent events that appear in any medium with a focus on the digital medium.
2.1.2 Media Consumption
When trying to understand the term of news media a distinction can be made between socalled hard and soft news. Hard news focusing on politics and world affairs whilst soft news encompasses entertainment content. Whilst arguably the amount of soft news has increased with the digitalisation of news, soft news has always played a part in newscasting and journalism. [29], [38]
The concept of hard and soft news also leads to the question of how and why people consume news media. The motivation for consuming news reaches from gaining information and knowledge to pure entertainment. Hard news is more commonly consumed to gain information and soft news more commonly for entertainment.
Studies have found that consuming media fulfills a variety of different purposes depending on the medium through which it is consumed. For example, reading an article in a newspaper fulfills the need of providing information whilst watching a segment on tv entails a social aspect and is linked to spending time with friends.
Reading arguably requires more effort from the user while watching a video or listening to a broadcast can be done more passively. [1]
Although some maintain that the actual behavior of news consumers has not changed with the birth of digital media, others find that the interaction and use of media have developed with the new medium, respectively [14], [28]. This argument is dependent on what we define as news media and usage or consumption. Mitchel and Boczkowski argue that the actual interaction of reading a newspaper article has merely shifted to a digital medium however, reading still occurs in the same manner [25]. This may be true when looking at newspaper websites as a host for digital news media. If we however include social media platforms one can debate that the interaction has changed.
Not only has the way we read articles changed but also how we get to them. Studies
show that vast amounts of US adults state social media as one of their main sources
of news [33]. The use of digital channels has not only changed the way we consume
media but also where. Reading the news is no longer a thing we do over breakfast
at home, it has become something many users consume ‘on the go’. Understanding
CHAPTER 2. BACKGROUND
where, how, and why the modern user consumes news will be further investigated in this research.
2.1.3 Shift to Digital Media
In order to understand the importance of media, we need to take a look into the role it has played in history. Some argue that media is the foundation of democracy as we know it today [36]. Print media opened up knowledge and information about politics and the economy to the broad masses. With news being so readily available and entering every household, it is truly democratising [27].
Figure 2.1.1: Circulation of daily newspapers in Germany, years 1991 to 2019 adapted from Statista
1The relevance of mainstream media has been questioned many times over the years.
With worries of print media dying completely and studies showing that there is a decline in print media readership [9]. Charts such as Figure 2.1.1 from Statista
1, show a rapid decline in newspaper readership in Germany. With the circulation of daily newspapers dropping by almost 50% within the last 28 years. A similar trend can be observed in most developed countries.
It has been reported that more than 50% of people in the EU read the written press at
1https://www.statista.com/statistics/380784/circulationdailynewspapersgermany/
CHAPTER 2. BACKGROUND
least once per week
2. Hence showing that print newspapers are an important source of news information for large parts of society. While there is a shift taking place and print media readerships are declining, more and more people are consuming news digitally.
Be it through reading an online newspaper, through their social media feeds, or from video platforms.
Print media has always been locally accessible at a low cost. With digitalisation, much of this information is now available for free on a much more global scale. However, with this improvement also arises the issue of information overload and a need for people to quickly scan and select which items are relevant to them.
2.2 Credibility
With 37% of Europeans stating they encounter fake news every day or almost every day, the credibility of news is a topic worth looking into
3. As we are not only interested in inspecting the changed interaction of users with media but also the impact these interactions have on perceived credibility it is vital to gain an understanding of the concept of credibility.
Credibility can be defined as the believability or the trustworthiness of the source or message, defined by the information receiver [23]. First, we have to make a distinction between the credibility of the source or of the message or both. This divide in credibility comes from the beginnings of credibility research which is much prior to the digital world. The source hereby being the person or organisation that is spreading the message and the message in case of news media being an article or a piece of news.
Arguably, in a digital world, a third element of credibility should be defined as the medium or channel of delivery [2]. However, this has a major overlap with the element of source, thus leading to the question of whether factors of source credibility should simply be adapted to a new digital source.
Early research focused on the credibility of a speaker has found source credibility to rely mainly on trustworthiness and expertise of the source [13]. There have been multiple scales of measurement developed to try and compare source credibility. Most of them are based on a variety of different characteristics of the source, however, usually varying in characteristics. Some being more or less applicable to a digital environment.
2https://www.statista.com/statistics/422680/europeusagefrequencyofthewrittenpress/
3https://www.statista.com/study/68765/fakenewsineurope/
CHAPTER 2. BACKGROUND
For example, measures of ethos such as by McCroskey looking at extroversion, character and composure apply more easily to a human source [22]. Other approaches, such as by Leathers identify the factors of competence, trustworthiness and dynamism, which may be more easily applied to a digital platform [24].
One commonly used index to define credibility is from Gaziano and McGrath. It consists of a 16factor analysis of which 12 factors are grouped. Some of the high
loading factors include: is fair or unfair, is opinionated or factual, can or cannot be trusted [11]. The three mentioned highloading factors will be used in this study to measure credibility.
Classical print media often builds credibility on the fact that the journalist or publisher is seen as credible therefore the article written or published by them is seen as credible.
In a digital age publisher and author of news can be challenging to identify, which makes this form of analysing credibility difficult. With the source being hard to identify the user relies on assessing credibility through the message. Message credibility is built up through many dimensions. According to Metzger et al. [24], the key components of message credibility are message structure, message content, language intensity, and message deliver .
While news websites have been found to be perceived as more credible than personal sites such as blogs this credibility is often based upon the design of a website rather than the credibility of the author or organization behind it [7]. Factors of credibility and their importance vary when building credibility in classical print media and digital media. Other studies have also found that one of the most mentioned characteristics when it comes to evaluating the credibility of a website is its design and information structure [8]. Since the focus of this research is on the impact of interactivity no analysis of the appearance or design of the prototypes is done. Rather, a point is made of keeping all styling to a minimum in order to focus on interaction.
2.3 User Interaction
2.3.1 Interactivity
In order to analyse how interactivity may have an impact on credibility, we need to
define interactivity. Interactivity has been defined as the degree to which users can
interact with content. It is not dependent on the medium and induces engagement by
CHAPTER 2. BACKGROUND
enabling users to communicate with technology or other users.[31] The range between human and medium interactivity has been made important by many researchers [6].
Figure 2.3.1 shows the four forms of interactivity as postulated by Chung [5] and used by Larsson [18]. Each form of interaction is described by its functions and examples.
For this research a slightly simplified look at interactivity will be taken, focusing on the differentiation between human interactivity and medium interactivity. The distinction has been added to Figure 2.3.1. Human interactivity consists of humanhuman and humanmedium interactivity and is also referred to as useruser interactivity.
Mediumhuman and mediummedium interactivity form medium interactivity also referred to as usersystem interactivity. This approach has also been previously used by Chung [6]. Usersystem interactivity enhances the interaction of the user with the medium, for example enabling navigation or manipulation of the medium to the users’
needs. Useruser interactivity allows the user to engage and connect with other users, often through direct or indirect communication between users.
Figure 2.3.1: Different types of interactivity, adapted from Chung [5] and Larsson [18]
2.3.2 Common User Interactions
Humans have read or consumed news and shared and discussed the information with
friends, family, or colleagues for centuries. However, how and with whom we interact
has changed given the new digital platform. Digital interactions have been split into
CHAPTER 2. BACKGROUND
two main groups: acknowledging reactions and sharing reactions [19]. Acknowledging reactions are commonly lessdemanding actions such as liking an article. Due to their simple to carry out nature they usually have higher engagement numbers from users.
Sharing reactions require slightly more effort from the user and thus are used less frequently. [17]
In this study, we are focusing on interactivity in the form of three key useruser interactions. All of which are commonly used in social media. With 48.3% of the world population using social media, analysing its key interactions and how they attract users and can be applied to other platforms seems relevant [34]. The three interactions:
liking, sharing, and commenting will be explained in the following sections.
Liking
Acknowledging reactions have become a commonplace interaction on social media platforms. The most widespread and commonly used one still being like. However, with large social platforms such as Facebook leading the way these interactions have become more nuanced. After much discussion about whether Facebook should implement a dislike button in 2016 the network redesigned its reactions to six nuances:
love, haha, wow, sad, angry and like
4. More recently Facebook has added a care reaction during the COVID19 Pandemic. These seven reactions can be seen in Figure 2.3.2. Soon after other platforms such as LinkedIn followed suit adding a slightly different assortment of easily accessible reactions. Typical news publishing platforms have however not adapted to this trend. Only sites such as Buzzfeed, which have a stronger focus on ‘soft’ news or entertainment content give their users an option to show their reactions.
Figure 2.3.2: Facebook Reactions
4Closely linked to the liking interaction are user rating and review systems, such as can be found on the content aggregation site Reddit. This again is an interaction which we commonly find in social media and rarely in news media. The most basic ones
4https://about.fb.com/news/2016/02/reactionsnowavailableglobally/
CHAPTER 2. BACKGROUND
consisting of simple up/down or like/dislike votes. More complex forms of user rating or review systems contain 5star rating scales or long comments with added images. A simple rating system and its effects will be tested in the prototype. However, a deeper more focused analysis of user rating systems is out of the scope of this project.
Sharing
Bergstrom [3] postulates that there is a reluctance in users to actively interact with digital news, which is why simple interactions such as liking and sharing are still the most common. With many platforms enabling easy sharing through social media or email. While sharing requires more than a simple click of an acknowledging reaction it is still a relatively low effort reaction. Due to the simple implementation on most modern digital news platforms, as can be seen in Figure 2.3.3 from the Guardian
5, sharing of an article can be done within a few simple clicks. Next to enabling simple sharing a count of the number of times the article has already been shared is provided.
This plays into the element of user rating and could have an interesting effect on credibility.
Figure 2.3.3: Sharing as seen on theguardian.com
5Commenting
The third common interaction we will look into regarding digital news platforms is commenting. In terms of effort required by the user, it is substantially higher than liking or sharing. For specific content pieces commenting is also available to the user, however, comments are usually monitored and mediated, as can be seen in a screenshot from the New York Times
6in Figure 2.3.4. If commenting is available it is however usually only on opinion pieces and in a moderated manner.
Not only has social media changed the speed at which breaking news stories can be shared with the public. It has also had an incredible impact on how quickly these stories can be spread due to the interaction users have with social media platforms.
Traditional news platforms cannot set off the type of media avalanche that Twitter
5https://www.theguardian.com/
6https://www.nytimes.com/
CHAPTER 2. BACKGROUND
Figure 2.3.4: Commenting as seen on nytimes.com
6or Facebook can cause, even if they have an international following. Whereby a post is potentially retweeted by thousands of people and thus seen and read by millions around the globe within a short period of time.
2.3.3 Changes in Interaction with Digital Media
It is indisputable that changes have taken place in the way we interact with news from classical to digital media. However, the radical change that may have been thought to take place in the level of interaction and participatory journalism has failed to happen [18]. Lack of user interaction can be observed in the fact that only very specific content in mainstream news media is available to comment on, often being more soft news.
Also, most digital news is still consumed in a mainly textual way not taking advantage of the multimediality that the digital medium offers.
This is true for large mainstream publishing houses that have moved their classical
print newspaper online. However independent news sites and blogs have come
into existence to challenge this. Understanding interactivity and the effects it has
on credibility could lead to developments on this front, which would push larger
publishing houses to experiment more with interactivity and the digital medium.
Chapter 3
Research Methods
3.1 Research Design with Design Thinking
The research was loosely based on the design thinking methodology, initially made prominent by Nielsen Norman
1. Design thinking is a userfocused methodology that has become extremely popular in businesses over the past years making it a term known not only by designers but in a variety of different fields. Due to this the concepts and methods have evolved and been adapted to suit the specific needs of the project.
Design thinking is an iterative process that consists of three main phases:
understanding, exploring, and materializing. Each of these main phases consists of two steps. As it is an iterative process there is the possibility of taking a step back at every point in the process or even starting from the beginning. [32]
The understanding phase consists of empathising and defining. During empathising the designer or researcher is trying to gain an awareness of the problem and the user.
As designing for a context is at the heart of design thinking this is a vital stage in the process and builds the foundation for later steps. In the defining stage, the previously gathered data from empathising is then gathered and analysed. It may be found that there is not yet enough data at which point a step back can be taken to reempathise.
The outcome of the defining phase is ideally a complete understanding of both the problem and the user. The understanding phase of this research consists of three elements. The first, being the background research used to understand and define the
1https://www.nngroup.com/articles/designthinking/
CHAPTER 3. RESEARCH METHODS
problem. The second coming from the qualitative research in the form of an online questionnaire. The last element of understanding is done in form of indepth user interviews and the following thematic analysis, both of which will be expanded on in subsection 3.4.2 and subsection 3.4.3.
Based upon these findings the phase of exploring begins. During which ideation and prototyping are key elements. Throughout the process, different types of prototypes may be created to test different elements or ideas. Initially, a prototype may be used to test an idea whereas later it can also be used to test something as specific as a designed UI element. Prototyping in this research was done through an interactive mobile newsfeed with varying levels of interactivity that will be explained in detail in section 3.3.
The final phase materialising is started through testing. Testing is a critical part of the design thinking process and often a point after which there has to be a step back in the process in order to make changes to the prototype or if the idea has been completely invalidated to go back to earlier steps. All of the steps are reiterated until a prototype is tested that meets the needs of the user. After which it can finally be implemented.
Meaning that the development process begins. In this project, design thinking is used as an aid in exploratory research. The focus is not creating a product, which is why the emphasis is largely on the understanding and exploring steps. There is however an element of prototyping and testing involved. Implementation would require many more iterations of the process to lead to an actual product.
Figure 3.1.1: Design Thinking from the Nielsen Norman Group
2CHAPTER 3. RESEARCH METHODS
3.2 Quantitative methods
3.2.1 Questionnaire
The main quantitative research method of this study was an online questionnaire created with Google Forms. Google Forms was chosen over other more complex online surveying tools as it covered all required functionality of multiplechoice, singlechoice, and scaled answering options. Sharing via link is simple and does not require any registration from the participant. Furthermore, the gathered data can be easily viewed and downloaded to analyse the results. The survey was sent out through multiple digital channels in order to try and reach a broad spectrum of people. Aiming for a reach of 100 participants focusing on people in Europe.
As part of the understanding phase of the design thinking process, the survey was used to gain a broader understanding of the way people consume digital news media. A general understanding of how people consume news media had been gained through background research. The questionnaire consists of four sections, which are introduced with a brief text explaining the nature of the research, what the information is being collected for, and how it will be used. Additionally, an estimation of the time the survey takes is given, to provide transparency to the participant. A contact detail, in the form of an email address, is given to all respondents in case they have any questions and would like to reach out. No incentive was given as the survey is very short and does not require more than five minutes of people’s time.
The first section of the questionnaire begins with basic demographic information about the respondent. This is used to see whether there can be a demographic link or a distinction between different demographic groups in answers. The demographic section identifies four key demographics: age, gender, level of education, and nationality.
Next, the second section consists of six questions that gain a basic understanding of how people consume digital news media. Reaching from language to frequency and access channels. The final question of this section goes into whether people pay to access digital news and whether they would consider this as an option.
The third section of the questionnaire focuses on credibility, starting with information
on perceived credibility on a general level then digging deeper into how often people
find news to not be credible. Finally looking into people’s reactions and interactions
CHAPTER 3. RESEARCH METHODS
with news and how they verify news. The survey concludes with a section on interaction and interactivity. This section contains two statements of agreement. Here a five
point Likert scale is used reaching from strongly disagree to strongly agree. While the fivepoint version of the scale has the issue that it may cause a tendency to select the neutral middle value, its common use makes it easy to understand and allows for a quick selection of an answer [21].
Throughout the survey, there is a focus on making answers as clear and unambiguous as possible. Making sure that everyone understands the question and answer in the same way thus leading to consistent and reliable results. For example when using scales reaching from never to always a definition of these terms is used, thus ensuring the same understanding. In the case of frequency of news consumption, this is shown in Figure 3.2.1. Otherwise rarely can have varying meanings depending on who is reading and answering the question.
Figure 3.2.1: Example Question from the online Questionnaire
The main question type used is single or multiple choice questions as these make further data analysis simpler and also ensure quick response times. In order to enable the user to still add further answers most questions have an ‘other’ option which allows free input from the respondent.
3.2.2 Statistical Analysis Fisher’s Exact Test
Fisher’s exact test is used to analyse the correlation of categorical information, meaning it can be applied to nonnumeric values. It is typically applied to small data sets, where the chisquared test is less accurate. Due to the higher accuracy in calculating the P
value Fisher’s test is preferred in these circumstances. Although it is commonly applied
CHAPTER 3. RESEARCH METHODS
to 2x2 contingency matrices it has been expanded to be used on larger matrices by Freeman and Halton in 1951 [10]. Digital computing has made it simple to apply to 2x3 or 3x3 matrices. The test is a measure of whether the data provides evidence against a null hypothesis. The Pvalue determined shows the probability of getting a difference as big as the one observed if the null hypothesis is true. Typically, a Pvalue of lower than 0.05 is taken to show significance. [12], [35]
The calculations of the Pvalue in this research were carried out using an online tool by VassarStats
3. The results are two Pvalues (P
Aand P
B) and the number of tables evaluated. P
Ais the probability of the observed array of cell frequencies plus the sum of the probabilities of all other cellfrequency arrays that are equal to or smaller than the probability of the observed array. Where P
Bthe probability of the observed array of cell frequencies plus the sum of the probabilities of all other cellfrequency arrays that are smaller than the probability of the observed array. Due to this P
Awill be the Pvalue we focus on in later research.
3.3 Prototype
The prototype creation and subsequent testing aimed to take a deeper look at the role interactivity has on credibility. This was done with three versions of an interactive digital newsfeed with varying levels and types of interactions.
All prototypes showcased a newspaper feed consisting of 32 news article teasers. Each teaser consisting of a headline and an image. No further information on the author, date, source, or category was given. The headlines were chosen on a randomly selected day during the prototype making and taken from the news aggregation site Google News. A news aggregation site was chosen over one specific newspaper in order to try and keep news bias and political tendencies low and to reflect global and not country
specific news. For the user to feel like they are looking at an actual news site rather than just a few articles, 32 articles were used. This leads to a more realistic and natural user experience overall. 32 news stories from 8 categories were selected, 4 from each category. The 8 categories were: US, World, Business, Technology, Entertainment, Sports, Science, and Health. Although each prototype used the same news articles the order was varied through random sorting of the articles in each newsfeed.
3http://vassarstats.net/fisher3x3.html
CHAPTER 3. RESEARCH METHODS
Table 3.3.1: The three different versions of the prototype with different interactions
The prototypes were made using the interface design tool Figma
4. They were designed to be as neutral and unstyled as possible. This was done to focus the feedback on the actual interaction rather than on the design. The fidelity of the prototype was kept closer to a wireframe than a finished and styled user interface. All elements were kept in black, white, and grey and the commonly used Android font Roboto was used.
The prototype was used to investigate both research question RQ1: What impact does user interactivity have on digital news media credibility? and RQ2: How are different user interactions applicable in the context of news media? In order to investigate this, three news feeds with varying degrees of interactivity were made.
Table 3.3.1 shows the added interactions for each of the three prototypes.
Version A consists of 32 news articles and a header. Each article is represented in form of a teaser consisting of an image and a headline. The only interaction is scrolling through the articles. The simple prototype can be seen in Figure 3.3.1.
Figure 3.3.2 shows version B of the prototype again has a headline and image for each article. Each article teaser has three added buttons, all of which consist of an icon and a number counting the interactions. The interactions are like, comment, and share.
All of which are used to enhance useruser interactivity. Liking is a simple, oneclick acknowledging interaction. Whereas commenting and sharing requires slightly more clicks and thus more effort from the user.
Figure 3.3.3 shows the third version of the prototype. In prototype C the user can search the newsfeed and filter by categories. These interactions allow the user to navigate and manipulate the content he can see. This enhances medium or usersystem interactivity.
4https://www.figma.com/
CHAPTER 3. RESEARCH METHODS
Figure 3.3.1: Prototype A, showing a scrollable newsfeed with no added interaction
Figure 3.3.2: Prototype B, showing a newsfeed with the interactions: like, comment, share
3.4 Qualitative Methods
3.4.1 User Testing Setup
User testing with the prototypes was carried out with 10 people. Due to time limitations, indepth interviews after the testing were only carried out with 5 of the 10 participants. The participants were previously recruited through the online survey.
This was done by asking participants who had filled out the survey if they were willing to take part in a user interview and prototype testing session.
Due to the current global situation and the time limitations of this research all of the
CHAPTER 3. RESEARCH METHODS
Figure 3.3.3: Prototype C, showing a newsfeed with the interactions: search and filter
testing was carried out via video call. This has some technical limitations but overall worked out well. Each session was conducted online using Google Hangouts as a video call tool. A link was sent to the participants enabling them to take part without having to download or register to any platform. After an introduction explaining the reason for the research and giving an overview of the planned setup of prototype testing and interview, the participants were asked if recording the session would be acceptable.
The recordings were used to later transcribe the interviews without having to focus on taking notes during the session. Due to the testing being conducted by one person, this allowed me to fully focus on the session. In total, the prototype testing and interview session were planned to take approximately 45 minutes. Half of this time was scheduled for the prototype testing and the other half for indepth interviews.
Table 3.4.1: Tasks for user testing
The testing sessions were started with prototype testing. The participants were sent
a link for each version of the prototype and a link after each prototype to answer the
fivequestion survey. Due to the small number of participants, every participant saw
every version of the prototype. With a larger group of participants, groups could be
CHAPTER 3. RESEARCH METHODS
formed to show only one version of the prototype to each group. The three previously mentioned interactive prototypes were randomly ordered for each participant. This was done so that learning throughout the testing process would not always have the same impact on results. Each participant was shown a prototype and asked to browse and then complete a task with each one. Due to the different levels of interactivity and to make the testing less repetitive the tasks were different for each version of the prototype. The intention of giving the users tasks was to get them to interact with each prototype more naturally. Getting them to focus on a task rather than just being shown a prototype and asked to evaluate it.
Table 3.4.2: Likert scale ratings in Prototype Survey
After completing the task and browsing the prototype the participants were asked to fill out a fivequestion survey. The order of the questions was randomized through Google Forms to cause less learning during the test set up. Four questions were used to determine the credibility of the prototype. One of which simply asked whether the prototype was credible or not. The others were key factors from Gaziano and McGrath on measuring credibility: fair, factual, and can be trusted [11]. This process was repeated after each version of the prototype.
3.4.2 Interviews
After completing the prototype testing session 5 of the test users continued to an interview. This was done as conducting interviews with all 10 test candidates would have been out of scope for this project. The time required to transcribed and analysed all the interviews would be very high.
The interview was conducted in a semistructured way. Based on an interview
guideline covering key topics of interest but allowing the conversation to flow naturally
during the interview process. Similar to the online survey the guideline consisted of
CHAPTER 3. RESEARCH METHODS
four main sections. These were: demographic information about the interviewee, basic news consumption, interaction with news, and credibility.
The first section giving a brief introduction to the person was used to start a conversation and get people used to the setup. Asking people to tell me a bit about themselves in order to feel more comfortable and less like they are in an interview setting. Next, the conversation was turned to how they consume news, from which medium, and digging deeper into the reason behind this. Even if the topics of the interview and questionnaire are similar the interview aided to understand why people interact with news in the way they do rather than just how. Enabling people to go into more detail and elaborate much further than a simple multiple or singlechoice question.
The second half of the interview consisted of questions on interaction and credibility.
Here going further into why people find news to be credible and how they react to news they do not find credible. In terms of interactivity, common interactions such as liking, commenting, and sharing were also discussed gaining a further understanding of why people chose to use them or not use them.
3.4.3 Thematic Analysis
Thematic analysis based on the Braun and Clarke approach was used to investigate the findings from the interview. This qualitative data analysis method is used to find patterns and themes in the data. It can be used across data sets in order to find commonalities. It was chosen as the method due to the systematic approach that is still flexible enough to adapt perfectly to the given data set and circumstances. [37]
To start with thematic analysis, all interviews are recorded and subsequently
transcribed. The transcripts are then used as the basis of the data. Thematic analysis
is a sixstep process beginning with familiarizing oneself with the data, the six phases
can be seen in Figure 3.4.1. The following phases consist of coding the data and then
searching for themes within these codes. These themes are then checked ultimately
concluding in a thematic map. Once the defined themes have been verified they are
further defined and given names. Finally, a report of the findings is produced.
CHAPTER 3. RESEARCH METHODS
Figure 3.4.1: The 6 Phases of Thematic Analysis as defined by Braun and Clarke [37]
Chapter 4 Results
4.1 Questionnaire
4.1.1 Demographics
The questionnaire was initially sent out on the 23rd of May 2020 and left open to responses throughout this research project. The last response was registered on the 13th of June 2020. In this time 88 people completed the survey. The survey was sent via Google Forms link and shared through social media channels and groups.
Due to this, it was not limited to a specific country or region. However, a majority of the respondents were situated in central Europe. The largest nationality to respond was 27.9% Germans, followed by respectively 12.8% Italians and Spanish. The largest group of nonEuropean respondents was 11.6% Indian.
59.8% of respondents were male and 39.1% female, with a further 1.1% preferring not to give information about their gender. Due to the chosen channels of outreach, the largest age group was between 26 and 35 years old consisting of 49.4% of respondents.
The secondlargest age group was between 18 and 25 years old and made up 36.8% of
answers. 43% of people answered that they mainly consume news in English. Again
43% of people said they primarily consume news in a language which is not their native
language, of which almost all of them were consuming news in English, with only 3
responses being an exception to this.
CHAPTER 4. RESULTS
4.1.2 Basic News Consumption
After having gathered basic demographic information on the respondents. the second section of the survey addresses basic news consumption and habits. Starting with finding out how frequently people consume news. Five options were given ranging from always (every day) to never. The responses can be seen in Figure 4.1.1. The vast majority of respondents said they consume news media daily or every other day, making up a total of 77.3%. With only 9.1% of people saying they never consume any news at all.
Figure 4.1.1: Frequency of news consumption
Next, the question of how people prefer to access news was asked. Five predefined options were given with a sixth ’other’ option that could be specified. Most users selected from the predefined answers however one user added Radio/Spotify to the list as can be seen in Figure 4.1.2. Digital access to news was by far the most common way to gain access to news, with a total of 63.6% of people choosing to access news through newspaper apps to websites. Websites here being preferred over apps. 8%
of people still preferred the classical means of television to consume news and 6.8%
choose to read nondigital print newspapers.
Leading on from the question of channels used to access news a more specific question
into which social media channels people have used in the past month was asked. Figure
4.1.3 shows that Youtube, Linkedin, and Instagram are the most commonly chosen
social media channels to access news. 12.9% of people do not use any form of social
media to gain access to the news.
CHAPTER 4. RESULTS
Figure 4.1.2: Channels of news consumption
Figure 4.1.3: Social media channels to access news
The news consumption questions were finalised by looking at whether respondents pay for the news they are consuming. Answers found that 78.4% of respondents never pay to get access to digital news. More than half of these would also never consider paying for news. Only 12.5% of respondents pay to gain access to digital news frequently.
4.1.3 Credibility
The second section of the questionnaire aims to assess how credible people find the
news. Starting with the question of how much people agree with the statement The
news I consume is credible. A fivepoint Likert scale ranging from strongly disagree to
CHAPTER 4. RESULTS
strongly agree was used. The most common answer was somewhat agree, followed in second place by feeling neutral towards this statement as shown in Figure 4.1.4. Most respondents, 61% find the news they are consuming to be credible.
Figure 4.1.4: Agreement with statement ’The news I consume is credible’
Taking a different approach at looking into credibility, the second question of this section asked people how often they see news online that they do not find credible.
Giving options reaching from never to always. As can be seen in Figure 4.1.5 the distribution of answers was broader with a less clear dominant answer than the previous question. Both rarely (once per week) and occasionally (once per day) having 23 responses each.
When comparing results from Figure 4.1.4 and Figure 4.1.5 a clear distinction needs to be seen in the phrasing of the two questions. One focusing on the news respondents actively consume and the other on the news they are encountering in general.
Interestingly this leads to different distributions.
Two questions were asked on how people react to or interact with news they do not find credible. The first asking what people do when they see noncredible news online.
The answering options given were multiplechoice so a variety of answers could be
selected. The predefined answering options were: ignore it, consume it in order to
gain an understanding, share it, report it, and unfollow/stop using the source. The
responses in Figure 4.1.6 show that a vast majority of people choose to ignore news
that they do not find credible. The second most common reaction is to unfollow
CHAPTER 4. RESULTS
Figure 4.1.5: Frequency of seeing noncredible news
or stop using the source. Where one respondee added that he would only do so if the source frequently published news he or she did not find to be credible, through using the ’other’ option. Only 8% of respondents said they would actively report the misinformation. 3 respondents used the ’other’ option to mention they would try to verify the credibility of the news through other sources.
Figure 4.1.6: Reaction to noncredible news
The survey section on credibility was completed by the question: How often do
you verify the news media you consume online through other sources? Again
giving respondents five answering options reaching from never to always. The
results in Figure 4.1.7 show the largest concentration of answers around rarely and
CHAPTER 4. RESULTS occasionally.
Figure 4.1.7: Frequency of news verification through other sources
4.1.4 Interaction
The previous survey section already started the introduction of the topic of interaction.
However, this section aimed not only to look at user interactions but also how they are perceived by the user. Trying to gain an understanding of whether some interactions are seen as more active or engaging than others. Starting with two questions asking people whether reading a news article is a passive activity and whether watching a news video is a passive activity. Answering options ranged from strongly disagree to strongly agree. The answers in Figure 4.1.8 show that overall watching a news video is seen as a more passive activity. Reading a news article is however seen as a more active interaction.
The final questions of the survey focused on how often people used certain interactions that are available on many digital news platforms such as liking, sharing, and commenting. All three questions’ answers are summarized in Figure 4.1.9 to see how they compare. Overall, it can be said that sharing and liking are much more frequently reported interactions than commenting. With sharing being slightly more common than liking. 67% of respondents said they never comment on news articles or videos.
The only frequent interaction users reported was liking.
CHAPTER 4. RESULTS
Figure 4.1.8: Reading a news article vs. watching a news video as a passive activity
4.1.5 Associations between Credibility and News Consumption Habits
Survey answers from questions such as how frequently people consume news, how frequently they share/like/comment on news, were analysed in relation to how credible they find the news they consume. This was done to gain a deeper understanding of the impact of interactions and habits on credibility. Contingency tables were created comparing the two answers in focus. Fisher’s exact test was conducted with these contingency tables, to find associations between survey responses. The resulting contingency tables, graphs, and Pvalues are elaborated in the following sections.
Association between Frequency of News Consumption and Credibility
To begin the association analysis answers of how frequently people consume news were analysed in conjunction with how credible they find the news they are consuming. For this, a contingency table containing responses to both questions was created. The initial table can be seen in Table 4.1.1. When looking at the contingency table that comes directly from the survey results of rated credibility of news and frequency of news consumption we find that multiple entries have values below 5.
These small sample sizes make chisquared distributions less well approximated ([35]).
The data was grouped in order for less entries to be below 5 ([15]). Instead of using the
five states of agreement, strongly agree and somewhat agree were merged to form
CHAPTER 4. RESULTS
Figure 4.1.9: Frequency of liking, sharing, commenting and creating content
Table 4.1.1: Association News Consumption Frequency and Credibility 5x6
agree. The same was done for strongly disagree and somewhat disagree, forming disagree. This resulted in three categories: agree, neutral and disagree. For the questions of how frequently users consumed news never and very rarely, rarely and occasionally, and frequently and always were grouped. Ultimately, this resulted in a 3x3 contingency table that can be seen in Table 4.1.2. As there were still values below 5 in the contingency table Fisher’s exact test was applied. The exact test and calculation of the Pvalue were carried out using a VassarStats
1tool. From this, two Pvalues are calculated which have been elaborated on earlier. Here we will focus on P
A.
Fisher’s exact test postulates a null hypothesis. In this case, being ‘frequency of news consumption has no impact on perceived credibility’. Therefore we test whether the frequency of news consumption has an impact on credibility. The calculated Pvalue shows us the probability of getting a difference of the same size or bigger than the one
1http://vassarstats.net/fisher3x3.html
