Indikatorer på forskningens inverkan på samhället genom Twitter-interaktioner: En forskares guide till spridningspraxis

(1)

MASTERUPPSATS I BIBLIOTEKS- OCH INFORMATIONSVETENSKAP AKADEMIN FÖR BIBLIOTEK, INFORMATION, PEDAGOGIK OCH IT

Indicators of societal impact of research through Twitter interactions

A researcher’s guide to dissemination practices

ANTONELLA FODERARO

©Antonella Foderaro

Mångfaldigande och spridande av innehållet

i denna uppsats – helt eller delvis – är förbjudet utan medgivande.

(2)

Svensk titel: Indikatorer på forskningens inverkan på samhället genom Twitter-interaktioner: En forskares guide till spridningspraxis

Engelsk titel: Indicators of societal impact of research through Twitter interactions: A researcher’s guide to dissemination practices

Författare: Antonella Foderaro

Färdigställt: 2020

Abstract: Research is usually evaluated through its input, output and impact. While the latter is very expensive in terms of time (a paper or patent publication can take years to be published), it can easily be measured through journals metrics, citations, grants, and conference presentations. Societal impact of research, on the contrary, is the most difficult concept to capture and that is why, in the past, it was considered only in terms of effect within the scientific community. Nowadays, the ubiquitous social media platforms have been increasingly studied in search of new indicators for measuring the immediate impact of research. In this study I explore linking practices and reactions to scientific output as they are posted on Twitter. Through the qualitative analysis of one Twitter conversation dy- namic about a topic of public interest and the comparison with quantitative analysis of individual tweets on similar topics, I searched for what makes a tweet about research more discussed within the scientific community and what makes it more successful in terms of societal impact by arousing citizens’ awareness, increasing their critical thinking and promot- ing their engagement. The results showed that engaging in conversations, specially in argumentation, increase the public benefits of science.

Nyckelord: forskning spridningspraxis, argumentation- steori, Twitterkonversation, forskningens inverkan

(3)

List of Tables

5.1 Cardiovascular . . . 32

5.2 Cardiovascular: Academic articles - Science news . . . 32

5.3 Environmental health. . . 34

5.4 Environmental health: Academic articles - Science news . . . 34

5.5 Glioblastoma . . . 36

5.6 Glioblastoma: Academic articles - Science news . . . 36

5.7 Climate change - Tweets type . . . 39

5.8 Climate change: Tweet type-> Tweet reply. . . 40

5.9 Climate change: Interaction type - Argument type . . . 40

5.10 Climate change: Source Type . . . 41

List of Figures

3.1 Toulmin’s Scheme . . . 15

(6)

Chapter 1

Introduction

Research is usually evaluated through its input, output and impact. The first consists on the resources invested to support scientific activities, the second is based on the results in term of novel knowledge (articles and patents), and the latter is based on how this knowledge affect the academic community and the society (Sugimoto and Larivière, 2018).

While the impact in the academic community is very expensive in terms of time (a paper or patent publication can take years to be published), it can easily be measured through journals metrics, citations, grants, and conference presentations. Societal impact of research, on the contrary, is the most difficult concept to capture and that is why, in the past, impact was considered only in terms of effect within the scientific community (Sugimoto and Larivière, 2018).

This work contains a study of users’ dissemination practices specifically related to science artifacts on Twitter. The main aim is to understand which kind of interaction and sources generate citizens participation. Societal impact of research is usually intended as the intrinsic value of science in terms of impact in the scientific community (through citations) and industry (patents), for example the discover of a vaccine. This work takes a different approach considering societal impact in a broader sense focusing on the recognition and acceptance of this value by citizens at large (Cassidy, 2019).

1.1 The research tradition

Give a definition of information science is not simple because there is a lack of agreement about it. According to Bawden and Robinson (2015), there are three main answers to this question. One considers information science as strictly related to computing, algorithms and technologies; a second highlights the relationship with entropy in information theory and physics; a third considers it as recorded in documents.

These answers are often overlapping in the many different information science fields and if a lack of an unique definition can be seen as a weakness in theory, it often results in

(7)

a strength when it comes to practice (Nolin and Åström, 2010). Scientometrics, in fact, is a field dedicated to analyze a wide range of data sources, employing also algorithms.

It allows research measurement and to build science indicators.

Traditionally, citations are one of the main indicator used to measure research, but they are increasingly been criticized for not giving the overall picture of impact. In 2010, this approach was challenged by Jason Priem who launched the term altmetrics with the purpose to introduce new measurements of research complementing citations (Sugimoto and Larivière, 2018).

With the emergence of Internet’s platforms a new kind of public sphere for discussion was created that did not exist before becoming an unprecedented opportunity for scientists to study human behavior and to assess research (Gunnarsson Lorentzen, 2016a).

Considering Twitter a significant platform in this contest, this study will focus on the activities and interactions on it. The purpose is to sketch a sort of researchers’ guide to dissemination practices on Twitter, that reaches both citizens and the scientific community.

1.2 Research problem

Nowadays, the ubiquitous social media platforms have been increasingly studied in search of new indicators for measuring the immediate impact of research and to try to differentiate works with high scientific value but low immediate societal impact, from researches with high scientific value and immediate societal benefits.

These studies however, do not take into account societal impact from the perspective of citizens in terms of science recognition and acceptance, and generally they do not consider data collected from Twitter conversations (Gunnarsson Lorentzen, 2016a).

The use of social media to share information in order to increase research impact, awareness about a topic and promote public support for a cause or policy grows hand in hand with the parallel but opposite intent to spread false news, support ideologies and to influence citizens’ choices. Therefore is important to address assessment of societal impact of research on social media platforms such Twitter, from a more critical, constructive, complete and comprehensive perspective.

In this work I study how science is shared and discussed on Twitter in order to see what kind of dissemination practice can be considered and adopted as a robust indicator of societal impact in the meaning previously explained.

1.3 Aim and research questions

The main objective of this study therefore is to explore linking practices and reactions to scientific output as they are posted on Twitter. More specifically, I search for indicators of societal impact through the interactions between researchers and the general public

(8)

regarding tweets referring to research. To achieve this goal, one Twitter conversation (509 tweets) about climate change and 3 samples of 300 single tweets about Public health will be studied with a mixed approach of qualitative (for the conversation) and quantitative content analysis (for single tweets).

The choice of the above-mentioned topics is motivated by the fact that they are of shared interest by both the communities. The main research questions are:

RQ1: What kind of web based research artifacts are referred to in Twitter discussions?

RQ2: How do interactions in these discussions play out with regard to type of research artifacts and how they are used in tweets?

RQ3: What characterize the research artifacts linked to, with regard to type of sources and how they are used in tweets?

Where a source is defined as any kind of web based resource that can be used for backing up a claim, such as a research article, a science news article, blog post, embedded chart, video, picture, etc.

Depending on the results obtained by answering these research questions, Twitter could be an usable tool not only to measure societal impact of research within the scientific community, but also to measure science impact on the public(s) (Cassidy, 2019), building a bridge between the two communities. This would be a strong incentive for:

• scientists to engage themselves in discussing research in social media instead than merely redistributing it

• practitioners and journalists to participate actively in research dissemination

• citizens to regain trust in science often hampered by the ideological use of social media, having at the same time more critical instruments to distinguish between different intents in sharing information and between opinions and facts.

1.4 Relevance for research and society

The target audience for my thesis is primarily formed by scientists, interested for example in: research assessment; interaction dynamics in social media; the topic discussed;

argumentation analysis; but also by policy makers, librarians, journalists, politicians, students, citizens, interested for example in founding reliable sources about the different topics discussed, in how science dissemination can be improved or what is the public/scientific opinion about such topics, etc.

(9)

Reading this study the audience will be aware about how dissemination of research in social media is important for public health and how this process can be more successful using:

• an understandable language to communicate with a non scientific audience (Horst et al., 2017)

• discussion instead than merely redistribution, regardless of the typology of the audience

• reliable, clear and understandable sources for sustaining or defending one’s claims.

Even if there is no simple causal relation between knowledge about science and being positive towards it (Horst et al., 2017), increasing citizens awareness about science methods, processes and practices, is vital at least for two reasons: the first is related to the survival of the research itself, since what has the most impact on society is considered more relevant and gets more funds; the second is that science must be able to speak and be understood by people to receive the necessary recognition, i.e. scientific research requires public trust (Brembs et al., 2013).

Nowadays is trust in science what can really do the difference in fighting ignorance and disinformation, therefore how it is disseminated through social media, but also by using other channels, should not be underestimated. The most effective strategy in this direction seems to be struggling to improve analytical thinking skills in the audience, in fact, people who are more used in exercising logical/critical thinking are less likely to accept conspiracy theories (Swami et al., 2014). One could claim that this is not what societal impact of research should measure, but how can science reach the society in a democratic world, if not through free recognition and acceptance?

(10)

Chapter 2

Background and Related Works

According to Boyd and Ellison (2007) social network sites are web-based services allowing people to:

(1) construct a public or semi-public profile within a bounded system, (2) articulate a list of other users with whom they share a connection, and (3) view and traverse their list of connections and those made by others within the system (p. 2).

This social network view is crucial for the future of research dissemination and impact, if integrated with that of knowledge networking, where people, resources and relation- ships, are assembled and shared for the purpose of creating value. In this perspective, creation and sharing processes must be encouraged rather than just accumulation of data (Seufert et al., 1999).

Even if the above-mentioned framework is actually created in order to integrate social networking and knowledge management in firms and therefore has a hierarchical structure, it offers many interesting inputs for this study. In fact, connectivity has always been one of the keys for creating new knowledge and could also be an effective tactic for improving research dissemination and impact.

Scientists are in all senses knowledge actors who create and transfer research not only between peers but also organizations and citizens. As scientists, they use Twitter as a tool to share and exchange information, create new collaboration, stay update and get inspired (Priem and Costello, 2010; Darling et al., 2013). They participate in discussion about research creating new content or simply redistributing it.

According to Kwak et al. (2010) Twitter is a powerful communication channel that consent to spread information very fast and to a large audience. Retweeting in fact reaches an average of 1, 000 users no matter what was the number of followers the original tweet had. This number is probably larger today with more accounts on the platform. Twitter has therefore the potential to be studied in multiple ways and one of them is, for example, considering it as an intermediary or as a mediator (Van Dijck,

(11)

2013). In the first case the participants play an active role and the tool is therefore an intermediary between them, while in the second case the role of the users is more passive and the tool influences social connections, relevance of information, etc.

2.1 Scholarly use of Twitter

Internet offers many novel, friendly and social ways to stay update on topics of personal and public interest. One of the most prevalent in everyday life is Social Media which has become an effective tool to disseminate information in real time (Thelwall et al., 2013b).

Social Media platforms allow users to create a public or private "profile" with authen- tic or invented names, to describe themselves, exchanging different kinds of messages and create or sharing content with other users or groups they are connected to. Scientists use different kind of Internet services to remain update about their research fields, such email alerts or register keyword in scholarly databases and they are, together with practitioners, undergraduates and the interested public, one of the more common typology of users that reads, and more likely shares, scholarly publications (Kurtz and Bollen, 2010).

Despite many researches have been conducted about scholarly uses of Twitter in different disciplines and contexts (Thelwall et al., 2013b; Holmberg and Thelwall, 2014;

Nelhans and Gunnarsson Lorentzen, 2016; Ricoy and Feliz, 2016), transforming this digital platform in an interesting source for research assessment (Thelwall et al., 2013a;

Liu and Adie, 2013; Bornmann, 2014), no studies have focused so far in detecting a dissemination strategy that benefit both the communities.

More specifically, there is an important gap between research argumentation and dissemination within the scientific community and the society. A recent study conducted by Díaz-Faes et al. (2019) tries to draw a picture of Twitter users interactions with research objects collected from the Altmetric.com database, identifying four dimensions such science engagement, social media capital, social media activity and science focus.

Their results, based on the analysis of scholarly outputs and the overall social media activity of scholarly agents, such for example number of tweets and mentions of scholars, confirm that impact cannot be measured only in term of citations, but must include other factors such "societal, environmental, political, health, and economic impact" (p. 14).

Therefore, searching for meaningful interactions from tweeters with high social media capital belonging to different communities, can be a better path to follow.

This demonstrates the urgency in the field in searching for a new set of indicators for assessing research impact on Twitter, focusing more on what characterize different communities and their interactions around scientific outputs rather than only number of shares (tweets and retweets).

However, If I look for example at colleges’ and universities’ practices on Twitter or at the use of the tool by hospital and other health institutions, it seems that these primarily prefer individual tweets and employ them as a news feed to a general audience with a scarce conversational intent (Linvill et al., 2012).

(12)

Scholarly communication, which is traditionally categorized as formal and informal communication (Barjak, 2006), should ideally (Shehata et al., 2015):

serve to inform other researchers about a research project, promote current research results among the scholarly community and inform a non-academic audience about the important results that were produced from scientific research (p. 429).

A recently approach offered by Hatchard et al. (2019) show that researchers are not alone in practicing scholarly communication on social media. The authors, through the analysis of linking practices, give a different perspective about the usage of Twitter by another important category of users sharing scientific content in order to promote their agenda, i.e. polices proponents. This is relevant for assessing societal impact of research because by sharing scientific evidences for sustaining one’s claim, policy advocates try to influence decisions that have a big impact on society, such in this case tobacco policies.

2.2 How to approach Twitter

The perspective chosen in this thesis is to consider Twitter as a mediator, i.e as a digital platform "which shapes the performance of social acts" (Van Dijck, 2013). From this point of view Twitter is not merely a facilitator of social acts, rather a computational architecture through which this activity is expressed. Two main approaches in collecting data from Twitter have been used in this study: the first, hashtag oriented, in which one or several hashtags are tracked in order to choose topics of interest for both the communities involved; the second, user oriented, in which a set of users are followed in order to capture the conversations dynamics. In both cases, random samples with high intensity of linked sources were selected in order to answer the research questions.

According to Gunnarsson Lorentzen (2016a), one of the problems with Twitter research, is the exclusion of follow-on conversation which prevents the collection of complete data able to answer complex research questions. In order to resolve this issue a method has been developed by Gunnarsson Lorentzen and Nolin (2017), which allows to capture and therefore to study and analyze a data set substantially more complete than previous methods. By collecting and analyzing follow-on conversations around a set of hashtags, the authors offer an effective framework for approaching and analyzing conversation dynamics and their life cycle. In this study, I used this framework, also adopted by Data4Impact consortium (Feidenheimer et al., 2018) which is the EU project providing the data for this work.

Nelhans and Gunnarsson Lorentzen (2016) findings are particularly relevant for my project because the presence of conversations in their data collection allows to compare the results obtained in this study and to draw robust conclusions. According to their results, considering only the number of users tweeting or retweeting a publication as an indicator for impact can be problematic, because there is a difference between the most

(13)

retweeted items and the most discussed ones in a conversation context. For this reason they propose

that an impact metric should be extended to include measures of visibility, spreadability and the ability to spark discussion to represent Twitter activity around scholarly work to be truly meaningful (p. 27).

In their qualitative content analysis, they found also that references to research articles were mainly used for self-promotion purpose, as conversation starter or as arguments in a discussion. If judging scholarly impact through Twitter data and conversations should be used with caution, the researchers agree that this process might be relevant for capturing social impact. Collecting conversations makes it possible for researchers to analyze reactions to scientific artifacts, accessing research impact from the perspective of a non- academic public.

In order to reach a non-academic audience, an interesting approach could be to create

"a collaboratory as a virtual learning community" (p. 265) as proposed by Ponti (2008).

She discusses the importance of collaboration between library and information science (LIS) researchers and practitioners in order to improve and integrate knowledge, exper- tise and expand participation of practitioners in research projects. This collaborative approach could be extended to online scientific communication on Twitter where researchers, librarians, practitioners discuss research with the final purpose of addressing

"the public(s)" (Cassidy, 2019).

2.2.1 Twitter: social media or information mediator?

Twitter allows users to create short statements named "tweets" limited to 280 characters and to share links to external web based artifacts to which I refer often in this thesis by using the term (re)sources. People can be able to read these updates by following each other or by creating a hashtag, which is a kind of metadata tag helping others to find content about a specif topic.

On Twitter, users do not need to present themselves and therefore, according to Hu- berman et al. (2008), this site focuses more on what you have to say than in who you are. Anonymity reduces social pressure and suggests that users may have different reasons, age and personality, for preferring Twitter than other social media, such for example informational purposes, older age, and need for cognition instead than socialization (Hughes et al., 2012):

For instance, information sought from Facebook may be obtained socially (i.e. by asking other users), whereas the information sought on Twitter might be more cognitively based, such as academic or political information that is best gained by reading source materials, for which links are often "tweeted". Equally, the correlations with Conscientiousness suggest

(14)

that informational use of Twitter may be goal-directed, perhaps seeking information relevant to work or study; whereas for Facebook, information seeking may be the manifestation of procrastination (p. 19).

This is relevant for this study because it highlights how Twitter is used more as an information mediator, than as a social media. Moreover discussing research in conversations and not just retweeting sources, allows even passive users (readers) to benefit of this digital platform as an instrument for improve their critical thinking by comparing their opinion whit facts or different points of view. In this context it is essential to keep in mind that as a mediator this tool necessarily exercises its influence, that is, it is never neutral (Bodle, 2010; Marres, 2015).

According to Boyd (2010), networked technologies can play a powerful role in con- trolling information and configuring interactions, through for example, putting in relevance popular users or topics and collapsing contexts. That is why, among other mo- tivations such need for completeness and interest in interaction dynamics, I choose to compare individual tweets with conversational tweets, because in the latter I can study them in their conversational contexts. This did not resolve, however, another very relevant problem which Boyd highlights, i.e. the problem that arises from the interaction with invisible audiences. In fact,

Not all audiences are visible when a person is contributing online, nor are they necessarily co-present (p. 49).

and this complicates further the communication because users do not know if the language adopted or the shared links used for sustaining their claims, are understandable for who is reading/listening or will later retrieve and reproduce the content.

2.3 Twitter and research assessment

Scientometrics is the field of study dedicated to analyze a wide range of data sources and allow for research measurement and build science indicators. As Sugimoto and Larivière (2018) explain, citations have been long criticized for not giving the overall picture of impact.

Measuring the impact of science on economy and society is a contemporary problem that requires new and more comprehensive indicators. An indicator is valid if it repre- sents faithfully the concept linked to it, and therefore, indicators based on social media data should follow the social media dynamics, e.g., being updated in real time (Sugimoto and Larivière, 2018).

Liu and Adie (2013) highlight how the use of digital scholarly communications tools is helping Altmetrics to offer a more complete approach in measuring the immediate societal impact of research. By aggregating online interactions created around individual

(15)

scholarly articles, Altmetrics assesses the impact of research within a particular community, the overall impact of a body of scholarly work, and the profile of the communities that discuss articles online.

Even though attention and popularity does not necessarily mean quality of research, Altmetrics qualitative approach is a remarkable attempt to access and display a rich and, until now, unofficial world of scholarly communication and collaborations. Furthermore, it provides a measure of economic and societal impacts of research in almost real time.

The latter was previously impossible due to the long time process through which scientific peer-reviewed articles and patents must pass to be reviewed, published, quoted and, in case of patents, eventually utilized.

As Nelhans and Gunnarsson Lorentzen (2016) and Gunnarsson Lorentzen et al. (2019) demonstrate, the collection and analysis of the data used by Altmetrics in order to create news indicators, should include not only Twitter mentions or single tweets, which can be easily multiplied by bots, but also "conversations as units of analysis". By including them, the impact of research can be measured through the number and the types of interactions that the discussion creates.

Shared interest and participation in discussing a peculiar article or issue cannot be easily manipulated (Marres, 2015), while popularity can be promoted by algorithms, meaningful interactions cannot be simulated. A conversation thread can be seen as a chain of tweets connected as replies to previous tweets all related to a specific topic (Gunnarsson Lorentzen and Nolin, 2017). By collecting conversations instead of just tweets matching hashtags or keywords, the quality and depth of the analysis increase.

If I look for example at similar studies conducted on public health topics through the analysis of single tweets only, it is clear that the results obtained through this method are either reliable and robust (Nambisan et al., 2015) or complete (Paul and Dredze, 2011).

Moreover, many studies conducted in order to evaluate the reliability of Twitter mentions for research assessment (as I mentioned previously in this chapter), demonstrate that tweets containing links to scholarly articles usually provide (Thelwall et al., 2013b):

little more than publicity, and so whilst tweet counts may provide evidence of the popularity of an article, the contents of the tweets themselves are un- likely to give deep insights into scientists’ reactions to publications, except perhaps in special cases (p. 1).

This is due, according to the authors, to the fact that the content accompanying the research object is mainly an echo of the article’s title or a brief summary of it created with the purpose of making the linked article more comprehensible for a wider audience.

The purpose in sharing a scientific paper is rarely critical and the presence of other scholars mentions is reducible to the authors of the paper being these tweets, in most cases, self-citations.

In the contest of scholarly communication, it is also interesting to study the difference usage and linking practices on Twitter depending on the scholarly discipline of belonging. Holmberg and Thelwall (2014) findings show, for example, that biochemists retweet

(16)

more than researchers in the other disciplines, scientists in digital humanities and cognitive science participate more in conversations, while scholars in economics share more links. The lack of conversational purpose in scholarly communication is evident in their results, but it seems that the intent in sharing scientific content on Twitter is generally popularizing science. This is demonstrate by the linking practices through which the users share a variety of content such science blogs and articles in news sites and popular science magazines.

These results are also confirmed by Ke et al. (2017) who show that researchers linking practices are almost the same as the general population (using YouTube, Instagram, Facebook, mainstream media, etc.)

However, scientists also have a distinct imprint of scholarly sites, such as generalists publications (i.e., Nature and Science) [...]. The popular pre- print server, arXiv, also occupies a prominent spot among the top 20 cited domains (p. 14).

Therefore the content shared by scientists can be highly heterogeneous, depending on professional or personal usage of the tool.

The lack of conversational purpose in scholarly communication on Twitter has been generally considered by all the above-mentioned authors as a practice of "unbearable emptiness" quoting Robinson-García et al. (2017), meaning that tweeting and retweeting scientific content without active engagement in discussing it, does not improve impact and cannot be considered a meaningful and reliable indicator in measuring it.

These considerations and results have guided my choices related to the theoretical framework and to the design of the method being very useful for the interpretation and discussion of the results obtained.

2.4 "Thinking through public(s)"

As Cassidy (2019) highlights in the Introduction of her review about how science changed its practice (in UK) in order to reach the public(s):

It is increasingly difficult to function in 21st century academia without tak- ing account of (or at least noticing) that small word for a big thing - public - a word in the unusual position of being used as a noun and an adjective, as well as a singular, plural and collective term (p. 1).

The relation between science and society is not a new problem, but after the publication of The Public Understanding of Science (Council, 1985) scientists had started to progressively rethink about the importance to reach a broader audience. The shift from

"the public" to "the publics" expresses, according to Cassidy, the way in which this concept is now thought. In fact, "the public" sounded more as an empty vessel to be filled

(17)

(Cooter and Pumphrey, 1994), while the term "the publics" takes in to account a rich variety of factors, such gender, ethnicity and class, social identities, personal experiences, putting all those factors in relation to the scientific topic.

This need to be understood has generate an increasing researchers engagement in changing academic practices (Packman et al., 2017), which includes a working in progress mixed strategy of collaboration and co-production activities, adapted and adopted according to the publics and the researchers involved (Burchell et al., 2017).

This approach, however, must take into account a peculiar characteristic of the concept the publics, i.e. that it is contingent, constantly changing and because it reflects the human diversity, it is also contradictory. Moreover, when it comes to communication, there is always a difference between the real publics and the imagined, what Litt (2012) calls the imagined audience: i.e. "the mental conceptualization of the people with whom we are communicating" (p. 331).

Communication in science changes not only when it is in public, official and presented within the scientific community compared to when it is shared in private, in its making, with colleagues, but also when it is addressing the publics. In this latter case, researchers need to have in mind also the broader meaning of the public as "the greater good or collective benefits" and interest (Cassidy, 2019). How every scientist in his field could and should engage himself/herself with the public(s) is a question not to underestimate, but how this engagement can be more effective in on-line communication, such on Twitter, is a complex and exciting challenge that I want to deepen in this study.

(18)

Chapter 3

Research Framework

Twitter is fundamentally a digital platform, a networking site, a mediator (never neutral) rather than an intermediary (Van Dijck, 2013). Researchers need to be aware of all these aspects when approaching it using an accurate critical framework that consent to inter- pret interactions, practices and the role played by networks. Therefore, relevant aspects of argumentation theory and analysis, linking practices and social network studies are included in this chapter.

In this study, the concept of interaction between users and their links sharing practices are central, because the perspective through which I will analyze the conversation thread is argumentative, and the approach for studying the linked sources is through their effectiveness in reaching the audience.

Since the same source is often used by different parts involved in the discussion, both for sustaining and criticizing a claim, dissemination based on single tweets could be not effective for reaching the target audience (both communities) and improve citizens awareness and critical thinking.

3.1 Argumentation theory and Toulmin’s schema

According to Van Eemeren et al. (2013), argumentation is a verbal and social activity consisting in speaking or writing sentences aiming at denying or justifying something, in order to persuade a present or possible audience. It is also an activity of reason in which emotions can be involved. The arguer attempts to show that there is a rational warranty which justify his/her position.

Argumentation starts when a standpoint, thesis or claim is perceived as controver- sial, creating one or more pro-arguments or one or more contra-arguments, for increasing/decreasing the rational acceptability of the claim. The goal of argumentation anal- ysisis to observe and document how statements are structured within a discursive text, and to assess their soundness (Eemeren et al., 1997).

In the book The use of Argument, Stephen Toulmin (1958) claims that arguments need

(19)

more than premises and conclusions to be considered. He proposes a more complex scheme which includes, beyond the traditional data and claims taken from formal logic, also other elements such as warrant, backing, rebuttal and qualifier (Fig. 3.1).

Figure 3.1: Example of additional elements in Toulmin. My illustration.

Premises/grounds or data consist of facts that support the claim. The warrant is an inference according to which the premises support the claim. The backing provides reasons supporting the warrant. Finally, the rebuttal highlights conditions of exception for the validity of the claim, while a qualifier expresses a degree of force that the data give to the claim by the warrant (Toulmin, 2003).

Toulmin’s scheme has had a continuing influence on argumentation studies because the emphasis placed on (Verheij, 2005):

• the argumentation’s context, i.e. warrants and their backings can differ from domain to domain;

• arguments can be subject to rebuttal, i.e. there can be cases of exception;

• arguments can have qualified conclusions;

• determining whether an argument is good or not involves not only formal but also substantive judgments.

When approaching argumentation theory in online interaction however, it is also very important to have in mind Herring and Androutsopoulos (2015) considerations about Computer-mediated discourse (CMD) and Computer-mediated communication (CMC):

In CMD, meaning is constituted and negotiated almost entirely through verbal discourse. This is especially true in textual CMD, in which context cues are reduced relative to face-to-face communication. [...] CMD users

(20)

also incorporate outside context to create meaning by paraphrasing, quoting, retweeting, or linking to other texts elsewhere on the Web (pp. 133, 136).

Intertextual reference strategyis expression of shared premises or beliefs (culture), and this helps to better understand the broad use and the usage of external sources linked to the tweets in online argumentation. These considerations lead to the next section, where three of the main traditional approaches to argumentation are presented and where it will become clear why Toulmin’s schema plays a crucial role when it comes to arguments formulation and evaluation both in CMC and in face to face interactions.

3.1.1 Traditional scholarly approaches

There are three traditional different scholarly approaches on argument: the rhetorical which focus on the process of persuading, the dialectical which examines the pro- cedures/methods/rules followed in arguing, and the logical which takes in to account premises and conclusions and their logical soundness (Ehninger and Brockriede, 2008).

These three standpoints are distinguished in terms of purpose and situation. The purpose consents to answer questions about effectiveness (rhetorical), rigor (dialectical) and soundness (logical) of the argument while the situation or context is real in the rhetorical approach, contrived in the dialectical and determined by the field of inquiry, in the logical approach (Benoit and Hample, 2012).

Because of these three different approaches, what makes an argument good or bad often depends on the perspective by which it is judged. In fact, whereas the logical approach prefers direct evaluation putting oneself as intended audience and asking whether or not the argument is cogent, the rhetorical and dialectical approaches tends to use in- direct evaluation, considering the intended audience as evaluator.

Logicians consider arguments as products and evaluate them according to certain cri- teria such as acceptability and relevance of premises and sufficiency to support the conclusion, while dialectic focuses on procedure - such people have to take turns when arguing, must consider seriously others’ reasons, and have to seek a rational outcome rather than a convenient one (Ehninger and Brockriede, 2008) - in order to regulate argumentative communication. Rhetoricians, finally, are concerned with arguments as a process whose ultimate goal is persuasion and therefore the argument is good if it is effective (Govier, 1999).

From which perspective and context the analysis of arguments is taken become therefore crucial for controversy studies which have as a goal to understand how debates end in a scientific context (Sismondo, 2011) and outside of it (Mazur, 1987; McMullin, 1987;

Gunnarsson Lorentzen, 2016b). In fact, as Gunnarsson Lorentzen (2016b) explains in his study about political controversies, presenting the case of Sweden’s December Agree- ment (2014), during these debates, participants are trying to find and expose potential weaknesses in the arguments of the opponents (Mazur, 1987) in order to persuade the

(21)

audience. In this context, the perspective is more rhetorical than logical or dialectical, and the conversation ends often with lack of agreement. Termination of a debate, in fact, can be achieved in different ways: resolution, in which agreement is achieved, closure, where the conversation ends with lack of agreement, and abandonment, where the participants lose their interest on it (McMullin, 1987).

In a political conversation, but also in many other scenarios where the different parts often belong to opposite ideological positions, closure is the normal way to terminate a debate, because the audience do not have common premises and do not share the same beliefs. In scientific conversation instead, resolution or abandonment, can be more common, if the discussion is within scientists who belong to the same domain, as Toulmin (2003) highlights. These considerations leads us from the importance of the context, to the role of the audience, which comes in the next section.

3.1.2 Argumentation analysis and the audience’s role

In the past, Aristotele’s syllogism and his general theory of reasoning was fundamental in argument evaluation because argumentation’s theories considered it good if the logical structure behind it followed the rules of logic (Bauer and Gaskell, 2000). Nowadays, the influence of mass media and the advent of internet, widened the audience involving not just scholars but the broad public.

Because argumentation is a human and social process, the concept of audience is essential, in fact an argument cannot be judged only by applying objective standards, but the audience, either particular or universal, plays the role of ultimate arbitrator (Benoit and Hample, 2012). Therefore, the focus is now shifting from the quality of the argument itself or its logical structure, to the quality of the audience (Blair and Johnson, 1987).

Even if this assumption taken alone can lead to relativism, posed in the context of this study, gives an important input in order to understand the interaction in social networks like Twitter, where the audience involved in discussing scientific topics should not belong entirely to the scholar community.

The importance of audience’s role in online argumentation cannot be underestimated because it heavily influences the purpose by which argumentation is used on social network. In fact, the larger is the audience, the harder become to find premises that such audience is prepared to accept, because argumentation seeks to persuade others to accept conclusions on the basis of premises they already consider valid.

In social platforms like Twitter the audience is usually the group of people who follows the user, but through retweet, this audience can be significantly broadened, becoming a mass. An effect of being exposed to a mass audience is basically to transform argumentation in fragmented interactions because the context of the conversation is lost to the audience (Boyd, 2010). This happens because Twitter does not immediately expose the audience to all the questions/answers in the conversation, and partially because the users becoming a mass, cannot share common premises.

(22)

Fragmented interactions is a problem with which Gunnarsson Lorentzen (2016a) faced in his dissertation. The author concluded that a response from one tweeter to another was shown only to the one who followed both, thus the audience became narrower when it comes to conversation:

Retweeting passes content on to a wider audience, which means that the number of people that will see the tweet increases. Contrary to this, replying restricts the content to smaller audiences, since only a specific subset of the followers of both the replier and the user replied-to will see the reply.

The fewer the joint followers of the two interacting users, the smaller the audience exposed to the conversation (p. 62).

Nowadays, things are not changed as we can read in Twitter documentation, in fact only relevant people, such as those who follow the person who replied and the person in the conversation, can see the reply in their timeline. However, when Twitter went over 280 characters, it also included the possibility to exclude hashtags, mentions and URLs from the count. As a result, this allowed replies to be more visible if the users were explicitly "tagged in" discussion. Therefore, retweets broaden the audience in, while the size of the conversation can be affected by a response within the "tagged" audience. This could partially explain why some comments remain unanswered and some arguments end without none of the contending parties being persuaded.

As underlined by James B. Freeman in the Critical Review of Govier (1999), it is not essential to an argument that it ends with winners and losers. In fact argumentation consists essentially in giving reasons and evidence for claims, all the more if they are perceived as questionable:

Eliminating argument would eliminate the possibility of our examining the reasons for our views and leave us vulnerable to various propagandistic strategies. By explicitly giving reasons rather than making implicit sug- gestions, we show respect for others and their intellectual autonomy. By trying to rationally persuade them, we need to take into account their beliefs and values, and this is to show respect for them. On the basis of the reasons given, they can judge whether or not to accept the conclusion. This is not to force anyone to accept a premise or a conclusion, and can be done without confrontational behavior or attempts at domination (p. 74).

From this point of view, it is therefore always fruitful to engage in argumentation even if it does not lead necessarily to persuasion.

3.1.3 Persuasion and the linked source’s role

Persuasion is the main purpose of rhetorical argumentation. According to Aristotele, a speech needs three elements: the speaker, the subject that is treated in the speech, and the

(23)

listener to whom the speech is addressed (Rhet. I.3, 1358a37ff.). That is why persuasion depends either on the character of the speaker; in the emotional state of the hearer, or in the argument itself (Rapp, 2010).

In the first case, the speaker has to be credible, in the second, he/she needs to arouse emotions having the power to changes the audience’s judgments and in the latter he/she must demonstrate the claim through reasoning or evidence. This purpose can be more easily achieved by involving emotions (Macagno, 2014) than by the others technical means, in fact to be scared or happy, people do not need to be trained, while to present valid arguments, knowledge on the field and efforts are needed.

On Twitter, there are different aspects that influence the credibility of the speaker such, for example, popularity (see next section), furthermore emotive language is used to obtain consensus (Macagno, 2014) and different kinds of external sources are linked to the conversations in order to increase/decrease the validity of a claim (Toulmin, 2003;

Eemeren et al., 1997).

A linked source could be the only counterargument used without any further comment. The use of sources and therefore their reliability become in online argumentation of great importance, because it can be also used, following Toulmin’s schema, as warrant, backing or rebuttal, in order to defend the advocated position and to persuade the audience.

The use of external sources in online interactions increases difficulty in active participation because requires that the audience involved in it has the information liter- acy needed in order to distinguish between reliable sources and "junk news" (Venturini, 2019). Moreover, if the source is scientific, some knowledge in Primary Scientific Liter- ature (PSL) is required in order to understand it (Koeneman et al., 2013).

From this, it follows that the kind of shared link narrows or widens the audience making participation more or less active and more or less specialized. Furthermore, depending on the intended audience, the linked source should be adapted as well as the scholarly approach on the argument.

3.2 Linking practices

As already underlined in section 1.1, there is always a certain degree of ambiguity and therefore a theoretical limitation, when it comes to define concepts in information science, because its novelty and multidisciplinary. As Pilerot (2012) accurately explains:

The meaning of a concept to a great extent depends on the practice it is used in. [...] for a language-game in which the central concepts are, for example,

"networks", "actors" and "ties", the concept of "sharing" becomes closely related to concepts such as "information flow", "dissemination", "information transfer" etc. Accordingly, the way something is conceptualised in an academic, research oriented text depends, at least partly, on how the object of conceptualisation is approached theoretically (p. 9).

(24)

What could constitute the unifying dimensions for the study of information sharing, might be therefore people’s doings and sayings, their common interests, goals or experiences, but also the use of shared tools and the engagement in similar tasks (Talja and Hansen, 2006). According to the research conducted by Pilerot (2012), 6 types of theoretical frameworks are been identified in studies of information sharing:

• Network analysis, with focus on flow, actors, and ties. Dominating concepts: information transfer, flow and exchange.

• Common ground, whit focus on common interests and mutual beliefs. Dominating concept: information sharing.

• Information ground, with focus on physical co-presence and opportunities for formal and informal information sharing. Dominating concept: information sharing

• Small worlds, with focus on common interests, shared norms and co-creation of meaning. Dominating concepts: information sharing and exchange.

• Social capital, with focus on motivation, incentives and reward systems. Domi- nating concepts: information exchange and sharing.

• Practice theories, with focus on interconnectedness, people, activities and mate- rial conditions. Dominating concept: information sharing.

In my study I should use as a mainly approach to analyze users practices in linking sources, the one that Pilerot identified as Small worlds supported by Huotari and Chatman (2001) and Jaeger and Burnett (2005), in which the focus is common interests, shared norms and co-creation of meaning and where the dominating concepts are information sharingand information exchange. According to Chatman (1999):

small world is a society in which mutual opinions and concerns are reflected by its members, a world in which language and customs bind its participants to a worldview (p. 213)

The motivation to prefer this approach over others, equally valid, would derive from the choice of the previous framework. In fact, the argumentation theory has among its presuppositions, building arguments on the basis of shared premises which means accepted or acceptable by the target audience. However, because in online argumentation the audience rarely remains the real and actual one (Boyd, 2010), I need to enrich this framework with some important concepts taken from Network analysis.

Social media support diverse information sharing, allowing communication between users with different Common ground. Because users can choose whom to follow, researchers commonly agree that social media users live in echo chambers. However, according to Shore et al. (2016) who studied a cross section of hyperlinks posted on

(25)

Twitter, only members of a tiny network core do exhibit cross-sectional evidence of polarization due to their popularity and activity. This means, according to the authors, that:

Predicted behavior depends on which part of the system you are looking at, but on average, Twitter accounts post more centrist information than they receive in their own time lines, undercutting the prevailing narrative of the social media echo chamber [...]. Instead, the widespread perception of such polarization may be the result of a network paradox, in which the behavior of nodes with a high degree is mistaken to be typical (p. 48).

This is relevant for this study when it comes to how science dissemination can be improved in order to increase citizens awareness and critical thinking. In fact, it is clear that information spreads always in and through a network which is subject to the influence of a visible and invisible audience (Boyd, 2010). However, this audience depends only partially on the personal choices and interests of the users, because it is regulated by the system, therefore, it is crucial for a broader science dissemination, to understand both.

3.2.1 Popularity, Virality and Credibility

According to Weng et al. (2014) who studied the predictability of successful memes (piece of information that replicates among people) in social networks, features based on network structure, provide crucial insights into virality.

As Bakshy et al. (2011) demonstrate, the neighbors of an individual in the network can be considered as potential audience, therefore users with a high number of followers, are more influential because their popularity. This leads, according to Jenkins et al. (2018) to the concept of spreadability, which refers to the potential to share content with other people, receiving the most retweets by a largest number of users.

This is particularly relevant for this study because popularity can be misleading for many reasons. From a user perspective, popularity can lead to trustworthiness which is, in rhetorical argumentation, a way to persuade the audience. Because of their popularity, users share information considered automatically credible, influencing other people views. Moreover, Twitter algorithms help popular content, making it more visible, further increasing its popularity.

As anyone can become popular, any news can become viral, therefore discussing research is more important than merely redistribution, when it comes to increase citizens awareness and critical thinking. As Wathen and Burkell (2002) explain, information can be used for learning and therefore just stored and recalled when needed, but also can be passed on to others and used to make decisions, which affect attitudes and behaviors.

Considering that credibility is crucial in order to persuade the audience, it becomes important to understand what factors influence users in believing or not what they are reading or listening. For doing that our argumentation framework is important because it shows that soundness of the content, credibility of the source or expertness of the

(26)

speaker/organization, and the characteristics of the audience itself, such common values and beliefs, are all decisive aspects.

However, the perception of cognitive authority is equated to influence, in electronic media (Olaisen, 1990) and this is not always based on credibility, but as we clarified above, on popularity, which is improved by the system. Moreover, funding shared premises and beliefs in order to adequately address a target audience, become difficult in a digital contest such Twitter, where this kind of delimitation is against the finally goal of the tool, which is to reinforce dissemination through retweets (Kwak et al., 2010).

What can help in this scenario, is to improve conversation and argumentation, rather than dissemination through single tweets as Gunnarsson Lorentzen (2016a) and Gun- narsson Lorentzen and Nolin (2017) findings suggest.

The downsizing of the audience in conversations, if on one hand is probably the main cause of fragmentation (Gunnarsson Lorentzen, 2016a), on the other hand it consents to put the interactions in a context given by the content relation and to produce meaningful interaction as I will discuss in the next chapter.

(27)

Chapter 4

Methodology

In this chapter I will briefly describe my choice of the main methodological approaches, giving more space to the analysis of data, for the reasons clarified in the next section.

Therefore, it is not to be considered as a conventional method chapter, rather you will find in it my general methodological considerations important for the study. I use content data in this project, which consist of the tweets produced by the users. The lack of thread analyses of Twitter conversations (Nelhans and Gunnarsson Lorentzen, 2016;

Gunnarsson Lorentzen and Nolin, 2017) justifies my choice of the data and adoption of qualitative analysis, rather than quantitative, when approaching conversation. What I want explore, in fact, is what kind of interaction offers a more complete and successful strategy to access and improve dissemination and societal impact of research. However, I use a quantitative approach when studying single tweets, because:

• the broader patterns within linking practices provide context for those used in the conversation thread

• I need to know which kind of research artifact is more frequently used in dissemination practices, and finally

• their usage is not conversational or argumentative, therefore using a quantitative method is more appropriate with the purpose of the users in sharing content, i.e.

distribution within the target audience.

According to White and Marsh (2006), content analysis is a flexible method that can be applied both in qualitative and quantitative research frameworks and allows to make replicable and reliable inferences from texts. Through this technique, the researcher is able to infer context from the text, and eventually answer the research questions. The main difference between quantitative and qualitative content analysis is that the first one is deductive in its approach, while the latter is inductive. In this research, both approaches are been used accordingly to the data and the questions posed.

(28)

4.1 Data collection and sampling

The project Data4Impact (EU H2020), involved in studying the impact of EU research funding on innovation development (Pukelis and Stanciauskas, 2018), provided the data used in this study. Two Twitter data collections on various topics related to Health, Demographic Change and Wellbeing Societal Challenge have been collected by the consortium: one based on keyword matching, and one based on conversations.

Data collection on Twitter is mostly focused on URLs to identify links to research, but it is not the only way to refer to research. Another way is to search for keywords and identify media files (images of charts, tables, etc.).

Conversation based data collection was performed during two weeks starting at Au- gust 23. The data comprise just short of 30000 tweets. The conversational threads have been selected only if the tweets mentioned the main scientific repositories available online (examples of search terms used for conversations are: ’dx doi org’, dx.doi.org, researchgate, academia.edu, arxiv.org, socarxiv.org , etc.).

Conversational threads were inferred from pairs of tweets / in reply to tweet identifiers.

From 17 threads with at least 100 tweets, one thread within the topic of climate change was selected on the basis that it included a variety of links to academic sources, blogs and embedded media files.

A follow-up three months data collection was performed starting at January 14, 2019.

In this case, a rolling scheme of 283 queries representing 137 topics was used for send- ing one query at a time. For this study, I focus on three of these topics: Cardiovascular, Environmental health and Glioblastoma. The search terms have been chosen according to the research topics which focused on public health (examples of search terms used for single tweets are: "air pollution" OR "environmental health" OR "environmental expo- sures" OR "environmental exposure" OR "air pollutants" OR "air pollutant"). Here, the collected data match queries only, meaning that conversations were not collected. From the three topics, 300 tweets with URLs were randomly selected for content analysis.

Since the data were provided and there has not been an actively participation in that process by the author of this study, this section dedicated to data collection and sampling, has to be intended as state of the art report based on the research work and method performed by the Data4Impact consortium.

4.2 Digital methods

According to Venturini et al. (2018), Digital Methods are the repurposing of the inscrip- tions generated by digital mediaconsenting the investigation of collective phenomena.

These methods have, as advantage, the use of data and computational capacities of online platforms and, as weakness, the difficulty to discern the phenomena investigated from the digital setting in which they appear (Venturini et al., 2018; Marres, 2015).

(29)

To obtain scientific findings, digital methods need an accurately secondary analysis, since there is always a risk to mistake the characteristics of the phenomenon to be studied with those of the medium (Venturini et al., 2018). Moreover, even if digital traceabil- ity provides a unexampled quantities of information to social sciences, the quality of such data for answering complex research questions, is often poor and complete data collection method has been lacking.

As Gunnarsson Lorentzen (2016a) highlights, two main research approaches has been used so far: user-based and hashtag/keyword-based, with detriment of the geographic- based approach, possible but not popular. Many studies have been conducted combin- ing two or more approaches, but these have not taken into account conversations. My project, using data provided by Data4Impact consortium collected through the method suggested by Gunnarsson Lorentzen and Nolin (2017), guarantees a more complete, reliable and less biased information.

In the second chapter I have already mentioned that the role played by Twitter in medi- ating information is not neutral (Van Dijck, 2013), but following Venturini et al. (2018) check-list for Digital Methods, I will present some of the methodological precautions taken in this study. As the authors suggest:

the key to securing the adequacy between observed phenomenon and repur- posed medium is to handle with care the relation between the scope of your research questions and the traces that you will use to investigate them (p.

4204).

This process called operationalisation (Moretti, 2013) consists in a constant effort to align research questions with digital media and device cultures (Weltevrede et al., 2016) allowing the researchers to have many different views on the object of study in order to reduce biases. This is possible even within the same platform (Venturini et al., 2018) and that justifies my choices: to compare data obtained by collecting single tweets with data taken from conversations; to use a mixed theoretical framework where operational- isationis attuned not just to the medium formats, but also to the medium practices of their users in term of discussing content and information sharing; and finally, to design the data analysis section searching for credibility of the sources and number of citations outside of the studied platform in order to follow the nature of the tool itself, which encourages dissemination across its borders.

4.3 Data analysis

Two different approaches were used in the analysis of the data. Quantitative content analysis when studying single tweets to find out which kind of sources were more frequently linked to, and qualitative content analysis to find out what kind of argumentation was used in discussing research and to which purpose the linked sources were adopted.

(30)

In this first quantitative approach I analyzed the sources, particularly I searched for source type. If the identified sources were scientific papers I searched further for number of citations in Google Scholar and shared Tweets through Altmetrics. For the articles with more than 12 citations within the same year of publication (2019), I verified also if the journals were open access. To avoid as much possible bias in the interpretation of the data, a reliability test has been done over 100 randomly selected tweets. These were coded a second time a few weeks after the coding of all tweets.

If the source was a science news, I searched for the scientific article by which it was inspired and repeated the same process described above. When approaching the conversation thread qualitative content analysis was applied. In this case three steps were done.

In the first step, the overall life cycle of the conversation was outlined by identifying different phases such sparkling, development, dynamics and closure of the argumentation.

In the second step, the types of tweets and responses were also coded in order to analyze users reactions, basically agreement and disagreement.

Inspired by Zubiaga et al. (2015) and Gunnarsson Lorentzen (2016b) and having in mind the theoretical framework, three dimensions were investigated: type of argumentation, type of response in relation to replied-to tweet, and context. The approach was data driven and the categories were developed and incremented if and when founded (Corbin and Strauss, 2008). The third step was identifying and coding the linked sources according to their type and here I used again the same quantitative approach applied in studying single tweets.

There are different necessary steps in qualitative content analysis, according to White and Marsh (2006). In this study, however, some of them had been already performed by the Data4Impact consortium, such collection of the data and sampling, so I did only the following steps:

• Coding

• Reliability checking (Cohen’s Kappa coefficient)

• Analyzing coded data

• Writing and discussing the results

In the next section I will present how the coding schema have been created and in the next chapter the results obtained.

4.3.1 Individual tweets and (re)source sharing

The analysis of 900 single tweets on different topics about Public health was based on a mixed method of information retrieval and bibliographic classification designed as follow:

• all the links were retrieved, analyzed and classified

(31)

• all the sources belonging to the categories scientific articles and science news were further analyzed in search of patterns related with the aim of this study

The tweets were all coded by the author of this study. The standard Cohen’s Kappa statistic was used to assess intra-coder reliability between the first and the second coding round of the 100 randomly selected tweets, giving 0.917 for the category and 0.874 for the usage. Intra-coder reliability, according to Sieben (2014) "checks whether the natural learning process during the coding procedure has a significant effect on the coding itself"

(p. 26). I coded first 100 randomly selected tweets, then these were coded a second time a few weeks after the coding of all tweets. Finally I measured the agreement between the obtained two rates. This method allowed to display a picture of what kind of scientific sources are more frequent in the selected data in the context of single tweets, where the goal of the users clearly appears not to be argumentation, but information transfer, sharing and exchange (Pilerot, 2012).

On the other hand, this method did not allow to put the picture in a context, in fact the only way to understand what kind of audience the linked source was intended to, is by analyzing the type of source itself. This meaning that a tweet mentioning a scholar paper by the title only or with a scientific comment, is with a high degree of probability directed to a scientific audience, while a tweet mentioning science new by title only or with a general comment, has a broader audience as a target. I analyzed three samples of 300 tweets each, on three different topics about Public health, Cardiovascular, Envi- ronmental health, Glioblastoma, and classified them by category and usage according to the follow:

4.3.2 Method in qualitative content analysis

From what has been outlined in relation to argumentation theory and analysis in the dedicated section, the procedure for coding the interactions in the conversational thread was designed as follows:

• all tweets with links to sources or with images embedded were analyzed

• all tweets within two reply steps from these tweets were also analyzed