Accessibility of Information Visualization for eParticipation

(1)

IN

DEGREE PROJECT INFORMATION AND COMMUNICATION TECHNOLOGY,

SECOND CYCLE, 30 CREDITS STOCKHOLM SWEDEN 2016,

Accessibility of Information Visualization for eParticipation

BOJANA DUMELJIC

KTH ROYAL INSTITUTE OF TECHNOLOGY

SCHOOL OF INFORMATION AND COMMUNICATION TECHNOLOGY

(2)

Page intentionally left blank.

(3)

Abstract

Visualizing information is a large field focussed on using a visual display of information to change or facilitate the user’s thinking. However, creating and interpreting data visualizations is still seen as a task for experts and can even be seen as a daunting task by the general public. Techniques like storytelling with data aim to make visualization more accessible for these users.

A fitting application of this approach is to eParticipation, which are processes during which citizens get involved in society and governmental decision making, through a digital medium.

In order to enhance the access that all groups have to the progress and results of these participations, an interesting lead is the visualization of the eParticipation processes and results. This would provide participants with insights towards the effect their participation had. Additionally, it would provide decision makers with deeper insight into the opinions and views of the people that will directly be affected by their decisions.

This project used the power of information visualization and storytelling with data, to design a concept that achieves this deeper insight into eParticipation results. This report describes the design, usability test and evaluation of a tool that shows the results from an eParticipation in a dashboard, while using a story format to help communicate the key insights. The goal was to accomplish a higher level of accessibility for this type of data for non data analysis experts, i.e., making the visualization easier to understand by everyone.

Keywords: information visualization, e-participation, storytelling, human computer interaction

(4)

(5)

Abstrakt

Att visualisera information är ett stort fält fokuserat på att använda en visuell presentation av information för att ändra eller underlätta för användarens tänkande. Att skapa och tolka datavisualiseringar ses dock fortfarande som en uppgift för experter och kan även ses som en svår uppgift för allmänheten. Tekniker som berättande med data strävar efter att göra visualisering mer tillgänglig för dessa användare.

Ett passande tillämpning av denna metod är e-deltagande, som är processer under vilka medborgare engagerar sig i samhället och regeringars beslutsfattande, genom ett digitalt medium. För att öka alla gruppers tillgång till framsteg och resultat av sitt deltagande, är ett intressant spår visualisering av e-deltagande processer och resultat. Detta skulle ge deltagarna insikter om effekterna av sitt deltagande. Dessutom skulle det ge beslutsfattare fördjupad insikt i åsikter och synpunkter från människor som direkt kommer att påverkas av deras beslut.

Detta projekt använde kraften i informationsvisualisering och berättande med data för att utforma ett koncept som uppnår denna djupare insikt i resultat av e-deltagande. Denna rapport beskriver design, användartester och utvärdering av ett verktyg som visar resultaten från ett e-deltagande i en instrumentpanel, när en berättelse formad för att kommunicera viktiga insikter används. Målet var att åstadkomma en högre grad av tillgänglighet för denna typ av data för icke-experter, vilket gör visualisering lättare att förstå för alla.

Nyckelord: informationsvisualisering, e-deltagande, berättande, människa-datorinteraktion

(6)

(7)

1 Introduction

The visualization of information is a widely applied technique that helps to translate data into a format that is more accessible to humans. Domains from finance to government to journalism, all use infovis to explore data and thereby gain new insights [8][9]. This wide range of professions relies on information visualization regularly, as it is useful for a multitude of reasons. There is however one domain that does not utilize the full potential of information visualization as it could. This domain is eParticipation, i.e., online processes during which citizens get involved in society and governmental decision making. This work shall focus on discovering the gap between these two research fields and then propose a possible tool that could bridge it through the application of the newest information visualization methods on eParticipation results.

1.1 Background

Information visualization, or infovis, is frequently described as a visual representation of data, meant to boost human thinking. Infovis is being studied by many different fields and is used for a multitude of reasons. One of these being that diagrams make communication easier [3]; as data visualization helps to illustrate the message behind the data. However, general audiences are not trained to analyze complicated graphs [1]. A recent approach, intended to increase the accessibility, is through storytelling with the data. Meaning that the viewer is guided towards the key messages that the information visualization is meant to convey, through the use of classic story elements [1][2].

With the latest advances in technology, eParticipation has grown into a research field that strives to increase citizens’ involvement in society and governmental decision making, through the use of technology. eParticipation has three main types: information sharing, citizen consultation, and active participation. Each type has a different approach, and each can be aided by various ICT tools, e.g., polls, surveys, and decision making aids.

Essentially, there are three stakeholders involved in an eParticipation process:

● The decision maker, the individual who benefits from the information gathered.

● The organizer, the person tasked with collecting the knowledge from the participants.

● The participant, the citizens that want to be involved and have their voice heard.

An eParticipation can be performed at the request of any party interested in gathering input from a specific group. A common practice would be government officials that would benefit from citizens input on a decision that needs to be made on the citizen's’ behalf. However, the decision maker can be anyone with a need for external input and will to ask for participation.

This is a wide range of people, spanning from government officials, who are responsible for city planning, to youth shelter managers, who want input on what the next event to organize should be [5].

1.2 Problem

The decision makers, who need the interesting information from this data to make a choice, are generally not trained data analysts and therefore can not acquire this information from the raw data that was gathered themselves. Thus, the collected data often ends up not being

(10)

used. There would be a need to hire an external expert for this purpose. This might not be within the budget of the government or organization that wanted the eParticipation.

Neither might be the expertise to set up and lead a participatory process [11]. If an expert visualizes the data, these visualizations can still be too complex for non experts to understand without additional information [3]. All the previously described eParticipation processes produce a big volume of data, which then needs to be analyzed and shown in a more accessible way. An example of combining information visualization with eParticipation is to facilitate the understanding of alternative choices or predicted future trends in decision making process by using visualizations [5].

This work focusses on the accessibility of information visualization, i.e., making information visualization easier to understand for non data analysts. Therefore, it addresses the part of communicating the results to all stakeholders; from the ideal participation tool process. The main hypothesis was: “Using storytelling in information visualizations would improve the accessibility to eParticipation results and their visualizations for the general public.“. The reasoning behind this hypothesis stems from the fact that most people are familiar with the narrative structure of a story [2], as it was initially constructed by Aristotle over two thousand years ago. Therefore, this well known concept should help to make the visualization easier to understand by anyone.

The secondary hypothesis was: “By applying the experience sampling method in order to gather eParticipation data, more useful visualizations could be created.”. The experience sampling theory was formed by Hektner [14]. He proposes that if participants are asked the same questions multiple times over a period of time, this would help gather more in depth data, which in turn would facilitate the creation of more effective visuals.

1.3 Purpose

The purpose of this report is to present the design and evaluation of a prototype that visualizes the results from an eParticipation in a more accessible way. Additionally, the in depth description of the process and the foundation of knowledge that was used to create this prototype is described. This foundation is based on an investigation of the relationship between information visualization and eParticipation, as well as the gap between these two fields. The goal of the prototype was to bridge this gap through the use of accessible information visualizations.

1.4 Goal

Specifically, the goal of this project was to design and test an information visualization tool for eParticipation data, that focuses on making visualization more accessible to the masses through the use of storytelling with data. Masses would refer to the general public, who are not data analysis experts.

The final design solution needed to cover three issues. Firstly, for the participants, this would be a way that allows for easier analysis of the progress and access to the results. Secondly, the organizer of the process needs to be able to do exploratory analysis of the data on their own. Thirdly, the decision maker should receive a summary report which will be an

(11)

explanatory analysis using the storytelling with data approach. However, the main purpose of the tool is to communicate the results in the most accessible way possible.

Although there is extensive research into the two fields separately, very limited studies were found that focus on the combination of both infovis and eParticipation. Therefore, the focus is on finding the common ground between the two fields and visualizing the specific types of data that are produced during eParticipations.

1.4.1 Benefits

An ideal tool would organize everything to do with data from the participation process. From the planning of what type of data is going to be generated, to the collection of this data during a participatory process, as well as the analysis of this data and the communication of results to all stakeholders. Afterwards, the visuals could be used to facilitate a discussions of these results amongst the participants. This work is a first step towards such a tool.

1.4.2 Ethics

The detailed description of the methods used, ensures the replicability of the work. The usability tests and pilot study held to evaluate the design ensures the dependability of the conclusion and confirmability that there are no personal assessments affecting the results.

The pilot study done at the end of the project ensures the validity of the work, by testing if the purpose was achieved. Furthermore, the wide range of resources that was consulted during the pre-study and the validation from the pilot study assure the quality of this work.

All the participants in the usability testing and the evaluation pilot study have voluntarily taken part in the interviews and were not given any compensation for their time. Each participant was explained what the data gathering was for and has given their consent. No private information was collected during this project.

1.4.3 Sustainability

In order to have sustainable development, the development process has been divided by the following aims: environmental, societal and economical [30]. This project contributes to these aims in the following ways:

● Environmental: This report does not directly impact the environment, however more digital participation would mean less paper would be necessary to record the results.

Additionally, less travel would be necessary for participants to have their voice heard, which would benefit the environment.

● Societal: The concept described in this report is meant to help increase the citizens involvement in governmental decision making, by making the communication of information and results easier to comprehend, the goal is to get the citizens more involved in the overall process. Through a better communication of results, decision makers would have more access to the public’s opinion. Therefore, one could argue that this would allow decision makers to make choices the public supports.

● Economical: By implementing the concept proposed in this report, the need to hire data analysis experts to analyze and visualize the results would be eliminated.

Additionally, aside from maintaining the software, there would be no extra cost for the use of the system once it was created. The one tool would facilitate a large number of eParticipations, with no extra cost of spreading an online tool to the participants.

(12)

1.5 Method

The work was performed using a qualitative research method. First, the theoretical dimensions of the related research were mapped. Second, a comparative analysis of existing systems was done. Together these two formed the pre-study. Based on an analysis of the related work and the existing systems, a list of observations was made to be the foundation for the design. Then, through an iterative approach of prototyping and usability testing, a design solution was created. Finally, this design was evaluated through a pilot study in order to validate if the purpose of the project was achieved.

1.6 Delimitations

This report limits itself to the design and validation of a possible tool for the communication of eParticipation results. The development of such a tool is beyond the scope of this work.

While participation can be applied to various fields and for various reasons, this work will focusses on the use of eParticipation for governmental processes, as it was defined by Macintosh [11].

1.7 Outline

The overall structure of this work takes the form of nine chapters. This report is built up as follows:

● Chapter 1 introduces the topic and main goal of the work. It also discusses the benefits this work provides as well as any ethical and sustainable development concerns involved.

● Chapter 2 discusses the related work this project builds upon. It lays out the theoretical dimensions of the two research fields, information visualization and eParticipation, and looks at how these two could complement each other.

● Chapter 3 is concerned with the method used to perform the work and described the full process.

● Chapter 4 holds the pre-study, comprising of a comparative analysis of existing tools and platforms that are currently being used to address the problem. This chapter forms the foundation upon which the rest of the work builds the problem solution.

● Chapter 5 introduces the proposed solution that was designed, as well as how it was created.

● Chapter 6 described the pilot study that was performed to evaluate the design and thus validate if it achieves the goals set in the beginning of the project.

● Chapter 7 reports on the results from the pilot study that was done and described in chapter 6 to evaluate the work and design presented in chapter 5.

● Chapter 8 looks back at the work and discusses it. The discussion includes observations on the work, i.e., the data used, the final design and the evaluation, as well as reflections on the ethics and sustainability related to the subject.

● Chapter 9 holds the conclusions, which holds a summary of the main parts of the thesis and the overall takeaways from this work. Additionally, the conclusion holds remarks for future work.

(13)

2 Background

Tufte [7] is one of the pioneers of data visualization. He described the objective of the field as helping humans see without words. Pousman [8] described information visualization as using a visual display of information to change or facilitate the user’s thinking. A more broader definition is provided by Card, Mackinlay and Shneiderman, a group of acclaimed human computer interaction researchers; they outline infovis as using visual representations, that were made on computers, to interactively amplify cognition [9].

Information visualization is used for a multitude of reasons and is essential for a wide range of domains. Pousman [8] reports how domains from finance to government to journalism, all use infovis to explore data and thereby gain new insights. He also describes how visualizations are expanding from our work lives to our everyday lives and imparts how humans are the original information processors. Pousman calls these everyday visualizations casual information visualization; these are computer systems that show personally meaningful information in a visual way that supports the user in everyday situations. As the use of infovis expands, so does the need for easily interpretable graphics by anyone.

An important aspect of infovis, is that diagrams make communication easier [3]. Data visualizations can be used to illuminate the messages within data. However, general audiences are not trained to analyze complicated graphics in order to find these messages [1]. While, through the rise of casual visualizations, general users are getting increasingly more familiar with infovis, the accessibility of these visualizations remains an issue.

Laramee and Kosara [10] attributed this accessibility issue to the lack of appropriate visual metaphors, which are used to map data to visual elements. In order to understand the visual, the viewer first needs to put effort into learning this new visual metaphor that was used to display the data. Each new visual metaphor requires a learning step in order to be understood. Visualizations of large data sets or multi-dimensional data, require more sophisticated metaphors. Viewers have higher difficulty translating these metaphors to understand these types of visualizations, as they first need to put more effort into learning this new visual metaphor that was used to display the data.

Interaction in infovis poses a similar issue [10]. Similar to the visual metaphors, each interaction requires another learning step from the user. Additionally, with each new visual metaphor, there is going to be a new interaction technique to learn as well. These issues make the accessibility of infovis harder for people who do not have a background in data analysis.

There is a large multitude of different information visualization types. One of the most general differentiators is the type of information that is being visualized. There are two main data types: quantitative data and qualitative data. Quantitative data is information that can be measured in numbers, while qualitative data is of the descriptive nature and can not be measured. The type of data determines what kind of visualization can be made with it.

A second differentiator in infovis is the type of analysis that is going to be undertaken with the visualization, which can either be exploratory or explanatory. In exploratory analysis, the

(14)

data set is visualized and the viewer is left to explore the data, on their own, to find the key messages the data contains. Explanatory visualizations, however, are created after the key messages in a data set have been found or were already know, and are therefore created in a way that clearly communicates these specific messages to the viewer.

A new approach to increase the accessibility of visualizations is through storytelling with the data; meaning that the viewer is guided towards the message that the information visualization is meant to convey by the visualization. Nussbaumer Knaflic [2] has played a key role in spreading this message. She has recently written a book that serves as a guide for making explanatory visualizations. It focuses on how to make or improve graphics in such a way that they clearly convey one or two specific messages to the viewer.

While Nussbaumer Knaflic has been coaching workshops on this technique for the past few years, she is not the only one to apply this approach. There have recently been multiple studies in the field, for example Battista [1], that also reinforce how storytelling with data visualizations can illustrate the message behind the data in a more accessible way.

A field where this storytelling with data approach could be beneficial, is electronic participation. According to Macintosh [11], eParticipation is defined as the use of ICT to support democratic decision-making processes. In general, a decision maker requires the input from another party to make a decision on a certain matter that involves this other party.

A decision maker can be anyone with a need for external input. This is a wide range of people, from government officials responsible for city planning to youth shelter managers that want input on what the next event to organize should be [5]. However, too little attention has been paid to the practical benefits that information visualization could have on eParticipation. Although there is extensive research into the two fields separately, very limited studies were found that focused on the combination of the two.

In a more recent study, Terán [12] builds upon work done by Macintosh to describe eParticipation as follows: “An emerging and growing research area that aims to increase citizens’ participation in order to promote fair and efficient society and government support, by using the latest technology developments.”. Terán explains the need for a eParticipation as it creates new channels that facilitate citizens participation and direct engagement in regards to issues and processes that will form their future society.

Macintosh [11] defines three levels of involvement in eParticipation: information sharing with citizens (eEnabling), citizen consultation (eEngaging), and active participation in policy making (eEmpowering). For each level the process is different and there exist various ICT tools to aid the process, like polls, surveys, decision making tools. Tambouris [12] expanded these levels to include tools. This resulted in the following five level model:

● eInforming, the government shares information with citizens

● eConsulting, the government collects public feedback

● eInvolving, the government works to understand the public’s needs

● eCollaborating, citizens actively work with the government to find solutions

● eEmpowerment, the citizens get to decide

Tambouris found that most existing ICT tools are created to share information, to poll opinions and to vote. Thus, most ICT tools are created for the eInform and eConsult participation levels. There is a gap when the other three processes are concerned.

(15)

Terán [12] builds on the framework created by Tambouris to create a tool, called VAAs, that can evaluate eParticipation processes. VAAs reconfirms that most existing ICT tools cover the eInfrom and eConsult participation level, but that there is a recent rise in tools that facilitate eInvolving and eCollaborating. These are tools enabled by advancements in technology, tools that can facilitate the online process of discussions and idea generations.

However, these eInvolving and eCollaborating processes involve idea generation and group discussions activities, which result in the production of volumes of text, which is qualitative data. Although, information visualization research tends to focus more on quantitative data visualization. At a recent workshop on the subject of participatory methods, which was attended by Europe’s leading participation experts and held by the Nexus Institute in Berlin, the consensus was that these tools and methods need to be created for the analysis and visualization of this eParticipation data in a more accessible way. As this data is in the end used by decision makers, who need the interesting finding in this data in order to make a choice, and who are generally not trained data analysis and therefore can not acquire this themselves from the raw data that was gathered. Therefore, there would be a need to hire a data analysis expert to translate the data into a more accessible form. However, if an expert visualizes the data, these visualizations can still be poorly made or too complex for non experts to understand [3]. Additionally, an expert may not be within the budget of whomever was in need of the participation process in the first place. Neither might the expertise to set up and lead a participatory process. [11]

Phang [5] suggests that using information visualizations to facilitate the understanding of alternative choices or predicted future trends, can help guide decision makers during the decision making processes. These are few possibilities that can be achieved through the application of the storytelling with data technique on eParticipation data. As influencing decision making and measuring the effect of the influence on a decision are complicated behavioral processes, this work shall focus on presenting the results from eParticipations in more accessible ways, to the general public.

(16)

(17)

3 Method

This project was done using the qualitative research method, an approach that uses the understanding of human behavior to create new hypotheses and theories, or to develop new computer systems [6]. As the goal of this work was to design a new information visualization tool, this method seemed the most suitable to achieve this goal.

Since this project aimed to improve a process that relies on the human cognitive ability, i.e., understanding of information visualizations, it was thought necessary to first gain a deep understanding of this process. Therefore, the first step was the pre-study, which analyzed related work and formed the foundation for the rest of this work. This approach with these investigations, which are then interpreted to form theories, was based on the qualitative research method [6].

Following the pre-study, a design for a tool was made based on the theories that were formed. This design phase was performed through iterative cycles of prototyping and usability testing. Finally, the tool was evaluated through a pilot study, in order to validate if the proposed solution supports the hypothesis.

Thus, there were three phases: the pre-study, the design and the evaluation phase. The rest of this chapter describes each in more detail, along with motivation on why these were performed.

3.1 Pre-Study Phase

The pre-study phase was the initial research phase and was performed in two parts, each with a different research method. First the analytical research was done and then it was followed by the empirical research.

3.1.1 Analytical Research

The analytical phase of the research was performed at the very start of this project. The goal of this phase was to gain a deeper knowledge of the relevant fields, outline them and find their main obstacles. Initial hypotheses were created to focus the research. The main hypothesis was “Using storytelling in information visualizations would improve the accessibility of the visualizations for the general public.”. This work focused on the visualization of eParticipation data.

Therefore, the two fields of interest were information visualization and eParticipation.

Recent studies were explored to obtain insights into their current states. A considerable amount of literature has been published on both of these topics. However, studies in which information visualization is used to improve the eParticipation have not been found. The purpose of this work is to use novel information visualization techniques, which are accessible to the general public, in order to improve the eParticipation process. The accessibility was important as the users involved in the eParticipations are not information visualization experts.

(18)

The findings of this research have been documented in chapter 2 Background. The theories that were formed have been used throughout the rest of this research, and are described in chapter 4 Pre-Study.

3.1.2 Empirical Research

The knowledge and insights gained during the analytical phase were used as the foundation for the empirical research phase. During this phase, the goal was to gain deeper knowledge and form well rounded theories. This was to be based on experiences and observations from real situations [6].

Therefore, a comparative analysis of existing visualization approaches and tools was conducted. This research was executed in an exploratory fashion. The internet was explored in order to find state of the art systems for information visualization, and are preferably marketed as systems that visualize participation data. The aim was to gain as comprehensive a view of the existing solutions, as was possible with the available resources.

Twenty of these state of the art systems were studied. During this exploratory research, the data collection was focussed on identifying the strengths and weaknesses of these systems, The observations and lessons learned were then harnessed to design a better system.

The complete setup of the analysis and the results can be found in chapter 4 Pre-Study.

Appendix A Comparative Analysis holds the table with all the data that was gathered.

3.2 Design Phase

The design phase aspired to iteratively create and improve the proposed solution. In order to reach a satisfactorily designed tool that has been user tested to validate its usability. Based on the literature study and the insights from the exploratory analysis, a tool for visualization of eParticipation data was designed and tested for usability. There were three iterations.

Each of these iteration started with the design of a prototype, which was then tested for usability, in order to improve the next prototype.

The tools used for this design phase were pen and paper, Sketch [28] and Invision [29].

Sketch is a digital design tool that focuses on user interface design and Invision turns static designs into clickable and interactive prototypes. The first prototypes were wireframes drawn on paper. In the second iteration, the wireframes were digitized into more high fidelity mockups as they were created in Sketch. In the third and final round, the mockups were imported into Invision, where they were made to be interactive.

To asses the design, these prototypes were tested for usability. Each iteration started with the creation of a new design prototype and was then followed by a usability test of this prototype. During the usability tests, users were asked to do a cognitive walkthrough, i.e., an interview while they interact with the prototype. An interview lasted between 30 and 60 minutes. Each prototype was tested with five to ten users. After each usability test, the theory behind the framework was reassessed and improved. In the next design iteration, the improvements were incorporated.

(19)

3.3 Evaluation Phase

After the design iterations, a final evaluation phase was held to validate the proposed design solution. A pilot study was done with two independent groups. Each group had five participants. Both groups were presented the same data, yet in different representations.

Thus, the only varying condition for the test was the way the results were presented to the viewer. One group used the solution proposed in this report, while the other group used a current state of the art representation.

During the pilot study, the users were asked to interact with the solution, and perform a set of five tasks with the data. Data collection was done through interviews. The information of interest that was collected, was whether the user managed to find the correct data, how long it took to complete the task, how difficult the user perceived the completion of the task and how aesthetically pleasing they found the visualization. The collected data was stored in an excel sheet, where it was analyzed and turned into graphs. Both the observations from the analysis and the graphs can be found in chapter 7 Results.

The analysis of the data was performed to determine whether the tool and visualizations fulfill the expectations set in the hypotheses, i.e., does the new design improve accessibility for non data analysis experts. Essentially, to evaluate if the project, and its proposed design, do actually achieve the goal set in the beginning of the project. This final test was done after the design phase was complete and in addition to the usability testing that was done during the design phase, in order to generally improve the prototype.

The participants during the usability testing in the design phase and the evaluation pilot study did not overlap. This was done in order to prevent an unfair advantage by participants having prior knowledge of the design or data.

(20)

(21)

4 Pre-Study

At the beginning of the project, the initial hypotheses were that the results from an eParticipation can be communicated more effectively through the following hypotheses:

1. Using storytelling in information visualizations would improve the accessibility of the visualizations for the general public.

2. By applying the experience sampling method to gather eParticipation data, more useful visualizations could be created.

These hypotheses specified the direction in which a new system was to be designed.

In order for the research to be performed in a more targeted and manageable way, the two hypotheses were split into the following smaller tasks:

- How storytelling improves information visualization.

- How classic elements of storytelling could be incorporated in visualizations.

- How to visualize qualitative data.

- How to show changes in data that was sampled over longer periods of time.

- How to compare the answers to the same questions, which were answered by independent groups.

To answer these questions, a literature study was performed of the scope, i.e., information visualization and eParticipation. Following this, a comparative analysis was performed of the existing tools and solutions to the problem. The knowledge acquired from this research was used to create the design scope and model the user’s needs.

Subsequently, these analyses were combined to form the foundation for when making design decision during the creation the solution presented in chapter 5 Design. The process and results of the pre-study are described in this chapter.

4.1 Literature Findings

There have been several studies in the literature [11][12][5] reporting that the most common activities during a participation process are information sharing, surveys, idea generation, discussion and voting. The type of data that this results in is mostly qualitative data from open ended questions. Therefore, the data to be visualized is in the form of

● text(s)

● collections of text(s), ranked by preference

● Likert scale rankings

● and other types of voting.

Based on the storytelling approach by Knaflic [2], which is supported by several research studies [1][22], the following steps lead to successful visualizations:

1. Know your audience. The first step in any proper human computer interaction process.

2. Get rid of clutter. This is an idea originally propagated by Tufte [7], to remove all distracting and unnecessary elements in order to draw focus to the important information.

3. Draw attention. Meaning the wise use of specific elements, like color and placement, to highlight specific messages.

4. Have a story. Using the mechanics of a story to lead the viewer through a visualization in a way that will highlight the key takeaway messages.

(22)

4.2 Related Work

The difference between image 1 and 2 shows the gap between the current way that eParticipation results are reported and what an information visualization using the storytelling with data approach looks like.

Image 1 shows a page from a UNICEF report of adolescent participation. The complete report is mostly comprised of text and some tables and graphs. It is 300 pages long, which means that getting information from this report is time consuming and not that accessible.

Image 2 shows a single screen with an engage graphic, tabs to navigate through the story and a limited amount of text. However, this visualization was created by a skilled data analyst and graphic designer.

The contrast between these two images validates the need for the design of a new tool to visualize the results from eParticipations. A tool that does not require a data analysis expert to create accessible results. A complication is that the data from eParticipation is qualitative, this textual data is harder to visualize compared to quantitative data.

" "

IMAGE 1 (left): Current state of the art for reporting for eParticipation taken from a UNICEF report [17]

IMAGE 2 (right): Information visualization using the storytelling with data approach from the New York Times on the Euro Crisis [24]

The subsection of data visualization that is focused on turning text into graphics is called textual visualization [15][16][20][21]. The research prototypes in this section turn into complex visualizations fairly quickly. There are numerous extreme examples out there. The main concern that is related to the accessibility of these new types of visualizations, is that each one of them is based on a new metaphor. Each of these new metaphors needs to be learned by every single viewer [10], which makes it harder to interpret the graphic. An example of such a visualization is Opinion Space shown in image 3, where comments are mapped into a space metaphor where positive and negative comments repel each other.

(23)

The second concern is that each of these metaphors is joined by new interaction techniques that go with it. These interaction techniques, again, need to be learned. The steep learning curves makes a lot of the prototypes not accessible to the general public.

"

IMAGE 3: Opinion Space [23]

4.3 Existing Solutions

In order to have a solid foundation for the design, a comparative analysis of existing solutions was performed. The goal of this analysis was to identify the strengths and weaknesses of competitive products and services. The internet was explored in order to find state of the art systems for information visualization, and are preferably marketed as systems that visualize participation data. The aim was to gain as comprehensive a view of the existing solutions, as was possible with the available resources.

In order to achieve the analysis goal, each system was inspected with the walkthrough method [4], i.e., a step-by-step review was performed of the system, during which its design was scrutinized to understand it and identify problems like missing functionality, other weakness in the design, as well as its strong aspects.

The Norman Nielsen heuristics for usability [31] were used as a guideline to navigate each system. These heuristics are ten principles that have been commonly used by user interface designers to improve their design since they were established in the 1990s. They have helped guide the search towards identifying the usability problems in the existing solutions.

This search identified strengths and weaknesses which were used as the foundation for the design a new tool, which was the goal of this project.

From the ten Nielsen principles [31], observing the following was the most insightful with regards to creating a tool with accessible infovis:

● Match between system and the real world: the system should speak the users' language, with words, phrases and concepts familiar to the user.

● Consistency and standards: users should not have to wonder whether different words, situations, or actions mean the same thing.

(24)

● Aesthetic and minimalist design: Every extra unit of information competes with the relevant units of information and diminishes their relative visibility.

● User control and freedom: users often choose system functions by mistake and will need a clearly marked "emergency exit" to leave the unwanted state without having to go through an extended dialogue. Support undo and redo.

● Recognition rather than recall: minimize the user's memory load by making objects, actions, and options visible. Instructions for use of the system should be visible or easily retrievable whenever appropriate.

These heuristics were especially useful as they simultaneously aided the design phase of this project that followed, i.e., helped create usable and aesthetically pleasing graphs that engage the viewer, as well as a template in which to put these graphs.

4.3.1 Analysis Setup

The analysis was performed by studying twenty different existing systems, that utilize some type of information visualization and are preferably marketed as eParticipation systems. The goal was to gather the strengths and possible improvements for each of these systems as explained earlier in this chapter. The internet was explored in order to find state of the art information visualization systems.

The test setup, for each system, was made out of the following steps:

1. The system’s website was analyzed to determine if the system indeed performed some form of information visualization and if this visualization was of eParticipation data, and it was thus of interest for this analysis

2. The system was used to gain a deeper understanding of its functionalities and analyzed with the walkthrough method.

3. Reviews of the tools were studied, in order to establish if they mention additional details that might have been overlooked in step 2. These reviews were found on the system’s website or on an unaffiliated and unbiased website.

4. The observations on the strengths and possible improvements for each tool were reported in a table that can be found in Appendix A. The type of participation process was also noted.

4.3.2 Observations

While the below is a summary of the observations, the complete findings can be found in appendix A. The study brought multiple interesting observations forward. Both strength and weaknesses. The main strengths found were:

● Information visualization dashboards are widely adopted to provide a coherent overview of the current state of a process or to present all the results in a single place.

● The systems that use a participatory approach seem to have higher engagement with the users, compared to systems without this approach. Participatory approach refers to the direct involvement of users in the creation of the results.

● Tailoring the data that is shown to the respective audience, results in more meaningful data visualizations.

● There is a wide range of available survey tools in the market.

(25)

The main weaknesses, that were found in the current systems, were:

● Generally very few qualitative data visualization systems. Most of the analyzed systems only visualize quantitative data. The ones that do qualitative data are usually beta experiments made during research projects.

● Survey tools generally provide a report of the results, with basic information visualizations.

● A high usage of pie or donut charts, these are visualization types that lower usability and accessibility [2].

● The participants rarely have influence over the results.

● Sharing of the results can be difficult, as few systems provide access to the raw data.

● Some systems have free access to their services, while more sophisticated ones have high price tags attached to them.

4.3 Observations

The knowledge acquired from the pre-study, i.e., the literature study and comparative analysis, has lead to the following theories, which are based on the observations and lessons learned during the pre-study.

The two main things to address in the design phase were found to be:

● Using the storytelling with data approach in the design of the system.

● Choosing accessible information visualizations to show the eParticipation data.

4.3.1 Key Aspects

The key aspects to consider in the design were:

- Using a dashboard representation -- The use of dashboards in information visualization is a common approach in order to provide an instant overview of the data. This same approach seemed appropriate to represent the results of an eParticipation.

- Including qualitative data -- The main weakness found in the comparative analysis was the lack of qualitative data visualizations. The literature study showed that this type of data is a significant part of the data gathered during eParticipations. Its inclusion in the design is therefore important.

- Defining which data is useful to visualize -- This needs to be as automated as possible as there is no data analyst involved.

- Translating the data into meaningful visualizations -- This needs to be as automated as possible as there is no data analyst involved.

- Creating coherent stories with the visualizations -- The tool needs to facilitate the creation of a story with a beginning, middle and end.

- Highlighting the key messages from the data -- This is an important aspect of the story.

4.3.2 User Analysis

There are three types of users in an eParticipation; the decision makers, the organizers of the participation and the participants themselves. Decision makers are the ones that are interested in the results. The organizers are the ones organizing the participation and the transfer of information. Participants are citizens volunteering their free time to participate.

(26)

Essentially, these are the three stakeholders involved in participations:

● The decision maker, the individual who benefits from the information gathered.

● The organizer, the person tasked with collecting the knowledge from the participants.

● The participant, the citizens that want to be involved and have their voice heard.

The assumption is that none of these users have an in depth knowledge of information visualization. Therefore, the designed solution needs to function without a data analyst expert. While most of the data analysis can be automated, some form of participation from a stakeholder, probably from the organizer, would still be necessary during the process in order to create a story from the data.

(27)

5 Design

This chapter describes the concept that was designed. The data that was used in this representation is from the results of an UNICEF participation [17]. The design was achieved through the iterative creation and usability testing of prototypes. The sequence of prototypes that has lead to the design presented here, can be found in Appendix B. This chapter described the concept behind the designed system, the reasoning behind the visualizations used, the usability testing that was done, and the overall process that has lead to this prototype.

5.1 Concept

The proposed solution is to use the storytelling with data approach in a dashboard like format, a panel that shows all the highlights in one place. Each box is meant to be a question from an eParticipation process, e.g., a survey, or a piece of key information. Image 4 shows an example of such a dashboard created from the UNICEF report on adolescent participation [17].

The dashboard is meant to tell the story formed by all the results of a participation. Like a classical story it has a beginning, middle and end; with key takeaway messages. These messages are the main findings in the data from the participation.

"

IMAGE 4: Dashboard View

There are three parts to this concept; view, compare and edit. The first part is viewing the dashboard with the final results from an eParticipation. The results are meant to be viewed by participants and decision makers, who want to understand the main findings from the

(28)

eParticipation. The dashboard serves as a summary report. For participants it is a way to view the effects of their input, and for decision makers it is a way to gain useful insights in order to make more informed decisions.

The second mode, compare, is to compare two dashboards in order to find differences. This is again meant to be viewed by the participants and decision makers. Dashboards can be made from data from one group that participated in the same process on multiple occasions, or be based on the same process performed by different groups, which could be varying in demographics. Ultimately, the comparison is meant to visualize the findings from the experience sampling method [14].

The third mode is editing. Editing is meant to be done by the organizers of the eParticipation.

The boxes can be rearranged on the dashboard, in order to formulate the desired story to be told. Additionally, boxes holding more important messages can be resized and colored with another color in order to make them stand out and put more emphasis on the box’s respective message. If there are multiple visualizations for the data type that the box holds, these could be interchanged as well. In case the data in a box holds little to no insightful details, the box can be hidden from the dashboard. If need be, additional boxes can be added from external sources, e.g., statistical data on the topic, or extra participation material, i.e., images, video or audio from the participation. A lot of the work to create the summary report from the results can be done by the system through automatic data processing.

However, the organizer’s input is still necessary to identify the key messages in order to construct a more insightful final report.

5.2 User Experience

When viewing a web page, the user’s focus goes over the page in a zig-zag viewing pattern, similar to the culture’s normal reading pattern [18, page 13]. The eyes scan the page, initially starting at the top left, then working to the top right, continuing down to the left and finally to the bottom right. They keep making this z-shape from the top of the page to bottom.

The dashboard is designed to make use of this natural human behavior. Therefore, when organizers edit the dashboard, the most important information is meant to be placed in the top left corner and then the top right. The importance decreases towards the bottom of the page. Image 5 illustrates the order in which a viewer would perceive a dashboard.

This way in which a user would perceive the dashboard, helps to cement the structure of the story being told with the dashboard. As a story has its beginning, middle and end, the zig- zag pattern would naturally lead the viewer through this story. Initially, the viewer would first see parts 1 and 2 shown in image 5, this is where the introductory information would be. The beginning of the story is thus the title of the eParticipation and a short summary. The viewer would then keep following the zig-zag pattern to 3, 4 and 5, which is where the middle part of the story would be. Then ending at part 6, there would be a conclusion. The key messages would be accentuated through either bigger sized boxes or color. Finally, the dashboard boxes are to be placed in an order that helps to convey the story in a logical order.

(29)

"

IMAGE 5: Zig zag Viewing Pattern

During the design of this tool, an effort was made to reduce chart junk as much as possible.

Minimalism is key to accessible visualizations [2]. Therefore, there are no unnecessary texts, borders, shadings or 3D effects.

Additionally, as people tend to process information better in chunks [18, p. 62], the results were grouped into boxes to indicate separation and cohesion. Each box can be enlarged to show more details.

There was an overall sparse use of color. Colors are used purposefully to draw attention to important information and to push less important details to the background. The charts are mainly made with varying shades of grey. The use of brand coloring and a complementary color, are used as the attention grabbers.

Interactions were kept simple and have been kept to a minimum. A box can be zoomed in for more details and a larger graphic. Additionally, graphics have tooltips, i.e., messages that appear when a cursor is positioned over parts of the visualization that hold more information that might not be easily inferred from the graphic.

5.3 Visualizations

As the discovery phase had identified for whom and for which purpose these visualizations are meant, i.e., communicating the results from an eParticipation to the participants and decision makers, the main challenge left to be solved in the design phase was choosing the proper information visualization types for the eParticipation data. Some would say this is an art on its own and could if applied properly increase the accessibility of the visualizations [25].

(30)

In a fairly straightforward approach, the choice is based on the type of data that is to be visualized [27][7][2]. As established in the pre-study phase [15][5], the data produced by eParticipation processes is generally in the following formats:

● Text

○ Single text(s)

○ Collection of text(s)

● Voting

○ Ranked lists

○ Ranked grid

○ Likert scale

The distinction between a single text and a collection of texts for textual data, was appropriated from Nualart-Vilaplana [15]. A few data visualizations that are possible with these types of data are shown in table 1.

All the visualization types that have been placed in table 1, have been chosen because of their high accessibility and because they map to the data types [2][26][15][16].

Table 1: Visualizations

Finally, the comparison feature enables more complex visualizations. In order to smooth the process of compare multiple graphs to identify changes, it was deemed more appropriate to combine certain graphs into one. These would be parallel coordinates graphs, as well as line and bar charts with multiple series. An example of a dashboard comparison can be seen in image 6.

Data Type Visualization

Text - Single - Raw data, Quotes

- Wordcloud - Bubble chart

Text - Collection - Wordcloud

- Bubble chart - Trees

- Sunburst

- Timeline of quotes Voting - Ranked List - Single Choice - Text

- Pictorial - Line chart - Bar chart - Tree map Voting - Ranked Grid - Multiple Choice - Text

- Scatter plot - Bubble chart

- Bar chart with multiple series Voting - Likert Scale - 100% horizontal bar chart

- Multiple series bar chart

(31)

"

IMAGE 6: Comparing dashboards

5.4 Process and Usability Testing

During this design phase, usability testing was used to improve the prototype after each iteration. There were three iterations. The first half was creating a design. Followed by the second half, which was a usability test of the created design.

There were three design iterations in total. A design period was the start of each iteration, in which prototypes of varying levels were created. After each design period, a test was held in the form of an interview, with potential users, to identify issues with the proposed design.

The issues range from small interface improvements to deeper usability problems that would impact a user’s understanding of the system or the accessibility of the visualizations.

The interviews were set up as cognitive walkthroughs. This type of test setup was established by the Nielsen Norman Group [4]. It is a fundamental part of human computer interaction and widely practiced. The focus is on an iterative process of frequently testing designs with small groups of people. Nielsen has proven that five participants will find 85%

of the usability problems. More elaborate research, i.e., with more participants, would be a waste of resources. Instead, those resources should be used to test future designs.

(32)

During the test, interviewees were asked to interact with the design, think aloud while doing so, and answer questions to determine if the design was understood. Each interview lasted around sixty minutes. Each design was tested with at least five users. The designs were iteratively improved based on the received feedback from the interviewees.

The interviewees were recruited from a cafe on the KTH Royal Institute of Technology university campus. They were selected based on them not being data analysis experts.

There was no overlap between this group of testers and the group that participated in the final evaluation, described in chapter 6.

This design process has led to the concept presented in this chapter and shown in image 4.

The whole sequence of prototypes can be found in Appendix B and the complete findings of all the walkthroughs can be found in Appendix C.

Images 7, 8 and 9 show the evolution that resulted from the process, through screenshots of the three iterations that were done. Image 7 shows the initial wireframe, sketched on paper.

The second iteration was created in Sketch, using random data to create the graphs, and is shown in image 8. The final iteration is shown in image 9, and was created using the UNICEF report on adolescent participation [17].

"

IMAGE 7: First Iteration

(33)

"

IMAGE 8: Second Iteration

"

IMAGE 9: Third Iteration

(34)

(35)

6 Evaluation

The final phase of the project was aimed at evaluating the design and assessing whether it upholds the hypothesis. After the design phase, a pilot study was done to accomplish this task. The main focus of this pilot study was whether the storyboard dashboard design that was described in chapter 5, does indeed ease the task of interpreting the results by communicating these results in a more accessible manner.

In order to test this hypothesis, a pilot study was set up. This pilot study setup consisted of two independent groups performing the same tasks, yet with one different setup condition.

This type of setup is called a between subject design, and it was propagated by Cook and Thomas [19]. Because there is only one varying condition in the test, this method allows us to keep the test groups small and still achieve significant results.

6.1 Setup

The varying factor in the setup, was the way the data was represented to the study participants. The first group used the dashboard design from chapter 5 during the test. The other group was using the current state of the art solution, which was a written report from a participation. The dashboard was created using the data from this same report, therefore the data shown was the same in both representations of the data.

The data used was from a report on participation, which was performed and compiled by UNICEF on a study that investigates adolescent and child participation in policy making on a global scale [17]. This report contained a collection of various eParticipation process, of which the most useful data originated from the surveys, interviews and discussion groups documented in the report.

6.2 Participants

Each test group consisted of five participants, therefore there were ten participants in total during the pilot study. Each test lasted thirty minutes, during which the participants were asked to answer a set of five questions about the data. The questions were formulated as tasks, which were aimed at finding key pieces of information from the information visualizations. For each task, the following was noted: whether the participant managed to find the correct answer, how long it took to complete the task, how difficult they perceived the task to be and whether the participant found the visualization aesthetically pleasing based on the Likert scale. Consequently, this pilot study setup means that comparing the results, of the two test groups, would answer which representation is better at communicating the eParticipation results.

All the participants in the evaluation pilot study have voluntarily taken part in the interviews and were not given any compensation for their time. They were selected based on them not being data analysis experts. It was explained to each participant for which purpose the data was being gathered and they have given their consent accordingly. No private information was collected. This pilot study setup ensured the dependability of the conclusion and confirmability that there are no personal assessments affecting the results.

(36)

Furthermore, there was no overlap between this group of testers and the group that participated in the usability testing during the design phase, described in chapter 5.4. This was done in order to prevent an unfair advantage, through familiarity with the concept or data, during the test.

6.3 Data

For the pilot study, one group of the participants was provided with pages 19 to 28 from the UNICEF report, as well as the front cover and executive summary on page v. These pages were selected since they hold a dense amount of key findings from the report.

The dashboard shown to the other group was the one shown in image 4. The creation of the dashboard was previously described in chapter 5. The data shown in this dashboard is from the same UNICEF report pages. The key messages from the report were selected and put into the storytelling dashboard format.

Both groups were asked the following five questions:

1. What was UNICEF investigating with this evaluation?

2. Which issue is better understood by staff, adolescent rights or development?

3. Why is the term “adolescent” an issue?

4. On which area do local and global management disagree the most on as lacking in guidance?

5. What are the four future priority areas for adolescent development?

For each of these questions the following data was to be collected:

● The participants answer.

● How many seconds it took to find the answer.

● How difficult the participant perceived finding the answer was, on a Likert scale.

● How aesthetically pleasing the participant perceived the representation of the answer to be, on a Likert scale.

The data was collected through interviews with the participants during the test and stored in a data sheet; along with their comments on the design and test. The analysis of the results is described in chapter 7 and discussed in chapter 8.

(37)

7 Results

The evaluation was performed as it was described in chapter 6. There were ten participants in total. Group 1 performed the test with the design created for this project and group 2 performed the test with the UNICEF report [17]. The test was done in a cafe on KTH campus. The environment was not completely free from distraction, but the noise was relatively low and the participants were in a stress-free environment, without distractions from other tasks. The participants were not compensated for their time. The rest of this chapter will compare the results collected during the pilot study from the two independent groups.

Each participant was asked to answer five question about the data they were shown. For each of the question the following four things were recorded:

● Correctness of the answer.

● Time in seconds to find the answer.

● Perceived difficulty of the questions, on a Likert scale where 1 was very difficult and 5 was very easy.

● Perceived level of aesthetics of the visualization, on a liked scale where 1 was ugly and 5 was beautiful.

Chart 1 shows the comparison of the percentage of incorrectly answered questions for each group, grouped by each question. Except from question 2, the participants from group 1 had a higher rate of correct answers for all the other questions.

"

CHART 1: Percentage of correctly answered questions by group

Chart 2 and 3 show the times it took the participants to complete each question. The variable ‘P’ stands for the participant. The design clearly lowered the necessary time to find the information for group 1. Which would indicate that the dashboard design makes it easier to find the relevant information.

(38)

"

CHART 2: Completion Time for each question by group 1

"

CHART 3: Completion time for each question by group 2

(39)

"

CHART 4: Level of difficulty from group 1

"

CHART 5: Level of difficulty from group 2

(40)

"

CHART 6: Aesthetically pleasing level by group 1

"

CHART 7: Aesthetically pleasing level by group 2

(41)

By comparing charts 4 and 5, it is evident that group 1 perceived the questions to be medium to easy to answer. However, the difficulty was rated to be significantly higher by group 2, as their ratings range from medium to high difficulty.

Lastly, from chart 7 we can conclude that group 2 mostly perceived the aesthetics of the report in a neutral way, while some participants rated it ugly. Chart 6 shows that group 1 generally perceived the design to be pleasant.

These findings, while preliminary, suggest that the design described in this report indeed does show promising results in regards to increasing the accessibility of the information visualization by applying the storytelling with data approach and creating aesthetically pleasing visualizations. However, further study is deemed necessary to determine this with more certainty.

The results do indicate that perhaps the the UNICEF report was too difficult to analyze. The participants from that group commented that they did not have the patience to read through the amount of text. Which is understandable, as it took considerably longer to analyze the report and create the dashboard and pilot study from it, than the participants spent analyzing it to answer the questions.

While the UNICEF report wasn’t the most ideal source of data, it was the only available open data source that held this type of information, i.e., extensive qualitative data from a participation. Other options were either not in English, were payed, were protected company information or were not open data.

(42)

Accessibility of Information Visualization for eParticipation

Accessibility of Information Visualization for eParticipation

BOJANA DUMELJIC

Abstract

Abstrakt

Table of Contents

1 Introduction

1.1 Background

1.2 Problem

1.3 Purpose

1.4 Goal

1.5 Method

1.6 Delimitations

1.7 Outline

2 Background

3 Method

3.1 Pre-Study Phase

3.2 Design Phase

3.3 Evaluation Phase

4 Pre-Study

4.1 Literature Findings

4.2 Related Work

4.3 Existing Solutions

4.3 Observations

5 Design

5.1 Concept

5.2 User Experience

5.3 Visualizations

5.4 Process and Usability Testing

6 Evaluation

6.1 Setup

6.2 Participants

6.3 Data

7 Results