Real-time visualization of a digital learning platform

Department of Science and Technology Institutionen för teknik och naturvetenskap

LiU-ITN-TEK-A--17/035--SE

Real-time visualization of a

digital learning platform

Kristina Engström

Mikaela Koller

LiU-ITN-TEK-A--17/035--SE

Real-time visualization of a

digital learning platform

Examensarbete utfört i Medieteknik

vid Tekniska högskolan vid

Linköpings universitet

Kristina Engström

Mikaela Koller

Handledare Jimmy Johansson

Examinator Daniel Nyström

Upphovsrätt

Detta dokument hålls tillgängligt på Internet – eller dess framtida ersättare –

under en längre tid från publiceringsdatum under förutsättning att inga

extra-ordinära omständigheter uppstår.

Tillgång till dokumentet innebär tillstånd för var och en att läsa, ladda ner,

skriva ut enstaka kopior för enskilt bruk och att använda det oförändrat för

ickekommersiell forskning och för undervisning. Överföring av upphovsrätten

vid en senare tidpunkt kan inte upphäva detta tillstånd. All annan användning av

dokumentet kräver upphovsmannens medgivande. För att garantera äktheten,

säkerheten och tillgängligheten finns det lösningar av teknisk och administrativ

art.

Upphovsmannens ideella rätt innefattar rätt att bli nämnd som upphovsman i

den omfattning som god sed kräver vid användning av dokumentet på ovan

beskrivna sätt samt skydd mot att dokumentet ändras eller presenteras i sådan

form eller i sådant sammanhang som är kränkande för upphovsmannens litterära

eller konstnärliga anseende eller egenart.

För ytterligare information om Linköping University Electronic Press se

förlagets hemsida

Copyright

The publishers will keep this document online on the Internet - or its possible

replacement - for a considerable time from the date of publication barring

exceptional circumstances.

The online availability of the document implies a permanent permission for

anyone to read, to download, to print out single copies for your own use and to

use it unchanged for any non-commercial research and educational purpose.

Subsequent transfers of copyright cannot revoke this permission. All other uses

of the document are conditional on the consent of the copyright owner. The

publisher has taken technical and administrative measures to assure authenticity,

security and accessibility.

According to intellectual property law the author has the right to be

mentioned when his/her work is accessed as described above and to be protected

against infringement.

For additional information about the Linköping University Electronic Press

and its procedures for publication and for assurance of document integrity,

please refer to its WWW home page:

Link¨

opings Universitet

Real-time visualization of a

digital learning platform

Kristina Engstr¨om Mikaela Koller Examiner: Daniel Nystr¨om

Tutor: Jimmy Johansson

Abstract

This master thesis report presents the development and results of a dashboard developed for Sanoma Utbildning. The learning platform, called SUD - Sanoma Utbildnig Digital, is a combination of exercises and reading material used by both students and teachers. The pur-pose of the application is to provide Sanoma Utbildning with valuable insights of the activity in SUD and was developed using agile meth-ods. The dashboard should be easy to understand for both employees at Sanoma Utbildning and possible customers. The user interface has been developed with help from theory and by conducting three user evaluations. The data is collected from logs produced by SUD and sent with webhooks to the dashboard. The result is a web based MEAN-stack application visualizing real time data and statistical data of the events in the learning platform.

Acknowledgements

We would like to thank Sanoma Utbilding and Frost for giving us the opportunity to do our master thesis at their companies. Especially thanks to Madeleine Shemweta for answering all our questions and Toste Junestrand for always making sure we were doing ok. Thanks to our supervisor Daniel Nystr¨om and a special thank to our tutor Jimmy for all your help! ♥

CONTENTS CONTENTS

Contents

1 Introduction 1

1.1 Sanoma Utbildning and SUD . . . 1

1.2 Background and problem description . . . 1

1.3 Research questions . . . 3

1.4 Limitations . . . 4

2 Background and related work 5 2.1 Information visualization . . . 5 2.1.1 Cognitive mechanisms . . . 5 2.1.2 Perception . . . 5 2.1.3 Data types . . . 6 2.2 Visualization techniques . . . 8 2.2.1 Pie chart . . . 8 2.2.2 Bar chart . . . 9 2.2.3 Line chart . . . 9 2.2.4 Area chart . . . 10 2.2.5 Scatter plot . . . 10 2.3 Usability . . . 11

2.3.1 Designing information graphics . . . 11

2.3.2 The Visual Information Seeking Mantra . . . 14

2.3.3 Colour . . . 14 2.3.4 Pattern of behaviour . . . 16 2.3.5 Scientific Evaluation . . . 17 3 Method 19 3.1 Development methodology . . . 19 3.2 User study . . . 19 3.3 Data retrieval . . . 20 4 Implementation details 21 5 Results 23 5.1 First evaluation . . . 23 5.2 Second evaluation . . . 24 5.3 Third evaluation . . . 25 5.4 Final prototype . . . 26 6 Discussion 30 6.1 Online users - pie chart . . . 30

6.2 Daily login activity - area chart combined with line chart . . . . 30

6.3 Weekly login activity - stacked bar chart . . . 31

6.4 Top three most used books - line chart . . . 31

CONTENTS CONTENTS

6.6 The charts overall . . . 32

6.7 Development process . . . 33 6.8 Frameworks . . . 33 6.9 User testing . . . 33 7 Future work 35 8 Conclusion 36 A Scientific evaluation 39 A.1 Evaluation 1 - Questionnaire . . . 39

A.2 Evaluation 2 - Interviews . . . 45

A.3 Evaluation 3 - Interviews . . . 49

B Sprints 52

CONTENTS CONTENTS

Abbreviations and explanations

Abbreviation Explanation

D3 D3 is a JavaScript library for visualizing data with HTML, SVG and CSS.

Git Git is a version control system. Git make it easier when working with code projects in group.

GitHub Github is used to save data in the cloud. This minimalises the risk of loosing code.

Heroku Heroku is a cloud application platform and is used to host the application online.

MEAN 2.0 It is a full-stack development toolkit of the JavaScript-based technologies MongoDB, Express.js, Angular 2 and Node.js. MongoDB MongoDB is a NoSQL database. MongoDB is free and open

source.

Papertrail A cloud base service where all the logs produced by SUD is saved.

Socket.io Socket.io is a Javascript library enabling real-time event-based communication.

SUD Sanoma Utbildning Digital, a digital learning platform for teachers and students in middle school.

Trello Trello is a web-based project management application. It was used to manage the backlog during the project.

Ultrahook Ultrahook is a webhook used locally.

Webhook A webhook is a HTTP callback that occurs when something is triggered.

1. Introduction

1

Introduction

This is a master thesis written at the Media Technology and Engineering -Master of Science in Engineering program at ITN1

, Link¨oping University. The thesis has been done in collaboration with Frost2

and Sanoma Utbildning3

. This following section will describe the background of the thesis work.

1.1

Sanoma Utbildning and SUD

Sanoma Utbildning (hereafter simply Sanoma) is a company situated in Stock-holm producing teaching material for teachers and students in middle school since 1993. All literature is written by teachers and almost all employees are former teachers. Sanoma are now testing a beta version of a new digital learn-ing platform in a number of schools providlearn-ing learnlearn-ing material and exercises called SUD - Sanoma Utbildning Digital.

SUD is a digital learning platform with complete teaching materials. Students can read theory and do exercises just like they would do in a physical book (see figure 1). There are many different types of exercises for the students to do. Teachers can follow the students’ progress and results, both in student and class level (see figures 2 and 3).

SUD is developed by Frost; a digital agency in Stockholm building mobile apps and web solutions. Sanoma is one of Frost’s clients and SUD is one of the projects Frost is developing for Sanoma.

1.2

Background and problem description

Sanoma wish to have an overview of what is happening in SUD. They would like to have both real time data and statistical data saved over time. A list of requirement specifications is shown below.

• Total number of users in SUD right now – How many of these that are teachers – How many of these that are students • Statistics saved over time of log-ins

1

Department of Science and Technology

2

www.frostdigital.se

3

1.2 Background and problem description 1. Introduction

Figure 1: This is how SUD looks in theory and exercise view for a teacher. In this figure, the exercise panel is expanded, visible on the right side. The teacher can see exercises for a specific chapter: Kapitel¨ovningar and interactive exercises: Interaktiva ¨ovningar. In the left hand side theory about algebra is visible.

Figure 2: This is the progress dashboard for a whole class. Every row represents a students progress in each of the chapters. The light blue colour in the progress bar represents unanswered questions and the dark blue represents answered questions.

– A day view – A week view

• Which of the books that are most used by the users • Schools using SUD and where they are situated in Sweden

• How many real time exercises that have been started during the past week

This information is supposed to be displayed with a dashboard on a big screen at Sanoma’s office. They want this dashboard for their own sake to know what is happening in SUD, but they also want to be able to show it to potential costumers.

1.3 Research questions 1. Introduction

Figure 3: This is the progress dashboard for a single student. On the top of the figure, progress bars for every chapter is visible. The light blue colour in the progress bar represents unanswered questions and the dark blue represents answered questions. Below the progress bars the progress for a particular chapter is visible, in this case the chapter 1. Statistik is selected. Each row is a sub chapter and each column is a exercise type. Each circle with a line means that the sub chapter does not have exercises of that type, see Tabeller/Uppv¨armning. A light blue circle is a exercise type with no progress, see Tabeller/Niv˚a ett. A circle with multiple colours is a exercise type with progress, see Avl¨asa och tolka diagram/Niv˚a ett. The colours have different meanings: green = correct, red = incorrect, yellow = unassessed, gray = question with key and light blue = unanswered.

The data from SUD will be obtained from user logs. Unnecessary data will be filtered out and useful data will be processed and saved in a database. The thesis will focus on how to visualize the data as user-friendly as possible. It has to be clear which data that is real time and which is statistical data. Since the dashboard mainly will be shown on a screen in an office, there will be none or limited interactions with the visualization. Therefore it has to be intuitive and easy to understand.

The project will be developed using agile methods and a user centered devel-opment technique. The sprints will be short and at least two user evaluations will be held.

1.3

Research questions

• How can data from a learning platform be visualized to give relevant information for the product owners to see how the product is used? • How can data from a learning platform be visualized to give relevant

information for possible clients interested to buy the product? • How can real time data and statistical data be combined together?

1.4 Limitations 1. Introduction

1.4

Limitations

The project will result in a web application, therefore it will require Inter-net connection from the user. The application will be developed with Google Chrome as main web browser since it is the browser that the developers prefers and is the most used web browser according to Sitepoint [1]. Therefore, possi-ble errors occurring in other browsers will not be considered in development. Except those provided from Frost, frameworks and development environments that are free of charge will be used in the project.

2. Background and related work

2

Background and related work

This section describes the theoretical background to the thesis work. It was used as a pre-study for the work.

2.1

Information visualization

Information visualization is the study of transforming abstract data to visual representations easy for a user to understand [2]. One great benefit with data visualization is the quantity of information that can be rapidly interpreted if it is presented well. Visualizations can also reveal properties that were not anticipated and give the opportunity to understand huge amounts of data. 2.1.1 Cognitive mechanisms

The brain structure and cognitive mechanisms have enabled humans to achieve the huge knowledge that exists today [2]. However a person working with thinking aid tools is much more cognitively powerful than a person only work-ing with the thoughts. The tools can mean workwork-ing with paper and pens or working with advanced computer-based systems such as MATLAB. Visualiza-tions are an increasingly important part of cognitive systems, partly because a person acquire more information through vision than all other senses com-bined. To improve a cognitive system, the search for data and possibility to see patterns must be easier.

Graphic representations of data must be designed by adapting to human sen-sory capabilities in a way that important data patterns and elements can be quickly perceived [2]. The important data should be represented with graphical elements that are visually distinct and therefore easy to find.

2.1.2 Perception

The human perception, the state of becoming aware of internal or external stimuli or events, has an important role in visualization [2]. The degree to which a data visualization effectively communicates depends on how well the visual perception is considered. The human visual system seek for patterns. But if the patterns are presented in ways that do not consider the perception by the user, the patterns become invisible or misleading. At first, the hu-man vision extract features, orientation, color, texture and movement patters. Second, active processes in the brain divide the visual field into regions and patterns such as regions of the same color or texture. By following the rules

2.1 Information visualization 2. Background and related work of perception, data can be presented in ways that makes the important and informative patterns stand out.

Further, the number of data components must be limited to seven at most in a graph [3]. More components cannot be held in short term memory and forces the user go back and forth between explanation and the components in the graph. Also, preattentive visual attributes should be used when encoding the data to make certain data stand out from the rest.

2.1.3 Data types

Generally data is categorized into two categories: quantitative and categorical. Quantitative data is the numbers that measure things and categorical data identifies what the numbers measure [4]. In a graph, the quantitative data represents the values and the categorical data can be text labels telling the user what the values represent. Graphs almost always include at least one quantitative scale and one or more categorical scales.

As presented in [4], the interval of the data must be handled with care, oth-erwise the data may be presented incorrectly giving the user misinformation, see figure 4. The graphs illustrate the division between ages in a company, presenting it differently depending on the intervals. The graph to the left has equal interval scales and the other one has unequal intervals. The intervals should always be equal for a correct presentation.

Figure 4: Both figures show the number of employees for different age groups working at a company. The left figure show the data scaled with equal intervals and therefore showing the correlations correctly. The right figure show the data scaled unequal and different from the left figure. Therefore, changing the scale interval may lead to misinformation.

Data presentation has a number of possibilities. The choice of which graph to use should be motivated by identifying the quantitative message type and understanding the relationship between quantitative data and graphical meth-ods.

2.1 Information visualization 2. Background and related work Data can also be time-oriented, meaning data that is connected to time. A re-quirement for achieving expressive and effective visualization is to consider the characteristics of the data to be presented [5]. Important aspects to consider are listed below:

• Linear time vs cyclic time: Linear time has a starting point and defines a linear time domain from past to future. Many natural processes are cyclic such as the seasons and to visualize such phenomena a cyclic time domain can be used.

• Time points vs time intervals: Time points have no duration in contrast to time intervals that has an interval scaled time domain such as days or years.

• Ordered time vs branching time vs time with multiple perspec-tives: Ordered time intervals describe things that happen one after the other. Branching time facilitates description and comparison of alterna-tive scenarios with multiple time strands branching out. Can be used for project planning and supports decision making where only one alterna-tive will happen. Time with multiple perspecalterna-tives allows more than one point of view.

Figure 5: Different representations of a time-oriented dataset with cyclic behaviour. Demon-strating the benefit with a cyclic graph and the importance of correct parameterization. Left: time series plot where it is hard to recognize repetitive patterns. Center: spiral graph with incorrect parameterization, making it hard to see if any patterns exist. Right: spiral graph with correct parameterization making it easy to recognize cyclic patterns.

When looking for cyclic time patterns in a simple time series plot it is easy to recognize recurrent peaks in time, see figure 5 (left). But it is hard to see if repetitive quantitative patterns exist. The spiral graph one the other hand focuses on cyclic characteristics by using a spirally shaped time axis. That may reveal unknown periodic behaviour but requires correct parameterization of the visualization method, see figure 5 (center and right). But finding the correct parameterization can be difficult for unknown data sets. One solution is to spot patterns when animating through possible parameter settings [5].

2.2 Visualization techniques 2. Background and related work All aspects listed are characteristics that influence the appropriateness of the visualization methods and is a must for a properly insight into time-oriented data [5]. Integrating appropriate interaction methods is also a key concern. In-teraction enable the user to re-parameterize and facilitate user tasks including navigation in time, search, comparison and manipulation.

Concluded, to achieve expressive and effective visual representations, it is re-quired to choose the appropriate techniques, parameterize them correctly and have useful interaction methods.

2.2

Visualization techniques

Large amount of data are generated and stored in various formats. The is-sue today is how to represent this data and extract valuable knowledge [6]. The goal of visualizing data is making huge amounts of complex information accessible and easy to understand for a user.

Choosing the correct graph is not an easy task, it has to give the correct message to the viewer. It is not the data, it is not the measure, it is the message [7]. If visualizing the data incorrectly it can actually end in tragedy. In Stephen Few’s The Information Cannot Speak for Itself [8] he mentioned that the 1986 explosion of the space shuttle Challenger [9] could have been avoided if the potential risks of O-ring failure had been presented in a better way. Now, most graphs and charts may not risk lives if being misleading but it is good to have in mind what could happen if a bad implementation is chosen. 2.2.1 Pie chart

A pie chart, also called circle graph, is divided into a number of sections describing a proportion in a whole quantity [6]. The proportions of the slices are usually shown in percentage which can be effective when comparing one slice with the rest of the slices in the pie chart. Figure 6 shows a basic pie chart.

Pie charts have received a lot of criticism. Mico Yuk et al [10] presents in their book Data Visualization for Dummies that searching for avoid pie chart in a search engine will generate more than 1 million entries. However, if used correctly pie charts can be very effective. There are two main points to be aware of when creating a pie chart:

• Do not have more than five slices - If the slices gets too small the viewer will ha hard time reading the graph and may risk to loose the main point.

2.2 Visualization techniques 2. Background and related work

Figure 6: A pie chart with six slices. Each slice have a different colour and the size in percent. The lilac slice is the biggest with 40% and the green is the smallest with 4%. The total percentage sums up to 100%.

• Do not have slices of relatively equal size - The point with a pie chart is to compare the size of the slices. If the slices are too equal in size, the pie chart will not give much to the viewer.

2.2.2 Bar chart

Bar charts are the most common visualization method and is mostly used for discrete data [6]. Each bar represent an object in the database, the length of the bar represent the value of the data set. This type of chart is good to use when comparing activities that occurs within a period of time [7], for example compare sales between 2000 - 2004 where each year represent a bar (see figure 7). Bar charts can easily be cluttered if comparing to many values. When the amount of bars is greater than eight, it should be considered using another type of graph.

2.2.3 Line chart

Line charts, just as bar charts, are one of the most commonly used in visu-alization [7]. Each dot in the line chart represent a value in the data set, a line goes through all dots to create a curve. A line chart’s message is clear, it shows weather a trend is increasing, decreasing or constant. With the usage of several data sets it is possible to compare different trends and see patterns, i.e if data set A is increasing, will data set B decrease and vice verse? See figure 8 for an example of a line chart. Just as with any graph, the number of lines should be limited. With too many lines the graph gets cluttered making it hard to read.

2.2 Visualization techniques 2. Background and related work

Figure 7: This is a bar chart with five bars each representing a year. The length of the bar is based on the value in the data set. A bar chart works very well when comparing values during a period of time. This particular chart could represent sales between 2000 - 2004. The red bar representing year 2005 has the highest value and the lilac bar representing 2000 has the lowest.

2.2.4 Area chart

An area graph is similar to the line graph but with the area between the line and the baseline coloured [7]. The most important segment should be placed at front and the others behind. Try to limit the segments to at most five. If there are more than five segments, keep the four most important separate and combine the rest to an ”others” category [7].

A problem with area charts is that any segment that is not upfront risk to get hidden behind other segments. One solution for this is to have transparent layers, see figure 9.

2.2.5 Scatter plot

The graph to use when showing correlation is the scatter plot [4]. In a scatter plot each data point from the data set is drawn one by one. In a scatter plot it is possible to identify patterns and clusters in the data. For example, if a value increases does the other value increases or decreases in a similar manner? It is also easier to identify data items that differ from the rest of the data [6]. In a scatter plot each point is defined by two parameters; position on the x-axis and on the y-x-axis. There is a similar chart called bubble chart where each point has a third parameter; the area of the bubble. A scatter plot is shown in figure 10.

2.3 Usability 2. Background and related work

Figure 8: A basic line chart comparing two trends between 2000 - 2008. The green line is decreasing between 2000 - 2005 and starts to increase after 2006. The blue line is mostly increasing between 2000 - 2007 and then a hard abatement after 2007.

2.3

Usability

The importance of designing good interfaces has become more apparent the last years. But the human mind is still the same, meaning visual hierarchy is still applicable and patterns based on fundamental design principles are as valid as they were twenty years ago [11]. However it is still not easy to design good interfaces because users’ expectations is higher than they used to be. If a site is not fast enough or is not designed as the users prefer, they will probably switch to a competitor’s site instead. Hence the cost of designing a mediocre interface is higher than it used to be.

2.3.1 Designing information graphics

Information graphics means data presented visually with the goal of giving knowledge to the user [11]. The purpose of information graphics for a user is to learn something. The designer need to know this and design thereafter. The user may want to find something specific or get an overview. The ability to search, filter and zero in on details is therefore crucial. Good interactive infor-mation graphics give the user the possibility to see how the data is organized, related and give possibility to rearrange and filter.

Further, a visual hierarchy build on the fact that the most important content should stand out the most and the least important should stand out the least [11]. The user should be able to distinguish the elements and deduce the informational structure.

2.3 Usability 2. Background and related work

Figure 9: Left: An area chart comparing three trends. The most important segment is the blue one, hence why it is placed foremost. Behind the blue one there is a green and then a yellow. A problem with area charts is that segments can cover other segments that are behind. A solution to this is to make the segments transparent, see the figure on the right.

Figure 10: This is a scatter plot. Scatter plots are used to easy identify clusters and patterns in data. In this figure there is one big cluster and one point far off to the right. In scatter plots the data has two parameters, position on the x-axis and position on the y-axis.

The hierarchy can be set by changing text attributes such as size, color, weight, position etc. A user can ignore things that they do not think they need to look at and only focus on the parts of the page that they think is important. To control the readers’ eyes as they scan a page, focal points are used. Focal points are spots the eyes cannot resist going to. That pulls the eyes to the right places in the right order. The visual flow in a page can be manipulated by using implied lines or a call to action button.

How to arrange the visual attributes is presented by Jenifer Tidwell [11]. She present the theory behind grouping and alignment that was developed early by psychologists in the 20th century and are still used when designing interfaces. The theory was put into a number of gestalt principles listed below.

2.3 Usability 2. Background and related work

Figure 11: The five different gestalt principles. Top left is showing proximity: grouping things together. Top right is similarity: the viewer group things depending on their shape, size, colour or orientation. Middle left displays continuity: the eye search for continuous lines and curves by combing smaller elements. Middle left shows closure: searching for rectangles and blobs in white space. Bottom: All previous principles mentioned above combined to one.

them.

• Similarity: If objects look similar - having the same shape, size, color or orientation, they will also be grouped and associated with each other by the viewer.

• Continuity: Our eyes want to see continuous lines and curves formed by the alignment of smaller elements.

• Closure: Our eyes also want to see closed forms such as rectangles and blobs of whitespace. The viewer can perceive an incomplete object by filling in the missing information.

These design principles are built on the human visual system and how it re-sponds cognitively to certain inputs as explained in sections 2.1.1 and 2.1.2. See figure 11 to understand the principles further.

2.3 Usability 2. Background and related work 2.3.2 The Visual Information Seeking Mantra

One popular guideline to follow when designing information visualization is ”The Visual Information Seeking Mantra”, presented by Ben Shneiderman in 1996 [12]. The mantra describes how data should be presented to users to be as effective as possible. The mantra follows: Overview first, zoom and filter, then details-on-demand. The mantra can be expanded to seven high level abstract tasks:

• Overview: Get an overview over the entire collection • Zoom: Being able to zoom in or out on point of interest • Filter: Filter out uninteresting data

• Details-on-demand: Select data and get more details • Relate: See relationships between data

• History: Being able to go back to previous steps • Extract: Allow users to extract data

Since it is guidelines, it only gives a direction on what to think about when designing information visualization. Not all of the tasks can be applied on all applications. Craft and Cairns discuss the importance of Shneiderman’s mantra in their paper ”Beyond Guidelines: What Can We Learn from the Visual Information Seeking Mantra” [13]. They describe the mantra as a guideline that serve as recommendation for a certain approach to a particular problem. Even though Shneiderman’s mantra has been mentioned in numerous papers there is no study to validate Shneiderman’s recommendations.

2.3.3 Colour

There are several factors to be aware of when choosing colour for an interface as presented by Jeff Johnson [14]. First of all, it is important to understand the human colour perception. The way the human visual system is designed makes it easier to distinguish differences in contrast between colours and edges than different levels of brightness. Edward H. Adelson [15] created the illusion shown in figure 12. It is a perfect example of the humans insensitivity to absolute brightness and the sensitivity to contrast. Even thought it looks like square A and B have two different colours they are actually the same. Square B appears to be brighter since it lays in the shadow of the cylinder.

The ability to differentiate colours depends on how they are represented. There are some factors to be aware of.

2.3 Usability 2. Background and related work

Figure 12: In the left image the shadow behind the cylinder creates the appearance that square A is darker than square B. However, by looking at the right image it is clear that this is not the case. This example shows the humans insensitivity to absolute brightness and the sensitivity to contrast. (Source: ©Edward H. Adelson, 1995 - Checkershadow Illusion, http: //persci.mit.edu/gallery/checkershadow)

• Paleness: The less saturation two colours have, the harder it is for humans to tell them apart.

• Colour patch size: The smaller an object is, the harder it is to see what colour it has. For example, if the lines in a line chart is to thin, the viewers will have a hard time separating which colour patch belong to which line.

• Separation: The more separated colour patches are, the harder it is to distinguish their colour from another.

When choosing colours for information visualization it is important to take colour blindness in mind. Approximately 8% of all men and 0,5% of all women have some sort of colour blindness [14]. Fortunately there are colour-blindness simulators4

to use when picking colours. Figure 13 shows the colours in one of the earlier prototypes in different types of colour blindness.

John Johnson[14] ends his chapter about colour with a few guidelines to have in mind when choosing colours:

1. Separate colours using saturation, brightness and hue

2. Use distinctive colours: Red, green, blue, yellow, white and black 3. Avoid to pair colours that color-blind people cannot separate

• Do: Dark red-light yellow, dark blue-light green

• Do not: Dark red-black, blue-purple, light green-white

4

Coblis color blindness simulator lets the user see how their images are per-ceived by those who are colour blind. URL: http://www.color-blindness.com/ coblis-color-blindness-simulator/

2.3 Usability 2. Background and related work

Figure 13: This figure shows how the colours in a early version of the dashboard would be perceived in three different types of colour blindness. Top left: The dashboard in original colours. Top right: Protanomaly - weakness for red colours. Bottom left: Deuteranomaly - weakness for green colours. Bottom right: Tritanomaly - weakness for blue colours.

4. Use colour with other elements: Combine colours with other ele-ment, for example colour and a symbol

5. Do not place two strong colours right next to another: Guideline 2 mentioned to use distinctive colours, but using them to close to each other can be quite disturbing, see figure 14

Figure 14: Combining four very strong colours, blue, yellow, red and green, is not a recommen-dation. It gets to bright which can be disturbing for the viewer.

2.3.4 Pattern of behaviour

John Johnson [14] presents how humans have a cyclical pattern that appears in many different activities: find a goal, execute to make progress towards the

2.3 Usability 2. Background and related work goal, evaluate the actions to see if it worked, repeat until the goal is reached. People runs through this cycle constantly and in different levels at the same time. Bigger tasks can be divided into sub tasks. As an example, a typical task for a student who studies at Campus Norrk¨oping; drink a cup of coffee. The primary goal is to drink coffee, but to get there the goal has to be divided into sub goals; i.e go to Studentfiket and buy coffee. These sub tasks can be divided into sub tasks.

Drink a cup of coffee: 1. Go to Studentfiket

(a) Calculate best route to Studentfiket (b) Follow route to reach destination 2. Buy coffee

(a) Stand in line for the coffee (b) Pour coffee in cup

(c) Pay for the coffee

After each sub task is finished, the sub task is evaluated to see if it helped to reach the sub goal. To have user cases like this when designing the layout of an application is good. If it is hard or tedious for the user to reach its goals, then the user might stop using the application, which is usually not the goal for the developers.

The main goal of the dashboard is for Sanoma to get an overview of their ap-plication. As mentioned in 1.2 Background and problem description, Sanoma among others would like to know the amount of online users in SUD right now and how many of these that are teachers and students. These could be considered as a sub goal. When Sanoma is trying to achieve this sub goal they do not want to confuse the values of number of users online right now with the value of users who logged in to SUD yesterday. Therefore it has to be apparent which value represent which goal.

2.3.5 Scientific Evaluation

Scientific evaluation is a key research challenge in information visualization [16]. The aims to perform an elevation study are many, see examples below.

2.3 Usability 2. Background and related work • Gives insight to why a special technique perform good

• Demonstrates that a new technique is better than an existing one Initially, a research question should be decided depending on the aim of the research, since that will specify what to investigate and how [16]. The next step is to decide whether to examine performance of the same participants or compare different groups. When using the same participants, the difference of experience, motivation and age is the same across all conditions. This is called a related design or a within-subjects design. When comparison is between groups their scores are unrelated, called an unrelated design or a between-subject design. What kind of design to use is dependent of the number of participants and the aim of the study. When testing how easy it is to perceive different elements on a display, it is enough to use a within-subjects design. A within-subjects design is known to be more sensitive to small differences between conditions and are more likely to detect differences if they exist. But if the first task will affect the result in the following, it is important to use the between-subjects design. The number of participants should not be less then 12-14 per group. It is also important to apply randomization and/or balancing the participants to groups and conditions properly. Otherwise the outcome of the evaluation may be meaningless.

The tasks in the evaluation need to be representative, meaning it should aim towards the intended users and have a data set as big and complex as the intended. Even though the test should strive to use real data, sometimes it is more suitable to use a synthetic data set to control structures in the data. Before conducting the evaluation, it may be relevant to collect background information from the participants. For instance, color blindness and stereo vision ability could affect the result. The participants should have understood what they need to know and this should be tested before beginning with the tasks. Whether to be with the participant during the test is another issue. It is usually best to leave the room and let the participant perform the test alone to reduce stress and avoid having the participant asking questions. But by not being present, it is harder to control what happens. A pilot study should be conducted to refine the evaluation.

3. Method

3

Method

This section describes the different methods used in the development process. It discusses the methodologies used in development, the conducted user studies and how the data was retrieved.

3.1

Development methodology

The project was developed with the agile development in mind. The project was divided in sprints of one to two weeks. Tasks was added and sorted in priority in a backlog. Before every sprint started, a planning meeting was held. A list of the project’s sprints can be found in appendix B

Some of the typical agile tasks were not used in this project since the team only consisted of two persons. Examples of these tasks are backlog refinement, stand up and sprint demo. Instead of sprint demos, the team had a close communication with the customer and showed the progress of the application continually.

Coding was done both in pair and separate. Git and Github were used for ver-sion control in the project. The team strived to always have the master branch as a working version of the application, and every new feature was developed in a separate branch. To make sure the code withheld a high standard, code reviewing was performed before merging with the code on the master branch. Trello is a web-based project management application used as backlog in the project. In Trello, the project was divided into boards with the current sprint, what was under development, what was done and what needed to be done ahead. The tool facilitated the planning of the project together with a Gantt chart.

In the end of each working day, the team documented what they had done in a diary. The diary helped when looking back at what the team had accomplished.

3.2

User study

User evaluations were conducted during the development process to ensure the final product would satisfy the customer’s needs and have a user-centered design by being easy to understand. Three evaluations were held, one ques-tionnaire and two sets of interviews.

A typical user of the dashboard is either an employee working at Sanoma, or a possible client. Depending on the role of the employee, the dashboard

3.3 Data retrieval 3. Method will be a tool or an interesting information board. For an employee working with support the dashboard will give valuable information about when extra resources is needed. An employee working with market will on the other hand use the dashboard to see how well it is used and to market the application for possible customers undecided whether to buy it or not. Most users will probably only use the dashboard out of curiosity.

First, a questionnaire (see appendix A.1) was sent out targeting a large number of people. The aim of the questionnaire was to complement the interviews held later in the process and to let some users do the evaluation alone with no stress, which was recommended in section 2.3.5. When closing the questionnaire 38 persons had participated.

With help from the questionnaire, the prototype was updated before the second evaluation was conducted (see appendix A.2). The evaluation was performed on a static prototype made in a prototype tool and was tested on the target group: the employees working at Sanoma. The aim of the evaluation was to test how easy it was to perceive different elements on the display. Ten persons participated.

The third evaluation was done with the real prototype and tested on the em-ployees at Sanoma and Frost, but with new participants. The aim was once again to test how easy it was to understand the elements on the dashboard by measuring how long it took for a user to understand the charts. Therefore the first impression was found as the most important task to test and it was decided to use a between-subjects design as explained in section 2.3.5. Ten people were interviewed.

3.3

Data retrieval

The data was collected from the logs produced by SUD. The application stored all logs in Papertrail, a cloud-hosted log management tool. In Papertrail, searches were saved for logs needed for the dashboard data, i.e when a user logged in, logged out and used a book. For every search a web hook was added. Every minute the web hook checked if there were any new data, if there were any it was sent to the dashboard. The data was stored in a mongoDB database.

4. Implementation details

4

Implementation details

This section discuss the frameworks and techniques used in development of the application.

As web framework, the MEAN framework was used. It is a full-stack de-velopment toolkit of the JavaScript-based technologies MongoDB, Express.js, Angular 2 and Node.js. The toolkit was used because it is well known and it build fast and maintainable web applications. MEAN.stack 2.0 indicates that Angular 2 is used instead of its predecessor AngularJS. Another benefit with using MEAN is the scalability which was an important aspect in the project. MongoDB is NoSQL database who uses JSON-like document with schemas. To manage the database Robomongo was used. Robomongo is a free to use mangement tool. Node.js is lightweight and efficient, it is written in JavaScript and built on Chrome’s V8 JavaScript engine. It is asynchronous event driven and built to build scalable applications. It is commonly used in web applica-tions and therefore easy to find help online. The Express.js framework is a Node.js framework, it is de facto standard framework for server development and API for Node.js. Express.js served as the backend framework in the dash-board. As front-end framework, the recently released Angular 2 was used. Angular 2 is an open source JavaScript framework to build web applications. The framework consists of components, making it possible to create custom HTML tags. TypeScript, Microsoft’s extension of Javascipt, allows the use of all ES2015 features, type checking and object-oriented features. Because not all browsers are compatible with ES2015 yet, TypeScript is compiled to JavaScript code. When working with real time updates, the framework had the advantage of only updating the affected graph instead of updating the whole page.

D3, Data-Driven Documents, was used for the visualizations in the project. It is a JavaScript library bringing data to life using HTML, SVG and CSS. As a complement to D3, the library Ngx-charts were used. It is an angular 2 and D3 chart framework. Because TypeScript did not accept all type checking from the D3 library, problems occurred when implementing examples from the D3 webpage. Therefore it was time saving using the Ngx-charts library for some of the graphs. Even though it was a fast way of creating graphs in TypeScript, it was restricted in how much it could be manipulated compared to the features offered in the regular D3 framework. Therefore it was not used to the graphs containing special features this framework did not offer.

The project required real time updates. For that, Socket.io and webhooks were used. A webhook is an HTTP callback that occurs when something is triggered; like an event-notification. The web application implementing webhooks will send a HTTP POST message to a given URL when a specified

4. Implementation details event happens. Instead of continually retrieve data without knowing if it is updated with new values, webhooks retrieve data only when needed. It is a great tool when working with real time data. In the project webhooks were implemented connecting Papertrail who hosted all logs from SUD to the web application. For receiving webhooks when running the dashboard locally UltraHook was used. UltraHook makes it possible to receive webhooks behind a firewall. There were some problems with the UltraHook crashing, but since it was only used locally it was never a problem when running the application live. Socket.io is a Javascript library for real-time web applications. It enables real-time event-based communication between web clients and servers. The library is popular among large companies such as Microsoft and Trello and is one of the most popular JavaScript frameworks on GitHub. It was therefore chosen to use for the project. When the application received a webhook from Papertrail, the Socket.io library forwarded the data to the client side and the graphs were updated with the new data.

5. Results

5

Results

This section describes the findings and modifications made from each of the evaluations conducted in the development of the dashboard. It also presents the final prototype of the dashboard.

5.1

First evaluation

A majority of the participants thought it was easy to distinguish the different charts. The questionnaire showed that the pie chart was the most popular and easiest to understand and the area graph was the most difficult. The participants wanted labels on the axises on all graphs and wanted to know the size of each slice in the pie chart, preferably in percentage.

Two of the total 34 participants were colour blind, but they did not find it hard to distinguish the colours. Most people liked the colour scheme in the dashboard, though some questioned the turquoise in the area chart and the colours on the lines in the line chart.

When questioning if the participants wanted each bar in the bar chart to be stacked instead of just the one for the current day, 78% said yes.

Some got confused that the line graph showed data for the days after the current day, it made it hard to understand which day that was the current day. One participant gave the suggestion to always have the current day as the last item on the x-axis.

After the questionnaire and meeting with the tutor the prototype was updated to prototype 1.1. The colour in the area graph was changed from a gradient to a solid green. The number of online users was added to the real time marker to make it more apparent that it showed the same value as the pie chart. A legend describing what the green area and black line represented was added. The heading was changed since some of the participants found it unclear. In the bar chart, every bar was changed to stacked. The bar representing the current day was given a darker colour than the other bars with the values displayed on the bar. To make it possible for the user to know the values for the other bars, a tooltip was added when hovering. The item for the current bar in the x-axis was rendered in bold to make it stand out.

In the line graph, the lines were changed to cyan, magenta and yellow after advise from the tutor. Just as with the bar chart the item for the current day was changed to bold. The grid line for the current day was also made more apparent. Finally, values and labels were added to the y-axis for clarification.

5.2 Second evaluation 5. Results

5.2

Second evaluation

Again, the participants found the area graph hard to understand because it both showed activity for today and statistical data. They did not notice the legend and thought the title was misleading. Another finding was that the mean value may be misleading when the value is calculated with days the schools have holiday.

Like the respondents in the questionnaire, the pie graph showing number of online users was perceived as easy to understand and very often noted as the most important graph. The bar graph showing number of online users the last week was found confusing for many respondents. Some respondents did not understand where the bar was that represented today. They meant that it was hard to understand that the graph visualized both the last week and the current week. They only wanted the days of the current week to be shown to understand better and faster. The same problem occurred in the line graph showing the books. From the questionnaire, the prototype had also been updated with a hovering function to the bar chart, but very few noticed that functionality.

In the line graph, the y-axis appeared as vague even though it was enough for most respondents only to see that the books differed from each other and not exact numbers. However it was recommended to use a categorical data that is more clear what it means and which is not free of interpretation like ”popularity” is. One respondent also noted that the graph may look strange when it is a large difference between the books.

Depending on the area of responsibility the respondent had at Sanoma, the answers were different on what level of importance the graphs had. The ones working at support implied that the area graph showing the login activity was the most important one. In contrast, the ones working in market and publishing found the graph unnecessary. They thought the line graph with the books and the pie graph with the number of online users were the most important ones.

The settings in the menu was thought as general settings changing all graphs, and therefore it was suggested that the settings were placed next to the con-cerned graphs. Lastly, the numbers shown was told to be hard to know if it represented a large or a small value. The respondent wanted some kind of per-spective. Many of the respondents wanted to be able to compare the current data with other days or weeks to see patterns.

After discussion with the client, it was decided that the pie chart showing number of online users was the most important visualization. Therefore it was placed in the top left corner because that is the place a user usually first look at. The large and bold text work as a focal point forcing the eye to go to that

5.3 Third evaluation 5. Results graph first as told in the theory about visual hierarchy in section 2.3.

Most respondents associated all bars in the bar graph with each other. That was probably due to the gestalt principle similarity listed in section 2.3 which may have made it harder to understand that they belonged to different weeks. By only showing data from the current week the risk of misunderstanding is reduced. This solution was implemented in prototype 2.0.

The settings to change the data shown was moved from the settings sidebar to the concerned graphs. In that way the setting was grouped together with the graph. And by having them close to each other, the user can more easily see the feedback. This is because the eyes do not need to shift back and forth as told in section 2.1.2 where it is explained that the short term memory aggravate the understanding of what elements are connected. All modifications made to prototype 2.0 can be seen in appendix C.

5.3

Third evaluation

The area graph was the hardest to understand. But that was partly because the data was misleading due to problems with the logout functionality and that the deployment hosting platform turned inactive if the application was not used. Other than that, the amount of time it took for the respondents to answer the questions was quicker this time and the amount of problems that occurred was far less.

One respondent with colour blindness thought the lines in the line chart was a bit hard to distinguish. It was proposed that the lines should be thicker for clarification. Generally the colours were told suiting, but one respondent thought a gray background may give a sad impression.

Another respondent thought the titles was to long to read and proposed they only should contain one word to make it as easy as possible for the users. When asked for more features to the dashboard, the respondents wished for visualization of more books, the ability to filter out data, number of licenses bought and the time each user is logged in.

In the UX prototype, the lines in the line chart was made thicker. But due to the limitations in the visualization framework, it was not implemented in the real prototype. The real prototype was updated with a few design modifica-tions since the respondents had noticed issues during the evaluation.

5.4 Final prototype 5. Results

Figure 15: The final version of the dashboard. It contains four different charts: one pie chart, a combination of an area and line chart, a bar chart and a line chart. The pie chart shows the distribution of online teachers and students, above the chart the total number of online users is visible in big letters. The combined area and line chart shows the login activity during the day. The line shows the real time activity for the current day and the green area chart shows the average activity on this week day four weeks back. The stacked bar chart shows the total number of logged in users during the past week. Each bar shows the division between teacher and student similar to the pie chart. The line chart shows the usage for the three most used books in SUD. Each grid line in the graph represents a day. The usage is measured in percentage.

5.4

Final prototype

The final implementation of the dashboard has four different charts, three of them visualize login activity on some level and the forth shows the usage of the three most popular books, see figure 15. An information-icon is located in the upper left corner, when clicking on it a panel slides out and shows information about the dashboard.

A pie chart was used as graph to represent online users, see figure 16. The whole graph represent the total number of online users and the slices are di-vided by teachers and students. Next to each slice is a popover displaying the percentage value. Instead of using regular boxes as legends two foxes are used. The graph showing the daily login activity is a combination of two charts, see figure 17. The real time data is a line that updates every minute showing the number of online users during the day. The statistical data is showing the average activity based on the current weekday four weeks back. The statistical data is represented by an area graph that is regenerated every day. The real

5.4 Final prototype 5. Results

Figure 16: The pie chart showing the distribution of teachers and students online right now. A label next to each slice shows the percentage amount. Above the chart the total number of online users is shown in numbers. Students is represented by orange and teachers by blue. Instead of using a regular legend with boxes, a orange and blue fox were used.

Figure 17: The combined area and line chart. The line is showing the login activity for the current day and is updated in real time. A vertical line with the current time and online users is showed next to the real time line. The green area segment represent the average login activity based on the current week day four weeks back.

time line has a vertical line with the current hour at the top and number of online users in brackets underneath.

A stacked bar chart was chosen as chart when showing the weekly login activity, see figure 18. When hovering above the segments a pop over shows the exact value of that segment. The colours of the teacher and student segments are the same as in the pie chart and the legend has two foxes replacing the regular boxes. The value on the x-axis represent weekdays and the value on the y-axis represent the total number of logged in users. The y-axis re-scales depending on the max value of the data set.

A line chart was used to represent top three most used books, see figure 19. The chart displays at most three lines were each line represent a book. When

5.4 Final prototype 5. Results

Figure 18: A stacked bar chart is representing the login activity during the week. Each bar represents a day, were the blue part is number of teachers logged in and the orange part is number of students logged in. Replacing an ordinary legend with boxes, two foxes were used. One blue for teachers and one orange for students.

hovering over a grid line a popover displays the values of the lines on that particular grid line. As with the bar chart, the x-axis represent the days of the week and the y-axis represent the percentage usage. The range on the y-axis goes from zero to the max value of the data set, which naturally cannot be higher than 100%. The y-axis re-scales depending on the max value of the data set.

The colours in the dashboard are rather soft. The background colour is a light gray and each of the charts are surrounded by a white box. Apart from the line chart in the daily login activity and the highlighted grid line in the book chart being black, no harsh colours were used. See figure 20 for all the colours used.

5.4 Final prototype 5. Results

Figure 19: A line chart with tree lines represent the top three most used books during the week. The usage is measured in percentage. The grid line for the current day is black to highlight which day it is.

Figure 20: This is the colours used on the dashboard. The orange colour represent students and the blue represent teachers. The green is used in the area chart representing average login activity during the day. The cyan, magenta and yellow is the colours used in the line graph. The light gray one is used as background colour. Other than these colours black and white was also used.

6. Discussion

6

Discussion

This section discuss the visualization techniques and frameworks used in the project together with alternative suggestions.

An important aspect when choosing visualization techniques was that the users were not familiar with visualizations. The visualizations needed to be simple to understand and this was considered in the process.

6.1

Online users - pie chart

The pie chart showing the number of online users was the most popular graph in all of the evaluations. As mentioned in section 2.2, the pie chart has received a lot of hate, probably because of it being miss used. However, in this case a pie chart is the perfect choice. Since it only compare two properties and the chance of both slices being equal size is small, it will not break any of the two main points in subsection 2.2.1. An alternative to using a pie chart would be to use a bar chart, either two separate bars or a stacked bar.

6.2

Daily login activity - area chart combined with line

chart

In the chart showing the daily login activity, the goal was to make it easy for the user to follow the login activity during the course of the day and at the same time being able to compare it with how it usually is. This will make it possible for the user to identify patterns and if something is out of the norm. This was the chart most participants in the evaluations thought was difficult to understand. Probably because it is a mix of two visualization techniques, a line- and an area chart. Instead of the real time data being represented as a line, it could also have been represented as an area chart. Most users have seen an area chart before and know how to read it. But as mentioned in 2.2.4, a problem occurring with area charts having several segments is that the one upfront might cover the ones in the back. By making the segments transparent the problem with the foremost layer covering the others can be reduced. See figure 9 for an example. Even if it is possible to see the green and yellow layer through the blue it is still not a perfect solution. By having the same visualization technique on both the real time data and the statistical data, it may have been harder to separate the data from each other. Therefore it was chosen to have two different visualization techniques.

6.3 Weekly login activity - stacked bar chart 6. Discussion

6.3

Weekly login activity - stacked bar chart

Instead of using a stacked bar chart showing the weekly login activity, one possibility would be to have two bars for each day, one representing teachers and one representing students. This would have doubled the number of bars from seven to fourteen, and according to section 2.2.2 it is not recommended to use more than eight bars. Also, the user would loose the ability to see the total number of users that day without adding the teacher and students bar. Another possibility is to display the the data in a spiral graph, see section 2.1.3. Since the data is cyclic a spiral graph could have been used. A spiral graph can be used to show periodical behaviors that are hard to see in a regular bar chart. However, a spiral graph need to be parameterized correctly to show patterns and is not something a common user have experience of. It might take to long for the user to interpret what the graph is displaying. As mentioned above, one important aspect is that the dashboard should be simple to understand, especially because the dashboard’s purpose is to get a quick overview of what is happening in the application. Therefore a spiral graph was not considered. For a more analytic application used by a few amount of people responsible to analyze the activity more deeply, a spiral graph may be a good choice.

6.4

Top three most used books - line chart

In the line chart visualizing the top three most used books, each of the books is represented by a line. A problem with this chart was to decide what quantity to have on the y-axis. The usage is measured by how many times users have interacted with the book. In the first prototypes no numbers were visible on the y-axis, this was changed to actual usage numbers after feedback from the first questionnaire. The first evaluation showed that this was not an optimal solution, what does it mean when a book has 168 usages on a Monday? Per-cental usage is easier to understand, hence the value on the y-axis was changed to this.

The choice of using a line chart was simple since the charts purpose is to compare three different data sets during a time span. A stacked bar chart could have been used as with the Weekly login activity-chart. However in the case with the weekly login activity, the user is interested in the total number of activity per day and how many of those that are teachers and students. This is not the case with the book chart. In this case the user would like to compare the usage of three books, they do not want to know the total percentage usage of these three books every day.

6.5 Colours 6. Discussion

6.5

Colours

Both the online now-chart and weekly activity-chart are displaying student and teacher data, therefore they both have the same colour scheme. SUD is internally called FOX among Sanoma and developers at Frost, hence there are foxes on the dashboard.

The colours for the daily activity and book chart were chosen so they would not be confused with the teacher and student colours. In the daily activity-chart the average has a light green colour and the real time line is black. The book-chart have soft versions of cyan, magenta and yellow.

One of the participants in the third evaluation thought that the gray colour could give a sad impression. Some other alternatives were tested, but none was as good as the gray one. The gray colour works very good with the other colours in the dashboard, when changed the other colours were perceived differently. If the background colour were to be changed, the other colours should be looked over as well.

In all of the evaluations a total of four participants were colour blind. One had some problem separating the blue and purple lines in the book chart, this could be fixed by making the lines thicker. Other than that none of the participants gave any negative feedback on the colours.

6.6

The charts overall

In the earlier prototypes the bar- and line chart showed data for seven days, e.g if the current day was Wednesday the charts would show data for the past Thursday - Sunday after Wednesday. The questionnaire and first evaluation showed that many was confused by this. To solve this there were two alterna-tives considered; 1. only show data from Monday to current day or 2. always show the current day outermost on the x-axis.

Choosing the first alternative means that the user cannot compare the current day with past days if it is Monday, however the previous problem with users confused by data after the current day wont be a problem any more. Choosing the second alternative will make sure that the user always can compare the current day with past days. A problem with this alternative is that is does not show weekdays in a regular order, this could cause the users to get confused. If a user want to see the data for Thursday in a bar chart he or she would probably look at the forth bar, but if the current day always is outermost Thursday may not be the forth bar. This could result in the user reading data from the wrong bar or having to read all ticks on the x-axis to find the sought bar. In the final prototype the first choice was implemented.

6.7 Development process 6. Discussion

6.7

Development process

In the early stages of the project Herkou was chosen as deployment platform. It was simple to use and have a free version available. The problem with using the free version of Heroku is that it falls asleep after 30 minutes of inactivity, that is if no web hook receives any call within 30 minutes. The app gets activated again when Papertrail send new data to the app. However, when being activated again it misses the call from Papertrail that awoke the application and no data is saved in the database. Suppose that every call from Papertrail would be 35 minutes apart, then the app would fall asleep between every call and hence only get woken up by the next call. If this was the case, no information would be saved in the database and the dashboard would show no data, which is far from desirable.

The problems with Heroku can be solved, but in the end Sanoma will probably use another platform when deploying the dashboard. Therefore the time spent trying to fix Heroku-problems might have been unnecessary.

6.8

Frameworks

Two visualization frameworks were used for the project, D3 and ngx-charts. At first, problems occurred when implementing D3 in TypeScript and alternatives were investigated. Ngx-charts was found easy to implement and time-saving. But as more features and design choices were to be implemented, ngx-charts was restricted and hard to manipulate beyond the given features. In the end, D3 was used for the most complicated chart and the ngx framework was used for all others.

Using the MEAN-stack as development toolkit turned out to be a good choice. Since it is well known and used by many, a lot of information and tutorials are available online. Angular 2 uses TypeScript instead of JavaScript, this caused some problems with libraries not yet supported by TypeScript. This was not a major problem since there were plenty of alternative libraries.

6.9

User testing

For the evaluations, it was chosen to use a between subjects design, see section 2.3.5. The advantages of testing the first impression was considered more important than testing if the same participants recognized the changes made to the dashboard in the second test. The dashboard’s purpose was partly to attract new customers, requiring the dashboard to be easily understood. An advantage to use a within subjects design is that the conditions are the

6.9 User testing 6. Discussion same among the participants. But the groups in the tests were deliberately chosen from different departments to have a bigger range of experience of visualizations. In that way, the dashboard could be tested if it satisfied both experienced and inexperienced users. The range of experience gave a wider amount of feedback but may have had a negative impact on the focus of making the dashboard easily understood. An experienced user may notice and focus on different things than an inexperienced user.

The evaluations included questions the participant answered, making the par-ticipant focus on the tasks rather than exploring the dashboard. Because of this, it was never investigated how the experience was for a user seeing the screen for the first time in the office. Hence it is not known if the dashboard is perceived interesting and easy enough for a user to stay and look at for a while. The reason for using questions in this case was to have more control and test the user scenarios.