Linköpings universitet SE–581 83 Linköping
Linköping University | Department of Computer and Information Science
Bachelor’s thesis, 18 ECTS | Cognitive Science
2020 | LIU-IDA/KOGVET-G--20/012--SE
The user’s experience of stress
and workload in videos with stalls
Användarens upplevelse av stress och arbetsbelastning i videos
med stalls
Wilhelm Brodin
Supervisor : Erkin Asutay Examiner : Kenny Skagerlund
Upphovsrätt
Detta dokument hålls tillgängligt på Internet - eller dess framtida ersättare - under 25 år från publicer-ingsdatum under förutsättning att inga extraordinära omständigheter uppstår.
Tillgång till dokumentet innebär tillstånd för var och en att läsa, ladda ner, skriva ut enstaka ko-pior för enskilt bruk och att använda det oförändrat för ickekommersiell forskning och för undervis-ning. Överföring av upphovsrätten vid en senare tidpunkt kan inte upphäva detta tillstånd. All annan användning av dokumentet kräver upphovsmannens medgivande. För att garantera äktheten, säker-heten och tillgängligsäker-heten finns lösningar av teknisk och administrativ art.
Upphovsmannens ideella rätt innefattar rätt att bli nämnd som upphovsman i den omfattning som god sed kräver vid användning av dokumentet på ovan beskrivna sätt samt skydd mot att dokumentet ändras eller presenteras i sådan form eller i sådant sammanhang som är kränkande för upphovsman-nens litterära eller konstnärliga anseende eller egenart.
För ytterligare information om Linköping University Electronic Press se förlagets hemsida http://www.ep.liu.se/.
Copyright
The publishers will keep this document online on the Internet - or its possible replacement - for a period of 25 years starting from the date of publication barring exceptional circumstances.
The online availability of the document implies permanent permission for anyone to read, to down-load, or to print out single copies for his/hers own use and to use it unchanged for non-commercial research and educational purpose. Subsequent transfers of copyright cannot revoke this permission. All other uses of the document are conditional upon the consent of the copyright owner. The publisher has taken technical and administrative measures to assure authenticity, security and accessibility.
According to intellectual property law the author has the right to be mentioned when his/her work is accessed as described above and to be protected against infringement.
For additional information about the Linköping University Electronic Press and its procedures for publication and for assurance of document integrity, please refer to its www home page: http://www.ep.liu.se/.
Abstract
The aim of this thesis was to investigate the user experience when watching different formats of video, specifically at the occurrence of a stall from the perspective of workload and stress. The three research questions narrowed the scope down to a comparison be-tween linear and branched video and the presence, or absence, of a playback bar in the user interface (UI). The measurements used were Nasa-TLX, Short stress state questionnaire as well as thematic analysis of participants free text answers. No significant differences in workload or stress was found between branched and linear video, as well as between the presence and absence of a playback bar. However, due to high number of non-completion participants the results were of questionable validity. The thematic analysis of the free text answers indicated a contrary story to the results from the statistical analysis. Further in-vestigation of the effects of workload and stress on stalls in video experience is therefore recommended. Finally, the thesis provide guidelines based on the lessons learned so to best address the remaining research questions.
Acknowledgments
Tack till Erkin Asutay, Niklas Carlsson och Kenny Skagerlund för all hjälp. Utan er hade inte denna uppsats varit möjlig.
Contents
Abstract iii
Acknowledgments iv
Contents v
List of Figures vii
List of Tables viii
1 Introduction 1 1.1 Motivation . . . 1 1.2 Aim . . . 2 1.3 Research questions . . . 2 1.4 Delimitations . . . 2 2 Background 3 2.1 Linear video . . . 3 2.2 Branched video . . . 3 2.3 Stalls in video . . . 3 2.4 Stress . . . 4 2.5 Workload . . . 6 3 Method 9 3.1 Study design . . . 9 3.2 Procedure . . . 10 3.3 Participants . . . 11 3.4 Non-completion participants . . . 13 3.5 Materials . . . 13 3.6 Pilot study . . . 15 3.7 Qualitative analysis . . . 15 4 Results 17 4.1 NASA-TLX video 1 analysis . . . 17
4.2 NASA-TLX video 2-3 analysis . . . 19
4.3 Short Stress State questionnaire analysis . . . 19
4.4 Thematic analysis of free text answers - Video 2-4 . . . 20
5 Discussion 23 5.1 Results discussion . . . 23
5.2 Method discussion . . . 26
Bibliography 29
A Appendix 32
A.1 Informed consent form . . . 32
A.2 Short Stress State Questionnaire . . . 33
A.3 Nasa Task Load Index . . . 36
A.4 Demographics questionnaire . . . 38
A.5 Free text questionnaire . . . 41
A.6 Branched video exposure questionnaire . . . 42
List of Figures
3.1 Distribution of participants average weekly time spent watching videos . . . 12 3.2 Distribution of participants average weekly time spent playing video games . . . . 12 3.3 Video player implementation, showing user interface at a branch point. . . 15 4.1 Nasa-TLX means and standard error of measurments from video 1. Excluding
List of Tables
3.1 Summary of the steps included in each part of the experiment. . . 11 3.2 The distribution of the participants education subjects by percentage . . . 12 3.3 The distribution of the participants previous exposure to branched video . . . 12 4.1 Total number of participants and number of participants excluding
non-completion participants in the analysis of the Nasa-TLX result from video 1. . . 18 4.2 Number of participants included in the analyses of the Nasa-TLX results from
video 2-3. . . 19 4.3 Total number of participants and number of participants excluding
1
Introduction
1.1
Motivation
The availability of smartphones and the Internets global connectivity has made watching videos a daily habit for many people around the world. This is made possible in part by the multiple platforms for sharing videos online. The platforms have removed previous bound-aries for creating sharing video content with almost all people connected to the internet, mak-ing it easier than ever before. The format of the majority of these videos are of a linear type. Insofar as that they have a beginning, a middle and an end and every time you start the video to watch it, the story of the video will take place the exact same way. The linear video format is however not the only format for video. Netflix has produced and published multi-ple movies with a non-linear format. One of them is the movie Black Mirror: Bandersnatch (Netflix, 2018). The difference between a linear video and a non-linear video, such as Black Mirror: Bandersnatch, is that the user has the option to influence the story of the video and make choices that effects the storyline of the video. Making it possible for each watching of the video to be distinctly different than the last one. An infinite number of new story lines are of course not possible. It is determined by the number of points in the video where the story branches out in different ways and how many different choices are available at each of these branch points.
The difference between these two video format, one linear and the other non-linear, begs the question of the users’ experience of them. More specifically if the option of interaction in branched video creates a higher workload or stress in the user compared to a linear video. Workload (Hart and Staveland, 1988) and stress (Lazarus and Folkman, 1984) becomes rele-vant in this context as they are both concepts that are dependent on an individuals capacity in a specific task. Since workload and stress becomes more salient in an individual as the requirements of a task comes closer to or over shoots the capabilities of the individual and simply watching a video could be considered as a low effort task. A difference between branched and linear video might be amplified by introducing a stall in the video viewing ex-perience. A stall is when the video is unexpectedly halted. A prolonged stall event increases the chance that a user no longer finds the quality of the video acceptable (De Pessemier et al., 2013). An interesting question to investigate then, is what the user experiences before they no longer find the video acceptable. This might be a question where linear video and branched video differs from each other.
1.2. Aim
Previous research on the users’ experience of branched video (Lindskog et al., 2019) inves-tigated different versions of playbackbars as well as the presence and absence of the playback-bar. A playbackbar, as the concept was used by Lindskog et al., is a user interface component located at the bottom of the video player and can include a visual cue for the progress of the video. Exposing the user to conditions both with and without a playbackbar could provide a tool to affect the task difficulty of watching a video with stalls.
Researching these queries could provide insights valuable to aiding the development the user experience of non-linear videos as this video format continues to grow and mature. A difference in stress and workload between branched and linear videos at stall events could give suggestions to whether these two video formats differ enough in the users’ experience of them that the user interfaces should be considered to have different design requirements. Or if the experiences are similar enough that no differentiation from a user interface design perspective is necessary to consider.
These points of query, difference in users experience, workload and stress between video formats, the presence and absence of a playbackbar and users experience in adjacency to stall events, form the base for the aim and focus for this thesis.
1.2
Aim
The aim of this thesis is to investigate what the users experiences when a stall occurs while they are watching a video and if that experience differs between the linear and non-linear video formats. This will be done by taking a perspective of cognitive science but leaning on engineering psychological aspects rather than usability. Meaning that the focus will lie on aspects such as stress and mental workload to capture the users experience. But also taking into account what the user express their experience to be. This will be investigated using the short stress state questionnaire (SSSQ) and NASA task load index (Nasa-TLX) to measure both the users state of stress as well as the workload connected to the task. Coupled with a questionnaire with free text answers, aimed to collect the users expectations and thought process within the problem solving of the tasks of finding the reason for the stall.
1.3
Research questions
The aim of the study can be reduced to the following research questions:
• Is there a difference in user reported stress and workload between linear and branched formats when a video stall occurs?
• Does the presence of a playback bar affect the user reported stress and workload of linear and branched video formats when a video stall occurs?
• Which themes are present in the users expressions regarding intentions, actions and expectations of the experience in the task of the experiment?
1.4
Delimitations
There are multiple formats of video within the linear video paradigm, as well as the branched video paradigm. The most prominent aspect being interactability. This thesis delimits linear videos to non-interactive videos. Non-interactive in the sense that user should not be able to effect the flow of the video by stopping and starting when the video is initially started. Same constraint is intended on the branched video format. However, branched videos must be interactive in the sense that a user is able to chose between different branches at the videos branch points. The instruments used to measure the user experience are delimited to stress and workload and therefore does not include other usability questionnaires. Instead it is the intention to collect this part of the user experience with the free text questionnaire.
2
Background
2.1
Linear video
The factor that delimits a video to linear in this thesis is its temporal format. A video which the content of that video is always in the same order, no matter the interactions with the user. It is the most common type of video available, with multimedia platforms such as youtube (Youtube, 2005) and vimeo (Vimeo, 2004) providing a place for users to upload and share videos online. A linear video can be interactive in multiple ways. Providing the user with functionality to for example pause, change playback speed of the video, or scroll thru the video with the aid of a playback bar. It can also be non-interactive, providing no way of affecting it. Going to a cinema is one example of a non-interactive linear video. The type of linear video used in this thesis is only of the non interactive type. In the sense that the user will not be able to stop and start, or scroll in the video once it is started.
2.2
Branched video
A branched video is a type of non-linear interactive video (Meixner and Kosch, 2012) where the user has the option to affect the story in the video. The branched variant refers to how users encounters points in the video were a choice is presented to them. The different op-tions at that point are branches in the story of the video (Krishnamoorthi et al., 2014). An implementation of this was done (Lindskog et al., 2019) by showing the user two squares with labels, describing the different choices. Each square embodying a different branch of the videos possible story lines. The user then picks one of the branches by clicking one of them. When the video comes to a branch point the video stops and waits for input from the user. The implementation of the playback bar took a somewhat different shape compared to a linear video. The representation of the multiple branches were visualised by branching the progress bar at the branch points. Thus conveying branched format of the video to the user.
2.3
Stalls in video
From a user perspective a stall in a video is the interruption of video content, also called buffering when the reason for the stall is insufficient bandwidth for data streaming (De Pessemier et al., 2013). There are two variants of stalls, planned and unplanned. A planned
2.4. Stress
stall in a branched video only includes when the video is temporarily stopped at a branch point while waiting for the user to input selection of branched to proceed with. Unplanned stall includes all other types of interruption of playback of the video. The length of the stall has been found to affect how willing the user is to accept different levels of video quality. One user study (De Pessemier et al., 2013) showed that there is a significant drop of in how proba-ble it is for a user to find the quality of a video acceptaproba-ble after a stall longer than 20 seconds. In a different user study (Mok, Chan, and Chang, 2011) it was found that the frequency of buffering (stalls) is a main factor for the quality of experience for the user.
2.4
Stress
There are multiple ways to measure the phenomena of stress. The two major categories of measurements are physiological and subjective. To measure stress physiologically it is com-mon to measure skin conductance as an index of sympathetic autonomic nervous system activity (Braithwaite et al., 2013), (Khalfa et al., 2002) and heart rate variation via heart-rate monitor, for example as done in the study by Shapiro et al. (2017).
Subjective measurements of stress entails the participants to express their experience of stress, often thru the means of a questionnaire. The answers can then be used to derive the stress state of the participant in the particular task being performed during the experiment (Helton, 2004). The division of stress into different states of stress is a part of the transactional theory of stress.
The transactional theory of stress and performance
The contemporary theory for subjectively measured stress places it in relation to performance in a task. Stress is then a result from how the individual believes the requirements of the task or the environment is greater than or exceeds the individuals capabilities (Lazarus and Folkman, 1984). The transactional part of the theory is how the different appraisals of the individual can be measured to be separate states of stress. These states then affect cognition in different ways (Hockey, 1997). Examples of how these different states affects cognition can be found in the research developing and using the Dundee stress state questionnaire (DSSQ) and the Short stress state questionnaire.
Dundee stress state questionnaire
DSSQ assumes that the experience of stress is dependent on a multitude of aspects such as for example emotion, work load, cognitive resources (G. Matthews et al., 1999). Therefore mea-suring stress should be done in multiple dimensions as well. This is done in the DSSQ with three different stress states: task engagement, worry and distress. To detect a participants stress state related to a specific task a pre/post-task questionnaire design is used. The partic-ipant answers the questionnaire before the task is performed. Then, immediately following the completion of the task the participant fills out the questionnaire again. The subjective stress state is then derived from the difference of the results from the pre and post question-naires (G. Matthews et al., 1999).
In previous research including the DSSQ, the questionnaire has been used to measure the amount of stress a particular task induces (G. Matthews et al., 2013). The tasks measured has mostly been related to humans operating or performing in a physical space. From operating vehicles (Funke et al., 2007), to air defence (Panganiban, Matthews, and Funke, 2011) and surgical procedures (Klein et al., 2012). It has also been used to evaluate tasks on computer displays (J. L. Szalma, 2011),(J. Szalma and Teo, 2012). In each of these instances the tasks measured took 12-24 minutes for the participants to complete. This is regarded as a short time for a task to be measured by the DSSQ (G. Matthews et al., 2013). The Dundee stress state questionnaire is a large and in some cases cumbersome tool to measures stress. This is
2.4. Stress
mainly due to it consisting of 90 individual questions for the participant to answer both in the pre-task measurement and the post-task measurement. This adds up to 180 questions having to be answered to get a full measurement.
Short stress state questionnaire
There is a shorter alternative to the DSSQ available, developed by William S. Helton (2004). This questionnaire is called the short stress state questionnaire and aims to essentially do the same work as the DSSQ with fewer questions and therefore in less time. The questionnaire is, much like DSSQ, a two part questionnaire, where a pre-task questionnaire of 24-items is presented to the subject. Measuring three aspects of stress: task engagement, distress and worry (Helton, 2004).
SSSQ - Task engagement
The stress state measured as task engagement can be recognized as affecting the following aspects of cognition. Energy level before task predicts task engagement to some degree. It also seems to predict task performance in attention tasks. Task engagement has also been measured to show how external sources of stress affects the participants, such as loud noises or sickness (G. Matthews et al., 2013). It also seemed to predict how willing a participant is to use automation in real world cases. Lower level of task engagement meant more likely to engage automation. A low level of task engagement for a task can be interpreted as a reason to overview the tasks monotony and also increase the challenge in the task. Both of these changes would make the task better suited for the users (J. Szalma, 2009).
SSSQ - Distress
The aspect of distress is closely related to worry as they are both measured to be included under the mental state of anxiety (G. Matthews and Campbell, 2010). However, dividing up the state in two gives the opportunity to distinguish between the stress originating from the moment, measured as distress, and the stress coming from the participants anticipation of the coming situation. Distress is correlated to lower scores in working memory tasks. It was also found to coincide with inhibition. More precisely, more distress lead to less ability to inhibit as an executive function. A task that induces the users to report high distress is a sign that the task needs to be modified to reduce the workload if it is to better suit the users (J. Szalma, 2009).
SSSQ - Worry
The stress state worry is closely related to the stress state distress, but worry is better de-scribed as the participants anticipation of stress or difficulties the coming situation. A partici-pant who measures higher in worry tends to perform worse on verbal tasks. This might be be-cause a participants worry is taking up some of the verbal short term memory (G. Matthews et al., 2013). Worry may influence a participants ability to switch between tasks. Worry seems to induce a participant to dislocate from the task at hand, which in turn reallocates resources from processes involved in solving the task to non-task related processes (G. Matthews et al., 2013). Tasks that prove to induce high scores in worry from the users should be redesigned to help the user focus and may also be changed to provide reassurance to the user, to better fit their needs (G. Matthews et al., 2013).
Implementation of the Short stress state questionnaire
The questions in the questionnaire are formulated to ask the subject to rate how similar their state of mind were in regards to the item. One item could for example be, "I feel dissatisfied".
2.5. Workload
See Appendix A.2 for full list of items. The subject answers the 24-items in a intuitive manner using a 5-point scale ranging from 1. "Not at all" to 5 "Extremely". After this the subject completes a task that the experiment is designed to investigate the different states of stress. Directly after the task is completed, the subject completes the post-task questionnaire. This is the same questionnaire again. The stress states of the individual due to the task is then produced by comparing the pre- and post-task questionnaires. If the change from pre-task to post-task is positive, meaning going from for example "Not at all" to "Somewhat", it indicates that the task has induced the stress state connected to that item. The items connected to the stress state worry should have a negative change to indicate that the stress state is induced. This is because the items measuring this state are constructed to either ask if the subject is thinking about others or something different than the task. The logic being that if the subject is thinking about something different than the task, the subject can not be worrying about it. For example the item "I am worried what other people think of me". If the participant answers high on the scale in the pre-task questionnaire and lower in the scale in the post-task questionnaire. This would be indicating that the participant first thought about other aspects than the task at hand before the task compared to after it. Which in turn would mean that the subject was not worrying about the task at hand before it took place since the subject was thinking of other things than the coming task. Another example of an item for the stress state worry is, "I’m daydreaming about myself". The same logic applies to it. If the subject was daydreaming more before the task than after it, the subject could not have been worrying about the task at hand.
The SSSQ has been used to measure different kinds of tasks often alongside with other measurements. It was used in conjunction with the Nasa-TLX questionnaire to asses the task of monitoring for hazards while driving an automated vehicle (Greenlee, DeLucia, and New-ton, 2018). SSSQ was used again with NASA-TLX to measure how different design of query interfaces affects stress, mental workload and performance (Edwards, Kelly, and Azzopardi, 2015). Shapiro et al. (2017) used it together with physiological measurements such as saliva cortisol, heart rate and variability, facial muscle movements and skin conductance levels to measure a simulated income inequality in a laboratory setting. The last example shows how different the tasks can be when using the SSSQ to measure stress state is the study by Lim and Kwok (2016). In it they investigate the effets of varying break lenght on attention and time on task .
Stress in videos and video stalls
The task of watching a video would on its own be unlikely to induce any of the three stress states, unless the content of the video creates an emotional reaction. Such as a scary movie or a boring movie. A users stress states might be influenced at a stall event, in particular the stress state distress. Especially if the user also experiences the workload to be higher since, as mentioned before, distress is an indicator that the workload of the task is too high (G. Matthews et al., 2013). This is most relevant in the case of how the presence of a playback bar influences the user experience. It might be the case that the playback bar reduces the workload for the user by making the state of the video player more salient and therefore lowering the users distress. There might be a difference between linear and branched videos in this regards as well. Since the case can be made that an interactive branched video puts more demand on the user compared to an non-interactive linear video.
2.5
Workload
Workload is the experience of effort that an individual has when performing a task. Workload is often more noticeable when the load is too small or too big in relation to the individuals capacity for the task. This results in problems with boredom and reduced vigilance when the task load is small and stress when the workload is to large (Hart, 2006). The distinction
2.5. Workload
between workload and stress made by Gaillard (1993 is that high workload does not neces-sarily lead to worse results, while stress is the end result of an insufficient match between the individuals workload capacities and the tasks workload demand. Which in turn can be experienced as negative emotions. High workload is not necessarily experienced in the same way. A subjective measure of workload can be done with the Nasa task load index.
Nasa-Task Load Index
The Nasa Task Load Index (Nasa-TLX) questionnaire was developed by Hart and Staveland (1988). It is a common instrument to measure workload (Hart, 2006). It is based on 6 ques-tions, ranking the subjects own experience of different aspects of the workload. This ranking is done by the subject in direct conjuncture to the task that is to be measured (Hart and Stave-land, 1988). Since workload is a complex phenomena, there are multiple dimensions of it. Nasa-TLX divides the concept of workload into six different dimensions. With one question in the questionnaire devoted to each one of them. The dimensions with descriptions based on (Hart and Staveland, 1988) are:
• Mental demand - The amount of mental activity that was required by the participant to complete the task.
• Physical demand - The amount of physical activity that was required by the participant to complete the task.
• Temporal demand - The amount of pressure experienced by the participant by the tempo required by the task.
• Performance - How successful the participant experienced themselves to be in complet-ing the task.
• Effort - How much effort overall was required by the participant to complete the task. • Frustration - How much frustration, in the form of insecurity, discouragement, stress
and annoyance was experienced by the participant.
The analysis of Nasa-TLX can either be done by looking at the dimensions separately or taking an average of them. This creates a measurement of the total average workload (Grier, 2015). This is called Raw-TLX as no weighting of the different dimensions are done, which would make some dimensions influence the result more than others. The original way of analysing the results from the Nasa-TLX was to pair the questionnaire with questions of which dimension had the most impact on the performance on the task. The score was then calculated by giving the more important task more weight in the calculation (Hart and Staveland, 1988). The removal of the weights on the calculation, making it Raw-TLX has been reported as having mixed results. In some cases it has been found to be more sensitive to the different kinds of workload in the task (Hendy, Hamilton, and Landry, 1993). In other cases it seems to be less sensitive (Liu and Wickens, 1994). While some have found it to be an equal measurement (Byers, Bittner, and Hill, 1989).
Workload in videos and video stalls
Workload as measured by Nasa-TLX could be quite insipid with regards to watching a non-interactive linear video. Since it is a passive task if nothing else is demanded of the user. In this case only the actual content of the video might create differences in workload as some story lines could be more difficult to follow than others. In the case of the interactive branched video format, it could be expected to have an higher demand on workload on the user com-pared to a linear video format. For example since the branched video interaction has time constraints to the choosing of a story line at the videos branch points. When considering a
2.5. Workload
stall event the case seems to be different. In one user study (Szameitat et al., 2009) it was found that small delays by about 1.6 seconds seemed to decrease the users performance and the users also reported the experience as less likable compared to the same task with no de-lays. The task in this study was more interactive than the task used in the current experiment. But performance decreasing could be an indication that the users workload increases, since one of the capacities that bounds the performance of a user is the users workload capacity (Hart, 2006). In a different user study frustration was measured with regards to latency in information availability (Nah, 2003). It found that users expect information to be available to them within 2 seconds of interaction. Longer waiting time than that increases frustration. This puts a demand on the user interface in the current study to make the information of what has gone wrong in with the playback of the video salient to the user, or an increased frustration score in the Nasa-TLX measurement should be expected. For example, this could be the case when a playback bar is not present for the user to see the current available buffer in the video player.
3
Method
3.1
Study design
The experiment in this thesis will use a mixed design. It consists of both between group and within subject design. The participants are first assigned to one of four conditions:
• Linear video, enabled playbackbar - (LP) • Linear video, disabled playbackbar - (LN) • Branched video, enabled playbackbar - (BP) • Branched video, disabled playbackbar - (BN)
The distinction between linear and branched video is already made in the theory section. The difference between enabled and disabled playback bar is simply that for the conditions that have the playback bar enabled, it will be visible and functional and to the conditions where it is disabled it will be not visible and therefore not functional. The condition with the branched video format and disabled playback bar will still be able to make choices thru clicking on the designated place on the screen. After the first two videos the participants then get assigned to a new condition, one of the three that they were not placed in before. Since the research questions of the study focused on the difference between subjects having a playback bar and not, as well as the difference between the video being linear or branched, we did not use all possible condition changes. The included condition combinations where:
• LP - BP • BP - LP • BP - BN • BN - BP • LP - LN • LN - LP
3.2. Procedure
Because of an unforeseen error in the experimental setup, one participant went through BP-LP conditions.
The decision was made to include this participant in the hypothesized comparisons be-cause the results showed no order effects in any condition.
3.2
Procedure
The procedure for the experiment consisted of 21 separate steps.
The 21 steps of the experiment procedure
1. Information about the experiment and informed consent procedure 2. Collecting of demographic data
3. Collecting pre-task SSSQ data
4. Presenting information1 and instructions for the task 5. Watching video 1
6. Collecting workload data with Nasa-TLX questionnaire 7. Presenting information2 and instructions for the task 8. Watching video 2
9. Collecting workload data with Nasa-TLX questionnaire 10. Collecting post-task SSSQ data
11. Collecting free text answers about task solving strategies 12. Presenting information2.1 and instructions for the task 13. Watching video 3
14. Collecting workload data with Nasa-TLX
15. Collecting free text answers about task solving strategies
16. Collecting questionnaire of previous knowledge of branched video 17. Presenting information3 and instructions for the task
18. Watching video 4
19. Collecting free text answers about task solving strategies 20. Informing the subject that the experiment was over. 21. Debriefing with the subject
3.3. Participants
The five parts of the procedure
The steps in the procedure of the experiment can be separated into 5 parts, with some parts overlapping others. The different parts of the experiment are summarized in Table 3.1.
1. The first part included the informed consent, information about the experiment and collection demographic data of the participants. This part was done in steps 1-2. 2. The second part, steps 4-6, was about Nasa-TLX.
3. The third part was intended to enable a stress state analysis, investigating if different conditions had different stress states after completing same the task with different in-terfaces. This part was done in steps 3-10.
4. The fourth part was intended to enable a within subject comparison of different condi-tions. As the participants changed conditions between step 6 and step 7. The steps for this part included steps 7-15.
5. The fifth and final part of the experiment was intended to give the participants an open view of what the experiment was intended to investigate. Steps 16-21 included asking about previous knowledge and experience of branched videos, explicitly informing the participants what would happen in the video. The experiment was then finished with a debriefing of the participant.
The steps in the five parts of the procedure
Parts Steps 1 - Info. 1-2 2 - Nasa-TLX 4-6 3 - SSSQ 3-10 4 - Nasa-TLX Change 7-15 5 - Explicit stall 16-21
Table 3.1: Summary of the steps included in each part of the experiment.
The decision was made to limit the SSSQ to the first two videos to avoid introducing biases from ordering effects. As well as to limit the length of the experiment.
A decision was also made to limit the use of Nasa-TLX in conjunction to videos 1-3 but not video 4. This was done as it was a priority to keep the length of the experiment within reason and having another measure of workload was perceived as having diminished returns since the task of video 4 was no longer an ecological task. It asked the participant to focus more on the problem solving strategy than the experience of watching a fourth video. More about the task can be found in Section 3.5 Materials.
3.3
Participants
A total of 43 individuals participated in the experiment. 21 female and 22 male. The age of the participants ranged from 19 years old to 34 years old with a median of 23 years old. The participants education level was 4% Upper secondary school, 57% Bachelor degree and 39% Master degree. The distribution of the participants subject of education is displayed in table 3.2.
The participants average weekly video watching time can be seen in figure 3.1 and the participants average weekly time spent playing video games can be seen in figure 3.2.
The participants previous experience with branched videos were also collected and is presented in table 3.3.
3.3. Participants
Education subject distribution Education subject Percent Information and
Commu-nication Technologies
39% Engineering, Manufac-turing and Construction
20%
Cognitive Science 20%
Natural Sciences, Mathe-matics and Statistics
7%
Education 4%
Social Science, Journalism and Information
4% Health and Welfare 2% Business, Administration and Law
2% Arts and Humanities 2%
Table 3.2: The distribution of the participants education subjects by percentage
Figure 3.1: Distribution of participants average weekly time spent watching videos
Figure 3.2: Distribution of participants average weekly time spent playing video games
The participants in the study where recruited using convenience sampling, which in prac-tice meant asking students at Linköping University to participate. Due to the covid-19 epi-demic during the experiment and the public health agency of Swedens recommendation to minimize close contact to others, the subjects where verbally made aware of the possible risk of participating. All partaking participants were constrained to partake in the experiment at location. All subjects provided informed consent before participating, see Appendix A.1 for the informed consent form that was used.
Participants previous exposure to branched video
Exposure Percent
Never seen one 46%
Seen 1-3 videos 46%
Seen 3-5 videos 4%
Seen 5+ videos 4%
3.4. Non-completion participants
3.4
Non-completion participants
The design of the study was intended to limit the amount of dependent variables to the con-dition assignments in the different parts of the experiment. Part of this was to try and keep the length of the videos and the length of the stalls the same for all participants. However, after the data collection was completed a problem regarding this emerged. It was discovered that some participants had found a way to start the video when a stall occurred. Making their total video length, as well as stall length, shorter than the other participants. The group of participants who found this bug, found it at different steps in the experiment. Therefore the exclusion of the participants had to be done on an analysis by analysis basis. Another non-completion issue that arose was that some participants had left some questionnaires un-finished. These where excluded on a analysis by analysis basis as well.
3.5
Materials
The materials used in the study are listed below with a short description of them. The mate-rial can be viewed in full in Appendix A.
• Forms
– Informed consent form, informing the participant of the contents of the experi-ment. See Appendix A.1.
• Questionnaires
– Demographic questionnaire, contains questions regarding the participants age, gender, education and habits of watching video and playing computer games. See Appendix A.4.
– Short stress state questionnaire, contains questions regarding the participants stress state. See Appendix A.2.
– NASA-TLX questionnaire, contains questions regarding the participants work-load. See Appendix A.3.
– Free text questionnaire, asks the participant to define what the problem in the video was, what they wanted to do to find out the reason for the problem, what they actually did to find out the reason for the problem and what, if anything, could have made the reason for the problem in the video easier to find. See Ap-pendix A.5.
– Branched video exposure questionnaire, asks the participants about their previ-ous experience with branched videos before this experiment and the name of a branched video if they have seen one before this experiment. See Appendix A.6. • Task information pages
See Appendix A.7.
– Task information 1, informs the participant that they are about to watch a video and that the task is to watch it and answer questions about the video later. It is deliberately vague toward the participant to try to capture an ecological reaction from the participant.
– Task information 2, informs the participant that they are about to watch a video again and that task in this video is to try to figure out the reason for the problem in the video. The thinking behind the minor change in this information is to make it more clear that it is a reaction to problem solving that is the focus but not giving away the focus of the experiment on video stalls.
3.5. Materials
– Task information 2.1, informs the participant that they are about to watch a video with a different video player and that task in this video is to try to figure out the reason for the problem in the video. This task is meant to be the same as the pre-vious task but with a new user interface or video format, to enable a comparison between them.
– Task information 3, informs the participant that they are about to watch a video with a stall in it and the task is to find out the reason for that stall. Making the purpose of the experiment more clear to the participant and giving the participant an opportunity to fulfill that purpose.
– Information 4, informs the participant that the test is over and that a debrief is coming next.
• Videos
The videos used in the study were cut from the open source video Big Buck Bunny (Goedefebure et al., 2008). A video showing a confrontation between Big Buck and three squirrels, which ends in favor of Big Buck. The intended audience for the video is a bit unclear as the visuals seem inspired by animated cartoons intended for children, but the story contains two scenes where the squirrels kill a butterfly. The video chosen for the study to make the results easier to compare to a study (Lindskog et al., 2019) in which the same video player was implemented.
The content presented to the branched and the linear conditions could differ. This was because the branched condition having options in the story and the linear condition being presented the same content as a participant in the branched condition who only chose the first option at the branch points. This was also the default path for the video. If a participant in the branched group did not make an selection at the branch point, the video player would automatically choose this path at the branch points.
The the videos differ in number of stalls, length of the stalls and video length.
– Video 1 - The length of the video was 2 min 40 seconds for both linear and branched. The branched video had two branch points, the first at 32 seconds in and the second at 1 min 36 seconds in. The two stalls were each 18 seconds long.
– Video 2-4 - The length of the video was 1 min 12 seconds for both linear and branched. The branch version had a branch point at 40 seconds into the video. The one stall was 24 seconds long.
Video player implementation
The video player implementation used in this study was based on the systems design in (Lindskog et al., 2019). The user interface had a playback bar with a progress bar in it that shows the coming up branches and the already chosen ones, but not the branches that were not chosen. When an branch point is near, the choices available to the user are made visible with two clickable boxes containing the name of their respective branch. See figure 3.3. The participants could interact with the video player in three ways. Starting the video by pressing the "P" key on the keyboard, and if the participant were in a branched video condition they could choose a branch at the branch point. The participants in a condition with a playback bar enabled could make it visible by moving the mouse pointer on the screen. The implemen-tation had a predetermined branch path that is defaulted to if the user fails to input a choice before the branch point and thus did not wait for the users input at the branch point.
Equipment and screen setup
The technical equipment use on in the experiment was a laptop (Asus model X550J) with a 15.6 inch LED 1920x1080 screen, Intel processor (i7-4720HQ), running Windows 8.1. The
3.6. Pilot study
Figure 3.3: Video player implementation, showing user interface at a branch point.
track pad on the laptop was disabled and instead an Exibel Wireless Optical Mouse (model SM-356AG) was used with the movement sensitivity setting at 1200 DPI.
3.6
Pilot study
Two pilot studies was conducted to test the method. The first pilot had 4 participants, 2 male and 2 female. In this pilot study two things in particular were being tested. The first was if the participants would understand what the task they were given was and the second was if they would understand that stalls was the problem in the videos. From the first to the second pilot study two things was changed or added. The free text questionnaires were reformulated to differentiate questions between what the participant wanted to do, what they actually did and if something could make it easier for them. The second change was to add a fourth video and a third free text questionnaire. The instructions to the forth video was intended to make a clear and final reveal to the participant that it was stalls that was the problem in the video. Thereby making the intent of the last free text questionnaire explicit. The second pilot study had 3 participants, all male. No further changes were made after the second pilot study.
3.7
Qualitative analysis
The study included a questionnaire with four questions, each asking the participants to an-swer in free text format. To analyse this data the qualitative analysis method thematic analysis was used.
Thematic analysis
Thematic analysis is a method developed for qualitative data with the purpose to locate, analyse and report patterns within the data (Braun and Clarke, 2006). The end result is a descriptive account for the overarching themes in the data-set. These themes can contain sub-themes within themselves. A theme has the purpose of encapsulating something in the data that relates to the research-questions, but must also be supported by a greater pattern within the data (Braun and Clarke, 2006). One answer from a participant does not constitute a theme. But many answers in multiple participant conditions and in conjunction to many videos does constitute the basis for a theme. A sub-theme then, is a finer nuance that is present within
3.7. Qualitative analysis
that theme. The descriptive nature of the result of the analysis has the effect of making the result unfit for theory building. The purpose is to make existing patterns within the data more clear, not to extrapolate from them (Braun and Clarke, 2006). The use of the thematic analysis in this thesis is to make the overarching themes in the participant answers more clear in relation to the research questions. Especially as a descriptive account for the participants experience of the stalls and the video players. As the SSSQ and Nasa-TLX measurements are designed to quantify the experience rather than provide a rich account of it. The combination of the two methods are intended to provide a fuller account for the different aspects a user can experience at a stall event when watching video, than either method is capable of on its own.
As it is possible to conduct thematic analysis in different ways, it is recommended to include the chosen method (Howitt, 2016). The following steps were used in the thematic analysis and are based on the steps presented by Braun and Clarke (2006).
1. Grouping data according to experimental conditions - The participants answers were grouped to provide a structure that conveys which answers came from similar UI set-tings and video format.
2. Familiarising with data - An inital reading of the data was conducted to construct an overview of the data.
3. Initial coding of data on an answer to answer basis - Each answer was coded to convey the expressed intent in the answer. If no existing code fit the answer a new code was created to convey the aspect of the answer.
4. Collecting and combining codes into themes - The codes were compiled and reviewed. If two or more codes were similar they were combined into one more inclusive code or a theme. Themes were constructed by the codes that were present as overarching themes across multiple participant groups and video formats.
5. Reviewing themes, defining and describe themes - A final review of the themes were conducted to ensure that they matched the answers included in the themes. In conjunc-tion to this a descripconjunc-tion of the themes were produced.
6. Reporting key themes - The themes were finally reported in the thesis on a theme by theme basis to focus on the overarching spread of the themes.
The results from the thematic analysis can be viewed in Section 4.4 Thematic analysis of free text answers - Videos 2-4.
4
Results
4.1
NASA-TLX video 1 analysis
The Nasa-TLX results from video 1 was analysed in order to determine if the conditions differed from each other. The results were analysed as Raw-TLX (Grier, 2015), which means that all dimensions were weighted the same. The four conditions averages and standard deviations on the six dimensions of NASA-TLX from video 1 were graphed to determine which conditions ranges were not overlapping.
Figure 4.1: Nasa-TLX means and standard error of measurments from video 1. Excluding participants with shorter stalls.
With regards to the conditions of interest, seen in Section 3.1 (Study Design), three cases were found where the two conditions standard error did not overlap:
4.1. NASA-TLX video 1 analysis
• Comparing BP and BN on the performance dimension • Comparing LP and LN on the performance dimension
Non-completion participants Video 1
Three out of four conditions had participants who did not complete all parts necessary for the Nasa-TLX score to be valid for them, due to an unforeseen problem with the implementation of the stalls in the videos. There were two sources for this: participants not completing the Nasa-TLX questionnaire and participants starting the video during one of the stalls in the first video. These participants were excluded from the analysis.
Division of participants in conditions for video 1
BP BN LP LN Total
Total part. 15 7 14 7 43
Excl. non-compl. part. 15 6 8 5 34
Table 4.1: Total number of participants and number of participants excluding non-completion participants in the analysis of the Nasa-TLX result from video 1.
The row of total part. shows the number of participants for each condition disregarding any non-completion issues. The row Excl non-compl. part. displays the number of par-ticipants in each condition after excluding the parpar-ticipants who either did not complete the TLX questionnaire or had a shorter stall in the first video in the analysis of the Nasa-TLX results from the first video. Only the participants who had no non-completion issues in their data were included in the analysis.
BP vs LP - Temporal
A Kolmogorov-Smirnov test showed that both conditions significantly deviated from nor-mality BP D(15) = 0.273, p < .01. LP D(8) = 0.443, p < 0.01. Homogeneity of variance was tested using a Levene’s test. The homogeneity differed significantly between BP and LP F(1, 21) = 6.82, p < .02. The Mann-Whitney U test was selected to determine if there was a sig-nificant difference between BP and LPs temporal scores. No sigsig-nificant difference was found BP(Mean = 2.33) and LP(Mean = 1.38) U = 38.5, z = -1.54, p = 0.17, r = -0.34.
BP vs BN - Performance
Normality was tested on both conditions data using a Kolmogorov-Smirnov test. The BP data was shown to deviate significantly from normality BP D(15) = .269, p < .01. While the data from the BN condition was distributed normaly BN D(6) = .187, p = .2. A Levene’s test showed that the variance in the two conditions did not differ from each other F(1,19) = .841, p = .37. Mann-Whitney U test was used to determine if there was a significant difference between BP and BNs performance score. No significant difference was found BP (Mean = 1.93), BN (Mean = 3). U = 30, z = -1.226, p = .27, r = -0.27.
LP vs LN - Performance
A Kolmogorov-Smirnov test was used to determine normality of the two conditions perfor-mance data LP was found to deviate significantly from normality LP D(8) = .367, p < .01. While LN was found to be normaly distributed LN D(5) = .3, p = .16. Homogeneity of vari-ance for each condition was tested using a Levene’s test. The varivari-ance in the data in the two conditions was shown to deviate significantly F(1,11) = 18, p < .01. A Mann-Whitney U test was used to determine if there was significant difference between the two conditions performance score. No significant difference was found U = 10.5, z = -1.441, p = .17, r = -0.4.
4.2. NASA-TLX video 2-3 analysis
4.2
NASA-TLX video 2-3 analysis
The Nasa-TLX results from video 2 and 3 were analysed using Wilcoxon signed-rank tests due to the non-parametric quality of the data. The number of participants were decreased as a result of participant non-completion issues. The purpose of the analysis was to investigate if there was a significant change in the workload experienced by the participants as they were assigned a new condition after the second video. For example a participant who is in the BP condition in video 1 & 2 changes to the LP condition for video 3 & 4. The number of participants in each comparison can be seen in Table 4.2.
Participants for Nasa-TLX video 2-3 analysis
BP vs LP 9
BP vs BN 13
LP vs LN 7
Table 4.2: Number of participants included in the analyses of the Nasa-TLX results from video 2-3.
No regards to the order to which the participant was assigned a condition. This meant that in the BP vs LP comparison both participants who first watched a branched video with a playback bar and the watched a linear video with a playback bar and participants who first watched a linear video with a playback bar and then watched a branched video with a playback bar are included in the same analysis.
The analysis was done on all 6 dimensions of the Nasa-TLX questionnaire. No significant results were found in this analysis.
4.3
Short Stress State questionnaire analysis
The results from the SSSQ analysed according to the procedure described by Helton and Näswall (2010) and Shapiro et al. (2017). With the purpose of identifying if there was a difference in stress state between the four main conditions. First the answers from the par-ticipants were translated to their corresponding numbers in the scale. Then an average was calculated for each participant and SSSQ dimension. The items categorization into a dimen-sion was done according to Helton and Näswall (2010). After this the ratio between the pre- and post-questionnaire for each participants dimension was calculated by dividing the post-questionnaires average result with the pre-questionnaire average result. This was done according to to Shapiro et al. (2017).
Non-completion participants Short Stress State Questionnaire
As mentioned before, an unforeseen problem with the implementation of the stalls in the videos created some issues with non-completion of the data collection with regards to some participants. Some participants experienced a shorter stall for video 1, video 2 or both of them. Some did not complete one or more of the SSSQ questionnaires, or a questionnaire between the SSSQ questionnaires. Making their stress state results non-comparable to the others.
Division of participants in conditions for SSSQ
BP BN LP LN Total
Total part. 15 7 14 7 43
Excl. non-compl. part. 14 6 7 3 30
Table 4.3: Total number of participants and number of participants excluding non-completion participants in the analysis of the SSSQ results.
4.4. Thematic analysis of free text answers - Video 2-4
The row of total part. shows the number of participants for each condition disregarding any non-completion issues. The row Excl. non-compl. part. displays the number of partici-pants in each condition after excluding the participartici-pants who either did not complete one of the SSSQ questionnaires or had a shorter stall in the first or second video. In the analysis of the SSSQ results, only the participants who had no non-completion issues in their data were included.
SSSQ - Engagement ratio
A Kolmogorov-Smirnov test was used to determine normality of the four conditions engage-ment data. No condition was found to deviate significantly from normality. But the LN condition was excluded due to the small number of participants LN participants = 3, BN D(6) = .133, p = .2, BP D(15) = .126, p = .2, LP D(7) = .258, p = .17.
Leneve’s test was used to determine if the variance in the Engagement ratio data deviated significantly. Engagement ration was found to not differ significantly F(3,27) = .288, p = .83. A One-way ANOVA was used to determine if there was a significant difference between the four conditions in Engagement ration. No significant difference between the conditions Engagement ratio was found F(3,27) = 0.212, p = .89, n2 = .02.
SSSQ - Distress ratio
A Kolmogorov-Smirnov test was used to determine normality of the four conditions Distress ratio data. No condition was found to deviate significantly from normality. But the LN con-dition was excluded due to the small number of participants LN participants = 3, BN D(6) = .188, p = .2, BP D(15) = .170, p = .2, LP D(7) = .139, p = .2.
Leneve’s test was used to determine if the variance in the Distress ratio data was shown to deviate significantly. The variance in distress ration was found to deviate significantly F(3, 27) = 3.109, p = .04. A One-way ANOVA was used to determine if there was a significant difference between the four conditions in Distress ratio. No significant difference between the conditions Distress ratio was found F(3,27) = 0.718, p = .55, n2 = .08.
SSSQ - Worry ratio
A Kolmogorov-Smirnov test was used to determine normality of the four conditions Worry ratio data. No condition was found to deviate significantly from normality. But the LN con-dition was excluded due to the small number of participants LN participants = 3, BN D(6) = .202, p = .2, BP D(15) = .106, p = .2, LP D(7) = .146, p = .2.
Leneve’s test was used to determine if the variance in the Worry ratio data was shown to deviate significantly. The variance in Worry ration was found to not deviate significantly F(3, 27) = 1.186, p = .33. A One-way ANOVA was used to determine if there was a significant difference between the four conditions in Worry ratio. No significant difference between the conditions Worry ratio was found F(3,27) = 0.516, p = .675, n2 = .06.
4.4
Thematic analysis of free text answers - Video 2-4
A thematic analysis of the answers from the free text questionnaire in relation to video 2-4 found a number of overarching themes. The themes can include answers from multiple par-ticipant conditions and questions and provides the fundamental categories of the parpar-ticipants answers. The questions the participants answered can be viewed in Section 3.5 Materials, or in full in Appendix A.5.
Themes
4.4. Thematic analysis of free text answers - Video 2-4
• Stall focused - This theme was found in the first question of the questionnaire, and includes answers that indicate that the user understood the problem in the video as the stall event. An exemplary answer to this theme was:
"the video froze a few times" - Participant in the BP condition after video 2.
• Content focused - The content focus theme describes the answers that indicated that the participants focused on the content of the video. This was either in the question of what the problem in the video was, but answers focusing on the video content were also present in the other questions as well. Since the contents effect on the participant is outside the scope of this thesis, these answers were grouped together. An exemplary answer of this theme was:
"The choices didn’t make sense in the given instance." - Participant in the BP condition in response to question 1 after video 3.
• Non-answers and unawareness - The answers which either indicated the participant not understanding the question, not answering the question, or indicating an unaware-ness of a problem at all were placed in this theme. This was similar to the content focused theme in so much that it indicated a deviation from the intended focus of the experiment. But this theme differed in that the answers gave no indication to where the participants thoughts were on the present questions. An exemplary answer of this theme was:
"Still no idea" - Participant in the BP condition in response to question 1 after video 4. • Deliberate non-interaction - This theme includes answers indicating that the participant
chose to wait or not interact with the video player. As the answers were often from question 2 and 3, they can be thought to show that the participant expected the problem in the video to solve it self. As the design of the experiment was such that the video would start again after a set amount of time, this was a valid strategy to start the video. However, it seems that the participants that have given this answer might not have understood the tasks of the experiment in the way that they were intended to when the experiment was designed. This is because the tasks were asking the participant to find the reason for the stall, not to undo the stall. An exemplary answer of this theme was:
"Ingenting, visste den skulle dra igång igen", Eng. Nothing, knew that it would start again. - Participant in the BN condition in response to question 3 after video 3.
• UI focused - The UI focus theme is a composite of answers that indicated that the user interacted with some part of the UI. The theme also includes answers that indicated that the user intended to interact with some part of the UI but was unable to. The answers indicate that the participant was focused on the expected and presented tools in the UI to solve the task. An exemplary answer of this theme was:
"move the mouse and click on the screen and to move the progress bar forward" - Par-ticipant in the LP condition in response to question 2 after video 3.
The theme also has two more precise sub-themes from question 4, which prompts the participant to give suggestions to what could have made the reason for the stall easier to find. Two themes occurred in those answers:
1. Adding loading symbol to UI - The reason given for this was to better indicate that the video had indeed stalled. An exemplary answer of this subtheme was: "a more clear way to show that the video was buffering, like a spinning circle" -Participant in the LP condition in response to question 4 after video 4.
4.4. Thematic analysis of free text answers - Video 2-4
2. Adding playback bar to UI - This sub-theme included answers that suggested something to visualize the timeline of the video. Interestingly enough this theme was present among user in both conditions with and without a playback bar in their UI. An exemplary answer of this subtheme was:
"If the progress bar showed up." - Participant in the LN condition in response to question 4 after video 4.
• Information seeking - This theme was present in the answers that indicated the partici-pant wanted to find out more about the state of the video player but did not direct the potential search to the UI or outside the video player. It suggest that some of the par-ticipants did not see a clear path to where the reason of the problem might lie and there fore did not indicate were they wanted to start too look for the missing information. An exemplary answer of this theme was:
"Jag försökte hitta information i spelaren men lyckades inte" Eng. I tried to find infor-mation in the player but did not succeed. - Participant in the LN condition in response to question 3 after video 4.
• Outside video player focus - Answers that indicated the user wanted to interact with systems outside the video player were included in this theme. This was mostly in the case of the user expressing a belief that the problem of the stall might be related to bandwidth or download speed. Suggesting that the participant ruled out the video player as an obvious problem for the videos playback. An exemplary answer of this theme was:
"Check the internet connection, and the computer to see if something had happened to either one of them." - Participant in the BP condition in response to question 2 after video 4.
5
Discussion
5.1
Results discussion
Thematic analysis discussion
The free text questionnaire and the following thematic analysis was conducted to answer the third research question, which themes are present in the users expression regarding in-tentions, actions and expectations of the experience in the task? The answers in the themes indicate different ways the participants understand the task in the experiment and they also give indication of different problem solving strategies. The participants show different un-derstandings of what is meant by a problem in a video by the presence of themes such as stall focused and content focused. The themes also gave clues to where the participants searched for answers to their understanding of the task. Some answers suggests that some participants understood the task not as find out the reason for the problem, but as solve the problem. The following quote from one of the participants answering the question, what steps did you ac-tually take to find out the reason for the stall? exemplifies this.
"I tried moving the mouse, this time nothing was showing on the screen though, I just pressed P, like I had done before on all the other videos" -Participant in the BN condition, in response to question 3 after video 3. See Section 3.5 Materials, or Appendix A.5 for the full question.
The answers shows the description of an attempted interaction, but when no feedback was received the participant completes the task in the same way as when there were a re-sponse in previous videos.
The two themes of non-answers and unawareness and deliberate non-interaction seems to be similar in the sense that both have answers that indicate the participant being uncertain of what to do. The difference then seems to be that the answers in the deliberate non-interaction shows signs of a conclusion that not interacting is a way to solve the task. While the answers from the non-answers and unawareness theme shows no such conclusion. This makes an-other understanding of how to solve the task salient. To deliberately not interact with the UI, should be considered an action in it self, which then point towards some participant using a problem solving strategy of simply waiting when encountering a stall in a video.
5.1. Results discussion
Two themes gave the indication that some participants did not think that what they needed to complete the task was inside of the video player. These themes were the information seek-ing focus and the outside video player focus. In some sense this idea is completely valid, as some answers in these themes came from participants in conditions that had UI that did not indicate the reason for the stall. However, the information seeking theme was comprised out of answers that did not direct themselves toward an interaction with the UI. Instead they were more broader in requesting information of different kinds. One answer that exemplified this was the following quote.
"Diagnose that the video in fact was paused for some reason." - Answer from a participant in the BP condition answering question 2 after video 3. See Section 3.5 Materials, or Appendix A.5 for the full question.
The theme of outside video player focus differed from the information seeking theme in that it lacked the broadness and was instead more specific in that the participant seemed to indicate that they believed that the answer to the task lied outside of the video player. The story these two themes tell are the openness of the participants to conduct problem solving outside of the direct interaction with a system. This can be an indication of were an user experience designer should make a decision of where to draw the line of what to include in the UX design and what to leave out. For example, one participant in the LP condition gave the answer "check the connection" to the question, what did you want to do to find out the reason for the stall? In a design process there is a decision available after this feedback. The decision being if the UI should include the function of checking current web-connection or to leave that functionality to the operating system outside of the video player experience.
The theme UI focused indicate a trend that both the conditions that had a playback bar and the conditions that lacked a playback bar was missing a buffer indicator. Even though some of the participants of the conditions without a playback bar expressed that a playback bar was needed there were more participants who expressed a need for a buffer indicator. As these results are from qualitative data, a conclusion to draw from this could be that both a playback bar and a buffer indicator are seen by the participants in this experiment as vital parts of the experience of watching a video when a stall occurs. But as thematic analysis provides a descriptive result a generalisation of the results should be done cautiously.
The thematic analysis provided insight to research question three, which themes are present in the users expression regarding intentions, actions and expectations of the expe-rience in the task? The users reported different intentions at the stall, mainly trying to start the video again, search for more information, both focused on components in the UI. Some reportedly opted to simply wait and do nothing. The actions the users reported overlapped greatly with their intentions if there were a playbackbar to support the actions. Clear themes of the users expectations were that of the presence of a playbackbar and also a buffer indica-tion icon.
Nasa-TLX results discussion
The Nasa-TLX measurments were used to answer research question one and two.
• Does the presence of a playback bar affect the user reported stress and workload of linear and branched video formats when a video stall occurs?
• Is there a difference in user reported stress and workload between linear and branched formats when a video stall occurs?
The results from the analysis of the Nasa-TLX measurements from video 1 showed no difference between the different dimensions of workload the questionnaire instrument mea-sures. There might be multiple reasons for this. The tasks given to the subjects might have
