IN
DEGREE PROJECT COMPUTER SCIENCE AND ENGINEERING, SECOND CYCLE, 30 CREDITS
STOCKHOLM SWEDEN 2020,
Designing and using gamification elements to improve students’
user experience in a video-based mobile language learning app
THOR GALLE
1 Abstract
With the rise of the smartphone industry, the domain of mobile-assisted language learning (MALL) has increasingly grown. A large number of language learning applications have been developed aiming to support individuals’ second language acquisition on various levels, e.g., by teaching vocabulary and grammar to improve reading and listening comprehension.
The viability of these applications has been examined in literature and shows overall positive but mixed results. On one side, their success is partly attributed to gamified design elements. These are reported to improve the user experience (UX) and boost learners' motivation. On the other side, the primary reliance on decontextualized vocabulary and grammar exercises is criticized.
In response, one such application, SVT Språkplay developed by the Swedish non-profit Språkkraft, incorporated television programs as a longer form of context. This introduced novel video-based learning functions. The first aim of this thesis was to start filling a gap in research by evaluating the usability and user experience of these functions. This was performed through user tests and interviews with seven second language students who used the app to learn Swedish over a period of at least two weeks. The second aim of the thesis was to improve the usability and user experience of the problematic learning
functions through a user-centred design process with the ultimate goal to improve learner support and vocabulary acquisition outcomes. The study participants consisted of doctoral researchers and students recruited from a basic Swedish course at KTH. They represent a demographic that benefits from learning Swedish to improve their job opportunities.
The initial evaluation results were analysed through the lens of the MUUX-E theoretical framework [10], a framework for evaluating the “user experience and educational features of m-learning environments”. The evaluation showed that the core vocabulary learning aids directly integrated into the video watching experience were perceived as useful.
Conversely, the gamified learning functions outside of the video watching experience were found to be scarcely used as intended.
The subsequent user-centered design process improved upon the design of problematic learning functions by adhering to the principles of the MUUX-E framework. Concretely, more varied contextualized vocabulary exercises were designed, more options for user customization were included and feedback and progress metrics such as “streaks” were highlighted. An evaluation of the design with the same participants as the initial evaluation suggests that these changes would improve the usability and user experience of the
application.
Further research should evaluate an implemented end-product based on the proposed designs in a real-life setting. In that case, its pedagogical merit should also be evaluated.
In summary, this thesis found that mobile video-based MALL apps such as Språkplay can provide usable and enjoyable language learning functions.
1
1.1 Keywords
mobile-assisted language learning, video-based language learning, usability, UX, usability, user experience and educational features of m-learning environments (MUUX-E), gamification
Abstract | 3
2 Sammanfattning
Med tillväxten av mobiltelefonbranschen har domänen för mobilassisterad språkinlärning (MALL) ökat alltmer. Ett stort antal språkinlärningsapplikationer har utvecklats för att stödja individers förvärv av andraspråk på olika nivåer, t.ex. genom att lära ut ordförråd och grammatik samt för att förbättra läs- och hörförståelsen.
Dessa applikationer har undersökts i litteraturen och visar positiva men blandade resultat.
Å ena sidan tillskrivit deras framgång delvis spelelementen. Dessa rapporteras förbättra användarupplevelsen (UX) och öka elevernas motivation. Å andra sidan kritiseras det primära förlitandet på dekontekstualiserade ordförråd och grammatikövningar.
Som ett svar skapades en sådan applikation, SVT Språkplay, utvecklad av den svenska ideella föreningen Språkkraft, vilken använder TV-program som en längre form av
språkligt sammanhang. Detta introducerade nya videobaserade inlärningsfunktioner. Det första syftet med denna uppsats var att börja fylla ett hål i forskningen genom att utvärdera användbarheten och användarupplevelsen av dessa funktioner. Det gjordes genom att genomföra användartester och intervjuer med sju andraspråkstudenter som använde appen för att lära sig svenska under en period av två veckor. Det andra syftet med arbetet var att förbättra användbarheten och användarupplevelsen för dessa inlärningsfunktioner genom en användarcentrerad designprocess med det slutliga målet att förbättra studentens stöd. Studiedeltagarna bestod av doktorander och studenter rekryterade från en
nybörjarkurs i svenska på KTH. De representerar en demografisk nytta av att lära sig svenska för att öka deras tillgång till den svenska arbetsmarknaden.
De första utvärderingsresultaten analyserades genom tillämpning av MUUX-E-ramverket, ett ramverk för att utvärdera “user experience and educational features of m-learning environments” [10]. Det visade att de grundläggande ordförrådets inlärningshjälpmedel som direkt integrerades i video upplevdes som användbara. Omvänt användes knappt alls de spelifierade inlärningsfunktionerna utanför videon.
Den efterföljande användarcentrerade designprocessen förbättrades vid design av problematiska inlärningsfunktioner genom att följa principerna i MUUX-E-ramverket.
Konkret utformades mer varierade kontextualiserade vokabulärövningar, fler alternativ för användaranpassning inkluderades och feedback- och framstegsmetriker som “streaks”
lyftes fram. En utvärdering av designen med samma deltagare som den första utvärderingen tyder på att dessa förändringar skulle förbättra användbarheten och användarupplevelsen.
Ytterligare forskning bör utvärdera en implementerad slutprodukt baserad på de
föreslagna designförbättringarna i en verklig miljö. I så fall bör dess pedagogiska meriter också utvärderas.
Sammanfattningsvis fann vi att videobaserade MALL-appar som Språkplay kan ge användbara och roliga språkinlärningsfunktioner.
3
2.1 Nyckelord
mobile-assisted language learning, video-based language learning, usability, UX, usability, user experience and educational features of m-learning environments (MUUX-E),
gamification
3 Acknowledgments
First of all I would like to thank Niss Jonas Carlsson, CEO of Språkkraft and industrial adviser to this thesis, for allowing me to examine and reimagine the app developed by his non-profit. The regular discussions we had and insights from Språkkrafts’ earlier R&D efforts were more than once an inspiration for my work. My views did not always align with the direction of the app development, so I appreciate Niss Jonas’ willingness to welcome a different perspective on the design of the app. I genuinely believe Språkkraft is
contributing to an important social challenge in an innovative way, and I am grateful that my thesis could contribute to that.
Next, I would like to thank my supervisor Olga Viberg for the continuous support in the planning and writing of this thesis. Her knowledge of the field and regular and precise feedback were an invaluable source of help. In this context, I should also mention the group supervision sessions where my peers Edward Lindén, Xinyan Luo and Martin
Wedberg taught me useful lessons, and where I could share my progress and concerns once in a while.
Finally, I want to express my gratitude to the seven study participants who voluntarily spent their time on interviews and user tests to help improve Språkplay. Without them, this study would have been difficult.
Stockholm, May 2020 Thor Galle
5
4 Table of contents
1 Abstract 1
1.1 Keywords 2
2 Sammanfattning 3
2.1 Nyckelord 4
3 Acknowledgments 5
4 Table of contents 6
5 List of acronyms and abbreviations 12
6 Introduction 1
6.1 Context of this work 2
6.1.1 Societal background 2
6.1.2 Learning language with television content 3
6.1.3 SVT Språkplay: the solution? 3
6.1.4 Reception of Språkplay 4
6.2 Problem 5
6.3 Purpose 7
6.4 Goals 7
6.5 Research Methodology 7
6.6 Delimitations 8
6.7 Summary 9
6.8 Structure of the thesis 9
7 Description of the examined application 10
7.1 Video watching functions 10
7.1.1 Control functions (pause, jump forward, jump backward) 10
7.1.2 The translation pop-up 11
7.1.3 The color system 12
Initial color determination 12
Changing word color 13
7.2 Active learning functions 13
7.2.1 Word test 13
7.2.2 Word list 14
7.3 Self-monitoring functions 15
7.3.1 Profile page 15
7.3.2 Activity statistics dashboard 15
8 Background 16
8.1 Video and language learning 16
8.1.1 Media and language learning 16
8.1.2 Learning with Closed Captions 16
8.1.3 Control and “glossed” captions 17
8.1.4 Implications for a video-based MALL application 17
8.2 Mobile assisted language learning (MALL) 17
8.2.1 Defining MALL 17
8.2.2 MALL with immigrants 18
8.2.3 Video-based learning in MALL 18
8.3 The role of gamification 19
8.3.1 Defining gamification 19
8.3.2 Game-based learning and gamification within MALL 19 8.3.3 The value of gamification for vocabulary learning 20
8.3.4 Implications for this project 20
8.4 Usability, User Experience (UX) and User-centered design (UCD) 20
8.4.1 Why and Usability and UX? 20
8.4.2 Usability and User Experience in MALL 21
8.5 Summary 21
9 Theory 22
9.1 The Double Diamond framework 22
9.2 The evaluation framework MUUX-E 23
10 Methods 24
10.1 Discover 24
10.1.1 Literature review 24
10.1.2 State-of-the-art review 24
10.1.3 Self-guided usability evaluation of the examined app 25
Initial evaluation 25
Cognitive evaluation 26
7
10.1.4 User test of the examined app 26
Participants 26
Initial survey 27
Meeting modality 27
User test process 27
Data analysis 28
10.1.5 Semi-structured interviews 28
Data analysis 29
10.2 Define 29
10.3 Develop 30
10.3.1 Low-fidelity prototype 30
10.3.2 High-fidelity prototype 30
10.3.3 Prototype evaluation with users 31
Group interview process 32
Data analysis 32
10.4 Ethics 32
11 Prototype development 33
11.1 Initial evaluation results of the examined app (Define) 33
Participants 33
11.1.1 General usability 34
User errors in the word test 34
The initial state of the color system 34
Efficiency of color switching 35
11.1.2 Web-based learning principles 35
Obstacles for the discoverability of learning features 35 Non-verbatim captions as imperfect learning media 35
11.1.3 m-Learning features 36
Useful translation pop-up 36
Missing ability to hide captions 36
Recommendations for learning & viewing are not personalised 37
Limited encouragement of active learning 37
Missing features to integrate with learning context 38
11.1.4 Educational usability 38
Clarity of the color system at first use 38
The clarity of learning goals 39
11.1.5 User Experience 39
Aesthetic appeal of the application 39
Gamified elements as a fulfiller of motivational needs 40
Emotional viewing context 40
11.1.6 Summary of the MUUX-E findings 41
11.2 Design of the improved prototype (Develop) 41
11.2.1 Low-fidelity prototype sketches 41
11.2.2 High-fidelity prototype 43
Home screen 43
Navigation bar & information architecture 44
‘Learn’ screen & word list management 44
Video watching screen 45
Collaborative caption quality assurance 46
Translation pop-up module 47
Word list screens 47
Word tests 48
Flash cards 49
Mixed exercises 50
Feedback and progression 52
12 Results 53
12.1 General usability 53
Video screen and transcript 53
New translation pop-up 54
Dashed underlines and hidden green underline 54
12.2 Web-based learning principles 54
Navigational architecture 54
Improving media quality by mismatch reporting 55
12.3 m-Learning features 55
Personalized “Continue watching” 55
9
Customizable word lists 56
Hiding captions 56
12.4 Educational usability 57
Modularised exercises 57
Video-based exercises 57
Learning goal reminders 58
12.5 User Experience 58
Aesthetic experience 58
Gamified elements 58
12.6 Summary 58
13 Discussion 59
13.1 Evaluation of the original app 59
13.1.1 Limitations 60
13.2 Design process and results 61
13.2.1 Limitations 61
13.3 Further work 62
13.4 Ethics 62
14 Conclusions and Future work 63
15 References 64
16 Appendix A: The initial survey for study participants 67
16.1 👋 Hey, thanks for your interest in the Språkplay study! 67
16.1.1 Do I have your permission? 67
16.2 A bit more about you 68
16.3 Done! 69
17 Appendix B: Semi-structured interview questions 69
17.1 A. Contextual inquiry 69
17.2 B. Learning scenario follow-up (examined application) 69
18 Appendix C: Prototype evaluation questions 7018.1 Generic Usability 70
18.2 Web-based learning principles 70
18.3 Educational usability 70
18.4 m-Learning features 70
18.5 User Experience 71
11
5 List of acronyms and abbreviations
CEFR common european framework of reference for languages DGBL digital game-based language learning
HCI human-computer interaction
L2 second-language
MALL mobile-assisted language learning MGBL mobile game-based learning
MUUX-E usability and user experience of mobile educational environments (evaluation framework)
UCD user-centred design
UX user experience
RQ research question
SVT Sveriges Television (company) SLA second language acquisition
6 Introduction
This chapter presents the specific problem that this thesis addresses, the context of the problem, the goals of this thesis project, and finally, it outlines the structure of the thesis.
In recent years, mobile-assisted language learning (MALL) has risen as a means to support learners’ second language (L2) acquisition in informal language learning on various levels [2,18,25,31]. Compared to other learning methods, MALL brings flexible use of time and space for learning, good alignment with personal needs and preferences and the possibility of keeping learning efforts ongoing while waiting or commuting [22]. Popular
state-of-the-art MALL applications such as Duolingo have shown considerable merit with regard to explicit learning outcomes such as vocabulary acquisition and grammar
understanding [2,25]. Contrarily, challenges were also identified, such as faltering learner motivation and persistence [2,15,25]. Moreover, prior research found limitations regarding the development of implicit skills such as listening and speaking comprehension [25].
The impaired viability of using MALL for improving implicit learning outcomes has been attributed to a primary reliance on decontextualized exercises on the individual sentence level [11,25]. According to Krashen, such exercises promote conscious language learning as opposed to subconscious language acquisition [21]. He states that methods encouraging the latter, such as the extensive reading of L2 texts [20], have been proven to perform better on communicative tests where real time, implicit language knowledge is required [21].
Recently, new mobile technology, such as developed by the Swedish non-profit organization Språkkraft 1attempts to cater to this critique by merging more extensive language context (television) into language learning activities. In their application SVT Språkplay , this concretely entails a combination of (1) an interactive video watching 2 experience with “glossed” captions, i.e. captions where the translation of a word can be requested at any time [27], and (2) exercises with digital game-based learning elements that are created from the context of the video transcript. In this process however, there remain challenges. First, the area of context-based MALL is largely unexplored in literature and little state-of-art precedents are available for comparison. At the same time, the
domain of L2 learning has known a growing body of research on video-based learning with glossed captions that suggests promising results regarding vocabulary acquisition
[26,34,37]. This calls for research of the applicability of these findings to MALL.
Second, the usability and user experience (UX) of other state-of-the-art MALL applications have been praised, notably of Duolingo [32,35,36]. Furthermore, digital game-based elements (DGBL) and gamification have been linked to their success in vocabulary learning acquisition outcomes and UX [4,11,25,32]. However, these game-based exercises are also the elements that were criticized before for being ‘decontextualized’. This raises the questions: are decontextualized exercises related to a good UX? What is the usability and UX of the contextualized exercises that Språkplay aims to provide?
1 The website of the organization may be found at https://sprakkraft.org
2 A website describing SVT Språkplay is found at https://sprakkraft.org/svt-sprakplay/
1
This degree project specifically takes up that last question. It studies the usability and user experience of the aforementioned novel vocabulary learning functions found in the MALL app SVT Språkplay, a smartphone app developed by Språkkraft. The app and the
non-profit behind it will be discussed in more detail in the section 6.1.3. A description of the learning functions of the app can be found in section 7. This thesis examines the app within the context of its use by immigrant students and researchers in Swedish higher education, a target group with a need for more immersive language learning, as will be explained in the next section. It aims to find out which learning functions provide a good usability and user experience, and how, from the human-computer interaction (HCI) perspective, they could be improved and extended to optimize second language learning.
6.1 Context of this work
6.1.1 Societal background
For immigrants, speaking the main language used in a host country is key to access job markets, social - and cultural facilities. Yet upon arrival, many of them do not speak that language. One such immigrant learner group in Sweden is foreign students and
researchers, for whom good skills in English are often sufficient to enter as a student or as a doctoral researcher in academic environments. Yet, not knowing Swedish brings the group to a clear disadvantage when looking for work. Statistics Sweden found in a survey with foreign-born, highly-educated people between the ages 24-64 that they are 22% less likely to be employed compared to native born Swedes [33]. A study based on this research concludes that “a lack of contacts was by far the most common reason why foreign born persons had difficulties in getting positions that they applied for” and that “the second most common reason was difficulty with the Swedish language.” [33 p. 23]. It is not only these immigrants that ‘suffer’, but also Swedish society. For example, the management of KTH in Stockholm, has recently made it a requirement for all foreign staff members to have a learning plan for Swedish [24]. The reasons given are twofold. First, less knowledge of Swedish may pose a risk in decreasing Swedish research outputs, reducing the relevance of research for the local society. Second, the administration of certain university roles, especially more senior ones such as “dean”, requires command of the Swedish language.
Not knowing the language may prevent researchers from growing in their academic career, and puts more pressure on Swedish-speaking employees to fulfill those roles.
From the aforementioned challenges, it follows that learning Swedish is critical for immigrant students and researchers that wish to work and quickly integrate in their host country. It removes barriers in both academic and industrial career paths. Additionally, it may be indirectly useful to establish contact with potential employers and build a network.
Finally, it would be beneficial for society.
To address this need, the Swedish government-sponsored program Utbildning i svenska för invandrare (SFI) offers free Swedish education for immigrants. However, this formal educational system is only one way to solve the problem, and it may not be sufficient.
6.1.2 Learning language with television content
Modern digital technology could increase the availability of tailored Swedish language content for second language learners. Most people in Sweden carry a smartphone wherever they go, which provides an opportunity to find and explore content such as news articles and television programmes outside of classrooms. One potential source is the public television provider SVT. SVT offers free programmes that can familiarize the watcher with real-life spoken Swedish, national news and Swedish culture.
However, watching regular television programmes may prove a steep challenge for basic language learners. Indeed, the Common European Framework of Reference for Languages (CEFR), a framework that categorizes levels of language mastery, deems this a challenge.
There are six CEFR levels: A1, A2, B1, B2, C1, C2. A1 denotes the basic level of an absolute beginner, while C2 denotes near-native proficiency in writing, speaking and listening. In the framework’s level descriptors, learners at the A1 level are not expected to understand any second language video material [5]. Learners at the A2 level should be able to identify changes of topic and certain points of commentary supported by visuals [ibib.]. For full comprehension, obstacles exist in speech that may be too fast, words and sentences that could be too complicated or the subject matter that may be too unfamiliar. Only at the upper-intermediate B2 level does the framework expect full comprehension of video material [ibid.].
This may lead basic learners to refrain from studying this challenging language content and to stick with the level-tailored material from formal education resources. But is this not a missed opportunity? How could the video content be adapted so that it fits the learner’s level?
Here technology may also provide an answer. In the last decade, mobile devices have been increasingly used to facilitate self-regulated second language learning. Apps such as Duolingo, Babbel and Quizlet with millions of users aim to teach vocabulary and grammar by means of interactive exercises, substituting or complementing formal education. In doing so, they achieve remarkable results. They also employ gamification, i.e., a way of designing experiences where progress, achievements and play are introduced to a
non-game setting. Could this mobile technology also be used to aid in the consumption of Swedish television programmes? Is there a marriage possible between this interactive, mobile way of learning vocabulary and mobile consumption of television content?
A non-profit organization in Sweden called Språkkraft partnered with the Swedish public broadcaster SVT to develop such an application, called SVT Språkplay. This degree project has been conducted in collaboration with Språkkraft and evaluated the usability and user experience of Språkplay.
6.1.3 SVT Språkplay: the solution?
Språkkraft is a Swedish NGO that aims to accelerate the integration of immigrants in Sweden by focusing on their development of Swedish language skills. Their main activity is the development of freely available digital applications to facilitate Swedish language
3
learning. They are primarily active in Sweden but are soon expanding their activities to other countries in the near future.
SVT Språkplay is a mobile app developed by Språkkraft and published in a partnership with SVT, the Swedish public broadcaster. It has 390 000 downloads as of April 2020 across the iOS and Android platforms and 50 000 monthly active users as of February 2020. It offers mobile access to Swedish television programs from SVT with aids in language learning: the closed captions can be translated to the language learners' first (known) language on-demand. This is the crucial feature that provides learners with the means to increase their consumption of Swedish media content. A description of all learning functions of Språkplay is found in the next chapter (7).
It is the intention of Språkkraft that by frequently using the app, learners could improve their vocabulary and grammar knowledge but also gain awareness of Swedish culture and customs and through the television programmes. But does this intention materialize?
6.1.4 Reception of Språkplay
Learners have been overwhelmingly appreciative of Språkplay. 80% of 762 user reviews on Google Play gave the app a 5-star review (on 5 stars) . They highlight mainly the concept of 3 combining learning with television watching, and reportedly gain benefits from it:
“This application teaches me more than school 👍” (translated by the author)
“Den här applikationen lära mig mer än skola 👍” (original) - Mohamad A. 1 June 2017, 5 star rating
“Really really nice idea and implementation. It makes it so much easier to learn new vocabulary. And the exercises are quite engaging. Great overall”
- Marco A, 1 February 2017, 5 star rating
“Fantastic way to help with my Swedish. I have found this much more beneficial than any SFI course I have taken. One of the most useful language apps I have encountered...love it!”
- Angelia S., 27 May 2017, 5 star rating
However, this is not the end of the story. Under this wide surface of appreciation, there are non-negligible problems to be found, including those that relate to the design, user
experience (UX) and usability of the app:
“Good idea. Great contents. Horrible UX. Supporting only African or Arabic languages mostly. Impossible to read the auto translation.”
̶ Christopher R, 24 April 2019, 2 star rating
3 Reviews retrieved Tuesday 21 April 2020 from the Apps Google Play store listing:
https://play.google.com/store/apps/details?id=se.svt.sprakplay.
Percentage calculated by comparing the pixel widths of the review distribution bars.
Number of pixels per star count: 5*: 100, 4*: 17, 3*: 0, 2*: 2, 1*: 7
“App has a lot of bugs but I have to give it 5 stars because the concept is awesome.”
̶ Anonymous, 10 August 2019, 5 star rating
“Awesome concept, a little buggy but hope it will be fixed soon. Layout as well needs makeover :)”
̶ Sharvil B., 23 September 2019, 4 star rating
“Wow, you have no idea how much I was looking for something like this. I think it opens in a web view. It is so slow at least on my old phone. I hope that it will change soon to like more app style with more features. Thanks a lot.”
̶ Farzad Z., 10 December 2016, 5 star rating
“Very good. I like this application [but] I hope that they have another [test] than the drag-and-drop test, it is a little dull [...]” (translated by the author)
“Jättebra. Jag tycker om applikationen [men] jag hoppas att de har annat [test] än drag och dropp test det är lite tråkigt [...]” (original)
- Jack A., 23 October 2016, 5 star rating
While users are reportedly satisfied with the main functionality, they stumble on bugs (often unspecified) and performance issues. After leaving out remarks that are less relevant to this thesis , there seem to be varied opinions regarding the exercises and learning tasks 4 that are available in the app, with one user calling them “quite engaging” and another hoping that “there are others than the drag-and-drop test” because it is “a bit dull” (freely translated). The aforementioned feedback reveals concrete issues, but statements like
“horrible UX” allude to a wider problem of a general usage experience that may not be satisfactory to some users.
6.2 Problem
The previous sections established that learning Swedish is critical for quick integration into Swedish society. At the same time, formal and traditional language learning systems are expensive in terms of time and financial cost and are typically limited to classroom environments. This imposes limits on the ways in which students can learn Swedish.
Mobile-assisted language learning (MALL) could provide an answer by making flexible learning possible and being accessible "anywhere and anytime" [11].
However, state-of-the-art mobile language learning technology is missing components of contextual language practice. Providing television content through this technology as attempted by Språkplay may bring a form or rich audiovisual context to mobile language learning, but in practice there is little academic evidence for the viability of this concept.
In literature, few studies were conducted focusing on video-based language learning in mobile contexts. Hsu et al. [14] compared the effects of fully-captioned, keyword-captioned and uncaptioned mobile video on the vocabulary acquisition of 5th graders learning
English as a foreign language. The study reported increased vocabulary comprehension in
4 The remark “supporting only African or Arabic languages” is not an issue that will be discussed in this thesis. Språkkraft supports mainly these languages since most immigrants in Sweden are from Arabic or African origin. The thesis was conducted with the English language as known language.
5
the case of captioned video. In another study, Hsu et al. [13] further evaluated different strategies of selecting keywords in captions on the motivation of mobile video learners, where a selection strategy adaptive to the learner’s skill was found to be most motivating for student learning. To the best of the author’s knowledge, no studies are available on usability and user experience of video-based mobile learning, and this thesis aims to fill this gap.
In the state-of-the-art as well, no other mobile video-based language learning solutions were found. Exceptions on non-mobile systems consisted of the web-extension “Language learning with Netflix” and the web application Lingopie . The former is a 5 6
community-made extension for Netflix. It adds multilingual captions and dictionary aids to Netflix videos. The latter affords similar functions to Språkplay, including a flashcard game. It is available as a web app and projected to be available on mobile devices.
Alongside the mentioned precedents, Språkkraft has released a novel concept with novel design decisions to combine DGBL techniques with television-watching. This introduces two challenges. First, given the lack of research and precedents it is unclear which of functions in Språkplay support users in their second language acquisition from a perspective of usability.
Second, the previous account of the reception of Språkplay suggests that the knowledge of user preferences and aims with regard to mobile game-based language learning in
video-based context may be incomplete, resulting in a “horrible UX” for one. For example, it remains a question what kinds of tests and “app layouts” the user would prefer.
6.2.1 Research questions
The aforementioned challenges and existing gaps bring us to the following research questions of this work:
RQ1: What is the usability and user experience of the vocabulary learning functions available in the examined application for students learning Swedish in higher education?
RQ2: How can gamification be used and designed in a video-based language learning application to improve (perceived) usability and user experience of vocabulary learning functions, for students learning Swedish in higher education?
The first question is concerned with the implementation of Språkplay as it has been available as of February-April 2020. It inquires the usability and user experience of the learning functions, specifically examining functions that aim to support vocabulary acquisition. The second question focuses on the game-based techniques that are already part of many MALL applications, and explores how they can be integrated with novel video-based MALL applications such as Språkplay to better conform with the needs and preferences of users. In terms of users, “students” should be seen broadly as both students and researchers, that is, all members of higher education who receive language education.
5https://languagelearningwithnetflix.com/
6https://lingopie.com/
Concretely, this work first inquires how the existing game-based functions have been used in Språkplay. Next, it aims to improve and extend them. RQ2 can therefore be split up in two sub research questions:
RQ2a) How well, in terms of usability and UX, do the video-based language learning functions in Språkplay based on gamification support the vocabulary acquisition for students learning Swedish in higher education?
RQ2b) How can they be improved by taking benefits of game-based learning?
6.3 Purpose
From a societal perspective, this degree project aims to contribute to the integration of foreign students into Swedish society by aiding in the research and development of novel digital tools that facilitate Swedish language learning.
From an industrial perspective, this project answers a need from the partner Språkkraft to evaluate the design of SVT Språkplay, and to design ways in which game-based learning techniques can be better matched with video-based learning to ultimately provide better learner support. A tangible outcome of this work for Språkkraft are designs that could be implemented in their app.
From an academic perspective, this degree project aims to identify the components of a successful integration of mobile game-based language learning with video-based language learning from a usability and UX perspective, which could provide a base for further research and development.
6.4 Goals
The general goal of this design-oriented research project is to study the usability concerns involved in the design of a video-based MALL application, with specific relation to its primary use for vocabulary learning purposes rather than entertainment purposes. It concretely examines Språkplay as a paradigm for a new set of MALL applications. Three sub-goals can be defined:
Firstly (1), this work aims to evaluate the usability and user experience of learning functions currently available in Språkplay. The functions may or may not involve digital game-based elements.
Secondly (2), this work aims to further design learning functions with gamification following the guidance of MALL and design research.
Thirdly (3), this work aims to evaluate the design produced in (2) in a user-centred way so that the functions match the expectations and preferences of users.
6.5 Research Methodology
The research questions were examined by employing a mix-method approach with complementing qualitative methods in a user-centred design process. This process was structured according to the Double Diamond framework, which will be further discussed in
7
section 9. Theory. As a part of the process, structured methods were used such as a
“cognitive walkthrough” and a semi-structured interview. Next to these, unstructured methods consisted of a free exploration of state-of-the-art apps and the examined app. For an overview of data collection methods, see table 1.
For methods with users, a recurring set of participants was recruited of students with the A2 CEFR language level from the course Svenska för Ingenjörer at KTH. More details on the data collection as well as data analysis methods are presented in section 10.Methods.
Targeted Research Question
Method Double
Diamond Phase
Users Primary Aim
RQ2 State-of-the-art review Discover no Design
RQ1, RQ2 Free app exploration Discover no Evaluation
RQ1 Cognitive walkthrough Discover no Evaluation
RQ1, RQ2a User test (30 min.) Discover yes (7) Evaluation RQ1, RQ2 Semi-structured interview (15 min.) Discover yes (7) Design RQ1, RQ2 Focus group discussion (1 hr.) Develop yes (4) Design
Table 1: Overview of data collection methods
6.6 Delimitations
To keep this study manageable as a degree project certain choices were made. Some are already embedded in the research question: the project only studies a video-based MALL app for a target group of students and researchers learning Swedish in higher education.
Inferences of results towards the learning of other languages, or towards other societal groups of learners are therefore not warranted.
Next, the evaluation and design in this work focused on scenarios in which vocabulary can be learned with a video-based context. While there may have been potential for implicit or explicit instruction on grammar, cultural knowledge or other language-related skills, these possibilities were ignored for the scope of this project.
The intention of evaluation and design needs to be clarified as well: the methods did not attempt to provide evidence for the language-pedagogical merit of the discussed learning functions, but rather aimed to understand the actual or perceived usability and user experience of the functions.
Finally, this degree project did not produce an implemented interactive application, but halted the user-centered design process with the design and evaluation of an interactive, high-fidelity mock-up. This imposes restrictions on evaluation findings of that design.
6.7 Summary
In this introduction it has been established that Språkplay is a new kind of MALL app, focused on L2 video context rather than decontextualized exercises. This approach seems to be widely appreciated among its users. Moreover, literature suggests the value of learning with glossed captions for increasing vocabulary acquisition. However, some usability and user experience issues remain in the application, as evidenced by user reports. This may stem from a limited understanding of user preferences and goals in the context of the app. In response, this thesis seeks to evaluate the application’s learning functions in terms of usability and user experience, and then to resolve these issues in design. The evaluation will be carried out in a user-centred way with students and
researchers learning Swedish in higher education, a demographic in need of Swedish skills to increase their job opportunities.
6.8 Structure of the thesis
Chapter 7 explains the features of the evaluated application.
Chapter 8 presents background knowledge from relevant literature and a review of state-of-the-art digital learning technology.
Chapter 9discusses background theory to be used in the methods.
Chapter 10 describes the methods used for data collection and data analysis in the user-centred design process.
Chapter 11 presents the intermediary evaluation results and design of the improved prototype.
Chapter 12 presents the results of the final prototype evaluation
Chapter 13 discusses the results and their connection to the research questions and their validity
Chapter 14 concludes the degree project
Chapter 15 lists the references used in this work Chapter 16 and next are appendices.
9
7 Description of the examined application
This overview describes the learning functions of the original version of the examined application, SVT Språkplay, as of April 202o. It is presented here as an understanding of these functions is helpful to grasp references to them in the following thesis chapters.
The presented screenshots are taken from the application running on an Android smartphone. However, the application is also available with like functionality and very similar layouts on Android tablets and iOS-driven devices.
7.1 Video watching functions
7.1.1 Control functions (pause, jump forward, jump backward)
Figure 1: video watching functions
The application has a video control interface (figure 1) resembling common video applications such as YouTube, looking at the pause button, with a few important differences.
The pausing behavior is similar to other video players, as is the bottom scrub-bar. Yet, in contrast to other video-watching applications, the app has more features related to captioning. In portrait mode, captions roll from bottom to the top as the video plays, and the previous two subtitles are shown together with the current one and the next two. It is possible to quick-scrub (jump) through the video by either clicking the left-arrow or right-arrow icons, or tapping the next or previous captions. In landscape mode, only the current caption is shown.
Next to the play/pause button, there is an “auto-translate” toggle button on the left, which is described further in the next section. On the right lies the “auto-pause” toggle, which will automatically pause the video at the start of every new caption.
7.1.2 The translation pop-up
Figure 2: video watching functions
When tapping a word, its translation is shown (see figure 2) together with the Swedish definition and links to more resources with information about the word, such as Google 7 Translate or the Folkets Lexikon dictionary. 8 9
In the top bar the canonical form of the word is shown (e.g. infinitive form for a verb), together with the CEFR level of the word (see next). On the right, a star icon affords a way to favorite the word (see later), play back a voice to hear the correct pronunciation of the word and to close the dialog. Closing the dialog will resume the video.
The pop-up can be opened in the following locations where words appear:
1. In video captions (here)
2. In word test transcripts (see Active learning functions)
3. In the zoomed detail view of the “visual vocabulary” visualization (see Self-monitoring functions)
4. In the word list (see Active learning functions) Tapping the bar will change the color of the word (see next).
7 For certain words Språkplay does not have a translation in the app, e.g. neologisms or slang words.
These may be more quickly found in external resources that are frequently updated.
8https://translate.google.com/ - a web app from Google for translation of words and full sentences between any two languages.
9http://folkets-lexikon.csc.kth.se/folkets/ - a community-built, creative-commons licensed Swedish Swedish-English, English-Swedish dictionary
11
7.1.3 The color system
Figure 3: video watching functions
At the core of the application is a system that gives a learning status to each Swedish word known to the application. This status ideally should reflect the degree of knowledge that the user has of the word. There are three statuses relevant to a user: a word can either be known (green), it can be unknown and of interest to be learned (yellow), or it can be
unknown and not of immediate interest to be learned (red). Initial statuses are based on an assumption and may not reflect the users’ actual vocabulary knowledge. By changing word colors, the user may adapt the system to their situation (see more below).
The color system appears in many functions of the application:
● as an underline in captions (see the previous section)
● as a status in a translation pop-up (see figure 3)
● as an underline in the transcript displayed in the word test, when no test is active (see Active learning functions)
● as base for statistics such as “words learned” in the profile page (see Self-monitoring functions)
One more way in which the color can change is when the user completes a video word test (see later). After correctly filling in a word to be learned there (yellow), the word gains a dot underneath. At the third dot, the yellow color is automatically switched to green.
Initial color determination
When a user first starts the app, she is required to choose a CEFR level that fits her current language knowledge. The initial status of words will be adjusted based on that level, i.e. it is based on the assumption that the user wants to focus on learning words on her current level now and leave more advanced words for later. For example,e.g.. common verbs such
as “att äta” (to eat) would be associated with an A1 level, and would be marked as green when the user selects that they are at the A2 level. Next, the app would set words from current level (A2) to yellow, and words from a later level (B1-C2) to red. This initial mapping is, importantly, an approximation of the vocabulary knowledge typically associated with that CEFR level. It will not reflect the user’s real vocabulary knowledge.
Changing word color
The color of a word (see next) can be changed by the user to better reflect the user’s knowledge. This is mainly achieved by tapping the bar of the translation pop-up after selecting the word somewhere (see previous).
One peculiarity is the change to yellow: this change also marks (stars) the word as
“favorite”, which adds it to the word list (see later). The star can be removed manually if wished. The learning status assigned to words for a particular user is embedded in the user’s Learner Profile and shared across the applications of Språkkraft.
7.2 Active learning functions
7.2.1 Word test
Figure 4: video watching functions
In the word test (see figure 4) the user can practice the unfamiliar words she encountered in a video.
The word test is a fill-in-the-gap test with translation hints. The whole video transcript serves as a text base, and the yellow (learn now) words are blanked out in multiples of about 5 (units). Words should be dragged from the word bank and filled in to the correct
13
gap. A user can choose the unit until where the test should be done. The word test is started from the pause menu of a video.
7.2.2 Word list
Figure 5: video watching functions
The word list (see figure 5) gathers words that were favorited (starred) with the translation pop-up for reference. It can be configured by sorting the words by date or learning status (green, yellow). Next, the words can be filtered by the video from which they came.
The “spaced repetition” functionality will sort the words according to the spaced repetition algorithm, in which older and less difficult words put at the bottom in order to exploit the psychological spacing effect, that states that knowledge is retained better when repeating it in spaced-out sessions [43]. Words list items can be tapped to show the sentence
wherefrom they were marked as favorite, and it is possible to open the video containing that sentence, or to delete the word.
7.3 Self-monitoring functions
Figure 6: Learning profile (left), visual vocabulary for level B1 (middle), activity page (right)
7.3.1 Profile page
The learning profile page (see figure 6, left) summarizes some app activity statistics.
“Points” denote the number of correct word fill-ins in the word tests. Next, accomplished achievements are also displayed here, and an overview of the status of all words known to the app (their CEFR level is first given by the users, then individual words can be changed).
In the visual vocabulary detail view (see figure 6, middle) each word associated with a certain CEFR level (e.g. B1) is represented with a rectangle. The user can zoom to see the words.
7.3.2 Activity statistics dashboard
The Activity page (see figure 6, right) displays certain activity statistics over time: the words read (1), words learned (2) and time spent in the app (3). It then offers a possibility to set daily goals for each of these statistics. Achieving one goal will increase the count of a
“streak”, which is lost (reset to zero) again if the user skips one day of using the app.
15
8 Background
This background section expands on several concepts mentioned in the introduction: video and language learning, mobile assisted language learning, gamification, usability, user experience and user centred design. All these are research areas with an important relation to the examined application or this thesis.
8.1 Video and language learning
8.1.1 Media and language learning
One way in which language can be acquired is through extensive media consumption, an activity that has been intertwined with language learning for centuries. Consuming second language media can be seen as a goal of learning that language, but also as an aid in learning. Consider for example how 19th century scholars have relearned to read Egyptian hieroglyphs by studying a multilingual decree carved on a stone, the Rosetta Stone [40].
More recently, Stephen Krashen developed a hypothesis relating the consumption of foreign-language media to the acquisition of vocabulary and spelling. His Input Hypothesis
“assumes that we acquire language by understanding messages. More precisely,
comprehensible input is the essential environmental ingredient” [20 p. 1]. A message may be transmitted by any medium, yet Krashen studied mostly textual media. Since the rise of audiovisual media however, messages have been extensively transmitted through media that is not primarily textual. This also sparked research into language learning with these new media.
8.1.2 Learning with Closed Captions
Particularly relevant for this thesis is research on language learning conducted with video-based media. Today, people may binge-watch a series on Netflix and follow their favorite vlogger on YouTube. Video content such as TV programmes, internet videos and films are available in enormous volumes at the tap of a finger, also in the examined application. This is material brimming with spoken language, and often also captioned with text. Vanderplank started researching language learning with video in the late 80s and is still active in the domain today. In the recent Handbook of Informal Learning [8], he recalls that “TV, film, and video were seen by second language acquisition (SLA) researchers as an excellent potential source of comprehensible input, since they fulfill Krashen's idea of exposure to natural language”. Indeed, he further discusses that more exposure to foreign-language video content is correlated with better L2 (i.e., second language learning) comprehension skills [8].
But not all exposure to video is equally helpful for learning. One important aspect is the presence and language of captions: “viewing captioned video material compared to uncaptioned or subtitled material increases comprehension and promotes the learning of vocabulary” [8 p. 341]. In this quote, captions translated to a known language are referred to as (L1) “subtitles”, while untranslated second-language (L2) captions are simply named
“captions”. Moreover, Vanderplank specified the most desirable type of captions by concluding that “full captions come out very well against keyword captions” [37 p. 116].
Full captions intend to render what is said in a video in full sentences, or if possible, verbatim. Keyword captions omit commonly occuring words such as prepositions and articles with the intention of lowering the cognitive reading load for learners [37 p.106].
8.1.3 Control and “glossed” captions
The positive results from learning with closed-captions should be nuanced by pointing to a dependency on the mode of learning. Vanderplank sheds light on this when he discusses a study on the benefits, affordances and downsides for individuals watching a range of foreign language DVDs (the EURECAP project [37]): “evidence from diaries kept by participants showed quite clearly that only if participants made full use of their control of viewing in terms of stopping, rewinding, and checking words and phrases that were difficult to follow or were unknown did they have any chance of taking in these words and phrases.” [8 p. 343].
Furthermore, recent research [27] has compared a form of captioning that integrates this
“checking of words” to other forms of captioning, namely full captions, keyword captions and no captions. The “glossed keyword captions” they used are defined as “keyword captions with access to meaning: each keyword is linked to its corresponding L1
context-bound translation” [26, p.8]. L1 is the first or known language of the learner, and the learner could see the L1 meaning of an L2 word by pausing the video (pressing the spacebar). The study found that glossed keyword captions clearly outperformed the other forms of captioning in terms of vocabulary acquisition (as measured by meaning recall).
This is a noteworthy result for this thesis, as the captioning of Språkplay can be seen as glossed full captions, where a meaning is shown by tapping a word in the captions. While the merits of glossed full captions are open for study, it seems that the combined reports of Montero Perez et al. (glossed keyword captions are best) and Vanderplank (full captions are preferred over keyword captions) suggest their value.
8.1.4 Implications for a video-based MALL application
Implications for the project at hand are twofold. On one hand, the previous section predicts the viability of the examined application: the concept of learning language from captioned video (with gloss) makes sense. Even though most related studies are based primarily on other technologies such as teletext TV, captioned DVDs and computer media players, transplanting this concept onto a mobile device should not be different in essence. On the other hand, this literature points to desirable technological affordances of the
video-playing environment. It is stressed that learner control functions, such as pausing, stopping and rewinding video, should be available
8.2 Mobile assisted language learning (MALL)
8.2.1 Defining MALL
Mobile assisted language learning (MALL) is a term that originated in the late 2000’s to denote language learning via mobile interactive or audiovisual technologies, which included devices such as handheld computers, tablet pc’s and mp3-players [23]. When
17
smartphones superseded these technologies in the last decade, many MALL studies have since focused on the use of mobile applications on smartphones (apps) for language learning [11,38]. Kukulska-Hulme (the author of a seminal MALL paper, [23]), defines MALL as “the use of smartphones and other mobile technologies in language learning, especially in situations where portability and situated learning offer specific advantages”
[22 p. 1]. She further mentions some specific advantages, of which the following are pertinent to the studied case listed: flexible use of time and space for learning, good alignment with personal needs and preferences, the possibility of keeping learning efforts ongoing while waiting or commuting. MALL may “support learners in reading, listening, speaking and writing in the target language, either individually or in collaboration with one another” [22 p. 1].
MALL has generally been seen as a sub-field of mobile learning (mLearning) [38] which was defined by Sharples et al. as a “process of coming to know through conversations across multiple contexts among people and personal interactive technologies” [39], as cited by Viberg & Grönlund [20 p. 1]. Kukulska-Hulme further characterizes mLearning as being ubiquitous and contextual, noting that the mobility of the learner is increasingly
highlighted over the mobility of the technology [22]. From these definitions it follows that the examined application can be situated in MALL. It specifically addresses reading and listening capabilities as mentioned by Kukulska-Hulme, but also combines this with a viewing aspect through the captioned video. The application may be used in mobile contexts such as commuting, and focuses on individual use.
8.2.2 MALL with immigrants
A number of studies have examined the uses of MALL for immigrant language learners specifically. Jones et al. [18] suggest that the examined MALL application helps
immigrants with acquiring relevant language learning skills and practice of these skills.
One other conclusion from this study is that MALL as practiced by immigrants was not performed “anywhere, anytime”, but situated with more specificity. The application they studied “was used when appropriate and convenient, i.e., when participants had sufficient time, reliable internet connection and a suitable location, such as at home. The concept of anywhere anytime learning idea is not always borne out in practice.” [18 p. 249]. A home was seen as a good location since “It allows learners a private space in which to practice and make mistakes and appears to address some known challenges, such as pronouncing words ‘correctly’, understanding local accents and understanding and using colloquial language which includes contractions. Such support and practice was motivating and improved confidence.” This highlights the benefits that MALL could have over learning in non-private locations, such as a typical classroom.
8.2.3 Video-based learning in MALL
A few studies have specifically investigated the opportunities that lie at the intersection of video-based learning and MALL. The section 6.2 Problem section already reported the results of two different studies ([13,14]), namely an increased vocabulary comprehension where (adaptive) captioned video was used on mobile devices, compared to non-captioned video. The first of the two studies motivated the integration of captioned video with a
MALL application a priori by arguing that, (1) learners nowadays spend more time on mobile devices than on PCs, and that (2) MALL has the flexibility to support in-class and out-of-class learning. However, there seem to be no studies comparing a video-based MALL approach to video-based learning with PCs, or to a different vocabulary learning method entirely (e.g., the reading of texts on mobile devices/PCs/books). Thus, while the individual evidence from video-based learning (see before) and MALL seems promising, more empirical research is required to ground the field of video-based MALL.
8.3 The role of gamification
The most popular mobile language learning application of today, Duolingo, is used by about 300 million people [41]. Its popularity has been partly attributed to “gamification”
[11,25,32,35], which is why this section describes that domain and its relation to MALL in more detail.
8.3.1 Defining gamification
In studies of MALL, gamification has surfaced as a desirable characteristic along with e.g.
flexibility [25]. But what is it really? Gamification can be defined as “the use of game design elements in non-game contexts” [7 p. 10]. The same paper further specifies it as a design practice that aims to incorporate “game elements”, rather than producing full-fledged games. The paper categorizes game elements into five levels of abstraction, where examples of the most concrete elements are “reward and reputation systems with points, badges, levels and leaderboards” (p. 1). More abstract elements include the consideration of “time constraints” and “limited resources”, or even processes to develop games such as
“playtesting” and “playcentric design”.
8.3.2 Game-based learning and gamification within MALL
In the domain of (language) learning technology, the use of game elements has been discussed with the term digital game-based learning (DGBL), of which mobile game-based learning (MGBL) is a subdomain [9]. This body of research has links to mLearning, MALL and is thus of relevance to this thesis. The difference of this concept with “gamification” as defined before may be that (M)GBL starts from a (mobile) game and looks at how learning is or can be achieved through that game. Gamification on the other hand, starts from a traditionally non-game context (e.g. language learning) and implements game elements in that context in order to increase outcomes such as the users’ motivation to learn [1], that is, it gamifies a non-game application.
In practice, this conceptual distinction is demonstrated in a study on the effects of using
“gamified features” in a “mobile game-based English vocabulary learning app” [4]. In this experimental study, two versions of the same app have been compared. One version has gamified elements such as leaderboards and minigames as a means of vocabulary assessment. It was thus called a “mobile game-based app”. In the other version, these elements were omitted. It did thus not qualify as a game-based app.
19
8.3.3 The value of gamification for vocabulary learning
The study above has found that specifically for (English) vocabulary learning in a mobile app, the use of gamified features has a strong positive correlation with both the acquisition and retention of new vocabulary [4]. In particular, the following were found to be the gamified elements that contributed most to this result: (1) a pre-established learning path that visualizes progression, (2) gamified assessment methods, e.g. a tic-tac-toe game where the learner can only move when she/he provides a correct answer to a vocabulary
(translation) exercise, (3) a ranking of learning peers (leaderboard), an element of social competition. Moreover, the study found that users spent more time in the app with gamified features, than the version without. Another study found that gamification elements such as progression tracking with experience points (XP) could be used to adapt the interface to the learners’ needs, e.g. when the learner wrongly guesses a word, it could be made to appear more in the app [13]. Similarly, a study on Duolingo [25] praised its XP leaderboards as a social gamification aspect. The presence of this element had a positive correlation with app usage, which was itself correlated with greater language mastery.
However, it was also noted that the feature “may have led some participants to focus less on language learning and more on reaching the top of the XP leaderboard” [25 p. 305].
8.3.4 Implications for this project
Since the aim of this project is to improve the usability and UX of the examined app with the ultimate goal of increasing vocabulary acquisition for its users, attention should be given to game design elements such as presented in the last sections. It is clear that these elements are connected to improved vocabulary acquisition on their own merit. In the next background section, they are also related to an improved UX.
8.4 Usability, User Experience (UX) and User-centered design (UCD) 8.4.1 Why and Usability and UX?
ISO 9241-210 states that good usability and UX are determinants of the technical and commercial success of any interactive application [17]. Therefore, gaining insights on how good usability and UX can be achieved in the domain of MALL is important. The ISO defines usability as the “extent to which a system, product or service can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use” [17 p. 17]. User experience is defined as a “person's perceptions and responses resulting from the use and/or anticipated use of a product, system or service” [ibid.]. Furthermore, the ISO specifies that good usability and UX are achieved by a user-centred design process (UCD). This is an iterative process of understanding the context of use, specifying user requirements, producing design solutions to meet those requirements and evaluating the designs against the requirements. As a result, all these steps would ideally also apply to the design of MALL applications.
When employing the concepts of usability and user experience in a UCD process, it is necessary to note that they are intertwined. The ISO specification notes that “usability, when interpreted from the perspective of the users' personal goals, can include the kind of
perceptual and emotional aspects typically associated with user experience. Usability criteria can be used to assess aspects of user experience.” [ibid.]. Ritter et al. further clarify the difference: “usability and usability engineering focus on task related aspects (getting the job done); user experience and experience design focus on and foreground the users’
feelings, emotions, values, and their immediate and delayed responses.” [29 p. 44]. Given this complementary yet overlapping connection, this work aims to evaluate both aspects simultaneously.
8.4.2 Usability and User Experience in MALL
Few research efforts have been focused on the examination of “user experience” (UX) or
“usability” within the MALL domain. Limited exceptions were found, starting with a 2016 blog article that puts forward gamification as a solution to UX challenges, taking Duolingo as a successful example [35]. Sendurur et al. in 2017 [32] discuss UX of Duolingo in more detail, finding for example that high usability, a step-by-step flow, gamification and immediate feedback positively affect the UX, while a lack of challenge and overuse of notifications can be detrimental. Finally, Triando and Arhippainen released a
design-focused study 2019 on the UX of a self-developed “mobile web game” to learn Viena Karelian, a Finnic language dialect, taking inspiration from Duolingo. This thesis aims to widen the limited insights on usability and UX in MALL, focussing on a video-based mobile learning app as opposed to Duolingo or a web-based solution.
8.5 Summary
The application Språkplay can be viewed from different angles covered in literature. From the angle of video-based learning, the glossed full captions and video control functions available in the app are promising for vocabulary acquisition. From the angle of MALL, the aspect of flexible use has been praised, but the context-of-use for immigrant learners has been found to be more localized than what would be expected from the tenet "anywhere, anytime" . Next, gamified design elements have been a part of many MALL applications and are cited as reasons for their success. Elements conveying progression and social competitive features were discussed most. Finally the concepts of usability and user
experience were brought forward as important factors in the success of an application, and the lack of studies relating to these factors in video-based MALL was noted.
The next chapter introduces two theoretical frameworks before discussing the methodology of this work in more detail.
21
9 Theory
This chapter briefly introduces two frameworks that were instrumental to the setup of this thesis. First, the Double Diamond is presented, a framework that implements the UCD process seen in the previous chapter. Next, the MUUX-E framework is explained, a framework to evaluate the usability, user experience and educational features of m-learning environments.
9.1 The Double Diamond framework
Figure 7: A visualization of the Double Diamond design process, adapted from Design Council UK [6]
The user-centred design process as seen in section 8.4 has been implemented in various ways by parties across the globe [3]. One interpretation is the Double Diamond framework defined by Design Council, an advising organisation to the UK government. The framework segments the user-centred design process in four phases: Discover, Define, Develop and Deliver [6], (see also figure 6). Importantly, Discover and Develop contain expanding methods for researching the problem domain (Discover) and designing a chosen solution (Develop). Define and Deliver respectively narrow down the scope of the problem domain (Define) and implementation paths (Deliver), for example by evaluation methods. In the context of this project it is important to note that the Discover phase mostly gains
information about an existing application, while the Double Diamond is generally used to design new solutions from scratch. Nevertheless, any user-centred design process expects iteration, which means that the process can also be applied to improve an existing app. In
