Linköpings universitet
Linköping University | Department of Computer and Information Science
Bachelor’s thesis, 16 ECTS | Computer Science
2020 | LIU-IDA/LITH-EX-G--20/071--SE
Development of a
Pick-Up-And-Play Gamification Geography
Game for Learnability
Utveckling av ett pick-up-and-play gamification geografispel för
lärbarhet
Ermin Pitarevic
Supervisor : Anders Fröberg Examiner : Erik Berglund
Upphovsrätt
Detta dokument hålls tillgängligt på Internet - eller dess framtida ersättare - under 25 år från publicer-ingsdatum under förutsättning att inga extraordinära omständigheter uppstår.
Tillgång till dokumentet innebär tillstånd för var och en att läsa, ladda ner, skriva ut enstaka ko-pior för enskilt bruk och att använda det oförändrat för ickekommersiell forskning och för undervis-ning. Överföring av upphovsrätten vid en senare tidpunkt kan inte upphäva detta tillstånd. All annan användning av dokumentet kräver upphovsmannens medgivande. För att garantera äktheten, säker-heten och tillgängligsäker-heten finns lösningar av teknisk och administrativ art.
Upphovsmannens ideella rätt innefattar rätt att bli nämnd som upphovsman i den omfattning som god sed kräver vid användning av dokumentet på ovan beskrivna sätt samt skydd mot att dokumentet ändras eller presenteras i sådan form eller i sådant sammanhang som är kränkande för upphovsman-nens litterära eller konstnärliga anseende eller egenart.
För ytterligare information om Linköping University Electronic Press se förlagets hemsida http://www.ep.liu.se/.
Copyright
The publishers will keep this document online on the Internet - or its possible replacement - for a period of 25 years starting from the date of publication barring exceptional circumstances.
The online availability of the document implies permanent permission for anyone to read, to down-load, or to print out single copies for his/hers own use and to use it unchanged for non-commercial research and educational purpose. Subsequent transfers of copyright cannot revoke this permission. All other uses of the document are conditional upon the consent of the copyright owner. The publisher has taken technical and administrative measures to assure authenticity, security and accessibility.
According to intellectual property law the author has the right to be mentioned when his/her work is accessed as described above and to be protected against infringement.
For additional information about the Linköping University Electronic Press and its procedures for publication and for assurance of document integrity, please refer to its www home page: http://www.ep.liu.se/.
Abstract
Gamification, immersion and flow are well known within game design, and more fre-quently considered within the development of educational applications. The simplicity and comprehensibility of an educational game can result into improved learning and a more enjoyable experience, as the focus on the gameplay increases with less instructions and a more straightforward approach for playing and learning.
The aim is to focus on the relevant areas in order to implement a pick-up-and-play gamification geography game for learnability, as well as understanding the considerations when developing an educational application of this kind. Moreover, the work mainly focuses on the implementation of relevant and central game design elements, including animations, correlated to the researched areas. Given the information and the implemen-tation, evaluation and analyzation were performed using Think-Aloud protocol combined with two scripted tests. Additionally, two slightly modified questionnaires were used, the Game Experience Questionnaire and User Engagement Scale Short Form, as a completion for any uncertainties or missing information after the Think-Aloud protocol.
The work has brought up the importance of understanding the correlation between the relevant areas and the game design elements, as well as discussed these in context of this work. Moreover, it describes the importance of the well-balanced usage of animations, which improves the dynamic and engaging experience, while also contributing to a clearer feedback. The Think-Aloud protocol have shown being very useful and effective for the evaluation and analyzation. However, the questionnaires were only useful to some extent, but overall ambiguous and confusing, mainly because of the short playtime by the participants, as well as the difficulties in interpreting the assumptions.
Acknowledgments
I would like to thank Erik Berglund and Anders Fröberg for patiently helping me throughout the work. Great guidance, useful advices and encouraging words that I appreciate. Also, a special thank you to the ones who have supported me and continuously do, you know who you are.
Contents
Abstract iii
Acknowledgments iv
Contents v
List of Figures vii
List of Tables viii
1 Introduction 1 1.1 Motivation . . . 1 1.2 Aim . . . 3 1.3 Research Questions . . . 3 1.4 Delimitations . . . 3 2 Theory 4 2.1 Gamification . . . 4
2.2 Gamification within Educational Context . . . 4
2.3 Immersion and Engagement . . . 5
2.4 Flow . . . 5
2.5 Animations in Educational Applications . . . 6
2.6 Pick-Up-And-Play . . . 6
2.7 Related Work . . . 6
3 Method 7 3.1 Technical Description of the Platform . . . 7
3.2 Development Process . . . 7
3.3 Testing and Evaluation . . . 10
4 Results 16 4.1 Implementation . . . 16
4.2 Think-Aloud Protocol and Scripted Tests . . . 21
4.3 Game Experience Questionnaire . . . 22
4.4 User Engagement Scale Short Form . . . 23
4.5 Categories Correlated to the Focus . . . 23
5 Discussion 24 5.1 Results . . . 24 5.2 Method . . . 27 5.3 Source Criticism . . . 28 6 Conclusion 29 6.1 Future Work . . . 30
List of Figures
4.1 Level selection screen, with level markers dropping down and other animations operating. . . 17 4.2 Feedback showing the incorrect and pressed marker, transitioning over to the
cor-rect marker. . . 17 4.3 Different themes, with the colors for the lives and score being adapted accordingly. 18 4.4 Progress bar, with additional information about the played level. . . 19
List of Tables
3.1 Overview of participants . . . 10 3.2 Scripted test, mainly focusing on comprehensibility, feedback and learnability . . 11 3.3 Scripted test focusing on learnability and gameplay . . . 11 3.4 Slightly modified core module of the Game Experience Questionnaire . . . 13 3.5 Scoring guidelines for the core module . . . 13 3.6 Slightly modified post-game module of the Game Experience Questionnaire . . . . 14 3.7 Score guidelines for post-game module . . . 14 3.8 Factors in the User Engagement Scale Short Form . . . 14 3.9 User Engagement Scale Short Form . . . 15 4.1 The two tasks being confusing for a few participants during the scripted test . . . 21 4.2 Mean score for the core module . . . 22 4.3 Mean score for the post-game module . . . 22 4.4 Mean score for the UES-SF . . . 23
1
Introduction
Mobile applications play a significant role in the smartphone functionality. The educational mobile applications have become more popular by the years, mainly due the portability and creativeness it brings to teaching and learning. Furthermore, games have shown being ef-fective learning tools, with continuous noticeable improvements over traditional methods, partly because of the responsiveness, speed and the constant usage of smart devices. [7, 18] Designing these educational applications is challenging, and there are multiple factors to de-rive from.
1.1
Motivation
Educational games, also called serious games, aims at influencing the behaviour of the player and improving their knowledge, rather than focusing on entertaining the player as in tradi-tional games. [5] Games are known for attaining the attention of players, which formed the idea of taking inspiration from games in order to improve education. This brings different challenges when designing educational games, such as understanding the users’ interaction with the game as well as the how the learning process should be developed. [11] Moreover, ensuring that interactions with the game is developmentally beneficial, is another challenge [23].
There are several ways of designing an educational game, and multiple factors must be taken into consideration. Gamification, the usage of game design elements in context other than entertainment, has shown increasing users’ performance and motivation [5]. There are different game design elements which can be applied to a game, such as points, badges and challenges. However, the area is complex and requires understanding of the userbase, context and having clear goals to properly apply these elements and achieve better game experience while increasing the learnability [13].
1.1. Motivation
Satisfying the intrinsic and extrinsic needs, is another important task for the game designer where the focus is on helping the user feel motivated and understand the meaning of the game. Using rewards and virtual achievements can contribute to the feeling of ownership, while applying risks of losing these rewards might change how the player value these. Moreover, making the player feel freedom of actions, self-expression and unpredictability can result into increased intrinsic motivation. [25]
Having achieved the satisfaction of the intrinsic and extrinsic needs, and by obtaining a good understanding of the mentioned factors while applying the elements correctly, a player might feel more engaged and immersed. The usage of good animations and other multime-dia assets are other key factors to increase the effectiveness of the applied game elements. Immersion refers to the time and effort a player is investing into the application, as well as how emotions and feelings are affected by it, which can be a measure of how interested and engaged the user is in the game.
A commonly used term referring to the feeling of optimal experience and enjoyment is flow. The concept has been adapted by many and different models have been proposed, but are in general very similar. One of these proposed models, GameFlow, consists of eight explained elements with the purpose to achieve enjoyment in games [24].
The simplicity and comprehensibility can lead to a more self-explanatory gameplay, with the goal of having the players focusing on the learning, rather than focusing on the instructions. Having less instructions leads in that way to a less interrupted player and to some extent increases the flow and learnability. [3]
There is a lot to take into consideration when designing an educational game and requires good planning and understanding of what is to be achieved. It is of interest to combine these areas to create a gamification learning game, based on learnability with a self-explanatory gameplay, in order to also achieve an engaging and motivational experience, as this can contribute to increased learning.
1.2. Aim
1.2
Aim
It has been shown that gamification, immersion and flow are well known within game de-sign, and more frequently considered within the development of educational applications. The purpose of this work is to mainly focus on these areas to implement a pick-up-and-play gamification geography game for learnability. Although the work focuses on implementing a geography game, the genre is not main focus and the work could be applied for different genres.
More specifically, the work should first further investigate these areas to understand their correlation and objective, as well as which game design elements are of relevance. Further-more, with this knowledge the game will be implemented to achieve the results strived for. Finally, an evaluation will be performed in order to analyze the implementation based on the relevant areas. The evaluation will be performed using a Think-Aloud protocol combined with scripted tests, finishing off with two questionnaires measuring experience and engage-ment. A discussion and conclusion are to be made based on the applied method and the given results.
As the aim is to implement an easily understandable geography game for learnability, the work should focus on students where this area is most common, without focusing on a specific age group. However, this could be broaden for a wider target group where the interest, or need, for geography learning or entertainment is present.
1.3
Research Questions
The following research questions will be processed in this thesis:
1. Which considerations should be taken into action when designing and developing an educational pick-up-and-play gamification geography game for learnability?
2. Which tools can be used for evaluation and analyzation of the implementation, based on the relevant areas? How useful are these in this work?
1.4
Delimitations
In order to do the research within its scope and given timeframe, it has been limited to fo-cus on the most central and relevant game design elements, but also because of the lack of research for other non-central game design elements. The evaluation and analyzation will be based on the application’s state at the end of the given timeframe, which will not be in its final state. Using react-native-maps is restricting the possibilities of using some elements to their full potential, such as animations.
2
Theory
This chapter begins by presenting a more detailed description of some theoretical and rele-vant areas, in order to give the reader a better understanding of the method and the result. Related works are then presented directly after these descriptions.
2.1
Gamification
The term gamification has no strict definition. However, Deterding et. al. [5] proposed a very good working definition of this term, where they described gamification as the usage of game design elements in non-game contexts. With the assertion of entertainment being the commonly expected usage of games, the gamification uses games for different purposes, such as engagement and motivation. Neither are there any clear advantages of limiting the term to any specific usage or purpose, since it can be applied broadly.
2.2
Gamification within Educational Context
There are many variables determining which type of game is the most suitable for the current learning content in question, making the assessment of the effectiveness of an educational game difficult. The purposes of applying gamification in education has mainly been improv-ing user engagement and experience, but the potential of it beimprov-ing used as a pedagogical tool for various purposes is inevitable.
It is important to have clear learning goals for the gamification project and thoroughly understand the userbase, as well as matching these goals together with suitable game design elements and mechanics. Understanding all these concepts and how to bond these together is challenging but valuable within the educational context, and this is where the creativity and imagination of the game designer comes into play. [1]
2.3. Immersion and Engagement
2.3
Immersion and Engagement
Immersion refers to the intensity of involvement within a game. There are different stages of immersion, with different barriers and where the players’ involvement is affected differ-ently. Entering first stage of immersion, engagement, the player must invest time, effort and attention before being able to move deeper into the levels of involvement and increase the immersive experiences. Moving deeper into the level of involvement the emotions, feelings and engagement becomes stronger.[2]
Appropriate combinations of game elements, associated with involvement and immer-sion, might lead to the players emotions being directly affected by the game. The feeling of developing expertise can, for example, be achieved by having clear controls and feedback corresponding appropriately. Likewise, expected rewards contributes to more effort invested into the game and triggers feelings.[2]
Marshall G Jones [9] defined engagement as the relationship between the intrinsic knowl-edge and the external motivation, which encourages the initial interest and continued use of a learning environment.
Building a truly engaging environment can be challenging. The environment’s motiva-tional features do have their impact on the learner’s interest. However, this applies in case of the learner not being intrinsically motivated, since the motivation derives from within the learner. Moreover, learners’ interest in the content or well-defined questions are reasons of why they choose to work in learning environments. Intrinsically motivated users can find the need of mastering the content, resulting into them spending hours learning in order to achieve that. Additionally, the usage of good animations and other multimedia assets are a key factor when designing and developing educational software, as they keep the users more interested. [9]
2.4
Flow
A commonly referred flow theory provided by Csikszentmihalyi [4], described flow as the feeling of optimal experience and is a model of enjoyment. Sweetser and Wyeth [24] adapted this concept and proposed an own concise model, GameFlow, with the purpose of helping game designers understand, design and evaluate enjoyment in games. GameFlow consists of eight elements with each including a set of criteria for achieving enjoyment in games. The player should be able to concentrate on the game. It should be challenging for the player and match the skill of the player, meanwhile skill development and mastery should be supported. Further on, players should feel control over their actions in the game, receive appropriate feedback and the game should provide clear goals. Playing the game, a player should experience immersion, meaning deep but effortless involvement. [24]
2.5. Animations in Educational Applications
2.5
Animations in Educational Applications
In some scenarios, animated graphic does not always seem to be more effective compared to static graphic. The usage of animations for learning is believed to bring more interest and is more visually appealing, which might increase the motivation. [10] Furthermore, animations used in applications have been appreciated, while appearing to improve learnability and user experience [17]. However, animations must be incorporated carefully and accordingly with consideration of the context, since they could cause a distraction for the learner and therefore affect the learnability. Another consideration is the performance impact animations have on the applications, which in turn affect the user experience and motivation. [10, 17]
2.6
Pick-Up-And-Play
The term pick-up-and-play is sometimes used in correlation with games to describe a game that is easy to understand in a short period of time. As the name suggest, a first-time player should be able to pick up the game and play it instantly. Comprehensibility, clarity and the learning curve are some of the main factors which affect whether a game is within the category of pick-up-and-play. Moreover, how self-explainatory and straightforward the gameplay is, how clear goals and how the usage of game design elements are some other factors to take into consideration.
2.7
Related Work
Based on empirical and expert evaluation, Morschheuser. et. al [13, 12] advanced the un-derstanding of designing gamification. The common game design mechanics and elements, including leaderboards, points, badges and challenges within educational environments do enrich motivation experiences and increase performance. However, the gameful experience is complex and does not only rely on these mechanics but requires, among other things, analysis of context and users. To achieve continues challenging, diverse experience and a fully immersed user, easier challenges are preferred. The satisfaction of the user’s instinct needs, such as mastering something, is a reason for invocation of motivational and influenced behavior. Clear objectives are essential as well as testing the gamification ideas frequently, analyzing them and evaluating the result. An analyzation of the context might help the understanding of possible target groups, their needs and design limitations. The authors highlight the importance of creative ideation and design phases, since there is a tendency of lacking detailed description of this important phase.
Tondello et. al [25] presented multiple gameful design heuristics aiming at helping eval-uation of a gameful system as well as identifying design gaps. The intrinsic motivation heuristics focus on affordances aimed at helping the users understand the meaningful goal with the system and satisfy their intrinsic needs of competence using challenges, goals or virtual achievements. The aesthetic experience can be improved by immersing the users into the system, by having a theme or a story they can be a part of or relate to. Intrinsic needs can also be satisfied through allowing freedom and self-expression within the system. Furthermore, motivating users through ownership and rewards can give the users a reason to interact with the system, which is a part of the extrinsic motivation heuristic. Including scarcity and loss avoidance can increase the motivation due to the change of how the users might value these rewards. A good addition to the mentioned heuristics is unpredictability, which can afford both the intrinsic and extrinsic motivation.
3
Method
This chapter will describe the details of the approach chosen to obtain to answer the research questions. Initially, a qualitative literature study was done in order to get a good under-standing about the focused area, which is covered in the theory chapter. Furthermore, the development process will be described focusing on the implementation.
Several participants were testing the game, where two scripted tests were followed in combination with a Think-Aloud protocol. An experience questionnaire and a short form engagement scale were used in order to get a deeper understanding for relevant categories. The results were analyzed in order to make a discussion, conclusion and answer the research questions.
3.1
Technical Description of the Platform
The development and implementation is not restricted to any programming language or any framework, with the only exception that it has to support some kind of graphics. For this work, the game has been developed using React Native [19] and Redux [22, 20] which is a good option with many benefits. Furthermore, the Expo [6] framework and platform has been used, which is a set of tools and services built around React Native, used to facilitate the development, build, deployment and iteration. To integrate with maps in React Native, react-native-maps [21] was used. It is a module containing React Native Map components for both iOS and Android.
3.2
Development Process
Initially, the base of the game was further worked on in order to have a fully working game with the core mechanism implemented to continue this work on. Furthermore, a qualitative literature study was done to gather a good understanding of gamification, immersion, flow, motivation and engagement, as these are the relevant areas focused on in this work. Based on the theory, we could understand that there is a correlation between the game design elements and the areas we chose to focus on. Additionally, the comprehensibility and a well-balanced implementation have an impact on these areas, as well as on the learnability.
3.2. Development Process
With this understanding, the following points were taken into consideration during the con-tinuous development:
• Main focus should be the learning and learnability
• A straightforward approach, both the navigation and the gameplay
• Overall dynamic and appealing, but non-complex design throughout the game
• Minimizing the amount of components, restricting the addition of only necessary and purposeful components
The above-mentioned points are mainly affected by the implementation of the game design elements in this work. Some other game design decisions have also been made. These do however depend on personal thoughts, opinions and other factors, which are out of context for this work.
3.2.1
Game Design Elements
During the continuous development, with the above-mentioned points considered, the fol-lowing central game design elements were focused on:
• Score • Life • Time bar • Progress bar
• Statistics (currently streak, total points, total time played and amount of completed maps)
• Feedback and animations
Some of the above-mentioned game design elements were a part of the base implementation: score, life and time bar. These were kept because of their relevance and contribution to the focused areas in this work. Moreover, additional changes were made to these, most impor-tantly making them animated and improving their aesthetic appeal. Furthermore, to achieve a more engaging and dynamic experience, all the game design elements were animated. The game design element’s animations are also supposed to contribute to the feedback, while a separate animation was implemented, specifically for the feedback, and is triggered when an incorrect marker is chosen. Two other animations implemented for their contribution to the dynamic and engaging experience were zoom transitioning and theme change.
Having well-balanced animations and appropriate positioning of game design elements and components were also focused on during the implementation, as well as only using purposeful animations that contribute to the focused areas. Testing and adjustments were done during the development to achieve a acceptable implementation for this work.
3.2. Development Process
3.2.2
Levels
There were levels including cities or seas. Each level was implemented with an ascending difficulty following the concepts of flow. The levels either started with the larger, more known cities and seas. If the player chose a level close to the own geographical position, the levels started with more local cities and seas.
In each level there were grouped cities or seas, which were defined as a sub-levels or rounds. These cities and seas were randomized within the sub-levels every time they were played. Completing a sub-level moved the player to the next sub-level, and completing all sub-levels resulted into completing the whole level. The transition between sub-levels were animated, with an addition of theme changes if the player achieves an amount of correct answers in a row.
3.2.3
Testing and Evaluation
However, as others are supposed to play and use the game, it is interesting to understand their experience and thoughts about the game, along with the how well we achieved the im-plementation we strived for. An evaluation was done in order to gather this understanding, based on their experience playing the game, which is further described in the next section.
3.3. Testing and Evaluation
3.3
Testing and Evaluation
This section will describe how the testing and evaluation was done on the implemented game at its current state.
3.3.1
Participants
A total of 65 individuals completed the testing of the game, with the majority of these being students with and without interest in geography. A more detailed overview of the partici-pants can be seen in table 3.1.
Table 3.1: Overview of participants
Amount Percentage (%) Gender - -Male 42 65 Female Experience 23 35 Profession - -Student 61 94 Teacher 2 3 Other 2 3 Completed - -Think-Aloud 65 100 Scripted test 65 100
Game Experience Questionnaire 52 80
User Engagement Scale (Short form) 55 85
Interest in geography -
-Interested 50 77
Not interested 15 23
3.3.2
Think-Aloud Protocol
Think-aloud (TA) protocols are one of the tools primarily used performing usability tests, mainly because it is not directly possible to observe what the user is thinking when testing the system. A user is asked to verbally express the reasoning during a problem-solving task performed on the tested system, where difficulties are of higher interest to perceive. The given information is then used together with relevant metrics in order to formulate improvements for the problematic areas causing the difficulties.[16] The analyzation of the given information was mainly focusing on the implemented components and how the game design elements contributed to the game in terms of the relevant categories. Moreover, it was also focusing on the learnability, comprehensibility of the game as these are correlated to pick-up-and-play, which this game strives to be categorized as.
In this thesis, a common traditional Think-Aloud approach was used, where the only in-teraction with the users was reminding them to keep talking in case of silence. The users testing the game were given multiple tasks to complete while thinking out loud, meaning explaining the thinking while performing these tasks. At the beginning of the testing, the users were given instructions of how the testing would be performed, as following:
You will be given tasks to perform in the game. Try to perform them as good as possible and do not ask any questions. The most important part is that you continuously talk us
3.3. Testing and Evaluation
3.3.3
Scripted Test
In combination with the Think-Aloud protocol two scripted tests were used, which included the given tasks for the participants to follow during the testing. The detailed tasks followed by the participants can be seen in table 3.2. During the testing every relevant and important expressed thought by the participant was documented for the evaluation later, mainly fo-cusing on the mistakes and delays when performing tasks, as these are typically caused by confusion or other difficulties.
The tasks were produced by focusing on covering all the relevant parts of the game, in order to get a good understanding of the implementation within the limited time. Addi-tionally, except for only observing any thoughts expressed about the game design elements, the participants were asked about their thoughts on these as well after finishing the tests. This to acquire a better understanding of how the game design elements contributed to the learnability, comprehensibility, immersion, flow and motivation, and how these could possibility be enhanced.
Table 3.2: Scripted test, mainly focusing on comprehensibility, feedback and learnability
Task 1 Navigate to the level selection
Task 2 Switch mode to seas and lakes
Task 3 Select level of choice
Task 4 Go back to the available levels
Task 5 Switch mode to cities
Task 6 Select level of choice
Task 7 Start that level
Task 8 Play two rounds
Task 9 Pause the game
Task 10 Continue the game
Task 11 Play until you win or lose the level
Task 12 Try that level again
Task 13 Play one round and go back to the level selection Task 14 Make sure you are in the cities mode Task 15 Navigate to the levels available in North America
Task 16 Go back to main menu
Task 17 Check your personal stats, including badges
Task 18 Go back to main menu
In order to get more information about the learnability and the actual learning, another sim-ple scripted test was used, as seen in table 3.3. The basic idea of it is to allow the participant finish one level, and see how the participant progress during the gameplay, as well as how the feedback affected the gameplay and learnability. After finishing the level, the participants were asked to play the same level again, which allowed further evaluation of how well they learned and progressed. During this scripted test, the participants’ progress was followed through out the played level, mostly by counting the amount of tries it took, in order to analyze how well they learned. This scripted test was not performed in combination with the Think-Aloud protocol, mainly to gather more information about focus related categories, such as immersion.
Table 3.3: Scripted test focusing on learnability and gameplay Task 1 Select a level of choice and start the level Task 2 Keep playing the level until you finish it Task 3 Play the same level again and finish it
3.3. Testing and Evaluation
3.3.4
Game Experience Questionnaire
The Game Experience Questionnaire (GEQ) [8], was devised for play testing games and measuring the different aspects of the individual experience the players have during the gameplay.
It consists of three different modules which are meant to be used directly after the game-session, and consist of multiple questions. The first two modules investigate the players’ feelings and thoughts during the gameplay, while the last investigates their feelings after they have stopped playing the game.
In this thesis, we focus mostly on the first module as it evaluates the game experience based on components we are interested in. The second module focuses more on psycholog-ical and behavioral involvement within the social experience in the game, which is not of our interest and therefore not used. However, the third module is focusing on the post-game feelings, which is of our interest since it can be associated to some of the components in the first module, which are relevant to our research area.
Analyzation of the questionnaire was done to check whether any of the assumptions in the GEQ should be changed or removed based on their relevance, in order to avoid sub-optimal scoring patterns. Each module consists of assumptions correlated with different components, such as immersion, flow and challenge. For each of these assumptions the players are scoring according to a scale from the lowest score, 0, to the highest, 4. The scoring for each module is then summed and categorized by the components, where the mean is the final score for each category. These scores can then be used in order to get an understanding of the players’ game experience for the different categories.
The first module, core module, originally have 32 assumptions. However, a few of these have been removed due to them not being relevant and not fully rewarding, or because of multiple of them being too similar in meaning. The slightly modified core module can be seen in table 3.4.
3.3. Testing and Evaluation
Table 3.4: Slightly modified core module of the Game Experience Questionnaire
Assumption 2 I felt like I was improving
Assumption 4 I thought it was fun
Assumption 6 I felt happy
Assumption 8 I thought about other things
Assumption 9 I found it tiresome
Assumption 11 I thought it was hard
Assumption 12 It was aesthetically pleasing
Assumption 13 I did not focus on things around/outside the game
Assumption 14 I felt good
Assumption 15 I was good at it
Assumption 16 I felt bored
Assumption 17 I felt successful
Assumption 19 I felt that I could explore things
Assumption 20 I enjoyed it
Assumption 21 I was fast at reaching the game’s targets
Assumption 22 I felt annoyed
Assumption 23 I felt pressured
Assumption 24 I felt irritable
Assumption 25 I lost track of time
Assumption 26 I felt challenged
Assumption 28 I was deeply concentrated in the game
Assumption 29 I felt frustrated
Assumption 30 It felt like a rich experience
Assumption 32 I felt time pressure
The points for each assumption were then divided into the respective category according to the guidelines in the Game Experience Questionnaire, as seen in table 3.5 bellow.
Table 3.5: Scoring guidelines for the core module
Component/Category Assumptions
Competence 2, 15, 17, and 21
Sensory and Imaginative Immersion 12, 19, and 30
Flow 13, 25 and 28
Tension/Annoyance 22, 24, and 29
Challenge 11, 23, 26 and 32
Negative affect 8, 9, and 16
Positive affect 4, 6, 14, and 20
With the same reasoning as for the core module, there were a few assumptions removed from the post-game module, resulting as in table 3.6. The scoring was done in the same approach as for the core module, following the scoring guidelines for the post-game module, which uses different categories as seen in table 3.7.
3.3. Testing and Evaluation
Table 3.6: Slightly modified post-game module of the Game Experience Questionnaire
Assumption 2 I felt bad
Assumption 3 I found it hard to get back to reality
Assumption 5 It felt like a victory
Assumption 6 I found it a waste of time
Assumption 7 I felt energized
Assumption 8 I felt satisfied
Assumption 11 I felt that I could have done more useful things
Assumption 13 I felt weary
Assumption 14 I felt regret
Assumption 16 I felt proud
Assumption 17 I had a sense that I had returned from a journey Table 3.7: Score guidelines for post-game module
Component/Category Assumptions
Positive Experience 5, 7, 8 and 16
Negative Experience 2, 6, 11 and 14
Tiredness 13
Returning to Reality 3 and 17
3.3.5
User Engagement Scale
Another well used tool for measuring user engagement is the User Engagement Scale (UES) [14]. There are cases where only selected parts of the UES are used, mainly because of the length of the original full version. Using the original version may lead to participant fatigue or attrition, especially when multiple trials are involved, or other questionnaires are used. Therefore, a four-factor model, User Engagement Scale Short Form (UES-SF) was devised which still defines the core concepts as in the original full form and was used for this project.[15]
The UES-SF could potentially be more ideal when multiple tasks are involved and is also a slightly more robust scale with less conceptual overlaps in some cases. The four dimensions used are aesthetic appeal, focused attention, perceived usability, and reward.[15] Like the Game Experience Questionnaire, there are different items, or assumptions, to be scored using a scale and the score is calculated using scoring guidelines.
Table 3.8: Factors in the User Engagement Scale Short Form
FA Focused attention
PU Perceived usability
AE Aesthetic appeal
3.3. Testing and Evaluation
In the UES-SF, the four factors consist of three assumptions (items) each. Each item should be given a score from 1 to 5, where 1 stands for strongly disagree and 5 strongly agree. The scoring is done for each factor where each item’s score within same factor is summed and then divided by the number of items, in this case three. [15]
Table 3.9: User Engagement Scale Short Form
FA-S.1 I lost myself in this experience
FA-S.2 The time I spent using Mappy just slipped away.
FA-S.3 I was absorbed in this experience.
PU-S.1 I felt frustrated while using this Mappy. PU-S.2 I found this Mappy confusing to use.
PU-S.3 Using this Mappy was physically or mentally demanding.
AE-S.1 Mappy was attractive.
AE-S.2 Mappy was aesthetically appealing.
AE-S.3 Mappy appealed to my sense.
RW-S.1 Using Mappy was worthwhile.
RW-S.2 My experience was rewarding.
4
Results
4.1
Implementation
This section will describe the game design elements implemented in this educational game. It will focus on the important parts related to the theory and explain the reasoning for imple-menting these.
4.1.1
Game Design Elements
Together with my supervisor and examiner, we discussed and planned the relevant game de-sign elements to implement in the educational game. This was done after acquiring enough information within the area of gamification, immersion, flow, motivation and engagement within educational gaming. The focus was on implementing the game design elements with-out them dragging too much attention towards them, but instead providing a better user experience as well as contributing to the related and relevant categories. Moreover, this also contributes to the aim, which is to increase the learning effectiveness, with a more engaging and motivating experience, while still keeping the gameplay comprehensible.
4.1.1.1 Animations and Feedback
Multiple animations were added to the game, mainly to contribute to clearer feedback, but also for the application to be more dynamic. The level selection has a similar view as in the actual gameplay, with markers and a map. In the level selection, the player has the freedom to navigate around and select a level of choice to play. Animations are implemented to create a more dynamic level selection, with the use of dropping markers, zoom transitions and other components animated accordingly, as seen in figure 4.1.
The intention with some of the animation is to make the players feel progress and that they are moving forward in the gameplay, by having theme changes and zoom transition-ings, as seen in figure 4.2 and figure 4.3. Each game design element was implemented with animations to increase the awareness of incorrect and correct answers, with the intention of
4.1. Implementation
Figure 4.1: Level selection screen, with level markers dropping down and other animations operating.
Animations were added to the markers, where they have a smooth transition during the gameplay, as well as providing animated feedback when choosing the incorrect alternative. This is done by first highlighting the pressed incorrect marker, and then transitioning to the correct marker and highlighting it, as seen in figure 4.2. We want the players to understand the misses, reflect and learn from them, to eventually finish the level. This part should not take too much time, just enough to serve its purpose.
Figure 4.2: Feedback showing the incorrect and pressed marker, transitioning over to the correct marker.
4.1. Implementation
4.1.1.2 Score
Every time a player chooses a correct marker, points will be added to the total for that level, where the points depend on the time left for each round. The player can lose points from the total by choosing the incorrect marker, but will not go under the minimum of zero points for that level. Except for acting as a feedback, the score is supposed to be a motivating factor in form of a reward. The total score will later be used for highscore, and possibly in connection with other rewarding elements further on in the development of this educational application. Additionally, the color for the score should sometimes be adapted to the theme for the visual and aesthetic appeal, as seen in figure 4.3. The positioning of the score is chosen so it does not distract or take focus from the actual gameplay, as seen in figure 4.2 and figure 4.3. The animations when losing points and gaining points are the same as for the life component, which are described in the next subsection.
4.1.1.3 Life
For each level there are different amount of lives given, depending on the difficulty of the level, which reflects the amount of times the player can choose an incorrect marker. After an amount of correctly picked markers in a row, the user is granted an extra life, with the exception of the amount of lives already being the maximum amount for that level. This should serve as a motivating factor that should make the player value the choice when se-lecting markers. As for the score, the color for the lives should sometimes be adapted to the theme and the positioning not distracting, as seen in figure 4.3. Choosing an incorrect marker should remove one life, which is done in combination with an animation. This should make the player aware of the life loss, without the player losing focus from the gameplay and the important feedback. A shake animation is used for this, as this usually is interpreted as a miss or incorrect answer. In figure 4.2, the the life can be seen more to the left during the shake animation in the left image, and then normally positioned again in the right image. A received life is animated by it bouncing in, hoping it will be interpreted as an addition or a gain.
4.1. Implementation
4.1.1.4 Time
During each round of a level, the player has a to choose a correct marker before the time runs out. The time is represented by a animated time bar, which decreases in correlation with the actual time, as well as changing color to indicate when it is crucial to choose a correct marker, which can be seen in figure 4.3. This element’s purpose is to increase the challenging factor, with the aim to increase the player’s focus due to the pressure as it is correlated to the score.
4.1.1.5 Progress Bar
To get an understandability of how the level went, a progress bar is used, seen in figure 4.4. This progress bar indicates how far in the level the player reached and is displayed as an ani-mated bar with the percentage of how many markers the player got out of the total. Progress bars are overall good and self-explainable graphical representations which are regularly used for progress feedback.
Figure 4.4: Progress bar, with additional information about the played level.
4.1.1.6 Statistics
Statistics were implemented, with the motivation of allowing the user to have an overview of the progress and contribute to the rewarding factor. It currently shows the progress, sums up points and displays the total time played, while further additions will be added.
4.1.2
Levels
Each round consisted of a group of markers, which were shown two at time, and increasing to three at the last rounds in a level. This keeps a enough challenging level with a slight increase towards the end to keep the intensity. To avoid the players to memorize the rounds and contribute to the learning, the markers were randomized each time a round was played. Usually, a person has better local knowledge, which lead to the decision of adjusting the level to the player’s geographical position if possible. This contributes to the flow and the challenging factor, as difficulty increases by time. However, allowing the players to have freedom of choosing different levels, rather than having them to unlock these, contributes to the engagement and decreases the risk of them quickly losing interest.
4.1. Implementation
Animations and theme changes between the rounds contribute to the dynamic experi-ence, allowing the players to feel they are advancing, making them more engaged and feel rewarded.
4.1.3
Implementation of the Game Design Elements
It is of importance to implement the game design elements without them gaining too much focus, as they are not the central part of the game, but should instead contribute to a better experience and engagement. These elements were placed in areas where they do not disturb the gameplay and the learning, typically in corners as the focus usually is on the centre of the screen. There should neither be too many distracting components, meaning adding compo-nents for fun and curiosity, as the this would negatively affect the learnability, comprehensi-bility and other mentioned important factors we aim for when developing this educational game. In this work, only the central game design elements are used, these are normally cov-ered within gamification and could variously contribute to related categories depending on the game design.
4.2. Think-Aloud Protocol and Scripted Tests
4.2
Think-Aloud Protocol and Scripted Tests
The majority of the tasks were completed without any difficulties, with an overall quick un-derstanding of the gameplay by there participants. However, a few participants did not quite understand some of the components and animations immediately, resulting into a slight con-fusion. This was noticed during the two tasks seen in table 4.1, either by the participants expressing confusion or by not completing the tasks without any mistakes.
Table 4.1: The two tasks being confusing for a few participants during the scripted test
Tasks Description
Task 15 Navigate to the levels available in North America
Task 16 Go back to main menu
During the scripted test focusing on learnability and the actual learning, seen in table 3.3, all of the participants were minimizing their amount of incorrect answers on the second try of the level, as well as fully understanding the gameplay. This was resulting into better results and amount of points compared to the first try of the same level. They also felt more focused, immersed and motivated to improve their result on the second attempt at finishing the same level with improved results. During the second scripted test, the majority of the participants were very focused, engaged and felt overall motivated and immersed during their gameplay.
4.2.1
Game Design Elements
The observation of participants’ expressions and comments regarding the game design ele-ments during the scripted tests and the Think-Aloud protocol showed how noticeable these elements were, as well as how they contributed to the game according to the participants.
4.2.1.1 Animations and Feedback
Even if the participants were quite focused on the gameplay and in completing the tasks, they reacted to the animations and feedback. The central animations for creating a more dy-namic experience, such as theme changes, zooming and positioning transitions, were mainly mentioned by the participants. These were experienced as the game feeling livelier and the participants feeling that they were progressing. The feedback animation that was the most noticeable was the one animated when an incorrect marker was chosen, which was mostly appreciated. However, it also was the most confusing one, which was observed during their gameplay. The confusion was noticed when the participants tried to press the correct markers during the actual feedback animation, later realizing that this was intended for them to learn from their mistake. Overall, the animations contributed significantly to the engagement and dynamic experience.
4.2.1.2 Score, Life and Time
Furthermore, the other game design elements, i.e. score, life and time, felt natural for the par-ticipants. For example, they noticed when they had one life left, when time was running out for a round as well as the color change on the timer. However, the animations for these were barely commented nor focused on. These animations could be more eye-catching according to some of the participants, which was also observed since not many reacted on losing or receiving a life, neither on the animation correlated to it, which also applies for the score and its animation. The combination of these elements with their correlation contributed to the fo-cus, challenging and the rewarding factor, as the participants were focused and valued each decision to avoid getting penalized for incorrect or slow answers.
4.3. Game Experience Questionnaire
4.2.1.3 Progress Bar and Statistics
The participants were positive to the progress bar and statistics, as they felt they got a good overview of their latest gameplay, as well as their overall statistics in the game. They mainly showed interest in knowing how far in the level they got. Some participants felt that the progress bar was enough, while other more competitive participants mentioned they would like to see even more statistics in form of achievements and correlated badges.
4.3
Game Experience Questionnaire
Both the core module and the post-game module, were answered by 52 of the 65 participants. Table 4.2 presents the calculated mean score for all the components in the core module. Like-wise, table 4.3 presents the same for the post-game module.
Table 4.2: Mean score for the core module
Component/Category Mean score
Competence 2.90
Sensory and Imaginative Immersion 2.47
Flow 2.10
Tension/Annoyance 0.49
Challenge 2.09
Negative affect 0.47
Positive affect 2.43
Table 4.3: Mean score for the post-game module
Component/Category Mean score
Positive Experience 1.5
Negative Experience 0.42
Tiredness 0.23
Returning to Reality 0.48
The participants did not show any noticeable difficulties in answering the core module. Some of the participants did however comment some of the assumptions, meaning scoring these depends on how they are interpreted, especially for the post-module which was more con-fusing according to them.
4.4. User Engagement Scale Short Form
4.4
User Engagement Scale Short Form
The User Engagement Scale Short Form was answered by 55 of the 65 participants. Table 4.4 presents the calculated mean score for all the factors.
Table 4.4: Mean score for the UES-SF
Factor/Category Mean score
Focused attention (FA) 2.28
Perceived usability (PU) 1.45
Aesthetic appeal (AE) 3.5
Reward (RW) 3.55
As for the Game Experience Questionnaire, the participants did not have any noticeable dif-ficulties in understanding and scoring most the assumptions. Similarly, a majority felt some of the assumptions were tougher to interpret and correctly score.
4.5
Categories Correlated to the Focus
Using a Think-Aloud protocol requires the users to focus on vocally expressing their thoughts during the testing, which in turn affects the focus on the actual gameplay. Therefore, the categories in the questionnaires correlated to the focus, such as immersion, flow and focused attention, are affected. This must be taken into consideration when analyzing the scoring of these categories, and cannot be fully reliable.
5
Discussion
5.1
Results
5.1.1
Game Design Elements, Think-Aloud protocol and Scripted Tests
The confusion with performing the two tasks described table 4.1 was mainly based on the participants’ misunderstanding of the correlated buttons’ function. Navigation to a different section in the level selection is done by pressing two buttons, with two arrows indicating di-rection. Trying a new game for the first time, is most likely the reason of the participants not directly understanding the functionality of these, causing the confusion and incompletion of the associated tasks. With most of the participants correctly predicting the functionality of the correlated buttons to these tasks, I do not consider this as an issue which should need a change in the implementation. Neither is the complexity of the level selection high, with very limited components for simplicity and comprehensibility. However, the arrows could be switched with the names of the next sections instead, which would probably result into a clearer understanding of the buttons functionality.
The animations with the intention of creating a more dynamic, rewarding and engaging experience showed being successful and very appreciated. Feeling progression and moving forward in the game were the results of these animations, leading to a more focused and engaged participant. It increased the clearness of the feedback and the awareness of the participants, without noticeably affecting the immersion, engagement or flow, but rather contributing to a more enjoyable experience. Therefore, it is of high importance to create a dynamic experience in an otherwise static and limited view, such as when using maps. Animations might need adjustments to optimize them for the best results, others might be unnecessary, while some might be missing. Evaluation is hence very valuable for acquiring this kind of information.
5.1. Results
Pressing an incorrect marker results in the game being paused for a short period of time, while the animation highlights the incorrect and pressed marker, then transitioning over to highlight the correct marker. This feedback was appreciated and worked as intended. A small confusion lead to the participants reacting too quick to the animation, instead of waiting it out. An option is to simultaneously highlight both the incorrect and correct marker in order to make the animation less confusing. A thought that arises is whether this would affect the understanding of the incorrect answer and therefore the learning, since the focus is then on the incorrect and correct marker simultaneously rather than one at time. Another option is to decrease the animation time and let the users get used to it over time, since there were no complaints or criticism about the current implementation of the animation.
Furthermore, during these animations the participants could also take in what they have learnt so far, rather than constantly and intensely playing, which could potentially lead to a less learnable and more demanding game.
Score, life and time are popular game design components, leading to players understanding these components naturally and keeping track of these during gameplay. The life, score and time are all correlated to the loss avoidance in this game, resulting in the player valuing the selection of marker, which in turn results into a more focused and learning player. Time pressure and life amount also contributes to the challenging factor, which is why it must be well balanced. They did, however, not keep as much track on when they received or lost a life, nor the score even though there were less intensive animations for this specific purpose. Successfully implementing these eye-catching animations without dragging too much focus from the actual gameplay or interrupting in a negative way, can be difficult and require modifications. The involved components are already kept track on, but the animations should increase the awareness and contribute to a more dynamic and immersive gameplay, without affecting the flow, engagement or any other correlated category. Since the animations of the lives and score did not fully serve their purpose, it will require few mod-ifications in order to achieve the mentioned results. In this case, the modmod-ifications needed might be as simple as increasing the intensity of the animations, or any other smaller changes. The progress bars were appreciated and served their purpose with visually providing feedback of the users’ progress after playing a level, and the animations to these were a good addition in order to create a slightly more dynamic and interesting view. However, the statistics were interesting for the participants but since the game is not fully developed and in its final state, it is not as relevant now. It should be further looked upon to have enough and relevant statistics both for the users and the maintainers.
5.1.2
Game Experience Questionnaire
5.1.2.1 Core Module
Beginning with the competence category, which got a score of 2.90, the participants felt that they were fairly improving, feeling good at it and quickly reaching the game’s target. Within this category, the difficulty was interpreting what successful in this case meant, which proba-bly lead to a more inaccurate result for this category due to the insecurity. However, consid-ering the early stage of the game and during the short time it was tested, the result shows that the game got an acceptable score for the competence category. It will require more time to get an even more accurate score and to certainly understand how well the users are learning using this educational game. Moreover, it would also be a very doubted result if this category got a too high score based on the first-time experience and the short testing done. This cate-gory was a good completion to the results and information gathered from the Think-Aloud protocol.
5.1. Results
Moving further to the sensory and imaginative immersion, it is relevant both during the early stage and the first-time experience, as well as for the final product. This category got a score of 2.47, where we strive for as high score as possible during early stages as well as for the final product. The assumptions related to this category is not the best fitting and most relevant for this specific educational game, which could potentially affect the scoring. An issue with these, in this context, is how rich the experience should be expected for this specific game, as well as how much they should expect to explore within this game. This does provide a good overview in how much the game could be improved within this category, but should not fully cover the immersion evaluation of the game.
The scoring for flow did give a decent understanding of how concentrated the partici-pants were during the testing. The reasoning is the same for the challenge category, where the score showed the game is moderately challenging and was neither too hard nor too easy. However, the participants did only test a limited amount of levels, which does not give a completely correct scoring. These results would be more relevant and correct if the game was played for a longer period of time, allowing the users to score each assumption more correctly, as a shorter test does not reflect the full experience. These are still of relevance since they can be used to gather a good understanding of how well the game is developed withing these categories, and which improvements should be considered in the continuous development.
The feeling of annoyance and tension was mainly based on the mistakes done when choosing an incorrect marker, which comes naturally when being focused and immersed in the game. We do not want this to be the main focus, but is of interest due of its relevance and correlation to other categories, such as immersion. To understand this category, an analyzation of what the annoyance and tension is based on is required.
Most of the participants felt a positive affection, where it seemed as a fun and enjoying way of learning geography, as well as being able to relate to the widely relatable maps used in smartphones today. It is expected for it to have some negative affections, since not every user will enjoy these kinds of educational games, mainly due not having any interest or generally not enjoying educational games. The importance here is that the positive affect is considerably higher than the negative.
For the current stage of the game, where the central and core parts are implemented, these results indicate that the current implementation and formation of the game are looking promising for achieving the targeted result.
5.1.2.2 Post Module
The scoring of the post-module is difficult to accurately interpret, meaning it is not a suitable evaluation for this game. This is mainly due it being tested quickly and based on the first-time experience, rather than after a good amount of playfirst-time invested into the game. Based on the results, the participants felt more of a positive experience, and did not feel weary nor had a problem returning to reality. Due to the difficulty in interpreting the assumptions, as well as this module not being fully suitable as it was used too early, these results are not being focused on too much.
5.2. Method
the GEQ and the same thinking is applied for this factor. Moreover, if the game is challenging it can affect the physical and mental demand, meaning the perceived usability factor is to some extent related to the challenge category of the GEQ. Based on the score for the aesthetic appeal factor and the rewarding factor, the game is being somewhat aesthetically appealing as well as rewarding. However, since some of the assumptions for each factor in this ques-tionnaire is woolly, the score is not fully representative. Another reason the score is not fully representative is the limited testing as mentioned in the discussion for the GEQ. Similarly as for the GEQ, these results somewhat indicate a promising and well formed game for reaching the targeted result.
5.2
Method
5.2.1
Implementation of Game Design Elements
The implementation of this educational game was based on gamification, learnability and comprehensibility, striving for an immersive, engaging and educational experience. This re-quired game design elements to be implemented, as they are the key factors contributing to these areas. Furthermore, the planning around the game design and the game design ele-ments was important. An insight in the correlation between gamification, immersion and flow is required in order to understand which game design elements are essential. These, together with animations, should be well thought out and clear enough to serve their pur-pose, in order to contribute to the simplicity and comprehensibility, as well as other relevant areas. Exaggerating can result into negative affects on the learnability and comprehensibility as there would be too much on the screen, resulting into a more complex and less straightfor-ward layout. Therefore, these elements were placed appropriately with enough space on the screen to server their purpose, in order to also minimize unnecessary components and de-creasing performance. However, after evaluating the implementation, further modifications were needed, as the expected results is not always guaranteed.
5.2.2
Think-Aloud protocol and Scripted Tests
The widely usage of the Think-Aloud protocol is not a coincidence, it is a simple and cost-efficient yet effective tool for gathering relevant information about a tested system. By only using traditional Think-Aloud protocol and allowing the tester to freely use the application, the testing can be time-consuming due to all irrelevant tasks being performed, which results into irrelevant information being gathered. Therefore, in combination with scripted tests, the maintainer can more efficiently focus on the fields of interest, meanwhile each step is easier to analyze.
In this study, the think-aloud protocol in combination with the scripted tests gave a good understanding of each relevant category. It was later easy to evaluate and analyze each relevant part in order to either understand where improvements were needed, or the reason of why parts worked as expected. However, even though this tool showed out to be efficient with relevant information collected, it is not fully reliable nor is the testing always straight-forward. The maintainer must partly correctly interpret the testers thoughts and actions during the testing, such as mistakes, as these are not always clear. Therefore, some additional tools might be needed to complete the understanding of uncertainties or parts not answered during the Think-Aloud protocol. All in all, the Think-Aloud protocol in combination with scripted tests is a recommended method to gather relevant information about a system when performing shorter tests, which can also be used in different stages of the development. The Think-Aloud and the scripted tests were the clearly the most giving tool in this study.
5.3. Source Criticism
5.2.3
Game Experience Questionnaire and User Engagement Scale Short Form
It is difficult to get a full understanding using a think-aloud protocol and scripted tests, which is the reason of why questionnaires were used in this study. They cover different categories based on the participants experience and engagement. Moreover, these could complete the missing parts of the information needed, which were difficult to interpret for the maintainer during the Think-Aloud testing. Furthermore, the usage of the Game Expe-rience Questionnaire (GEQ) together with the User Engagement Scale Short Form (UES-SF) felt reasonable to get a more precise result to analyze. However, several of the assumptions in both the questionnaires were difficult to interpret. Multiple assumptions were very similar, which lead to shortening these questionnaires as I wanted to keep it short and concise. These questionnaires gave an additional understanding of the relevant categories developing the game, but are more useful when the participants have been playing for a longer time to score the assumptions more precisely.Some of the categories had understandable and relevant assumptions which were giving additional insights in how the participants experienced the game. Therefore, the question-naires were to some extent useful, but not needed for this study as they did not contribute as much as expected. Moreover, these would probably be more useful for other types of games, or for evaluations where the participants have been using the game for a longer time. Another idea is to customize the questionnaires to the tested system, resulting into them being more comprehensible and more relatable to the tested system. This means that the GEQ and UES-SF would work as templates instead of using these directly.
5.3
Source Criticism
Scientific and peer-reviewed sources describing the relevant theory have been used in this work, ensuring the accuracy of the information. However, there is still some ambiguity re-garding gamification, as well as some of the other relevant areas. Typically, the same game design elements are brought up, and there is no clear evidence of their usefulness and effec-tiveness within the areas. This results into difficulties in completely interpreting these areas, leading to more testing and evaluation in order to understand these elements. Moreover, the questionnaires were not completely evident and required some adjustments for this work.
6
Conclusion
The aim was to implement a pick-up-and-play gamification geography game for learnability, mainly focusing on gamification, immersion and flow. More specifically, the work should have initially investigated the necessary and relevant areas, as well as the relevant game design elements. Given this information, the game should have been implemented to achieve the results strived for. Finally, an evaluation and analyzation of the implementation should have been done using a think-aloud protocol in combination with scripted tests, as well as using two questionnaires for measurement of the experience and engagement. The results and correlated discussion have shown that this work have answered the associated research questions. The research questions to be answered were:
1. Which considerations should be taken into action when designing and developing an educational pick-up-and-play gamification geography game for learnability?
A method was proposed, focusing on gathering knowledge within relevant areas to competently apply the most central game design elements, including animations. Im-portant factors related to the implementation were brought up, mainly focusing on the relevant game design elements and their correlation to the researched areas. Further-more, a method for performing evaluation and analyzation was proposed with the pur-pose of gaining an overview and understanding of the implementation, as well as how and where improvements could possibly be done.
2. Which tools can be used for evaluation and analyzation of the implementation, based on the relevant areas? How useful are these in this work?
In order to perform the evaluation and analysis of the implementation within the rele-vant areas, the used tools were
• Think-Aloud protocol, combined with scripted tests
The Think-Aloud protocol combined with the scripted tests was highly useful. It gave the most relevant information, out of the three tools, and contributed to a good understanding of the implementation and the participants thoughts. How-ever, it is not always for the maintainer to interpret everything correctly and could need a complementary tool, such as a questionnaire.
