Application for innovative gamified e-service

Degree: Bachelor Thesis 180 HP Handler: José María Font

Teknik och samhälle

Datavetenskap

Examensarbete

15 högskolepoäng, grundnivå

Application for innovative gamified e-service

(Applikation för innovativ

e-service

)

Abstract

Motivation from students that studies at higher educational institutes can sometimes be minimalistic at different times. This can depend on different reasons, such as boring lectures, bad assignments or that the student does not feel the social belonging to their class. With this study, we want to help the students by increasing their motivation with the use of gamification in an e-service. Through a collaboration with Marcus Lindstedt that studies game development at Malmö Högskola, we created an e-service that contains gamification elements to motivate students. The prototype that is created is not supposed to replace the student's current e-service but should be seen as a complement to increase motivation. To test if the students felt more motivated after using this prototype, we gave 10 students the opportunity to test it during a ten minutes time period. The results from the students were then gathered from a questionnaire that consists of a number of questions regarding this prototype and motivation.

Sammanfattning

Motivation hos studenter som studerar på högskolor eller universitet kan ibland vara minimal vid olika tillfällen. Detta kan bero på olika anledningar, så som tråkiga föreläsningar, dåliga uppgifter eller att man bara inte känner att man passar in i sin klass. Genom denna studie vill vi hjälpa studenterna att få mera motivation genom användandet av spelelement i deras e-service. Genom ett samarbete med Marcus Lindstedt som studerar spelutveckling på Malmö Högskola, så skapade vi en e-service som innehåller spelelement för att motivera studenterna. Denna prototyp är inte i första hand tänkt att byta ut studenternas nuvarande e-service utan skall först ses som ett komplement för att höja motivationen. För att testa om studenterna känner sig mer motiverade efter användandet av prototypen så satte vi ihop test där studenterna fick prova på att använda denna prototyp under en tio minuters period. Resultaten från studenterna är sammanställda genom en enkät som består av ett antal frågor angående denna prototyp och motivation.

Contents

1 INTRODUCTION ... 1 1.1PURPOSE ... 3 1.2RESEARCH QUESTIONS ... 3 1.3RESEARCH SCOPE ... 3 1.4EXPECTED RESULTS ... 4

1.5DISPOSITION OF THE THESIS ... 4

2 METHODOLOGY ... 5

2.1SYSTEMATIC LITERATURE REVIEW ... 5

2.2APPLYING STATE-OF-THE-ART GAME DESIGN GUIDELINES... 6

2.3SUGGESTION TOWARDS A GAMIFIED SYSTEM ... 6

2.3.1FRAMEWORKS CONSIDERED FOR DEVELOPING WEB APPLICATIONS ... 7

2.3.2CHOOSING A POSSIBLE WEB FRAMEWORK ... 7

2.4DEVELOPING WITH ANGULAR 2 ... 8

2.5GAMIFICATION ELEMENTS IMPLEMENTED ... 9

2.6EVALUATING THE APPLICATION ... 10

3 SOLUTION ... 12

3.1OVERVIEW OF THE SYSTEM ... 12

3.2COMPONENTS IMPLEMENTED IN THE APPLICATION ... 12

3.3GAMIFICATION ELEMENTS IMPLEMENTED ... 14

4 RESULTS ... 17

4.1RESPONSIVENESS RESULTS FROM THE DEVELOPMENT ... 17

4.2RESULTS OF THE QUESTIONNAIRE ... 18

4.3SUMMARY OF THE QUESTIONNAIRE RESULTS ... 20

5 ANALYSIS AND DISCUSSION ... 20

5.1GAMIFIED APPLICATION = MORE MOTIVATION? ... 21

5.2POSSIBLE TO IMPLEMENT A GAMIFIED E-LEARNING SYSTEM? ... 21

5.3DID WE GET GOOD USER COVERAGE? ... 22

6 CONCLUSIONS AND PROPOSITIONS FOR FURTHER RESEARCH ... 23

1 Introduction

According to Europe 2020 indicators, there is an increasing rate of students who drop out of higher institute educations (European Commission. 2015). Reasons for the increasing dropout rates have been discussed at great length by researchers, especially within the area of computer science (Saurabh Pal. 2012 and Tinto. 2006) and the reasons for student dropouts may vary. Some students feel that the education did not live up to their expectations or that the courses in their education were too challenging. Other students drop out due to lack of motivation (Europeancommission.2015). To address the increasing rate of student dropouts’ due to lack of motivation, we have investigated key terms surrounding motivation, e-learning, and gamification, to be able to develop a gamified application.

Motivation in learning is a key aspect to analyze when discussing student dropout rates. Motivation can be divided into two specific categories called intrinsic motivation and extrinsic motivation. Intrinsic motivation refers to actions taken for self-interest and curiosity. Extrinsic motivation concerns motivation from recognition or grades rather than motivation out of curiosity (Bartel and Hagel. 2014 and Llorens-Largo et al. 2016). Studies have also shown that computer games increase student motivation (Ibrahim and Jaafar. 2009).

E-learning

As the problem in question concerns student motivation, it is necessary to understand the concept of electronic learning. Electronic learning(e-learning) is about learning over distance by using a medium like the internet, which is completely virtualized (Lara et al. 2014). E-learning has both positive and negative features which need to be taken in to account. One of the weaknesses with e-learning is lack of a concrete teacher who has little to no follow-up with the students. This leads to a lack of personal supervision by the teacher. Supervision of student behavior is essential for detecting problems. On the other hand, e-learning has several strengths such as flexibility, interactivity and focus on the learner (Mihaela Cojocariu et al. 2016). Classcraft is one e-learning tool designed as a Massive- Multiplayer-Online (MMO) that uses game mechanics to engage students and evolve the learning experience (Classcraft. n.d.). Students are divided into teams and are rewarded for helping each other. Classcraft uses both negative and positive reinforcements, like giving students the possibility to earn the right to eat in class or get an answer handed to them on an exam. Kahoot is another e-learning tool that is designed as a game show, where the teacher acts as the host (Kahoot. 2017). Kahoot is widely used in elementary and secondary schools. In the US, twenty million students use Kahoot on a monthly basis (Singer. 2016). Both Classcraft and Kahoot incorporate gamification elements in their creations. Gamification will be further described in the following section.

Gamification

Gamification has turned out to be somewhat of a new research area in computer science and extensive amount of research have been published (Hamari et al. 2014). Deterding et al (2011) define gamification as the use of game design elements in non-game contexts. The purpose of this usage is to make them more engaging and to increase the fun (Bartel and Hagel. 2014). Gamification has been implemented in several studies to find out if it can have any effect on student results (Fotaris et al. 2016). Results have shown improved student engagement and higher student attendance in courses where gamification has been used (Ibáñez et al. 2014).

The authors (Barata et al. 2013) performed a long-time study on engaging engineering students with gamification. In their study, they evaluate how gamification affects students that attend college in one specific class. The first year, they held the course without any gamification elements. The second time they held the course they implemented it with a gamified version to be able to compare the two versions of the course. This version contained experience points, levels, challenges, and

badges. Their analysis showed a compelling increase in engagement, dedication, and satisfaction from the people attending the second year when the gamification elements were implemented into the course.

Other research on the subject of gamified learning has also been done by (Krause et al. 2015). Their paper focused on online courses, adding gamification and social aspects to help fight the retention rates and increase learning success. The gamification elements they implemented where achievements, score points, leaderboards and avatars for the students. The social aspects were a quiz game where the students could battle each other. They conducted experiments with three conditions. These conditions were: the course without any gamification elements or social elements, the second condition with gamification and no social elements, and the third condition with gamification and social elements. Their second condition showed an increase in student score with 23% and the retention lowered by 25%. The third condition, with the added social elements, showed a 40% increase in score and 50% reduced retention.

More research on the subject in developing a gamified e-learning system has been done by (Utomo and Santoso. 2015). In their research, they present a pedagogical agent with the purpose to motivate the students and provide adaptive information towards the students. The pedagogical agent gave the students information on what they need to focus on, based on the activity of the students using the e-learning system. Other gamification elements implemented in their system were assignments and quizzes, a progress bar and badges. They analyzed their system by gathering data from the students that tested the system. The results of their testing indicated that gamification elements and a pedagogical agent are two possible solutions to help motivate students in an online e-learning system.

Badges is a gamification element that can be used in education to help improve student results. To further understand the meaning of badges, one may compare it to the meaning of achievements in computer games. An experiment conducted by (Hakulinen et al. 2015) has shown that badges had a significant behavioral impact and that students were positive to gamification design elements such as badges. The development of a gamified application can be done in multiple ways, either as a native mobile application or as a web application.

Frameworks for development

To acknowledge the need for different development systems, we first have to analyze the diversity of mobile operating systems used. According to the Internet foundation in Sweden (2016), the most preferred phone in Sweden is the iPhone, with 41% mobile users. Second place comes Android devices with 37% of users. The remaining 3% use another model than the aforementioned two ones (Davidsson and Findahl. 2016). This leads to another problem, namely the challenges of developing mobile applications across multiple devices with different operating systems. Developing different devices are time-consuming and costly, which leads to developers having to redesign and re-implement most of their creations. This makes cross-platform development with HTML5, CSS, and JavaScript an interesting and possible way to deal with this problem (Joorabchi et al. 2013). This is a key aspect when evaluating the possibility to maximize the target audience. At the same time, one of the possible drawbacks when developing non-native applications is that the application does not acquire the same access to hardware integrations as a native application. This may affect the performance and functionality of the application (Jobe. 2013). But non-native applications which only consume information and don't need access to the hardware are suitable for development with JavaScript (Jobe. 2013).

When discussing web application development using JavaScript, one encounters several frameworks which expand the functionality of JavaScript. Angular 2 is one such framework with improved speed and performance (Angular 2. 2016). In Angular 2, components are the main way to build specific elements and logic on the web-page. Components are composed out of four different

files: the shaping of information (CSS), the structure of the information (HTML) and two TypeScript files. One TypeScript file is in place for testing the code and the other one is in place for the interactivity on the web-page. Services are used for reuse of non-view related logic, which can then be used by different components. (Angular 2. 2016). Angular is a part of the full stack development environment called MEAN. MEAN is a combination of tools (MongoDB, Express, Angular, and Node.js) which all run on JavaScript (Štajcer et al. 2016). According to research, MEAN is the new hip trend in web development, followed by a large and enthusiastic community (Louridas. 2016).

1.1 Purpose

The purpose of this thesis is to gain a deeper understanding of how to create a gamified cross-platform application. We also strive to study the effects of using a gamified cross-platform application to investigate student motivation in higher education, which can help motivate students at higher education institutes. To investigate the problem with student motivation in higher education, we will create an application named Clash of the MAHs that uses gamification elements to increase student motivation. The application will be developed to cover the largest number of students possible. Furthermore, student motivation at higher education institutes will be investigated through a questionnaire.

1.2 Research questions

To grasp an answer to the problem discussion, the following research questions have been formulated: RQ1: How can a gamified application help motivate students at higher education institutes?

RQ2: How can the services that are provided by the student's current e-learning system be implemented in a gamified application?

RQ3: How can a cross-platform gamified application be developed in order to have maximum user coverage?

1.3 Research scope

All of the aspects mentioned in the introduction and problem domain have been analyzed and used as the basis for the development of our application (Clash of the MAHs). This analysis has been incorporated into the design document provided by Lindstedt (2017). The document in question is what was conducted from Lindstedt’s (2017) state of the art analysis of game mechanics. It is new research on the topic and is highly applicable to our creation of a gamified application. To give a short description of the design document, it includes aspects of UI design that shows what were needed to be incorporated. Another aspect included in the design was the mini-game. The design describes the look and feel of the game and which game mechanics that should be included. The document also gave examples of gamification elements that would help increase motivation.

As this is a large problem area with multiple vantage points, all aspects have not been covered in this thesis due to time constraints. We have decided to limit the scope of our thesis to the development of a cross-platform application to help combat the motivational problems for students at Malmö Högskola. Furthermore, we are not focusing on any specific students or certain programs and will be using the students that are available at Malmö Högskola, as opposed to other or even all higher education institutes when we are performing the tests. This is because students at Malmö Högskola uses itslearning as e-service tool and it’s also convenient for us as attending students at Malmö Högskola.

1.4 Expected results

The creation of a prototype application will be implemented, and this will give us the possibility to use it as a testing tool to answer the research questions. We assume that the production of this application as a web application will give us the highest student coverage. The students whom will use this application will feel more motivated to complete their assignments, which in turn could lead to a lower student dropout rate. Some of the challenges could be to incorporate the empirical findings from the literature into elements in the development. Other challenges could be to make the application approachable to a large number of different student groups, not only to students who prefer to play games.

1.5 Disposition of the thesis

The disposition of the thesis is organized in the following way. The upcoming chapter will describe the methodology of the study before moving over to the third chapter, where the solution of our development will be defined. In the fourth chapter, the results will be presented. In the fifth chapter, the analysis and discussion will be presented. In the sixth chapter, the conclusions will be discussed as well as propositions for further research.

2 Methodology

This chapter is aimed to describe the research methodology for this thesis. The strategy that is going to be used for this thesis is design and creation (figure 1). The thesis will focus on the creation part, meaning the creation of a gamified application since the design for the application will be carried out by Lindstedt (2017). The decision to use design and creation as a strategy was because of the need to create a gamified application from the viewpoint that there has not been a previous gamified application at Malmö Högskola. Using design and creation gives something tangible to show like a gamified application, instead of just theories and knowledge (Oates. 2006). For these reasons, it was the best choice for us. The creation method for this gamified application is pair programming because it increases the quality of the code and reduces the time spent coding due to less error-prone mistakes (Williams et al. 2000). Finally, the evaluation process will be conducted using questionnaires to collect data. The data will be evaluated using the Likert scale (McLeod. 2008). As an alternative data generation method, interviews have also been considered for gathering information on the IT-artifact. However, interviews were discarded because they are time- consuming, may lack reliability and may also be misleading (Oates. 2006). It is difficult to predict whether the results would have changed if we would have conducted the interviews instead. Altogether, the results from our development of the application and questionnaires will give us answers regarding our hypothesis and research questions.

Figure 1: Design and Creation Process Modell (Vaishnavi and Kuechler. 2004)

2.1 Systematic literature review

The systematic review of the literature was created from our research questions. The questions were divided into specific keywords. The keywords were then analyzed, and synonyms of the words were added to our search strategy. With a combination of keywords and focus on the research questions stated in section 1.6, we conducted our searches. We used the approved academic search engines to conduct our searches. These were ACM and IEEE and Libsearch. The combinations led to multiple search results that could range from several thousand to a couple of hundred. When the results were in a manageable range, this could vary from 80 as most to lower ranges. An initial read-through of abstracts that related to our research questions was conducted. Here we focused on articles that reference the keywords. When an article was selected from this initial read it was examined again with a more extensive look. Below you can see an example of a search made with this method.

IEEE

Keywords: gamification AND learning
 Criteria: only journal


Results: 14


This search yielded in two research articles shown below:

Llorens-Largo, F., Gallego-Durán, F.J., Villagrá-Arnedo, C.J., Compañ-Rosique, P., Satorre- Cuerda, R and Molina-Carmona, R. 2016. Gamification of the Learning Process: Lessons Learned. IEEE Revista Iberoamericana de Tecnologias del Aprendizaje 11, 227–234.

Ibáñez, M.-B., Di-Serio, A and Delgado-Kloos, C. 2014. Gamification for engaging computer science students in learning activities: A case study. IEEE Transactions on Learning Technologies 7, 291– 301.

2.2 Applying state-of-the-art game design guidelines

The design document provided by Lindstedt (2017) has been analyzed and key areas have been implemented in accordance to the document. We used this design document because it was new research done on the subject and was designed to suit our application's needs. We have decided to exclude aspects which were out of the scope of the problem discussion while re-evaluating other aspects. Examples of the design implemented from the document can be seen in figure 2. These design images have been used as a template on how to implement the UI.

Figure 2: Design for study hall, dashboard and arena

The figure above represents a small part of the design created by Lindstedt (2017). The design document includes more design, like game mechanics for the arena and all achievements. This will be discussed in more detail in later parts of the thesis.

2.3 Suggestion towards a gamified system

In the design and creation process model, the first step is to gain awareness of the problem. This has been implemented in our problem discussion section. The next step is to give a suggestion to the problem. The step of giving a suggestion is where a tentative idea to solve the problem is presented (Vaishnavi and Kuechler. 2004). Since the design and content of the gamified application are generated by Lindstedt. (2017) state of the art design research documentation, we have had the chance to focus fully on the implementation of the application. Using the design document, we started by identifying everything the gamified application would need to be implemented. Overviewing the design documentation, we quickly identified the need to implement a front-end to display the user data and a responsive UI to cover as many students as possible. This was carried out as a response to

time restrictions and to the need of creating a gamified application that has the most coverage of students at MAH. The restrictions made it necessary to use a framework that was easy to use. Another requirement of the framework was to be able to develop a UI that was both responsive and could be used on multiple devices.

2.3.1 Frameworks considered for developing web applications

Finding out what framework to use can be difficult since

there are multiple different choices to choose from. Using Codeanywhere (Hrgović. 2015) (figure 3), we limited ourselves to three different frameworks, (Angular. 2010), (React. 2013) and (Ember.JS. 2011).

To validate the information from Codeanywhere, we used StackOverflow. StackOverflow is a question- and answer forum where developers can find technical information (Hart and Sarma. 2014). Searching for the different frameworks and getting high results would imply that they are frequently used and might be possible for us to use. On StackOverflow, we specifically looked at how many questions were being asked about the specific framework (figure 4). Searching for AngularJS we also found (Angular 2. 2016) and added it to our consideration. Combining Angular 2, which is the new version of AngularJS, it stood out as the winner of the number of searches. Combined Angular 2 and AngularJS had 272 621 questions asked on StackOverflow, compared to React which had the second most asked questions at 83016. We believe this shows that Angular is used more frequently.

Figure 4: Number of questions asked on StackOverflow regarding our frameworks.

2.3.2 Choosing a possible web framework

Due to the needs to cover the largest number of students at MAH we came up with three important requirements that the framework chosen must have:

• Ease of use • Browser support

Figure 3: Top 10 used JavaScript frameworks as reported by Codeanywhere

• Responsiveness

Ease of use is motivated by the time restraints for the thesis, meaning that the framework must not take too long to learn and should be easy to use or at least have good documentation. Secondly, the need for multiple browser support is motivated by the need to get as much coverage of students as possible. Thirdly, the responsiveness requirement is needed in order to receive as much coverage of students as possible. We have assigned points to the different characteristics ranging from one to three were three is the best and one is the worst. An evaluation of the three requirements is presented in figure 5.

In the ease of use test, we examined the documentation and if there were any developer guides or tutorials available. We found that all frameworks had similar documentation and code guides. Therefore, they all got three points, with the exception of (AngularJS. 2010) since that one missed out on a quick start guide. AngularJS also pointed to (Angular 2. 2016) giving the feeling that Angular 2 was preferred to use, which awarded AngularJS with only one point. All the different frameworks supported Chrome, Safari, Firefox and Internet Explorer with the exception for (Ember.JS. 2011) where we could not find any evidence for support in Chrome. Ember.JS got two points while the rest got three.

The four frameworks that we tested all supported our requirements for the ability to be developed responsively. Therefore, they were awarded three points. Angular 2 and React got the same amount of points, 9 each. We picked Angular 2 because we felt that it was a bit easier to learn, simply by comparing the code examples from both.

Figure 5: Results from the evaluation of requirements on different frameworks.

2.4 Developing with Angular 2

This section describes how we use Angular 2 to develop the application. The application was developed using components and services. Components described in the introduction were used to display the different design elements from Lindstedt’s (2017) design document. Components needed to be implemented from the design document were the homepage, login, character creation, dashboard, account settings, achievements, arena, fight, profile page, user settings, study hall, FAQ, and friends online. Every component listed were explained in detail in the design document, showing the design which was implemented in the HTML and CSS files. The responsive design that we implemented was missing in Lindstedt’s design document, here we the researchers used our knowledge gained from previous courses at Malmö Högskola. The design document also described how the student was supposed to interact with it, which was implemented in the components associated TypeScript file. The interaction in the fight component was described in more detail, due to the need of implementing the game mechanics. Here, Lindstedt’s design document described the

flow of a match, finishing conditions of the game and how the gameplay works. This was done in the fight components TypeScript file.

Services were used in this application when the student needed to send requests to the back-end. These requests were sent using AJAX. Different request that can be sent is an update, show and store character or user data. Services are also used to handle session data from the user or to validate information from the student.

2.5 Gamification elements implemented

This part will describe the different gamification elements that were implemented in the application. Every gamification element implemented can be found in Lindstedt’s (2017) design document. The gamification elements found from the design document where experience points, levels, achievements, a student-specific character with health points (hp), damage points (damage) and a character image, different rewards from handing in school assignments and a game where the students can battle each other using their characters.

Experience points are awarded to the student when he or she hands in an assignment, battle a friend or the boss and wins that game round. Passing an assignment will reward the student with 20 XP that the character uses. Experience points are needed to be able to level up the character. The student starts at level 1 and has the possibility to level up to 50. Every level requires a different amount of experience points and the amount of experience required to level up also increases on the different levels. Every level also comes with new character hp and damage increase (figure 6).

Figure 6: Example from Lindstedt’s design document regarding the level, experience points, health and damage

Leveling up the character comes in handy when battling other students or the boss in the arena. In the arena, the student uses their characters to battle each other, to measure their strengths gained from turning in assignments and being active in school. This feature of the application is meant for the student to have some fun, while still being able to compare their progress in school against other students (figure 7). The more the student study and hand in assignments the better their character gets and can more easily beat the other students.

Figure 7: Displays the arena from Lindstedt design document

Figure 8 shows an example of the achievements the students are able to collect during their time at school. Achievements are one gamification element that is a big motivational aspect of the application. The student is able to collect a lot more of these achievements not included in figure 8 during their study time at Malmö Högskola. The students are also going to be able to use social media to post about their newly gained achievements.

Figure 8: Displays some of the achievements that the student can collect

2.6 Evaluating the application

A test of the application that evaluated the responsiveness was conducted by the researchers. The test includes a set of four mobile phones, two tablets and two versions of desktop computers. The outcome of the responsiveness test was to exclude any graphical anomalies that could appear in the various test devices. A scale of responsiveness was created by using three keywords. The keys used to represent the performance of responsiveness of the various devices were Poor, Acceptable and Outstanding. Outstanding was the highest criteria possible. As a standard for testing, web pages were chosen as benchmarks. As such, this automatically gave web pages the key outstanding. Acceptable, on the other hand, was a medium rating. The rating would be awarded to devices which only showed minimal graphical changes when it came to the benchmark standard. Poor was the lowest rating, given to devices that had large graphical changes from the benchmark standard. A second test was also

conducted to find out if the application worked on different systems, such as Windows, Apple devices, and Android devices and also to find out if it worked on different internet browsers. The browsers tested were Google Chrome, Internet Explorer, Firefox, and Safari.

Clash of the MAH was evaluated by allowing 12 students to test the application during a 10-minute test session. Before they started the testing, they were given a brief explanation of the application and the usage of it. After that, they received information on how to access the application. After the testing, they were asked to fill in a predefined questionnaire. The questionnaire was designed to gather factual data and opinions about the application. The questions asked were both open questions and closed questions. In the section with the closed questions, the tester got to choose from a set of predefined questions. In the section with the open questions, the tester got to fill in their own answer. In this way, we strived at following a suggested technique for questions designed for questionnaires (Oates. 2006).

3 Solution

This section describes the implemented gamified application. First and foremost, we will describe the system in a high-level overview. Secondly, we will present the different components that the application was created with. Furthermore, the solution from the development and the gamification elements will be described by the presentation of illustrative screenshots.

3.1 Overview of the system

Figure 9 displays the overview of the system. Angular 2 was used to create the UI for the students. The UI is created by the use of components as the main way to build and use specific elements and logic on the webpage (Angular 2. 2016). The guard is used to make sure that the student is logged in before accessing certain components whereas the services are used for reuse of non-view related logic that can be used by different components. Angular 2 communicates to the server by using Ajax. The server then responds with a JSON callback. The server also has a connection to a database where we have stored our application data.

Figure 9: Overview of the system, showing a high level of what the system contains

3.2 Components implemented in the application

The application implements the following features specified in the design document, presented by Lindstedt (2017). The application is built with 15 different types of components, displayed to the right in figure 10. The different components from figure 10 were gathered from Lindstedt’s design document. The first component that the student will encounter is the home page, which contains the login component, a welcome message to the student and a button to the FAQ component that displays frequently asked questions. Every component in the application also displays the navbar component, which holds two buttons. One of the buttons is used for logout and the other one for navigating back to the dashboard. The character- creation component is the second page that the student will encounter, but only the first time that they log in to the application. Here, they are asked to choose a character image and a character name (figure 11).

Figure 11: Displays the character creation

When the student has successfully chosen a character, they are redirected to the dashboard component which is displayed in figure 12. The dashboard is the main navigation page that the student encounters after they have logged in. At number (1), the student can navigate to the arena which is displayed in section 3.3. If the student clicks (2), they get redirected to the study hall described in figure 13. Number (3) shows the student's character name, the image of the created character, current health points, and the character's level. A click on the character image takes the student to the profile component where they can handle their user account. Number (4) shows the number of friends’ online (figure 12). By clicking here, the student is directed to a component displaying other students online.

Figure 13 presents the study hall page. This page shows the student’s progress from MAH ́s e-service system. Number (1)

displays where the student can check how many assignments they have turned in, how many assignments they have passed and also how many classes they have attended. Under “user stats”, the

Figure 12: Dashboard of the application, including buttons to the Arena (1), the Study hall (2), the student’s profile (3) and friends online (4)

current experience points are displayed. Number (2) shows two buttons. The first one is the achievements button used to inspect all the achievements that the student has earned, while the second button is a back button directing the student back to the dashboard. Number (3) shows the completed assignments, it includes the course that the student is attending, what type of assignment, the reward for completing it and the date when it needs to be turned in. Lastly, number (4) displays the assignments that the student has to correct, with the same information as in assignments to complete, with the exception of a message from the teacher explaining what is needed to be fixed to be able to pass the assignment.

Figure 13: Study hall, displaying the student’s stats (1), buttons (2), assignments complete (3) and assignments to be corrected (4)

3.3 Gamification elements implemented

In this section, the solution from the implementation of gamification elements in the application will be demonstrated. First presented is our solution from the implementation of achievements then we will present the gaming part of the application. Figures 14-16 show the solution from the gamification elements implemented into the application. An example of a gamification element is the action of receiving a reward when completing a task, the reward being an achievement and experience points.

In figure 14, the achievements component is presented. This is where the student can view their completed and uncompleted achievements. The achievements are displayed with a name, a description on how to acquire the achievement and the reward the student gets. An example is “Play hard, study harder”. In order to collect this achievement, the student needs to hand in their

first assignment in class. This will award the student ten experience points. There is also a social aspect implemented in the achievements, where the student has the possibility to tweet their acquired achievements.

Figure 15 displays the arena page. Inside the arena, the student uses their character to challenge other students in a fight. In number (1) and number (2), the student's character statistics and image of the character are shown. From the arena page, the student can navigate to either fight a friend or fight

Figure 14: Example of achievements in the application

the boss. This is done by pressing one of the buttons to the left of number (3). Further explanation of the fighting is described further down (figure 16).

Figure 16 displays the fight page. The fight is a version of the game rock, paper, and scissors. The student chooses a move by pressing one of the alternatives shown in number (2). The icons on the buttons describe the different moves. The single sword is the strike, Shield icon is the protect and double swords are the haymaker. This is further described in the list below.

• The single sword is a strike attack that deals 95% of the character's base damage. 


• The shield protects the character and reduce 65% damage taken from a strike or reflects 20% of the base damage when blocking a haymaker attack. 


• Double swords are a wild haymaker attack that deals 150% of the character's base damage or misses when being blocked
When both the student and the opponent have chosen a move and the game time is zero, both players ́ actions will be executed.

Depending on the chosen moves, health is deducted from both players ́ health bars. Health bars are displayed in number (1). This then leads to an algorithm that establishes the winner and loser of the fight. Number (3) shows the character image chosen by the student (figure 16).

Figure 16: Fight vs boss or player, this displays the player's health bars (1), user moves (2) and character images (3)

4 Results

First, in the results section, we will display the results we got from our tests regarding the responsiveness in the prototype. After the responsiveness results, we will show the results of our tests that were carried out on the students at MAH. Lastly, we will summarize the results we gathered.

4.1 Responsiveness results from the development

This section will provide the results of the development when it comes to responsiveness and browser support. The tested devices where Mac, iPhone, Windows Computer, Android Phones, and two different tablets and they all passed the working test without any problems. The browsers Google Chrome, Internet Explorer, Firefox and Safari that were tested could also run the application without any problems. Figure 17 shows the results of the responsive test described in section 2.6 made on the application. The result indicates that tablets and mobile phones with larger screens and better resolution got a positive responsive result.

Figure 17: Table of responsive application test

An example of a device that got a high score was the Apple iPhone 5 (figure 18). In this device everything scaled well, everything was placed where it was supposed to, so the user could easily use the game's features as intended. Conversely, mobile phones with smaller screens and lower

resolution had a more significant disadvantage when it came to responsive changes. An example of this was the Samsung Galaxy Y device (figure 18). In this figure, you can clearly see that the images on the buttons stops following their containers shapes and start to move away from the buttons. Also, everything that should be displayed on the screen isn't there, the student would have to scroll to be able to use all of the game's features, which is not practical when using the device.

Figure 18: Displays Samsung Y and iPhone in the responsiveness test

4.2 Results of the questionnaire

This section will provide the results of the questionnaire with tables and outtakes from the answers given by the students. The tests were conducted on both mobile phones and computers, on a variety of students from different areas at Malmö Högskola. The majority of the test group studied a computer science-centric education. The other test subjects came from a more social oriented education, like International Relations and a dentist program. The results of the 12 students who answered the questionnaire are shown in the tables below. The table shows the Likert scale part of the questionnaire. The results will not specify which kind of device each student used. Instead, all the results will be analyzed together. Neither will the gender of the students be specified. The questionnaires were answered by students that were between 20 - 32 years old.

Tabell 1: Likert scale results of questionnaire

The quantitative results of the questionnaire (table 1) showed that the students felt the user interface intuitive. With nine students scoring a four or higher in this field, which were 75% of the students. Only three students felt that the user interface was more or less intuitive. The results of the question “Did you find the game fun?” showed that the students felt that the game was fun with 58.3% scoring it four and higher. Three students scored this question with a number below three. Only one student felt more or less that the game was fun. The results of the motivation question showed that six students felt more motivated, three felt more or less motivated and three felt no difference in motivation from before using the application. This resulted in that 50% of the students felt more motivated, 25% more or less and 25% did not feel any motivational change when they had used the application.

The results of the qualitative questions gave some interesting feedback. One student said “If this was a part of my time at Malmö Högskola I would probably do better than I am. I like grinding games and becoming overpowered, so I would hand in assignments to get overpowered.” Another also said, “I love winning and my performance in school makes me win more!” Another student said, “Since the game was boring I could not care less about doing assignments to improve my character progression.” This fell in line with another quote that said, “Do not understand why there is a game in the middle of everything.”

The results of the achievements part of the questionnaire gave some interesting results. One student said “Achievements is something that makes me work extra for it. It is nice to achieve things it makes you feel good.” Another said, “I would like more achievements for performing well in

school.” Another student said, “Do not care about them”. The achievements question showed that the majority of students felt that achievements were something that could improve their results and motivate them to hand in assignments. This is clearly shown in the statement from one student. The student said “I really enjoy the idea that I could share my progress on twitter. I like showing off, especially when it is things I can feel proud about, like handing in assignments and such. Sticking it to the slackers.”

The results of the question if the students could compare our application against their current e-learning platform (itslearning) showed that our application was accepted with great encouragement. One student said “I would rather use this application in everyday life. It appears to be quicker nicer and more efficient”, another student said this, “love the game”. But there were also students that had other opinions on which application they would use. An example is given by a student “It’s Learning, Easier to understand” and by another student “It’s Learning has a better overview.” When it came to the question if they could compare It’s Learning to our application and that they provide the same features. Students stated that our application had the majority of features which is included in itslearning. An example of a feature that was missing according to several students was the teacher bulletin. Another student also pointed to the connection aspect of their current e-learning platform as something missing in the application. One student stated, “I am not sure, but I do not think I could get in contact with others, like a teacher and other students in your service.”

We then had a question in the end where the students could give additional insight into the application. This is what some of the students said “It is a nice concept. I really think that gamification could improve the students’ motivation and improve their grades. Especially if it was possible to create guilds and high score boards.” Another student said “Do not have the game there. Have motivational comments when turning in assignments instead.”

The answers from the students also gave a great indication of the state of the application when it came to the development. The students uncovered some small bug in the application that will be investigated in later iterations of the application. They also gave productive feedback on what they wanted to make it a more rewarding experience. One student said, “I want a shorter amount of time for each turn.” This was supported by another student that said “The game was a bit slow. 10 seconds felt like an eternity.”

4.3 Summary of the questionnaire results

The results of the questioner indicate that a majority of students felt that the application was something worthwhile and it had a positive effect on them. The results of the achievements question indicated that the students felt that achievements were a nice feature that could improve their motivation to complete assignments and work harder to achieve all the set of achievements. The results from the quantitative questions also indicate that this application is a viable substitute for the current e-learning platform at Malmö Högskola. With a majority of the student feeling that they would rather use the application over itslearning. The overall results showed that the majority of the test group was positive about the application.

5 Analysis and discussion

In this section, we will we will go through our research questions and analyze what we have done to answer them. We will discuss our results found in section 4 and how the results answer our research questions.

5.1 Gamified application = more motivation?

This part will describe what we found out during our research on the subject of the creation of a gamified application. Here we will analyze and discuss our results to be able to answer our first research question, how can a gamified application help motivate students at higher education institutions? In order to answer our first question, we created a prototype application. We started by doing a comprehensive analysis of different frameworks to find out what type of framework that would suit our demands in the creation of a gamified application. We found out that Angular 2 was one framework that fulfilled our requirements in the creation aspect of the gamified application.

We then incorporate gamification elements that Lindstedt (2017) design document provided us with. We choose to use Lindstedt’s design and gamification elements because its purpose was to gather good design principles and provide it to us in our implementation of the application. This work from Lindstedt was done to help us answer the motivational question and gave us the opportunity to fully focus on the development part. From the design document we implemented, achievements, experience points, levels, and characters. If we had done our own design research, we might have missed some important design features or gamification elements. One can argue that if we had done the design research, the results may have changed for the better or to the worse, it is hard to tell the outcome. But if we hadn't had time to implement the crucial features of the application it might not have been as good as it is, and the implementations of the application might have suffered from more bugs and not being functional enough to conduct tests on. The results from this work described above are displayed in chapter three, here we display our work done on the creation of the prototype application.

To be able to answer the first research question the application was tested on 12 different students from Malmö Högskola. The students got to spend 10 minutes on testing the application and the results are displayed in more depth in chapter four. The results showed the potential of our application and the use of gamification for increasing the motivation of students. The increasing motivational aspect of gamification is strengthened by (Utomo and Santoso. 2015), (Ibáñez et al. 2014) and (Barata et al. 2013). They all got positive results in increasing student motivation, engagement, and dedication. Krause et al. (2015) also indicate in their results that gamification reduces the retention rates in half. Everything mentioned in this analysis is a good indication of that a gamified application is a possible way to fight decreasing motivation for students at higher educational institutions. However, this result may have differed if we had performed this test on students during a course or semester. If we had performed a longer test study, it would also have given the results more credibility and probably more feedback on the prototype. Which could have been used in later iterations on improving the prototype application and giving it more functionality.

5.2 Possible to implement a gamified e-learning system?

The second research question was, how can the services that are provided by the student’s current e-learning system be implemented in a gamified application? To be able to answer this question we had to implement some similarities with the current e-service, itslearning. In collaboration with Lindstedt’s (2017) design document, we set out to implement a study hall. This is the part of the application that has a resemblance to itslearning. Here the student can hand in their assignments and check what assignments they have to correct or are approved on. This is also connected to the gamification part where the user gets experience points and achievements when completing assignments.

When we examined the results from our tests, the students felt that the application was similar or close to their current e-service. Most of the students even preferred to use our application instead of itslearning. However, some students felt that is was missing some key features that weren't implemented in the gamified prototype. According to some students, these missing key features were the information from teachers, course documents, and course information. Since the point of this

prototype not is to fully replace its learning it is expected not to have all the features that itslearning has implemented. If this study would have been carried out during a longer time period, we are sure that the students would have found more features they felt were missing. But at this time this tool's purpose is to be a complement to itslearning and to be used in correlation with itslearning.

With all this information in mind, we feel that we can answer our second research question and that it is possible to create a gamified application with features incorporated from itslearning. These results showed that it is possible for a gamified application to provide the same services as itslearning or any other student e-service tool. As stated before our research shows that itslearning has not been the best e-service tool to use since most students that tested the contribution presented in this work preferred it instead of itslearning. So, we believe that anyone interested in gamifying their e-service or using gamification, in general, can learn from this work presented.

5.3 Did we get good user coverage?

The third research question was, how can a cross-platform gamified application be developed in order to have maximum user coverage? To answer this question, we investigated frameworks in different areas of development. The areas we investigated were native application and mobile applications. With our findings, we could see that a web application had the possibility to obtain the highest user coverage. The results from our study showed that the application could be used on all modern media devices, from mobile phones to ordinary computers. This includes different operating systems like Windows, IOS, and Android. The reason for developing our application towards a web framework became relevant by analyzing data of the most commonly used mobile devices in Sweden (Internetstiftelsen i Sverige. 2016).

To get these results we did a test on multiple devices to see if the usage of the application as possible. The tests were performed by testing the application on a Windows computer, a Mac computer, IOS devices and Android devices using different web browsers. Here we could see that developing using a web-centric approach made it possible to run the application on both operating systems, also on Android and iOS devices without developing for different native applications. The drawback of developing the application in this way was that no native specific extras could be used (Jobe. 2013). By native specific extras, we mean notification in the mobile as an example. If the literature would have shown a larger disparity between mobile usages, a consideration towards making a native application might have been made. But because the mobile usage in Sweden is divided into blocks, Android and iOS the choice became clear that a web approach would give us the largest user coverage possible. The results also showed that developing with a web framework made it possible for the students to easily open the application with just internet access and a web browser.

6 Conclusions and propositions for further research

This study is based on the desire to help motivate students, with the use of gamification. This study is also trying to answer if it is possible to use gamification in an e-service environment. Ourambitionofthisstudyistoprovideresearchifgamificationhasanyimpact on student’s motivation and also provide guidelines on how to implement an e-service that uses gamification.

In this study, we present Clash of MAH. A gamification application prototype that can be used to help motivate students. With the help of Clash of MAH, the students can check their assignments from their different classes, hand in their assignments and correct assignments. Just like with itslearning, but with the difference that they will feel more motivated when using our prototype. Because of the gamification aspects implemented in the application. The study consists of students that have tested the prototype during a set time period of 10 minutes and the results are based on their feedback.

The contribution we leave to the science is the developed application which contains gamification elements and guidelines on what to consider when developing a gamified application. We also contribute to the results we got from our tests carried out on the students at Malmö Högskola. Although the results we present are promising, we recognize that the system that is developed needs to be tested during a longer time period and this will be discussed in the propositions for further research.

Propositions for further research would be to do a longer study on this subject to really investigate if this gamified application increases the motivation of students. The best would be to conduct this study during a course or even better a term and with more students involved in the tests. There also needs to be a defined base condition, which the results can be compared against. The tests should be carried out on multiple groups of students, some groups that use the gamified application and other groups that only uses the itslearning e-service during a set time period. The results that would be presented from a long-time study would be a better indication if gamification actually can motivate students during a longer time period, which our study does not answer. There is also work that can be done to fully develop the presented prototype to cover all the functionality offered by the current e-learning system, with the addition of the presented gamification model that, according to the results shown, may help increasing student motivation during the complete higher education period. We believe that gamification is one possible way to motivate students, but more research on the subject is needed to fully validate how well it works and when to use it.

Bibliography list

Angular 2. One framework. - Angular.https://angular.io (Accessed 2017-03-20).

AngularJS. Super heroic JavaScript MVW Framework. https://angularjs.org (Accessed 2017-03-20). Barata, G, Gama, S, Jorge, J, Goncalves, D, 2013. Engaging Engineering Students with Gamification, in: 2013 5th International Conference on Games and Virtual Worlds for Serious Applications (VS-GAMES).

Bartel, A and Hagel, G. 2014. Engaging students with a mobile game-based learning system in university education. Global Engineering Education Conference (EDUCON), 2014 IEEE. IEEE, pp. 957–960.

Class craft. Why gamification?https://www.classcraft.com/gamification/ (Accessed 2017-02-16). Davidsson, P and Findahl, O. 2016. Internetstiftelsen i Sverige, Svenskarna och internet 2016. http://www.internetstatistik.se/rapporter/svenskarna-och-internet-2016/ (Accessed 2017-02-24). Deterding, S., Dixon, D and Khaled, R. 2011. From game design elements to gamefulness: defining gamification, Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments, 2011, pp. 9–15.

Ember.js. A framework for creating ambitious web applications. https://emberjs.com (Accessed 2017-03-20).

Fotaris, P., Mastoras, T., Leinfellner, R and Rosunally, Y. 2016. Climbing up the Leaderboard: An Empirical Study of Applying Gamification Techniques to a Computer Programming Class. Electronic Journal of e-Learning 14, 94–110.

GitHub. 2017. Rep for creation of app for gamified learning.

https://github.com/andersfrid/exjobb (Accessed 2017-04-06).

Hakulinen, L., Auvinen, T and Korhonen, A. 2015. The Effect of Achievement Badges on Students’ Behavior: An Empirical Study in a University-Level Computer Science Course. iJET 10, 18–29. Hart, K and Sarma, A. 2014. Perceptions of Answer Quality in an Online Technical Question and Answer Forum, Proceedings of the 7th International Workshop on Cooperative and Human Aspects of Software Engineering, CHASE 2014. ACM, New York, NY, USA, pp. 103–106. doi:10.1145/2593702.2593703.

Hrgović, K. Codeanywhere. 2015. Top 10 Most Used JavaScript Frameworks.

https://blog.codeanywhere.com/top-10-most-used-javascript-frameworks/ (Accessed 2017-03-29). Ibáñez, M.-B., Di-Serio, A and Delgado-Kloos, C. 2014. Gamification for engaging computer science students in learning activities: A case study. IEEE Transactions on Learning Technologies 7, 291– 301.

Jobe, W. 2013. Native Apps Vs. Mobile Web Apps. International Journal of Interactive Mobile Technologies (iJIM) 27. doi:10.3991/ijim.v7i4.3226.

Joorabchi, M.E. Mesbah, A and Kruchten, P. 2013. Real Challenges in Mobile App Development, 2013 ACM / IEEE International Symposium on Empirical Software Engineering and Measurement. Presented at the 2013 ACM / IEEE International Symposium on Empirical Software Engineering and Measurement, pp. 15–24. doi:10.1109/ESEM.2013.9.

Krause, M. Mogalle, M. Pohl, H. Williams, J.J. 2015. A Playful Game Changer: Fostering Student Retention in Online Education with Social Gamification, in: Proceedings of the Second (2015) ACM Conference on Learning @ Scale, L@S’15. ACM, New York, NY, USA, pp. 95–102.

Lara, J.A., Lizcano, D., Martínez, M.A., Pazos, J and Riera, T. 2014. A system for knowledge discovery in e-learning environments within the European Higher Education Area – Application to student data from Open University of Madrid, UDIMA. Computers & Education 72, 23–36. doi:10.1016/j.compedu.2013.10.009.

Lindstedt, M. 2017. Best practice free-to-play game mechanics: Impact on gamification of higher

education. Malmö Högskola.

Llorens-Largo, F., Gallego-Durán, F.J., Villagrá-Arnedo, C.J., Compañ-Rosique, P., Satorre-Cuerda, R and Molina-Carmona, R. 2016. Gamification of the Learning Process: Lessons Learned. IEEE Revista Iberoamericana de Tecnologias del Aprendizaje 11, 227–234.

McLeod, S. A. (2008). Likert Scale. www.simplypsychology.org/likert-scale.html (Accessed 2017-03-24).

Mihaela Cojocariu, V., Lazar, L and Lazar, G. 2016. The Ambivalence of Strengths and Weaknesses of E-Learning Educational Services. Brain: Broad Research in Artificial Intelligence and Neuroscience, Vol 7, Iss 3, Pp 55-74 (2016) 55.

Oates, Briony J. 2006. Researching Information Systems and Computing. London: SAGE publications Ltd.

React. A JavaScript library for building user interfaces. https://facebook.github.io/react/ (Accessed 2017-03-20).

Saurabh, Pal. 2012. Mining Educational Data to Reduce Dropout Rates of Engineering Students. International Journal of Information Engineering and Electronic Business, Vol 4, Iss 2, Pp 1–7 (2012) 1.

Singer, N., 2016. Kahoot App Brings Urgency of a Quiz Show to the Classroom. The New York Times.

Tinto, V. 2006. Research and practice of student retention: What next? Journal of College Student Retention: Research, Theory and Practice 8, 1–19.

Utomo, A.Y., Santoso, H.B., 2015. Development of gamification-enriched pedagogical agent for e-Learning system based on community of inquiry, in: Proceedings of the International HCI and UX Conference in Indonesia. ACM, pp. 1–9.

Vaishnavi, V and Kuechler, B. 2004. Design Science Research in Information Systems http://desrist.org/desrist/content/design-science-research-in-information-systems.pdf (Accessed 2017-03-12).

Williams, L., Kessler, R.R., Cunningham, W and Jeffries, R. 2000. Strengthening the case for pair programming. IEEE Software 17, 19–25. doi:10.1109/52.854064.