On the development of an educational math game

(1)

Examensarbete

15 högskolepoäng, grundnivå

On the development of an educational math game

En studie i skapandet av ett lärandespel i matematik

Sebastian Hovenberg

Ehab Okal

Fakulteten för teknik och samhälle

Datavetenskap

Examen: Kandidatexamen 180hp Handledare: Mia Persson

Huvudområde: Datavetenskap Andrabedömare:Gion Koch Svedberg Program: Spelutveckling

(2)

(3)

Abstract

There are many different ways to implement learning activities in the modern

educational system as seen in schools nowadays. But in an era rapidly being digitized, the educational system has possibly not really managed to catch up or succeed in implementing these digitalization’s in a meaningful and effective way. This inability to digitize has been growing more prevalent by each passing year with droppings in mathematical performance, as shown in the latest result for the tests carried out by PISA [1].

Out of this enlightenment, the Swedish government set out funding research projects in order to understand and examine the reasons behind this drop. But this study will not focus on the PISA result. In this study, we investigate whether educational games could be an answer to negate or halt these dropping in performance.

In particular, the main aim of this study is to design and implement an educational game centered on solving mathematical problems in a new way not often seen in educational games, by implementing design choices mostly seen in highly developed video games.

The results of this study show that there is some evidence that confirms that more highly developed games could be an excellent way of learning and possibly also the next evolution in the educational system seen in schools. However, our results also show that the educational part of such an educational math game really has to be flexible and challenging enough for the players or students in order to have them coming back to the game and learn more.

(4)

(5)

Sammanfattning

Det finns många olika sätt att skapa inlärningsaktiviteter i det moderna

utbildningssystemet. Men i en tid som snabbt digitaliseras har utbildningssystemet kanske inte alltid lyckats med att implementera digitaliseringen på ett meningsfullt och effektivt sätt. Denna oförmåga att digitalisera har blivit alltmer utbredd för varje år som passerar med en minskning i matematisk prestanda hos studenterna, vilket tydligt framgår av senaste resultaten i PISA [1].

Utifrån ovannämnda observationer utarbetade den svenska regeringen ett finansierat forskningsprojekt för att förstå och undersöka orsakerna till denna nedgång i

matematisk prestanda. Denna studie kommer dock inte att fokusera på

PISA-resultatet utan istället kommer vi att undersöka huruvida pedagogiska spel kan vara ett möjligt sätt att hantera ovannämnda nedgång i matematiska prestanda.

Syftet med denna studie är att skapa ett pedagogiskt spel som är centrerat på att lösa matematiska problem på ett nytt sätt, närmare bestämt på ett sätt som oftast inte ses i pedagogiska spel; detta kommer att göras genom att implementera designmönster som oftast ses i högutvecklade videospel.

Resultaten av denna studie visar att det finns bevis som bekräftar att mer

högutvecklade spel kan vara ett bra sätt att lära sig, och därmed eventuellt kanske nästa steg i skolsystemets utveckling. Resultatet visar dock även att den pedagogiska delen av spelet måste vara flexibel och utmanande nog för att få spelaren eller

(6)

(7)

In light of the aforementioned issue, an idea was born to investigate whether it is possible to create an educational math game that could serve as a helping tool for students. There are already thousands of learning games available to all kinds of platforms, yet there are almost none that is compatible to use in free form in education. Moreover, almost every single learning games are designed to a specific topic, mostly math and language. However, there are some entertaining games that have stories with adventures combined with the gameplay which have been proved to drive the players wanting to learn more [2, 4, 7]. But the aforementioned games are still very restricted and can only be used in few areas.

In an earlier study, Rowe et al. [6] conducted an experiment of creating an educational game called Crystal Island with a focus on narrative-centered learning environments, which was conducted to help the students to learn more about biology. By using both the post and pre-tests the research team could then see if the students managed to learn something by playing the game. The findings of the study show and promotes that with a design of a broadly effective gameplay activities for narrative-centered learning environments such as investigations, being able to talk to Non-Playable Characters(NPC) and trying science equipment’s contributes to an effective problem solving mentality, and greatly improves the learning outcome and sustained engagement for all the students.

Moreover, in [3] Gee arguments that good games do not have to be educational for being an educational game. Instead, the game only has to be self-explained in such a way that the player can understand the fundamental on how the game can be played and thus being able to achieve the skills and knowledge for completing the progressive harder task [3,4].

In another study conducted by Gee (see [7]), he argues that it is possible to create an educational game that could benefit for schools and that the most and foremost part that video games can contribute with here is that they can be more forgiving if a player fails to complete a task. It is much quicker to restart a video game and retry until it is completed than handing in an essay or a test that could take up to a week to be checked. Or, as Gee formulates the derived conclusions in [7]:

“The cutting edge is realizing the potential of games for learning by building good games into good learning systems in and out of classrooms and by building the good learning principles in good games into learning in and out of school whether or not a video game is present. “

(12)

2 1.2 Main aim of our study

In this study, we investigate whether educational math games could be a solution for negating the aforementioned drop in the Swedish PISA results.

In particular, the main aim of this study is to identify and obtain a deeper understanding of appropriate design choices while creating an educational math computer game, and how our choices here actually impact a player´s overall satisfaction and interest to continue playing the game.

Specifically, in this study we will seek the answer to the following question:



Is it possible to develop an educational math computer game that is both

satisfying and interesting for the player to play?

Note that from the aforementioned research question, it follows that one of the objectives of our study will be to propose a suitable design pattern to create a game that are both satisfying, interesting and educational in the same time for a player.

In particular, note that in order to achieve the aforementioned aim, we will need to develop a new educational math game, implementing appropriate design patterns, well known to facilitating the educational, satisfying and interesting aspects in a game. And, as a next step, evaluate whether our proposed game actually fulfills the

aforementioned so called non-functional requirements. During this evaluation phase, we will collect both qualitative and quantitative data from observations of how the educational elements in our proposed game where perceived by a set of test participants, and how it affected their overall judgment with respect to the educational, satisfaction and interest aspects of the game. We will collect the aforementioned data by observing and recording the behavior of the participants testing our new educational math game, and also by collecting additional data in form of a survey.

1.3 Audience

Intended audience for this study are video game developers, video game researcher but also researchers in education who are interested in the implementation of video games as a possible learning tool in classrooms. Our intention is that the reader of this study should be able to understand the structure, methods, models and frameworks used for the design of the research process, how and why the game was implemented in a certain fashion and finally how the results of the study was collected and evaluated in order to integrate these findings into future works and iterations of the created

(13)

3 1.4 Delimitations

Our objective is not to go through all kinds of game designs, but rather to propose one possible design promoting the aforementioned aspects in an educational math game. Also, our proposed educational game restricts to a PC platform game, since other platform’s possible restrictions in game controls would not allow our game to be played as we wanted it to be played (this observation will be discussed in detail in upcoming chapters).

Worth mentioning here is also that we are not investigating any gains or abilities in mathematical proficiency among our test participants, since the main focus of our study is investigate how our test participants reacted and perceived on the design choices and patterns of our proposed educational game.

(14)

4

2. Theoretical background

2.1 Important elements for our study

Since our study is aimed both towards video game developers and teachers that are either interested in the game development process or the generated results from the evaluation of our proposed educational game, it is important to address the elements and framework that were used to structure our study, and our developed educational game herein. In the rest of this chapter, we will subsequently describe three elements that are particularly important during the rest of our study.

2.1.1 Previous results

The first important element used in our study are some related research results generated by Gee [4, 7, 20]. In particular, Gee has dedicated a lot of research in order to establish a design framework that can be used in video game development in order to enhance the learning experience of a game. Moreover, Gee also argues how video games can be a useful tool supplementing traditional educational system. In particular, in our study we will use two earlier research studies by Gee; one that addresses six reasons to why video games are good for learning [7], the other provides 16 guidelines recommended to follow when creating an educational game [4].

2.1.2 Mechanics-Dynamics-Aesthetics (MDA) framework

The second important element used in our study is the

Mechanics-Dynamics-Aesthetics framework (MDA framework for short), described as follows in [16]: “MDA is a formal approach to understanding games - one which attempts to bridge the gap between game design and development, game criticism and technical game research.” The MDA framework was developed at Game Design and Tuning Working at the Game Developers Conference at San Jose 2001- 2004, and the main aim of this methodology is to a have a more formal approach regarding the subject of game research [16]. In particular, the purpose of this methodology is to clarify and strengthen the iterative processes for developers, scholars and researches alike, which allows the studying, decomposing and finally the designing of a broad array of game designs and game artifacts [16].

MDA is an abbreviation for:

 Mechanics: The components that stands for data representation and

algorithms. For example, pressing the space bar will result in the character jumping a certain height and landing back on the ground through an algorithm that simulates gravity.

 Dynamics: Is the run-time behavior of the mechanics acting on the player’s inputs and the output it generates over time. For example, when the player presses the shoot button the game character fires of the gun.

 Aesthetics: Is the emotional response the game evokes in the player. For example, there is an emotional response in a game scenery such as a lush forest or a post-apocalyptic wasteland.

(15)

5 2.1.3 Design patterns

The third important element used in this study is the implementation of design patterns [19], which is a formal approach used during the development and testing of different game designs during the development phase of a video game. Design patterns can be described as follows [19]:

“Patterns are simple collections of reusable solutions to solve recurring problems.” Design patterns are traditionally expressions of problem-oriented thinking, with each pattern being described as a problem which occurs often in a certain environment. The aim of design patterns is to find a solution to the described problem in a way that can be reapplied “millions of times”, without ever doing it the same way twice [19].

The reason behind using design patterns is to establish and provide the development team with a shared design vocabulary which allows for [19]:

 Better communication among the development team.

 Documentation of the insights of the development team, and organization of individual experience as written knowledge.

 The analysis of the design choices of the development team, as well as the design choices of others, with the purpose of conducting comparative criticism, re-engineering or maintenance.

The documentation of design patterns is usually performed by using a template containing the following four essential elements [19]:

1. Name: In order to make the shared vocabulary among the development team easy to understand, it is important to name the created pattern.

2. Problem: A description of the problem, including the inherent trade-offs and the context in which the problem occurs.

3. Solution: A description of the solution to the problem, and the general arrangement of entireties and mechanisms that can be used to solve the problem.

4. Consequences: Each solution has its own trade-offs and consequences, with some solutions either causing other problems or amplify existing ones.

Therefore, is it important to balance the costs and benefits, and also compare other solutions, before making a design decision.

In this study, design patterns promoting the aspects satisfying and interesting in the context of video games are of particular interest to us. Note that the definitions of the aforementioned aspects satisfying and interesting for this study are derived from a series of design patterns [19] commonly seen in well renowned and highly developed video games, and are listed as 16 different design aspects as mentioned in James Paul Gee’s article Good video game and good learning [4]. Additionally, the MDA framework [16] will also be used to define the aforementioned terms, since this framework is used to developed and improve video games in a way that is both satisfying and interesting for the player playing the game.

(16)

6

The main focus of the aforementioned design patterns is to catch the attention of the players and drive them to continue on playing the game until completion (note the relation to aforementioned design aspects satisfaction and interest).

The following design patterns are well known for promoting the aforementioned aspects [16]:

- Well implemented and meaningful game mechanics are considered to be the most important design pattern when developing video games, since video games as a medium are centered on the interaction between the player and the game. Therefore, it is important to convey a meaning behind each action.

- Interesting and satisfying game story without any plot holes. Video games are in many ways just like books. They have a beginning, protagonist, antagonist, clash of interest, transitions and an ending, but unlike the more passive notion seen in books (as described by Plato in Phaedrus [4, 45]), games reflects the actions and decisions of the players, and thus, allows the player to become both the reader and the writer of the story. - Immersive and eye catching game world. This design pattern leans toward

the aesthetic part of video game development and plays a significant role in setting the scene of the story of the game. This design pattern is the style in which the game projects itself to the player, i.e., 3d / pixels.

- Clever and intuitive gameplay is closely linked to game mechanics but instead addresses the design of the puzzle elements and their solutions.

For a detailed description on how we actually deploy design patterns in our study, see chapter 3.

2.1.4 On the relevance of aesthetics in video games

The aesthetics in video games is an important aspect when it comes to developing a game that is considered “fun” by the end user [16]. But instead of using the word “fun” in our study, we will use the more directed vocabulary from [16]:

1. Sensation Game as sense-pleasure

2. Fantasy Game as make-believe

3. Narrative Game as drama

4. Challenge Game as obstacle course

(17)

7 6. Discovery Game as uncharted territory

7. Expression Game as self-discovery 8. Submission Game as pastime.

As for the educational game for this study it will consist of: Sensation, Fantasy, Discovery, Companionship and Challenge, see 2.1.5 below for a more detailed

explanation of these keywords. Note that the aforementioned keywords will be used as a compass in order to define our gameplay and describe the dynamic and mechanics aspect of our gameplay.

2.1.5 Dynamics

The design choices for the educational game need to be evaluated in order to establish their usefulness and adaptability with the general vision and design plan of the game, dynamics work to create aesthetic experiences for the game. Therefore, we will

continuously evaluate our developed artefacts:

 Sensation: The theme for the game world need to be made in fashion that directly catch the players interest, that is why the design of the game world will be focused on immersing the player in a world filled with mysteries and interesting visual effects that will serve as eye catchers [16].

 Fantasy: In order to supplement the sensation aspect for game play, the setting and the design for the game will be set in a fantasy world. This enables more freedom since the game world does not need to adhere to a certain set of worldly rules and law.

 Discovery: One way to catch the player’s interest is to create an environment that encourage the player to explore and discover in a mysterious and uncharted game world [16]. Our assumption here which will later be

evaluated is that this will drive the player to continue on to play the game and not run away after realizing it is a math game.

 Companionship: Is just like Fellowship but instead of interacting with fellow player in the game world, this game will instead have a companion that will follow and help the player.

 Challenge: The challenge aspect for this game will be the mathematical questions that need to be solved in in order to progress forward in the game, the solving of the math problems will not be made in the traditional fashion, i.e., inputting numbers directly to solve the problem. Instead it will be made in puzzle fashion by lifting or moving cubes with numbers on them, this is in order to make the gameplay a little more interesting.

(18)

8 2.1.6 Mechanics

The mechanics for the educational game are the various actions, behaviors and control mechanisms. The mechanics available for the player will be the standard control functions as seen in nearly all modern first person shooter, i.e. the ability to move around, jump and interact with game object either by lifting them or pushing them. Regarding the control mechanisms for the game it will be based on a first- person perspective in a 3d world and controlled with a keyboard and mouse just as the mentioned inspiration for this game.

(19)

9

3. Method

Our study will rely on the following elements and methods/approaches:

 Worldview

 Design science

 Mixed methods methodology

3.1 Worldview

Worldview (or philosophical foundation) has been described as “a basic set of beliefs that guide action” [21]. In particular, it is how the researcher and author view the world around them, and also how they react and perceive the different elements in that certain world. In essence, these views draw from a mixture of different

assumptions and variables such as predetermined facts, past experience, academic standards and ideals [14]. Many different views have been formed to better understand and address the different problems that are at hand and in our study, we will deploy the well-known so called pragmatic worldview which can be described as follows:

“Instead of focusing on the methods, researchers emphasize the research problem and use all approaches available to understand the problem (Rossman et al. [13]).”

Furthermore, the pragmatic worldview is generally used to address “The consequences of a series of actions, problem-centred or real-world practice oriented” [14].

Based on our analysis and planning for this study, we realized that the

aforementioned pragmatic worldview was the most suitable option for our study since, as mentioned above, it allows the mixture of both quantitative and qualitative

methods, conducted in either sequential or concurrent ways [13]. Hence, this gives us the freedom to choose any method, technique, or procedures of research that best suits the needs and purpose of the study. In other word, we will be able to focus on which mixture of methods that provides the best understanding needed to investigate and solve our research question under consideration [14].

However, there are some disadvantages with using a pragmatic worldview; one is how to come up with a reasonable mixture between quantitative and qualitative

assumptions, because of the difference in their data collection, interpretation and conclusion. Another problem is when trying to connect quantitative data, which is numeric based, with qualitative data obtained from observations or interviews, and still manage to develop an acceptable analysis and interpretation of the combined data. Moreover, this view is open-ended which can become an issue when planning the research [14].

(20)

10 3.2 Design science

The approach and structure of our study will be based on design science [12] for information systems, with a MDA view or lens [16] for evaluating our design choices during development of our educational game. Furthermore, in order to collect and analyze the necessary test data obtained from the evaluation of our proposed game a concurrent mixed methodology [14] will be employed.

Van Aken et al. describe design science research as follows in [9]:

“Design science research can be defined as research based on the approach of the design sciences, that is, research that develops a valid general knowledge to solve field problem. “

Design science research (see, e.g., [9, 12, 13, 25, 26]) is a problem-solving process, often used in development and improvement of information systems, with its framework mainly used for conducting research in areas such as engineering and computing [13]. In particular, the design science framework focuses on the relevance of the research problem under consideration and its subsequent contribution to related subject, since the main outcome of this framework is forwarding the

development (or improvement) of a certain information system or a process related to a technical domain [25]. Since the goal of design science is utility and moreover, since our study is tailored towards information system such as video games, this approach is considered suitable for our study.

In [26], Peffers et al. was the first to provide the following general guidelines for design science practitioners: 1. Design of artifact 2. Problem relevance 3. Design evaluation 4. Research contributions 5. Research rigor

6. Design as a search process 7. Communication of research

Out of the aforementioned guidelines, Offermann et al. [25] further defined and evolved the process of design science into two work packages; one intended for publication and another self-contained, and together they are summarized into the following three subparts: problem identification, solution design, and evaluation.

In our study, we will deploy the aforementioned recommended structure by Offermann et al. [25], together with a specialized publication work package with the MDA

(21)

11

See Fig.1 below for summary of our deployed research method.

Figure 1: The specialized publication work package with MDA framework and design patterns, which summarizes and illustrates the deployed research method of our study.

3.3 Problem identification

3.3.1 Hypotheses and research question

As stated above, the research question of our study can be formulated as follows:



Is it possible to develop an educational math computer game that is both

(22)

12

From the aforementioned research question, we derive the following hypotheses:  Is there a relationship between the average time spent playing video game per

week and the completion time of our game? (Relationship)

Note that the aim of this hypothesis is to investigate whether our developed educational game requires any past gaming knowledge in order to understand how to play it. The intention of our developed game is to be playable without any gaming experience.

 Does the age of a participant play a role in the overall performance and ability to complete the game? (Relationship)

Note that the aim of this hypothesis is quite similar to the aforementioned one, but here we investigate whether the age of the test participant plays a factor. This in order, as mentioned above, investigate whether our developed game is playable regardless of age.

 Have we managed to create a demonstration game that takes approximately 15-25 min to complete?

Since a testing/presentation session generally is between 15-25 minutes, the aim of this hypothesis is to investigate whether it is possible for us to develop such a game demo, fulfilling all the aforementioned constraints (e.g., a satisfying and interesting educational math game).

 How did the participants experience our game in comparison with their past experiences with other educational game? (Relationship)

Note that this hypothesis is included in order to investigate our test participant’s experiences with our proposed game, in relation to their past experiences with educational games.

 Did the choice of theme, design and gameplay style make our game unique and interesting for the players? (Supporting the research question.)

Note that the main aim of this hypothesis is to investigate whether our selected design patterns actually contribute to an interesting and satisfying educational math game. Also note that since our game is developed in fashion akin to the aforementioned inspirational games, it is important for us to establish that our proposed game is viewed as a unique contribution to existing games, even though our developed game employs similar game mechanics.

Furthermore, it is also important to note that the answers to the aforementioned hypotheses will provide us with valuable information whether the age and earlier gaming experience of our test participants have an impact on their performance, and also experience, of our proposed game. A knowledge about the general composition of our test sample, and any relationship between certain factors within our study, is important when it comes to handle potential error sources in our obtained primary data.

(23)

13 3.3.2 Literature study

The main aim of our literature study is to provide us with an in-depth insight into previously research results related to our posed research question (see section 3.3.1).

Our literature study was conducted as follows. We started by first searching for related research studies relevant to our subject under consideration. Here we were using the following online databases: Google scholar, ACM digital library, and IEEE Xplore. Moreover, we use the following search terms for our search: games, game and math, mathematical games, design pattern, educational games, learning, students, and digital education. Then, by skimming over the abstracts, introductions and discussions sections of our selected research articles, we decide whether the articles were relevant for our research study. If so, a continuation was made based on the reference list of the article; this is in order to obtain more information regarding the subject the study was addressing.

In particular, our literature study also provided us with several articles addressing design science [9, 13, 25, 26], hence providing us with valuable guidelines on how to structure the development process of an information systems (such as video games) in a research study.

Moreover, in order to develop an educational game that is convinced as both satisfying and interesting by a player, articles addressing the development (and improvements) of design patterns were selected for further investigation. In particular, design patterns [19] and the MDA framework [16] were used for our study. Moreover, several articles by James Paul Gee (see, e.g., [3, 4, 7]) were studied to understand why and how video games should be used as an educational helping tool for students.

Finally, the book titled Research Design Qualitative, Quantitative and Mixed Method Approaches and Educational research: planning, conducting, and evaluating

quantitative and qualitative research by Creswell [14] provides us with an invaluable framework for our study, since the aforementioned book actually covers every step from the planning, research design, how to collect the necessary data, how to link together and interpret collected data, to how one actually conclude and validate our findings.

See our list of references for a complete list of found earlier related research studies and books.

In summary, our conducted literature study provides us with several related research studies, confirming the relevance of our posed research question. Moreover, it also provides us with a solid understanding and background (or, more specifically, a framework) on the subject of developing, testing and evaluating educational math games.

(24)

14 3.4 Solution design

3.4.1 Artefact design and implementation

The artefact design, i.e., the design of our educational game, was decided based on the information and ideas obtained during our aforementioned literature study but also from our earlier experiences on game design and game development. In particular, when it comes to the overall design of our proposed educational game, we draw inspiration from three well renowned games, namely Portal [27], Anti-Chamber [28] and Amnesia [29]. Specifically, already very early during the planning phase of our study we decided that our proposed educational game would be inspired and designed in the same fashion as the aforementioned games. The reason behind this is first of all that all of these three games are so called puzzle games, i.e., their gameplay

mechanics revolves around a certain set of puzzle related actions that need to be used or implemented in order to advance further in these games. Another reason is that all of the aforementioned three game are played through a first-person perspective in a 3d world space, which requires a higher degree of production quality compared to regular educational games, which usually are based on flash animation or still pictures. Moreover, all of these three games are also developed by using specialized in-house versions of the MDA framework (which will be deployed also in our study), and last but not the least, all these three games have an interesting and gripping story, which is one of the main factors why they are consider to be good games by players and video games reviewers alike [46, 47, 48, 49].

In our opinion, the most spectacular with the three aforementioned games is the way the development teams for these games managed to implement design choices and patterns promoting our two desired aspects of a game, namely an interesting and satisfying game. In particular, the design patterns relating to the puzzle element, control mechanics and the way the story of the game is told are of interest in our study. Therefore, several design patterns inspired by these three games will be implemented during the development of our educational game, see chapter 4 for a more detailed discussion on this matter.

The implementation of the artefact of our study will be creating an educational game by following the important elements (e.g., design patterns) mentioned in chapter 2, together with the 16 guidelines set by Gee in [4].

Finally, in order to fine-tune and improve our developed educational game as a whole the MDA framework will also be used. The implementation of the artefact and its subsequent design choices and patterns will be discussed in further details in upcoming chapters of this study.

3.4.2 Choice of game engine

The aforementioned three games (i.e., Portal [27], Anti-Chamber [28] and Amnesia [29]) are all developed with the help of so called game engines. A game engine is software framework that enables the effective creation of video games and other related media. There are many different version of game engines used in the

(25)

15

usually based on: the type/genre of the game, cost and availably of the engine and finally, past experience of the development team.

The game of our study will be developed by using the game engine Unity [30]. The main reason for choosing Unity is based on our previous experience working with this engine, but also its usage of C# for writing scripts. Moreover, its versatility and ease of usage compared to other game engines (such as for example Unreal Development Kit [31]) was also important for our choice here. Another advantage with Unity is the cost and availability of the engine, since the business model for Unity is free to download and use if the developed game earns less than $100,000. But if the game earns more than this sum, the development team has to subscribe to a monthly service (Unity Professional edition [30], with a cost of $75/month).

3.4.3 Modeling tools and software

Since the Unity game engine only provides the necessary framework for programming the logic and rules for the game, additional software is needed in order to develop (or in this case model) the necessary 3d models that will be used in our game. The software that were used to model the 3d world and its contents in our study are 3d Studio Max [32] and Blender [33]. Moreover, for coloring (or texturing) the 3d models Photoshop with the Quxiel [34], which is an add-on for Photoshop that makes the texturing easy, was used.

3.4.4 Development model

When developing any software video game or other applications, it is always important to structure the development process around a certain software development model. The model that will be used for developing the educational game in our study is the well-known agile development model [41] with some Extreme programming (XP) [41]. The choice of the aforementioned models where based on our need of rapid

implementation and testing of different designs for our proposed educational game. 3.4.5 Version handling

In our study, we used Dropbox [35] for storing and version handling of our

educational game and its related assets. Dropbox is a cloud storage service that allows the uploading and sharing of program files and other file types between different users, thus allowing us to work on the game from different angles. For example, while one of us is programming the scripts, the other one models the 3d objects for the game.

(26)

16 3.4.6 Platform

Our proposed educational game was developed for PC/Mac mostly because of the planned control design which is a keyboard and mouse, but also since PC/Mac have better computational and graphical performance compared to smartphones.

3.5 Evaluation of the artefact

During the development process of our educational game, we will continuously test and evaluate our design choices. In particular, this will be conducted in-house (i.e., alpha stage). Note that this early and continuous involvement of the end user (i.e., in our case, our selected test participants) in the development process is in line with the deployment of modern agile development methods.

There are several well-known benefits with such an early involvement of the end users, e.g., this will provide us with invaluable feedback in the process of developing a game that better fits to our end users’ wants and needs of an interesting and satisfying educational math game.

3.6 Evaluation

3.6.1 Mixed methodology

There are many different types of methods and design framework when conducting a research study. The methods of choice are generally based upon the specific type of research problem and what kind of data it relies on, but also how the researchers decided to conduct their research study and the expected outcome.

There are some different method designs at hand, one example here is the

performance centered study which relies mostly on numeric based quantitative data in order to analyze and understand if a series of changes impacted the performance of a certain thing or subject. The other well-known method design relies instead on observations and interviews (i.e., based on qualitative data) in order to find a certain relationship between a subject or participants’ behavior and their historical and social background.

However, some studies require both a quantitative and qualitative approach and then, the researchers need to apply a so called mixed method research design [14, 24]; this in order to have a distinct framework on how to address the needs and requirements for both qualitative and quantitative research.

Since mixed methodology relies on both quantitative and qualitative research

methods, a study applying the aforementioned mixed approach must also emphasize the order in which the quantitative and qualitative data is collected. For example, if the data collections were made in a sequential order (i.e., first a qualitative data collection and later a quantitative, or vice versa) or in a transformative order (i.e., first collecting either qualitative or quantitative data in order to base a secondary data collect of the opposite approach) or lastly, both qualitative and quantitative data are collected at the same time (also denoted as concurrent data collection) [14, 24].

(27)

17

Since our study relies both on qualitative and quantitative assumptions in order to answer our aforementioned research question (i.e., whether our design choices here impacted our test participants’ overall satisfaction and interest to continue playing our educational math game), a mixed method approach will be implemented in our study. Regarding the emphasis of the study it will be equal divided between the two

approaches.

3.6.2 Data collection

In our study, a traditional behavioral science research (see, e.g., [8, 14, 24]) will be used to collect the necessary primary data. Moreover, the primary data of our study will be collected in a concurrent fashion, first by silently observing our test

participants playing our proposed educational math game (only assisting them when they get stuck).

Furthermore, in order to obtain first hand qualitative data our test participants were encouraged to think loud, i.e., speak up their thoughts while playing our game. After completing the game, our test participants were also asked to fill out a survey (see appendix A) based on a mixture of both qualitative and quantitative questions about their enjoyment, educational level and perception of our proposed game. Finally, a short interview, based on the aforementioned observational data, was conducted with a selected group of our test participants. Note that the main aim of the interview is to flesh out our test participants’ experience of our proposed game and in this way, gain a better understanding about their thoughts and impressions of the game that they just played.

3.6.3 Selection of test participants

As mentioned above, the evaluation of our proposed educational game will be

performed by a selected group of test participants, which will simulate the end user for our proposed educational game. Since the intended sample group for our study is elementary school students between the ages of 8-12 years old, we contacted some local schools and presented the idea of our study. However, due to complications with arranging such testing sessions together with the local high schools (due to some school regulations), we instead had to include test participants for our study outside our intended main end user group. Specifically, we set up an inquiry about

participating in our study and from this we ended up with a group of 14 test participants in the age of 18-48 years old, constituting mostly family members and close friends (mostly video game development students and LAN gamers), for our study.

(28)

18 3.6.4 Data collection -- biases and pitfalls

There are some important aspects that must be taken into consideration while

conducting the data collection process. First, possible biases that our test participants may have while testing and answering the question should be minimized. For example, does a high score of a test case actually means that a test participant enjoyed our game only for other reasons than the intended ones.

Another important aspect is the gender diversity of our data sampling. Specifically, there is a lack of female participants our study, which unfortunately diminishes the validity of the test data since the female perspective and feedback is unknown for the moment. The reasons for the lack of female participants are mainly attributed to the low number of females in the video game development program, or that we simply missed them while conducting the data collection.

Last, but not the least, the qualitative data collection was conducted in Swedish. Later the obtained data was translated into English by us. This is important to consider since language differences may have consequences, e.g., concepts in one language may be understood differently in another language. Hence, in order to translate the qualitative data in a correct way (i.e., not losing any meaning in the process) we

followed the recommendations set by van Nes et al. in [15]. In particular, van Nes et al. discuss the challenges of such language differences in qualitative research studies.

3.6.5 Data analysis

The analysis of the collected data for our study is conducted by using the convergent concurrent design [37, 50], which is an analysis process that first analyses both quantitative and qualitative datasets separately in order to compare or relate the results and make an interpretation whether the results support or contradict each other. In our study, the convergence will be performed by quantifying the qualitative data, which means translating textual information into numerical data in order to compare it with the quantitative dataset. In our study, the quantification of the

qualitative data was performed with the help of MAXQDA 12[38], which is a qualitative data analysis software used to assign or code the textual information in order to later on analyze the frequency of the code in the collected dataset. Furthermore, the statistical analysis of the quantitative dataset of our study was performed with the help of IBM SPSS [39], which is quantitative data analysis software used to analyze the relationship between different variables in a dataset. Finally, the two datasets of our study will also be directly compared with each other, by supporting their results with statistical tendencies or qualitative themes.

(29)

19

(30)

20

4. Analysis, design and implementation of our game

In this chapter, we well describe our initial analysis of the educational math game we are about to develop in our study. Moreover, in this chapter we will also break down the design of developed educational game into a set of conceptual entities, where each component was implemented by deploying one or more well-known design patterns. Note that each of the aforementioned design pattern is well-known for facilitating one or more attribute; in particular, we will explain why the implemented design patterns are necessary to make the educational game interesting and satisfying.

Moreover, in order to provide a general picture of our proposed game design we will also describe how the aforementioned patterns work in its entirety.

4.1 Analysis of educational games

There are many kinds of educational games out in the market today, but very few of them have succeeded in being perceived by the player as being both interesting and satisfying. One possible reason for this is the way these games are developed, with restrictive design choices in the game design, the lack of re-playability and dull and uninspiring game world and story [6].

Therefore, we decide to break the norm of how these games are normally developed, i.e., in a fashion comprising pure puzzle elements, quiz, point and click, and etc. Instead, we will focus on creating a game that uses the standard first person perspective as see in modern first-person shooter game (FPS for short) just like Counter strike [39] in combination with a simulation gameplay to enhance the educational aspect of the game. Specifically, to identify the “how” to create an educational game we chose to go

through Gee’s 16 game mechanical principles on what makes a game “A good video game” for educational purposes [4]. A good game for learning, according to Gee, is a simulation game that will reflect the area it affects. Here, the players will be able to explore, test, examine and experiment just to get a hang of what they can do and what they cannot do in the game. However, note that creating a simulation game does not mean that it is relevant by its own unless it is constructed in a user-friendly fashion which enables a coherent interaction between the player and the game.

Furthermore, one of the key points when developing a game is to give the players a clue on what role they have in the game, i.e., who they are and what they can do to influence an outcome in the game [4]. In particular, we found that the theme of the game is a very important and quite critical part of a game since it affects the player’s first impression of the game. By knowing what and how the game should be played, it should now have an appropriate corresponding theme; however, this should not be too revealing or it will ruin some undiscovered excitements of the game.

Rai and Beck argue that instead of creating a learning game, it can be helpful to create a learning environment with game-like elements as learning games have had difficulty in keeping students' interest for longer periods, which limits their long-term learning [5].

(31)

21

As discussed above, everything that makes a so-called identity is also what makes a game. Hence, the structure of our developed game will be a combination of a presented way of solving a puzzle (on-demand), this by having a narrative environment which provides the necessary information needed to solve the puzzle. But we will also allow the players to have the freedom to try and solve it in their own way (just in time); this in order to portray a degree of freedom which indeed is a key factor in making the game play of our developed game more interesting since it involves an element of exploration. Gee summarizes the above discussion as follows:

“Good games give information on demand and just in time, not out of the contexts of actual use or apart from people's purposes and goals, something that happens too often

in schools.” [3]

Note that while the Design Pattern process can be used to evaluate the implemented design choices of the developed game, the MDA framework can be used to improve and fine tune the developed game as a whole.

4.2 Setting up the Game

We started by brainstorming a theme that would fit the game with an important rule which is that the theme should not mimic a school environment. Instead the theme needs to be more interesting and eye catching in order to keep the player (student) coming back for more, and the more they come back the more they will eventually learn.

The theme of this game is inspired from the scientific theory behind the Big Bang [42]. The motivation behind picking the aforementioned theme is the mystery and scientific value it has. The Big Bang theme will also serve as our premise for the story of our developed game, with the story revolving around what caused the Big Bang to happen. The answer to this question will be the presented goal to the player, in which they have to explore, examine and solve mathematical puzzles in order to answer what could have caused the Big Bang.

In addition, we believe our game play needs to be “freer” and more “forgiving” than several earlier attempts; this in order to encourage the player to experiment and explore more and hence learn for their mistakes without being punished (i.e., as a failure in exam could be). In order to achieve the aforementioned gameplay, the following design patterns are implemented in our educational math game:

(32)

22

4.2.1 Design Pattern: Fail to learn (forgiving gameplay)

Problem: A lot of games penalize the player when failing to complete a certain presented task, and this can discourage some players since it reminds them of the same feeling when failing a test in school. This problem will occur throughout the game in the puzzle section of the game, but also in the exploration and examination sections.

Solution: In order to solve this problem, we will not implement any penalty system in our developed game. Hence, there will no life systems, time limits or anything alike as seen in the aforementioned games (i.e., Portal [27], Anti-Chamber [28] and Amnesia [29]). Instead, the gameplay is implemented in a way that allows the player to have unlimited tries to solve the presented puzzles, and also being rescued by a so-called NPC when falling of a platform (which will be explain into further details in the Teacher NPC design pattern in this chapter).

Consequences: The consequence of this design choice is that some players will either find the gameplay “too easy” or, more importantly, solve the mathematical puzzle by “brute forcing”, i.e., trying different solutions without thinking in order to solve the puzzle as quickly as possible.

4.2.2 Design Pattern: Pseudo unlimited movement space (freer gameplay)

Problem: Unfortunately, all games have the necessary evil pattern of so-called invisible walls, which are used to limit the player’s movement inside the game. This pattern is implemented in different ways, ranging from simple rooms in which the player move around to debris blocking a certain path. The reason behind the implementation of this pattern is to divert the players focus and attention the essential elements and mechanics inside of the game, but also to limit the game world (and the subsequent work need to develop the game world).

Solution: The solution we implemented in our developed game is to let the player move around on floating platforms, which are placed in a game world rimiest of a starry night or galaxy dust. This aesthetic design choice limits the amount of interest of the players since they are surrounded by the same horizon, and if they decide to walk outside and fall of the platform the Teacher NPC design pattern will rescue them.

Consequences: The consequence of this design choice is that some of the players will eventually find out that the freedom offered in our developed game is fake or pseudo, and this realization will unfortunately diminish the player’s interest in exploration. But our design pattern will still be better received by the players, since our invisible wall are not really present in our game thanks to the platform structure of the game world.

(33)

23

4.2.3 Design Pattern: Predictable Consequence nr.1 [19]

Problem: The player has to perceive failure in movement (or input) as a consequence of their mistakes, and not as a random or a predestined event in the game.

Solution: The player cannot take a meaningful decision to act (or not to act) if the result of the possible action cannot be anticipated. Therefore, it is important to convey meaning behind every reaction our developed game has to the player’s action, since a meaningful player decision is an informed decision: the player has to be able to guess the result of their action before doing it. The goal of the game mechanic, which will be implemented in our game, is to communicating these predictable behaviors in the following manners:

Earth-like physics: The game world and all the objects residing in it are influenced by gravity and Newton’s three laws of motion [43]; the decision for using this mechanics is that it is based upon real life experience which the player already has some

knowledge about.

Standard first person “shooter” (FPS for short) control schema: Our developed game will follow the control schema and mechanics which are used in several modern FPS games, with the following controls:

The projection of the game world is projected through the player’s “eyes”, i.e., the player plays as the game character and sees what the game character sees. Moreover, our developed game is a so-called single player game, i.e., there is only one player in a game instance.

Consequences: Any faulty implementation in the two aforementioned solutions will break the immersion in our game which will only tend to confuse the player, since the games reaction will be unpredicted and this will greatly damage both the gameplay and the satisfaction level of our game. Furthermore, it becomes harder to surprise the player with other design patterns which temporary distorts the aforementioned design pattern, since the distortion will either be received by the player as a fun and

interesting supplement or just confusing and strange.

Keyboard:  W = Move forward.  S = Move Backwards.  A = Move Left.  D = Move Right.  Space = Jump.

 E = Change the objects value +/- i.e. 3 to -3.

Mouse:

 Pan up = Look up

 Pan down = Look down

 Pan left = Look left

 Pan right = Look right

 Holding left mouse button = Interaction with a game object

(34)

24 4.3 Introduction

An important part of all games is the introduction level which describes how and what to do in order to play the game. According to us, this part of our game must be very similar to what is done in school, i.e., when a teacher introduces the students to a new subject such as mathematics. But the learning principles are different and vary with factors such as motivation, recognition and learning environment. A skillful teacher is defined as a person who, for example, can easily show how to solve a certain problem in many different ways, while a good game designer creates a number of different aids for the player to use in order to solve the problem [2].

For our developed educational game the MDA framework companionship aesthetics will be used together with the following design pattern:

4.3.1 Design Pattern: Teacher NPC (or Sture)

Problem: The decision of including a so-called teacher NPC was made out of the insight that most of earlier educational games lacked this kind of feature, which made the aforementioned games feel empty and lonely.

Moreover, the effectiveness of this design pattern when it comes to informing the player on what and how things work in the game world, and how fun this feature can be, can be seen in the aforementioned game Portal [27]. In the aforementioned game, the main antagonist Cell encourages the player to continue solving deadly puzzles with the main reason being “it is for science” [27].

Solution: Our solution is to introduce a fictional little gas ball by the name of “Sture", who is a fictional non-playable character (NPC), see figure 3 below. Sture’s task is to be the player's teacher, guide and friend. Note that the implementation of our character Sture is in line with the following recommendation on how to create successful learning environment:

“Narrative-centered learning environments offer the potential to be effective tools for promoting content and problem-solving learning gains by providing students with

engaging, interactive learning experiences.” [6]

Consequence: If Sture is poorly implemented the entirety of our game would suffer, and both interest and satisfaction levels would diminish. Since Sture is one of the key pillars in our educational game, any erratic behavior of Sture could possibly damage or amplify other design patterns.

(35)

25 .

Figure 3: The companion “Sture” acts as a guide and a friend during the whole game.

4.3.2 Design Pattern: Sture’s Movement (or privileged movement)

Problem: It is important to note that the player of our developed game should not be able to interfere with Sture’s movement or other in-game entities such as the platforms which the player stands on. This design pattern is implemented in order to keep the aforementioned Predictable consequence nr.1 design pattern in check, and also to hinder the player from finding game-breaking exploitations [19].

Solution: Our solution for this problem is to first of all lock all static game objects (such as the doors the player walks through, the platforms the player stands on and the structures of the puzzles) after they appear. The second solution is to give Sture the ability to fly around unhindered by the player and other game object, without falling down as the player would do.

Consequence: This implementation can sometimes be viewed as being a very heavy-handed way of ensuring protection of the game world and its game objects. But it is unfortunately a necessary evil in order to insure the games structure and consistency. The player might also perceive this movement constraint to only platforms as

annoying and disappointing, compared to Sture who can fly around without any problems.

(36)

26 Design Pattern: Sture Weenie

Problem: The player might lose their sense of direction with respect to how our game world unfold, which is a particular problem often seen in player driven gameplay such as in our developed game [19].

Solution: Our solution for this problem is to have Sture becoming the guide of the players throughout the game world and also to divert the players focus towards the important elements presented inside our game. This is archived by implementing audiovisual cues often seen in other games such as: a glowing point which the player has to move towards, a sound bite indicating confirming the player’s action, or the level itself refolding to a certain path to show where the player needs to go.

Consequence: The player may become dependent on this guidance system (i.e., Sture Weenie), and the player will be confused if this design pattern is poorly implemented or even suddenly omitted inside our game. Therefore, it is important to first focus on developing a number of solid Weenie design patterns, which are both coherent and consistent to the presented game world. But also in order to later on chain these Weenies together to form a clear and uninterrupted way through our game world. Note that, according to Clarke-Wilson [44], the term Weenie was coined by Walt Disney. He suggested that when designing massive 3D environments, such as theme parks, it is important to lead the visitors through these environments the same way as one trains a dog - by holding a wiener and leading the dog by its nose.

4.4 Design Pattern: Give the player a goal

Problem: In most learning games, you already know what the goal will be. But this knowledge will in fact make the game less interesting, just like as knowing the ending of a book or movie.

Solution: Our solution for this is to hide the main goal as much as possible; this in order to encourage exploration among our players when playing the game. Hence, we will not immediately reveal that the game is a math solving game. Therefore, we created goals that are not so obvious, even if our players know that the game is going to be about solving math questions. The aforementioned game pattern was also discussed as follows in [3]

“Good games confront players in the initial game levels with problems that are specifically designed to allow players to form good generalizations about what will work

well later when they face more complex problems. Often, in fact, the initial levels of a game are in actuality hidden tutorials.” [3]

(37)

27

Consequence: The consequence of the aforementioned design pattern is based on how obvious the initial goal of the game is for the player, since a failure in conveying the meaning and purpose behind the aforementioned goal can lead to confusion which greatly diminishes the player interest and satisfaction of the game. Therefore, it is important to have other well implemented design patterns supporting this one (such as the Teacher NPC/Sture and Weenie design patterns previously mentioned, see figure 4).

Figure 4: Sture tells the player to move to the glowing point.

4.5 Design pattern: Pseudo choices

Problem: We included a small experiment in our game, more specifically, we would like to investigate whether it would be possible to trick the player by presenting an illusion of some choice and options in our game. The main aim here is to break up the linear structure of our developed game, i.e., a structure of our game where there is only one way to get from point A to point B.

Solution: This design pattern occurs in our game when the player is confronted by two doors with the choice of going through either the green or the blue door in order to get to the next part of the game. By deploying this design pattern, we let the player believe that the game world is filled with mysteries and that their choices mean something in the game world. This in order to give the player a feeling that anything can be explored in the game world, thus minimizing the feeling of linearity.

Another reason to why we choose to implement this feature was the lack of any character customization in our game, which, according to Gee [3], is a necessity to make the player feel more like of producer than a viewer of the game. In other words, the player must have the ability to play the game in their own way (see figure 5 below).

(38)

28

Consequence: However, the aforementioned design pattern can actually become a so-called double edge sword when implemented in our game, since we can easily lose the illusion of choice if this design pattern is not well implemented. However, this can be prevented to some extent by implementing randomization together with this design pattern. For example, the door which actually leads to the next part of the game can be the green one for one play session, whereas for another play session it is actually the blue one, and so on (see figure 5).

Figure 5: Choices are being given to the player to let them do their own decisions.

4.6 Design pattern: Transition

Problem: The transition in a game is important and has to be smooth for a maximum stimulation between the stages of the game. By making it “smooth” we mean that the player has to be able to identify themselves as a part of the transition rather than just jumping from one part to another part of the game without knowing how it happened. Another thing about smooth transition is that it helps the player to put behind a part of the game mentally and to recognize where they are at the moment.

Solution: The transition section in our game is implemented by having a flashy tunnel section, which is unique between every transition section in our game. This transition idea comes from how movies and books handle transitions, i.e., how they manage to increase the curiosity of the reader (or viewer) on what will happen next if they will proceed further on (see figure 6).

(39)

29

Consequence: The transition can be done at any part of the game, whenever the game changes from one state to another. But the most important part, as mentioned above, is that the player must be a part of that transition. Like in movies or books, if you lose the reader (or the viewer) during a bad transition then they will lose interest.

Figure 6: A great transition will give a player the motivation to continue the discovering.

4.7 The cube solving puzzle part

When the players have completed the basic tutorial and the introductory part of our game, and have familiarized themselves with the controls, they will be introduced to the first task of the game.

The first task our game consists of a simple puzzle game which uses multiple cubes and a platform. The purpose of the platform is to provide the player with a number, which the player has at disposal in order to complete the puzzle. Furthermore, the role of a cube is to hold a number. On a platform, there are several such cubes placed and the main aim for a player is to select a number of cubes in such a way that the total sum of the selected cubes digits is equal to the sum written on the platform.

Note that before the player can do anything, the teacher NPC, i.e., Sture, approaches and inform the player that it is possible to pick up the cubes and to put them on the platform. Specifically, the first task includes the following elements:

 Multiple cubes, where each cube has a number (or digit) printed on it,

 a platform, where each platform has a number (or digit) printed on it,

 The power to lift and throw the cubes at will, by aiming on the cube and then holding the left mouse button.

On the development of an educational math game

Examensarbete

15 högskolepoäng, grundnivå