Gamification of a Physics Simulation Tool

(1)

Teknik och samhälle Datavetenskap

Examensarbete 15 högskolepoäng, grundnivå

Gamification of a Physics Simulation Tool

Gamification av ett verktyg för fysiksimulering

Alexander D. Baldwin

Simon J.O. Dahlberg

Examen: Kandidatexamen 180 hp Huvudområde: Datavetenskap Program: Spelutveckling

Handledare: Olle Lindeberg Andrabedömare: Bengt J. Nilsson

(2)

Abstract

Gamification – the use of game elements in non-game contexts – has been shown to be an effective way of creating more enjoyable and engaging user experiences. Many applications of gamification rely on a limited subset of game elements such as points, badges and leaderboards – techniques which have been criticised for not being representative of what makes games enjoyable. Research in psychology suggests that people are most effectively motivated by factors that appeal to their personal satisfaction, such as fun, rather than external factors such as rewards and prizes. This thesis explores the use of game design elements and related 3D technologies in the creation of a tool for the construction and simulation of physical models. The implementation of such a tool is described, focussing on the integration of game design elements that produce a fun and engaging experience for the user. A study is performed to determine the effects of the use of game elements on users’ enjoyment when interacting with the simulation tool. It was found that by appealing to themes users can relate to and which are contextually appropriate for the application, game elements can help promote an enjoyable user experience.

(3)

Sammanfattning

Gamification – bruket av spelelement i icke-spelsammanhang – har visat sig vara ett ef-fektivt sätt att skapa mer underhållande och engagerande användarupplevelser. Många tillämpningar av gamification förlitar sig på en begränsad delmängd av spelelement som poäng, emblem och resultatlistor – tekniker som kritiserats för att inte vara representativa för vad som gör spel underhållande. Forskning inom psykologin indikerar att personer blir mest effektivt motiverade av faktorer som vädjar till deras personliga tillfredsställelse, såsom nöje, snarare än externa faktorer som belöning och priser. Detta arbete utforskar bruket av speldesignelement och relaterade 3D-teknologier i utvecklingen av ett verktyg avsett för att skapa och simulera fysikaliska modeller. Implementationen av ett sådant verktyg beskrivs med fokus på integrationen av speldesignelement vilka bidrar till en un-derhållande och engagerande upplevelse för användaren. En studie utförs för att validera effekten av bruket av spelelement på användarnas nöje när dessa interagerar med sim-ulationsverktyget. Det observerades att genom att använda teman som användaren kan relatera till och som är kontextuellt passande för applikationen i fråga kan spelelement förstärka en njutbar användarupplevelse.

(4)

Acknowledgements

We would like to thank Hilding Elmqvist for his support and guidance, as well as everyone from Dassault Systèmes who worked with us to make this thesis possible.

(5)

1 Introduction

1.1 Background

Increases in the computational power of modern computers have led to advanced scientific simulation/visualisation tools being accessible to the general public, but science education has been slow to adopt the use of interactive simulations. Wieman and Perkins [36] are vocal proponents of using interactive simulations as an educational tool in science and argue that traditional forms of education fail to provide students with an understanding of science by suppressing their interest in the subject and failing to engage them. Wieman and Perkins argue that interactive simulations are an effective complement to traditional media in science education – an idea that is supported by research into the use of simulation tools to engage college students [24]. According to Wieman and Perkins, the most important features of an engaging interactive simulation tool are:

• Highly interactive animation

• An appealing environment and sophisticated graphics • Simple and intuitive controls

• Connections to real-life objects [36, p. 291]

Dymola is an environment for modelling and simulation of complex physical systems from diverse engineering fields [20]. Through a JavaScript interface Dymola allows the user to specify and simulate models in a browser-based application, allowing new functionality to be provided or existing functionality to be hidden, thereby facilitating the creation of new tools. While Dymola is a complex application targeted at professional users with experience of programming in the Modelica language [19], on which its models are based, the JavaScript interface allows tools targeted at less experienced users, such as students, to take advantage of the simulation capabilities of Dymola in a simpler environment.

Research shows that an effective way of promoting user-engagement and interest in learning environments is the incorporation of elements from digital games [30]. The use of game-elements in other contexts, usually referred to as gamification, has been an increas-ingly popular research topic in recent years, with many studies applying its principles in an educational context [12, 31]. The stated motivation for the use of (digital) gamification is often to increase user engagement and, consequently, user retention in software systems [9].

Few attempts have been made to produce a formal definition of gamification and most of the available research on gamification appears to be based on Deterding et al.’s [9] definition:

‘Gamification’ is the use of game design elements in non-game contexts.

Huotari and Hamari [14] attempt to define gamification from a marketing perspective – a tool to facilitate value creation. Their view of gamification deals with applications invoking the same kind of psychological and behavioural responses that games do, whereas Deterding et al.’s focus is on the use of elements from games in an application rather

(8)

than the outcome [12]. Deterding et al. criticise Huotari and Hamari’s definition for be-ing too narrow and underplaybe-ing social and experiential aspects of games. Nonetheless, the use of gamification as a marketing tool has become popular, with proponents such as Gabe Zichermann enthusiastically promoting the idea of using gamification for mar-keting [40]. Zichermann and Linder’s book “Game-based Marmar-keting. Inspire Customer Loyalty Through Rewards, Challenges and Contests” [42] describes means by which game mechanics can be used for marketing, with a focus on points, leaderboards and rewards.

The limited array of game elements used in applied gamification research has drawn criticism, particularly from the games industry and the field of game studies [31] and, consequently, gamification as a whole is sometimes viewed in a negative light. Game designer and critic Ian Bogost calls Zichermann “the gamification movement’s Dark Lord”, criticising what Zichermann calls “key game mechanics” as not representative of what makes games fun and describing gamification as “exploitationware” – taking advantage of customers by replacing real incentives for loyalty with fictional ones [6]. Game designer Margaret Robertson takes a similar perspective to Bogost in stating about the prevalence of points and badges in gamification: “What we’re currently terming gamification is in fact the process of taking the thing that is least essential to games and representing it as the core of the experience.” [27]. Robertson feels that the term “pointsification” would be more appropriate when describing typical applications of gamification. Seaborn and Fels [31] contend that Bogost’s view of gamification is too focussed on marketing applications despite the fact that gamification has been used in numerous other areas and point out that gamification is generally viewed more positively outside of the fields of game studies and game design.

In addition to design-elements from games, game-related technologies such as game engines and 3D-engines can be useful in non-game contexts and have successfully been used in the development of simulation tools on numerous occasions [4, 13, 25]. Game engines often boast advanced graphical features with realistic lighting and shadows and support for importing complex animated and textured 3D models, which can be used to provide the appealing environment and sophisticated graphics Wieman and Perkins consider important. Depending on the type of simulation tool to be created, other common features of game engines such as built-in physics engines and artificial intelligence systems can also be useful. These types of features are often provided in a development environment which supports rapid production, commonly with little need for programming.

1.2 Related Research

This section examines previous research on gamification and the application of game-related technology to simulation tools.

1.2.1 Game Elements

The majority of available research on gamification uses either Deterding et al.’s definition, no explicit definition or a hybrid of multiple definitions [31, pp. 23-24]. When using Deterding et al.’s definition as a starting point, it is important to consider what a “game design element” is. Huotari and Hamari’s definition sidesteps this issue by putting the

(9)

focus on the added value arising from the use of “feedback and interaction mechanisms” [14] rather than the specifics of what those mechanisms actually are. Regardless of which definition is used, the designer of a gamified system/service must have some idea of what a game element is and how it can be used.

In an attempt to define game elements, Deterding et al. note that it is not useful to define elements which must be present for a system to be considered gamified, since there exists no such set of elements that accurately represents all styles of games. They prefer instead to talk about elements that are characteristic of games:

. . . elements that are found in most (but not necessarily all) games, readily associated with games, and found to play a significant role in gameplay. [9]

The authors divide game design elements into five levels of abstraction, shown with examples in Table 1.

Level Description Example

Game interface design pat-terns

Common, successful inter-action design components and design solutions for a known problem in a con-text, including prototypi-cal implementations

Badge, leaderboard, level

Game design patterns and mechanics

Commonly reoccurring parts of the design of a game that concern gameplay

Time constraint, limited resources, turns

Game design principles and heuristics

Evaluative guidelines to approach a design problem or analyze a given design solution

Enduring play, clear goals, variety of game styles

Game models Conceptual models of the components of games or game experience

MDA (Mechanics Dynam-ics AesthetDynam-ics); challenge, fantasy, curiosity; game de-sign atoms; CEGE (Core Elements of the Gaming Experience)

Game design methods Game design-specific prac-tices and processes

Playtesting, playcentric de-sign, value conscious game design

Table 1: “Levels of Game Design Elements” from Deterding et al. [9, p. 12]

In addition, Deterding et al. draw a distinction between game design elements and game-related technologies, such as the use of graphics/game engines or input devices like game controllers – classing the former as a part of gamification but not the latter, as they are widely used outside of game-related contexts [9].

(10)

In two literature reviews investigating the results of empirical studies on the effects of gamification, both Seaborn and Fels [31] and Hamari et al. [12] report that points, badges and leaderboards were the most common forms of game-element (or motivational affordance, in Hamari et al.’s terminology) studied. Seaborn and Fels recommend the study of more diverse game-elements as an avenue for future research, citing the early state of gamification research as a potential reason for this repeated focus on a limited number of elements.

1.2.2 Intrinsic and Extrinsic Motivation

A number of attempts have been made to construct models or frameworks for the applica-tion of gamificaapplica-tion, mostly taking inspiraapplica-tion from Ryan and Deci’s research on intrinsic and extrinsic motivation [7] or the related concept of Self Determination Theory, which deals with the behaviour of people who are motivated by purely internal factors [29]. In-trinsic motivation refers to the motivation to perform an activity because it is inherently satisfying, rather than because of some separate consequence or reward [28]. Ryan and Deci describe instrinsic motivation as follows:

Intrinsic motivation energizes and sustains activities through the spontaneous satisfactions inherent in effective volitional action. It is manifest in behaviors such as play, exploration, and challenge seeking that people often do for no external rewards. [7, p. 658]

Extrinsic motivation refers instead to activities performed with the specific intention of achieving an external outcome, such as a reward or praise. According to Self Determination Theory, extrinsic motivation can vary in the degree to which it is autonomous (coming from a person, rather than external sources), since a person may perform actions with perceived personal benefits (rather than purely for the sake of personal satisfaction) without external influences [28].

Ryan and Deci’s review of papers studying the effect of extrinsic rewards on intrinsic motivation concludes that, generally, “tangible rewards had a significant negative effect on intrinsic motivation for interesting tasks” [7] and a later article on Self Determination Theory states that people who are self-motivated rather than externally controlled have “more interest, excitement, and confidence, which in turn is manifest both as enhanced performance, persistence, and creativity” [29, pp. 658-659]. This negative impact of extrinsic motivators suggests that it might be more successful to employ game elements that appeal to a user’s intrinsic motivation rather than the commonly used points and badges, which imitate tangible rewards.

1.2.3 Gamification Frameworks

In his design philosophy for gamification [39], Gabe Zichermann recognises the potentially negative effects of extrinsic motivation, but instead of focussing on intrinsic motivation directly, he uses the Self-Determination Theory idea that extrinsic motivators can be con-verted to intrinsic motivators if they are “meaningful, pleasurable and consistent with a person’s worldview”, stating:

(11)

The introduction of carefully selected extrinsic rewards, built around a design that speaks to intrinsic motivational states (sometimes not the ones most closely aligned with the behavior we seek to change), is the most powerful design model we have today.

Consequently, the two books Zichermann has co-authored on the subject of design strategies for gamification [42, 41] have a focus on extrinsically-motivating game elements such as points, leaderboards and badges, with an emphasis on the social aspects of gamified systems where users compete for status (exactly the kinds of gamification that people like Bogost [6] and Robertson [27] criticise).

Nicholson [21] attempts to solve the issue of the negative effects of extrinsic motivators on intrinsic motivation by conceptualising a framework for “meaningful gamification” with a focus on the end-user rather than the organization providing the service. He also provides his own definition of meaningful gamification: “the integration of user-centered game design elements into non-game contexts”. Besides recommending a focus on intrinsic motivators, Nicholson’s framework stresses the importance of context in the use of game elements, referring to the concept of situated motivational affordances [8], which describes how the motivational effect of a system element depends upon the background of the user and the context in which it is used in the system. Nicholson concludes that meaningful gamification relies on using elements that users with a wide variety of backgrounds can relate to in the right context within a system. In his examples of applications of meaningful gamification, Nicholson suggests removing scoring/rule-based elements and focussing on play, referring to this as “playification”. Deterding et al. refer to this concept as “ludification” and consider gamification a subset of ludification [9, p. 13].

Aparicio et al.’s [1] methodology for gamification is based more closely on Self Determi-nation Theory, basing the choice of game elements on those matching the central sources of intrinsic motivation: autonomy (the sense of personal choice), competence (the need for challenge) and relatedness (the need for social interaction). Game elements listed as examples in each of these categories are:

• Autonomy: profiles, avatars, macros, configurable interface, alternative activities, privacy control, notification control.

• Competence: positive feedback, optimal challenge, progressive information, intuitive controls, points, levels, leaderboards.

• Relation: groups, messages, blogs, connection to social networks, chat.

Numerous studies on gamification refer to Malone’s paper [18] on using ideas from games in effective user interface design. While written long before the coining of the term gamification, Malone’s ideas still have a lot in common with the aforementioned definitions, particularly Nicholson’s framework for user-centered meaningful gamification. Malone focuses on the user’s enjoyment, defining three principle heuristics for designing enjoyable user interfaces: challenge – the activity should have a clear goal and an uncertain outcome, fantasy – the interface should be “emotionally appealing” and use metaphors that the user can relate to, and curiosity – the interface should provide the right level of

(12)

informational complexity in order to be novel or surprising while still understandable; it might also use “sensory curiosity”, which refers to the use of audio and visual effects as decorations, to enhance fantasy or as a means of representing aspects of a system.

While some of the papers reviewed by Seaborn and Fels cite Cunningham and Zicher-mann [41] or ZicherZicher-mann and Linder [42] as bases for their application of gamification [31, pp. 23-24] – likely due to their focus on the concrete marketing benefits of the style of gamification they describe – none cited the approaches of Nicholson, Aparicio et al. or other theoretical frameworks, suggesting that more research is needed in order to evaluate whether or not these models are effective.

1.2.4 Gamification in Practice

As discovered by Hamari et al. [12] and Seaborn and Fels’ [31] literature reviews, the prevalent form of gamification in academic research today uses points, badges and leader-boards as incentives to increase enagagement and create competition in social contexts, but a number of studies also use elements of fantasy (story/theme), challenge (goals) and curiosity (audio/visual feedback) either in conjunction with the other elements or instead of them.

Liu et al. [17] integrate thematic elements into their study of the effects of gamification on an application designed to decrease energy usage in the home. They argue that the user is no longer just a “cog in an organizational machine”, but rather a partner and that the focus today is in user engagement in order to produce a lasting relationship between a system and its users. As such, their design focus is on motivating users and their study is underpinned by Ryan and Deci’s work on intrinsic motivation. The authors conclude that a topic or theme must be of interest to the user for gamification to be successful, which is in line with Deterding [8] and Nicholson’s [21] ideas about the importance of context when choosing how to implement game elements. Liu et al. also believe that (while important) elements like badges and leaderboards are insufficient to create a successful gamified experience – what is important is to make the application feel like a game, with an emphasis on fun and aesthetic qualities.

Another article which supports the importance of context in the application of gami-fication is Downes-Le Guin et al.’s study of the effects of gamifying a questionnaire [10]. Elements of fantasy were introduced in the form of a storyline, attractive graphics and a stylised avatar representing the respondent. While the gamified questionnaire was generally considered more fun than non-gamified versions, it was not shown to increase engagement and had a significantly lower response rate than non-gamified versions. A questionnaire was probably not the right context for this type of gamification (or possibly any gamification at all).

Li et al.’s [16] implementation and evaluation of a gamified tutorial system for a CAD package is interesting in its use of a consistent theme and background story throughout, as well as its use of audio and visual feedback. The user is given the role of constructing parts for the Apollo space program – a theme which is supported by the use of attractive graphics and progressive-disclosure to reveal the story gradually. Their description of the system highlights the desire to make it feel like an actual game, with elements described

(13)

as “arcade style” and compared to puzzle games. Li et al’s article is also one of the few studied for this thesis that references an actual game as a source of inspiration: Tetris [23]. The authors’ choice of game elements is heavily informed by Malone’s [18] strategy for effective user interface design.

Finally, Berengueres et al.’s study of a gamified recyling bin [2] is interesting because, unlike the majority of gamification studies, it did not involve the use of points, badges, leaderboards or any similar elements, relying instead on audio and visual feedback alone. The study reports a threefold increase in the usage of the gamified recycling system, sug-gesting that it is possible for gamification strategies not centered around points systems to be successful.

1.2.5 Applications of Game Technology

While Deterding et al. [9] exclude applications of game-related technologies from their definition of gamification, game-related technology has application in the development of non-game applications and has been used to enhance simulations in numerous ways. It is often used as an easy way of enhancing the graphical fidelity of a system and, because of the rapidly evolving nature of the game development industry, game engines are commonly kept up to date with advances in graphics technology [4].

Bijl and Boer present a survey of the usefulness of various game engine features in simulations from the perspectives of validation, analysis and marketing [4], which suggests that the marketing perspective benefits most from the use of features from game engines. The survey also indicates that, regardless of the perspective, textures, lighting and shadows are the most useful features. Features such as file converters and exporters as well as the various configuration possibilities of game engines are also presented as potentially useful features but were not included in the survey.

Bijl and Boer also describe a tool for simulating a shipping-container terminal (to be used for training purposes) which utilises the 3D-engine Ogre3D’s [22] graphical capabilities in order to present a realistic depiction of the system, increasing its usability by applying the engine’s support for camera and character control as well as for general performance improvements. The engine’s 3D animation features are also cited as beneficial in training scenarios using the simulation tool [4].

Several other examples of the use of game technology in a simulation context exist, but lack the detailed evaluation of specific features described above. Prasithsangaree et al. [25] use the game engine Unreal Engine [35] to develop a front end for running military simula-tions, utilising the engine’s networking capabilities to allow for distributed simulations as well as the engine’s visualization of skeletal dynamics, Newtonian physics and explosions. Hu et al. [13] use the game engine Unity’s [34] physics engine as the basis for running and displaying mechanical simulations as well as saving time on implementation by reusing predefined components of the engine.

(14)

1.3 Research Question 1.3.1 Background

The bulk of research on gamification suggests that gamification is an effective tool for increasing user engagement and enjoyment of software systems [12]. However, research has been focussed on the use of a limited subset of game elements (sometimes referred to as pointsification [27]), which encourages users to compete for points or complete tasks for rewards despite the psychological foundations of motivation research suggesting that intrinsic motivators are more effective [7]. On top of this, gamification has frequently been regarded as a layer on top of an already functional application, rather than a core part of it (e.g. Blohm and Leimeister’s gamified service bundles [5]). Additionally, game-related technologies have been shown to be useful when constructing simulation software [4]. 1.3.2 Aims

The thesis has two primary aims:

1. To investigate whether the use of game elements is effective in creating an enjoyable user experience in a tool for simulating the behaviour of mechanical systems. 2. To investigate whether a 3D environment using game-related technologies can aid in

the understanding and ease of use of such a simulation tool. 1.3.3 Limitations

While one of the potential uses of simulation tools is in science education, this thesis does not consider any specific educational benefits of the application of gamification and/or game-related technologies.

2 Method

This section describes the method used in the investigation of the aims stated in Sec-tion 1.3.2.

2.1 Prototype Development

A software prototype, Playmola, in the form of a browser-based application for simulation and visualisation of the behaviour of mechanical systems was developed. This prototype serves two main purposes: firstly to demonstrate a design strategy for gamification of a simulation tool based upon intrinsically motivating factors and the ways that game technology can support this and secondly to evaluate the benefits of using game elements in such an environment.

(15)

2.2 User Study

Based upon the prototype, a small-scale user study with eight participants was carried out in order to gather user experience data. Tullis and Albert [33] describe the three main characteristics of user experience as follows:

• A user is involved

• That user is interacting with a product, system, or really anything with an interface • The users’ experience is of interest, and observable or measurable [33, p. 4]

Tullis and Albert also describe a possible differentiation between usability and user expe-rience, where usability is considered to be the users’ ability to use a system to carry out a given task successfully and user experience is considered to be the broader view of the users’ interaction with the system, including thoughts, feelings and perceptions stemming from that interaction.

Since the aim of the study was to gather user experience data rather than to gauge educational or productivity benefits, results are largely in the form of self-reported data from the users relating to their experiences and perceptions when using the application. Tullis and Albert describe the importance of self-reported data in usability studies as follows:

Self-reported data give you the most important information about users’ per-ception of the system and their interaction with it. At an emotional level, the data may tell you something about how the users feel about the system. [33, p. 123]

As such, the format of the test (further described in Section 4) consisted of a number of tasks to be performed while under observation, followed by a questionnaire. Numerous general usability scales for post-testing questionnaires exist, as described by Tullis and Albert, but most focus on general usability, productivity and understanding of a system, rather than the aesthetic or emotional responses of interest to this study (responses which are, of course, also influenced by usability and understanding of the application). The questionnaire was inspired by the aforementioned scales, as well as tailor-made metrics used in previous studies of gamification like Li et al.’s study of a gamified tutorial system [16], which also examines user enjoyment.

2.2.1 Questionnaire

Qualitative data were collected through open-ended questions in a questionnaire adminis-tered after the testing session was completed.

Quantitative data were collected through Likert scale-based rating questions (as de-scribed by Tullis and Albert [33, p. 123]) in the post-testing questionnaire.

(16)

2.2.2 Observation

In addition to self-reported data, testing sessions were observed and recorded. During each testing session one of the test organisers primarily focussed on writing down observations of the users’ interaction with the system as well as the users’ verbal comments in order to gather further qualitative data on user behaviour and perceptions, which were then used to provide additional context when analysing responses to the questionnaire. Some specific factors observed were as follows (all based on the observer’s subjective judgement):

1. Apparent enjoyment. 2. Apparent frustration.

3. Difficulty when controlling/navigating the application. 2.3 Discussion

A disadvantage of using a tailor-made format for user experience testing, rather than an established scale such as the System Usability Scale (described by Tullis and Albert in [33, p. 137]) is that it is not possible to quantify the usability of the system as a whole and compare it to known benchmarks. However, since the study does not address on usability specifically and focuses on parts of the system in isolation (specific game elements and 3D techniques), rather than the application as a whole, this approach may be more appropriate than using an existing scale.

Another concern is the size of the study. While there is little consensus on the number of participants required for a high level of confidence in the results of a usability study, available research suggests that a small number of users is sufficient in most cases [15]. Since the aim of this study was not to exhaustively uncover problems in the prototype’s usability, but rather to determine general trends in how enjoyable and engaging it is, a small number of participants is sufficient.

The choice to use questionnaires rather than interviews in collecting qualitative data was based on a number of factors. Preece et al. [26] recommend using a questionnaire rather than an interview when studying specific design features of a system, which was accomplished by examining users’ responses to particular game elements, rather than the system as a whole. The use of well-designed open-ended questions allows a similar depth of data to be gathered as when using interviews, when combined with the results of observa-tions. Using a questionnaire also allows the test participants to be left alone (or supervised from a distance, so assistance can be offered if necessary) while filling out the answers, in the hope of encouraging honesty.

The idea of measuring intangible concepts such as fun is controversial since said con-cepts are highly subjective and not easily quantifiable. However, considering the benefits of intrinsically motivating factors, like fun, shown in research on gamification, measuring fun is highly relevant. Wixon [38] argues that if subjective experiences are to be consid-ered unmeasurable then there is no point in developing software designed to be enjoyable. Within the field of interaction design, Preece et al. [26] include fun in their definition of

(17)

user experience and consider it worth measuring not only by itself but also in relation to usability.

In their list of current methodological limitations in academic studies of gamifica-tion [12, p. 3029], Hamari et al. criticise the lack of non-gamified control groups and the application of multiple game elements simultaneously for making it impossible to de-termine the exact impact of gamification (or specific game elements) on a system. Seaborn and Fels [31, p. 29] also recommend that future research on the subject of gamification should attempt to isolate its effect, preferably with comparative studies. The simulation tool developed in this work does not however lend itself easily to a comparative study, since gamification is applied as a core aspect of its functionality, unlike many prior studies which focus on the application of gamification as as a layer on top of an already functional system (see, for example, Blohm and Leimeister’s concept of “gamified service bundles” [5]).

3 Implementation

In order to test whether game elements chosen to be intrinsically motivating along with a design strategy with gamification and the use of game technology at its core rather than as a separate layer of functionality can be effective in creating engaging simulation tools, a prototype named Playmola was produced.

Playmola is a tool for constructing physical (specifically mechanical) models, simu-lating their behaviour and playing back the animated results directly in a web browser. Simulations are performed by the modelling and simulation tool Dymola [20], through a JavaScript interface.

Playmola’s target user is a university undergraduate with an interest in physics and some experience of university level education in physics or applied mathematics.

3.1 Functionality

Dymola is an environment for modelling and simulation of complex physical systems from diverse engineering fields, which are described using the Modelica language [19]. Models in Dymola are constructed in a traditional 2D view where components are represented by icons connected by lines, or written directly in Modelica. When run, simulations can output a 3D animation, showing the behaviour of the model (see Figure 1).

Playmola is based on Dymola, but limits functionality to a small subset of mechan-ical components and provides a simplified 3D interface with the combined functions of constructing, simulating and animating models.

A model in Playmola consists of a number of components that can be selected from a menu (see Figure 2) and dragged into the scene. Each component has a number of connectors – points at which it can be connected to another component.

Each component in a model has a number of associated parameters that can be set using an input form as shown in Figure 3. Changing the component parameters is often directly reflected in the 3D environment – for example, changing the length parameter for a box will make it longer. The components used in Playmola are simplified versions of components from the Modelica standard component library, dramatically reducing the

(18)

Figure 1: Two screenshots of Dymola, the left showing a model represented by icons connected with lines and the right showing the animated output of a simulation.

Figure 2: Screenshot of Playmola, show-ing the menu of available components.

Figure 3: Screenshot of Playmola, show-ing the input form used to change a com-ponent’s parameters.

number of parameters the user can change to a subset of key parameters that allow the user freedom to experiment without becoming confused by an overload of information. These kinds of constraints are recommended by Podolefsky et al. in their discussion of the use of simulation tools in physics education [24]: “constraining what students can do reduces cognitive demands and frees up resources for sense making and development of an expert-like mental framework.”

While constructing a model, Playmola is in one of two modes: 3D View (see Figure 4) or Exploded View (see Figure 5), each representing a different level of abstraction. 3D View is the highest level of abstraction, where the only components visible are those representing

(19)

three dimensional bodies (boxes, cylinders or complex shapes) and these components are directly connected to each other (without intermediate lines or other components). All other components are hidden. Exploded View shows all components pushed apart in the manner of a technical exploded-view drawing/schematic. Connections between components are shown as lines in the same manner as in Dymola (see Figure 1).

Figure 4: Screenshot of Playmola, showing the 3D View for constructing models.

Figure 5: Screenshot of Playmola, showing the Exploded View for constructing models. Models constructed in Playmola are converted to Modelica code and sent via a JavaScript

(20)

interface for simulation by an instance of Dymola running on a server. If Dymola reports that the simulation was successful, further queries are sent to the server requesting inter-polated values of variables from the simulation (for example, angles and positions) to be used in animating the model. These animations can then be played back in the same view used for creating the model, as shown in Figure 6.

Figure 6: Screenshot of Playmola, showing a model mid-animation.

3.2 Use of Game Elements

Game elements used in the prototype are separated into categories inspired by Malone’s heuristics for designing enjoyable user interfaces [18].

3.2.1 Fantasy

Malone argues that the concept of fantasy, a system that evokes mental images of make believe objects and/or situations, is “probably the most important feature of computer games that can be usefully included in other user interfaces” and recommends using it to appeal to users on an emotional level. Playmola takes advantage of fantasy in a number of ways:

Theme/Story

The user is placed in the role of an enthusiastic amateur inventor, tinkering with machines in a small run-down workshop or garage (see Figure 7). Deterding [8] and Nicholson [21] stress the importance of using game elements that are appropriate for the user’s background and in an appropriate context. Since the main function of the system is to build and run simulations of mechanical models, a theme based around building/experimenting is

(21)

contextually appropriate. The setting was chosen to be recognisable to as wide a range of people as possible, but by never explicitly stating what the setting is, the user is allowed to draw his/her own conclusions about it based upon the graphical cues provided. The graphical cues in question include a workbench, numerous cardboard boxes, a screwdriver, a radio, dirty concrete flooring, dirty plaster walls and a corrugated metal ceiling.

Figure 7: Screenshot of Playmola, showing the application’s theme: a workshop, as well as the interactive elements in the scene: a radio for muting/unmuting music and a “big red button” for starting a simulation.

In addition to appealing to user’s emotions, the use of the workshop scene has several benefits for navigation and construction of models. It provides a clear sense of which direction is down in order to prevent disorientation when navigating in the 3D environment and also provides a reference for the measurements of the objects in the scene (for example the measurements of a box placed in the scene can be compared to the measurements of the workbench which is always present in the scene). Objects in the scene cast shadows which provide a visual indication of their relative locations.

Metaphors

In addition to using fantasy to induce an emotional response, Malone points out that using fantasies analogous to real-life objects or concepts (metaphors) can make a system easier to use. Wieman and Perkins [36] also highlight the importance of using “connections to real life objects” in the creation of engaging simulation tools. In Playmola, a number of objects in the scene are interactive and serve as metaphors for functionality: a 3D model of a radio can be clicked to mute or unmute the background music, a 3D model of a “big red button” [3] can be clicked to start a simulation (see Figure 7).

(22)

Audio/Visual effects

Music, sound effects and visual effects used in Playmola are chosen to fit the theme and thereby enhance the sense of fantasy. For example, when objects are connected a “welding” sound is played along with a visual effect representing sparks, see Figure 8. While this may not be physically realistic for the kind of connection being made, the intention is to evoke an atmosphere consistent with the workshop theme. Music played while a simulation is being run uses metallic “clanking” noises to contribute to the atmosphere.

Figure 8: Screenshots of Playmola showing visual feedback. On the left: an animated highlight/translucency effect when an object is selected. On the right: visual effect resem-bling sparks when two objects are connected. The sound of welding is played along with the effect.

3.2.2 Curiosity

Malone defines “sensory curiosity” as an effect invoked by audio and visual effects in a game and suggests that it can be used in three main ways:

• As decoration – i.e. to enhance aesthetic qualities of the application. Wieman [36] supports this in writing about the importance of “an appealing environment and rea-sonably sophisticated graphics” when making an engaging simulation tool.

• To enhance fantasy.

• As a representation system – either through the use of metaphors or as a means of providing feedback in response to events.

Playmola’s aesthetics are enhanced by background music and sound effects fitting the application’s theme, coupled with the visual effects shown in Figure 8. These audio/visual

(23)

effects also provide additional feedback in response to user interaction. When connecting objects, valid connections are highlighted in a variety of ways (depending on the method of connection), some of which are shown in Figure 9. A clicking sound plays to indicate that a connection has successfully been made. Figure 8 also shows interaction-driven visual feedback in the form of a visual effect meant to represent “welding” which is played when two objects are connected.

The application’s background music also changes dynamically, based on the state the user is in. While creating a model (the majority of the time spent using the application), unobtrusive piano-based background music is played. In order to signify the change in state when a simulation starts, the music changes to a similar but more energetic tune, with industrial “clanking” noises, to reinforce the workshop context. If the simulation fails, the music will return to the normal background music (along with an error tone).

Figure 9: Screenshot of Playmola, showing visual feedback when connecting objects. Avail-able connections are highlighted and the connection-line changes colour when it is near enough to a valid target to form a connection.

3.2.3 Challenge

A common element of proposed gamification frameworks is challenge [1, 5], something that Malone presents as one of the key features of games that can be used in effective user interface design. Challenge is usually defined as the presence of tasks with clear goals (and often with feedback regarding how close to achieving a goal the user is). Since the focus of this study is on intrinsically motivating factors and the purpose of the application is not to complete specific predetermined tasks, explicit goals are not used. Challenge is however still present in the system by virtue of users setting their own goals: creating models which can be successfully simulated and which exhibit the expected behaviour.

(24)

While not specifically an application of gamification, this element of challenge may have positive benefits on the users’ enjoyment of the system.

3.3 Use of 3D Technology

Wieman and Perkins describe sophisticated graphics as one of the critical features of an interactive simulation [36], so tools facilitating the creation of attractive 3D environments were required. Numerous engines and libraries exist for creating 3D applications and, as described in Section 1.2.5, game engines have previously been used for creating simulation tools.

Due to the need to run directly in a web browser without a plugin, Playmola required an engine either written in JavaScript or capable of exporting to JavaScript. three.js [32], a popular JavaScript 3D rendering API based on WebGL, was chosen for Playmola’s devel-opment since it provides the required graphical functionality and Dymola already includes functions for exporting 3D models in a format compatible with three.js. While not specif-ically a game development framework, three.js contains the graphical features expected from such a framework, including: lighting, shadows, texturing and import of 3D models in a variety of formats. The main feature missing from three.js that would have facilitated development is a graphical editor for positioning objects in a 3D scene – a feature which is present in most modern game engines.

Another promising choice was the game engine Unity [34], but support for exporting code which can be run natively in the browser was not included until after development of Playmola began, thereby ruling it out. Larger-scale game engines such as Unreal En-gine [35] also allow development of applications that can run natively in a web browser, but also provide a large amount of extra functionality not required for an application of Playmola’s scale and have a steep learning curve.

4 User Study

This section describes the format of the user study introduced in Section 2.2.

Eight participants took part in the user study, consisting of two mechanical engineering students with experience of courses in physics and mathematics, four first-year game-development students who had completed a course in game physics and two third-year game development students who had completed an introductory physics course. Due to the nature of the simulation tool’s intended user – a person interested in physics and with some experience of education in physics and/or applied mathematics – and the fact that the prototype assumes some knowledge of linear algebra and mechanics, test subjects were chosen based on having taken at least one university-level course in physics.

Each test session lasted between 40 minutes and one hour and consisted of the following process:

• Playmola was introduced and described as “a tool for constructing and running me-chanics simulations”. Care was taken not to describe the tool as a game or to use game-related terminology.

(25)

• A brief (approximately 10 minute) introduction of Playmola was given, in which one of the test organisers followed a scripted overview of controls and functionality and demonstrated how to construct two simple models – a pendulum and a box sliding down an inclined plane. This tutorial-style introduction took the place of a traditional built-in software tutorial, which was not present in the prototype and was intended to speed up the initial learning-process.

• The user was then given a short period of time to get acquainted with the tool and was encouraged to ask questions. The test organisers used their judgement to decide when the user was ready to progress to the next stage, generally taking between 10 and 15 minutes. To aid the familiarisation process, the user was given a printed sheet of pictures and descriptions of the functionality of all of the components present in the system (again as a replacement for a built-in tutorial or guide).

• The user was given two tasks to complete in order: a wheel rolling down an inclined plane and a model of a swing from a fairground ride. These models were chosen firstly for their simplicity, to allow testers with limited knowledge of physics to complete them and for the fact that the second model would produce movement in all three dimensions, allowing the effectiveness of the 3D interface to be properly studied. For each task, the user was presented with an image of the final model and a schematic of the components to include and how they should be connected. This stage took between 10 and 20 minutes to complete.

• Upon completion of both tasks, the user was asked to fill in a questionnaire con-sisting of a mixture of ranking-based questions rating a statement according to a five-point Likert Scale (e.g. from Strongly Disagree to Strongly Agree), yes/no ques-tions and quesques-tions requiring longer written answers. This stage took between 10 and 15 minutes to complete.

During the testing sessions, the second test organiser made written observations of the behaviour and actions of the user. An audio-recording of each test session was also made.

5 Results of the User Study

This section presents the results of the user study described in Section 4. 5.1 Ranked Questions

Nine ranked questions (see Figures 10-18) were answered using a five-point Likert Scale, from “Strongly Disagree (1)” to “Strongly Agree (5)” (with the exception of question 9, see Figure 18). All questions were answered by all participants.

Users were given the opportunity to expand upon visual feedback in relation to question 1 (see Figure 10). No comments were made on the use of visual effects like the sparks effect seen in Figure 8. Several users commented on how the Exploded View construction mode made it easier to understand how their models worked. One user found it useful being able to simulate and construct in the same view, so that changes to parameters could be made

(26)

directly after watching the animated simulation. Some users felt that more visual feedback was needed, for example to indicate the cause of the error when a simulation fails.

Users were also given the opportunity to expand upon audio feedback in relation to question 2 (see Figure 11). Responses were varied, but most users were indifferent to the application’s use of music and sound effects. All realised that there was sound, but one commented that he hadn’t heard any sound except for the background music, which he didn’t consider to have any effect on his ability to understand the application. Several users stated that the sound effect played when a simulation fails was useful.

Figure 10: Question 1 Figure 11: Question 2

(27)

(28)

5.2 Other Questions

Qualitative data were gathered from the responses to the remaining survey questions and are presented here. All answers have been translated from Swedish.

10. How much physics/mechanics have you studied at university level?

All users had some experience of physics and/or mechanics, where at minimum one course had been taken and the most experienced were two mechanical engineering students with regular experience of physics at university level.

11. Do you feel you understood the concepts involved in the tasks?

All users wrote that they understood the concepts involved after some time exper-imenting and with the help of the information we provided along with the tasks. Those with more experience of physics were more confident in their understanding, while several of those studying game development stated that they would not have been able to complete the tasks without (verbal and written) assistance.

12. How did it feel to navigate in Playmola? Was the interaction natural or forced? The test participants were quite consistent in agreeing that navigating the interface had somewhat of a learning curve, but most felt that the time provided in the testing session was sufficient to become used to it. Only one user felt that significantly more time would be required to become familiar with navigating the interface. Consistently criticised was the inability to move the camera left and right (instead the camera rotates around a fixed point and can be zoomed in and out), which several testers felt was necessary when attempting to construct a large model. One user also had difficulty getting used to the different functions provided by the application’s two different construction modes (3D View and Exploded View, see Section 3.1).

Several testers noted that more information in the form of instructions or tool-tips would have made the interface easier to understand.

13. How would you describe the application’s setting/theme?

All eight users were able to accurately describe the setting as a “workshop” or a similar term (words used included: “workshop”, “tool-shed” and “garage”) conveying the idea of an environment where things are made – particularly an informal or personal setting, rather than an organised large-scale factory.

In addition to identifying the setting, several of the users also described enjoying the setting, using words like “charming”, “pleasant” and “great”. One user in particular described how the environment actively enhanced the user experience: “You come into a calm environment with calming music which isn’t distracting. It makes it easier to concentrate on what you actually want to do.”

One user also mentions a game, Garry’s Mod [11] (a sandbox game, where the player has freedom to do as he/she pleases without explicit goals, not dissimilar to the kind of free experimentation Playmola allows), as a reference point for his interpretation of the setting.

(29)

14. Do you think the theme was suitable for the application?

Everyone felt that the theme fit the application well. One user commented that the experience of using Playmola wouldn’t have been the same without the theme: “Very fitting for sitting and experimenting with the program. Without the theme it probably wouldn’t really have had the same feeling.” Another felt the theme helped him focus on the tasks: “You get into your little workshop and can concentrate. There’s nothing to distract you or make you lose focus.”

15. Did you enjoy using Playmola?

All but one user described themselves as enjoying Playmola including three users who expressed an interest in continuing to use it, including both mechanical engineering students. One user (a game-development student) felt that a tool like Playmola would be useful to help him understand mechanics: “Very entertaining. I could see myself using a tool like this for testing out ideas or learning about how objects behave in certain mechanical situations.”. The remaining user didn’t feel he had been given enough time with the program to formulate a decision about whether or not it was enjoyable – also mentioning that it didn’t feel like a program that was supposed to be entertaining, but rather a tool for work.

16. Do you think the application’s complexity was suitable for your current skill level? Seven out of eight users answered “yes”. No opportunity was given to expand upon this answer.

17. How many hours per week do you spend playing games?

All the users play games regularly. The user who played games the least estimated 2-3 hours per week and the user who played the most estimated 20-30 hours per week. Most played between 5 and 15 hours per week.

18. What kinds of games do you usually play?

Out of a small selection of genres (strategy, action, puzzle, sport, role-playing and other) the most popular were action (7 users) and puzzle (5 users) and the least popular was sport, with no users reporting that they play sports games.

19. I would use Playmola again. All users answered “yes”.

20. Playmola has made me more interested in physics/mechanics. Seven out of eight users answered “yes”.

21. General Comments

Several users reiterated their enjoyment of using Playmola and a number mentioned that it could be useful as a learning tool (it was not presented as such): “I hope this will be used in the future for engineers and physicists to be able to perform

(30)

simulations in an educational context” and “it would be a really nice tool to use for simulating problems when studying”.

One user felt there was not enough content in the application and that more help from built-in instructions and tool-tips was necessary.

5.3 Observation

Every user test was accompanied by observations focussing on the interaction between the user and the system. One of the test organisers was primarily responsible for writing down these observations.

Two different ways of starting a simulation are available to the user, either by clicking a 3D object in the scene or by clicking a more traditional HTML button in the GUI. A majority of the testers used the traditional GUI button most of the time, rather than the 3D button in the scene. In some cases it was not entirely conclusive, with some users switching between the one and the other. One user used the button in the scene exclusively. The mouse can be used to change the camera’s orientation. Some of the users appeared confident using this method to orient themselves in 3D space. This made it easier for them to orient models and connect them together. A user’s ability to orient the camera with the mouse did not necessarily correspond to their impression of the ease of navigation. Some users rarely changed the camera perspective, except when absolutely necessary (because an object was hidden behind another object), while others rotated the camera much more naturally. Both mechanical engineering students made little use of the camera controls, while the third-year game development students were much more likely to rotate the camera to see a model from a different angle.

When a simulation fails in the application, an “error” sound is played. All of the participants understood the first time they heard this sound that the simulation had failed. It is interesting to note that one user claimed not to have heard any sound except for the background music, despite having responded as anticipated to the error sound when using the application. On the other hand, several users became confused when their simulations succeeded but did not result in any movement (because of the physical properties of the model they had created) – assuming that it was the result of a bug in the program.

The application allows the user to switch between two modes when constructing models (see Section 3.1), each providing access to different functionality. Most users spent almost all their time in the Exploded View, which provides more functionality and some became confused by the functionality missing in 3D View (fewer components are available). A small number switched regularly between modes in order to better visualise the complete model.

6 Analysis

This section presents an analysis of the results of the user study. Firstly, the effects of specific game elements are analysed according to the categories of game elements defined by Malone [18], as discussed in Section 3.2. Secondly, the effects of using 3D techniques are analysed. Finally, the overall effects of gamification on user experience are analysed.

(31)

6.1 Fantasy

6.1.1 Theme/Story

The theme of a workshop was used, making use of visual and audio elements to invoke a specific atmosphere with the goals of being recognisable and contextually appropriate for the kinds of activities performed in the application (experimenting with physical mod-els). Answers to the question “How would you describe the application’s setting/theme?” revealed that the application’s implementation of a setting was effectively conveyed, with every user independently identifying the setting as a tool-shed, garage or a workshop. Users also commented on how the atmosphere of the program invited them to create things and experiment, encouraging the sort of playful/experimental behaviour that gamification can be used to motivate. Most of the users did feel that the theme contributed to their enjoy-ment of the application (see Figure 18) and this is further supported by comenjoy-ments made about pleasant atmosphere of the setting. One user also commented that the theme had made it easier for him to concentrate on the tasks - stating also that the application would not have been as enjoyable without the setting.

6.1.2 Metaphors

The application’s use of visual metaphors takes the form of interactive objects designed to resemble real-life objects (as recommended by Malone [18] and Wieman and Perkins [37]) with easily recognisable functions - implemented here in the form of a the radio and the “big red button” as shown in Figure 7. Users were asked not to interact with the radio (which toggles the application’s audio), since the effects of audio were being studied, but the function of the button (starting a simulation) was explained and demonstrated. While it would have been interesting to be able to analyse whether or not users chose to disable the music, it was decided in advance of the testing sessions that data regarding user responses to the context-sensitive music would be more useful. The interface also included a more traditional HTML button with the text “Simulate”. As noted in section 5.3, only one user consistently used the interactive object rather than the traditional button and others used it very infrequently. This is unsurprising since the traditional button is more recognisable to a regular computer user, even though none of the users felt the 3D object was out of place in the application’s setting. Additionally, the position of the 3D object moves as the user adjusts the camera angle (since it is attached to the scene’s workbench), possibly making it harder to locate than the traditional button, which is always in the same screen location. It is also possible for other objects placed in the scene by the user to obscure the interactive objects, making them harder to use. The one user who did use the 3D object rather than the traditional button most of the time was also observed to reposition the camera very infrequently, ameliorating the aforementioned problems in this case, but not representing behaviour consistent with the way most users interacted with the application, or the anticipated behaviour.

(32)

6.1.3 Audio/Visual Effects

The background music received only one comment related to its relevance to the theme of the application, stating that it contributed to a relaxed setting.

No user commented specifically on the use of a sound effect as an indication of a failed simulation, with one even stating that he had heard no sound effects apart from the background music during his use of the application. This contradicts the observations made of the users since every user was observed to react to the sound effect. This contradiction may be related to the sound effect being short and dissonant, catching the attention of the user as it plays but not making a lasting impression. It should also be noted that sound effects conveying error or success messages are not unusual in applications and certain not restricted to gamification, so the fact that the users are used to hearing this kind of sound effect may have made them less noticeable.

6.2 Curiosity 6.2.1 Decoration

Decoration refers here to the use of audio and visual effects to enhance the application’s aesthetic qualities. While it is difficult to draw any conclusions about the precise con-tributions of isolated decorative elements on the users’ perceptions of the application’s attractiveness, comments made about the theme and audio/visual feedback provide some indications. In addition to correctly identifying the application’s theme, users described it as “charming” or “pleasant” and one explicitly commented on purely decorative elements:

...it was fun that you put energy into small things like a radio or boxes in a corner. Dirty and broken windows made it a bit ‘mysterious’. It would have been extremely boring to have the usual standard 3D environment which never ends.

The same user also felt that the interface was difficult to navigate and was the only user to explicitly state that he felt he hadn’t been given enough time with the application, indicating perhaps that an increased focus on decorative aspects actually had a negative effect on understanding the application. Another user said about the setting: “probably needs to be made a bit more cosy”, suggesting that further work might need to be done to make a more aesthetically pleasing environment.

Comments about the application’s music and sound effects were mixed. One user described the music as “nice”, but another felt that it was a little annoying after hearing it for 45 minutes. There were no comments about the aesthetic qualities of the sound effects, most of which were not actively perceived by the users at all. A lack of comments about music and a general lack of realisation that there were sound effects connected to certain actions may indicate that the application’s audio hit a good balance between being contextually appropriate (and providing useful feedback) and being distracting – this says little about the aesthetic qualities of the audio though.

(33)

6.2.2 Representation System/Feedback

Seven of eight users agreed or strongly agreed with the statement of question one: “Visual feedback made it easier to understand Playmola” (see Figure 10), suggesting that the use of visual feedback was effective. However, responses to a follow-up question giving an opportunity to expand upon this question revealed that users interpreted the question in different ways, commenting on Exploded View or the animated results of a simulation rather than on the use of visual effects like sparks or the selection highlight.

Responses to the statement of question two: “Audio feedback made it easier to under-stand Playmola” (see Figure 11) were varied, with the average slightly agreeing with the statement, even though in a follow-up question none of the users had any specific comments to make about the effects of audio on their perception of the application.

6.3 Challenge

No challenges, in the sense of tasks with clear goals, are present in the application. The tasks presented to the users during the user study did provide some level of challenge but will not be analysed from a gamification perspective since these are not an inherent part of the system. Some insight into the users’ subjective experience regarding how difficult, frustrating and fun they considered the experience to be can be found in Section 6.5. 6.4 Technology

The use of a single, unified 3D view for construction and animation of models - instead of the disconnected 2D view for construction of models and 3D view for animation of simulation results seen in Dymola - appears useful from the perspective of helping new users understand the application. All users agreed that the 3D environment made it easier for them to visualise the models, one pointing out that the unified view made it easy to tweak parameters in a model and get direct visual feedback reflecting the changes. Observations also showed that the ability to view the complete model in 3D allowed users to quickly discover the source of any issues.

In addition to the benefits of a unified view, the majority of users agreed that the use of shadows and lighting helped them to position the models, demonstrating that more advanced graphical features present in modern 3D libraries or game engines can be useful when constructing 3D simulation tools - agreeing with Bijl and Boer’s conclusions [4]. Given the increased difficulty of navigation in a 3D environment compared to a 2D one, these kinds of visual aids may be crucial in helping users understand how to manipulate and position objects in a 3D scene.

Despite the positive aspects, it took users some time to become accustomed to navi-gating in the 3D environment, possibly corresponding to the users’ familiarity with games and/or game development, since the mechanical engineering students seemed to take longer to get used to using the system than the game development students. A user’s ability to navigate the system did not, however, correspond to the user’s expressed opinion of the ease of navigation at all times, with one user who struggled with the controls describing the navigation as “very easy” and one who was used the controls more naturally describing it

(34)

as “complicated at first” (this user was also the only one to describe the tasks as frustrating - see Figure 16).

6.5 User Experience

When considering how enjoyable the users found Playmola, it is interesting to consider their perceptions of what kind of application they were using. Throughout the user study, Playmola was described to users as a “tool” rather than a game, but responses to the questionnaire and other observations made it clear that some users had the impression that it was in fact a game. One user in particular consistently referred to “the game” in his responses the the questionnaire, while a number of users verbally referred to Playmola as a game during and after the sessions. Another user wrote “It felt more like a work-tool than an entertainment program”, implying that he was under the impression that the principal purpose of the application was entertainment. All users were aware that the application was developed by game development students, which may have led to assumptions about the nature of the work, but confusion about the nature of the application may also have been a result of it feeling like a game, due to the use of game elements and experimentation-based functionality which resembles “play” (it should also be noted that a general person may not appreciate the differences between “games” and “play” and use the word “game” when experiencing playful environments).

Questionnaire responses confirm that users found the tasks fun to complete (see Fig-ure 17) and, when asked how much they felt that the theme had contributed to their enjoyment, the users responded with an average of 3.65 on a scale from 1 (not at all) to 5 (significantly). This, along with observations and comments, suggests that the gamified setting played a role in users’ enjoyment of the application. While, on average, the users found the tasks neither easy nor difficult, all but one agreed that they were not frustrating, suggesting that the tasks were of a difficulty level suitable to not negatively affect enjoy-ment. Malone’s [18] ideas on the importance of challenge in good user interface design support that an experience targeted correctly to the users’ abilities (that is, challenging, but not frustrating) can lead to a more enjoyable experience.

All users stated that they would use the application again and two expressed an in-terest in being able to use it again before being asked, further reinforcing results that the application was enjoyable to use. Additionally, seven of eight said that Playmola made them more interested in physics/mechanics - a desirable result given that one of the aims of gamifying an application is to increase users’ engagement and interest [9] (in this case by making it more enjoyable for them to use).

7 Discussion

The use of gamification appears to be a successful strategy for developing enjoyable simula-tion software with users responding well to the presence of a theme and the visual stimuli. Malone’s suggestion of using the concept of fantasy to appeal to users’ emotions [18] is supported by the results and the use of a relatable environment did have a positive effect on the user experience.