Teknik och samhälle
Datavetenskap och medieteknik
Examensarbete 15 högskolepoäng, grundnivå
Mixed-initiative quest generation
A mixed-initiative quest tool for the evolutionary dungeon designer
Eric Grevillius
Elin Olsson
Examen: Kandidatexamen 180 hp Handledare: Alberto Alvarez
Huvudområde: Datavetenskap Examinator: Reza Malekian
Program: Datavetenskap och applikationsutveckling Slutseminarium: 28/8 - 2020
Abstrakt
Ända sedan 1960-talet har idén om ett symbiotiskt partnerskap mellan dator och människa presenterats och att detta partnerskap kan ge lösningar bättre än människan ensam kan. Detta symbiotiska partnerskap har spridit sig till processuell generering (PCG) genom angripningssättet “blandade initiativ”, där människa och dator turas om bidra med lösningar. Inom spelutveckling kan viss innehåll skapas bättre av en generator och en annan del av en människa. Denna forskning fokuserar på att tillämpa den “blandade initiativ” metoden för att skapa uppdrag i “dungeon” spel, genom den utvecklade artefakten kan användaren skapa uppdrag i “Evolutionary dungeon designer” (EDD) för att designa nivåer av spelgenren “dungeons”. Artefakten använder en generator för att ge användaren automatisk genererade förslag. Generatorn har utvärderats genom ett “expressive range”-experiment som utvärderade dominansen av de aktioner som fungerar som byggstenar för uppdragen. Utöver experimentet genomfördes en användarstudie för att utvärdera artefaktens användbarhet. Mottagandet av artefakten i användarstudien var positivt. En majoritet av deltagarna upplevde en ökad kreativitet och beskrev artefakten som ett resurseffektivt verktyg för spelutvecklare, som bidrar med snabba lösningar och hjälper till att motverka inspirationsblockeringar.
Abstract
Ever since the 1960s the idea of a symbiotic partnership between computer and man has been laid out, suggesting a partnership can provide solutions better than man alone can. This symbiotic relationship has been branched out to procedural content generation (PCG), through it’s “mixed initiative” approach, taking turns to provide suggestions. Within game development, some content is better created by a generator, and some by a human. This research focuses on applying the mixed initiative approach in quest creation in dungeon games, through an artefact that lets the user create quests in “Evolutionary dungeon designer” (EDD) to design dungeons in dungeon games. The artefact developed uses a generator to provide the user with automatic generated suggestions. The generator has been evaluated through an expressive range experiment to investigate the dominance of the actions which acts like building blocks for the quests. In addition to the experiment a user study was conducted. The result of the user study was that the experiences relating to the artefact were positive. A majority of the participants experienced increased creativity and described the artefact as a resource efficient tool for game developers, providing fast solutions and helping reduce inspirational blockages.
Abstrakt 1
Abstract 1
1. Introduction 3
2. Related work 4
2.1. RPG, adventure games & dungeons 4
2.2 Quests 4
2.3 PCG & Grammars in dungeon games 5 2.4 Mixed-initiative co-creation 6
3. Methodology 7
3.1 Design Science 7
Guideline 1: Design as an Artifact 7 Guideline 2: Problem Relevance 8 Guideline 3: Design Evaluation 8 Guideline 4: Research Contributions 8 Guideline 5: Research Rigor 9 Guideline 6: Design as a Search Process 9 Guideline 7: Communication of Research 9
3.2 Evaluation 9 3.2.1 Experiment 9 3.2.2 User study 9 3.3 Methodology discussion 10 4. Artefact 10 4.1 Design 11 4.1.1 EDD 11 4.1.2 Introduction 11 4.1.3 New Tiles 11 4.1.4 Actions 11 4.1.5 Generator 12 4.2 Workflow 12 4.3 User Interface 12 4.3.1 EDD navigation 12 4.3.2 Quest-tool implementation 13 4.3.2.1 Action tiles 13 4.3.2.2 Action panel 13 4.3.2.3 World panel 14 4.3.2.4 Suggestion panel 14 4.3.2.5 Sequence panel 14 4.3.2.6 Toggle Menu 14 4.3.2.7 Erase & Back 14
5. Results 18 5.1 User study 18 5.1.1 Design 18 5.1.2 Users 18 5.1.3 Process 18 5.1.3.1 Introduction 18 5.1.3.2 Instruction 18 5.1.3.3 Questions 18 5.1.4 Limitations & threats to validity 19 5.1.5 Result of user study 19 5.1.5.1 Pre-questionnaire 19 5.1.5.2 Manual creation of quests 19 5.1.5.3 Automatic creation 20 5.1.5.4 Mixed creation 20 5.1.5.5 Automatic suggestions 21 5.1.5.6 Quest actions 21 5.1.5.7 Usability 22 5.1.5.8 Creativity 22 5.1.5.9 Overall experience 22 5.1.6 Analysis of the user study 23
5.2. Experiment 25
5.2.1 Expressive range 25
5.2.2 Method 25
5.2.3 Result of the experiment 26 5.2.4 Analysis of the experiment 28
6. Discussion 29
7. Conclusion & Future work 35 8. Acknowledgements 35 9. References 37 10. Appendices 40 Appendix A: Questionnaire 40
1. Introduction
The increasing usage of Procedural Content Generation (PCG) in both research and industry [1] has shown successful results regarding the efficiency of the game development process [2] but also to generate endless variations of a game, therefore making games “infinitely” replayable [3]. For example animation and the environment is taking a large part of a development budget [2], which PCG can produce solutions efficiently. PCG can generate game content quickly, however some parts are still best made by humans[4]. One solution is using a mixed-initiative approach. Which focuses on taking turns that are negotiated rather than determined by a single party regarding the modality of interaction [5], in the case of game development, where the computer and human co-creates a solution to a problem. However, the two actors' contributions do not need to be the same [6].
Examples of previous game development tools using mixed-initiative co-creation (MI-CC) are Sentiment Sketchbook [5], Tangara [7] and the evolutionary dungeon designer (EDD) [8]. EDD is a mixed-initiative dungeon designer for making dungeons for adventure games. The tool lets the user manually design rooms with enemies, treasures, chambers and an overall world structure that connects the different rooms. For the computing party of MI-CC, EDD uses evolutionary computations to procedurally generate content suggestions. The two parties collaborate and the evolutionary algorithms provide the user with alternatives, based on symmetry or similarity criteria. Currently, EDD does not have any narrative or story, which limits EDD’s potential, functionality and creative usage for the user. Narrative and quests are important to bring meaning into a game for the player as presented by Howard [12]. On a bigger perspective, several research on quest generation using PCG have been
conducted such as Parberry & Doran [9], Braualt et al [31], Doran [10], but none have been through mixed-initiative, thus making this research a starting point for a new research area to be explored within both PCG, mixed-initiative and narratives. Not only adding contributions to the research community but for the video game industry and game designers as well. Our research could for example, benefit smaller game development companies, with having a tool that uses the time and resource advantages of PCG, but still letting the designer make their own mark with their preferred style with the development tool. As one challenge of PCG in games is that it could ultimately lead to uninspiring and uncreative content [27], our research will contribute to shifting from an automatic procedurally generated content to mixed authorship with the creative collaboration in focus.
This paper will focus on developing and evaluating a quest feature implementation to EDD using design science as a methodology. Previous research regarding procedural generated quests and puzzles has been conducted [1], [9], [10], [11], and will act as a research base for quests structure and PCG quests. The tools quest generation will be based on previous research conducted by Parberry & Doran with their classifications of RPG quests [9]. However, our tool will be done with mixed-initiative and not only automatically generated as many quest generations have been, such as Parberry & Dorans[9] and
Breault et al. [31]. Our tool lets the user place different quest actions, such as “GOTO”, ”KILL”, “STEAL” [9] in a list. This list will be generated in the same order as the player has clicked them, thus creating a sequence of quests. The procedurally generated actions will be based on grammars and thus have pre-defined sequence rules, inspired by both Dorman’s [20] work on action-adventure games and Parberry & Doran [9]. The generated quests will display the different series of quests available to the user in a list. The tool scans the room, thus eliminating
quests that have a clear pre-condition, such as for the “KILL” action an enemy must have been placed. The user has the option to either use the generated actions, to manually place out different actions or use a combination of both to build up a full quest. As opposed to previous quests generation, we will focus on the turn-taking and creativity the mixed-initiative system will achieve during the development process and the outcome, rather than generating as many quests as possible automatically.
Due to the previously presented research gap, the following research questions have been identified:
● How can mixed-initiative be featured in a quest tool for EDD?
● How can a mixed-initiative quest tool improve the quality of quest creation?
● How can a mixed-initiative quest tool be used to help designers when designing dungeon games?
In the following sections, we will present the previous work in the related research fields, research methodology, results, discussion, and conclusions.
2. Related work
2.1. RPG, adventure games & dungeons
The genres where quests appear are mainly adventure games, action-adventure games, and role-playing games (RPG) [12]. Both RPGs and adventure games have a rich storyline, detailed characters, and involve exploration [13]. RPGs are a genre developed from the pen and paper role-playing games such as Dungeons and Dragons [4], where dungeons first were introduced as game levels within the genre. Dungeons as game content can be defined as a single level or set of levels that are set in an underground complex and is connected through an overworld with cities or a wilderness [14]. The cities can act as a replenishment zone where the player can
do trades using found items or item upgrades. Dungeons can contain enemies, treasures, hidden passages, puzzles [14], decorations, and Non-playable characters (NPC) [4], thus creating space that allows the player to explore the unknown areas[14]. Dungeons consist of five different types: connecting rooms, rooms & corridors, labyrinths (unicursal structures), mazes (multicursal structure), and open areas [14]. The different types appear to a various extent in different games. The dungeon’s topology could either be dense or scarce, depending on the amount of traversable game space in a dungeon’s layout [14]. Dungeons are a popular level design, especially within PCG [4], [13],[15], where it has been present in popular games ever since the 1970s [16].
2.2 Quests
The word quests come from the Latin word “questare” meaning “to seek”, which suggests a goal-oriented search for something of value [12], Other definitions are Aarseth’s “a game that depends on mere movement from position A to position B” [13] and Howard’s “A quest is a middle term, a conceptual bridge that can help to
join together many two-part or binary pairs [..] these include game and narrative, gaming and literature, technology and mythology and meaning and action” [12] . Quests are usually a part of role-playing games, however, in other games, the word “mission” is sometimes used instead [16]. The history of quests in games trails from a tradition of quests narratives in literature-form, such as The Odyssey and the quest for the Holy Grail. This link between quest narrative, quests, and quest games has been proposed by Howard [12], arguing that quests unify both meaning and action. The meaning hails from strategic actions with thematic, narrative and personal implications, and actions being those that are meaningful for the player on the level of ideas, personal ambitions, benefits to society, and spiritual authenticity. Howard argues that game designers can produce meaningful actions
by taking inspiration from strategies derived from quest narratives [12]. Quests can be linked together like a chain to advance in the game’s story further [16] and give structure by limiting the player’s available choice through providing access to certain areas only in a specific order(s) [16], thus making the game designer take control of the players’ agenda [13]. Besides being a fundamental element for narrative progression[17], quest affects the amount of space needed for a game’s landscape [10], [13]. The landscapes in quest games can be either linear, semi-open, or open and the design is structured by the types of quests featured in a game [13].
There are three basic quests types as described by Aarseth [13].
● Time ● Place
● Objective-oriented
These three quest types can be combined to create additional quests, thus making the total 7 different types.
● Time + place (“Get there before”), ● Time + objective (“Get it before”), ● Place + objective (“Get there and”) ● Time + Place + objective (“Get there
before.. and…”).
Besides these seven types, a further classification for RPG quests has been made [9]. Parberry & Doran found 9 different “motivations” from NPC´s which resulted together with a specific strategy in a “verb-noun” pair, for example, “steal supplies”. This derived into twenty atomic actions [9]. A quest can be broken down into several side-quests [9], and be optional and unrelated to the main story-line [15], [18].
2.3 PCG & Grammars in dungeon games
PCG can be defined as “The algorithmical creation of game content with limited or indirect user input”, defined by Togelius et al. [19]. Content can be for example levels, maps, game rules, stories, items, quests, weapons, and characters. However, NPC AI
is not considered as content [4]. Examples of games with procedurally generated content and dungeons are The Elder Scrolls V: Skyrim, Diablo, and Rogue [4]. Skyrim uses PCG to create missions and adventures [27], Diablo to create maps and the type, number, and placements of items and monsters[15]. Rogue is a classic example of early use of PCG with dungeons and even spawning a genre called Roguelikes [4]. In Rogue, the dungeons, placements of items and creatures are procedurally generated every new game [16]. These successful games and the unique challenges in their design have made dungeons an active and attractive PCG subject [4].
PCG has several advantages such as games can be produced faster and cheaper, thus making it possible for smaller teams without resources of large companies to create content rich games [27]. It can also increase replayability and become a design tool to assist designers, such as mixed-initiative systems [4]. The usage of PCG has been investigated by Aruajo & Souto through a case study of No Man’s Sky and state of the art study [27]. Aruajo & Souto have proposed three recommendations for PCG in games with the desired effect.
● Basic - When PCG is used to generate content, thus making developers able to work on predetermined templates instead of creating from scratch. Aruajo & Souto argue smaller studios with no or little PCG usage should focus on shorter games with interesting core design.
● Intense - When PCG is used to increase game time and enjoyment but does not detract from possibilities provided by the game. ● Core - When PCG is used as a core
element of the game. This requires a larger fine turning to make sure the players interest in the game is kept over the curse of the gameplay.
However, there are challenges of PCG, as Aruajo & Souto presents, flawed implementations and lack of quality in the core mechanics. While PCG can offer a great quantity content wise, it could ultimately turn into uncreative and uninteresting content. The challenge when creating an interesting world is to balance between the need for a long game and to fill it with interesting stories and elements, thus arguing for a content quality over a content quantity approach [27].
Games where the world is procedurally generated, encourage exploration, while games, where levels are procedurally generated, encourages replayability [15]. There is different type of replayability as Smith presents [28], reacting in a surprising environment (the game plays different content at each attempt), building generator strategies (experiencing different content and having the opportunity to build strategies around the content generator) and practicing in different environment (the game lets the player experience new challenges but of the same kind) [28].
There are a variety of methods for procedurally creating dungeons. One category being constructive methods, producing only one output per run, which differs from the search-based methods which use evolutionary algorithms to search for good content according to Darwinian evolution principles. One constructive approach is using grammars [4], for example, Dorman’s [20] conducted research on using two-layered grammars to generate both gameplay and game space [4]. Dividing missions and spaces as separate structures and generating the content in two individual steps [20]. Missions are generated using graph grammars, creating a non-linear structure suited for exploration while extended shape grammar generates the corresponding space required [20]. Van der Linden et at. have proposed using gameplay grammar-based levels to generate dungeon levels, being able to significantly improve
the design of procedurally generated levels [21].
The relation between plot and space have been further presented by Kybartas & Bidarra, with focus on the degree of automation for story elements. This resulted in five categories: automated space, constrained space, space simulation, space modification, manual space that builds a gradient between automatic and manual generation [31]. Kybartas & Bidarra argue that even with breaking down narrative in sub-components, the goal of an automatic narrative creation tool there would still be a large presence of a human author, as so, benefitting mixed initiative methods but opening up new research questions for fully automatic methods [29].
For quest generation Parberry & Doran has categorized quests based on NPC’s motivations with the goal of autonomous generation [9]. Other research within autonomous generation is Ashmore & Nitsche [30], investigating a player centric quest generation, where the progression through level generation is achieved with “key and lock” structure. The player engages in the quests to find the “key” to overcome the obstacle, thus blocking the players progress in a flexible way. Ashmore & Nitsche argues this results in a bridge between the generated space and the quests [30].
More quest generation research has been conducted by Breault et al. following Parberry & Dorans action classification. The findings were that their developed engine was capable of creating quests similar to human written ones, and because the engine generates quests based on the world state at the time of generation, the creation of possible quests increases as the game progresses [31].
2.4 Mixed-initiative co-creation
In 1960, J.C.R Licklider expressed a suggestion that a symbiotic partnership between men and electronic computers can
provide intellectual operations much more efficiently than man alone can [22]. The term mixed-initiative was first used by Jaime Carbonell [4] in 1970, describing a new type of computer-assisted instruction (CAI) system [23]. Carbonell presented two important aspects, context and relevance. The computer-generated answers in his quiz styled system were to be contextually relevant and with the relevant information only [4].
Although no consensus has been reached about the term “mixed-initiative”, Novick & Sutton presented the idea of three aspects regarding the term. Choice of task, Choice of speaker and choice of outcome [5]. Choice of task being determining what the conversation is about, choice of speaker, the turn-taking dialogue and choice of outcome, the allocating of decisions to achieve the task [5]. The majority of mixed-initiative PCG systems focuses on choice of speaker, with the computer providing support during the design process [4].
Yannakakis et at. defines mixed-initiative co-creation (MI-CC) as “both the human and the computer proactively making contributions to the problem solution, although the two initiatives do not need to contribute to the same degree” [6]. Yannakakis et at. have presented links between MI-CC and the theories of computational and human creativity through evaluating the game development tool Sentient Sketchbook [24]. The results suggested that tools such as Sentient Sketchbook not only enable human creativity but foster it. Therefore making the approach useful for strategy game level design and making autonomous computational systems explore the possibilities guided by human decisions during the process [6]. Other mixed-initiative game developing tools are Tangara [7], Ropossum [25] and EDD. This research will extend upon the current state of EDD, adding a narrative and storyline aspect, thus increasing the tools functionality.
3. Methodology
In a similar way to how previous authors of EDD have followed the principles of design science, this paper will perform the process of constructing and evaluating a feature for co-creating quests to the already existing system EDD. To answer our research questions we have used design science as a research methodology.
3.1 Design Science
Design Science is a research methodology that acts with respect to technology. Focusing on creating and evaluating IT-artefacts that solves identified organizational problems, thus making the methodology an inherently problem-solving process. These artefacts may vary from software, formal logic to rigorous mathematics, but often results in innovations that define ideas and practices where analysis, design, implementation, and use of information systems can be efficiently and effectively accomplished. The four types of IT- artefacts are constructs, model, methods and instantiation. Our research will be an instatisation, which represents a working system that can demonstrate how the previously mentioned artefacts or ideas can be implemented in a computer based system [33], which in this research will be the mixed-initiative system.
Therefore, we will be following the 7 guidelines of Design Science as described by Hevner et al. and describing how it is applied. [26].
Guideline 1: Design as an Artifact
“Design-science research must produce a viable artefact in the form of a construct, a model, a method, or an instantiation” . The result of a design and science research must address an important organizational problem and be described efficiently with its implementation and application.
Our artefact will be a mixed-initiative system to create quests within EDD. The artefact displays the user’s created dungeon in a world view with the available quest options and generated quests. The artefact will be used for evaluation of the overall experience with EDD and thus evaluating mixed-initiative systems for game development. The artefact will be further explained in section 4, with an in depth description of the tool and the process behind its creation, describing its implementation and application within the mixed-initiative, quest generation and PCG research fields.
Guideline 2: Problem Relevance
“The objective of design-science research is to develop technology-based solutions to important and relevant business problems”. With acquiring knowledge and understanding of unsolved and important business problems, the research plays a major role in enabling effective business processes to achieve goals of business organisations, such as increasing revenue and decreasing cost [26].
PCG has shown to be an efficient solution to game development with the opportunity to develop games faster and cheaper and possibility to increase replayability [4], thus opening up solutions for smaller companies with limited resources [27]. Not only producing content, PCG can be used together with humans as a turn taking initiative, and not only assist, but collaborate and foster human creativity [6], through providing solutions that may be unexpected but valid [4].
Our artefact will address the opportunities the mixed-initiative offers game developers, using the resource efficiency of PCG together with quest generation and fostering human creativity to increase revenue and decrease costs while still keeping the games interesting and creative, moving away from automatic quest generation research to one with co creativity in focus.
Guideline 3: Design Evaluation
“The utility, quality, and efficacy of a design artefact must be rigorously demonstrated via well-executed evaluation methods”. The utility, quality and efficacy of the artefact will be evaluated though mix-method evaluation. The evaluation will follow the design evaluation methods proposed by Hevner [26]. The experiment will evaluate the artefact through a dynamic analysis, thus studying the artefact in use for its dynamic qualities[26], which will be the quality of the quest generator. The user study will be evaluated through experimental evaluation, [26]. The motivation behind using two evaluations are the possibility to use the data as compliments to each other, evaluation of both the artefact’s technical functionality and its efficacy and utility as a tool for game designers. The user study will be further explained in section 5.
Guideline 4: Research Contributions “Effective design-science research must provide clear and verifiable contributions in the areas of the design artefact, design foundations, and/or design methodologies”. The artefact will enable game designers to use the resource efficiency of PCG while generating quests, but with the option on how much of the generated solutions they would want to use, thus having the opportunity to create only manual quests, fully generated or a mixed of both. The mixed-initiative tool will allow game designers to create their own dungeons and add quests, progressively creating a sequence of quest actions and thus adding a narrative to the dungeon. The users would thus have control of the gameplay, having the possibility to decide on factors such as the progressions linearity or focus on exploration by the order and placement of the quests in the dungeon.
Guideline 5: Research Rigor
“Design-science research relies upon the application of rigorous methods in both the construction and evaluation of the design artefact”.
The research will address how the need of the artefact is experienced by game developers, and focusing on how well the artefact works. The artefact will have a theoretical base through being based on previous quest generation research and through following research methodologies and user study methods. The artefact will be iteratively tested and emphasis will be made on that system will follow logical quest structures found within RPG and action adventure games.
Guideline 6: Design as a Search Process
“The search for an effective artefact requires utilizing available means to reach desired ends while satisfying laws in the problem environment”.
The development of the artefact will be through an iterative process. Once the main functions are implemented, which is the possibility to make a sequence of quests, both manually and using some or all of the generated options, the process will continue iteratively to tweak and test the artefact, making sure the generated quests works as planned and delivers a satisfying result. The iterative workflow will allow a progressing process throughout the development with problem solving and exploring alternatives solutions.
Guideline 7: Communication of Research
“Design-science research must be presented effectively both to technology-oriented as well as management-oriented audiences”. This paper will describe the artefact regarding how it was developed, its functionality, structure and how it should be implemented, thus creating a knowledge
base for further improvement and development regarding the research, but also opening up for repeatability regarding this particular research project.
3.2 Evaluation
The evaluation of the artefact will be conducted through an experiment and a user study as described in section 3.1. This decision is based on Shaker et al. argues that a hybrid approach with both a bottom up (evaluations through players) and a top down (expressively measures) method together will provide a more holistic approach to game content evaluators[32], thus resulting in a understanding of both what the generator do through decided metrics and if the system is suitable for the designers .
3.2.1 Experiment
To evaluate content generators Yannakakis & Togelius argues that a content generator can be evaluated in three ways, directly by the designer or indirectly by either human players or AI agents [15]. In addition to be evaluated by human players through the user study, it will be evaluated through the experiment directly by us. While the method of evaluation generally is ad hoc in PCG research [16], Shaker et. al argues that “regardless of the method followed, generators are evaluated on their ability to achieve the desired goals of the designer” [4], thus the evaluation method will be conducted and designed to achieve the goals we have on the generator. The experiment process will be further described in section 5.2.2.
3.2.2 User study
The evaluation of the artefacts usability and functionality will be evaluated through a user study, which will be further described in section 5. When evaluating PCG systems, Shaker et. al argues that when making a PCG system, “we are also creating a large amount of content for players to experience, thus it is important to be able to evaluate how successful the
generator is according to players who interact with the content”[30]. In our case, EDD is a development tool for designers and not for actual players, but we argue that the actor interacting with the system, in our case, the designer, still is the interactive actor and thus we have decided to do an evaluation with actual potential users of the artefact. An additional advantage of conducting a user study is that it evaluates the aspects that cannot be objectively measured, such as aesthetics and playing experience [15]. In addition, previous studies on mixed-initiative systems have been conducted through a user study, such as the first version of EDD[35], its second follow-up study [36] and Sentient Sketchbook [24].
3.3 Methodology discussion
Due to the nature of the research subject being an IT artefact that could be used to provide a solution to a business problem, we felt design science was the optimal research methodology. Our quest implementation will thus become an invention that defines ideas and practices, as described in section 3.1. In our case the ideas and practices are a quest tool and a mixed-initiative approach. The main advantages we discussed when deciding on design science was its dominant usage in previous research, where this research both is based upon and inspired by. Additional advantages is the “learning by making” approach through its problem solving nature [33], resulting in an iterative process. We argue that this iterative process would be beneficial for providing and generating potential solutions and important insights regarding the research during the development process. However one of the main disadvantages of following design science are the lack of generalization [33] regarding the research, in which alternative methodologies have advantages. For our research suited alternative research methodologies would be surveys or a case study.
The advantages of survey would be the possibility to gain much generated data
from designers, and through having a systematic and standardized method would result in the possibility to generalise from the sample size to larger population [33]. However we believe in order to answer our research question, a depth rather than a breadth is preferred because we want to get to know what the designers think of the tool and its actual potential usability. Another advantage of survey would be the possibility to investigate the actual needs and requirements of a mixed-initiative quest system for game developers. This survey would then be possible to function as a base for a design science research regarding the system, in contrast to our study’s current foundation on literature studies. This would additionally develop an artefact that is based on actual needs and requirements rather than our interpretation of previous research.
Another alternative methodology would be case study. In case of a case study, a study could be conducted and testing mixed-initiative as a theory, introducing the system for a group of developers and investigating over time how the tool affects their way of developing, following them during a period of time. The drawbacks would be that this approach would be very time consuming [33], and would not be possible to conduct during our available time frame. However, a case study could be possible to conduct as a follow up to this research, testing it’s intentional use in practice as a real life evaluation [33].
The validity of the user study and experiments will be further discussed in section 5.1.4 and 5.2.2, describing the limitations and threats to validity.
4. Artefact
This chapter will present the quest tool as an artefact by explaining the different components implemented, such as the design choices, the user interface, functional structure and explaining the workflow in EDD with the quest tool.
4.1 Design 4.1.1 EDD
EDD is a mixed-initiative development tool that lets the user design 2D grid sized rooms by placing objects as enemies, treasures, walls and connecting the rooms to make an overall dungeon structure. EDD presents the user with automatically generated suggestions for the rooms based on game design patterns (GDP). GDP is structured through a hierarchy including meso, micro and macro patterns [36]. Meso patterns related to a larger design structure, micro consists of thin level “slices” and macro is the occurrence and sequence of micro patterns[36]. EDD uses micro- and meso patterns to generate the room suggestions. EDD’s micro patterns are enemy, treasure, chamber, corridor, connector, entrance and door, and meso patterns are ambush, guard chamber, treasure chamber and guarded treasure[35]. The PCG suggestions are using an Interactive Constrained MAP-Elites generic algorithm [8]. The algorithm maintains a map of good suggestions, rather than one single best solution. The algorithm continuously reflects the manual changes in the edited rooms, thus constantly displaying new suggestions[8]. The user can create rooms with size 3x3 up to 20x20 and connect them in a graph like an appearance. The smallest allowed dungeon is a dungeon consisting of two connected rooms[8].
4.1.2 Introduction
The quest implementation is based on EDD and thus extends to the current state of EDD. The quest implementation lets the user create one sequence of actions to create a quest. These actions are based on research conducted by Parberry & Doran [9] as described in upcoming sections.
4.1.3 New Tiles
EDD has been extended with more tiles, these tiles are:
● NPC: Our quest actions are based on research regarding NPCs motivation [9], and thus they are added in the tool.
● Item: Items are the subject of many quests, and thus an graphic representation was needed.
These tiles together with the pre-existed enemy and boss tiles have been intertwined with the actions, resulting in the “unlocking” mechanism of different quests. (see table 1).
While enemies are part of EDDs evolutionary algorithm, NPC & Item tiles are not part of the algorithm and thus will need to be manually placed out and nor will they be affected by the algorithm.
4.1.4 Actions
19 different actions have been added to the quest implementation. These are based on Parberry and Dorans research [8]. However in some cases, we have made the decision regarding if the actor represented in the action is friendly (NPC) or hostile (enemy), this will be evaluated in the user feedback. The actions have different prerequisites (table 2). With the actions “unlocking” through the user placing the necessary tiles. The user has the option to select what type of action and then what position on the map the action will take place. This is done by selecting a specific floor tile, marking that position on the map. However, Exchange, Give, and Take requires two positions due to its two subject nature (an actor + an item).
Besides manual placement, the user has the option to instead pick a suggested action from the generated actions from the right panel. After deciding these options, the user will need to press the “+” button on
the bottom panel, this will add the action to the quest sequence.
4.1.5 Generator
The generator generates actions that make up the generated quests based on grammar. The generator's grammar-structure is displayed in table 2 and based on Doran and Parberry´s research [8]. The generator is based on categories discovered in previously mentioned research. The grammatical rules can be found in table 2. The grammatical rules include terminal symbols, which means the rules at some point will terminate, thus making the quest generation suggestions to end. If the generator no longer can generate quests based on the active quest sequence, the user receives feedback that the quest is not compatible to further generate quests, and gives the user suggestions on how to continue and to overcome this limitation. 4.2 Workflow
The quest implementation is accessible from the world view (fig. 1). In the world view the user can decide on whether to either start by using the room view or directly to the quest implementation (fig. 1). The quest implementation and room view is structured to work iterative. This allows the user to firstly create a room, view the available quests actions for that room, and iterate back and “unlock” more quests or change the layout structure of that room. In addition, an overall iteration between world view, room design and the quest implementation is possible (fig. 1). Any changes made on the dungeon or in a room will be reflected in the three “designers” (world view, room design and quest implementation) through their utility of the same world map.
Figure 1 displaying the workflow between EDD and the quest tool.
4.3 User Interface 4.3.1 EDD navigation
The flow between the collaboration between EDD and the quest tool is demonstrated in fig. 1. EDD's starting view starts with an option to pick a category and then “create your world” The quest implementation is reachable from EDDs world-view as a button, called “add quest”. While in the world view the user can select how many rooms they want their dungeons to consist of, the size of the rooms, the position of the doors and the overall structure of the dungeon. When clicking on a room the user comes to the room editor where they can place tiles and create their rooms (fig. 1). The available tile options are visible in the left corner, and showing the new additions, NPC and Item (fig. 1). These tiles are stylistically based on EDD already existing tiles and pixel art in RPG and action-adventure games. The manual placement of the tiles are done by clicking the tile’s image and then the position in the room. The tool will display red if the placement is not allowed. The room the user creates is saved and visible in the quest tool through using the map the world-view shows (fig. 1). The user does not have to place any tiles or design a room before using the quest implementation, however placing both enemies, items, NPCs affects what actions that are available as described in section 4.1.3.
4.3.2 Quest-tool implementation
The user interface for quest-tool implementation follows the room-view’s design, and consists of four panels; the left-hand toolbar, the right-hand generated suggestions, the central map-view, and the bottom quest-panel (fig. 7).
4.3.2.1 Action tiles
Since EDDs world map is reused in the quest tools, all of EDDs pre-existing visual elements remain the same in the quest view. An action takes up 1x1 square which is the general size of tile elements in EDD, however a boss requires 4x4 tiles, though this is still treated as 1x1 entity, and thus an 1x1 tile of the 4x4 is required to be selected. Once an action is placed on the room manually, the available tiles where the action can be placed will be displayed in green (fig. 2). When the user selects a suggestion from the generator, the available tiles will instead be displayed in purple (fig. 3).
Figure 2 displaying an user manually selecting the “listen” action, therefore the NPC will be highlighted in green to display its availability.
Figure 3 displaying a user selecting a generated suggestion, “give”, the generator automatically selects the required tiles and highlights them in purple.
4.3.2.2 Action panel
The action panel is displayed in fig. 7.1. The panel consists of buttons for each respective action (19 actions). The tile placed in the room reflects the available actions, which is previously discussed in section 4.1.2. This panel represents the manual placements and manual aspect of the mixed initiative approach of the tool. If an action is not “unlocked” based on the prerequisites, it is disabled and the button is colored a dark gray. If an action fulfills its prerequisites it is enabled and turns light gray and white borders and text (fig. 4).
Figure 4 displaying two rooms with no added tiles. Since ”goto” and “explore” only requires a floor tile, it is “unlocked” and therefore enabled. The rest of the buttons are “locked” and therefore displayed as the same color as the background.
4.3.2.3 World panel
The world panel is displayed in fig. 7.2. The panel displays the reused EDD world view. The world view displays the user’s dungeon and the tiles placed. The selected actions will be visible on the map (fig. 2). 4.3.2.4 Suggestion panel
The suggestion panel is displayed in fig. 7.3. The panel displays the generated actions from the generator. Once the generator generates a suggestion, it will be displayed in a list. The list is clickable, and once an action is selected it is highlighted in a lighter gray (fig. 5). To add the generated suggestion to the quest sequence, the user needs to click on the “+” symbol, therefore adding it on either last place or the desired place in the sequence. While the generator is “working”, a loading symbol is displayed, indicating to the user to wait.
4.3.2.5 Sequence panel
The sequence panel is displayed in fig. 7.4. The panel displays the actions the user has selected. In order to add a sequence to the list, the user either must manually select an action and its desired position, or select a suggested action, however both these options require the user to manually press the “+” button.
Each button in the sequence is clickable and interchangeable. If an action in the sequence is selected, the user can select the action desired from the action panel to be exchanged. When an action button is selected, removal is done by pressing delete on the user’s keyboard.
4.3.2.6 Toggle Menu
The toggle menu is displayed in fig. 7.5. The menu consists of two buttons, toggle help and toggle path.
Enabling toggle help results in several help dialogs being displayed for the user. The dialog informs the user that a placement must be picked in a room, but in addition
informs the user that a certain action was added to the sequence. The alerts fade and disappear after 4.5 seconds.
Toggle path displays the shortest path from the user’s characters through all placed positions to the recent placed. This is done through a modified Dijkstra's algorithm. The path is displayed in light blue (fig. 6).
Figure 5 displaying a dialog message informing the user to pick a position in the room.
Figure 6 displaying the shortest path from the character (left corner) to a “goto” action (recent action) placed in the next room. 4.3.2.7 Erase & Back
The button “erase” and “back” is displayed in fig. 7.6 and 7.7. “Erase” erases the quest sequence while “back” directs the user to the world view (fig 1).
Figure 7 displaying the UI of the quest artefact with the panels, 7.1 Actions, 7.2 World View,
7.3 Suggestions, 7.6 Erase Button and 7.7 Back Button.
Table 1
Action Prerequisites (from Doran & Parberry’s research) [8]
The prerequisites implementation in EDD (the “unlocking”) Capture “Somebody is there” A NPC or boss/enemy must be placed.
Damage “Somebody or something is there” An item or NPC must be placed. Defend “Somebody or something is there” An item or NPC must be placed. Escort “Somebody is there” A NPC must be placed.
Exchange “Somebody is there, they and you have something” A NPC and an item must be placed (requires two positions). Experiment “Something is there” An item must be placed.
Explore “none” An available floor tile.
Gather “Something is there.” An item must be placed.
Give “Somebody is there, you have something.“ A NPC and an item must be placed (requires two positions). Goto “You know where to go and how to get there.“ An available floor tile.
Kill “Somebody is there.“ A boss/enemy must be placed. Listen “Somebody is there.“ A NPC must be placed. Read “Somebody is there.“ A NPC must be placed. Repair “Somebody is there.“ A NPC must be placed. Report “Somebody is there.“ A NPC must be placed.
Spy “Somebody or something is there.“ A NPC or boss/enemy must be placed. Stealth “Somebody is there.“ A NPC or boss/enemy must be placed.
Take “Somebody is there, they have something.“ A NPC and an item must be placed (requires two positions). Use “There is something there.“ An item must be placed.
Table 1 displaying the actions together with Dorans & Parberry´s[8] prerequisites and how the actions and the previously mentioned prerequisites have been implemented in EDD. This indirectly explains the “unlocking” - describing what tiles that must be placed for an action to be available. Note that “Goto” & “Explore” do not have any special tile prerequisites besides available floor.
Table 2
Production rules Actions knowledge* ["<get>","<go_to>","give"] ["<spy>"], ["<go_to>","listen","<go_to>","report"] ["<get>","<go_to>","use","<go_to>","give"] comfort* ["<get>","<go_to>","give"], ["<go_to>","damage","<go_to>","report"] reputation* ["<get>","<go_to>","give"], ["<go_to>","<kill>","<go_to>","report"], ["<go_to>","<go_to>","report"] serenity* ["<go_to>","damage"], ["<get>","<go_to>","use","<go_to>","give"], ["<get>","<go_to>","use","capture","<go_to>","give"], ["<go_to>","listen","<go_to>","report"], ["<go_to>","take","<go_to>","give"], ["<get>","<go_to>","give"], ["<go_to>","damage","escort","<go_to>","report"] protection* ["<go_to>","damage","<go_to>","report"], ["<get>","<go_to>","use"], ["<go_to>","repair"], ["<get>","<go_to>","use"], ["<go_to>","damage"], ["<go_to>","repair"], ["<go_to>","defend"] conquest* ["<go_to>","damage"], ["<go_to>","<steal>","<go_to>","give"] wealth* ["<go_to>","<get>"], ["<go_to>","<steal>"], ["repair"] ability* ["repair","use"], ["<get>","use"], ["use"], ["damage"], ["<get>","experiment"] equipment* ["repair"], ["<get>","<go_to>","give"], ["<steal>"], ["<go_to>","exchange"] subquest* ["<go_to>"], ["<go_to>","<QUEST>","go_to"] go_to ["explore"], ["<learn>","go_to"] learn ["<go_to>","<subquest>","listen"], ["<go_to>","<get>","read"], ["<get>","<subquest>","give","listen"] get ["<steal>"], ["<go_to>","gather"], ["<go_to>","<get>","<go_to>","<subquest>","exchange"] steal ["<go_to>","stealth","take"], ["<go_to>","<kill>","take"] spy ["<go_to>","spy","<go_to>","report"] capture ["<get>","<go_to>","capture"] kill ["<go_to>","kill"]
Table 2 displaying the grammatical rules. The columns marked with asterisks are identified as “motivations” by Doran & Parberry [8], but are used as a starting point for the quests. The “< >” indicates the next production rule to be taken, and actions without “< >” is the terminating action.
5. Results
5.1 User study
In order to assess the relevance of the quest implementation and the overall heuristic experience of EDD, an user study has been conducted. The goal of the user study is related to design science guidelines regarding design evaluation. The implementation’s utility, efficiency and quality will be covered in a set of questions and tasks, thus investigating whether the tool will be a useful addition for game designers when developing dungeon games. The study’s validity and reliability will be further discussed in section 5.3.5.
5.1.1 Design
The user study consisted of 7 user tests where the user followed a set of tasks and answered questions regarding the tools functionality, experience with the tool and overall experience with the tool implementation to EDD. The test structures are available as appendix A. The themes of the user study was to investigate the tool’s functionality and the possibility to co-create together with the generated material, thus relating to if the system would be a potential efficient and beneficial solution for game developers. The test consisted of a pre interview questionnaire and an interview. Both the questionnaire and interview followed guidelines described by Oates [33].
5.1.2 Users
We invited 6 persons that either work in the game development industry or are currently studying or finished studying game development. We have not limited the users to only game developers, but invited people that work within the industry such as level designers and game designers, which is roles included in game development teams. The users were mainly recruited through convenience sampling and through snowballing and thus a nonprobability sampling. No testers had
any previous knowledge or experience with EDD.
5.1.3 Process
The interviews were conducted through a structured interview in written form, where the users were sent a document with information on EDD and the interview questions in a web form to answer. 3 questions directly reflected on the 3 tasks the testers were instructed to follow.
5.1.3.1 Introduction
The users were given a document which describes the purpose of the user study, the aim of this thesis, how to download the program, a brief introduction on EDD and a short overview of the interview’s structure and task.
5.1.3.2 Instruction
The users were asked to complete 3 tasks that covered the tool’s functionality. These were:
● Manually creating a quest sequence
In order to evaluate how the manual creation of quests are perceived and for the possibility to compare to automatic and mixed initiative.
● Automatically generating a quest sequence
In order to evaluate how the user experiences the automatic generation related to its usability and functionality. This will be compared to manually and mixed.
● Creating a mixed sequence
The core research subject of this paper relates to how the user experiences the mixed-initiative tool, thus this will test our research questions.
5.1.3.3 Questions
The questions consisted of 17 questions of which two were closed ended questions and
