Generation, evaluation, and optimisation of procedural 2D tile-based maps in turn-based tactical video games

(1)

Author: Ioseff Griffith

Supervisor: Johan Hagelbäck Semester: VT 2016

Subject: Computer Science

Bachelor Thesis Project

Generation, evaluation, and

optimisation of procedural 2D

tile-based maps in turn-based

tactical video games

(2)

Abstract

In modern times, procedural content generation sees frequent use in video games, producing anything from graphics to maps and quests. This study focuses on how these techniques can be used to produce forest maps for tactical role-playing games, and how this is applied in particular to Starshard, one such game. The main points addressed are how well the forestry growth simulation algorithm previously implemented in Starshard compares to algorithms based on other popular techniques, how to utilise evaluation and optimisation to produce correct and strategically interesting maps without human interference, and whether or not players can easily perceive a difference between procedural maps and maps designed by humans. With high demand for new algorithms to automate content generation in order to help smaller teams of developers and maintain interest in content post release, there is much value in delving further into less frequently explored aspects such as tile-based maps or player perception of procedural content. In order to answer the research problems, a controlled experiment comparing four forestry generation algorithms was performed, in addition to a study of literature to implement evaluation and optimisation algorithms, and a survey to gauge player reactions to the produced maps. Ultimately, the growth simulation proves to be more successful than alternative algorithms in reducing number of errors in maps. In addition, the results of the survey showed that while players are capable of reliably picking out human created maps, their ratings for procedurally generated maps were not much lower than these and they could frequently mistake procedural maps as being produced by humans.

Keywords: procedural content generation, tactical role-playing game, evaluation, optimisation, forestry generation

(3)

1 Introduction

In recent years, procedural content generation (PCG) has seen use in many video games, from the high budget AAA to the increasingly popular independently developed titles.

Role-playing and tactical role-playing games are two genres which have also begun to incorporate many procedural aspects, such as world building and event or quest generation.

This study will focus primarily on the procedural generation of forests within tile- based tactical role-playing games, their evaluation, and optimisation for play.

1.1 Background

Procedural content generation (PCG) is often used to refer to “the algorithmical creation of game content with limited or indirect user input” [1]. Some of the desirable properties of procedurally generated content are: [2]

• Novelty: content should contain elements of randomness and unpredictability.

• Structure: content should contain larger structures and not just random noise.

• Interest: this combination of randomness and structure should engage the player.

• Speed: generation should be quick.

• Controllability: it should be possible to generate content based on a set of parameters.

In this case, when “game content” is spoken of, it is used to refer to any of the content that is included in a video game. This can range from levels and maps to stories, characters, and music [3].

PCG has been present in video games since as early as the 1980s [4]. Until recently, however, the instances of its use have been few and far between. With demand for new releases and additional content for existing video games rising, new methods are needed to produce this content while keeping development costs low [5] and production efficient [3]. For this reason, its popularity has surged in recent years and the concept has become prevalent throughout most, if not all, genres of video game, and even finds some use within the film industry [6]. With this increase in usage, many aspects of modern video games are often assisted in some way by PCG, with many games built almost entirely atop such algorithms. Popular modern games which extensively implement PCG include Minecraft [7] and Dwarf Fortress [8], amongst a vast many others.

Role-playing (RPG) and tactical role-playing games (TRPG), form two such genres in which procedural content generation has begun to thrive. Since as early as 1974, tabletop role-playing games have been prevalent within popular culture [9]. Soon after this, the first digital role-playing games began to surface. RPGs are often considered to be very hard to define as there is an extreme variety within the genre itself, however most contain similar elements such as character progression and narrative elements, among others [10]. Tactical role-playing games differ from other similar genres such as real-time strategy games in that they include elements of traditional role-playing games as previously described, but using a turn-based strategic battle system. Much like RPGs, there is a large degree of diversity within TRPGs, encompassing games such as Final Fantasy Tactics [11] and Valkyria Chronicles [12].

Within these tactical role-playing games, players field groups of characters (often referred to as “units”) to face off against other such groups of units. Battlefields can vary

(6)

from basic maps such as those employed in real-time strategy games, to isometric tile- based grids. These are generally referred to as “maps” or “fields” and the majority of gameplay takes place within these battlefields.

1.2 Starshard

In Starshard, players pit their individually customised armies against each other or the AI as seen in Figure 1.1. These battles can take place both on maps that are generated procedurally at runtime and pre-built maps designed by human hands. Battles are generally small-scale and include five to eight units on maps that vary in scale from 30x30 to 60x60.

There is currently no defined upper limit on map size and armies can grow with player progression.

Figure 1.1: A small-scale battle in Starshard

1.2.1 Movement

For the purpose of optimising maps for play, it is important to consider the movement capabilities of individual units. Maps in Starshard can feature terrain with greatly varying height between neighbouring tiles in order to provide the player with strategic opportunities and to represent natural formations such as hills or mountains. Certain units, however, do not have the capability to climb cliffs over a set height. This can be seen in Figure 1.2, where the unit can only move in three of the four cardinal directions due to the higher cliff to the north. As such, plateaus over a certain height have the potential to become inaccessible to such units and could render entire sections of the map unreachable to players only fielding these units. In order to ensure universal fairness in maps, all segments separated by height must be connected by at least one path where tiles differ in height by only one unit of measurement.

1.2.2 Obstacles

There are many types of obstacles defined in Starshard; these can range from trees, rocks, and buildings, to water and lava. Not all obstacles are immutable, for example trees and

(7)

Figure 1.2: An image demonstrating the movement restrictions the Evoker unit.

rocks which can be destroyed by certain types of attacks and yield resources for the player.

For this reason, when evaluating terrain for optimisation, only indestructible obstacles are considered to require optimisation. A selection of destructible objects can be seen in Figure 1.3.

Figure 1.3: A selection of destructible objects.

1.2.3 Unit Placement

At the beginning of each match, a player must deploy their units on any of the tiles in predefined regions that are “owned” by them, as seen in Figure 1.4. Blue overlays represent positions in which units can be placed, whereas red overlays are tiles on which they cannot. Each unit has a “cost” and the player is given a maximum allotment of points to spend on these units. During this stage of the battle, units can be removed and replaced at will.

(8)

Figure 1.4: Unit placement positions.

1.2.4 Tactical View

In order to allow the player an easier overview of the map and the current situation in order to make strategic and tactical decisions, a “Tactical View” mode is provided. While this mode is active, player units, enemy units, and obstacles are displayed in visually distinguishable colours, tiles and obstacles are rendered semi-transparent, and height differences are represented by shifts in colour, as seen in Figure 1.5. Future iterations are planned in which further information about maps is added to this view, including factors such as temperature and other risks.

1.3 Forestry generation algorithms

While Starshard currently implements a forestry generation algorithm based on the simulation of forestry growth, three other algorithms are also under consideration to fulfil this role. This section presents an overview of the concepts behind these four algorithms.

1.3.1 Growth simulation

The original algorithm for generating a pseudo-realistic placement of trees for procedural maps was partially based upon two conceptual algorithms described by Kerr [13], and in passing, by Dapper [14]. This algorithm involves the use of a simulation which steps through a number of years in an attempt to emulate the growth of a natural forest over a set period of time.

A number of parameters are defined apart from the layout of the terrain and existing obstacles in order to run the simulation. An initial seed count is specified which defines the number of trees that will be spread around the map at the start of the simulation. In

(9)

Figure 1.5: Tactical View.

addition to this, the number of cycles the simulation will run for and a list of the types of vegetation that can be generated by the simulation are provided.

Each type of vegetation is defined by a set number of parameters which influence the simulation. These are: maximum age, which is the number of cycles after which the tree will be removed from the simulation and considered “dead”; mature age, which defines the number of cycles after placement must pass before the tree can begin scattering seeds;

seed radius, which is the maximum distance seeds can be scattered from the tree; kill radius, which is the radius within which any younger tree will be “killed” and removed from the simulation due to lack of light; age, which is the number of cycles the tree has existed for; an identifier used to specify what type of tree should be generated in the game itself; the fertility of the tree, or how many seeds it can scatter in a single cycle; and the survival chance of the tree which is a randomly generated float value used to determine the likelihood of the tree’s survival with each passing cycle.

1.3.2 Random number generation

One potential tactic for producing a map with randomly placed trees would be to use pure random number generation while still taking into account which tiles can be planted on (e.g. grass). This algorithm was chosen due to the fact that it is quite possibly the simplest algorithm to implement. In order for this algorithm to fit with the definition provided for procedural generation, a few specific adjustments need to be made. The first is to ensure that a seeded random number generator is used so that the patterns might be repeated if the same parameters are given [15]. The second is that it must be restricted by some factor (in this case tiles which can and cannot be planted on) so that it is not purely random.

A concern in the case of this algorithm would be occurrences of the so-called “birthday problem” [16], which states that in a set of randomly chosen individuals, there is a high likelihood for some pair to share the same birthday. This same logic can also have an

(10)

influence on the positioning of randomly generated coordinates as picking numbers from within a certain range means that it is quite likely for some of the numbers to end up in close proximity to each other [17]. For this reason, clustering of trees is quite likely, though this can be seen as either a positive (if the trees cluster in a pleasing manner), or a negative (if the trees overlap and cause artefacts).

1.3.3 Perlin noise

Perlin noise was developed in 1983 by Ken Perlin [18]. It is a type of gradient noise which is most commonly implemented as a two, three, or four dimensional function. An n-dimensional grid is defined with each node assigned a random gradient unit vector with n dimensions. Next, the vectors from the given point to the surrounding points on the grid are calculated. A dot product is taken between the gradient and distance vectors to find the final influence values. Interpolation is then performed between the values in order to find a weighted average before a fade function is applied to smooth the transition between gradients.

The use of Perlin noise is quite prevalent throughout game development [19] and is often used for procedural content generation as it can increase realism in a pseudo-random fashion. This makes it very suitable for determining patterns such as the placement of trees or other landscape features.

1.3.4 Cellular Automata

A cellular automaton features a collection of gird cells with a value in each cell. The automaton evolves through a number of steps based on the states of neighbouring cells and a set of rules [20]. The number of different states for each cell is generally specified beforehand and each state is often represented by an integer. The simplest cellular automata feature only two different states, often referred to as “black” and “white” [21].

There are many ways in which the rules can bind a cellular automaton. At its most basic level, only the nearest neighbours - those directly adjacent to a given cell - affect a cell.

1.4 Previous research

Much literature has been produced describing a variety of algorithms for the implementation of procedural aspects to video games in many different genres. Procedural role- playing and tactical role-playing games are sometimes discussed, with particular focus on map generation and quest generation. In his paper on PCG for strategy/role-playing games, Buck [22] explores the world generation system for Project Death and Taxes. He makes use of fractal-based generation and Markov chain methods to present a method for generating maps and ultimately concludes that there are many limitations in the maps produced by the implemented algorithms.

Grey and Bryson [23] explore procedural generation for non-player character interac- tions using Behaviour Oriented Design to allow for the procedural generation of dynamic quest systems. They conclude that while the method is basic, it shows enough merit to warrant further investigation and development. In their 2011 technical report, Doran and Parberry [24] describe a prototype quest generator using a structural analysis of typical quests and their flows. From their research they conclude that quests gravitate towards a well defined structure which could be used as a basis for procedural generation.

(11)

Research in other related genres has been even more extensive. Mark et al. [25]

discuss generating 3D caves on the GPU using a pipeline consisting of three major com- ponents, building tunnels around a set of structural points generated from an L-system.

The conclusion reached is that the presented solution is effective at producing believ- ably natural cave systems, though improvements could be made in the controllability of the generation. Cannizzo and Ramírez [26] presented approaches to generating procedural maps and dynamically adapting characters for the multiplayer on-line battle arena (MOBA) genre. The present a series of algorithms for the production of such maps and a character adaptation system which will adapt progression to the playing style of the player.

Their conclusion is that, while the proposed systems have a long way to go in terms of optimisation and design, they do provide proof that MOBAs can incorporate elements of dynamically generated content. Togelius et al. [27] discuss a search-based approach to PCG through multiobjective evolution. Experiments and a user study were conducted in order to investigate the generation and evaluation methods. They ultimately conclude that the paper demonstrates that complete and playable strategy maps can be produced through multiobjective evolution and that the methods described could be applied to offline map generation with only a little extra work.

1.5 Problem formulation

The purpose of this thesis project is to evaluate and expand upon the PCG methods already implemented in Starshard [28], a 2D tile-based tactical role-playing game built in the Unity engine. At this stage, Starshard features basic algorithms for the procedural generation and evaluation of maps, including forestry, bodies of water, and vegetation.

Particular focus should be placed upon the generation of forestry and related terrain, along with the evaluation methods used to determine whether maps are playable and defect free and the optimisation algorithms utilised to eliminate these defects. Defects in this context refer to instances wherein sections of terrain may be inaccessible to the player or objects and obstacles are spawned in positions that defy logic. The degree to which players can distinguish between maps generated through PCG algorithms and those devised by human minds is also of great interest in the problem area.

1.6 Motivation

At its current stage in development, Starshard already features many aspects of procedural generation in order to enhance its gameplay. One of these involves the procedural generation of forestry using a purpose built algorithm. However, the value of its currently implemented forestry generation algorithm is as yet unproven and thus would benefit from a comparison to other algorithms built upon common concepts within procedural generation. This comparison could also serve to provide an overview on the performance of these different algorithms for implementation in future video games. A total of three alternative algorithms were chosen for the comparison based on their frequent use within the video game industry. These were random number generation, Perlin noise, and cellular automata.

The evaluation of maps generated through PCG is also an area of great interest [29], and much work [30] has been produced in seeking algorithms to automate this proce- dure. A need for such algorithms is also present in Starshard in order to allow maps to be generated in their entirety without exterior human influence. This thesis will explore and present such automatic evaluation methods in the context of 2D tile-based maps gen-

(12)

specialised optimisation algorithms. As only the four forestry algorithms were changed between experiments, only the difference in running times for these was of interest in the controlled experiment.

In addition, it is important to judge the extent to which players of the game (or ob- servers) can distinguish between procedurally generated content and human designed content, as the use of procedural content to replace human made content implies that the generated content should be of similar quality. If a player can point out differences in quality between the two types in terms of aesthetics, playability, plausibility or potential for strategic play, then they can most likely be easily distinguished.

1.7 Research Questions

RQ1 How do maps generated by the currently implemented forestry algorithm compare to those generated by other algorithms in terms of generation time and number of defects?

RQ2 How can we ensure that maps are defect free and strategically interesting without external human interference?

RQ3 Do players easily perceive an aesthetic or other differences between maps generated through PCG and those designed by a human?

It is expected that the currently implemented forestry algorithms will be more effective at avoiding defects than most, but slightly less efficient in terms of running times. Players are expected to find maps generated procedurally to be at least moderately appealing, in most cases to a similar degree as those generated by humans.

1.8 Scope/Limitation

Since there is an endless number of ways to approach PCG, it will not be possible to implement and compare all possible methods for generating procedural forestry or evaluation and optimisation algorithms. It will also not be possible to provide a playable version of the game, and as such, evaluation of maps from a human perspective will be limited to what can be judged through screen captures.

As Starshard is built upon the Unity engine, quantitative results from the experiments will be contingent on the implementation and efficiency of the engine itself.

1.9 Target group

The primary target group of this thesis is developers interested in implementing procedurally generated forestry and accompanying evaluation and optimisation algorithms in a tile-based video game. This can also be expanded to those with interests in other aspects of procedural generation and how players perceive differences between content produced by a human and content produced through PCG.

(13)

2 Method

In this chapter, the scientific approaches to satisfy the research questions are explained.

Some discussion of the reliability and validity of the thesis is presented, in addition to any ethical considerations that need to be taken into account.

2.1 Scientific approach

In order to establish the performance of the forestry generation algorithm initially implemented in Starshard, a controlled experiment producing quantitative data was required such that the algorithms could be compared in a controlled environment. Providing an answer to the second research question - how to ensure that maps are defect free and strategically interesting without external human interference - was a problem that would require the design and implementation of a purpose-built artefact created through design science for use in Starshard. These scientific methods were augmented by a survey which also served the additional purpose of providing quantitative data in order to answer the third research question.

2.2 Forestry Generation Experiment

A total of three algorithms were designed and implemented based on common procedural generation techniques for comparison with the algorithm implemented in Starshard’s Unity-based engine. Each of these algorithms was incorporated into the game and run separately as part of the overall procedural generation subsystem. For the purpose of this experiment, the independent variable was the algorithm being used to generate the procedural forests, of which there were a total of four options. The dependent variables were the times taken for the generation algorithm to produce the tree map and the number of defects produced. This experiment was designed in order to compare the number of defects produced and ensure that none of the selected algorithms would exceed a certain threshold in terms of generation time, preventing them from being viable in offline or online map generation. In total, the experiment was run ten times with each algorithm.

In this context, defects most commonly referred to placement of trees in such a way that segments of the map are isolated from each other with no possible paths or trees placed directly adjacent to each other, resulting in clipping of their sprites. It could also refer to cases in which very few trees were generated, no trees were generated, or trees were placed in impossible positions, e.g. on rocks, or water. Defects within the generated maps was determined using a simple automated system similar to the evaluation algorithms that would seek out and register any aspect considered to be a defect in the map.

As the generation of the forest itself comprises only a fraction of the entire process involved in creating a procedural map, it should take an equivalently fractional amount of the overall generation time in order to be deemed suitable for online content generation.

Since the overall generation time of a map in Starshard is already quite low, the maximum suitable threshold is defined as being 5ms for the purpose of this experiment.

2.3 Evaluation and Optimisation Algorithms

For the purpose of implementing algorithms to evaluate and optimise procedurally generated maps created by the above forestry generation algorithms, a study of literature relating to evaluation and optimisation was conducted. Research materials were selected

(14)

based on topics relating closely to the procedural generation of maps, with tile-based solutions being preferred where possible. Results were selected primarily from databases of scientific journals and evaluated based on their relevance and feasibility for implementation within Starshard. This research was used in order to design the algorithms and a survey was used in order to provide quantitative data relating to the success of the implemented algorithms. Evaluation and optimisation in this regard refer to evaluating a map in order to discover any potential defects, and optimising it to remove these defects.

2.4 Survey

In order to establish whether or not the preceding algorithms fulfil their purpose when presented to a human audience and establish the extent to which potential players can observe differences between procedurally generated maps and maps produced by human designers, a quantitative survey was conducted. This survey made use of a 1-5 rating scale which asked participants to rate a series of screen captures based on set criteria. Each of the ten screen captures presented an image of a map produced either by one of the forestry algorithms implemented as part of the experiment, or by a human designer. The survey contained two images for each of the four PCG algorithms, and two designed by a human.

The specific criteria for evaluation were aesthetics, potential for strategic play, whether it seemed like the map was implausible or unplayable, and how likely the individual would be to want to play such a map. These criteria were selected as they were of most interest for the purpose of designing content in the context of Starshard and can be judged with a fair degree of confidence from screen captures. The full survey can be seen in Appendix A.

Selection of participants was carried out based on their interest in video games. The survey was distributed to a variety of video game communities, some with focus on the specific genres of Starshard, others without. In order to ensure that no validity threats would arise due to the inability to provide a working build of the game to participants, the survey was prefixed with a thorough description of the game and its mechanics and provided a comparison to two well known and widely played games which use very similar gameplay systems. This ensured that most, if not all, respondents would have an understanding of the key factors before rating the screen captures.

The resulting data from the survey was used to perform a quantitative statistical analysis based on the responses to each screen capture.

2.5 Reliability and Validity

This thesis is reliable because repeated execution with the same conditions would produce similar results. However, a few potential threats to this reliability exist. These threats are the timer used in the experiments, the use of the Unity engine, and the methods for determining defects within generated maps. In order to minimise any potential for error in regards to the timer, the experiment was built in such a way as to ensure that the timer would measure the execution time of the algorithms specified in the experiment only. No other statements, code blocks, or algorithms were included within the period recorded by the timer. As many versions of the Unity engine exist on several platforms, the results of the experiments in one version could potentially differ to those in another should optimisations or bugs be introduced between versions. Determining the presence of defects within maps could also have presented threats to the reliability of the study as not all types of defects can be easily identified by evaluation algorithms.

(15)

In order to ensure that this thesis is valid, several measures were taken. To prevent problems relating to construct validity, all terms are thoroughly defined in order to ensure that there can be no misinterpretation of the contents. While not all aspects of Unity can be controlled leading to potential internal validity threats, this was minimised by restrict- ing the experimentation to only one segment of the code being run. This was also done in order to try and minimise the effects of other processes running on the same machine, though the effect could not be entirely negated. Depending on the individual implementing the algorithms and the language/libraries being used, performance could vary. For the purpose of this thesis, all algorithms were implemented in C# by an experienced program- mer with strong knowledge of the language, using only the standard libraries and those provided by the Unity engine.

To ensure that the results of the survey were valid, a broad range of respondents were chosen and given a thorough explanation of the game, including comparisons to well known and widely played games. In order to address potential validity threats in the survey due to its linear nature, pairwise comparison was considered as an alternative.

This, however, was ultimately rejected as pairwise comparison of ten different screen captures would lead to far too many questions for the average respondent to answer in a single survey. Screen captures were selected with a minimal amount of bias by picking the first two of each type of map generated with the camera roughly centred on the map.

Future work could be made more reliable and valid by providing respondents with access to the game itself in order to allow them to make more informed judgements as certain criteria, such as strategic potential, might be hard to judge from screen captures alone.

2.6 Ethical considerations

The main ethical consideration within this thesis was the handling of data retrieved from the survey. In order to protect the privacy of respondents, the survey was anonymised such that no names or connections that could identify individuals would be published.

(16)

3 Implementation

In this section, the implementation of the four procedural forestry generation algorithms will be discussed and detailed, along with the design and implementation of the evaluation and optimisation algorithms. Subsections are prefaced with theoretical knowledge that is considered important to understanding both the thesis and the implementations.

3.1 The Unity Engine

The Unity Engine forms the backbone of Starshard’s implementation, on top of which all further systems are built. These systems include battle systems, dialogue systems, the procedural map generation system, and a plethora of others. The first version of Unity was released on June 6th, 2005 with the goal of creating a professional and affordable game engine for amateur game developers [31]. To date there have been five versions of Unity, with the latest supporting all major platforms [32] including Windows, Mac, iOS, PlayStation 4, Xbox One, and Wii U to name a few.

Unity not only provides functionality to simplify rendering, but also provides its own coordinate systems and physics within a three-dimensional space. While Starshard does not make use of the physics engine, it does rely heavily on the coordinate system, as shown in Figure 3.6, in order to position and render all objects that are part of the game.

Figure 3.6: A demonstration of Unity’s three-dimensional grid space

3.2 Forestry generation algorithms

Here, the implementation of the four algorithms for the procedural generation of forestry will be discussed. These four implementations were the original growth simulation algorithm previously implemented in Starshard, one generated using pure random number generation, one based on Perlin noise, and one making use of a cellular automaton.

3.2.1 Growth simulation

The algorithm begins by selecting several points at random (up to the specified initial seed count) around the map that have been designated as “plantable” (seeds can grow in

(17)

these places) and randomly placing one of the defined vegetation types at this point with their current age and survival chance also randomly chosen between set values. Each vegetation type is given an equal probability of being chosen at this stage.

With this done, the algorithm iterates for the set number of cycles, removing any trees that meet the criteria for “dying”, spreading seeds from those that are considered mature, and increasing the age of every tree. A pseudocode representation of the algorithm’s cycle can be seen in Algorithm 1.

Data: The number of cylces to run for Result: A tree map

for i ← 0 to cycles do

foreach Tree tree in the map do increase tree age;

if tree is older than max age then remove tree;

continue to next loop;

end

if tree is mature then

for j ← 0 to treef ertility do

spread seeds up to max radius and add new trees to the map;

end end

if tree is dominating another then remove another;

end end end

Algorithm 1: Pseudocode growth simulation algorithm.

This algorithm is designed specifically to avoid issues wherein trees are placed adjacent to each other, causing problem for players when moving through the forested areas. A sample map produced by the algorithm can be seen in Figure 3.7.

Figure 3.7: A sample forest generated using the tree growth simulation algorithm.

(18)

3.2.2 Random number generation

The algorithm begins by creating a list of all tiles which are not currently obstructed and are considered “plantable”. From this, a random number of points is selected, up to the size of this list. A loop is then run in which points are selected from the previous list using Unity’s random function, a random tree is selected from an array of trees, and this is placed in that position on the map. At the same time, the coordinate is removed from the coordinate list so that no two coordinates may be chosen twice due to an effect such as the birthday problem. A pseudocode representation of the algorithm can be seen in Algorithm 2.

Data: The map and a list Result: A tree map

foreach coordinate coord in map do if coord is plantable then

add coord to list;

end end

treecount = random number between 0 and list length;

for i ← 0 to treecount do pick a random tree type;

place the tree at a random point from list;

remove the point from list;

end

Algorithm 2: Pseudocode random number generation based algorithm.

A sample map produced by the algorithm can be seen in 3.8. Much clustering is displayed, with very few paths between trees for players to utilise.

Figure 3.8: A sample forest generated using the random number generation based algorithm.

(19)

3.2.3 Perlin noise

This algorithm makes use of the Perlin noise function implemented in the Unity engine.

Two layers of Perlin noise are combined using the coordinates of each tile on the grid. One layer is subtracted from the other in order to create more gaps within the larger bodies of forestry. A threshold is defined for each type of tree, taking into account the maximum value generated during the creation of the Perlin noise layers. Areas below any of the thresholds are left without trees. The algorithm then iterates across all cells in the grid, and if a tree can be placed, it uses the value generated by the layers of noise to determine what type of tree (if any) is placed based on the thresholds. A pseudocode representation can be seen in Algorithm 3.

Data: The map Result: A tree map

foreach coordinate coord in map do

calculate two different Perlin noise values based on different offsets;

subtract one from the other;

compare the final value with the current largest and replace if larger;

end

calculate the threshold values for each type of tree based on largest;

foreach coordinate coord in map do if coord is plantable then

if Perlin value at coord is above one of the tree thresholds then place a tree of the corresponding type at coord in the tree map;

end end end

Algorithm 3: Pseudocode Perlin noise algorithm.

A sample map produced by the algorithm can be seen in Figure 3.9. Clustering is once again present, with clumps of trees distributed in large groupings.

Figure 3.9: A sample forest generated using the Perlin noise algorithm.

(20)

3.2.4 Cellular Automata

For the purpose of the algorithm implemented here, the rules state that a cell is affected by eight of its neighbours - the four directly adjacent, and those which are also diagonally accessible. The algorithm is partially based on one outlined by Rakaska [33], expanded to include more states in order to accommodate the differing types of trees.

An initial set of percentages is used to perform a random fill of parts of the map, providing a starting point for the automaton. These percentages represent the relative number of each type of tree that should be present in the map. As the borders of the map are most often used to designate the spawn points for player armies, these are initially disregarded by the random fill and only the centre parts of the map are filled based on the percentage.

Once this step is completed, the cellular automaton runs for a defined number of cycles in order to stabilise the pattern. In each cycle, each tile is examined and its state is altered based on its eight neighbours. The simple rule states that if a total of five or more adjacent tiles (including the tile being evaluated) have the same state, where each state represents either an empty tile, or a type of tree, then the tile itself should also switch to that state. This allows the map to evolve based on the states of adjacent grid cells. In the initial implementation, only three states are designated since only two types of trees exist.

This would, with the expansion of assets in the game, increase depending on the number of tree types. Once the automaton has completed its cycles, the states are evaluated and the corresponding tree is placed in each tile. A pseudocode representation can be seen in Algorithm 4.

Data: The map Result: A tree map

calculate and assign a tile state based on percentage fill for each tree type;

end

for i ← 0 to cyclecount do

check the states of the eight adjacent tiles;

if five or more tiles share the same state then change the state of coord to match that of tile;

end end end

place a tree based on the final state of coord;

end

Algorithm 4: Pseudocode Cellular Automaton algorithm.

A sample map produced by the algorithm can be seen in Figure 3.10.

3.3 Evaluation and Optimisation

The following subsections deal with the evaluation and optimisation algorithms implemented in Starshard, in addition to common methods that are often used to perform these two task. Research was conducted beforehand to identify these methods and how they could apply to Starshard, after which the algorithms discussed in the following subsec-

(21)

Figure 3.10: A sample forest generated using the Cellular Automaton algorithm.

tions were implemented as described.

3.3.1 Evaluation

The evaluation of procedurally generated maps is a broad and complex topic, with almost as many different implementations as there are games which make use of procedurally generated content. Much research has also been conducted in the area, focusing on maps generated for all manner of games and utilising a large variety of techniques for the evaluation. The purpose of this section is to detail the evaluation steps taken by the procedural map generator in Starshard and the rationale behind their choices.

Evaluation of procedural maps can be broken into two groups of techniques: human- based techniques such as play testing and forced rank evaluations, or metric-based techniques which evaluate based on a set of parameters [34]. Of these two, only metric-based techniques are suitable for online generation as human-based techniques require feedback from humans before they can be released. For this reason, Starshard’s evaluation methods are based around a metric approach as one of the primary motivations for the use of procedural content is to provide new content immediately on player request without human interference.

Liapis et al. suggest one method for evaluation that partially imitates the benefits of human-based techniques while being metric-based: the use of procedural personas [30].

These procedural personas would represent archetypal player behaviours and be evolved on maps from the game in question. By using these personas, it would be possible to evaluate both the quality of the map for players with different styles of play, and also whether the map is playable at all. However, using these personas for evaluation and optimisation presents the risk of maps being optimised for one particular style of play and render it uninteresting for others.

Another frequently used method for evaluating maps is to accept or reject maps based on specific parameters by using metrics such as fitness functions [5]. These often involve evaluating the map based on certain criteria and returning scalar or vector representations of their suitability for use in the game. From this, maps can be judged based on their fitness and evolved or mutated further until their suitability reaches acceptable levels.

There are many metrics by which procedural maps can be evaluated. In the case of

(22)

Starshard, we can break down the desired criteria into:

(a) accessibility of all regions of the map to all units (b) no defects present

(c) acceptable visibility of the majority of the map (d) possibility for strategic play

(e) fairness in the size of unit placement regions

In order for a map to be regarded playable, it is important that players be able to access all significant regions of the map (a) with any of their units. This means that no indestructible objects or impassable terrain may be allowed to block off sections of the map. Since the jump height of units is variable, all sections of the map must be reachable by some path which does not exceed height differences of +/-1. Since all units can destroy objects which are considered destructible, these will not be considered to be barriers which prevent accessibility.

A second component to playability, in addition to aesthetics, comes in the form of ensuring that there are no defects present in maps (b). Trees, for example, should not be placed too close to each other, or be offset, in order to prevent z-fighting or other undesirable visual aspects. Obstacles should also not be placed in impossible positions and buildings or other structures with entrances, should be placed in a reasonable manner, with entrances accessible.

Due to Starshard’s two-dimensional nature, rotation of maps is currently not possible.

For this reason, it is important that the higher regions of the map be placed towards the back and the lower regions towards the front, such that the map is not completely obscured from the player’s view (c). Starshard features a tactical view mode designed to allow players to see through obstacles to some degree, therefore a small degree of obscured content is acceptable.

Maps which feature possibilities for strategic play (d) are also important to ensure that players are given the best gameplay experience. These include elements such as choke points or high ground advantages. While not absolutely mandatory, the presence of one or more of these elements separating armies is desirable in a good map.

An additional parameter that was initially considered to be of importance to evaluating maps in Starshard was the accessibility of resources and the “fairness” of their distribution between the players based on previous work by Togelius et al. [35]. It is easy to see how resources could provide advantages in real-time strategy games such as Starcraft, but in Starshard, resources provide no benefits during a match, and are only valuable once the match has concluded. For this reason, their accessibility in maps does not affect the player’s ability to have a “fair” match.

Instead of this, however, “fairness” is represented more in the amount of unit placement squares presented to the player for initially setting up their army (e). As obstacles positions are generated procedurally and players are always placed at opposite ends of the map, some method is needed in order to ensure that one player does not end up with a significantly smaller allotment of space in which to initially deploy their army.

3.3.2 Performing the Evaluation

The algorithms designed to perform evaluations of maps in Starshard are partially based on the use of fitness functions, with the ultimate goal being that the optimisation algorithms then make use of these in order to work towards improving the scores and rendering the map more playable.

(23)

Evaluation of maps in Starshard begins by checking to see if the map is rotated in such a way that an acceptable level of visibility is achieved (c). This is achieved by calculating the average heights of each of the four edges of the map. Once these have been calculated, the highest and second highest averages are chosen as a reference to be later passed to the optimisation algorithms for rotation. In order to preserve the pseudo-natural terrain of the map, it is not desirable to modify the map such that it becomes a smooth slope - some degree of variations in height could be important to maintain interest in the map.

However, there can be rare occasions in which the higher edges are directly opposite each other with a dip in the middle. In these cases, it may sometimes be necessary to perform some degree of modifications to the height of the terrain.

Once the correct rotation of the map has been determined, all unblocked tiles on the map are added to a set, and a flood fill is used to determine the accessibility of all regions of the map (a). This flood fill will access all tiles from a start point randomly chosen from the initial set and add them to an individual “region” set. The flood fill is implemented such that it will not expand through obstacles, or past height differences that are too high for certain unit types. These sets, each of which contain isolated regions of the map, are kept for use in the optimisation algorithms, where the number of individual regions es- sentially represent the results of a fitness function; the more regions present, the worse the score. A pseudocode representation of the algorithm can be seen in Algorithm 5.

Data: The map

Result: A list of sets which represent isolated regions foreach coordinate coord in map do

if coord is unblocked then

add coord to set initialT iles;

end end

while initialT iles is not empty do

pick a random coord from initialT iles as a starting point for this region;

perform a flood fill which takes into account obstacles and height differences;

remove all coord reached by the flood fill from initialT iles;

add all coord reached by the flood fill to a set for this region;

end

Algorithm 5: Pseudocode algorithm for identifying isolated regions in the map.

This flood fill partially performs the third step of the evaluation - to determine the presence of defects (b). To complement this, an additional scan of the map is performed in which objects are checked to ensure that their positioning is plausible. This involves registering any objects that are placed on tiles they cannot be placed upon, such as trees on water, and any larger objects that are placed adjacent to each other in a manner that could cause z-fighting or other undesirable artefacts. As buildings are not currently present in the portions of the procedural generation incorporated in this thesis, no implementation has been provided to ensure their correct placement at this stage.

Identifying potential for strategic obstacles such as choke points is not a simple task (d). In Starshard, focus is placed more on adding such obstacles during the optimisation stage, where new pathways through the map can be constructed which can fulfil such purposes.

The evaluation of unit placement tiles (e) involves first checking all available positions

(24)

within the initially designated region. This initially designated region is roughly a quarter of the map per player on opposite sides of the map. Any tiles which are occupied by an obstacle are immediately discounted. If any of the remaining tiles fall into regions that are blocked by objects, destructible or not, they are also discounted as potential placement tiles due to the disadvantage that would be incurred by spending turns to free the unit. A comparison of the remaining two values, representing the numbers of placement tiles available to each of the players, provides a measure for the fairness in this aspect.

3.3.3 Optimisation

Optimisation in this context refers not to improving running times of algorithms or de- creasing resources used, but to the systematic removal of defects and steps taken to ensure that the previously stated desired criteria are met in each map.

One popular method for performing this kind of optimisation is through the use of multiobjective evolutionary algorithms. Applied to optimisation, the goal of these algorithms is to optimise a set of two or more functions that measure different features desirable in the solution [36]. Since these features often conflict on some level and no solution will be optimal for all of the features, a set of optimal solutions, referred to as the

“Pareto optimal set”, is produced.

Standard forms of evolution are also often used to optimise procedural content [37].

Many of these algorithms involve evolving multiple solutions and comparing their fitness in a tournament, where the more suitable map is chosen and proceeds. These come in many forms, including various genetic programming algorithms [38], or algorithms that fall under the umbrella of evolutionary programming [39].

Some aspects of the optimisation of maps for Starshard are too simplistic to merit the use of evolutionary algorithms. Achieving an acceptable level of visibility (c), for example, is accomplished trivially by rotating the entire heightmap such that the two highest edges are towards the back. In the event that the two highest edges are directly opposite to each other, the heightmap is discarded and a new one is generated. Minor differences in height or obstacles obscuring parts of the map are considered acceptable due to the availability of the tactical view mode.

In order to eliminate the potential of z-fighting resulting from obstacles placed too close together (b), a similarly trivial method is used. By varying the positioning of neighbouring objects on the z-axis by a fractional amount, these effects can be completely eliminated.

The removal of larger defects (b), ensuring that all parts of the map are connected (a), improving on strategic opportunities where possible (d), and ensuring a fair number of placement tiles for each player (e), are all handled using a simplified form of evolutionary programming [39][40]. The values provided by these aforementioned criteria are used as the results of the fitness functions and mutations come in the form of creating new pathways through the map by removing obstacles or altering terrain height.

Carrying out the mutations is performed using a modified version of the A* search algorithm [41], which determines the path of least cost between two unconnected segments of the map. As the goal of the path is to connect regions with minimal adjustments to the map itself, indestructible obstacles, impassable tiles, and differences in height incur higher costs in the pathfinding algorithm such that they will not be included as part of the path unless no other option is present. Of the three, obstacles have the lowest cost as it is preferable to remove these as opposed to altering the heightmap or terrain. Destructible obstacles are not considered to be barriers to progress in this instance and regions con-

(25)

taining only one or two tiles are considered too insignificant to require connecting to the rest of the map.

Once a path has been determined between each of the regions in the map, it is tra- versed, and any indestructible obstacles, differences in height, or impassable tiles en- countered along the path are modified to ensure that they no longer impede access to the regions. Indestructible obstacles are removed to open the pathway, height differences are eliminated by raising or lowering the tiles leading up to the jump to provide a smooth ramp, and impassable tiles are bridged using one of the many “bridge” type tiles. A pseudocode representation of the algorithm can be seen in Algorithm 6.

Data: The map

Result: A mutated version of map

select points p1 and p2 from neighbouring regions;

use A* to determine a path between p1 and p2;

foreach coord in the path do

if there is an indestructible obstacle at coord then remove the obstacle;

end

else if there is an impassable height difference at coord then

alter the heightmap along this segment of the path to provide a natural slope;

end

else if there is an impassable tile at coord then replace the tile with a bridge tile;

end end

Algorithm 6: Pseudocode algorithm for connecting isolated regions.

Interestingly, the process of creating these paths between regions often helps to in- troduce potential strategic elements to the map, particularly in the form of choke points.

Since these narrow paths become the only way to proceed between separate regions, they form bottlenecks. While this is good for strategy, it is not necessarily desirable for the sole access points between regions to be choke points in every single map - this would become repetitive and uninteresting. For this reason, multiple paths are often selected between two regions to allow for multiple points of access, or paths are widened to reduce the bottleneck.

The last part of the mutation is to ensure a fair distribution of placement tiles without these placement regions extending too far out into the map. This is done by removing nearby destructible and indestructible objects to provide more space for a player with less placement tiles. At the same, time, the number of placement tiles granted to the opposing player may be slightly reduced while ensuring that a minimum of one fifth of the map is granted for placing their units.

Once this is complete, the map is re-evaluated using the same criteria to see if it has reached a suitable stage for acceptance based on the values of the fitness functions. If not, further mutations are applied in order to evolve the map once again, with the criteria of ensuring map interconnectivity being given priority. Due to the methods used to connect the fragmented regions and balance the numbers of placement tiles, there is very little conflict in the criteria and the number of cycles should generally be relatively low.

(26)

4 Results

This chapter presents the results of the controlled experiment conducted on forestry generation using the four procedural algorithms in addition to those of the survey. The controlled experiment was run a total of ten times for each of the four algorithms.

The execution times and number of defects produced for the four algorithms can be seen in Table 4.1. Execution times are represented in milliseconds rounded to four decimal places. Standard error is included to provide a more concrete illustration of the confidence intervals within each variable for all four algorithms and its effect on the significance of the results. The experiment was performed using the Unity 5.1.2f1 Editor to run the game and produce procedural maps on a laptop running Windows 8.1, with 8GB of ram, an i7-4710MQ 2.5GHz, and a 4GB NVIDIA GeForce GTX 860M.

GS RN PN CA

Run Time Defects Time Defects Time Defects Time Defects

1 2.2735 0 1.2440 296 0.3802 78 0.9661 33

2 0.6195 0 0.1158 12 0.1856 195 0.4795 87

3 2.1742 0 0.1324 6 0.0924 66 0.4881 69

4 1.4242 3 0.2095 501 0.0936 57 0.4996 80

5 2.2851 1 0.1276 54 0.1453 246 0.5156 88

6 0.9865 0 1.2329 244 0.4237 304 1.0059 90

7 1.8237 0 0.1680 347 0.1121 27 0.4623 63

8 1.0892 0 0.2046 19 0.1051 150 0.4758 65

9 1.3372 0 0.1996 280 0.0911 70 0.5637 70

10 2.3910 1 0.1832 407 0.0936 83 0.4836 44

Average 1.6404 0.5 0.3818 216.6 0.1723 127.6 0.5940 68.9 Std. Err. 0.1812 0.3073 0.1295 57.4372 0.0358 29.2207 0.0597 5.9823 Table 4.1: Execution times in milliseconds and number of defects produced for the four algorithms, where GS refers to growth simulation, RN to random number generation, PN to Perlin noise, and CA to cellular automata.

Figure 4.11 and Figure 4.12 provide complementary charts depicting the averages of execution times and number of defects produced respectively.

4.2 Literature Study

Several sources have been deemed as being particularly relevant to the evaluation and optimisation algorithms as a result of the study. These include the work of Liapis et al.

on procedural personas for evaluating procedurally generated maps [30], the discussions of Togelius et al. on fitness functions and multiobjective evolutionary algorithms [5], additional works on multiobjective evolutionary algorithms by Saetrom et al. [36], and a more general overview on the subject of evolutionary algorithms for procedural content generation by Ashlock [37].

Parts of many of these sources were incorporated in the design and implementation of the evaluation and optimisation algorithms as discussed in the implementation chapter,

(27)

Figure 4.11: Average execution times for the four algorithms shown as a graph with standard error.

Figure 4.12: Average number of defects produced by the four algorithms shown as a graph with standard error.

and a few show great promise for future work with improving these algorithms even further.

(28)

4.3 Survey results

In this section, the results of the survey are presented with average values for each of the criteria shown for the different screen captures. A total of 27 responses were recorded from a range of communities and individuals. The survey was submitted to several communities interested in video games and displayed publicly for a period of two weeks.

Communities were selected based on their general focus, with one being built around tactical role-playing games similar to Starshard, and another around independent video game development. As the surveys were posted publicly for an extended period of time, there is no way to deduce the exact number of individuals to whom it was submitted. Full results can be found in Appendix B. A mapping of screen capture numbers to the algorithms used to create them can be seen in Table 4.2. A 1-5 Likert scale was used to rate each of the four map criteria, with 1 being poor, and 5 being excellent. The average results for each of these can be seen in Table 4.3.

Screen capture Produced by 1 Cellular automata

2 Growth sim

3 Perlin noise

4 Random number

5 Human designer

6 Cellular automata

7 Growth sim

8 Perlin noise

9 Random number

10 Human designer

Table 4.2: Mapping of screen capture numbers to their method of production.

Screen capture Aesthetics Strategic Potential Plausibility Playability

1 2.78 3.11 3.89 3.19

2 3.33 3.07 3.56 3.30

3 3.22 3.22 3.96 3.37

4 3.22 3.19 3.37 3.44

5 3.56 4.15 3.63 3.93

6 3.26 3.26 2.81 2.81

7 3.48 3.44 3.74 3.74

8 3.04 2.89 3.56 2.63

9 2.74 2.96 3.48 2.74

10 3.85 3.93 3.78 3.74

Table 4.3: Average scores for Aesthetics, Strategic Potential, Plausibility, and Playability.

Figures 4.13, 4.14, 4.15, and 4.16 provide complimentary charts for aesthetics, strategic potential, plausibility, and playability respectively, including a confidence interval. In these figures, H refers to maps designed by a human, GS to growth simulation, RN to random number generation, PN to Perlin noise, and CA to cellular automata.

Table 4.4 shows the results of whether or not players thought individual screen captures were created by a human designer.

(29)

Figure 4.13: Average ratings for Aesthetics shown as a graph with confidence interval.

Figure 4.14: Average ratings for Strategic Potential shown as a graph with confidence interval.

Table 4.5 and Figure 4.17 show the responses to how often respondents stated they play video games. Table 4.6 shows which genres respondents stated they usually played.

Table 4.7 shows which TRPG game series respondents have played. Tables 4.8 and 4.9 show the gender and age distribution of respondents respectively.

(30)

Figure 4.15: Average ratings for Plausibility shown as a graph with confidence interval.

Figure 4.16: Average ratings for Playability shown as a graph with confidence interval.

Screen capture Yes No

1 37% 63%

2 40.7% 59.3%

3 59.3% 40.7%

4 25.9% 74.1%

5 66.7% 33.3%

6 33.3% 66.7%

7 55.6% 44.4%

8 51.9% 48.1%

9 33.3% 66.7%

10 74.1% 25.9%

Table 4.4: Percentage values for whether respondents thought a screen capture was designed by a human. Rows in bold indicate the maps that were designed by a human.

(31)

Frequency Number of Respondents

Daily 11

Weekly 6

A few times a month 4

Occasionally every year 4

Never 2

Table 4.5: How often respondents play games.

Figure 4.17: Pie chart representation of how often respondents play games.

Genre Number of Respondents

RPG 15

TRPG 4

Online 6

Strategy 12

Other 15

None of the above 4

Table 4.6: Which genres respondents tended to play.

Series Number of Respondents

Disgaea 3

Final Fantasy Tactics 6

Final Fantasy Tactics Advance 3

Fire Emblem 4

Valkyria Chronicles 5

XCOM 7

Other 4

None 13

Table 4.7: Which TRPG series respondents had played.

Male Female 70.4% 29.6%

Table 4.8: The gender distribution of respondents.

(32)

Age Range Number of Respondents

0-12 0

13-18 0

19-25 17

26-32 6

33-39 2

40+ 2

Table 4.9: The age distribution of respondents.

(33)

5 Analysis

In this chapter, an analysis of the data collected through the experiments and survey is presented. Forestry generation algorithms are compared based on their suitability for online procedural map generation and number of defects produced. The data from the survey is used to provide insights on how respondents reacted to the maps produced by these different algorithms, how successful the evaluation and optimisation algorithms were at improving maps, and to what degree the individuals were able to differentiate between procedurally generated maps and those produced by a human designer. Statistical analysis is utilised where relevant.

By making use of a One-Way ANOVA analysis, the statistical significance of the data produced by the controlled experiment could be determined. The One-Way ANOVA test was chosen as the data for the samples should be normally distributed, though the presence of some outliers may have had an impact on skewing the results away from normal distribution. Based on the P-values, which are displayed in Table 5.10 and Table 5.12, the differences are statistically significant as the P-value is <.0001 for execution time and 0.001111 for number of defects produced. An additional Tukey HSD post-test, the results of which can be seen in Tables 5.11 and 5.13, shows where this significance lies.

ANOVA refers to the analysis of variance test in which multiple sets of data are compared in order to determine if significant differences exist between them. In the results tables, SS refers to the sum of squares, MS to the mean square, and P to the significance of the data. A P value of less than 0.05 indicates that there is a statistical significance present in the data. Tukey’s HSD post-test is used to compare two sets of data in a manner similar to a t-test in order to determine where to find this statistical significance in the data.

The data on execution times shows that, while all four algorithms are fast enough to be suitable for online content generation, the execution time of the growth simulation algorithm is significantly longer than that of any of the other three which are relatively close in comparison. Of these, the Perlin noise algorithm appears to be the quickest.

Some outliers in the data for execution time can most likely be attributed to unavoidable background processes running on the test machine, or other subroutines being performed by the Unity engine.

Source SS df MS F P

Treatment 14.028796 3 4.676265 36.91 <.0001

Error 3.7999 30 0.1267

Total 20.017038 43

Table 5.10: Results of the One-Way ANOVA analysis on the execution time of the four algorithms.

From the results on number of defects produced by the algorithms, we can see that the growth simulation and cellular automata algorithms produce a relatively stable number of defects, whereas Perlin noise can be moderately erratic, and the values for random number generation are completely unpredictable with a much higher standard error. The growth

(34)

First Algorithm Second Algorithm P Growth Simulation Random number <.01 Growth Simulation Perlin noise <.01 Growth Simulation Cellular automata <.01

Random number Perlin noise non-significant Random number Cellular automata non-significant Perlin noise Cellular automata non-significant

Table 5.11: Results of the Tukey HSD post-test on the execution time of the four algorithms.

simulation algorithm is far ahead of the others in terms of limiting the number of defects, only rarely producing a single digit number, whereas the remaining three frequently display far higher figures.

These results are not entirely surprising; similarities in the nature of the growth simulation and cellular automata algorithms and the way in which they evolve based on proximity over a number of cycles mean that it makes sense that the two would exhibit similar properties in the stability of defect numbers. The high number of defects displayed by the cellular automaton can be attributed to the fact that it spreads using only adjacent tiles, resulting in clusters of trees that can be too close together. Perlin noise, due to being gradient noise, also exhibits a large amount of clustering as the gradual change in values between coordinates will mean large, tightly packed regions exceed the thresholds required for placing trees.

Source SS df MS F P

Treatment 251785.4 3 83928.4667 7.14 0.001111 Error 317260.1 27 11750.3741

Total 628773.6 39

Table 5.12: Results of the One-Way ANOVA analysis on number of defects produced by the four algorithms.

First Algorithm Second Algorithm P Growth Simulation Random number <.01 Growth Simulation Perlin noise non-significant Growth Simulation Cellular automata non-significant Random number Perlin noise non-significant Random number Cellular automata <.05

Perlin noise Cellular automata non-significant

Table 5.13: Results of the Tukey HSD post-test on number of defects produced by the four algorithms.

5.2 Survey

Kruskal-Wallis analysis of the data produced by the survey on the four rating criteria, seen in Tables 5.14, 5.16, 5.18, and 5.20, shows that the data has statistical significance.

(35)

This was used in place of One-Way ANOVA analysis as data from the Likert scale is not usually normally distributed, thus requiring the use of Kruskal-Wallis tests instead. Pair- ings which produced significant results in the Mann-Whitney U-test post-test are shown in Tables 5.15, 5.17, 5.19, and 5.21.

The Kruskal-Wallis test is similar to ANOVA, though non-parametric in nature. This means that the data used need not be normally distributed, as survey data is generally not.

It is used to determine if at least one set in the samples dominates another in a statistically significant way. Similarly, the Mann-Whitney U-test can be used in place of the t-test for data that does not follow normal distribution in order to find the specific sets with statistically significant differences.

H df P

26.105 9 0.001964

Table 5.14: Results of the Kruskal-Wallis test on the aesthetics scores from the survey.

First Screen Capture Second Screen Capture P

1 2 0.0230

1 5 0.0047

1 7 0.0060

1 10 0.0002

2 9 0.0230

2 10 0.0410

3 10 0.0170

4 10 0.0210

5 9 0.0043

7 9 0.0060

8 10 0.0063

9 10 0.0002

Table 5.15: Significant results of the Mann-Whitney U-test on the aesthetics scores from the survey.

H df P

40.851 9 <0.0001

Table 5.16: Results of the Kruskal-Wallis test on the strategic potential scores from the survey.

Generation, evaluation, and optimisation of procedural 2D tile-based maps in turn-based tactical video games

Author: Ioseff Griffith

Supervisor: Johan Hagelbäck Semester: VT 2016

Subject: Computer Science

Bachelor Thesis Project

Generation, evaluation, and

optimisation of procedural 2D

tile-based maps in turn-based

tactical video games

Contents

1 Introduction

2 Method

3 Implementation

4 Results

5 Analysis