Master of Science in Engineering: Game and Software Engineering May 2018
Generation and evaluation of collision
geometry based on drawings
Albin Bernhardsson
Ivar Björling
This thesis is submitted to the Faculty of Computing at Blekinge Institute of Technology in partial fulfilment of the requirements for the degree of Master of Science in Engineering: Game and Software Engineering. The thesis is equivalent to 20 weeks of full time studies.
The authors declare that they are the sole authors of this thesis and that they have not used any sources other than those listed in the bibliography and identified as references. They further declare that they have not submitted this thesis at any other institution to obtain a degree.
Contact Information: Author(s): Albin Bernhardsson E-mail: albg13@student.bth.se Ivar Björling E-mail: ivbj13@student.bth.se University advisors: Dr. Prashant Goswami
Department of Creative Technologies Dr. Veronica Sundstedt
Department of Creative Technologies
Faculty of Computing Internet : www.bth.se
Blekinge Institute of Technology Phone : +46 455 38 50 00 SE–371 79 Karlskrona, Sweden Fax : +46 455 38 50 57
Abstract
Background. Many video games allow for creative expression. Attractive Interac-tive AB is developing such a game, in which players draw their own levels using pen and paper. For such a game to work, collision geometry needs to be generated from photos of hand-drawn video game levels.
Objectives. The main goal of the thesis is to create an algorithm for generating collision geometry from photos of hand-drawn video game levels and to determine whether the generated geometry can replace handcrafted geometry. Handcrafted ge-ometry is manually created using vector graphics editing tools.
Methods. A method for generating collision geometry from photos of drawings is implemented. The quality of the generated geometry is evaluated and compared to handcrafted geometry in terms of vertex count and positional accuracy. Ground truths are used to determine the positional accuracy of collision geometry by calcu-lating the resemblance of the created collision geometry and the respective ground truth.
Results. The generated geometry has a higher positional accuracy and on average a lower vertex count than the handcrafted geometry. Performance measurements for two different configurations of the algorithm are presented.
Conclusions. Collision geometry generated by the presented algorithm has a higher quality than handcrafted geometry. Thus, the generated geometry can replace hand-crafted geometry.
Keywords: vectorization, image processing, segmentation, line simplification
Sammanfattning
Bakgrund. Många datorspel möjliggör kreativa yttringar. Attractive Interactive AB utvecklar ett sådant spel, i vilket spelare ritar sina egna nivåer med hjälp av papper och penna. För att ett sådant spel ska vara möjligt måste kollisionsgeometri genereras från foton av handritade spelnivåer.
Syfte. Syftet med examensarbetet är att skapa en algoritm för att generera kollisions-geometri från foton av handritade datorspelsnivåer och fastställa om den genererade geometrin kan ersätta handgjord geometri. Handgjord kollisionsgeometri skapas manuellt genom använding av redigeringsverktyg för vectorgrafik.
Metod. En metod för att generera kollisionsgeometri från foton av ritade datorspel-snivåer implementeras. Kvaliteten av den genererade geometrin evalueras och jämförs med handgjord geometri i fråga om vertexantal och positionsnoggrannhet. Grun-dreferenser används för att fastställa positionsnoggrannheten av kollisionsgeometri genom att beräkna likheten av den skapta kollisionsgeometrin och respektive grun-dreferens.
Resultat. Den genererade geometrin har en högre positionsnoggrannhet och i genomsnitt ett lägre vertexantal än den handgjorda geometrin. Prestandamätningar för två olika konfigurationer av algoritmen presenteras.
Slutsatser. Kollisionsgeometrin som genererats av den föreslagna algoritmen har en högre kvalitet än handgjord geometri. Därmed kan den genererade geometrin ersätta handgjord geometri.
Nyckelord: vektorisering, bildbehandling, segmentering, linjeförenkling
Acknowledgments
We would like to thank our supervisors: Dr. Prashant Goswami and Dr. Veronica Sundstedt for their expert advice, support and encouragement throughout the thesis. We would also like to thank Simon Hyltegård for reviewing the handcrafted geometry and Bo Bernhardsson for his feedback on the report.
Contents
Abstract i Sammanfattning iii Acknowledgments v 1 Introduction 1 1.1 Background . . . 21.2 Aims and objectives . . . 3
1.3 Contribution . . . 4 1.4 Limitations . . . 4 1.5 Overview of report . . . 5 2 Related work 7 2.1 Preprocessing . . . 7 2.1.1 Kuwahara filters . . . 7 2.1.2 Bilateral filter . . . 8 2.1.3 Median filters . . . 8 2.2 Quantization . . . 9 2.3 Tracing . . . 9 2.4 Vertex reduction . . . 10 3 Method 11 3.1 Collision geometry generation . . . 11
3.1.1 Preprocessing . . . 11 3.1.2 Quantization . . . 13 3.1.3 Tracing . . . 16 3.1.4 Vertex reduction . . . 18 3.2 Evaluation . . . 19 3.2.1 Measurements . . . 20 3.2.2 Performing measurements . . . 21 3.2.3 Experiment . . . 22 3.3 Implementation details . . . 26 4 Results 29 4.1 Shape . . . 29 4.2 Vertex count . . . 30 4.3 Processing time . . . 30 vii
4.4 Memory usage . . . 30
5 Analysis and discussion 35 5.1 Method of quality analysis . . . 35
5.2 Analysis of quality results . . . 36
5.3 Analysis of performance results . . . 37
5.4 Validity threats . . . 37
5.5 Sustainability and ethics . . . 38
6 Conclusions and future work 39 6.1 Future work . . . 39
References 41
A Tables of results 45
B Ground truth images 49
C Perturbed images 53
D Generated geometry 57
Chapter 1
Introduction
Many modern video games allow their players to express themselves creatively. In Minecraft [1] players can create three-dimensional structures through placing blocks in the game world. Minecraft: Education Edition [2] has been used in schools, for example by tasking students with designing a city while taking into account problems like sustainable development. Through the Block by Block [3] project, a collaboration between the creators of Minecraft and the United Nations, it has also been used as a way to engage urban communities in the design of public spaces.
Some games enable players’ creativity by allowing them to draw objects in the game world. In Crayon Physics Deluxe [4], players draw physics objects in order to solve puzzles. In Kirby: Canvas Curse [5] the player is tasked with drawing the way the protagonist travels in order to clear the levels. In Draw a Stickman: Epic [6] the player can customize the appearance of the game by drawing their own player character and items in the game.
Some games allow the players to draw their own levels. In Line Rider [7], the player draws lines on which a character in a sled then rides. Using simple tools, players can draw elaborate levels including various stunts. The Draw-Play game series [8, 9, 10] are platformer games in which the player has to draw the platforms on which they can stand in order to clear the levels.
These games allow the player to draw digitally, either using a computer mouse or a touch screen. By generating collision geometry based on the player’s drawings, the drawn objects or levels can be interacted with. Collision geometry in two-dimensional games is a collection of line segments describing the outline of an object. This can be generated by tracking the position of the mouse cursor and creating line segments between the current position and the position of the previous frame. If a touch screen is used, the position where the player touches the screen can be tracked rather than the mouse cursor.
Attractive Interactive AB is currently developing a game in which players can draw their own levels using pen and paper. The drawings can then be played by taking a photo of them. However, to make these levels playable, collision geometry is needed.
Generating collision geometry from photos of hand-drawn video game levels can not be done in the same way as when using digital tools. Temporal information is not available in a photo and so the position of the user’s pen can not be tracked over time. Instead, another method of generating collision geometry is needed.
Collision geometry can be handcrafted using vector graphics editing tools. How-ever, the process of manually creating geometry is tedious, especially for complex
2 Chapter 1. Introduction levels with many details. Furthermore, Attractive Interactive AB wants Paper Game to be user friendly. Thus, a playable level should be provided effortlessly after taking a photograph.
No published solution for generating collision geometry from color photos was found. The focus of this thesis is to create an algorithm for generating collision geometry from photos of hand-drawn video game levels and to evaluate the quality of the generated collision geometry. Since no previous method was found, it is evaluated by comparing it to handcrafted collision geometry.
1.1
Background
Attractive Interactive AB is a video game developer based in Karlskrona and founded in 2017. Attractive Interactive AB is developing a game called Paper Game in which players are able to express themselves creatively. Paper Game is a mobile application which enables players to create their own platformer game levels by taking photos of levels drawn on paper using color markers. In Paper Game, the players are able to assign functionalities to colors. This could be used to, for example, use black to draw platforms on which the player can stand and use red for dangerous objects that harm the player on contact. Attractive Interactive AB believes that creating video game levels using pen and paper is a fun and an intuitive way of expressing creativity, particularly for children. Attractive Interactive AB considers whether Paper Game could be used as an educational tool in schools, introducing schoolchildren to art, programming and game design.
Paper Game is developed using the game engine Unity [11] which is a cross-platform game engine for creating two-dimensional and three-dimensional games. Unity supports a collection of functionalities required in most types of game devel-opment.
Attractive Interactive AB is interested in a method to generate collision geometry from photos of hand-drawn video game levels, drawn on paper using color markers. Additionally, the collision geometry generated should be generated in a format sup-ported by Unity.
Collision shapes represent the boundaries of an object and enable physical inter-action with other objects. Geometrical primitives like boxes and circles are examples of collision shapes which can be used in order to represent the boundaries of ob-jects. However, in order to represent the outline of more complex shaped objects, line segments can be used. In order to generate collision geometry that accurately represents the boundaries of complex shaped objects and is supported by modern game engines, the collision geometry is represented as a collection of polylines, each representing the outline of an object.
No published method for generating collision geometry from color photos was found. However, it was identified that the process of generating collision geometry can be split into several smaller steps. The steps are explained below and an overview of the steps can be seen in Figure 1.1.
Preprocessing This step mainly aims to reduce noise and texture of the image while preserving its edges. Noise are color artifacts which appear during photography.
1.2. Aims and objectives 3 Quantization This step extracts the binary layers by quantizing the image from the preprocessing step into a limited amount of colors.
Tracing This step generates collision geometry by tracing the binary layers from the quantization step.
Vertex reduction This step takes the highly detailed collision geometry from the tracing step and removes vertices in order to reduce the vertex count while attempting to maintain the shape as closely as possible.
(a) Input image. (b) Filtered image. (c) Binary layers.
(d) Collision geometry. (e) Reduced collision geometry.
Figure 1.1: Overview of the steps performed when generating collision geometry. The input image (a) is filtered in order to reduce noise (b). The filtered image is quantized into binary layers (c). The binary layers are traced, generating collision geometry (d). The collision geometry is processed in order to lower the vertex count (e).
1.2
Aims and objectives
The aim of the thesis is to create an algorithm generating collision geometry from photos of hand-drawn two-dimensional video game levels and to evaluate the quality of the generated collision geometry by comparing it to handcrafted geometry from the same photo.
The thesis question is:
4 Chapter 1. Introduction
level such that the generated geometry can replace handcrafted geometry of the same photo?
In order to replace handcrafted geometry the generated geometry needs to fulfill the following two criteria:
• The generated geometry has to match the shape of the geometry in the photo equally well or better than the handcrafted geometry.
• The generated geometry has to have a similar or lower vertex count than the handcrafted geometry.
The thesis is divided into the following objectives:
• Develop an algorithm that generates collision geometry from photos of hand-drawn video game levels.
• Create handcrafted geometry from the same photos.
• Develop an algorithm that evaluates the quality of collision geometry in relation to a ground truth.
• Evaluate the quality of the generated geometry and the handcrafted geometry. • Measure the performance of the algorithm in terms of processing time and
memory usage.
1.3
Contribution
A method is proposed where collision geometry is generated from photos of hand-drawn two-dimensional video game levels using color labels. Color labels are used to quantize the photo into foreground layers and a background layer. The foreground layers are interpreted as silhouettes representing the shapes of the objects in the photo. The outlines of the silhouettes are traced in order to generate the collision geometry. The collision geometry is generated in a format which is supported by modern game engines like Unity.
The quality of generated collision geometry is evaluated by comparing it to the quality of handcrafted geometry. The quality of the collision geometry is measured by rasterizing the created geometry and comparing the rasterized image against a ground truth image.
1.4
Limitations
Whether the collision geometry would be usable as a video game level is not eval-uated. For example, if a user draws a platformer game level in which the player is required to jump across a gap but the gap is too wide for the player to cross, this would make the level impossible to clear. This is not evaluated as these conditions depend heavily on what game the collision geometry is to be used in. The proposed method is intended to be generally applicable, regardless of genre or game specifics.
1.5. Overview of report 5 Attractive Interactive AB believes that the process of creating video games levels using pen and paper is intuitive. Furthermore, they consider using the proposed method in their game for learning purposes. However, it is not part of the thesis evaluating whether the process of creating video game levels using pen and paper is intuitive or whether the proposed method could be used in a game for learning purposes.
1.5
Overview of report
The remainder of the report is divided into the following chapters:
Chapter 2 presents previous work related to each of the steps for generating colli-sion geometry.
Chapter 3 presents the proposed algorithm and the method for evaluation. Chapter 4 presents the results of the evaluation.
Chapter 5 presents a discussion of the results obtained in the evaluation. Chapter 6 presents the conclusions of the thesis and areas for further study.
Chapter 2
Related work
No published method for generating collision geometry from color photos was found. However, related work exists which presents methods that generate geometry from images.
Rotaru et al. [12] propose a method which generates two-dimensional polygonal objects from ultrasound images. Their method could be used in order to generate collision geometry. However, their method is limited to grayscale images and can not generate concave geometry. Rotaru et al. [13] present a further developed method which is able to generate concave geometry. However, their method is still limited to grayscale images.
Birdal and Bala [14] present a novel method for vectorization of raster images. Their method is able to generate geometry from color images. However, using their method, the generated geometry is represented using geometric curves which are not supported by modern game engines. Thus, it can not be used as collision geometry. Dou et al. [15] present a method for generating facial geometry from an image. The method consists of two steps. First, geometry is estimated from landmarks using partial least-square regression. Then the face is reconstructed using coupled-dictionary learning. The method is shown to outperform four state-of-the art tech-niques. However, they generate 3D geometry and the method is specialized for facial geometry and can not handle general geometry of any shape.
Although no method for generating collision geometry from color photos was found, established methods for each step of generating collision geometry exist. In the following sections, methods for each step are presented. In Chapter 3, it is concluded which method(s) is most suited for generating collision geometry.
2.1
Preprocessing
Linear low-pass filters are best known to reduce noise and texture in images [16]. However, linear low-pass filters reduce all high-frequency components, including edges and corners. According to Papari, Petkov, and Campisi [17], non-linear low-pass filters have been developed that reduce texture and noise while preserving edges. Some non-linear low-pass filters that preserve edges are presented below.
2.1.1
Kuwahara filters
The classical Kuwahara filter introduced by Kuwahara et al. [18] was initially de-signed for smoothing and enhancing RI-angiocardiography images. Kuwahara filters
8 Chapter 2. Related work work by dividing the area around a pixel into subregions and finding the mean value from the subregion with the lowest variance. The classical Kuwahara filter [18] uses four rectangular subregions with overlapping edges, as seen in Figure 2.1. Other vari-ants of the classical Kuwahara filter have been developed in order to achieve different effects [19]. Changes made to the classical Kuwahara filter have been done: alter-ing the shape from which the subregions are generated[19, 20], changalter-ing the weights inside the subregions [16, 21] or adjusting the subregions size during operation [22]. The Kuwahara filter can not be directly applied to color images with multiple channels. The variance for each subregion in each channel can differ, thus causing ambiguity where the center pixel is determined by several subregions. Pritamdas, Singh and Singh [23] approximates the variance given a multi-channel image using
σ2 = 1 m × n m×n X i=1 ||xi− xmean||2
where σ2 is the variance, m × n is the size of the subregion, x
i is the ith pixel value
in the subregion and xmean is the mean of the subregion. By approximating the
variation, the Kuwahara filter can be applied to color images.
a
b
c d
Figure 2.1: Visualization of the classical Kuwahara 5x5 kernel using four rectangular subregions (a,b,c,d) around the center pixel. The center pixel is replaced by the mean value of the subregion with the lowest variance.
2.1.2
Bilateral filter
According to Thai et al. [24], the bilateral filter was introduced by Tomasi and Manduchi [25] based on the non-linear low-pass Gaussian filter developed by Aurich and Weule [26]. The bilateral filter works by adding weights to the standard low-pass Gaussian filter which are calculated based on the color difference between the pixels [24]. The bilateral filter results in sharp edges by preserving pixels with similar color [16, 27].
2.1.3
Median filters
Median filters are effective at reducing high-frequency salt-and-pepper type noise [27, 21, 28]. Median filters work by calculating the local median around each pixel and replace the pixel value with the median value [21].
2.2. Quantization 9 The classical median filter uses a static kernel size. The major disadvantage of the classical median filter is that it can only reduce noise in small scale areas without blurring effects [21]. The adaptive median filter enables adjustment of the kernel size depending on input [28], therefore making it more applicable for large scale area noise reduction compared to the classical median filter [21, 28].
2.2
Quantization
Color quantization methods can be divided into unguided methods, such as k-means, and methods where a palette is already known. For this thesis a palette may be provided by the user. Thus by calculating the distance between the color of the pixel to be quantized and the colors in the palette, the closest color in the palette can be picked.
There are multiple methods of calculating the distance between two colors. One method is to calculate the Euclidean distance in RGB space but more advanced methods exist that are designed to more closely match human perception. The International Commission on Illumination developed the CIE76 [29] ∆E∗ measure,
which is the Euclidean distance in CIELAB space rather than in RGB space. This metric was improved in CIE94 [30] to take perceptual non-uniformities into account, and improved further in CIEDE2000 [31].
2.3
Tracing
Line segments, circles, curves or polygons are geometric shapes which can be used instead of pixels in order to represent an image. The process of generating geometric shapes which represent the pixels of a raster image is called tracing.
Kolesnikov [32] and Wenyin and Dori [33] present several methods for tracing raster images. These tracing methods are categorized as thinning based methods [34] or non-thinning based methods [33].
Thinning based methods work by eroding the raster image in order to extract one-pixel-width raster skeletons which are traced into chains of line segments [34, 35]. Thinning based methods lose shape information when the image is eroded [35].
Non-thinning based methods work by calculating the medial axis for the edge pixels which is used in order to generate line segments [33, 35]. Non-thinning based methods have problems reconstructing junctions [35].
The thinning based methods and the non-thinning methods generate skeleton geometry. Skeleton geometry represents the shape of an object by containing multiple line segments with varying width.
Potrace [36] is an open-source application and library that performs image tracing for binary images. Potrace generates Bezier curves in order to represent the pixels of a raster image.
Maple [37] presents a method called marching squares, which is a modification of the algorithm called marching cubes by Lorensen and Cline [38]. The marching cubes algorithm was developed in order to create a medical tool helping physicians under-stand the complex anatomy of the body by generating three-dimensional surfaces
10 Chapter 2. Related work from multiple scanned magnetic resonance images [38]. The marching squares algo-rithm generates two-dimensional polygon geometry which represent the area of an object [37]. Maple [37] uses the marching squares algorithm in order to approximate the area of two-dimensional objects.
Xiong, Feng and Zhou [39] present a GPU-parallelized algorithm for contour-tracing of binary images. The contour-contour-tracing algorithm works by scanning each line in the image in order to find the boundary pixels. The image is traced by connecting the boundary pixels for each adjacent line.
2.4
Vertex reduction
Whereas the results of the tracing step will produce collision geometry, the amount of vertices may be too high to be of practical use. A higher amount of vertices means that more work needs to be performed in order to check whether a collision has occurred. For performance reasons a low amount of vertices is desirable as long as the shape of the outline can be maintained. The purpose of the vertex reduction step is thus to process the geometry in order to reduce the number of vertices.
Collision geometry can be treated as a polyline, a connected series of line seg-ments. There exist several line simplification algorithms that simplify polylines. Some examples are the Douglas-Peucker algorithm [40], the Visvalingam-Whyatt algorithm [41] and the Zhao-Saalfeld algorithm [42].
Shi and Cheung [43] evaluate the performance of several line simplification al-gorithms in terms of positional accuracy and processing time. The alal-gorithms eval-uated were Nth point routine, Perpendicular distance, Reumann-Witkam, Opheim, Visvalingam-Whyatt, Douglas-Peucker, Distance between points, Lang and Zhao-Saalfeld. They conclude that Douglas-Peucker has the highest positional accuracy but also the worst execution time, with a time complexity of O(n2) where n is the
number of points in the polyline. Lang and Zhao-Saalfeld are proposed as algorithms with a good compromise between positional accuracy and processing time, with a time complexity of O(n).
Liu and Chen [44] evaluate the performance of 4 line simplification algorithms: Step, line segment filtering, Douglas-Peucker and perpendicular distance. Like Shi and Cheung, they find Douglas-Peucker to produce the results with the smallest mean error, but also note its poor performance in terms of processing time.
Chapter 3
Method
The thesis consists of two parts: developing a method of generating collision ge-ometry from a photo and evaluating the performance of the method in comparison to handcrafted geometry. The method is evaluated by measuring the shape, vertex count, processing time and memory usage.
The implementation of the generation method is described in Section 3.1.
The evaluation is performed by implementing a method for evaluating the quality of collision geometry based on a ground truth. An experiment is conducted using this method on a data set of ground truth images and comparing the quality of the generated geometry to that of handcrafted geometry. The evaluation is described in Section 3.2.
3.1
Collision geometry generation
The collision geometry method takes a color image as an input and generates a list of outlines in the image. Each outline is described as a list of points forming line segments around an object in the image.
This is done in four steps previously presented in Section 1.1: preprocessing, quantization, tracing and vertex reduction.
Two different configurations are implemented:
• A configuration aimed at providing the highest quality geometry. This config-uration is referred to as ‘Quality’.
• A configuration aimed at providing the fastest results. This configuration is referred to as ‘Time’.
An overview of the process for both configurations can be seen in Figure 3.1. The Time configuration skips the preprocessing step and uses a faster, but less accurate, quantization method. Both configurations use the same tracing and vertex reduction methods.
3.1.1
Preprocessing
OpenCV [45] is an open source computer vision library that implements several well-established image filters, such as Gaussian blur and Sobel. Wherever applicable, OpenCV’s implementation is used in order to ensure a state-of-the-art implementa-tion.
12 Chapter 3. Method
Quality
Time
(a)
(b)
(c)
(d)
(e)
(f)
Figure 3.1: An input image (a) is processed. If the Quality configuration is used, the image is preprocessed (b) and quantized using CIEDE2000 (c). If the Time configuration is used, the image is quantized using linear Euclidean distance (d). The quantized image is then traced (e) and the number of vertices is reduced (f).
Two filters are used in the processing step. First a median filter is applied in order to reduce salt-and-pepper noise, efficiently removing deviating pixels in the image. The median filter is implemented using the one available in OpenCV [45] which is the classical median filter. The median filter is applied to the image using a 3x3 kernel. Secondly a Kuwahara filter is applied in order to enhance edges, reduce texture and remove additional noise in the image. The classical Kuwahara filter is implemented as no Kuwahara filter is available in OpenCV. The Kuwahara filter is applied to the image using a 5x5 kernel. The effects of the filters can be seen in Figure 3.2.
(a) Input image. (b) Median filter applied to the input image.
(c) Median filter and Kuwa-hara filter applied to the input image.
Figure 3.2: The input image (a) is processed in order to reduce noise by applying a median filter (b) and enhance edges by further applying a Kuwahara filter (c). Salt-and-pepper noise is reduced, textures are smoothed and edges are enhanced.
Kyprianidis, Kang and Döllner [19] conclude that Kuwahara filters are able to preserve boundary shapes in low-contrast regions better than other edge preserving filters such as bilateral filters and mean shift segmentation filters. Therefore, the
3.1. Collision geometry generation 13 Kuwahara filter was preferred over the bilateral filter. Thus, the bilateral filter is not used in the preprocessing step.
3.1.2
Quantization
In the quantization step, a color image is split up into several binary layers. Each of these binary layers can then be vectorized in the tracing step. The user supplies a list of colors, along with a background color. A layer is created for each color and each pixel is assigned to whichever layer’s color is the closest.
Two methods to determine the distance between two colors are implemented, linear Euclidean distance and CIEDE2000. Linear Euclidean distance is used in the Time configuration and was chosen for its speed of processing, whereas CIEDE2000 is used in the Quality configuration and was chosen as it closely matches human perception.
Linear Euclidean distance
Each color is treated as a three-dimensional vector and the Euclidean distance be-tween those vectors is calculated.
d = |~c1− ~c2|
Images in common file formats are by default stored in sRGB space. sRGB is a color space designed to match the gamma curve of CRT monitors and common viewing conditions, and is defined by international standard IEC 61966-2-1 [46]. The intensities of the color channels are encoded using a gamma curve. In order to get back to linear space, where the distance is calculated, the colors need to be transformed using the inverse of the gamma curve. This transformation is given in the IEC 61966-2-1 standard.
Clinear = Csrgb 12.92, Csrgb ≤ 0.04045 C srgb+0.055 1.055 2.4 , Csrgb > 0.04045 where C is R, G or B.
Performing these calculations for every pixel can be expensive, especially calcu-lating the power function. However, since each color channel is treated individually and with 8 bits per channel, there are only 256 possible input values. A lookup table is used to avoid having to perform any calculations at runtime.
CIEDE2000
CIEDE2000 ∆E∗ is a metric of the difference between two colors in CIELAB space.
Thus, in order to use it, the colors must first be converted from sRGB to CIELAB. First, the colors are transformed from sRGB to linear RGB space as described in the previous section. The colors are then converted to CIE XYZ as per IEC 61966-2-1 [46]. Additionally, a scaling factor of 100 is used to normalize the XYZ values in order to enable conversion to CIELAB. This scaling factor depends on the
14 Chapter 3. Method normalization of the reference white, which is explained later. The transformation is given by X Y Z = 100 0.4124 0.3576 0.1805 0.2126 0.7152 0.0722 0.0193 0.1192 0.9505 Rlinear Glinear Blinear .
The CIE XYZ colors can then be converted to CIELAB, as per the international standard ICC.1:2004-10 [47], using the following transformation:
L∗ = 116f Y Yn − 16 a∗ = 500 f X Xn − f Y Yn b∗ = 200 f Y Yn − f Z Zn where f (t) = (√3 t, t > δ3 t 3δ2 + 4 29, t ≤ δ 3 δ = 6 29.
Xn, Yn and Zn are the reference white. The Illuminant D65 is used, wherein
Xn= 95.047
Yn= 100.000
Zn= 108.883
when normalized to Y = 100.
D65 is a standard illuminant representing average daylight in Western and North-ern Europe. It is proposed by the IntNorth-ernational Commission on Illumination as the standard daylight illuminant, unless there are specific reasons for using another illu-minant [48].
Once two colors in CIELAB space are obtained, the ∆E00value can be calculated.
This process is described by Sharma, Wu and Dalal[49]. For two colors, L∗ 1, a ∗ 1, b ∗ 1 and L∗ 2, a ∗ 2, b ∗
2, the difference can be calculated in three steps.
1. Calculate C0
i and h 0 i:
3.1. Collision geometry generation 15 Ci,ab∗ =p(a∗i)2+ (b∗ i)2 ¯ Cab∗ = C ∗ 1,ab+ C ∗ 2,ab 2 G = 0.5 1 − s ( ¯Cab∗ )7 ( ¯Cab∗)7+ 257 ! a0i = (1 + G)a∗i Ci0 =p(a0i)2+ (b∗ i)2 h0i = ( 0, b∗i = a0i = 0 tan−1(b∗ i, a 0 i), otherwise where i = 1, 2. 2. Calculate ∆L0, ∆C0 and ∆H0: ∆L0 = L∗2− L∗ 1 ∆C0 = C20 − C10 ∆h0 = 0, C10C20 = 0 h02− h0 1, C 0 1C 0 2 6= 0; |h 0 2− h 0 1| ≤ 180 ◦ (h02− h0 1) − 360, C10C20 6= 0; (h02− h01) > 180◦ (h02− h0 1) + 360, C10C20 6= 0; (h02− h01) < −180◦ ∆H0 = 2pC10C20sin ∆h 0 2 .
16 Chapter 3. Method ¯ L0 = (L∗1 + L∗2)/2 ¯ C0 = (C10 + C20)/2 ¯ h0 = h01+h02 2 , |h 0 1− h 0 2| ≤ 180 ◦; C0 1C 0 2 6= 0 h0 1+h02+360◦ 2 , |h 0 1− h02| > 180◦; (h01+ h02) < 360◦; C10C20 6= 0 h01+h02−360◦ 2 , |h 0 1− h 0 2| > 180 ◦; (h0 1+ h 0 2) ≥ 360 ◦; C0 1C 0 2 6= 0 (h01+ h02), C10C20 = 0
T = 1 − 0.17cos(¯h0− 30◦) + 0.24cos(2¯h0) + 0.32cos(3¯h0+ 6◦) − 0.20cos(4¯h0− 63◦) ∆θ = 30exp ( − ¯ h0− 275◦ 25 2) RC = 2 s ¯ C07 ¯ C07+ 257 SL= 1 + 0.015( ¯L0− 50)2 p 20 + ( ¯L0− 50)2 SC = 1 + 0.045 ¯C0 SH = 1 + 0.015 ¯C0T RT = −sin(2∆θ)RC ∆E00= s ∆L0 kLSL 2 + ∆C 0 kCSC 2 + ∆H 0 kHSH 2 + RT ∆C0 kCSC ∆H0 kHSH . The parametric weighting factors kL = kC = kH = 1 are used.
3.1.3
Tracing
In the tracing step, the binary layers generated in the quantization step are traced, generating geometry.
None of the tracing methods found were developed for the purpose of generating collision geometry. Therefore, several methods presented in Chapter 2 are not directly applicable for generating collision geometry.
The thinning based methods and non-thinning based methods generate skeleton geometry. Skeleton geometry represents the internal structure of objects, rather than the outlines of objects. Thus, skeleton geometry can not be used as collision geometry.
Potrace generates geometric curves. Curves can be used in order to represent the boundary of an object. However, it is not supported by modern game engines. Line segments could be extracted using the curves. However, Potrace is licensed under the GNU General Public License, which requires that all derivative works also be licensed under the same license. Attractive Interactive AB does not wish to use that license. Thus, another method for tracing was preferred.
The marching squares algorithm generates two-dimensional polygon geometry representing the area of an object. The outlines of the generated geometry can be
3.1. Collision geometry generation 17 extracted in order to be used as collision geometry. Thus, a modified marching squares algorithm can be used in order to generate collision geometry.
The GPU-parallelized algorithm presented by Xiong, Feng and Zho generates line segments at the outlines of objects. Thus, the algorithm can be used in order to generate collision geometry.
Generating geometry
Both the modified marching squares algorithm and the GPU-parallelized algorithm generate geometry that can be used as collision geometry. No published sources were found comparing the performance of the algorithms. However, the modified marching squares was selected because of its apparent simplicity. Furthermore, a tutorial was found implementing the classical marching squares algorithm [50]. The presented code in the tutorial was not used. However, the information presented in the tutorial contributed to further understanding of the classical marching squares algorithm.
The classical marching squares algorithm generates two-dimensional triangles rep-resenting the area of an object. However, collision geometry should represent the outline of an object. Therefore, modifications to the classical marching squares algo-rithm are made in order to make the algoalgo-rithm generate line segments representing the outline of an object instead of triangles representing the area of an object. The difference of the geometry generated by the classical marching squares algorithm and the modified marching squares algorithm can be seen in Figure 3.3.
(a) (b) (c)
Figure 3.3: Illustration of the geometry generated using the classical marching squares algorithm and the modified marching squares algorithm. The binary image (a) is traced using the classical marching squares algorithm (b) and the modified marching squares algorithm (c).
Classical marching squares algorithm
The classical marching squares algorithm [37] is used to triangulate two-dimensional scalar fields. The marching squares algorithm can be applied to a binary image in order to generate geometry, as seen in Figure 3.4.
For every 2x2 block of pixels in a binary image, squares are generated in one of sixteen configurations depending on the pixel values in the square. Each square-configuration generates a different set of geometry, as seen in Figure 3.5.
There are two ambiguous square configurations where the pixels in the opposite corners of the square are active (configuration 5 and 10), as seen in Figure 3.6. The geometry generated by the ambiguous configurations is either connected or split,
18 Chapter 3. Method
(a) (b) (c) (d)
Figure 3.4: Illustration of the classical marching squares algorithm. The binary input image (a) is traced using marching squares (b). The classical marching squares configurations are calculated for each square using the pixel values in each block (c). The final geometry is visualized (d).
Figure 3.5: Visualization of the marching squares configurations for 2x2 pixel blocks [37].
depending on implementation. The algorithm implemented splits the geometry gen-erated for the ambiguous configurations.
Modified marching squares algorithm
The process of generating collision geometry using the modified marching squares algorithm is illustrated in Figure 3.7. The modified marching squares algorithm only generates geometry at the outline of an object.
3.1.4
Vertex reduction
The vertex reduction step takes the highly detailed geometry from the tracing step and removes vertices in order to reduce the vertex count while attempting to maintain the shape as closely as possible.
Each outline is processed separately. The two points farthest from each other are found and the outline is treated as two polylines between these points. Thus, line simplification algorithms can be used on the two polylines.
The line simplification algorithm selected was Douglas-Peucker [40] as it has the highest positional accuracy according to Shi and Cheung [43] and Liu and Chen [44].
3.2. Evaluation 19
(a) Configuration 5 (b) Configuration 10
Figure 3.6: Illustration of ambiguous configurations using the marching squares algorithm. The geometry generated is either split or connected.
(a) (b) (c) (d)
Figure 3.7: Illustration of the modified marching squares algorithm. The binary input image (a) is traced using marching squares (b). The modified marching squares configurations are calculated for each square using the pixel values in each block (c). The final geometry is visualized (d).
The Douglas-Peucker algorithm works by first selecting the first and last points of the polyline. A line between the two is constructed and the point with the largest perpendicular distance is selected. If the perpendicular distance between this point and the line is below a given threshold value, all vertices between the start and end point are removed. If the distance is larger than the threshold, the selected point is kept. The polyline is divided into two, one between the start point and the selected point, and one between the selected point and the end point. The algorithm is then performed recursively on these two new polylines.
The process can be seen in Figure 3.8. A higher threshold means allowing a larger error. This allows more vertices to be removed, but also reduces the positional accuracy. Selecting the threshold value is a balance between vertex count and quality.
3.2
Evaluation
No method was found which generates collision geometry from photos of hand-drawn video game levels. Therefore, the generated collision geometry was compared to handcrafted geometry. In order to do so, methods to measure the desired qualities are needed and need to be applicable to both generated geometry and handcrafted geometry alike.
20 Chapter 3. Method
(a) The initial polyline. (b) The point is above the threshold and is kept.
(c) The polyline from the start-ing point to the selected point.
(d) The point was below the threshold and was removed.
(e) The polyline from the se-lected point to the end point.
(f) The final result.
Figure 3.8: An example of line simplification using the Douglas-Peucker algorithm.
3.2.1
Measurements
The desired qualities that are measured are how well the geometry matches the shape of the input data and the vertex count. These are the main focus of the thesis.
Additionally, processing time and memory usage are also measured. These are only applicable when generating geometry, not when handcrafting geometry.
Shape
Hori and Doermann present a method for evaluating the performance of vectorization techniques [51]. A ground truth vector image is rasterized and perturbed, for example by introducing noise. It is then used as the source for the raster-to-vector technique to be evaluated. The output of the technique is then compared to the ground truth using a matching algorithm that measures the similarity between two vector images. The matching algorithm proposed by Hori and Doermann is intended for line images such as schematics and is not directly applicable to outlines. One part of the algorithm considers features such as beginning and end points and line crossings. Outlines are continuous and do not have beginnings or ends, nor should the outlines cross. However, whereas the matching algorithm is not applicable, the overall method of evaluating a perturbed ground truth is still applicable to outlines.
An overview of the process can be seen in Figure 3.9. Instead of starting with a ground truth in vector form, as Hori and Doermann, a raster image ground truth is used. It is then perturbed and collision geometry is created based on the perturbed image. The collision geometry is rasterized and the rasterized image is compared to the raster ground truth.
The ground truth images and the matching algorithm used are described in Sec-tion 3.2.3.
3.2. Evaluation 21 (a) (b) (c) (e) % (d)
Figure 3.9: Visual representation of collision geometry matching evaluation process. Ground truth (a) is perturbed to represent a photo of a drawing (b). Collision ge-ometry is generated for the perturbed image (c). The collision gege-ometry is rasterized (d) and then compared to the ground truth (e).
Vertex count
The vertex count for the collision geometry is measured by summing the vertex counts of all outlines in every layer.
Processing time
The processing time is measured by taking a timestamp at the beginning and end of the process and calculating the difference.
Memory usage
The Windows API is used in order to measure the memory usage. By calling GetProcessMemoryInfo, information about the memory usage of the current pro-cess can be queried. However, simply calling this function after the propro-cess has finished is not sufficient, as memory is both allocated and deallocated during the processing. Instead, a separate thread is created which runs parallel to the process-ing thread and continuously measures the current memory usage. The maximum amount of memory used during the entire process is stored and reported.
3.2.2
Performing measurements
The tests were run on a computer running 64-bit Windows 10, with a 3.2 GHz Intel Core i7 CPU and 12 GB RAM.
In order to get accurate results, the tests were performed five times. The tests were performed multiple times because the computer runs several processes at once. The process which executes the test might be competing with other processes. Thus, the performance of the program might be effected. Therefore, the measurements are performed five times in order to reduce variance. The memory is measured during a separate pass from the processing time, as not to affect the processing time measurement.
The deciding factor for the vertex count of the generated geometry is the threshold value used during vertex reduction. A higher vertex count allows for a more accurate
22 Chapter 3. Method approximation of the geometry. The threshold value is selected so that the gener-ated geometry would have, on average, a lower vertex count than the handcrafted geometry.
3.2.3
Experiment
In the experiment the measurements are taken and the generated geometry is com-pared to handcrafted geometry. The overall process can be seen in Figure 3.10. The shape is measured much like in Figure 3.9 with the exception that both handcrafted and generated geometry is created from the perturbed image and compared to the ground truth. (a) (b) (c1) (c2) (e) % (d1) (d2)
Figure 3.10: Visual representation of collision geometry matching evaluation pro-cess. Ground truth (a) is perturbed to represent a photo of a drawing (b). Collision geometry is traced manually (c1) and generated (c2) for the perturbed image. The collision geometry is rasterized (d1)(d2) and then compared to determine how well it matches the ground truth (e).
Ground truth
The ground truth images are raster images using only the colors in the given color palette. In a platformer level with platforms and dangerous areas, the ground truth image would consist of white, black and red. There is no blending between the colors; every pixel is either fully white, fully black or fully red. This means it can be compared directly against the rasterized geometry in the comparison step.
A data set was provided by Attractive Interactive AB containing photos of hand-drawn platformer game levels made by players. The data set was used as reference when creating ground truth images in order to make the ground truth representative of real levels made by players. The ground truth images are used in order to evaluate the quality of the generated and handcrafted geometry. The ground truth images can be seen in Appendix B.
Perturb
In the perturbation step multiple filters are applied to an image in order to make it look like a photo of a drawing. The data set provided by Attractive Interactive
3.2. Evaluation 23 AB was used as reference when designing the filters making the ground truth images look like photos of hand-drawn video game levels. An overview of the perturbation process can be seen in Figure 3.11. The steps are explained in more detail below.
A The source input image. One of the images in Appendix B.
B The color channels are created based on the source image. Background (white) pixels are replaced with black whereas all other pixels are left as they were.
~cB =
(
( 0 0 0 ), ~cA= ( 255 255 255 )
~cA, ~cA6= ( 255 255 255 )
C An alpha channel is created from the source image. Background (white) pixels are treated as fully transparent whereas all other pixels are fully opaque.
aC =
(
0, ~cA = ( 255 255 255 )
1, ~cA 6= ( 255 255 255 )
The alpha channel will be used later, in the blending step, and is used to determine which parts of the image are colored in using a marker and which parts of the image are uncolored paper.
D-E The color channels and the alpha channel are blurred using a Gaussian filter. The Gaussian blur implementation in OpenCV is used, with a Gaussian standard deviation of σ = 1.5. The OpenCV implementation calculates the kernel size auto-matically based on σ.
F-G The color channels and the alpha channel are downscaled using linear inter-polation. Applying a Gaussian blur and downscaling reduces aliasing in the image.
H A marker texture is loaded and tiled to fit the dimensions of the image. The marker texture was created by taking a photo of an area of a paper that has been filled in using a marker. By creating a different texture, for example using a crayon, different pens could be evaluated.
I The color channels are divided by the alpha channel.
~cI =
~cF
aG
This is done in order to allow the different colors to blend during the blurring and downscaling steps while avoiding mixing with the black background color. Without this step, the features in the finished image would have darkened borders.
24 Chapter 3. Method
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
(j)
(k)
(l)
Figure 3.11: Overview of the perturbation process. (a) Source image. (b) Color channels. (c) Alpha channel. (d) Gaussian blurred color channels. (e) Gaussian blurred alpha channel. (f) Downscaled color channels. (g) Downscaled alpha channel. (h) Marker texture. (i) Color channels divided by alpha channel. (j) Alpha channel multiplied with marker texture. (k) Paper texture. (l) Final blended result.
3.2. Evaluation 25 J The alpha channel is multiplied with the marker texture. During the blending step, L, this will allow the paper texture to show through the marker lines rather than them being a solid color on top of the paper texture.
aJ = aGm
K A paper texture is randomly selected. It is loaded and scaled to fit the dimensions of the image. It is mirrored horizontally and vertically at random.
L The final results are achieved by blending the paper texture with the color chan-nels according to the alpha channel.
~cL= aJ~cI+ (1 − aJ)~p
The perturbed ground truth images can be seen in Appendix C. Handcrafted collision geometry
The handcrafted collision geometry is created using the vector graphics design tool Inkscape [52]. Inkscape enables manual placement of vertices and supports loading raster images. The perturbed image being traced is placed as a background layer and is traced by placing vertices on top of it.
The handcrafted collision geometry can be biased by intentionally creating bad geometry, making the generated geometry appear better in comparison. In order to ensure that the handcrafted geometry is not biased, an external reviewer is asked to validate it. The reviewer has a bachelor’s degree as a technical artist and has experience working with game development. The reviewer is an employee at a game development company in Karlskrona. The company has released one game which the reviewer worked on. The reviewer is deemed experienced enough in game devel-opment in order to evaluate the collision geometry.
Each perturbed image is presented to the reviewer with an overlay visualizing the collision geometry. The reviewer is able to explore each image freely in the vector graphics design tool Inkscape, the same tool in which the geometry was created. The reviewer concluded that the handcrafted geometry is not biased.
Rasterize
The rasterization step takes an SVG vector image as input and ouputs a raster image. Each group, determined by a <g> tag, in the SVG file is treated as a layer and a single object. This object is filled according to the even-odd rule, as defined in the SVG specification [53]. To determine if a given point is inside or outside the object, a ray is constructed with the point as origin that extends infinitely in any direction. The number of path segments crossing this ray is counted. An odd number means the point is inside the object, whereas an even number means the point is outside. An example can be seen in Figure 3.12.
By performing an even-odd check for each pixel in the output image, the color of every pixel can be determined. However, performing such a check requires checking against all line segments in the image for every pixel. The time complexity is O(whn),
26 Chapter 3. Method
Figure 3.12: Visualization of the even-odd rule. Since the ray intersects an odd number of line segments, the point is determined to be inside the object.
where w is the width of the image, h is the height of the image and n is the number of line segments in the image.
This can be significantly improved by carefully selecting the direction of the ray. If a positive horizontal ray is selected, the ray for the first pixel in a row will pass through every other pixel on that row. By storing the results of the ray against line segments intersection test for the first pixel in a row, they can be reused for all other pixels in that row. Thus, all line segments only have to be iterated through once per row. This improves the time complexity from O(whn) to O(wh + hn).
The output from the rasterization step can be seen in Figure 3.13.
Figure 3.13: A rasterized version of the image from Figure 3.12.
Compare
In the comparison step, the rasterized vector image and the ground truth are com-pared against each other. The percentage of matching pixels is calculated. For this reason, it is important that the two images being compared have exactly the same dimensions. The ability to scale images during the rasterization step means this can always be ensured.
3.3
Implementation details
The collision geometry generation was implemented as a C++ library with a C ABI. A C ABI was chosen as it can be called from many other programming languages,
3.3. Implementation details 27 making the library more widely accessible. The full source code is available online under the MIT license [54].
Chapter 4
Results
The generated geometry using the Quality configuration can be seen in Appendix D. Both configurations were used in the tests. The Quality configuration is presented as ‘Generated (quality)’ in the tables, whereas the Time configuration is presented as ‘Generated (time)’.
Seven perturbed images were used, all with the dimensions 1280×720 pixels: Al-bertsEasyBana, BoingBoing, Candysky, Flippersgame, Grimstar, Hus and SaltySym-metry. The peturbed images can be seen in Appendix C.
4.1
Shape
The shape comparison results for handcrafted and generated geometry are presented in Figure 4.1. Geometry generated using the quality-oriented configuration yields the lowest error rate compared to the other methods of creating geometry.
AlbertsEasyBanaBoingBoing CandyskyFlippersgame
Grimstar Hus SaltySymmetry Picture 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 Error (%) Handcrafted Generated(quality) Generated(time)
Figure 4.1: The percentage of mismatched pixels between the rasterized geome-try and the ground truth (lower is better). Geomegeome-try generated using the quality-oriented configuration yields the lowest error rate compared to the other methods of creating geometry. Values can be seen in Table A.1.
30 Chapter 4. Results
4.2
Vertex count
The vertex count for the handcrafted and generated geometry can be seen in Fig-ure 4.2. The number of vertices of the generated geometry is, in most cases, lower than the vertex count of the handcrafted geometry.
AlbertsEasyBanaBoingBoing CandyskyFlippersgame
Grimstar Hus SaltySymmetry Picture 0 500 1000 1500 2000 Vertex count Handcrafted Generated(quality) Generated(time)
Figure 4.2: The number of vertices in the created geometry (lower is better). The number of vertices of the generated geometry is, in most cases, lower than the vertex count of the handcrafted geometry. Values can be seen in Table A.2.
4.3
Processing time
The total time taken to generate geometry for the different configurations can be seen in Figure 4.3. The time-oriented configuration is faster than the quality-oriented configuration in all cases.
The processing time of the individual stages can be seen in Figure 4.4 for the quality-oriented configuration and in Figure 4.5 for the time-oriented configuration.
4.4
Memory usage
The maximum amount of memory used during the generation process for the different configurations can be seen in Figure 4.6. Both methods of generating geometry use the same amount of memory.
The memory usage of the individual stages can be seen in Figure 4.7 for the quality-oriented configuration and in Figure 4.8 for the time-oriented configuration.
4.4. Memory usage 31
AlbertsEasyBanaBoingBoing CandyskyFlippersgame
Grimstar Hus SaltySymmetry Picture 0 500 1000 1500 2000 2500 Processing time (ms) Generated(quality) Generated(time)
Figure 4.3: Total processing time of the different configurations (lower is better). The time-oriented configuration is faster than the quality-oriented configuration in all cases. Values can be seen in Table A.3.
AlbertsEasyBanaBoingBoing CandyskyFlippersgame
Grimstar Hus SaltySymmetry Picture 0 200 400 600 800 1000 1200 1400 1600 Processing time (ms) Preprocessing Quantization Tracing Vertex reduction
Figure 4.4: Processing time of the different steps using the quality-oriented config-uration (lower is better). The quantization step takes the longest time to processes compared to the other steps. Values can be seen in Table A.4.
32 Chapter 4. Results
AlbertsEasyBanaBoingBoing CandyskyFlippersgame
Grimstar Hus SaltySymmetry Picture 0 20 40 60 80 100 120 Processing time (ms) Quantization Tracing Vertex reduction
Figure 4.5: Processing time of the different steps using the time-oriented configu-ration (lower is better). The preprocessing step is not performed when using the time-oriented configuration. The vertex reduction step takes the longest time to processes compared to the other steps using the time-oriented configuration. Values can be seen in Table A.5.
AlbertsEasyBanaBoingBoing CandyskyFlippersgame
Grimstar Hus SaltySymmetry Picture 0 2 4 6 8 10
Memory usage (MiB)
Generated(quality) Generated(time)
Figure 4.6: The maximum amount of memory used during processing (lower is bet-ter). Both methods of generating geometry use the same amount of memory. Values can be seen in Table A.6.
4.4. Memory usage 33
AlbertsEasyBanaBoingBoing CandyskyFlippersgame
Grimstar Hus SaltySymmetry Picture 0 2 4 6 8 10
Memory usage (MiB)
Preprocessing Quantization Tracing Vertex reduction
Figure 4.7: The maximum amount of memory used during the different steps using the quality-oriented configuration (lower is better). The tracing step uses the most memory compared to the other steps. Values can be seen in Table A.7.
AlbertsEasyBanaBoingBoing CandyskyFlippersgame
Grimstar Hus SaltySymmetry Picture 0 2 4 6 8 10
Memory usage (MiB)
Quantization Tracing Vertex reduction
Figure 4.8: The maximum amount of memory used during the different steps using time-oriented configuration (lower is better). The tracing step uses the most memory compared to the other steps. Values can be seen in Table A.8.
Chapter 5
Analysis and discussion
In all of the images used, seen in Table A.1, the generated geometry using the configuration focused on quality has a lower error rate than the configuration focused on processing time. Both configurations have lower error rates than the handcrafted geometry in all images.
The probability that the error rate of generated geometry is higher than the error rate of handcrafted geometry is calculated in order to determine whether generated geometry can replace handcrafted geometry.
5.1
Method of quality analysis
The mean error rate for each method of creating geometry is calculated using µ = 1 N N X i=0 xi
where µ is the mean, xiis the value of the ithsample and N is the number of samples.
The sample standard deviation σ is calculated using
σ = v u u t 1 N − 1 N X i=1 (µ − xi)2
where µ is the mean of the samples, xi is the value of the ith sample and N is the
number of samples. The probability that the error rate of generated geometry is higher than the error rate of handcrafted geometry is calculated using
P (G > H) = P (G − H > 0) = 1 − P (G − H < 0)
where G is the population of error rates for generated geometry with the normal distribution G ∼ N(µG, σG2) and H is the population of error rates for handcrafted
geometry with the normal distribution H ∼ N(µH, σ2H). G − H is normally
dis-tributed with mean
µ = µG− µH and variance σ2 = σ2G+ σ2H. Hence, P (G > H) = 1 − P G − H − µ σ < 0 − µ σ = 1 − Φ −µ σ where Φ is the distribution function of the standard normal distribution.
36 Chapter 5. Analysis and discussion
5.2
Analysis of quality results
The mean and standard deviation for each method of creating geometry are presented in Table 5.1. The probability that the error rate of the generated geometry is higher than the error rate of the handcrafted geometry is presented in Table 5.2. The probability density functions of the error rate for generated geometry and handcrafted geometry are presented in Figure 5.1.
There is a 3.53 percent probability that the generated geometry using the quality-oriented configuration has a higher error rate than handcrafted geometry. A statis-tically significant difference is observed, therefore a conclusion can be made: the generated geometry using the quality-oriented configuration matches the shape of the geometry in the photo better than the handcrafted geometry (p < 0.05). Thus, the first criterion of the thesis question is fulfilled.
No conclusions can be made whether the generated geometry using the time-oriented configuration matches the shape of the geometry in the photo better than the handcrafted geometry as the difference is not statistically significant (p > 0.05).
Handcrafted Generated (quality) Generated (time)
Mean 0.5798 % 0.4028 % 0.4978 %
Standard deviation 0.0711 % 0.0673 % 0.0610 %
Table 5.1: The mean error rate and standard deviation of the handcrafted geometry and different configurations (lower is better). The mean error of the handcrafted geometry is higher than the generated geometry. The values are calculated using the results presented in Table A.1.
Generated (quality) Generated (time)
Φ 3.53 % 19.06 %
Table 5.2: Φ is the probability that the error rate of the generated geometry is higher than the error rate of the handcrafted geometry (lower is better). The values are calculated using the results presented in Table 5.1.
The average vertex count for the different configurations and the handcrafted geometry can be seen in Table 5.3.
The generated geometry has on average a lower vertex count compared to hand-crafted geometry. Therefore, the second criteria of the thesis question is fulfilled: the generated geometry has a lower vertex count than the handcrafted geometry.
Handcrafted Generated (quality) Generated (time)
Average vertex count 1334 1139 1143
Table 5.3: The average vertex count for the handcrafted geometry and different configurations (lower is better). The generated geometry has on average a lower vertex count than the handcrafted geometry. The values are calculated using the results presented in Table A.2.
5.3. Analysis of performance results 37 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Error (%) 0 1 2 3 4 5 6 7 8 9 Density of probability Generated (quality) Handcrafted
(a) Error rate for generated (quality) and hand-crafted geometry. 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Error (%) 0 1 2 3 4 5 6 7 8 9 Density of probability Generated (time) Handcrafted
(b) Error rate for generated (time) and hand-crafted geometry.
Figure 5.1: The probability density functions for generated geometry and hand-crafted geometry (lower is better). There is a higher probability that the generated geometry has a lower error rate compared to handcrafted geometry. (a) illustrates the probability of error rate for handcrafted geometry and generated geometry us-ing the quality-oriented configuration. (b) illustrates the probability of error rate for handcrafted geometry and generated geometry using the time-oriented configuration.
5.3
Analysis of performance results
The total processing time differs significantly between the quality-oriented config-uration and the time-oriented configconfig-uration, as seen in Table A.3. By comparing the processing time for each step in Table A.4 and Table A.5, it is evident that the processing time of the quantization step is significantly lower for the time-oriented configuration. Additionally no preprocessing is performed for the time-oriented con-figuration, further reducing its total processing time.
Both configurations use about the same amount of memory, around 10 MiB, as seen in Table A.6. By comparing the maximum memory usage for each step in Table A.7 and Table A.8, it is evident that the tracing step uses the most memory. Both configurations use the same tracing method. Thus, the maximum memory used is analogous for both configurations.
5.4
Validity threats
A matching algorithm was developed in order to evaluate the quality of the generated geometry compared to the quality of the handcrafted geometry. An advantage using a matching algorithm is that the experiment is scalable and the error rate is quan-tifiable. However, a disadvantage is that the error rate calculated by the matching algorithm might not match human perception. Another evaluation method was dis-cussed where a survey was going to be used instead of a matching algorithm, asking participants whether they preferred generated geometry or handcrafted geometry. However, the evaluation method using a survey was not favored because participants had to be found and the matching algorithm was believed to give more persistent
38 Chapter 5. Analysis and discussion and accurate results than that of a survey with few participants.
The data set provided by Attractive Interactive AB contains photos of hand-drawn platformer game levels where red and black felt tip pens/markers are used on white paper. The game for which the levels were created uses black for platforms on which the player can stand and red for dangerous objects which the player must avoid. Although the data set contains different drawing-styles, the evaluation of the method is limited to the combination of pen and paper used in the data set. Preferably, a diverse type of reference photos should have been used with video game levels drawn using different types of paper and markers.
The data set provided by Attractive Interactive AB was used as reference when designing the filters making the ground images look like photos of hand-drawn video game levels. The filters introduced noise and light discontinuities found in the data set. However, all combinations of noise-types and light discontinuities can not be recreated. No validation whether the processed images looked hand-drawn was made. However, they are presented in Appendix C.
The handcrafted geometry could be biased by intentionally creating bad geometry, making the generated geometry appear better in comparison. To mitigate this, an external reviewer was asked to determine whether the handcrafted geometry was biased.
5.5
Sustainability and ethics
Sustainability can be divided into environmental sustainability, concerning the use of natural resources, social sustainability, concerning the development of people and cultures, and economic sustainability, concerning equal economic growth.
No relevant issues regarding environmental and social sustainability were identi-fied. In regards to economic sustainability, only free and open source libraries were used and the source code for the implementation was made publicly available. This makes the method more widely accessible.
No relevant ethical concerns were identified. Since no user study was performed, there were no issues relating to privacy or potential harm to test subjects.
