INOM
EXAMENSARBETE TEKNIK, GRUNDNIVÅ, 15 HP
STOCKHOLM SVERIGE 2019 ,
Investigating How Impostor
Techniques are Used to Optimize the Performance of Video Games
TOR DAGSSON SUNDMAN JOEL THUNANDER
KTH
SKOLAN FÖR ELEKTROTEKNIK OCH DATAVETENSKAP
Investigating How Impostor Techniques are Used to
Optimize the Performance of Video Games
TOR DAGSSON SUNDMAN JOEL THUNANDER
Degree Project in Computer Science, DD142X Date: June 18, 2019
Supervisor: Christopher Peters Examiner: Pawel Herman
School of Electrical Engineering and Computer Science
Swedish title: Studie av hur impostor techniques används för att
optimera prestandan av spel
iii
Abstract
The demand for visual quality in video games is higher than ever. Video games approximate real objects via the tessellation of triangular polygons.
Complex objects with a lot of detail require denser tessellations in order to
be represented properly. However denser tessellations require more perfor-
mance which can prevent the game from running properly in real time. Thus
arises the problem of reducing the complexity of 3D objects in games while
simultaneously preserving visual quality. An impostor technique approaches
this problem by reducing the rendering cost. This project constructs a game
engine and applies two existing impostor techniques: NP3D impostors and
2D impostors. A third impostor techniques is introduced where 3D objects
are combined with 2D impostors and NP3D impostors. These techniques are
then evaluated in terms of their effect on visual fidelity and performance. 2D
impostors and NP3D impostors are revealed to be equally good at preserv-
ing visual fidelity but have different effects on performance. Using only 2D
impostors is revealed to reduce CPU load. Using only NP3D impostors is re-
vealed to reduce GPU load. The third technique is revealed to be completely
ineffective.
iv
Sammanfattning
Det ställs idag högre krav än någonsin på den visuella kvaliteten i tv-spel. Rik- tiga objekt approximeras i tv-spel med en tessellation av triangulära polygoner.
Komplexa objekt med mycket detalj kräver en tätare tessellation för att kun- na representeras korrekt. En tätare tessellation är däremot mer resurskrävande och detta kan hindra spelet från att köra ordentligt i realtid. Därmed uppstår problemet att reducera komplexiteten av objektet utan att försämra den visuella kvaliteten för mycket. Ett tillvägagångssätt är impostor-tekniker som minskar kostnaden av att rendera 3D objekt. Detta projekt konstruerar en spelmotor (en integrerad utvecklingsmiljö utvecklad för att skapa spel) och tillämpar två existerande impostor tekniker: NP3D impostors och 2D impostors. En tred- je impostor-teknik introduceras även där 3D object kombineras med 2D im- postors och NP3D impostors. Dessa tekniker evalueras därefter baserad på hur de påverkar visuell kvalitet och prestanda. 2D impostors och NP3D impostors visar sig vara lika bra på att bevara visuell kvalitet men påverkar prestandan olika. Att endast använda 2D impostors visar sig reducera CPU användning.
Att endast använda NP3D impostors reducerar GPU användning. Den tredje
tekniken visar sig vara fullständigt ineffektiv.
Contents
1 Introduction 1
1.1 Research Question . . . . 2
1.2 Hypothesis . . . . 2
1.3 Scope and Delimitations . . . . 2
2 Background 3 2.1 Hardware . . . . 3
2.1.1 Central Processing Unit (CPU) . . . . 3
2.1.2 Graphics Processing Unit (GPU) . . . . 3
2.2 Computer graphics . . . . 3
2.2.1 Rasterization . . . . 3
2.2.2 Polygons . . . . 4
2.2.3 World Space, Object Space, and View Space . . . . . 5
2.2.4 Depth Buffer . . . . 6
2.2.5 View Frustum . . . . 7
2.3 Impostor Techniques . . . . 7
2.3.1 Sprite Maps . . . . 7
2.3.2 Texture Resolution . . . . 7
2.3.3 Changes in Camera Angle . . . . 7
2.3.4 Visual Popping . . . . 8
2.3.5 2D Impostors . . . . 8
2.4 OpenGL . . . . 8
2.4.1 Framebuffer Object (FBO) . . . . 8
2.4.2 Vertex Buffer Object (VBO) . . . . 8
2.4.3 Vertex Array Object (VAO) . . . . 9
2.4.4 Mip Maps . . . . 9
2.5 Measuring Performance . . . . 9
2.5.1 Frame Time . . . . 9
2.5.2 CPU Load . . . . 10
v
vi CONTENTS
2.5.3 GPU Load . . . . 10
2.6 Related Work . . . . 10
2.6.1 Non-Planar 3D (NP3D) Impostors . . . . 10
2.6.2 Combining 3D Objects with Impostors . . . . 10
3 Method 11 3.1 Constructing a Game Engine in OpenGL . . . . 11
3.1.1 Introducing a Near Sector, Middle Sector, and Far Sector 11 3.1.2 Generating Sprite Maps . . . . 11
3.1.3 Generating Bounding Boxes . . . . 13
3.1.4 Generating Impostor Sprites . . . . 14
3.1.5 Selecting the Appropriate Sprite . . . . 14
3.1.6 Rendering the Sprite to the Canvas . . . . 16
3.1.7 Creating NP3D Impostors . . . . 17
3.2 Tests . . . . 17
4 Results 20 4.1 Using Only 3D Objects . . . . 20
4.2 Using Only 2D Impostors . . . . 20
4.3 Using Only NP3D Impostors . . . . 21
4.4 Using the Combined Technique . . . . 21
4.5 Using Only 2D Impostors and Varrying the Number of Sprites 22 4.6 Distilled Information . . . . 22
5 Discussion 25 5.1 Using the Combined Technique . . . . 25
5.2 Using Only NP3D Impostors . . . . 26
5.2.1 Lighting with NP3D . . . . 26
5.2.2 Sprite Maps . . . . 26
5.2.3 Multiple Sectors . . . . 26
5.3 Using Only 2D Impostors . . . . 27
5.3.1 Known Directions . . . . 27
5.3.2 Locating Sprites . . . . 28
6 Conclusion 29
Bibliography 30
Chapter 1 Introduction
Most popular video games are comprised of interactive 3D environments sim- ulated in real-time. The demand for rich visual detail and high realism of these environments has increased rapidly over the past years and continues to do so.
Meeting these demands while maintaining good performance is difficult, since the objects in the environment can be very complex and exist in great numbers.
The difficulty lies in preserving the visual detail in the environment while si- multaneously keeping the performance under control so that the game can run in real-time. 3D environments and the objects that exist in them are made up of polygons and for a 3D object to have a lot of detail, it must be composed of many polygons. An increased number of polygons subsequently increases the performance overhead. Thus arises the problem of reducing the complexity of 3D objects in games while simultaneously preserving their visual quality.
There are a number of approaches to optimize the performance of a real- time simulation. One way to improve performance is to reduce the total num- ber of polygons present in the 3D environment. This is done via a method known as Level of Detail (LOD) which reduces the complexity of a 3D ob- ject. LOD can improve performance in terms of frames per second, however it can also lead to greater memory usage as well as negatively impact the vi- sual detail of the object by producing unwanted artefacts (Ulicny et al. 2004).
A popular LOD approach is the impostor technique. This technique centers around replacing 3D objects with simple 2D images of themselves, thereby significantly reducing geometrical complexity while preserving visual detail.
1
2 CHAPTER 1. INTRODUCTION
1.1 Research Question
In the context of real-time interactive 3D simulations, how do different impos- tor techniques compare in their ability to optimize performance while simul- taneously preserving visual fidelity?
1.2 Hypothesis
According to research done by the work of Kavan [1], combining impostors with real objects can make it appear like an environment has many high vertex count models present. Normally this would be done by having a scene where you first have a layer with real models and then after that you use imposters.A problem that arises is that you have to base your impostors on looking good enough that you will not notice the closest imposters being impostors. We think that by implementing three layers,where the first layer uses models the second layer uses a higher quality impostor and the third uses a worse impostor could save a lot of compute power. Because the characters in the back do not have to have the same quality as the ones in the middle.
1.3 Scope and Delimitations
The project intends to investigate existing methods to optimize the perfor- mance of real-time simulations based on the impostor technique.
This project does not conduct an empirical evaluation of visual fidelity via a user study. It should be noted that the visual quality is judged by the authors themselves. It is outside the scope of the project to conduct an empirical eval- uation of the visual fidelity via a user study. The visual quality of impostors is judged by their resemblance to their respective 3D objects. The resemblance is judged by looking at the lighting and perceived depth of the impostor. These aspects were chosen because 2D impostors completely remove depth from the object, and depth also affects shading.
Due to resource limitations this project does not factor in any animation in
the impostor techniques.
Chapter 2 Background
2.1 Hardware
2.1.1 Central Processing Unit (CPU)
The CPU is responsible for executing the instructions of a computer program.
In a game, it performs calculations related to simulating the environment, such as physics simulations and matrix transformations.
2.1.2 Graphics Processing Unit (GPU)
The GPU is a specialized processor designed for rendering images. The CPU uses the GPU to render the images of the environment that are displayed on the computer screen.
2.2 Computer graphics
2.2.1 Rasterization
Images can be stored in various formats, however before an image can be dis- played on a computer screen it must first be rasterized. Rasterization is the process of converting the view space of the environment to a rasterized image that can be displayed on the screen. A rasterized image is essentially a grid of pixels.
3
4 CHAPTER 2. BACKGROUND
Figure 2.1: The pixels on the screen are colored to match the lines and curves on the screen.
2.2.2 Polygons
In computer graphics, 3D objects are represented with a tessellation of trian- gular polygons forming a mesh. This representation is a way of approximating the shape of real objects. Triangles are used because they are the simplest poly- gon, having only three vertices, and any arrangement of three vertices in 3D space will always be coplanar. This makes triangles the only 2D polygon that can be used to approximate a 3D object. Furthermore, any 3D surface can be approximated via the tessellation of triangles. The quality of the approxima- tion depends on the density of the tessellation; a denser tessellation results in a better approximation but also requires more polygons. This is exemplified in figures 2.2 and 2.3. Simpler objects such as spheres and cylinders require relatively few polygons. However more complex geometries such as human bodies can require millions of polygons to create a good approximation. Cre- ating good approximations is especially important in games that use visual quality to attract customers.
For rendering purposes (such as during rasterization), it is important to differentiate between the two faces of a triangle (rarely both faces of a polygon have to be rendered). This makes it possible to specify that only the outside of a 3D mesh should be rendered but not the inside which helps save computational resources.
To move a 3D object the belonging polygons have to be scaled, trans- formed, and rotated in different ways. This means transformation matrices must be applied to each and every vector that needs to be transformed. The polygons must then additionally be transformed from the local space of the 3D object to a 2D representation in the view space on the screen. These opera- tions must all be done several tens of times per second, therefore an increased number of polygons in a 3D environment results in a greater performance cost.
This process can become a problem when a very large number of com-
plex objects are simultaneously present in an environment, since the perfor-
mance cost increases with the total polygon count. As the performance cost
approaches the limit of what the hardware can handle, the game begins to slow
CHAPTER 2. BACKGROUND 5
Figure 2.2: A denser tessel- lation creates a smoother 3D mesh with good resemblance to a real human head.
Figure 2.3: A sparser tessella- tion creates a rougher 3D mesh with poor resemblance to a real human head.
down. Game developers must therefore find ways to decrease the total polygon count without significantly lowering the visual quality.
2.2.3 World Space, Object Space, and View Space
The world space, object space, and view space are tree Euclidean spaces in which the positioning of all the elements in a 3D environment is specified.
The object space is local to a single object and holds all the coordinates that
define its vertices. The world space relates to the entire 3D environment that
contains all the objects. The objects are placed in the environment by moving
them from their local space into world space. The world space is observed
6 CHAPTER 2. BACKGROUND
through a camera, and the part of the environment that can be seen through the camera is known as the view space.
2.2.4 Depth Buffer
Usually there is more than one object occupying the screen at the same time, several of which are overlapping each other in different ways. A rendering application programming interface (API) can draw each object triangle-by- triangle, but this will overwrite the pixels that were drawn for the previous triangle and can lead to visual artifacts. An example of this is shown in figure 2.4. Therefore it is important to consider the correct order of the overlaps in order to avoid such visual artifacts when drawing the objects on the screen.
The solution is to use a depth buffer for each object to record the depth in- formation of the corresponding pixels that are within the view space. When multiple objects are competing over the same pixel on the screen, the object with the nearest depth for that pixel will be drawn.
Figure 2.4: A denser tessel- lation creates a smoother 3D mesh with good resemblance to a real human head.
Figure 2.5: A sparser tessella-
tion creates a rougher 3D mesh
with poor resemblance to a real
human head.
CHAPTER 2. BACKGROUND 7
2.2.5 View Frustum
The view frustum is a region that determines how 3D objects are transformed from world space to pixels on the screen. The view frustum is bounded by a front- and back clipping plane. Any objects (or parts of objects) outside the front- and back clipping plane are not rendered.
2.3 Impostor Techniques
Impostor techniques aim to reduce the performance-overhead of real-time ren- dering while preserving as much visual detail as possible. It revolves around replacing a 3D object with a different representation, termed “the impostor”, that has less complexity but still resembles the original object as much as pos- sible. Resemblance is mainly achieved by texturing the impostor with an image of the original 3D object.
2.3.1 Sprite Maps
The images used to texture the impostor are referred to as sprites and can be pre-generated or rendered dynamically. Pre-generated impostors involve gen- erating images of the object from many different angles and organizing them on a single image referred to as a sprite map. A sprite is later selected from the sprite map which best represents what the corresponding 3D object would look like from the current viewing angle. The disadvantage of pre-generated impostors is a higher usage of texture memory, since the sprite maps must be stored in memory during the simulation.
2.3.2 Texture Resolution
The texture resolution of a sprite indicates the number of pixels it consists of.
Using sprites with higher texture resolution produces impostors with higher visual quality but also consumes more texture memory.
2.3.3 Changes in Camera Angle
Since a sprite is only visually representative of an object from a particular an- gle, the visual error increases when the viewing angle of the impostor changes.
If the change is too substantial, a new sprite must be selected from the sprite
map that better represents the object.
8 CHAPTER 2. BACKGROUND
2.3.4 Visual Popping
Switching to a different sprite can be visually noticeable if the difference be- tween to adjacent sprites is too great. This can be counteracted by including more sprites on the sprite map to allow for more subtle transitions. However, more texture memory will be consumed.
Visual popping can also arise if a 3D object transitions to an impostor while observed by the camera. It can look like the impostor “pops” into the position of the 3D object, or that the 3D object, conversely, “pops” into the position of the impostor. Visual popping is problematic as it can have a negative impact on immersion and perceived realism.
2.3.5 2D Impostors
2D impostors is a concept that lies within the impostor technique and revolves around replacing a 3D object in the environment with a textured quadrilateral.
The texture is a sprite selected from a pre-generated sprite map. Impostors have an advantage in terms of performance since the complexity of a object is reduced to as little as two polygons.
2.4 OpenGL
OpenGL is an API for developing 2D and 3D graphics applications. The game engine in this project is created in OpenGL.
2.4.1 Framebuffer Object (FBO)
A framebuffer object is one of the two kinds of framebuffers in OpenGL (along- side the default framebuffer). A framebuffer is a collection of buffers (such as depth and color buffers) that can be selected as a rendering destination. A FBO provides users with the option to define their own framebuffers. There are two kinds of framebuffers in OpenGL. One is the default frame buffer, the other is the framebuffer object. The framebuffer object is a collection of buffers (such as depth and color buffers) that can be selected as a rendering destination. A FBO provides users with the option to define their own framebuffers.
2.4.2 Vertex Buffer Object (VBO)
Sending data from the CPU to the GPU is a relatively slow process. Vertex
buffer objects are used to tackle this problem by allowing a large amount of
CHAPTER 2. BACKGROUND 9
vertices to be sent and stored at once in the GPU.
2.4.3 Vertex Array Object (VAO)
A vertex array object is a recipe for what is contained inside the VBO, so that when the GPU reads from the VBO it knows what it is supposed to read. For example, if you are going to send in a 3D vector and a float into the vertex shader you first let the VAO know the size and contents of the vector followed by the size of the float. In that way the GPU knows that it is first going to get a vector and then a float and can read that information.
2.4.4 Mip Maps
Mip mapping is the idea of creating smaller versions of textures so that as the impostors become smaller and smaller (further away from the camera) the texture changes to one that fits the object. This is done because otherwise the API will do it on its own, often leading to visual artifacts.
2.5 Measuring Performance
The performance of a game is measured by analyzing how the hardware is being used. CPU load and GPU load are selected as two measurements since the bottleneck for the possible total polygon count is determined by the CPU or GPU. The reason is that there is a limit to how actions that can the processed by the CPU every second. Likewise there is a limit to the number of polygons that can be rendered fast enough by the GPU.
2.5.1 Frame Time
Frametime is a good way to measure how well a game is running since it shows
how long it takes to render one frame. This is used instead of frames per
second (FPS) because frametime grows linearly while FPS does not. If one
millisecond of the rendering is cut then that is an absolute fact, while if there
are three more frames per second it is very different based on what the previous
frame rate was.
10 CHAPTER 2. BACKGROUND
2.5.2 CPU Load
CPU load is measured in percent and indicates how much of the CPU’s capac- ity is currently being used. There is no support for multi-threading in OpenGL meaning that it can only utilize a single core on a multi-core processor. The CPU load for an Intel i7 processor is 17
2.5.3 GPU Load
This measured in percent and indicates how much of the GPUs capacity is currently being used.
2.6 Related Work
2.6.1 Non-Planar 3D (NP3D) Impostors
NP3D impostors is an LOD method presented by Yee and Davis [2] that works by texture-mapping an image of the 3D object onto a simplified version of the original 3D mesh. Since the image is not texture-mapped onto a simple quadri- lateral, the impostor can be viewed from more angles than just the camera- angle from which the image was derived. References? The results from using NP3D were a reduced amount of texture memory and a decrease in popping when using the same amount of sprite in the sprite map
2.6.2 Combining 3D Objects with Impostors
Previously Rachel McDonnell and O’Sullivan [3] have attempted to combine
3D objects with an impostor technique. The results were a compromise in
performance and quality, where the quality fell off a bit but at an increase in
performance.
Chapter 3 Method
A 3D game is constructed in OpenGL which is then used to run a series of tests. The test scenario consists of a large number of static objects in world space lined up next to each other in a pattern resembling a rectangle. The camera is positioned in the environment so that all the objects are observable.
A varying number of copies of a 3D object are lined up and a series of tests are run to evaluate and compare the performance of the game engine when applying different impostor techniques.
3.1 Constructing a Game Engine in OpenGL
3.1.1 Introducing a Near Sector, Middle Sector, and Far Sector
The approach of Christiansen [4] was followed to generate the sprite maps.
3.1.2 Generating Sprite Maps
One of the biggest problems with making the sprite map was that a naively implemented sprite map wasted a lot of space. This is because generating images from the perspective directly above an object will lead to images like those shown in figure 3.5. To tackle this issue an algorithm was devised (see figure 3.3) that placed the images so that there was a high priority on getting many samples when looking straight at the character while getting less and less sprites the higher or lower it goes, ending when looking straight below or straight up at the character where there only is one sprite.
11
12 CHAPTER 3. METHOD
Figure 3.1: Sample-images indicating how a naively implemented sprite map can waste memory. Taking samples of the model from different angles as indicated by the bottom images leads to redundancy.
Figure 3.2: Angles from which sprites are taken.
When making the spritemap one takes the object and places it in the object space at origin. Then put a unit sphere centered on the origin and traversing the circumference of different angles and take photos of the object (as indicated in figure 3.2).
Figure 3.3: Pseudocode for calculating sprite angles.
The first thing done to generate the images is to calculate the angles at which the “photos” needed to be taken. There has to be at least three different angles (figure 3.2) to take it from, if it is chosen that we take three angles, photos are taken from the top, one from the bottom and one from the side.
It is then decided how many photos should be taken from the “equator line”
which is the middle most photo that will always be taken from the 90 degree
CHAPTER 3. METHOD 13
angle. When we have calculated the number of photos that will be taken from the equator line photos from other angles are generated using sin(angle) ∗ numberof photosf romequatorline. It is important to remember that at the top and bottom (sin(0) and sin(180)) one will get 0. To counteract that add 1 image in the case where the number of photos is 0. Now that the number of photos and the angle are calculated, the direction and angle of each photo that is going to be taken can be stored and sent to the impostor generator.
Figure 3.4: Sprites are taken from the equa- tor line. They are also taken from above and below with an equal distance. Less sprites are taken directly above and directly below.
3.1.3 Generating Bounding Boxes
When capturing the impostor from the object two problems arose, the first is how to know what part of the screen to capture, the second is what is trans- parent and what is not. To find a solution to the first problem, a bounding box is created for the object. The bounding box is made during the reading of the object from storage, that way a min and max xyz is available to us which cre- ates a box around the object. To know what part of the screen to capture is to make sure that the entire bounding box is inside of the capture square (figure 3.5) as seen from the red dots. The green square is to show the impostor cap- ture zone. Since the new bounding square is dependant on where the camera is looking the bounding box coordinates is multiplied with the matrix gener- ated from looking at the object and finally multiplied with the direction vector generated in the earlier step to get its proper coordinates. With the new bound- ing square we know exactly what should be captured. To solve the problem with transparency one must be able to decide what is void and what is color.
When showing the object in the impostor generator stage the background is
cleared to be black (0,0,0,0), but when then read from the fragment shader the
fragment shader will not know the difference between black or void. This is
solved by storing and sending in a depth buffer or Z buffer. The depth buffer
will store what depth each pixel is at and as such will be able to tell whether
something is in the void because then the Z buffer will store a -1 (uninitialized)
14 CHAPTER 3. METHOD
at that index. It should be noted that this will cause the sprites themselves to be stretched (figure 3.6) since there is no way to store different sprite sizes, but when the image is put on top of the smaller square (impostor canvas) it will look normal.
Figure 3.5: left bounding box of model, right bounding box of sprite, red dots are min max xyz.
Figure 3.6: All sprites are the same size, but stretched out.
3.1.4 Generating Impostor Sprites
Now that we have the box ready and now what angles to take each sprite photo from it is time to actually generate the sprites we do this by saving the position of the box in world space. After that we bind a framebuffer object so that we have the information in our gpu memory when we have rendered the image to it. We then send the bounding box information into our impostor generator shader, which in turn will set the position of the vertices for the entire screen, basically filling the entire screen with what we want to see by making two triangles that fill the entire screen. We then send that information over to the fragment shader that draws the object onto the fbo, this needs to be done for each sprite and saved so that we can use an index to seperate them.
3.1.5 Selecting the Appropriate Sprite
Now that the frame buffers are ready with different sprites the next step is to
render sprites on the screen. The first task is to decide how many impostors we
CHAPTER 3. METHOD 15
need. In most cases impostors are drawn in the thousands if not tens of thou- sands, because of this it is important to decide how many impostors are going to be drawn because otherwise the array will have to regenerate itself when it reaches its maximum(std::vector in c++) and allocate double the memory of its previous size. Without deciding the size both the memory and a proces- sor will take longer time. For every impostor the impostor sprite to use must be calculated, this is done by implementing a design from Risser [5], first the camera direction needs to be changed so that it is looking as if it were in object space. This is done by taking the view and object matrix and multiplying them, inversing it will get the camera matrix to be in object space. Multiplying the matrix with a 4D vector will get the last row and as such the translation of the camera, also known as the camera direction. The camera is now looking at the impostor as if it were in object space and needs to normalize so that it can be compare to how the vector looked when the camera was looking at the object during the creation of the impostor sprite. Comparing the cameras direction matrix with the different direction vectors of the sprites is done by getting the dot product of every sprites direction vector against the current camera direc- tion. The closer to 1 the result is, the more similar the two views are to each other, because the dot product is the cosine of the angle, so theoretically if we get a 1 it is the exact same direction. This task is the most intensive part of program and could be optimized.
Figure 3.7: Two triangles made from geom-
etry shader are followed and reconstructed.
16 CHAPTER 3. METHOD
Figure 3.8: The left image of shows the impostors from far away. The different sprites being used can be seen by the differently colored areas. The right image shows the impostors up close.
3.1.6 Rendering the Sprite to the Canvas
To render the impostors the approach of Milan [6] was used. Now that the correct sprite has been found the final task is to draw it to the canvas, in order to do that the first thing that needs to be done is to get the original positioning of the canvas during the impostor generation, once the values have been gained multiplying that with the object matrix of the impostor for all four points(the corners) of the canvas and the canvas points will be in correct rotation and translation. With the values of the canvas calculated it is now possible to send the vertex points and the framebuffer object into the impostor vertex shader.
The vertex shader has little to do in this stage since there are no vertices, only
the four points of the canvas have been generated. with no edges and only
vertex points one can generate proper vertices with edges with a geometry
shader. Routing the canvas point data to the geometry shader will take the
four points generated for the canvas and put them out into the world (figure
3.7), then make two triangles that will result in the canvas and connect the
texture coordinates so that the fragment shader knows where to start and end
the texture as well as which sprite it is going to render. With all that data
calculated one only needs to send the data to the fragment shader so that it
can draw the vertices asked for and using normal fragment shader code, and
the impostors will show up on the screen (figure 3.8), now that the impostor
is generated on the canvas the only two requirements left are a need for depth
CHAPTER 3. METHOD 17
and lighting, so that the 2D impostor looks the same as its object counterpart.
In order to give the object depth changes need to be made to the fragment shader of the impostor. Using the approach by LAWLOR [7] we already have the z-value map that was generated during the generation of impostors and add it to the canvas during the fragment shading stage, this should create the illusion that each pixel stands out of the canvas. To calculate the lighting we need a normal map generated during the impostor generation stage, when we capture the scene we also capture a matrix with the normal values, later during fragment shading adding lighting can be done using normal lighting styles such as blinn phong.
3.1.7 Creating NP3D Impostors
Creating NP3D objects is rather simple as it only requires you to have things done before you start the simulation, you need to have a down scaled version of your current object. One can use most 3D object editing software to reduce the amount of polygons that existed on the character loaded in. To get the impostor images the same procedure is done as for 2D impostors, but with a bigger gap between each photo because less photos are needed during NP3D [2]. In order to apply the impostor images to the lower resolution model requires us to have texture mapping for the model, this is done by putting the low resolution model into the imposter generator, but instead of storing images of the low resolution model the location of each vertex is stored so that we can use that to map the impostor to. When both the impostor generation and texture mapping has been done, rendering the NP3D models is done in the same way as a 2D Impostor, except we send in the entire mesh instead of the 4 points of the canvas, upon generating the vertex data we send it to a normal fragment shader for final rendering.
During early tests it was noticed that sometimes a model would sway be- tween two planes of impostors this would cause rapid switching between tech- niques causing a lot of popping, to counteract that a rule was made, if you cross to a new sector you have to to travel 3 units into the old sector before it switched back.
3.2 Tests
for our tests only one computer was used, the computer had a nvidia asus
geforce RTX 2080 and a intel I7 9700 which is during the time this thesis was
written considered highest tier hardware. The test scene was set up so that all
18 CHAPTER 3. METHOD
the characters start 1 unit(where 1 unit is the width of the character away from each other) they then move in a random direction with a distance of 8 units, no requirement for collision detection was set. To test our hypothesis and to compare the different impostors we created five tests. The first four tests were to to create a baseline and to try out our hypothesis. the first test was a base- line for comparing the different impostor techniques. The test was only normal models rendered to the screen using a rasterizer. To test the performance of 2D impostors we have a scene with only 2D impostors, the impostors used 160 sprites to create the sprite map. To compare NP3D we used a test scene with only NP3D, it used 52 sprites. To see how the hypothesis holds up a test scene was created with all three types in the same scene. To determine what type of character to draw half circles were created from the camera which were 15 and 30 units (the width of one character) in circumference, each circle chose the type of character that was to be rendered in the scene. If the character was within 15 units it would be the real object known as the near sector, 30 units it would be NP3D which is called the middle sector and after that we use 2D impostors in what will be called the far sector.
Figure 3.9: Models and NP3D impostors.
In order to compare NP3D with 2D impostors it was also chosen to observe
how 2D impostors behave with an increase in sprite count, this being because
a decrease or increase in sprites will drastically change how 2D impostors
behave, a lower amount of sprites will increase popping but decrease the frame
time. The amount of characters used was 62500 for that test. During the tests
we noted a lot of popping, this will not be mentioned in results since this is
highly subjective and difficult to show in image format, but will be discussed
in the discussion section
CHAPTER 3. METHOD 19
Figure 3.10: Lines indicating where the rendering techniques change.
Chapter 4 Results
4.1 Using Only 3D Objects
When using no impostor we received the expected results, the CPU was fine not being able to work at a 100
4.2 Using Only 2D Impostors
When using 2D impostors we quickly were able to asses that there was a great improvement to the frame time, but when we increased the amount of impos- tors beyond a specific amount the CPU started to bottleneck. This is because of all of the 4x4 matrix multiplications required to find out the position and rotation of each impostor.
20
CHAPTER 4. RESULTS 21
4.3 Using Only NP3D Impostors
NP3D acted much like the paper said it would, easing the workload of the gpu while increasing the cpu memory because of the texture map needed for the new object.
4.4 Using the Combined Technique
When we were using both objects and impostors we quickly noticed that the results started of a bit in between but then went more and more the way of impostors, this was of course because as we added more and more characters they were put further away from the screen and as such only increased the impostor value
Using NP3D as a middle plane resulted in the slightly better version of
the chart regarding 3 objects, to begin with i was simply cutting away the 3d
objects and keeping the cheaper ones, making us able to run 62500 in 412
ms which was a much better outcome when the middle plane was set as half
of the near plane only, the processor was the bottleneck much like with 2D
impostors.
22 CHAPTER 4. RESULTS
4.5 Using Only 2D Impostors and Varrying the Number of Sprites
With different amount of sprites we can see an almost linear increase in fram- etime versus amount of sprites. This is because we just have to go through each sprite to see if it matches with the character.
4.6 Distilled Information
The following graphs compare the tests and show how the different techniques
perform in different areas.
CHAPTER 4. RESULTS 23
24 CHAPTER 4. RESULTS
Chapter 5 Discussion
5.1 Using the Combined Technique
The biggest problem we found while trying out the hypothesis and combining the three different rendering techniques was the popping. While NP3D is used to alleviate some of the popping when rotating, it still does pop when it goes from NP3D to impostor or to real object. When Dobbyn [8] combined their impostors they only had two kinds of transitional pops from impostor to object and object to impostor. With the test it had doubled the amount of popping, since now the scene had 2d impostor to NP3D vice versa and NP3D to object and vice versa. This caused the objects to pop all over the screen making it very noticeable because the human eye is trained to spot changes.
When it came to the performance the overall results were poorer when there was less than 900 characters because the cpu memory was tasked with both storing the NP3D Impostor and the 2D impostor texture, although this could perhaps be stored in the same variable somehow. At 900 characters we could see that we still had a decent decrease in frame time compared to just using normal models and the scene had a better use of the gpu compared to how the 2D impostor balancing the values to not have a bottleneck anywhere
When we were designing the middle plane which the NP3D would use we had hoped we could use NP3D to remove some of the ugly popping that happens when 2D impostors were turning, but we quickly noticed that is cost too much to replace 2D impostors with NP3D when it was much cheaper to simply add more sprites to the equator line.
25
26 CHAPTER 5. DISCUSSION
5.2 Using Only NP3D Impostors
NP3D is a higher polygon count version of 2D impostors so when we used NP3D only we saw that we got a more balanced use of the cpu and gpu but at the cost of poorer frame time. There was no popping when the characters were moving during only NP3D
When using NP3D you are trying to apply a texture made for a 3D character on a simplified mesh of that same character, this leads to compressing of the image on several locations making it look a bit weird, this is not so noticeable on low detail objects with little pattern like a tree. While on characters it was much more noticable.
5.2.1 Lighting with NP3D
When we were implementing impostors we wanted to have some light react with it we achieved this by having a z buffer that we then calculated on to get the light onto the 2D impostors, when we were using NP3D this comes with the normal rasterizer meaning you do not have to do any extra calculations making it much better because you do not have to call other shaders for that purpose. This adds simplicity to creating the characters and less loading of different shades, something that can be very beneficial, since loading shaders can be very expensive.
5.2.2 Sprite Maps
With impostors you have to decide how many sprites you want on the equator line, if you have too few it will be noticable, too many and you will be wasting a lot of processor power. Np3d does not need this because you apply it on a 3d mesh, so it instead becomes a question of how good the mesh should be, something that you can actually calculate based on pixel size, we did not do this ourselves.
5.2.3 Multiple Sectors
In the test scene with combined impostor techniques there are three planes that
have enormous steps in between them. far sector has two polygons medium
sector 10.000 polygons and near sector having 40 000 polygons, one of the
things we could do to perhaps alleviate the popping is to simply make the steps
smaller, this would also make it cheaper by having the objects slowly change
CHAPTER 5. DISCUSSION 27
to the next step. This would mean that we had 10 different medium sector with different polygon counts on the NP3D.The texture mapped onto the original object is mapped onto the NP3D impostor. The problem is that the NP3D mesh has a smaller volume than the original that the mesh is intended for. This leads to artifacts. What we have found is that NP3D is not a suitable method for simplifying meshes near the camera. Originally NP3D is not designed to be part of a middle plane, but is more designed as a way to have less popping on impostors, so when we added them but had them closer the visual artifacts started appearing, much like 2D impostors have a lot more popping the closer they are. The cost of adding more sectors to the middle sector area is almost none, as all we have to do is save the mesh angles. If we use half the meshes vertices for each sector we will atmost increase the the cpu memory by twice as much because we will keep on halving the cost of each mesh which will atmost go towards twice the size because of sum of 1/i where i starts at 1 and goes to infinity which will go towards 2, where 1/i is how big the current mesh is compared to the original
5.3 Using Only 2D Impostors
During the tests it was quickly apparent that the cpu was bottlenecking when it came to the 2d impostors, when checking what took so much processor power, the highest amount of work time was spent on 4x4 matrix multipli- cations which is a O(n
3) operation. doubling the amount of sprites only dou- bled the frame time this means that the number of sprites we have is what costs impostors the most amount.
5.3.1 Known Directions
Currently when the impostor map is created about half of the sprites are taken
at the equator line. In most cases where you look at impostors the camera
position does not change so we keep the same perspective over the impostors,
eg. the camera is in a crowd of people so we see everyone from the equator line
no one is either above or below the camera, or we are looking down on a battle,
every view is from above so we do not need the equator line or anything below
it. By adding these constraints we can optimize how 2D impostors work to a
great extent because we can often remove more than 50 % of all the sprites,
which is what costs us in the long run.
28 CHAPTER 5. DISCUSSION
5.3.2 Locating Sprites
Currently in order to map the sprites we are using a rather poor system where we use the dot product between vectors that we had to do 4x4 matrix multipli- cations on, this is extremely expensive. One theory to improve this would be to switch over to a normalized cube to have better mapping that would easen the workload of the processor, but take up more ram, which currently is unaffected by the impostor generation and usage.
Figure 5.1: A different way to take photos for the sprite map, using a normal
mapped cube.
Chapter 6 Conclusion
From our tests we can see that our hypothesis was wrong, and combining the three rendering techniques caused both a worse result and more popping. Us- ing either only NP3D or only 2D imposters gave a better visual look and better performance.
NP3D is recommended to use in games, this is most likely because in games there are not alot of characters, what you gain by removing some of the rotation popping it takes by being many times bigger polygon wise, lead- ing to it limiting the amount of objects that can be in the scene because the graphic card has to work a lot harder to render each character. When trying to compare NP3D with 2D impostors the results showed that 2D impostors performed better with unanimated character, but one of NP3Ds strengths is its ability to not cost more when animated. Simply put our tests were unintention- ally made to work against NP3D and make their results look worse then they were.The advantage of using 2D impostors is a reduction in object complexity with a trade-off in used texture memory. NP3D impostors eliminates a lot of the texture memory because it requires less sprites in its sprite map. When rendering multiple different models there will be a breaking point, where you have too much texture memory and it will be better to use NP3D. When one is deciding which impostor technique to use, you firstly have to see what type of character is used and ly if GPU or CPU is taking up the most in your simu- lation, then decide on which impostor to use.
29
Bibliography
[1] Ladislav Kavan. Polypostors: 2D Polygonal Impostors for 3D Crowds.
2008.
[2] Jerry Yee and James Davis. Crowd Rendering with Non-Planar 3D Im- postors. 2008.
[3] Simon Dobbyn Rachel McDonnell and Carol O’Sullivan. LOD Human Representations: A Comparative Study. 2005.
[4] Kenneth Rohde Christiansen. The use of Imposters in Interactive 3D Graph- ics Systems. 2005.
[5] Eric Risser. True Impostors. 2005. url: https : / / developer . nvidia . com / gpugems / GPUGems3 / gpugems3 _ ch21 . html (visited on 09/30/2010).
[6] Erik Milan. “Impostors, pseudo-instancing and Image maps for GPU Crowd rendering”. In: The international Journal of Virtual Reality 6 (2007), pp. 35–44.
[7] ORION SKY LAWLOR. “IMPOSTORS FOR PARALLEL INTERAC- TIVE COMPUTER GRAPHICS”. PhD thesis. University of Illinois at Urbana-Champaign, 2005.
[8] Simon Dobbyn. “Geopostors: a real-time geometry impostor crowd ren- dering system”. In: ACM Press 1 (2005), pp. 95–102.
30
TRITA-EECS-EX-2019:401
