Comparing Perception of Animated Imposters and 3D Models

Comparing Perception of Animated Imposters and 3D Models

OLIVER ERIKSSON

WILLIAM LINDBLOM

Comparing Perception of Animated Imposters and 3D Models

OLIVER ERIKSSON, WILLIAM LINDBLOM

Bachelor in Computer Science Date: June 23, 2020

Supervisor: Christopher Peters

Abstract

In modern 3D games and movies, large character crowds are commonly rendered which can be expensive with regard to rendering times. As character complexity increases, so does the need for optimizations.

Level of Detail (LOD) techniques are used to optimize rendering by reducing geometric complexity in a scene. One such technique is reducing a complex character to a textured flat plane, a so called imposter.

Previous research has shown that imposters are a good way of optimizing 3D-rendering, and can be done without decreasing visual fidelity compared to 3D-models if rendered statically up to a one- to-one pixel to texel ratio. In this report we look further into using imposers as an LOD technique by investigating how animation, in particular rotation, of imposters at different distances affects human perception when observing character crowds.

The results, with regards to static non rotating characters, goes in line

with previous research showing that imposters are indistinguishable

from 3D-models when standing still. When introducing rotation, slow

rotation speed is shown to be a dominant factor compared to distance

which reveals crowds of imposters. On the other hand, the results

suggest that fast movements could be used as a means for hiding flaws

in pre-rendered imposters, even at near distances, where non moving

imposters otherwise could be distinguishable.

Abstract

I moderna 3D-spel och filmer är rendering av stora mängder karaktärer vanligt förekommande, vilket kan vara kostsamt med avseende på renderingstider. Allt eftersom karaktärernas komplexitet ökar så ökar behovet av optimeringar. Level of Detail (LOD) tekniker används för att optimera rendering genom att reducera geometrisk komplexitet i en scen. En sådan teknik bygger på att reducera en komplex karaktär till ett texturtäckt plan, en så kallad imposter.

Tidigare forskning har visat att imposters är ett bra sätt att optimera 3D-rendering, och kan användas utan att minska visuell trohet jämfört med 3D-modeller om de renderas statiskt upp till ett förhållande av en-till-en pixel per texel. I den här rapporten tittar vi vidare på imposters som en LOD teknik genom att undersöka hur animering, i synnerhet rotation, av imposters vid olika avstånd påverkar mänsklig iaktagelseförmåga när folkmassor av karaktärer observeras.

Resultaten, med hänsyn till statiska icke-roterande karaktärer, går i linje med tidigare forskning och visar att imposters inte är urskiljbara från 3D-modeller när de står stilla. När rotation introduceras visar det sig att långsam rotation är en dominerande faktor jämfört med avstånd som avslöjar folkmassor av imposters. Å andra sidan tyder resultaten på att snabba rörelser skulle kunna användas för att dölja brister hos förrenderade imposters, även vid små avstånd, där stillastående imposters annars kan vara urskiljbara.

Nyckelord

Imposters, Rendering, Rendering av folkmassor,

Iaktagelseförmåga

LOD Level of Detail

PBR physically based rendering FPS frames per second

RPM rounds per minute

Contents

1 Introduction 1

1.1 Background . . . . 1

1.2 Purpose . . . . 2

1.2.1 Goal . . . . 3

1.3 Problem . . . . 3

1.3.1 Hypothesis . . . . 4

1.3.2 Research questions . . . . 4

1.4 Methodology . . . . 5

1.5 Delimitations . . . . 6

2 Theoretical Background 7 2.1 Polygon meshes . . . . 7

2.1.1 Level of Detail (LOD) . . . . 7

2.1.2 Lighting . . . . 8

2.2 Imposters . . . 10

2.2.1 Imposter lighting . . . 10

2.2.2 Texture resolution . . . 11

2.2.3 Viewing angles . . . 12

2.2.4 Visual popping . . . 12

3 Method 14 3.1 Implementation . . . 14

3.1.1 Object loading . . . 15

3.1.2 Viewpoints . . . 16

3.1.3 Level manager . . . 19

3.2 User study . . . 21

4 Evaluation 23 4.1 Results . . . 23

4.1.1 Distance . . . 23

4.1.2 Rotation speed . . . 24

4.1.3 Participants self evaluation . . . 25

4.2 Discussion . . . 27

4.2.1 Conclusion . . . 28

4.2.2 Future Work . . . 29

References 30 A User study 32 A.1 Video . . . 32

A.2 Form . . . 32

A.2.1 Instructions . . . 33

A.2.2 Questionaire - For each scene . . . 34

A.2.3 Questionaire - Self evaluation . . . 34

A.2.4 Answers . . . 35

Chapter 1 Introduction

Games, movies and general crowd simulations

are constantly improving in quality and there is high competition in these fields. Movies, games and simulations have to look and behave realistically and be visually appealing.

To render a realistic large scene of complex characters is on the other hand a difficult problem, as it requires a lot of computational power to preserve character details. Characters, or meshes, rendered at a distance may sometimes be more complex and detailed than what can be rendered onto the screen. Depending on the screen resolution and the size/complexity of the mesh, it can be wasteful to render and process a full character mesh when only parts of it can be rasterized onto the limited number of pixels on the screen. It is therefore interesting and sometimes also necessary to apply LOD techniques, to reduce the complexity of characters without losing overall quality in the rendered scene. One such LOD technique is imposters, which is the technique this study will apply, following a comparison of fidelity with more detailed character models.

1.1 Background

A mesh is constructed by triangles, squares or general polygons, and

is a discrete representation of an object. The amount of polygons in a

high detail mesh in a modern real-time rendered game may be more than 100,000, and rendering crowds of characters of such complexity may be hard on consumer hardware while keeping high and consistent frame-rates.

In contrast, when observing large crowds at a distance, an observer cannot always perceive all of the details of the rendered crowd. This problem is both due to the human perception, which cannot perceive too much information at one time, and computer hardware. Screens have a limited number of pixels and a limited refresh rate, which causes complex characters when drawn small to be minified and approximated onto the limited number of pixels.

Rendering optimization is interesting since it affects both science and entertainment. In scientific simulations we may want to be able to display as much detail as possible in relevant areas, and leave redundant information out to keep complexity low in less relevant parts. When it comes to entertainment, we often want to be able to render many high detail characters with high fidelity while still being able to have reasonable render times in both movies and games. To enable this, we can try to reduce complexity of meshes as long as it does not affect the general appearance of the game/movie.

The problem of rendering complex characters can be reduced to rendering imposters, which in practice are textured quadrilaterals, more commonly refered to as textured quads. Imposters are used to replace complex models by rendering a texture representation of the model onto a flat plane, something that reduces the vertex count of the model, which in turn can improve performance. A problem with imposters is that characters are likely to be observed from any angle, and a texture is only a 2D representation of one single angle of an object. To be able to visualize a continuous representation of the object from any angle, an infinite amount of textures would in theory be required.

1.2 Purpose

This report presents work within the domain of computer graphics, in

particular crowd rendering and imposter fidelity. The purpose is to

CHAPTER 1. INTRODUCTION

address imposter fidelity from a user’s perspective in contrast to the more commonly analyzed optimization point of view. There is no doubt according to previous research that imposters can be used to reduce polygon count in a scene, and therefore speed up rendering times [2].

In reality though, we often want the scene not only to render fast, but also to look appealing as a whole. It is therefore necessary to study the effect of replacing high LOD models with lower LOD models with regard to the character’s fidelity based on a user’s perception.

1.2.1 Goal

In the best case scenario with respect to pure performance, all characters in a scene should be replaced with imposters, as that would reduce the polygon count by a great amount. Though in reality, imposters are just textured quads which makes shading of imposters hard. The quality and look of an imposter can therefore, depending on the implementation, be different compared to a 3D model when exposed to dynamic lighting, animations and movements.

Given the fundamental differences between 3D models and imposters, the goal of the study is to get an understanding of how noticable rotating/moving imposters actually are from 3D models. The end goal of such research is to be able to tell when and how imposters can be used as a non noticable mean of optimizing crowd rendering.

1.3 Problem

With a constant amount of pre-generated textures representing one character from different angles, fidelity is being evaluated of the simplified character compared to a full 3D model in a simple scene. The investigated problem is therefore whether a user is able to differentiate vertically rotating characters in form of imposters from rotating 3D models at different distances in a scene.

Imposter fidelity and differentiability has been investigated before by

JHamill et al [3]. In that study, animated virtual humans and buildings

represented as imposters and higher LOD models, are compared

with respect to imposter effectiveness in real-time rendering. The

study performed was done by comparing user perception using a psychophysical evaluation method. What was done was to evaluate at what distance the users noticed changes between imposters and geometric models when displayed statically without any movement or animation involved.

1.3.1 Hypothesis

Due to the loss of detail for any character, both imposter and 3D- model, at increasing distances, we believe that as the distance between characters and the camera increases, the 3D-models will be harder to differentiate from the imposters. The results should therefore show a decrease of participants able to differentiate the crowds as the distance increases. It is also believed that rotation speed will affect the fidelity of imposters at a closer distance due to visual popping and lack of viewing angles. At a longer distance such effects may be harder to notice and thus the speed will not be as relevant at larger distances.

1.3.2 Research questions

With regard to the problem of differentiating imposters from 3D models, the question to be answered is how distance and rotation/animation speed affects user perception of pre-rendered imposters versus 3D-models. To answer the question, the following sub-questions need to be answered:

1. How much does the distance to the models contribute to differences in perception between crowds of 3D models and imposters?

2. How much does the rotation/animation speed contribute to differences in perception between crowds of 3D models and imposters?

3. What are the most common factors that make a user able to notice imposters in a scene?

Previous research has shown that imposters’ distance to the camera,

which in fact is the pixel to texel ratio of the imposter in the scene, does

matter for static image imposters [3]. Preserving a 1:1 pixel to texel

CHAPTER 1. INTRODUCTION

ratio should yield a satisfactory result in terms of imposter fidelity.

On the other hand, little has been said regarding how animation affects perception of imposters, and whether a 1:1 pixel to texel ratio is necessary to preserve fidelity if the characters are moving. This makes the research question particularly interesting since pre rendered imposters could benefit, saving for example texture memory, from not having to preserve a 1:1 pixel to texel ratio in all possible scenarios. By introducing animations, there is a possibility that imposter blurriness and lack of detail could be hidden by applying animations.

Moreover, it is interesting to observe whether rotation has any significant impact in differentiating imposters from 3D models. If introducing animation does not cause any significant differences, imposters could possibly be used in games and movies for other purposes than background elements, which is how they are often used today [4], to create a more living environment at a low performance cost.

1.4 Methodology

To compare 3D-models and imposters, a game engine in which the scene occurs, and an evalution method of the result is needed. The game engine can either be developed from the ground up, or an already existing application can be used. By developing the environment from the ground up, it guarantees full controll of all parameters in the scene in which the study is performed. Hence parameters such as lightning, shadows, and background can easily manually be tweaked for the experiment, and only the bare necessities have to be implemented.

In contrast, an existing game engine already has many built in features.

Therefore using an existing game engine makes it easier and and less time consuming to set up the enviroment for the scene. With an existing game engine, there is no need to implement necessities such as an event system, shader compilation, object loading and data to GPU- submissions. The scene enviroment in this paper was implemented using Unity 3D.

To evaluate within what distance from the camera most users notice

differences between rotating imposters and rotating 3D models, a quantitative study from the perspective of the pixel to texel ratio can be performed. The result with that evaluation strategy would answer where the critical point, for replacing 3D models with imposters, is in theory. In this report however, the question is where the user cannot notice the difference between imposters and 3D-models. Hence a user study is a suitable evaluation method.

1.5 Delimitations

In the study, character animation is limited to rotation around the vertical character axis. The study does therefore not compare any sophisticated movement, such as animations of individual body parts, between the imposters and 3D-models. The scene also takes place in a single colored lanscape without level differences in the ground plane.

Moreover, no other objects than the compared character crowds are be visible in the scene.

When talking about imposters in the study, if nothing else is mentioned, it is referred to pre-rendered imposters. That is, an imposter whose texture was rendered before the application was run.

There exists imposter techniques where the textures are rendered to an off-screen buffer at run time, but no such system was implemented and tested during this study.

All light is limited to one single static light source. The one light source

in the scene is positioned behind the camera pointing in the face of

the characters when they are looking into the camera, making the

characters never appearing to be in shadow. Moreover, no shadow-

casting is done on characters appearing behind others.

Chapter 2

Theoretical Background

This chapter will present essential background information and concepts to understand the study and interpret the results. The basics of mesh structures will be explained including mesh lighting.

Moreover, imposters as an LOD concept is explained and defined.

2.1 Polygon meshes

A common way to represent a 3-dimensional graphical object in a computer is by using a polygon mesh. A polygon mesh consists of a set of vertices, edges and faces which when combined define the shape of a 3-dimensional graphical object. The more vertices an object has, the more complex and detailed the object is. High detail meshes may seem desireable in a real-time applications, but they come with the cost of being memory expensive and slower to render than characters with lower LOD. Reducing polygon count is a way to reduce memory requirements and optimize rendering times in an application, but may result in the overall scene being perceived as less detailed.

2.1.1 Level of Detail (LOD)

In 3D rendering of crowds, different levels of detail are used to optimize

rendering times to be able to render more characters each frame

and maintain a steady frame rate. What LOD means in practice is that models of different quality and structure are used in different scenarios. It may not always be necessary to render the highest LOD models, so lower LOD models can be used and still have the characters being perceived equally by the observer of the character or crowd.

Selecting an LOD of a model can be done based on different criterions such as model distance to camera and model movement/animation speed/complexity. For example, it may not be necessary to render a high detailed model far in the distance, as all details may not be visible anyway on the limited amount of pixels on the screen. Moreover, if the character is moving very fast, the user may not be able to detect all details in a high detailed mesh, so a lower LOD mesh could be used instead while maintaining the same overall perceived quality of the character [4].

2.1.2 Lighting

When rendering images, lighting is crucial to achieve a realistic look of the rendered objects. Both reflections, refractions and shadows exist due to light travelling and interacting with objects before reaching the human eye. More advanced and physically based rendering (PBR) calculations, such as reflections and refractions, require ray tracing to be performed, which is nothing we will touch in this report. On the other hand, when rendering with rasterization, one can still achieve specular, diffuse and ambient lighting effects using Phong shading [6][1].

Ambient lighting

Ambient lighting is a mean to achieve global illumination. What this kind of lighting does is to give all objects that are not in direct lighting a slight color other than pitch black. In other words, ambient lighting mimics light being reflected on other objects in the scene so that nothing is not pitch black just because it is not directly hit by a light source.

In computer graphics, ambient lighting can be done in different

sophisticated ways, but the simplest form of ambient lighting can be

CHAPTER 2. THEORETICAL BACKGROUND

done by simply adding a small amount of color to each pixel when rendering, usually some tint of gray.

Specular lighting

Specular lighting, or specular highlight, is a highlighting effect that causes an object to be perceived as shiny. Specular lighting is basically a reflection of the incoming light on the object surface, which in computer graphics is calculated using the surface normal, the light position, the camera/eye position and the light and material colors.

l

= l

∗ c

∗ (ˆv · ˆr)

where v = p

− p

l = p

− p

r = l − 2 ∗ (l · ˆn) ∗ ˆn

where l

is the specular light vector calculated by component wise multiplication between the source light and the material color multiplied by the dot product between the reflection vector r and the view vector v.

Diffuse lighting

Diffuse lighting is caused by surface roughness, and looks like the light is being smoothly spread out when reflected on the object surface.

Diffuse lighting is the effect that occurs when light hits fabric or any

non shiny surface, where shadows and light smoothly blends on the

surface of the rough material. In computer graphics, diffuse light is

calculated using the surface normal and the direction of the incoming

light. Basically, the diffuse light is how intensely a surface is exposed

to a certain light source.

l

= l

∗ c

∗ (ˆn · −ˆl)

where l

is the diffuse light vector calculated by component wise multiplication between the source light and the material color multiplied by the dot product between the normalized normal vector ˆ n of the surface and the inverse of the normalized light direction ˆl.

2.2 Imposters

Imposter rendering is an LOD technique where geometric detail is replaced with image detail to reduce complexity and polygon count of a character in a scene. An imposter is basically an image texture of a character from a certain angle rendered onto a flat polygon, often a simple quad of four vertices [4].

The texture of the imposter can either be pre-rendered, or rendered at run-time, but in this study we will only focus on pre-rendered textures as real-time rendering textures requires a more sophisticated system/application.

2.2.1 Imposter lighting

An imposter can simply reduce a mesh from having more than 100 000 polygons, down to four simple vertices, which can increase frame rate in a real-time application significantly. Though a significant speed up, imposters come with the cost of just being a 2D-representation of an object, making light calculations such as specular- 2.1.2 and diffuse lighting 2.1.2 not possible. Light calculations depend on surface normals, and a quad has only one surface normal and at most four unique vertex-normals, which cannot be used to correctly calculate specular or diffuse effects of an individual pixel on a pre-rendered imposter texture. With that said, there exists techniques to add lighting effects to pre-rendered imposters, but it is hard to get the lighting as accurate as that on the corresponding 3D-model.

Due to this, the study performed will try to minimize the effect of light,

CHAPTER 2. THEORETICAL BACKGROUND

as lighting is rather tricky to get equal when comparing imposters to 3D-models. It would not be possible to completely remove light from the scene when using shaders that calculate diffuse lighting, as that would make the objects completely black. When rendering 3D-models in the scene of the study, the same light angle and intensity will be used as when pre-rendering the textures for the imposters, to make sure they are preceived as close to equal as possible while still keeping some effects of diffuse lighting.

An alternative would be to use flat color shading, which means that only the color of the texture/model is used, and no light color, when rendering. On the other hand, that sets up a more unlikely scenario as most scenes and applications do use light sources and apply diffuse and specular lighting to objects. Moreover, the characters will under flat color shading look rather flat, which itself could have an effect on the user’s perception of the characters.

2.2.2 Texture resolution

Another factor that affects perception of imposters is texture resolution. The lower the resolution of the imposter texture, the harder it is to perceive the character and detect character details [9]. This in turn makes it easier to detect differences between a lower resolution texture and a 3D-model where the texture is uv-mapped to the surface and the color between vertices is interpolated smoothly.

A lower resolution imposter would have to be place further away from the camera to be perceived equally as a 3D equivalent, which also makes camera distance a relevant factor for perception when using imposters as an LOD technique [9].

Texture filtering

When an imposter is drawn at different distances into the scene, the

texture has to be scaled to fit the size of the character. In an optimal

scenario, the dimention of the texture and the size of the imposter is

equal, so each color unit/texel on the texture can be mapped to exactly

one pixel on the screen. Though this is not always the case, and the

imposter or target where the texture should be rendered may be either

smaller or larger than the size of the texture.

To solve this problem, texture filtering is used to filter the texture and approximate the color of a pixel given the surrounding texels in the area. A maginifiaction filter is used when the texture has to be stretched to fit the larger geometry, and a minification filter is used when the texture has to be shrinked to fit the smaller geometry [7]. The filtering method used for magnification or minification is essential to how the result will look, and common such methods are nearest-neighbor interpolation, bilinear filtering, trilinear filtering and anisotropic filtering.

2.2.3 Viewing angles

As imposters are 2D representations of a single object using an image texture, it is also only a representation of an object from a single angle and in a single position. When rendering imposters one therefore has to capture several images from different angles of the object, to be able to display a more continuous and complete representation of the object as a whole. The more angles the object is captured from, the more character variety can be illustrated with the imposter when the object for example is being rotated.

When pre-generating imposters, the number of viewing angles is directly proportional to the number of pre-generated textures of the object. On the other hand, one can also dynamically generate imposters at run time. This means that the imposter texture is rendered to an offscreen buffer from whatever angle is needed for that specific imposter at a given frame. Using this method, there is no need to pre-determine which angles of the object that should be visible. When rendering new imposters each frame, K. R. Chrisiansen states that no difference between imposters and 3D models can be seen, though it can affect performance rendering new imposters each frame [2].

2.2.4 Visual popping

Visual popping is what may occur when the LOD of an object is

changed. If the visual difference is too big between two distinct levels of

detail, the jump between those levels may cause the user to notice the

loss or gain in detail. In turn, this can steal focus from more important

details or objects in the scene. When working with LOD the purpose is

CHAPTER 2. THEORETICAL BACKGROUND

to optimize render times without reducing the perceived quality of the scene noticably. That is, we want to avoid visual popping as much as possible while still gaining a valuable increase in performance.

Visual popping may also occur when changing the viewing angle of an imposter. The rotation or animation of an imposter caused by switching between two similar, but not equal, positions of a character on a texture, may cause visual popping if the difference in position is too big. Thus increasing the number of viewing angles may reduce visual popping.

Dynamically generated imposters, which can represent any view

angle or animation of an object, can also cause visual popping due

to imposters not being re-generated each frame. One approach

for rendering dynamically generated imposters is to have a change

threshold in for example view angle, to determine when to generate

the imposter again [2]. If this threshold is too big, so the change in

movement of the impostor is too big, visual popping may occur.

Method

The experiment was divided into two parts, implementing the enviroment where the scene took place and perform a user study to evaluate the research question. To render the image textures to the impostors, the modelling application Blender was used. The rendered textures were then imported to Unity 3D where the scene was implemented. The user study was evaluated by asking questions regarding the character fidelity in the scene, where the users answered whether they could spot differences between imposters and 3D-models in different scenarios.

3.1 Implementation

First of all, the Unity documentation was used to get an understanding of how create applications in the Unity Game Engine [8]. This section will describe how the scene was created using Unity and what design decisions we made with regard to the scene layout.

To answer the research questions, a set of features were required by the scene, including:

• Simulating a crowd mixed of impostors and 3D-models.

• The characters should be able to rotate at different speed.

CHAPTER 3. METHOD

• A user should be able to view the characters from different positions.

3.1.1 Object loading

A few C# scripts were written to be able to load a 3D-model into Unity from a .blend file and render the character in the scene. The loaded mesh resource, or so called prefab, was used to instantiate multiple characters into the scene at different positions.

To render the imposter, a unit square mesh was created with side length one. Onto the mesh, UV-coordinates were mapped to each corner of the square to be able to map a corresponding texture to the square. As all textures were pre-rendered, any of them could be loaded and added to the square to finish off the imposter and render it into the scene.

To match the imposter with the 3D model in size, the 3D model was

scaled to unit height based on its bounding box dimensions. Also, as

the 3D-model had its feet on the origin, the model had to be translated

to be centered around the origin like the imposter’s unit square. Last,

when transforming a prefab in Unity the whole asset is transformed, so

to make sure the asset was not rescaled each time the asset was loaded,

a simple check was made to see if the object had already been scaled

down to unit length.

float max = prefab.GetComponent<Renderer>() .bounds.max.y;

float min = prefab.GetComponent<Renderer>() .bounds.min.y;

float scale = max - min;

if (scale > 1) {

p.transform.localScale /= (scale * 1.1f);

//Transform to correct position

p.transform.position = new Vector3(0f, -1 / (scale * 1.1f), 0f);

}

When selecting the character for the study, we selected a model that had human features such as head, arms, legs and feet, but on the other hand we did not mind that the character did not have a very detailed face. As long as the user would be able to recognize its features and somewhat relate to the object they are observing.

The most important thing from a technical point of view was that the mesh was large enough, and had enough detail/vertices, so that making imposters out of it would actually make sense in a real application. In reality if the mesh is small enough, creating imposters of the character may not result in a noticable performance boost, which only creates an unnecessary overhead and code complexity.

3.1.2 Viewpoints

To generate the image textures for the pre-rendered impostors the 3D-

model was imported to Blender. In Blender a virtual camera was placed

to capture images from different angles of the model. The camera

was placed to capture a full length portrait of the model and was then

rotated 15 degrees horizontally at each step around the model. This

resulted in 24 viewpoints of every 15 degree angle of the model. The

number of 24 viewpoints was chosen mainly for simplicity reasons, as

Blender’s built in camera rotation rotates 15 degrees. At the same time,

CHAPTER 3. METHOD

24 is a reasonably large number of textures. Previously 32 viewpoints have been tested, but in that scenario the models did not rotate and the models were only viewed two side by side, and not in crowds.

Therefore, we estimated that we could have fewer viewpoints while still maintaining imposter fidelity at a high level [9]. The 24 textures were stored as png and had an image resolution of 1920x1920.

In the application, one of the 24 character viewpoints is selected based

on the angle the character is viewed from. To animate a rotation, the

24 textures are simply cycled through in a linear way, and one texture

at a time is selected based on how many degrees the character model is

rotated at a certain point in time.

private const int NUM_TEXTURES = 24;

private const float MAX_ANGLE = (360f / NUM_TEXTURES) / 2;

private const float MIN_ANGLE = -MAX_ANGLE;

public override void RotateCharacters(float deg) {

for (int i = 0; i < m_Characters.Length; i++) {

m_Rotations[i] += deg;

if (m_Rotations[i] > MAX_ANGLE) {

m_Rotations[i] = MIN_ANGLE +

(m_Rotations[i] - MAX_ANGLE);

m_CharTexIndices[i] += NUM_TEXTURES-1;

m_CharTexIndices[i] %= NUM_TEXTURES;

}

m_Characters[i].GetComponent<Renderer>() .material.mainTexture =

m_Textures[m_CharTexIndices[i]];

} }

Rotating all characters is done each frame, which for the 3D-models

implies incrementing it’s rotation around the y-axis. For the impostors

it means selecting the appropriate texture based on how much the

character is rotated. Using 24 textures, we implemented the texture

selection so that for example texture 0 is visible when the character is

rotated between -7.5 and 7.5 degrees, texture 1 between 7.5 and 22,5,

etc.

CHAPTER 3. METHOD

3.1.3 Level manager

To set up the scene, a level manager was necessary to be able to manage different levels of distance to the characters and rotation speeds. A level therefore consists of two integers, representing the number of imposters and the number of 3D-models, a floating point value storing the speed of rotation and a floating point value representing the distance from the camera where the characters should spawn.

Given a set distance of the camera along the z-axis, the characters were spread out in x-direction randomly within a fixed number of units.

To prevent characters from spawning on the exact same position, the characters were also randomly positioned in z-direction ±1 unit from the fixed z-position.

Last, every other character is chosen to rotate either left or right, to make the crowd appear more random. This is based on the result of Odhner et al (2019) that it may be easier to spot differences between two crowds given they are equal. Therefore, making the crowd more random may make the imposters harder to spot [5].

Setup

In the application, four different levels of distance and four different levels of rotation speed were selected, creating 16 unique scenes. The rotation speeds were 0, 4, 8 and 16, representing the number of degrees the character is being rotated each frame. Running the application on an Nvidia GTX1070, we were able to run the application at 75 frames per second (FPS). Using speeds 4, 8 and 16 degrees of rotation per frame corresponding to 90, 45 and 22.5 frames per lap, the different speeds at 75 FPS correspond to 50, 100 and 200 rounds per minute (RPM) respectively.

Regarding the distance, the camera in the scene was placed at a base z- position of -5. The characters in the scene were then positioned around z-positions 0, 4, 6 and 12, in different levels. As the imposters were pre-rendered, the perspective of the imposters was equal no matter the distance, which is not true for a 3D-model, whose perspective/view angle changes depending on the distance to the camera.

As can be seen in the image 3.1.1a, though subtle as the characters

(a) Furthest position with flat perspective

(b) Furthest position with perspective from above.

(c) Closest position with flat perspective

may appear very small, when the 3D-model on the left is rendered at a distance one cannot fully see the top of the character’s feet. On the imposter on the other hand, the top of the feet are slightly visible and the toes are appearing to point down slightly, because of the perspective the image was captured from. To solve this, we decided to move the camera in y-direction based on how far away the characters were rendered, to compensate for the skewed perspective on the flat imposter. We found a camera.y = character.z/2 to be a good estimate to make the two sides appear as equal as possible. In 3.1.1b, the camera is therefore positioned at z = −5, y = 6 as character position is 12 in that level.

One can notice a difference between the right imposter and the left 3D-model in 3.1.1c looking at the feet, though at this distance we found them to be similar enough perspective-wise so no camera- repositioning was done when the characters’ position was 0.

Due to the camera positioning, the actual distance from camera to the

CHAPTER 3. METHOD

characters is √

(5 + character.z)

+ (character.z/2)

, where character.z is either 0, 4, 6 or 12. The result is the characters being positioned at approximate distances 5, 9.2, 11,4 and 18.0 from the camera. For simplicity reasons, when presenting and discussing the rotation speeds and character positions in the evaluation, we will still label them using 0, 4, 8 and 16 for rotating speeds and 0, 4, 6 and 12 for the distances.

3.2 User study

To evaluate fidelity in terms of similarity and distinguishability between pre-rendered imposters and 3D-models, a user study was conducted based on the created crowd simulation tool. To answer the research questions proposed in 1.3, the levels in the application were set up to test the effect of different rotation speeds and distances.

The user study then consisted of watching 16 scenes and one introduction scene in form of video clips. The introduction scene was made up to make the difference between a 3D model and an imposter clear, and in connection to that the users were asked whether they could perceive differences between the displayed characters, which they were expected to do. Moreover, they were asked which one of the two characters they prefered from an esthetic point of view, and what key features they perceived different between the imposter and the 3D model.

Each scene including the introdiction was displayed for 10 seconds after which a 5 second break was made for the user to pause the video and answer three questions. The following three questions were asked to the users in connection to each level:

• Which character crowd did you perceive to be of lower quality (imposters)? (Left/Right/The crowds appeared equal)

• Which character crowd did you prefer from an esthetic point of view? (Left/Right/Neither)

• If you noticed something special in the scene that made one

side stand out particularly, please write a short comment. (Text

answer)

When all scenes had been watched, the users were asked general questions about how hard or easy they found it to be to differentiate the imposters from the 3D-models, what made them see the difference when they saw it. Moreover, they were asked whether there were any particular distances or movement speeds they found to be particularily revealing and whether their perception of what differs imposters from 3D-models changed after watching the movie compared to after just having watched the introduction.

In the user study, there were 21 participants in total where all were

in the age group 20-35. The full user study is found in appendix A

(4.2.2).

Chapter 4 Evaluation

In this chapter the results are presented with respect to character position and speed together with the participants self evaluation. The results aim to point out which factors are contributing to participants being able to differentiate imposters from 3D-models. As part of the evaluation, the results are then also discussed and validated.

4.1 Results

Given the 16 different scenes shown in the user study, data was summarized and averaged based on the characters’ positions and rotation speed respectively to get an understanding of how each of the two factors were contributing individually.

4.1.1 Distance

The results from the user study with respect to distance can first of all be stated to not completely fall in line with the hypothesis. In contrary to what was believed, users were more likely to spot imposters at larger distances than when the characters were closer to the camera.

An average of 50.6% correct answers at character positions 0 and 4

compared to an average of 59.5% correct answers at positions 6 and

12.

(a) User answers, averaged by position, of whether imposters were

perceived correctly, incorrectly or equal.

(b) Users’ self evaluation of which distances they thought the imposters

were easiest to spot

The users themselves on the other hand perceived it to be far easier to spot imposters at closer distance. The user self evaluation diagram in 4.1.1b states that 19.05% and 14.29% of the users found it hard to spot imposters at positions 6 and 12, which goes in line with the hypothesis that users would find it hard to spot imposters at a distance. The actual data on the other hand shows particularly that users were very good to spot imposters at distance 6, where 72.6% correctly spotted the imposters, 22.6% perceived imposters and 3D-models to be equal and 4.8% incorrectly thought the 3D-models were of lower quality.

On the other hand, looking at positions 0 and 4 isolated, they pretty well line up with the users’ self evaluation at 1 (Near) matching position 0 and 2 matching position 4, around 60% and 40% respectively, which could imply that something in the scene happens around position 6 making the imposters stand out particularly. One can also observe a significant drop again in the number of correct answers from position 6 to position 12, which is expected. At position 12, 46,4%

spotted the imposters correctly, 47.6% thought they appeared equal and 5.95% answered incorrectly. At position 6 on the other hand, 72,62% answered correct, 22,62% perceived equal and 4,76% answered incorrectly.

4.1.2 Rotation speed

At speed 0, when the characters were standing still, 29.76% correctly

spotted the imposters, 17.86% incorrectly thought they spotted the

CHAPTER 4. EVALUATION

(a) User answers, averaged by movement speeds, of whether imposters were perceived correctly,

incorrectly or equal.

(b) Users’ self evaluation of which movement speeds they thought the

imposters were easiest to spot

imposter and 52.38% perceived the crowds to be equal. At speed 4, when the characters were rotating at 50 RPM, 91.67% correctly spotted the imposters, 2.38% incorrectly spotted the imposter and 5.95% stated the imposters and 3D-models appeared to be equal. At speed 8, when the characters were rotating at 100 RPM, 75% correctly spotted the imposters, 2.38% incorrectly thought they spotted the imposter and 22.62% percieved the imposters and 3D-models to be equal. At speed 12, when the characters were rotating at 200 RPM, 23.81% correctly spotted the imposters, 14.29% incorrectly spotted the imposters and 61.9% percieved the imposters and 3D-models to be equal.

Compared to the user self evaluation on distance, the users rather accurately perceived how hard or easy different rotation speeds were to differentiate compared to the distribution of correct answers. The two graphs 4.1.2a and 4.1.2b both show that users find imposters standing still hard to differentiate from 3D models. From there, which also goes in line with our own hypothesis, users will find it harder to differentiate imposters and 3D models the faster they rotate, up to a point where they are perceived equal by a majority of the users.

4.1.3 Participants self evaluation

At the end of the user study, the participants commented on what

factors they found made the imposters and 3D-models differ. The

comments overall could be divided into two groups. One group, the

majority of the participants, perceived the rotation speed and the

difference in rotation smoothness between the imposters and 3D-

models to be most significant. The other group of the participants

perceived the quality of the imposters in general to differ from

the 3D-models, which was more revealing. In these comments

lighting, smoothness and shininess was mentioned to differentiate the

imposters from the 3D-models. One participant stated that it was

harder to diffentiate the quality between the crowds when the distance

was larger. Another participant stated that as the distance increased,

the rotation speed became a more significant factor.

CHAPTER 4. EVALUATION

4.2 Discussion

In contrary to our hypothesis the result did not show that an increase in distance made it harder for the participants to differentiate the imposters from the 3D-models. As shown in the result there is an anomaly in the otherwise expected falling trend, at position 6 where almost 73% spotted the imposters. Why the difference between the crowds is so apparent at that distance is hard to tell. But we believe other factors than distance played a major roll here. Since the imposters were pre rendered, the resolution of each image was constant independent of the character position. At a larger distance it is likely that texture magnification and minification together with aliasing are contributing factors to the results. When the imposters are rendered at a distance, texture minification has to be applied, and depending on the aliasing and minification algorithms used the results may differ compared to the result of rasterizing the detailed 3D mesh. To minimize these consequences, mipmapping techniques could be applied to the imposter textures to make the quality better at minification.

Looking at the study from a broader perspective, the participants in the user study was a fairly homogeneous group. A vast majority of the 21 participants consisted of other computer science students, many of which having an interest in playing video games. Whether or not the results are affected by this is hard to tell, but the participants in general may have been more sensitive to quality changes in the scenes than the average man or woman.

As a consequence of the on going covid-19 pandemia, the user study had to done remotely by the participants watching the video on their own computer. Therefore the result might have been affected by the differences in hardware used by the participants, together with us not being able to control that all participants downloaded the video and watched it on the highest resolution in full screen, instead of watching it in the web browser in lower quality.

What can be said about the results is that there is still strong evidence

that static imposters if implemented properly is a good way of reducing

geometric complexity without losing fidelity in a scene. Even though

the results based on distance did not match the hypothesis, as users

were more likely to spot imposters at a distance, one can still see that

no matter the distance imposters are generally hard to spot when not moving at all.

If animation should be implemented as part of a pre-rendered imposter system, it is necessary that the difference in movement between two key frames is small enough or the time between two following key frames is short enough, so that the visual difference between two key frames is not causing any flickering. Our results show that for a rotation in 15 degrees per key frame of the imposters, a time of 16 degrees of rotation per frame was necessary for imposters to be equal to 3D-models in terms of animation. This is likely due to that for each frame, the 3D- model rotates about the same amount as the imposter does between two keyframes.

To answer the research question, distance seems to affect perception of imposters in some extent. To say the least, users themselves thought they had a harder time spotting imposters at a distance, even though the results said they were more accurate when the imposters were further away. The reason for this may be that a user could be more likely to try hard if the the characters are smaller and the difference between the characters is smaller. On the other hand we are aware that the imposters used, which were pre-rendered and exported from Blender, had major flaws and did not look equal to the 3D-models if you observed them for long enough. One major problem was shading, as light calculations may be done differently in our Unity application versus in Blender, and it was hard to match the settings in the two programs. Another problem is perspective, as pre-rendered imposters are captured at a fixed distance, the perspective of the imposter is not changed as the imposter moves further away, which would be the case of a 3D-model.

4.2.1 Conclusion

Rotation speeds slower than 1:1 keyframe per frame seem in the end

to be the most contributing factor to differentiate imposters from 3D

models when compared against distance. 3D models rotating slower is

theoretically equal to imposters having more viewpoints, which cannot

be added if imposters are pre-generated. This will lead to visual

differences when crowds of imposters and 3D-models are rotating at

RPM. In a scenario using real-time rendered imposters on the other

CHAPTER 4. EVALUATION

hand, animation speed should not contribute as much to perception differences, as any position of the 3D-model can be projected at any time, at a small cost of performance [2]. On the other hand, the results show that even for pre-rendered imposters, a fast enough rotation speed can be used to hide visible flaws.

As users are more likely to perceive imposters and 3D-models to be equal when moving fast than standing still, while static imposters already are proven to be a good LOD technique [3] in a 1 1 pixel to texel ratio, pre rendered fast moving imposters could very well be used in movies and games. As long as imposters move fast enough in relation to the visual difference between two animation key frames, the visual fidelity of imposters is comparable to that of 3D-models.

4.2.2 Future Work

As no effort was put into creating realistic shading for the imposter, future work could involve improving quality of the imposter by applying more sophisticated shading techniques and normal maps.

Something that would be interesting to look at is whether there could be found any threshold where improvements to pre-generated textures closes up on imposters rendered to textures in real time with respect to perception, while still being better performance wise.

Moreover, it could be interesting to see if results would be similar when

applying animations of individual body parts, instead of just applying

rotations to the characters. As this study only tests animations in terms

of rotations, an extension to including rigid body animations would be

useful for future work in the field of imposters.

[1] Basic Lighting.

https://learnopengl.com/Lighting/Basic-Lighting, May 2020.

[2] Christiansen, K. R. The use of Imposters in Interactive 3D Graphics Systems. Tech. rep. Department of Mathematics and Computing Science Rijksuniversiteit Groningen, 2005.

[3] Hamill, J., McDonnell, R., Dobbyn, S., and O’Sullivan, C.

Perceptual Evaluation of Impostor Representations for Virtual Humans and Buildings. Tech. rep. Image Synthesis Group, Computer Science Department, Trinity College Dublin, Ireland, 2005.

[4] Luebke, D., Cohen, J. D., Watson, B., Reddy, M., Varshney, A., and Huebner, R. Level of Detail for 3D Graphics. Elsevier Inc, 2003.

[5] Odhner, N. Lindberg and Freme, C. J. “Impact of Crowd Density and Camera Perspective on User Sensitivity Towards Impostor Resolution Degradation in Crowd Simulators”. B.S. Thesis. KTH ROYAL INSTITUTE OF TECHNOLOGY, June 2019.

[6] Phong, B. T. Illumination for Computer Generated Pictures.

Tech. rep. Communications of the ACM, 1975.

BIBLIOGRAPHY

[7] Texture mapping.

http://www.glprogramming.com/red/chapter09.html, May 2020.

[8] Unity User Manual. Unity Technologies.

https://docs.unity3d.com/Manual/index.html, Mar. 2020.

[9] von Eckermann, J. “How users differentiate imposters from real

models”. B.S. Thesis. KTH ROYAL INSTITUTE OF

TECHNOLOGY, June 2018.

User study

Following section lists all content related to the user study. Both form instrutions handed to the test users, the video used for the study and the answer spreadsheet compiled from the users’ answers.

A.1 Video

The video used to perform the user study. The content was pre- rendered from Unity and downloaded by the participants.

https://drive.google.com/open?id=16l6emf_

idSKk7XhiFhAGbLoPFoRQC-Kx

A.2 Form

The form for the user study was created using Google Forms.

https:

//docs.google.com/forms/d/e/1FAIpQLSf67UXa-V6jTq-e_

rvxuJPxiODMZsvtnjMsmdlft4XX-Zd10g/viewform?usp=sf_

link

APPENDIX A. USER STUDY

A.2.1 Instructions

In video games and movies it is often necessary to display crowds of complex characters in a scene. Examples could be battles in action movies with many fighting people, or a background filled with trees in a 3D game.

When rendering many characters at the same time, some characters can be replaced with lower detailed characters to increase performance.

One such technique is imposter rendering. An imposter is, in contrary to a 3D-model, only a simple square plane with an image representation of an object pasted onto it.

In this experiment, we are testing how perception of imposters, opposed to 3D-models, is affected by distance and movement speed.

The purpose is to get an understanding of how imposters are perceived in crowds compared to 3D models.

You will watch a 4 minute pre recorded movie in full screen with 17 different scenes including a preview/introduction scene.

The video is found here:

https://drive.google.com/open?id=16l6emf_

idSKk7XhiFhAGbLoPFoRQC-Kx Download the video and run from your computer. Do NOT watch it in the browser as it will be shown in lower resolution. Also remember to watch the video in full screen, for whatever screen you have.

Each scene is displayed for 10 seconds, and after that there is a 5 second pause until the next scene appears. You should pause the video during the 5 second break where the scene has no characters, to answer the questions in your own pace. In each scene, you will see two crowds.

One crowd consisting of 3D-models, and the other of lower quality imposters. Your task is to tell if there is a difference between the two crowds, and which you prefer esthetically.

Do not rewatch a scene twice, even if you felt you did not have enough time to look carefully. We are evaluating the fidelity of the characters in the scenes, and not your individual ability to spot differences between the crowds.

You may respond to the questions in either Swedish or English.

A.2.2 Questionaire - For each scene

Which character crowd did you perceive to be of lower quality (imposters)?

• Left

• Right

• The crowds appeared equal

Which character crowd did you prefer from an esthetic point of view?

• Left

• Right

• Neither

If you noticed something special in the scene that made one side stand out particularly, please write a short comment.

A.2.3 Questionaire - Self evaluation

In general, how hard was it to spot differences between the left and right crowds?

• 1 (Very easy)

• Easy

• Medium

• Hard

• Very hard

What part of the character do you think was the most revealing that it was of lower/higher quality? E.g feet, arms, face, the character as a whole, or whatever made you notice the difference.

Do you perceive distance or movement speed to be the

dominant factor of whether imposters are perceived as lower

APPENDIX A. USER STUDY

quality?

• Distance

• Movement speed

• Neither

If neither: What do you think differs imposters from 3D- models the most?

Were there any distances

where the imposters were particularly easy to spot? Select all alternatives where you could spot the difference most of the time.

• 1 (Near)

• 2

• 3

• 4 (Far)

• No, all distances were hard

Were there any movement speeds where the imposters were particularly easy to spot? Select all alternatives where you could spot the difference most of the time.

• 0 (Standing still)

• 1 (Slow)

• 2 (Intermediate)

• 3 (Fast)

• No, all movement speeds were hard

A.2.4 Answers

All participants’ answer data was compiled and processed in a Google

spreadsheet.

https://docs.google.com/spreadsheets/d/

1awlQv1CFSapE9Xm-6xmbB1jTng3o0noo_GDXmQrCrP4/edit?

usp=sharing

TRITA-EECS-EX-2020:387