Integrating Motion with Real-time Fur Simulated using Shells and Particles

INOM

EXAMENSARBETE DATALOGI OCH DATATEKNIK, AVANCERAD NIVÅ, 30 HP

STOCKHOLM SVERIGE 2019 ,

Integrating Motion with Real-time Fur Simulated using Shells and Particles

LEIF TYSELL SUNDKVIST

Referat

Att simulera och rendera p¨ als ¨ ar ett koncept som ofta f¨ orknippas med kom- plicerad geometri och l˚ ang renderingstid. Ibland ¨ ar det dock ¨ onskv¨ art att rep- resentera p¨ als i realtid och den h¨ ar rapporten kommer att unders¨ oka en vanlig teknik f¨ or p¨ als kallad skal. Att anv¨ anda skal f¨ or att rendera p¨ als m¨ ojligg¨ or f¨ or en ¨ overtygande illusion av volymetrisk p¨ als utan att vara beroende av ett h¨ ogt antal polygoner. Tekniken ¨ ar dock mindre l¨ ampad f¨ or fall s˚ asom l˚ ang p¨ als och av denna anledning introduceras s˚ a kallade partiklar som ett kom- plement till svagheterna med skal. Rapporten fortskrider med en diskussion kring anv¨ andningen och implementationen utav skal, partiklar och hur man

˚ astadkommer en bra blandning mellan dessa tv˚ a tekniker, med st¨ od f¨ or r¨ orelse

som baserad p˚ a objektets ythastighet. Resultatet ¨ ar en implementation av p¨ als

v¨ al l¨ ampad f¨ or realtidsbruk, i synnerhet d˚ a p¨ alsen ¨ ar l˚ ang.

Abstract

Simulating and rendering convincing fur is a concept in computer graphics

often associated with complex geometry and long rendering times. However,

there are situations in which a real-time representation of fur is desired. In

this report, we explore a technique for rendering fur for real-time applications

popularly known as shells. Fur shells provide a convincing illusion of volumet-

ric fur without being dependant on a large amount of polygons. Despite this,

the technique is less suited for cases such as when the fur is long, and so we

introduce particles as a complement to this weakness. We discuss the usage

and implementation of shells, particles and how to achieve a convincing blend-

ing between the two, supporting movement achieved by evaluating the surface

velocity of the objects. The result is an implementation of fur well suited for

real-time use, particularly in cases such as with long fur.

Words of acknowledgement

I would like to thank my supervisor Bj¨ orn Thuresson for providing me with

invaluable assistance and encouragement.

Contents

1 Introduction 7

1.1 Problem . . . . 8

1.2 Aim . . . . 9

1.3 Scope and Delimitations . . . . 9

2 Relevant theory and terminology 10 2.1 FPS . . . . 10

2.2 Niagara . . . . 10

2.3 Rendering pipeline . . . . 10

2.3.1 The Vertex Shader . . . . 11

2.3.2 The Geometry Shader . . . . 11

2.3.3 The Pixel Shader . . . . 11

2.4 RGB . . . . 12

2.5 Tangent Space . . . . 12

2.6 Tessellation . . . . 12

2.7 Quad . . . . 12

3 Prior work 13 3.1 Volume textures . . . . 13

3.2 Layered slices . . . . 13

3.3 Shells and fins . . . . 18

3.4 Procedural noise . . . . 18

3.5 Particles . . . . 19

3.6 Fur Motion . . . . 21

3.6.1 Mass Spring Systems . . . . 21

3.7 Reflection . . . . 22

4 Methodology 22 4.1 Tools and development . . . . 22

4.1.1 Hardware . . . . 22

4.2 Implementing shells . . . . 22

4.2.1 Procedurally generating fur textures . . . . 23

4.3 Particles . . . . 26

4.4 Blending particles with shells . . . . 28

4.5 Fur Motion . . . . 31

5 Results 36 5.1 Evaluation . . . . 36

5.2 Performance . . . . 36

5.3 Aesthetics . . . . 39

6 Discussion 43 7 Method Criticism 44 7.1 Blending . . . . 44

7.2 Fur motion . . . . 45

8 Future work 45

9 Conclusion 45

1 Introduction

Rendering fur is arguably one of the more costly aspects of animation in terms of rendering time. Animal fur typically consists of millions of individual strands of hair and calculating the appearance and behaviour of realistic fur takes seconds at best.

Figure 1: Rendering fur geometry is an expensive task in terms of rendering time and takes seconds at best. This mink fur hat consists of geometric hair strands and was rendered in 4.49 seconds by Andersen et. al [2].

However, whereas a film is essentially a sequence of rendered images, or frames, interactable applications such as video games require each frame to be rendered in real-time. A game running at 30 frames per second expects each frame to render in approximately 33.3 milliseconds. As such, more viable techniques in terms of rendering time are employed within game development in order to achieve a higher level of immersion for the user whilst retaining visually appealing aesthetics. One such technique is called shells, the idea of which is that a mesh is covered with a series of concentric duplicates, hence its name.

The effect is easily visualized; imagine a stack of glass panes, each decorated

with a dotted pattern. When placed atop of each other, the dots on the glass

align and suddenly give the illusion of volume. This is the fundamental idea of

shells.

Figure 2: The left image shows a series of cylinders represented by their geometry.

The right image shows the same cylinders represented using shells. Note that the number of triangles is drastically reduced, yet the illusion of volume is still present when using shells.

One of the strengths of using shells is that the number of layers may vary depending on the desired level of detail. For instance, the cylinders on the right in the image above were rendered using ten shells. This report will explore the concept of shells, how it is used and its weaknesses. More detailed information about the theory and practices behind shells along with images of shells with a higher shell count will be featured in the Prior Work section.

Aside from the aesthetics of fur, this report will also explore how to bring it to life by implementing simple motion as a result from velocity, gravity and wind.

The reason for including motion in this report is to prove that it is possible to incorporate motion with the use of particles and shells. Thus, the calculations for fur motion will not be taken into account in the evaluation process. Read more about this in the Results section.

The implementation will be evaluated side-by-side with a plugin for Unreal Engine called Neofur. Neofur is discontinued and no longer supported, but as it uses the technique of shells as well as having been a widely popular tool for accomplishing fur in game development, it is a suitable tool for comparison.

Neofur also features fur motion, albeit based on spring physics, which will be described in more detail in section 3.5.2.

1.1 Problem

Whilst the technique of shells has been proven feasible for rendering convincing real-time fur, the techniques has weaknesses that will be addressed in this report.

The main problem is that when rendering long fur, the gaps between the

shell layers become very problem. A solution to this is to simply increase the

number of shells. However, instead of using more shells, additional primitives

can be used in conjunction with shells to outbalance this weakness.

Figure 3: These two images show a sphere with fur simulated using Neofur. The left image has a fur length of 10 cm and the right a fur length of 40 cm; both spheres have 10 shell layers. Note that for longer fur, the gaps between the shells are prone to be more distinguishable.

Though the technique known as fins (see section 3.3) could be used to alle- viate this problem, in this report the aim is to investigate the use of particles instead. The hypothesis is that particles would offer a solution to similar to fins in that they complement the use of shells at certain angles but that they offer a greater degree of freedom regarding distribution and behaviour than fins do, as fins are by definition extruded along the silhouette of the mesh.

1.2 Aim

The aim of this project is to investigate the use of shells in combination with particles for real-time fur rendering as an alternative to solely using shells. A prototype based on the research in this report will be developed, capable of supporting fur motion as a result from gravity, velocity and wind.

Note that in this project the fur will be based first and foremost upon an implementation using shells together with particles. As for fur motion, instead of mass-spring systems, a simple force based on the surface velocity of the fur object will be used. This technique will be used for both the shells as well as the particles comprising the fur.

The desired outcome is an implementation that provides a visually convinc- ing simulation of fur feasible for real-time applications. Real-time in this case refers to a frame rate of at least 60 FPS. The technique of shells has by them- selves proven to be feasible for real-time use, and so this research will strive to reach an implementation that performs similarly to a native solution using only shells.

1.3 Scope and Delimitations

As this project aims towards a good representation of long fur, some clarity

regarding what is considered ”long” fur is necessary. As for Neofur’s implemen-

tation as well as the implementation developed in this project, both uses Unreal

Engine and thus the same metrics (one Unreal length unit corresponds to 1

cm). The short fur for the examples of this report is around 10 cm in length,

while long fur is around 40 cm in length. The particles in this report will have

a constant length of 40 cm.

The prototype will be delimited to simple geometry (cubes, spheres, etc) for fur rendering. In order to evaluate fur motion resulting from motion, meshes will be assigned simple armatures in the form of one or two joints. The texturing of the meshes will not be using the approach of lapped patches, as described by [5]. Instead, the meshes will be assumed to already be UV unwrapped.

Lighting will not be explored in the implementation, which will only focus on the actual implementation of shells, particles and the motion of the two.

The reason for this is that the two scenes use deferred rendering, and is related to the profiling, which will be described in the results section. Furthermore, the experiments will not include any anti-aliasing enabled. The fur pattern and colour may differ between Neofur and the implementation from this project.

There is no specific reason for this difference, but as the scope of this report is to investigate and evaluate the use of particles in combination with shells as well as the motion of fur, it does not play any relevant role.

2 Relevant theory and terminology

In this section, a couple of terms related to the field of computer graphics and the rendering pipeline will be explained. Some of these expressions will not be mentioned in the report, but are closely related to the subject and as such may prove to be of interest.

2.1 FPS

FPS, short for frames per second, is a self-explanatory term for how many frames are output at each second in an animated sequence. For instance, a movie typically runs at 24 FPS whilst a video game runs at 60 FPS. A higher FPS implies more calculations per second and thus requires more capable hardware, but provides a smoother experience to the player.

2.2 Niagara

The Niagara Editor is Unreal Engine’s toolkit for creating and managing visual effects, replacing the previous Cascade. Niagara is very powerful in the way that it allows for tailoring visual effects to a high degree as a result of being modular; when creating effects, the user may append a wide array of different parameters for controlling the behaviour and appearance of the effects [1]

2.3 Rendering pipeline

The rendering pipeline typically includes two principal shaders; the vertex shader

and the pixel shader. However, the pipeline may include the optional geometry

shader as well. This report, however, will only be discussing the general prin-

ciples and usages of the pixel and the vertex shader with the exception of the

geometry shader, which will be discussed in the Prior work section.

Figure 4: The OpenGL rendering pipeline as illustrated in The OpenGL Programming Guide, version 4.3 [11].

2.3.1 The Vertex Shader

The Vertex Shader is part of the vertex processing stage and is responsible for processing vertex data. This processing includes calculating vertex normals, transforming vertex coordinates into view space before passing them to the pixel shader or, if specified, the geometry shader. A typical example of how the Vertex Shader may be used is with bump mapping, i.e. faking bumps in the surface with the use of a texture. The sampling of the texture is done in the vertex shader and the colour of the vertex is adjusted based on whether the vertex will receive light or not after the bump map has been applied [11].

2.3.2 The Geometry Shader

After the vertex shader comes the geometry shader, an optional shader used for handling entire primitives and not single vertices. As the vertex shader only receives data in the form of single vertices, processing of faces consisting of multiple vertices is left to the geometry shader. The geometry shader receives entire primitives as input and its capabilities include manipulating vertices based on information about neighbouring vertices or creating new geometry based on the input received from the vertex shader [11]

2.3.3 The Pixel Shader

The pixel, occasionally referred to as the fragment, shader belongs to the ras-

terisation stage. Whilst the vertex shader handles individual vertices, the pixel

shader’s responsibility is to calculate the colour of each pixel on a surface that

has been rasterised and is typically based on the lighting within the scene along

with parameters inherent to the surface material. The pixel shader may also be

used to disclose whether a pixel should be drawn or not, and thus may terminate

the processing of the pixel in a process called fragment discard [11].

2.4 RGB

RGB is an abbreviation of Red, Green and Blue and is commonly used for representing colours in computer graphics. RGB values are often represented as three-dimensional vectors, with values ranging from either 0 to 1 or 0 to 255.

2.5 Tangent Space

The tangent space is the space originating from a point on the surface of a manifold. A vector in tangent space could, for instance, represent the velocity of the object at the current point [12].

Figure 5: The tangent spaces of a number of points on a M¨ obius strip manifold. Note that due to the non-orientable nature of this particular manifold, the tangent space switches direction as as the strip is twisted and united with the opposite orientation.

This image is a reference from The OpenGL Programming Guide, version 4.3 [12].

2.6 Tessellation

Tessellation is a technique within rendering for reading primitives and generating subdivided variants for subsequent processing in the rendering pipeline. For instance, the GPU handles quad primitives by tessellating them into triangles [10].

2.7 Quad

A quad is a geometric primitive essentially representing a polygon with four

vertices, hence its name.

3 Prior work

In this section, a brief overview of previous research for rendering fur will be discussed. As this report focuses on the implementation of fur in real-time, only literature and theory from such works will be referenced. In order to best describe the concept and theory behind shells, the previous research will be presented in a chronological order, starting with the early concepts that eventually became state-of-the-art today. At the end of the section a reflection will take place, where the previous research will be discussed.

3.1 Volume textures

In 1989, Kajiya and Kay proposed a technique for rendering fur using a concept denoted texels, three-dimensional data structures storing information about the surface frame as well as the lighting model. These texels were then rendered along the surface of a model using ray tracing, and resulted in highly realistic renderings of fur. The technique, however, was costly (and still is by today’s standards), and thus ill-suited for real-time applications. [4]

Figure 6: A teddy bear covered in fur rendered using Kajiya’s technique of ray tracing volumetric texels.[4]

3.2 Layered slices

As Kajiya and Kay’s concept of texels rendered using ray tracing was not vi-

able for real-time applications, the technique was later refined by Lengyel, who

simplified the concept by ”slicing” the three-dimensional data into a series of

transparent textures. These textures were then placed in concentric layers with

some spacing in-between, above the surface of the base mesh. This resulted in

an illusion where it appears as though the fur is actually geometric.[6]

Figure 7: The idea of shells is easily visualised if each layer is interpreted as a transparent sheet with some pattern on it, shown by these illustrations from Lengyel’s work on real-time fur. The image to the left shows a geometric representation of fur, and the one to the right the equivalent fur but rendered using shells [5].

As the technique of shells required different textures for each shell layer, Lengyel’s approach revolved around filtering a sample of geometric hair into a number of different textures. These textures would then be mapped onto each shell layer [6].

Figure 8: Lengyel’s approach of filtering a representation of geometric hair into a

series of 16 layers, which would then be mapped onto the shells.

To illustrate the concept and efficiency of using shells, the following para- graphs will feature a number of examples with cylinders acting as fur. The cylinders are coloured from red to green for two reasons. One is so that the depth of the strands may be more distinguishable, and the second is due to vertex colouring, used in the implementation of the prototype in this project.

For more information about the vertex colouring and implementation of shells, see section 4.3.1.

Figure 9: A set of cylinders consisting of 1600 vertices and 3100 triangles.[6]

Whilst a mesh of this scale in itself is not complicated to render, one must

once again consider the fact that fur consists of thousands, possibly millions,

of individual strands. As one cylinder in itself is comprised of 120 triangles, a

thousand of them would thus amass to 124,000 triangles. Layered slices help

alleviate this by creating an illusion of volume and cutting down the cost of

rendering triangle.

Figure 10: The cylinders on the right were rendered using the layered slices technique, in this case with ten layers, using the texture on the left. Note that the black wireframe is only present to demonstrate that the layers are simple planes with transparent textures of the cylinders as seen from above.

Rendering the cylinders using layered slices is an efficient way to give the illusion of volume while retaining a low number of polygons.

Figure 11: Another representation of the aforementioned cylinders using layered slices. In this case, the cylinders consist of 40 layers, and with a total of 80 triangles, as opposed to the mesh cylinders with 3100 triangles.

However, the technique of using layered slices had flaws in the form of gaps

between layers that became visible when viewed from a grazing angle.

Figure 12: Rendering slices of a three-dimensional object yields the illusion of volume when viewed from a distance. When viewed from an angle, however, visible gaps between the layers become apparent. This problem becomes more prominent with longer fur.

Layers along the silhouette of the model were difficult to distinguish for the viewer and to counter this, Neyret and Meyer improved the technique by introducing sets of slices in additional directions to complement the layers [7].

Figure 13: Similar to how the layered slices in the previous examples used textures of the cylinders as viewed from above, textures of the cylinders as viewed from the side (left) can be utilised to alleviate the problem of gaps between each layer (right).

Upon rendering, the angle between the view direction as well as the surface

normal was then calculated in order to determine which of the sets of slices to

render. [7]. This will be discussed in greater detail in the next section about

shells and fins.

3.3 Shells and fins

Lengyel’s use of the term shells, referring to the layered slices, was popularised and is still used today in referral to the technique [6]. He later expanded upon the addressing of visible gaps between shells along the silhouette of the mesh by formalising the use of textures with different directions, deducing that only two types of textures were required, the first type for shells and the second for so-called fins, planes rendered along the silhouette of the mesh. Whilst shells use textures representing fur as seen from above, fins required textures of fur as viewed from the side.[5]

Figure 14: Whilst the concept of fins differs from the one of shells, the sampling of the texturing is similar. Whereas shells utilise a texture sampled from a texel from above, fins utilise a texture sampled from the side. This image shows Lengyel’s illustration of how fins are extruded alongside the shells [5].

Today, the use of shells and fins is considered state-of-the-art within fur rendering as the technique has proven efficient for real-time interactions, for instance video games.

3.4 Procedural noise

Perlin and Hoffert demonstrated how procedural noise textures can be used to

simulate mesh surface topology due to its topological nature and rendered using

ray-marching. They argued that the benefits of using noise for such purposes

is that oftentimes complex surface models are awkward at best to describe, and

that noise is a useful way of approximating such complex models [8].

Figure 15: This image shows how a plane mesh (left) receives a surface model based on the topology of a noise texture (middle). The result is a plane where the geometry has been manipulated into a bumpy surface (right). These models were rendered using Blender.

Using noise for describing the surface model proved efficient in that various parameters are easily controlled, such as frequency and amplitude. The vertex offsetting is typically done in the vertex shader, where the texture is sampled and the world position of the relevant vertices are offset by an amount that corresponds to the sampled value.

3.5 Particles

This section will explain the fundamentals of particle systems. Particle systems

are often used to represent fluid, or dynamic, effects such as smoke or fire or

clouds or water. Whilst a typical geometrical object in 3D space is represented

by the bounds of its geometry, a particle system’s bound is defined by the

extent of the particles, and particles are essentially geometrical primitives such

as squads or entire meshes themselves.

Figure 16: This image depicts Reeves’ illustration of particle system with a spherical generation shape. The particles spawn inside the sphere bounds, where they may move around depending on their behaviour [9].

Whilst the behaviour of the particle system may be controlled, such as the spawn rate for the particles, the rotation of the system itself or the particle emission direction, etc, the behaviour of each particle can be controlled as well, such as the velocity, life-time (ie. time before the particle is destroyed), collision and so forth. However, aside from more dynamic effects such as smoke or fire, particles may also be used for representing fuzzy surfaces such as fur, or grass.

In such cases, the particles are spawned instantly on a given surface with an

appropriate alignment as well as a longer lifetime [9].

Figure 17: These images were made using Blender and features a cube with a simple building texture. Using Blender’s particle system, a number of particles were spawned over a plane surface, where each particle is a duplicate of the building to the left. This is a typical example of a static particle system, unlike dynamic where particles move around in a containing volume.

3.6 Fur Motion

3.6.1 Mass Spring Systems

In 2006, Banisch suggested a technique for fur and grass movement with the use of mass spring systems. The idea was that each layer of shells would be connected with springs placed on the vertices, after which the offset of each layer was calculated using explicit and implicit Euler interpolation. Addition- ally, Banisch also suggested that in order to reduce performance cost, only the vertices on the skin as well as the topmost shell layer would be fitted with spring points. As such, the offset of the layers in between would be interpolated between these two layers.[3]

Figure 18: This image shows Banisch’s idea of simulating fur motion using springs.

The vertices on the base mesh is connected to the vertices in the outermost shell

layer. The shells in the left image are displayed linearly and the shells to the right

interpolated to appear as though they bend more naturally [3].

Banisch found that simulating fur using mass-spring systems did indeed perform in real-time. The following image shows a patch of grass consisting of 64 shell layers and with a resolution of 640x480. Simulating the grass ran at around 25 FPS, but Banisch did note that most of the time went to rendering the grass shells and not calculating the motion. The simulation was run on a platform equipped with a NVidia Geforce 5700 FX, Intel P4 1.7GHz and with 512 MB RAM [3].

3.7 Reflection

Using shells does indeed seem to be a feasible technique for rendering fur in that it does not rely on complex geometry. Furthermore, it has also been proven to be suitable for simulating motion, as Banisch showed. However, it does still have weaknesses in the form of long fur, and whilst an approach to confront this problem would be to use fins, fins are restricted in the way that they are distributed and placed. For instance, fins are extruded from the base mesh and as such is dependent of the geometry of the base mesh. Particles, on the other hand, may serve the same purpose but can be distributed freely over the surface, and particle systems offer greater control over each individual particle.

Lengyel’s early work of filtering geometric hair into sequences of textures has a weakness in the form of memory consumption. A large number of shells would imply a large number of textures. However, as fur is topological in nature, an interesting idea would be to, instead of relying on multiple textures, combine the use of noise textures with the use of shells. The implementation in this report will be based on the research covered in this section. Read more about how these techniques are used in the Methodology section ahead.

4 Methodology

4.1 Tools and development

The development and implementation in this project will be done using Unreal Engine 4. Unreal Engine 4 features a graphics as well as physics engine. As such, this project will focus on implementing the technique of shells and particles using Unreal’s built in shader language as well as its Niagara particle system tool.

4.1.1 Hardware

The implementation and benchmarking was performed on a computer running Windows 10 64-bit with a NVidia GeForce GTX 1080 GPU, an Intel Xeon CPU E5-1650 3.60GHz and 64 GB RAM.

4.2 Implementing shells

Before delving deeper into the implementation of the shells, it is important to

disclose that the shells will not be generated during runtime of the fur simu-

lation. Instead, it is assumed that the input mesh already consists of a base

mesh along with a number of submeshes representing the shells, where each

layer of shells are assigned a vertex colour of their own. The mesh beneath the

layers of shells is referred to as the base mesh, whilst the mesh that the shell layers are based on is called the growth mesh. The reason for having a base mesh and a growth mesh is that the base mesh may be a more highly defined mesh consisting of more polygons. The growth mesh, on the other hand, serves only as a mesh for representing the fur using transparent textures and so does not require as high an amount of faces. The purpose of the vertex colouring in within the growth mesh and thus the shell layers is to be able to deduce at which offset from the base mesh that the shells will be placed. In this project, the height will be based on the red channel of the vertex colour, and so so the vertex colour for the red channel will vary in the range of 0 to 1. For clarity, this value will henceforth be referred to as c v .

Figure 19: A base mesh with four shells, where each shell is assigned a red vertex colour value ranging from 0 to 1.

In this project it is assumed that the vertex colouring increases linearly between each layer. As such, the layers in a mesh consisting of a total of four shells would thus be represented by the values 0.25 for the innermost layer, 0.5 for the second, 0.75 for the third and 1.0 for the topmost layer. These values will be of significance in the following sections.

4.2.1 Procedurally generating fur textures

Furry surfaces are often complex in nature and determining a surface model

for describing fur is a tedious task, and using multiple textures for each layer

would consume an unnecessary amount of memory. For this reason, it is conve-

nient to instead utilise textures representing perlin noise, with their topological

properties, for the same purpose [8].

Figure 20: Procedural noise is a useful technique for representing three-dimensional data in a two-dimensional format due to its topological nature.

Procedural noise was used in combination with the vertex colouring of the shells to produce the illusion of topology. For example, by letting each pixel on the surface of the shells sample the noise texture at that point, one may achieve a convincing illusion of a procedurally deformed surface. Let o represent the opacity at a point p, c v the vertex colour and n c the sampled noise value at a given point. To determine whether or not a point should be visible or not, the point’s opacity p is given by

o =

( 1, if n c − c v > 0 0, otherwise

Applying this calculation on a series of shell layers yields the illusion of a

bumped surface. The image below shows a mesh consisting of a series of shells,

where each shell layer samples the noise texture, gradually becoming more and

more transparent the higher the shell is offset from the base mesh.

Figure 21: This image shows a plane with 16 shells and rendered with a gradually increasing transparency. The higher up the shells are offset from the base mesh (the bottom-most layer), the more masked the texture becomes. The red and green colours represent the vertex colours of the shell layers. This image was rendered using Blender.

This technique may thus be applied to meshes for representing furry surfaces.

It is important to note that fur consists of longer strands, and so the shell offset

should be increased. Furthermore, the tile scaling of the texture should be

increased as well, resulting in thinner bumps, or strands. The tile scaling may

be adjusted; a higher tile scaling, i.e. thinner strands would perhaps be more

suitable for more realistic fur.

Figure 22: A sphere with a number of 32 shells and a fur length of 10 cm. Note how the strands grow thinner towards the top; a direct result of the aforementioned calculation for the opacity o, where the vertex colour, which could be interpreted as the offset of each shell, determines how much should be ”peeled off” from the opacity for each layer.

4.3 Particles

The fur particles were implemented using Unreal Engine’s Niagara particle sys-

tem, and randomly placed over a designated region of the mesh. In the following

examples, this mesh will be represented by a simple sphere. Each particle is rep-

resented by a sprite with a masked material of a tuft of hair as viewed from the

side. The particles are set to always be aligned with the viewer, i.e. adjusting

their rotation around their surface normal axis.

Figure 23: The left image shows a texture of fur from the side. This texture is in turn used as a opacity mask for the particles spawned around the surface of the mesh. The image to the right shows the distribution of the particles over the surface.

Note that the particles are rendered using their black-and-white image for more clearly demonstrating their purpose.

As the texture of the fur used for the particles is black and white, it is suitable for use as a transparency mask for the particles. The particles in the following examples have been appropriately assigned a colour matching the colour of the fur shells.

Note that the automatic alignment of the particles results in a problem that

when viewing a surface with particles from straight above, it becomes apparent

that the particles are simple planar primitives. Similar to how shells has an

apparent weakness in the form of visible gaps along the silhouette of a mesh,

this too is a weakness of particles when viewed from above.

Figure 24: Using only particles by themselves yields good results when viewing the object along the silhouettes, but when viewed from an angle perpendicular to the surface the illusion of volume is lost when it instead becomes apparent that the hair particles are simple quads.

In order to attain a good compromise between the two techniques, a com- promise is required that will be described in more detail in the next section.

4.4 Blending particles with shells

A blending between particles and shells can be achieved by taking into consider- ation the viewing angle from the camera. The basic idea is that the fur particles will only be visible on regions of the mesh that is observed from grazing angles.

Consider a direction vector v _eye pointing from the surface of the mesh to- wards the viewer and the surface normal vector v _n . The opacity o _p of the particles can be controlled using the following statement

o p = (v eye · v normal ) ^k

(1)

where k p is a constant representing the rate at which the opacity fades for

the particles. The higher the value of k p , the more the particle’s opacity will

fade when approaching the silhouette of the mesh.

Figure 25: As the value of k

increases, the less particles will be displayed towards the center of the mesh. The leftmost sphere was rendered with a k

value of 2, the middlemost 4 and the rightmost 8.

It is important to note, however, that due to the billboard nature of the par- ticles (they maintain their rotation so as to always be aligned with the viewer), the value of k p should be considered with caution. Should the object rotate around the view vector, for instance, the particles closer to the silhouette of the mesh will ”orbit” along a larger radius and so will align towards the viewer at a slower rate. The particles closer to the center, on the other hand, will rotate along a smaller orbit and align at a quicker rate which may be obvious to the viewer.

As opposed to particles, fur shells will not be rendered along the silhouette of the mesh, and will instead only be shown in areas not in the proximity of the silhouette. Similar to equation (1), the opacity o _s of the shells can be controlled in the following fashion

o _s = 1 − (v _eye · v _normal + c _s ) ^k

(2)

where k s is a constant representing the strength of the opacity fade for

the shells. The higher the value of k s , the more the opacity will fade as the

perpendicularity between the surface normal and the viewing angle becomes

smaller, i.e. the perimeter of the area will become sharper. The value c s is a

constant value used for adjusting the size of this region.

Figure 26: As the value of k

increases, the less the shells will be visible towards the silhouette of the mesh. The leftmost sphere shows the result of using a k

value of 1, the middlemost 2 and the rightmost 4. The constant c

was omitted, i.e. set to 0 in these examples.

When adjusting any one of the constants k _p and k _s , the other should be taken

into consideration in order to achieve as natural a blend as possible. In other

words, a good initial approach for achieving a natural blend would be to fade the

opacity of the shells towards the silhouette of the mesh whilst simultaneously

fading the opacity of the particles towards the centre of the mesh with a similar

but inverse rate.

Figure 27: This blend between shells and particles was achieved by using the values of k

= 2, k

= 1.5 and c

= 0.15.

4.5 Fur Motion

Fur physics and motion was achieved by evaluating the velocity of the fur ob- ject at each pixel and applying a bending force proportional to this velocity.

However, Unreal Engine provides no trivial way of evaluating the velocity, and so the steps taken to achieve this will be described in this section.

The following paragraphs detailing the implementation of the fur motion was

implemented using a basic sphere mesh. The following image shows a wireframe

representation of the sphere as well as its UV layout.

Figure 28: The sphere used during the implementation of the fur motion and its corresponding UV layout.

In order to sample the surface velocity of the pixels of the sphere, it is assigned an emissive material where the emissive colour of each pixel corresponds to its world position. The reason for choosing an emissive value is that it permits values outside of regular diffuse colour intervals, as all values outside of this range is instead treated as the intensity of the emission. This procedure is repeated at each frame.

Aside from storing the velocity of the object in this fashion, the texture containing the previous frame’s positional values is stored and subtracted from the texture containing the current frame’s positional values.

Figure 29: The two images above represent the UV map of the sphere where each pixel represents the world position in the form of emissive RGB values. The left image represents the sphere located at the centre, (0,0,0), whilst in the right image the sphere has been moved one positive unit along the global Y axis, located at (0,1,0). In the right image, note that the green values in the colours have increased.

Subtracting the values between these two textures would thus yield a veloc-

ity at each pixel as the difference in colour represents the difference in world

position. The result is a texture coloured by the difference in world positions.

Figure 30: This image represents the change in position, i.e. the velocity of each pixel for the sphere using RGB colours for each pixel. The reason for the green colour is explained by the fact that the sphere has been moved a distance of one unit in the Y direction in world space corresponds to a distance vector of (0, 1, 0) in XYZ space, which translates into an RGB colour of (0, 1, 0); green.

When this positional difference has then been calculated in the vertex shader, it is then divided with the frame delta time after which a velocity can be derived.

This velocity may then be used in combination with other forces, such as wind, or gravity, and added together to form a combined force vector that can be applied to the fur. It is desireable that the fur strands do not bend in a linear fashion, and so the rate at which they bend will be based on the height of each layer. For this purpose it is useful to use the vertex colouring c v as a substitute for the layer of each shell. The force F applied to each vertex is thus

F = (F _v ∗ s v + F _g ∗ s g + F _w ∗ s w ) ∗ c ^k _v (3) where F _v represents the force of velocity, F _g gravity and F _w wind. s _v , s _g and s _w are scalar values denoting the scale at which the aforementioned force vectors should be applied. Furthermore, c _v is once again the vertex colour of each layer and k is a scalar value for determining the bend factor of the strands. This calculation results in an effect where the higher the shell layer, the stronger the bend will become. Shells close to the base mesh will be offset by a less amount, whilst the outermost layer will be offset the most. The resulting force vector is then added to the world position offset of the vertex in the vertex shader.

As an example of this effect, imagine the scenario where the fur is only

affected by the force of velocity, i.e. F = (F v ∗ s v )c ^k _v . For simplicity’s sake, let’s

denote F v = (1, 0, 0) and s v = 1. The following images represent the effect of

applying no velocity at all versus the cases where k varies.

(a) k = 1 (b) k = 2

Figure 31: In image a) a linear offset can be observed when the value of k is 1. In b), where k > 1 the fur is offset in an exponential fashion, i.e. bending more when closer to the tip and less when close to the root.

Figure 32: In this image with a wireframe representation the sphere is rotating

around its local X axis. In this example, the fur is only affected by the velocity

as a result of the spheres rotation. Note that the offset, or bending, of the layers

increases closer to the outermost shell layer. This sphere uses 64 shells for more

clearly demonstrating the shell behaviour, and the particles are not included in this

image.

Figure 33: This image shows the sphere moving in an oscillating motion, from left to right. The particles have been omitted so that the motion of shells can be more easily observed. The right image shows a wireframe representation of the same sphere and motion.

Applying this motion to the particles results in a motion similar to the one of the shells. However, there is one limitation to the behaviour of the particles. Because the particles are represented by quads consisting of two triangles, four vertices in total, and the motion is based on offsetting the vertices, the displacement for the particles result in a sheering behaviour.

Figure 34: Particle motion as a result of the surface velocity of the object. The left image shows the particles moving as a result of rotation, and the image to the right as a result of the sphere moving back and forth in an oscillating motion. The shells are disabled in case to more clearly demonstrate the particle behaviour.

This shearing can be alleviated by having particles consisting of more ver-

tices, for instance by using tesselation, as illustrated by the image below. The

implementation of bending particles, however, will not be addressed within the

scope of this project.

Figure 35: By tessellating particles and introducing more vertices, the particle may be bent in the same manner that the shells are; vertices closer to the tip bend stronger than the ones at the root.

5 Results

5.1 Evaluation

Aside from development, Unreal Engine 4 was used in this project for evaluating the implementation in terms of aesthetics and performance. As a state-of-the- art baseline, an Unreal Engine plugin known as Neofur was used. Neofur, albeit today discontinued, is a plugin used for creating real-time fur suitable for video games using the shells technique and features fur motion based on mass-spring systems.

The evaluation for Neofur was performed using Unreal Engine version 4.9, and the evaluation for the implementation created in this project using version 4.21.

Before delving into the performance section, it is important to disclose that the performance analysis will be based solely on the rendering aspect of the fur.

The reason for this is that this report is mainly about the effects of combin- ing shells with particles. Furthermore, experiments performed during testing showed that the performance loss of incorporating fur motion as described in this report was insignificant when compared to the cost of the actual rendering, despite varying the texture resolution of the motion textures.

5.2 Performance

The following statistics were gathered using RenderDoc, a tool commonly used

for GPU profiling. RenderDoc works by capturing frames from a running ap-

plication and by analysing how long it took for every draw call to be made.

Figure 36: RenderDoc is a powerful tool for GPU profiling, capable of displaying the time it took for various draw calls to be made.

The image above shows the RenderDoc output from a captured frame from the scene used in the tests. For calculating FPS in this report, the Scene draw call will be considered, as it is the draw call responsible for drawing all the objects within the scene. As an extra measure, the BasePass draw call will be considered as well, though in milliseconds. The BasePass draw call is the call in which material properties such as roughness, diffuse as well as metallic colour are drawn to the G-buffer, after which the lightning will be calculated at a later stage. However, this report is once again not focusing on the lighting aspects of the simulation and as such those stages will be omitted for the BasePass parameter. The reason for considering the two metrics FPS and BasePass is that there may be differences in the two versions of Unreal that causes the Scene call to differ more or less, but the BasePass only considers the sphere so that may be more just. However, solely depending on the base call as a metric for FPS would not be deemed appropriate, as a BasePass is often significantly faster to render than the whole scene in itself.

As the two scenes that will be evaluated in the bench-marking is assumed to be identical in terms of polygon counts and lighting, so is it assumed that comparing the time taken for the Scene draw call is a just comparison. The scenes were comprised of only a spot light and the spheres in question.

Note that RenderDoc does not guarantee exact timing results and that the results may vary depending on the machine on which the tests were conducted.

More about this will be discussed in the further work section.

The tests were made with a sphere represented by a base mesh consisting of 960 triangles and a growth mesh of 960 triangles. Each particle consists of two triangles. This means that a sphere with 32 shells and 500 particles would be comprised of

960 + 32 ∗ 960 + 500 ∗ 2 = 32680 (4) triangles in total.

The fur noise texture used for both the solution from this project and Ne-

ofur’s had a resolution of 512x512 pixels. Aside from this resolution, material

properties such as specularity, ambient occlusion, roughness, etc, were equal between the two scenes.

The following graphs shows different test results from the implementation in this project with varying numbers of shells and particles in tandem with the solution using Neofur. Note that the evaluation using shells and particles will use 10, 32 and 64 shells and up to 2000 particles, whilst the Neofur solution will compensate for the lack of particles by using up to 128 shell layers. The bench-marking results will be discussed in terms of aesthetics in the Aesthetics section ahead.

0 10 32 64 128

100 150 200 250 300 350 400

Number of shells

F rames p er second (FPS )

Benchmarking fur using shells and particles versus Neofur’s solution

0 particles 500 particles 1000 particles 1500 particles 2000 particles

Neofur

From this graph it can be observed that the average frame rate decreases

as the shell and particle count increases. An interesting note about the test

results is that the difference in performance is wider when the shell count is

low, in contrast to when the shell count is 64 when it can be observed that the

FPS seems to converge around 180 FPS. Neofur’s solution performed generally

better with the only exception being with 0 particles and 10 shells, where Neofur

was observed with a performance of 357 FPS and the solution using shells and

particles performed with 370 FPS.

0 10 32 64 128 0

0.5 1 1.5 2 2.5

Number of shells

BaseP ass (ms)

Benchmarking fur using shells and particles versus Neofur’s solution

0 particles 500 particles 1000 particles 1500 particles 2000 particles

Neofur

Evaluating the BasePass performance yielded similar results as the FPS evaluation. However, note that the Neofur solution performed as well with 10 shells as it did with 32 shells. The reason for this was never discovered and is likely due to small differences in performance that RenderDoc was not capable of distinguishing between. Aside from this, Neofur once again performed better than the solution using shells and particles, with the difference being with 10 shells and 0 particles (0.45 ms) versus Neofur with 10 shells (0.89 ms).

In summary, the fur rendered using Neofur proved to yield a generally lower BasePass value and higher frame rate. Despite being generally slower than Neofur’s solution, these bench-marking results show that it is indeed feasible to use particles in combination with shells for achieving real-time performance.

5.3 Aesthetics

As the problem of long fur has already been established in this report, and thus

made clear that long fur without the use of particles requires a high amount of

shells, the examples hereafter that were simulated using Neofur will feature 128

shells, rendered with an average of 139 FPS as according to the bench-marking

evaluation in the previous section. The solution using shells and particles will

consist of 10, 32 and 64 shell layers and 1500 particles. In addition to the

visual comparison, and evaluation of the performance of each example will be

performed.

Figure 37: The left image sphere was simulated with 64 shells and 1500 particles.

The right sphere was simulated using Neofur and with 128 shells.

The appearance of the fur using particles provides a comparable represen- tation of long fur, when compared to the result made using Neofur. The image above shows that a number of 64 shells almost eliminates the gaps between the shell layers, and gaps that persist are complemented by the introduction of particles. Though both alternatives both look aesthetically convincing, consider the fact that the solution using 64 shells and 1500 particles was rendered with an average FPS of 185, in contrast to the Neofur solution with an average of 213 FPS; Neofur thus performed approximately 15% better in terms of FPS.

Lowering the number of shells even further also proved to yield convincing results, as can be seen in the image below.

Figure 38: This is the same image as the previous one, but the sphere to the left has been replaced with a sphere consisting of 32 shells.

Reducing the number of shells from 64 to 32 yields is barely noticeable.

Once again, the particles around the silhouette cover up the gaps between the

individual shell layers. Using 32 shell layers along with 1500 particles, averaging around 222 FPS, performed quite similarly to the solution from Neofur with 128 shell layers, with an FPS increase of approximately 4%.

Figure 39: The same image as the previous ones, but the left sphere now uses 10 shells.

Rendering the sphere with 10 shell layers and 1500 particles performed with an average of 253 FPS, thus performing approximately 19% better than Neofur in terms of FPS. When using 10 shells, however, gaps become visible. However, it is not the gaps at the silhouette of the fur that is visible, but instead along the boundary where the particles are starting to become visible. Thus, the problem of long fur persists, which will be discussed in greater detail in the conclusion section.

When using particles, it is important that the amount of particles specified

is high enough so that it is dense enough that the blending between particles

and shells is good. Too few particles will also result in uneven shading between

the shells and the particles. The following image shows the effect of using too

few particles.

Figure 40: It is important to consider the amount of particles lest gaps and missing patches of fur will be visible. This fur was rendered using 200 particles.

As for the motion, the solution based on surface evaluation proved convincing

as well, with a behaviour very similar to the motion exhibited by Neofur and

using springs. However, the motion for the particles does not look quite as

convincing, as the particles do not bend in a fashion such as the shells do. More

about this will be discussed in the Future work section.

Figure 41: The fur motion using shells and particles (left) compared to the fur motion using mass-spring systems (right).

6 Discussion

The desired outcome of this project, as stated in the Aim section, was to imple- ment a visually convincing simulation of fur suitable for real-time applications, and with a similar performance to the solution using only shells. Whilst the test results showed that the solution using shells and particles did not perform quite as well as the one that Neofur provided, they did show that it is indeed reasonable to consider using particles for long fur in real-time, and so the prob- lem is deemed to have been addressed as none of the tests exhibited frame rates of lower than 150 FPS.

As for using particles together with a low shell count, as shown in the results

section, one could see that the problem of gaps between layers persisted despite

the attempt to blend the shells with the particles. To remove the gaps com-

pletely is a challenging task, and though one may be tempted to use as few shell

layers as possible, one should recall that when compared to actual geometric

objects, shells generally allow for a much lower poly count. A shell count of

around 32 may very well be suitable for most types of fur, even longer ones;

with particles one can achieve a natural blend between the two in a similar to way to how fins behave, and particle systems are easily customised and very flexible. The implementation of fur motion in this project also proved that par- ticles may as well be subject to motion as shells are, and that the two techniques may move naturally with one another. One interesting aspect of achieving fur motion using motion textures, as described in this report, is one texture may be used for multiple instances of fur. One fur instance requires two motion textures, one for the past state and one for the current state and each with the dimension of 512x512 pixels. Consider the idea that instead using textures of size 1024x512 pixels; now two fur instances may reference one half of the texture by indexing. This is an idea that could possibly save performance in terms of all the additional parameters required to store image files. Ten fur instances would thus together share a texture of resolution 5120x512 pixels.

The reason for why the results differed so prominently between the solution using shells and particles in contrast to Neofur’s solution was never determined.

In theory, the implementation of shells within this project and Neofur’s imple- mentation of shells should not differ in any major way. There may be a wide array of factors contributing to the differences in performance, factors that along with the reliability of RenderDoc as a bench-marking tool were never investi- gated prior to conducting the tests.

As most images of fur in this report represent fur from up close, the problem of gaps might not be as evident when viewed from afar. Furthermore, the tiling of the fur is yet another parameter that can be tweaked and adjusted in an attempt to alleviate this. However, larger surfaces of fur with a silhouette that is generally not visible does not depend as much upon the use of particles as surfaces where the silhouette is more exposed. For such surfaces, the sole use of shells would suffice. These are all factors that allow for tweaking. For instance, an interesting idea for character artists in game development would be to use a mix between solely shells in some regions of the character mesh, perhaps regions with shorter fur, and shells together with particles in other areas with longer fur.

Lighting, glossiness and other material parameters were never explored within the scope of this project; such parameters may as well be explored further with particles in order to achieve a more natural representation of fur. Parameters such as height, hair flow, density and root and tip colouring can for instance be controlled with the introduction of appropriate textures.

7 Method Criticism

This section will discuss aspects of the methodology of the prototype that may yield unexpected or even unwanted behaviour, along with appropriate counter- measures.

7.1 Blending

The blending between particles and shells was done without any culling whatso-

ever. This means that even though shells or particles may not be visible, these

triangles are still processed and rendered transparent by the shader. A possibly

better approach would be to implement an actual culling of these regions; tri-

angles that are not visible or should not be visible at all are completely omitted by the shader. This could possibly have resulted in a performance gain of some degree, but was never explored in the scope of this project.

7.2 Fur motion

The fur motion in this project was implemented without any restrictions to how long the fur can stretch. This means that if the difference in world position between frames results in a velocity that is very high, i.e. the object is moving a large distance in a short span of time, the resulting motion on the fur would be quite large. This was never addressed in this project, but could be alleviated by introducing some mechanism for clamping the world position offset of the vertices in the shader. Furthermore, as stated in the methodology section, the fur particles were not tessellated in any way which resulted in the particles bending in a linear fashion. If these particles would consist of more triangles, the vertices could be offset in a fashion more similar to the shells, and thus appear to move even more naturally together with the shells.

8 Future work

Using particles for simulating fur introduces another aspect of customising the fur furtherly. Similar to how shells may use different types of noise textures for representing different patterns of fur, particles may utilise different texture masks. For instance, particles may use textures of more sparse fur, or more dense. In addition to this, anti-aliasing may also yield interesting results for the particles. As textures representing hair are typically high-detailed in order to best represent the fine strands, these details result in pixelated strands when rendered. To alleviate this, anti-aliasing could be utilised; however, anti-aliasing introduces an additional post-processing cost and as such should be used with this in mind.

Note that the fur in this report consisted of a furry sphere. It would be interesting to further elaborate on this research by experimenting on other types of meshes as well, including more organic meshes such as animals.

For more convincing particle motion it is advised to tessellate the particles further in order to have them bend more naturally. In this report the particles were not subdivided in any way, and so they sheered rather than bent.

As for the problem of long fur that persisted despite the use of particles, one could utilise a technique called shell bias. A shell bias is simply the process of distributing the shells using a mathematical formula. For instance, a shell bias could be used for having a dense distribution of shells along the tips and more sparse closer to the base mesh. This could, perhaps, alleviate the problem of long fur further, as it is the tips of the fur that is prone to gaps, and not the layers close to the base mesh.

9 Conclusion

The results from this project show that it is indeed feasible to use a combination

of shells and particles for real-time purposes, and that the technique of using

surface velocity evaluation for fur motion provides a convincing representation of fur motion. However, as the the particles were not tessellated in any way, they were also incapable of bending. As such, whilst the shells bent in a natural way, the particles did not and as such did not look as convincing.

Using shells and particles were proven to be more costly than solely using shells, as displayed by the implementation from Neofur. There may be multiple reasons for this, but it is most likely because the material properties between the solution in this project and the one from Neofur differs, as the calculations for fur motion are considered insignificant in comparison with the rendering and, as disclosed in the discussion section, due to irregularities concerning the bench-marking using RenderDoc.

The problem of long fur was alleviated by the introduction of particles.

However, using too few shells still resulted in gaps between the layers that were

not blended with the particles. As such, the problem of gaps between the layers

persists, but this project has proven that it may be remedied.

References

[1] Niagara editor. https://docs.unrealengine.com/en-us/Engine/Niagara. Ac- cessed: 2019-03-25.

[2] Tobias Grønbeck Andersen, Viggo Falster, Jeppe Revall Frisvad, and Niels Jørgen Christensen. Hybrid fur rendering: combining volumetric fur with explicit hair strands. The Visual Computer, 32(6):739–749, Jun 2016.

[3] Sven Banisch. Making grass and fur move. Jan 2006.

[4] J. T. Kajiya and T. L. Kay. Rendering fur with three dimensional textures.

SIGGRAPH Comput. Graph., 23(3):271–280, July 1989.

[5] Jerome E. Lengyel, Emil Praun, Adam Finkelstein, and Hugues Hoppe.

Real-time fur over arbitrary surfaces. In 2001 ACM Symposium on Inter- active 3D Graphics, pages 227–232, March 2001.

[6] Jerome Edward Lengyel. Real-time fur. In Bernard P´ eroche and Holly Rushmeier, editors, Rendering Techniques 2000, pages 243–256, Vienna, 2000. Springer Vienna.

[7] Alexandre Meyer and Fabrice Neyret. Interactive volumetric textures. In George Drettakis and Nelson Max, editors, Rendering Techniques ’98, pages 157–168, Vienna, 1998. Springer Vienna.

[8] K. Perlin and E. M. Hoffert. Hypertexture. In Proceedings of the 16th An- nual Conference on Computer Graphics and Interactive Techniques, SIG- GRAPH ’89, pages 253–262, New York, NY, USA, 1989. ACM.

[9] W.T. Reeves. Particle system-a technique modeling a class of fuzzy objects.

ACM Transactions on Graphics, 17:359–376, 01 1983.

[10] Mark Segal and Kurt Akeley. The OpenGL Graphics System: A Specifica- tion. The Khronos Group, Inc., Mar 2010. Version 4.0.

[11] Dave Shreiner, Graham Sellers, John M. Kessenich, and Bill M. Licea-Kane.

OpenGL Programming Guide: The Official Guide to Learning OpenGL, Version 4.3. Addison-Wesley Professional, 8th edition, 2013.

[12] Michael D. Spivak. Calculus on manifolds: A modern approach to classical

theorems of advanced calculus. 1965.

TRITA -EECS-EX-2019: 450