Självständigt arbete på avancerad nivå
Independant degree project - second cycle
Datateknik
Computer science
Communication tool in virtual reality – A telepresence alternative
An alternative to telepresence – bringing the shared space to a virtual environment in a virtual reality
Marcus Ekström
MID SWEDEN UNIVERSITY
Department of Information Systems and Technology (ITS) Examiners:
Mikael Gidlund, mikael.gidlund@miun.se Internal Supervisors:
Stefan Forsström, Stefan.Forsström@miun.se External Supervisor:
Johan Moritz, Johan.Moritz@xlent.se Author:
Marcus Ekström, maek1212@student.miun.se Degree programmes:
Master of Science in Computer Engineering Main field of study:
Computer Engineering Semester, year:
Spring, 2017
Abstract
Videoconferencing is one of the most common telepresence methods today and educational videos is rising in popularity among distance learners. Traditional videoconferencing is unable to convey gestures and mutual eye contact between participants. This study aim to propose a Virtual Reality telepresence solution using game engines. The system was designed with Unreal engine 4 based on a platform comparison literature study with the game engines Unity 3D and Unreal Engine 4 in focus.
A literature study confirmed the effectiveness achieved in VR is comparable to the effectiveness in face-to-face meetings. The suggested solution implements whiteboard functionality from a real-life perspective, confirming it is possible to include new functionality and directly transfer old function- ality to the VR system from the communication systems today. The system was evaluated based on the response time, packet loss, bandwidth, frame rate and through user tests. The evaluation shows it is possible to design a telepresence system with VR capable of passing the Turing Test for Telep- resence. The participants of the user tests did not experience discomfort and they were positively inclined to the telepresence system. Though, discomfort may emerge if the VR character is used with a common office workstation. Future studies in this topic would involve modifications of the third person camera, making the head’s rotation follow the direction of the camera view and implementing movable eye pupils on the VR character using the upcoming eye-tracking accessory.
Keywords: Telepresence, Virtual Reality, VoIP, Unreal Engine 4, Conference, Presentation, Head
Mounted Display, HTC vive, Communication tool
Acknowledgements / Foreword
Thanks to Jordan Dobosc for contributing with a 3D office model and YuriNK for providing with the code for the IK-body without networking support.
• For supporting with the theoretical parts, thanks to:
– Stefan Forsström (Stefan.Forsstrom@miun.se) – Johan Moritz (Johan.Moritz@xlent.se)
– Christffer Öhman (Chrisoffer.ohman@xlent.se)
• For the support with implementation questions regarding VR in Unreal Engine 4 in general, thanks to:
– Morpho aka VizualTek (https://forums.unrealengine.com/member.php?471345-VizualTek).
– MordenTral (https://forums.unrealengine.com/member.php?4285-mordentral).
Contents
Abstract ii
Acknowledgements / Foreword iii
Contents iv
List of Figures vi
Listings vii
1 Introduction 1
1.1 Background and Problem Motivation . . . . 1
1.2 Overall Aim . . . . 2
1.3 Research Questions . . . . 2
1.4 Scope . . . . 2
1.5 Contributions . . . . 2
2 Theory and Literature Study 4 2.1 Telepresence . . . . 4
2.2 Virtual Reality . . . . 4
2.2.1 Virtual Reality Best Practises . . . . 5
2.3 Virtual Reality Hardware . . . . 5
2.3.1 HTC Vive . . . . 5
2.3.2 Oculus Rift . . . . 6
2.3.3 PlayStation VR . . . . R 6 2.3.4 Comparison Summary . . . . 6
2.4 Steam Subsystem . . . . 7
2.5 Inverse Kinematics . . . . 7
2.6 Evaluation of Quality of Experience . . . . 7
2.7 Related Work . . . . 7
3 Methodology 9 3.1 Approach . . . . 9
3.2 Prototype Assessment . . . . 10
3.3 Tools . . . . 10
3.4 Research Methodology . . . . 10
3.5 Implementation Methodology . . . . 11
3.6 Evaluation Methodology . . . . 12
3.6.1 Response Time and Packet Loss . . . . 12
3.6.2 Bandwidth . . . . 13
3.6.3 Frame Rate . . . . 13
3.6.4 User Tests . . . . 14
4 Platform Comparison 15 4.1 Areas of Interest . . . . 15
4.2 Unreal Engine . . . . 15
4.2.1 Memory . . . . 16
4.2.2 Networking . . . . 16
4.3 Unity 3D . . . . 16
4.3.1 Memory . . . . 17
4.3.2 Networking . . . . 17
Contents
4.4 Comparison Summary . . . . 19
4.5 Game Engine Choice . . . . 19
5 Prototype Implementation 21 5.1 Main Menu . . . . 21
5.1.1 Create Conference . . . . 22
5.1.2 Find and Join Conference . . . . 22
5.2 Player Controller . . . . 23
5.3 Player Controllable Pawn . . . . 23
5.3.1 Inverse Kinematics Application . . . . 23
5.3.2 Creating Interactable Cubes . . . . 24
5.3.3 Painting on Interactable Cubes . . . . 25
5.3.4 Positioning Interactable Cubes . . . . 25
5.3.5 Destroying Interactable Cubes . . . . 26
5.3.6 Teleport . . . . 26
5.4 Measurements Setup . . . . 26
5.4.1 Response time . . . . 26
5.4.2 Bandwidth . . . . 26
5.4.3 Frame Rate and User Tests . . . . 26
6 Results 27 6.1 Prototype . . . . 27
6.2 Evaluation . . . . 29
6.2.1 Response Times and Packet Loss . . . . 30
6.2.2 Bandwidth . . . . 31
6.2.3 Frame Rate . . . . 31
6.2.4 Usability Test Results . . . . 33
7 Discussion 35 7.1 Prototype . . . . 35
7.2 Evaluation . . . . 35
7.2.1 Response Time and Packet Loss . . . . 35
7.2.2 Bandwidth . . . . 35
7.2.3 Frame Rate . . . . 36
7.2.4 Quantitative Evaluation . . . . 36
7.2.5 Usability Test Results . . . . 36
7.3 Ethical Aspects . . . . 36
8 Conclusions 37 8.1 Future Work and Recommendations . . . . 38
References 39
A Source code over function: TwoBoneIK 44
List of Figures
2.1 Overview of the HTC Vive’s motion controller buttons . . . . 5
2.2 Overview of the Oculus Rift’s Touch controller buttons . . . . 6
3.1 Research work flow overview . . . . 11
3.2 Implementation work flow overview . . . . 12
3.3 Response time and packet loss work flow overview . . . . 13
3.4 Bandwidth work flow overview . . . . 13
3.5 Frame rate work flow overview . . . . 13
3.6 User tests work flow overview . . . . 14
4.1 Performance usage with Unreal Engine 4 on standby . . . . 16
4.2 Replication using the blueprint editor in Unreal Engine 4 . . . . 17
4.5 Replication in Unity3D using SyncVars . . . . 17
4.3 Remote Procedure Calls in Unreal Engine 4 using blueprints . . . . 18
4.4 Performance usage with Unity on standby . . . . 18
4.6 Remote Procedure Call definition in Unity3D using C# . . . . 19
4.7 Remote Procedure Call in Unity3D using C# . . . . 19
5.1 Network tunneling through steam’s server to circumvent firewall blockages . . . . 21
5.2 Overview of the system mechanics . . . . 21
5.3 Flowchart of the Refresh Conferences macro . . . . 22
5.4 Switching character on the client-side . . . . 23
5.5 Illustration explaining the Inverse kinematics variables . . . . 24
6.1 Main menu presented directly to the user upon startup . . . . 27
6.2 Create conference view in the application . . . . 28
6.3 Find conference view in the application . . . . 28
6.4 Inside a conference room in the application . . . . 29
6.5 Virtual Reality character with a client in the background . . . . 29
6.6 Response times to client for 1-3 connected clients to a server with standard deviation as error bars . . . . 30
6.7 Packet loss in percentage for a client with 1-3 connected clients to a server . . . . 30
6.8 Upload bandwidth for 1-3 connected clients with standard deviation as error bars . . . . 31
6.9 Download bandwidth for 1-3 connected clients with standard deviation as error bars . . . . . 31
6.10 Timings with the lenovo laptop as reference . . . . 32
6.11 Timings with the VR-computer as reference . . . . 33
6.12 Response results from the modified version of the Post-Study System Usability Questionnaire from 0 - 6 where 0 is strongly disagree and 6 is strongly agree. The full question description is presented in Table 6.13 on page 34 . . . . 34
6.13 The question description corresponding to the questions in Figure 6.12 on page 34 . . . . 34
Listings
Listings
A.1 Function: TwoBoneIK . . . . 44
1 Introduction
Advances in communication systems have come far, videoconferencing is now being taken for granted.
Videoconferencing is one of the most common telepresence methods today [1], the concept of telepresence is to give the user the feeling of being in a different location [1]. Communication systems is applicable in every field from education to relationships, business and sports to emergency situations. Communication solutions are essential to enable independent lifestyles, particularly for the elderly. The elderly in coun- tries like Sweden prefer to live independently in their residence for as long as possible. To promote such lifestyle, it is important to provide supporting tools. Although there are already similar tools available today, they often enhance the feeling of isolation [2]. Virtual Reality (VR) can facilitate the communi- cation medium and in general, improve the social presence of a person, thereby also decrease the feeling of isolation. Fortunately there is a strong focus on telepresence in research and studies. Telepresence is able to fill the gap which face-to-face provides, that remote communication is lacking [3].
Telepresence solutions is common among distance learners, and the use of video for distance education is rising in popularity. Distance learners have educational video as the preferred choice compared to other types of learning materials [4]. Videoconferencing enables distance education though videoconferencing may also cause students to be discouraged by the lack of social presence and interaction with the other students and teachers. The use of educational video is most prominent among computer science and engineering students [4]. Distance learners are not the only group meriting from interaction between participants, participants in business meetings also often require interaction between one another [5, 6].
For videoconferencing, the interaction between participants is limited to audio and video communication.
The introduction of VR unlocks this interaction constraint, and the participants are no longer bound to a specific location. VR solutions are different from the common telepresence solutions where the partici- pants are allowed to feel present at the location without being there. In a VR solution the participants are transferred and present in a shared space within the virtual environment, thereby greatly increasing the interactivity possibilities.
1.1 Background and Problem Motivation
VR has the capability to visualise shapes and objects in depth, thereby showing0 more information to the audience. Additionally, with VR it is also possible to navigate around and through objects, thereby visualising concepts on a whole new level. By utilising the technology of VR, images for illustrations are no longer the ideal choice. Now real three dimensional models can be instantiated in a VR environment, allowing people to see and interact with them as in real life. Instead of screen-sharing techniques, as with Adobe connect presentations, communication and presentations in general will now be closer to reality, as a virtual world can be simulated and the participants can be immersed in it. A major drawback with videoconferencing and screen sharing techniques is the difficulty to change perspective to the presenter, or to the direction the presenter is wanting the attention to be at. It is a commonly acknowledged problem, mentioned and tackled by many researchers. A problem with videoconferencing is the difficulty of reading a persons body language, which is important because a person’s emotions can be detected through body language [7, 8]. The body language is essential to communication, in optimal cases, up to 70 % of the communication may be non-verbal [9]. Body posture and body movement plays a major role in our communication and therefore there is a need to improve the current remote communication technologies.
Distance education is often carried out through audio and video calls and it is often more challenging
than face-to-face education. Distance learners have a higher dropout rate in comparison to face-to-
face students [10]. There is a challenge for distance educational programs to be both accessible and
interactive, this is seen as a major drawback of distance education [10]. The traditional web-based
(distance) education is inadequate to involve student’s interests and enthusiasm. By involving students
by interaction, their interest and enthusiasm increases. Therefore the current videoconferencing systems
are insufficient for distance learners. The lack of an immersive learning environment is reflected as a
1.2. Overall Aim reduced learning initiative for students and teachers [11]. The lack of emotions between students and teachers can greatly be reduced by introducing a VR learning environment, thereby also increasing the teaching efficiency by removing the spatial and temporal restrictions [11]. In conclusion, there is a need for better communication tools that closer resembles reality. The communication tools are required in multiple areas, not only for distance education and business areas.
1.2 Overall Aim
The point of this study is to design, implement and evaluate a VR solution for remote communication, such as presentations, meetings and conferences. Interactions between participants in today’s solutions (of remote communication systems) are strictly limited, and their field-of-view is limited to the size of the view-port or screen-share. This study aims to remove these limitations by introducing a new remote communication tool for virtual environment. By immersing participants into the virtual environment, their presence and perception in the virtual environment increases, thereby increasing the interaction possibilities and enabling new features. The interaction possibilities, such as gestures and mutual gaze between two participants offered by face-to-face meetings are unavailable in the current communication technologies. This study aims to solve this problem by introducing and evaluating a new telepresence platform for VR.
1.3 Research Questions
A new tool for remote communication and telepresence is required to resemble face-to-face communication, therefore VR is a plausible solution. The third dimension in the VR simulates the real world better in contrast to Voice over IP (VoIP) solutions and it enables new functionality. The research questions of this study are motivated by the research problem. The research questions are listed as:
1. How can such telepresence system be designed with Virtual Reality? Which currently available platforms support this?
2. How can Virtual Reality improve the user experience and effectiveness during remote communica- tion? Can the respective functionality be transferred from reality and other systems directly to the Virtual Reality system?
3. How can the use of avatars be applied with the technology in a virtual environment? What pro et contra does this achieve?
4. What possibilities are there to include new functionality to the system with Virtual Reality?
5. How does the system performance affect the user experience?
1.4 Scope
The main focus of this study is to investigate telepresence solutions with virtual reality. The study aims to answer the questions defined in Section 1.3 and the goal of the implemented prototype is to make the participants feel immersed in a VR. The evaluation of the prototype will determine if the system is prone to simulation sickness and how the user experienced the system. This research will evaluate the performance of the implemented prototype by measuring the response time, packet loss, frame rate in frame time and the bandwidth. Networking security and security in general is outside the scope of this thesis.
1.5 Contributions
The presented thesis presents a proof of concept solution for a telepresence solution in virtual reality using
Unreal Engine 4 and the HTC Vive head mounted display. The prototype utilises avatars to provide a
surrogate for the user and with the use of inverse kinematics, the users movements is replicated to the
surrogate’s movements. The user is provided with functionality to interact with other participants and the
environment through grabbing, moving, creating, removing and painting on objects. The user may move
around freely in real life and the movements is replicated to the virtual environment. The avatars can
convey body language, gestures and also solve the problem of achieving mutual eye contact. The thesis
also presents an evaluation of the presented solution along with a extensive literature study answering the
research questions defined in Section 1.3. The evaluation of the system confirms the proposed solution
1.5. Contributions
may be used with a typical office workstation without the HMD, although for the use of the HMD, a
computer with a stronger graphic card is required. The participants using the system during the usability
test did not experience motion sickness and they were positively inclined to the system.
2 Theory and Literature Study
The theory required for this study is presented in this Chapter and the current state of art of telepresence is presented in Section 2.7.
2.1 Telepresence
Telepresence is defined in a paper regarding evaluation methods of telepresence systems by Mathias Jo- hanson [12], as the mutual feeling of presence at the same location by two or more participants while in reality, the participants are at different locations. In the paper by Johanson, he proposed a Turing Test for Telepresence (TTT). To pass the TTT at a distance of 2 meters between participants in the virtual environment, a set of audiovisual requirements are defined. It is required to achieve a framerate of 65 fps, a resolution of minimum 1920x1080 pixels and to provide enough visual fidelity, a minimum of 24 bits per pixel is required. For the audio, the participant can hear frequencies up to 20 kHz [12], which according to the Nyquist-Shannon sample theorem, requires a 40 kHz sampling rate to achieve (the Nyquist Rate [13]). The samples also require a strength of 40 dB, with the state of the art audio technology, is achievable. The author mentioned that the TTT is too challenging for today’s applications [12], but the requirements will serve as guidelines to this telepresence study.
Similar to Johanson’s definition in [12], telepresence is defined in a paper about a mixed reality telep- resence system by Fairchild et al. [14] and in a empirical study about the effectiveness of telepresence by Standaert et al. [15], as the feeling of being present in an environment derived from technology.
Similarly, Maimone et al. defines telepresence as the feeling of being present at a remote location or with a remote person [16]. Systems which attempt to replicate face-to-face meetings on remote locations are often referred to as telepresence systems [14, 15]. The same article expresses a common problem with similar systems being unable to capture the Non Verbal Communication (NVC). The problem often arises in video-conference systems where the camera and the physical display cannot share the same physical space, therefore makes it difficult for the participant to look into the camera and the display concurrently.
[14] recommends VR systems for dealing with the problem, the method is also said to be successful when conducting collaborative tasks. Although, the study expresses concerns about full body tracking being a bigger challenge when comparing to limiting the system to simple head and hand tracking. Other studies agree on the spatial and temporal disturbance being problematic among telepresence studies [17–19].
Willem et al. [15] states in a study about the effectiveness of telepresence in business meetings “As the functionality of the medium increases, it is found to be more effective” they also point out, there is no significant difference in effectiveness between face-to-face meetings and technology-based meetings. They question if it is possible to go beyond face-to-face meetings. The importance and effectiveness of telepres- ence systems are dependent on the objective of the meeting. Certain objectives merit more from meetings using telepresence technology than other objectives. Though, regardless of objective, telepresence does not have a negative impact on efficiency. The use of telepresence is especially merited when the point of the objective is related to building trust and relationships, communicate feelings and emotions as well as complicated discussions, like explaining concepts or resolving conflicts and issues. In fact, telepresence is comparable to face-to-face meetings in terms of efficiency [15]. There is a need for tools supporting remote collaboration, especially for large interactive spaces [18]. Virtual whiteboards is a popular collaboration tool for visualising ideas and collaboration with people. The use of seamless virtual whiteboards in a continuous space with the participants while preserving the common eye contact and gestures has been in research for some time [20–22].
2.2 Virtual Reality
VR is sometimes referred to as immersive computing technology (ICT). VR is often defined as the
technology which enables users to become immersed into a virtual environment. Interaction within the
2.3. Virtual Reality Hardware virtual environment is critical to many VR systems. VR started off around 1965 and was presented in an essay by Ivan Sutherland, although VR has not received a lot of attention until recently. One of the main challenges - the end-to-end latency - were addressed already in 1999 [23]. VR is already in use in multiple areas, [24] mention VR is used in ceramics, aerospace, surgical and defence applications.
Pausch et al. [25] has demonstrated that VR can be used to increase the user performance in search- related tasks. The performance in search-related tasks is dependant on how fast the brain can process the information flow. This study proves it is possible to achieve a more efficient information gain using VR.
With respect to telepresence and communication studies this enables teachers and presenters to explain complex situations and scenarios efficiently.
Users may become subject to motion sickness when working within the virtual environment, that is one of the major challenges when working with VR [26]. The same article mentions the HMD (Head Mounted Display) needs to be lighter than the current Oculus Rift DK2 at 0.44 kg and it is required to provide a higher resolution than 960x1080 per eye to avoid nausea. Nausea may be experienced due to the human body treating the virtual environment experience as poison. Therefore causing the user to feel sick as an attempt to purge the poison from the body. Although research shows people may grow resistant towards such sickness over time [27], it is still considered a major issue.
The choice of using VR to develop a telepresence system is strengthened by an article on gaming and VR-technologies by Gouveia and Patricia [26]. They conclude VR with the Oculus Rift can increase the feeling of presence, as well as connecting and engaging users in ways that may not be possible outside the VR environment. Slater et al. mentions virtual limbs can be owned as a part of the person’s own body and the realism of those are not of big importance [28]. Viviana and Luis found children felt uncomfortable using avatars of the opposite gender in a study related to avatars [29].
2.2.1 Virtual Reality Best Practises
According to Epic Games, Simulation Sickness (SS) may be caused by misleading scales of objects in the virtual environment, objects are recommended to be viewed in a range of 75 to 350 Unreal Units which corresponds to 0.75 to 3.5 meters from the player’s camera [30]. It is also important to maintain the framerate in a VR experience to prevent SS, the HTC Vive has a target frame rate of 90 FPS along with the retail version of the Oculus Rift [30]. When it comes to the best practises, Epic Games also mentions a list of things to avoid, for example they mention developers are bad test subjects to SS as they grow resistant to the effect, which confirms with [27]. SS, enjoyment and presence is suggested to be dependent on a function of the Field of View (FOV) [31], and the the FOV should not be tampered with manually as it is dependent on the HMD’s physical geometry and may cause discomfort for the user [30].
2.3 Virtual Reality Hardware
There are various kind of existing hardware supporting VR applications, this section describes three popular hardware choices and puts them in comparison.
2.3.1 HTC Vive
Figure 2.1: Overview of the HTC Vive’s motion con- troller buttons 1
The HTC Vive is powered by a OLED panel [32, 33] and features a 90 Hz refresh rate, thereby fulfilling the 90 frame rate require- ment, along with a 110 degree FOV and 2160x1200 pixels resolution [34], which exceeds the 960x1080 pixels resolution which caused users to experience nausea [26]. The HTC Vive has 32 sensors for accurate tracking [34].
The pair of motion controllers packaged with the HTC Vive has a 24 sensors each. The pair of base stations which packaged with the HTC vive provide 360 degrees of motion track- ing and they are easily synchronised wirelessly [35]. The HTC Vive costs about 9500 SEK.
The price is retrieved from Prisjakt [36]. The
1
Img by mitchemmc, source:
https://forums.unrealengine.com/showthread.php?
106609-Steam-VR-Template&p=507476&viewfull=
1#post507476
2.3. Virtual Reality Hardware Vive’s base stations, also called lighthouses,
emitting lasers which are captured by the HMD and motion controllers, which in turn, computes their own locations in the physical space. The Unreal Engine 4 key-mapping and
the buttons of the motion controllers is illustrated in Figure 2.1 on the preceding page.
2.3.2 Oculus Rift
A direct competitor to HTC Wive, the Oculus Rift may be purchased packaged with the Oculus touch which is the corresponding version of the HTC Vive motion controllers [37]. The Oculus Rift with the Oculus Touch included reaches close to 10400 SEK. It is important to observe the price of the Oculus Rift as the price is the combined price of the headset and the Oculus Touch, as no package deal was found, meanwhile the HTC Vive is already in a combined package with the motion controllers. The prices are retrieved from Prisjakt [38, 39]. The Oculus Rift features a combined 2160x1200 pixels resolution which exceeds the 960x1080 pixels resolution requirement like the HTC Vive. Similarly 90 Hz refresh rate, a 110 degree FOV, a OLED panel. The Rift has their own version of the HTC Vive’s base station, although they work differently. They are named "constellation" and they detect the HMD location in the physical space by using an optical sensor [32, 33]. The Unreal Engine 4 key-mapping and the buttons of the Oculus Touch is illustrated in Figure 2.2
Figure 2.2: Overview of the Oculus Rift’s Touch controller buttons 2
2.3.3 PlayStation VR R
PlayStation VR, also commonly called Sony PlayStation VR (PS VR) is targeting console gamers. The R PS VR is required to be used in conjunction with the PlayStation 4 instead of a computer, therefore the information is not available in Table 2.1 on the following page [40]. The PlayStation VR provides a resolution of 1920x1080 pixels (960 x 1080 pixels per eye), the refresh rate of the display is at 90 Hz. The PS VR also has 6 motion sensors, a 100 degree FOV [40, 41]. The PS VR comes at a minimum price of roughly 4700 SEK [42].
2.3.4 Comparison Summary
Table 2.1 presents a table of the minimum specifications for the various of VR hardware, the specifications are retrieved from [37, 43].
The PS VR requires the Playstation 4 console and has no minimum specifications. Although the PS VR is the cheapest among the three, it has lower resolution and less motion sensors which may negatively affect the user tracking. The hardware requirements of the Rift is lower although they are roughly in the same price range. The Oculus Rift and the HTC Vive are similar in their hardware specifications although the Vive is superior to the Rift in the sense of free-movement as the Vive features a full 360
2
Img by mitchemmc, source:
https://forums.unrealengine.com/showthread.php?131000-Unreal-Button-Mapping-names-for-Oculus-
Touch&p=644385&viewfull=1#post644385
2.4. Steam Subsystem
Parameter HTC Vive Oculus Rift Sony PlayStation VR
GPU GeForce GTX 970 Geforce GTX 960 N/A
CPU Intel i5-4590 Intel i3-6100 N/A
RAM 4 GB 8 GB N/A
Video Output HDMI 1.4 or Dsiplay Port 1.2 HDMI 1.3 N/A
HMD Price (SEK) 9500 10400 4700
Table 2.1: Recommended computer specification for the HTC Vive, Oculus Rift and the PS VR
degree tracking whilst the Rift recommends a tracking of 180 degrees [32]. The HTC Vive offers two accessories of particular interest for the future of this study, the Vive Tracker which allows real world items to be brought into the virtual environment [44]. Secondly, the HTC Vive has an upcoming accessory which enables eye tracking [45, 46]. The accessories unlocks new potential which may greatly improve the proposed telepresence system.
2.4 Steam Subsystem
The steam subsystem provides a range of functions to setup a software available to the Internet while tun- neling the network through Steam’s servers, therefore also foregoing the server’s need of port-forwarding.
The Steam subsystem exposes a range of interfaces which is also offered by Valve’s Steamworks SDK. The main interfaces of interest for this study is the matchmaking and the voice chat interface [47]. Although an steam APP-ID is required to develop systems using the Steam Subsystem, it is an simple one-time setup.
2.5 Inverse Kinematics
Inverse kinematics is often used to recreate the body structure and animations. It is used to define the rotation and location of joints based on two or more known joints, i.e. the location and rotation of a body’s end effector. It is often used to create character animations in computer science. The goal of inverse kinematics is to make the recreated structure seem realistic [48]. There are three known approaches for computing the inverse kinematics; the algebraic approach, the geometric approach and the numeric approach. The algebraic approach performs multiple equation solutions therefore a lot of matrix computations. The idea of the geometric approach is to utilise knowledge about the body structure meanwhile the numeric method is an iterative method to guesses and successively interpolate to minimise the error [49]. Daniel Holden presents a well explained approach to the application of inverse kinematics in [50]. The explanation includes an implementation example using the cosine rule to return the angles of the two joints in a two-joint IK setup.
2.6 Evaluation of Quality of Experience
A psychometric evaluation study of a questionnaire which aim is to evaluate the user satisfaction after participating in case-based user tests suggests others to use the questionnaire, with additional modified items if required [51].
2.7 Related Work
Although ideally having meetings face to face is the most efficient and preferred way especially for col- laboration, it may not always be possible for multiple reasons such as economic, time etc. In the cases of face to face meetings being out of the question there is a need for a comparable alternative approach.
Telepresence is a term often used in contact with the alternatives. Telepresence systems aim to substitute face to face meetings when alternative approaches are requested. This section presents some telepresence systems.
Allen et al. mentions in [14], that video can capture most of the non-verbal communication, however it is
difficult to capture eye contact between participants, as the participant and the camera cannot have over-
lapping physical locations. The subject of topic may also be out of view when referenced and thereby the
meaning may be lost [14]. Another study on a mixed reality telepresence system presents an integration
of the telepresence "withyou" and a Mars simulator, they are using a background/foreground detection
to only capture the participant using 10 cameras [14]. The "withyou" telepresence system mentions eye
2.7. Related Work gazes can be accurately estimated while the cameras is at maximum within 30 degrees apart, therefore the original "withyou" implementation requires 12 cameras to render a 360 degree view accurately. However, this is unpractical, as according to the study, 12 HD cameras require a bandwidth greater than 10 Gb/s uncompressed image data [17]. The accuracy of gaze estimation can be measured by conducting a game between pairs of participants as in [22], one user would be gazing on a coordinate in a grid, while the other participant guesses which coordinate in the grid the first user is staring at. The horizontal and vertical offset can then be calculated from the results, and provide a measurement on the gaze estimation accuracy. A different immersive telepresence solution using a omni-directional camera with a 360 degree view is presented by Luis et al. The Panoptic camera prototype features 49 cameras with 640x480 pixels resolution each. The solution generates an omni-directional image from the 49 images captured using the Panoptic camera, the omni-directional image having a resolution of 1024x768 pixels. Two major draw- backs with the system is the low frame rate below 25 frames per second and the low resolution. Although the system reduces the latency by allowing the user to move the head around due to the omni-directional image, the article itself mentions that the system is unsuitable for real-time video streaming. Therefore, it is not a candidate for a live telepresence system [1].
Carpeno is a telepresence study and a combination of two previous studies cAR/PE! and Coeno. Carpeno is a limited system where participants are unable to move freely because an Apple iSight camera is seated on top of a display which presents a shared working space. The iSight camera captures the participant but is limiting the movements of the participant due to it’s static location. It is an innovative system but it is lacking the non-verbal communication in sense of head-tacking gaze and a larger workspace awareness. The evaluation of the study showed the users was satisfied using the system, although they had difficulties grasping where the other participant was looking [52].
Andrew et al. uses extensive equipment such as multiple depth-sensors to scan the two environments between two participants and head-worn seen-through displays to present the other participant in respec- tive environment. Although the idea and implementation gives a realistic results, the implementation induces a large latency because the implementation transfers a video-feed through the network [16]. A similar approach is used by Kim et al. with augmented reality where they update a virtual avatar’s head rotation based on a captured gaze [19]. The study updates the head rotation if the avatar’s owner has been staring at another participant for a certain amount of time based on an empirical threshold.
The empirical threshold is derived from two factors. First the threshold has to be low prevent the other participant from experiencing lag in the person’s head rotation. Secondly, to make sure the avatar doesn’t appear to be looking at a third person in the scene when the avatar’s owner is rotating the head to the second person while passing by the third person. The system successfully captures the mutual eye contact between participants though, it requires all participants to wear special track-able helmets and there is no implementation of a shared working space.
MeBot [53], PRoP [54], Anybots’QB, VGo [55], [56] and [57] are examples of existing telepresence robots.
The robot is remotely controlled by a user from a distant location while also conversing with other users
near the robot. Though, [57] main focus is within health care, primarily among the elderly. Yuka Kato
also presents a mobile telepresence robot system in [58], the user remotely controls a robot through a
virtual space in a browser. The robot sends the live feed to a server which constructs a virtual model of
the feed. Although these telepresence studies using robots are relevant, they require special equipment
dedicated specifically for telepresence nor do they provide the presence of a face-to-face meeting, therefore
they are no substitutes for this study.
3 Methodology
The approach to the research questions defined in Section 1.3 is described in this chapter. The project used agile methodologies. By delivering small continuous releases, the project promised to always have a working prototype. Regular meetings with supervisors have encouraged discussions and provided feedback on the progress thereby increasing the quality. A backlog of potential features was for the implementation of the prototype. To ensure the focus was on the prioritised items, the backlog was refined continuously.
3.1 Approach
This section will first recap the research questions and then briefly describe the approach taken to answer them.
1. How can such telepresence system be designed with Virtual Reality? Which currently available platforms support this? First off a literature study which compared the available major platforms capable of supporting VR applications were conducted. The literature study aimed to compare the game engines Unity and Unreal Engine to determine the most suitable engine for this study.
A prototype was implemented as a proposed solution of the VR communication system using Unreal Engine 4, which was chosen based on the platform comparison study in Chapter 5. The ideal system would be a replication of reality, therefore the system requires similar functionality to be implemented to those of a presenters capabilities in reality. As collaboration is an area of interest for the system, there was a need to prioritise interact-able functionality such as drawing. One of the most basic requirement to resemble reality is voice channels and the support of online networking (multiplayer). Although voice channels is a part of the implemented solution, it is not part of the prototype evaluation. With the state of the art audio technology, it is not considered a challenge to achieve a sufficiently good voice chat to satisfy users need.
2. How can Virtual Reality improve the user experience and effectiveness during remote communi- cation? Can the respective functionality be transferred from reality and other systems directly to the Virtual Reality system? An extension of the literature study combined with the implemented prototype showed whether the respective functionality can be transferred directly and revealed how the VR system can improve the user experience and effectiveness. The system’s evaluation showed how well the functionality can be transferred.
3. How can the use of avatars be applied with the technology in a virtual environment? What pro et contra does this achieve? The literature study investigated the potential of avatars inside virtual environments and their effects. The avatar served as the role of a surrogate for users in the im- plemented prototype. The prototype along with the use of avatars was evaluated using user tests followed by a questionnaire.
4. What possibilities are there to include new functionality to the system with Virtual Reality? The implemented prototype was assessed using a questionnaire which purpose was to evaluate the system from a user’s perspective, and to identify needs for new features. The first prototype aimed to represent communication in real life while encouraging collaboration using support for whiteboard illustrations.
5. How does the system performance affect the user experience? It is required to measure the frame rate, response time, bandwidth and the network jitter. These parameters is required to be compared to the research retrieved from the literature study to answer the final question. The performance metrics of the system is the based on the frame rate and the networking response time. Frame rate and latency is recommended to be measured, controlled and reported in all VR research [59].
[22] conducted a game to measure the response time between two participants, although the game
is unfeasible for this study as it requires two sets of the HTC Vive, it verifies its importance and
relevance.
3.2. Prototype Assessment The frame rate cannot simply be measured by using the equation:
1Second
DeltaSeconds (3.1)
where delta seconds is the time since the most recent tick. It is not recommended because it may result in a different frame rate depending on where the computations are done because the head mounted display has it’s own refresh rate. The frame rate is required to be equal or greater than 90 frames per second to prevent motion sickness. The frame rate data will be plotted to a graph and analysed to verify if it meets the requirements.
Latency is performance metric used by others [14] and it is a standard way of measuring commu- nication systems. A low latency is required for a smooth user experience, it is often referred to as the end-to-end delay in a system. Latency has an adverse effect on the performance and comfort of the user immersed in the Virtual Environment (VE) [59]. The latency and the network delay are a part of the response time as the response time is dependant on both [60]. The network condition may greatly affect the level of immersion, network delay and network jitter are known problems to telepresence systems. It has been proven participants dissociate their own hands with surrogates when experiencing delays down to 300 ms [61], therefore it is crucial to achiave a good response time. The response time provides a real measurement on the latency and network delay of how users interact with the virtual environment. The response time and the bandwidth along with the packet loss will be captured for the system. The variance of the response time can roughly be considered as the network jitter.
3.2 Prototype Assessment
The prototype was assessed using a questionnaire where the participants answered the questions using a 7-point likert-scale after performing the user tests. The responses has been normalised and summarised in a Figure for further evaluation. The questionnaire will mainly determine the usability of the prototype.
The questionnaire follows the Post-Study System Usability Questionnaire (PSSUQ) [51], with minor customisation. The results has been compared to an existing telepresence system called Carpeno [52].
The wording in the questionairre was carefully chosen as wordings may have unwanted impact on the respondents. Only clearly understood words was used, slang, technical, intensifying and judgemental wordings was avoided. No hidden assumptions took place. These guidelines worked to prevent the respondent from being subject to bias [62].
3.3 Tools
Certain tools are required to conduct this study and to recreate the evaluation results. The tools is required is Unreal Engine 4, HTC Vive HMD, Steam software and 4 computers with specifications as presented in Table 3.1. A game engine is required, this study used the Unreal Engine 4, although a different engine may be used to create similar results. Visual Studio is currently required to be used in parallel with the Unreal Engine 4 to modify and compile the C++ code. This study also used the HTC vive head mounted display, which may also be substituted with a VR HMD of equal or similar performance such as the Oculus Rift.
Device Processor Graphic Card Memory
VR-computer Intel I7 7700k GTX 1080 16 GB 24000 MHz
Lenovo Thinkpad W540 I7 4810MQ Quadro K2100M 24 GB 1600 MHz
Computer 3 Intel I5 4690k GTX 760 16 GB 1800 MHz
Computer 4 Intel I5 6600k RX 490 16 GB 1800 MHz
Table 3.1: Specifications of the computer system
3.4 Research Methodology
This research research followed a sequence of steps as illustrated in Figure 3.1 on the following page. The
first steps started about conducting an extensive literature study and thereby identifying the problems
encountered by related research and pinpointing the competitors of telepresence systems. The competitors
is mentioned in Section 2.7. The literature study involved reading through multiple articles related to
remote communication such as telepresence studies. The literature study provided insight on the focus
3.5. Implementation Methodology points of the various studies, therefore giving a hint to what this study should focus on. An extended literature study comparing the two major game engines Unreal Engine 4 and Unity3D, provided the basis to the decision of which game engine to use for the implementation the VR telepresence solution in this study. The platform comparison is documented in Chapter 4. Following the extensive literature study, the research questions were defined based on the knowledge obtained through the literature. The overall aim of this study was to design and implement a VR solution for remote communication such as telepresence. Therefore to ensure the most essential features were implemented, a product backlog was defined and prioritised as presented in Table 3.2 under Section 3.5. The implementation was guided through the product backlog. The implementation workflow is described more detailed in Section 3.5. The implemented solution were then evaluated by the metrics response time, packet loss, bandwidth, frame rate and user tests. Section 3.6 explains the evaluation method in detail. Finally the data retrieved from the system evaluation were processed and analysed.
Figure 3.1: Research work flow overview
3.5 Implementation Methodology
1 Virtual environment 2 VR Support
3 Start-up menu 4 Online sessions 5 Create & join sessions 6 Audio communication 7 Controllable character 8 Toggle VR/non-VR 9 Create objects 10 Move objects 11 Paint on objects 12 Push and pull objects 13 Rotate objects
14 Prepare measurements 15 Do measurements 16 Process measurements Table 3.2: A summary of the docu- mented backlog
Figure 3.2 on the following page illustrates the implementation work flow for this study. The work flow was based on multiple it- erations of iterations. The implemented features were chosen from the prioritised product backlog as illustrated in Table 3.2. The top most features had the highest priority. The firstly implemented feature were the virtual environment itself along with the project creation inside the Unreal Engine 4. VR support were sequentially added onto the project followed by a start-up menu which the user firstly sees upon application start. The menu allows the user to create and join sessions.
Each and every implemented feature went through an accep-
tance test to verify it is functioning as expected. Once the first ac-
ceptance test passed, the features which required networking sup-
port transitioned from the implementation stage to the network
replication stage. The network replication stage involved replicat-
ing the functionality throughout the network so that all connected
clients would experience the same reality. Once again the network-
ing stage went through an acceptance test to verify it is working
as expected. If bugs were detected, they were corrected and in
3.6. Evaluation Methodology worst case, the feature had to be redesigned, thereby going back
to the feature implementation stage. Finally, as this telepresence system has support for multiple users, it is important to optimise the network load. Therefore this study included a optimisation stage in the implementation phase which also proceeded onto an acceptance test. Although, the acceptance test for the network load mainly consisted of manual code analysing. The optimisation stage was the final stage in the implementation phase for each feature. The feature were then considered deployed and a part of the final prototype.
Figure 3.2: Implementation work flow overview
3.6 Evaluation Methodology
The system was evaluated on the response time, packet loss, bandwidth, frame rate and through user tests. The user tests was evaluated using a system usability questionnaire. Subsection 3.6.1 through Sub- section 3.6.4 describes the work flow in detail. Each evaluation apart from the user tests was conducted by increasing the quantity of connected clients to measure the clients affect on the evaluation. It is essen- tial to highlight the server and the connected clients was connected to separate networks throughout the quantitative measurements. All data-capturing measurements was conducted over a 5 minute time inter- val to prevent the affect of outliers on the average values. Finally the measurements was conducted on the VR-computer and the Lenovo Thinkpad W540 (Lenovo laptop). The Lenovo laptop represents a common office workstation. The VR-computer had the HMD activated throughout all of the measurements.
3.6.1 Response Time and Packet Loss
The response time in this study is the time taken for a message to be sent to the server and returned back. The response time and the packet loss measurements was conducted while successively increasing the connected clients from 0 to 2 clients connected excluding itself. The response time was only captured from the reference of a client as it is considered to be the worst case, because the server requires one less RPC to replicate it’s changes to all connected clients. The response time includes the time taken for processing, queueing, transmission and propagation delay. It is calculated as:
M time = t receive − t sent (3.2)
Where M time is the response time, t sent is when the message was sent and t receive is when the message was returned. The response time and packet loss was measured concurrently as depicted in Figure 3.3 on the following page. The packet loss is then calculated by subtracting the messages sent, denoted M sent
with the messages received, denoted M received . It is calculated as:
M loss = M sent − M received (3.3)
Packet re-transmission is not considered in this study as it is measured over a unreliable protocol. Al-
though re-transmissions are ignored in this study, precaution is taken by matching t sent with correspond-
ing t receive for each message.
3.6. Evaluation Methodology
Figure 3.3: Response time and packet loss work flow overview
3.6.2 Bandwidth
Figure 3.4 illustrates how the client’s and the server’s bandwidth was measured. A set of samples of the average bandwidth B u was calculated for all connected users u ∈ [1, 3]. All connected users was in an idle state inside a conference room to prevent interference. B u was calculated for all packages p ∈ [1, N ] over the time T . The measurements was done with T = 10s and the average was measured with a minimum of 17 samples for each B u .
B u =
p
P
i=1
p i
sizeT (3.4)
The client’s and the server’s bandwidth was calculated identically, although for the client there was 2 additional clients connected rather than 3. The sum of the maximum number of connected users remained 3. The bandwidth was split into upload rate and download rate.
Figure 3.4: Bandwidth work flow overview
3.6.3 Frame Rate
The frame rate is captured using the "startFPSChart" console command which is built into the Unreal Engine 4 game engine. It is a build-in function defined in the UEngine class within the ChartCreation.cpp of the engine source code. The measurement was conducted as with the bandwidth, in an idle state to ensure it is reproducible and to prevent interference from external factors.
Figure 3.5: Frame rate work flow overview
3.6. Evaluation Methodology 3.6.4 User Tests
5 Persons conducted user tests and evaluated the experience through a set of questions based on the PSSUQ [51]. The participants was instructed to equip the HTC Vive HMD and test the system freely.
The participant was then asked to play tick-tack-toe as the server with a second participant whom is connected to the conference as a client from a different machine as the setup in Figure 3.6. The participant wearing the HMD was instructed to draw the tic-tac-toe board and mark the choices of both participants using the system’s painting tool. The participant not wearing verbally instructed their choice to the VR-user. Due to a lack of participants, some participants had no partner and therefore some answers on the PSSUQ were rendered as not-applicable. The participants was requested to answer the PSSUQ honestly and truthfully as it is anonymously. The purpose of the tick-tac-toe gameplay was to encourage the participant to test the system’s functionality.
Figure 3.6: User tests work flow overview
4 Platform Comparison
Unreal Engine and Unity are the two most apparent choices when it comes to game engines. It is difficult to say which game engine is better than the other one, choosing the game engine that works the best is one common hurdle for developers today [63–65]. Unreal Engine 4 and Unity 3D are the leading choices among game engines, this Chapter will put the two in comparison to help developers in making a wise choice. To compare the two, it is required to also look at previous comparisons of Unreal Engine 3 and Unity 3D. Section 4.2 and Section 4.3 describes the unique properties and features of Unreal Engine 4 and Unity 3D and Section 4.4 summarises the key points of both game engines.
4.1 Areas of Interest
This section describes the different areas of interest used by previous research to compare Unreal Engine 3 with Unity. It is required terminology to understand the remaining comparison.
• Fidelity
A world can be created which looks realist but does not function as expected, in that case it may have a strong audio and visual fidelity yet a low functional fidelity. The level of fidelity mainly depends on the narrative, audio and visual contents, the content’s behaviour and the interaction medium. The audio and visual fidelity is of particular relevance to this study, users feel more motivated and engaged in higher fidelity games and simulations [66].
• Accessibility
For a game engine to have high accessibility, it requires an easy to use interface for both users with limited experience such as designers as well as experienced developers. The accessibility is also of relevance to developers exploring new grounds [66].
• Networking
Games with online social networking interaction is often of interest and it is required to be considered early because it affects the nature of a game. The communication between players in a game is a part of the networking area’s responsibility to implement [66].
• Composability
Composability is used to define how much support the game engine has to reuse components created and the capabilities of importing existing components and modules from the engine and external sources [66].
4.2 Unreal Engine
Unreal Engine 4 is supposed to have a shallow learning curve and Unreal Engine is used on many AAA titles 1 [67], thereby contributing to the accessibility. The Unreal Engine 4 provides additional function- ality above Unity 3D when it comes to rendering (spatial effects, shadows, lighting and texturing effects) [66]. The increase in functionality enables potential in the perception of feeling immersed - feeling present in the real scene. The sense of realism is a function of both auditory and visual fidelities in virtual envi- ronments [68].
Unreal Engine 4 has support for their own visual scripting language called Blueprints [69], and JavaScript through a plugin named Unreal.JS [70]. C++ is also an alternative for performance demanding tasks, and the C++ classes and functions can be exposed to the Blueprints. [71]. Packaging and cooking blueprints to native C++ code to experience performance boosts is a fully supported feature with the recent release of Unreal Engine 4.15. Cooking blueprints to C++ reduces the overhead caused by the execution of blueprints [72].
1
A game with the AAA title is given to games with very large budgets
4.3. Unity 3D For VR development, Epic Games provides a VR editor which allows editing from within the virtual environment. Thereby reducing the difficulty of positioning, scaling and transforming objects to perfec- tion. By working while immersed into the virtual environment, the developer receives instant feedback, thus making it easier to create realistic scenes [73]. Tim Sweetney, the founder of Unreal Engine claims Unreal Engine has a strong focus on supporting the developers to finalising the shipping of a product, as a result, the source code of the game engine is open source, allowing developers to solve their own arising problems [74]. Unreal Engine 4 is recommended to use for 3D games and applications and it has the most support from the community when it comes to development for VR [64].
4.2.1 Memory
The Unreal Engine 4 requires large amount of memory allocation in relative to Unity 3D, the memory consumption used by Unreal Engine 4 is depicted in Figure 4.1. The Unreal Engine 4 also requires the Epic Games Launcher, although this can be closed once the engine has been launched, it requires additional memory and CPU power in the start-up phase. The testbed of Figure 4.1 is the Lenovo laptop. In game development, memory allocation is an important performance factor. To prevent memory leakage, the allocated memory can either be collected using the Unreal Engine 4’s garbage collector [75] or it can be freed manually through C++ [76].
Figure 4.1: Performance usage with Unreal Engine 4 on standby
4.2.2 Networking
Unreal Engine 4 is using the client-server model which means the server is responsible for all the "real"
updates of the world meanwhile the client is simply recreating the world based on the information sent by the server [77]. When the client interacts with the world, this interaction is sent to the server, usually by calling a remote function on the server-side such as "move-left" or similarly. By enabling the replication Boolean of an actor in Unreal Engine 4 as depicted in Figure 4.2 on the next page, the actor will automatically be synchronised from the server to the connected clients. Though, this synchronisation is not bi-directional, it only synchronises from the server to the client. Actions from the client is sent to the server through replicated events using Remote Procedure Calls (RPC) [78]. This also means, if Client A wants to interact with Client B, Client A needs to first send a RPC to the server which in turn, communicates to Client B. Clients cannot directly interact with one another. Figure 4.3 on page 18 is exemplifies how a RPC may be defined and executed using Unreal Engine 4. To call a RPC from a client to all connected clients, there are 3 requirements. The event must be called, in the illustration this requirement is illustrated as "Request Print Message" which is called from the begin play event and replicated to the server. Sequentially, the server calls the Print Message function which is in turn, replicated to all connected clients. Unreal Engine 4 denotes this as a multicast from the server. Only the server have authority to communicate with other clients. The clients receiving this RPC executes the function as defined. The replicated events in the illustration are all set to reliable. An event may also be set to unreliable, which is generally recommended to use for cosmetics and less important events.
4.3 Unity 3D
Unity is also free although if a revenue of $100k is reached, the subscription plan is required to be
upgraded or all of the Unity software copies needs to be destroyed and the project content may no longer
be updated [79].
4.3. Unity 3D
Figure 4.2: Replication using the blueprint editor in Unreal Engine 4
Unity is known to have a shallow learning curve and is easy to use for beginners [67], thereby Unity has a strong accessibility. Unity 3D offers two main programming languages, C# and JavaScript. Both of them are popular programming languages. Developers are commonly acquainted with one or the other. The documentation is in most cases sufficient [80]. This contributes to the shallow learning curve of Unity3D.
4.3.1 Memory
Figure 4.4 on the next page Shows the standby performance consumption used by Unity 3D, it can be observed Unity is using considerable less memory compared to Unreal Engine 4. Unity only requires about 30% memory consumption of what Unreal Engine 4 does. Although with today’s cheap prices for additional memory, these consumption numbers are no issues. The testbed of Figure 4.4 on the following page is the Lenovo laptop. Unity’s approach to prevent memory leakage is similar to Unreal Engine 4’s method, they also use garbage collection. Although, Unity does not provide a method for the developer themselves to free the memory, but they do offer a function call to request the garbage collector to clean up unused memory, the function call is depicted in [81].
4.3.2 Networking
Most of the networking in Unity3D is easiest done using the High level scripting API [82]. Although for server to client synchronisation, Unity uses a technique called SyncVars, which resembles the replication in Unreal Engine. Figure 4.5 depicts the definition of a SyncVar in C# [83].
Figure 4.5: Replication in Unity3D using SyncVars
However, Unity3D also supports RPC like Unreal Engine 4. To execute a RPC within Unity3D, the
function needs to first be defined as illustrated in Figure 4.6 on page 19 using C#. Finally the RPC
4.3. Unity 3D
Figure 4.3: Remote Procedure Calls in Unreal Engine 4 using blueprints
Figure 4.4: Performance usage with Unity on standby
may be executed in C# as illustrated in Figure 4.7 on the next page. As seen in Figure 4.7 on the
following page, the second parameter of the RPC execution defines where the RPC is being executed, in
this example, it is being executed on all connected clients.
4.4. Comparison Summary
Figure 4.6: Remote Procedure Call definition in Unity3D using C#
Figure 4.7: Remote Procedure Call in Unity3D using C#
4.4 Comparison Summary
The learning curve is subjective and it is often discussed and bias towards Unity. The learning curve has been improved with the release of the Unreal Engine 4 [67], and both Unity and Unreal Engine 3 have a learning curve of medium difficulty and both engines provides similar documentation [66]. This does not directly portrait Unreal Engine 4 as the easier game engine to learn, it proves both to be sufficiently adequate.
The audio and visual fidelity, accessibility, networking, composability and functional fidelity stats is summarised in Table 4.1, the data is retrieved from [66].
Parameter Unreal Engine 3 Unity 3D
Audio and visual fidelity 5 4.8
Accessibility 2.7 3
Networking 5 3
Composability 5 1.7
Functional Fidelity 4.3 5
Table 4.1: Estimated values of a comparison of the selected game engines
The two game engines differ among features and the differences in features of relevance to this study is depicted in Table 4.2. The differences is of the free version of both Unreal Engine 4 and Unity 3D.
Parameter Unreal Engine 4 Unity 3D
Virtual Reality Editor Yes No
Performance Profiler Yes Yes
Default VR Project Template Yes No
Replication Support Yes Yes
Visual Scripting Yes No
Garbage Collection Auto and Manual Auto Table 4.2: Functionalty comparison of Unreal Engine 4 and Unity 3D
4.5 Game Engine Choice
The choice of Unreal Engine 4 as game engine is motivated by the development speed gains provided by
using the visual scripting language blueprint while not losing in performance as the demanding scripts
may be cooked to native C++ code. In comparison to Unreal Engine 4’s competitors such as the rival
Unity 5, Unreal Engine is open source. The development of the engine is faster as it is being open
source, as multiple developers throughout the world improves and solves the existing bugs together. This
portraits the Unreal Engine 4 as future proof with strong community support.
4.5. Game Engine Choice Unreal Engine 4 is ahead of Unity 5 when it comes to graphics which is a strong motivator as it helps making the virtual environment resemble reality without requiring experience in design. This study’s focus is on VR, and the motivation will therefore be based on the game engine’s potential to create VR applications. Unity does not come with VR boiler template 2 , whilst Unreal Engine 4 does. The Unreal Engine also comes with an immersing VR editor which improves the potential of creating a realistic VR environment.
The statistics observed in Table 4.1 on the previous page and the data in is also strong motivators to why Unreal Engine is the correct choice, the composability and networking is of particular advantage to this study. The extensive memory consumption by Unreal Engine is no issue with the computer specifications available for this study, as the memory consumption only affects the development stage, it is irrelevant for the end user. Although, the garbage collection advantage in Unreal Engine is of interest as it gives the developer more control of the memory consumption during run-time. The functionality comparison table Table 4.2 on the preceding page enables a quick and smooth introduction to VR by using the default VR project template.
2
