DSC2018EuropeVR B. Blissing et al.
Exploring the suitability of
virtual reality for driving simulation
B. Blissing and F. Bruzelius
Swedish National Road and Transport Research Institute; SE-581 95 Link ¨oping; Sweden, e-mail: {bjorn.blissing, fredrik.bruzelius}@ vti.se
Abstract - Head mounted displays (HMDs) is an emerging technology and the availability of affordable systems
is growing fast. Replacing projector and large screen solutions with head mounted displays may appear as an appealing solution. However, inherent properties and technical limitations of these systems need to be understood and considered before making the leap to virtual reality.
This paper outlines some of the most fundamental limitations of head mounted displays relevant to this context, both from a technical and human factors perspective. Desirable properties of scenarios and types of studies are deduced, based on these limitations. Finally, a meta analysis is performed on the feasibility of transferring simulator studies found in the literature to platforms with head mounted displays.
The results suggest that a noticeable amount (40%) of the investigated simulator studies could likely have been performed with head mounted displays. This number could be increased further with technical advances in display resolution, display technology, reduction in latency, etc.
Keywords: Virtual Reality, Head Mounted Display, Driving Simulation
Introduction
Previous usage of virtual reality in the automotive context has mostly concerned engineering design, vehicle ergonomics, styling, interior lighting, and per-ceived quality [Kem14]. Meanwhile, driving simula-tors using projecsimula-tors or large display systems have been tasked with driver training and most of the stud-ies relating to human factors.
As virtual reality technology has become more and more ubiquitous, the interest for using head mounted displays (HMD) in driving simulators have surged. However, one cannot simply replace the traditional displays with a virtual reality solution without consid-ering the potential effects that it may have on the re-sults. Virtual reality technology comes with its own set of limitations and drawbacks that need to be con-sidered before moving away from traditional display solutions. In addition there are setups where HMDs are used while driving real vehicles, which requires significant understanding of the technology.
Methodology
In this paper we give a summary of the benefits and challenges of virtual reality in a driving simu-lation context. We then review conference papers published within the simulator community (the Driv-ing Simulation Conference and the Road Safety and Simulation Conference) from the last five years and try to identify studies which could be suitable for vir-tual reality. We limit the selection of papers to studies requiring an interactive environment. This includes most driving simulator and test track studies, but ex-cludes naturalistic studies.
The selected studies from the driving simulation com-munity have been supplemented with generalized studies from the virtual reality community, which have explored human factors in virtual reality in a more broad sense.
Benefits of HMDs
Using HMDs instead of projectors or other large dis-play systems have the additional benefit of being more compact and the hardware cost is lower. The installation is also much simpler compared to multi-projector systems.
HMDs used to be exotic hardware which required large investments and technological skill to use. With the new wave of HMDs targeted towards home users the investment cost has dropped radically, but to use an HMD still require technological skill to avoid com-mon pitfalls.
Binocular depth cues
Most HMDs uses binocular imaging, i.e. two different images for the left and right eye. Binocular imaging give strong depth cues, especially up close. The ef-fect diminishes with distance with an upper limit of about 40 meters [Bad14].
Stereo projectors can present binocular imaging as well, either by using active shutter glasses or glasses with polarizing filters or band-pass filters. Regret-tably, there is usually some light leakage leading to crosstalk [Woo12], i.e. some of the image intended for one eye get displayed to the other eye as well. This can in turn reduce the perceived image quality and the sense of presence.
-163-Exploring the suitability of virtual reality for driving simulation DSC2018EuropeVR
Presence
Since the user of an HMD is completely surrounded in the virtual environment the level of presence is in-creased. Improved presence suggests that the user behavior would be more similar to real world action, compared to other display solutions were presence is rated lower [Bow07].
Completely virtual environment
One major benefit of immersing the user in a virtual environment is the ability to change every part of the environment as desired. This makes studies involv-ing different interior designs effortless compared to changing the real physical environment in the vehi-cle.
Challenges of HMDs
Virtual reality technology is not without challenges. Some of the most fundamental limitations in the con-text of driving simulation are listed below.
Resolution & Field of view
The current generation of HMDs uses displays and lenses capable of an angular resolution of 10-15 pix-els per degree. In comparison, projector based sim-ulators usually have an angular resolution of 30-40 pixels per degree. The low resolution of an HMD can make objects hard to discern at distances were they would be clearly identified in real life. It could also reduce the legibility of road signs and other impor-tant information, both inside and outside of the vehi-cle cabin.
According to the Commission Directive 2009/112/EC any applicant applying for or renewing a driving li-cense need to the minimum binocular visual acuity, with corrective lenses if necessary, of at least 0,5 when using both eyes together. This visual acuity corresponds roughly to 30 pixels per degree, which makes the current generation of HMDs fall well be-low the limit where you are deemed fit to drive. The horizontal field of view of most current genera-tion HMDs are between 90-100 degrees. A healthy individual have a horizontal field of view of about 200 degrees. This makes any task where the user is go-ing to detect objects in the peripheral hard to per-form when using an HMD. The Commission Directive 2009/112/EC states a lower limit of 120 degrees hor-izontal field of view to be considered fit to drive.
Lenses
The lenses used in the HMD can either be conven-tional lenses or Fresnel lenses. Fresnel lenses can achieve the same focal length as conventional lenses by dividing the lens in to concentric rounded sections. Since each section is moved down Fresnel lenses require considerably less mass and can be made thinner than conventional lenses. The drawback with Fresnel lenses is that they can produce optical arti-facts know as Fresnel rings. Depending on the den-sity of the sections these rings can be seen as ei-ther distinct rings or as bright haze. The effect is par-ticularly visible when displaying bright objects on a
dark background. It is therefore not advisable to use HMDs which employs Fresnel lenses when perform-ing dark scenarios, such as night time drivperform-ing.
Latency
Latency is the time delay from the input to output in a system. Low latency in virtual reality has been proven to be important for cognitive functions such as sense of presence, spatial cognition, and awareness [Mee03, Pap12]. When latency increases, the user experiences decreased visual acuity, decreased per-formance, decreased presence and is less suscep-tible to training [All01]. Increased latency is also as-sociated with increased levels of simulator sickness [Buk12] and stress [Man04].
Accommmodation-Vergence Conflict
HMDs uses binocular imaging but the convergence distance is fixed. This means that even though your eyes are presented with images of objects that are supposed to be at different distances, your eyes are still converging at the same angle to keep the image in focus. Since convergence is one of the cues that the brain uses to infer distance this results in a sen-sory conflict. This can lead to eye strain, blurred vi-sion, and headaches [Shi11]. There are examples of early research prototypes of optical systems which use varifocal or multifocal displays to eliminate this effect [Mat17]. But until these types of systems are generally available it is advisable to try to minimize the necessity of shifting focal distance when using a fixed focal HMD.
Tracking inside full motion simulators
The most common tracking technology to track the position of the HMD is to use inertial tracking [Bli16a]. However if inertial tracking sensors is used inside a full motion driving simulator these sensors will pick up the accelerations from the movement of the user inside in the vehicle as well as the accelerations from the motion system as well. This problem can in the-ory be solved by subtracting the inertial information from the motion system [Fox00]. In practice this en-tails bypassing the tracking algorithms developed by the HMD vendor, thus often also bypassing some of the low level latency combating techniques placed there by the vendor.
Simulator Sickness
There are multiple theories why motion sickness occurs inside virtual reality; sensory conflict theory [Rea75], evolutionary theory [Tre77], postural insta-bility theory [Ric91], rest frame hypothesis [Pro03], and eye movement theory [Ebe94].
There are individual factors which affect the suscepti-bility for motion sickness such as health status, previ-ous experiences, gender, age, and the users own ex-pectations [Jer16]. There are also hardware factors which may contribute to motion sickness in virtual re-ality, i.e. latency, tracking errors, wide field of view, low refresh rate, flicker, accommmodation-vergence conflicts, as well as the weight and fit of the HMD. Using a motion system may help alleviate simulator
DSC2018EuropeVR B. Blissing et al.
sickness, but only if done well otherwise it may have the opposite effect.
The most common way to measure motion sickness is via the Kennedy Simulator Sickness Questionnaire (SSQ) [Ken93], where the users are asked to rate 16 different common symptoms on a 4-point scale. The trouble with this questionnaire is that it is hard to quickly administer during a test. Another option is to use the Fast Motion Sickness Score [Kes11], where the user is asked to rate their level of motion sickness once per minute.
A recent study comparing the influence of non-VR (2D, stereoscopic 3D) and VR (HMD) was performed in a fixed base simulator [Wei17]. The study focused on lane change maneuvers. The results showed no significant difference in neither driving performance nor physiological responses, although reported sig-nificant increase in simulator sickness for the HMD-condition.
Completely virtual environment
One major drawback of having a completely virtual world is that you no longer can see your own body. Especially problematic is the fact that you no longer see your own hands and fingers. Tracking of hands and fingers can be done and re-introduced in the vir-tual world. This can be done either by attaching track-ing equipment to the hands or by contact free tracktrack-ing using image based systems.
Another drawback of having a completely virtual world is the lack of haptic feedback from the virtual interior. Solutions may include replicating the interior shape and adding real buttons and knobs to give the user the correct haptic feedback. Although this coun-teracts the one greatest benefit of having the full vir-tual environment to begin with, i.e. cost and ease of adaptation.
Review of simulator studies
To evaluate the potential transferability of driving sim-ulation studies to virtual reality and HMDs, we re-viewed all the papers published during the last five years from the two largest conferences within the simulator community. A total of 734 papers were re-viewed. 203 of these papers concerned interactive driving simulation studies where a test subject was either driving or riding along in a simulated vehicle. The studies were categorized into different factors:
Driving behavior – This included what types of
driv-ing maneuvers the subject was expected to perform and how this might affect their head turning behav-ior. The required lateral motion was split into three levels low, medium and high. Studies that featured low level lateral motion was typically studies which included driving on a highway without any quick side maneuvers. Studies were the subject drove on a ru-ral road or similar were categorized as medium level, while lane change maneuvers or intense city driving were categorized as high level.
Head turns were categories in a similar fashion. Stud-ies performed driving on a highway without traffic or where simulators with a single display in front of the driver was used was categorized as the lowest level.
Driving on a highway or rural road with low traffic den-sity was categorized as medium level and driving in dense traffic or complicated city environments were categorized as high level.
Motion sickness – If the study listed how many of
the subjects that were affected by motion sickness and if so, which type of method that was used to measure it. The most common case was that report-ing of rate of motion sickness was omitted, which ac-counted for 148 of all published studies. 15 papers reported detailed motion sickness ratings using SSQ, FMS or similar rating systems. 3 papers simply re-ports that all the subjects suffered from motion sick-ness in some form. The rest of the papers only pub-lished how many of the subjects that suffered motion sickness at all.
Interactivity with hardware – If the experiment
con-ditions required subjects to interact with hardware in-side the cabin such as buttons or knob in the instru-ment cluster or other third party hardware.
Virtual reality studies
We found ten papers which already used HMDs. Five of these studies where technology evaluations where the technical details of virtual reality was ex-plored. Two of these concerned a bicycle simulator [Suz15, Suz17], while the other three concerned driv-ing a real car while weardriv-ing an HMD either sittdriv-ing in-side the car [Jud12, Bli16b] or remote driving [Ric15]. Of the remaining five papers, two were exploring sim-ulator sickness [Ayk14, Col16] and one concerned how to interact with tangible interfaces while using virtual reality [Las17]. The final two papers were more traditional user studies; one study regarding de-tection of motorcyclist by car drivers [Boe17] and one study were pedestrians used a smartphone at street crossings [Sob17].
Transferable scenarios
Scenarios that would be feasible to transfer into vir-tual reality without special considerations are scenar-ios with low lateral motion and few head turns. The scenario should also preferably not require the sub-ject to interact with any hardware inside the cabin. Although the best candidates would be the scenar-ios where the low lateral motion, few head turns and no interior interaction is required. Calculating the in-tersection of segmented data gives us 84 (41%) po-tential candidates studies which probably could be moved into a simulation environment where an HMD is employed as display.
Conclusion
Although most of the driving simulation studies are still performed using simulators equipped with pro-jectors or large displays, it is possible to use HMD systems instead. One does need to be mindful of the current drawbacks of the technology and design the simulation scenarios accordingly. This study sug-gests that despite the current limitation of the HMDs, as many as 40% of the reviewed studies could have been performed using an HMD.
-165-Exploring the suitability of virtual reality for driving simulation DSC2018EuropeVR
Acknowledgments
This project is mainly funded by the VINNOVA/FFI project Chronos 2. Additional funding have also been provided by the Swedish National Road and Trans-port Research Institute.
References
R. S. R. Allison, L. R. Harris, M. Jenkin, U. Jasiobedzka, J. E. J. Zacher and K. Street,Tolerance of temporal delay in virtual en-vironments, in IEEE Virtual Reality 2001 Proceedings, 247–254,
IEEE Computer Society, Yokohama, Japan, 2001.
B. Aykent, Y. Zhao, F. Merienne and A. Kemeny,Simulation sick-ness comparison between a limited field of view virtual reality head mounted display (Oculus) and a medium range field of view static ecological driving simulator (Eco2), in Proceedings
of the Driving Simulation Conference 2014, Driving Simulation As-sociation, 2014.
D. R. Badcock, S. Palmisano and J. G. May,Vision and Virtual Environments, in K. S. Hale and K. M. Stanney, eds., Handbook of
Virtual Environments: Design, Implementation, and Applications, 39–85, CRC Press, Boca Raton, FL, USA, second ed., 2014, ISBN 9781466511842.
B. Blissing,Tracking techniques for automotive virtual reality,
Tech. Rep. 25A-2016, Swedish National Road and Transport Re-search Institute, 2016.
B. Blissing, F. Bruzelius and O. Eriksson, Driver behavior in mixed and virtual reality – a comparative study, in Proceedings
of the Driving Simulation Conference 2016 VR, 179–186, Driving Simulation Association, Paris, France, 2016.
S. Boets, C. Desmet, D. Knowles, A. Pommer and S. Kaiser, De-tection of motorcyclists by car drivers with and without mo-torcycling experience. A virtual reality driving simulator study including eye-tracking., in Proceedings of the 2017 Road Safety
& Simulation International Conference, The Hague, The Nether-lands, 2017.
D. A. Bowman and R. P. McMahan,Virtual Reality: How Much Immersion Is Enough?, Computer, vol. 40(7): 36–43, 2007.
T. J. Buker, D. A. Vincenzi and J. E. Deaton,The Effect of Ap-parent Latency on Simulator Sickness While Using a See-Through Helmet-Mounted Display: Reducing Apparent La-tency With Predictive Compensation, Human Factors: The
Journal of the Human Factors and Ergonomics Society, vol. 54(2): 235–249, 2012.
F. Colombet, A. Kemeny and P. George,Motion sickness com-parison between a CAVE environment and a HMD, in
Proceed-ings of the Driving Simulation Conference 2016 VR, Driving Simu-lation Association, Paris, France, 2016.
S. Ebenholtz, M. M Cohen and B. Linder,The possible role of nystagmus in motion sickness: A hypothesis, Aviation, space,
and environmental medicine, vol. 65: 1032–5, 1994.
E. Foxlin,Head tracking relative to a moving vehicle or simula-tor platform using differential inertial sensors, Proceedings of
SPIE: Helmet and Head-Mounted Displays V, vol. 4021: 133–144, 2000.
J. Jerald, The VR Book: Human-Centered Design for Virtual Reality, Association for Computing Machinery and Morgan &
Clay-pool, New York, NY, USA, 2016, ISBN 978-1-97000-112-9. V. Judalet, S. Glaser, V. Kocher and D. Charondiere,Virtual reality for real driving: a tool to fill the gap between simulators and test tracks, in Proceedings of the Driving Simulation Conference
2012, Driving Simulation Association, Paris, France, 2012. A. Kemeny,From Driving Simulation to Virtual Reality, in
Pro-ceedings of the 2014 Virtual Reality International Conference, VRIC ’14, 32:1–32:5, ACM, New York, NY, USA, 2014.
R. S. Kennedy, N. E. Lane, K. S. Berbaum and M. G. Lilienthal,
Simulator Sickness Questionnaire: An Enhanced Method for Quantifying Simulator Sickness, The International Journal of
Aviation Psychology, vol. 3(3): 203–220, 1993.
B. Keshavarz and H. Hecht,Validating an Efficient Method to Quantify Motion Sickness, Human Factors, vol. 53(4): 415–426,
2011.
A. Lassagne, A. Kemeny, J. Posselt and F. Merienne,Comparing tangible and fully virtual haptic systems for HMI studies in driving situations, in Proceedings of the Driving Simulation
Con-ference Europe 2017, 163–166, Driving Simulation Association, Stuttgart, Germany, 2017.
K. Mania, B. D. Adelstein, S. R. Ellis and M. I. Hill,Perceptual sen-sitivity to head tracking latency in virtual environments with varying degrees of scene complexity, in Proceedings of the 1st
Symposium on Applied perception in graphics and visualization -APGV ’04, study 1, 39–48, ACM Press, New York, New York, USA, 2004.
N. Matsuda, A. Fix and D. Lanman,Focal Surface Displays, ACM
Trans. Graph., vol. 36(4): 86:1–86:14, 2017.
M. Meehan, S. Razzaque, M. C. Whitton and F. P. Brooks, Ef-fect of latency on presence in stressful virtual environments,
in IEEE Virtual Reality, 2003. Proceedings., vol. 2003, 141–148, IEEE Computer Society, 2003.
G. Papadakis, K. Mania, M. Coxon and E. Koutroulis,The effect of tracking delay on awareness states in immersive virtual en-vironments: an initial exploration, in Proceedings of the 10th
International Conference on Virtual Reality Continuum and Its Ap-plications in Industry, 475–482, ACM Press, Hong Kong, China, 2012.
J. Prothero and D. Parker,A Unified Approach to Presence and Motion Sickness, in L. J. Hettinger and M. W. Haas, eds.,
Vir-tual and Adaptive Environments: Applications, Implications, and Human Performance Issues, chap. 3, 47–66, CRC Press, Boca Raton, FL, USA, 2003.
J. Reason and J. J. Brand,Motion sickness, Academic Press
London ; New York, 1975, ISBN 0125840500.
G. E. Riccio and T. A. Stoffregen,An ecological Theory of Mo-tion Sickness and Postural Instability, Ecological Psychology,
vol. 3(3): 195–240, 1991.
B. Ricaud, R. Lietar and C. Joly,Are Virtual Reality headsets effi-cient for remote driving?, in Proceedings of the 5th International
Conference on Road Safety and Simulation, University of Central Florida, Orlando, FL, USA, 2015.
T. Shibata, J. Kim, D. M. Hoffman and M. S. Banks, The zone of comfort: Predicting visual discomfort with stereo displays,
Journal of Vision, vol. 11(8): 11–11, 2011.
A. Sobhani, B. Farooq and Z. Zhong,Immersive Head Mounted Virtual Reality based Safety Analysis of Smartphone Dis-tracted Pedestrians at Street Crossing, in Proceedings of the
2017 Road Safety & Simulation International Conference, The Hague, The Netherlands, 2017.
M. Suzuki and T. Yai,Riding Performance Evalution of Cycling Simulator ”Morics” Considering Rider’s Controllability, in
Pro-ceedings of the 5th International Conference on Road Safety and Simulation, University of Central Florida, Orlando, FL, USA, 2015. M. Suzuki, K. Miyanoue and T. Yai,A Fundamental Study about Naturalistic Cycling on Simulator ”MORICS1”, in Proceedings
of the 2017 Road Safety & Simulation International Conference, The Hague, The Netherlands, 2017.
The Commission of the European Communities,Commission Di-rective 2009/112/EC, Official Journal of the European Union, vol.
223: 26—30, 2013.
M. Treisman, Motion sickness: an evolutionary hypothesis,
Science, vol. 197(4302): 493–495, 1977.
F. Weidner, A. Hoesch, S. Poeschl and W. Broll,Comparing VR and non-VR driving simulations: An experimental user study,
in 2017 IEEE Virtual Reality (VR), 281–282, IEEE, 2017, ISBN 978-1-5090-6647-6.
A. J. Woods, Crosstalk in stereoscopic displays: a review,
Journal of Electronic Imaging, vol. 21(4): 040902, 2012.
