Linköping University | Department of Computer and Information Science (IDA) Bachelor thesis, 16 ECTS | Innovativ Programmering Spring Semester 2020 | LIU-IDA/LITH-EX-G--20/051--SE
Linköping University SE-581 83 Linköping +46 013-28 10 00, www.liu.se
Evaluating an ARCore application to get an image of the
state of AR technology today
Utvärdering av en ARCore applikation för att få en bild av AR
teknologi idag
Oliver Ekstrand
Sebastian Lundqvist
Supervisor: Jody Foo Examiner: Peter Dalenius
- 1 -
Evaluating an ARCore application to get an image of the
state of AR technology today
Oliver Ekstrand
Sebastian Lundqvist
Oliek365@student.liu.se
Seblu114@student.liu.se
ABSTRACT
Augmented reality is an old technology that is still far away from being perfect. It is also quickly being improved upon and the state of AR today has come a long way from AR just a couple of years ago. New big players have recently introduced their tools and have made it easier than ever to develop AR applications. In this study we look at what established methods (if any) there are for AR evaluation, develop AR evaluation methods that fit our needs, carry out the evaluation and analyze the collected data. We also note some important things to think about when working with AR to increase tracking and recognition stability. The recommendations are: try to have reference images with high scores, have reference objects that are distinct enough from one another to not be mixed up and make sure that the visual for the reference image matches the visual for the reference object in its intended viewing environment. INTRODUCTION
With studies finding evidence that immersive elements increase visitor’s interest in an exhibition [1], there comes a need to know both the possibilities and the limitations that AR as a technology presents today. AR in cultural heritage has been a topic of interest for several years, as exemplified by Damala’s case study in 2007 [2] and later, the survey by Bekele et al. in 2018 [1].
In this study, we collaborate with Östergötlands Museum on an application that can fit in as part of or as one of their exhibitions. Speaking to an art curator at Östergötlands Museum about their current projects revealed that finding a way to tell different stories with varying perspectives seemed to be of interest. The museum’s collection consists of many paintings where not only the motifs portrayed could be of interest, but also the painter, the era in which it was painted and the importance to specific social movements can be relevant to convey as well [Klein, Personal Communication]. Finding a way to exhibit all these stories could be a way to attract a larger audience with different interests in art.
In this study, we aim to get an image of the current state of AR technology by creating an ARCore application that augments pictures set up on a wall and analysing its image recognition and tracking abilities. In doing this we hope to help augmented reality experience designers make more informed decisions and point AR developers towards features that can be improved.
Research Problem
• Evaluate augmented reality performance by measuring tracking stability and how it is affected by movement and surface textures.
Delimitation
This study will focus solely on ARCore and more specifically will evaluate its augmented images module. BACKGROUND AND RELATED WORK
What is AR?
Azuma [3] defines Augmented Reality (AR) as a system that has the following three characteristics:
1. Combines real and virtual environments 2. Has to be interactive in real-time 3. Has to be registered in 3-D
Therefore, a movie with blended virtual elements does not count as it is not interactive, and Virtual Reality (VR) is not AR as VR is entirely virtual [3].
While augmented reality is not a new technology, with the first AR-prototype system being created in 1968 at MIT [4], it is a quickly evolving field with many recent developments. Several recent AR-surveys look at development tools like ARToolKit1, Vuforia2 and Wikitude 3and as such are losing some of their relevance as the two tech giants Apple and Google entered the field with their AR-development solutions ARKit4 and ARCore5 as late as 2017 and 2018 respectively. 1 http://www.hitl.washington.edu/artoolkit/ 2 https://developer.vuforia.com/ 3 https://www.wikitude.com/ 4 https://developer.apple.com/augmented-reality/ 5 https://developers.google.com/ar
ARCore
ARCore is Google’s platform and according to Google it focuses on three key capabilities [5]:
• Motion tracking, which means that the phone can understand and track its position in relation to the world
• Environmental understanding, which means that the phone can detect and locate all types of surfaces.
• Light estimation, which allows for estimations of the environments current lighting
Because of the collaboration with Östergötlands Museum, the study will focus on the image recognition feature of ARCore, “Augmented Images”, a feature that allows the creation of AR applications in which 2D images can be recognized and responded to.
A developer simply has to supply a reference image for ARCore. This is done by adding the pictures to an image database, either before compilation or at runtime e.g. by taking a picture. The database does store the image itself but a compressed representation of it, thus pictures of a higher resolution do not lead to better results.
The recognition here is quite simple and relies entirely on points of high contrast, and as such images with few distinct features or pictures with repetitive features are harder to recognize and should be avoided if possible [5]. When a reference image is given to the database given in, ARCore gives the image a score, representing the quality of the image as a reference.
AR in Cultural Heritage
Some surveys are studying the relevance of AR-applications in cultural heritage settings. Bekele et al. [1] define five potential purposes for AR-applications in cultural heritage. One of the categories being exhibition enhancement, meant to enhance the experience for visitors to an exhibition. This enhancement implies the superimposition of virtual elements upon the real world. Some studies mentioned in this survey show evidence that interest in an exhibition increases when immersive elements are added. This points to the potential for a museum to increase its attractiveness by augmenting its exhibitions with digital media.
There are however noted limitations to the adoption of AR in cultural heritage that can be divided into technological limitations, content complexity and human factors. Some of the mentioned limitations pertain to sensor tracking that could be improved and potential file sizes that may need to be downloaded from the internet [1].
Several studies focus on the question of whether AR technology is the best solution for the given problems [1,2]. One question asked is whether AR is the best solution as a
museum guide or if an audio guide would be a better solution [6]. Another focus in older articles was the lack of common AR design guidelines [2], which is something that has changed in recent years when seeing the very similar design guidelines offered by Google’s ARCore [7] and Apple’s ARKit [8]. According to Damala [2], evaluation points of AR-applications as museum guides can be divided into three categories: technology chosen, interactive content and logistics/administration [2]. Under these categories, much of the weight of these assessments is carried by application development issues, content curation and logistical concerns for AR-experience delivery. Which is understandable given the nature of AR-experiences in cultural heritage, but it brings the focus away from the technological stability of augmented reality.
Only one of the studies mentioned by Damala, Dünser et al. has responsiveness/feedback and error tolerance as criteria. This guideline discusses the effect technological limitations such as system lag, slow tracking speed and tracking stability have on AR-application design [2,9]. Tracking stability, in particular, is vulnerable to unreliable sensor data. While a combination of different tracking methods can help mitigate inaccuracies through hybrid tracking, the small errors can accumulate to create noticeable tracking errors [1,9]. The unreliability of AR-tracking technology is a limiting factor in experience design. With AR-platform creators putting a big emphasis on immersion [7,8], designers have to consider and design around the technological limitations to achieve a desirable level of immersion. Therefore, there is a need to evaluate the technology and attempt to establish the limitations augmented reality has as a technology to help experience-designers make more informed decisions.
Virtual Object Stability Evaluation
Outside of cultural heritage, there are a few system stability evaluations, focusing on a few different areas; There is a study focusing on hologram stability for Microsoft HoloLens by Vassallo et al. [10]. “Hologram” being Microsoft’s term for their projected virtual objects. The study evaluates HoloLens for potential use in a clinical setting by testing how actions that are expected to occur in such a setting affect hologram stability. These actions are defined as “Walking”, “Sudden Acceleration”, “Occlusion” and “Object Insertion”. Of these actions, the most relevant are “Walking” and “Sudden Acceleration” since the application used in our study has no support for recognizing objects placed in front of or inside the virtual object. Only “Walking” and “Sudden Acceleration” involve moving with the device equipped. “Walking” involves turning and walking away from the virtual object and then back to see if the virtual object is still in place. “Sudden Acceleration” involves suddenly moving the head to the side and back to interrupt spatial mapping. The accuracy for the placement for the virtual object was measured by marking the corners
of the virtual object’s placement in the real world and then measuring the distance between the markings made after an action is performed [10].
In a study made for a potential app for use in an NPP (Nuclear Power Plant) maintenance setting, there was a performance evaluation intended to consider the feasibility of an assistant AR-application in such a setting. The evaluation was performed in a room with no imagery that could be confused with the tested markers and an average lighting of 1050 lux in the room. The performance metrics tested were: the time needed to process and recognize a marker, and impact on frame rate; the distance where the AR-device stops being able to recognize the markers; and the impact of camera rotation [11].
In a study evaluating an AR-system intended for military use in a natural setting, five metrics are defined to measure system performance by observing a virtual object over time. These metrics are named: “Jitter” meaning high-frequency movement and looks like shaking, “Wander” meaning low-frequency movement that looks like the object wanders away, “Lag” referring to an object falling behind when the device moves, “Bounce” referring to the object moving up and down when the device carrier starts or stops walking, and “Accuracy” which measures the accuracy of the position compared to the real position it is anchored to. Of the five metrics, “Accuracy” is the only test where the AR-device would be mounted on a soldier. In this test, the soldier would carry out a military exercise in a natural setting while the system is being evaluated. For the other metrics, the AR-device is mounted on a machine which carries out any movement necessary to acquire test data [12]. These tests focus on 2D virtual objects and the metrics, therefore, do not fit this study since 3D objects have a third plane to consider. A 2D object can, for example, not “Wiggle” or “Rotate” as they are described (see Appendix). “Drift” is similar to “Wander”, but “Wander” is a metric meant to examine if the virtual object moves when the AR-device is still, something that is not relevant for this study. “Drift” also considers movement in 3D space, which “Wander” does not.
METHOD
This thesis consists of creating an application in Unity with the help of the ARCore SDK and evaluating the tracking stability in said application measured by the effect of the following parameters:
• Device movement • Surface textures
Light conditions, surface textures and sensor measurement error accumulation are discussed as being the main issues behind tracking instability [2,9]. While light is an interesting parameter to consider, due to the limitations in this study, the primary focus is on movement, while glossy
and matte surfaces are tested to see if there is an effect on tracking and recognition performance.
Implementation
The application was developed on Windows in Unity 2019.3 with ARCore SDK for Unity v1.16.0. The necessary coding was done in C# and the minimum android SDK was 7.0 (API level 24), the lowest version possible for "AR Required" apps. And this was done to reach the widest possible compatibility level, and thus the most potential users. The application was built for Android 9.0 (API level 28) on the phone used for testing. The code is based on Google's example for Augmented Images that are added with the ARCore SDK for Unity6.
The application recognizes images that have been added to its database and then it tracks them. This is visualized by creating four corner pieces that mark the corners of the recognized image and a thin cuboid, acting as a nameplate, showing the name of the specific image. The text used in the nameplate is retrieved from a text file.
Evaluation
The evaluated application creates a virtual object attached to a real object found by a reference image given to the application. This virtual object was observed in an evenly lit environment and the performance of the app was evaluated by its recognition speed and tracking stability in the testing environment. If the virtual object managed to stay in place during device movement it was considered stable, while displacements were categorised based on the severity and type of visual instability. The tests were done on a Samsung S20+ running Android 10 (API level 29) and were recorded with stock screen capture software on the device, with no other apps running in the background and the captured videos are available in a playlist on YouTube7.
The tested environments consisted of an empty room with stable lighting conditions where all reference objects were set in a determined position.
Light conditions were measured with a lux-meter iPhone application from Arbetsmiljöverket8 to make sure all reference objects received an even amount of light. All reference objects had light conditions of roughly 300 lux (±10%) according to Arbetsmiljöverkets application.
6 https://github.com/google-ar/arcore-unity-sdk/releases 7
https://www.youtube.com/playlist?list=PLTKPnYv9PtIui5 V54Z7GK52ILPPkEA30T
Different surface textures were tested by adding reference images of both glossy posters and matte newspapers and observing the effects they had on tracking.
To simulate the real movement the device would be used in, the device was handheld while carrying out the movements to be tested. To approach a good average speed while the device is handheld, the time it takes for the image observed in the camera to move from one point to another was measured. The points were marked in the testing environment. This testing method allowed both parallel movement and device rotation to be tested.
Four tests were carried out with the handheld device. Two of these consisted of walking at two different speeds while the other two consisted of rotating at different speeds. The walking tests consisted of walking along a 7,7-meter line that was placed 2 meters away from the reference objects, see Figure 1. Two different average speeds were tested, one with an average speed of 0,5 meters/second and the other with an average speed of 0,3 m/s - these were designated as “Walk” and “Slow Walk” respectively. The rotation tests consisted of standing 2 meters away from each wall when measured at a right angle, see Figure 2. These were also carried out at two different speeds and were designated as “Rotation” and “Slow Rotation”
respectively. “Rotation” had an average rotation speed of 10 degrees per second while “Slow Rotation” had an average rotation speed of 8 degrees per second. Every walking test ended with a rotation after “Annan Reklam” and finish on “Fogde”, this would technically make “Fogde” a rotation test instead of a walking test and was therefore considered a special case.
Ten reference images were used in the test. Six of these were posters with a glossy surface and a similar colour palette, while the other four were newspaper pages with a matte surface, see table 1.
All tests were recorded and analysed to write down the time taken for image recognition and mark visual inconsistencies. Image recognition time was calculated by subtracting the time for the first frame in which a portion of the reference object appeared from the time for the frame in which the virtual object appeared. Visual inconsistencies were observed and graded according to a subjective appreciation of how intrusive they were and the type of inconsistency. The levels of severity were graded as minor, medium and major; while the types of inconsistencies were described as “Wiggle”, “Drift”, “Rotation” and “Image not recognized”.
Table 1: Reference Images
Jayne Simon
Wash Zoe Kaylee Mal
Vertikal Reklam
Ljus Reklam Annan Reklam
All data was recorded in spreadsheets9 where image data, test data, calculated time averages and criteria terms are further described (see Appendix).
RESULTS
All posters had varying levels of recognition times and rates. If “Vertikal Reklam” is excluded, all newspapers had a 100% recognition rate with a lower average recognition time than the posters, see Table 2.
Considering the visual inconsistencies, the “reklam” images, excluding “Vertikal Reklam”, were the most stable, with the only inconsistencies found in “Walk 2” and “Rotation 2”. All the posters often showed visual inconsistencies throughout most tests. “Fogde” however, which also was a newspaper, did show some inconsistencies.
Only one poster, “Mal”, was recognized in all tests, but also had a relatively high average recognition time of about three seconds. There were seven instances of major inconsistencies in the “Walk” tests, three of which were for “Mal” and “Simon”, while four were for “Fogde”. In “Slow Walk” there were ten instances of major inconsistencies across all eight tests, four of which were connected to “Mal”, two were connected to “Zoe” and one was for “Simon”. In “Rotation” tests there was one major inconsistency for “Mal”, while “Slow Rotation” had one major for “Mal” and one major for “Wash”.
DISCUSSION Results
Of all the image examples given, the matte newspapers scored the best across the board according to ARCore, ranging between 60 to 100, while the monotone glossy posters had their highest score at 15, with most of them averaging 2.5. There was an intention to try two more glossy posters that had a score of 100 and 75, but presumably because of their size, were not recognized at the 2-meter distance decided on for the tests. “Vertikal Reklam” was a matte newspaper page that had a relatively
9
https://docs.google.com/spreadsheets/d/1PakBP8y5bTjQ7J
ZX4XoO14DNWqF-XWdWpsPufvpAhYQ/edit?usp=sharing
high score at 75 but was only recognized in one test, “Walk 3”, despite its appearance in all tests. Our guess for the reason behind the lacking recognition of “Vertikal Reklam” is that it loosened its grip to the wall and fell (after “Walk 3”), which meant that we had to set it up again and the result might have been that the reference image did no longer match the object on the wall. Unfortunately, the time restrictions for the study did not allow for testing uniquely with “Vertical Reklam” without moving the AR device to test this theory. Another theory which seemed to work for the two posters we decided to not use in the study, is the effect of the real size of the objects pictured in the reference images. “Vertikal Reklam” had a real width of 27 cm while the unused posters had widths of 22 and 33 cm, with the height of the 33 cm poster being 22 cm. The posters that were used in the study had a width of 30 cm, but also had a height of 60 cm which might have made all the difference. This could explain it only being recognized in one test before it fell.
Looking at the data, there seemed to be a connection between the score given to each image and their resulting stability. The images with higher scores tended towards not showing any inconsistencies and showed perfect recognition rates, while the data for the posters showed variations in performance. The posters only had one entry, “Mal”, that had a 100% recognition rate. The average recognition time for “Mal” however was among the highest at 3.3 seconds. On the other side of the spectrum, “Kaylee” had a recognition rate of about 79% while the recognition time was on the lower end at 1.8 seconds. With this in mind there could be a connection made between the expected stability of an object and their given score, but more so a general expectation of instability rather than a direct connection to either visual inconsistencies or recognition
Figure 2: Overview of the Rotation test area Figure 1: Overview of the Walking test area
rate. A study with bigger resources willing to find a more objective conclusion on the impact of surface textures on stability could make glossy and matte versions of the same image while keeping the same object dimensions to potentially get more representative results.
Two different types of movements were tested in this study. These movements required a user to hold the device in their hand while moving and focusing the camera image on the augmented objects. The decision to not use a setup that could hold a stable speed while carrying the device is one grounded in the intentions for how the application is meant to be used, which in this case means handheld while walking around an exhibition. The chosen speeds used in the test, however, could have used an even slower speed or even a test where the user stands still for a while looking at every augmented object, something that might come closer to the way a museum visitor might behave. These might have provided interesting data that could have given the application a fairer chance at recognizing each referenced object.
The performance could have been negatively affected by running the screen recording software while testing, but as all tests were done on the same phone, under the same conditions the results should be comparable.
There was only one instance where an object seemed to have been mixed up with another. This is what we assumed
happened during the only instances of major drift failures seen in “Slow Walk 8”. In “Slow Walk 8”, “Simon” had already been recognized but seemed to respawn again on top of “Mal” together with “Zoe” and “Wash” in what seems to be the same plane of rotation. One assumption is that the mix-up occurred because of the similar colour palettes found in “Simon” and “Mal”. Another thing that could have happened is that the application lost the anchors for the augmented objects, which may have led to the virtual objects drifting away.
Method
In the literature found in this study, there was a lack of established methods for augmented reality application evaluation from a technical point of view. Most studies seemed to focus on usability or were made to fit a more specific solution. Even if there was some inspiration to be gained from the more specialized evaluations, there were a lot of methodological questions that had to be designed for this study. The study by Ishii et al
.
[11] had an evaluation in an evenly lit, empty room for the sole purpose of testing their system, a setup used in this study. The study by Roberts et al. [12] defined several types of visual inconsistencies that did not fit this study’s needs but inspired the categorisation of errors. The study on HoloLens by Vassallo et al. [10] focused on the impact of movement on the tracking, which is something this studyAverage Results
Image Name Fogde Annan
Reklam
Ljus reklam
Vertikal reklam
Mal Kaylee Zoe Wash Simon Jayne
Real Width 0,55m 0,55m 0,55m 0,27m 0,3m 0,3m 0,3m 0,3m 0,3m 0,3m Image Quality (ARCore)* 60/100 100/100 80/100 75/100 0/100 0/100 5/100 5/100 15/100 0/100 Recognition Rate 100% 100% 100% ~ 5.26% 100% ~ 79% ~ 79% ~ 68.42% ~ 73.68% ~ 84.21% Average Recognition Time (Overall) 1.163 1.360 1.584 0.944 3.304 1.845 2.505 4.157 3.231 2.544 Average Recognition Time (Walking) 1.163 1.083 1.212 0.944 3.603 1.788 1.343 2.055 4.358 2.378 Average Recognition Time (Rotation) N/A 1.637 1.955 N/A 3.005 1.902 3.668 6.260 2.104 2.711
focuses on. HoloLens uses different techniques from ARCore however, which only made the “Walking” and “Sudden Acceleration” relevant for evaluating ARCore. In essence, the point of the HoloLens study was to observe HoloLens ability to deliver an accurate spatial mapping so that the Hologram remains in position after certain interruptions. This study, however, focuses on how well ARCore works in movement while still keeping the augmented images in frame.
In this study, we decided to make a subjective analysis of what we perceived to be different degrees of severity in the visual inconsistencies found. The scale of this part of the study could have been increased by making an empirical study with many participants that could have graded the different visual inconsistencies according to their own biases. This would allow a study to grade the performance according to what is more generally perceived as intrusive or immersion breaking.
Considering that an important factor in whether an object will be recognized or not is the time given for recognition, the way the walking tests were setup could be troublesome. The walking tests included a small rotation at the end of the walking segment, which might have led to a bias towards standing still before rotating. Continuing walking after the last object in the walking tests might have given a more even chance to appear for all virtual objects. In this case, however, the recognition times for the “Annan Reklam” images were so fast that the potential lingering would have made no difference.
The Work in a Larger Context
Damala mentions in her survey that there is a risk when using AR, that the medium takes away focus from the work of art it is supposed to augment [2]. In implementing a virtual guide in a museum instead of hiring human guides, there is a risk of human guides losing potential jobs taken over by smartphones. At the same time, it makes it easier for an art curator to tell the stories they want to tell when a visitor wants to experience them.
CONCLUSION
Augmented reality as a technology is still a work in progress with a lot of work that is needed to improve tracking performance and image recognition. This requires augmented reality experience designers to consider the limitations found in the technology by making decisions that can decrease the chances for visual inconsistencies to occur. Some of these are to try to have reference images with high scores, having reference objects that are distinct enough from one another to not be mixed up and make sure that the visual for the reference object matches the reference image - considering potential changes in lighting conditions and object surface texture.
Future Work
This paper focuses on ARCore to get an idea of the state of AR today. A more expansive study could have made similar apps through other big actors in AR such as Wikitude SDK and Apple’s ARKit while doing the same experiment to compare and analyse the data. A study more interested in seeing what is considered acceptable when it comes to visual inconsistencies could carry out a survey asking testers to rate different occurrences of visual inconsistencies based on their intrusiveness. Finally, a study with an interest in the effects of different surface textures on image recognition and tracking, could make different degrees of matte and glossy versions of the same images and make a study similar to this one.
REFERENCES
[1] M. K. Bekele et al., "A Survey of Augmented, Virtual, and Mixed Reality for Cultural Heritage",
Journal on Computing and Cultural Heritage, vol.
11, No. 2, pp. 7:1-7:36, 2018. Available: https://dl.acm.org/doi/10.1145/3145534. [Accessed: Feb. 15, 2020].
[2] A. Damala, Interaction Design and Evaluation of Mobile Guides for the Museum Visit: A Case Study in Multimedia and Mobile Augmented Reality. Ph.D. [Dissertation]. Paris, France: CNAM, 2009. [Online]. Available: TEL
[3] R. T. Azuma, "A Survey of Augmented Reality",
Presence: Teleoperators and Virtual
Environments, vol. 6, No. 4, pp. 355-385, 1997.
Available:
https://www.mitpressjournals.org/doi/10.1162/pres .1997.6.4.355. [Accessed: Feb. 15, 2020].
[4] M. Billinghurst, A. Clark and G. Lee, A Survey of Augmented Reality. Boston, MA: Now, 2015 [5] Google, “ARCore overview”. [Online]. Available:
https://developers.google.com/ar/discover. [Accessed: May 4, 2020].
[6] K. Chang et al., "Development and behavioral pattern analysis of a mobile guide system with augmented reality for painting appreciation instruction in an art museum", Computers &
Education, vol. 71, pp. 185-197, 2014. Available:
https://www.sciencedirect.com/science/article/abs/ pii/S0360131513002868. [Accessed 4 May 2020]. [7] Google, “Augmented Reality Design Guidelines”.
[Online]. Available:
https://designguidelines.withgoogle.com/ar-design/. [Accessed: May 4, 2020].
[8] Apple, “Augmented Reality,” Augmented Reality -
System Capabilities - iOS - Human Interface Guidelines. [Online]. Available:
https://developer.apple.com/design/human-
interface-guidelines/ios/system-capabilities/augmented-reality/. [Accessed: May 4, 2020]
[9] A. Dünser et al., “Applying HCI principles to AR systems design,” in Proc. of the MRUI'07: 2nd
International Workshop at the IEEE Virtual Reality 2007 Conference, March 11, 2007, Charlotte, NC [Online] Available: UC Research
Repository,
https://ir.canterbury.ac.nz/handle/10092/2340# [Accessed: May 1, 2020]
[10] R. Vassallo et al., “Hologram stability evaluation for Microsoft HoloLens,” in SPIE Medical
Imaging 2017: Image Perception, Observer Performance, and Technology Assessment, Feb. 11-16, 2017, Orlando, FL [Online]. Available:
SPIE Digital Library,
https://www.spiedigitallibrary.org/conference- proceedings-of-spie/10136/1013614/Hologram-
stability-evaluation-for-Microsoft-HoloLens/10.1117/12.2255831.short. [Accessed: 8 May. 2020].
[11] H. Ishii et al., "Development of Marker-based Tracking Methods for Augmented Reality Applied to NPP Maintenance Work Support and its Experimental Evaluation," in NPIC and HMIT
2006: 5th International Topical Meeting on
Nuclear Plant Instrumentation, Control
and Human-Machine Interface Technologies (), Nov. 12-16, 2006. Albuquerque, NM, American
Nuclear Society, Eds., LaGrange Park, IL: American Nuclear Society, 2006. pp.973-980. [12] D. Roberts et al., "Testing and evaluation of a wearable augmented reality system for natural outdoor environments," in SPIE Defense
Commercial Sensing: Head- and Helmet-Mounted Displays XVIII: Design and Applications, April 29- May 3, 2013, Baltimore, MD [Online] Available: SPIE Digital Library,
https://www.spiedigitallibrary.org/conference- proceedings-of-spie/8735/87350A/Testing-and- evaluation-of-a-wearable-augmented-reality-system-for/10.1117/12.2015621.short
.
[Accessed 15 May 2020].APPENDIX
