Examensarbete på kandidatnivå, 15 hp Systemvetenskapliga programmet
SPB 2020.33
REALITY CHECK
A review of design principles
within emergent XR artefacts
2
Abstract
3
Acknowledgments
Many thanks to Mikael Wiberg for his truly invaluable mentorship!
A warm thanks to Anders Markstedt for this opportunity, and the team at CGI for providing great advice, jokes and working environment - despite current events. And a special dedication of love to my friends and family that had to endure all my
4
Table of Contents
1. Background ... 5
2. The problem area ... 6
3. Purpose of the study ... 7
4. Related research ... 8
Assessment of current XR design guidelines ... 8
5. Methodology ... 10
5.1 Research through design ... 10
5.2 Prototype testing methodology ... 11
6. Results ... 12
6.1 Prototype design ... 12
6.2 Practical prototype testing ... 14
7 Discussion ... 16
7.1 Are current design paradigms within XR sufficient? ... 16
7.2 A quick evaluation of physical/tactile feedback within XR ... 17
7.3 Limitations of the study ... 18
8. General conclusions ... 20
5
1. Background
Virtual, augmented, and mixed reality environments as a whole are growing at an incredible pace, seeing rapid adoption within a multitude of areas (Chuah, 2019). But it is especially within the enterprise space that the most promising developments are slated to happen in the near future (Davis, 2020), likely due to how the high cost of hardware associated with these technologies are more manageable within the corporate sectors, along with the risks associated with being an early adopter of the tech.
The commodification of these extended realities (XR1) are slated to be the next big
revolution in how we interface with, interact and experience digital artefacts on a fundamental level. Through either replacing our perceived reality completely with a virtual one or augmenting and mixing the real world with digital artefacts, the gap between the digital and physical is quickly closing and blurring the previously clear lines between the two are getting equally blurry. This seems to be the trend for new technological developments in this century overall, with specifics such as the Internet of Things (IoT) promising to enabling having sensors in nearly everything – both virtually and physically. (Cerf, 2015)
With how the current coronavirus epidemic is playing out, there is now more emphasis than ever on our need for efficient ways to work and collaborate anywhere, making the push for proper virtual replacements and/or enhancements of our environment just as strong. Particularly virtual meetings is an area where the utilisation of XR has been estimated to accelerate tremendously, especially within the corporate sector. (Coronavirus: Dutch work
on XR technology shows promise in pandemic scenario, 2020)
And while this is in many ways great news for the associated technologies, in terms of the capabilities they might bring and the positive changes this could enable, I would like to pose the question if this fast rate of development does not come with its own caveats and problems in terms of what compromised are made to create the end product. (Verner, 2008)
1 There is some controversy regarding the usage of XR to mean extended reality, as the original usage of
6
2. The problem area
There is already a plethora of best practice handbooks, guides, and examples available for us to utilize today. (Staci, Madrid and Harris-White, 2017; Unity, 2019; Microsoft, 2020) But how do we know that these principles are in fact the most applicable ones when it comes to creating good and usable XR artefacts? Are we just re-using old principles out of convenience, without really considering how they specifically apply/relate to or affect XR artefacts (Choi et al., 2006). Upon quick examination of these recommendations, it appears that a lot of the design choices and advice presented are sorely lacking in references to academic research or any form of otherwise acquired knowledge; they are simply presented as is. (Unity, 2019; Microsoft, 2020) While there is no doubt that there is indeed an absolute plethora of practical and truly learned knowledge within these resources - and they are no less useful because of it – it is still a point of concern regarding the lack of cited sources.
This goes in line with findings that establish how research artefacts inherently differ from practical design artefacts. (Zimmerman, Forlizzi and Evenson, 2007) In broad terms, research aims to arrive at what is ultimately right within the given context and current knowledge. So, while an artefact produced in a practical setting is of course often striving for the very same, it might be occurring within a completely different environment under whole new constraints and limitations. Both are widely regarded as correct methods in their own terms, but the underlying reasoning and methodology means that they both have inherent flaws in the resulting artefacts. An academic research artefact might be well intentioned in its strive for finding the most correct way to accomplish something, but once it meets the many variables of actual applications in a practical setting, this might not be the case anymore.
In a similar vein, the constant pressure of quickly developing a commercially viable product may place considerable constraints on the artefacts produced by active design practitioners. Time as well as specific considerations from customers, along with market fluctuations (such as the coronavirus pandemic) is likely to shape the resulting artefact in ways that could potentially end up being less than optimal. (Verner, 2008)
Additionally, while both methods have these perceived flaws inherent to them, they may also carry their own inherent strengths as well; combining the sentiments of both schools of thought and process might end up combining these strengths as well. (Zimmerman, Forlizzi and Evenson, 2007)
7
3. Purpose of the study
The first goal of this thesis is to evaluate some of the current published design principles within XR artefacts and to see what academical research and/or practical knowledge there is which gives them credibility. The study also intends to explore if there are any alternate methodologies and theories that could be used instead, such as alternate design (Linehan
et al., 2014) or even previously established concepts such as how different kinds of feedback
affects the interactions (Freeman et al., 2017).
The intended contributions are to fill a perceived gap within current literature and research on current practices regarding XR-developments, providing for the perceived need for a “reality-check” concerning if current methodologies are in fact the best that they could possibly be in the high-pressure environments that they are being utilized (Verner, 2008). It will also include an experimental prototype that deviates from these contemporary practices, to try and explore how experimentation beyond might give new insights which hopefully applies to both practitioners and researchers alike. This prototype also serves as a way of attempting to bridge the gap between academic knowledge and the type of practical knowledge that the active software industry has acquired from building and testing their products in the field (Zimmerman, Forlizzi and Evenson, 2007).
8
4. Related research
Assessment of current XR design guidelines
In this section we will briefly look at the related research an attempt to ascertain how UI and interactions within XR is currently being implemented by industry leaders such as Unity and Microsoft. Are there any cited design methodologies and concrete research that stipulate these design choices? (Rossi et al., 2020)) Do they follow any other contemporary or novel theories regarding interaction design? Are these designs at all tested and verified on users, or mostly directed towards developers as means to quickly create a novel XR application? (Gabriel, 2008)
The intent with these assessments is not to deliberately stifle or hamper XR development and adoption rates, but rather aid and encourage even further development. If current design paradigms are indeed valid and have backing in research, then the paper will further explore what more could be done upon this basis. If there are any valid criticisms, then suggestions on how to mitigate these will be offered, from both related research and design methodology as well as any novel knowledges that will be gained during the prototyping stages. There will also be an attempt of assessing what larger consequences this line of thinking may have upon the end product in terms of quality and longevity. (Verner, 2008)
Extended reality (XR) is the collective name for augmented, mixed and virtual reality, which makes it sort of an umbrella term for the ways that we are moving computing away from fixed screens, making digital artefacts have a spatial and volumetric space within the real world, or a fully digital world that we display in its stead, or somewhere in between. It is a constantly evolving definition, wherein the technology used to display digital artefacts in our own or a completely artificial world is the common denominator. Virtual reality can be defined as the complete replacement and immersion into a digital world in lieu of the physical one, augmented reality as the anchoring and placement of digital artefacts into our real world, and mixed reality as the intersection between the two – but these definitions are quickly changing and becoming more granular as the technology and use-cases are rapidly evolving. (Parveau and Adda, 2020)
Upon examination of the guidelines and recommendations of industry-leaders such as Unity and Microsoft, it is quite apparent that the avenue of XR design is still in its infancy. Some even refer to VR interface design as “the wild west” due to how uncharted the territory is.(Staci, Madrid and Harris-White, 2017)
When reviewing guidelines publicised by two industry-leaders (Unity, 2019; Microsoft, 2020)as well as an independent article (Staci, Madrid and Harris-White, 2017), a few common points were found. A lot of the main points are all highly centred around the user, ensuring the interactions and experiences are pleasant, always keeping them in control and comfortable using the technology. Several of these points are also in accordance with established academical research. Especially the notion of ensuring that the body ownership illusion (BOI) is maintained at all costs, is incredibly valuable insights for when we attempt to design our own prototype at the later stage.(Bergström, Kilteni and Slater, 2016)
9
as how a low rate of frames-per-second (FPS) often correlates inversely to poor responsiveness which very often results in a high discomfort of the user. This is generally accepted as proven by research (Ware and Balakrishnan, 1994) though few or none of the cited guidebooks and articles link directly to studies like these. However, there is also some good advice given on how to utilize these constraints as a way of enhancing the enjoyment and fun of the XR experience (Jerald, 2015). Notable examples would be the highly minimal designs of XR content such as the game Superhot VR, which utilizes models with very few polygons even in the representation of the humanoid avatars; this way achieving great practical performance on a majority of hardware while still creating an minimalistic and memorable look, perhaps akin to the visual design choices of early videogames games such as Pong and Tetris.
Interestingly, alternate design methodologies such as fictional/speculative design is already utilized within XR content. A great example is the usage of magic, which is often used to abstract the otherwise difficult or tedious to explain elements of digital artefacts, such as objects coming in or out of existence with a mere gesture of our hands or the press of a few keys. (Rasmussen, 2013)
Particularly in some sources (Staci, Madrid and Harris-White, 2017) the references are almost circular, referring to other handbooks and guidelines which in turn also lack any form of academical testing, other than the notion that it seems to have worked well-enough for the relevant company that it was decided to be issued as a guideline or put in a handbook document. Since the reasoning behind many of these guidebook choices are not explicitly specified and sourced, it leaves us questioning why and how something has been done a certain way. This also produces a difficulty when it comes to re-producing or iterating upon a certain method or design, possibly leading to inconsistencies that may have a very real effect on the end product.
10
5. Methodology
5.1 Research through design
Utilizing what we have learned from the previous assessment, a prototype that implements these contemporary design choices will be created through the concept of research through design specifically adapted for human-computer interaction (HCI). In essence, the adaptation of research through design for HCI stemmed from a perceived need for some form of concrete methodology which could unite the practical and academical research artefacts within HCI community, through adapting a formerly suggested methodology for the more specific intricacies of HCI design. One of the major contributions from this methodology is four criteria through which we can evaluate a design artefact: process, invention, relevance as well as extensibility. The process should be as rigorously documented as possible, to enable the most accurate recreations possible for other design researchers, along with the rationale of why specific methods are chosen over others. The invention highlights the need for researchers to demonstrate that the design brings something new to the table, showing what advancements it is making. The relevance is important because of the nature of HCI, since positivist measurements of success is difficult when it comes to this area, instead focus is on proving that the design provides an improvement over the existing ones in a given context. The extensibility ensures that there are ways that this design can be picked up, rationalised, and be further improved on by other researchers and designers. (Zimmerman, Forlizzi and Evenson, 2007)
For this specific example, the end-goal is a functional prototype done in conjunction with a company which is active in the intended market for XR products, to ensure this artefact reaps the benefits of both academical and practical design process principles and lines of thought, while also having the aforementioned guidelines which ensure we review the resulting artefact through sound and established criteria. This essentially means that the artefact which we design will be an attempt to explore what is previously thought to be known, by designing something new with this knowledge as the basis.
The prototype will primarily be developed by myself as part of the explorative study, but in close conjunction with both material help and professional expertise of the employees at a local office of the IT consultant firm, CGI. The company will provide the necessary hardware for XR as well as software and interaction design advice required for this process, in exchange for first-hand access to the knowledge gained within this study. This prototype will also utilize some pre-made assets distributed by the leading companies, in order to accelerate and streamline the initial design iterations somewhat and to see what could be done differently. Through this, we ensure that the final methodology and results are connected to the practical design principles rather than just the academical, in order to try and gain the benefits of both, as research through design intends to accomplish.
11
5.2 Prototype testing methodology
Once the prototype reaches a level of functionality in that is satisfactory in relation to the time constraints inherent to this study, it will be tested against several practitioners within software development and design at the aforementioned company. These users already have some prior experience both developing for and working with XR artefacts, which means that the prototype will be tested against contemporary norms and practices, seeking to test in what manner either of the prototype concepts reach their intended effects. Quantitative test results and large amounts of reliable data is however not the focus, but rather more qualitative assessments in order to see what potential avenues of further exploration these experimental interaction methods might enable. As was discussed in the previous section, it is of high importance that these methods are as detailed as possible, in order to facilitate reliable recreation and understanding of the method itself and why it was chosen. In these initial stages of prospective designs, it could be argued to be more valuable to gain qualitative data to discern if the designs have potential and should be explored further, or if a different approach should be considered. (Cooper, 2012) Additionally, it is simply not practical to try and accomplish large-scale quantitative data collection of this kind of prototype, given the highly specific (and often expensive) hardware setup required. There are also some highly limiting outside factors that have come to play in the choice of evaluation method, but this will be expanded upon further in the later sections.
Therefore, the main method for this testing will effectively be a freeform application of usability testing, wherein the participants interact with the artefact, while we both observe and converse with them in an attempt to discern how the interaction feels and works in a practical sense. (Rubin, 2008) Data will be gathered both through observing the interactions as well as an organic interviewing of the participants both during and after the interaction testing. It is of key importance here to let the participants voice their own thoughts and feelings on the interactions, rather than have them be influenced by the interviewer to give any form of non-objective feedback. Our goal is to discern if this type of interaction might feel intuitive and responsive enough to warrant further, large-scale exploration and testing, not to discern if this specific artefact is the best way to utilize tactile feedback within XR-artefacts. In order to comply with the research through design methodology, it is highly important to document how this is done practically, in order to maximise the ability for other researchers and practitioners to be able to both understand and/or recreate the testing environment and procedures if needed. (Zimmerman, Forlizzi and Evenson, 2007)
12
6. Results
6.1 Prototype design
Upon attempting to design a new prototype utilizing novel ideas, the avenue of physical feedback within XR became a huge concern. For example, let us take the most classical and ubiquitous of interactive elements – a button. These are still used extensively, likely because they are a commonplace element in both physical and virtual interfaces and therefore easily recognizable by practically every user. However, the current implementations when using hand recognition all have one major point of contention – lack of physical feedback when interacting with.
Figure 1. Hand UI example courtesy of LeapMotion; notice the “floating” nature of the UI elements. (LeapMotion, no date)
13
Figure 2. Image showcasing the prototype ready for testing.
This discovery led to the main point of the experimental prototype – to explore if physical feedback can enhance the feeling of precision and responsiveness when utilizing button elements within virtual reality. Since most explorations was done using hand tracking, as this is a quite common interaction method within emergent XR spaces (Jerald, 2015), an idea of utilizing the users’ own hands to provide physical feedback was formed. Practically, this was also very feasible due to the developer resources provided by the supplier of the hand tracking module (formerly Leap Motion, currently Ultraleap), enabling quick and simple prototyping and design iterations. All of the prototyping was done in Unity, since it is a common engine with large market share within XR applications, is compatible with the aforementioned developer resources, but also because this is the particular engine that the local developers and designers already have plenty of experience working with.
14
Figure 3. The rotation function of the prototype used to manipulate an object.
A note is to be made here that this particular design makes multiple assumptions about the user which may or may not always be viable; such as the user having all 5 fingers intact, both hands available, as well as full range precise control of their fingers. If this interaction model is to be iterated upon further, considerations and options should be highly considered to increase accessibility, ensuring that no user is left out due to pre-existing conditions or physical variations. For example, a survey of surgeons who had lost their fingers showed that a vast majority of them claimed no professional disability and continued to practice surgery (Brown, 1982). This means it should be very important that any potential XR tool they might have work with should not be hindered by their physiology, when their own professional ability is not.
6.2 Practical prototype testing
15
The general test methodology itself was very freeform, with users allowed to experiment with the prototype on their own volition, request guidance as they felt it was needed, or abort the testing at any point if they felt any form of discomfort. Responses were gauged by the designer and recorded in the same spontaneous manner as the testing itself was conducted, recording positive and negative reactions as they occurred or were realised later. Participants were encouraged to voice their own reactions and feelings as they arrived during the testing, and special emphasis was put on providing objective feedback rather than trying to sugar-coat or dismiss any negative experiences. After the participants felt like they had explored the prototype to their liking and expressed everything they could think of, there was a brief and freeform post-interview to try and ensure that all aspects of how the interaction felt was covered – from how intuitive pressing elements on your hand was, to how distracting or not XR content itself feels, to what degree the discrepancies between what they felt and saw was affecting them. Generally, all feedback given was recorded by the interviewer during the process and after and is summarized below.
16
7 Discussion
7.1 Are current design paradigms within XR sufficient?
In the short review of related research, guidelines, and resources available for XR designers, it quickly became obvious that a lot of the focal points stem from practical and functional constraints and learnings associated with such a novel and spatial field of work. In practice, they are indeed invaluable for anyone attempting to create a practical and usable experience, relying heavily on hands-on testing and user feedback to evaluate novel products and ideas. This poses the issue that some have with design through research methodology when it is done primarily for practical means; is this truly the best way to do things, or are they guided by outer necessities such as time constraints, budget choices and specific requests by the customer? (Gaver, 2012) Of course, we will not deny that we live in a reality here, where plenty of constraints and limitations exists – even within the “magical” field of XR – therefore anything we design must indeed be very mindful of these practicalities or it simply will not work.
This is where academic research and practical experience designing something with only exploration and product design in mind, can differ wildly. I am not here to proclaim that one is better than the other, but I will press the point that they both have inherent strengths, as this one the basis of why one could/should attempt the research through design approach. (Zimmerman, Forlizzi and Evenson, 2007) For example, I would like to point at the current use of previously only fictional technologies such as brain-machine interfaces (Jantz, Molnar and Alcaide, 2017) and invisible, mid-air physical feedback (Kim and Schneider, 2020) as examples of just how quickly technical limitations can dissipate thanks to the development of novel technologies. If we are suddenly capable of utilizing brain-waves to interact with a digital artefact, there are of course brand-new limitations to consider, but they are likely to be completely different from those inherent in the hand-tracking technology utilized in this study’s prototype. So why should we let the limitations of today steer our interaction designs for tomorrow?
It is because of this that I would strongly implore experimentation and imagination beyond the limitations imposed by current technologies – if any form of true longevity and resilience is to be desired in the final product. Simply put, to design an XR product only for and around the technologies that are available as of right now, might as well be on par with planning its premature obsolescence.
17
7.2 A quick evaluation of physical/tactile feedback within XR
Due to the ongoing pandemic it was unfortunately not possible –practically nor ethically – to attempt any large-scale testing of the physical feedback model. Regrettably, the small data sample of only 5 participants is not enough to draw any concrete conclusions from and so we will not attempt to do so – however, the testing still fills another intended purpose of the study.
The responses gathered here further reinforce that we should indeed try to continuously experiment with different interaction methods than the established ones, due to how strongly the limitations of current technologies and hardware control the design of these models. One of the experienced users proclaimed that the lack of tracking accuracy is exactly why actors such as Leap Motion opted to display their interface as floating objects with only audiovisual feedback; these floating, abstract objects are already so vague/abstracted that there is basically no way to experience any form of dissonance between the representation of your hand and the virtual button. While this might be sufficient for most actions, one could pose the question if it is truly the best practice for every kind of interaction that is typically dedicated to button elements. Actions that have a large impact and should not be able to trigger unintentionally (such as deleting or any other kind of action that cannot be easily undone without some frustration) as well as those that require a high degree of granular precision; fine positional placement, small rotations and scale changes within an editor, etc. We could also consider the impact of making some of these actions nearly subconscious; few of us are born with the ability to type on a keyboard, yet it is still something that eventually comes very naturally to a lot of us due to how we use physical feedback to enable and register inputs.
One should also point out here that the testing methodology was very explorative and in no way determinative in its nature. Instead, in order to fully examine if physical feedback indeed gives finer precision and better feel than the current models, something akin to A/B-testing in a variety of environments and tasks could be attempted. This A/B-testing could also be improved with a much larger group of users, whose previous experiences of XR content are more clearly defined, so we can be sure if we are testing the interaction itself among veteran users, or it maybe we are just testing the novelty of XR itself among junior users. We should also strive for more elaborate and structured post-test interviews in order to try and ascertain as much qualitative data as possible. This was regrettably not possible within our specific tests, as the current circumstances provided a lot of tight time constraints on all participants – we are incredibly happy that they found the time to participate at all!
18
If we now finally take a step back to the cited research through design methodology, we can reasess the prototype results through the 4 criteria that it provides. From a process perspective, it is hopefully easy to see why it was decided to deviate from the established and typical UI element placement in mid-air, and instead – through what was learned in the related research regarding tactile feedback – attempt to place these elements so that it is not only sight which informs the user about the state of an element. In terms of invention, this is currently likely to be the weakest aspect of the prototype, due to a lack of substantial and qualitative data from which we can conform or deny if this way of designing is in fact different from the current XR design conventions. This ties into the point of relevance, which I feel has been argued for sufficiently with how the related research and guidleines available all explain how to do something, but not why. Hopefully, this prototype can showcase the importance of clearly explaning why and how certain design choices were made, all in order to increase the likelyhood of reproducing the design successfully and thus gaining any of the inherent benefits. Finally, we can look at extensibility, which I have already argued is one of the main takeaways from this design artefact. While there certainly are incredibly practical reasons for not placing the UI like this today, it is clear that limitations in hardware is the reasoning behind this design choice, and not any form of scientifically derived improvement. Once the technology inevitably moves past these limitations, it is incredibly important to revisit previous designs and see if the pathways of designing beyond may lead to a better design and thus an improved artefact.
If there is to be any critique towards the singular method chosen here, we could also argue that speculative design could have significant contributions in ensuring that we do not design for technical or sociological limitations of today, in the products of tomorrow. Considering how novel and unexplored XR and the following lack of established design methologies specific to it, the imaginative approach could hold an enormous potential for enabling more natural and intutive interations within the artefacts. (Dunne and Raby, 2013) Maybe simulating physical buttons turns out to be passé for interactions through brain-machine interfaces, and we instead seem to prefer with making a flower bloom or wilt away when we choose to delete a file within an XR environment?
7.3 Limitations of the study
This study has deliberately chosen to focus solely on XR that utilizes some form of method to display content in a volumetric 3D-space, typically referring to stereoscopic AR/VR headsets. While AR is rapidly growing within the mobile space, the lack of interaction methods and “true” 3D-space within these devices lead to only a simulated spatial/volumetric experience, since humans typically sense depth through the stereoscopy our pair of eyes provide us with.
19
the farthest one will lose almost all of its tracking. This has implications for when attempting to design an interaction model that utilizes the users’ hands as tactile feedback for actions, meaning that the limitations shape the interaction model and design implementations. Once technical flaws such as these are eventually overcome, it could be incredibly important that we remember to revisit these design stages, for perhaps a greater implementation is possible when the limitations of the past are no longer there shape it as strongly.
While utilizing the described freeform method of usability testing was the only reasonable methodology given the constraints of the current circumstances, it becomes clear that the user testing of XR-artefacts could be argued being in dire need of evaluation methodology specific to the field. Gauging user responses in this environment through observation alone limits several otherwise important factors in human communication, since their faces are mostly obscured by some form of headset device that both distract them from the interviewer as well as hide their natural facial expressions that might hint at what they’re feeling but are not fully able to express during the active testing. Overall, the freeform and natural approach might be fitting for XR-artefacts to capture as much intuitive data as possible, but more rigorous and XR-specific methodology for the testing is still in high demand if the research is to generate reliable knowledge upon which both academical and practicing HCI designers can successfully utilize and build upon.
20
8. General conclusions
Current best practices and advice regarding XR design and interaction are in general solid advice, albeit supported to varying degrees by both academic research and real-life experiences.
However, a majority of these are still heavily shaped around (or at the least influenced by) the current limitations inherent in the technologies that facilitate XR content and interactions.
XR-design can be argued to lack established methodologies that concern the specific ways that XR differs from other digital content and artefacts. Further testing of conventional or novel methodology is strongly encouraged, to see if they can be adapted or accepted as ways to reliably generate knowledge across academical and practical settings alike.
As these technologies so rapidly progress, it could be suggested that these accompanying design guidelines and methodologies should be continuously revised and used as basis for further experimentation, to enable better experiences and perhaps even a better longevity/resilience for the resulting XR products.
A rudimentary exploration of utilizing users on hands as physical feedback showed some indications as a potential avenue for more responsive and accurate interactions – however this still requires much further and more rigorous testing before any concrete conclusions should be drawn.
21
References
Bergström, I., Kilteni, K. and Slater, M. (2016) ‘First-Person Perspective Virtual Body Posture Influences Stress: A Virtual Reality Body Ownership Study’, PLOS ONE. Edited by M. Longo. Public Library of Science, 11(2), p. e0148060. doi:
10.1371/journal.pone.0148060.
Brown, P. W. (1982) ‘Less than ten—Surgeons with amputated fingers’, Journal of Hand
Surgery. Elsevier, 7(1), pp. 31–37. doi: 10.1016/S0363-5023(82)80010-5.
Cerf, V. G. (2015) ‘Prospects for the internet of things’, XRDS: Crossroads, The ACM
Magazine for Students. Association for Computing Machinery (ACM), 22(2), pp.
28–31. doi: 10.1145/2845145.
Choi, Y. S. et al. (2006) ‘Are “universal design resources” designed for designers?’, in
Eighth International ACM SIGACCESS Conference on Computers and
Accessibility, ASSETS 2006. New York, New York, USA: ACM Press, pp. 87–94.
doi: 10.1145/1168987.1169003.
Chuah, S. H.-W. (2019) ‘Why and Who Will Adopt Extended Reality Technology? Literature Review, Synthesis, and Future Research Agenda’, SSRN Electronic
Journal. Elsevier BV. doi: 10.2139/ssrn.3300469.
Cooper, B. (2012) Challenging the qualitative-quantitative divide explorations in
case-focused causal analysis, Continuum. London, New York : Continuum. Coronavirus: Dutch work on XR technology shows promise in pandemic scenario
(2020) ComputerWeekly.com. Available at:
https://www.computerweekly.com/news/252483466/Coronavirus-Dutch-work-on-XR-technology-shows-promise-in-pandemic-scenario (Accessed: 26 May 2020).
Davis, N. (2020) What experts expect from enterprise augmented reality and virtual
reality in 2020 - Unity Technologies Blog, Unity Blog. Available at:
https://blogs.unity3d.com/2020/02/12/what-experts-expect-from-enterprise-augmented-reality-and-virtual-reality-in-2020/ (Accessed: 12 March 2020). Dunne, A. and Raby, F. (2013) Speculative Everything Design, Fiction, and Social
Dreaming, The MIT Press.
Freeman, E. et al. (2017) ‘Multimodal feedback in HCI: haptics, non-speech audio, and their applications’, in The Handbook of Multimodal-Multisensor Interfaces:
Foundations, User Modeling, and Common Modality Combinations - Volume 1.
ACM, pp. 277–317. doi: 10.1145/3015783.3015792.
Gabriel, R. P. (2008) ‘Designed as designer’, in Proceedings of the Conference on
Object-Oriented Programming Systems, Languages, and Applications, OOPSLA. New
York, New York, USA: ACM Press, pp. 617–632. doi: 10.1145/1449764.1449813. Gaver, W. (2012) ‘What should we expect from research through design?’, in Conference
on Human Factors in Computing Systems - Proceedings. New York, New York,
USA: ACM Press, pp. 937–946. doi: 10.1145/2207676.2208538.
Ishizawa, F., Sakamoto, M. and Nakajima, T. (2018) ‘Extracting intermediate-level design knowledge for speculating digital–physical hybrid alternate reality experiences’,
Multimedia Tools and Applications. Springer New York LLC, 77(16), pp. 21329–
22
Jantz, J., Molnar, A. and Alcaide, R. (2017) ‘A brain-computer interface for extended reality interfaces’, in ACM SIGGRAPH 2017 VR Village, SIGGRAPH 2017. New York, NY, USA: Association for Computing Machinery, Inc, pp. 1–2. doi: 10.1145/3089269.3089290.
Jerald, J. (2015) The VR Book, The VR Book. Association for Computing Machinery. doi: 10.1145/2792790.
Kim, E. and Schneider, O. (2020) ‘Defining Haptic Experience: Foundations for
Understanding, Communicating, and Evaluating HX’, in Proceedings of the 2020
CHI Conference on Human Factors in Computing Systems. New York, NY, USA:
Association for Computing Machinery (CHI ’20), pp. 1–13. doi: 10.1145/3313831.3376280.
LeapMotion (no date) Unity Modules: Interaction Engine. Available at:
https://leapmotion.github.io/UnityModules/interaction-engine.html (Accessed: 27 May 2020).
Lévesque, V. et al. (2011) ‘Enhancing physicality in touch interaction with programmable friction’, in Conference on Human Factors in Computing Systems - Proceedings. New York, New York, USA: ACM Press, pp. 2481–2490. doi:
10.1145/1978942.1979306.
Linehan, C. et al. (2014) ‘Alternate endings: Using fiction to explore design futures’, in
Conference on Human Factors in Computing Systems - Proceedings. New York,
New York, USA: Association for Computing Machinery, pp. 45–48. doi: 10.1145/2559206.2560472.
Makin, L., Barnaby, G. and Roudaut, A. (2019) ‘Tactile and kinesthetic feedbacks improve distance perception in virtual reality’, in IHM 2019 - Actes de la 31e Conference
Francophone sur l’Interaction Homme-Machine. New York, New York, USA:
Association for Computing Machinery, Inc, pp. 1–9. doi: 10.1145/3366550.3372248.
Microsoft (2020) Start designing and prototyping - Mixed Reality | Microsoft Docs. Available at: https://docs.microsoft.com/en-us/windows/mixed-reality/design (Accessed: 14 May 2020).
Parveau, M. and Adda, · Mehdi (2020) ‘Toward a user-centric classification scheme for extended reality paradigms’, Journal of Ambient Intelligence and Humanized
Computing, 11, pp. 2237–2249. doi: 10.1007/s12652-019-01352-9.
Rasmussen, M. K. (2013) ‘Magical realities in interaction design’, in TEI 2013 -
Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction. New York, New York, USA: ACM Press, pp. 125–128. doi:
10.1145/2460625.2460644.
Rossi, S. et al. (2020) ‘Do Users Behave Similarly in VR? Investigation of the User Influence on the System Design’, ACM Transactions on Multimedia Computing,
Communications, and Applications, 16(2), pp. 1–26. doi: 10.1145/3381846.
Rubin, J. (2008) Handbook of usability testing how to plan, design, and conduct
effective tests, Wiley Pub.
23
https://medium.com/@oneStaci/https-medium-com-ux-vr-18-guidelines-51ef667c2c49 (Accessed: 14 May 2020).
Unity (2019) Classic design principles to inspire VR and AR creation - Unity. Available at: https://unity3d.com/how-to/XR-classic-design-principles (Accessed: 30 April 2020).
Verner, J. M. (2008) ‘Quality software development: What do we need to improve in the software development process?’, in Proceedings - International Conference on
Software Engineering. New York, New York, USA: ACM Press, p. 1. doi:
10.1145/1370099.1370100.
Vetenskapsrådet (no date) Forskningsetiska principer inom
humanistisk-samhällsvetenskaplig forskning. Stockholm.
Ware, C. and Balakrishnan, R. (1994) ‘Reaching for Objects in VR Displays: Lag and Frame Rate’, ACM Transactions on Computer-Human Interaction (TOCHI), 1(4), pp. 331–356. doi: 10.1145/198425.198426.
Zimmerman, J., Forlizzi, J. and Evenson, S. (2007) ‘Research through design as a method for interaction design research in HCI’, in Conference on Human Factors in
