Virtual prototyping of physical space - The value of presence, place and direct communication in prototyping

(1)

Virtual prototyping of physical

space

The value of presence, place and direct

communication in prototyping

Andrée Rehnberg

Author: Andrée Rehnberg (af1266) Email: andree.rehnberg@gmail.com

Supervisor: Henrik Svarrer Larsen Email: henrik.svarrer.larsen@mau.se

Interaktionsdesign Bachelor

22.5 ECTS Spring 2018

(2)

Abstract

This thesis explores the use of virtual reality as a prototyping tool for design practitioners to utilize when designing artifacts situated in physical places. Methods used in the study are qualitative and derive from the field of interaction design. The topic is explored through literature review on the attributes of virtual reality and sense of place, presented in the theory chapter, and is connected to the empirical research throughout the design process chapter. The theories presented strongly indicate that places affect human behavior, making them a substantial part of the user experience when interacting with designed artifacts in specific locations, which is echoed though the target audience who voice the need to prototype design concepts in the context of place. The study culminates in two kinds of virtual reality prototypes, dubbed contextual prototypes, that are suggested to have different use cases depending on what the purpose for prototyping is.

Keywords: Prototyping; Virtual reality; Augmented virtuality; Immersion; Presence; Direct communication; Place;

(3)

Acknowledgements

Big thanks to inUse and all its employees who agreed to be interviewed and participate in usability tests. I am extra grateful to Emma who provided guidance and put me in contact with the right people.

I would like to express my gratitude to Henrik for supervising me through the design process and thesis writing.

And finally, Ida, I would not have been able to write this thesis without you there to support me. Thank you for helping me get through this and keeping my spirit up even though you have had your own thesis to deal with.

(4)

1 Introduction

With the technological developments of the last decades, interaction design and architecture have begun to interweave in the sense that computing technology has gone from being situated in designated rooms to something that today is present in both the built environment and in our pockets (Dalton, Schnädelbach, Wiberg, & Varoudis, 2016). Digital interactions are becoming ubiquitous throughout the places we inhabit so understanding how to design these interactions whilst keeping the attributes of the place as a central part of the design process becomes imperative. Interactions with digital artifacts can affect how we perceive a place and places can affect how we perceive designed artifacts or services (section 2.2), this implies that the two correlate, but what does it mean to interaction designers in their practice?

inUse, a user experience design company, is working with a field they have chosen to call Human Spaces where they design digital products and services situated in physical places. Important to their work in the field is understanding how the context of physical places affects human behavior and in extension the artifacts they design (inUse Experience, n.d.). To further their understanding of this, inUse has become curious about how they may use virtual reality (VR) for communicating the context of place. The company has identified that whilst there is a great deal of research available on how to design for VR, there is a lack of research on how VR might be used to usability test when designing artifacts and services for physical space – indicative of a gap in knowledge.

In interaction design, prototyping is a large part of the practice, designers need the ability to prototype for different situations, to evaluate their ideas and to communicate the future states of products and services (Preece, Rogers, & Sharp, 2015; Saffer, 2010). Naturally, this entails that the bigger the prototyping toolbelt of the designer, the more capable she is in handling the situations that comes with the practice. VR differs from other technology in the sense that it can immerse its users in environments they are not physically located in – a concept called presence which is discussed further in section 2.1.3. When connecting the theory of presence with the need found at inUse of contextualizing their design concepts in the places they’re supposed to inhabit, the question of how VR can be used for prototyping arises.

1.1 Purpose

This thesis is explorative in nature and focuses on how interaction design practitioners can use VR as a prototyping tool when designing artifacts situated in physical places, which can currently be considered a gap in knowledge as little research has been done on the matter. The gap is

(7)

investigated using qualitative research methods common to interaction design with the aim to identify what problems design practitioners experience using other methods of prototyping for physical space that might be solved using VR as a prototyping tool. Theory is used throughout the thesis to understand the material being prototyped with (VR) and to understand how places and people relate to each other and how this relationship may affect our perception of designed artifacts. The research culminates in a set of VR prototypes which are evaluated through usability testing in their ability to solve the issues found through empirical research and theory. The contribution of the thesis is to expand knowledge in the field of interaction design on how VR may be used to improve design practice when prototyping for physical space which in extension also affects the built environments of society and the everyday lives of people.

1.2 Target group

The target group of the thesis are design practitioners who work with design concepts aimed to be situated in physical locations. This target group is reached through inUse where employees involved with Human Spaces projects have been interviewed to identify their needs and perceived problem areas when prototyping and usability testing concepts bound to a physical place. These employees have also participated in usability tests to evaluate and ground the usefulness of the VR prototypes in their design practice.

1.3 Delimitations

As prototyping in interaction design is stated to be a method that can be carried out in a wide range of ways depending on what needs to be evaluated, (Houde & Hill, 1997; Preece et al., 2015; Saffer, 2010) the VR prototypes produced in this thesis should not be considered solutions which would fit all design practitioners or situations. Instead, as a delimitation, the empirical research conducted in this thesis is limited to design practitioners of inUse and the VR prototypes produced in the design process are based on real projects previously conducted by them - this makes the results focused specifically around their practice.

Saffer (2010) states that prototyping is a form of communication of what the future state of a product or service might be which is how it is also looked at in this thesis. Emphasis is put on prototyping for evaluating design concepts, internally or with stakeholders, in the middle or later stages of the design process and a delimitation is set to not focus on the early stages of the design process when prototyping for ideation.

Another delimitation lies in the technology used to display the VR prototypes produced, so called visual displays. VR can be displayed using head-mounted displays, hand-held displays and world-fixed displays to a user (Jerald, 2015). The prototypes produced in this thesis are displayed using head-mounted display technology as it is considered the most immersive by completely

(8)

shutting out the real environment when used which increases the likelihood of the user feeling a sense of presence (section 2.1.3).

1.4 Ethical conduct

This thesis follows the ethical guidelines from the Swedish Research Council (2017). These guidelines include, but are not limited to, the right for individuals who participate in the study to be anonymous, truthfulness and transparency about the results of the research and to not share confidential information of the individuals or companies involved in the study.

1.5 Research question

In what ways can the sense of presence in a virtual environment, as induced by virtual reality technology, be used by designers to prototype design concepts bound to physical places?

2 Theory

The theories and literature in this chapter were chosen to gain insight on the qualities of VR technology, to inform design decisions and to understand the design problem. The theory is connected to the empirical work throughout the design process to analyze issues found in interviews and usability tests.

2.1 Virtual reality

VR has boomed in popularity in recent years due to the rise of commercially available products and has therefore gone from being a technology only available in research labs to something that is gaining traction and is finding its ways into our homes and offices (Jerald, 2015). But as the technology is only just gaining mass availability we are now in a situation where we are figuring out its use. What sets VR apart from much other technology is that it is immersive, can enable a sense of presence, is built out of virtual environments and that it communicates to us directly rather than indirectly and all this needs to be unpacked to understand how VR can be used for prototyping in interaction design.

(9)

2.1.1 Virtuality continuum

Figure 1 - The virtuality continuum (Milgram & Fumio, 1994)

VR is experienced through digitally rendered environments which can take many shapes. The virtual continuum (Milgram & Fumio, 1994) in Figure 1 gives an overview of how much of the real world is present in any observed environment. On the two extremes of the continuum is the real environment and the virtual environment. Real environments are described as the real world i.e. the world that we exist in, whilst virtual environments are described as environments that contain only virtual content, with no elements gathered from the real world (Jerald, 2015; Milgram & Fumio, 1994).

Any environment that mixes elements from a real environment and a virtual environment is considered a Mixed Reality (MR) environment. Amongst these we find Augmented Reality (AR) and Augmented Virtuality (AV) (Milgram & Fumio, 1994). In AR the real world is observed in real time with virtual elements rendered into the real environment. In recent years this has become a popular medium used with smartphones where the real environment is observed through the camera (Jerald, 2015). An example of AR is present in the social media platform Snapchat (Snap Inc., 2018) where virtual figures can be placed upon the real world as seen in Figure 2.

(10)

In AV, real world elements are captured and implemented into a virtual environment. Unlike AR, the real-world elements are not seen in real time by the observer as they are only captured as snapshots (Jerald, 2015; Milgram & Fumio, 1994). An example of AV is Google Street View (Google, n.d.) where 360° photos of places around the world are captured and the viewer is allowed to look, zoom and move around between different photos using interactive icons as seen in Figure 3.

Figure 3 - AV in Google Street View displayed in a web browser (Google, n.d.)

This section informs the study about the various kinds of environments of VR. This thesis focuses on the use of AV in design work due to the advantage of being able to represent a physical place together with a suggested design concept without needing to be physically present at the location as needed in AR.

2.1.2 Immersion

In the field of VR, immersion is described as the degree to which the technology and the VR application can engage the user through sensory stimuli. The level of immersion is determined through six characteristics - extensiveness, matching, surroundedness, vividness, interactability and plot (Jerald, 2015).

Jerald (2016, p. 45) defines these characteristics in the following way: “Extensiveness is the range of sensory modalities presented to the user (e.g., visuals, audio, and physical force).

Matching is the congruence between sensory modalities (e.g.,

appropriate visual presentation corresponding to head motion and a virtual representation of one’s own body).

Surroundness is the extent to which cues are panoramic (e.g.,

(11)

Vividness is the quality of energy simulated (e.g., resolution,

lighting, frame rate, audio bitrate).

Interactability is the capability for the user to make changes

to the world, the response of virtual entities to the user’s actions, and the user’s ability to influence future events.

Plot is the story—the consistent portrayal of a message or

experience, the dynamic unfolding sequence of events, and the behavior of the world and its entities.”

This definition gives an overview of how VR environments and experiences may be designed to facilitate prototyping in the field of interaction design. Whilst some characteristics such as vividness may be restricted by the technology itself (e.g. the display resolution of a head-mounted display cannot be increased by the designer), other characteristics such as extensiveness, interactability and plot can be designed to a greater degree which enables an objectively higher level of immersion. However, Jerald (2016) states that whilst these objective characteristics can in many cases be designed to increase the level of immersion in the VR experience, perceiving and interpreting the sensory stimuli presented by VR is a highly subjective experience known as presence.

The characteristics of immersion are used in the design process as a guide for making VR prototypes more immersive and for analyzing issues found during usability tests.

2.1.3 Presence

The International Society for Presence Research (2000) states that the term presence (or telepresence) is commonly referred to as a sense of “being there” in a virtual environment, a statement which they expand upon by explaining that presence is …

“… a psychological state or subjective perception in which even though part or all of an individual’s current experience is generated by and/or filtered through human-made technology, part or all of the individual’s perception fails to accurately acknowledge the role of the

technology in the experience.”

In this definition a distinction is made between where the individual is subjectively experiencing herself being present (i.e. the virtual environment) and where she is objectively present (i.e. the physical environment). When the individual is experiencing presence in the virtual environment she no longer acknowledges the VR technology presenting the virtual environment. The International Society for Presence Research (2000) states that the user of VR technology is almost always able to identify the technology used to deliver the VR experience due to e.g. the weight of the head-mounted display

(12)

on her head or a lack of vividness in terms of immersion, but that the knowledge about the technology is to some degree overlooked. When experiencing a virtual environment the technology is not perceived as being involved in the experience. The subjective experience of being present in a virtual environment leads the individual to experience temporary amnesia of the real world and feeling highly present in the virtual environment makes the individual consider the environment a place visited rather than a place observed (Jerald, 2015).

Users can lose their sense of being present in the virtual environment which is called a break-in-presence. These can occur for example if the head-mounted display suddenly stops tracking the head-motions of the user which breaks the immersion of matching between what the user sees and what she expects to see. A break-in-presence is stated to destroy a VR experience (Jerald, 2015).

Presence is a function of the immersion and the user’s perception – it is not guaranteed even though you have designed with the characteristics of immersion in mind but it also cannot be achieved if you have not (Jerald, 2015). Presence is largely what sets VR, and specifically head-mounted displays, apart from other mediums such as mobile phones, large displays and sketches since although you can observe and perceive other environments using these mediums they are not able to create a sense of “being there” in the unique way that VR technology enables the user to. What is gathered in this section is that VR can create a sense of presence in ways which other materials cannot. Presence allows for forms of communication that does not require active interpretation from the user which are discussed in the following section.

2.1.4 Communication

Although much can be said to describe what VR is, Jerald (2016) argues that above all, VR is communication. This statement can be put into comparison to how the method of prototyping is described by Saffer (2010) – that prototyping in the field of interaction design is communication about the future state of a design. These descriptions suggest that VR can be used in the field of interaction design as a prototyping tool, but to do so it proves useful to define how communication in VR differs from communication in other mediums.

VR is stated to communicate to the user in what is known as direct communication, which in turn consists of structural communication and visceral communication (Jerald, 2015).

Direct communication is communication that is experienced without the need for active interpretation. When we as humans go about our daily lives we do not need to actively interpret our environment, instead we experience it directly without the need for an intermediary (e.g. text, pictures) to describe

(13)

what we see, hear and otherwise sense. The same can be said for when a user is present in a virtual environment. In these cases, the intermediary visual display used to deliver the VR experience is not perceived by the user, as described in the previous section, and instead the virtual environment is experienced directly without the need for interpretation by the user. Structural communication is a big part of how we as humans situate ourselves in the environments we inhabit in terms of physics (Jerald, 2015) – our bodies always relate to other objects in the world and how different objects relate to each other, e.g. directly seeing an artifact that is 3x4x2 meters in size is not the same as reading about an artifact that has the same dimensions. Visceral communication is described as our automatic emotional responses, it is our immediate response before we have had the ability to process and interpret an experience as opposed to the later retelling of our experiences (at which it is instead transformed to indirect communication). Our automatic emotional responses are said to be triggered by both places and personal experiences (Jerald, 2015; Turner & Turner, 2006).

Mediums and prototyping tools such as phones and paper sketches only allow for indirect communication since they cannot create a sense of presence. When reading a book or watching a movie you can interpret the environment or experience described but your presence is still situated in the sofa, turning pages or watching TV, which stands in contrast to the direct communication afforded by VR technology.

Prototypes in interaction design are often described in ways which indicate that they communicate indirectly, especially when the prototype is of low fidelity (Preece et al., 2015; Saffer, 2010). VR opens the possibility for interaction designers to build prototypes with direct communication and to do so when the place the design concept is situated in is of relevance, as they often are in the Human Spaces projects of inUse. VR prototypes could therefore allow users to experience designs situated in places directly, rather than indirectly which has the benefit of increasing the understanding of the user:

“Whatever the industry, VR is largely about providing understanding—whether that is understanding an entertaining story, learning an abstract concept, or practicing a real skill. Actively using more of the human

sensory capability and motor skills has been known to increase understanding/learning for some time.” - Jerald

(2016, p. 12)

The theory of communication is used in this thesis to understand problems voiced by the design practitioners about their prototyping experiences and to analyze the results of usability tests of VR prototypes.

(14)

2.2 Spaces, places and virtuality

Spaces and places are considered separate things in the fields of architecture and urban planning and literature on the theme of place conclude that for a space (a physical location) to be considered a place it needs to, in some sense, become meaningful to the individual - this is known as a sense of place (Turner & Turner, 2006). In short, this can be described as place = space + meaning but meaning is a broad term that needs to be defined when put into the context of space and place.

Harrison & Dourish (1996) differentiate space and place like this: “While spaces have up and down, left and right, places

have yesterday and tomorrow, good and bad.”

This extract defines space as the physical characteristics of a location (e.g. tall buildings, heavy traffic) whilst places are defined as dynamic entities which can change with time and that we have personal feelings about. If connected to the theory on direct communication, space could then be considered as the structural communication (how we relate to the objects around us) and place could be considered the visceral communication (how a space might feel to us).

Different places afford different activities, our social interactions differs from place to place and we personally fill these places with memories and associations (Turner & Turner, 2006). Being in a place is itself an experience which has connotations for how we might perceive a designed artifact situated in that place. Furthermore, Buxton (2007) states that no design exists in a vacuum – that things such as location, time and mental state play a role in how we use artifacts. This indicates a need for interaction designers to not only be able to prototype the artifact of design but to also be able to situate it in its respectful place as it provides context for its use. The place theory suggests that the user experience is a function of both the place and the interaction with the designed artifact itself as illustrated in Figure 4.

(15)

Figure 4 – Diagram illustrating that the user experience is a function of the sense of place and interaction with the designed artifact.

In an ideal situation, designers should have access to the place they are designing artifacts for when prototyping, keeping aspects of both place and interaction central to more clearly be able to evaluate the user experience. This is supported by Saffer (2010) who states that the place of interaction should ideally be replicated when designing services. Although this is ideal, it is not always possible given situations that can hinder access or the ability to properly conduct design work, such as places where there are considerable amounts of foot traffic. In these cases, the places instead need to be replicated.

In augmented virtuality, elements from the real world are captured and placed in a VR environment, often together with digitally rendered objects. The space itself can be captured using 360

°

cameras to shoot equirectangular images or videos as shown in Figure 5 and capturing the soundscape of a space has been found to greatly increase the sense of place in the virtual environment (Turner, Mcgregor, Turner, & Carroll, 2003) allowing designers to build virtual environment with greater extensiveness in terms of immersion. Adding a digital representation of the artifact of design to the augmented virtuality environment could allow designers to prototype the user experience in a richer way by involving the context of place.

(16)

Figure 5 - Equirectangular image (360° panorama)

The space and place theory echoes throughout this thesis both in the findings from communicating with the target group and when building the prototypes.

3 Methods

This chapter outlines the design process model used throughout this study and details what methods were used and how they were interpreted by the author.

3.1 Applied research

Since the general aim of this thesis is to investigate how design practitioners can better their practice using technologies on the virtual continuum (Milgram & Fumio, 1994), using an applied research methodology is needed. Muratovski (2015) states that applied research is a type of research where the purpose is to reflect upon and evaluate your own work or the work of your organization and to suggest and test ideas for improving the practice. inUse is the company to which this research has been applied. By using qualitative research methods, the intention has been to learn about what their Human Spaces department strives to achieve, what methods they use and what they find problematic in their practice. The research provides an empirical knowledge foundation on which VR prototypes could be produced that are grounded in the practice of the target group. Throughout the design process, the findings from the applied research is connected to the theory of chapter 2 to further inform design decisions, analyze findings and evaluate prototypes. The design process of this applied research study follows a model which is detailed in the following section.

(17)

3.2 Design process model

Figure 6 - Applied research model (Muratovski, 2015)

The applied research process model in Figure 6 is used in this project to drive the research forwards in a structured manner. The model contains four main stages which are planning, acting, monitoring and evaluating. The process is iterative which means that it is repeated after going through the stages to incorporate the knowledge obtained in the previous round of iteration. Saffer

(18)

(2010) points out that designers rarely get their designs right from the start and that the iterative process is necessary for making sure that design manages to solve the problem it sets out to solve for the intended target audience.

Since no formal explanation of these stages are given by Muratovski (2015), the following interpretation has been made of them:

Plan – the initial plan does not need to include detail or be based on a clearly defined research question or hypothesis. Instead it can be sparked by a general idea on how one’s own design practice might be improved (Muratovski, 2015). In the case of this study the initial plan was produced by inUse where they had an idea about VR technology being useful in their work with Human Spaces. In subsequent rounds of iteration, the plan was revised to incorporate the new knowledge.

Act – after the plan has been devised you can start acting on it, this stage was where insights on the design problem was gathered using research methods such as literature research and interviews. The act stage was also interpreted as the part of the process where the designer can prototype physical or digital artefacts which functions as suggestions for how to improve one’s own work. Monitor – the monitor stage is where you test the and evaluate your suggestions for improving practice, which in the case of this study were VR prototypes. In this study the method used in the monitor stage was usability testing which was conducted with the target audience which enabled the research to be grounded in their practice. During the monitor stage, issues and suggestions for improvements with the VR prototypes were identified which could later be used in subsequent rounds of iteration.

Evaluate – evaluating is the last step in a round of iteration where the knowledge and findings gathered from previous steps are evaluated to establish if the suggestions for how to improve practice had its intended effect. In the evaluation stage the findings where be connected to theory to better understand the results of the monitor stage.

3.3 Interviews

The brief from inUse for conducting this study was open-ended, it had no set end goal in mind but was instead exploratory in nature meaning that they wanted to investigate how they might use VR technology as a tool in their own work when designing for physical spaces. Because the company had not themselves identified a problem they needed to have solved this was the first step in my design process and was done mainly through interviews.

Preece, Rogers & Sharp (2015) state that it is imperative to consult with representative members of the target group in the interaction design process to understand their experiences and expectations. In this thesis the target group are interaction designers and user experience designers so

(19)

interviewing select employees at inUse allowed me to better understand how they today work in their Human Spaces projects and to get an understanding of what they feel can be difficult in this field. It also allowed gauging what knowledge they currently have about VR technology and what possibilities they could see in using it in their work.

Interviews in this study have been semi-structured which Preece et al. (2015) describes as a mix between unstructured interviews, which in many ways mimic the dynamics of regular conversations that are open and explorative in nature, and structured interviews which instead follows a strict set of questions with the intention of getting a clear set of answers. Semi-structured interviews allowed the use of both open ended questions that the interviewee could expand upon and for spontaneous follow up questions to be asked when needed (Muratovski, 2015).

Before conducting the semi-structured interviews, moderator scripts (Saffer, 2010) were prepared for each interviewee containing questions relevant to their position at the company. However, many questions were overlapping between interviewees to identify common themes amongst them. Effort was made to ensure neutral questions in the moderator script so interviewees were not nudged to give a specific answer (Preece et al., 2015).

To account for ethical conduct, each interview started out by explaining the aims of the interview along with the option to participate in the interview anonymously (Preece et al., 2015). The interviewees were informed that the interview should preferably be audio recorded and asked for their permission to do so (Muratovski, 2015). The audio recordings were complemented with note-taking to capture main findings in a structured manner. The audio recordings were subsequently transcribed (Muratovski, 2015), which proved useful when time had passed since the interview and the data needed to be re-visited.

Interviews were conducted through online voice call services when the interviewee was located at a remote physical location and face-to-face when the interviewee was able to meet in person.

The interviews ties into the act stage of the design process model in Figure 6.

3.3.1 Interviewee selection

Three individuals at inUse were interviewed, Pontus, Anders and David, neither of whom felt the need to be anonymous and all agreed to be recorded. Interviewing employees of the company served to keep the study grounded in their practice.

Pontus has the title director of service design and is one of the employees most involved in the company’s work with Human Spaces. He was sought out as an interviewee to gain insight on the field of Human Spaces.

(20)

Anders is a user experience designer with an informal reputation at the company of being a prototyping expert which is why he was sought out as an interviewee to hear his take on the topic of VR as a tool for prototyping. David is the director of design and innovation at the company and was sought out as an interviewee as his job involves further developing the methods used in the company’s design practice as well as for his experiences in Human Spaces projects.

3.4 Literature review

Literature review is the process of studying what is already known in the field of interest using data that other people have collected, recorded and analyzed (Muratovski, 2015). By conducting literature review on the field of interest you not only become well-informed about what is already known in the field but you can also identify what is not yet known and needs further research which Muratovski (2015, p. 32) claims is “commonly known as identifying the ‘gap’ in the knowledge”. Identifying the gap in knowledge through literature, as done in chapter 2, has been critical as it provides relevance to the study.

Muratovski (2015) lists six types of literature review commonly used in research of which thematic and methodological has been applied in this thesis. A thematic literature review is one where common themes and topics of the field of interest is researched for the researcher to become well-informed and able to present the selected themes in written format. In this thesis thematic reviews have been conducted on the themes of augmented virtuality, presence, immersion, communication and place. A methodological literature review is where the research revolves around examining various methods and how they might be used to study issues relating to the field of interest. In this thesis, methodological literature reviews have been done to both structure the thesis-writing process itself and to become more well-informed about how methods such as prototyping.

The knowledge gathered from the literature reviews has, together with empirical research, been used to ground design decisions and to analyze findings during interviews and usability tests.

3.4.1 Studying related works

Parts of the literature review has revolved around not just gaining insights into previously mentioned themes and methodologies but also exploring what work has previously been done, i.e. in what ways has VR technology previously been used as a tool for prototyping. Previous work and literature review are often used as interchangeable terms in academia but in this thesis the need to distinguish the two arose as literature review was interpreted to define what something is whilst related work was interpreted as being examples of how that something has been used.

(21)

3.5 Storyboards

Storyboards are described as a means to visualize a scenario where a product or service is in use (Preece et al., 2015; Saffer, 2010). In this thesis storyboarding was used to map out key moments in a scenario which would later be recreated in a VR experience but also to figure out the positioning of objects in the virtual environment before building it. Two types of storyboards were drawn on paper, the first type was traditional — e.g. crude sequential sketches on paper and the other type is a top-down type of storyboard useful for designing linear VR experiences called 360

°

block diagrams (Fictum, 2016).

3.5.1 360° block diagrams

360° block diagrams place the user experiencing the virtual environment in the center of a circle with relevant objects positioned around the user to map out how the user and objects should be positioned in relation to each other, an example of this can be seen in Figure 7.

Figure 7 - Example of a 360°block diagram (Fictum, 2016).

Sequential block diagrams were drawn in the early process of designing VR experiences to map out how the user should move in the virtual environment, where object should be located and what interactions should be presented at which stages of the experience.

3.6 Prototyping

Saffer (2010) states that in interaction design, prototypes are tools for the communication of design concepts and that they are imperative in the design process since many people will have difficulties understanding a design until given the ability to see and test a prototype of it. Preece et al. (2015) further

(22)

expands upon this by claiming that what constitutes as a prototype can be a wide range of things, from paper sketches of an interface to complex digital 3D renderings of physical products. No matter the fidelity they can all be of value to the designer and the purpose of the prototype will have a role in selecting what kind of prototype to build (Preece et al., 2015). This is supported by Houde & Hill (1997) who state that prototypes rarely prototype the full functionality of a product but rather select parts of it depending on what needs to be evaluated.

Prototyping has been a central theme throughout this thesis since the brief given by inUse revolved around exploring how VR technology might be used for prototyping design concepts and interactions in physical places. To explore this, VR prototypes have been built that explores how attributes like immersion, presence and direct communication as described in chapter 2 can meet the needs of the design practitioners at inUse and prove useful when designing concepts situated in physical places. The VR prototypes built in this thesis are closely based on a previous project inUse has conducted to further ground the study in their practice of design practitioners.

Low-fidelity and high-fidelity are terms that are commonly associated with prototyping in the field of interaction design (Preece et al., 2015; Saffer, 2010). The terms are used to describe the degree to which the prototype is meant look and behave like a final product. Saffer (2010) explains that prototype fidelity is a continuum which Pernice (2016) expands upon by presenting three areas of a prototype that can either be low or high in fidelity, these are interactivity, visuals and content. The VR prototypes produced in this thesis have been high-fidelity in terms of interactivity and visuals but low-fidelity in content since it is the attributes of the VR medium that is evaluated in subsequent usability tests rather than the concept presented in the prototype.

Saffer (2010) points out that there are differences in prototyping a service and prototyping a product as the people and places involved are important in services. Therefore, a service prototype is not able to come alive unless users are able to walk through the process of using it and ideally so in the actual environment the service is meant to exist or one that closely mimics it. Saffer (2010) therefore suggests creating scenarios and role playing them as a method of prototyping scenarios. The concepts of mimicking environments and creating scenarios (Preece et al., 2015; Saffer, 2010) that users can walk through are ones that has been adopted in the building of VR prototypes in this thesis.

The connection explored in this study between prototyping as a form of communication and the direct communication of VR (section 2.1.4) provides relevance to the field of interaction design as it relates to our practice. The prototyping ties into the act stage of the design process model in Figure 6.

(23)

3.7 Usability testing

Usability testing is a method of evaluating the usability of a product or prototype by inviting users to test it, observing how they interact with it and asking them questions relating to the design (Muratovski, 2015; Preece et al., 2015; Saffer, 2010). In this study the prototyping phase was followed up with usability testing to validate if the VR prototypes work effectively for communicating design concepts in places.

The usability tests were carried out using a test plan which Saffer (2010) describes as a “route” through the product that tests functionality and feedback in the prototype. The test plan contained questions that could prompt users through the prototype but were formulated in a neutral way as any opinions about the design should come naturally from the user and not be nudged by the test facilitator (Saffer, 2010).

In usability testing it is common practice that users are asked to think-aloud when they interact with the design, this is done to gain insight into why the user interacts with the design in the way she does, in the moment of interaction (Preece et al., 2015). However, the choice was made to avoid the think-aloud technique for any usability test of VR experiences out of suspicion that being asked to talk about what you are experiencing to someone outside of the virtual environment would cause a break-in-presence as described in section 2.1.2. Participants were instead allowed to explore the virtual environments without interruptions and were interviewed about their experience after testing the design to allow them to remain present in the virtual environment for the duration of the test. The subsequent interview was semi-structured and followed a moderator script containing neutral questions (formulated to not nudge the participant) relating to the VR prototype that had been tested.

To account for ethical conduct, participants were informed of their right to be anonymous and asked for permission to capture audio recordings of the interview and video recording of what happened on screen in the head-mounted displays during testing of the prototypes as this is normally not visible to anyone outside of the VR headset. The video and audio recordings were transcribed and analyzed to be revisited later. Moreover, Muratovski (2015) lists physical discomfort as a potential risk that needs to be addressed when conducting ethical research and VR is known to cause nausea in some individuals (Fictum, 2016; Jerald, 2015), especially if certain immersion characteristics (section 2.1.2) are substandard in the experience. Individuals with epilepsy are advised to not use VR at all due to the risk of seizures (Fictum, 2016; Jerald, 2015). Participants in the usability tests were therefore informed of the risk of nausea beforehand and asked to stop the test if it was experienced, they were also asked to not participate in the test if they had previously experienced epileptic seizures.

(24)

The usability tests functioned as the monitor stage of the design process model (Figure 6) and was useful for understanding where the users struggled with the prototypes as well how future iterations might be improved to better suit the practice of the design practitioners which ties in to the evaluation stage of the applied research model in Figure 6.

3.7.1 Test participant selection

Like in the interviews, the participants of the usability tests were employees of inUse. Because the VR prototypes built were based on a previous project of the company, two employees were selected as participants, David and Jenny, both of whom had been involved in that project which enabled keeping the research grounded. Neither of the participants stated that they wanted to be anonymous and both agreed to be recorded.

4 Design process

In this chapter, the process and findings of the empirical work conduced in the study is detailed and throughout connected to the theory of chapter 2.

4.1 Related works

Related works has been researched to explore what has already been done in the field of VR in relation to prototyping. What was found was that although there is a great amount of research available on presence, communication and VR, there is a gap in knowledge on how VR, and specifically augmented virtuality, can be used by user experience designers as a tool for prototyping. Below is a selection of previous academic work that has made use of the theories of chapter 2, but not in the context of interaction design and prototyping as this could not be found.

In summary, studying related works informed this thesis by suggesting that the knowledge contributed is new and that it holds relevance because of the understanding provided by the direct communication of VR technology.

4.1.1 BENOGO – Being there, without going

The BENOGO project was an EU-funded research project coordinated by the university of Aalborg, Denmark, that aimed to develop innovative technologies to enable individuals to feel present in real places without the need to physically travel there. The project made use of image-based rendering technologies that enabled them to generate detailed 3D-environments based on real places using a series of photos (European Commission, 2005). The environments produced in the project are

(25)

considered augmented virtuality environments as they build on material gathered form the real world.

Turner & Turner (2006) evaluated the relationship between the literature available on sense of place (section 2.2) and the sense of presence (section 2.1.3) in the virtual environments produced in the BENOGO project. The research found that test participants in the study often described the environments through the physical characteristics of the space and that they often voiced that they wanted the ability to move around and carry out activities relevant to the place in question. Some environments were reported to trigger memories and associations as they were similar to ones they had personal experiences with. In one test, the participants were given a scenario in advance in which they were to act as a security guard sitting at her desk observing the environment. This scenario was found to have a striking effect and to be enjoyable by the participants as the role of the participant involved actively looking and listening for suspicious behavior. This finding is echoed by Jerald (2016) who defines story as an immersion characteristic (section 2.1.2).

In summary, VR experiences in the BENOGO project were able to communicate structurally as the well as viscerally to the participants and setting a scenario was an effective means to increase the immersion of the experience. Moreover, in all tests, the participants wanted the ability to move around and take part in activities afforded by environments suggesting this may be a key element in generating a sense of place in users.

4.1.2 Mixed reality for electrical construction

Chalhoub & Ayer (2017) have studied the use of augmented reality in the field electrical construction, a field which predominantly makes use of written documentation to communicate instructions on how to construct electrical conduits for specific locations. The goal of the study was to explore how mixed reality can be used for communication in the construction industry. It is stated that as much as 30% of the value of a given project goes to waste if the communication is ineffective. Tests were conducted with the aim of comparing how quickly test participants were able to complete two similar conduits when using paper documentation and when using augmented reality which communicated the conduit as a digital 3D-model rendered into the physical environment. The authors also documented the number of mistakes made by the participants when the different methods to communicate the conduits were used.

The results of the test statistically showed that the augmented reality visualization was more effective in communicating how to construct the conduit than the paper documentation. The visualization of augmented reality was observed to be easier to comprehend by the participants resulting in quicker construction with significantly fewer mistakes made.

(26)

In this case, the direct communication (section 2.1.4) afforded by the augmented reality experience was proven to be more effective than the indirect communication afforded by the paper plans. The results point towards the effectiveness of using VR technologies that communicate in a direct manner in design practice as well when a design is situated in specific place due to the ease of comprehension it affords. This is supported by Jerald (2016) who states that VR is largely about providing understanding.

4.2 Interviews with inUse

As this study aims to explore how the design practitioners of inUse may use VR technology in their practice it has been vital to do empirical research at the company to gain insights to their practices. Interviews (section 3.3) have been conducted with a focus on learning more about how employees at inUse conduct their practice with an emphasis on projects that lies within their Human Spaces field as it is described as one where the themes of place and human behavior are tightly interwoven (inUse Experience, n.d.). This description relates back to the theory on place described in section 2.2. Moreover, interviewees were also asked about their personal experiences with VR and what ideas and opportunities they saw regarding its use in their design practice.

In depth interviews were conducted with three employees at the company – Pontus, Anders and David, who were introduced in section 3.3.1.

4.2.1 Summary of interviews

When asked to explain the field inUse calls Human Spaces, Pontus explained that it is a field that lies right on the intersection between service design and architecture but underlines that the field cannot be considered new. In his view, many service- and user experience designers already unknowingly conduct work that could be considered part of Human Spaces when they create design concepts that are not purely digital but are situated in physical space. Pontus states that places can shape human behavior and that the focus of their work with Human Spaces puts an emphasis on not just the design concepts but also the places they are situated in due to digital interactions become more ubiquitous in our everyday life and environments.

The interviewees were asked about their experiences with VR at which it became clear that they had very limited personal experience with it and next to no experience working with it in a professional capacity, neither at inUse or in previous workplaces. Pontus pointed out that he felt like VR was still a technology in it is infancy in the sense that people are still figuring out what to use it for. He had observed that the public perception of the technology in recent years seemed to lean towards VR being an entertainment platform (primarily gaming) but that we had yet to realize its potential in relation to practice - in Pontus’s own words (authors translation): “We are still looking for that killer app or use case where it adds value”. This perception seemed

(27)

to be indirectly shared by both David and Anders as when they spoke about their experiences with VR they recounted primarily gaming experiences. Despite the entertainment focused perception of the technology, the designers interviewed were still able to envision cases where the technology could be of use in their practice, specifically to experience a design concept in the first person.

Pontus states that when prototyping concepts for the built environment he sees the options of either doing it quickly using low-fidelity with paper sketches and scale models or to outsource it and have high-fidelity CAD-models developed - both options he states comes with their own pitfalls:

“The drawback with today’s tools for prototyping for the built environment is that you can either do it relatively

quickly with a scale model, but that makes it very difficult for us as fully-grown people to interpret how it

would feel to be that 2 cm tall human model walking around inside the scaled down environment. The other

option is that if you put that much time and effort into building a high-fidelity, highly detailed CAD-model, then you’re already so far along in the process that it becomes

difficult to make changes to the concept.” - Pontus (authors translation)

360° photography was suggested by Pontus as a way of enabling design practitioners to quickly re-create a place thus potentially bypassing issues relating to the cost/benefit of high-fidelity prototyping and difficulties in interpretation with low-fidelity prototypes. As an employee specializing in prototyping, Anders also found that the general strengths of prototyping were that it enabled the designer to test ideas before any part of the concept was put into production as well as enabling communication of design concepts better than any presentation or design specification. He also found that prototypes provided a better basis for discussion. Anders was generally positive to the idea of using 360° photography but could also see that it had certain drawbacks:

“If you’re working with a public environment and want to add for example a large display then that [360° photo]

would be a really nice way of visualizing how it would look and be positioned, it feels like a good way to do it …

the drawback with it is that in some cases the environment may not exist yet then you’d instead need a

3D-model of the environment.” - Anders (authors translation)

David retold the process he went through when designing an interactive installation which was to be situated in a public place with considerable

(28)

amounts of foot traffic. During this process he found it difficult to remember physical attributes of the place – what one was able to see from different angles in a place, how high the ceiling was or what the general scale of the place was. David found that it took many visits to the place he was designing for before he was able to get a decent mental image of the space. In much the same way as Anders, David argued that prototyping is a helpful tool in supporting our imagination of the future state of whatever it may be that they are designing, but just like Pontus argued, designing for physical space comes with its own sets of challenges which makes interpretation more difficult both for the designer and stakeholders.

The interactive installation David spoke of consisted of large digital displays situated on the ground and ceiling of a large room and David used several prototyping tools to help him interpret his concept, including bodystorming and making video scenarios. He also had an ambition to develop a 3D visualization of the place and concept and present it to stakeholders, but this was never done due to a lack of time and knowledge of how to build them. David envisions that VR prototypes could be used to prototype several different things including how the design may look and feel in the environment as it changes between night and day or during different seasons of the year. He also stated that it could be used to visualize a scenario of the design being used in a comparable way to video scenarios and said the following regarding a potential VR scenario in comparison to a video scenario:

“The video scenario we produced for the project still requires a lot from the person watching it, it is difficult to

understand how the design feels in the space. A virtual reality scenario could maybe make this a bit clearer. It could help the observer understand how one moves in the environment and how big the object is. But I do think that the concept needs to be more concrete in virtual reality,

video probably allows for more openness.” - David (authors translation)

In summary, all interviewees were generally positive to the use of VR as a prototyping tool as they could envision it immersing the observer in the environment in ways other prototyping tools cannot. It was identified that for the VR prototypes to be valuable they needed to be relatively quick to make as the value the prototype can add to the design process diminishes exponentially with the time it takes to produce. A possible solution involving 360° photography was suggested that could aid designers working with Human Spaces to quickly prototype whilst taking both the place and the design concept into account. David also provided insights for what VR prototypes may prototype in the context of the interactive installation project he had been involved with. The VR prototypes produced in the later stages of

(29)

the design process were closely based on the interactive installation David had described.

4.2.2 Connection to theory

The findings from the interviews ties in well with the theory of VR and sense of place as outlined in sections 2.1 and 2.2 as well as what has been stated about prototyping in the field of interaction design in section 3.6.

When the interviewees spoke about Human Spaces it was clear that they saw place as something that affects human behavior, that the design of places shape how we interact amongst each other as well as how we may interact with designed objects in much the same way outlined in the place theory (section 2.2). Moreover, they showed an awareness of places being dynamic rather than static and indicated a need for being able to situate a design in different states of a place.

Using augmented virtuality environments, rather than fully virtual environments (section 2.1.1) was suggested when the use of 360° photography was brought up. The interviewees believed that this would kill two birds with one stone – that it would assist in creating a sense of presence (section 2.1.3) when prototyping and that it could cut down on the time and effort involved in producing the prototype, thus increasing the value gained from the prototype. The caveats identified in using 360° photography was that the place being designed for would already need to exist which is not always the case and that major structural or architectural changes in the environment could prove difficult to apply to a photography - design concepts would instead need to be applied to the space “as is”.

Issues with the indirect communication (section 2.1.4) of many prototyping tools were also brought up in the interviews with interviewees stating that they had found paper sketches and scale models problematic when trying to interpret how a design would look and feel when situated in a physical location. VR was considered a potential solution to this due to its direct communication which would allow the observer to experience the place and design directly rather than having to imagine the two together, providing greater understanding. This finding is in line with the data from Chalhoub & Ayer (2017) in section 4.1.2 where the direct communication of VR technology was found to make interpretation easier when doing electrical construction.

4.3 Conceptualization & testing

The interview with David inspired the making of a VR prototype closely based on interactive installation concept he had been involved with. The intent of mimicking this concept was to make it easier for employees participating in the later usability tests to relate to the concept and to ground it in their practice. The prototype was however situated in a different geographical

(30)

location than the project David was involved with and has key elements changed to better suit the affordances of the place.

The choice was made to design the prototype in the style of a storyboard with the user walking through a sequential scenario of using the design concept situated in its place. Making the prototype function like a storyboard was a choice made due to the interest shown during the interviews in making VR scenarios and because of VR being known to be a powerful tool for storytelling (Fictum, 2016; Jerald, 2015) with research showing that the direct communication of VR even enables storyliving – a sense of “living” a story rather than being told one (Maschio, 2017). Furthermore, research on the BENOGO-project (section 4.1.1) found that the use of scenarios greatly increased engagement in VR experiences.

The concept being storyboarded revolved around the experience of public transportation and the act of waiting for the bus. The place chosen for the design to be situated in was Värnhemstorget in Malmö which acts as one of the larger bus stations in the city. The concept involves a large display situated in the ground near the bus station which can be interacted with by walking on it. The display contains a map of the region of Skåne and as you walk across places marked on the map you get information about them and how to get there using public transportation.

4.3.1 Drawing storyboards

As a first step, the concept was drawn as a traditional storyboard (Figure 8). The drawn storyboard was never intended to be shared with anyone but rather to map out how the scenario would play out and to identify the key scenes and moments of the story which would later be used when making the VR prototype. Fictum (2016) states that sketches and storyboards function as a first step to understanding the narrative when designing VR stories.

(31)

Figure 8 - Hand drawn storyboard.

More than anything, the storyboard explores the role the design plays in the user’s life (Houde & Hill, 1997) as it follows a main character and explains what he is doing at the place, how he interacts with the design and how it affects him. These were aspects that would later be communicated in VR. In addition, a storyboard consisting of 360° block diagrams (section 3.5.1) was made with the intention of mapping out the positions of different elements on the different scenes of the augmented virtuality environment. The block diagram storyboard (Figure 9) matched the frames of the traditional storyboard but was instead useful for understanding what the user would see and be able to interact with from the first-person perspective given by VR instead of the third-person perspective drawn in the first storyboard.

(32)

Figure 9 - 360° block diagram "storyboard"

4.3.2 First virtual reality prototype

When the scenario had been planned out in terms of story and positioning in the 3D-space using storyboards, the VR prototype was ready to be produced. The prototype was built using A-Frame (Mozilla, 2018), a code framework which enables creating VR experiences using HTML and JavaScript that can run in web browsers. Links to all VR prototypes produced in this study can be found in the appendix.

Since the aim was to build an augmented virtuality environment, the first step was to capture the environment of the bus station at Värnhemstorget using

(33)

360° photos. Several photos were captured using the block diagram storyboard as a guide for knowing where to capture the photos to progress the “frames” of the scenario in VR. It was was determined that only capturing photos from the place was not extensive enough in terms of immersion (section 2.1.2). Sound plays a large role in how humans perceive their surroundings which was why the soundscape (Turner et al., 2003) of Värnhemstorget was also recorded. In the audio recording you can hear buses arriving and departing, people chatting in the distance and seagulls squawking overhead - sounds common to the place in question. The goal of the recording was to make the environment feel more like a place as opposed to a space (section 2.2).

A limitation of using 360° photography is that although it appears as a 3D environment when wrapped around the observer’s field of view in the head-mounted display, the image format is in fact only 2D as seen in Figure 5. In effect, this means that it will not allow users to physically walk around in the environment. To overcome this limitation a script was written that allowed users to teleport (relocating to a new location without physically moving) between different points in environment (different 360° photos) which is a technique commonly used in VR when needing to travel longer distances than the technology or physical environment allows (Jerald, 2015). The points the user was able to teleport between were indicated using map pins placed in the environment which teleported the user when clicked.

In addition to capturing elements of the place being designed for, visual elements representing the design concept also needed to be developed to be placed in the environment. A map of the region of Skåne was designed using the website Snazzy Maps (Atmist, 2018) and an image of it was downloaded which would represent the large display mounted in the ground. As detailed in the drawn storyboard, the concept would allow users to interact with the map display by walking on it, at which the map would display information dialogs about the location you were standing on, these information dialogs were designed in photo editing software.

As shown in the drawn storyboard of Figure 8, the user would towards the end of the scenario message a friend on his phone suggesting they travel to one of the locations observed on the map display on the weekend. The phone and conversation were visualized in photo editing software and was scripted to automatically show up in the user’s field of view and begin once the user reached a certain “frame” of the VR storyboard. The script also played an audio clip of the sound of typing on a smartphone keyboard to signal that the user was composing a message as well as a common notification sound which played when the user received a response to her message.

(34)

Figure 10 - Extract of key frames from the VR storyboard.

Figure 10 shows an extract of the key frames in the storyboard where the user can observe the map display at a distance and whilst standing on it, receiving information about the map location she is standing at. The final frame shows the phone interaction where the user sends a question to her friend asking if they should travel to one of the locations the user has stood at and received information about.

4.3.3 First round of usability testing

The prototype was tested on David and Jenny to ground it in their practice (section 3.7.1). The prototype was presented on a smartphone mounted in a Google Cardboard headset. The aim of the usability test was to evaluate if users would feel a sense of presence and place (sections 2.1.3 and 2.2) and if this type of VR prototype would have been useful in their own practice using the interactive installation project as a point of comparison.

When starting the experience, both participants looked around in their environment and very quickly recognized that they were standing at Värnhemstorget. Although they both stated that the resolution and general feel of the head-mounted display would benefit from being of a higher quality, suggesting a lack of vividness (section 2.1.2), they both also felt that the 360° images paired with the soundscape were effective tools for both creating an immersive experience which created the illusion of being at a different location. “I found it fascinating that holding this piece of cardboard to my face could make me feel like I was at Värnhem” David stated (authors translation), which suggests that he experienced both a sense of presence and place.

When asked if they believed that this kind of VR storyboard would have been useful in their own practice when working with the interactive installation project Jenny stated that she felt it could facilitate in the communication with