DEGREE PROJECT IN COMPUTER SCIENCE AND
ENGINEERING, SECOND CYCLE, 30 CREDITS
STOCKHOLM, SWEDEN 2020
Investigating presence in
remote meetings; a case
study testing extended
reality (XR) technology
Olivia Almgren
KTH ROYAL INSTITUTE OF TECHNOLOGY
ABSTRACT
During times with global pandemics and climate change, the need for companies to be able to conduct their business without travelling is essential. Upholding social distancing and complying to restrictions on travel both globally and nationally have not only forced everyone to conduct their business from home but to do so regardless of technological maturity. While also doing so for an unforeseeable future. In times of change, resilience is key. Having more durable and resilient teams and workers are essential now, tomorrow, and most likely in the future too. Improving the usability of remote collaboration has never been as important.
Disregarding the fact that this has been a forced act of measure from the government, the aftermath of this for many companies will surely include reduced costs for travel, improved efficiency and reduced environmental impact. Undoubtedly, there is incentive from a business perspective, but what are the effects from the user's perspective? Derived from previous literature on presence, video communication, quality of experience (QoE) and interaction, this case study set out to examine the following research questions; What current factors influence the remote meetings of employees in a telecommunication company? In what way can extended reality (XR) technology potentially improve their experience? Extended reality (XR) technology refers to all real-and-virtual combined environments including augmented reality (AR), mixed reality (MR) and virtual reality (VR) and other areas that exist among them.
With obtained data from interviews and a series of tests, the results indicate that factors from every category; human, system and context factors influence the QoE. Additionally, in its current state, XR technology does not provide enough, especially in terms of quality, to significantly improve anything for these employees. The XR technology has potential to heighten the experience in respects such as mobility, but for presence and social context it did not gain much attraction.
SAMMANFATTNING
Under tider med en global pandemi och klimatfo ra ndringar a r behovet att fo retag ska kunna bedriva sin verksamhet utan att resa stort. Att uppra ttha lla social distansering och fo lja restriktioner pa resor ba de globalt och nationellt har inte bara tvingat alla att bedriva sin verksamhet hemifra n utan att go ra det oavsett teknisk mognad. Det go rs ocksa fo r en ofo rutsebar framtid. I tider av fo ra ndring a r det viktigt att vara anpassningsbar. Att ha mer ha llbara och
motsta ndskraftiga team och arbetare a r viktigt nu, imorgon och troligtvis a ven i framtiden. Att fo rba ttra anva ndbarheten av samarbete pa distans har aldrig varit lika viktigt.
Om man bortser fra n det faktum att detta har varit en pa tvingad a tga rd fra n regeringen, sa kommer konsekvenserna fo r ma nga fo retag sa kert att inkludera la gre kostnader fo r resor, fo rba ttrad effektivitet och minskad miljo pa verkan. Utan tvekan finns det incitament ur ett affa rsperspektiv, men vilka a r effekterna ur anva ndarnas perspektiv? Uppbyggt fra n tidigare litteratur om na rvaro, videokommunikation, kvalitet av upplevelsen (QoE) och interaktion, syftar denna fallstudie till att underso ka fo ljande forskningsfra gor; Vilka aktuella faktorer pa verkar distansmo ten fo r ansta llda i ett telekommunikationsfo retag? Pa vilket sa tt kan XR-teknik (Extended Reality) potentiellt fo rba ttra deras upplevelse? Med Extended reality (XR) -teknologi menas alla reala-och-virtuella kombinerade miljo er inklusive Augumented Reality (AR), Mixed Reality (MR) och Virtual Reality (VR) och andra omra den som finns mellan dem. Insamlade data fra n intervjuer och en serie tester visar att faktorer fra n varje kategori; ma nskliga, system- och kontextuella faktorer pa verkar QoE. I sitt nuvarande tillsta nd, ger XR-tekniken inte tillra ckligt, sa rskilt inte na r det ga ller kvalitet, fo r att avseva rt fo rba ttra na got fo r de ansta llda. XR-tekniken har potential att fo rba ttra upplevelsen na r det ga ller exempelvis ro rlighet, men fo r na rvaro och social kontext har den a n sa la nge inte mycket att bidra med.
Investigating presence in remote meetings; a case study testing
extended reality (XR) technology
Olivia Almgren
KTH Royal Institute of Technology Stockholm, Sweden
oliviaal@kth.se
ABSTRACT
During times with global pandemics and climate change, the need for companies to be able to conduct their business without travelling is essential. Upholding social distancing and complying to restrictions on travel both globally and nationally have not only forced everyone to conduct their business from home but to do so regardless of technological maturity. While also doing so for an unforeseeable future. In times of change, resilience is key. Having more durable and resilient teams and workers are essential now, tomorrow, and most likely in the future too. Improving the usability of remote collaboration has never been as important.
Disregarding the fact that this has been a forced act of measure from the government, the aftermath of this for many companies will surely include reduced costs for travel, improved efficiency and reduced environmental impact. Undoubtedly, there is incentive from a business perspective, but what are the effects from the user's perspective? Derived from previous literature on presence, video communication, quality of experience (QoE) and interaction, this case study set out to examine the following research questions; What current factors influence the remote meetings of employees in a telecommunication company? In what way can extended reality (XR) technology potentially improve their experience? Extended reality (XR) technology refers to all real-and-virtual combined environments including augmented reality (AR), mixed reality (MR) and virtual reality (VR) and other areas that exist among them. With obtained data from interviews and a series of tests, the results indicate that factors from every category; human, system and context factors influence the QoE. Additionally, in its current state, XR technology does not provide enough, especially in terms of quality, to significantly improve anything for these employees. The XR technology has potential to heighten the experience in respects such as mobility, but for presence and social context it did not gain much attraction.
INTRODUCTION
Presence is one crucial element to fully experiencing a virtual environment. It is a subjective experience and can be impacted by a variety of contextual factors. The term presence has been used and studied in many forms, but distinct subcategorizations are made, two of which are telepresence and social presence [14]. More than four decades ago Marvin Minsky [11] started discussing the term telepresence in regard to instruments that could feel and work so much like our own hands that we do not notice a significant difference, achieving the sense of being there. Building on this and taking Minsky’s term one step further, Hollan and Stornetta [8] suggests going beyond being there. For a long time, face-to-face interactions have been seen as the baseline because of the richness and variety manifested in them. They suggest that we should aim towards building tools that users prefer using more than the face-to-face physical interaction, beyond being there. Social presence differs to telepresence in the sense that it is about the degree to which the medium supports awareness of others, the interpersonal relationships and achieves a sense of being with another [14] rather than being there [11] or beyond [8]. In most technologies today, the real-world is used as a baseline. With the aim to try to replicate what makes a face-to-face interaction successful in the new technologies and better it. Dourish et al. [5] questions the real-world baseline and points to it not always being an appropriate point of comparison for new media technologies. The complexity and intricacy that constitutes communicative behaviors is something that grows over time. When considering media spaces and the connection that these spaces create between individuals, one must also consider the wider perspective where these individuals are parts of more and bigger social groups. Dourish et al. corroborates such findings and explains that when the medium changes, the mechanisms also change, but what remains is the communicative achievements. Looking to real-world interactional practice for a set of design guidelines
for media environments is common and consequently, there are important lessons to be learned when comparing mediated communication and face-to-face interaction. Even so, Dourish et al [5] argue that using the real-world baseline to evaluate the efficacy and value of media spaces is to miss the point since the media space world should also be viewed as the real world. Meaning that it is a place where real people, in real working relationships, engage in real interactions. The mediated interaction is a developed skill, not an adapted form of real-world, face-to-face interaction. Users tend to employ resources and develop skills which allow them to overcome some of the issues that are identified as troublesome in these technologies. By creating new patterns of behavior organized around the nature of the medium itself, users find ways to make it work for them. Additionally, McLuhan [12] reflects on technology as an extension of the body. Every technological extension has the effect of amputating or modifying some other extension. An example of an amputation would be the loss of the art of penmanship through regular correspondence when the telephone became an extension for the voice. The extensions have, according to McLuhan, always been at the forefront of attention. Consequently, the amputations are usually ignored, minimized, or forgotten. Thus, the eminence of the extension overshadows the consequences. For instance, now we praise the advantage of being able to communicate via video that is made available via telephone and computer, but what is being amputated in the process? McLuhan extends and states that people generally do not want to be reminded of the negative aspects of things, that the resulting amputation can cause equally or more harmful effects then the extension can bring positive. McLuhan also believed that this way of doing it is at our own peril.
Minsky, Holland and Stornetta, Dourish et al and McLuhan's discussions are all becoming even more relevant when moving closer to the umbrella term of so-called extended reality (XR). The XR technology that will be used in this study, the AR smart glass called Microsoft HoloLens, rather realistically incorporates 3D hologram images into the user’s field of vision while simultaneously keeping the vision of the glass wearers own reality. In other words, the glasses superimpose digital information onto a user’s field of view [3].
To investigate presence and other factors influencing remote meetings and in what way XR technology could improve this experience, this study is divided into two parts. The first, aims at investigating what issues people are currently facing in a telecommunication company and what needs are not fulfilled in remote meetings as they are today. Derived from the first part of the study, the second part aims at testing whether the current needs and factors can be helped or improved upon by leveraging of XR technology.
RELATED RESEARCH
Churchill [4] explains the concept of “social glue” and says that in order for communication technology to be considered as a “social glue,” it must bring people together and keep them in a sort of social dynamic. The technology should adhere people to each other and/or to social places by acting as a facilitator to strengthen those relations. On the contrary, technology can just as easily hinder such connections and instead of it being the necessary adhesion it takes the form of a solvent. Churchill also states that the introduction of new social technologies is a part of a bigger whole, including social relationships, which influence how a technology is received, adopted, adapted, and ultimately owned. One key component in the formation of social relationships is communication. More than half of the communication being transferred in interaction between people is nonverbal. Nonverbal cues such as paralanguage, body language, posture, facial expression, eye contact, proxemics, haptics, chronemics, pheromones etc. all contribute to the outcome of a physical interaction. The degree to what extent a telecommunication medium can support a feeling of social presence have been found to depend on additional cues that an interface can support [7]. Furthermore, what is also important to note is that most of these nonverbal cues happen unconsciously. Trying to make technology that assists such mechanisms, that naturally occurs, is difficult due to the simple fact that the mechanisms probably change depending on the technology provided and the context they are used in. The mechanisms are conditional to the context.
Hollan and Stornetta [8] further reinforces the claim that these mechanisms might not be inherently transferable across different types of media. Meaning, what works in a physical interaction, might not be successfully transmitted into the realm of a technology and what
works in a specific media might not work in another one. Similarly, [1] find that different contexts result in striking differences in users´ attitudes and practices, independent from the technology used. Meaning that using the same technology across different scenarios does not work because of the differences in context. Given the different reactions to the same video technology, depending on context, this further emphasizes the notion that the technology is not and should not be the core or point of departure. Instead of technology being the core, [1] suggests using a shared experiences perspective to support users' dynamic behavior. And from the activities and shared experience look at how technology can help enable them.
As further reinforced by [16], needs and constraints vary significantly across different types of scenarios and a one-size-fits-all approach is not sufficient. Which might explain the various approaches implemented in this particular field of research. So instead of trying to look at the mechanism and the technology this study aims at looking at this from the user’s perspective, what their needs and goals are, and in what ways those are not met sufficiently.
Distributed meetings have extensive amounts of dimension to them that are all contributing factors towards the experience needed to be mediated through technology. Various studies in the field have been conducted to identify the user requirements and specific design dimensions that are crucial for the interactions occurring in remote meetings [16] [1] [19] [15] [9]. [19] researched the top problems people were having with remote meetings and categorized these findings as audio, behavior, and technical problems, which were all correlated to the meeting effectiveness. Some stated factors were, users not being able to view material, poor audio quality and participants not checking for understanding. In their study they also examined the meeting type, size, duration and configuration and the participants location and role in correlation to the effectiveness of the meeting. All these factors were found to be related to how effective the meeting was. Similarly, [16] proposes a framework consisting of seven design dimensions with associated characteristics for understanding the contexts in which telepresence technologies might be used. The seven dimensions are initiation, physical environment, social environment, mobility, vision, communication, and independence. The physical environment is where the system will be used,
and the social environment is the relationships among all participants. Additionally, mobility is said to be the amount of movement the participant will engage in while using the system and vision is what the users see and how they visually adjust their focus. All aspects constitute their framework.
[15] states that in order to create an environment for successful interaction, the purpose of the interaction needs to be recognized. They also state that the increase in remote working indicates that face-to-face interaction becomes much more important for team building, motivation and sense of loyalty and belonging. They could also make the conclusion that physical and organizational factors both contribute to successful interaction. Fayard and Weeks [6] discuss environmental affordances in informal interactions. They define environmental affordances by how the physical and social characteristics of an environment jointly influence the perceptions and behaviors of actors. Fayard and Weeks [6] argue that to understand the relationship between environments and interactions, we need to take seriously the idea that the physical artifacts and the social constructions of organizations are mutually constitutive.
The facilitation of remote teams has been proven difficult in various different aspects [13]. Building and developing teams and settling on decisions over a distance are two aspects mentioned. In regard to short-term remote work [9] found that most teams were able to execute and operate without sizable effects on the relationships between employees. However, in a long-term perspective what has also been found is that the durability of virtual team working is dependent on commitment and personal trust relationships. Supporting the sense of social presence is therefore important. Otherwise, these relationships may dissipate over time without collocated face-to-face social interactions.
In telecommunication the so-called Quality of experience (QoE) is used as a way to measure the performance in a system [2]. The measure is based on human perception and the QoE can be impacted by a variety of different factors. These so-called influence factors are defined as: “Any characteristic of a user, system, service, application, or context whose actual state or setting may have influence on the Quality of Experience for the user”. By identifying and classifying these factors the aim is to
provide a definition for QoE which could be relevant for the largest possible set of application fields. Influence factors that have been stated to constitute the QoE is human, system and context. These three groups often overlap and relate to one another. The human influence factors are stated to be socio-cultural and economic background, gender, age, expectations, needs and goals, other personality traits etc. System Influence Factors can be content, media, network or device related. Meaning for instance resolution, delays, or size of screen. Context influence factors refers to physical, social, economic, temporal, task, and technical contexts. These classify elements such as interpersonal relationships, location, task type, space and more. These factors often overlap and have a mutual synchronous impact on the QoE. Additionally, research have shown [17] how once a system has enabled users to engage in an interaction, participants will be quite forgiving about quality degradations, until it brings them out of the flow. Meaning that to estimate the QoE of participants, knowing for instance only the system factors, would not be enough. It is also necessary to know the users and understand what their needs and goals are in order to build software that can actually balance system factors for the current situation. What many of the related research has aimed to accomplish is to develop such dimensions or put forward a framework as a tool for designers and researchers to facilitate communication across many domains. To continue the framework provided by [2], which is deemed to be the most comprehensive, the insights from this study will be classified according to the above influence factors. The aim is to maintain a structure that can be easily transferable for future research.
METHOD
To investigate what factors currently influence remote meetings and in what way XR technology could improve this experience the following methods were applied. The research study was initiated with 18 semi-structured interviews with employees who have been, or just started, working remotely. Meaning that they work from their own home. Respondents were predominantly based in Sweden, however all of them had both national and international remote meetings on a continuous basis. A thematic analysis of the interviews was performed with an inductive approach. Meaning that the
data was analyzed with the goal of finding reoccurring themes and patterns but doing so directed by the content of the data [10]. The themes extracted from the interviews are then connected to previous literature and are categorized with the help of the structure provided in the QoE paper [2]. To address the findings from the interviews and examine the potential of XR technology improving on the contextual factors extracted and categorized in part one, a series of user tests were conducted. A total of 9 user tests were composed, all with current employees from the company. All the employees were based at the office in Stockholm and each test was carried out in a controlled lab environment. The testing method and design is based on the ITU-T Recommendation Protocol [18]. This standardization document is intended to define interactive evaluation methods for measuring the impact of audiovisual communications. A detailed description of the prerequisites of the test can be read in appendix 2. Each test consisted of the moderator, a user and two sets of Duplo Lego. The two Duplo sets could be built in two different ways but with the same number of parts. The Duplo parts were spread out on a table in front of the user in a way that enabled them to conduct the tests without obstruction. This setup was replicated on the moderator’s side. When wearing the HoloLens glasses the user can see a square stage, approximately 100x100 cm in size, that is used for bidirectional communication over a low latency, high bitrate connection [Figure 1].
Figure 1: Through the glasses the participant sees this square, which in this instance shows the Duplo Lego pieces in its finished form on the moderator’s side.
In the square the user can see the life size hologram of the moderator. The user can also see the table with the building blocks.
In the first condition the user was asked to put on the HoloLens glasses and follow the moderator’s instructions to build one of the models out of the Duplo blocks [Figure 2]. Through the glasses the user could see the moderator, which did not wear glasses, and the Duplo blocks that the user was going to try to replicate [Figure 1].
Figure 2: A user wearing the Microsoft HoloLens glasses doing the first condition of the user testing.
The blocks were taken apart between the two conditions. In the second condition the same set up was applied and the user was again asked to follow the instructions of the moderator, but this time through a video-call. The same blocks were used but were built in a different way. Following the completion of the test the users were asked to be part of a semi-structured interview, reflecting upon their experience with the two conditions.
RESULTS
This first part of the study resulted in several factors influencing the quality of the experience in remote meetings conducted with video-calls. The extracted factors will be presented in the following section, including examples from interviews and test results. Something to be made clear is that many of these factors often interrelated and overlap to one another and should not be seen as independently isolated. A summary with the key quotations can be read in appendix 1.
Human Influence Factors
Human influence factors include individual values, needs and goals, motivations, preferences and sentiments,
attitudes, and personality traits and will be discussed in this section.
The respondents were asked to reflect upon their experience of working remotely. Many of the respondents naturally started explaining the disadvantages of having remote meetings in comparison to the physical face-to-face meeting. Firstly, one respondent reflected on the ability to fully see how the other people react to the things being talked about in the meeting:
“In a face-to-face meeting you can immediately see through a whole range of signals, you know body language or how the person is behaving or the attentiveness of the group or the reactiveness of the participants, you can immediately tell if a certain proposal has a certain traction, if there is a good chance of it being agreed or modified or that there is no chance in hell that it would get through.”
In many of the answers provided by the different respondents there was a clear need for a more vivid response then what is currently being supported by video only. Almost everyone mentioned that they had a preference of being able to see how the other person was reacting in a more holistic sense, something which they felt were not fulfilled remotely with video-call only. Some were also ardent to talk about the sentiment of a room. Describing that this “feeling” or “mood” that occurs when they meet physically is hard to grasp when they are not in the actual “room” with the people they are having the meeting with.
"Remote meetings makes it hard to pick up on nuances. Not only in verbal language but also in body language.”
The mood of the room is, similarly to the reactiveness discussed above, explained to be more clearly distinguished through non-verbal communication such as one’s body language, facial expression, behavior and so on. However, as illustrated in the quote above the nuances that occur can be hard to pick up on in remote meetings. The attentiveness, engagement and the motivation of the people that are in the meeting can strongly be affected by the lack of interactivity that several respondents’ current remote meetings are having:
“What it boils down to is that you must keep people engaged, keep their attention. If new
technology could bring us closer to the level of interactivity of a face-to-face meeting, it would help.”
The following respondent also describes that this lack of interactivity directly decreases the effectiveness of the meetings and refers back to the need for a reaction in order to be able to motivate and engage the participants:
“I would also say there is a decrease in effectiveness due to the lack of interactivity. Because when you interact on a face-to-face basis you know immediately what works and what does not. You see the kind of closure and opening that your counterpart in a debate is giving you, so its incentive, if you have some skills. An experienced person can adjust his argument, can adjust his story and a lot of things depending on the reaction he gets.”
The attention of the people in remote meetings is not only a product of the interactivity, reactiveness or lack of sentiment but can also be affected by the medium itself. The screen constitutes a distraction for the user and even if the content and the people that are in the meeting are of high interest, the attention might be disrupted because of a single notification on the screen.
System Influence Factors
The system influence factors reflect the technically produced quality. They are divided into four classifications, listed below, to distinguish their different properties.
Content-related
A common thread among the respondents was that they wanted to be able to see the other persons in the meeting. When it comes to visual information, the amount of detail as well as the amount of motion in the scene is described to be vital for many of the respondents. Simply sitting there and watching something static without any movement was not desirable content.
Network-related
Many respondents explain how they often have been forced to turn of the video in order for the meeting to work without delays and loss of information due to bandwidth issues. The occurrence of such errors has a negative impact on the QoE and has resulted in the means of action being to turn off what is possible to not waste precious time and execute what was intended from the start, unfortunately at the cost of the
experience. Several respondents have expressed that technically it has not been as disturbing as they had expected and that it has been working rather smoothly with some hiccups:
Media-related
The media synchronization is described to be an issue. Having to install an array of different tools to make the meeting flow as desired is not optimal. The respondents also explained that different computers are used for different projects. For a person working remotely, this creates a situation that might be very hard to work around if there is no easy way of solving this. This might simply be a matter of lack of understanding for the media at hand or the fact that it does not support the needs of that particular user. Either way, this kind of intricacy is unnecessary and creates extra hurdles, especially for the following user:
“Sometimes I use different computers for different projects. Now, a lot of the time you have to install external software for everything to work as you wish. Wishes it to be easier sharing documents, hand raising, point of show-hard to know when and what you are showing, some sort of docs per minutes, those things are to separate now.”
Device-related
Respondents also considers the impact the display size of the device has on the overall experience. Explaining the fact that often, meetings revolve around viewing some type of document or presentation. Due to the device that this meeting is viewed from, it is not possible to view each participant simultaneously as the document. The display is simply too small for both the content and the people to be shown in a satisfying way for the respondent. Which relates to the content-related factor mentioned above. Furthermore, a latent challenge and apprehensiveness that has become even more apparent for respondents during this particular time is the actuality of wearing devices on the body. Having to touch or wear devices that are not used only by yourself is seen as unhygienic, especially during a pandemic.
Context Influence Factors
The context influence factors are also divided into categories namely physical, social, economic, temporal, task, and technical contexts.
Physical context
Working and having meetings in the location and space that is considered the home has made some of the respondents realize that they tend to change their routines in order to not look a certain way. Being more wary of how they represent themselves through the scenery of their location, they make sure they stay on top of cleaning and do not have “inappropriate” things standing in the bookshelves or around themselves:
“You change the routines when you work remotely from your home, you need to fix a representable environment, clean more. The boundaries become more fluid.”
That the boundaries are more fluid when you are working remotely are further confirmed by several users that describe how they reposition themselves more inside their own home. Whereas in the office they are often glued to their desk.
Respondents also talked about the difficulty of being two people working from home simultaneously. It has become clear in this study that most households do not possess access to more than one in-the-home-office-setup if that. This is vividly described by the following respondent:
“I sat at my kids crafting table and now all my notes are covered in glitter, it is a bit tricky when we are both working from home.”
Task context
As mentioned by other respondents in the device-related section, often the task creates a context in which the device is not sufficient to support it. This respondent reflects on tasks that are predominantly occurring, especially in this company:
“The thought is that everyone should be in this meeting room and everyone will use PowerPoint, but that is not a law of nature, it is just a paradigm that we are stuck in.” “Everything is so revolved around presentations, not around the interaction itself.” The tasks that used to occur physically have also been transmitted to the remote realm but, according to the respondents, without really having worked in the first place. They explain that the interactions are what make the meetings “memorable” and are easier “to relate to at a later stage”. The task is not the best in its pre remote meeting state and becomes even worse when other
influence factors arise that make the task even less memorable and relatable.
Temporal context
The concept of working time has shifted and similarly to the boundaries of location and space becoming fluid, working time has sustained a blurred border. This does not only mean that workdays can be longer, but it also makes it apparent that for many people “workdays are no longer 9-17”. The following quote illustrates one respondent’s reflection on this:
“Private time is seriously reduced. If you are on a business trip you go out on this battle and you know once this is over, I go home. In this case it actually intrudes into your private zone, because if you are in China you would have to lock yourself out from your spouse and kids because you have to connect to the meeting.” This quote clearly reflects on both the physical and the temporal context. However, the temporal influence factor can be hard to overcome. The consequence of having a global company with employees all over the world is that scheduling remote meetings during a time of day that works for everyone is rather difficult. Many respondents express that the differences in time zones is often a constraint. Naturally, the more companies and people are working remote, the higher the demand is for “the golden slot”. The two-hour window which works best for a global participant group.
Social Context
During an experience, the interpersonal relations define the social environment. Social ties are a part of interaction and therefore it is important to consider how different people are involved in such an experience. A majority of the respondents touched on the importance of initiating relations with people physically before having remote communication taking place. One respondent touched upon this while also describing that it applies to both people that are working internally at the company and external connections:
“Meeting people physically before feels like a prerequisite for the digital meetings to work. I have also noticed this internally, when it comes to people you do not know it gets choppy. Usually, you plan some type of get together where everyone meets face-to-face, then you notice that it works much better afterwards. Then you can tell on the voice who is talking, you
know how they act, you know how they are and that is very telling. If this situation would continue, there is always a natural rotation of people and there will appear people that you don’t know and that don’t know you.”
Many also described that relations inherently have a need to be maintained. In order to build and keep trust you need to nurture those relationships.
Having meetings remotely also bring, what one respondent refer to as intimacy because they talk more to “private individuals”. Since the setting is more personal and people feel more at ease in the comfort of their own home, the respondents sense a feeling of intimacy because they are invited into this sphere that they have not seen before. This intimacy is further acknowledged by one respondent that also discusses the risk of falling into a loneliness that is not very healthy:
“With my colleague in Australia, we decided to grab a tea to catch up. He was sitting inside, then he went outside to show his backyard, there was an alpaca standing there. No matter what, it is boring to just be sitting by yourself all the time. The social element, it is a risk you know, getting blunted out socially.”
Many of the respondents that were used to working from a remote setting describe how they no longer feel as if they are “the bad ones”.
“I am no longer just a voice in a box, no longer an evil thing that just creates problems.”
Now that everyone is conducting their work and having meetings from home there is no longer that huge disparity between the remote workers and the office workers. The playing field has been evened out and now everyone is on the same level, which also seems to have grown the acceptance level if something were to go wrong.
Overall, the results have shown that most of the respondents were all able to execute their work from a remote setting but with various concerns. Firstly, the lack of nuances that has become expected in remote meetings paves the way for attention loss and creates a lack of motivation. Many feel that they do not get the desired reactions to conduct the meeting in the most beneficial way. Interactivity is not supported adequately, and the screen can be a source of distraction and vision impairment. The respondent feels like they must have met physically first otherwise communication becomes
much more difficult. They also account for a bigger feeling of loneliness but on the contrary also more intimacy when they get to see the more private side of their colleagues.
To further examine the key influence factors established in the above section, the following part brings forth the results from the user testing.
USER TESTING
To investigate if leveraging XR technology could improve the key factors brought forward in the previous section, a series of user tests were conducted. The key issues that were mentioned most frequently and by the most amount of people will be investigated further. After each test, the participants were part of a semi-structured interview in which they were asked to reflect on their experience with the conditions. The following section presents the results from those interviews.
Human Influence Factors
The most pronounced issue and the overarching theme of a majority of the answer were that the quality of the XR solution simply was not good enough. Even if some of the parts could have been up to par or even better than the video-call the poor quality was too distracting for most of the participants. So even if the participants were asked about the human influence factors such as body language and attention, they could not distinguish those shifts or activities clearly enough to be able to reflect about them in the most optimal way.
When discussing the reactiveness, body language and facial expressions one respondent explained that it felt like a non-human conversation. Even though it is possible to see the other person, you cannot feel that connection or see the mimics of the face. Which made it impossible to read facial expressions or guess what the person was saying or doing. Many said it felt like talking to an “avatar”.
“Through the hologram it felt like you were an AI figure or something very far away. It was not real. You looked like an AI instructor, over there, which you can’t see really, and that felt really distant.”
On the other hand, a majority of the respondents reflected on the fact that in XR you get a more real-life approximation of the person you are talking to. Being able to see the full length of the person made it more
“life-sized” and “real”. Which was a positive note in comparison to the regular video-call views that the participants were used to, were most of the time they were only able to see the colleagues face and shoulders. This was moving closer to the more holistic view that they wished for earlier in this study.
The user discussed that when the goal is to reach agreements, doing workshops and working creatively on a whiteboard, they could see the potential with the XR technology, because it affords a different interactivity level. But as it is now, the quality needs to improve significantly.
System Influence Factors
As previously stated, the quality is a central issue with the XR condition. The connection was not good enough and the participants could see black outlining, jitter and shadows when wearing the glasses, which made it a lot more difficult to distinguish those essential signals. The resolution unfortunately created “shadows and shapes” that created confusion for the eyes, which is visible in Figure 1. This was the hardest thing to accept as part of their experience according to the participants.
When wearing the glasses, the participants are not dependent on a screen in the same sense as they are when they are on a video-call. However, there is a square that is visible in which you can see the person you are talking to or other objects depending on how you move your head [Figure 1]. One participant did not appreciate this element but could see an advantage in the possibility of a higher personal control, meaning that having a screen like with video-call, makes everything more static, whereas the mobility is much higher with XR. It is not as restricting as being tied to a chair and a desk, both the user’s mobility in the physical space and virtual space has increased:
“The little square in Hololens is a pain. It is like having a screen sort of. But that is the difference, also I can decide what is shown in the image and when there is video you can’t. That is an advantage, that you can look around even though you can’t see everything.”
However, an observation made during the test was that not all users were prone to moving in the first condition. Several remained quite static, despite standing up and having space to move their head and body. This could be due to the affordance of the task
itself, affordance of the system or simply a matter of habit from the users, but worth noting, nonetheless. A majority of the participants stated that the weight of the glasses was not acceptable. They were described as “not very comfortable” and “to heavy”. The glasses are also a lot more “invasive” than they would have liked.
Context Influence Factors
The physical context was hard to distinguish in these user tests because of the fact that they were conducted in a controlled lab environment. The temporal context is also not discussed since it was no differentiation of time between the participants. In terms of the task context many of the participants could not see the benefit of having XR for their everyday tasks and meetings:
“I don’t think it works, working together, taking notes, writing something, it doesn’t work unfortunately. There may be, workshops I can imagine are possible to do, but not for my everyday tasks. It should really be a task where you need the depth feeling. More practical tasks would benefit from this.”
Even though there were deviations in this matter a majority still did not view the task in the XR experience as being executed in a preferable way than with the video. Even though the task had a higher practicality level to it than the usual work tasks of the participants. As mentioned previously, XR has a more real-life approximation which would suggest that it could act as a bridge between the highly suggested element of meeting a person physically before having remote meetings. However, the fact that the reactiveness, facial expressions, body language and other nuances in non-verbal communication did not come through due to the lack of quality, indicate that it would not work this way in the state it is now.
When testing the two conditions, it became clear that more errors were made by the users when building the blocks with the help of the XR technology. The average completion time in the condition with the XR technology was also longer. This implies that, the quality made it harder for them to view what was what and where.
DISCUSSION
The aim of this study was to investigate what issues employees in a telecommunication company are currently facing and what needs are not fulfilled in
remote meetings in their current state. Derived from the first part of the study, the second part aimed at testing whether the current issues could be improved with the use of XR technology. The research questions examined were: What current factors influence the remote meetings of employees in a telecommunication company? In what way can extended reality (XR) technology possibly improve their experience?
The results show that the overall perception is that the XR technology in its current state does not improve the current issues and factors deemed as troublesome by the users. Hollan and Stornetta's [8] aim of going beyond being there, having tools that the users prefer using then the physical face-to-face interaction, is not close to being fulfilled. The bad quality is the biggest contributor to the overall perception being so low. The XR technology also did not support the interpersonal relationships nor achieve the sense of being with another [14] more so than the regular video-calls were. Therefore, social presence was not attained or improved.
Dourish et al [5] questions the use of the real-world baseline and whether the usage of that causes the discussion as a whole to lose focus. When interviewing users about their needs, goals, and perceptions a majority tend to reflect upon their experiences in comparison to the physical real-world face-to-face interactions. This is what the users naturally turn to and use as their point of departure. It is hard to see how that cannot interweave into the consideration when that is their considered as their utopia. Furthermore, this sort of mindset must especially be perilous when it comes to designing for users. Dourish et al states that users will find a way to work around issues, which indeed might be true. However, it is an assumption that is putting aside the fact that we are creating technologies with the intent of them being used by people. Yes, humans are adaptive creatures but that should not be the baseline assumption either. Like one respondent neatly put it: “Some people are just as prone to change as the royals.”
To continue, a viable next step would be to find a way for those, seen as mandatory by the respondents and by previous research, initial physical meetings and in the long run a way to maintain them [9]. Given the results from this study, this was one of the bigger concerns for the users. Thus, the importance of having a solution or software in place that can support this is judged to be essential for the future. XR technology could assist this
notion, if more practical task would be the solution for this. However, this would have to be researched further in order to judge what could work or not and also that is, given that the quality is raised to an acceptable level according to the users.
Furthermore, combining the user perspective while also looking at the shared experiences [1], could be a beneficial way of continuing to examine this. Given that one of the primary goals in a work setting like this is to execute specific tasks one might need to look at what the users goal is and what experience need support in order for that goal to be reached. Overall, the studies of the problems [19], dimensions [16], environment [15] [6], all aim for practically the same goal of reaching an understanding of how successful interactions are created and supported. However, the lack of a unified language is motivating the use and aim of trying to maintain a structure with the help of the QoE framework. Churchill [4] reflected on technology as social glue and the need for it to bring people together in order for it to be considered that. Throughout this study, current video communication technology has shown to have bonding properties which have helped employees continue their work from a remote location. When comparing the current standard of video-meetings with an XR meeting solution, there was a dominant realization that the actual quality of video-meetings is very adequate. Which was not entirely the case when the respondents, in the first part of this study, discussed the questions related to the system influence factors. The necessary adhesion is there but, according to some users it is hanging by a thread in some respects. The longevity is seen as questionable and not as a sustainable solution in regard to the human influence factors and social context in particular. There is, as mentioned previously, an incentive in terms of economic and environmental sustainability to enhancing the experience and means of remote collaboration. A decrease in business travel for this telecommunication company in particular would mean less money spent, inefficient time reduced, and the carbon neutrality goal a lot closer to fruition. At least in the short term, however, the equation is not that simple. The effects on social sustainability are the most unpredictable part of this. We have this technology that could indeed be of much help, especially now. But to paraphrase McLuhan [12], what are we amputating in the process? The participants in this study have reflected on the scarcity of physical connections starting to create
social bluntness and loneliness. These are serious topics that needs to be considered and addressed in every aspect of life, not only in the creation and development of technology. The ethics around this matter is a challenge for employers and employees alike. This study has shown how users must change routines and their private sphere being on display more than they might want it to be. How this way of working would affect people, need to be researched, especially in a longer-term perspective.
This research study is subject to limitations. The data collection was conducted internally at a global telecommunications organization, meaning that the findings should not be considered to be generalizable outside of this scope. Furthermore, the initial semi-structured interviews touched upon remote meetings in a general sense in terms of participant number, meaning that the respondents could reflect upon meetings that were one-to-one, one-to-several, to-one, several-to-several or everyone individually remote. These factors will have had an impact on their perception as is shown in some examples mentioned above. Additionally, the tests in themselves were only conducted between two persons, one of which was the moderator in each case. The main reason for this was constraints in technology, space, and governmental restrictions. In addition to the visual holograms, the experience also enables voice communication. However, it is important to note that the audio was not taken into consideration in the user testing due to limitations of the technology and space of where the tests were conducted. Admittedly, audio can play a significant role in the the quality of these experiences. Therefore, the recommendation for future research is to also examine this factor more closely. Also, to reach more of an ecological validity, conducting these tests in a setting that is more natural for the users should be something to aim for. In that way it is possible to document what they do and how they behave in a more natural way that might not be influenced by the lab context.
CONCLUSION
This study has presented the current factors that influence remote meetings of employees in a telecommunication company and divided them into human, system, and context factors according to the QoE. Based on the findings in the initial interviews, user testing was conducted to examine the possibility of
leveraging XR technology to improve their experience. The results indicate that XR technology does not provide enough, especially in terms of quality according to the users. There are potential benefits when the work tasks are more interactive in themselves and could leverage from a more interactive tool, then it could improve the experience.
The socially situated contextual factors are not seen as better in neither of the two conditions and needless to say the real world is still the utopia in this aspect. When analyzing presence from this contextual perspective the achievement of a sense of being with another [14], being there [11] or beyond [8], has not been reached in the most desirable way. For the context that has been presented in this study the added value of the XR technology is not high enough for there to be an incentive for the telecommunication company to take aboard this as a standard tool for a variety of employees. However, given a future scenario where among other things, the level of quality is improved, the results might be significantly different.
To conclude, paraphrasing McLuhans [12] reflection on amputations. Now that we praise the advantage of being able to communicate via video or other technology, reflecting on what we might be amputating in the process is vital in order to not only know the consequences but also to understand what needs might need to be fulfilled in the future.
REFERENCES
[1] Brubaker, J. R. et al. 2012. Focusing on shared experiences:
Moving beyond the camera in video communication. Proceedings of the Designing Interactive Systems Conference (DIS ’12), June 11-15, 2012, Newcastle, UK, 96– 105. doi: 10.1145/2317956.2317973.
[2] Brunnstro m, K. et al. 2013. Qualinet White Paper on Definitions of Quality of Experience Output from the fifth Qualinet meeting, Novi Sad. European Network on Quality of Experience in Multimedia Systems and Services (COST Action IC 1003), (March 2013), 1-17. Retrieved March 15th,
2020 from:
https://hal.archives-ouvertes.fr/hal-00977812/document
[3] Chuah, S. H. 2018. Why and Who Will Adopt Extended Reality Technology? Literature Review, Synthesis, and Future Research Agenda. (December 13). Retrieved March 15th 2020 from: http://dx.doi.org/10.2139/ssrn.3300469
[4] Churchill, E. 2009. Introduction: Social glue. (November
2009). Retrieved March 15th 2020 from:
https://link.springer.com/content/pdf/bfm%3A978-1-84882-727-1%2F4%2F1.pdf
[5] Dourish, P. et al. 1996. Your place or mine? Learning from long-term use of audio-video communication, Computer Supported Cooperative Work, 5(1), 33–62. DOI: 10.1007/BF00141935.
[6] Fayard, A, and Weeks, J. 2007. Photocopiers and Water-Coolers: The Affordances of Informal Interaction. Organization Studies, vol. 28, no. 5, May 2007, 605–634. doi:10.1177/0170840606068310.
[7] Hauber, J. et al. 2005. Social Presence in Two-and Three-Dimensional Videoconferencing, Proceedings of ISPR Presence 2005, September 21-23, 2005, London, UK, 189–
198. Available at:
http://ir.canterbury.ac.nz/handle/10092/352.
[8] Hollan, J. and Stornetta, S. 1992. Beyond being there. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, May 1992, Monterey, California. 119– 125. doi: 10.1145/1940761.1940870.
[9] Nandhakumar, J. and Baskerville, R. 2006. Durability of online teamworking: Patterns of trust, Information
Technology and People, 19(4), 371–389. DOI:
10.1108/09593840610718045.
[10] Nowell, L. S. et al. 2017. Thematic Analysis: Striving to Meet the Trustworthiness Criteria. International Journal of Qualitative Methods. vol. 16, no. 1, October 2, 2017. doi: 10.1177/1609406917733847
[11] Marvin Minsky, 1980. Telepresence. (June 1980). Retrieved
March 15th, 2020 from:
<http://web.media.mit.edu/~minsky/papers/Telepresenc e.html>.
[12] Marshall McLuhan. 1964. Understanding Media: The Extensions of Man. (1964). Retrieved March 15th from:
https://web.mit.edu/allanmc/www/mcluhan.mediummess age.pdf
[13] Mittleman, D. et al. 2000. Best practices in facilitating virtual
meetings: Some notes from initial experience. Group Facilitation: A Research and Applications Journal, 2 (January
2000), 5-14.
https://www.researchgate.net/publication/260284235_Be st_practices_in_facilitating_virtual_meetings_Some_notes_fr om_initial_experience
[14] Oh, C. et al. 2018. A Systematic Review of Social Presence: Definition, Antecedents, and Implications. Frontiers in Robotics and AI, October 2018, Orlando, Florida. doi: 10.3389/frobt.2018.00114
[15] Oseland, N. et al. 2011. Environments for successful
interaction, Facilities, 29(1), 50–62. DOI:
10.1108/02632771111101322.
[16] Rae, I. et al. 2015. A framework for understanding and
designing telepresence. Proceedings of the 2015 ACM
International Conference on Computer-Supported
Cooperative Work and Social Computing (CSCW ‘15), March, 2015, Vancouver BC, Canada, 1552–1566. DOI: 10.1145/2675133.2675141.
[17] Schmitt, M. et al. 2018. Towards Individual QoE for Multiparty Videoconferencing, in IEEE Transactions on Multimedia, vol. 20, no. 7, July, 2018, 1781-1795. doi: 10.1109/TMM.2017.2777466.
[18] The International Telecommunication Union. 2000. ITU-T Recommendation P.920 - Interactive test methods for audiovisual communications. May 18, 2000, Geneva,
Switzerland. Retrieved March 15th from:
https://www.itu.int/rec/T-REC-P.920/en
[19] Yankelovich, N. et al. 2004. Meeting central: Making distributed meetings more effective, Proceedings of the ACM Conference on Computer Supported Cooperative Work (CSCW ’04), November, 2004, Chicago, Illinois, 419–428. doi: 10.1145/1031607.1031678
APPENDIX 1
RESULTS
Human Influence Factors
“In a face-to-face meeting you can immediately see through a whole range of signals, you know body language or how the person is behaving or the attentiveness of the group or the reactiveness of the participants, you can immediately tell if a certain proposal has a certain traction, if there is a good chance of it being agreed or modified or that there is no chance in hell that it would get through.”
“Physically you can see how people react, when you are digital you lose this part.”
“It is hard to have meetings when you can’t get a feel for the mood in the room so to speak. You might be sitting and tensing up in your separate locations and then the meeting becomes tense even though it doesn’t really need to be, when you meet physically you can read the body language and such also.”
"Remote meetings makes it hard to pick up on nuances. Not only in verbal language but also in body language.” “What it boils down to is that you must keep people engaged, keep their attention. If new technology could bring us closer to the level of interactivity of a face-to-face meeting, it would help.”
“I would also say there is a decrease in effectiveness due to the lack of interactivity. Because when you interact on a face-to-face basis you know immediately what works and what does not. You see the kind of closure and opening that your counterpart in a debate is giving you, so its incentive, if you have some skills. An experienced person can adjust his argument, can adjust his story and a lot of things depending on the reaction he gets.” “In a physical meeting you are watching the other people, but in a remote meeting you are looking at the screen and then it is easy to get distracted, an email or chat can pop up. Even if you are highly interested in what is being said this is a distraction.”
System Influence Factors
Content-related
“Being able to see the other person is a requirement for me, then you also get a similar effect as the physical meetings.”
Network-related
“The bandwidth loss forces you to turn off the video.”
“It is working surprisingly well, what you notice sometimes is that the broadband is not enough.”
Media-related
“Sometimes use different computers for different projects. Now, a lot of the time you have to install external software for everything to work as you wish. Wishes it to be easier sharing documents, hand raising, point of show-hard to know when and what you are showing, some sort of docs per minutes, those things are separate now.”
Device-related
“Even if you have a document, often you might look at the document being shared and if not that would mean that you would need a very big screen to be able to see everyone.”
“I wish I did not have to have any devices on my body or touch a lot of things, especially now during the virus.”
Context Influence Factors
Physical context
“You change the routines when you work remotely from your home, you need to fix a representable environment, clean more. The boundaries become more fluid.”
“I move around a lot at home depending on the work task, I sit by the desk, on my couch, walk around, which I don’t do if I am at the office.”
“I sat at my kids crafting table and now all my notes are covered in glitter, it is a bit tricky when we are both working from home.”
Task context
“The thought is that everyone should be in this meeting room and everyone will use PowerPoint, but that is not a law of nature, it is just a paradigm that we are stuck in.”
“Everything is so revolved around presentations, not around the interaction itself.”
Temporal context
“Workdays are longer, but it is easier taking breaks.”
“Private time is seriously reduced. If you are on a business trip you go out on this battle and you know once this is over, I go home. In this case it actually intrudes into your private zone, because if you are in China you would have to lock yourself out from your spouse and kids because you have to connect to the meeting.”
“There are time zone constraints, there is only one slot we can use, called the golden slot, which happens in the early afternoon EU time 3 to 5, Stockholm time.”
Social Context
“Meeting people physically before feels like a prerequisite for the digital meetings to work. I have also noticed this internally, when it comes to people you do not know it gets choppy. Usually you plan some type of get together where everyone meets face-to-face, then you notice that it works much better afterwards. Then you can tell on the voice who is talking, you know how they act, you know how they are and that is very telling. If this situation would continue, there is always a natural rotation of people and there will appear people that you don’t know and that don’t know you.”
“You need to get to know the person first in order to build trust. And this should be refreshed every two months or so in order to maintain that relationship.”
“It is more intimate meeting your team digitally when they are at home, you become more private individuals.” “With my colleague in Australia, we decided to grab a tea to catch up. He was sitting inside, then he went outside to show his backyard, there was an alpaca there. No matter what, it is boring to just be sitting by yourself all the time. The social element, it is a risk you know, getting blunted out socially.”
“I am no longer just a voice in a box, no longer an evil thing that just creates problems”.
“People that used to work remotely, were often forgotten, they are not anymore. We used to use photos to not forget about those people.”
“It is working great now that everyone is sitting remotely, people also have a higher acceptance if something goes wrong.”
RESULTS RESULTS PART 2
Human Influence Factors
“The lack of quality means that you cannot distinguish the subtle shifts in position and body language anyway.” “With the hologram you can’t see the mimics of the face, so it is like having a person in front of you, but it is like an avatar, not a real human being. So, I cannot read your facial expressions, mimics or guess what you think or what you said or so, you can’t do that.”
“Through the hologram it felt like you were an AI figure or something very far away. It was not real. You looked like an AI instructor, over there, which you can’t see really, and that felt really distant.”
“When the goal is to agree on something, oftentimes you want to work interactively against a whiteboard or other type of material, here it could probably help and be better.”
System Influence Factors
“What you lose with the holo is the resolution, and that you have the shadows and shapes on the pieces, so you couldn’t really tell which were which.”
“The little square in HoloLens is a pain. It is like having a screen sort of. But that is the difference, also I can decide what is shown in the image and when there is video you can’t. That is an advantage, that you can look around even though you can’t see everything.”
“The glasses are a lot more invasive; you have something on you, and it is not the world's most comfortable thing.”
Context Influence Factors
“I don’t think it works, working together, taking notes, writing something, it doesn’t work unfortunately. There may be, workshops I can imagine are possible to do, but not for my everyday tasks. It should really be a task where you need the depth feeling. More practical tasks would benefit from this.”
APPENDIX 2
Parameter: Amount: Specifications:
Distance to camera Max 2 meters.
Kinect camera 2 Includes the camera, a Kinect USB adapter, and a power supply (110-230V, 50-60Hz) for each camera.
Gaming Laptop (6 core i7) 2 Includes a 200W power supply (110-230V, 50-60Hz) for each laptop.
Camera tripod 2 Holding the Kinect cameras.
HoloLens 2 HoloLens v1 including USB chargers (110-230V, nb with US power recetacles).
Ethernet cables 4 CAT 6 or better.
WiFi routers (AC3200) 2 Includes a power supply (110-230V, 50-60Hz) for each router.
