Beyond the Big Red Button: Science Fiction as a Resource For Generating Novel Interaction Design Concepts For Emergency Situations

(1)

(2)

Abstract

In this thesis I take up the challenge to see whether designers would benefit from using science fiction in order to extend their resources when generating novel interaction design concepts for emergency situations. I discuss the relationship between the nature of fiction and design, and trademarks for emergency situations. I choose four scenes from different media types to analyse, further ideate and evaluate in order to derive final concepts, which I submitted to user testing. I argue that designers may well take science fiction into

(3)

Acknowledgements

I would like to thank Simon Niedenthal for his support, Jonas Löwgren for his inspirational conversation, and my fellow classmates for asking the right questions during my work on this thesis. I would like to show my gratitude towards my colleagues, who were kind enough to be there when needed, and last but not least my wife for giving me the hope and opportunity to submit this work.

(4)

1. Introduction

I love science fiction.

I especially enjoy the ideas science fiction presents about futuristic interfaces and

interactions. This is the reason why, while reading the book Make It So (Shedroff & Noessel, 2013), I was amazed by the massive efforts taken to analyse these interfaces and compile them as a comprehensive list of interaction design lessons.

Impressed as I was, however, a realisation soon hit me: that the potential for design work in this area goes well beyond analytics. Sketching, prototyping, creating scenarios, in short: designing should also constitute a part of how designers research and gain knowledge about novel concepts.

In this thesis, I attempt to fill the aforementioned academic gap by presenting reasons for using science fiction media to generate novel interaction design concepts. There are at least five theories guiding this effort.

First, we have to look at the crossroads of interaction design and science fiction (Schmitz et al., 2008 and Shedroff & Noessel, 2012).

Second, the work of Dunne & Raby (2013) capturing speculative design as a critical catalyst for exploring implications of advancements made in science and technology. Their broad design program focuses mainly on probing how the world could be.

Third, the notion of design fiction (Bleecker, 2009), with its more practical approach for looking at science fiction movies - specifically science fiction and science fact - from a designer’s perspective.

Fourth, designing as a way of acquiring design knowledge (e.g. Gaver, 2012; Stappers, 2007; Löwgren et al., 2013).

And finally, the practical approach of science fiction prototyping (Johnson, 2011) as a way for engineers to prototype their ideas into short science fiction stories.

In spite of these theories, up until now there have not been many academic works focused on researching multiple science fiction media as ways of generating novel interaction design concepts from an interaction designer’s point of view. There is much more to explore for a designer in such a context.

(6)

For example, a science fiction movie, apart from the many futuristic gadgets, flashy effects and interesting storylines involved, is potentially filled with many possible design scenarios. If we approach scenes from science fiction media as design scenarios, we can apply interaction design methods to gain design knowledge from fiction and utilise that knowledge in real-life scenarios.

It is because of this that I decided to take a designerly approach to science fiction media - including films, animation, comics and texts - so that I could determine whether the genre is capable of becoming a design resource to interaction designers in any way.

During the analysis of the research material, it was my objective to select interactions according to a common theme instead of random choice. I ultimately chose the theme of emergency situations as depicted in science fiction as it was present in all four genres of science fiction media I chose to focus on, thus linking the interactions together both academically and thematically.

Upon defining this theme, I narrowed the scope of this thesis to the analysis of emergency interactions in science fiction, the interfaces involved, and potential improvement of the interactions depicted there. I aimed to go beyond the conventional interpretation of emergency situations, the big red button and look for more subtle design knowledge.

Research Question

My research question therefore looks into whether designers can use science fiction media as a resource for generating novel interaction design concepts for emergency situations.

(7)

2. Research Framework

In order to get a better understanding of the research question, we have to cover five relevant areas discussing science fiction and/or design research, and emergency situations. Following a brief introduction of these areas, I will give examples of related works and then address of the overall methodology of this thesis.

2.1. Science Fiction and Interaction Design

There are many ways science fiction and interaction design connect to each other, one of them being from the Human-Computer Interaction (HCI) perspective. The comparative literature in the field of HCI (Schmitz et al., 2008) surveys HCI designs in recent science fiction movies in order to relate it to existing technologies and prototypes in HCI research. The authors’ efforts bring the genre of science fiction and interaction design much closer together by noticing their interrelated influence on each other; however, the main focus of their work is categorisation, and they include only motion picture material.

Another connection point between science fiction and interaction design is presented by Shedroff and Noessel (2012). They reach out to science fiction movies and TV shows from a more practical point of view, investigating how interfaces in science fiction movies can inform interaction design in the real world. In their research, they examine and analyse interfaces and interactions in detail to extract valid design knowledge from them.

Interestingly, Shedroff and Noessel also chose to exclusively examine motion picture material, excluding the animation, comics or textual genres as they argued that these genres do not satisfy formal criteria for analytics (i.e. audiences cannot see and hear it over a specific time period). They approach their chosen source material in four different ways: (1) Bottom-up, where a specific user interface (UI) is analysed in detail; (2) Top-down, putting tags on shows; (3) Looking for similarities and emerging themes; and (4) Apologetics - in other words, ways to forgive an interface, and how it could work as shown (according to its authorial intent). The authors later conclude that this last approach is the most fruitful for gaining valuable

interaction design insights.

These authors bring interaction design and science fiction closer together by stating the obvious: that the relationship between the two fields is unequal. Science fiction is not bound by practical constraints (like feasibility, viability, usability etc.), there is no clutter - similarly to a technological company’s vision of the future (e.g. Apple’s Knowledge Navigator, 1987 or Microsoft’s Productivity Future Vision, 2011). Altogether, their recognition of the genre setting expectations for audiences about what interfaces they would believe to be useful is a

(8)

As Löwgren et al (2013) suggest, “interaction design is about exploring possible futures”; in other words, science fiction and the speculative aspect of interaction design can thus be seen as closely related to one another.

2.2. Speculative Design

This speculative aspect of the design process is addressed by the academic approach of speculative fiction as presented by Dunne and Raby (2013; Dunne, 1999) and to a lesser extent by Auger (2013).

Dunne and Raby (2013) expressed an interest in using possible futures as tools to better understand the present, and to discuss the kind of future people do and do not want. This is their primary reason for holding speculative design to be an inspiring way of sparking debate and discussion. According to their conclusions, (fictional) future scenarios should always be feasible in terms of scientific probability and the foreseeable route leading to it. In their research, they also introduce the notion of functional fiction as a way to “pull technological developments into imaginary but believable everyday situations so that we can explore possible consequences before they happen” (ibid p. 57).

Why it is worthwhile to draw inspiration from fiction can be understood better if we look at Dunne and Raby’s diagram, which is in turn inspired by Stuart Candy’s presentation, where we can see Probable, Preferable, Plausible and Possible futures (Figure 0). The focus of this thesis falls mostly under or out of the Possible category, i.e. that which may extend a

designer’s repertoire. Moreover, looking at possibilities outside of a Probable scope, in which most academic and practical designerly work is done (Dunne & Raby, 2013:3), may well hold promising insights to this project.

Similar to Dunne and Raby’s notion of believability, the importance relating to every detail and aspect of the fictional prototype appears in Auger’s work (2013) as well. He created the project Audio Tooth Implant as such an example together with Loizeau in 2001. Together the two researchers explored the implications of blending the fictional to the reality as much as possible, creating discussion around it in that way.

(9)

Figure 0. Preferable, Probable, Plausible and Possible futures (Dunne & Raby, 2013).

Overall, both of these research pairs are primarily concerned with critical reflection, opening for debate and discussion, provocation and using rhetoric to present their notions.

2.3. Design Fiction

Slightly less critical and academically ambitious, the theory of design fiction (Bleecker, 2009) investigates the joint continuum of design, science fact and science fiction. Bleecker

researches how science fiction can be a “purposeful, deliberate, direct participant in the practices of science fact” (p.6). He sees it as a way of probing, sketching and exploring ideas, a method for “imagining and materializing future habitable worlds” (p.86).

In addition to Bleecker, Sterling (2009) also acknowledges design fiction, but sees it as the notion of where science fiction thinking opens itself to design thinking. Possessing science fiction writing skills himself, he defines it as the “deliberate use of diegetic prototypes to suspend disbelief about change” (2013); in other words, creating a conceptual place, supported by (diegetic) prototypes, which has a defined purpose.

We can conclude that both these efforts have science fiction in focus specifically, and both present their findings from a designer’s point of view, which make them very relevant to the current thesis.

(10)

2.4. Design Research

The notion of reflection in action (Schön, 1987), together with that of “seeing-as” as a way of using existing repertoire when carrying out design processes, is no novelty for researchers, as is involving practical experiments along the way to produce valid design knowledge.

Research through design (Zimmerman et al., 2007; Gaver, 2012) argues that valid design knowledge is produced while practising design. The act of creating prototypes is the core of this practice, where “through realising ‘products’, designers absorb knowledge from different directions and confront, integrate and contextualise this knowledge” (Stappers, 2007:84). As pointed out by Gaver (2012), artefacts are the holders of design knowledge, bearing implicit theories in themselves.

However, there exists a fluid space between abstraction and artefacts for intermediate knowledge (Löwgren & Höök, 2012), where - amongst many others methods and tools - guidelines and heuristics, experiential qualities, and concepts may all be found.

To sum up, there is a definite probability for gaining design knowledge by applying design methods, such as sketching and prototyping.

2.5. Emergency Management and Safety-Critical Systems

As mentioned before, the common theme for all interactions analysed in this thesis is that of emergency situations in science fiction, which is why the following design implications as suggested by Carver & Turoff (2007) prove relevant as a shortcut in the vast field of design criteria and safety-critical systems. Although covering the whole breadth of safety-critical systems is not in the scope of the thesis, an essential clarification is presented here.

Safety critical systems are software programs whose failure may result in damage or loss of human lives, equipment or fiscal resources, or harming the environment severely (Bowen & Stavridou, 1993; Knight, 2002; Haider & Nadeem, 2013). Well known applications exist in the areas of aviation, medical devices, ammunition and nuclear systems for example. However, these papers are concerned mainly with error prone software engineering (e.g. Bowen & Stavridou, 1993; Knight, 2002), which has a foundational but limited contribution to the thesis.

We can find a survey of design suggestions for emergency management at Carver & Turoff (2007), with a more abstract perspective, concerned mainly with communication and human factors, which can be more relevant to this thesis. According to the researchers, in emergency situations people would need to, for example, receive key information rapidly, make fast decisions based on this information and even have the option to occasionally improvise (Figure 1). They filter the available literature to identify key properties in emergency

(11)

of being in control or high motivation. They find several challenges for the design of such systems, such as:

● “Providing ways to obtain accurate and timely perceptions of reality through communication structures that track and facilitate open exchange of information, including feedback from the incident site from both trained observers and victims. ● Designing to enhance the ability to focus attention without interruption, and to

require a minimum of effort to carry out a task.

● Designing to encourage or facilitate creativity and improvisation on both an individual and team basis.

● Providing mechanisms to support building trust between the individuals in the team, many of whom have not worked together in the past.

● Being able to anticipate and predict when more relevant information will be obtained” (pp. 34-35).

Figure 1. Each phase of the emergency management process (Carver & Turoff, 2007 p. 35).

Carver & Turoff (2007) also summarise three interface design tools being developed when they touch upon HCI.

The first tool is information prioritization, as of what rules are used to prioritize the situational information and whether the sender or the user defines them or managed in a dynamic way depending on the context and time of use.

(12)

The second such tool, decision support and modelling tools, in order to help to make the decision and providing support once the decision has been made.

Third, another tool is representation of a common operating picture, in terms of interactive visualizations of the current situation and resources.

According to the researchers, coordination, collaboration, communication, information overload, networks and automated systems are also parts in emergency management systems that one should consider and plan. They are all major areas worth a separate thesis each; therefore I mention only a reference to them here.

It should be also mentioned that human factors in safety-critical systems are also a huge field that is elaborated by many researchers. For example, it is discussed in a general introductory level from many perspectives (Redmill & Rajan, 1997), from risk analysis (Cacciabue, 2000) and humans as a source of error point of view (Chambers, 2005).

As my design focus was centred on emergency situations, management, human factors and safety-critical systems, the above aspects are especially relevant to this thesis.

Having set up the research framework within Science Fiction and Interaction Design, Design Fiction, Speculative Design, Design Research and Emergency Situations, I will now go on to mention some of the related works in these fields.

2.6. Related Works

The analytical work of Shedroff and Noessel (2012), albeit already introduced above, is undeniably the most relevant to this research topic with its exhaustive focus on interactions within science fiction media. Their primary way of extracting design lessons is from the analysis of interfaces and the relevant interactions, limiting themselves however to the genre of motion picture material.

Another relevant work is the research around the concept of science fiction prototyping by Johnson (2011). Aiming to help engineers imagine future technologies and explore the relationship between society and technology, Johnson proposes a methodology of creating science fiction short stories as a way of sharing this knowledge. These narratives are supposed to help construct imaginary future scenarios with the implications and possible consequences of an actual technology.

Unfortunately, the value of Johnson’s knowledge contribution is problematic to define. He suggests creating a story around a technological implication or science fact, however, after reading such stories, the quality of them is difficult to perceive. (This is mainly due to the fact that storytelling is its own art form, requiring a deeper understanding and knowledge, and as a professional from another field it is fairly difficult to deliver captivating original stories).

(13)

Nevertheless, the effort is greatly appreciated as an example for a path not really worth taking.

Bleecker presents one example for generating knowledge based on science fiction

(knowledge) in the form of a design work kit (Design Fiction Product Design Work Kit 0-TBD-D012, Figure 2). According to the game, designers can define the core elements of a fictional product, to a fictional design brief, with the cards helping to define what the product would actually be.

Figure 2. The Design Fiction Product Design Work Kit 0-TBD-D012 (Near Future Laboratory, 2012)

2.7. Research Method

In order to find out whether we can use science fiction media as a resource for generating novel interaction design concepts for emergency situations, we have to look at a variety of source material first. This is why I chose to examine different science fiction media types or genres, in order to see what differences exist between them and what concepts could be generated based on them. These four media types are: text, comics, animation, and motion picture.

After defining the scope of this thesis as illustrated above, I selected scenes from these aforementioned media types and generated new interaction concepts based on them. As the purpose of the thesis is to look at science fiction as a resource and to answer the question of whether it is possible to generate novel interaction design concepts from them, including extensive user testing in the research process did not seem vitally necessary. However, some user tests were carried out asynchronously in order to see what different people could conclude from the final concepts, and to look at their general validity.

(14)

The process of working with a source material followed the pattern below: I. Select a scenario from the chosen medium

II. Analyse the scenario • Action breakdown • User Goal

• Analysis III. Prototype the scenario

• Bodystorm insights (Experience prototype) IV. Ideate around the prototypes to realise many alternatives V. Present final concept

• Evaluation • User test • Use cases

For “Analyse the scenario”, I used a methodology inspired by Shedroff & Noessel (n.d.), more specifically number 5 and 6 were taken into consideration. While “Action breakdown” and “User goal” are based on the prior, and “Analysis” can be loosely associated with the latter.

5. “Provide a description of the interaction.

a. Start by providing more detail about what happens than you had room for in the title. b. Include the following information in the description. The description should be narrative,

not a bullet list, but be sure to provide this information in that narrative.

i. The name of the user and position or title if relevant to the interaction (like General Dodonna)

ii. The character’s goals in using the interface, or at least the task they are trying to accomplish.

iii. The actions they take during use.

iv. What they perceive during use to understand what’s going on v. The results of use.

6. Now (finally) do a little analysis. This is more free-form than the prior steps, but here are some questions to consider as you write your thoughts. Consider every level of the design: microinteractions, tasks/workflow, user goals, and broad social consequences of the design.

a. Does it make sense? If not, what’s broken about it? Do the users have what they need to understand what’s going on? Do they have the right controls to tell what changes they want?

b. How would it would it work in the real world?

c. Does it fit with the rest of what I know about the show’s diegesis (how things work in their reality, not ours)? Does it conflict?

d. What metaphors does it build on? What mood does it set?” (Extract from appendix i) For my prototyping analysis, I chose a method similar to design improvisations (Oulasvirta et al., 2003) and experience prototyping (Fulton Suri & Buchenau, 2000), in order to have a better understanding of the user’s point of view.

(15)

The final concepts were then defined based on the ideation, and evaluated according to real life criteria (Carver & Turoff, 2007), with points and brief explanations given to each of them. The exact criteria were derived from their findings listed above, broken into logical pieces and what they generally mentioned in their paper about human factors in such situations (see below). I added a basic point system, to gain an overall picture how would the concept, as such, function as a system. The points were calculated in a scale of 10 as the following: 1-3: Poor, 4-6: Sufficient, 7-9: Good and 10: Excellent. The thirteen criteria were the following:

1. Obtaining information rapidly

2. Deciding quickly based on this information 3. Having space for improvisation

4. Providing the feeling of being in control 5. Obtaining accurate information

6. Obtaining timely perceptions of reality

7. Tracking and facilitating open exchange of information

8. Providing information from the incident site from both trained observers and victims 9. Enhancing the ability to focus attention without interruption

10. Requiring a minimum of effort to carry out a task

11. Encouraging or facilitating creativity and improvisation on both an individual and team basis

12. Supporting and building trust between individuals unknown to each other in the team 13. Support for being able to anticipate and predict when more relevant information will

be obtained

User testing were carried out with functional prototypes, presenting the concepts through these mid-fi artefacts and leaving the most of the discussions to imagination as of where they could see their potential. The reason behind this choice was to get feedback on validity and potential of the concepts and their interactions in an informal workshop setting. Both user tests were recorded, transcribed and analysed in order to gain reasons for the validity and potential of the concepts. I presented these concepts in two rounds, first a one-to-one interview with one person, and at the second round of user testing I discussed the concepts with a group of people.

I conclude the discussion by mentioning brief use cases for each concept, to provide insights into what environments they could be best utilised in, without detailing them exhaustively. Furthermore, I reflect upon the novelty factor of each concept.

(16)

3. Design Process

In order to obtain a satisfying answer to whether we can use science fiction for generating novel interaction design concepts, we must first find the material we are going to work with and detail the theme chosen in this thesis. By doing this can we start the process of

generating new concepts based on the four media types.

3.1. Defining Source Material And Theme

As noted also by Shedroff and Noessel (2012), defining the source material for such a project presents several challenges, as the genre is fairly big and constantly growing. Moreover, having different media types within the overall science fiction genre made it even harder to tackle the problem.

I examined many films, animation, comics and short texts while trying to define my exact source material; luckily, I had prior information and received some guidance about which could be potentially relevant. As stated earlier, I ultimately decided to filter the vast amount of available literature according to a single theme, so that the examples and prototypes I analyse would contain pre-existing similarities, which connect them academically and thematically. One significant challenge presented by this extensive genre was to find relevant source material that was suitable for creating prototypes. At a certain point during my research, I considered analysing media types based on one another, e.g. movies based on certain texts, so that I could prototype the same interaction found in both media types and then compare and contrast the results. However, owing to the shortage of material that I was either already familiar with or which has a reasonably short text base; I ended up abandoning this approach. I then started examining science fiction from the perspective of augmenting the human body, so that I could group the material accordingly. Initially, I also considered building several small prototypes based on the scenarios sourced in the material. However, it quickly became apparent that such a project would require resources far beyond the scope of the current thesis, so I had to let go of this approach as well.

This was the road that finally led me to my chosen theme of emergency situations, with a mindset of creating lower fidelity prototypes based on the sourced scenarios and then

developing them into final concepts. This theme turned out to be more rewarding, in terms of almost every science fiction story having a climax, i.e. a point where there is a major conflict or life-threatening situation, thus providing a linked base for analysis. Moreover, the theme of emergency served as a narrative to the discussion in the thesis. These situations also

possessed other common identifiers such as time pressure, potential risks, the need to find a way out etc. which all carry the potential for comparative analysis and prototyping.

(17)

Another specifically problematic aspect of conducting research within such an extensive genre as science fiction is that of labelling. Some motion pictures labelled as science fiction, or even based on science fiction texts, often have another dominant themes or topics, and despite engaging storylines, possess no or merely a couple of interactions or interfaces worthy of analysis (e.g.: A Scanner Darkly (2006), based on Philip K. Dick’s novel of the same title (1977)).

The process of finding relevant source material to this project took longer than expected, especially in case of the literary fiction/text media type. This was due to both my sketchy background knowledge and the time-consuming nature of selecting material from the vast genre of science fiction that possess prototypable interactions fitting the focus of the thesis. For my analysis and prototyping ventures, I chose a scene from each science fiction media type which was created in the recent past; I believe that it was necessary to select recent works in order to possess the right references and larger context as a user. In case of analysing media created much earlier, too much time would have been spent with reasoning about why and how certain objects or interactions were related to the times when they were fictionalized. Finally, yet another reason for choosing scenes from different media types was to see if there were any differences in the process of building prototypes and the details of the resulting final concepts.

I treated the selected scenes as design scenarios for design concepts, which need further elaboration. It is important to declare an obvious fact: that any interaction within a fiction serves a narrative. There would never be a “usable” or quick tool if there should be more tension according to the story, and this is especially valid in emergency situations.

Nevertheless, I examined the scenes as “real” scenarios, as in viewing them each as a separate design proposal. This is why I analysed them critically and evaluated them based on academic standards and criteria.

It should also be acknowledged that the genre of motion pictures is more performative in nature than that of texts. I do not treat each medium differently, however, this aspect is worth keeping in mind, to see what complications might appear because of it.

At the end of each iteration, I derived a novel interaction design concept based on the given media. As the selected scenes all had the common theme of emergency situations, it was foreseeable that my explorations would have several similarities and thus simplify the overall design process.

(18)

Taking all reasons mentioned above into consideration, I finally chose the following pieces of media:

1. Motion picture: Star Trek by Paramount Pictures (2009, action, adventure) 2. Animation: Despicable Me 2 by Universal Pictures (2013, adventure, comedy) 3. Comics: The Wake by Scott Snyder and Sean Murphy (2013, horror)

4. Text: Accelerando by Charles Stross (2004, drama)

We have seen the consequences of choosing several science fiction media types and

concluded to having a theme for the investigation in this thesis. In the following, I will discuss the selected scenes from each of these media types in more detail.

3.2. Generating Novel Interactions For Emergencies Based on

Science Fiction Media

There are four science fiction media types that I discuss in this chapter: motion pictures, animation, comics and text. The scenarios are viewed consistently as (real life) design scenarios, starting with a critical analysis and understanding of what is happening in them, then followed by insights from prototyping, ideating and presenting a final concept.

3.2.1. Emergency Scenario From Motion Picture: “Damage Report”

3.2.1.1. Synopsis

The USS Kelvin spaceship is under attack, receiving considerable damage from her attacker. At the deck, the captain of the ship requests a damage report from the engine room, where a crewmember reports back to him.

(19)

3.2.1.2. Scene Analysis What happens in the scene?

The captain initiates communication with engineering personnel, (supposedly) by pressing a button and requesting the damage report. We see the responding crewmember running to a terminal and reporting the status of the ship while he is there. He maintains communication with the captain through talking into the Communicator, a handheld device unique to the Star Trek series (see Star Trek original series, 1966-69) while looking and interacting with a screen via a keyboard.

Action breakdown:

1. Captain requests communication via an input button (not visible, only audio feedback); after the feedback, he requests a damage report.

2. Crew member responds via the Communicator, which we first hear from the captain’s location, then we see him making his way to a terminal.

3. At the terminal, the crew member reports the availability of the weapons system and what the remaining main power is. In the meanwhile, he starts to type on the

keyboard. User Goal

Acquire knowledge of the spaceship’s current damage level. Analysis

In real-life emergency situations, communication is short and purely informative. This is represented adequately in the scene.

As part of the legacy of Star Trek, the Communicator is used from the personnel’s point of view. The question is, then, how it can be productive in emergency situations, as it blocks one hand, thus making it more difficult to navigate in an emergency situation? Unless in case of an emergency, the communicator would already have information about the data in question, so if the terminal is blocked the data would still be accessible.

Only the captain has a fixed communicator, which requires him to be in one position at all times if he wishes to initiate and maintain communication. All officers in the control room have headsets at the USS Kelvin only (Figure 4). This would make sense if, for example, it would establish a priority line directly to a specific personnel, which would override every communication at the time for both parties.

It is also worth mentioning that every time the captain wishes to communicate, he interacts with the keypad in his chair, and every function has its distinct audio feedback.

(20)

These pieces of information raise the question of whether the information in the control room could be in any way accessible to the captain. That way, there is no requirement to call any external sites for information in the first place, thus saving valuable time. The only qualitative data the crew member provides is that he has never seen anything like the current

happenings. In real life, this is more likely to happen as well - to have this information as near to the commanding officer as it can be.

Also on this note, and probably because of similar franchise legacy reasons, there are some other scenes in the motion picture with inconsistent usage of communication ways. For example, officers in the command room in the USS Kelvin use headsets, while the captain has no means of communication via devices or otherwise. Later on, on USS Enterprise, there are no headsets at all.

Communication transcript Captain: “Damage report.”

Personnel: “Warp drive has been knocked down. Never seen anything like it. Weapons offline, main power 38%”

(21)

3.2.1.3. Prototyping

Figure 5. Bodystorming the scene with common devices.

I chose to conduct design improvisation (Oulasvirta et al., 2003) sessions after watching the movie clip few times. My insights were the following:

● The phone (i.e. “Communicator”) is in the way if I have to type and communicate at the same time.

● I could drop the phone on the way: insecure communication.

● In case of emergency, all information should already be accessible as soon as possible, I should not have to look for it (call or type).

● I knew where to run, but what happens if the path is blocked? Would I not know where to go, would I be disoriented in any way?

● It is physically exhausting to jump around and run to terminal: maintaining communication is difficult because lack of breathe.

● How objects are situated in the surrounding area and how people are feeling nearby has a great effect on problem resolution.

● If I am an expert of the specific area (engineering room), I can probably evaluate the situation by only looking around.

(22)

3.2.1.4. Alternatives to the scene Alternatives could be, for example

● If the screen would constantly display all status information necessary in such situations.

● If the captain would already see this information at hand.

● Having the communicator as a special device that transmits other information as well. ● The terminal would give access to some extra information.

Figure 6. Verbal prompt from captain, response from crew members next to him

(23)

As a preliminary question, I asked why it is necessary to make contact outside of the control room. According to common sense, this information should be available immediately, so that the user would have to expend as little effort as possible to receive these details. There could be at least two different scenarios.

One solution (Figure 6) could be that the captain asks his officers around him for the necessary information, as this detail should be accessible for them at all times.

Another solution (Figure 7) could be that in case of emergency, these details would be visible within the control room, so that everybody inside could easily assess the current status of the ship.

3.2.1.5. Final concept

I chose the notion of automatic interface change upon emergency (as seen as in Figure 7). This would mean that the state of the UI would change when there is emergency detected and the most important information would be visible for the users on a display. Alternatively, there could be a spare display that becomes visible only upon emergency situations.

Emergency situations are determined based on sensor readings. For example, if a ship is attacked on sea waters, then factors could be other approaching vessels around the ship (pirates), amount of water within the ship (heavy raining, sinking) or whether fire hoses were activated. This would allow any crew member (or even passengers) to deal with a potential emergency situation better, if there are digital user interfaces available for helping deal with it.

(24)

Evaluation

Emergency Criteria Points Text

Obtaining information rapidly 10

As the information appears when there is an apparent emergency situation, receiving information should be effortless.

Deciding quickly based on this information 9

The opportunity is facilitated by the concept for a decision. However, it must be noted that making an informed decision is solely

dependent on whether the information shown is relevant to the emergency at hand.

Having space for improvisation 3

With such a visual interface, improvisation is limited as it is a one-way information provider. The given information might provide space for improvisation, acting as a base for future interactions.

Providing the feeling of being in control 7

Having the information without asking provides a way of being in control when an emergency situation happens. However, because the system is fully automatic, some control has been already taken away from the user.

Obtaining accurate information 9

With (only) important information visible, there is a solid chance that it is accurate. However, it remains a chance, not a fact, depending on the relevance of information, as noted before. Obtaining timely perceptions of reality 10

As one of its core functionalities, this is the strongest point of the concept.

Tracking and facilitating open exchange of

information 4 The system tracks and facilitates information exchange in a one-way manner Providing information from the incident site

from both trained observers and victims 0

There is no feedback from people, only automated, so this criteria is not applicable to the concept.

Enhancing the ability to focus attention

without interruption 7

Due to the fact that the UI changes to better facilitate an emergency situation, there is less chance that the user would get distracted. However, the interface merely provides information, so the user's attention has to be interrupted at the point of taking action based on the received information.

Requiring a minimum of effort to carry out a

task 9

If the task is to get informed about the emergency, then this score would be a 10. However, this is only the starting point, when

(25)

the user can decide upon a following chain of actions. Therefore it makes the initial states of the emergency more fluent without the concern of the latter happenings. Encouraging or facilitating creativity and

improvisation on both an individual and

team basis 3

It does nothing more than provide information rapidly which may or may not facilitate creative usage (decision) based on the information. Supporting and building trust between

individuals unknown to each other in the

team 5 The sense of trust might be supported by the common knowledge of the current status.

Support for being able to anticipate and predict when more relevant information will

be obtained 9

As the UI updates itself automatically when there is more updated information available, the system notifies the user on relevant information. It is up to the UI to visualize more relevant information and how to implement it, the possibility is given.

TOTAL 7.1

As we can see from the evaluation, the main strength of this concept is providing real-time information from the site to people with decision-making power. This would also result in making decisions much more quickly, which would be beneficial for an ultimate resolution. However, there are still some pain points within the concept, particularly in the areas of having space for improvisation, providing (qualitative) information from people on site and encouraging creativity and spontaneity.

Additionally, this concept may become exponentially more valuable if the original UIs in question were considerably sophisticated/complex, with a fairly big number of options to choose from. This way the emergency interface would be simplified, to offer only

fundamentally necessary actions. Although this notion may be counter-intuitive, as users have to be already aware of (i.e. have to have learned) the nuances of the original interface, otherwise they would not be able to handle it.

Regarding the novelty of this concept, its very existence is quite questionable. There is very small fraction about it that may prove new, for example the contextual change; however that could also be argued otherwise. If we take a simple case of popups or alert boxes, the concept seems quite trivial without any strong signs of novelty. It could be argued that this may be thanks to the fact that movies, as a genre have more closure (McCloud, 2008).

(26)

User test

First, the majority of users tried to make sense of the UI, both the graphical layout and the interaction that may trigger the change. Most of the times they asked if it is a machine that they might already be familiar with (using it for regular work).

“I think that there is a big difference if you have to be able to work in it and trigger the emergency that way. It reminds me of the boss key for games…”

From the user test, it was apparent that people saw notifications as a main strength for the concept.

“I see this as a kind of a notifying system. You practically see it in all the movies, like ‘Security breach’ or ‘Access denied’. You also see a big red popup sign saying this in the middle of the screen. That is just so annoying, if you think of it.”

Users found this notion of notifications puzzling when trying to solve the situation at hand. “It would be nice to have less text on the screen and more pictograms. For example, when there is an emergency in films, you see biohazard on every screen. There could be a burglar icon when there is a robbery [laugh].”

However, all users specifically mentioned that getting useful notifications/information is of utmost importance.

“The system should tell me what to do and where the emergency happened. It’s annoying when the interface just changes without me doing anything. I just don’t expect it [...] I don’t feel in control. The sound is just annoying. I want useful information.” [sic]

They saw a potential for offering options to act upon.

“For example, if I’m an IT manager and my site was hacked, then probably I want to be notified by this. And not only that, but I also want to have actions, like tracking down, filtering, you know, something relevant.”

Another potential risk identified was that if this information is visible for all users, then probably they would all try to solve the situation at hand, which could lead to complications. Users preferred to have a manual switch or a way to turn the alarm off.

Use cases

Based on the qualities of the concept and the results of the user test, this concept could be utilised at the following areas:

(27)

Massive compounds. As people would get this feedback no matter where they are in the

building, it would be beneficial for all personnel to receive notifications about the situation. The concept would probably have to be adapted in a way that only people with acting capabilities receive this notification, so that they can react to the given scenario and no one else without acting power would do something faulty.

Soundproof places. Due to the fact that these places omit sound, there is high potential for

visual notification in an emergency situation. Soundproof, on the other hand, could mean many areas including any control rooms that are separated from the actual site (see nuclear power plants), which were designed in this manner from the start (see after evaluation).

Deaf community areas. In places where people would have some form of audial disability, it

would be ideal to have visual notifications at hand - especially when users have to act upon the given circumstances. However, it is a question of how novel this design could be in this community as they already have their own mechanisms to deal with such situations.

Nuclear plants. In areas where the emergency depends on many variables, and to support

decision-making under pressure this concept could suffice. Supporting decision-making on the other hand is something that has its own subfield within safety-critical systems (e.g. Endsley, 1995; Naderpour, 2014) thus needs much more elaboration than this space allows

(28)

3.2.2. Scenario From Animation: “Escape From Emergency Situation”

A rogue minion attacks Margo and Agnes in their own house. The girls run to the control panel on the chair and press a button to activate the elevator to the secret underground facility, in order to lose their attacker.

3.2.2.2. Scene Analysis

Margo only punches a big red button on the chair that lights up, and a little screen displays a Gru logo with up and down arrows as well. She doesn’t look at the button after she presses it, and the interaction looks simple - she punches it with her fist and activates it that way. Even though the artefact has three other buttons, it seems that she triggers the elevator

successfully. The elevator takes some time to initialize, but they can successfully escape (for the time being).

User goal

Escape from an attacking intruder. Actions breakdown

1. Margo pushes the button on the chair.

2. The elevator prepares itself for them to step inside.

(29)

Analysis

This interaction is seemingly very simple, and quite intuitive, something that we can imagine doing in real-life cases of danger. There are, however, some interesting points raised by this scene.

One question could be how the control panel detects which button was used. It is possible to imagine that if somebody were in an emergency, we would want the system to detect that if the big red button is pressed - no matter how many other buttons are touched - then the (emergency) sequence will start up.

Another question could be why the elevator initiates so slowly. In a real-life situation like this, users would probably expect the necessary element (i.e. elevator) to be ready within a very short time period. This is obviously not a real concern for any motion picture/animation. In real life, there could be many possible scenarios, mostly without the built-in elevator or angry minions. However, this presents a really good opportunity to investigate children’s options in responding to a similar scenario without any adult intervention. This in itself could be its own topic for another type of academic research.

3.2.2.3. Prototyping

Figure 10a. Bodystorming. Figure 10b. Rapid prototype of the device. Insights from bodystorming were the following:

● It makes a big difference if there is another person else with the initiator (i.e. dragging somebody with me), because the other person has to be taken into consideration while running (e.g. keeping the pace).

(30)

● I was not really looking for anything else than the big button; so presupposition that some text or sign on the button that signifies what it is for might help the interaction proved to be false.

● I pressed at least one other smaller button on the panel while pressing the big one. ● In the scenario, the path to chair was clear, but what if it would not be? In real life,

what other options would there be?

● Small buttons flied everywhere after I punched the control panel/keypad

● After pressing the button, I wanted immediate action to happen, to be in a safe place (evacuate)

● In the animation two children used it; what if there are two adults, or even more? Would it be even more difficult to handle the situation?

3.2.2.4. Alternatives to the scene

As seen from the sketches in Figure 11, I was thinking about where (other) possibilities could arise for triggering emergency protocol (escape). I had several reasons for this: for example, if the intruder/intrusion itself could be spotted sooner, then people could escape earlier as well, which could be beneficial in an emergency situation. There are three scenarios covered in the sketch below.

Figure 11a. Different possibilities for triggering emergency protocol

(31)

First, the emergency (escape trigger) button is much bigger than the rest, so there is minimal chance for triggering a different action. However, what would be different from the big red emergency “shutdown” button we are already familiar with is the notion of sensory

(biometric) data feeding into a central system.

Second, a sensor in the window would trigger an escape mechanism or alarm system when broken from outside. This raises the question about if or when accidents occur, how they would differentiate that instance.

Third, external cameras are monitoring the outer parts of the house, so when a suspicious pattern is spotted (by an AI algorithm), then it would alarm the people inside to be aware of the situation.

The concept I chose is that several/all buttons pressed would trigger an emergency protocol. In case of the chosen scene above, it should not matter if Margo presses all the buttons, once she pushed the emergency button. Ideally, this would be helpful for beginners or users without prior knowledge of the interface in order to use it without any background knowledge.

There could be at least two ways of initiating emergency. (1) If there is a dedicated button for triggering, then it does not matter if surrounding buttons are pressed too; (2) If more than a certain number of keys (e.g. 6-8) are pressed then system triggers emergency protocol. Evaluation

The system is not designed for obtaining information, but for execution. With this in mind, users can receive the information about the systems themselves in a fast manner.

Deciding quickly based on this information 3

Keeping in mind that providing information is not within the scope of this concept, users can still decide quickly. There might be

complications because users might not be aware of the designed functionality.

The design of the system provides limited space for improvisation, in the form of having multiple ways to trigger the same effect.

(32)

In case of emergency, the system could let the user feel in control; however, when not in an emergency situation, the protocol can be triggered accidentally (unless another dependency is introduced to it).

If by information we mean the usage of the system (buttons), then the system provides sufficiently accurate information.

Obtaining timely perceptions of reality 0 As there is no feedback from the system, this criteria is not applicable. Tracking and facilitating open exchange of

information 0 As there is no information exchange in the system, this criteria is not applicable. Providing information from the incident site

from both trained observers and victims 0 Like before: as there is no information exchange in the system, this criteria is not applicable. Enhancing the ability to focus attention

without interruption 6 There is sufficient potential in the solution for the user's attention not to be interrupted. Requiring a minimum of effort to carry out a

task 8 There is very minimal effort to push the button(s). Encouraging or facilitating creativity and

team basis 3 There is a limited space for improvisation, with multiple ways to trigger emergency protocol. Supporting and building trust between

individuals unknown to each other in the

team 2

All users possess the same information in seeing what happens after one button is pushed.

Support for being able to anticipate and predict when more relevant information will

be obtained 0 As there is no feature of information retrieval in the system, this criterion is not applicable.

TOTAL 3.3

As seen from the evaluation, this concept does not really qualify as a sufficient emergency system, mainly due to the fact that it is designed for one simple execution and around the idea of inaccurate user input. However, it has its merits of being simple and somewhat problem-free.

(33)

Although the concept as a system received the lowest points altogether, the part of the concept where pressing any button including the emergency button and still initiating only the emergency protocol can be seen as a novel concept. It raises some questions and poses implications. For example, in what situations would it be tolerable to trigger, what functions do the nearby buttons do in what environment we are in a who would trigger it (e.g. children, elderly people).

User tests

From the user tests, it was apparent that people did not see the interaction as intuitive, as they usually do not use keyboards/keypads for slamming.

“If I work there then probably I have the knowledge already, but otherwise I wouldn’t know how to do that [trigger emergency].”

“When I am playing and get really pissed off by other people or cannot pass a level, then I punch the keyboard anyway.”

Users started to experiment with popular keys (such as space and return) and other key combinations generally. They viewed the interface as easy-to-use, in both positive and negative ways and some users had differing opinions about it.

Mikael: “I like it that it’s so easy to start the emergency, but it probably would lead to a lot of false alarms. People would misuse it. I saw that happening when I worked at my previous workplace. You would have to call in each time when there would be an alarm. ” Tobias: “I would actually think that it’s not a false alarm. Because it was manually triggered [...].”

One user saw a potential for using it for “immeasurable things” because of the human trigger aspect. They also felt in charge of the action and satisfied with the immediate feedback from it.

Users identified potential weak points about the concept, centred on the triggering method. “I would have to know about it before I can use it, right? There is no way I could tell that this is how it should be used.”

“The problem with this, is that anybody can trigger it. Also, I would have to be close by to it to get it working. Are the buttons close by or far apart? I can see this happening only when they’re close.”

There was also a discussion if the input device would not be a computer’s keyboard (as in the prototype).

(34)

“If there would be only buttons, then I would sequentially press them and then slam it, unless there is a big red button.”

One user also noted that they would have to be awake in order to trigger the alarm. Use cases

People with special needs. It would be more optimal for sensitive users, such as children,

elderly or mentally challenged people to operate such a concept owing to its ease of use and no prior knowledge of the functionality itself.

Non life-threatening situations. Users could hide their gaming window while at work (i.e.

“boss key”), express their frustration in gaming. User tests showed signs that it could also be used to abort a cash withdrawal at a bank terminal for example.

(35)

3.2.3. Scenario From Comic: “Closing an underwater door from within a carrier”

Figure 12. Closing an underwater door from a mini submarine (The Wake #4, 2014; 2013-2014 © Vertigo Comics) 3.2.3.1. Synopsis

A group of divers are attacked in deep water by alien creatures, so they decide to swim back to the underwater marine base. They ask Mel in the accompanying small submarine to close the door behind them before their attackers can get in. Mel closes the door from within the boat’s control booth so they can escape from the attackers.

3.2.3.2. Scene Analysis User Goal

Closing an underwater door. Actions

1. Mel types something on the control surface of the boat. 2. Mel moves a handle inside of a mini submarine.

(36)

Analysis

In an emergency situation, users should be able to accomplish their goal as fast and efficiently as possible due to the time pressure and the complications of the situation. For example, doing two different actions might not be an ideal scenario, unless there is absolutely no way to close the door otherwise. The question still remains: why is an additional interaction (i.e. handle) necessary?

The notion of remote control is one worth highlighting. It introduces many potential questions, such as where it would make sense to place remote controls, how it could be implemented in a way that is not mistaken by e.g. the local (i.e. submarine) emergency triggers, or the possibility of causing deliberate emergencies from off site. However, resolving the emergency situation remotely could help people who are not able to trigger the

emergency on site manually.

Diegetically, the interactions are coherent: we can find control panels granting access to opening and closing any doors in the underwater base throughout the comics.

In a real-life scenario, users would probably have to close the gate in an emergency situation manually, with no wireless access to underwater controls, at least not from a moving

submarine. It also raises the question whether it is a sound concept to place complex electronic devices underwater.

Prototyping

Figure 13a,b. Bodystorming steps required for closing the underwater door Insights from the bodystorming:

● The need for using two different devices felt too complicated and demanding in case of an emergency.

(37)

● Handle had a satisfying embodied experience while using.

● I pushed the handle too hard, it might break if pushed too sudden - not too suitable for emergency situations

3.2.3.3. Alternatives to the scene

Less interaction needed to close door. E.g.: push a button and the door closes. This button could also be near the door itself.

Figure 14. Alternative solution for a possible emergency button in two different places This Figure 14 sketch initially probed the idea of having an emergency button in a remote place, such as the control panel of a submarine. This solution raises the question of conflicting with a possible emergency device designed for the submarine itself.

Another reason why it would be complicated to have such a remote trigger for a door is that one could close or open the door without taking notice if there are people passing through it, unless a remote emergency closing button would associate itself automatically to the nearest door. This way, the default interaction would be to open or close the door nearby.

However, it would also make sense to have the emergency control device locally, where it would be needed. This way, people would take less time figuring out what actions they need to accomplish in order to close a door, and would make it much more secure from external hacks.

In case of placing the device next to the door, the issue remains that this local device has to be in reach no matter where a person might swim in. Also, it has to be decided that on which side of the door should the buttons go. Therefore, if the door is bigger than a user could reach

(38)

both sides at the same time, there need to be at least two buttons for the emergency operation to complete.

As a final concept, I chose to have sensors on users feeding location and vital information into an external system that would decide whether to initiate emergency protocol. In other words, the system would know where (tracked) people are, and if they are alive. Additionally, there would be a manual trigger as well to empower people on site, to trigger the same effect. For example, this would mean that - in case of the chosen scenario - biometric data from the divers would feed into the system that handles the door’s emergency protocol, so when they all enter, the door would close (or when they hit the emergency close button) automatically. Evaluation

The concept is more about providing the information rapidly than receiving it. Also, there is no need for obtaining information, unless the protocol is triggered manually. However, should such a situation would take place, there would be no problems with obtaining the information.

Deciding quickly based on this

information 10

There are decisions required from the user, unless upon activating the manual trigger, in which case the possibility for quick decisions are high due to the fact of the visible buttons.

With the manual triggers there is some space for improvisation, otherwise the system automatically takes the lead in the decision making process.

In such a case where control is delegated to an automatic system, it is questionable if there is a feeling of being in control from the user's perspective. Otherwise, with the manual trigger, this is naturally given.

Owing to the lack of an information provider (feedback) system, it is problematic to decide whether any accurate information can be obtained at all. Receiving accurate information from the manual trigger would not be a problem, however, that is not in the scope of this scenario

(39)

either.

Obtaining timely perceptions of reality 3

Similarly to accessing accurate information, timely perceptions are also problematic to gain due to the lack of feedback. Through pressing the manual trigger, there is timely perception of reality.

Tracking and facilitating open exchange

of information 4

Tracking information is a considerable part of the concept. However, there is no information facilitator (one that gives feedback at the location), so we cannot talk about exchange of knowledge. Providing information from the incident

site from both trained observers and

victims 7

Intelligence from victims is fed into a system, directly from the victims. It would pass this criterion excellently, if there would be a way for observers to send and receive this information. Enhancing the ability to focus attention

without interruption 10

The focus of the users (victims) is uninterrupted, as the automatic systems take care of both the decision and the corresponding action. Requiring a minimum of effort to carry

out a task 10 There is almost no effort at all required from the user to solve the emergency situation. Encouraging or facilitating creativity and

team basis 3 Only a manual trigger would provide some form of improvisation in the situation.

Supporting and building trust between individuals unknown to each other in the

team 5

There is no explicit support for building trust between unknown people per se, however, there is no need for knowing each other for the system to function. Team communication in itself is vital in terms of providing the information about the system and this is why the concept does not pass this criterion with good marks.

Support for being able to anticipate and predict when more relevant information

will be obtained 6

If this information means to know if e.g. all people have passed the gate, then the concept handles it fairly well (i.e. closing the gate). However, there is no support for gaining intelligence.

(40)

From the evaluation, we can conclude that the concept is strong on requiring minimal effort, understanding the situation and deciding quickly on the information given. This is mainly due to the fact that the system is capable of taking care of decision-making by itself.

However, there is no support for information exchange or communication, nor is there provided space for improvisation. Furthermore, obtaining information from the user’s perspective is limited to sensory observations. The information collected by the system does not appear in output anywhere, hence failing to provide users with it on site.

The reason for combining bio-monitor and location data in this concept can be illustrated by the diving example. If a diver would loose her life outside of a gateway, than it would be necessary to close down once each diver got in safely who are alive.

Collecting such invasive information as a heart rate, location and other information is only justified by its life saving capabilities. We could consider extreme sport for example. If there would an accident, authorities would know about it right when the mishap occurs. Another reason for using such a concept is by choice, people would decide to accept the

consequences (logging these information). It has to be noted that all details about life circumstances supposed to be protected under a privacy policy that a user agrees upon. User test

The majority of users expressed their thoughts around the nature of the emergency detection.

“I think the strongest part of it is that it’s automatic and manual. I can choose to trust the system or not.”

Some users could imagine this concept being utilised for detecting changes in people’s behaviour, such as a lie detector. They reasoned that there would be fewer amounts of false alarms due to the fact that there are both automatic and manual triggers. However, most of the users saw the automatic solution as an opportunity or aid to decide whether to trigger the system manually or not.

For some users the concept felt unnecessary and too complex, with a surplus of sensors feeding in.

“I don’t know, it just feels off. Why would I need feedback from sensors, when I can decide my own if there is an emergency. Obviously, it would make sense if I wouldn’t be there.” There was a brief discussion about the triggering.

(41)

“It’s too easy to trigger it. I can see people misusing it to send false readings. For example, there is this way for doctors to tell if the patient is lying...”

One user mentioned that there is no means of triggering the emergency if the user is unconscious, and that because of the human factor, the reaction time could be too slow for resolving the current situation.

Another user also suggested that having only a manual trigger would be more intuitive. For example, having the input in a non-trivial position, in order to prevent misuse and restrict access, would simplify the process and result in an efficient resolution of potential crises. Use cases

Diving. In order to determine and track the divers’ vital conditions, there could be an action

taken, such as an adjustment in the oxygen tank or pulling up the diver, while keeping the option open for the manual trigger for the divers themselves.

Evacuation. When fleeing from a building, gates could help detect which areas to seal and

where people haven’t been evacuated from yet.

3.2.4. Scenario From Text: “Sudden appearance of an uninvited guest at a dinner”

"Sirhan stops trying to tell the air what to do and focuses on his clothing instead. Fabric reweaves itself instantly, mapping itself into an armoured airtight suit that raises a bubble visor from behind his neck and flips itself shut across his face. ‘City please supply my grandmama with an

environment suit now. Make it completely autonomous.’

The air around Pamela begins to congeal in a blossom of crystalline security, as a sphere like a giant hamster ball precipitates out around her.

Sirhan dines with his grandmother Pamela, when an orangutan falls onto the table and disturbs them. Sirhan’s clothes automatically go into self-defence mode and he requests an “environment suit” for Pamela. For analysis, I chose the latter interaction.

3.2.4.2. Scene Analysis User Goal