New ways to interact with devices to change the gaming experience

(1)

New ways to interact with devices

to change the gaming experience

Kim Jankowski

Interaktionsdesign Bachelor

22.5HP Spring/2020

(2)

Abstract

Gaming experience refers to the players’ overall experience when playing a game. Both game and controller have an impact on it. This project explores how different element arrangements on a controller can affect the players’ gaming experience. Four lo-fi prototypes were constructed and used to both test the reaction of players, but also to educate them into the possible elements included in controllers. Participants were then invited to design their own controller while reflecting about aspects like embodiment,

immersion, or latency. Throughout the whole process participants reported about their previous experiences with controllers through an interview. The research showed that the elemental arrangements did in fact affect the players’ gaming experience but also that there is a cultural understanding of games and controllers that constrains the possibility of new designs. The project also explored the limitations of conducting research on distance, and reflects about how remote design can be conducted.

(3)

Acknowledgments

I would like to show gratitude and say thank you to: My family for all the support during the thesis work.

David Cuartielles, my supervisor, for all the help, guidance, and feedback through this thesis.

All the people who participated in the user testing and helped to explore the topic of controllers.

(4)

1 Introduction

This project aims to analyze how element arrangements can influence the gaming experience of users. There already are different types of game controllers that are somehow standardized from a user perspective. Controllers have evolved from being merely a joystick with a single button to having many buttons, joysticks, and variations of sensors (Mustaquim & Nyström, 2014).

Furthermore, other categories of controllers include: motion controllers like Nintendo Wii and VR controllers, or natural mapped ones as the guitar in Guitar Hero (McGloin, Farrar, & Krcmar, 2011). Those controllers transform the gaming experience for their respective games. Compare, for example, to play Guitar Hero with a gamepad versus to do so on the official guitar-shaped controller.

Controllers are constantly evolving. Each new generation of game consoles comes with a new controller that will change the user’s gaming experience provided by the previous controller type. In many cases, players end up playing the same games on different consoles and therefore on a variety of controllers. This leads us to wonder how the gaming experience changes because of the change of controller. How can we, designers, have an open conversation with users about how those controllers could be created? These questions open up for the research question presented in the following section.

For the sake of reducing the amount of variables to control, this project will explore the use of different types of controllers on a single game. This is a simple navigation game called the Binding of Isaac, the game will be presented in greater detail in chapter 5, design process. The game is employed during the user testing phase where different prototypes are used to see how those change the gaming experience. This game has been chosen because of its simple control scheme, so that even gamers who have no experience with this type of game could be part of the testing.

This project explores how the gaming experience changes for different controllers. Participants in the research process are asked to test paper prototypes of controllers, as well as to create their own. None of the prototype controllers here presented look or work like existing controllers. They are conceptual objects used to test how these new devices types can affect the user while playing games.

1.1 Aim

The project aims to investigate how different controllers, with the help of interaction design, can change a person’s gaming experience. This will be

(7)

done through the process of creating different controllers that will be tested in different ways to explore gaming experience.

1.2 Research question

1.2.1 Main research question

How could the gaming experience be affected by different controller element arrangements?

1.2.2 Secondary research questions:

What does the prototype’s feedback mean or indicate? How can users be part of a co-design process where they will design their own game controllers?

1.3 Target group

As there are gamers in all ages there is no specific target age for this project but during the testing phase, the testers were aged all from 20 - 40 years old. The project excludes people that have any disabilities as there are more specific considerations to be taken that might not translate back to people without disabilities.

1.4 Ethical consideration

The project follows the ethical standard as formulated in good research practice (Swedish Research Council, 2017). The user testers are informed about what the project is before participating. The user testers participate freely and they are informed before the tests which information will be used and what will be anonymous. They give oral consent that recording the testing is permitted.

While gaming has positive effects, especially to cope with boredom and limited activities during the 2020 Covid-19 isolations around the world. On the other hand, sitting in front of a screen all day playing is not good for the user's health because you do not move around. However, some games challenge that notion/idea. For example, some games let you move your body around making you more active when playing those kinds of games. These games exist primarily in VR and for the Nintendo Wii and incentivize the user to move their body along with the game.

With most things, risks or negative effects can be found. Gaming is no exception to this. Though there are risks with it the project is still going to investigate gaming and work with it. During the project, these risks are mitigated by not having the players in the same position for many hours at a time, so no harm will come to the users when they play the game.

(8)

1.5 Delimitation

The focus is not on the game in the project but how it feels playing it on different devices. The focus is on the interaction with the device and if it changes how they play the game and if so in which way.

The second limitation is the game that is going to be used for user testing. This limit is there because there are too many games and games genres out in the world. Setting this limitation will make it easier to sketch and prototype because the focus is on one game and not a million different ones.

The third limitation due to covid-19 is going to be the number of people that can be tested and the quality of the prototype(s). The prototypes need to be some simple lo-fi prototypes that the users can test. So, it needs to be something simple and easy to use for testing. For this project, paper prototypes are chosen because they are easier to use and construct.

The game used for this project is ‘The Binding of Isaac’. This game was chosen because it uses the most industry-standard controls for a game. That is the keyboard to play with.

1.6 Explorative research process

The chosen design process is not going to be to find a solution it is instead about exploring controls to play a game and see what can be added to existing knowledge about controllers and how they affect the gaming experience. This research process was chosen because this project is not going to end with a conclusive result. Because this research process is about “potentially useful knowledge-building in a new or lonely area (Beall, 2002)”. The explorative research process is going to help with explaining why there is no final prototype for the project as the goal is to widen the knowledge of how controllers change the gaming experience for users.

2 Background

Chapter two highlights what a controller is, what elements are included, and what different aspects of a controller can change the gaming experience for the player.

2.1 What is a controller?

A game controller is a device that is used mainly with consoles but may also be used with computers. There are different controller designs out in the world, the most common ones are from Sony and Microsoft. There is a third manufacturer Nintendo that is the oldest of these three they have designed different game controllers with distinct differences from one another. Each

(9)

of these manufacturers also has its own console line for their controller: Sony for PlayStation1_{(figure 1), Microsoft for Xbox}2_{(figure 2), and Nintendo}3_has,

for example, Nintendo Wii and Nintendo Switch (figure 3). Sony and Microsoft mainly releasing a new version of their console with small changes to their game controller. However, Nintendo has developed different game controller designs for each console in comparison to Sony and Microsoft. Furthermore, PlayStation and Xbox have set the standard of what controllers should be like with buttons, analog sticks, and triggers (Natapov & MacKenzie, 2010). These are only three elements that typically exist in a controller. Looking further, it can be seen that more modern controllers can include touchpads, tilt control, remote pointing, in addition to the previously mentioned three (Ramcharitar & Teather, 2017).

Figure 1. PlayStation 4 Controller

1_{https://www.playstation.com/sv-se/}

2_{https://www.xbox.com/en-US/?xr=mebarnav}

(10)

Figure 2. Xbox One Controller

Figure 3. Nintendo Switch and Nintendo switch controller

The game controllers are either wireless or connected with a cable to a console or computer. However, there are combinations of controller and console that takes away the need for an extra controller because the controls are already implemented into the console. This combination can be seen in the Nintendo DS and Sony’s PSP. Additionally, motion sensors controllers have started to appear in-game controllers, especially in VR controllers such as the HTC Vive4_{(figure 4), Nintendo Wii, and PlayStation Move, this allows}

the player to move around while playing instead of sitting.

(11)

Figure 4. HTC Vive

Additionally, the term controller is not only used for gamepads but for the keyboard and mouse too. This is due to the similarities between both types of controllers. For example, with the way, you navigate a character and look around. This is seen with the left analog sticks in modern gamepads as it works in the same way as the W, A, S, and D keys with that it lets the player move the character forward, left, downward, and right while the right analog stick acts as the mouse as it is used to look around (Gerling, Klauser, & Niesenhaus, 2011; Natapov & MacKenzie, 2010).

2.2 What is a controller element?

A controller’s elements are things like buttons, joysticks, feedback elements, motions sensor, etc. Yet these are only some elements that can exist in a controller as different controllers would have different elements.

The Xbox One controller will be used to see different elements that are in a controller and how they are used. Firstly, the buttons are used to be pressed and there is a visualization of what the button will do when pressed. is the buttons have feedback to them when pressing it in the form of both sound and a stop. Additionally, the different elements exist in a controller to make it functional. But also, to provide the players with different information during gameplay. Secondly, the joysticks are there to both there as an element to navigate, move a character, and look around depending on which one is used. With the joysticks only being able to move in different directed in their housing and they have an extra element to them that lets the player press them down for extra action. Additionally, in there are triggers, d-pad, haptic feedback, and more in a controller.

Comparing the Xbox One controller to an HTC Vive that is a VR controller and headset combination. There exist different elements in both making them work differently to one another (figures of these are seen in the previous section). With the Xbox one controller already being explained I go to the HTC Vive. It has some similar elements such as buttons, triggers, and vibrators it. But the things that make it stand out are the motion sensor element that lets the game mimic the player movement of the hands and the headset that has a screen built into it. There are different variations of VR

(12)

controllers and standard controller, they can have different elements to them and arrangements. Every element as seen in the controllers are there for a specific reason and that is to help the player know that they are doing something. With the element’s arrangement being explained in the following section.

2.3 What is a controller element arrangement?

A controller has different elements to it as explained in the previous section and those elements are arranged in a way to utilize them as best as possible in a controller. Looking at the Xbox One controller yet again but more specific you notice that there are eight buttons, two triggers, two joysticks, one d-pad, haptic feedback, and more to it. These elements are arranged in a specific way with the controller’s design to utilize them as best as possible. Each controller from different manufacturers has a different arrangement for them. Looking at Xbox One and PlayStation 4 controller it is seen that they share nearly the same elements, with the PlayStation having a touchpad and Xbox One a d-pad as extras to them. Except for that one difference in both they share almost the same elements with some different arrangements to them. Knowing that there exist different controllers indicated that there would exist different arrangements of the elements to them. With each controller, the arrangement can change to where a button is placed, or it includes something as a motion sensor to it. Every element in a controller is there to give the player options to how it should be used and what they will receive from doing an action. Since the goal is to have the users design their own controllers, they have to experiment with these arrangements of elements by themselves. Different prototypes of controllers were built that have different elements and arrangements to them. This is what the prototypes are built to let the player experience different elements and ways they can be arranged. This will be discussed in chapter five.

3 Theory

Chapter three highlights the different theoretical aspects that go into analyzing a controller. What is looked at when one is created and what is thought of.

3.1 Naturalness

“‘Natural’ refers to the extent to which users of interactive technology perceive the interactivity to be predictable, logical, or in line with expectations. ‘Naturalness’ is therefore conceptualized as a psychological

(13)

state dependent on both technology and individual differences.” (Skalski, Tamborini, Shelton,

Buncher

, &

Lindmark

, 2011).

Standard controllers do not usually have a naturalness to them and because of that, they do not feel unique when using them. But there exist controllers that do have naturalness to them. What makes natural controllers unique compared to the standard controllers, is that they are designed to be used for specific games. Controllers that have a natural mapping to them are usually designed and constructed to look like the real artifact, and they are seen in-game such as music and sport (McGloin et al., 2011).

Depending on how natural mapped a controller is, it will affect the player with regards to how used they are to the real artifact. Take for example a standard gamepad it is not natural mapped at all compared to the guitar from Guitar Hero that is it. The naturalness is because the guitar controller is constructed to look and feel like a real guitar letting the player get the feeling of playing a real guitar. Given how it looks and feels compared to the corresponding artifact, a natural mapped controller is not just a controller that is used for all types of games as a gamepad. This phenomenon also exists in, for example, the Nintendo Wii remote while playing a sports game like baseball where you swing the controller giving the feeling of swinging a baseball bat without it being designed to match a real baseball bat.

“The nature of natural mapping motion capturing controllers embodies the concept of spatial presence by transferring real-world movements and behaviors of gamers directly into the virtual environment. The more natural the controller is perceived to be by the gamer, the greater levels of perceived spatial presence” (McGloin et al., 2011). Meaning that the naturalness of controllers affects the player's immersion into the game. This is seen with the previously mentioned guitar controller and for example a steering wheel.

3.2 Immersion

Immersion, as defined by (McGloin et al., 2011), is “a feeling of virtual involvement”. Meaning that immersion is mostly obtainable through something like a VR. Because the player gets to see the game in the first-person view, means that they see the same way as they see in real life. The way the VR gives immersion is through a screen in front of the player's eyes that is embedded into the google that is put on. The way VR is constructed is so that the only visible thing is the screen with no way of seeing what is around you. This results in the player becoming immersed in the game. Immersion can be understood as a feeling that the player gets of virtual involvement in a game (McGloin, Farrar, & Krcmar, 2013).

What also affects immersion is the controller for the game because the user’s action is replicated in the game (Birk & Mandryk, 2013). For example, controllers like a steering wheel for racing games would function as a steering wheel in a car, then there is the Nintendo Wii controller that can be used to

(14)

replicate the feeling of swinging a baseball bat(Williams, 2014). These controllers may be used with games that let the user see from the first point on view on a screen. This is not the same as with the player putting on VR headset to where the only thing, they see is the game and nothing from the outside.

3.3 Embodiment

Embodied interaction is the tangible and gestural interaction that a person has with artifacts (Maher & Lee, 2017). Embodied interaction can be seen when using, for example, a Nintendo Wii because while using it the player will move their body around and thus making it embodied (Poels, IJsselsteijn, Kort, & Iersel, 2010). The Nintendo Wii controller works as an embodied controller because it engages to move and not sit still as a standard game controller encourages. This embodiment is not only seen with a Nintendo Wii controller but also in newer technologies as the VR. Where in VR you need to move your body to do different things making it thus embodied.

“Embodied interaction design creates an environment that is activated by gesture and actions on objects and therefore induces cognitive effects that traditional user interaction does not” (Maher & Lee, 2017). Embodied interaction can be seen later in the project with some prototypes because they encourage the user to move around.

“Embodied interaction is the creation, manipulation, and sharing of meaning through engaged interaction with artifacts” (Dourish, 2001 as cited in Waddington & Wakkary, 2010). Meaning that embodied interaction comes from the engagement with artifacts. This engagement with artifacts is seen in the mentioned controllers but at the same time, it can be applied to something that does not require the user to move their whole body but only a part of it. This is seen when a person interacts with nearly all the objects around them. This is further explained with “embodiment is the common way in which we encounter physical and social reality in the everyday world,” (Dourish, 2001 as cited in Waddington & Wakkary, 2010). Indicating that most of the objects around us can be used for embodied interaction in some way.

3.4 Affordances

Affordances exist as a part of everything and as Gibson (1979) describes it, that in the environment there exist objects that have different affordances depending on their manipulation. Furthermore, affordance is not only about the environment but a cohesive between the environment and the user (Gibson, 1979). Meaning that affordance on for example a keyboard would be to press down a key, but the user’s knowledge is what dictates what they will do. This can give different manipulations of how the keyboard can be used in that it can be used for writing a mail or it can be changed for gaming. Changing the environment of how the keyboard is used changes how the

(15)

interaction with it will be. This talk about the keyboard works with Gaver’s (1991) explanation of affordances:

“It implies that the physical attributes of the thing to be acted upon are compatible with those of the actor, that information about those attributes is available in a form compatible with a perceptual system, and (implicitly) that these attributes and the action they make possible are relevant to a culture and a perceiver.”

The affordance of an object is determined by the user’s culture and how they see it. Meaning that an artifact's purpose and design can be changed depending on the user knowing about it. Where one would, for example, play with two different game controllers, one being an Xbox one and the other a keyboard, where they have different designs that change their affordances. A person that has only played with a keyboard can have a problem understanding how the Xbox one controller works because of their differences and how they are used. Here comes the culture part in that a user can perceive how the gamepad works for they have played games with the keyboard while a non-gamer could have a harder time understanding the gamepad and how it works. All of this goes into the design of the artifact. The culture affects the understanding of different artifacts with experiencing similar things earlier in their life.

When designing an artifact, it should be designed in the way that it is easy for the user to understand how it works so the right action is taken (Norman, 1988). The design of the artifact needs to convey to the user what they should do with it. Take, for example, a mouse that is used with the computer, whether a person would not have used one, could make it difficult to understand how it works with its right/left click and the scrolling wheel from only looking at it because there are no labels to describe what each button does on the mouse. But at the same time, whether a person would feel it, they would start to figure out how it works to learn how to use it for different actions. Tying everything together it is understood that affordances are different for each person depending on their culture and the environment they have used the artifact in. The artifact should also indicate to the user what it can do with either letting the user touch it or with only looking at it. The better the design is of the artifact; the fewer labels need to be on it. Indicating that good design should make it easy for people to understand how the artifact works without the extra help of labels on it.

3.5 Feedback and feedforward

When designing an artifact, the designer should think of what feedbacks and feedforwards should be implemented into the artifact. Feedback, as defined by Wensveen,

Djajadiningrat

, and

Overbeeke

(2004), is that it “can be defined

as ‘the return of information about the result of a process or activity’”. Meaning that everyday artifacts in one way or another inform us of the action

(16)

taken with it. In video game controllers it can be seen and felt for example through the press of a button and looking at the screen. Through pressing down a button the player can feel it through the button stopping at a location, giving a sound, and even through vibrations in some circumstances. Looking at another type of controller the keyboard where there exist different types of keycaps that works differently with for example travel distance of the buttons to register it being pressed, sound when pressing, and light. With a bigger variety of keyboards and functions, it is possible to find one that works with a user’s preferences in what type of feedback they want from it. With newer keyboards, it is seen that the light used is not only used to light it up but also to let the user know that they have pressed a key with it changing color. Feedforward, on the other hand, is not about giving the user response from and action, instead, it indicates to the user of what their action will do for example the key on the keyboard showing that the letter ‘A’ is going to appear when that button is pressed. Feedforward defined by Wensveen et al., (2004) is “before the user’s action takes place the product already offers information, which is called feedforward”. This can be seen on the mentioned keyboard with the lettering on it that indicates to the user that pressing down this button will result in this action (Heyer, 2018).

Other types of feedforward can be seen in the gamepad with the joysticks, with the left one always being for moving and the right one for looking around. But even if someone would not know that there is a possibility that they would understand that moving the left joystick forward would make something move forward. But at the same time, the joystick indicates that they can be pushed around with how they are shaped. The buttons indicate that they can be pressed with how they look and that pressing this button with that icon on it will correspond with the command that is for that one in the game.

3.6 Gaming experience

The gaming experience is something that a player feels when playing a game. Different factors are involved with how a player gets a gaming experience but mainly it is what Calvillo Gámez, Cairns, and Cox (2009) say, “the gaming experience is the one-to-one relationship of the player with the game; once the player starts engaging with the game”. Meaning that as soon as the player engages the game the player's gaming experience starts to get affected. The core element of the gaming experience is something that includes two things and those are the video game and the user’s interaction between it and the user itself (Calvillo-Gámez, Cairns, & Cox, 2010). These two elements are what defines the core of the gaming experience.

Depending on how the controllers are designed by the manufacturer and used by a player, it will have an impact on how the gaming experience changes (Brown et al., 2010). This can be seen in, for example, the guitar for Guitar

(17)

Hero where it will have one kind of gaming experience for the game it was designed for compared to using it for another game.

3.7 Summary

To summarize chapter three, the different terms that are explained in this chapter are all connected when looking at the perspective of a game controller and how it should work except for naturalness. Because that one affects game controllers that are meant for specific games and that are constructed to match that game. A controller is used differently by people that will result in a change of its affordance, take for example the guitar from Guitar Hero. It is a natural mapped controller for Guitar Hero and that makes its affordances different from using it to, for example, a shooting game. Looking further into the controller it has the required feedback and feedforward to it but does not have the immersion part to it that a VR headset has for the player. Yet, the embodiment exists with it in the way that the player stands up with the guitar controller and moves around while playing. While the immersion part, as mentioned earlier is not so visible, it is there, in that the player feels like they are a guitar player in a band that stands on a scene and plays for a crowd. Every little aspect of all the mentioned terms can be seen in the guitar controller with some being clearer and some less. All of these ends with the player's gaming experience because these things on a controller are there to let the player get as good of a gaming experience as possible from the controller side.

4 Methodology

Chapter four deals with the methods applied during the project. I used prototypes, sketches, and cultural probes to approach users and desk research, interviews, and co-design to gather information, as well as the wizard of Oz technique and acting to make a systematic analysis of the fieldwork. The following content will explain each one of the methods in greater detail

4.1 Desk research

I conducted a literature review to understand and gain insights about video game controllers. This gave me insights into how different aspects of controllers can change the players’ interaction with games. Literature review increases the knowledge around the research field that is worked upon. (Muratovski, 2016), as well as help finding out what has been done in the field, and what other researchers have found when testing. In the specific case of this thesis, desk research gave me a view of how others look at controllers,

(18)

how they are used, how people feel when using them while gaming, and how they affect the gaming experience.

I found out that different poses affect the users’ immersion in the game. In a study done by Peña and Chen (2017) one can see how users who played games standing compared to sitting got a greater immersion feeling [into the game] than those who were sitting down. These findings got me to define the research question behind this thesis: “How do different game controllers or devices compare amongst each other in terms of gaming experience?”

4.2 Interviews

Conducting interviews allowed me to understand and learn what users know about game controllers. Conducting interviews helps with gathering qualitative data as well because a small number of people were asked in-depth questions about controllers to learn as much as possible from the interviewees. Instead of structured interviews, conducting in-depth interviews and semi-structured interviews let the interviewees answer the question. (Muratovski, 2016).

To gather as much information as possible, the interviews were carried out as semi-structured interviews. Using this kind of interview method allows for more follow-up questions to be asked that are outside of the initial ones (Muratovski, 2016), the questions are open-ended ones which will also let the users answer the question more openly than a normal question would allow (McIntosh & Morse, 2015).

4.3 Sketching

Sketching is used to explore different ideas, for example, new designs of controllers that can be used to play games with (Agustin et al., 2007). Sketches can visualize different ideas of artifacts and how they are going to work; as sketching helps the designer visualize the idea, allowing for input from outside sources. As a result, suggestions, criticisms, and possible changes will come about and will make refining prototypes more possible (Buxton, 2007). Sketches are not only used to see how the prototypes would work but also to get a feeling of the size they would have. Sketching was useful when drawing the controllers, making it easier to draw and change the buttons without the need to draw a new sketch for every little change.

Drawing different sketches helps with exploring different ideas for prototypes because the drawing will make it easier to see whether an idea will work, look, and if it is even possible to build it. Sketches were used to learn how prototypes could be used in different scenarios and to get input from outside sources. As mentioned earlier sketches are disposable ideas, because of that they are easy to use with not only ideas but also to look back at and see what worked in one sketch and take that part and put it on something else (Buxton, 2007).

(19)

4.4 Prototyping

Prototypes are the physical manifestation of a sketch that can be both digital and physical with the latter being used in the project. Prototypes differ from sketches in that they are physical and can show the designer how the idea is going to look and feel (Houde & Hill, 1997). Prototypes, as defined by Houde and Hill (1997), are “an evolving design, and to explore options”, meaning that prototypes are used to explore different design ideas. Every prototype can have both small and big changes as a result of testing. The prototypes are not only supposed to show how it works to users but also to let them test it and receive feedback.

This project uses lo-fi fidelity prototypes to convey the idea to users. Meaning that the prototypes will be constructed from cheap material and either no or very limited technology. In this project, they are going to be constructed through the paper with no technology to them. As a result, they are regarded as lo-fi prototypes that have been used during the project. Using lo-fi prototypes is beneficial in terms of quick and cheap construction of prototypes using materials such as paper. (Lim, Stolterman, & Tenenberg, 2008). Using paper for the project is a way to show how the prototypes are going to look. Paper is easy to manipulate, it gives information to the user from just looking at it and it also can be drawn upon and changed (Sellen & Harper, 2003).

One thing to remember during the building and testing of the prototypes is that they would not have the same kind of feedback as a hi-fi prototype would have. As a result of the way the prototypes are constructed, there is no possible way for the users to feel the sort of feedback of pressing a button downwards also known as inherent feedback (Wensveen et al., 2004). This is the downside of using lo-fi prototypes of only paper, but they are easy to build and send to users for testing. While the feedback part of the prototypes is not clear the feedforward is. Inherent feedforward is the clearest form of feedforward that can be seen on the prototypes. Because inherent feedforward indicates to the user what the different buttons can do while looking at them. For example, some buttons can be pressed in the prototypes showing to the user that this button will do this even if they do not get any feedback from pressing it. Functional feedforward will inform the user about the prototype with the look of it. This will indicate that it will be used for playing, pressing buttons, and holding it in the hands. But as there are different prototypes all users will probably not get what some of the prototypes can be for because they have never played with a controller that utilizes that kind of controls. That makes it hard to understand what the functional feedforwards of the prototypes are going to be. (Wensveen et al., 2004)

(20)

4.5 Co-design

Co-design is when users get a proactive role during the design process for example in the project where users will design their own controllers (Sanders, 2008). Letting the users participate in designing their controller with how it looks and works results in new ideas of how controllers can work as the users are experts from their own experience from playing with controllers. This also indicates what kind of solution for controllers’ users would like to see. During this project, the co-design sessions are conducted one on one during user testing because meeting in a large group is not a possibility during the pandemic. Co-design usually involves different actors that collaborate during co-design sessions to come up with a new design, ideas, and to learn from each other. Co-design is used to learn and understand as much as possible in different fields that collaborate to come up with a solution to the problem. Conducting a co-design session can be used to provoke the users with designs to get them to have a critical thought of the design (Mogensen, 1992; Sanders & Stappers, 2014). Meaning that they are given for example different designs as in this project to let them understand the prototypes but also let them start thinking in different ways about controllers. The prototypes they received were not only used to be tested but also to ‘provoke’ the user into thinking differently about controllers and how they can be used.

4.6 Cultural probe

The cultural probe is a method used to gather data about users that are, for example, in different areas where the researcher cannot be at to gather data or in this case when there is a pandemic going on that restricts the researcher's way to travel. As a result, my method was to send a package to the users with items in it for them. Examples of different tasks are taking pictures, Writing letters or postcards, or writing assignment (Gaver, Dunne, & Pacenti, 1999). In this project, I used cultural probes as a way to send prototypes to users for testing. Therefore, I designed a workflow that would allow me to collect data from experiments that could be conducted long-distance.

The cultural probes for the project did not contain a package with different items in it for the user to do things with. Instead, it contained prototypes for the users to test. While the probes did not have tasks for the user to do before user testing. Using the probe only during user testing helped with making so that the users would use them as intended. A cultural probe, as explained by Gaver et al. (1999) is a technique that is sent out to users to gather the information that will be used to come up with design ideas.

The probe sent to users would not be finished with the buttons meaning that the users needed to fill them up with how they wanted them to be. If they would have done it before the testing, they could have misunderstood what the prototypes are for. At the same time, if the probes would have been

(21)

complete for the users, it would make it hard to analyze the prototypes because they would have tested it in a way it was not meant to. Resulting in making it harder to interpret what they mean (Gaver, Boucher, Pennington, & Walker, 2004). This is the risk with cultural probes: that the user can mean one thing with for example a picture while the researcher understands it in another way. There is no clear way to communicate for the user what they mean with things (Gaver et al., 2004).

4.7 Wizard of Oz and acting

The user testing part of the project was done through the use of both Wizard of Oz and acting. The wizard of Oz part was used because the prototypes were not functional on their own and were in the need of a ‘wizard’ to control them. Having a ‘wizard’ makes so that the prototypes interaction with the prototypes goes through to what they want to do (Dahlbäck, Jönsson, & Ahrenberg, 1993). The Wizard of Oz method is described by Steinfeld, Jenkins, and Scassellati (2008) as a way to explore human behavior to be the designer assumes the role of the technology. This method was used because of the low fidelity of the prototypes as they would not give the action the user wants on their own.

During user testing, the users would talk out loud with what they were doing with the prototype while playing. For example, if they wanted to go right and shoot left, they were saying something like go right and shoot left while they were mimicking the same thing on the prototype. Instead of playing the game normally with themselves controlling what they want to do, it would instead go through me. Meaning that while they are talking loudly what they are doing I would press the corresponding action letting the character do that action. Using this method in the game helped with giving the user one kind of feedback while at the same time it would force them to do the same thing as what they are saying with the prototype. While this is happening, the users would do the acting part of pressing the buttons even if they knew that it does not work.

The acting part, however, was done through the users using the prototypes as if they would work as a normal controller. As the game does need to be played with a working controller and the paper prototypes are not working controllers they are there to give the sense of how the controllers will look and work, through the players acting that they are working. With acting, it means that the games will be played through me, while the user not only does the required movement for the controls but also says out loud what they are doing so that it will happen on their screen. (Smith, 2016).

4.8 Summary

To summarize chapter four, I have introduced the different methods used along the process of researching game controllers. I have been in the situation of having to mix different research methods to adjust to the specific

(22)

contextual circumstances that needed to be done through the current circumstances. Such as having to change the methods to work online with users.

5 Design process

Chapter five is about the design process of the prototypes and user testing. It is about the game that was used with the testing of the prototypes, how the prototypes were constructed, and what the users thought of them. During the user testing, five users agreed to take part. A timeline can be seen of the design process beneath (figure 5).

(23)

5.1 The Binding of Isaac

The game used for testing the different prototypes is called the Binding of Isaac (see screenshot figure 6). The game can be found on the Steam platform. It belongs to the rogue-like game genre, "is a subgenre of role-playing video games characterized by a dungeon crawl through procedurally generated levels, turn-based gameplay, tile-based graphics, and permanent death of the player character” (

Roguelike

, n.d.). This makes

so that the player will never know what lies ahead of him. Yet, it is not only the environment and enemies that are randomly generated it is also the items that the character can use. The goal of each room is to defeat the enemies in order to progress further and the goal of the game is to defeat the final enemy, also known as “the boss”. In the game, there are different items — coins, keys, bombs, cards, pills, passive items, and active items— that are activated in different ways by the pressing different buttons. As my goal is to evaluate the use of variations of self-designed game controllers, having a single game experiment with simplifies the process. Observing how different players respond to the controllers will further inform the design process.

Since each new game will be different from the previous ones, as they are randomly generated, each user will have a different experience when playing. Meaning that there can be users that do not get the opportunity to test all of the items in the game. The game was chosen because it can be played with both keyboard and gamepads allowing for both sitting down and stand up playing postures. Furthermore, the game’s control mechanism is simple and easy to understand after just some minutes of gameplay. This represents an advantage for testing the prototypes. The player should quickly embrace the game dynamics for a more productive collaborative prototyping session.

(24)

5.2 Prototypes design

In the design process, different phases took place; different ideas became prototypes and got changed after a testing phase. After the testing phase, the number of prototypes decreased from eight to four, and instead of having the tester build the prototypes, they would be sent to the users through the mail.

5.2.1 Interviews

Conducting interviews with people was done online through a platform called Zoom. The interviews were done after the desk research to learn more about controllers from users’ and how they interact with it while playing.

As expected, the majority of the interviewees either played with a PlayStation controller or an Xbox controller as their main gamepad device. This is because they grew up with one of the manufacturer’s consoles and controllers making it their preferred one. The interviewees even preferred to use the gamepads with the computer instead of the keyboard and mouse, this is because they grew up playing console games and when the opportunity came to use the same controller with the computer they would take that. Interviewees would not abandon the keyboard and mouse as that for gaming gives the player more options to assign different buttons to different things. Most of the players choose their controller depending on what game they would play.

The interviewees prefer to play computer games with an external controller because it is easier to play with, it is more compact, easier to sit in front of the screen and they don’t need to move their hands all the time compared to playing with keyboard and mouse. With a keyboard and mouse, it is mentioned that it requires a lot of movement to press the buttons compared to a gamepad where the hands fall into the right place. This can be why most of the interviewees preferred to use the gamepad for the computer as they do not need to move their hands as much.

Furthermore, another aspect that needs to be taken into consideration after the interviews is the user's pose while playing. Meaning for example that the user was usually sitting down while playing with either the PlayStation or the Xbox controller and with the Nintendo Wii controller they would usually stand up while playing. Because the Nintendo Wii is constructed with a motion sensor in its controller and with that engaging the user to stand up while playing its games.

5.2.2 Sketching

After conducting the interviews and analyzing the answers the project would go into the sketching phase. This phase is where the prototypes' ideas came to life. The inspiration to come up with the different prototype controllers came from what the interviewees said their preferred controller is and from what the desk research showed about controllers. Additionally, looking at

(25)

poses while playing helped with realizing how differently sketches could be used when they become prototypes.

What the interviews and desk research revealed was that people are interested in playing games more with a gamepad than keyboard and mouse if they have an opportunity for it. So, when it goes into starting to design different controllers it goes with thinking first, how they should be controlled, their size, and the placements of the buttons. The best thing became to combine different aspects of controllers that already exist and come up with something new. Sketches are easy to visualize an idea, they were used to draw different variations of ideas and see whether they could work. If they did not work maybe some aspects from the idea could be used with another idea and that is how it ended with eight different sketches that became four prototypes in the end.

5.2.3 Constructing the prototypes

After having sketched on different ideas for controllers that can be used and tested, it came to building them. It ended with eight sketches it started with the same number of prototypes but ended with four. As the prototypes are going to be all lo-fi, the need for technology was not needed for construction making it easier and faster to build them. The prototypes became different combinations of some controllers and some are new to test how they would work. This will be explained further down the chapter.

All the prototypes share some similar aspects to them as that all of them do have buttons on them. But instead of already being mapped like on a normal controller they are left blank for the user, to let themselves mark out how the buttons should look. After the construction of the prototypes, it was time to send them to the user tester through the use of cultural probes, but before that, there will be an explanation of the prototypes feedback and feedforward.

5.2.4 Cultural probe

The cultural probe package (figure 7) contained all the prototypes that the users needed for testing without the need to spend time building them and figuring out whether they are built correctly. Five packages were sent out to five user testers that had answered that they could participate in the user testing. The packages contained five parts that would together become four prototypes. Sending this out to the user testers made so that they did not need to build the prototypes as they were nearly finished, instead, they only needed to mark out how the buttons should look. Leaving the buttons blank allows the users to give them their symbols, colors, or something else to indicate what the buttons do. The colors could have been e.g. red for the buttons that are for shooting, green for walking, and yellow for the rest of the buttons. With symbols there existed arrows for both shooting and walking. The user would do so that they would know what the buttons did.

(26)

Figure 7. Cultural probe package with the prototypes

5.3 Prototype testing

As prototypes have been sent out to users, the design process moves further into how the user testing was conducted. As the probes were sent out, the beginning of the user testing phase begun with the users receiving the probe and opening it to see if everything is there. Because they have gotten a mail before with information about the probes and to know their address. That was deleted after the user testing and was only used for sending out the probe.

5.3.1 User testing

Before conducting the user testing with five people there was a test session with one user to test the concept of the prototypes. The test session revealed that the workshop was too long and needed to be re-worked. The re-work became to remove some of the building aspects to shorten the workshop time to focus on the gameplay instead.

Doing the user testing online gave the opportunity to test the lo-fi prototypes but also gave some problems for me and the user. The problems that came up was for example internet connection and a delay with the user doing an action and it pops up on their screen. This delay affected the users with how they perceived the controllers because instead of action happening immediately when pressing a button, it would take around two seconds for the users’ command to happen.

After explaining some of the problems that the user will feel during the user testing an explanation of the game was given with what it is and how it is played. As there is this delay for the users’ actions to be recognized the game

(27)

would be played with invincibility. Meaning that the character in the game will not take any damage from the enemies.

Furthermore, the testing would be in different phases with them testing all the prototypes before questions were asked. But before testing each prototype they would first test the game through playing it with a keyboard to get an understanding of how the controls work. Each prototype would take around 10 minutes to let the user test so that they get a feeling of how it works. After the keyboard test, the prototypes got tested with the users marking the buttons so that they would know what each does before testing that prototype. Lastly, the users would draw their controller that would work to play games with. It was said that they could draw any kind of controllers that will be explained later.

5.3.2 The user testing setting

As it is not easy to keep the same setting for each user because of it being online and everyone has different computers that they are sitting in front of. The only setting that can be affected would be that they have a place to both sit down in front of the computer on a chair with leg space and the other to have enough clearance to stand in front of the computer while at the same time seeing the screen. Those two are the clear settings for each user to have. While all the user testers did not have a camera, they would use their phone as a camera during the test.

5.3.3 Poses

While user testing the prototypes, it was not only to see how they work while the user is playing. But also, to see if different poses affect the user's gaming experience.

5.3.3.1 Standing

During user testing, some prototypes were used while the users needed to stand up. Those prototypes were the sticks, the bracelet, and the watch. While using the prototypes the users not only needed to move their arms but their legs as well and by doing this, they commented that they felt a greater immersion with the game than sitting down. This is because they felt like they became closer to the character they were controlling. Instead of sitting down and controlling the character with the hands, they needed to move their whole body to the direction they wanted to go and shoot.

The most immersive prototype was the bracelet. That made the user move the character with their feet and use items with their hands. By doing this they got the feeling that they were not only controlling the character but more becoming it. Showing that when playing games while standing up does change how the user understands the game and that their experience changes.

(28)

5.3.3.2 Sitting

While sitting down the users also used three prototypes to test. Some prototypes could be used both while sitting and standing. The ones that were used to test the sitting aspect were the gamepad, the sticks, and the watch prototypes. As it is seen two of those were also used while standing up that is because they both were constructed so that the user could use it both ways. When the users were sitting down while playing with the prototypes, they commented that they felt it easier to play the game sitting down than standing. But also, they felt more relaxed during their gaming session instead of constantly thinking of where their feet are in relation to the buttons. When they were sitting down it was possible to see that they did not exactly worry about where the buttons are compared to standing where they looked down a lot to see where what is.

5.4 The prototypes

5.4.1 Gamepad:

The first prototype resembles the existing gamepads (figure 8) like the ones from PlayStation or Xbox. One change is that instead of having two joysticks, there’s a bigger single joystick with extra buttons on the front. The positioning of the joystick and buttons are done in this way to have improved accessibility because if they would have been closer to the middle, they would be harder to reach. There are also two triggers at the back for the user to use as well. The buttons on the controller worked the same for all users, but the joystick would work differently, with some either having it like the ends of the joystick was buttons or they would have it like a trackpad where they move their thumb around without lifting it up.

The controller does not have any physical buttons if referring to already existing ones that have sound and feel when pressing a button. Not having this feeling puts users in a confused state because they do not know whether they have pressed the button or even pressed the right one because of the lack of feedback from the gamepad itself. Even without the correct feedback, users still felt like they were playing on a gamepad and it felt natural to use for them. As one user said, “Comfortable and could chill in the chair while playing” (figure 10) indicating that the user could take it easy while playing with this prototype with a sitting pose in the chair.

(29)

Figure 8. The gamepad prototype

5.4.2 Sticks with buttons on them:

The second prototype (figure 9) got its inspiration from motions controllers such as the Nintendo Wii controller and VR controllers. The sticks are constructed to be held in each hand with the buttons located for the thumps to rest on. The prototype is inspired to be a motion controller it is supposed to be used with the user standing up and playing but it can also be used sitting down for people that are not able to play while standing up. Being able to use this both standing and sitting down comes from the controls being the movement of the arms meaning that it can be moved in different directions. The sticks are cylinder-shaped so that they fit better into the hands of the users. But unfortunately, the length of the sticks is a harder problem to fix because people have different hand sizes making so that a short stick would not work with someone that has big hands and the same thing the other way around.

(30)

Figure 9. The sticks prototype

5.4.3 Bracelet with square cards:

The third prototype took its inspiration from the arcade game ‘dance dance revolution’ controls with the addition of bracelets for pressing buttons (figure 10). Having the controls for both moving and shooting on the floor was not a unique way to control a character but users mentioned that the bracelets and how they worked felt unique to them. The controls were not unique for the users it was the bracelet as the have never played a game where you need to active things with slapping the opposite wrist with your hand. One user said that “It felt kinda neat when using powers” showing that this user likes how the bracelet was incorporated and used.

As the sticks felt immersive to some users this one gave an even greater immersion because of how the character in the game was moved and how items were activated. One user said, “I liked them because it was feeling that you were the Isaac” indicating that the immersion is bigger with these controls than with others. The ones that understood the controls liked it but those who did not have a problem with both moving their feet around and remembering to activate items. Diving deeper into why users did not like this one as much as the others are because of the complexity of the controls because of the previous controls were simple in how they operated. With this one having added the feet for additional control to let the users move their body and not only their hands and arms. Another part why users had a problem using it is because the prototype is constructed with only paper and that made so that the square cards moved around on the floor making it harder for the user to hit it each time.

(31)

Figure 10. The bracelet prototype with both the bracelet and the movement buttons

5.4.4 Watch with square cards:

The fourth and final prototype (figure 11) got its inspiration from the same place as the previous one, car pedals, and smartwatch. Having this much inspiration for the prototype gave something that can both be used while standing or sitting. The square cards are placed in the way it is, to test another kind of control where you need to move the feet to the front to shoot and activate items, while the watch is used to move the character around. Changing how the users would use their feet and arms made it in one way easier for them to see where they were pressing with their feet. The watch made it easier to control the character as the users did not need to move around their hands a lot as everything was in one place. One user even said that they would like to have a button in the middle of the watch to activate an item with.

The addition of the watch as a controller made so that the user would react faster comparing it to the previous prototype. One user said, “I had the movement on the watch, so I reacted faster”. Indicating that having a compact control makes it easier for the user to react faster in the game compared to having it spread out like on the previous one.

(32)

Figure 11. The watch prototype with both the watch for character movement and the square buttons for shooting and activating items

5.5 Co-design

For participants to warm up and design their game controller and because we are in a distant situation, they were first given an easy game and with a simple game mechanism to learn how to control it. And then they were given fake prototypes to play the game with, so they would go into the mindset of how these fake controllers work. After the user testing, the time came for them to think of their idea for a game controller. The game sets up the framework they are going to play and learn from, and then the tools which are scissors, paper, and pens. The participants were shown four different paper prototypes that they tested, this was an invitation for being playful and imagining uses of new interactive things built with paper. As they had tested and understood how the prototypes work, I asked them to design their own game controller which set us off for the co-design session. All I did up to this point was in preparation for this moment where I could tell the participant to please design their own game controller, out of what they thought is important for a controller to have.

Every participant designed their controller that could be used to play the same gave with, it became interesting to see how different each controller is from the rest. All controllers implemented a way to move and shoot, but after that, the number of buttons changed for each different design. If a participant only used buttons during the gameplay, then their controller would only have two buttons instead of four because they never needed to use the other two. It looks like the participants got manipulated when constructing their

(33)

controllers by how their gameplay went. I further elaborate this idea in the following paragraph.

One interesting part of the participant’s controller design is that they made them form what they are comfortable using while playing. Meaning that they took inspiration from either one of the prototypes of what they usually use when playing. Participants designed both big and small controllers that they would like to play with if they had the opportunity to. While it became clear that the framework given to the participants (same game, same design elements) played a role in the design, their own personal experiences also became visible in their artefacts. What I find relevant about this is that the gaming experience that would come out from using these controllers would differ depending on the game they would be used for. Also this design exercise imposed no limitations on how easy those design would be to implement for real, as the participant’s role was not to design a completely working controller but designing new controllers focusing only on how they would like them to look and work.

5.6 Player feedback

The most common player input was about how the prototypes did not have any feedback to them with no feeling of pressing or even hovering a finger over a button. Having no feedback made it hard for all of the users and for all of the prototypes. Participants would like to have something more than only seeing the feedback on the screen. Because even if they were saying that they were going up they may have pressed the wrong button. How the participants thought of the prototypes can be seen in the table in figure 12.

The participants said that they liked the prototypes of controllers I suggested, but that they should not be used with this kind of game. For example, the sticks worked conceptually, but did not have the right connection to the game — affordances — they mentioned how it could eventually work in other game genres. They got the question as to what games or genres the controller could be used for and unfortunately the users could not give an answer to what kind. This could indicate that maybe the participants are not that experienced with different game controllers, or alternatively, that they did not like the controller but gave a biased answer (as in trying to be “nice”).

The users said that they liked the controllers but that they should not be used with this kind of game. One prototype that was like this where the sticks where they worked but did not have the right connection to the game but could work in other genres. They got the question as to what games or genres the controller could be used for and unfortunately the users could not give an answer to what kind.

Furthermore, if we look at how different poses have affected the players while testing I conclude that they prefer to sit down compared to standing. Data shows that while sitting down they felt more relaxed and it felt easier for them

(34)

to play the game. While standing, they instead felt that they were getting more immersed into the game, because they moved with the game character and not only their hands. What I can extract from this is that there are different playing moments that are associated with the players’ pose. Some games require immersion, thus they are nor as “relaxed” as others. Game controllers should reflect this situation. Not every controller will fit every playing scenario.

Keyboard Gamepad Sticks Bracelet Watch

User tester 1 Can use more fingers that will put less strain on my fingers. Felt like a controller. Feels like I need to be at more places at once because I use less fingers. Felt like a VR controller. Felt immersive when using it but the motion controls felt awkward when using.

Was the most immersive one but I had slower reaction speed. Felt neat when using power/items and it felt unique to use.

Liked that the character movements were on the watch. Felt like I had more freedom with this one

User tester 2 Felt normal to use Most comfortable to use Uncomfortable because of the hand movement made it feel awkward Clumsy to use. Needing to use hand for items was distracting while playing because I needed to look down a lot to see where my feet where Clumsy to use even with fingers on the watch all the time. While playing it was easier to use it while sitting compared to standing. There was no need to think about the bodyweight while moving like on the bracelet. User tester 3 Felt normal to

use

Comfortable and could chill in the chair while playing

There was more flexibility with the sticks that could make it good

Felt that you

were the

character in the game. Hard to do

Easier with the controls on the watch

(35)

Figure 12. Table of users input of the prototypes

5.7 Summary

To summarize chapter five about the design comprises the user testing and co-design session for the different uses. During my first user testing with my first user it became clear that I had included too many prototypes for the testing process resulting in a too long testing time. I revised the user testing to include fewer prototypes that resulted in shorter testing times. I sent the prototypes by mail so that users would not need to build them themselves which further shortened the first half of the sessions (dedicated to learning about the different affordances). The user testing leads up to the co-design session where participants would design their own controller departing from the experience of the prototypes.

I documented the whole process, and produced a table (figure 12) including the main outcomes from testing.

for other kind of games.

different actions with the feet at the same time. And I felt more alert when playing.

compared to the bracelet

User tester 4 Felt normal to use

Felt most like what I am used to Clumsy to use when e.g. needing to move and shoot in opposite directions. Hard to move around and remember where each button is on the floor. Felt weird to use the controls because I am not used to control games in this way User tester 5 Felt normal to

use

After using the keyboard this was the most normal

controller to use

Fun to use with moving the arms around but confusing to remember which stick was for moving and shooting

Felt unique to have

activation on the wrists but moving and shooting was hard to do with the feet.

Felt a little random with having a control on a ‘watch’ but it was easy to use

New ways to interact with devices to change the gaming experience