Analog input in gaming: Investigating the possibilities of new controller affordances in video games

(1)

Analog input in gaming:

Investigating the possibilities of new controller affordances in

video games

Pim Ostendorf

Interaction Design One-Year Master

Thesis Project I KD643A 15.0 credits May 23, 2018

(2)

(3)

Acknowledgements

I would like to start by thanking my friend, Johanna Westerlund, for her dedication in creating the sprite art required to create the prototype on such a short notice. I don’t want to think about the abominations I would have drawn up if she had not been able to help me out.

Furthermore, I want to thank my friends at Spelenshus Malmö, who were willing to take time out of their day to playtest and discuss the prototype I created. My thanks also goes out to my classmates for their time and input in the experiments and playtesting.

Also, my thanks to my supervisor, Per Linde for his enthusiasm, passion and his inexplicable ability to make me say things that sound clever.

Last but not least my thanks go out to David Cuartielles for his input and advice when I got stuck. I hope your keyboards will turn out amazing.

(4)

Abstract

This report covers the eight week design project for the Thesis Project 1 and it looks at both the process and the results of this project. Through an iterative prototyping process a prototype was created to answer the research question: “What gameplay mechanics are required for a fighting style video game to allow for analog input in its character movements and do these mechanics allow for more natural controls and mastery?” Together with research into academic writing in the fields of neurology, embodied design and game design it was concluded that an analog control scheme, within the context of a video game that allows for that type of affordance, gives the user more natural control over the character they are playing. This was validated by

playtesting the prototype with several users. However, the richness of the mastery of a game relies heavily on the implemented game elements in a finalized game and was therefore unanswered in this project. There were also constraints and game mechanics identified for game designers to keep in mind when designing a game that relies on this type of analog input. The project also raised a number of questions and new design opportunities that were unable to be explored due to the limited scope of the project.

(5)

1. Introduction: Aim and Research Question

1.1 Background

With our advances in technology and manufacturing processes our input devices for video games have changed drastically as well. Where the game consoles of old were generally limited to binary inputs that could only approximate movements in the most basic of ways. The

advances in digital sensors with a range of values (from now on referred to as analog sensors), touch controls and motion technology allow much finer control in our video games. However, most video game genres have not caught up with this and are still using the binary inputs that the old controllers dictated.

I saw an opportunity to explore this side of controller mapping to see if I can employ these advances in the standard video game controllers to create a more natural and intuitive way of controlling fighting games.

I am especially looking into the genre of fighting games. This genre evolved from their roots in arcade machines to more generalised machines like pc and consoles today. Many expert players nowadays prefer to play the game on so called fight sticks.

(8)

These controllers emulate the physical layout of the older arcade machines. However, there are also players that use more standard controllers. The gameplay of these games revolves around close combat against either an AI or another human opponent. The player can input different combinations of button presses to make sure that their in game character comes out victorious. These different inputs can be chained together to create something called a combo, a

combination of moves that results in a often preprogrammed movement that the character performs. Most of the times these combos get executed without any further input from the player.

Figure 2. A standard Xbox One controller. The trigger buttons are highlighted.

Figure 3. The Xbox One controller with the analog sticks highlighted.

There is also a sizable community build around fighting games of both expert and professional players as well as enthusiasts who watch broadcasted fights between professional players online. There are tournaments organised where people compete against each other for

prizemoney or glory and in some cases both. This community could possibly be a point to take into account since there are a lot of people that really love this genre of game and have been playing these games for a very long time. Meaning that drastic changes to the way the games are played might not catch on. However, there is also an upside since the community existing means that ideas spread fast and the community can develop moves together out of the freedom created by this new mapping.

1.2 Aim

The aim of this thesis is to determine what aspects of mechanics are needed to facilitate the physical affordances that newer controllers have. Getting to the point where these findings (type

(9)

of interactions) are implemented in the context of an actual video game lies beyond the scope of this project.

The research question is posed as follows: How can we integrate kinesthetic design principles in fighting games controls to allow for more mastery and granularity?

In this research question I define the kinesthetic design as a goal where the user should have the ability to visually observe a motion and being able to replicate this motion without further clues or guidence. As such the principles of kinesthetic design are design conventions that are generally used for full body motions applied to acting on a small device.

A future vision of this type of interaction would be a game where the movements made by the user on the controller are mapped to the screen in a non-binary way. This analog input allows for greater mastery on the users part as well as giving the user a set number of interaction points, eliminating the need to learn what move is bound to what button. Instead the learning of moves/actions should come as intrinsically as the learning of hand gestures, incrementally more complex and building on each other.

2. Theory

2.1 Literature review

2.1.1 Game and Play

When looking at game design we first need to define what a game is as well as what play entails. In the chapter on “Play as Research” by Eric Zimmerman _{(2003) they conclude their text} with: “To design a game is to construct a set of rules. But the point of game design is not to have players experience rules - it is to have players experience play.”

In chapter 7 of their book _{(Salen & Zimmerman, 2004, p. 11) define a game as the following: “A} game is a system in which players engage in an artificial conflict, defined by rules, that results in a quantifiable outcome.”

In chapter 22, page 11 they define play as: “The play of a game is the experiential aspect of a game. Play in a game occurs as the game rules are set into motion and experienced by the players.”

They also conclude that games are a subset of play as play is a far wider concept than games generally are. But play is also an element of games. It is a part that makes the game a game as the definitions quoted above state. In this sense the two definitions are entangled and hard to separate. However, in this thesis I want to dive deeper into the design that lies beyond the

(10)

definition of play and games landing on a level where I am less concerned about definition of what I am working with and more with the embodiment of the experience I want to facilitate. This embodiment is the coupling between the actions on the controller and the actions displayed on the screen.

2.1.2 The appeal of games and embodiment

When thinking of video games in the future there is generally this idea of total immersion, with the increased availability of virtual reality to the general public and advances in haptics it may seem like the traditional way of experiencing games is on its way out. However this viewpoint generally revolves around the ideal that games should be authentic experiences that are grounded in our real world. The paper by _{(Farrow & Lacovides, 2013) further explores this idea} of full immersion and embodiment. However when I look from the perspective of a gamer that is focussed on mechanics driven games it aligns well with one of their final statements in their conclusion: “Part of the enjoyment of watching a horror film or playing a soldier in a first person shooter may stem from the very fact that these experiences do indeed differ from what they would be in the “risky, moody real world” where we are much more vulnerable: it is arguably our lack of embodiment that forms the basis of the appeal.” Games do not always need to strive to a perfect full body simulation of a certain concept. Sometimes a game just needs to be a relaxing time without physical exertion.

2.1.3 Natural interactions

The word natural can have many meanings. In the context of video games what is natural may not always align with what is perceived natural in the real world. In the paper by _(Urgesi, Candidi, Fabbro, Romani, & Aglioti, 2006)_{they describe the correlation between watching} motions happen on a screen and the activation of the parts of the brain that would normally be activated during the execution of that motion. The naturalness I want to achieve with this remapping should be as close as possible to the motion displayed on the screen, effectively allowing the users to understand a movement by looking at the visuals.

In chapter 11 of their book “Biological Psychology: An Introduction to Behavioral, Cognitive, and Clinical Neuroscience” _{(Marc Breedlove & Watson, 2013) talk about the concept of mirror} neurons and again the ability for humans to activate the same area of the brain when seeing someone else perform a movement. Giving a strong indication that we can use these concepts in developing our control scheme to fit closer to the human brain.

2.1.4 Experience of mediated embodied play

In their paper _{(Tholander & Johansson, 2010) talk about eight interactional qualities that are of} importance for peoples engagement and embodied interaction. One of the qualities

“Open-ended response” is of special interest to us as it tends to be completely omitted by fighting games as they are now. All actions made on the controller and the mapped actions in

(11)

game are pre-determined by the developer. By using analog inputs and mapping I want to give the user a more open-ended response.

In a similar sense the paper written by _{(Nixon & Bizzocchi, 2013, p. 11) says: “However, upon} closer examination, this interaction provides a powerful tool for reaffirming players’ connections to the character they are controlling and their immersion within the virtual world through the arousal of affect and interest.” The interaction they speak of is the actions performed by the player on the analog stick. In Heavy Rain the user has to perform contextual movements on the analog stick to progress the story and complete puzzles. In turn these contextual movements further engaged the user in the virtual world.

In his book “Where the Action is: The Foundations of Embodied Interaction” _{(Dourish, 2004, p.} 100)_{he defines embodiment as: “Embodiment means possessing and acting through a physical} manifestation in the world.” In our case this would be the controller for our video game. Where the user acts through the controller to influence the virtual world. The virtual world would normally be ungraspable but the physicality of the controller allows the user to act in this virtual world. On page 138 he also states: “Coupling is how an intentional reference is made effective.” In our case this coupling is the intentional chain between the movement on the controller and the eventual output on the screen. The efficiency with which this coupling is made and the length of the chain are directly correlated to the experience of the user.

2.1.5 Motion Controls

Motion controls would at first seem like a very interesting avenue and the literature on controller mapping in video games by _{(Skalski, Tamborini, Shelton, Buncher, & Lindmark, 2010) and} (McGloin & Krcmar, 2011)_{has a very optimistic outlook on motion controls and their ability to} change the way we interact with games. However, 12 years after the initial launch of the Wii the landscape of mainstream video games genres has changed very little, with Nintendo even going back to more standard control methods for their new Switch console. _{(Rogers, Bowman, &} Oliver, 2015)_{alludes to the idea that motion controls might not be the most natural mapping in} the context of video games. This is further confirmed in the paper by _{(Farrow & Lacovides,} 2012, p. 8)_{who conclude that: “Technologies and interfaces which promote using more of the} body than just the hands to interact with digital worlds do not necessarily promote a sense of embodiment.” Context of the game needs to match the context of the game.

From the commercial examples we have seen using motion controls, the games were all

simulations in some sense. Wii Sports simulates different sports and games. These games used the motion controls as their central premise and core gameplay element. However not all video games are simulations. Fighting games are a power fantasy and, when one gets sufficiently proficient at them, a mechanical puzzle. The player tries to outsmart the AI or opponent through a combination of mechanical skill and strategic choices. In this state the power fantasy aspect gets pushed to the background and the player is focused entirely on the mechanics of the game and how to employ them to become victorious over their opponent. In this scenario the physical

(12)

execution of these mechanics gets pushed to the background as well. As opposed to simulations where the player is continuously engaged by the physical motion since it is the principal gameplay element in the game.

2.1.6 Buttons in gameplay

In the paper by _{(Griffin, 2005), in which they describe the state and function of the button in} video games, they say: “Current buttons are not suitable for intimate, performance-based play. They are incapable of capturing the nuance of corporeal expression. The hand’s movements are situated in time and space while the biased-switch is instantaneous.” Despite this paper being quite dated it encapsulates the problems with the binary input system that is so prevalent in video games really well. There is no space for exploration within the movement themselves while our bodies do not find themselves in such a binary space.

2.1.7 Neurological limits

Chapter 18 of _{(Marc Breedlove & Watson, 2013)’s book describes the ability of the brain to} enhance or suppress the firing rate of neurons depending on where the attention of that specific person lies. This brings us to an interesting conundrum, when we want our user to perform fine grade motor actions on the controller they have to divide their attention to it. However, how many tasks can the brain keep up with before it become overwhelmed and starts suppressing the firing rate of its neurons, effectively hampering the users ability to perform the tasks given to them. When consulting Jules Erkens, a PHD student in cognitive neuroscience and fervent gamer, they speculated that users could potentially increase the amount of fine motor tasks they could perform at the same time through training. Whether this is applicable and engaging in the context of a video game is something that is beyond the scope of this project.

2.2 Examples

There are a couple of games that have used analog control concepts, such as “Getting Over It with Bennett Foddy” _{(Foddy, 2017) and “Chivalry: Medieval Warfare” (Torn-Banner-Studios,} 2012)_.

In the case of Getting over it, the player controls their character with the mouse, moving a sledgehammer in order to scale a mountain. This gives the player a lot of granularity in their movements since the movements made by the players hand and arm is directly translated into the game. It gives the game a very high skill ceiling and allows for a lot of mastery and

experimentation. Where a first-time player might have to invest several hours to complete the game, expert players have been able to complete the game in less than 2 minutes. _{(Apok et al.,} 24 april, 2018)_{This illustrates the amount of proficiency users can build up with this method of} controls.

(13)

Figure 4. A screenshot of “Getting over it”. (Source: gameplay.tips)

Chivalry: Medieval Warfare is interesting as well because it uses a hybrid form of control. The players use the mouse buttons to initiate a different type of attack. But using the mouse

movements expert users can change the position of the upper body. This can be used to catch out opponents or extend the range of an attack giving the user the upper hand. Again, the added dimension of free movement allows the user to reach a greater level of mastery.

2.2.1 Skullgirls analysis

The game Skullgirls _{(RevengeLabs, 2012) is a fighting game set on a 2-dimensional plane,} where two characters are engaged in close quarter combat. For this analysis we presume that the player is using an Xbox One controller (or similar) and the standard key bindings for the game, since Skullgirls allows the player to remap their controls.

The core game mechanics of Skullgirls revolve around beating the opponent by draining their health bar through combat. The player can press a variety of buttons on the controller to make the character they are controlling perform certain moves. These moves can vary from punches to kicks to throwing projectiles. The directional movement of the character is controlled through the analog stick. In Skullgirls the player can string these button presses and directional

movements with the analog stick together to create something called a combo. These combos are high powered moves that get performed after the user input the correct button combination and their characters performance meter is sufficiently filled. The performance meter fills up gradually when landing successful attacks on your opponent. (The curved bar around the

(14)

character portrait in the top.) It is also interesting to note that the button presses needed to perform these combos are not hidden but rather can be found in game through the pause menu. This is a clear indicator that the combos are not meant to be a secret that the player can

discover through play but rather an integrated mechanic that the player is expected to use.

Figure 5. A screenshot of two characters in Skullgirls ready to face each other.

(15)

As I wrote previously most fighting games can be classified as a mechanical puzzle instead of a simulation and Skullgirls is no different in that. Although mastering the motions needed to control the character can be a challenge, there is a point when the player is more concerned about the strategic and tactical mechanics of the game rather than the physical execution of the moves that go into executing those moves. It is not the execution of the motion that takes the central stage like in simulation games like Wii Sports _{(Miyamoto, Tezuka, Asuke, Imaizumi, &} Matoba, 2006)_{but rather the underlying balance between the characters and skill of the users} playing the characters. Your way to victory is a puzzle that you need to solve and then execute on.

Skullgirls controls find themselves along two axis of play. The first axis being the directional movement of the character. That being either jumping up, crouching down or moving left to right. The second axis is the activation of the characters fighting moves, punching or kicking. These axis together form the control method for the game. They are generally combined with each other but each axis inherently excludes itself from conveying more inputs than two. Skullgirls sometimes requires the player to press two buttons at the same time to trigger one of the combos but most combos rely on a combination of the two axis together.

3. Methods

3.1 Fly on the shoulder

While conducting my experiments I always made sure to pay especially close attention to the movements and actions the user took. This is also the reason why I prefer to record the audio during an experiment so I can prioritise taking notes of movements and actions over recording what the user is saying. In that sense my approach differs from the “Fly on the wall method” as described in the “The field guide to human- centered design”. _{((Firm) & IDEO.org, 2015) They} described the method as shadowing your/the users and submerging yourself in their world. While my method is loosely based on theirs I propose the name “Fly on the Shoulder” as I am closer to the user that is being observed and at some points even interacting with them. My main focus is still on their movements and their behaviours.

3.2 Loose interview

In combination with the “Fly on the shoulder” method I also tend to conduct loose interviews. From experience I have found it useful to debrief the people I ran the user test on. This gives us the time and more importantly the context to have an interview where I am able to ask them about behaviour that they displayed or further discuss why they feel that they did certain things. The format tends to be pretty loose since I adapt most of the questions depending on the things they did or how they acted during the “Fly on the shoulder” section of the experiment. I have

(16)

found that it is especially useful to conduct the interview when the concept that is being tested is still there and ready for use. The user generally has an easier time expressing what they are thinking because they can act and point out things in the prototype.

3.3 Iterative prototyping

In their chapter on “Play as Research” _{(Zimmerman, 2003, p. 176) Eric Zimmerman talks about} iterative design in game development as the constant need for playtesting and making sure that throughout the development of the prototype it is played and validated.

Chapter 2 page 1 of their book “Rules of Play: Game Design Fundamentals” _{(Salen &}

Zimmerman, 2004)_{they state: “Iterative design is a method in which design decisions are made} based on the experience of playing a game while it is in development.”

I used this iterative design method heavily during the creation of my prototype. With the time for this project being as short as it was I had to rely on myself to continuously playtest what I had made and compare this to what I knew as a gamer and what I knew as a designer. However, I also had to keep in mind the goals for my eventual playtesting session with other users and the things I wanted to get out of it. This makes the iterative process two fold. On the one hand I am continuously updating and testing the interaction to make it feel as right as possible to me. While on the other hand being iterative in the way the prototype facilitates the later playtesting.

3.4 Playtesting with users

In their paper _{(Eladhari & Ollila, 2012) discuss several different styles of playtesting. During the} creation of the prototype I used a modified version of their ad-hoc playtest variant. For my user playtest I went with a variant called a focus test. On page 402 it is described as follows: “In a focus test, a group of potential players are probed about their perceptions, opinions, beliefs, and attitudes toward the prototype.”

I felt that this was especially prudent because it allows me to not only use the users for their initial impression but also get their views on the future of the prototype and where they would like to see this concept go in the future.

(17)

4. Design Process

4.1 Research

During the research phase I looked for academic work done on video game controller

mappings. The goal for this research was to get a feeling for what research had already been done as well as to find academic articles to support my idea of analog based input controls in video games. Narrowing down the potential influences was not without difficulty, since there were many interesting threads to pursue in the literature. From neurology to design philosophy there are a lot of different fields coming together.

4.2 Re-framing

4.2.1 Original Framing

In the original framing I defined my research question as: “How can I integrate kinesthetic design principles in fighting games controls to allow for more mastery and granularity?” The context of this research question was focused around the genre of fighting games. This was originally chosen because the fighting game genre has a lot of broad bodily movements that are represented by binary button presses. This was an opportunity where I wanted to innovate or at least assess why this innovation had not taken place yet.

4.2.2 Reframing Experiment

In order to test and validate my gut feeling that just remapping the controls of a fighting game, to something else that is more natural and analog, will not work, I set up a small experiment. Using the game Skullgirls _{(RevengeLabs, 2012) and a tool called JoyToKey (JTKSOFT, 2017) I} remapped the controls to use the analog sticks to punch instead of the “normal” binary buttons on the front of the controller. I also mapped the character movement to the shoulder buttons since we are now using the analog stick for the fighting moves.

(18)

Figure 7. Overview of the mapping used for the experiment.

I tried this mapping and the original (default) mapping with six users. Three of the users played video games regularly, the other users only played when they were young or never played at all. The users played around in the training mode of the game where there was no immediate threat. That way there was less pressure on the more novice users to perform and it gave them space to explore the new mapping. I observed player behaviour and asked for their opinions and thoughts. I was particularly interested in the following questions:

● Which mode of mapping did you prefer? ● What made more sense to you?

● Why did it make more sense to you?

In the creation of the experiment I already ran into issues with our original framing. The fighting game as a genre is very rooted in its arcade origins to the point where there is no room for the game to facilitate analog inputs. This made me take some shortcuts in the new mapping such as leaving out the kicks that the player could perform in the original mapping.

The more expert users in the experiment had more trouble adjusting to the new controller mapping than the novice users. The expert users all told me that the normal controller mapping was their preferred one and that the troubles they had with the new one mostly came down to what they were used to from previous games. The experts all said that moving with the shoulder buttons required more attention than they would normally invest when moving the character with the analog stick. The feedback from the novice users was more disperse in opinion on the new mapping, with one user saying that it felt more natural but it did not add anything in the context of the game. Another user thought it would take longer to become proficient in the new mapping since you could no longer mash the buttons and win.

Looking at these results as well as my own experiences setting up the experiment I can

conclude that my initial idea of remapping controls to allow for more mastery in a chosen genre of video game was naive. The physical affordance of the controller cannot be directly imported

(19)

into a game without the game itself having mechanics that take advantage of these physical affordances.

In my new framing we are going to investigate if a game that has these affordances for analog inputs to control the characters body movements is more natural and if it allows for a higher amount of mastery for the user. I also decided to switch over to a Steam controller instead of the more standard Xbox controller that was previously used. The two touchpads on the Steam controller should create a better physical affordance for the user and break with the

preconceived notion on how video games should be controlled.

Figure 8. A Steam controller. The two black circles are its touchpads.

Figure 9. The back of a Steam controller with its shoulder buttons and triggers.

The research question could be reframed as follows: What gameplay mechanics are required for a fighting style video game to allow for analog input in its character movements and do these mechanics allow for more natural controls and mastery?

When I changed my framing the target group for my thesis also changed. Where before it was mainely meant as an improvement for the players of fighting games, now the project is more aimed toward game designers themselves.

4.3 Prototyping cycle

After the reframing I needed to create a prototype to validate and investigate my ideas. Because the idea of having analog input controls in video games is difficult to imagine the prototype had to be reasonably high fidelity. Without a base level of polish in the prototype it would be

unusable and the user would be unable to give their opinion on the aspects of the prototype that I wanted to test. Making video games takes a lot of time so I decided that instead of

implementing game elements I would just focus on the movement of the arms and some very basic character movement. These basic movements were continuously applied and then tried

(20)

by myself until I got them to a point where I felt that it was good enough to enable what I wanted to investigate.

4.4 User validation

The next part of the project was user validation. I was able to ask several people from Spelens Hus (A game society in Malmö, Sweden) to playtest my prototype. These users were all expert users. However since I wanted to playtest with novice users as well I ended up asking several of my classmates as well. The playtesting itself was done in several different locations. Having the prototype running on a laptop meant I was not bound to testing in a set location but could meet my users in places that were the most convenient for them.

5. Main Results and Final Design

5.1 Central Concept

My central concept revolves around a tech demo made in Unity 3D that uses the Steam controller to control a character by giving the user direct control over its arms. This setup I enabled me to see how people responded to such a control scheme and what mechanics of it did and did not work.

5.2 Qualities in use

The aim for this concept was to provide the users with a framework to create their own set of highly personalized moves and gameplay tactics. Through this I want to avoid the limitations that are normally set by developers and create a system where in-game motions can be related back to real life motions without the need for a cheat sheet or knowledge of which button does what.

5.3 Final Design

I created the prototype in order to validate my idea. It also served as a way for me to get a grasp on the gameplay elements needed to facilitate these analog inputs in a video game. The

prototype is very much a tech-demo, and not the prototype of a video game. It has no actual gameplay elements but shares a lot of mechanical similarities to “normal” fighting games. Such as a 2D environment with movement along a horizontal axis.

(21)

Figure 10. A screenshot of the prototype during playtesting.

5.3.1 Creation of the prototype

The prototype was made in the Unity game engine. _{(Unity-Technologies, 2018) It consists of a} single sprite sheet with the individual body parts separated. I decided to go with a bone

structure approach for the animations of the character. A game like Skullgirls, uses predefined sprite animations to give their characters the illusion of

movement. However for what I wanted to create, a prototype where the user has free control over the motions the character makes, having preconstructed animations was not going to work.

Using a library called Anima2D I created a humanoid skeleton that the sprites would be attached to. Moving the bones in the skeleton would move the sprites as well. This way the animation of the character is left to the computer that simulates the

skeleton. This allows the user to not be bound by the restrictions of having premade animations.

The integration of the Steam controller was a hassle but ultimately well worth it. The Steam controller is special in the sense that it does not behave like a standard xinput controller. Xinput is the Windows API that handles most of controller inputs on the platform. The Steam controller however is set up to be able to simulate several different input devices. Mice, keyboards

(22)

and other controllers can all be simulated using this controller. This did mean that it took a while to set it up in a way that Unity would be able to make use of the inputs. The Steam controller will think that any program that is not launched through the Steam distribution platform is a desktop program that requires normal mouse input. The solution in the end was to launch the Unity editor through the Steam distribution platform. This allowed Unity to be able to read the inputs from the Steam controller as its own game instead of thinking it is just a desktop application. The next step was to fine tune the controller itself.

5.3.2 Controller fine tuning

I tried several different settings for the touch pads. In the end I found a setting where the controller simulates two analog sticks on its touch pads with no deadzone and an exponential response curve.

The exponential response curve for the touchpads are important in order to perform small motions in the center of the touchpads, which is key to controlling the character well. With a linear response curve even the slightest twitch of the thumb on the touchpad would move the arm in a way that felt out of proportion for the movement that was performed. When touching on the outside of the touchpads the motions that are being performed tend to be bigger as well making the enlarged response from the exponential curve more appropriate.

The deadzone on a controller is a setting where the input value must exceed a certain threshold value in order to generate any output. This was especially useful in older controllers where there would sometimes be some drift, random values generated by inaccuracies in the controller, on the values the analog stick was sending back when in its neutral state. However, since I needed a lot of accuracy in these touchpads and the neutral state of the touchpad is simulated instead of physical, (like in an analog stick) I could get away with completely disabling the deadzone in the touchpads. This allowed the users to

make small movements in the center of the touchpad without weird sticking issues.

The values that I got from the touchpads were directly related to the Inverse Kinetic Points (IKP), on the skeleton (The blue circles in figure 11). When the user touches the

touchpad on the left side the IKP will move to the left as well. Moving the IKP updates the skeletal pose and moves the arm and hand that is linked to that IKP. However during creation of the prototype I found that the controller outputs values in a square instead of a circle. This made the movement of the arm feel very strange and unnatural. I wrote a

(23)

small script that clamped the output values of the controller to a circle and this greatly improved the movement of the arms.

5.3.3 Creation of the scenes

For the creation of the first scene I linked the arm IKPs to the two touchpads on the controller and mapped the movement of the character to the trigger buttons on the back of the controller. The second scene had the character use a two-handed sword as a weapon. The bone structure of the character had to change in order to facilitate the swords movement. I extended the right arm by adding another bone onto it and mapped the sword sprite onto that bone. While trying out the clamped input with the sword I found that it felt awkward compared to the fists. The movements became very rounded and it felt like there was no way to angle the blade. I reverted it back to the square input and this felt a lot better, however the skeleton is still clearly not optimised for this type of weapon. The control over the sword was mapped to the right touchpad on the controller. I picked the right touchpad because this follows the standard controller

convention of putting the fighting moves on the right side of the controller. The triggers were used for movement in this scene.

The third scene was a copy of the second one except that I now mapped the movement of the character to the left analog stick and touchpad. By using both the pads I would be able to test the perceived difference of using an analog stick or a touchpad.

I set up the scenes in this way because it allowed me to test multiple aspects at once. The idea that two axis of play is the most a person can comfortably handle. The difference between using one or two hands for character movement. And how much of an influence the learned behaviour from other games is when approaching more conventional controls.

5.4 User test

For this user test I set up the prototype on both my laptop and my desktop computer at home in order to be able to visit potential users as well as to have them come over to me. The users were presented with a window on which the prototype was running and a Steam controller to control the demo. The screen interaction and spoken audio was recorded using OBS studio software. _{(Bailey, 2018) I also made notes of users behaviour.}

The user test itself was split in several sections. The first part was comprised of some introductory questions about the users gaming behaviour. How is their current gaming behaviour? What was their gaming behaviour like in the past? What type of video games did they play?

After this the users got an introduction to the prototype application itself and the controller. I explained the parts of the prototype that I was interested in. This being the motion that you

(24)

make on the controller and the output motions that the character can perform. I also explained what parts of the controller did what, without telling the user how the underlying system worked.

Figure 13. A picture of a playtest.

Next I evaluated three different scenes. These scenes were evaluated by letting the user get to grips with the controls for a couple of minutes before asking them to perform certain tasks. Such as moving backwards while swinging both arms. Putting up a guard in front of the characters face or stabbing downwards with the sword.

In the starting scene the user has control over both arms in a fist fighting style. The movement of the character itself was controlled by the triggers on the bottom left and right side of the controller. The two touchpads on the front of the controller were bound to one individual arm (left to left and right ot right). When moving the thumbs to the side of the touchpads the character on the screen would stretch their arms in a similar fashion.

The next scene had the user use a similar control scheme but instead of controlling the two fists with the two touchpads they used the right touchpad to control a two handed sword (and the limbs attached to it). The movement of the character was still the same with it being mapped to the trigger buttons on the controller.

The final scene was identical to the previous one besides the fact that the movement of the characters whole body was now mapped to the left analog stick and touchpad instead of the triggers on the back. This is approaching a more conventional controller setup where the left part of the controller is used for movement and the right part is used for actions in games.

(25)

At the end of the scenes I asked the users about their overall feelings on this type of control scheme. I also asked them how they would see themselves playing a game like this in the future when the design had been perfected. This time was also used for further discussion of things that came up during the validations of the scenes.

One of the playtests can be viewed here: _{http://host.pimostendorf.nl/playtest1.mp4}

5.4.1 User test results

From the user test there were several actions and things noted by the users that were of particular interest.

The animations need to make sense from an anatomical perspective. This was a point that was brought up in every user test. With the movements of the character being completely reliant on the calculations of the bone structure of that character, there was the possibility that when the user would act on the touchpads in a certain way, the character would make moves that would be anatomically impossible for a humanoid to make. The degree in which this bothered the users varied but all indicated that it influenced their experience in some way. Ranging from being unable to take the prototype seriously to not being able to accurately achieve the motion that they wanted to achieve.

There was a preference for positioning over timing based gameplay controls. A lot of the users indicated that they were intrigued or excited by the fact that the prototype relied on positioning instead of timing for its core gameplay controls. That is, instead of remembering what button to press, the position of your thumbs on the touchpad was what mattered. Some of the users also indicated that they liked the fact that this seemed to slow the game down in speed, where accuracy seemed more important than raw speed.

When asking the user to perform tasks like moving backwards slowly while jabbing with both fists they felt like it was too hard to perform. Putting in this extra axis of control seems to be too much for most users. Users would either move backwards quickly by just holding down the trigger instead of fingering it or they would forget to move one or both arms. While asking them to do the same exercise using the sword scene there was no issue. The users would be able to move slowly and in a controlled manner while still able to perform tasks with the sword. This control was increased when using the touchpad for the movement instead of the triggers. In the case of the more practiced users they would prefer the analog stick over the touchpad. All of them indicated that this was because it was a learned skill for them and it brought the controls closer to the controls that they were used to.

The more novice users expressed that they felt that this method of control was more intuitive. The fact that there were only two or three points of contact on the controller through which you

(26)

could control most of the characters movements really appealed to them. They told me that this meant that there was no need for them to remember what actions were mapped to what button. Users that played a lot of video games have a harder time adjusting to different movement schemes, but pick things up faster as well. Having played other video games made it harder for experienced users to adapt to the new control scheme but they also found the different

affordances in the prototype more quickly than the other users while being more vocal about the perceived strangeness of the controls. However all of them claimed that this strangeness came from the conventions they learned from previous games.

The touchpads seemed to perform well and give the users more control over their movements. There was the suggestion of having the touchpads be even more concave to give the users a better sense of where they are on the touchpad. The users liked how the size and the lack of pushback of the touchpads gave them more control over the movements of the character on the screen.

6. Conclusion

Let's look back on the research question: What gameplay mechanics are required for a fighting style video game to allow for analog input in its character movements and do these mechanics allow for more natural controls and mastery? There are several elements in this question that had to be answered.

From our user test as well as the research done in neurology on mirror neurons we can

conclude that an analog control scheme is indeed more natural and intuitive for the more novice user to use. Expert users that played a lot of video games felt that the controls were strange in the beginning but quickly picked up on the way the system worked after that. This initial feeling of strangeness was caused by the learned conventions that they carry with them from other games. Mastering the analog control scheme will require less cognitive memory in the sense that the user will no longer be required to remember and internalise what button presses perform what action. Instead the user will be required to explore their fine motor movements with their thumbs and internalise motions instead. This effectively cuts out the step where the user would have to get to grips with what action is on what button and can immediately go into exploring the movements and their effects on the screen. This makes the game easier to just pick up and play. Deep mastery of the controls like exhibited in the example games earlier in this paper will rely mostly on the game itself. Something that was not able to be tested in this project.

However, in order for this analog control scheme to make sense the games context has to match and facilitate this control method. From the experiments conducted I can determine several key points that designers would have to keep in mind creating their games.

(27)

You can not exceed the two axis of play. Having the user perform fine motor actions on three axis of play makes the controls hard to the point where many users cannot perform the tasks that were given to them. Neurology confirms this as well as an inherit biological trait in the human brain. However it seems that having two fine motor actions with a holding action does work. This would allow you to integrate things like modifiers or filters for your axis of play that could give the players and even bigger range of motion.

The pace of your game has to match the fact that you are now designing a game that relies on positioning and not on timing. While timing based gameplay can be very fast, position based gameplay needs to be more thought out and methodical in order to prevent it from becoming a so called “spam fest”. The “spam fest” refers to a scenario in a video game where the easiest way to win is to rapidly execute the same moves over and over effectively not giving the other player or AI room to react. This is generally considered by both gamers and developers to be undesirable since it removes all nuance from the gameplay.

While developing the prototype there was a lot of tinkering going into making the Steam controller work together with Unity. In the end this was very worth it as the tinkering and exploration of the controller as a design material opened up new design opportunities and facilitated ideas that would be hard to realise on a standard game controller. As a game designer it can be hard to step away from controller conventions because in order to maximise your profits your game will need to be able to be played on as many platforms and with as many input devices as possible. However if you want to create something that breaks with convention and explores new ground your input device might have to break convention as well.

All in all I have been able to find that an analog control scheme with positioning gameplay mechanics is an intriguing field for exploration and yielded positive results in playtests. It

however comes with some caveats and requires the game designer to be aware of the changes in game context that this type of control scheme brings with it.

7. Discussion

A design project like this on such a short scale brings along a lot of constrictions and limitations. Even with the positive results from playtests and interesting prototyping iteration process this project produced more questions and exciting new possibilities for me than it answered. As I lacked the time to properly address every question that popped up.

A point that was brought up during one of the playtests was the idea of tutorialisation. While standard tutorials for games that have more preprogrammed moves are pretty straight forward it provides me with an interesting conundrum in the case of the analog input controls. Since I want the user to feel free to experiment, just teaching them moves, the way most games do it

(28)

their own moves. The trick would be to teach the users the concept of using the analog inputs without resorting to examples but still make it understandable.

The application of an analog control scheme in a fighting style game made a lot of sense when looking at it on paper. However, there are a lot of different genres of video game as well as games that straddle the line between the genres. How relevant this type of input controls will be in these other genres is something that has to be experimented with.

There is also the question about breaking the conventions that exist within the mainstream gaming industry. The traditions and conventions especially in communities that are established around a specific genre can have a very hard time adapting to new design conventions that break with those traditions. An excellent example of this is Dota 2 that still supports the original legacy keybinds that the Warhammer III mod demanded of the players to learn due to technical limitations. A portion of Dota 2s playerbase still swears by the old legacy keybinds. This

adherence to tradition could impede the rate of acceptance of this new controller scheme. My solution would be to integrate the analog system slowly, starting by replacing one axis of play and over the span of several games move to a more full analog representation making the users acclimatise over time. Another large hurdle to the introduction of an analog controller scheme is the commercial aspect. Most major game developers do not develop gaming hardware. This limits new groundbreaking developers of analog controller scheme to either conform to what is available or severely hamper their potential sales when picking an option exclusive to a

controller that is not in widespread use yet.

The physicality of the controller is also something that warrants further investigation. While the Steam controller already breaks with the standard controller design by replacing two of its inputs with touchpads, there is obviously room for improvement. A user suggested that making the touchpads concave inwards would allow the user to get a better idea of where they are on the touchpads without having to look down to see where they had placed their thumbs.

Improvements like this could make a control scheme like this even more viable.

One thing that this project did not address was the integration of actual gameplay elements. The rules, conflict and results as described by Katie Salen that make the game a game. Since this prototype only touched upon a very small subsection of the rules that make up a game, there are still a lot of things to fill in, in order to make it into a full game. I have no doubt that these elements will have to be specially designed for this new input system as well. I will probably further develop the prototype to the point where it's playable as a game to see how these elements will pan out. Because instead of sating my curiosity this project only made me more curious!

(29)

8. References

Apok, coveredinbees, Tairoon, FireSwan, Lockness, BRODERICK, & Senhs. (24 april, 2018). Speedrun. Retrieved 24 april, 2018, from

https://www.speedrun.com/Getting_Over_It_With_Bennett_Foddy

Bailey, H. “jim.” (2018). Open Broadcaster Software (Version 21.1.0) [Windows]. Retrieved from https://obsproject.com/

Dourish, P. (2004). _{Where the Action is: The Foundations of Embodied Interaction}. MIT Press. Eladhari, M. P., & Ollila, E. M. I. (2012). Design for Research Results. _{Simulation & Gaming},

43(3), 391–412.

Farrow, R., & Lacovides, I. (2012). “In the Game”? Embodied Subjectivity in Gaming Environments. _{The Philosophy of Computer Games Conference, Madrid 2012}.

Farrow, R., & Lacovides, I. (2013). Gaming and the limits of digital embodiment. _{Philosophy &}

Technology, ₂₇(2), 221–233.

(Firm), I., & IDEO.org. (2015). _{Field Guide to Human-Centered Design}.

Foddy, B. (2017). Getting Over It with Bennett Foddy (Version 1.0) [Windows]. Retrieved from http://store.steampowered.com/app/240720/Getting_Over_It_with_Bennett_Foddy/ Griffin, S. N. (2005). Push. Play: An Examination of the Gameplay Button. In _{Proceedings of}

DiGRA 2005 Conference: Changing Views – Worlds in Play. Georgia Institute of

Technology.

JTKSOFT. (2017). JoyToKey (Version 5.9) [Windows]. Retrieved from _{https://joytokey.net/en/} Marc Breedlove, S., & Watson, N. V. (2013). _{Biological Psychology: An Introduction to}

Behavioral, Cognitive, and Clinical Neuroscience. Sinauer Associates Incorporated.

McGloin, R., & Krcmar, M. (2011). The Impact of Controller Naturalness on Spatial Presence, Gamer Enjoyment, and Perceived Realism in a Tennis Simulation Video Game. _Presence:

Teleoperators and Virtual Environments, ₂₀(4), 309–324.

Miyamoto, S., Tezuka, T., Asuke, S., Imaizumi, M., & Matoba, T. (2006, December 8). Wii Sports. Retrieved from _{https://www.nintendo.nl/Games/Wii/Wii-Sports-283971.html} Nixon, M., & Bizzocchi, J. (2013). Press X for Meaning: Interaction Leads to Identification in

(30)

RevengeLabs. (2012). Skullgirls (Version 1.0) [Windows]. Retrieved from _{http://skullgirls.com/} Rogers, R., Bowman, N. D., & Oliver, M. B. (2015). It’s not the model that doesn’t fit, it’s the

controller! The role of cognitive skills in understanding the links between natural mapping, performance, and enjoyment of console video games. _{Computers in Human Behavior}, ₄₉, 588–596.

Salen, K., & Zimmerman, E. (2004). _{Rules of Play: Game Design Fundamentals} (pp. 414–415). Massachusetts Institute of Technology.

Skalski, P., Tamborini, R., Shelton, A., Buncher, M., & Lindmark, P. (2010). Mapping the road to fun: Natural video game controllers, presence, and game enjoyment. _{New Media & Society}, 13(2), 224–242.

Tholander, J., & Johansson, C. (2010). Design qualities for whole body interaction. In Proceedings of the 6th Nordic Conference on Human-Computer Interaction Extending

Boundaries - NordiCHI ’10. https://doi.org/_{10.1145/1868914.1868970}

Torn-Banner-Studios. (2012). Chivalry: Medieval Warfare (Version 1.0) [Windows]. Retrieved from _{http://tornbanner.com/chivalry}

Unity-Technologies. (2018). Unity (Version 2017.4) [Windows]. Retrieved from https://unity3d.com/

Urgesi, C., Candidi, M., Fabbro, F., Romani, M., & Aglioti, S. M. (2006). Motor facilitation during action observation: topographic mapping of the target muscle and influence of the

onlooker’s posture. _{The European Journal of Neuroscience}, ₂₃(9), 2522–2530.

Zimmerman, E. (2003). Play as Research. In B. Laurel (Ed.), _{Design Research: Methods and}

(31)

9. Appendix

9.1 Github repository for the prototype

The code and project files for the prototype can be found at the link below. Please note that the thesis branch of the repository will contain the prototype in the state as described in this paper. The master branch may deviate from the prototype as described.

https://github.com/SethSenpai/mekanism/tree/thesis-branch

9.2 Images

Generic Fight stick [Online image]. Retrieved May 22, 2018, from

https://www.amazon.com/Fighting-Stick-V3-PS3-playstation-3/dp/B002QXMV9M Getting over it [Online image]. Retrieved May 21, 2018, from

Analog input in gaming: Investigating the possibilities of new controller affordances in video games