Interaction Concepts in a Rotary Haptic Device for Browsing Throug Lists in

(1)

M A S T E R’S T H E S I S

2006:315 CIV

TOBIAS SVENBERG

Interaction Concepts in a Rotary Haptic Device for Browsing Throug Lists in

an Integrated Infotainment System

MASTER OF SCIENCE PROGRAMME Media Engineering

Luleå University of Technology Department of Human Work Sciences

Division of Engineering Psychology

2006:315 CIV • ISSN: 1402 - 1617 • ISRN: LTU - EX - - 06/315 - - SE

(2)

Preface

The spring of 2006, during which I worked with this master thesis, was a very educational and fun few months for me. Even though it was at times very frustrating work programming the test environment and using a prototype rotary haptic device every hurdle I passed was riddled with joy and satisfaction.

I would especially like to thank Annie Rydström, my supervisor, who guided and supported me throughout the course of the work and kept me from making to many mistakes. Annie, you did an excellent job and I consider my self truly lucky to have had you as my supervisor. It is hard to believe I was your first “victim”.

Everyone at Volvo Cars department 94331 and my fellow master thesis workers at Open Arena Lindholmen, thank you for all the help, support and answers to my many and sometimes silly questions. Thank you all for a great few months, I can only hope that you enjoyed my company as much as I did yours.

Christer Johansson at Volvo technologies, thank you for helping me with the programming part of my master thesis and for letting me use your hardware communication script.

______________________________

Tobias Svenberg

(3)

Abstract

The purpose of this thesis was to design and evaluate different interaction concepts for navigating through long lists of information in an integrated infotainment system. The interaction was made with a rotary haptic devices . Does the haptic feedback provided through the device affect user performance? During the course of this master thesis a test environment was built and designed to examine just that. Through preliminary investigations four interaction concepts were designed for the test. In the first concept a haptic click was provided for each list entry, in the second no clicks were provided, the third concept was designed as a jogg-shuttle and the fourth had an acceleration script. The test environment was created using Macromedia Director and a rotary haptic device supplied by ALPS. A usability test was carried out in May 2006 using 12 participants and the

“Labb-Jakob” driving simulator at Open Arena Lindholmen. The participants got to solve tasks while driving on a simulated road. The list lengths in the tasks varied between 10 and 400 steps The participants performance was measured in time, turn over errors and workload. In between each test round the participants were asked to fill in a NASA-RTLX form, a form designed for measuring the subjective workload of the participants.

The results showed that participants performed best with concept 1 in shorter tasks, and performed best in longer tasks with concept 4. The participants performed significantly worst through out all task ranges with concept 3. When looking at turn over errors, that is the number of times per task the target is passed, participants performed significantly better with concept 1 than with concept 3 and better but not significantly so than the other two interaction concepts. The NASA-RTLX forms revealed no significant differences between the four interaction concepts.

The results of the test show that interaction concepts in rotary haptic devices can indeed have an effect on user performance in the human-machine interaction both to the better and to the worse.

An interaction concept such as concept number 4 which performed well in the test can enhance user performance. Based on the results of this master thesis the recommendation is to investigate an interaction concept with clicks and an acceleration script. This would hopefully result in an interaction concept with the precision of concept 1 and the speed of concept 4.

(4)

1 Introduction

The introduction section of this report covers the background to and the definition and delimitations of this master thesis.

1.1 Background

More and more comfort systems such as telephone, navigation and DVD has become available in the driver environment of today and thus an extensive change in the user interface follows. Several car manufacturers have developed integrated one-display systems where all the functions are controlled by just a few buttons or a touch screen. It is a challenge to develop a user interface that can be used without compromising the drivers' ability to safely convey the vehicle. In order for the driver to keep eyes on the road as much as possible, car manufacturers are trying to integrate the use of other senses in the interaction with a system. In this thesis the use of touch is investigated, how interaction concepts in a rotary haptic device can be used to facilitate the human-machine interaction when navigating through long lists of information.

1.2 Volvo Car Corporation (VCC)

Volvo was founded 1924 by Assar Gabrielsson and Gustaf Larson and only three years later the first series manufactured car, the ÖV4, rolled out of the factory in Gothenburg. VCC has strong ties to the city of Gothenburg and still today, over 80 years after Volvo was founded, the main office still resides there even though ownership of the VCC shifted in 1999 to the USA based Ford Motor Company. Today VCC is a well known car brand with a solid reputation of quality and safety and Volvo cars are being sold almost everywhere in the world.

1.3 Purpose

The purpose of this thesis was to study how different interaction concepts in rotary haptic devices can be used to improve user performance in the user-interface interaction for navigation through long lists of information in an infotainment system.

1.3.1 Aim

The aim of this thesis was to compare different interaction concepts in rotary haptic devices, for navigating through long lists of information. Also to determine what interaction concept, if any, is the best. The system must be adapted to the driver environment and take in concern such issues as usability, mental workload and safety. The interaction between the driver and the system must not compromise the drivers' ability to safely convey the vehicle.

(7)

1.3.2 Framing of the problem

How can interaction concepts in a rotary haptic device be used to ease navigation through large lists of information? More specified;

• How can interaction concepts in a rotary haptic device be used to enhance performance in the human-machine interaction?

• How can interaction concepts be used to decrease the amount of search time in a structured search of a long list of information?

1.4 Delimitations

This thesis only covers how different interaction concepts can be used in rotary haptic devices to improve the performance in the user-interface interaction for browsing lists in integrated infotainment systems.

There are also delimitations imposed by the hardware available since the rotary haptic device it self is only a prototype model. All interactions are made with a rotary haptic device that can be turned and pushed. The rotary haptic device it self is also technically limited by many factors such as slow communication speeds, slow updating times and limited number of feedback possibilities at any given time. These technical limitations made it impossible to examine all interaction concepts that where proposed for the test.

(8)

3

2 Theoretical Background

This section covers the vital theory concerning the areas which are important to this thesis such as human-machine interaction, human information processing, attention, haptics, graphical user interfaces, attention, perception, intermodal relations and structured search.

2.1 Human-Machine Interaction (HMI)

HMI regards the research of how humans interact with machines, and how user interfaces can be designed to make the machines more usable to humans. With increasing complexity and technical advancements the need for understanding how humans perceive and interpret information from their environment and how humans based upon that information make decisions increases dramatically. Solid understanding of how humans interact with user interfaces can be used to optimize the interaction by primarily making the best possible use of human strengths.

There are countless examples of products that have bad user interfaces, ranging from high-tech products such as VCR players, computers and cell phones to basic everyday objects such as doors.

A classic example of bad user interface design is that if something as basic as a door needs to have the words “push” or “pull” written on the handle to ensure that the users operate the interface correctly, the interface needs to be re-designed (Norman, 1990).

According to Wickens and Hollands (1999) the motivation for the development of HMI is a result of practical needs, technological advancements and linguistic developments. The need for understanding how humans interact with machines was revealed during World War II during which emphasis was placed on training humans to interact with machines such as fighter planes.

But there was plenty of proof that this approach with highly trained operators was not working, planes crashing into the ground for no apparent reason being the most obvious. To solve this problem, experimental psychologists were assigned to analyze the user interface and give recommendations for improvement. The psychologists approached the problem from what today is an engineering psychology point of view basing their recommendations on psychology and cognitive science.

2.2 Information Processing

Information processing is a term used to describe the entire process from sensory stimuli to a response based upon those sensory stimuli. There are many information processing models such as biological reductionism, modularity of perception and direct perception (Coren, Ward and Enns 2004) though in this thesis the model of choice is the one described in Wickens and Hollands (1999) and is mentioned in Ward and Enns (2004), where human information processing relies on a levels-of-processing analysis. In this approach each stage of sensory processing, from the moment the stimulus is registered until a response based on that stimulus has been made, is systematically analyzed. As shown in Figure 1 human information processing is divided into several stages. The individual steps will be covered more in detail in sections 2.2.1-2.3.

(9)

Figure 1. Christopher D. Wickens information processing model

2.2.1 Short-Term Sensory Store (STSS)

In order for us to perceive an event or information in our environment the stimuli must first gain access to the brain, you must receive stimuli of at least one modality regarding an event in order to respond to that event (Wickens and Hollands, 1999). Each modality has its own STSS, and thus immense amounts of information pass through the STSS most of which is never recoded. The information stored in STSS is not immediately discarded; it is stored for approximately 500 ms for visual stimuli and 2-4 seconds for auditory stimuli (Wickens and Hollands, 1999). This is why we can recreate the information a stimuli carries even if our attention was not directly aimed at the source of that stimuli when it arose.

2.2.2 Perception

In order for sensory stimuli to have meaning the information must be interpreted. This interpretation can basically be divided into bottom-up processing and top-down processing.

Bottom-up processing is also known as data driven processing and is driven by sensory input while Top-down processing is driven by prior knowledge (Wickens and Hollands, 1999). The selection and interpretation that takes place within the perception process can rely on both internal and external factors. Internal factors are, are as the name suggests, factors that comes from within such as our needs, experiences, expectations and emotions. The external factors are however reliant on the source of the stimuli such as size, strength, frequency and intensity (Wickens and Hollands, 1999; Coren, Ward and Enns, 2004). How we perceive our surroundings differs between

(10)

5

different modalities. Since haptic and visual perception is the two modalities most important to this thesis only these will be covered in depth (see section 2.4 and 2.6).

2.2.3 Working Memory

Working memory is the memory we use all the time to get by. Problem solving, decision making and action preparation are all carried out in the working memory (Wickens and Hollands, 1999).

Working memory is a short-term attention demanding storage area that we use to temporary store new information under very short periods of time (Wickens and Hollands, 1999). This memory is very limited in storage capacity and time only capable of storing 7±2 chunks of data for less than 10 seconds (Miller, 1956). Besides these limitations working memory is also very fragile and very susceptible to interferences.

Since working memory is not designed to store any information for longer durations of time the information has to be coded into the Long-term memory in order to not be lost, this is the process we call learning (Wickens and Hollands, 1999).

When designing a user interface it is important that the amount of information that the user need to store in his or her working memory does not exceed the storage capacity and storage time capacity.

If there is a need to display much information there must be a system to support the users working memory.

2.2.4 Long-Term Memory

As Figure 1 shows, long-term memory is used both before and during the perception stage. Our long-term memory is used in these stages as a registry for comparing stimuli with previously encoded memories. This is what allows us to recognize, for example, the sound of a dog barking without having to spend time figuring out what is the cause of that auditory stimulus. When information is stored in the long-term memory through learning, rehearsal or training it can take on a number of forms. According to Wickens and Hollands (1999) and Danielsson (2001) certain memories become procedural, how to do something and other memories become declarative which is knowledge of facts. There is also the distinction between semantic memory that is general knowledge, such as word meaning, and episodic memories which is the memory of specific events.

According to Wickens and Hollands (1999) and Danielsson (2001) long-term memory, unlike the other types of memories, has little or no limitations as to storage capacity and storage time and at little decay. There are however other factors imposing limitations upon the long-term memory such as forgetting. Forgetting often visualizes itself as losing a part of or an entire memory or failing to recall something in the right order. A common reason to forgetting is failing to properly code the memory from the working memory to the long-term memory to begin with. Interference is another cause of failing to recall a memory. Things that can interfere with long-term memories are similarities, where internal similarities gives rise to problems recalling, proactive interference where recalling is interfered by information presented earlier than that which is being recalled and finally retroactive interference where recalling is interfered by information presented after that which is being recalled. Another reason to forgetting is the lack of memory cues, which is something that aids the recollection of an event or a task.

In interface design it is important that the learning process, that is the coding into the long-term memory, is thought of in the creation process. Avoid factors that give rise to interferences such as

(11)

similarity, make good use of memory cues and finally utilize effective memory coding methodologies such as repetition and association.

2.2.5 Response Selection and Execution

According to Wickens and Hollands (1999), when understanding of a given stimuli is achieved often a response based upon that stimuli is selected and executed. The selection of an action can be interpreted as what we intend to do based upon the information of the stimuli, and the execution is what actually was done. Even though the selected action is the correct one the execution may not achieve the desired result since the execution of an action is often more complicated than the selection of that action.

2.2.6 Feedback

As shown in Wickens and Hollands (1999) and Norman (1990) feedback, when it comes to interface design, is information sent to the user by the interface about what has been done and what it has accomplished. As shown in Figure 1 feedback is the reconnecting loop from execution to STSS. This feedback can come in many forms and modalities such as a tone when you press a telephone button or the feel of the lock on a hatch when it closes. It is very important that the feedback is correct, relevant and that its timing is logical, which in many cases is immediate.

The haptic knob often integrated in new infotainment systems is a good example how, in this case multimodal, feedback can be used to ensure the user that the action taken has a result. A turn of the knob clockwise you feel a snap and the graphical user interface (GUI) moves the same amount of snaps clockwise as you feel in the knob. A plausible error in feedback to this example is that you would feel a random number of snaps for each step GUI make, or that the GUI rotates in the opposite direction of the knob.

2.3 Attention

Even if vast amounts of information pass our STSS we still have problems with missing information if we get distracted. This is the result of human attention, described by Wickens and Hollands (1999, page 69) as "One of the most formidable bottlenecks in human information processing". Human attention can be divided into:

• Selective attention

• Focused attention

• Divided attention

According to Wickens and Hollands (1999), Coren, Ward and Enns (2004) selective attention is just as the name suggests selective, we chose what information we want to perceive. This is both a strength and a weakness in human information processing. In today's society when the amount of information is so vast we need to filter down the incoming information to that which is necessary in the given situation (Danielsson, 2001).

However, we sometimes select information sources that are inappropriate for a situation and the user focuses his or her attention on the wrong thing (Wickens and Hollands, 1999). Choosing to pay attention to manipulating the car stereo instead of keeping the attention on the road is a classic

(12)

7

example. This phenomenon is referred too as cognitive tunneling and has been the reason of many accidents.

Focused attention describes situations where we need to focus on a single source of information and ignore distractions. That is, when we are actively focusing on a source of information. To focus on a single source of information is very tiring for humans and as a result we tend to avoid it if we can. According to Wickens and Hollands (1999) divided attention is more natural to us and is thus the state of attention we use the most. As with selective attention there are both strengths and weaknesses. The strength of divided attention is that we can perform several tasks simultaneously.

This ability is especially useful in situations that are highly demanding such as driving a car in rush-hour traffic and talking to a passenger at the same time. We can parallel process information and thus be able to process a lot more information at a single time. However when a need for divided attention is forced upon us rather being optional we have difficulties coping. These difficulties are not limited to any single modality when divided attention is required within that modality; we have difficulties with dividing attention within a single modality for all our senses.

Divided attention is a lot easier for us when we need to divide our attention between different modalities such as auditory and visual information. Dividing attention is also made easier for us when one or more of the processes needing attention are automatic rather than controlled. An automatic process is a process that takes less attention to perform, such as typing for a seasoned secretary.

2.4 Haptics

Haptics is defined as the science of touch (www.dictionary.com) and derives from the Greek word haptikos, from haptesthai, which means to grasp or to touch. According to Hatwell, Streri and Gentaz (2000) the sense of touch is different from other senses such as hearing and vision in two very distinct ways. Firstly it is contact dependant; the body needs to be in contact with an object in some way in order for us to feel it. And secondly the size of the receptors; the receptors that are affiliated with the sense of touch covers the entire human body. Also, touch is sequential in the meaning that we gather one piece of information of a given object at the time. This does not mean that it is temporal such as our auditory sense since auditory stimuli carries a special meaning such as the words in a sentence and changing order distorts the stimuli. The order in which haptic information about an object is gathered is not done in an imposed order since the order in which the information is perceived makes little difference when it comes to exploring the characteristics of an object using touch.

Since the main haptic sensory receptors of adults without disabilities are the hands, touch can provide spatial information such as localization, direction and distance. This spatial information is largely redundant to vision (Hatwell, Streri and Gentaz, 2000). The intermodal coordination between vision and touch will be covered more in depth in section 2.1.6. Haptics can be divided into tactile and kinesthetic information. Tactile information regards such information that is perceived by the skin such as temperature, texture, pain and vibrations (Hatwell, Streri and Gentaz, 2000) and (Heller and Schiff, 1991). Kinesthetic information concerns bodily position, weight, movement of the muscles, tendons, and joints. Haptic perception, what we feel when we actively manipulate an object, is actually a combined image of many different things.

(13)

Figure 2 Manual exploration, Ledeman and Klatzky (1987)

According to Hatwell, Streri and Gentaz (2000), Heller and Schiff (1991) and Colwell, Petrie and Kornbrot (1998) we perceive the object through both the position of our fingers and muscles as well as the skin of our fingers. We gather haptic information about an object through manual exploration by touching and squeezing an object (Figure 2). This manual exploration allows us to gather information about texture, hardness, temperature, weight, shape but also vibrations. Manual exploration is active haptic perception where haptic information about an object is actively gathered. Passive haptic perception on the other hand is haptic information that we do not gather actively such as the pressure exerted from a chair onto a person sitting in that chair.

2.4.1 Haptic Devices

Haptic devices are devices that convey haptic information to the user through a haptic interface.

When it comes to haptic devices there are two types: devices that provide kinesthetic feedback, also known as force feedback devices, and tactile feedback devices. Kinesthetic devices are probably the most common considering their popularity in the gaming industry. Most modern game consoles support kinesthetic feedback through "rumble packs" that use rotating weights to convey feeling from a game. Kinesthetic devices often use small servos to create resistance or pull in a lever or a similar device. Tactile devices are devices that convey feelings such as heat, pressure and texture to the user. The use of this type of devices has been almost limited to convey the sense of contact with a virtual object in virtual environments to a user.

As suggested above the most common use of haptic devices are computer and or entertainment related. There is however other important fields of application for haptic devices. Medical

(14)

9

applications such as remote surgery are under development and have been tested, and there are numerous applications to aid disabled persons such as Braille computer interfaces.

2.4.2 Haptics in Driver Environments

According to Steenbekkers and Van Beijsterveldt (1998) touch is a very suitable modality to incorporate into secondary car functions controls. This because the sense of touch does not degrade with age as vision and hearing does. Since the amount of elderly drivers increase rapidly it would be logical to further the use of haptics in driver environments. In the BIONIC project, Porter, Summerskill, Burnett and Prynne (2005) investigated the use of haptics in driver environments using both visually impaired and sighted people in their research that resulted in a series of guidelines in eyes-free design of secondary driving controls. Although the alternative controls designed in the BIONIC project are very different from the ones available to this thesis work there are a few guidelines that can be related to the single haptic rotary device controlling the system of this thesis work.

1. Be clear and distinct. Provide each group of functions with a distinct feel and avoid tactile information on the surrounding surfaces that can be confused for a control.

2. Provide redundancy. Redundant information, both tactile and visual, aids in human decision making. By granting several cues of many modalities that point to the same response makes the human-machine interaction more efficient.

3. Provide tactile feedback. Correct and well-timed feedback of control operation is critical to human machine-interaction to ensure the user that the manipulation carried out has given results and what results that have been achieved.

The fact that investigations have proved that the use of haptic devices in user interfaces decrease user errors, frustration and mental workload makes the use of touch ideal for driver environments.

Not only is it non-dependant upon age (Hatwell, Streri and Gentaz, 2000), it also provides another mode of information to the driver, information that allows for more eyes on road time.

2.5 Graphical User Interfaces (GUI)

A Graphical User Interface (GUI) is the user interface of software displayed on a screen such as a computer program or the graphics displayed on the screen of an ATM machine, and there are rules as to how it should be designed. A good GUI is primarily easy to use, practical, efficient and aesthetically appealing. There are many similarities between GUI design and the design of hardware user interfaces, UI design. Many issues that need to be taken into concern in UI design such as the Gestalt laws, structured search (see section 2.8) and human strengths and weaknesses also need to be considered when it comes to GUI design. There are also additional rules and principles applied to GUI design both traditional print and moving images theory as well as design rules for products play a role in GUI design.

According to an article by Cortes (1997) the use of affordances, metaphors and manipulations are very important in GUI design. An objects affordance is what the physical look tells the user how it can be manipulated. If an object has multiple affordances or affordances that give the wrong cues to the user interaction errors are likely to occur. Therefore it is very important in GUI design, as well as in UI design, that the affordances of an object are correct to minimize the amount of user interaction errors. In successful GUI design the affordances of real world objects are exploited to

(15)

make best use of the users prior knowledge of real world objects to further the human-computer interaction, this is done through analogies.

According to Cortes (1997) manipulations refer to how we in the real world actively manipulate objects in our environment. The documents do not magically appear in the desired folder, they have to be put there. If active manipulation is available in a GUI it greatly improves the user interaction.

Cortes (1997) set up principles of good GUI design that can be summarized to:

1. Consistency. A consistent design through out the system and between systems. Icons, choices, feedback, design needs to be consistent within the system. This is important and aids the user to faster achieve a basic understanding for the system. Do not break established GUI concepts such as menu placing and established icons.

2. Limit the possibilities of error when dealing with input. Limit the input choices by fading out those choices that are not available at the given time and use forms if you need to have encoded data such as dates or social security numbers.

3. Provide well-timed and relevant feedback in adequate amounts. The system must provide feedback to the user so that the user is knows if a choice or action has resulted in any events. It is important that the user knows which steps, especially critical ones that have been performed.

4. Use strong metaphors. When metaphors are used correctly even modal behavior that forces the user to perform tasks in a special order modifying the expected responses from the user, can become subtle and feel natural as a consequence of the metaphor. Also, by using strong and efficient metaphors the user will instinctively know what the correct course of action is since the affordances are directly picked from the real world.

5. Make the GUI self-evident. A good measurement of a great GUI is that which novice users rarely or never need to use help functions or refer to manuals. The goal is to create a GUI that is in no need of an explanation, where the interface has become transparent, so the user can focus on the task at hand instead of the GUI.

6. Allow safe exploration. Humans are naturally curious and this can be utilized to further the users’ interaction with the system. If the system invites to and rewards exploration of the system the user will experience both the thrill of discovery and the thrill of reaching the desired result unaided. Make this exploration safe by allowing undo or providing a safety net behind destructive choices that could result in lost data.

The most important thing when designing a GUI as well as an UI is user tests. User tests provide vital information that can allow the evasion of problems with the product early on, before changes are impossible or very expensive.

2.6 Visual perception

The eyes allow us to perceive depth, contrasts, shapes, color and brightness so it is the eyes that give us the main spatial information about the layout of the environment in which we reside.

Visual perception differs from haptic perception in the way that it is not sequential like haptic perception is. The Gestalt laws illustrate this very well. The Gestalt laws originate from the theory that the whole is greater than the sum of the parts. It has been shown that the human mind makes a leap, through a cognitive process, from comprehending the individual parts of an object to grasp the whole (Dabbagh, 1999). According to Monö (1997) the Gestalt laws are stated as follows:

(16)

The law of proximity (Figure 3) dictates that objects that are close to each other tend to be grouped together. This is true even if there are large differences amongst the objects.

Figure 3. Gestalt law of proximity (http://www.sapdesignguild.org).

The law of similarity (Figure 4) dictates that objects that are similar to each other tend to be grouped together. This is useful in designing user interfaces where, as an example, buttons controlling different types of functions can stand out if they look different from each other disregarding of their location.

Figure 4. Gestalt law of similarity (http://www.sapdesignguild.org).

The law of closure (Figure 5) dictates that objects are seen as grouped together if they tend to complete some entity. Human visual perception strives to complete Gestalts that are not complete.

Figure 5. Gestalt law of closure. http://chd.gse.gmu.edu

11

(17)

The law of good continuation (Figure 6) dictates that we tend to group objects that follow a smooth line or a curve. This ability allows us to interpret maps and mosaics (Monö, 1997).

Figure 6 [E5]. Gestalt law of good continuation (http://www.sapdesignguild.org).

The area factor (Figure 7) makes us perceive smaller enclosed areas more clearly than larger ones.

According to Monö (1997) this is why we see the Swedish flag as a yellow cross on blue background and not four blue squares on a yellow background.

Figure 7. The area factor.

The law of symmetry (Figure 8) dictates that symmetric contours tend to be grouped together into figures. In the figure below the white areas stand out as figures from the black background because the outlines of the white figures are mirror images of each other. This is not the case with the black shapes and thus the black is seen as background (Dabbagh, 1999).

Figure 8. Gestalt law of symmetry (http://www.graphics.cornell.edu)

(18)

The law of common movement (Figure 9) states that we tend to group elements that have common movement even if they differ in shape and size.

Figure 9 Monö (1997) Gestalt law of common movement.

The law of experience (Figure 10) states that certain Gestalts are visible if you have the experience needed to understand the picture at hand. Prior knowledge of what a car looks like is required in order to recognize the figure below as car.

Figure 10. Illustrating the Gestalt law of expertise, a Fiat Panda (http://www.treddi.com).

The Gestalt laws are frequently used in GUI design to give structure to an interface. The structure achieved by applying the Gestalt laws to, as an example a virtual control board, furthers the usability of that user interface since it takes into account human visual perception.

2.7 Combining haptic and visual cues

User interfaces of computer applications often only convey information to the user using one modality, vision. Humans are well adapted to make good use of multimodal information at the same time. This is not taken into account in most GUI designs. A problem that arises with growing interface complexity is information overload which shows in difficulties for the user to attend to all relevant parts of information conveyed by the system (Brewster, 1997). According to Campbell, Zhai, May and Maglio (1999), adding haptic feedback to an interface that concerts with the visual information increases user performance. They also showed that if the feedback from different modalities differed, the haptic feedback did not further the interaction and stated, “What 13

(19)

you feel must be what you see” (Campbell, Zhai, May and Maglio, 1999, page 5). Oakley, McGee, Brewster and Gray (2000) found that the addition of haptic feedback to an interface did not significantly decrease the amount of time needed to complete a given task. They did however find a significant change for the better in user errors, frustration and subjective workload.

2.7.1 Intermodal relations

According to Heller and Schiff (1991) vision tends to be initially relied upon and also being the main information source for adaptation. It tends to dominate the other senses and that dominance shows it self when the stimuli of the different modalities are in conflict. When it comes to conflicting information about the shape of an object vision is completely dominant over touch but when it comes to spatial information there is a compromise between vision and touch even though vision is still dominant over touch. Heller and Schiff (1991) also state that observers tend to rely on the modality that is the most appropriate for the given situation such as touch for texture, vision for spatial information and audition for temporal rate.

2.8 Structured search

Structured search is a search process in which information that may help guide search is available to the user. Tests have shown that there is a linear relationship between the number of entries in a list and the search time to find a specific item in that list, given that the entries are in random order. Even though the list in the usability test carried out within the confines of this master thesis was not in random order this still highlights the need for structure in the list when it is to be browsed by users.

The fundamental goal of when designing systems that utilizes structured search is to structure the menu so that the desired selection can be done in the least possible average time. There are of course methods that can be used to decrease the search time such as organizing the entries alphabetically, moving the most commonly chosen entries to the top of the list, highlights and shortcuts. These are all methods that convey cues to the user about the list structure and thus decreasing search time, which in a driver environment means less time in which the driver does not have eyes on the road.

3. Preliminary investigations

The preliminary investigations for this master thesis work contains of benchmarking in order to get an understanding for existing solutions and brainstorming to generate new ideas regarding interaction concepts.

3.1 Benchmarking

In order to get a solid overview of existing interaction concepts, a benchmarking process was carried out on relevant technologies. Infotainment systems of other car manufacturers were investigated along with mp3 players.

(20)

15

3.1.1 Infotainment systems

As a part of the benchmarking process the infotainment systems of other car manufacturers were examined. Only the functions relevant to this thesis were investigated, that is navigating through long lists of information. The infotainment systems examined was the systems in car models from Volvo, SAAB, Lexus, Acura, Ford, Renault, MB, Audi and BMW.

When investigating the infotainment systems of these nine car manufacturers it soon became evident that there are basic types of infotainment systems and that the difference between these different types is due to the control layout. The layout of the controls reflects the design of the GUI. Table 1 shows how the infotainment systems of the different car manufacturers controlled.

Table 1. Infotainment controls available in different car models.

Choice buttons Control Stick Direction controlpad Enter/OK control Haptic rotary device Phone pad Rotary device Shortcut buttons Touch screen

Acura TL V6 3.2 A5 X X X X X

BMW 325i X X

Ford Escape Hybrid X X X X

Lexus RX400h X X

MB E270 CDI X X X X X

Renault Scenic X X X

SAAB 9-3 R4 1,9 TDi X X X X

When it came to navigating through lists all of the manufacturers had a similar approach to the problem. In order to limit the number of entries in the list the possibility to write the name of the desired entry was available. This writing was done using on screen keyboards controlled by the controls available to the car model. There was a choice to write all of the name or a piece of it, except on the MB where the full name had to be entered, and the list was limited to the entries beginning with the letters entered. In some cases such as the Renault, SAAB and MB the entries that did not begin on the letters entered was still in the list thus no exclusions where made in these systems to decrease the amount of entries in the list simply a navigation using an on-screen keyboard. BMW and Lexus utilized full exclusion to reduce the number of list entries, this approach where perceived as more logical then the one adopted by SAAB, Renault and MB.

The entry selection was displayed in two main ways. The first used a highlight that moved through the list and when it reached the top or the bottom of the list scrolled, SAAB, MB and Renault had this approach. In the other type, which Lexus implemented, the highlight was fixed in the center of the list and the list scrolled under the highlight with each manipulation. Fast browsing through the

(21)

list was achieved by either pushing down the button for browsing downwards in the list upon which the system first made one step and then started auto scrolling through the list for the duration of the push. All systems except SAAB had this approach. In the SAAB fast browsing was achieved by simply rotating the rotary device faster. List entries were displayed either framed as in the Lexus or the Acura or as text on a colored background such as SAAB, Renault, MB and Ford.

Only one of the car models tested during the benchmarking process has a haptic rotary device installed, the BMW 325i. Even though the iDrive system in the BMW 325i as a whole is quite complicated and not very user friendly much due to the number of steps required to perform even the easiest of tasks such as audio balance. The iDrive system displays its highlight much like the SAAB does with the difference that the highlight only moves within one screen filled with entries.

Once the highlight reached the top or bottom of the screen the list starts scrolling in various speeds depending on how far the haptic rotary device is turned.

The infotainment systems could basically be divided into two groups; those controlled by few controls such as Acura, BMW, Lexus, Renault and Saab and those that are controlled by a lot of controls such as the Ford and the MB. A common denominator for all the systems is that they feel computer inspired. Pictures of the investigated infotainment systems can be found in appendix A.

As far as interaction concepts goes the infotainment systems that used a rotary device all worked the same, one click resulted in one step in the graphical user interface.

3.1.2 Mp3 players

Upon closer examination of the more popular Mp3 players on the market now such as the iPod or the Creative Zen series it was soon found that they where very much the same. When navigating through lists they all worked basically the same way. An initial choice, such as artist or album, was made to limit the number of entries, and then the navigation tool available was used to browse through the list. Certain Mp3 players where easy to navigate through list with such as the iPod or the Creative Zen sleek where the maneuvering was done by stroking a touchpad while others where harder to operate such as the Creative Zen Xtra which had angle and time driven interaction concept.

The list browsing in the Mp3 players was just like in most infotainment systems, very computer inspired.

(22)

4. Method

This section covers the methods used throughout the work of this master thesis. There where only three segments during which a specific method was used, during the brainstorming, the programming and during the test.

4.1 Brainstorming

The brainstorming was carried out as follows. The group was first divided into two groups; Men and women. Each member in each of the two groups where given a paper and a pencil. The two groups where then given the topic of the brainstorming process and given two minutes to quickly sketch down their ideas on the paper given to them. When two minutes had passed the participants passed the paper on to the next member in their group and were asked to continue on the paper they received. This was repeated until each paper in each group had been used by all members in that group. After this the two groups where given half an hour to come up with two concepts each which they where then to present to the other group. After the presentation the word was free and the concepts conceived where openly discussed, criticized and improved. This method proved very successful and gave birth not only to the interaction concepts to many strong new ideas concerning list sorting and haptic feedback. These ideas where left out of this report as they do not have anything to do with this master thesis. The topic of the brainstorming was how haptic feedback can be used to make the browsing of lists easier.

During the brainstorming and the discussion that followed, four interaction concepts where found to be important to compare. See figure 11.

17

Figure 11. The concepts selected for the test. The circle represents the rotary haptic device.

(23)

oncept 1 was designed to mimic a common mechanical rotary device conveys a click in the

oncept 2 can be described as concept 1 without clicks. This interaction concept is common in

oncept 3 which can be found on many VCR players is unlike the other three concepts. Instead of

oncept 4 has only one difference to concept 2, an acceleration script. This concept works

he test environment was created in Macromedia Director using Macromedia Lingo script. Lingo

he first step in the construction of the test environment was to create an overview of the program, C

device for each step taken in the graphical user interface. The clicks are 20 degrees in length and are as stated earlier directly connected to movement the graphical user interface and the feedback is the same no matter what direction the device is turned. Concept 1 is angle driven. This type of interaction concept is very common in car stereos but is generally standard interaction concept that can be found in many technical devices.

C

volume gauges on stereos. No clicks, just an even resistance all over. Concept 2 is just like concept 1 angle driven.

C

rotating the device it is held at an angle that is not equal to 0. Larger angle results in higher scrolling speeds. The device strives to be at 0 and thus providing an even resistance if turned away from 0. Concept 3 is time driven, the list scrolls as long as the angle is separated from 0. If the device is turned clockwise the list will scroll down and vice versa for counter clockwise.

C

somewhat like an overdrive that can be found on certain cars. When the rotary haptic device is being turned slowly concept 4 is exactly like concept 2, 20 degrees rotation resulting in one step in the graphical user interface. However as the turning speed increases the number of degrees needed for a step in the graphical user interface decreases. So if the device is being turned fast the number of steps resulted is larger than what would have been if the gauge was turned the same amount of degrees at a lower speed. Concept 4 is angle and speed driven.

4.2 The test environment

T

is an object oriented programming language just like Java or C++. Roughly, a program made in an object oriented language consists of several classes which consist of objects doing different things within the program. The test environment was created with an object oriented programming language using an object focused methodology. There was never any need for creating classes and thus using a strict object oriented methodology. Instead the test environment consists of objects, like pieces of a jigsaw puzzle, which together made a complete program, the test environment.

T

a blueprint (Figure 12) which defines what an object is supposed to do, what it can receive and what it can send. The reason for this step is that it makes the programming process much easier.

By deciding what comes in and out of an object and to where those signals go at an early stage in the software development the programmer gains the freedom to change an object at free will as long as it can receive and send the same messages without risking the integrity of the software as a whole. This step, if done properly, also gives the software structure and thus making it relatively easy to add new functionality to the software if needed.

(24)

Figure 12.An overview of the test environment.

Figure 12 is an overview of the code created specifically for this test environment which excludes the communication scripts which handles all the communication with the rotary haptic device.

Start Environment initiates the communication code and defines all variables that are needed immediately upon start of the test environment. File I/O reads a text file that contains the list that is displayed in the test environment and sets each row in the text file in its place in a vector. This vector is later used by the ReadSetText object which handles the text, places it in the right position in the GUI. ReadSetText also keeps the text updated and changes the colour of the text if the text is highlighted. The keylistener object listens if any keys are pressed down on the keyboard and compares the signal from the keyboard to a list. If there is a match with the signal from the keyboard and an item in the list then a specific action is taken. In this case the keylistener was used as a switch so that the test leader easily could switch between the different interaction concepts.

The four interaction concepts where assigned one object each, each object defining the specific interaction concept to which it was assigned. The hardware communication script, which was provided by Volvo technologies, handles all communication with the rotary haptic device and keeps vital information such as angles stored in global variables. The slider was also given an object specifically tracing and translating its movement and position.

4.3 The experimental setup

This section covers experimental setup for the test carried out during the course of this Master thesis.

4.3.2 Test rig

The test was carried out in the “Lab-Jakob” driving simulator (Figure 13) at Open Arena Lindholmen, Gothenburg.

19

(25)

Figure 13. The test environment “labb-Jakob” at open arena Lindholmen.

4.3.3 Driving simulation

The driving simulation used during the test was a simple winter environment with little traffic (figure 14). A very clean and simple environment that yields no surprises, similar to driving on a country road.

Figure 14. The driving simulation used in the test. Provided by Volvo Technologies.

4.3.2 In-car user interface

The car that was part of the driving simulator is the front seats and hood of a Volvo XC90. The simulated car is an automatic so the only controls the user needed to focus on for conveying the vehicle was the accelerator and the brake. The controls related to the graphical user interface were one LCD display and one rotary haptic device (Figure 13)

(26)

4.3.3 The Graphical User Interface

The Graphical User Interface (Figure 15) was designed to be simple and clean with focus on the test.

Figure 15. The Graphical user interface used in the test.

The graphical user interface contains 7 text fields of which the slider can reach number 2 to 6.

Fields number 1 and 7 are merely there to give a preview of what is to come. The names in the list are all the names in Sweden which have a names day ordered in alphabetical order. The book icon and the text “contacts” are merely there to put the test environment into context. Browsing a contacts list is a possible task in integrated infotainment systems.

4.4 The test

This section covers procedure, equipment and planning of the test carried out within the confines of this master thesis. The test was carried out in May 2006 at Open Arena Lindholmen.

4.4.1 Participants

A total of 12 participants, 6 females and 6 males, took part in the experiment.

4.4.2 Procedure

A within-subjects design was used in the test which means that every participant tested every interaction concept. To eliminate the learning factor from the results the interaction concepts were counterbalanced (table 2) so that each concept would have the benefit of being the last and the drawback of being first equally many times. To ensure that all test participants would get the same information in the same way a manuscript was used (appendix B). Each participant was given 2 minutes to practice driving within the driving simulator before the test commenced. For each concept the participants were asked to find a series of names in the list using the rotary haptic device and told to push the device when they have highlighted the requested list item. All tasks were given to the test participants while driving. The entire test was recorded with a video camera.

21

(27)

The test participant information form each participant had to fill can be found in appendix C.

During the test measurements of time, turn over errors and workload were taken.

Table 2. Condition counter balancing.

TP nr Order of Conditions

1 2 3 1 4

2 3 4 2 1

3 4 1 3 2

4 1 2 4 3

5 3 1 4 2

6 1 2 3 4

7 2 4 1 3

8 4 3 2 1

9 2 1 3 4

10 1 4 2 3

11 4 3 1 2

12 3 2 4 1

(28)

5. Results

This section covers the results of the test carried out during this master thesis. The results displayed here is the analysis of the data obtained during the test, for the raw data see appendix D.

5.1 Time

For every list length (10, 25, 50, 100, 200, and 400) in all concepts two time scores were recorded for each participant, downward and upward list searching. The average of downward and upward list searching for each participant were used in the analysis. Repeated measures single factor ANOVAs were conducted for each list length.

0 10 20 30 40 50 60 70 80

0 100 200 300 400 500

List length

Time (s)

Concept 1 Concept 2 Concept 3 Concept 4

Figure 16: The task time data is plotted for every list length (10, 25, 50, 100, 200 and 400 items). Error bars in the figure depict the standard deviations.

For all list lengths there were significant differences between the concepts. As can be seen in Figure 16 the participants performed worse with Concept 3 than with the other concepts in all tasks. Figure 16 also show that the participants performed similar in the short ranges with concept 1, 2 and 4, concept 1 being slightly better in the shortest tasks. In the task length 25 and 50, concepts 1, 2, and 4 was almost identical with average participant performance in time differing less than a second. In the tasks of length 100 and 200 concept 4 continued on the trend hinted at 50 with the participants performing better, though not significantly so, than what they performed with concepts 1 and 2. The test participants performed better with Concept than with concept 2 at task length 100 by an average of 3.08 seconds but at 200 the difference between the two decreased to only 0.32 seconds. At task length 400, the participants significantly outperformed the other three concepts with concept 4 averaging 10.76 seconds better than concept 1, 11.56 seconds better 23

Interaction Concepts in a Rotary Haptic Device for Browsing Throug Lists in

M A S T E R’S T H E S I S

TOBIAS SVENBERG

Interaction Concepts in a Rotary Haptic Device for Browsing Throug Lists in

an Integrated Infotainment System

Preface

Abstract

Table of contents

1 Introduction

2 Theoretical Background

3. Preliminary investigations

4. Method

5. Results