Augmenting the Remote Control : Studies in Complex Information Navigation for Digital TV

(1)

Augmenting the Remote Control:

Studies in Complex Information

Navigation for Digital TV

(2)

ISBN 91-7373-940-5 ISSN 0345-7524

Department of Computer and Information Science Linköpings Universitet

SE-581 83 Linköping, Sweden

(3)

(4)

(5)

(6)

(7)

Abstract

The transition to digital TV is changing the television set into an entertainment as well as information supplier device that provides two-way communication with the viewer. However, the present remote control device is not appropriate for navigation through the huge amount of services and information provided by the future digital TV, presumably also a device for accessing the Internet. One possibility for coping with the complex information navigation required by TV viewers is an augmentation of the interaction tools currently available for TV. Two approaches to such an augmentation are investigated in this thesis: linking paper-based TV guides to the digital TV and enhancing the remote control unit with speech interaction.

Augmentation of paper-based TV guides is a futuristic research approach based on the integration of paper-based TV guides into computation technol-ogy. This solution provides interactive paper-based TV guides that also function as a remote control for the TV. A prototype system is developed and explorative studies are conducted to investigate this approach. These studies indicate the benefits of integrating paper-based TV guides into the TV set. They also il-luminate the potential to provide innovative solutions for home information systems. Integrating familiar physical artefacts, such as paper and pen into TV technology may provide easy access to information services usually provided by PCs and the Internet. Thus, the same augmentation needed for TV as an entertainment device also opens up new communication channels for providing society information to citizens who do not feel comfortable with conventional computers.

The thesis also reports on studies of speech interfaces for TV information navigation. Traditional speech interfaces have several common problems, such as user acceptance and misinterpretation of user input. These problems are investigated in empirical and explorative studies with implementation of mock-ups and running research systems. We have found that the pragmatic solution of augmenting remote control devices by speech is a suitable solution that eases information navigation and search.

(8)

(9)

Acknowledgements

I am deeply grateful to many people for their support during this long journey. It is not possible to mention all of them, but I want to take the opportunity to express my sincerest gratitude to those who ought to receive special attention.

I would like to express my deepest gratitude to my supervisor Sture H¨agglund for his support, criticism and encouragement. I would also like to thank Arne J¨onsson and Kevin McGee for their generous contribution of time and expertise. Many thanks to Hans Marmolin for starting up this thesis project.

This research has been supported by the Foundation for Advancement of Knowledge and Competence (KK-stiftelsen) through the Industry Research School in Applied IT and Software Engineering at Linkping University with Nokia and Santa Anna IT Research Institute. Support was also received from the HMI-graduate school and IDA.

Special thanks to Lillemor Wallgren, Birgitta Franz´en, Helene Wigert and Britt-Inger Karlsson for their administrative support. I also wish to extend my gratitude to the TUS group for their computer support.

I would like to thank my colleagues for the joint effort on some of the studies; Erik Berglund, Magnus B˚ang, Pontus Johansson, Anders Larsson, Jonas Lund-berg and Pernilla Qvardfordt. I also wish to thank Lars Ohlsson and Sophia H¨ager for the joint effort on Paper V.

Thanks to my fellow PhD students at the Division of Human-Centered Sys-tems. In particular, I would like to thank David Dinka for always being there, especially when things were down. I also wish to thank Fredrik Arvidsson, Christina ¨Olvingson, and Sonia Sangari, it is always a pleasure talking to you! Thank you Ola Leifler for your Latex support!

My gratitude goes also to my wonderful friends. Special thanks to Maria Minassian for listening and encouraging me. You are always close despite the long distance!

Then of course, thanks to the LUPA board, the fantastic adept group, and my supporting as well as encouraging mentor Ewa Ljunge.

Finally and most of all, I would like to thank my wonderful family. My two fantastic sisters Susan and Sally – who are my friends and supporters. You are the best sisters anyone can wish for. Special thanks to my parents in law Margareta and B˚age. I always look forward to the Sunday dinners. My sweet Hoffe – you are the best dog! My adoring parents Shawshan and Akram – thank you for always being there especially when I needed you, for your enormous

(10)

support, and for believing in me! I have been very fortune to have you. My loving husband Erik – thank you for supporting me when I was in doubt and when things were really difficult! Thank you for all your help and scientific support! You have generously contributed with your time even in situations when you did not have any time. Thank you for sharing my life! You are my everything.

This thesis is dedicated to Shawshan, Akram, and Erik. I love you.

Aseel Berglund Link¨oping in April 2004

(11)

List of Papers

Papers Included in this Thesis

The following five papers are included in the thesis and will be referred to by their Roman numerals. The papers are printed in their original state except for format changes and some minor errata corrections. Papers I and II were published before I got married when my sir-name was Ibrahim:

I. Ibrahim A., Lundberg J. and Johansson J (2001) Speech Enhanced Remote Control for Media Terminal. In Proceedings of Eurospeech’01, Aalborg, Denmark, Volume 4, pp. 2685 – 2688.

II. Ibrahim A. and Johansson P. (2002) Multimodal Dialogue Systems for Interactive TV Applications. In Proceedings of the 4th IEEE International Conference on Multimodal Interfaces 2002 (ICMI’02), Pittsburgh, USA, pp. 117 - 122.

III. Berglund A. and Johansson P. (2004) Using Speech and Dialogue for Inter-active TV Navigation. Submitted to the international journal: Universal Access in the Information Society (UAIS), Springer-Verlag Heidelberg. IV. Berglund A. and Qvardfordt P. (2003) Error Resolution Strategies for

In-teractive Television Speech Interfaces. In Proceedings of the Ninth IFIP TC13 International Conference on Human-Computer Interaction, INTER-ACT 2003, Z¨urich, Switzerland, pp. 105 – 112.

V. Berglund A., Berglund E., Larsson A., and B˚ang M. (2004) The Paper Remote: An Augmented TV Guide and Remote Control. Submitted to the international journal: Universal Access in the Information Society (UAIS), Springer-Verlag Heidelberg.

Other Publications by the Author

The following published papers are not included in the thesis. However, Paper III is an extension of Paper VI. Papers VI, VII, and VIII were published before I got married when my sir-name was Ibrahim:

(14)

VI. Ibrahim A. and Johansson P. (2003) Multimodal Dialogue Systems: A case study for Interactive TV In Proceedings of the 7th ERCIM workshop (User Interfaces for All), 2002 in Paris, France, LNCS 2615, pp. 209 – 218. VII. Hultcrantz J. and Ibrahim A. (2002) Contextual Workshops: User

ipation in the Evaluation of Future Concepts. In Proceedings of Partic-ipatory Design Conference 2002 (PDC2002), Malm¨o, Sweden, pp. 344 – 348.

VIII. Lundberg J., Ibrahim A., J¨onsson D., Lindquist S., and Qvarfordt P. (2002) ”The snatcher Catcher” -an interactive refrigerator. In Proceed-ings of the second Nordic conference on Human-computer interaction (NordiCHI 2002), Aarhus, Denmark, pp. 209 – 212.

(15)

Introduction

1 Motivation

The primary multimedia electronic device in the home is the television (TV) set with its input device, the infrared remote control. During the last two decades, the transition to digital TV has changed the television concept; new services and applications are provided. There are attempts to integrate the computer and the TV to provide a home terminal that functions as both an entertainment as well as an information supplier device (Voutsas and Halverson, 2000). These attempts have faced various problems, e.g. economic, technical and usability difficulties.

One of the main problems that faces the home terminal, i.e. the future TV, is the complex information navigation through the remote control device. This problem is caused by the attempts to design the future TV as a computer but with the same conceptual model of the so-called ’traditional TV’ which com-plicates the interaction. For example, Internet pages are navigated by moving a marker controlled by means of four arrow keys on the remote control. Thus, compared to the ’traditional TV’ which consists mainly of TV channels, future TV devices will contain very large amounts of information and provide complex functions through which viewers can make use of the information and services. As an example, let us consider the user of a common digital TV application; the Electronic Program Guide (EPG), that provides information about TV pro-grams and channels. The user’s goal is to find an interesting movie and watch it. The user has to first search for movies, either by browsing or through some key-based search functions. Having found a set of satisfying alternatives, the user must analyze the description of each movie and decide which one is most suitable. Throughout these steps, many things can go wrong. For example, the user may not find what s/he is looking for, and may not succeed in entering search text. Moreover, the steps can take a long time to accomplish and the user may feel that it is a waste of time. The complexity of these steps is caused by the complexity of the interaction due to the huge amount of information controlled by the traditional interaction device and model; the remote control, which is better suited for the ’traditional TV.’

Most TV development has been technology driven (Eronen, 2003). In gen-eral, home technology providers have sound knowledge of the technology they

(16)

Figure 1: Digital and interactive TV systems usually consist of a set-top box (STB) connected to the TV set. Generally, two remote control devices are included: one for controlling the TV set itself and one for the set-top box.

produce but not on the social context in the use of technology (Venkatesh, 1996). There are some studies focused on understanding household and domestic envi-ronments (Harper, 2003, Hektor, 2001, Lee and Lee, 1995, O’Brien et al., 1999, Morrison and Krugman, 2001, Venkatesh, 1996). These studies provide impli-cations for the future of domestic appliances.

As pointed by many researchers, augmentation of existing artifacts is a strat-egy that can be applied to help users (Mackay, 1998). Help is important for universal usability, which is about making information services usable for ev-eryone (Stephanidis and Savidis, 2001). Help for future TV usage is needed by more and more people because TV is one of the most important devices for the home and is used by a wide range of people.

In this thesis we argue that augmenting existing tools instead of adding new ones is a suitable approach to helping users interact with the future TV. The existing tools we focus on in this thesis are paper-based TV guides and the remote control device.

2 Problem Description

This research started at Nokia Home Communications in Sweden which de-veloped set-top boxes (digital box receivers) and media terminals (digital box receivers and interactive TV devices) for the home. These devices are elements in digital and interactive TV systems that provide, among other things, digi-tal TV channels, Internet services, games and communication applications, see Figure 1. In other words, these devices are the home terminal of the future, they are entertainment and information suppliers for the homes. This research started with the problem that Nokia Home Communications was facing: The

(17)

ture TV is designed as a computer with the ’traditional TV’ interaction model. This approach causes complex information navigation by TV viewers through remote control devices that tend to be big, consisting of many buttons, and not so easy to use (Nelson, 1999). Figure 2 shows a few examples of existing remote control devices to illustrate how they can differ in design, shape, functionality, size and ease of use.

One possibility to cope with complex information navigation is to investigate augmentation of interaction tools available for TV today. In this thesis, two approaches to such an augmentation have been investigated: linking paper-based TV guides and the digital TV as well as enhancing the remote control device with speech interaction.

TV guides printed on paper are common and familiar to TV viewers, how-ever, they are currently isolated from TV sets. Paper Remote is a concept for linking such guides with the digital TV. This is a futuristic research approach focused on adding computation augmentation to paper-based TV guides. The Paper Remote concept connects paper-based TV guides with the TV set and allows people to access TV content with a digital pen. Thus, this future TV guide functions both as an interactive paper-based guide and as an interaction interface for TV sets. In this thesis, we have studied the current usage of TV guides (Paper V). We have, then, developed a prototype system and conducted explorative studies to investigate the benefits of the concept (Paper V). Using paper as an interface to technology has already been studied, however, mainly in work contexts (see Section 4.2).

This thesis has also explored a realistic commercial solution to the com-plex information navigation. The solution is focused on enhancing the exist-ing interaction through speech interaction. The research is focused on design issues from the user’s point of view and not from the technological point of view. The research is based on user studies of both working research systems and mock-ups. Previous studies confirm the benefits of speech interfaces in general (Marshall, 1992, Martin, 1989, Visick et al., 1984). However, to our knowledge little is known about speech interfaces for TV sets. This thesis re-ports on the experiences and lessons learned from designing speech interfaces for the TV environment that is social and noisy. Unlike the computer environ-ment, the TV is characterized by a big screen with not very high resolution, and placed at a distance in front of the viewer. We have studied the potential of speech command input (Paper I), the benefits of spoken natural language and dialogue for TV navigation (Paper II and III), and strategies that can be applied when the viewer input is misrecognized by the system (Paper IV).

3 Research Question

The simple days of 1-3 channels of TV are gone. Researchers are now facing the daunting task of developing an easy-to-use and multifaceted interaction system for the future TV providing natural interaction for a highly heterogeneous view-ing audience. In addition, the development also includes examinview-ing the design

(18)

Figure 2: Different types of remote controls.

of such a system using the tools available in the TV context today. The overall research question addressed by this thesis is:

How can commonly available TV artifacts or elements of the TV view-ing context be developed to provide a desirable interaction design for the complex and multifaceted future TV?

The elements of the TV viewing context include: the TV room, the remote control device, the paper-based TV guide, gestures, eye tracking and the VCR. This thesis is limited to two of these areas, namely:

− What are the prospects of integrating familiar physical artifacts, such as paper and pen, with TV technology?

− How suitable are speech interfaces in general and natural language in par-ticular for TV information navigation?

We have studied how users of future TV may react to the interaction avail-ability through interactive paper-based TV guides and speech. We have con-ducted experimental and explorative studies with implementations of mock-ups and running research systems.

4 Related Work

In this section we will provide a brief review of TV, paper and speech interfaces.

4.1 Brief history of TV interfaces

Television is one of the central entertainment devices in homes

(Herigstad and Wichansky, 1998). For example, in the United Kingdom 99% of households had TVs in 2000 – 01 (ONS, 2002). During the last two decades, most of the development of TV has been directed towards digitalization as well as integration with computers (Frenkel, 1989, Zong and Bourbakis, 2001, Seel, 2003). Thus, new concepts have been exploited, such as interactive digital

(19)

TV (Jacobs and Dransfield, 1998, Frenkel, 1989, Kim, 2001, Lougos et al., 2002, Nelson et al., 1995, Press, 1993). Despite the fact that television is one of the central devices in households, relatively few publications address inter-active digital TV in system design literature. The development of domestic devices in general and TV devices in particular have been driven by techno-logical advances rather than by understanding the interaction between people and technology (Taylor and Harper, 2003). A small number of studies have been conducted to explore television viewing and the relationships between television and people’s daily lives (Adams, 2000, Clancey, 1994, Cohen, 2002, Conway and Rubin, 1991, Gauntlett and Hill, 1999, Heeter and Greenberg, 1988, Taylor and Harper, 2003, Lee and Lee, 1995, Logan et al., 1995, Silverstone, 1994, O’Brien et al., 1999, Jensen and Toscan, 1999).

Interactive digital TV is an entertainment device that offers TV channels, Internet services, such as banking, e-mail, and e-shopping and other interactive applications, such as games, electronic program guides and pools (Furht, 1996, Dransfeld et al., 2000, Kerschbaumer, 2000). There are attempts to evolve the future TV into an enhanced entertainment as well as information supplier de-vice. However, the interaction tools with this device will still be the traditional remote control device. The degree of interactivity varies from a simple remote control button press to sending information back and forth to service providers by means of a return path (Perera, 2002, Gill and Perera, 2003a). According to Toscan and Jensen, interactive TV is a combination of technologies that allows viewers greater control over what they watch and when they watch it (Toscan and Jensen, 1999).

Nevertheless, there are three main problems that face interactive digital TV. First, the paradigm for user interaction is drawn from the menu-driven world of personal computers (Ali and Lamont, 2000, Klein et al., 2003, Koller et al., 1997). Therefore, people with limited computer experience, the elderly in particular, are excluded. Second, the interaction design is poor (Freeman and Lessiter, 2003). Therefore even people familiar with new technologies find interactive TV non-intuitive and exhibiting inconsistencies (Gill and Perera, 2003b, Klein et al., 2003). Last, the increased number of TV channels make the navigation and selec-tion difficult, thus channel surfing and informaselec-tion gazing become impractical (Black et al., 1994, Chorianopoulos, 2003, Ehrmantraut et al., 1996) Currently, the main ways of changing channels on interactive digital TVs are:

− Scrolling by using the remote control, which can be time consuming (Ehrmantraut et al., 1996).

− Entering channel numbers from the remote control, which results in mem-ory overload for more than 50+ channels.

− Using an on-screen TV guide application called Electronic Program Guide (EPG) which is usually complex [ (Daly-Jones and Carey, 2000),

(Ehrmantraut et al., 1996, Taylor and Harper, 2003)]. EPGs are oper-ated by point-and-click maneuvers from the remote control or wireless keyboards.

(20)

These problems result in complex viewer interaction that can cause frustra-tion and irritafrustra-tion. Therefore, it is essential to deal with these problems. Ease of use is one of the most important factors for interactive TV (Black et al., 1994, Choi et al., 2003, Freeman and Lessiter, 2003, Kang, 2002, Jan and Loes, 2001). Another important factor for adopting interactive TV in the home is enjoyment (Choi et al., 2003). In general, previous research has shown that entertainment is relatively important as regards technology adoption in the home environment (Venkatesh and Brown, 2001) and TV is one of the most important entertain-ment devices in the home

(Herigstad and Wichansky, 1998).

There are many approaches to solving these problems. One approach fo-cuses on the remote control units that control the TV interfaces. Ease of use of TV interfaces is affected by the design of these units. Some attempts have been made to develop usable remote controls (Logan, 1994, Logan et al., 1994, Logan and Lenzi, 1995, O’Modhrain and Oakley, 2003). However, the start-ing point for these attempts has been based on the so-called ’traditional TV’ where the number of channels and services is not as great as it is for the future TV. There have also been attempts to build altered remote controls (Bretan and Kroon, 1996, Enns and MacKenzie, 1998, Robertson et al., 1996), (Myers et al., 2002, Thoma and Nissler, 1999). Another approach that has been applied to solve the problem applies personalization technology to ensure that the right people are provided with the right information (Smyth et al., 2002, Goren-Bar and Glinansky, 2004, Westerink et al., 1998). Applying speech is an additional approach (Portolan et al., 1999, Goto et al., 2003). The latter is studied in this thesis. A more innovative approach that has not been previously studied is merging TV and digital paper to provide interactive paper-based TV guides that also function as a remote control for the TV set. This approach is a second factor studied in this thesis.

4.2 Paper interfaces

In general, many researchers have noticed the significant merits of paper. It is ubiquitous, highly portable, easy to use in a wide range of environments, inex-pensive, can be annotated easily, and provides excellent readability properties (Hansen and Haas, 1988). So far, paper as an interface has mainly been stud-ied in the work environment (Johnson et al., 1993, Sellen and Harper, 1997, Sellen and Harper, 2001) and compared with computer interfaces

(Hornbek and Frokjer, 2003, Harper et al., 1991). However, to our knowledge little is known about the importance of paper in the home environment and the TV area.

Systems and prototypes have been developed to demonstrate the potential of augmented paper, such as the PaperLink system which is an electronic filling system for paper documents (Arai et al., 1997), and the Protofoil system that is a document management system that can be accessed by both paper and computer workstations (Rao et al., 1994). There are variations of augmented desk systems that make a physical desk more like a computer workstation and

(21)

that support computer-based interaction with paper documents (Wellner, 1993, Koike et al., 2001, Mackay et al., 1993, Ishii and Ullmer, 1997). A typical ex-ample is the DigitalDesk (Wellner, 1993).

Previous research has shown that the use of paper in the modern world is persistent because of paper’s physical properties, not just despite these proper-ties (Johnson et al., 1993). Johnson et al. (1993) claim that paper has utility that will not disappear with the increase in the electronic world, therefore the main goal should be integrating, not removing paper from the electronic world. Furthermore, Koike et al. show that the integration of paper and digital infor-mation is effective (Koike et al., 2001). Sellen and Harper predict that paper in support of reading tasks will be one of the hardest paper-based tasks to shift to the digital world (Sellen and Harper, 1997). They also identified four affor-dances of paper supporting the flexible interweaving of reading and writing; the paper’s tangibility, its spatial flexibility, its tailorability, and the manipulability of paper (Sellen and Harper, 2001).

4.3 Speech interfaces

In general, previous research has shown that there are many benefits of speech interfaces in various areas, such as computer, telephone, and web-based inter-faces (Damper and Wood, 1995, Newell et al., 1990, Stifelman et al., 1993), (Kurematsu, 1992). However, little is known about whether speech interfaces are suitable for TV interfaces and about how to design a system appropriate primarily for entertainment purposes and secondarily for information search purposes.

Previous research has reported a 20-40% efficiency increase using speech systems compared with other interface technology, such as key board input (Martin, 1989, Visick et al., 1984). Furthermore, research has also shown that speech increases productivity and improves accuracy (Marshall, 1992). Apply-ing speech-only interfaces may not be beneficial for the TV set that includes a large screen and remote control units. There are various ways of combining modalities when building multimodal interfaces. Combining speech with graph-ical interfaces is a common strategy for multimodal interfaces involving speech (Mane et al., 1996). Multimodal interfaces can ease the access to a system since they permit a broader range of users to use the system than the traditional in-terface (Oviatt, 2002). Efficiency is another important advantage of multimodal interfaces (Cohen et al., 2000).

To our knowledge, applying speech interfaces in general and multimodal interfaces in particular to the TV environment is not common. Little research has been done on the TV context (Goto et al., 2003, Hauptmann et al., 1995, Portolan et al., 1999). More study is necessary to investigate how best to design such interfaces for the TV set. This thesis focuses on speech design issues for digital TV.

(22)

5 Research Method

This section will deal with the methods used in this research, the choices that have been made and the consequences of these choices. The research is divided in two types of studies: explorative and empirical. Section 5.1 describes the methods applied for data collection in general, discussing their strengths and weaknesses. Section 5.2 presents which of these methods were used in the 5 different papers included in this thesis.

5.1 Data Collection Methods and Techniques

The methods and techniques used in this thesis are interviews, observations, workshops, Wizard-of-Oz techniques, and prototypes as well as research sys-tems.

Interviews

The interview is a central technique in qualitative research that provides rich data (Bryman, 2001, Kvale, 1996, MacNealy, 1997). Facts and opinions of po-tential users are discovered in interviews. Interviews can be unstructured or semi-structured (Kvale, 1996). The objective of the interviews is to understand the participants’ point of view. In unstructured interviews, the interviewer has a list of topics to cover, while in semi-structured interviews the interviewer has an interview guide. It is important that questions in the interview guide are sufficiently specific but not leading (Kvale, 1996).

Observations

Observations can often provide useful additional information about the studied topic. Participants can be observed while they interact with a system to collect data about reactions (Preece et al., 1994) and to capture descriptions of the way activities are performed (Newman and Lamming, 1995). The method allows the observer to view what users actually do and to focus on areas of interest. A disadvantage of observations is that the participants’ behavior may be altered because of the presence of the observer. Observation may be direct or indirect. With direct observation, the investigator is actually present during the task while with indirect observation the task is viewed by some other means such as a video recording.

Workshops

Workshops are useful for generating group creativity by gathering a group of people and focusing on a problem or an issue. Workshops can be considered as structured brainstorming. Workshops can be conducted in different phases (Arvidsson et al., 2002, Jungk and Mullert, 1996). One of the benefits of work-shops conducted in groups is that more data can be collected in the same amount of time compared to methods based on individual participation. This advantage

(23)

is characteristic for methods in which data is collected using qualitative meth-ods conducted in groups such as focus groups. Moreover, a group discussion can generate new ideas because when one person starts talking about an idea it can trigger associations, thoughts, and ideas in other people. However, the dynamic of the group can affect the participants negatively. In the presence of dominant persons there is a risk that their opinion will affect the rest of the group and consequently the discussion will take another direction than it would have in another group constellation. In such cases, the workshop leader has to be more active.

Focus groups

The focus group method is a comprehensive research method that is used to collect data through structured and controlled group interviews (Morgan, 1993, Morgan, 1997). Information and insights are collected through interaction be-tween the moderator and the group, as well as bebe-tween group members. One criticism of focus group methods is that they often do not include real users but gatekeepers, such as supervisors, managers and others who made decisions about the products (Hackos and Redish, 1998). In a focus group session, groups tend to compromise a lot, in a way that makes their decisions always end up in some kind of no-man’s-land, conformity (Beyer and Holtzblatt, 1998, Brown, 1988). Thus, the decision the group takes is actually something no single member of them will stand behind. This is common in groups of people that do not know each other well, everybody wants to be polite, especially when the experience of the discussed issue differs a lot between the group members. It is also pos-sible that the group, if they do not have that much experience of the topic for the discussion, start to make decisions that are much more extreme than the group members would express if they had been interviewed alone, polarization (Beyer and Holtzblatt, 1998, Brown, 1988). This is simple to avoid if the focus group is used only to obtain the participants’ opinions and not force them to make decisions that will be representative for the group.

Wizard-of-Oz techniques

In research, there are several situations in which implementing a complex system is not possible. Simulating a system is common in such situations. According to Ericsson, two of the most common arguments for building a simulated sys-tem (Ericsson, 1999) are: First, the syssys-tem is too costly to build. Therefore, research is needed to investigate the usefulness and need for the system before designing it. Second, the technology required to build the system is not avail-able, but within reach. The purpose for the simulated study is to evaluate the need for future research.

Thus, the main benefit of this approach is that ideas and concepts can be evaluated before they are implemented. In the Human Computer Inter-action domain, the Wizard-of-Oz approach has been described as an exper-imental evaluation technique (Coutaz, 1994), or as a rapid prototyping for

(24)

Figure 3: The human operator (to the right) simulates the system when the partici-pant (to the left) interacts with the system.

systems that are costly to build or require new technology (Landauer, 1987, Maulsby et al., 1993)

In this approach, a human operator (a wizard) simulates the system and acts the way the system would act (Dahlb¨ack et al., 1993, Fraser and Gilbert, 1991). The wizard mediates the interaction while participants are led to believe that they are using a real system. However, in fact the system is simulated by the wizard. A typical Wizard-of-Oz experiment setup is depicted in Figure 3. The wizard is located in a room adjacent to the room where the study is conducted. This approach has, of course, disadvantages and potential problems. How-ever, Research in the domain of natural language has shown that people who know about the human operator behave differently and use different dialogue strategies than those who believe they are using a real system (Dahlb¨ack et al., 1993, Fraser and Gilbert, 1991). The main potential source of validity and/or relia-bility problems for this approach is the human operator – the wizard. The operator’s expectations cause a bias. The human operator can become biased due to the hypotheses used in the study. This can be avoided by providing the operator with a formal set of behavior guidelines and not with a set of hypothe-ses. These guidelines help to reduce the bias. Another solution is to use an operator who is not involved in the research, an operator who is not aware of the goals of the experiment.

An important issue regarding the Wizard-of-Oz technique in general is ethics.

(25)

Participants in the experiment think they are interacting with a real system and it might be argued that this is unethical. However, the obvious risk is that by telling the participants about the experiment setup their behavior would be affected. However, Fraser and Gilbert (Fraser and Gilbert, 1991) suggest that Wizard-of-Oz experiments should tell half the truth instead of an outright lie and leave the participants draw their own conclusions.

Prototypes and research systems

It is important to involve the users in the early stages of the design process in order to get feedback and to understand their needs as well as expectations (Jacobson et al., 1992, Schuler et al., 1993). Prototyping is an iterative ap-proach used throughout the entire development process (Avison and Fitzgerald, 1995) and is necessary to make sure that the developed system will meet users’ needs (Nielsen, 1993). A prototype serves as a communication medium between the developers and the users (Jacobson et al., 1992). Further, prototypes make it possible to test a work practice that does not yet exist and reveal future possibili-ties resulting from the new system (Newman and Lamming, 1995). With proto-types, different design alternatives can be tested (Jacobson et al., 1992). Mock-ups are one kind of prototype and are often used in the early stage of the devel-opment process (Muller et al., 1997). Advanced mock-ups can vary from either simple paper sketches to advanced working prototypes (Muller et al., 1997).

We have built working prototypes that have been used in controlled ex-periments with Wizard-of-Oz techniques to gather information for quantitative analysis (see Paper I and IV). We have also built two working research sys-tems that have been used to gather information for qualitative analysis. The first system is an advanced EPG dialogue research system (see Paper II and III). The second research system is called Paper Remote which is an interactive paper-based TV guide (see Paper V).

5.2 Methods Applied in the Different Papers

For some issues in this thesis, a quantitative approach has been natural, while a qualitative approach has been natural for others. Two types of studies have been conducted: explorative user studies and experimental studies. The explo-rative study approach is used to explore design implications and prospects of concept design that are new and require more knowledge. The data collected using this approach is based on participants’ needs, experiences, attitudes and expectations. In contrast, the experimental study approach is applied to eval-uate issues that are difficult to attain knowledge about using solely theoretical analysis. Therefore, these issues are evaluated empirically.

Explorative user studies

Paper II, III, and V presents results from two explorative user studies to provide initial design insights. Figure 4 illustrates the work process. These studies began

(26)

Figure 4: The work process of the explorative user studies reported in Paper II, III, and V.

by surveying the literature that raised research issues. To our knowledge, little research has been performed in the area. Therefore, we decided to conduct explorative studies to obtain knowledge that would form the foundation for upcoming work. According to Basili (Basili, 1996) the software-engineering discipline requires a cycle of model building, experimentation and learning to uncover or develop knowledge. For this reason, research systems were built and used in the studies to demonstrate the concepts for the participants. The systems were tested by the participants and qualitative data were collected with interviews, observations, workshops and focus group meetings.

Results presented in Paper II and III are based on observations and inter-views while results in Paper V are based on interinter-views, workshops and focus group meetings. In all these papers, running research systems were developed and used in the studies.

Empirical studies

Paper I and IV reports on two empirical studies. The process of the research is described in Figure 5. The work began by surveying the research literature that raised certain questions. Answers to these questions could not be based on theoretical analysis. Therefore, we decided to evaluate these questions em-pirically. Prototypes were implemented and evaluated from the Wizard-of-Oz approach. There are two major reasons for choosing this approach. First, the nature of the problems in Paper I and IV are not yet suitable for full-scale implementation and evaluation since we still need to collect more information about the situation. This approach helps us to observe different users, needs and behavior as much as it supports the collection of empirical data that per-tains to our research questions. Second, it has been shown that this method is viable for similar research, especially for natural language communication with

(27)

Figure 5: The work process of the experimental studies reported in Paper I and IV.

computers (J¨onsson and Dahlb¨ack, 1988, Carroll and Aaronson, 1988). The problem of validity and/or reliability caused by the human operator was managed differently in these studies. In Paper I, clear guidelines were provided while in Paper IV the operator was not aware of the goals of the experiment.

Ethics is an important issue for Wizard-of-Oz techniques in general. We believe that our experiments do not deceive the participants since:

− After the experiment, the aim and actual setup of the experiment were explained to the participants as well as the reasons for using the Wizard-of-Oz technique.

− Participants’ permission to use the collected data was requested and re-quired. If participants did not give permission, the collected data was to be destroyed and not used. This did not happen in our experiments. The empirical studies were supplemented with individual post-test interviews to collect the participants’ assessments of the concept and experiences from interacting with the interface. It is important to mention that the purpose of the interviews was not to evaluate the prototype . What was explored was the participants’ subjective response to such types of interfaces.

6 Contributions

The research reported here has contributed to understanding:

− Exploration of merging TV technology and digital paper and the prospects of integrating paper interfaces into home technology (explained in Paper V).

− TV habits of the Swedish audience and their usage of TV guides. Four types of TV watching were identified: TV as a background, TV as a main activity, TV as a partial activity, and TV as an accompaniment activity. The levels of TV guides usage were also identified as well as important qualities for current TV guides. These are presented in Paper V.

− The appropriateness of speech for viewer interaction with future TV in-terfaces. We have investigated spoken commands, natural language and

(28)

Figure 6: Current interaction model is based on direct manipulation by means of remote control (left). Future interaction is based on the Paper Remote concept as well as on speech input in addition to direct manipulation by means of remote control

dialogue interaction. Results of Paper I indicate the benefit of providing spoken commands for information navigation. Further, Paper II shows how dialogue and natural language input is suitable for information navi-gation.

− Description of situations when speech is not appropriate and when the remote control is more suitable (explained in Paper III).

− Potential error resolution strategies that can be applied when speech rec-ognizers misinterpret the users’ input. These strategies can also be applied to other home technology systems with a large screen and a distance be-tween it and the user, see Paper IV.

We have also identified two interaction models for TV based on research results. The first model is presented in Paper V and indicates the potential of interactive paper-based guides that are also integrated into the TV system, see Figure 6. The second model is presented in Paper II and concerns a system with spoken natural language input in combination with visual output. The model is based on a dialogue approach in which two partners communicate with each other about a subject. The model suggests a clear distinction between the involved partners and the discussed subject, see Figure 4.

7 Summaries of the Papers

Five papers that address various interaction design aspects of future TVs are included in this thesis. On a more detailed level, the papers can be divided into two groups: one that addresses speech interaction (Paper I, II, III, and IV) and one that addresses the Paper Remote concept (Paper V). The contribution of each paper is described in this section.

(29)

Figure 7: The three-entity interaction model for multimodal TV dialogue sys-tem.

7.1 Paper I: Speech Enhanced Remote Control for Media

Terminal.

In Proceedings of Eurospeech’01, Aalborg, Denmark, Volume 4, pp. 2685 – 2688. Co-authored by:

– Jonas Lundberg, Department of Computer and Information Science, Link¨oping University, S-581 83 Link¨oping, Sweden, email: jonlu@ida.liu.se

– Jenny Johansson, Nokia, Diskettgatan 11, SE-583 35 Link¨oping, Sweden, email: jenny.johansson@nokia.com

This paper focuses on the navigation difficulties inherent in the huge number of television channels. The aim of this study is to investigate whether or not spoken commands could solve the navigation problem. In this study, two input techniques are tested empirically: remote control and speech input.

The study showed that subjects preferred to use shortcuts rather than nav-igate, and that the speech command group could use shortcuts in situations where the remote control group had to resort to navigation. However, subjec-tive data showed that people preferred to have a remote control unit.

In summary, this study indicates that speech command input is more effec-tive as the steps required to complete tasks are fewer and shortcuts can be used more often in the speech condition. However, subjective data indicate that peo-ple are more satisfied with the remote control input. In conclusion, multimodal interaction, in which speech input complements the remote control unit is rec-ommended. Speech control should not be a substitute for the remote control device.

(30)

7.2 Paper II: Multimodal Dialogue Systems for

Interac-tive TV Applications.

In Proceedings of the 4th IEEE International Conference on Multimodal Inter-faces 2002 (ICMI’02), Pittsburgh, USA, pp. 117 – 122.

Co-authored by Pontus Johansson, Department of Computer and Information Science, Link¨oping University, S-581 83 Link¨oping, Sweden,

email: ponjo@ida.liu.se

This paper focuses on revealing the design knowledge of a multimodal elec-tronic program guide for interactive TV. Multimodal interface is defined as a combination of spoken natural language and visual representation. In this study, a running research system is developed and tested. A qualitative study based on observations and interviews was conducted to examine the users’ experiences of the system interaction. The system was designed by adding speech interaction to an already existing commercial electronic program guide controlled by means of a remote control.

Study results indicate that spoken natural language input combined with visual output is preferable for TV applications. Furthermore, user feedback requires a clear distinction between the dialogue system’s domain result and system status in the visual output. Consequently, an interaction model is pro-posed and consists of three entities: user, domain results and system feedback, see Figure 4. This model is created from a dialogue approach in which two partners (here the user and the system) communicate with each other about something (here TV channels, programs, descriptions, etc.)

7.3 Paper III: Using Speech and Dialogue Systems for

In-teractive TV Navigation

Submitted 2004 to the international journal: Universal Access in the Informa-tion Society (UAIS), Springer-Verlag Heidelberg.

Co-authored by Pontus Johansson, Department of Computer and Information Science, Link¨oping University, S-581 83 Link¨oping, Sweden,

email: ponjo@ida.liu.se

This paper focuses on the complex information navigation required by TV viewers. The paper focuses on providing spoken natural language and dia-logue in combination with visual representation for information navigation in the interactive TV context. A running system is built and users’ reactions are evaluated. Data is collected by testing the prototype, observing subjects and interviewing them afterward. User experiences are analyzed and structured into five categories that capturing essential aspects of TV interaction. These cate-gories are; interaction style, information load, data access, effectiveness and in-teraction initiative. From the analysis, design considerations relevant for speech and dialogue information systems for television interfaces emerge.

The results of this study indicate that dialogue interfaces have a significant

(31)

potential for TV interaction. However, results also indicate that the remote control device may still play an important role. A highly heterogeneous viewing audience can access the TV set by providing both speech and remote control interaction.

7.4 Paper IV: Error Resolution Strategies for Interactive

Television Speech Interfaces.

In Proceedings of the Ninth IFIP TC13 International Conference on Human-Computer Interaction, INTERACT 2003, Zürich, Switzerland, pp. 105 – 112. Co-authored by Pernilla Qvardfordt, Department of Computer and Information Science, Linköping University, S-581 83 Linköping, Sweden,

email: perqv@ida.liu.se

This paper explores possible error resolution strategies in the case of mis-recognizing user input to a TV set. From previous research two techniques have been identified: repetition of input by the user, called here correction, and choice of a suitable interpretation from several alternatives, called here clari-fication. From these two techniques, four alternative strategies suitable for a TV environment are designed. These four strategies are tested experimentally with a Wizard-of-Oz method: one with repetition (correction) and three based on choosing the best alternative from an n-best list given in audio and visual mode, alone or combined (clarification audio, clarification visual and clarifica-tion audio & visual).

An evaluation of the four error recovery strategies shows that people prefer clarification to correction for error correction. Between visual and audio clar-ification strategies, the visual n-best list provided better performance because it offered the flexibility to say the number of the item or to repeat the input while audio clarification was found tedious by the participants. When audio and visual clarification were combined, no further performance advantage was found and participants felt that the additional audio information was distracting.

In summary, results show that displaying an n-best list gives the most effi-cient interaction in a TV environment. Furthermore, redundant audio feedback does not influence the performance.

7.5 Paper V: The Paper Remote:

An Augmented TV

Guide and Remote Control.

Submitted 2004 to the international journal: Universal Access in the Informa-tion Society (UAIS), Springer-Verlag Heidelberg.

Co-authored by Erik Berglund, Anders Larsson, and Magnus B˚ang, Depart-ment of Computer and Information Science, Link¨oping University, S-581 83 Link¨oping, Sweden, email: eribe, andla, magba@ida.liu.se

TV guides printed on paper are commonly used to search and find infor-mation about TV channels and programs. This paper focuses on studying the

(32)

potential of augmenting such guides by linking them with TV technology. Three studies are reported in this paper.

The first study focuses on studying the actual usage of paper-based TV guides and how this usage is related to actual TV watching. People who usually use paper-based TV guides were interviewed about their usage habits, needs, and expectations. Results show that paper-based guides are easy to access, read and find. However, they are passive, isolated from the TV environment, and do not provide cognitive help. Thus, disadvantages and advantages of such guides are related to the physical properties of paper. The disadvantages can be eliminated by adding digital technology to provide interactive and TV-connected paper-based guides. This study provides design implications for a new concept called Paper Remote which is an interactive paper-based TV guide and a remote control for the TV. Consequently, a working research system was developed according to these implications and used in the two subsequent studies.

The second study focuses on evaluating the Paper Remote concept. Work-shops are conducted in which the prototype is tested and the concept is evalu-ated both individually and in groups.

In the third study, existing guides are criticized and users’ interests of future TV guides are examined. This study is explorative where adapted cooperative scenario building future workshops are applied.

Results of this paper indicate that pen and paper can, to some extent, be suitable for TV interaction. The properties of paper-based TV guides, based essentially on properties of paper in general, are desired. The Paper Remote concept preserves the properties of paper and at the same time adds interac-tion and computainterac-tion technology which augment viewers’ interacinterac-tion. Another interesting result is the conceptual model of the Paper Remote, see Figure 6. Further, integrating paper and pen with TV technology may provide easy ac-cess for information services usually provided by personal computers and the Internet.

8 Summary and Concluding Discussion

Viewers and users of digital TV face the problem that information navigation by means of remote control is complex. The transition to digital TV has changed the television set into an entertainment as well as information device that pro-vides two-way communication with the user. However, the present remote con-trol device is not appropriate for navigation through the huge amount of services and information provided by the future digital TV, presumably also a device for accessing the Internet. One possibility for coping with the complex informa-tion navigainforma-tion required by TV viewers is an augmentainforma-tion of interacinforma-tion tools available today, based on already existing TV elements and artifacts.

The research reported in this thesis took its departure in an interest for enhancing the remote control with speech interaction for TV information nav-igation. Traditionally, speech interfaces have exhibited several common prob-lems, such as user acceptance and misinterpretation of user input. We thus set

(33)

out to study prospects for improving navigation in the information space of-fered by digital TV by using speech interaction and dialogue. Possible solutions were investigated in empirical and explorative studies with implementations of mock-ups for Wizard-of-Oz studies as well as usability studies of running re-search systems.

Our studies confirm the conjecture that the pragmatic solution of augment-ing remote control devices with speech may provide a suitable means that eases information navigation and search. In particular, a result of our studies is docu-mentation of how the remote control device can be augmented by adding speech and connected dialogue technology. The thesis also illuminates the importance of understanding user attitudes towards speech interaction in order to design acceptable dialogue interfaces. An interesting result of the explorative studies is the indication that the structure of the underlying interaction model may be im-portant for the perceived ease of understanding. Thus, sometimes a three-entity model for interaction, with a clear distinction between the dialogue partners on the one hand and the object domain of the dialogue on the other is preferred. More research is, however, needed in order to explore the importance of this concept.

Nevertheless, the main focus of this thesis project was gradually transferred into a study of more futuristic approaches to substitute the traditional remote control device, thus allowing augmentation and integration of other common artifacts in the home with digital media. TV guides printed on paper may provide such a solution with the potential to also function as a remote control for the TV. In order to study this concept, we developed a prototype system, the Paper Remote, and conducted explorative studies to investigate the potential of this approach. The idea of the Paper Remote is to prepare for a future with ’digital paper’ realized either by an electronic reading device, such as the Anoto pen or by paper with printed electronic circuits. The studies in this thesis used Anoto technology and specially prepared TV guides printed on paper.

User studies that linked digital paper to the TV for everyday information navigation illuminated the potential to provide innovative solutions also for home information systems. Integrating familiar physical artifacts, such as pa-per and pen, into TV technology may provide easy access to information services usually provided by PCs and the Internet. Thus, the same augmentation nec-essary for TV as an entertainment device also opens up new communication channels for providing community information to citizens who do not feel com-fortable with conventional computers. The solutions and results of the studies in this thesis provide a first step in this interesting direction.

In summary, the research presented in this thesis contributes to the task of providing interaction solutions to the complex information navigation required by future TV viewers. These solutions are based on the augmentation of already existing TV elements and artifacts.

(34)

Refrences

Adams, W. J. (2000). How people watch television as investigated using focus group techniques. Journal of Broadcasting & Electronic Media, 44(1):78 – 93.

Ali, A. and Lamont, S. (2000). Interactive television programs: Current chal-lenges and solutions. In Proceedings of the 8th Annual Usability Profession-als’ Association Conference, UPA’00, pages 14 – 18, North Carolina. Arai, T., Aust, D., and Hudson, S. E. (1997). Paperlink: a technique for

hyper-linking from real paper to electronic content. In Proceedings of the SIGCHI conference on Human factors in computing systems, pages 327–334, At-lanta, Georgia, United States. ACM Press.

Arvidsson, F., Ihlstr¨om, C., and Lundberg, J. (2002). Visions of future news - consensus or conflict? In Proceedings of the 25th Information Systems Research Seminar in Scandinavia, Bautahoj, Denmark.

Avison, D. and Fitzgerald, G. (1995). Information Systems Development: Methodologies, Techniques and Tools, 2nd _{ed. London: McGraw-Hill}

Com-panies.

Basili, V. R. (1996). The role of experimentation in software engineering: past, current, and future. In Proceedings of the 18th international conference on Software engineering, pages 442 – 449, Berlin, Germany. IEEE Computer Society.

Beyer, H. and Holtzblatt, K. (1998). Contextual Design: Defining Customer-Centered Systems. San Francisco: Morgan Kaufmann.

Black, A., Bayley, O., Burns, C., Kuuluvainen, I., and Stoddard, J. (1994). Keeping viewers in the picture: real-world usability procedures in the de-velopment of a television control interface. In Conference companion on Hu-man factors in computing systems, pages 243 – 244, Boston, Massachusetts, United States. ACM Press.

Bretan, I. and Kroon, P. (1996). Concurrent engineering for an interactive TV interface. In Conference companion on Human factors in computing systems, pages 117–118, Vancouver, British Columbia, Canada. ACM Press. Brown, R. (1988). Group processes: Dynamics within and between groups.

Ox-ford: Blackwell Publishers Ltd.

Bryman, A. (2001). Social Research Methods. Oxford, UK: Oxford University Press.

Carroll, J. and Aaronson, A. (1988). Learning by doing with simulated intelli-gent help. Communication of the ACM, 31(9):1064–1079.

(35)

Choi, H., Choi, M., Kim, J., and Yu, H. (2003). An empirical study on the adop-tion of informaadop-tion appliances with a focus on interactive tv. Telematics and Infomatics, 20(2):161–183.

Chorianopoulos, K. (2003). The digital set-top box as a virtual channel provider. In CHI ’03 extended abstracts on Human factors in computing systems, pages 666–667. ACM Press.

Clancey, M. (1994). The television audience examined. In Journal of advertising research, volume 34, pages 78–87.

Cohen, J. (2002). Television viewing preferences: programs, schedules, and the structure of viewing choices made by Israeli adults. Journal of Broadcasting & Electronic Media, 46(2):204–221.

Cohen, P., McGee, D., and Clow, J. (2000). The efficiency of multimodal in-teraction for a map-based task. In the Proceedings of the Applied Natu-ral Language Processing Conference (ANLP’00), pages 331–338, Seattle, Washington.

Conway, J. C. and Rubin, A. M. (1991). Psychological predictors of television viewing motivation. Communication Research, 18(4):443 – 463.

Coutaz, J. (1994). Evaluation techniques: Exploring the intersection between HCI and Software Engineering. In Proceedings of the Workshop on Software Engineering and Human Computer Interaction (ICSE’94 Workshop on SE-HCI. Tylor, R. N. and Coutaz, J., editors, Lecture notes in Computer Sience, vol 896,.

Dahlb¨ack, N., J¨onsson, A., and Ahrenberg, L. (1993). Wizard of oz studies – why and how. Knowledge-Based Systems, 6(4):258–266. Also in: Readings in Intelligent User Interfaces, Mark Maybury & Wolfgang Wahlster (eds), Morgan Kaufmann, 1998.

Daly-Jones, O. and Carey, R. (2000). Interactive tv: a new interaction paradigm? In CHI ’00 extended abstracts on Human factors in computer systems, pages 306 – 306, The Hague, The Netherlands. ACM Press. Damper, R. I. and Wood, S. D. (1995). Speech versus keying in command and

control applications. International Journal of Human-Computer Studies, 42:289–305.

Dransfeld, H., Pemberton, J., and Jacobs, G. (2000). Quantifying weighted expert opinion: The future of interactive television and retailing. Techno-logical Forecasting and Social Change, 63(1):81–90.

Ehrmantraut, M., H¨arder, T., Wittig, H., and Steinmetz, R. (1996). The per-sonal electronic program guide – towards the pre-selection of individual tv programs. In Proceedings of the fifth international conference on Infor-mation and knowledge management, pages 243–250, Rockville, Maryland, United States. ACM Press.

(36)

Enns, N. R. N. and MacKenzie, I. S. (1998). Touchpad-based remote control devices. In CHI ’98 conference summary on Human factors in computing systems, pages 229–230, Los Angeles, California, United States. ACM Press.

Ericsson, M. (1999). Supporting the Use of Design Knowledge - An Assess-ment of ComAssess-menting Agents. PhD thesis, Dissertation No. 592, Link¨oping, Link¨oping studies in science and technology.

Eronen, L. (2003). Digital television for all: User preferences and designers’ ? views on what would suit the user. In N. Carbonell and C. Stephanidis (Eds.): User Interfaces for All, LNCS 2615, pages 179–186. Springer-Verlag Berlin Heidelberg.

Fraser, N. and Gilbert, N. S. (1991). Simulating speech systems. Computer Speech and Language, 5:81 – 99.

Freeman, J. and Lessiter, J. (2003). Using attitude based segmentation to better understand viewer’s usability issues with digital and interactive tv. In Proceedings of the 1st European Conference on Interactive Television: from Viewers to Actors? (EuroITV), pages 19 – 27.

Frenkel, K. A. (1989). The next generation of interactive technologies. Com-munication of the ACM, 32(7):872–881.

Furht, B. (1996). Interactive television systems. In Proceedings of the 1996 ACM symposium on Applied Computing, pages 7–11. ACM Press.

Gauntlett, D. and Hill, A. (1999). TV Living: Television, culture and everyday life. Routledge.

Gill, J. M. and Perera, S. A. (2003a). The accessibility of interactive digital television. In Craddock G M, McCormack L P, Reilly R B & Knops H T P Assistive Technology - Shaping the Future. IOS Press, ISBN 1383 813X, pages 500 – 504.

Gill, J. M. and Perera, S. A. (2003b). Accessibility of universal design of inter-active digital television. In In Proceedings of the 1st European Conference on Interactive Television: from Viewers to Actors? (EuroITV), pages 83 – 89.

Goren-Bar, D. and Glinansky, O. (2004). Fit-recommend ing tv programs to family members. Computers & Graphics, 28(2):149–156.

Goto, J., Komine, K., Kim, Y.-B., and Uratani, N. (2003). A television con-trol system based on spoken natural language dialogue. In Proceedings of Proceedings of the Ninth IFIP TC13 International Conference on Human-Computer Interaction, INTERACT 2003, pages 765 – 768, Z¨urich, Switzer-land.

(37)

Hackos, J. T. and Redish, J. C. (1998). User and Task Analysis for Interface Design. New York: John Wiley & Sons.

Hansen, W. J. and Haas, C. (1988). Reading and writing with computers: a framework for explaining differences in performance. Communication ACM, 31(9):1080–1089.

Harper, R. (2003). Inside the smart home. Springer-Verlag London Limited. Harper, R. H. R., Hughes, J. A., and Shapiro, D. Z. (1991). Working in harmony:

An examination of computer technology in air traffic control. In Bowers, J. M. and S.D.Benford, e., editors, Studies in Computer Supported Coop-erative Work: Theory, Practice and Design, pages 225 – 234. Amsterdam: Elsevier.

Hauptmann, A. G., Witbrock, M. J., Rudnickyand, A. I., and Reed, S. (1995). Speech for multimedia information retrieval. In UIST-95 Proceedings of User Interface Software and Technology, pages 79–80.

Heeter, C. and Greenberg, B. (1988). Cableviweing. Norwood, NJ: Ablex Pub-lishing Company.

Hektor, A. (2001). What’s the Use? Internet and Information Behaviour in Everyday Life. Dissertation, Link¨oping. Link¨oping University, (Department of Technology and Social Change).

Herigstad, D. and Wichansky, A. (1998). Designing user interfaces for television. In CHI 98 conference summary on Human factors in computing systems, pages 165–166. ACM Press.

Hornbek, K. and Frokjer, E. (2003). Reading patterns and usability in visual-izations of electronic documents. ACM Transactions on Computer-Human Interaction, 10(2):119–149.

Ishii, H. and Ullmer, B. (1997). Tangible bits: towards seamless interfaces between people, bits and atoms. In Proceedings of the SIGCHI conference on Human factors in computing systems, pages 234–241, Atlanta, Georgia, United States. ACM Press.

Jacobs, G. and Dransfield, H. (1998). Scenarios for interactive tv – europe’s uncertain future. Long Range Planning, 31(3):396–405.

Jacobson, I., Christerson, M., Jonsson, P., and Overgaard, G. (1992). Object-Oriented Software Engineering: A Use Case Driven Approach. ACM Press: Addison-Wesley Publishing Co.

Jan, V. D. and Loes, D. V. (2001). Searching for the holy grail: Images of interactive television. new media & society journal, 3(4):443 – 465. Jensen, J. F. and Toscan, C. (1999). Interactive Television: TV of the Future or

(38)

Johnson, W., Jellinek, H., Leigh Klotz, J., Rao, R., and Card, S. K. (1993). Bridging the paper and electronic worlds: the paper user interface. In Pro-ceedings of the SIGCHI conference on Human factors in computing systems, pages 507–512, Amsterdam, The Netherlands. ACM Press.

J¨onsson, A. and Dahlb¨ack, N. (1988). Talking to a computer is not like talking to your best friend. In Proceedings of The first Scandinivian Conference on Artificial Intelligence,, Tromso, Norway.

Jungk, R. and Mullert, N. (1996). Future Workshops - How to Create Desirable Futures. Institute for Social Inventions, London.

Kang, M.-H. (2002). Interactivity in Television: Use and Impact of an In-teractive Program Guide. Journal of Broadcasting and Electronic Media, 46(3):330–345.

Kerschbaumer, K. (2000). Fulfilling the promise: The cable industry is poised to capitalize on the concept’s long-awaited potential. Broadcasting & Cable, 7(10):22 – 34.

Kim, P. (2001). New media, old ideas: The organizing ideology of interactive tv. Journal Of Communication Inquiry, 25(1):72–88.

Klein, J. A., Karger, S. A., and Sinclair, K. A. (2003). Digital televi-sion for all - a report on usability and accessible design. Available at: http://www.digitaltelevision.gov.uk/dtv for all.html.

Koike, H., Sato, Y., and Kobayashi, Y. (2001). Integrating paper and digital information on enhanceddesk: a method for realtime finger tracking on an augmented desk system. ACM Transactions on Computer-Human Interac-tion, 8(4):307–322.

Koller, F., Burmester, M., and W¨ohr, A. (1997). User interfaces for interac-tive tv – a case study with end users. In Proceedings of the ECMAST 97 conference, pages 327–341.

Kurematsu, A. (1992). Future perspective of automatic telephone interpreta-tion. IEICE Transaction Communication, 1:14 – 19.

Kvale, S. (1996). Interviews: An introduction to qualitative research interview-ing. Thousand Oaks, CA: Sage Publications.

Landauer, T. K. (1987). Psychology as a mother of invention. In Proceedings of the SIGCHI/GI conference on Human factors in computing systems and graphics interface, pages 333 – 335, Toronto, Ontario, Canada. ACM Press. Lee, B. and Lee, R. S. (1995). How and why people watch tv: Implications for the future of interactive television. Journal of Advertising Research, 35(6):9–18.

(39)

Logan, R. J. (1994). Behavioral and emotional usability: Thomson consumer electronics. In Wiklund, M. E., editor, Usability in Practice: How com-panies develop user friendly products., pages 59–82. Academic Press, Inc., Boston.

Logan, R. J., Augaitis, S., Miller, R. H., and Wehmeyer, K. (1995). Living room culture - an anthropological study of television usage behaviors. In Proceedings of the Human Factors and Ergonomics Society.

Logan, R. J., Augaitis, S., and Renk, T. (1994). Design of simplified television remote controls: a case for behavioral and emotional usability. In Proceed-ings of the Human Factors and Ergonomics Society 38th Annual Meeting, pages 365 – 369.

Logan, R. J. and Lenzi, L. E. (1995). Innovations in rca user interface design. Design management journal, 6(4):16 – 20.

Lougos, C., Vassilopoulou, K., and Vrechopoulos, A. (2002). Interactive digital tv services - viewers’ perceptions. In Proceedings of the eBusiness and eWork 2002 Conference: Challenges and Achievements in E-Business and E-work.

Mackay, W., Velay, G., Carter, K., Ma, C., and Pagani, D. (1993). Augmenting reality: adding computational dimensions to paper. Communications of the ACM, 36(7):96–97.

Mackay, W. E. (1998). Augmented reality: linking real and virtual worlds: a new paradigm for interacting with computers. In Proceedings of the work-ing conference on Advanced visual interfaces, pages 13–21, L’Aquila, Italy. ACM Press.

MacNealy, M. S. (1997). Toward better case study research. IEEE Transaction on Professional Communication, 40(3):182 – 196.

Mane, A., Boyce, S., Karis, D., and Yankelovich, N. (1996). Designing the user interface for speech recognition applications. ACM SIGCHI Bulletin, 28(4):29–34.

Marshall, J. P. (1992). A manufacturing application of voice recognition for assembly of aircraft wire harnesses. In Proceedings of Speech Tech/Voice Systems Worldwide, New York. Media Dimensions, Inc.

Martin, G. L. (1989). The utility of speech input in user-computer interfaces. International Journal of Man-Machine Studies, 30(4):355–375.

Maulsby, D., Greenberg, S., and Mander, R. (1993). Prototyping an intelligent agent through wizard of oz. In ACM Conference on Human Factors in Computing Systems, INTERCHI’93. ACM Press.

(40)

Morgan, D. L. (1993). Successful focus groups: Advancing the state of the art. Newbury Park, California: Sage Publications, Inc.

Morgan, D. L. (1997). Focus Groups as Qualitative Research. Focus Groups as Qualitative. Thousands Oaks, California: Sage Publications, Inc.

Morrison, M. and Krugman, D. M. (Winter 2001). A look at mass and computer mediated technologies: Understanding the roles of television and computers in the home. Journal of Broadcasting & Electronic Media, 45(1):135–161. Muller, M., Haslwanter, J. H., and Dayton, T. (1997). Participatory practices

in the software lifecycle. In M.G. Helander, T.K. Landauer, P. P. E., editor, Handbook of Computer-Human Interaction, 2nd_{ed., pages 255–297. Elsevier}

Science B.V., Amsterdam.

Myers, B. A., Bhatnagar, R., Nichols, J., Peck, C. H., Kong, D., Miller, R., and Long, A. C. (2002). Interacting at a distance: measuring the performance of laser pointers and other devices. In Proceedings of the SIGCHI conference on Human factors in computing systems, pages 33–40. ACM Press. Nelson, M. (1999). Remote controls. DigiPoints: The Digital Knowledge

Hand-book, 3(4).

Nelson, M. N., Linton, M., and Owicki, S. (1995). A highly available scalable itv system. In Proceedings of the fifteenth ACM symposium on Operating systems principles, pages 54–67. ACM Press.

Newell, A. F., Arnott, J. L., Carter, K., and Cruickshank, G. (1990). Listen-ing typewriter simulation studies. International journal of man-machine studies, 33(1):1–19.

Newman, W. M. and Lamming, M. G. (1995). Interactive System Design. Addison-Wesley Longman Publishing Company Inc.

Nielsen, J. (1993). Usability Engineering. Academic Press, Inc.

O’Brien, J., Rodden, T., Rouncefield, M., and Hughes, J. (1999). At home with the technology: an ethnographic study of a set-top-box trial. ACM Transactions on Computer-Human Interaction (TOCHI), 6(3):282 – 308. O’Modhrain, S. and Oakley, I. (2003). Touch tv: Adding feeling to

broad-cast media. In Proceedings of the 1st European Conference on Interactive Television: from Viewers to Actors? (EuroITV).

ONS (2002). Family spending: a report on the 2000-01 family expenditure survey, london: the stationery office.

Oviatt, S. L. (2002). Multimodal interfaces. In The Human-Computer In-teraction Handbook: Fundamentals, Evolving Technologies and Emerging Applications, J. JACKO AND A. SEARS, Eds. Lawrence Erlbaum Assoc., Mahwah, NJ. Chap. 14, pages 286 – 304.

Augmenting the Remote Control : Studies in Complex Information Navigation for Digital TV