Easy to Read Digital Equipment for Older, Visually Impaired Users with Cognitive Decline

(1)

Easy to Read Digital Equipment for Older,

Visually Impaired Users with Cognitive Decline

Kristian Mellgaard Nielsen

Master’s Thesis in Innovation and Design, 30 credits Course: ITE500

Examiner: Yvonne Eriksson Supervisor: Ulrika Florin

Expert Supervisor: Björn Westling

(2)

Abstract

This master’s thesis explores what could be the usability affordances of tactile interfaces, imbedded with physical commands, and voice interfaces, when used by older people that have visual impairments and cognitive declines. The aim is to contribute to the development of more accessible media. Through an adapted design thinking approach and qualitative methods, relevant literature, benchmarking on existing technologies, and empirical data, it has showed that the usability affordances of the two interfaces differ from each other. The presence of various usability affordances is found to be decided by individual perception and severity of impairments. Thus, each interface accommodates some users better than others. The outcome of the master’s thesis can be used for future developers of accessible media when exploring new interfaces that could be better suited for people with disabilities.

Keywords: Accessible media, voice interface, tactile interface, usability affordances, visual impairment, cognitive decline.

(3)

Acknowledgements

First, I would like to sincerely thank all the people from both MTM and their partners for you great collaboration and always open mindset when I have been in dialogue with you. A better collaboration partner is hard to find. Next, I would like to thank the supervisors from MDH, you have all believed in my abilities and pushed me with kind moral support. Without you I would not have been able to make the results that I have – thank you. I also thank my family and friends for the unconditional support throughout my studies. In good and bad you have always been there, which I cannot thank you enough for. At last I would like to thank Laura Brihuega García for being an imminence support during this project and in general – you make me a better person.

(4)

Introduction

For many decades, new technologies have emerged and changed people’s lives, making a positive impact (Duque et al., 2019). However, the development has arguably been made with a focus on the ‘many’, which might have resulted in solutions that are not accommodating the skills, limitations and disabilities of the ‘few’. This has had an impact on the accessibility1_of media, as news media changed towards digital platforms, making it difficult for some people to access it. It is explained that: “For most people, technology makes things easier. For people with disabilities, technology makes things possible” (National Council on Disability, 1993). Although the potential is big, technology still needs to be adapted when it comes to media so that mentioned statement can become reality.

User-centered Design is an approach among many others that focus on developing right solutions for a specific target users (Duque et al., 2019). But as many designers are often focusing on typical users2_{, it has also made technologies, and media in specific, more difficult} for certain groups with disabilities. Of great importance in this context, it expected that there will be a growing older population in Sweden with gradual loss of senses over the coming years (Eriksson, 2016). It can thereby also be expected that more people will experience difficulties accessing media through the current developed technologies. According to Duque et al. (2019) the field of design for older users also grows as a result of this, as peoples life expectancy is increasing. We therefore enter a new epoch, where focus will be much more on developing universal design, however to start this movement developers and designers have to act. The bigger issue thereby consists of including older people, with certain impairments/disabilities, into the world of technology. Stigmatizations on age and different disabilities might have led to different design problems or conclusions. Assumptions have left some designers to think that non-tech savvy people cannot use certain technological devices and their needs have thereby been excluded from the design (Pew et al., 2004). Evaluating and developing existing solutions might however have great potential, as the technology might not be too far away from accommodating many more people with just a few alterations. For finding

1_{Accessibility is in this context concerning news, information etc. and therefore includes both text and}

technology.

(7)

the true needs of the older, one needs to include a holistic view that focuses on the specific situation of the user instead of having a fixed view on categories of users (Eriksson, 2016).

(8)

Background

As people are different and thereby have different requirements, media in general will not be as easily accessible for everyone (cf. chapter 1). Media is in this context seen as the communication outlets or tools used to deliver information. Sweden’s parliament has developed an agency for accessible media called Swedish Agency for Accessible Media (Myndigheten för Tillgängliga Medier, hereafter mentioned as MTM), whose purpose is to give equal rights to all people when it comes to media (MTM, 2021a). MTM is a public authority that has the task to make medias accessible for all people in Sweden. “MTM produces and distributes literature and papers in accessible formats for people with reading impairment and also actively participates in the development of available media” (MTM n. y.). From their point of view, and thereby also the Swedish parliament, reading in any form is considered a democratic right for all people, which is why they work on making texts accessible for everyone (MTM, 2021a). This is also the overall problem area that this thesis will work within, that not all people have the same accessibility to media because of different causes.

Initially the project ‘Easy-to-read digital reading equipment’ was established in 2019 internally in MTM to update their current reading device, the so-called Victor Stratus 4 (see figure 1). Victor Stratus 4 is a DAISY player device that can read books, papers etc. via CD or files, out loud. It is used by people that cannot read either newspapers or books because of various reasons. The request of substituting this device for a more ‘up-to-date’ solution comes from MTM. The reason for this is partly because the device is around twenty years old and has not been kept up to date to the new formats and possibilities that the market can offer (U Wikman 2020, personal communication, 22 September). The terms for accessing medias cannot be considered equal as people that are devoted to use a Victor Stratus, which does leave them with the same possibilities as other people.

MTM have continued the development of the Victor Stratus 4 but concluded that the users need some things easier to use, which the current form is not supporting (U Wikman 2020, personal communication, 22 September). A new solution should first of all meet the needs of people with biggest reading impairments but also be suitable for expanding to other groups of MTM’s clients (U Wikmann 2020, personal communication, 22 September). The reason behind this that if it can help those with the biggest needs, it can theoretically help everyone. A perspective that I agreed very much with and makes sense as it is possible to build on this base if some

(9)

people have higher requirements. A successful thesis project will therefore contribute to equality for everyone when accessing media.

Figure 1: Daisy spelare – Victor Stratus 4

2.1. Establishing Collaboration on Thesis Project

MTM and Mälardalen University have an established agreement of collaboration (MTM, 2019), which means that the two institutions have shared interests within the area of accessible media. MTM’s project was therefore presented to the students on the Master’s Program in Innovation and Design in the fall 2019. The project was initially brought up by a group of students including myself, however we realised that due to the projects’ complexity, it needed more time to come up with more valuable knowledge. Therefore, I contacted Björn Westling who is a part of both the MTM organisation as well as Mälardalen University. We initiated a further collaboration on the same project, which would become this Master's Thesis project (B Westling 2020, personal communication, 10 September).

On different meetings, both parties and relevant stakeholders were introduced and updated on the project progress. Different physical reading devices have since the start of the project been analysed by MTM but none of the options have been assessed to be sufficient (U Wikman 2020, personal communication, 22 September). Based on their initial pre-study, it has shown that the most interesting platform to continue with is the Android platform in the physical form of a

(10)

tablet. Currently MTM and Textalk3_{have developed a prototype in the form of an app including} an overlay. This will be included in this thesis work.

In close dialogue, I and MTM have discussed how the thesis work can contribute to the MTM project without replicating. It was discussed that the project and its overall aim will also be the aim of this thesis (further elaborated in chapter 2.3). However, the thesis will only take part in specific aspects and areas where all parties find it relevant to conduct extra research, and thereby having a narrower focus. This is due to the individual deadline between the parties, as well as the extent of the MTM project, which is much bigger than the what can be done in a Master’s thesis project. The decision was therefore to keep the thesis work close to the MTM project (U Wikman 2020, personal communication, 10 October), which theoretically should ensure that synergies are utilized. Separating the thesis and MTM project, but still keeping them connected in terms of sharing found knowledge and results will make it possible still to contribute. Both parties saw a possibility to reach out of the MTM project, and explore the possibilities of a voice interface but related to their prototype. Thereby the thesis will contribute to the field of innovation and design with valuable knowledge for developers that wish to improve the accessibility of media. Comparing the two interfaces, voice and tactile/haptic, will contribute with knowledge about where each solution thrives related to the user needs and challenges. To get a more accurate outcome of the thesis, it was decided that a narrowed target group would be necessary. The decided focus was thereby to look closer into the users that are older, with visual impairments and have a cognitive decline, as they constitute a big part of MTM’s user base (U Wikman 2021, personal communication, 10 February). These definitions all contribute with some characteristics of the target group and are further described and elaborated in chapter 2.5.1.

2.2. Motivation

First of all, my motivation for the collaboration came from the first contact I had with MTM on my first semester of the Master’s Program in Innovation and Design in the fall 2019. From this

3_{Textalk is a company that develop holistic solutions that help publishers (amongst MTM) get more, and reach}

(11)

first collaboration I had good communication and saw a great potential for continuing to work together as the outcome of a thesis project will be very dependent on this.

Secondly, my motivation comes from a more personal experience, as I have had a grandpa who used a similar device. Had it not been for such a device, it would have been nearly impossible for him to be updated on news or read any books. The solution had a very positive impact on his day and contributing to other people's life by improving this solution even further, is a very motivating thing for me. In several situations I remember coming home to my grandparents to visit them, where my grandfather would always want to discuss the latest news or even report about the latest books he had been listening to on the device. Many talks were a direct result from listening to the reading device and without this, the whole relationship might have been different. Not only did the device make it possible for my grandfather to listen to books and keep himself up-to-date on the latest news, which in my personal opinion is a fundamental right for everyone, it also creates a better connection to the people around him. Improving such a reading device, or as in this case developing a better one, is therefore from my perspective something that can give a lot of value, not only for the user itself but also the people around. The project was thereby very appealing and easy to see myself spend many hours on, as my thesis project. Also, from a professional side I am as an innovator and researcher curious to explore new and better ways, that can improve the daily life for people. This specific challenge is very appealing as it includes a strong focus on the users and their needs. A challenging process where collaboration with experts as well as staying updated with the user is key to the success of the project.

2.3. Problem description, Aim and Objective

The identified problem is that the emerging news media platforms, are difficult to access for older people with visual impairment and cognitive declines. Previously the formats that news and other media comes in was supported by the Victor Stratus 4 (see figure 1) but as new formats have appeared with new technologies, it no longer supports the most important formats. Therefore, new and better alternatives to the old reading device, the Victor Stratus 4, need to be found so that the target group can access media on equal terms with other people.

Existing technologies are the obvious choice to explore as they might only need smaller alterations to fit into the user needs of the target group. However, the knowledge area on usability of the two interfaces, tactile/haptic or voice, related to the target group is barely

(12)

knowledge to the field of innovation and design as well as developers such as MTM when trying to improve accessibility of media.

This work thereby first aims at exploring how new interfaces, in terms of technology, are being developed to accommodate the user needs. This will locate areas of improvements as well as increasing knowledge on target group and their needs. Exploring the user needs are therefore an important part of the aim. Secondly it aims to explore the usability affordances for the target group when using the two interfaces, tactile and voice interface.

The objective is therefore to translate the usability affordances into a theoretical guideline that can help future providers when assessing which interface that can suit the individuals within the target group best (see target group in chapter 2.5.1).

2.4. Research Question

To guide the research, following two research questions have been developed:

• How are the two types of interfaces, tactile reading devices imbedded with physical commands, and voice interface devices, developed to accommodate the needs of the target group?

• What could be the usability affordances for the target group when using tactile reading devices, imbedded with physical commands, and voice interface devices when accessing media, respectively?

2.5. Scope and Delamination

MTM and other stakeholders have stated that there is a need to investigate the usefulness of using tactile interface versus using voice interface device on mentioned target group (further elaborated in chapter 2.5.1). The project’s scope is therefore these two interfaces and delimits itself from other variants of reading aids (such as Braille). In terms of reaching a valid conclusion, it has been decided to have a more specific target on the group of users who are older, visually impaired and have a cognitive decline. The study does not take the specific degree of a condition into consideration as the outcome might become too complex and the results ungeneralizable. It does however recognize that individualities exist (inter terms of disabilities etc.), but will only consider more generalizable matters as this study is made in the field of design and not in the field of for example medicine. Furthermore, the study concerns

(13)

the usability from the target group’s perspective, the end-users, and is therefore devoted to this perspective, rather than for their relatives, caregivers, etc.

Due to the pandemic situation and the travel restrictions caused by Covid-19, the research has been conducted in Denmark as I have been situated there. However, the collaboration has been done primarily with MTM, situated in Sweden. This means that the research has been performed in both Denmark and online with stakeholders in Sweden. A quick cultural comparison between the two countries show, according to Hofstede Insights (2021), that the two countries are indeed very comparable. So, although there might be smaller variations between the countries, collecting data in both countries and merging these should not have any big implications or reason for not being able doing so.

The outcome is obviously made to create valuable for MTM and their partners, however the outcome is not only limited to these. The scope is therefore to create knowledge that can be used by various developers as well as researchers within the field of innovation and design, when developing accessible media.

2.5.1. Target Group

In collaboration with MTM, it has been agreed to define the target group as older people with visual impairment and cognitive decline. From their expertise, these criteria are what usually defines the target group, and those with the strongest needs (MTM, personal communication, 2021). Although it is preferable first to identify the context instead of the user, this has been done as these are the user’s MTM have identified from their past experience. Still, the user needs are very important and have to be identified.

Aging and the term ‘older’ is in this context important as it is understood to be something that has had an effect on sensorial capabilities. Seen from a phenomenological perspective, an individual can or cannot consider themselves as ‘older’ no matter age. However, in this context it is mainly related cognitive and visual functioning, that is decreased as a result of aging. But the term is also important in the perspective of acceptance and understanding of technologies, as this might also have an influence on the result of this work. It is thereby not a stigmatization of older people, that they are not tech savvy, but a recognition that their perception and use of technology is simply different than other age groups. Although individuality has a big role to play, there is a pattern showing that when people reach middle age, their level of comprehending verbal meaning is decreasing (Pew et al., 2004). Being older therefore has a

(14)

close connection to cognitive decline, but contributes with other traits such as perception of technology, and thus it is necessary to include the term ‘older’.

Next it is important to know what is more specifically meant by a cognitive decline or with other words cognitive impairment. CDC (Center for Disease Control and Prevention) describe cognitive impairment to be “when a person has trouble remembering, learning new things, concentrating, or making decisions that affect their everyday life” (CDC, 2021). Yale University (2021) describe cognitive, learning and neurological disabilities as a neurological disorder that impact how well people comprehend information. This understanding therefore implies that tasks in general are more difficult for people with cognitive declines, if not impossible, depending on the complexity. American Psychology Association (2003) more specifically specifies cognitive impairment into four categories as listed in table 1 below.

Severity of Cognitive Impairment

Mild Able to function on a daily basis with minimal

level of support

Moderate Able to take care of themselves but might

need vocational support and support with household

Severe Limited communication skills, but often have

the ability to understand speech

Profound Not able to live independently and need help

and supervision throughout the whole day Table 1: Overview – Severities of Cognitive Decline (American Psychology Association,

2003).

Defining visual impairment, Yale University (2021) state that they can vary over a range from mild or moderate vision loss on one or both eyes to substantial or complete loss of vision on both eyes. IBOS4_{define being visually impaired as having reduced vision, that cannot be} corrected by glasses, lenses or other help remedies (IBOS b, 2021). It is further elaborated that

(15)

a visual impairment cannot exclusively be a diagnosed by a doctor such as glaucoma or by looking at a person’s visual acuity. As they state, it is important also to focus on the individual’s ability to do various tasks as these show the true impairment (IBOS a, 2021).

Visual impairment is showed by the fact that the person cannot distinguish details at normal distance, called lowered central vision, or is overlooking things, called field of view defects (IBOS b, 2021). From the definition and characteristics, visual impairment will therefore obviously be an obstacle as they will have a difficult time distinguishing the different buttons on any given device. Blind people are defined by IBOS as if one has no sense of light (IBOS b, 2021). Therefore, this research will delimit itself from stating the user’s specific impairment. It will however try and include the different levels as an overall indication. The criterion for people included in the target group have a visual impairment that cannot just be corrected by glasses and having an impact on the person’s ability to read books and use digital reading equipment.

Fundamentally, the definition of the target users depends on visual impairment and cognitive declines. The degree of the user’s impairments is something that cannot be precisely specified, only theoretically. The people within the target group are therefore theoretically, in one way or another, placed on the lined area as illustrated in figure 2. The figure illustrates that although the target group has been characterized by impairments/disabilities, there are big variations depending on the individual. It is important to notice that the illustration does not take the term ‘older’ into consideration as it is a partly included in cognitive decline. Yet it is still separated as it also relates to individual perception towards technology.

(16)

Figure 2: Visualization of the users’ combined severity of impairment – Visualized by Kristian Mellgaard

In Denmark there are no exact numbers on how many blind or visually impaired people there are due to principal and historical reasons, neither cognitive impaired. However, the estimated number according to The Institute for the Blind and Partially Sighted (IBOS) is that 65.000 people are visual impaired (IBOS a, 2021). Of those people, around 90% are more than 75 years (IBOS b, 2021). In Sweden, it is estimated by Funka5_{(n.y.) that at least 30.000 people are} severely visually impaired or completely blind whereas from the number of people enrolled in the national vision centres, around 130.000 people are counted as visually impaired. Combined, the two countries thereby have approximately had 195.000 people that are visually impaired. Of these it is difficult to estimate who has a cognitive impairment, however as the request of defining the focus group also with cognitive impairments from MTM concludes the relevance for studying this focus group. So, although the study is devoted to people with both visual impairment and cognitive decline, which is a relatively small group of people, it still has potential to contribute with knowledge that can affect other groups.

(17)

Literature Perspectives

According to Bella and Hanington (2016) literature reviews are an integral part of academic papers. They are a useful component to collect and synthetize existing research (Bella and Hannington, 2016), which gives valuable insight on the given topic. This also justify bringing in relevant literature as yet another contributor of perspectives on the situation, which widens the perspective of the research (Kumar, 2013). These perspectives have been used as inspiration for collecting literature that can broaden the perspective on the topic. Throughout the project, relevant literature appeared from new searches and from people that had found new papers etc. It has been necessary to split these searches for literature into separate searches, as it has been nearly impossible to find papers that cover all subjects at once. Mainly, the search has been divided into two categories, one for voice interface and another for devices embedded with physical commands. Then another two searches have been performed in each of these reviews, where one focuses on catching literature that concerns visual impairments and another on cognitive declines. Although these insights are technically separated, they are still very useful for the research and conclusions as the different outtakes will be merged. So even though one paper might only cover research on one part of the topic, it can still help creating a more whole picture of the target group of this work.

Beside the structure of the literature search as explained, there has also been used snowballing to find other relevant papers that were not found in the search. MTM have also provided useful insights from a similar organisation in New Zealand, that has done research on voice recognition technology. Scopus and Google Scholar have been used as search engines and most of the collected on 17th_{of March to 5}th_{of April 2021 (see search strings in Appendix A).} Ultimately, the combined literature will be able to cover the use of tactile interface devices and voice interface technology on older people with visual impairments and/or cognitive declines. Lastly, a short research has been conducted in usability as this will define how usability is seen in this project. The literature has been provided by experts on the area rather than conducting a comprehensive literature research. This has provided sufficient material to state the overall frames of usability. These will then later be used in comparison with the empirical findings, to find the usability affordances of the target group.

(18)

3.1. Tactile Interface

Ease of Use

Obviously, standard touchscreens can be very difficult to handle for visually impaired or blind people, simply because it is difficult, or nearly impossible, to register the output from a screen. There are however research that shows, that adding vibrotactile and/or auditory feedback can successfully improve the use of the visual impaired user (Horton et al., 2017). Another study also shows that auditory output enables especially visually impaired people to interact with a touch screen (Hsiang et al., 2012). It therefore also seems that the device itself is not sufficient and needs other types of feedback for the visual impaired user. Without any other attributes, the user has too little navigation when using the device, and some sort of auditory or haptic feedback that can guide the user is necessary. In the example with Yuan-Hsiang et al. (2012), they invented a solution that gave the visual impaired user an auditory feedback, which showed to sufficiently help the user navigating the device/prototype.

Haptic & Tactile Aids

According to Khan and Khusro (2019) vibrotactile and haptic feedback connected to

touchscreens has the potential of making the device easier to handle. It can guide the visually impaired user and could make it possible to do tasks that could not be performed without vibrotactile and haptic feedback. It is further mentioned that these aids are still to be

considered as aids that complement a system (Khan and Khusro, 2019). Simplicity or at least guidance through the interface is therefore still critical as tactile overlays or variants of haptic feedback can only contribute but not stand alone. A complex system will thereby continue to be difficult for visually impaired users even though it has helpful tactile and/or haptic

feedback. Also, the aids have to match the profile of the specific user. Relating this to the target group of the study, it is not enough to have one set of user-interface components as they will not be fit for all (Khan and Khusro, 2019).

Impediments

According to Nystedt (2019), technologies have the potential of helping people with cognitive impairments. There are however different impediments that leaves them unused. One comes from the internal barriers such as mental imitations, linguistic and verbal comprehension (Nystedt, 2019). Elementarily, cognitive impaired users often fall to understand or recognize basic computer characteristics. Also, multiple steps to do certain action are very much a challenge as they have restricted memory and thereby cannot do too complex tasks (Nystedt,

(19)

2019). As the external barriers we have the fact that many new technologies are not designed for these users. They therefore have a higher focus on intuition and appearance rather than user needs for these people (Nystedt, 2019). This does however still leave room for the technologies to be adapted, as there are different ways of coping with these issues. Regarding external barriers Ghiani et al. (2009) has showed that merging existing technology with relevant aids, in this case vibrotactile feedback, it is possible to accommodate the impaired users’ needs. Also, when it comes to the internal barriers, simple guided apps to an existing technology, as in the case with the study of Yuan-Hsiang et al. (2012), can simplify the use of these technologies and thereby making them easier accessible. In both cases it has however needed a strong focus on the users’ needs and just as important their situation.

Reducing Errors

As visually impaired people seem to have a more difficult time handling devices and managing commands to make actions on a device, exercise might improve their skills drastically. Yuan-Hsiang et al. (2012) found that after a few exercises, their visually impaired participants, in a test of a smartphone-based digital talking book player, decreased both the time consumed on tasks and errors made. The use of a physical device for visually impaired people thereby implies that the tasks in comparison with a regular user might take longer time. However, with practice, the time to perform the tasks has the potential to be reduced (Yuan-Hsiang et al., 2012).

3.2. Voice Interface

User Independence

As the technology has developed, Voice Recognition Devices and similarly ECU’s (Environmental Control Units) can today significantly improve the quality of life for users with physical or functional disabilities. According to Noda (2018), the ability to control or manipulate the surroundings is something that highly improves the disabled users’ life in general, and thereby improves the quality of life. This also implies that the demands on family, carers or attendants is lowered (Akour, 2020; Noda, 2018). As a result, the user is more independent, which has an effect on quality of life. Their studies concerns surroundings, however the overall outcome is that when the user have the possibility for doing things themselves without assistance, their quality of life increases. Lancioni et al. (2020) reaches somewhat the same results, stating that users with intellectual and visual disabilities also rate voice recognition technology very high. Also the staff of these participants rated the system

(20)

positively (Lancioni et al., 2020). According to Garg et al. (2020), voice assistants are able to cover visually impaired people’s needs.

Errors are Insignificant

Comparing older users with young adults and adults, the older seems to experience more errors (Barros and Seabra, 2020). But although the technology might not catch every word the user says, the systems are now developed so well that the user has easy access back to a known starting point (Hae-Duck J. Jeong et. al., 2014). The referred study by Hae-Duck J. Jeong et. al. (2014) concerned visually impaired people in general, and had clear evidence that the structure of most modern systems allows the users to move back to a known position. This meant that the visually impaired user, and users in general would not end up being stuck, neither need a screen to recover from this.

Mixed Complexity

Kim (2018) concludes that simple commands can easily be performed by visually impaired people, however there are still experienced issues both before, under, and after using applications. Signing up with username, password, and similar is still almost not supported by the system, making these commands nearly impossible for the user (Noda, 2018). This is the same results that appeared in the studies of Barros and Seabra (2020) as they state that older people express tasks with multiple interactions are more difficult. These implications are therefore only occurring when more complex tasks have to be performed. When the tasks or needs are simple, there are no real complications as the system is in general low on complexity.

Intuitive

The technology does not seem to bring up any bigger problems for the user, as the easiness and various options allows the user to progress with simple response (Lancioni et al., 2011a). The referred study by Lancioni et al. (2011a) was a study performed with two women with multiple disabilities, also visual impairment, on a system that is now around ten years. It also means that the software already then was sufficiently developed to contribute to a better life, as the system could easily be navigated by the two users. The same results was later replicated also by Lancioni (et al., 2011b), which showed that the users with multiple disabilities, including visual impairment, rapidly learned how to use the system and did not need external help. This was a result of the intuitive interface and the systems’ ability to catch spoken sentences. This is a strong asset of voice recognition technology, that there is a very natural aspect to it. For humans it comes very natural using the spoken language and technology is now supporting this, which

(21)

makes it possible for human and machine to interact (Nuanmeesri, 2020). The system does therefore not require a specific command but is able to catch the non-linear phrases that it is presented by people. The studies do however not take cognitive decline into consideration, where some issues might appear.

Issues with Portability and Low Voice

Although the literature shows that the technology has as many upsides, there is a clear indication that there is a missing option of portability of such system. As in the case of the study performed by Lancioni et al. (2011b) it is something still to be improved, something that is still relevant today. Lancioni et al. (2011b) suggests that the technology should be designed in a compact manner that can then be brought with the user. However, the implication of being in an open room, outside of one’s home, has not been addressed.

Another limitation for voice recognition technology, that is recently addressed by Ismail et al. (2020) is its issues detecting low voice. In general, Ismail et al. (2020) found that their solution was 97% accurate tested on older people, patients and disabled people. However, if the user’s voice is affected by illness or is not clear enough, the system cannot detect the commands that it is given and will in these situations therefore not be suitable (Ismail et al., 2020).

Time per Task

Cognitive decline has an influence on the use of voice recognition technology as people with cognitive declines use longer time than regular users, however their tasks are still, to some extent, facilitated (Barros and Seabra, 2020). In a usability assessment of Google Assistant and Siri by Barros and Seabra (2020), it was found that older people were typically twice the time doing tasks compared to a regular user. Although they seem to do the same number of tasks in the end. The research also showed that the older had some tasks that were much more difficult that others such as adding a predetermined reminder in the calendar and sending a pre-established message with WhatsApp (Barros and Seabra, 2020). Whereas consulting the system for a weather forecast seemed rather easy (Barros and Seabra, 2020). Yet, they did not address users that have any severe cognitive declines. It therefore only relates to people with mild cognitive declines.

Missing ‘Follow-Up’ Function

In an assessment of Google Assistant, Amazon Alexa, Apple Siri, and Microsoft Cortana, it is shown that on parameter such as effectiveness and usefulness, the systems are mostly scoring

(22)

as the ability to, after the interaction is finished, to propose a suggestion and execute other related activity, all of the systems had a rather poor feedback. This could be a problem especially for people with a cognitive decline as they might not have the ability to continue the interaction with the system to follow up for other relevant feedback. They on the other hand might need a system had provides them the relevant feedback without having to think of what action to take next.

3.3. Central Theoretical Concepts

3.3.1. Usability

According to EU, “Usability” is something that determines the quality and acceptance of a system (EU, 1993). It is therefore something very important to think of when designing or investigating interfaces on various devices. To frame usability in relation to this project, and what it means, relevant literature has been collected and compared.

Usability comes from ‘user friendly’, which in the 1980’s had been used in broad sense but was by many seen as insufficient, and therefore the appearance of ‘usability’ (Nigel Bevan et al., 1991). There are according to Nigel Bevan et al. (1991), several definitions of usability depending on which perspective that is taken. They mention mainly three views on how usability should be measured, which includes a product-oriented view, user-oriented view, and user performance view. However, the complete definition must encompass all these views (Nigel Bevan et al., 1991). They define usability as: “the ease of use and acceptability of a system or product for a particular class of users carrying out specific tasks in a specific environment; where ‘ease of use’ affects user performance and satisfaction, and ‘acceptability’ affects whether or not the product is used.” (Nigel Bevan et al., 1991). Federici and Borsci (n.d.) defines in a simpler way, as usability is evaluated “by the quality of communication (interaction) between a technological product (system) and a user (the one who uses that technological product).”.

From these perspectives’, usability can be understood as how easy a given technology is to use. Not to confuse it with ‘user friendly’, as it is more complex with several factors such as context, acceptability, environment, comfort, time spend, and satisfaction (Federici and Borsci, n.d.; Nigel Bevan et al., 1991). These are obviously connected to one another but still they stand for different aspects that need to be considered for assessing usability. The literature does however not mention the importance of each aspect, which must be determined to be different depending

(23)

on the target group. This must therefore be discussed in the specific case, or addressed by either expert in the field, or from the users themselves. Below is the table that shows the found parameter that affects usability.

Found factors of usability

Context Acceptability Environment Time Spent Satisfaction Comfort

Table 2: Found parameters of usability – Made by Kristian Mellgaard

3.3.2. Affordances

James Gibson introduced in the 1960’s the concept of affordance. Although he uses the term at an early stage in several papers as well as a book from 1966, the best-known definition comes from the book The Ecological Approach to Visual Perception, published in 1979. Gibson (1979) explain that affordances are what environment offers the animal. This can be what it provides the animal for good as well as ill. Affordances can therefore be seen from the initial definition made by James Gibson to be something that substances, medium or surfaces affords or provides the human/user to do or not do.

Masoudi et al. (2019) are some amongst other that have interpreted Gibson’s concept. They suggest that there is an interaction between what Gibson call environment and animal, or what can be understood in this context as environment and user (Masoudi et al., 2019). Affordance can in this way be said to be rather simple, as for example a screen without touch function can be perceived to afford touch, although there is no feedback. Norman (1999) explain that, “Although all screens within reaching distance afford touching, only some can detect the touch and respond to it. Thus, if the display does not have a touch-sensitive screen, the screen still affords touching, but it has no effect on the computer system.” (Norman 1999, p.39). Affordances does therefore relate to for example clicking on a device is meaningful, useful action, with a known outcome (Norman, 1999).

(24)

With the mentioned above, it is thereby suggested that affordance is not relating to the expected action neither output that is for example shown on a tablet when the user is interacting with the device. This is something that can be discussed whether or not is true, as it can be argued that symbols, colors, touch function, etc. are also proving something to the user. In this work, it is therefore argued that these outputs and actions are also linked closely to affordances as they are seen as part of the wider picture.

Lastly, individual perception also plays an important role and must be considered. (Masoudi et al., 2019) describes that affordances are also relative to the size of the individual. They thereby relate to the individual’s perception which relates to the mind, but also to the individual’s physical abilities, shape, etc.

3.3.3. Connection Usability and Affordance

From the previous sub-chapters, usability and affordance can be seen as closely connected as affordances can help indicating different aspects of usability. Connected, it is therefore possible to investigate the affordance’s’ influence on usability. Affordances have been defined as something that the environment provides the user. Affordances therefore becomes a critical concept, which enables me to later explain and assess what could be the usability affordances of the two interfaces, voice and tactile.

3.4. Summary and Part Conclusion

The literature has shown that both types of interfaces, tactile and voice, can contribute to independency for the user as well as lighten the burden of relatives of the end-user. When it come to the positive sides of the physical command devices, there are clear positives about its function when other sensorial aids are added. Visual impaired people need guidance when using these, which can be met my audio output or vibrotactile feedback. This is something that is necessary as studies has shown that with a visual impairment it can be very difficult, if not impossible, to access a touchscreen that does not come with any feedback. Guidance is likewise something that is important for accommodating users with cognitive declines as too many steps or unfamiliar linguistic becomes big obstacles. Also, for visually impaired people it becomes very important that the device does not need too many steps to perform tasks, as these can lead to confusion and in the end leave the user stuck. The time to perform tasks seem to be affected by both cognitive decline as well as visual impairment in comparison with typical users. This might come from external barriers as the development of the existing technology has been

(25)

focusing on appearance and intuition rather than these users’ needs. Still, smaller alterations or developments can make them easier to access.

From the collected literature, different patterns have shown both positives and negatives. The two solutions show that each has a great potential to help these user needs as represented by collected positive aspects seen in table 3.

Physical Command Device Voice Recognition Technology

Mobility Natural way of interacting

Tactility Cannot get stuck

Possibility of adding aids Easy to learn

Surroundings does not influence interaction Basic tasks are easy to complete

Table 3: Positive aspects in consideration of visual impairments and cognitive declines – Made by Kristian Mellgaard

Generally, it can be concluded that the absence of usability guidelines for the target group is partly due to the fact that people with cognitive disabilities have diverse needs. They vary depending on the individual, making it difficult to create universal guidelines (Nystedt, 2019). As an outcome, both solutions show negative sides that does not accommodate these users, some more critical than others. As mentioned above they both have positive sides that also applies for the regular user, which in this case also turns into less fortunate aspects. These are listed below in a random order.

Physical Command Device Voice Recognition Technology

Needs added aids (improved tactility, audio) Low voice Multiple interactions Tactility

Recovering from mistakes Not possible to add aids

(26)

Table 4: Negative aspects in consideration of visual impairments and cognitive declines – Made by Kristian Mellgaard

Previous research has shown different positive and negative aspects in consideration to the users’ disabilities and most importantly their needs. This intel is very much a part of the total understanding of the target group, however a more in-depth exploration on these insights will be necessary for a better understanding.

Lastly, the collected literature on usability has shown that context, acceptability, environment, time spend, and satisfaction, can be important to assess the usability of any given device or interface. The aspects do however not show which is more important as these are more general and there is therefore still a task on figuring the importance of these factors. These will be addressed in the empirical data seen in chapter 6.

(27)

Benchmarking

In order to improve the understanding of the existing voice interface and tactile interface technologies that are currently available on the market, I have performed a benchmarking overview. This chapter includes an overview of two voice interface systems, Google Assistant and Amazon Alexa, and two tactile interface system, DORO phones and MTM’s new prototype Dagny. The two voice interface systems are some of the most used voice recognition assistants on the market, and can give a well-founded background explanation of the potential that such systems can provide. DORO provides insight in a product assortment of phones that have had great success the past years in the senior segment. And last, Dagny can provide insight in some of the newest developed systems developed specifically for the defined target group.

4.1. Touch Interface

4.1.1. DORO

DORO is well known for their simple phone with big digits that enable seniors to use functions on a phone that they could not do on a regular phone such as an iPhone. Although DORO are famous for their big digits, their solution is consisting of something much more when looking at their webpage. All in common is that they are tailored for older people, which means that they come easy to use, extra loud, clear sound, large separated keys, and high visual contrasts (DORO, 2021).

As mentioned, DORO has several phones and smartphones in their range, two of them are shown in figure 7. The simplest phone consists of only a few buttons and no complicated functions and menus. Basically, it has been boiled down to the most important functions. The numbers that usually constitute a bigger part of the keyboard have been replaced with only three buttons that each will make a direct dial to the number it has been programmed for (see figure 7).

The smartphone version comes with much more functions that almost reminds of a normal smartphone. However, it differentiates itself as it features conversational navigation (Doro, 2021). Google Assistant is also a part of the software, which helps the users find something that they cannot find by themselves (Doro, 2021). The software on the smartphone is the well-known Android, which has been altered to simplify the complexity of the user interface

(28)

by DORO, but rather changed and altered to meet the user needs. They have thereby used existing technologies but focused on designing these so that they fit into their target group. This has provided a different kind of innovation that is for example seen on Google Home and Amazon Alexa.

Figure 7: DORO Phones – Easiest phone version on left and Smartphone on right Response by Doro

With all DORO mobiles there is free access to what DORO call ‘Response by DORO’, which is a function that sends out alert out to responders simultaneously (DORO, 2021). A press of three seconds on a button on the backside of the phone will then automatically call the people that are programmed as responders on the mobile, these could for example be relatives. They will then be able to give remote assistance to adjust settings on the phone that the user itself could not find. In situations where the user has difficulties, responders will thereby be able to help them without having to go there physically. Figure 8 shows how the app looks from the responders’ side and which adjustments they can help out with.

(29)

Figure 8: Screenshot of Doro App – ‘Response by Doro’

4.1.2. MTM Prototype – Dagny

MTM have in collaboration with Textalk, developed an app called Dagny, that can read the news out loud and works on all Android platforms. The app has 12 different versions/templates, which means that the user can decide how many and which functions they wish. These are being called ‘templates’ where template 1 (see figure 9) is the simplest with the options, listen, volume-up and volume down. Template 12 comes with twelve functions (see figure 9) with extra functions such as changing to a new article, changing the pace of speech, listen from the start etc. The app is limited to news and comes with different levels of complexity, which can be chosen by or for the user. Its intended use is on bigger screens, making it possible to use for those that still have some visual abilities left.

(30)

Figure 9: Screenshots of MTM’s prototype ‘Dagny’ that works on Android devices. Template

1 on left, template 5 in the middle, and template 12 on the right.

The app therefore comes with different functions, although some are present in all templates. The permanent functions and selectable ones are listed below.

Selectable functions:

- Increase/decrease speech pace - Next/previous article

- Next/previous paper - Where am I?

- Listen from beginning - Next/previous section - Next/previous sentence Permanent functions: - Listen/stop listen - Volume up - Volume down

Tactile Overlay and Audio Output

The interface consists of touch on the screen but also an overlay is being developed. The first prototype of these overlays, that are supposed to fit the specific template, is so far developed on a basic level only for template 12 (see figure 10). The idea behind the overlay is to give the user some guidance when they use the device. The added tactile overlay thereby adds a dimension to the system, which can accommodate visual impairments as well as physical impairments, so that they are pressing the intended button.

(31)

Further guidance comes via audio output when buttons are pressed. The app will then tell which button has been pressed to inform the user. Some of the templates comes with a button that tells the user where they a located in the system. By pressing the information symbol, the user will be told what they a listening to and in which paper.

Figure 10: On left - overlay seen alone. On right – overlay seen on top of tablet displaying MTM prototype ‘Dagny’ template 12.

4.2. Voice Interface

4.2.1. Google Home/Assistant

Google Home/Assistant is a system that can detect human voice and act on this input with either auditory output or actual actions. Auditory output can come in form of speech that can provide information, news, etc. Other units can also be connected to the system, such as thermostats that can change the temperature in a room or changing the light setting. It is almost only the imagination and the user’s skills that decides its limits (Google, 2021a). The system makes it possible to control the devices via voice only, and can recognize individual voices. It can thereby be programmed so that it only acts on the voice of certain users (Raphael, J., 2021). Beside

(32)

The System

The control unit comes in the form of a Google Home smart speaker, however the system also works on an Android phone or tablet (Google, 2021b). As a full package it can control all items that have a smart function but it can also work only to control the specific tablet or smartphone (Google, 2021c). It is designed to cover more or less all the potential need for all users, which means that the potential for the system seems unlimited.

Key features of the system are that it: • Is provided on different platforms

• Can be adapted to the specific needs and wished commands • Can be programmed to only react to specific persons • Can control adaptable units in the home

• Access Google as the search engine

Figure 3: Various commands – Google Assistant/Home Built-in Intents (BIIs)

The commands do not have to be ‘spot-on’ as the system takes the context into consideration. The programming of any app as developer, is therefore simple, as Google itself takes the various ways of giving this exact order into account, making the system catch such commands almost no matter what is being said. Maintaining these BIIs is not something that the app inventor would have to do themselves as Google keeps these up to date (Google, 2021c).

(33)

Relevant add-ons

As something extra, different equipment such as speakers and headsets can be used to control the device. The extensions can therefore reach over a big range of equipment that might be relevant for the user in terms of making the usage easier.

The system can also be modified so that the specific app opens and acts depending on the user’s commands via voice. Meaning, that it is possible to build functions into an app so that when the user for example say ‘Google, read the news out loud’ the system will register this and open the specific app and read the news out loud. This is instead of opening standard apps and the chosen/developed app will thereby get priority on the system.

4.2.2. Amazon - Alexa

Alexa is in many ways like Google Home/Assistant as it is a system that can register the voice and return with information or even do commands. It is linked with thousands of devices that can use the system (Marr, 2021). The system comes with features such as controlling the home, answering a wide variety of questions and can even help shopping online, which is one of the features where Alexa is stronger from for example Google Home (Amazon, 2021a). The system registers the voice and then directly sends this information to the sky to Alexa Voice Services (AVS), which then interprets this and pairs it with commands it understands. Alexa therefore have limitations when there is no access to WIFI, whereas Google Home is still operational, although very limited. This also means that the device is very dependent on the internet connection. If the connection is lost, the system is not working. Further, the system is only in English, which excludes a lot of users, and especially for this specific research, however the functions and system is still interesting to interrogate further. The developers are right now working on adding different languages to the system, such as Swedish, although it will not be implemented any time soon (MIT, 2021)

Music and Entertainment

Alexa covers a wide range of listening needs such as music, podcasts and radio, and audiobooks (Amazon, 2021b). Simple commands such as “Alexa, what free audiobooks are available?” will make the system come back with suggestions for the user, however it also requires the user to give rather specific commands.

(34)

Figure 4: Alexa – Music and Entertainment commands

Several different commands are available when for example playing music. These commands enable the user increase or decrease the volume, pause and play, change song, and even put specific settings for bass, treble and equalizer (Amazon, 2021b). It is supported by Amazon Music but can also support other services when connected to these streaming services (Amazon, 2021b).

Figure 5: Alexa – Music and Entertainment commands News and information

By questions such as “Alexa, what’s in the news today?”, it will return with selected news from the specific day. However, the system can also deliver a daily news update that is personalised to its user. The preferences can be set to specific sources so that the system will generate a short news update only of headlines from these sources (Amazon, 2021c).

Natural Language Generation and Processing

A strong asset is the artificial intelligence, also called natural language generation (NLG), which is a system that gives back natural sounding verbal responses from data that is put into the system. The system thereby makes the outcome very easy and understandable, which makes

(35)

the device much easier to understand. Therefore, the system holds the promise and does not need any other kind of interactions such as swiping or typing as it will understand the commands its being told. This is the case almost no matter how the user gives the command as its AI can understand the very nonlinear way that we people talk in (Marr, 2021). This also gives security as the user can trust the system to understand them and not require people to do physical commands.

(36)

Methodology

5.1. Design Thinking and Qualitative Research

This Master’s Thesis is carried out from a Design Thinking approach. It utilizes its iterative approach to get a deeper understanding of both the situation and the general problem from the user’s perspective. It was originally invented and introduced by the design consultancy IDEO as an explicit design methodology aiming to foster creativity (Thoring; Müller 2011, p. 137). Design Thinking is according to Dorst (2011) used in a variety of fields and helps dealing with problems from a research perspective. It is identified as a paradigm and has created and exciting approach to challenges from a variety of fields (Duque et al., 2019). Brown (2008) defines the methodology as follows: “Design thinking can be described as a discipline that uses the designer’s sensibility and methods to match people’s needs with what is technologically feasible and what a viable business strategy can convert into customer value and market opportunity”. Its holistic perspective and iterative process is what accommodates the users’ needs and ultimately creates a solution or outcome that contributes with value (Eriksson, 2016). Information thereby comes from relevant user-centered methods, that focuses people’s needs with what is technologically feasible.

The process that has been followed in this project is visualized in figure 11. The process has as mentioned not been linear as new insight constantly changes the picture, making it necessary at times to move back in the process. These steps therefore lie within the visualized process and are further elaborated below.

Figure 11: Used Design Thinking Process Visualized by Kristian Mellgaard

(37)

Figure 12 visualize the continuous iterative process that is constantly performed. Moving from the activities in context on the user’s side and next interrogating how these needs a met by technology. This explains that Design Thinking Process is anything but linear, although there are several suggested phases presented by Dorst (2011) amongst others. The practical process in a much fuzzier process where a flow of information and knowledge constantly changes the picture on each side as represented in the figure below.

Figure 12: The iterative process of this research: from Carrol (2006), Benyon (2015) Visualized by Kristian Mellgaard

In combination with the mentioned Design Thinking Approach, this study is conducted as a qualitative research. Qualitative research defines methods that are most suitable for collecting and analysing data (Sharan B. Merriam and Elizabeth J. Tisdell, 2016). It focuses on meaning in context and acquiring data that is a sensible instrument for the underlying meaning, which humans are best at according to Sharan B. Merriam and Elizabeth J. Tisdell (2016). Thus, this study will build on data collection from qualitative methods that can be used to find underlying meaning when interpreting the collected data. The importance of understanding the experiences from users is something that can best be understood from interaction and interpretation rather than numbers. Reality is in this research considered to be something that is socially constructed, which means that there are no single observable reality (Sharan B. Merriam and Elizabeth J. Tisdell, 2016). There are thereby multiple realities and interpretations of a single event (Sharan B. Merriam and Elizabeth J. Tisdell, 2016). I have as a researcher thereby not had the task to “find” the knowledge, but rather construct it. This also empathize the importance of being critical to one’s own findings as a researcher and be open towards others’ interpretation.

(38)

Combined, the two creates a strong foundation for exploring the topic that will give depth to the research. This ensured from relevant qualitative methods that has a focus on the user’s needs and situation, and acknowledging exploration through experiences and interpretation by me as a researcher. It thereby allows me to collect various experiences and perspectives and interpret these into conclusion that can be used as inspiration for future developers (cf. project objective).

5.2. Methods for Data Generation

5.2.1. Card Sorting (interviews)

Card sorting is a method for creating user comprehension and make meaningful categorizations (Bella and Hannington, 2016). Participants are provided with several picture cards that each has some sort of symbol, picture or text. They are asked to categorize these in relation to the topic. In the process it is important to observe not only the categories that the participant is working out but also listening to their reasoning behind their actions (Bella and Hannington, 2016). Thereby the understanding of different aspects can be better understood by the researcher.

The purpose of this exercise for this project is to bring in experts that have relevant background knowledge on the focus group. It is the intention to make them point out relevant aspects and perspectives, that can deepen the understanding as researcher. Within the research, this step has been conducted as a part of the empathize and define phases. The exercise enables the participants to express their experience and knowledge about the users and their needs. It is an important tool that enables me as a researcher with limited knowledge on the area, to gain important insights through categories pointed out and explained by the participants. It has been done in collaboration with experts, as the target group has been impossible to establish contact with due to Covid-19.

Two participants have been doing the exercise. Participant 1.1 is a widow to a person that fitted into the described target group and has therefore valuable insights in this person’s life and daily struggle when accessing news, paper, etc. Participant 1.2 is a nurse that have worked at a nursing home where the person has had hands-on experience with various older people and also several that fits under the target group. The workshop was thereby based on two different perspectives, which was thought to contribute to a fuller understanding of the users.

(39)

Figure 13: Participants sorting out cards

The sessions were conducted at their private homes and a recording device was used so that I had the option to re-listen what was said, as writing down notes in the moment was not manageable to do alone. Also, pictures were taken to both document the exercise but also to easier remember the session when I had to re-listen to the recordings as well as to remember what pictures and categories the participants had chosen. Both exercises took around 30 minutes as this was expected to be an appropriate time for gathering the most essential thoughts and ideas related to the subject. Less time might have meant that some thoughts would be left out while more time might have distorted the focus on what is most important.

First, they were both asked to make categories that made sense for them, simply to become familiar with the cards. Next, they were asked to name new categories that were related to the target group and their situation when accessing media. The categories were then asked to be explained and supplied with picture cards that they saw fit to describe these. As a relatively inexperienced interviewer I tried to make the participants elaborate why they created the different categories and describe the emotions with the chosen picture cards.

5.2.2. Personas