The Role of Conversational Interfaces in the Future of Digitaland Technology

(1)

IN

DEGREE PROJECT COMPUTER ENGINEERING, FIRST CYCLE, 15 CREDITS

STOCKHOLM SWEDEN 2020,

The Role of Conversational

Interfaces in the Future of Digital and Technology

Examiner

Fredrik Kilander Academic Adviser Mira Kajko-Mattsson Industrial Adviser Kenneth Andersson

TUNA GERSIL

ISMAIL HILAL

(2)

Abstract

Conversational interfaces (CIs) have been a trending topic in recent years. As of the last decade, CIs have emerged with the aim of simplifying human-machine interactions and found a wide use case in the market. For example, Siri and Google Assistant are some of the most well-known CIs developed by the tech giants Apple and Google. The digital landscape has evolved from web, to mobile apps, to recently CIs. Nowadays, CIs, in particular chatbots and voicebots, are becoming increasingly common. Whether navigating the web or messaging on a phone, it is likely that CIs have been confronted offering the user help.

However, CIs have not managed to reach a large-scale use. Furthermore, the reasons regarding the challenges faced by CIs as well as their usability are not greatly explored.

In this thesis, we explore the most relevant uses of CIs and the reasons hindering a widespread use of CIs. Our goal is to provide an insight into CIs’ uses and list the reasons regarding the challenges faced by CIs. The research study followed a mixed method approach connecting an explorative qualitative literature study, a survey and an interview.

The data was collected by using a systematic mapping approach for it being more suitable for conducting an effective literature review. The survey and the interview were conducted in order to confirm the findings.

According to our research, it was found that the most common use cases of CIs were in customer service, sales, travel and bookings, education, healthcare and as voice assistants. The most prominent challenges faced by CIs were poor usability, language processing and understanding, speech recognition and natural language generation and security and privacy. As a conclusion, the future looks promising for CIs, however, they need to be furher researched and developed in order to help them reach a widespread use in the future.

Keywords: chatbots, voicebots, voice assistants, conversational interfaces, artificial intelligence.

(5)

(6)

Sammanfattning

Konversation Gränssnitt (CIs) har varit ett trendande ämne de senaste åren. Sedan det senaste decenniet har CIs kommit fram i syfte att förenkla interaktioner mellan människor och maskiner och har hittat ett brett användningsfall på marknaden. Det digitala landskapet har utvecklats från webb, till mobila appar till nyligen CI. Numera blir CIs, i synnerhet chatbots och voicebots, allt vanligare. Vare sig du navigerar på webben eller meddelanden i en telefon, är det troligt att CIs har konfronterats med att erbjuda användaren hjälp.

CIs har dock inte lyckats uppnå storskalig användning. Dessutom är orsakerna till de utmaningar som CIs står inför och deras användbarhet inte utforskas i hög grad. I den här avhandlingen undersöker vi de mest relevanta användningarna av CIs och orsakerna till en utbredd användning av CIs. Vårt mål är att ge en inblick i CI: s användningar och lista orsakerna till de utmaningar som CIs står inför. Forskningsstudien följde en blandad metodstrategi som ansluter en utforskande kvalitativ litteraturstudie, en undersökning och en intervju. Uppgifterna samlades in med hjälp av en systematisk kartläggningsätt för att göra dem mer lämpliga för att genomföra en effektiv litteraturgranskning. Undersökningen och intervjun genomfördes för att bekräfta resultaten.

Enligt vår forskning konstaterades att de vanligaste användningsfallen för CIs var kundservice, försäljning, resor och bokningar, utbildning, sjukvård och som röstassistenter. De mest framstående utmaningarna för CIs var dålig användbarhet, språkhantering och förståelse, taligenkänning och naturlig språkgenerering och säkerhet och integritet. Sammanfattningsvis ser framtiden lovande ut för CIs, men de måste undersökas och utvecklas ytterligare för att hjälpa dem att uppnå utbredd användning i framtiden.

Nyckelord: chatbots, voicebots, voice assistants, konversation gränssnitt, artificiell intelligens.

(7)

(8)

Acknowledgements

We would like to thank our advisers Mira Kajko-Mattsson and Fredrik Kilander at KTH Royal Institute of Technology. Their support and feedback provided throughout the thesis was of great value. We would also like to send a special thanks to Kenneth Andersson, Chief Product Officer at the company The Mobile Life, for his warm welcome to the company as well as for his continued follow-up and support during the time of this commissioned work.

(9)

(10)

1. Introduction

The world has always been led by human commands. Many believed that technological advances would make everyday tasks easier, such as booking a trip or controlling the indoor temperature at home. Advances in technology have contributed to ease people’s lifestyles. For example, CIs, software interfaces providing people with the capability of interacting with a computer based on human terms [4], have emerged with the aim of simplifying human-machine interactions. They are of great importance when it comes to people being busy with other activities, for example as in the case of driving. CIs also aim to help businesses in assisting customers in various ways [1].

CIs are mainly of two types: chatbots and voicebots. Chatbots use natural language to simulate a human conversation with a user via text, usually through websites or mobile applications [38]. Voicebots, on the other hand, use speech recognition technology to interpret natural language commands. They work differently to chatbots by maintaining the interaction between the user and the machine more natural and smoother since it gives the feeling that the user is interacting with a real person.

As of the last decade, CIs have found a wide use case in the market. They have enhanced the user experience across different industries. This has been linked to many advantages they provide such as being always available and supporting multiple languages. An additional advantage is that they are independent of the platform. This means that any device with internet access can be used to employ a CI, whether it is a laptop, a tablet or phone.

The digital landscape has evolved from web, to mobile apps, to recently chatbots and voicebots interfaces. The improvement of these interfaces, however, does not mean that this newly emerging technology is going to replace the old one. While there is a gradual shift in market share and behavior, any new technology does not entirely replace the previous one. Rather there are certain scenarios and use cases that work best with each new technology. There are also new challenges coming with these new technologies.

However, few studies have focused on usability of CIs and the challenges facing them to highlight the technology’s limitations.

1.1. Problem Statement

The problem is that the reasons regarding the challenges faced by CIs as well as their usability are not greatly explored. CIs have been limited to some use cases. Even though some studies have been made on CIs, in particular chatbots and voicebots, more research is still needed on what is hindering their improvement. Therefore, the challenges

(11)

1.2. Purpose

The purpose is to explore CIs most relevant uses and the reasons hindering a widespread use of CIs.

1.3. Goal

The main goal is to provide an insight into CIs’ uses and list the reasons regarding the challenges faced by CIs.

1.4. Research Method

The research study followed a mixed method approach connecting an explorative qualitative literature study, a survey and an interview. Data was collected by using a systematic mapping approach for it being more suitable for conducting an effective literature review. A survey and an interview were conducted in order to confirm the findings. Having done the initial research, we went further into investigating different CIs.

Our study included chatbots in various areas and voicebots as voice assistants. Our investigation method aimed to gather information from trustworthy sources.

1.5. Commissioned Work

This thesis work was done under the supervision of the company The Mobile Life, a company with a vision of driving innovation for the connected world [3]. The company has many years of diverse industry experience of providing useful services and products.

Developing mobile applications is their main focus, where their combined apps have been downloaded millions of times.

1.6. Target Audience

Anyone, from business owners to developers, who finds technological advancements and specifically CIs, in particular chatbots and voicebots, interesting can find a good amount of information for their interest in our paper. Although our work is easily comprehensible by people who lack knowledge about CIs, it especially targets those who want to use and develop CIs in their businesses.

1.7. Scope and Delimitations

Since this paper was written during a degree project course at KTH Royal Institute of Technology, our biggest limitation was the time given. Even though we would like to look in great detail to all of the challenges that CIs face, we are going to be focusing on some to make our work of better quality and deliver more reliable results. Another limitation was

(12)

the resources that we could access and the assessment of how reliable they were. In addition to these, finding the right people for the survey was a challenging task to accomplish due to the fact that not everyone has used a chatbot or a voice assistant.

1.8. Benefits, Ethics and Sustainability

This study enhances knowledge of CIs uses and the challenges faced by them among anyone interested in CIs field. In particular, developers and business owners who are interested in developing or integrating CIs in their work can benefit from this study.

As this study includes a survey and an interview, the informed consent of the participants was obtained. The anonymity of the participants was preserved thereby the confidentiality was respected. The data collected was used solely for the purpose of this research and not for external purposes.

The implementation of CIs can be used to handle processes, such as booking confirmations that require paperwork, electronically. Therefore, they contribute to sustainability by saving valuable resources.

1.9. Thesis Outline

The following demonstrates the chapters of this thesis and their contents:

Chapter 2: Background: This chapter provides the extended backgrounds and aims to familiarize the reader with the general concept of CIs.

Chapter 3: Methodology: This chapter describes the research strategy, research phases and research method conducted in the study.

Chapter 4: Results: This chapter presents the findings which were gathered through the examination of various literature as well as a survey and an interview.

Chapter 5: Analysis and Discussions: This chapter analyses and discusses the findings collected from the literature study, the survey and the interview.

Chapter 6: Conclusion and Future Work: This chapter draws conclusions based on the results and presents the future work.

(13)

(14)

2. Background

This chapter is divided into a number of subsections each of which aims to make the reader familiar with the general concept of CIs. Section 2.1 provides general background information about CIs. Section 2.2 presents the benefits of CIs. Section 2.3 explains the underlying core mechanisms such as Natural Language Understanding (NLU), Natural Language Processing (NLP) and Natural Language Generation (NLG). In Section 2.4, Artificial Intelligence (AI)-driven chatbots are described. Section 2.5 describes rule-based chatbots. Moreover, Section 2.6 covers voicebots. Finally, Section 2.7 examines related work to reflect on what has already been researched within the CIs field.

2.1. Conversational Interfaces

A CI can be defined as an interface providing people with the capability of interacting with a computer, based on human terms [4]. An even simpler definition of a CI is a way of communication for people to tell the computer what to do. In the early ages of communication, people had to input commands to the computer in the form of some specific syntax to do the same thing. CIs can be seen as more social compared to the traditional technological applications, from the perspective of the user who messages, contacts and invites. CIs are independent of the platform. They can be integrated into a number of different platforms including Google Assistant, Alexa, Cortana, Facebook Messenger and many others. Two main types of CIs are chatbots and voicebots.

A chatbot, as stated by Oracle [6], is a computer program that communicates with the user by simulating a human conversation to deal with their queries. Before chatbots were invented, people had to navigate through graphical interfaces to search for what they were looking for or to achieve a certain task. However, after the invention of chatbots, it introduced a new way of interacting with the software. Many websites now implement a chatbot which makes the process of searching the website easier. This is because with the chatbot use, the need to search through the graphical user interface is eliminated.

The user simply uses the chatbot’s interface to make it understand their intent and do the task for them.

A voicebot is used to achieve a similar function of a chatbot but relies on speech recognition technology to interpret natural language commands [7]. Voicebots provide a more human-like communication between the user and the machine. Therefore, the user feels as if they are talking to a real person rather than to a machine. Consequently, the conversation becomes smoother and more natural for the user.

(15)

CIs, in particular chatbots and voicebots, have contributed with many benefits to the modern world. To provide an insight into these benefits, we present some of them in the following section.

2.2. Benefits of Conversational interfaces

One of the main and easily visible benefits of CIs is that they are available 24/7. Unlike human customer service representatives that are available for a number of hours throughout the day, CIs provide fast answers and solutions to customers all round the clock. This has a great advantage for companies to boost online sales. A study conducted on a number of companies showed that fast response rates increase business’ revenue [70].

Another advantage is that CIs save customer’s time when searching for something [1].

The time users spend navigating websites in search for products or services can be greatly reduced as the users can be guided to what they are looking for by the bots. This contributes to the customer's satisfaction which is crucial in the modern world where brands and businesses are in increased competition.

In addition to that, CIs benefit businesses in a number of ways by saving them valuable time that is usually spent on repetitive tasks such as simple customer support. By using CIs to automate and provide fast simple responses, businesses can better use their resources [1]. An example is that instead of directing every query to a customer support human agent, only complex queries, that are unsolved by the bots are directed to a human representative. This also contributes to providing faster response rates for customers.

It has been shown that CIs provide a number of benefits for both their users and employers. To better understand how this is done we first begin by examining NLP before moving on to chatbots and voicebots.

2.3. Natural Language Processing

Humans, unlike machines, express themselves in complicated ways. The way we communicate either verbally or written, goes much deeper than what an individual word means. There are countless ways to express an idea. This is due to the fact that modern languages have so many different words and even the use of the same words can have different meanings depending on the context of use. Therefore, a system such as NLP, is needed to handle these different scenarios and at the end provide the machine with the capability of understanding what was meant by the cluster of words.

(16)

As defined in the article [8], “NLP is a theoretically motivated range of computational techniques for analyzing and representing naturally occurring texts at one or more levels of linguistic analysis for the purpose of achieving human-like language processing for a range of tasks or applications.”. In simpler words, NLP is the mechanism that makes it possible for machines to understand human language, also known as the natural language. This means that humans do not need to know any programming languages to interact with the machines. Two core elements of NLP are NLU and NLG.

NLU works at the core of NLP to provide the machine with the understanding of a natural language input. However, what NLP does is different from processing the input word by word but it rather evaluates the sentence in the way humans do. As discussed in the article [10] understanding natural language involves three stages. Firstly, the words’

meanings are determined. Secondly, the meaning of the sentence is determined as a whole by paying attention to the order of the words and the grammar. Finally, the meaning of the sentence is evaluated according to its context and domain of use thereby completing the understanding phase.

NLG being the second core element of NLP, is responsible for generating output. This is the mechanism that comes into play after NLP and NLU successfully manage the understanding and the processing of the input. At this point, it is clear for the machine what the user’s intention is and an output should be generated accordingly. NLG as described by [11] completes the communication process by choosing the wording, checking the grammar and generating a word or a phrase as a response to the query.

NLP is usually used interchangeably with AI and Machine Learning (ML). However, they are different from each other. As stated in the article [5], NLP and ML are parts of AI. AI helps machines in solving complex problems that are of great importance for humans.

NLP is the system which makes it possible for machines to understand the way humans communicate in both written and spoken language. ML is a software system which enables the machine to learn and develop from its own observations and previous experiences. Figure 1 shows the relation between NLP, NLU, NLG, ML and AI.

(17)

Figure 1. Representation of AI, ML and NLP [5]

With regards to these, there are two main types of chatbots as presented by [1]: AI-driven or NLP-based chatbots and rule-based chatbots. The first type of chatbots make use of an underlying AI mechanism, whereas the latter type is more of a hard-coded version.

2.4. AI-Driven Chatbots

AI-driven chatbots are based on the concept of NLP since these chatbots employ NLP as their core mechanism as well as ML [1]. AI-driven chatbots are able to simulate a human conversation. They are good at spotting the intention of the user and arrange the responses accordingly. For example, when a user responds by saying “Why not?” as a response to a suggestion, the chatbot interprets this as a confirmation rather than a question. In addition to that, AI-driven chatbots are mostly equipped with ML to reinforce their development over time thereby improving their ability to provide useful responses.

2.5. Rule-Based Chatbots

Rule-based chatbots are programmed for specific tasks and therefore can only answer the type of queries they are hard-coded for [1]. However, these tasks can have a varied range of complexity, all from simple to complicated queries. On the other hand, it is very challenging and, in most cases, almost impossible for rule-based chatbots to handle all the cases in a particular situation. Therefore, it is generally not very hard for users to spot a rule-based chatbot’s deficiencies.

2.6. Voicebots

Voicebots are another kind of CI that use speech recognition technology to interpret natural language commands. Voicebots can be thought of as voice assistants. Voice assistants are digital interfaces that imitate a human to human conversation flow. They use sound as a means of communication and eliminate the need for a graphical interface.

(18)

As with the case with AI-driven chatbots, voicebots are built on AI and ML in addition to voice recognition technology.

Voice assistants have software for listening to the sound around them. When a keyword is recognized by the software, a mechanism is triggered to wake up the voice assistant.

Following the wake up process, the user’s voice is recorded, packaged and sent to a dedicated server for further analysis and interpretation [9]. As a response, the server provides the voice assistant with the information it needs to be able to answer the user, or accomplish a certain task [9]. Voice commands are getting more and more supported by various services. As mentioned in [9], Internet-of-Things manufacturers are also adding compatibility for voice commands in their devices.

Siri, Google Assistant, Alexa and Cortana are some of the most well-known voice assistants developed by the tech giants Apple, Google, Amazon and Microsoft respectively. They are integrated into smartphones and home speakers and provide a number of functions, including controlling home devices, playing music and answering questions. The assistants have integrated voice matching technology to identify the user and personalize requests.

2.7. Related Work

CIs have shown to be beneficial in various fields. In particular, chatbots and voicebots, have witnessed many advances over the past years as stated in [29]. It has been shown in [12] that CIs are able to persuade users to change their attitudes positively towards regular exercise by successfully simulating a human conversation. Moreover, CIs help in increasing the knowledge of students when used in education [43]. Vehicles drivers have also benefited from CIs. [60] concluded that natural voice interfaces in vehicles lower distraction effects on drivers and provides a better user experience.

The strengths of chatbots in the service sector were investigated in [42]. The purpose of the paper was to familiarize the readers with the concept of chatbot-based CIs and how convenient they are for humans to use since they use natural language. The paper, however, focuses on a specific use case of CIs, in particular in the case of when a user interacts with services such as public administration, health and home automation. It is discussed that using chatbots contribute to a positive user experience by providing simple and enjoyable conversations. Additionally, CIs have been proven in [61], [62] to be beneficial in meeting customer demands thereby improving customer satisfaction. It is further emphasized in [42] that reducing users’ effort, providing fast responses and being always available are the key advantages of CIs.

(19)

CIs have changed the way information is accessed as demonstrated in [63]. It is further stated that this is particularly beneficial to visual and motion impaired people, aged people and children. This means that CIs provide easy access of information to all from young people who still do not know how to read and write to elderly and disabled people.

However, chatbots are not only limited to providing service but may also be used for social good [64]. It is stated that chatbots are able to provide low cost and easy access to medical support, influence voting behavior and help companies in engaging in beneficial social implications such as providing remote areas with internet communication.

On the other hand, CIs face some obstacles along the way. [42] states that CIs do not always provide their users with the right answers as some users may be reluctant to provide CIs with personal information. Besides, it is stated in [42] and [65] that customizing the chatbot in relation to the user is of significant importance. However, [65]

argued that personalizing experiences can complicate design strategies.

Additionally, [44] outlined some of the important development issues facing CIs. One of the main issues being linked to spoken language understanding is that it is challenging to handle noisy environments and a varied speaker population [44]. [66] further emphasized that CIs lack the support of vernacular language as well as understanding dialects which are of significant importance as part of speech recognition. It is further stated in [44] that most of the research within the spoken language systems field has the focus of the input side only. The output is mostly neglected even though it is very important for such systems [44]. Without a proper spoken language generation which sounds natural, the impression left on the users cannot be expected to be of high quality.

In the light of these related works, we expanded our knowledge to a broader extent. Their findings are relevant to our problem domain and were taken into consideration when writing this thesis.

(20)

3. Methodology

This chapter introduces the reader to the methodology followed throughout this thesis.

Section 3.1 presents the research strategy. Section 3.2 lists the research phases which start from the problem statement and end with conclusions. Section 3.3 gives a detailed explanation of the research methods that were used and why they were chosen. In section 3.4 we discuss the research criteria considered when gathering data. Furthermore, in Section 3.5, the validity threats are presented but motivated for in discussions. Finally, in section 3.6, ethical requirements that the researcher is expected to follow have been discussed.

3.1. Research Strategy

We chose the design science paradigm, a research methodology based on results, since it is relevant for the Information Systems field. As stated in the article [13] “The design science research paradigm seeks to create innovations that define the ideas, practices, technical capabilities, and products through which the analysis, design, implementation, management, and use of information systems can be effectively and efficiently accomplished”. Specifically for our case, design science was used to help us in defining CIs practices through analyzing their technical capabilities and the challenges facing them.

The research strategy was built up on research phases. The phases consist of six parts, problem statement, literature study, results, survey and interview, analysis and discussions and conclusions. The research methods selected were a qualitative literature study based on a systematic mapping approach, a survey and an interview. Following the research methods, the data collection method was presented. In order to carry out the data collection, certain criteria were needed. Finally, the validity threats were presented in this chapter but motivated for in the discussions chapter.

3.2. Research Phases

This thesis consists of six different research phases. First, the problem statement was formulated. It was followed by an explorative qualitative literature study for collecting relevant data. The results were then presented accordingly. Having presented the results, a survey and an interview were conducted to confirm our findings. Afterwards, the data was gathered, analyzed and the discussions were brought up. Finally, the conclusions were drawn and reflected upon. The research process followed was believed to produce high-quality knowledge that could be of use for future research. Figure 2 depicts the phases conducted in order to carry out this research.

(21)

Figure 2. The Research Process

3.3. Research Methods

This study followed a mixed method approach connecting an explorative literature study, a survey and an interview. A qualitative research design was chosen to help in exploring the topic. This is the method which when applied, provides the researcher with broad knowledge of the field. This way the reader is introduced to the research topic by making use of many credible resources on the internet. Data was collected by using a systematic mapping approach for it being more suitable for conducting an effective literature review as stated in the article, “Systematic Mapping Studies in Software Engineering” [41]. In addition to the literature study, a survey as well as an interview were conducted in order to confirm the findings. The survey targeted chatbots and voice assistants’ users in general. Finally, the interview conducted with an expert in the field was used in order to increase the validity of our results. The research methods used for collecting the data are the following.

3.3.1. Literature Study

A literature study is a way of gathering information from many sources such as books, websites, articles and other academic sources. After the initial step of gathering information is completed, the information can be analyzed and processed in a way that is beneficial to make meaningful points and relevant conclusions. In brief terms, a qualitative research only uses textual or non-numerical data. When it comes to exploratory research, such as our study, qualitative research is considered to be a better fit. The reason is that this type of research is used to learn and gain a deep understanding about a certain topic [14]. Qualitative research focuses on ideas, opinions and trends with the aim of delving deep into the subject and analyzing it in a careful manner [14].

The literature study was based on a systematic mapping study [41]. In order to gather and analyze useful data, a number of different steps were carried out. The steps included defining the problem statement, conducting primary search, selecting relevant literature,

(22)

classifying relevant literature and extracting data. Figure 3 presents the steps followed in the literature study process.

Figure 3. Literature study process

Defining problem statement Since the CIs field is a broad subject that requires a good amount of background knowledge, it was important to identify the main topics that contribute to the necessary knowledge. Some of the topics that were of significant importance, such as NLP and NLU, have been researched more extensively to make sure that these key concepts are brought to the reader in an easy to understand manner. A few other topics that were not identified as core topics, were investigated more briefly given the time limitation. The goal was to look for the most investigated topics with CIs uses and issues. For a thorough understanding of the subject, some websites and blogs were used. They were useful as the topic studied is fairly new and witnesses rapid advancements.

Conducting primary search When conducting the primary search, a number of databases were considered. Academic search engines, such as KTH library, Google Scholar, ResearchGate, ScienceDirect and IEEEXplore were the main tools used to find relevant source material. Many sources needed to be examined and decided on whether they were useful for the research field. In our case, the information was gathered from various different sources, including books, journal articles and academic papers. We focused on having recent sources which were not older than 5 years with a few exceptions. Our initial search with Google Scholar using the keywords “conversational interfaces”, “chatbots” and “voice” yielded countless results. Therefore, we made it more specific by using the keywords “chatbots”, “voice” and “conversational interfaces”

interchangeably with “use cases”, “challenges” and “user experience”. This has in turn yielded thousands of results, where the first 50 results of each search were selected.

(23)

Selecting relevant literature Elimination of the selected papers depended on the exclusion criterion. This narrowed down the results to 214. The remaining articles’ content was evaluated with respect to the inclusion criterion, leading to the selection of 53 sources in the end.

Classifying relevant literature The selected sources were classified into two categories, one being CIs usability, the other being challenges faced by CIs. Some sources fit both of the categories and were therefore placed in both.

Extracting data Having sorted the relevant literature into two separate categories, the data extraction process was initiated. The data extraction process was linked to our exploration criteria which had usefulness and challengingness in focus. This process then led to adding subcategories for CIs usability and challenges faced by CIs. The formation of these subcategories was based on the frequency of their appearance in the literature.

Some subcategories included the same articles as they were convenient to be used in both.

3.3.2. Survey

A survey is a great way of learning different people’s opinions about the subject the information gathering is done for. In our case, we made use of a survey to understand people’s experience with the use of CIs. The survey was created on an online-based survey platform called SurveyMonkey [37]. The choice of the platform was considered appropriate due to its ability to create a survey that could be deployed on a link and is easily accessible from mobile devices. In addition to this, the platform provides useful analysis tools for the surveyor.

Selecting the user sample was based on the judgmental, also known as purposive, sampling type [67]. The judgmental sampling type is a non-probability sampling method since the random selection of users is done by the researchers. According to this sampling type, the selection of the participants is based on the research requirements, thereby excluding others who do not fulfill the requirements. In our case, the selection criteria were that the participants were required to be older than 18 years old and have used either a chatbot or a voice assistant.

The survey included 53 respondents of various ages and backgrounds who used CIs such as chatbots and voice assistants. Our target groups consisted of users including students, academicians and families. The survey was sent via WhatsApp and Messenger to our peers who used CIs. In addition to this, an academician was contacted and asked if she could share the survey with her peers. This way, the survey results were randomized through both the age difference and the background difference among the

(24)

surveyees. The survey consisted of 9 questions. One rating question and 8 multiple choice questions. The survey helped us gain a deeper insight of the benefits and issues encountered with the use of CIs. Besides, it reinforced our understanding of the situation and therefore led us to make a better conclusion.

Due to the outbreak of the Coronavirus at the time this thesis was written, an advantage with conducting a survey was that it was administered remotely via an online link, making it easily available on mobile devices. However, a disadvantage could have been that respondents might not have provided accurate answers due to boredom or lack of memory on the subject.

3.3.3. Interview

An interview was needed to assess the relevance of our results. The interview was conducted with Kenneth Andersson, Chief Product Officer at the company The Mobile Life, who has more than 10 years of experience with mobile applications and conversational interfaces of various technologies. He has worked in several managerial positions in different technological companies and has supervised the development of many CIs and analyzed their progress.

The interview was conducted over a video meeting through Google Meet. The interview was semi-structured as we had open-guided questions leading to a free-flowing conversation. The results were discussed and his views were noted down.

An advantage with conducting the interview with an expert was that it improved our findings’ conformability and reliability. However, a disadvantage with conducting an interview is the interviewer effect, which is a distortion in the interviewee’s response due to the way the interviewer presents the questions [40]. To minimize the interviewer effect, we selected an expert with vast experience in the field and asked clear-worded questions [68].

3.4. Research Criteria

Our thesis explores CIs usability and the current challenges faced by CIs to help in setting grounds for further studies. Therefore, different criteria for selecting the literature and extracting the data from it were needed. Having examined several criteria, we have concluded that the following criteria fit our needs the best.

(25)

3.4.1. Literature Study Criteria

This section presents the exclusion and inclusion criteria used for narrowing down the number of our primary search results. In addition to this, the exploration criteria were used for extracting data from the selected literature.

● Exclusion Criterion

Insufficiency was the exclusion criterion used for excluding the literature from the primary search. If the keywords only appeared in the title or the abstract and there was no further examination made throughout the article, it was thought as it is not sufficient and therefore excluded. Any literature that had only the abstract section available and required payment for full access, even when using KTH institutional login, was excluded. Additionally, literature available only in the form of slide presentations were also excluded.

● Inclusion Criterion

Content analysis was the inclusion criterion used for including the literature from the primary search. Our selected literature included academic papers, journal articles, books and reports. When we came upon similar studies, we only included the most recent ones. Relevant research findings were used as grounds to aid in further research. The content analysis included comparisons of keywords or content. In particular, we looked upon if the keywords were discussed in the different sections of the articles reviewed.

● Exploration Criteria

Usefulness was the fundamental exploration criterion considered for CIs’ use cases. Usefulness refers to the attribute of being of high quality and helpful for a specific topic. Our exploration of the literature reviewed consisted of assessing the CIs usefulness linked to the subject studied. This was done in the following way: if CIs had an overall positive impact within the specific field, they were considered as useful.

Challengingness was the fundamental exploration criterion considered for the challenges faced by CIs. Challengingness indicates how challenging a certain aspect is for CIs to be in widespread use. We examined the selected literature in terms of the issues and hinders that CIs face with their use and development. This was done in the following way: if a problem was discussed in several sources, it was considered to be one of the main challenges.

(26)

3.4.2. Survey and Interview Evaluation Criteria

Relevance was the fundamental evaluation criterion that was considered throughout the thesis. Relevance reflects upon the state of being closely connected or appropriate. We assessed the results' relevance linked to the subject studied. This was done by conducting a survey and an interview to see how relevant the results were with accordance to our research question.

The survey was evaluated according to the following criteria:

● Appropriateness of the surveyee was one criterion. The surveyee was considered appropriate if they were older than 18 years old and have used either a chatbot or a voice assistant.

● Usage context was another criterion taken into consideration. This criterion evaluated the overall experience with the CIs, the purpose the CI was used for, any issues experienced with the CIs’ use and the best feature associated with the CI in relation to our findings. It was used for obtaining an overview of how CIs are used and the user experince they provide.

The interview was evaluated according to the following criteria:

● Appropriateness of the interviewee was one criterion. The interviewee was considered appropriate if they had more than 10 years of experience with developing or analyzing CIs.

● Finding’s coverage was another criterion used to evaluate the interview. It covered how relevant and useful the results appealed to the interviewee.

3.5. Validity Threats

In order to put forward a scientific approach throughout the thesis, we followed the principles presented in Shenton’s article “Strategies for ensuring trustworthiness in qualitative research projects" [17]. Shenton further refers to Guba’s validity threats constructs [18], the validity threats can be considered from four different perspectives which are credibility, transferability, dependability and confirmability. These are further discussed under discussions.

Credibility is a measure of trustworthiness in terms of the responses that are gathered from the participants in an interview, a questionnaire or a survey [18]. Transferability is a measure of generalizability of the findings [18]. Dependability is a measure of reliability [18]. Conformability is a measure of objectivity [18].

(27)

3.6. Ethical Requirements

Existing literature is regularly used to backup or support new research studies. However, when conducting a research, it is important that ethical requirements of the research strategy are considered. Researchers should not use a one-sided approach, especially when thoughts and opinions on the matter are important. This could then produce a biased result. When conducting our literature study, we always took into consideration both sides of an argument.

Another ethical aspect to take into consideration is that the reading should be done with caution. Researchers need to carefully read others' work to better understand and reflect on it. Skim-reading may lead to misinterpreting the results or conclusions [19]. The quality of the work presented is also better assessed when read carefully. Since we are two authors working on this thesis, the literature was always read carefully and assessed by both of us individually, making sure we had the same understanding of the writer’s reasoning.

The survey was made anonymous, therefore the surveyees did not worry about their opinions being linked to them personally. They were also given the right to withdraw from the research. Furthermore, when conducting the interview, we obtained the informed consent of the interviewee on sharing his name and opinion. We also avoided using deceptive practices, such as the interviewer effect by asking clear-worded questions.

(28)

4. Results

This section presents the findings which were gathered through the examination of various literature as well as a survey and an interview. The most common use cases of CIs were presented in Section 4.1. The findings about the challenges faced by CIs were listed under Section 4.2. In addition to this, the survey results were presented in Section 4.3. Finally, Section 4.4 demonstrates the interview conducted.

4.1. Use Cases

Conversational interfaces were examined as two different types, one being chatbots and the other being voicebots. According to the research conducted we found the most common use cases of chatbots to be in customer service, sales, travel and bookings, education and healthcare whereas the most common use of voicebots was found to be in voice assistants. Out of 53 articles studied, the results were obtained according to the following:

● Twenty-six articles mentioned chatbots use in customer service. Our exploration criterion usefulness was covered in 5 out of the 26 articles.

● Fifteen articles mentioned chatbots use in sales. Our exploration criterion usefulness was covered in 4 out of the 15 articles.

● Fourteen articles mentioned chatbots use in travel and bookings. Our exploration criterion usefulness was covered in 2 out of the 14 articles.

● Eight articles mentioned chatbots use in education. Our exploration criterion usefulness was covered in 2 out of the 8 articles.

● Eleven articles mentioned chatbots use in healthcare. Our exploration criterion usefulness was covered in 3 out of the 11 articles.

● Twelve articles mentioned voicebots use in voice assistants. Our exploration criterion usefulness was covered in 4 out of the 12 articles.

The number of articles which mentioned each sector out of the total number of articles is demonstrated in Figure 4.

(29)

Figure 4. Frequency of use cases appearance in the literature

In order to present the use cases of chatbots in different sectors, it was convenient to use a table to gather the important points together thereby making the results more compact and easier to read. Table 1 demonstrates the findings related to the chatbots use cases.

Table 1. Chatbots use cases

Chatbots in Customer Service

➢ lead to greater customer satisfaction [21].

➢ bring loyalty and favorable purchase intentions [24], [21].

➢ build stronger customer relationships by handling customer requests quickly and efficiently [26].

➢ give the customer a better insight of product description [27].

➢ provide enjoyable conversations [28].

(30)

Chatbots in Sales

➢ influence customers' decision-making process [21].

➢ prioritize what the person would like to get out of the service and easily provide product information [30], [31].

➢ handle customers’ emotions well, making them more likely to purchase the products [33].

Chatbots in Travel and Bookings

➢ simplify the process of booking through websites [48].

➢ enhance the online travel booking experience [47].

➢ use machine learning to assist travelers with the tasks of finding out the best deals, booking trips and getting information on the place they are visiting [48], [47].

➢ are preferred by many over a human agent for travel bookings [47].

➢ encourage a very high user interaction by saving users time and money [47].

➢ are found to be more preferred when they also serve as a concierge [47].

Chatbots in Education

➢ still need to be further developed to be considered at a comparable level with a student’s favorite teacher [35].

➢ due to their cost-efficient aspect, the process of integrating chatbots into education has sped up [35], [36].

➢ make it easier for a student to be informed about the critical events in their student life such as learning the deadlines for assignments and their final grades on a course [35].

Chatbots in Healthcare

➢ making self-diagnosis chatbots easily available for patients can cut down medical costs [49].

➢ can have negative consequences for patients who self- diagnose regularly [50].

➢ make the process of handling many patients more efficient and help the overall health sector save resources [51].

➢ serve well as appointments schedulers and information providers [50].

(31)

➢ can be used for screening and follow-up of cancer patients [51].

➢ are able to lower depression levels among the patients [51].

➢ are proven to be beneficial for the users as it is shown that they encourage people to have a healthier lifestyle by engaging them with the required physical activities [51].

For voicebots being a different type of a CI other than chatbots, it was convenient to use a separate table to gather the important points together. Table 2 demonstrates the findings related to the voicebots use cases.

Table 2. Voice use cases

Voicebots as Voice Assistants

➢ different voice assistants support different features, however, they all can be used to achieve the same basic tasks including sending and reading text messages, making phone calls, providing basic information, setting alarms, playing music and entertaining their users [9].

➢ home automation, which is achieved by integrating IoT- devices with voice assistants, is considered one of the main uses of voice assistants [52].

➢ 90% of the new vehicles sold by 2028 are going to have a voice assistant integrated in them [55].

➢ can be especially helpful for blind people or people who have limited mobility [56].

➢ can aid users who suffer from memory difficulties, dyslexia or speech impairments [56].

4.2. Challenges Faced by CIs

According to the research we conducted, usability issues, language processing and understanding, speech recognition and natural language generation and security and privacy were the most prominent challenges faced by CIs. Out of 53 articles studied, the results were obtained according to the following:

(32)

● Nineteen articles mentioned issues regarding usability and user experience. Our exploration criterion challengingness was covered in 4 out of the 19 articles.

● Fifteen articles mentioned issues regarding language processing and understanding. Our exploration criterion challengingness was covered in 3 out of the 15 articles.

● Nine articles mentioned issues regarding speech recognition and natural language generation. Our exploration criterion challengingness was covered in 3 out of the 9 articles.

● Sixteen articles mentioned issues regarding security and privacy. Our exploration criterion challengingness was covered in 4 out of the 16 articles.

The number of articles which mentioned each issue out of the total number of articles is demonstrated in Figure 5.

Figure 5. Frequency of challenges appearance in the literature

In order to present the challenges faced by CIs in different sectors, it was convenient to use a table to gather the important points together thereby making the results more compact and easier to read. Table 3 demonstrates the findings related to the challenges

(33)

Table 3. Challenges faced by CIs

Usability and User Experience

➢ voice assistants seem to have poor usability [57].

➢ chatbots are usually limited to a specific use case [2].

➢ usability problems occur mainly because a great percentage of people prefer communicating with a real person instead of a chatbot [58].

➢ half of online customers prefer communicating with a human rather than a chatbot regarding complex issues [53].

➢ digital channels are becoming more popular for customer service inquiries [53].

➢ the top digital channels are chat, phone, email and social media with chat taking around half of the market [53].

Language Processing and

Understanding

➢ NLP and ML algorithms are used to improve the ability of AI software [15].

➢ NLU is the core issue regarding the NLP problem [45].

➢ it is quite difficult to embed human emotions into CI agents [45].

➢ technological advances have led to continuous improvements in emotion recognition and detection systems [45].

➢ it is important to have low-resource languages, languages with very little data available on them, in the main focus to be able to target more people [45], [16].

Speech Recognition and Natural Language Generation

➢ it is challenging for CIs to meet the needs of a varied speaker population when dealing with children or non- native speakers who may have strong accents or dialects [44], [20].

➢ CIs need to cover open vocabulary as anticipating what vocabulary or words the user will use is extremely hard if not impossible [44], [16].

➢ users respond differently depending on the response from the system [44]. Unclear responses from CIs lead to more varied inputs from users [44].

(34)

Security and Privacy

➢ a voice assistant can be used by malicious users to facilitate unauthorized access to the private information of the owner [9], [22].

➢ voice assistants can open a new way for a new kind of attack where the attacker can pass ultrasonic commands through broadcast media to easily activate the user's device [9].

➢ a privacy concern for many users is that voice assistants are always recording audio [9].

➢ building trust in AI is achieved by reliability and transparency [39].

➢ personalizing online interactions raises privacy concerns among customers [34].

4.3. Survey Findings

In this section we present the survey results. Question 1 evaluates the appropriateness of the surveyee criterion. Questions 2 to 9 evaluate the usage context criterion.

(35)

Question 1 shows that the age span was from 18-54. The first range, 18-24, was only a 7 years gap compared to the other ranges of 10 years gap. This was due to the fact that the age span 18-24 is a critical interval between young adolescence and late adulthood [69]. The highest percentage of respondents was between 18 to 24 years old.

Question 1

(36)

Question 2 shows that from a scale of 1 to 5, the average rating was 3.7. The highest percentage of respondents rated their experience with 4 stars.

Question 2

(37)

Question 3 shows that the most common uses for a chatbot were customer service and travel bookings with 24 responses each.

Question 3

(38)

Question 4 shows that the most common issues with using chatbots were keeping users away from a real person and unhelpful responses.

Question 4

(39)

Question 5 shows that the best aspect with using a chatbot was being available 24/7. This was indicated by 57% of the respondents.

Question 5

(40)

Question 6 shows that the most common cases for a voice assistant were when hands- busy and messaging and making calls. Using voice assistants when hands-busy accounted for the highest responses.

Question 6

(41)

Question 7 shows that the most common issues with using voice assistants were misunderstanding requests and accuracy in executing commands. Misunderstanding requests dominated the highest number of respondents.

Question 7

(42)

Question 8 shows that the best aspect with using a voice assistant was convenient use when hands are busy. This was represented by 54% of the respondents.

Question 8

(43)

Question 9 shows that 96% of the respondents thought that CIs will become more common in the future.

Question 9

(44)

4.4. Interview

The interview was handled by the following questions, in relation to our evaluation criteria as presented on Table 4.

Table 4. Evaluation criteria and interview questions related to them

Appropriateness of the interviewee

1. What is your profession?

2. Do you have more than 10 years of experience with CIs?

Finding’s coverage

3. Do you think our findings cover the usability aspects of CIs as well as the challenges faced by them?

Kenneth agreed upon that the findings were relevant and in line with his expectations.

He, however, highlighted that even though chatbots can be faster and more efficient, they are not always the best choice for collecting data. Sometimes filling webforms is more convenient. He gave the example that many users hesitate to provide personal payment details in a conversation and this is where a webform could be of better use.

Another point was that in some cases, chatbots can waste the user's time by collecting data and at the end being unable to resolve the query thereby redirecting to a human agent. Filling a webform and sending it directly to a human agent in this case is simpler.

(45)

(46)

5. Analysis and Discussions

This chapter discusses the user experiences with the main use cases of CIs and the challenges faced by CIs based on the results collected from the literature review, the survey and the interview. The problem was that the reasons about the challenges faced by CIs as well as their usability are not greatly explored. After an extensive research, it was found that the most relevant uses and the reasons hindering a widespread use of CIs are many. Section 5.1 presents the most relevant use cases of CIs. Section 5.2 discusses the most prominent challenges faced by CIs. Lastly, Section 5.3 reflects upon the threats of validity.

5.1. Use Cases of CIs

CIs have been a topic of great interest during recent years. Due to their flexibility which helps them fit into different sectors, they have become a fast-developing industry in the modern world. However, different types of CIs, such as chatbots and voicebots, have been limited to different use cases. Chatbots are used mostly in websites and messaging applications. The most common use cases of chatbots include customer service and sales, travel and booking and education and healthcare. Voicebots have found their most common use in voice assistants and home speakers. According to the survey conducted, it was also highlighted that customer service, travel bookings and voice assistants were among the top uses of CIs.

5.1.1. Chatbots in Customer Service and Sales

Today, customers spend more and more time in digital environments. Brands from all over the world are trying to adapt to this radical change from interacting with the customer face-to-face to also expanding their services to digital environments. This is a mandatory shift and in most cases a great opportunity for brands [21].

Brand managers always look for new ways to improve customer experience by taking the customer preferences into account [23]. The long-established service agent interactions occur when there is a face-to-face customer and employee communication. The customers now prefer shopping online as it is more effective [25]. In addition to these, the online services are available 24/7. This is of significant importance for targeting more customers than a regular service agent, who works nine-to-five, could serve on a daily basis. Therefore, digital marketing has also led to greater customer satisfaction.

Research has shown that when a service feels reliable and responsive, positive emotions in customers are usually triggered thereby potentially influencing their decisions [32], [33].

As a result, this promotes favorable behavioral intentions. This was proven in one study

(47)

and thereby cannot be generalized to all service sectors as it was conducted on an optometry clinic. To be better considered as a generalization, carrying out similar studies across different services would be required.

The advances in digital marketing and AI are the main drivers of globalization of companies in modern time [21]. CIs and especially chatbots are becoming more commonly used to help companies reach out to more customers while at the same time meeting both the customers’ and companies’ expectations and goals [21]. These technologies are helping customers save time and effort on shopping. Hence, the chatbots integrated in social media platforms can build better customer-brand relationships through gathering and sharing opinions on new products. In other words, communication quality is the key to support these marketing efforts [21].

It was found to be that communication quality is built on accuracy, credibility and competence [21]. It is further stated that these can be achieved by CIs in the context of interaction, entertainment, customization, trendiness, innovativeness and problem- solving dimensions. These aspects help build better customer relationships by saving time through quickly providing requested information and even possibly giving advice [21].

The studies that were made on chatbots, however, are not many or can be improved upon, in particular the online communication between customers and brands can be researched further. Therefore, it is reasonable to investigate communication in terms of chatbots more.

5.1.2. Chatbots in Travel and Bookings

Whether people travel for business or leisure, booking trips using the internet has not always been as easy as today. Back in the days, people used to depend on human travel agents to complete their bookings. This trend has started to change lately, moving to online booking. However, people have experienced many problems when booking online.

It was shown in a study [46] that people tend to leave their booking process mostly due to encountering a complicated booking user interface or the booking website being not optimized for certain devices. Others have reported that they had faced complications with online booking as they had to search between four to seven websites before finding the right deal [47]. On the other hand, once the travel arrangements were completed, receiving booking confirmation and information was not perceived as a problem by many [47].

Chatbots can help the travel industry by enhancing the online travel booking process in several ways such as finding the best deal or the shortest flight route. It was found in a

(48)

study [47] that chatbots provided a useful online travel experience. This was due to that chatbots were able to ease and aid the process of booking and organizing the trips.

Modern chatbots use machine learning to assist their customers with the tasks of finding out the best deals, booking trips and getting information on the place they are visiting [48], [47]. This is done by empowering the chatbots with natural language processing capabilities leading to a much better, natural user experience and finding the relevant information easier. Indeed, chatbots are considered more valuable if they can help the customers save effort and money.

Another interesting fact is that more and more people are starting to choose the web instead of consulting the people they know for deciding on where to go [47]. Thereby, chatbots that can serve as a concierge are becoming more preferred. Using chatbots for travel arrangements seems to be the case for the future.

According to our interview findings, CIs are not always the optimal solution booking trips.

In particular, sometimes filling webforms could be a more reasonable choice. Webforms are more convenient in the case where personal information needs to be collected such as personal payment information. Users are not always comfortable with sharing such information through a conversation.

5.1.3. Chatbots in Education and Healthcare

Today, the industries that employ chatbots are increasing in number. The education industry is no exception. The advancements in technology allowed chatbots to be used in education and healthcare as well.

The anticipated advantages of chatbots being in the education sector are wide-ranging.

The launch of the AI concept in typical classrooms has not been as fast as the other sectors due to the education sector being more traditional and less adaptable to newly introduced techniques [35]. Nowadays the benefits and cost-efficient aspect of chatbot use in education has become clearer for the teachers and therefore the process of integrating chatbots into education has sped up. One aspect that helped this process to speed up was the fact that how fast the young generation could adapt to the changes [35], [36].

As in other sectors, chatbots have come into play in the healthcare sector as well. An individual approach and patient experience are what matters for the modern healthcare providers. Chatbots can provide customized and personalized experiences for patients.

They can be of great benefit for both the patients and the healthcare professionals for various reasons. Some of these include providing patients with information regarding their