Artificial Intelligence in Healthcare

IN

DEGREE PROJECT INDUSTRIAL ENGINEERING AND MANAGEMENT,

Artificial Intelligence in Healthcare

Ajanth Vaseigaran

Gobi Sripathy

Master of Science Thesis TRITA-ITM-EX 2021:326 KTH Industrial Engineering and Management

Artificiell Intelligens inom Sjukvården

Ajanth Vaseigaran

Gobi Sripathy

Examensarbete TRITA-ITM-EX 2021:326 KTH Industriell teknik och management

Master of Science Thesis TRITA-ITM-EX 2021:326

Artificial Intelligence in Healthcare

Ajanth Vaseigaran Gobi Sripathy Approved 2021-06-08 Examiner Christofer Laurell Supervisor Niklas Arvidsson

Commissioner Contact person

Abstract

Healthcare systems play a critical role in ensuring people's health. Establishing accurate diagnoses is a vital element of this process. As sources highlight misdiagnoses and missed diagnoses as a common issue, a solution must be sought. Diagnostic errors are common in the emergency departments, which has been recognized as a stressful work environment. Today's industries are forced to deal with rapidly changing technological advances that result in reshaped systems, products, and services. Artificial Intelligence (AI) is one of such technologies that can work as a solution to diagnosis issues but comes with technical, ethical and legal challenges. Hence, the thesis intends to investigate how AI can affect the accuracy of diagnosis as well as how its integration in healthcare relates to the technical, ethical and legal aspects. The thesis begins with a literature review, which serves as a theoretical foundation and allows for a conceptual framework to be formed. The conceptual framework is used to select interviewees, which results in 12 interviews with professors, researchers, doctors and politicians. In addition, a survey is conducted to obtain the general public’s opinion on the matter. The findings present that AI is already mature enough to make more accurate diagnoses than doctors as well as release burden from medical practitioners in the form of administrative tasks. One obstacle is the incomplete data available since laws hinder sharing of patient data. Furthermore, the AI algorithms must be fit for all social minorities and not demonstrate racial discrimination. The European AI Alliance was established in 2018 with the aim to keep the technology in check. Similar initiatives can be created on a national- and regional level to maintain some form of control over its proper use.

Examensarbete TRITA-ITM-EX 2021:326

Artificiell Intelligens inom Sjukvården

Ajanth Vaseigaran Gobi Sripathy Godkänt 2021-06-08 Examinator Christofer Laurell Handledare Niklas Arvidsson Uppdragsgivare Kontaktperson Sammanfattning

Sjukvårdssystem utgör en avgörande roll för att säkerställa människors välmående och hälsa. Att fastställa korrekta diagnoser är en viktig del av denna process. Enligt källor är feldiagnoser och uteblivna diagnoser ett vanligt problem och bör därför lösas. Diagnostiska fel är vanligt förekommande på akutmottagningar, vilka karaktäriseras som en stressig arbetsmiljö. Dagens industrier tvingas hantera snabbt föränderliga tekniska framsteg som resulterar i omformade system, produkter och tjänster. Artificiell Intelligens (AI) är en av sådana tekniker som kan fungera som en lösning på diagnosfrågor. Dock kommer den med tekniska, etiska och legala utmaningar. Examensarbetet avser därför att undersöka hur AI kan påverka diagnosens precision samt hur integrationen i vården relaterar till de tekniska, etiska och legala aspekterna. Rapporten inleds med en litteraturstudie, vilket fungerar som en teoretisk grund och bidrar till att skapa ett konceptuellt ramverk. Det konceptuella ramverket används för att välja intervjupersoner, vilket resulterar i 12 intervjuer med professorer, forskare, läkare och politiker. Dessutom genomförs en enkätundersökning för att få allmänhetens åsikt i frågan. Rapportens resultat visar att AI redan är tillräckligt utvecklad för att göra en mer precisionssäker diagnos än en läkare samt kan avlasta läkare i form av administrativa uppgifter. Ett hinder är att den data som finns tillgänglig är ofullständig på grund av lagar som hindrar delning av patientdata. AI-algoritmerna måste dessutom vara lämpliga för alla sociala minoriteter och inte leda till rasdiskriminering. European AI Alliance grundades 2018 med målet att hålla tekniken i schack i förhållande till de etiska och legala aspekterna. Liknande initiativ kan skapas på nationell och regional nivå för att bibehålla någon form av kontroll över dess korrekta användning.

Acknowledgement

We would like to extend our deepest gratitude to our supervisor Niclas Arvidsson who supported us through the process with meetings and discussions. We are also grateful to our examiner Christofer Laurell for his comments and suggestions during the seminars. We would like to thank all professors, researchers, doctors and politicians for the interviews, which contributed to the findings of the thesis. Especially the doctors who, despite their heavy workload during the pandemic, distributed time for the interviews. Many thanks to the 102 respondents who participated in the survey and contributed to the findings. We also appreciate the help from our opposition for constructive feedback throughout the seminars.

Table of contents

2. Literature review and Conceptual Framework 4

2.1 Transformation in healthcare 4

2.2 Big data 5

2.3 Artificial Intelligence 5

2.3.1 Machine learning 6

2.4 AI in healthcare 8

2.4.1 AI’s potentials and contributions to healthcare 8 2.4.2 AI’s limitations and difficulties in healthcare 10

2.5 Ethics 11

2.6 Laws & regulations 13

4.1.4 Interaction with medical practitioners 42 4.1.5 Divided interests 43 4.2 Ethical concerns 44 4.2.1 Consent 44 4.2.2 Trustworthiness 45 4.2.3 Responsibility 47

4.3 Laws & regulations 48 4.3.1 Patient integrity vs patient safety 48

4.3.2 Cybersecurity 50

5. Discussion 51

5.1 AI, ethics and laws & regulations in healthcare 52

5.2 Developed framework 56

6. Conclusions 57

6.1 Future research and limitations 58

6.2 Implications 59

6.2.1 Theoretical implications 59 6.2.2 Implication for healthcare practitioners 59 6.2.3 Implication for healthcare organisations 60

References 61

Appendix 67

A. Survey 67

List of Figures

Figure 1. UN sustainable development goals (UN, n.d.)

Figure 2. Examples of big data sources in healthcare (Bohr and Memarzadeh, 2020). Figure 3. Definition of AI inspired by Wu (2019).

Figure 4. Conceptual Framework.

Figure 5. Illustration of the research process.

Figure 6. The steps of qualitative data analysis inspired by Miles and Huberman (1994). Figure 7. Gold standard (Funk, 2015).

Figure 8. Presentation of the participant's experience regarding misdiagnosis and missed diagnosis.

Figure 9. Illustration of survey participants’ experience at emergency departments. Figure 10. Presentation of patients’ worst experience at emergency departments.

Figure 11. Collaboration and Experience Sharing, inspired by Professor Funk’s presentation. Figure 12. Illustration of case-based reasoning inspired by Professor Funk’s presentation. Figure 13. Illustration of participants’ willingness to share their patient data.

List of Tables

List of Abbreviations

Artificial Intelligence (AI)

Artificial Neural Network (ANN) Computed Tomography (CT) Deep Learning (DL)

Electrocardiogram (ECG) Electronic Health Record (EHR) European Union (EU)

Extensible Markup Language (XML)

General Data Protection Regulation (GDPR)

Information and Communication Technologies (ICT) Internet of Things (IoT)

Karolinska Institutet (KI) King's College London (KCL)

Kungliga Tekniska Högskolan (KTH) Machine Learning (ML)

Magnetic Resonance Imaging (MRI) National Health Service (NHS)

Nationell Patientöversikt (NPÖ) [in English: National Patient Overview] Natural Language Processing (NLP)

Optical Coherence Tomography (OCT) United Nations (UN)

List of Glossary

Adversarial attack

Adversarial attacks take advantage of anomalies in machine learning models' learned parameters. An attacker tests a target model by making small changes to its input until it behaves as desired. Balloon angioplasty

The technique of balloon angioplasty is used to open narrowed or blocked arteries. It involves inserting a catheter into an artery with a balloon attached to it. The balloon is inflated at the spot where plaque deposits have closed off or narrowed the blood flow channel.

Blockchain

Blockchain is a decentralised method of storing data in such a way that it is difficult or impossible to alter, hack, or trick it.

Cybersecurity

It is the art of preventing unauthorized access to networks, computers, and data, as well as the practice of ensuring information's integrity, confidentiality, and availability.

Diagnosis

The process of deciding which disease or disorder causes a person’s signs and symptoms. Internet of Things

It is a generic term for things that have built-in electronics and connection. IoT is devices that can talk to each other and share important information with humans. IoT connects the digital and physical worlds with each other.

Triage

1. Introduction

This section begins with presenting the background on healthcare systems and the problem of diagnostic accuracy in Sweden today. Research questions are formulated based on the problem and this section concludes by presenting the delimitations of the study.

1.1 Background

The healthcare systems have a key responsibility in caring for people’s health and are vital to having healthy societies and individuals. Such systems have been around ever since human beings have tried to take care of their health and cure diseases. According to Donev, Kovacic and Laaser (2013), the main objective of healthcare is twofold. The first is to respond in accordance with what people are expecting from it and the second is to respond equally to everyone.

Determining the right diagnosis is a crucial part of achieving those two objectives of healthcare. The diagnosis is based on a synthesis of information from and about the patient, which is interpreted by a caregiver. A specific treatment can then be selected based on the diagnosis established (NE, n.d). In Sweden, 10 to 20 percent of all serious injuries in healthcare have an incorrect, delayed or undetermined diagnosis as the cause. The diagnostic errors are most prevalent in the emergency department, a work environment that is identified as stressful (Andersson, 2020). High patient flow and increased length of stay are linked to increased mortality, decreased patient safety, increased number of misdiagnoses, increased risk of complications and dissatisfaction in patients (Cheng, 2016). The term “lex Maria” refers to reporting incidents that may have caused or have caused substantial injury to a patient (Socialstyrelsen, n.d.). Between 40 to 50 percent of all lex Maria registrations consist of cases from the emergency department and primary care (Andersson, 2020).

1.2 Problem formulation

Previously, healthcare was mainly about the private doctor-patient relationship but is now made up of a complex network with both human and non-human actors. This network consists of components such as hospital information systems, health records and databases. Digital technologies have enabled efficient management of complex networks by improving aspects such as connectivity, communication and flow of information (Belliger & Krieger, 2018). However, misdiagnosis and missed diagnosis are still occurring despite the technological advancements in the industry, especially in the emergency department. This is a department that is known for facing various cases during time pressure. Those who are responsible for overseeing healthcare are presented with challenges because of the worsening outcomes of healthcare systems. Clinical entrepreneurs, politicians and scientists are increasingly arguing that AI will be a crucial part of the solution. The technology of AI is shown to have great benefits in different sectors, also in healthcare. However, it faces challenges related to at least three aspects: technical, ethical and legal aspects (Morley et al., 2019).

1.3 Purpose

The purpose of this study is to investigate if AI can affect the accuracy of diagnosis in healthcare. Further, the study will proceed by examining if AI is an appropriate solution in healthcare and more specifically the emergency department in relation to the technical, ethical and legal aspect.

1.4 Research questions

To fulfil the purpose of the study, the following research questions are formulated.

RQ1 How can AI affect the accuracy of diagnosis?

RQ2 How does AI in healthcare relate to the technical, ethical and legal aspects?

1.5 Delimitations

healthcare has many aspects that need to be taken into consideration. It is not only the ethical, legal and technical aspects. The implementation of AI in healthcare requires a change process, which in turn can imply challenges on an organisational level. These factors, such as costs, organisational behaviour etc. have not been processed in this thesis. Furthermore, a delimitation could take place to study one common disease or sickness in emergency care. However, this study has a more general approach and touches upon common diseases and sicknesses.

1.6 Sustainability

The three components of sustainability are: social, economic and environmental. Since social sustainability includes human health, life expectancy and medical services, the thesis connects to the social dimension of sustainability throughout the report (Kuhlman & Farrington, 2010). The UN's sustainable development goals work as a blueprint for achieving a better future. The sustainable development goals are presented in Figure 1.

Figure 1. UN sustainable development goals (UN, n.d).

2. Literature review and Conceptual Framework

In this chapter, the literature review is presented. The literature review starts with a brief account of the transformation in healthcare and continues with the concept of Big Data. Moving on, Artificial Intelligence is described in a general point of view and the section also includes Machine Learning, from both a general and from a healthcare perspective. Thereafter, some of AI’s potentials and contributions as well as its limitations and difficulties concerning healthcare are presented. The limitations and difficulties include AI’s challenges in regards to ethics, and also laws & regulations. The general view on AI in healthcare continues with a section about its implementation in the emergency department. This chapter concludes with a conceptual framework based on the literature review.

2.1 Transformation in healthcare

The world is under transformation and lots of changes are being seen in the technology aspect as well as people’s lives. Concerning life expectancy, it is expected to experience a major change even if it has been developing in the last decades. By 2050, 1.5 billion people are expected to reach the age of 65 and above, which is more than double of what it was in 2019. The United Nations (UN) and World Health Organization (WHO) estimate that by 2025, the majority of all illnesses are related to chronic, comorbid conditions. These types of conditions often times call for continuous and increased attention from healthcare workers. The healthcare cost is also expected to increase radically as a consequence of the ageing population. Hence, there is a need to adapt to these changes and for the healthcare processes to cope with the growing healthcare demand (Bohr & Memarzadeh, 2020).

2.2 Big data

Big data is defined as large, complex and diverse datasets. It encompasses volumes of collected data from different sources, velocity in terms of the speed of processing and generating data according to the demand, and a variety of different formats of datasets. Since the acquired data is often unstructured and contains errors, it must be filtered, organized, and validated before it can be analyzed. This is well-aligned with the datasets generated and related to healthcare. Therefore, to process and concretize big data into useful information is crucial in order to provide transparency, improved performance, customized action and replace/support human decisions with automation (Bohr & Memarzadeh, 2020).

Figure 2: Examples of big data sources in healthcare (Bohr and Memarzadeh, 2020).

Big data in healthcare, as shown in Figure 2, can be gathered from multiple different sources. It is not required to be obtained from established sources such as research studies, electronic health records (EHR), or medical equipment. Also, data from search engines and social media may be quite useful in determining correlations (Bohr & Memarzadeh, 2020; Lalmi & Adala, 2021).

2.3 Artificial Intelligence

computer-controlled robot to perform tasks commonly associated with intelligent beings”. It is mentioned that the term is frequently applied to developing systems that have qualities as humans to be capable of reasoning, discovering meaning, generalizing, or learning from past experience. This occurs through massive data (Copeland, 2020). Since human abilities are rooted in a broad sense, AI can also be defined as in Figure 3 (Wu, 2019).

Figure 3. Definition of AI inspired by Wu (2019).

2.3.1 Machine learning

neural networks (ANNs), natural language processing (NLP) and deep learning (DL) are the most common (Bohr & Memarzadeh, 2020).

The artificial neural network is used to optimize data. It predicts the output with the use of input data. ANNs are built up of three layers, input layer, hidden layer and output layer. The input layer gathers the data and transfers it to the hidden layer where patterns are extracted with the use of mathematical models. When the data has been processed in the hidden layer, the result is produced and presented in the output layer. The typical work process of the ANN is that it takes 70 % of the input data to build a network, 15 % to train itself and the remaining 15 % to test itself to generate the output layer (Shahid, Rappon & Berta, 2019). The advantage of ANN is that literature-based and experimental data can be linked and integrated. It also has the ability to learn with an inductive approach from training data, operate with missing information and generalize to related undetected data (Kustrin & Beresford, 2000).

NLP is the technology that aids computers to understand and process the human’s natural language. With the use of NLP, the written text can be processed and understood, even though the text is formulated in numerous ways or does not follow logical and consistent rules of the language. In the healthcare area, NLP is used to transform unstructured clinical text data into structured data. For example, extracting data from varied text-based sources such as medical records and clinical notes. These are often unstructured and hard to understand for computer programs and thus NLP can be advantageous in extracting the appropriate information to match the narrative (Bohr & Memarzadeh, 2020). Despite its widespread use in clinical settings, symbolic NLP is not portable due to differences in clinician reporting practices. As a result, if the NLP part isn't portable, any AI that uses it for feature extraction will run into similar problems (Wen et al., 2019).

2.4 AI in healthcare

This part begins by describing the potentials and contributions AI brings to healthcare. Thereafter the limitations and difficulties of AI in healthcare are presented. The limitations of AI are further described in subchapters 2.5 and 2.6.

2.4.1 AI’s potentials and contributions to healthcare

AI has the ability to provide a developed approach to the care processes and techniques. Heart disease is the leading cause of death in the world. The death toll is increasing, as of 2019, it accounted for one-third of all deaths (Preidt, 2020). A research that was conducted over nine years, involving 243 hospitals and about 600 000 heart attack cases showed that around one-third of the cases were initially misdiagnosed. The study also looked at different factors that affected the survival rate. Some of these factors included delays in getting treatment, availability of specialist hospital facilities and staffing. Dr. Mike Knapption, Associate Medical Director at the British Heart Foundation said: “It’s important to get the diagnosis right straight away, if the correct diagnosis is delayed this will, in turn, delay the treatment. Time is critical; for the best outcome patients need the correct treatment as quickly as possible” (British Heart Foundation, n.d.). A way to check potential heart diseases is to record a patient’s heart electrical signal activities over a long period of time. It can be recorded by an electrocardiogram (ECG). An ECG allows to accurately see anomalies, however, to see these anomalies may take more than the 10-second ECG recording that is standard in hospitals. With the help of portable ECG devices, new sensing technologies have enabled the possibility to monitor patients' heart conditions on a long-term basis. The different technologies can measure a patient round the clock and transmit the information to a cloud service to be stored. This creates huge data sets, which are not very helpful for the medical practitioners as they generally do not have enough time or resources to go through the recordings. In order for doctors to detect heart problems based on large data sets, which are proven to be more accurate, reliable heart anomaly detection algorithms need to be developed. Many studies have incorporated deep learning models and published new algorithms to classify heartbeats (Li & Boulanger, 2020).

embedded in smartphones entails the possibility to detect high-risk moles. This will facilitate individuals to seek a doctor at an early stage (Troyanskaya et al., 2020). According to a study conducted with the help of an international group of 58 dermatologists, including 30 experts, AI outperformed the dermatologist in diagnosing skin cancer (Haenssle et al., 2018).

With the help of EHRs, combined with machine learning, there is a huge potential to improve the understanding of cancer. EHRs contain several million patients’ data over many years. There is a massive amount of data that can be used to improve the understanding of cancer. For example, screening for breast cancer is only recommended for women between the ages 50-75 years old and for lung cancer, it is only recommended for smokers. Studies have been made that describe how hormonal birth control, cholesterol abnormalities and immune or gynecological inconsistencies have an impact on the risk of getting breast cancer. Models that are EHR-based can study new cases, identify risk factors and link them with formerly established factors to set risk scores, which are more accurate. This will improve early detection and also increase the screening quality for cancer. Thus the use of machine learning algorithms with EHR data will abolish the general screening guidelines to be replaced with more intelligent and personalized recommendations. An example of a non-invasive method to detect cancer is by radiology. This method can create many false-positive and false-negative assessments. EHR data can contain information about several different abnormalities, fluctuations, problems, infections, etc. These can be fed in machine learning models to derive relations of different cancers and be combined with a radiological assessment. In this way, the false-positives and false-negatives will decrease and the accuracy of cancer diagnosis will increase (Troyanskaya et al., 2020).

In 2016, a project was conducted by Moorfields Eye Hospital NHS Foundation Trust and DeepMind (today owned by Google Health). They partnered to detect and diagnose serious eye conditions from 5000 optical coherence tomography (OCT) scans. The sample size constitutes the weekly amount of OCTs scans at the hospital. Based on the National Health Service (NHS) referrals, the AI focused on 53 key diagnoses parameters. The results showed that the AI had an accuracy of 94 % and reaches or exceeds a variety of eye disease experts after only training on 14,884 scans (Holm, Stanton & Bartlett, 2021).

AI has many administrative capabilities in healthcare. The high workload is an indicator of a stressful environment which ultimately affects the quality of care and patient outcomes. Research has shown that administrative tasks extensively contribute to workload and time pressure for healthcare workers (Hazarika, 2020). The average nurse in the US spends about 25 % of work time on regulatory and administrative activities. AI can be applied in a variety of areas to simplify the processes related to documentation and management of medical records, consequently reducing the administrative burden (Davenport & Kalakota, 2019).

Researchers have used AI to forecast health events. In different studies, AI has been used to develop models to predict disease outbreaks. For example, prediction models for outbreaks of dengue disease are built up by different ML algorithms and sensing data including remote (satellite or aircraft sensors) and local (data measured on e.g. rainfall) data. Researchers have also used AI to predict Malaria and Zika virus outbreaks with accuracy greater than 85 % (Schwalbe & Wahl, 2020).

2.4.2 AI’s limitations and difficulties in healthcare

Despite all the benefits the technology of AI presents, there are difficulties and limits to overcome before it can be implemented. There are existing worries regarding factors such as safety, the lack of transparency, interpretation of results as well as AI-user interactions (Esmaeilzadeh, 2020; Quinn et al., 2020).

clinician-patient relation. Patients have to reduce their fear against AI devices and clinicians need to become proficient in interaction with AI systems in order to achieve an efficient healthcare delivery. (Esmaeilzadeh, 2020)

In order for an AI to function properly, information about prior knowledge needs to be provided by an expert in the domain field. This means that the model designer inserts his/her expertise into the model architecture and makes sure the model learns in an optimal way. This is difficult because one of the problems with healthcare cases is that they are complex and vaguely defined. For a model designer to guide the AI is a complex task, since the reasoning from doctors does not follow a certain praxis. Furthermore, real-world examples provided by the training data is beneficial. The AI model is provided as a general model to adhere to the observed data. This can be handled in an acceptable way, even with unclarities, by finding statistical patterns from the training data. In this way, the algorithm created by the domain expert of the AI model is updated to relate to the imperfections. However, data from the healthcare application are not always complete. They can be scarce, which can give a false image of reality and give rise to models that are not generally representative for all patients (Quinn et al., 2020).

2.5 Ethics

There are severe ethical challenges and implications that come with utilizing machine learning in healthcare for decision-making. These are associated with principles such as accountability, transparency and justice (McCradden et al., 2020). Today, the caregivers are responsible for the decisions they make and must be able to justify their actions. This means that if a care provider causes harm to a patient by establishing a clear misdiagnosis, that care provider will be held accountable. A problematic scenario is if the algorithms from the machine provide a diverging diagnosis than a caregiver. This might result in clinicians being biased towards interpreting the evidence in a way that is in line with the diagnosis by the machine’s algorithms (Grote & Berens, 2019). The principle of justice is treating like cases alike and different cases differently. When it comes to ML, it is unfitting to consider one result similar for two patients when one of them runs a greater risk of receiving error results (McCradden et al., 2020).

Large datasets are essential for the use of AI technology in public health. The data related to health is regarded as one of the most sensitive information about a human being. When using AI systems, the patients’ personal data will probably be stored and shared across the network. Challenges regarding privacy, safety and governance are created because of the concerns regarding eventual data breaches. The patients are also anxious about information being gathered without their consent nor knowledge since AI systems require huge datasets to work properly. The models and algorithms of the machine must be limited with respect to the anonymity of the patients in order to hinder patients from getting psychologically and reputationally harmed. (McCradden et al., 2020; Esmaeilzadeh, 2020).

If the AI algorithms are uncritically adopted, they may develop into the repository of the collective medical mind (Arnold, 2021). Hence, in order to deliver effective healthcare, trust between healthcare systems and the public needs to be in place. This also refers to people's perception that one cannot rely on AI's predictive and diagnostic models. The fact that most patients do not understand the structure of AI and how it works may result in reduced trust in the technology. AI models, such as deep learning can increase the lack of transparency related to AI systems (Esmaeilzadeh, 2020). Worries regarding the consistency of accuracy exist since AI may encounter patient scenarios that it has not been trained on. Furthermore, as AI systems may collaborate with smart wearables, such as smartwatches and pulse trackers, these gadgets must be accurate as well. In a recent study, Fitbit PurePulse Trackers proved to not provide a precise estimation of the users’ heart rate as it diverged from the readings of ECG by an average of 20 bpm. The trust of the public has to be gained in order for a successful implementation of AI in healthcare (Hamid, 2016).

biases are inevitable in predictive programs because of the overrepresentation of social minorities in the process of pattern recognition (Esmaeilzadeh, 2020).

2.6 Laws & regulations

The European Commission has highlighted AI’s need for an appropriate legal and ethical framework. Therefore, in 2018, a European initiative took place resulting in creating the European AI Alliance. This initiative ensures a forum to discuss AI and its challenges. When the European AI Alliance was set up, an independent expert group was formed to oversee and work as a steering group in the European AI Alliance. The expert group is also responsible for policy development regarding AI. In 2019, they published a framework named “Ethics Guidelines for Trustworthy AI”. The guidelines’ seven key arguments in regard to ensuring trustworthy AI are: “(1) human agency and oversight, (2) technical robustness and safety, (3) privacy and data governance, (4) transparency, (5) diversity, non-discrimination and fairness, (6) environmental and societal well-being and (7) accountability”. (Gerke, Minssen & Cohen, 2020)

In the field of medical devices, two EU regulations were issued in 2017. The Medical Device

Regulation [2017/745—MDR; see Art. 123(1) of the MDR] and the Regulation on in vitro diagnostic medical devices [2017/746—IVDR; see Art. 113(1) of the IVDR]. Both of these were

entered into force to ensure “high level of safety and health whilst supporting innovation”. Another law that entered in all EU Member States is the General Data Protection Regulation (GDPR-2016/679). The GDPR concerns the protection of personal data of an individual and is applied to the “processing of personal data in the context of the activities of an establishment of a controller or a processor”. The definition of “processing” is defined in the GDPR as “any operation or set of operations which is performed on personal data or on sets of personal data, whether or not by automated means,” including collection, structuring, storage, or use [Art. 4(2) of the GDPR]. The “controller” is “the natural or legal person, public authority, agency or other body which, alone or jointly with others, determines the purposes and means of the processing of personal data [Arts. 4(7) of the GDPR]. For example, controllers have to, according to Articles 22(1) and (4), 13(2)(f), 14(2)(g), when personal data are collected with the existence of automated decision-making, inform data subjects of its use and in what way. This includes profiling, which means to evaluate a person’s personal data to analyze or predict behavior. (Gerke et al., 2020)

Act [Regulation (EU) 2019/881] issued in 2019. When the act entered into force, a framework was developed to ensure that information and communication technologies (ICT) services, products and processes have a high level of cyber resilience and cybersecurity (Gerke et al., 2020).

AI differs from general technologies in healthcare in the sense that it is able to learn from the real world automatically. In this way, it is challenged on a regulatory level. For legal authorities, medical workers and health services it is fundamental to provide safe and high-quality healthcare. AI in healthcare results in algorithms that are not explainable in decision-making. It also leads to continuous changes with the automatic updates. Prior decisions that are not approved may change. Hence there is a need for special policies and guidelines that adheres to these. The safety and behaviour of AI algorithms can be a potential barrier for AI to fully be implemented in healthcare. Also, the concerns related to liability have to be addressed since it is not clearly distinguished who the responsible part is when errors occur related to AI technology (Reddy, Allan, Coghlan & Cooper, 2019). Therefore, informed consent, cybersecurity, data protection, algorithmic behaviours, safety and liabilities and other regulatory barriers need to be addressed when implementing AI in healthcare. It is also a topic that needs to be further addressed with public and political discussions (Gerke et al., 2020).

Patient Act [2014:821)

The purpose of the Patient Act is to strengthen the patient's position and clarify it, make the patient more involved and self-determined and promote patient integrity. The goal is good health and care on equal terms and the patient with the largest needs should take precedence. According to the Patient Act, care shall be easily accessible and the patient must receive an urgent medical assessment if the need exists. If the patient does not receive care within a certain time, the patient can take advantage of the care guarantee and receive care with another care provider at no extra cost. The patient should be told when he or she can be expected to receive care, how the care and treatment will look like, what complications and side effects that may occur. The patient should also know how aftercare will look like and how the patient can prevent illness or injury (Ericsson, 2015).

the information. Personal data must be designed and processed so that the integrity of the patient and others mentioned in the record are respected. (Ericsson, 2015)

The patient must consent to care and the patient's self-determination and integrity must be respected. If the patient does not agree, information is provided on what the consequences of non-care or treatment can be. If the patient is unable to give consent, for example, due to unconsciousness, treatment is still given if acute and serious danger to the patient's life arises. If the patient requests a permanent care contact, one should be appointed to meet the patient's needs for, among other things, security and continuity. A patient who has suffered a healthcare injury must be informed about the occurred injury and what measures the care provider will take to prevent similar situations from happening again. The patient should be informed that it is possible to make a report to the Inspectorate for care if he/she has any complaints about the care (Ericsson, 2015).

Patient Data Act [2008:355]

“Informationshantering inom hälso- och sjukvården ska vara organiserad så att den tillgodoser patientsäkerhet och god kvalitet samt främjar kostnadseffektivitet. Personuppgifter ska utformas

och i övrigt behandlas så att patienters och övriga registrerades integritet respekteras. Dokumenterade personuppgifter ska hanteras och förvaras så att obehöriga inte får tillgång till dem” [Information management in health care must be organized so that it meets patient safety

and good quality as well as promoting cost efficiency. Personal data must be designed and otherwise processed so that patients’ and other data subjects' integrity is respected. Documented

personal data must be handled and stored so that unauthorized persons do not have access to them] (SFS 2020:1042)

A caregiver, an authority, a county council or a municipality, is responsible for how personal data is processed. Only personal data that is needed to be able to document the care of, among other things, the patient and perform administration related to the patient, may be treated. Plan, follow up and evaluate the operation and get statistics are other ways of processing personal data. The purpose of keeping patient records is primarily to ensure that patient care is good and safe. A patient record contains information that is important for the patient and for the operation in terms of development and follow-up. It is important to see if the care has gone right and according to legal requirements. It is also a source of information for research. (Ericsson, 2015)

possible and signed by the person responsible for the journal entry. If a journal copy is requested, this must be documented in the patient record, who received it and when it happened. Information in the patient record must not be deleted or made illegible, if a correction is made, it must be clear who did it and when. The patient record must be kept for at least ten years after the last note was entered. (Ericsson, 2015)

In a patient record, the patient's identity and background to the care, what diagnosis was made and what measures were planned and taken shall be included. It must also contain information about which position taken by choice of treatment and which information provided to the patient or patient representative. It must also be stated whether the patient has chosen to refrain from treatment or care. If the patient thinks that something is wrong in the medical record, this should be noted. The language in the patient record must be written in Swedish as well as being clear and easy to understand for the patient. Only the people who participate in the care of the patient or the ones who need information for their work have the right to take part in patient records. The caregiver has the right to check if any unauthorized people have read the patient's medical record. If the patient opposes that another care unit at the same care provider has access to the patient's medical record, the information must be blocked. The lock may be lifted if the patient agrees to it. It can also be lifted if the patient is unable to give consent and the information in the record may be relevant for patient care. If there are several authorities within the same municipality or county council who provide health care, they may have direct access to personal data processed by another authority within the same municipality or county council. If the patient opposes coherent record keeping, personal data cannot be made available to other caregivers. Information that the journal has blocked data and who blocked them may be shown to other healthcare providers. The patient should be informed about what cohesive record-keeping means before the patient's information is made available. If the patient opposes this, the data should be blocked, the block can on the patient's request be revoked at any time. Unobstructed information will be made available to care providers who are connected to the system with coherent record keeping. A care provider may make data accessible to others if a patient is unable to take a position or by assuming that the patient would not oppose this. (Ericsson, 2015)

and names may be used where coded personal data is not sufficient. When the personal data is no longer needed in a register, they must be eliminated unless an archival authority has decided that the data may be retained for historical, statistical or scientific purposes. (Ericsson, 2015)

2.7 Emergency department

Time and accuracy of diagnosis are of the essence in emergency departments. An environment usually identified as stressful with increasing wait times as well as diagnostic errors (Grant & McParland, 2019). Today, doctors acquire information from various sources such as patients, imaging, laboratory or monitoring. Consequently, experience is improving their accuracy of making decisions from the retrieved data. The sources may differ between hospitals, hence experience may vary from doctor to doctor. AI with its sub-branches (for example machine learning) presents promising potential to support decision-making and enable a fair healthcare delivery (Barh, 2020).

Triaging patients via algorithms in the emergency department is one of AI’s applications. A previous study that investigated triage waiting times and wrong triage results highlighted a mean success rate of 94 % for five disparate data mining models. Furthermore, machine learning techniques with a dataset containing 1205 patients’ information was used to predict emergency patients in pediatric emergency care. The objective was to reduce waiting times and it turned out to produce accurate as well as valid outcomes. Both hospital triage and prehospital environments with remote triage systems are used with machine learning methods. The symptom-based triage of AI has been tested and was discovered to be safer than human decisions. (Barh, 2020)

Table 1. Characteristics, problems and the aid of AI in the emergency department.

2.8 Conceptual framework

AI’s integration to healthcare and more specifically to the emergency departments is hampered by a number of obstacles. Although, since the scope of the study is delimited to the technical, ethical and legal aspects, the literature review focuses on these topics. Based on the research topic, problem formulation and literature study the following framework has been established with the main purpose of structuring the findings.

The literature shows that AI has several potential benefits in healthcare, for instance, AI can operate as assistance to doctors. It can also save healthcare professionals time from doing administrative work. Studies have proven that AI can forecast health events with great success. However, the literature also presented the limitations which included the risk for malfunctioning machines. Furthermore, communication barriers may occur in the human-machine relationship and the face-to-face interactions will also be reduced. Another issue is data quality, which is incomplete according to the literature study (Syeda-Mahmood et al., 2016; Davenport & Kalakota, 2019; Schwalbe & Wahl, 2020; Esmaeilzadeh, 2020).

The legal challenges include laws such as Patient Act, Patient Data Act and GDPR that have a negative impact on data quality. Since storing and sharing data is difficult in healthcare, AI does not have the conditions to function effectively. However, initiatives have been taken towards AI by the European Commission. They have formed a European AI Alliance that is responsible for developing policies and creating a framework for the usage of AI in healthcare (Gerke et al., 2020; Ericsson, 2015).

The development of the emergency department is affected by the three pillars “AI”, “Ethics” and “Laws & Regulations”. The framework in Figure 4 not only illustrates how the different parts are connected but are also used for structuring the coming chapters. The arrows on the framework illustrate that in order for AI technology to reach the emergency department, it not only has to meet the technical requirements but also interact and agree with the legal and ethical aspects.

3. Method

This section describes how the report intends to answer the research question. In the beginning, the focus is put on the research design, describing the methodology of this paper. Moving on, this chapter presents how the data collection is made, the data analysis and the research quality.

3.1 Research design

This paper follows the approach of a qualitative and quantitative study to investigate the research question. A qualitative study is a meaningful representation of words, talk and texts and is great for addressing “how” questions rather than “how many” (Pratt, 2009). Furthermore, a qualitative method can preferably be used in instances where complex phenomena need contextualisation, which in turn enables an in-depth and increased understanding of the study. (Hennik, Hutter & Bailey, 2020). Another typical aspect of qualitative study is using a variety of data gathering techniques (Saunders, Lewis & Thornhill, 2015). A part of this study is quantitative. With a quantitative method, numerical data is collected in order to investigate relationships between variables (Saunders et al., 2015). This qualitative and quantitative method is aligned to the research aim of this paper, which is to investigate how the accuracy of diagnosis affects healthcare and how AI in healthcare relates to the ethical and legal aspects. While some may claim that combining several methods takes more time, it has been demonstrated that the benefits typically exceed the disadvantages in terms of obtaining a more thorough picture of the investigated phenomena (Shah and Corley, 2006).

In this study, an inductive approach to the problem is adopted. An inductive approach means engaging in the information and details of data to find significant patterns, themes and inter-relationship. Using an inductive design is characterized by initializing with literature to increase the knowledge about the research area (Creswell, 2013).

3.2 Research process

combined with the survey data. Lastly, the results and literature review were connected to give rise to a discussion. This resulted in an adjusted framework and finally the conclusion of the study.

3.3 Data collection

The data gathering in this paper was obtained from primary and secondary sources, including literature review, interviews and a survey. The literature helped increase the overall knowledge of the subject and also facilitated the understanding of the findings made in the interviews. According to Saunders, Lewis and Thornhill (2015), independent sources are preferable when collecting data. Since the data in this report were collected through different methods, it adds further perspective on the researched phenomena. This enables triangulation (Creswell, 2013)

3.3.1 Secondary data

In order to create a solid baseline of knowledge in current research, a literature review was conducted. The literature review is based on secondary sources such as books and academic articles in the chosen areas of research. Artificial Intelligence and Big Data were two of the subjects included in the literature review. The literature review also included studies that addressed these concepts in context with healthcare. Moreover, a limited amount of information was retrieved from non-journal or book sources. These are state-owned websites and encyclopedias. Since there was a difference in reliability between sources, various literature was investigated to only make use of the valid ones. If a source was partial, it was not used in order to keep the paper credible (Durach et al., 2017).

3.3.2 Interviews

Swedish Research Council were followed (Vetenskapsrådet, 2002). The length of the interviews ranged from approximately 20-65 minutes.

Professor Hossein Azizpour

Researcher 1 is an assistant professor in ML at KTH Division of Robotics, Perception and Learning. The main area of expertise for Professor Azizpour initially was computer vision but it gradually shifted to ML with the advent of DL. The research in DL was applied to visual recognition, in other words, to understand what an image or video contains. He also did a postdoc at Science for Life Laboratory where he did a project on mammograms for breast cancer in collaboration with Karolinska Institutet (KI).

Professor Pawel Herman

Professor Herman has a wide research interest that revolves around the brain, trying to better understand certain cognitive aspects of the human brain capabilities, involving memory, attention, and decision making. He builds network models and tries to tackle some of the questions at the border of biology, cognitive neuroscience and psychology with the use of computation, mathematical and simulation techniques. His favourite application of ML and pattern recognition is in the medical domain and diagnostics is a part of it.

Professor Peter Funk

Professor Funk completed his PhD at Edinburgh University in AI and has been leading the AI group at Mälardalen University since 20 years. Prior to this Professor Funk has worked for Ericsson with R&D in the field of AI for communication systems. He has a strong track record in both national and international research projects in the healthcare sector.

Professor Magnus Boman

Professor in Intelligent Software Services at School of Electrical Engineering and Computer Science in KTH. Professor Boman has worked with AI for 30 years, of which the last 15 years within healthcare applications. He works at KI two days of the week, one of them to lead the project named “AI at KI ” and the other to work with the Precision Medicine Task Force which is about getting precision medicine at the clinic. Professor Boman also works for two Universities in London, University College London (UCL) and King's College London (KCL), doing health-related work.

Researcher Sophie Monsén Lerenius

several years of experience from the pharmaceutical and med-tech industry as well as management consulting.

Researcher Fredrik Karlsson

Researcher Karlsson has a biomedical degree and he provides support to KTH's researchers, schools and centers on research ethics issues, primarily with regard to ethical review and export control.

Politician 1

Politician 1 is a regional councilor with responsibility for healthcare. Politician 1 is the vice-chairman of the healthcare committee in one of the regions in Sweden. The politician does not possess a medical education but has a master's degree in economics from Stockholm School of Economics. A large part of the politician’s professional life has consisted of working in the business sector, both in own companies as well as being employed.

Politician 2

Politician 2 is a regional councilor and vice chairman of the regional board with specific responsibility for digitalisation.

Politician 3

Politician 3 is a regional councilor and chairman in the healthcare committee in one of Sweden’s regions.

Doctor 1

Medical intern at Sankt Görans Hospital, currently working at the surgeon department. Here the doctor encounters all types of surgical cases, from trauma to broken bones as well as emergency cases. In the emergency room, the doctor makes assessments of whether the patients should be admitted to get surgery or whether they should go home.

Doctor 2

Medical resident at Huddinge Hospital, doing a specialist service in internal medicine, meaning that the doctor takes care of all types of internal medical problems, much of the time in the emergency department.

Doctor 3

A summary of the interviewees, their profession and mainly covered topics are presented in Table 2. Since the interviewees are selected based on the conceptual framework, they have been marked with different colours that indicate which pillar of the framework they were initially selected for. Whereas the colour blue represents AI, green laws & regulation and yellow ethics.

Table 2. Summary of the primary data collection

Interviewee referred to as Profession Mainly covered topic Professor Azizpour Professor AI, Ethics

Professor Herman Professor AI

Professor Funk Professor AI, Ethics, Laws & Regulations Professor Boman Professor AI, Laws &Regulations Researcher Lerenius Researcher AI

Politician 1 Vice Chairman Laws & Regulations, AI, Ethics Politician 2 Vice Chairman Laws & Regulations, AI Politician 3 Chairman Laws & Regulations

Doctor 1 Medical Intern Ethics, AI Doctor 2 Medical Resident Ethics, AI Doctor 3 Medical Resident Ethics, AI

Researcher Karlsson Medical Resident Ethics, Laws & Regulations

3.3.3 Survey

of the survey were based on answers from 102 respondents, out of which 67 % were men and 33 % women. The majority of the participants (97 %) were within the age group of 16 to 29 years old.

3.4 Data analysis

The steps of qualitative data analysis by Miles and Huberman’s (1994) was utilized in the paper. The authors mention data reduction, data displays and conclusions drawing/verification as the three activities that constitute the data analysis, seen in Figure 6. After data collection, the following step was data reduction, which consisted of choosing, simplifying and summarizing the gathered data. The qualitative data was interview transcripts. Even though data was reduced in this step, it was still saved until the end of the study in case any information needed to be traced back. Data displays were about systematically summarizing the data. This was done by coding to separate and present the collected interview data. The coding did not only facilitate the presentation of information, but it also ensured diving deeper into selected categories that were relevant to the study. The conclusion drawing/verification process was simplified by the systematic presentation of the data. The preceding steps, by analysing and interpreting the data, ensured drawing conclusions based on the findings as well as discussing implication and limitations in relation to the collected data.

Figure 6. The steps of qualitative data analysis inspired by Miles and Huberman (1994).

3.5 Research quality

Blomkvist and Hallin (2015) mention validity and reliability as two aspects to evaluate the rigour of a research.

The validity refers to the extent that a study is investigating the intended area of research. The results from the study should have a procedure that leads to an accurate and adequate demonstration of reality (Collis & Hussey 2009). In order to achieve high validity of a study, Gibbert et al. (2008) mention that the research should have an apparent chain of evidence. The validity also increases when utilizing data triangulation with various sources (Gibbert et al. 2008). In this study, the chain of evidence is illustrated in the research process. It is achieved by motivated delimitations, methodological choices, critically evaluating sources and reviewing the literature, and motivating how literature and results support the conclusions. Furthermore, the constructed research is based on data collected from different data sources and strategies, which ensures a broader and complementary view of the research problem. The different data gathering methods used in this study were the literature review, interviews and survey, which ensure having different angles to look at the same phenomenon.

Furthermore, the study went through peer-reviews by students as the report developed. This is something that increases the validity since opinions on the thesis are provided from an objective standpoint (Blomkvist and Hallin, 2015).

3.6 Research ethics

The four principles according to Swedish Research Council were followed as a guideline in the study. 1. Information requirement, 2. Consent requirement, 3. Confidentiality requirement and 4. Good use requirement. (Vetenskapsrådet, 2002).

1. The first principle involves describing the purpose of the study to the participants. This happened two times before the interviews. First when contacting them but also right before the interview. This means that the participants were well aware of the nature of the study and also given information about how the shared information would be handled. The participants of the survey were given a short introduction of the subject with pros and cons about AI in healthcare. This ensured that all respondents were given knowledge about the subject before answering the survey.

2. By providing purpose and information about the interview at first contact it ensured that interviewees had all the necessary information to decide whether to participate in the interview or not. The survey participants were not obliged to answer and did it voluntarily. Also, all interviewees were asked permission to be recorded and were also informed on how the interview material would be used.

3. Confidentiality is upheld in regards to the wishes of the interviewed person. In this study, the shared information from the doctors and politicians were handled with integrity and confidentiality. The interviews with the researchers and professors were handled in consent with the participants as the option to be anonymous has been presented to all. Also when using quotes and referencing with names, the participants were asked for consent beforehand.

4. Results

This section is structured based on the three pillars in the conceptual framework: AI in healthcare, ethics and laws & regulations. For each theme, corresponding findings from the interview- and survey data are presented. The emergency department permeates the whole chapter and will not be described individually.

4.1 AI in healthcare

In the emergency department, the challenges are said to be varying patient assessments as well as lack of time and staff. Patients describe their symptoms in different ways and the symptoms themselves are also shown in different ways. The lack of time and personnel is not only due to covid-19, but there are generally more people seeking care and the population is also getting older. According to the interviewed doctors, the most common reasons why people seek aid at the emergency department are fractures, abdominal pain, testicular problems, chest pain, breathing difficulties and dizziness. The impression of AI in healthcare is positive among the interviewed researchers, professors, doctors and politicians in the thesis. Professor Boman highlights that hundreds of thousands of people visit emergency departments every year and if AI at least can improve the care for one percent, it should be implemented because that still has a great impact. AI has the potential to relieve medical practitioners from the heavy administrative tasks by automating processes. Researcher Karlsson emphasizes on correctly using AI to reach a stage where people have the time to reflect on their organisation and what they do best. Technical development is said to be slow-going, but as the basic obstacles are overcome it will go very fast as the potentials are enormous. The following are the perspectives of some of the interviewees on AI in healthcare:

“ I think it's absolutely positive, it's something that is going to happen. And it's going to happen relatively fast ” (Professor Azizpour)

“ [...]artificial intelligence will completely revolutionize healthcare. Once we overcome the basic obstacles it will go very fast and the potential is enormous, a paradigm shift in healthcare.”

(Professor Funk)

“ [...] To be able to get to the correct diagnosis faster, I think that AI can have a very large positive impact ” (Politician 2)

“ There are benefits depending on how you implement it. Maybe as some type of triaging that can screen out those who are critical and those who are less urgent in the emergency room ”

(Doctor 1)

“ The conclusion was that AI, the program itself, was so sensitive that it with the control group could find fractures that the human eye can not do ” (Doctor 2)

The implementation of AI in healthcare is a scary thought for many because of the risks it brings. However, Researcher Karlsson believes the potential benefits are worth taking the risk for:

“I think the benefits are so strong that it is worth the risk, so to speak, that AI is sometimes used incorrectly” (Researcher Karlsson)

4.1.1. Current practice

According to Politician 1, today’s systems are from the 20th century and are outdated, hence a lot of resources are invested in healthcare. Swedish healthcare has very high quality but low efficiency and productivity in terms of queues and availability. Politician 1 means that after you enter the system, you get great care. Doctors and other healthcare professionals spend much of their working time logging in and out of various systems as well as doing administrative tasks. 140 hours per month is said to be needed in order to keep up with all the medical advances. The importance of being updated to new knowledge is highlighted from the following quote:

“[...] Anyone who is very experienced today, maybe went to education in the 80's and the medical development goes so fast ” (Politician 1)

The region Politician 1 represents is about to import a whole new healthcare information system, however it has been postponed because of legal issues. The politician mentions the current situation with separate systems within the region and how all of them can be replaced by a single system. The new system is intended to act as decision support for all types of healthcare staff as everything will be stored in one same system which will allow everyone to be updated on new knowledge.

Professor Azizpour also points out that there is a shortage of specialists in almost all fields of medicine. This results in, among other things, long queues to see specialists. He refers to various published papers and describes the possibilities of AI today to help with first filtering the patients or help the specialist to take less time on a new patient.

In current practice in the medical field golden standards are often used when treating patients, illustrated in Figure 7. If there is a way to make 70 % of all patients well then the hospital or the government decides that this method should be used for all patients with these symptoms. If a new medicine or treatment is found, which ensures 74 % of all patients get well. Then this will be the new standard, changing from the other method. The problem with this, as Professor Funk mentions, is that there is a high risk that 26% of the patients don’t get the treatment best suited for them. No personalization is made to ensure all patients get the most efficient treatment. He explains that aspects such as symptoms, diet, genes and gender can be indicators which treatment is the most efficient. Thus, finding out who gets well from treatment A and who gets well from treatment B has the potential to give all patients a personalized treatment most efficient for them. Professor Funk mentions that he has talked with practitioners and that they don’t dare to go outside the recommended procedures/golden standard, since, if something goes wrong, when diverting from them, they may end up in trouble which will not happen if they follow golden standard. This means that practitioners are knowingly treating a patient in a non-optimal way. He sees AI as a future powerful tool to ensure every patient gets a personalized treatment most optimal for them in the future.

In the survey result, seen in Figure 8, 102 people are asked if they ever received a clear misdiagnosis or missed diagnosis by a doctor. 27,5 % answered “yes” and 10,5 % answered “maybe”. This means that approximately 40 %, four in ten, of the asked group are either unsure or have not received correct medical care.

Figure 8. Presentation of the participant's experience regarding misdiagnosis and missed diagnosis.

Figure 9. Illustration of survey participants’ experience at emergency departments.

The next question answered by the participants in the survey is what their worst experience has been at the emergency department. This is a multiple-choice question that entails that more than one answer can be given. Furthermore, this question is not locked, which means that participants that have not had a bad experience, or for any other reason, are not obliged to answer. The results in Figure 10 indicate that long waiting time is the number one bad experience followed by misdiagnosis and bad conduct towards the patient.

Doctor 3 explains that one of the biggest challenges in the twenty-four-seven emergency department is the high patient flow, which can shift day by day. In order to handle the high patient flow, the most vulnerable patients are taken care of first. In cases where reinforcement is needed, colleagues can be sent down since there is an on-call staff. However, the staffing is not always sufficient due to the high workload. Doctor 3 believes that with an increased patient flow comes a greater risk when making assessments. The doctor also mentions issues with administrative tasks that burden the practitioners that perhaps AI can help with:

“Take this example with a heart attack. If you come in with a heart attack, then you should fix a heart attack with balloon angioplasty. Then they have to go to a specialist, a cardiologist who does balloon angioplasty like this. Then you must contact this person and you must also contact

nurses. You must at the same time call and prepare a place for the patient in one ward because when the patient is finished he or she should not come back to the emergency room but move on to another ward. Such a flow and system, AI may be able to help simplify and improve “ (Doctor

3)

Doctor 3 mentions that a fast lane that handles this type of inquiries is desired and perhaps can be taken care of with the help of an AI. Right now, the staff makes lots of calls back and forth to get things done.

4.1.2 Decision support

All of the interviewees share a similar opinion that AI should be used as support to the medical practitioners and not make decisions independently, at least in the near future. Politician 2 explains how a digital doctor (AI) possesses the ability to handle thousands of journals and detect rare diseases compared to a doctor that encounters such diseases only once or twice in his/her whole career. However, Politician 2 mentions how healthcare personnel will not be replaced by AI since healthcare is not only based on hard data, but also soft. Something that the AI cannot understand with the algorithms today. Hence an interplay between the soft and hard values can result in more efficient care, which is supported by the quote below:

“[...] I believe that the interplay between the soft values that a natural person can add together with the hard facts that can be found digitally, then we get a more efficient and better care”

Collaborative Agents

Figure 11. Collaborative Experience Sharing framework, inspired by Professor Funk’s presentation.

Professor Funk presented a picture similar to Figure 10 and stated that it is possible to use AI for collaborative experience sharing. He explains that medical knowledge doubles today in less than a year and for a human to keep track of all the latest medical results and findings, even within one's narrow area of expertise, has become impossible. Keeping up to date with all new medical knowledge is beyond human capability. This makes tools for information sharing essential to improving healthcare.

knowledge doubles today in less than a year1_{[...] even if you are an expert in your field, as a}

specialized medical doctor or researcher, you will not be able to acquire all relevant information you need in a lifetime” (Professor Funk)

Doctor 2 has been involved in a project where AI was tested in the cardiac emergency department at Danderyd. All patients with chest pain received a tablet containing an AI program that asked questions. Depending on the patients' answers, it replied by saying if they could go home or should be admitted. Doctor 2 elaborates that patient assessment is based on anamnesis, i.e. based on the questions asked. Here, the collaboration and experience sharing between agents, seen in Figure 11, presents promising opportunities since the most optimal questions can be shaped. This ensures that doctors and medical staff not only ask the right questions but also that no important question is left out. Doctor 2 also mentions that despite AI’s capability to ask questions by itself, it still lacks the qualities to see, feel and hear:

“The patient assessment itself is very much from the anamnesis, i.e. according to the questions you ask. And you can get a lot out of it. There can AI be used” (Doctor 2)

Solving Similar problems

Professor Funk mentions that a doctor who is about to treat a patient with certain symptoms can insert the symptom and patient profile including co-morbidity into the computer system and receive an overview of similar cases from the past. The system can then show in percentage what diagnoses previous patients have received and the treatments accordingly. It should be able to illustrate when the diagnosis was wrong as well as other details that may be utilized to treat other patients. It is of importance that the system is able to admit when it does not know, for example when patients seek medical care with symptoms that the AI has not encountered before. From the computer’s findings, it is still up to the doctor to analyze the output from the AI system and decide how to treat the patient. Professor Funk elaborates by presenting a scenario:

“[...] the system can also tell you, ‘Oh, I see the treatment used in your hospital patients get better after two weeks, they're back at work [...] In Switzerland in hospital X, they use a different

method and these patients recover fully after one week’. Clinicians suddenly can get access to information, very specific information, which maybe take years until it is easily accessible

outside the research team or hospital” (Professor Funk)

Professor Funk exemplifies the relevance of such a system today by describing that doctors, 20 years ago, could share information by discussing rare cases with each other on coffee breaks. 1_{Reference given by Professor Funk after the interview: “Densen, P. (2011). Challenges and} opportunities facing medical education. Transactions of the American Clinical and