Accelerated Behavioural Adaptation through Targeted Training Programs : the Case of Highly Automated Driving

Accelerated Behavioural Adaptation through

Targeted Training Programs – the Case of Highly

Automated Driving

Martin A. H. Krampell

Master’s Thesis in Cognitive Science

Department of Computer and Information Science (IDA) at Linköping University

and the Swedish National Road and Transport Research Institute (VTI)

Supervisor: Magnus Hjälmdahl (VTI)

Examiner: Arne Jönsson (IDA)

ISRN: LIU-IDA/KOGVET-A–16/001–SE

Upphovsrätt

Detta dokument hålls tillgängligt på Internet – eller dess framtida ersättare – under 25 år från publiceringsdatum under förutsättning att inga extraordinära omständigheter uppstår. Tillgång till dokumentet innebär tillstånd för var och en att läsa, ladda ner, skriva ut enstaka kopior för enskilt bruk och att använda det oförändrat för ickekommersiell forskning och för undervisning. Överföring av upphovsrätten vid en senare tidpunkt kan inte upphäva detta tillstånd. All annan användning av dokumentet kräver upphovsmannens medgivande. För att garantera äktheten, säk-erheten och tillgängligheten finns lösningar av teknisk och administrativ art. Upphovsmannens ideella rätt innefattar rätt att bli nämnd som upphovsman i den omfattning som god sed kräver vid användning av dokumentet på ovan beskrivna sätt samt skydd mot att dokumentet ändras eller presenteras i sådan form eller i sådant sammanhang som är kränkande för upphovsmannens litterära eller konstnärliga anseende eller egenart. För ytterligare information om Linköping Uni-versity Electronic Press se förlagets hemsida http://www.ep.liu.se/.

Copyright

The publishers will keep this document online on the Internet – or its possible replacement – for a period of 25 years starting from the date of publication barring exceptional circumstances. The online availability of the document implies permanent permission for anyone to read, to download, or to print out single copies for his/hers own use and to use it unchanged for non-commercial re-search and educational purpose. Subsequent transfers of copyright cannot revoke this permission. All other uses of the document are conditional upon the consent of the copyright owner. The pub-lisher has taken technical and administrative measures to assure authenticity, security and accessi-bility. According to intellectual property law the author has the right to be mentioned when his/her work is accessed as described above and to be protected against infringement. For additional infor-mation about the Linköping University Electronic Press and its procedures for publication and for assurance of document integrity, please refer to its www home page: http://www.ep.liu.se/.

c

Abstract

A prominent issue in the field of automotive research is the apparent lack of consideration given to the potentially safety-critical differences between novice and experienced users of Advanced Driver Assistance System (ADAS). Conducting experiments with novices only often results in the generation of unrepresentative findings, as these new systems often come with a lengthy adaptation period following their introduction. Running experiments with experienced drivers, however, is difficult, as these are often few and far between, if they even exist. To alleviate this discrepancy, and to help researchers acquire participants more akin to experienced drivers, even before a system has been launched, the approach of AcceLerated Behavioural Adaptation through Targeted tRaining prOgramS (ALBATROS) is proposed. It aims at training drivers in the use of the system, ideally giving them a level of experience similar to experienced users of said systems.

A framework for the ALBATROS approach is presented, as is the development of a proof-of-concept training program following this approach. Likewise, a mock-up ADAS, that provides drivers with both longitudinal and lateral support of the vehicle, dubbed the Driver Assist (DA), is presented, for which the training program (the DATP) is developed. The current study presents an experiment designed to validate the efficacy of the DATP, and ultimately, the ALBATROS approach itself. The current study concludes that DATP-trained drivers display significantly improved understand-ing of the DA system followunderstand-ing trainunderstand-ing and are significantly more likely to retake control in critical situations, than are untrained drivers. Thus, the ALBATROS approach appears a viable approach in giving drivers a better understanding of an ADAS system. However, whether the DATP succeeded in creating drivers similar in experience and understanding to real experienced users of said sys-tem, and if so, exactly how similar, is still unknown. More research is needed, specifically, studies comparing experienced users with those having been trained with the ALBATROS approach.

Acknowledgements

I wish to extend my sincere thanks to my supervisor, Magnus Hjälmdahl, for introducing me to this exciting field, for his guidance, and for having the patience for my seemingly endless quick stops by his office.

I am indebted to Ignacio Solís, for going above and beyond in helping me with my analysis of the data, but also for the many coffee-breaks and intriguing conversations about the inner workings of the human mind.

I also wish to thank Björn Lidestam, Katja Kircher, and Björn Peters for their most timely assistance during my stay at VTI. Jonas Andersson Hultgren and Erik Olsson deserve much praise for their fine work developing the simulator software and scenarios required for this project, and have my humble thanks. I am also grateful to all my participants for their time and effort offered to this study.

I want to thank my opponent, Partik Johansson, for his excellent feedback and constructive cri-tique.

Last but not least, I wish to thank my colleagues at VTI, my friends at the University, and my family, for being there. You all know who you are.

Contents

2.2 A possible solution . . . 7 2.3 Driver training . . . 7 2.4 Transfer of training . . . 8 2.5 Training inputs . . . 9 2.5.1 Trainee characteristics . . . 9 2.5.2 Training design . . . 10 2.5.3 Environment . . . 10 2.6 Training outputs . . . 10 2.7 Instructional strategies . . . 11 2.7.1 Meta-cognitive approaches . . . 11 2.7.2 Road commentary . . . 12

2.8 Measuring training effectiveness . . . 12

3 The Driver Assist Training Program 15 3.1 Driver Assist . . . 15

3.2 Developing the training program . . . 15

3.3 Driver Assist training evaluation . . . 17

4 Method 19 4.1 Participants . . . 19

4.2 Design . . . 19

4.3 Procedure . . . 19

4.4 Stimuli and materials . . . 19

4.4.1 User manual . . . 19 4.4.2 Questionnaires . . . 20 4.4.3 Training program . . . 20 4.4.4 Evaluation test . . . 21 4.5 Apparatus . . . 21 4.6 Ethical considerations . . . 21 5 Results 22 5.1 Questionnaire data . . . 22 5.1.1 Attitudes towards DA . . . 22 5.1.2 Knowledge of DA . . . 22

5.1.3 Motivation to participate in training . . . 23

5.2 Quantitative evaluation data . . . 24

5.3 Qualitative evaluation data . . . 25

5.3.1 Correct understanding and generalisation of DA functioning . . . 26

5.3.2 Failure to perceive or interpret environmental cues . . . 26

5.3.3 General feeling of uncertainty . . . 27

6 Discussion 28 6.1 ADAS training program evaluation method . . . 28

6.2 The verdict for the DATP and the ALBATROS approach . . . 29

6.3 Experimental approach . . . 30

6.4 Implications . . . 31

7 Conclusions and future work 32 References 33 Appendices 39 Appendix A (Driver Assist User Manual) . . . 39

Appendix B (Requirement Specification) . . . 43

Appendix C (Experiment Protocol Training Group) . . . 46

Appendix D (Experiment Protocol Control Group) . . . 48

Appendix E (Training Content Specification) . . . 49

Appendix F (Questionnaire Training Group) . . . 57

List of Abbreviations

ACC . . . Adaptive Cruise Control

ADAS . . . Advanced Driver Assistance System

ALBATROS . . . AcceLerated Behavioural Adaptation through Targeted tRaining prOgramS BA . . . Behavioural Adaptation

DA . . . Driver Assist

DATP . . . Driver Assist Training Program HAD . . . Highly Automated Driving HP . . . Hazard Perception

HPT . . . Hazard Perception Test

JCTF . . . Joint Conceptual Theoretical Framework (of BA in response to ADAS) KSA . . . Knowledge, Skills, and Attitudes

LKA . . . Lane Keeping Assistant SA . . . Situation Awareness

SAGAT . . . Situation Awareness Global Assessment Technique TNA . . . Training Needs Analysis

1

Introduction

Fully automated vehicles are not yet a reality. Highly Automated Driving (HAD) and Advanced Driver Assistance Systems (ADAS) have in recent years seen a major proliferation, but despite their often promising functional descriptions, the human driver still constitutes the most critical compo-nent in the driving task (Banks & Stanton, 2016). There is still need to further our understanding of the interaction between human drivers and these new automated systems, that handle many, but far from all, types of driving situations (Merat et al, 2014). Many new issues arise as a result of changes to the road-vehicle-user system caused by the introduction of automation, and drivers are inevitably required to handle new types of driving situations (Saffarian, de Winter, & Happee, 2012).

The degree of automation in vehicles has rapidly gone from providing merely longitudinal support (e.g. ACC) to combining other functionalities, such as lane keeping assistance (LKA) – progressing the automation from level 1 to level 2. In some cases they might even constitute limited self-driving (level 3; see NHTSA, 2013). Yet, in the case of limited self-driving, the driver is still expected to remain available for occasional control, and needs to remain vigilant (Merat et al, 2014).

Drivers of highly automated vehicles, however, have been shown to attend less to the road, and be more inclined to perform non-driving related tasks, thus being less prepared to intervene should they need to (de Winter et al., 2014). It has been postulated that the failure to retake control is, among others, due to an underdeveloped system understanding, indeed a lack of experience with the functioning and limitations of the automated system (Saffarian, de Winter, & Happee, 2012). Human factors researchers need to ensure that drivers are able to comprehend the capabilities of these systems (Merat et al, 2014).

However, researchers are systematically limited in their choice of study participants; to the few drivers having had time to acquire some level of experience with the system in question, or to novice drivers with no previous experience with the system at all. Experienced drivers may not even exist at all, in the case of pre-launch validations, which leaves novices as the only remaining alternative to researchers. Using drivers with no previous experience with a HAD system might seem a reasonable alternative, however, many researchers have argued for the potentially safety-critical differences between novice and experienced users (e.g. Lai et al., 2010; Larsson, Kircher, & Hultgren, 2014; Piccinini et al., 2014; 2015). For research to achieve valid results and assessments, such a potential disparity should not go overlooked.

There might be a potential remedy available to researchers. Drivers could be trained through the use of a specifically targeted training program, to foster an accelerated behavioural adaptation towards the ADAS under study. Hypothetically, trained drivers would then exhibit attitudes and be-haviours similar to experienced users of these systems, mitigating the discrepancy between novice and experienced users of new system.

Education and training of drivers has already been proposed as a preventative strategy to minimise any adverse effects of introducing new ADAS (e.g. Saffarian, de Winter, & Happee, 2012), but could likewise be used in training drivers for this research purpose. Such an endeavour would potentially enable more valid research, and also help further our understanding of how real drivers learn and adapt to new ADAS.

1.1

Research questions

The main research questions addressed in the current study are as follows:

• Is it possible to foster an accelerated behavioural adaptation through the use of targeted training programs?

• How can the effects of this type of training be measured?

1.2

Purpose

The main purpose of the current study was to investigate the possibility of instigating and fostering AcceLerated Behavioural Adaptation through Targeted tRaining prOgramS (ALBATROS) for drivers with no previous experience with an ADAS system. This included the investigation of how to best target the underlying psychological components and processes of behavioural adaptation, how to best structure the training approach itself, as well as how to evaluate and validate training effec-tiveness. In line with the main purpose, the current study provides an exemplar training program for a selected contemporary ADAS to stand as a proof-of-concept for the ALBATROS approach, that could be empirically evaluated.

1.3

Research approach

The current study constitutes the third and final phase of a research collaboration project between VTI and the Cognitive Science Master’s program at Linköping University. The project, of which this thesis is the final deliverable, has been running for one and a half years and been conducted within the scope of the three university courses; 729A46 (16 credits), 729A64 (10 credits), and 729A80 (master thesis; 30 credits).

This research project assumed a top-down (exploratory) approach in the investigation of acceler-ated behavioural adaptation in combination with a bottom-up approach to the development of the training program. An abductive approach was also assumed, which dictates that the most likely conclusion is to be chosen when faced with an incomplete set of premisses. For the current study, the use of an abductive approach was critical, as the field (of accelerated behavioural adaptation) not only lacked empirical examples, but also lacked any sort of general theoretical framework. As-sertions made from this line of reasoning led to the specification of the ALBATROS approach, but were also applied and tested through the use of an empirical experiment.

1.4

Limitations

The experiment in the current study aimed at evaluating the efficacy of a proof-of-concept training program, but was limited in its overall scope. A larger and more diverse experiment would, for example, have been desirable. The selection of literature was also limited to the presented fields, and fields such as pedagogy, for example, are not included.

2

Background

The last 20 years has seen the successful introduction of many passive safety measures in road vehicles, such as airbags, crumple zones and anti-lock breaking systems. However, the margin of improvement for similar measures appears to be declining, and the trend has shifted from crash mitigation to crash prevention (Piccinini et al., 2012). Active safety measures are one of the more popular new developments in the automotive industry, often promising increased comfort in ad-dition to an increased safety. One such type of active safety measure is the category of Advanced Driver Assistance Systems (ADAS) (see Piccinini et al., 2012; Larsson, 2012).

Besides the ADAS designed mainly to increase safety (i.e. lane departure warning, collision avoid-ance systems), two additional ADAS categories can be specified: systems that assist drivers in lateral (e.g. Lane Keeping Assistant; LKA) or longitudinal control (e.g. Adaptive Cruise Control; ACC). The most recent trend in the industry are systems that provide a combination of both LKA and ACC (cf. de Winter et al., 2014). However, these systems still fall short of making the vehicle autonomous, often requiring the driver to maintain an equal level of vigilance and attention in the driving task, as manual driving would otherwise demand (cf. Larsson, 2012). Furthermore, these state-of-the-art systems often have a plethora of inherent limitations, and presupposes the driver’s ability to intervene in critical situations (Stanton, Young, & McCaulder, 1997).

The driver does indeed still constitute a critical component in the road-vehicle-user system, yet these new types of ADAS often offload little more than just physical workload, leaving the driver re-sponsible for overall system safety (Banks & Stanton, 2016). Contemporary research in the field of traffic psychology highlights the importance of acknowledging human variability in the deployment of new systems, arguing that drivers will adapt and change their behaviour over time, following changes to the road-vehicle-user system (Wege et al., 2014; Larsson, 2012).

This process is often referred to as Behavioural Adaptation (BA; e.g. Wege et al., 2014). BA was initially defined as the process that occurs after the introduction of changes to the road-vehicle-user system leading to consequences in behaviour unintended by the designers (OECD, 1990; Piccinini et al., 2014; Manser, Creaser, & Boyle, 2013; Rudin-Brown, & Jamson, 2013). Recently, how-ever, this definitions has by many researchers been altered to constitute all behavioural changes to driver behaviour following changes to the road-vehicle-user system, intention notwithstanding (e.g. Rudin-Brown & Parker, 2004; Wege et al., 2014). A fundamental requirement for this process to occur is that the driver perceives the change and has the ability, as well as the motivation, to act upon it (Wege et al. 2014).

However, the relatively straightforward nature of the BA definition betrays its underlying com-plexity, as it has proved difficult to both quantify or evaluate this process (cf. Manser, Creaser & Boyle, 2013). Several models accounting for BA and its underlying components have previously been proposed (e.g. Rudin-Brown & Parker, 2004), but no single accepted model seems to exist. As a result, Wege et al. (2014) presents the "Joint Conceptual Theoretical Framework (JCTF) of Behavioural Adaptation in Response to Advanced Driver Assistance Systems" following a detailed review of existing BA literature. This framework aims to combine the plurality of contributing fac-tors, and constitutes the most comprehensive framework of BA to date. The JCTF was therefore chosen to act as the model for BA in the current study (see Figure 1).

Among the constituent components, the mental model has been upheld as the key contributing factor by many researchers (Bellet et al., 2009; Beggiato & Krems, 2013; Huth et al., 2014; Larsson, 2012; Piccinini et al., 2012; Wege et al., 2014; Xiong et al., 2012). Furthermore, trust (Beggiato & Krems, 2013) as well as risk perception and situational awareness (Bellet et al., 2009) appear to be significant in the forming of BA. Wege et al. (2014) maintains that drivers continuously develop their mental model during the interaction with the ADAS, and only after an extensive learning period will the mental model of the system be stable enough to produce consistent driving behaviour.

Figure 2: Driver behavioural stability and behavioural adaptation following the introduction of a new ADAS to the road-vehicle-user system (adapted from Manser, Creaser & Boyle, 2013)

Indeed, the critical aspect for the process of BA is the time-frame in which it is assumed to work; yet, little is still known regarding this factor (Lai et al., 2010; Larsson, 2012; Wege et al., 2014; Xiong et al., 2012). However, BA has previously been characterised into three stages; immediate, short term, and long term (see Manser, Creaser & Boyle, 2013) based on the level of concurrent BA, and can be mapped with regard to the driver behavioural stability (cf. Wege et al., 2014) over time (see Figure 2).

Clearly, BA needs to be seen in the context of time, in addition to the critical contributing factors pertaining to its formation. Understanding the long term behavioural changes caused by the intro-duction of new ADAS is critical moving forward (cf. Lai et al., 2010). In this regard, research on ADAS is in need of catching up.

2.1

The issue at hand

Many researchers have voiced their concerns that differing levels of ADAS experience gives rise to potentially safety critical differences in behaviour (e.g. Lai et al., 2010; Larsson, Kircher, & Hult-gren, 2014; Piccinini et al., 2014; 2015). The difference is said to be especially prevalent comparing novices with experienced users of such systems. New users of an ADAS might be completely without a proper understanding of how the systems works, and without the ability to realise it (cf. Kruger & Dunning, 1999). Questionable new ADAS functionality coupled with the apparent importance of system understanding, highlights the rising issue of introducing new ADAS without consideration of the required learning period involved. The overarching safety issues arising from the lack of experience not only constitutes a potential problem for the safety of all road users, but represents an often insurmountable hurdle for researchers trying to study driver behaviour surrounding new ADAS.

Studying the effects of ADAS on novices is – relatively speaking – quite easy, and comprises the ma-jority of studies conducted on contemporary ADAS (cf. Lai et al., 2010). However, by evaluating new ADAS only with system novices, conclusions drawn regarding behaviours and safety will only really apply to how other novices will use that system, indeed before any BA could have occurred (Piccinini et al., 2015). Novices might simply not have encountered a sufficient number of haz-ardous situations, and therefore cannot anticipate hazards linked to the use of the system (Isler, Starkey, & Williamson, 2009) resulting in potentially unsafe behaviours.

Actually experienced system users, however, are often few and far between – if they even exist – as new systems are often limited to the most exclusive car makes and models. Researchers thus find themselves at an impasse. They can either choose to find a few qualified experiment participants, but at the expense of statistical power, or opt to use ordinary and inexperienced drivers and run the risk of achieving unrepresentative, albeit perhaps significant, results.

Drivers need time to experience the system in order to undergo BA, and so update their mental model of the changed road-vehicle-user system. Following BA, users might adopt an unsafe be-haviour and use the system in unintended ways, or, conversely, adaptation may lead to a safer behaviour compared to what studies on novices otherwise might have us believe (see Beggiato & Krems, 2013). Without the proper scientific measures, we simply cannot know for certain.

2.2

A possible solution

Is there a remedy to this issue? Driver education and research has shown that skills related to haz-ard perception (Horswill et al., 2013; Isler, Starkey & Shepphaz-ard, 2011), visual search (Isler, Starkey & Sheppard, 2011; Vlakveld et al., 2011) and situation awareness (Walker et al., 2009), to name a few, can be trained through targeted training interventions. Comparatively, training programs as short as 40 minutes have been shown to effectively improve higher order cognitive abilities (e.g. Yamani et al., 2016). Therefore, it may be possible to put novices through a relatively brief train-ing program, givtrain-ing them an artificially higher level of experience. A traintrain-ing program designed to equip novices with the necessary appreciation of the system may also reduce the measurable differ-ences between novices and experienced drivers. Such a training program should if possible focus on the underlying processes used in the appropriate tasks (Cuenen et al., 2015; Anderson, 1994), highlighting the possible pitfalls of system use, elucidating a variety of system limitations. Ideally, such a training program would also foster a natural accelerated BA towards the ADAS.

Instigating and fostering BA appears to hold the key to a successful implementation of an ADAS training program designed to reduce the measurable difference between novices and experienced users. However, it requires that the provided training program allows trainees to naturally adapt their own behaviour, rather than conform to some predetermined ideal of how experienced users should act. Indeed, a presumptive training program should display an objective picture of the functioning of the system, and let trainees themselves modify their behaviour in response to their updated mental model.

The idea of training novices in this fashion appears to be a somewhat novel conception. A call for investigation into this issue has been voiced by several researchers (e.g. Beggiato & Krems, 2013; Koustanaï et al., 2012; Piccinini et al., 2014; 2015; Lai et al., 2010; Larsson, 2012), but no scientific investigation of this potential solution has as of yet been conducted.

Giving novices a higher level of experience, and in so doing, fostering BA towards the system, can be seen as a learning problem. A cognitive perspective on learning implies a dynamic interaction between the learner and the environment (Vandenbosch & Higgins, 1996). The updating of the mental model, indeed the learning process, can further be aided through the use of tried and tested learning methods based in training development. This forms the basis for the AcceLerated Behavioural Adaptation through Targeted tRaining prOgramS (ALBATROS) approach.

2.3

Driver training

The field of Driver education and training can be used as the foundation to the development of an ADAS training program. In this regard, Hatakka et al., (2002) present a hierarchical framework for driver education, illustrating that the goals and motivations of the driver interact bilaterally with lower level skills such as vehicle control and traffic handling. The authors maintain that for driver training to be successful, the motivational levels have to be targeted as well as the lower, skill based ones. BA has also been argued to occur simultaneously on many levels of this functional hierarchy (e.g. Summala, 1997). To this end, the use of active training methods are proposed, targeting motivational factors as well as other higher order cognitive abilities such as hazard perception and risk assessment, in addition to the lower levels (Mayhew & Simpson, 2002; Lonero, 2008).

An area of specific interest to driver education is research on simulator-based training. Simulators are widely used as both training and research tools in many fields, there among the automotive one (see Carsten & Jamson, 2011; Pollatsek et al., 2011). Goode, Salmon, and Lenné (2013) review the field of simulator training, drawing upon evidence from the automotive, aviation, and medical training domains, to conclude that simulators offer effective training tools. Vlakveld et al., (2011) commends simulator use, as it allows trainees to experience situations otherwise not possible in real world driving (e.g. crashes and near crashes). The incorporation of simulator-based training therefore appears critical to the ALBATROS approach.

2.4

Transfer of training

Transfer of training has been identified as one of the most critical components to consider in the design and development of training (Bell & Kozlowski, 2008; Grohmann, Kauffeld, & Beller, 2014; Bhatti & Kaur, 2010; Cuenen et al., 2015). Transfer can be defined as the degree to which trainees generalise and maintain the knowledge, skills and attitudes gained from training (Baldwin & Ford, 1988; Salas & Cannon-Bowers, 2001; Vlakveld et al., 2011).

Figure 3: The chosen model for the transfer process, originally in (Baldwin & Ford, 1988)

Baldwin and Ford (1988) propose a model for the process of transfer (illustrated in Figure 3) which, since its inception, has been restated (Ford & Weissbein, 1997) and praised (Renata-Davids et al., 2014). This model was chosen as it represents one of the earliest models to fully capture all factors deemed important by contemporary training research (cf. Salas et al., 2006). The model highlights

three types of training inputs that influence learning and retention (training outputs), and further generalization and maintenance (conditions of transfer). According to Baldwin and Ford (1988), all of the training input factors have a direct effect on the learning and retention (indicated by full lines), whilst trainee characteristics and environmental factors only have a hypothesized relation to the conditions of transfer (indicated by dotted lines). The training design factors, and the training outputs, are hence of most importance to consider in achieving transfer of training, as they retain a direct relation to transfer.

Furthermore, transfer of training has many similarities to BA, and the conditions of transfer (gen-eralisation and maintenance; see Figure 3) are closely linked to the behavioural stability attained after long term BA (see Figure 2). Achieving this stability through the use of training is, therefore, not only conceivable, but one of the core concepts training research has concerned itself with for the last two decades; indeed the focus of the ALBATROS approach.

However, there is a distinct lack of empirical evidence for the efficacy and transferability of any training approach (cf. Moskaliuk, Bertram, & Cress, 2013; Lonero, 2008). This issue is not only limited to the automotive industry, as other fields such as construction equipment training (Dun-ston, Proctor, & Wang, 2014) and process control training (Kluge et al., 2009) suffer from a similar shortage of empirical evidence. The importance of basing the prospective ALBATROS approach on a scientifically proven foundation becomes all the more apparent, specifically regarding the training input factors specified in the model for the transfer process (see Figure 3).

2.5

Training inputs

2.5.1 Trainee characteristics

An important realisation for training development is that trainees have widely different learn-ing styles. For example, the concept of ’cognitive style’ (see Ridlearn-ing & Sadler-Smith, 1997) de-scribes how trainees differ in the way they structure and process information. According to Riding and Sadler-Smith’s (1997) model, trainees are located somewhere on a two-dimensional matrix of Wholist–Analytic and Verbaliser–Imager; either preferring to see the whole picture versus de-constructing the issue at hand, or receiving this information verbally versus seeing it represented visually, respectively. For any prospective training program to be truly inclusive, they argue, it must accommodate all different cognitive styles. A key recommendation provided by the authors is that such training should provide a balanced approach, one that combines appropriate elements for all type of cognitive style, thus remaining inclusive to all trainees.

Furthermore, motivation has been identified as a critical modulator of training effectiveness, and as a highly potent mediator to transfer (Bhatti & Kaur, 2010; Grohmann, Kauffeld, & Beller, 2014; Baldwin & Ford, 1988; Renata-Davids et al., 2014). Thus, Renata-Davids et al. (2014) propose that trainees be told about the motive of the training content, so as to establish proper report with trainees and improve motivation. It is imperative that trainees believe in the effectiveness of the training, as training outcomes have been correlated to the motivation to learn (Salas & Cannon-Bowers, 2001). Bhatti and Kaur (2010) argue that perceived content validity (i.e. believing in the content of training) affect both learning and transfer, and recommend that training designers test

for perceived content validity as part of training, as it could provide a good indication as to why training fails or succeeds.

Moreover, self-efficacy (the degree to which a person believes in their own ability to change them-selves or their surroundings) has been found to be instrumental for both learning and transfer (Baldwin & Ford, 1988; Gist, 1997; Bhatti & Kaur, 2010; Salas & Cannon-Bowers, 2001). Closely related to this concept is that of locus of control: the perception of causality between actions and any subsequent outcomes – effectively whether outcomes depend on any actions performed, or on luck or any other such external factor. Locus of control has been shown to greatly affect transfer of training, and is hypothesised to affect a trainees ability to change their own behaviour (Huang & Ford, 2012).

2.5.2 Training design

Training design concerns itself with the incorporation of learning principles and the use of es-tablished training methods (Bhatti & Kaur, 2010; Ford & Weissbein, 1997; Renata-Davids et al., 2014). Baldwin and Ford (1988) argue that transfer is facilitated when learning consists of the general rules and principles that underlie the to-be-trained task. Training should also provide var-ied opportunities of use, such as many different situations and contexts, so as to maximise transfer (Baldwin & Ford, 1988).

Ford and Weissbein (1997) propose the use of meta-cognitive and error learning approaches as potent and effective training design components. Training of meta-cognitive skills, they argue, improves learning due to the increase in self-efficacy. Accordingly, error learning is said to be effective at developing the driver’s mental model of the system, as erring in a task challenges assumptions and leads to the reshaping of the mental model. Error learning also gives trainees feedback on their performance, a critical component in learning (cf. Baldwin & Ford, 1988).

2.5.3 Environment

Environmental aspects have not been studied extensively, but some evidence exists as to their importance (cf. Baldwin & Ford, 1988). With regard to the achievement of transfer, Salas and Cannon-Bowers (2001) argue that delays between training and potential use of any learned skills or knowledge will result in a significant reduction in transfer. Trainees must be given ample op-portunities to test out their newly learned skills within a reasonable time-frame following training. Such opportunities are to be seen as part of the (training) environment.

2.6

Training outputs

Gist (1997) elucidates the process of learning in an extensive review of training design and learn-ing principles. She argues that trainees move from the cognitive realm of declarative knowledge (requiring high conscious processing) to the behavioural realm of procedural knowledge (requir-ing far less directed attention) when learn(requir-ing a new task. Most learn(requir-ing situations begin with the memorisation of declarative knowledge – pertaining to an explicit and verbal understanding of an

object or process; reciting how the system works, for example. When this knowledge has been proceduralised – pertaining to a more automated type of knowledge, where recall and retention require less conscious effort – knowing how a system functions is instinctual.

This distinction entails a major difference between knowing what to do and actually being able to do it. Knowing something for a fact does not necessarily imply a change in behaviour (Gist, 1997). Simply training drivers in the explicit functioning of a system is therefore not sufficient, as some form of proceduralisation appears needed for true learning and retention. There is, however, some evidence that the repeated use of declarative knowledge in a specific context will lead to procedu-ralisation (Clark & Estes, 1996) – in essence providing support for the age-old adage "learning by doing".

2.7

Instructional strategies

The field of training design has in the last few decades transitioned from viewing trainees as passive onlookers, to participants, playing a critical role in their own learning (Bell & Kozlowski, 2008). This distinction forms the basis for the concept of active learning. Bell and Kozlowski (2008) up-holds the engagement of meta-cognitive abilities as well as the constructive framing of errors (i.e. error learning), as being critical aspects of the active learning approach, echoing other training design researchers (e.g. Baldwin & Ford, 1988; Ford & Weissbein, 1997). Active learning has also been linked to improved transfer compared to more traditional learning approaches (Bell & Kozlowski, 2008). Furthermore, training programs that encourage exploration and an active de-velopment of the mental model are highly effective (Frese et al., 1988). Moreover, a multi-modal approach (i.e. using multiple modalities) has been shown to be especially effective for training (Butcher, 2006).

Two types of active learning approaches were identified in the broader training literature: meta-cognitive approaches and road commentary. Salas et al. (2006) call these types of learning ap-proaches instructional strategies.

2.7.1 Meta-cognitive approaches

Meta-cognition can be described as a person’s grasp and control of the cognitive process that allow for the active monitoring, evaluation, and regulation of other cognitive processes (see Kim, Park, & Baek, 2009). In learning contexts, this activity is often referred to as self-regulated learning, in which the trainee directs attention towards the learning task by virtue of their own goals and moti-vations (Gist, 1997). Meta-cognitive strategies thus empower trainees to take control of their own learning in a meaningful way (Kim, Park, & Baek, 2009). Furthermore, meta-cognitive strategies have been proven to be effective in different learning situations, for example in improving situa-tion awareness (Soliman & Mathna, 2009), self-assessment (Kruger & Dunning, 1999), or social problem-solving (Kim, Park, & Baek, 2009).

Applying a meta-cognitive learning approach is especially important for the use of error learning. Failure (i.e. an error) is often more attributionally ambiguous compared to success, and so easier to attribute to external factors (Kruger & Dunning, 1999; Huang & Ford, 2012). Being able to reflect

on your own actions becomes all the more important in realising the causality of actions in error learning (i.e. self-efficacy and locus of control).

Kim, Park, and Baek (2009) provide a few concrete examples of proven meta-cognitive strategies, of which two were chosen based on their suitability.

• Modeling is a self-monitoring strategy where trainees observe others being engaged in the to-be-trained task, which highlights how others are thinking and acting in these situations. • Thinking aloud is a verbal expression strategy intended to highlight otherwise inaccessible

or ignored mental processes to trainees, and so help with self-monitoring and self-evaluation.

2.7.2 Road commentary

Closely related to meta-cognitive approaches is the concept of road commentary. Road commen-tary is often used in conjunction with pre-recorded material such as video recordings of real or simulator-based driving situations. Basic psychological research suggests that actually performing a task carries no significant benefits over simply watching someone else perform it (Abrahamse & Noordzij, 2011). However, Giannini et al. (2013) argues that videos alone are not sufficient as training tools, and require additional accompanying verbal descriptions (i.e. road commentary) to yield a sufficiently high informative value.

Two types of road commentary were identified in the driver training literature:

• Proactive listening requires trainees to actively scan for relevant objects or happenings, whilst expert commentary guides the attention and provides immediate feedback for the ongoing task (see Castro et al., 2016).

• Self-commentary requires trainees to continuously scan for relevant objects or happenings, whilst simultaneously verbally communicating any such perceptions to the training supervi-sor (see Isler, Starkey, & Williamson, 2009).

Comparing the instructional strategy of proactive listening to modeling, and likewise self-commentary to thinking aloud, it appears they have many similarities. One might even argue that they target the same underlying learning principles, but are only described in different terms in different do-mains. Regardless, they are highly unifiable in the current training context, and are regarded as synonymous for the current purposes of training development.

2.8

Measuring training effectiveness

An essential part of training (program) development is the generation of performance metrics (see section 3.2). Because the mental model has been quoted as the most critical component to training development as well as to the process of BA, the mental model could also hold the key to training evaluation. However, there does not appear to exist any methods to directly measure the mental model.

Correlated measures to the mental model are instead prevalent in traffic research. One such po-tential correlate of the mental model is the psychological construct of Situation Awareness (SA), which can act as a window to the mind of the driver. SA can be defined as the perception and com-prehension of elements (level 1 & 2, respectively) and the projection of their future states (level 3) in the environment (Endsley, 1995; Matthews et al., 2001; Vlakveld, 2014). SA has been proven to largely depend on the mental model and be developed with increased experience (Scialfa et al., 2013), making it suitable for the present purposes. Furthermore, SA has been identified as a critical component to driving performance and decision making (de Winter et al., 2014; Endsley, 1995; Jackson, Chapman, & Crundall, 2009; Matthews et al., 2001; Young, Salmon, & Cornelissen, 2013). Some researchers argue that SA represents a key element in many cognitive processes of the driver, their behaviour especially (Bailly, Bellet, & Goupil, 2003).

A common measure of SA amongst researchers is the psychological construct of Hazard Perception (HP), which to a large extent is based on SA (Bailly, Bellet, & Goupil, 2003). HP can be defined as the driver’s ability to detect dangerous situations (i.e. hazards) on the road ahead (Wetton, Hill, & Horswill, 2013; 2011; Malone & Brünken, 2015). Some have even explicitly called HP the situational awareness of dangerous situations (Jackson, Chapman, & Crundall, 2009; Vlakveld, 2014). A Hazard Perception test (HPT) often involves having participants observe a video sequence, and click a button or the screen itself when they identify objects that have the potential of creating a hazard (e.g. Wetton, Hill, & Horswill, 2013).

In order for drivers to be adept at HP, they must have accurate expectations (i.e. a developed mental model of the driving situation) of when and where hazards are likely to occur (Scialfa et al., 2011). For the current context of ADAS training, this includes when and where the limitations of the system might result in critical situations. It is believed that novice drivers have an impoverished mental model, and so perform worse on HPTs than do more experienced drivers (Scialfa et al., 2011). Indeed, HPTs in general are successful at discriminating between novice and experienced drivers (Malone & Brünken, 2015; Scialfa et al., 2011; Wetton, Hill, & Horswill, 2013) and are, for the most part, considered valid measures of driving expertise. HP is also the only skill that has been consistently correlated with crash risk (Wetton, Hill, & Horswill, 2013).

However, standard HPTs have been criticised for their ambiguity of only requiring participants to identify potential hazards, and not explain exactly how things might unfold (Jackson, Chapman, & Crundall. 2009). As such, HPTs, might not truly measure all three levels of SA, but arguably only level 1 (perception) and possibly level 2 (comprehension). These HPTs do not require participants to know the future states of the identified elements, but only identify and highlight what might constitute a hazard in the present context. As a measure of the mental model, therefore, it appears ill suited.

An alternative, albeit highly related, measure is the Situation Awareness Global Assessment Tech-nique (SAGAT) which has previously been used in both aviation and automotive research (Endsley, 1988; de Winter et al., 2014; Vogel et al., 2003; Wetton, Hill, & Horswill, 2013; Young, Salmon, & Cornelissen, 2013). This measure requires the participant to monitor a simulator or video record-ing, often with the task of remaining vigilant. At a specific time, unbeknownst to the participant, the recording is halted and the screen is blanked, at which point the participant is asked a series of questions regarding the situation they were just in and what might be about to happen. The answers to these questions are later analysed and scored by the experimenter, and compared to

what was actually happening in the situation, so as to get an objective measure of SA (see Endsley, 1988; Wetton, Hill, & Horswill, 2013).

Categorically speaking, it could be argued that SAGAT can be a form of HPT; however, the HPT has become synonymous with the participant-driven approach of pointing and highlighting potential hazards. SAGAT, in comparison to HPTs, replaces this criticised selection of hazards with a more specific measurement of the driver’s understanding of the road situation, as it forces drivers to explain any potential hazards and not merely point them out. SAGAT thus appears suitable for the present purposes, as it measures all three levels of SA, providing an objective measure of SA, and potentially, the mental model.

3

The Driver Assist Training Program

3.1

Driver Assist

In order to develop a training program there is need to specify what ADAS this training program should target. Inception of an altogether unique ADAS was favoured above drawing too much inspiration from existing systems, due to the risk of potential infringement on existing intellectual property of vehicle manufacturers. The Driver assist (DA) was therefore developed from the ground up. The DA is an assistive combined-features system, assisting drivers with both longitudinal (i.e. ACC) and lateral (i.e. LKA) control of the vehicle. The DA works for roads with clearly marked lanes, and between speeds of 40-120 km/h. The driver can set a desired speed and minimum time headway, but will otherwise have to retake control to perform more advanced actions, such as overtaking or changing lanes. DA also has a limited braking capacity (40 %) and the driver has to take over if any more forceful braking is required. The driver is technically in control of the vehicle, and thus responsible for the driving task, and has to remain vigilant and ready to regain control of the vehicle at all times.

As with any contemporary ADAS, the DA has its own set of inherent limitations. The DA uses a radar sensor and camera mounted in the top middle of the windshield to assess the roadway in front of the vehicle. Therefore, small objects (motorcycles, low trailers, etc.), stationary objects, or objects not directly in front of the vehicle, might not be recognised in time (if at all), and are all situations that drivers should look out for.

See Appendix A for a detailed description of the system, in the form of the User Manual (N.B. in Swedish) used to introduce the system to participants. A fully functional DA mockup was built on the simulator software at VTI to be used in the current study.

3.2

Developing the training program

The development of the so called Driver Assist Training Program (DATP) follows the development method presented in the training technology selection, design, and implementation model by Salmon et al. (2012). The model represents one of few coherent processes for training design and devel-opment to date (cf. Salmon et al., 2012) and is designed to guides training designers in the process of developing, designing, and evaluating training programs. The method is adapted in several key ways to better fit the practical constraints of DATP development, and are listed below.

• The entire process is streamlined. Cross-dependencies are reduced considerably between tasks, and there are also dedicated deliverables for applicable tasks. The original model fol-lows the common ADDIE (Analysis, Design, Development, Implementation, and Evaluation) process, however, an iterative style was instead favoured for certain segments of the process (cf. Visscher-Voerman & Gustafson, 2004).

• A new task is created by combining the definition of technology requirements and the iden-tification of organisational, task and trainee characteristics which results instead in the spec-ification of training program requirements.

• The Technology identification, evaluation and selection block has been merged into one task and runs parallel to the Training program definition block. This task feeds back to the require-ments specification as well as towards the design of training.

• A feedback loop is added between the definition of evaluation methodology and the devel-opment of training content, ensuring training and evaluation is developed in tandem.

Figure 4: The development process for the DATP, adapted from Salmon et al. (2012)

The DATP development process is illustrated in Figure 4. The full description of the DATP devel-opment process can be found in Krampell (2016). Below follows a condensed summary describing the different tasks in the process.

The training program definition block begins with a Training Needs Analysis (TNA), and represents one of the most critical components in training development (Lintern & Naikar, 1998; Renata-Davids et al., 2014; Salas & Cannon-Bowers, 2001). The TNA systematically identifies the learning needs for the to-be trained task (Baldwin & Ford, 1988; Gould et al., 2004; Salas et al., 2006; Salmon et al., 2012) and results in the specification of the Knowledge, Skills, and Attitudes (KSAs) needed to perform this task. This also acts as basis for the further specification of the training objectives and aims.

The block of training technology identification, evaluation and selection had previously been con-ducted (see Krampell, 2015) and had generated some requirements with regard to the available and suitable training technologies. This, together with the identified KSAs and learning objectives and aims, forms the basis for the requirement specification. The requirement specification

speci-fies critical, important, and optional requirements for the development and design of the training program, and can be found in Appendix B.

These requirements, in turn, leads to the specification of the training procedure (in the form of a training protocol, which can be found in Appendix C). The training protocol specifies the type of learning segments and instructional strategies to be used on a minute by minute basis in the training program. An alternate ’control group’ version of the training protocol (lacking the training segment) was also specified once the experimental design was known for the evaluation process, and can be found in Appendix D.

The development of content is based on the previously identified technology requirements, as well as the training protocol, and specified the training content. This content takes the form of a scenario description, specifying the situations, contexts, objects, and general happenings of all the scenarios for the training program (see section 4.4.3) and can be found in Appendix E.

The content specification then leads to the development of the evaluation process (see section 3.3) and the resulting specification is included in the specification of scenarios to be developed on the simulator software (see Appendix E). Once completed and implemented on the simulator, the pilot test is performed, and any changes are integrated into the final training program and evaluation used in the current study. The delivery and evaluation of the training program is the main focus of the current study, and is described in detail in the following chapters.

3.3

Driver Assist training evaluation

The development of evaluation not only constitutes a critical component in the overall training program development process, but also acts as the quality assurance and validation for the training program itself. As such, it not only has to represent a scientifically valid method, but also measure the correct underlying components of the learning task, to provide ample insight regarding the effectiveness of the training program.

To this end, a SAGAT style (see Endsley, 1988) test was developed (see section 2.8 for the theo-retical underpinnings of this test, or section 4.4.4 for a detailed description of the evaluation test itself). However, the development of such a test is no trivial matter, as the literature concerning the development is limited (Wetton, Hill, & Horswill, 2011), albeit some general principles for their development do exist. A critical component of both SAGAT and HPTs is the importance of anticipatory cues (Vlakveld, 2014; Wetton, Hill, & Horswill, 2011). The abilities of HP and SA are more about the detection and understanding of latent hazards, than the recognition of acute threats that require immediate reaction (Vlakveld, 2014). These so called anticipatory cues – sometimes called foreshadowing objects – facilitate an early prediction of possible events, a component criti-cal to these tests in discriminating between novice and experienced drivers (Jackson, Chapman, & Crundall, 2009; Malone & Brünken, 2015; Wetton, Hill, & Horswill, 2011).

Furthermore, transfer of training has been identified as critical to both the development of training, but also to the success of training (e.g. Baldwin & Ford, 1988; Bell & Kozlowski, 2008). For a training program to be successful, it has to instigate transfer. Therefore, it became apparent that transfer be the focus of training evaluation, or more specifically, that evaluation should test for situations and contexts separate from those trained to assess the degree of transfer. If the

training program is successful, trainees will perform better in situations that rely on a common set of underlying principles (in this case with regard to the use of DA). The training evaluation scenarios developed (see Appendix E) are of similar characteristics to those trained, but still different enough to remove the direct benefit of just having been trained.

The scenarios developed for the training evaluation test were evaluated by a group of subject mat-ter experts from VTI, to assure their general function, but most importantly, to assure that the anticipatory cues were sufficient. With the scenarios validated, and so the evaluation process, the experiment itself could commence.

4

Method

4.1

Participants

A total of 24 students (16 male, 8 female) aged 20-37 (M= 24.91, SD = 3.66) from Linköping Uni-versity participated in the current study, and were assigned to one of two experimental conditions: the control group or the training group. Gender-wise, participants were split equally between the two experimental conditions. All participants were required to have held a Swedish drivers licence for at least a year (M= 6.29 years, SD = 3.74) and to have normal or corrected eyesight and hear-ing. An additional inclusion criteria was that participants did not have any previous experience with an ADAS that provides both lateral and longitudinal support. As compensation, participants in the control group received one cinema coupon (approx. value 110 SEK), whereas participants in the training group, received two.

4.2

Design

The current study employed a between-group design. The independent variable was the training condition, and to some extent the initial attitudes towards learning of the DA system. The depen-dent variables were the self-reported scores in the evaluation, and to a lesser extent the responses to the post-training questionnaire for the training group.

4.3

Procedure

Participants were tested on a single session lasting approximately one hour for the control group, or approximately 2 hours for the training group. Upon arrival, participants were greeted and offered a selection of beverages from the coffee machine. They were then led to the simulator, where they were presented with a written consent form that included some general participatory information (see ethical considerations below). Both groups were given the Driver Assist user manual (see user manual below) and were instructed to read it carefully. After reading the manual, all participants filled out an initial questionnaire (see questionnaires below). After this point, the training group completed the training program (see training program below) and were again required to answer the attitudes part of the questionnaire. Both groups then took part in the evaluation (see evaluation below). Following evaluation, participants could ask questions about the experiment. This post-evaluation questions session also concluded the experiment.

4.4

Stimuli and materials

4.4.1 User manual

All participants were required to read the user manual following the introduction to the experiment. The user manual is a comprehensive introduction to the Driver Assist, and explains its intended use, controls, activation and deactivation, as well as its limitations. The user manual was inspired by

manuals and descriptions of similar systems, and intentionally presents information in a concise, albeit somewhat ominous, fashion. Although the initial information presented in the user manual constitutes a critical component to the training program itself, it was decided that all participants should receive a coherent introduction to DA, to ensure validity of the evaluation. The user manual does not include all use-cases or limitations, and as such should not render the training program redundant.

4.4.2 Questionnaires

All participants completed a short questionnaire inquiring to their general driving habits and to their attitudes towards the DA system and their own knowledge and learning of the same. The questionnaire answered by the training group had a few extra questions inquiring to their general attitudes and motivation of taking part in the training program (see Appendix F) as compared to the control group (see Appendix G). Participants in the training group were after the completion of training, but before evaluation, again required to complete the part of the questionnaire inquiring to their attitudes and knowledge of the DA system. This was so as to measure any potential changes in attitudes towards the DA system following completion of the training program.

4.4.3 Training program

The training program consisted of 14 scenarios designed to highlight different limitations of the DA system. Each scenario took about 3 minutes to complete, and depicted either highway, rural, or sub-urban driving situations. The scenery in the scenarios was heavily inspired by real world roads from around Linköping municipality (see Appendix E for a detailed specification of the scenarios). The training program consisted of two major segments: a video-based training segment and a simulator segment. The videos in the video-based segment were replays recorded and displayed on the same simulator set up as the simulator segment. The only difference in presentation between the two was the lack of active control in the video-based training segment (see Appendix C for a detailed view of the training program design).

The video-based training segment, in turn, consisted of a passive segment, where expert road com-mentary was provided by the researcher and to which participants were instructed to actively listen (i.e. continuously trying to model their own understanding to the provided commentary), as well as an active segment, where participants themselves had to think aloud to what was happening on screen. Seven scenarios constituted the video-based training, split four-three between the passive and active segments, respectively.

The simulator segment began with a practice period consisting of two practice scenarios in which participants were able to familiarise themselves to the different road conditions (highway and rural), to the simulator itself, and about halfway through both of the scenarios, to the activation and use of the DA system. Following the practice period, 5 scenarios were presented to drivers, and in which they were in complete control of the vehicle but were instructed to use the DA system as much as possible. Training was concluded with the completion of the simulator training.

4.4.4 Evaluation test

The evaluation test consisted of 12 scenarios delivered using a SAGAT style presentation (see End-sley, 1988; Vogel et al., 2003). Half of the scenarios were critical situations where the DA would be operating at or beyond its limitations, whilst the other half were situations similar to the critical ones, but within the operational limits of the DA. The criticality of each scenario was validated by a group of subject matter experts at VTI. Following this validation, two scenarios (numbered 10 and 12) were removed from the test as they did not meet the required standards for the test.

For each scenario, and right after each black-out (see section 2.8 for a SAGAT description), partic-ipants were asked: "how much do you want to take over control of the vehicle right now?" (on a scale from 1 to 10), and furthermore, to motivate their answer as thoroughly as possible. Answers were recorded to later be transcribed by the researcher. The ordering of evaluation test scenarios was predetermined and the same for each participant, although presented in a random order of critical and non-critical situations.

Before the evaluation test was initiated, participants received a short introduction to the evaluation, including a verbal explanation of what would be measured and how they were to act. Participants were to supervise the system in the scenarios presented, and at the point of the black-out convey their general inclination of wanting to intervene. The approach was illustrated with two practice scenarios – one critical and one non-critical – where the researcher indicated to the happenings on screen, and what might constitute appropriate answers following the black-outs.

4.5

Apparatus

The simulator used consisted of a Thrustmaster T300 RS steering wheel mounted on a normal desktop, connected to a PC running VTIsim (in-house VTI simulator software). Participants sat approximately 60 cm from a 27 inch LCD monitor with a resolution of 2560 x 1440 and with a refresh-rate of 60 Hz. The field of view was adjusted accordingly to appear as natural as possible from this viewing angle distance.

4.6

Ethical considerations

Participation in the study was voluntary and could be discontinued at any time during the exper-iment. Participants were informed that participation would be anonymous and that any collected data would be handled with complete confidentiality. All participants signed a consent form prior to the experiment. The experiment followed the ethical principles and guidelines regarding the use of training programs provided by the American Psychological Association (2002).

5

Results

5.1

Questionnaire data

5.1.1 Attitudes towards DA

The self-reported attitudes towards DA, including the driver’s likelihood of use given the opportu-nity, and their interest in DA in their next car purchase, did not see any notable change following training (see Figure 5). Shapiro-Wilk was used to confirm that the data was normally distributed. A notable difference was visible in the likelihood of use question between the control group (M= 3.00, SD= .95) and the training group (M = 3.75, SD = .97). An independent-samples t-test was conducted which indicated that the difference was not significant; t(22)= 1.915, p = .069. It can be postulated that the difference observed between the two groups was due to the different instruc-tions given to the groups, in that the training group would be receiving training, and would thus get the opportunity to learn more about the system. The training group, despite having received the same information about the DA as the training group at this point, but not promised any more opportunities to learn, were perhaps not equally optimistic of their own ability regarding the use of the system.

Figure 5: Mean self-reported scores for questionnaire questions pertaining to the drivers attitudes towards DA. Error bars represent standard error of mean.

5.1.2 Knowledge of DA

Regarding the data pertaining to the driver’s self-reported knowledge of DA, Shapiro-Wilk was used to check the parametricity of the data, and only a few of the variables were normally distributed. A non-parametric statistical approach was therefore used analyse the data.

Figure 6: Mean self-reported scores for questionnaire questions pertaining to the drivers knowledge towards DA. Error bars represent standard error of mean. *= p < .05, ** = p < .01

Wilcoxon Signed-Ranks test was used for the intra-group comparisons of the two conditions of pre- and post-training. The results indicate that drivers self-reported their general understanding of DA was significantly higher following training (Z= -2.879, p = .004). The drivers had signifi-cantly higher self-reported confidence in using DA on their own (Z= -2.209, p = .027). However, a significant difference for the appreciations of DA limitations was not found (Z = -1.890, p = .059) comparing the pre- and post-training conditions. A paired-samples t-test was additionally conducted comparing all the conditions, with very similar results.

The question regarding the understanding of DA limitations did not indicate training had had any effect on the understanding the limitations of the system. This could be explained by the general focus on limitations in the DA user manual, which might have conveyed a sufficient number of lim-itations for participants to report similar scores on their subjective appreciation of the limlim-itations, before and after training.

5.1.3 Motivation to participate in training

Bhatti and Kaur (2010) recommends testing for the participant’s motivation to participate in train-ing, quoting its importance to the potential success of training programs. Generally, the partici-pant’s overall self-reported interest and motivation in the training program appears high, including their motivation to participate in the training program (M= 4.33, SD = .65), their belief in the efficacy of the training program itself (M = 4.17, SD = .58), as well as their interest in the DA system (M= 4.25, SD = .62).

5.2

Quantitative evaluation data

Figure 7 shows the mean self-reported inclination to retake control for each scenario. One data point (a score of 10 belonging to the control group for Scenario 11) was removed from the data set, as the participant clearly stated that their inclination to retake control concerned the desire to manually overtake the vehicle in front, regardless of how the DA system would or would not have handled the situation.

Figure 7: Mean self-reported scores for each scenario in the evaluation test. Error bars represent standard error of mean. **= p < .01

The Shapiro-Wilk test was then used to test the parametricity of the data, which indicated that only two scenarios were normally distributed. The non-parametric Mann-Whitney U test was therefore used to measure the differences between the training group and the control group for each scenario. The analysis yielded but one significant difference (U= 28, p = .008) between the two groups for Scenario 7.

The variability in scores across the two experimental groups, and the relatively small sample size, are likely the reason why only one scenario showed a significant difference between the two groups.

The scenarios were therefore instead categorised by their criticality into two conditions, critical and non-critical scenarios, to check for general interactions between level of criticality and the level of training (see Figure 8). Scenarios 2-7 were by design critical and scenarios 1, 8-14 (scenarios 10 & 12 were since previously removed) were by design of much less criticality (see section 4.4.4 for

Figure 8: Comparing mean self-reported scores for inclination to retake control between critical and non-critical scenarios. Error bars represent standard error of mean. **= p < .01

a description of the evaluation test). The means of the two groups, the training and control group, were then compared based on this categorisation. A repeated measures 2x2 ANOVA with the level of criticality as the within-subjects variable and the experimental group as the between-subjects variable was used to look for main and interaction effects in the data. An interaction effect was found between both independent variables, showing higher scores for the trained group in only the critical scenarios; F(1)= 10.008, p = .005.

5.3

Qualitative evaluation data

After each scenario in the evaluation, participants were, in addition to their inclination to retake control, required to report a motivation to their answer (see section 4.4.4 for a detailed description of the evaluation test). These motivations ranged from one laconic sentence to several minutes worth of rich commentary. The median motivation length across all participants was three sen-tences, often including a description of the current situation as well as what the DA system would, or would not, do. These responses were transcribed by the researcher, and categorised by experi-mental group and scenario.

The thematic analysis process was then applied to the transcript, following the guidelines to quali-tative research provided by Creswell (2013). This approach elicited some general concepts evident in the data. These concepts were categorised into three themes, and have been labelled as: "Correct understanding and generalisation of DA functioning", "Failure to perceive or interpret environmen-tal cues", and "General feeling of uncertainty", and are described in detail below. Certainly, these