Detection and prediction of lane-changes: A study to infer driver intent using support vector machine

(1)

Detection and prediction of lane-changes:

A study to infer driver intent using

support vector machine

STAFFAN BENGTSSON

(2)

(3)

Detection and prediction of lane-changes

A study to infer driver intent using support vector machine

Staffan Bengtsson

Master of Science Thesis

MMK 2012:08 MDA 429

KTH Industrial Engineering and Management

Machine Design

SE-100 44 STOCKHOLM

(4)

(5)

Master of Science Thesis MMK 2012:08 MDA 429

Detection and prediction of lane-changes:

A study to infer driver intent using support

vector machine

Staffan Bengtsson

Approved

2012-03-28

Examiner

Mats Hanson

Supervisor

Bengt Eriksson

Commissioner

Scania CV AB

Contact person

Joseph Ah-King

Abstract

The master thesis in this report has been made in cooperation between Scania CV AB in

Södertälje and the department of machine design at KTH in Stockholm. Scania CV AB is a

leading manufacturer of heavy trucks, buses and coaches, and industrial- and marine engines.

Increasing road safety demands for vehicles in development has resulted in a new generation of

safety systems, called Advanced Driver Assistance Systems (ADAS). These new systems do not

only try to mitigate the effects of a crash, they also try to prevent them from happening. To avoid

that ADAS will interfere during controlled maneuvers by a driver, it is necessary to determine

the intent and current actions of the driver from sensor measurements. This important

information will help to improve the decision-making for safety systems of when to engage in

assistive actions.

In this study, the possibilities to detect and predict lane-changes from patterns in sensor

measurements have been made using a truck. The main objectives were to decide how to

approach this type of problem without using turn signals, estimate the accuracies that can be

achieved and determine which sensors that are required to solve the task.

From the concept evaluation it was determined to use the pattern recognition technique support

vector machine (SVM) for the task. To train and test the algorithm real vehicle data was used,

recorded from a truck during motorway driving. From this data the algorithm was able to

correctly classify, with accuracy of 82%, between lane-keep and lane-change actions. The

algorithm could also quickly detect a lane-change before the first wheel crossed the lane

marking.

With the set-up of sensors available in the test vehicle, no significant patterns were found in

driver actions prior to a lane-change (the intent). Therefore, an alternative approach was

suggested that aimed to determine what the driver should prefer to do in a specific situation from

contextual data. This additional information helped to reduce the number of false positives with

8 percentage points when classifying between lane-change and lane-keep actions.

(6)

(7)

Examensarbete MMK 2012:08 MDA 429

Detektion och prediktion av filbyten:

Härledning av en förares avsikt med

support vector machine

Staffan Bengtsson

Godkänt

2012-03-28

Examinator

Mats Hanson

Handledare

Bengt Eriksson

Uppdragsgivare

Scania CV AB

Kontaktperson

Joseph Ah-King

Sammanfattning

Examensarbetet som behandlas i rapporten har utförts i samarbete med Scania CV AB i

Södertälje och institutionen för maskinkonstruktion på KTH i Stockholm. Scania CV AB är en

ledande tillverkare av tunga lastbilar, bussar samt industri- och marinmotorer.

Med allt tuffare krav på trafiksäkerhet för fordon under utveckling har en ny generationen av

säkerhetssystem växt fram, Advanced Driver Assistance Systems (ADAS). De här nya systemen

försöker inte bara att minska effekterna av en olycka utan även förhindra dess uppkomst. Det

ställer stora krav på att systemen enbart ingriper när det behövs och inte motarbetar föraren

under normal körning. Därför är det av intresse att undersöka hur en förares handlingar och

intentioner avspeglas i sensorvärden för att i slutändan kunna förbättra säkerhetssystemens

beslutsförmåga i olika situationer.

I projektet har en studie gjorts för att se om det finns tydliga mönster i sensorvärden utöver

blinkers, inför och under ett filbyte på motorväg med en lastbil. Syftet med studien var att

undersöka hur den här typen av problem kan lösas, estimera vilka noggrannheter som kan uppnås

samt ta reda på vilken uppsättning sensorer som behövs för att lösa problemet.

Utifrån den konceptevaluering som gjordes valdes metoden support vector machine (SVM) för

detta mönsterigenkänningsproblem. För att träna upp och testa algoritmen användes riktig

fordonsdata insamlad under motorvägskörning. Algoritmen kunde utifrån datat med 82%

noggrannhet korrekt detektera om föraren utförde en filhållnings- eller filbytesmanöver.

Algoritmen kunde också snabbt detektera en filbytesmanöver innan det första hjulet korsade

filmarkeringen.

Med tillgängliga sensorer i testfordonet kunde inga mönster hittas i förarrelaterade sensorvärden

inför ett filbyte som skulle avspegla dennes intentioner. Därför föreslogs en ny metod för att

istället avgöra vad föraren borde vilja göra givet en situation genom att titta på omvärldsdata.

Med hjälp av denna information kunde antalet felklassificeringar mellan filbytes- och

filhållningsmanövrar minskas med 8 procentenheter.

(8)

(9)

Nomenclature

Abbreviations

ABS Anti-locking Braking System

ACC Adaptive Cruise Control

ACT-R Adaptive Control of Thought - Rational ADAS Advanced Driver Assistance System

AEB Automatic Emergency Brake

CAN Controller Area Network

ESP Electronic Stability Program

GPS Global Positioning System

HMM Hidden Markov Model

LDW Lane Departure Warning

ROC Receiver Operating Characteristics

RVM Relevance Vector Machine

SVM Support Vector Machine

Glossary

False negative When a positive event is classified as negative. False positive When a negative event is classified as positive. True negative When a negative event is classified as negative. True positive When a positive event is classified as positive. Action The process, or state, of acting or being active.

Intent Having the mind, attention, or will concentrated on something, or at some end or purpose. Intention and intent are synonyms. Infer To derive something as a conclusion from facts or premises.

(10)

(11)

1 Introduction

This report is the result of a master thesis work at Scania CV AB in S¨odert¨alje and the de-partment of machine design at KTH (the Royal Institute of Technology) in Stockholm. The following introduction will provide the background, objectives, delimitations and method for this thesis.

1.1 Background

In heavy vehicles there are many safety systems that aims to mitigate the effects of an accident and to support the driver in performing a vast array of tasks. These are well-known systems such as airbags, Anti-lock Braking Systems (ABS) and Electronic Stability Program (ESP).

Increasing road safety demands for vehicles in development has resulted in a new generation of safety systems, called Advanced Driver Assistance Systems (ADAS). These new systems aims to prevent an accident by either alerting the driver or actively taking control of the vehicle, making ADAS act more like a co-driver system. Typical systems are; Blind Spot Detection (BSD) that warns the driver when a vehicle is in the blind spot of the side-view mirror, and Automatic Emergency Brake (AEB) that ensures emergency braking in critical situations.

With the development in this area there is one main problem to solve. How can these ADAS accurately make a decision of when to alert the driver or actively take control of the vehicle? Since ADAS should only activate when needed, not interfere when the driver correctly operates the vehicle, nor unnecessarily warn for irrelevant objects or situations. Misbehavior that may lead to annoyance and discomfort for the driver. For example, imagine that the driver intends to change lane. In this scenario it would be annoying if a Lane Departure Warning (LDW) system frequently misinterprets the situation and sends out a warning for what it beliefs is an unintended lane-change. Another example could be if a BSD-system unnecessarily sends out warnings for vehicles in the blind spot when the driver has no intents of changing lanes.

This creates the need of a system that with high accuracy can determine the intents and actions of drivers in heavy vehicles from sensor measurements. Information that will improve the decision-making of ADAS of when to engage in assistive actions, but also making them more adaptive when adjusting their outputs based on both the scenario and intents and actions of the driver.

1.2 Objectives

In heavy vehicles from Scania there are several sensors that either directly or indirectly measure driver behavior, the intents and actions. The purpose of this thesis is to investigate how these intents and actions manifest in sensor data, during and prior to a lane-change on motorways. Also, how they can be detected using a pattern recognition algorithm. This vital information about the driver would aid cooperating ADAS in their decision-making of when to engage in assistive actions regarding lane-changes.

Practically, the main objectives are therefore to determine the necessary sensors to infer driver actions and intents (prior and during lane-changes), and to find a suitable mathematical approach to solve this problem. To test the approach and estimate accuracies that can be achieved will sensor data from a Scania truck be used. At the end of the project present the selected method, required sensors and the estimated accuracies in a report.

(14)

1.3 Delimitations

For this thesis, only common scenarios (the reason behind a driver’s intent and actions to change lane) that occur on limited-access motorways including entrance and exit ramps will be handled. The drivers are assumed to not be distracted nor drowsy when operating the vehicle. They are also assumed to follow existing norms, laws and regulations, e.g. reverse driving is excluded since it is not allowed on motorways. The only exception is the usage of turn signals, since a research by Olsen et al. showed that turn signals were only used in 66% of all recorded naturalistic lane-changes [1]. The testing will also only use sensor data from commercially available systems in Scania trucks today. If necessary, additional sensors will be recommended for future work.

1.4 Method

The workflow for this thesis included six elements; literature study, concept evaluation, concept selection, collecting data, testing and analysis.

The first task was to conduct a literature study of related research to determine the different approaches available for this type of problem. The information gained formed the theoretical framework and how the work should progress. A concept evaluation was then made to determine the advantages and disadvantages for the available approaches when applied to the problem. In combination with this were common scenarios on motorways and required sensors researched. The concept selection then resulted in selecting the approach that was best adapted to the problem, sensors and scenarios.

To obtain vehicle and driver data for the development of the selected algorithm, sensor measurements were recorded in a Scania truck. Field trips were organized for this purpose using five different drivers. The data was recorded using a laptop with the software CANalyzer1

connected to the built-in CAN-network in the truck.

With the selected approach and recorded data, a test-system was built using Matlab2_.

Toolboxes combined with functions written by the author were used to construct and tune the pattern recognition algorithm. This was followed by testing and analysis to determine the accuracy of the solution by running off-line tests with collected data.

1_{CANalyzer version 7.2.38, Vector Informatik GmbH} 2

(15)

2 Theoretical framework

This chapter further describes the problem followed by the process of concept evaluation and concept selection to find a suitable algorithm for the implementation.

2.1 Defining driver intent and driver actions

The theory of reasoned action by Fishbein et al. suggests that a person’s actions (behavior) is determined by his or hers intent to perform that action [2]. In this theory, intent is viewed as the cognitive representation of the action and determined by three elements, the attitude toward the specific behavior, subjective norms and perceived behavioral control, see Figure 1.

Figure 1: The theory of reasoned action.

Since it is not possible to directly determine the mental processes described by Fishbein et al., it must instead be inferred at small timescales from actions resulting in changed sensor values. The goal is to find specific patterns in these measurements representing how the driver assess and prepares for a maneuver (the intent) and how the driver operates the vehicle to obtain this intent (the action). This is a continuous process that frequently updates given new situations and additional information. For example, the intent and action to change lane may be succeeded with a new intent to change lane again given the new circumstances. Often is the intent also changing multiple times before an resulting action due to changing scenarios.

Vehicle safety systems that incorporates information about the driver in its decision-making often only looks at driver actions, sometimes only values from a single sensor, trying to deter-mine when the system should suppress warnings. These are systems that aims to achieve high accuracies in their decision making but often have problems with high rates of false positives (situations when the systems warns when it should not). High rates that sometimes leads to the driver disabling the safety system out of annoyance.

By using an approach that both tries to detect the actual action but also predict it using multiple sensors, should be able to outperform existing solutions, both regarding accuracy and speed. An approach that also would be more robust when adopting a more holistic view using sensor information from the driver, vehicle and environment.

When using multiple sensor inputs are complicated dependencies introduced that are difficult to determine for expert configured rule-bases. Therefore it is desired to find another approach to solve these types of problems. There are techniques for this within the area of supervised learning in pattern recognition. These approaches can use samples of sensor data to train an algorithm to automatically create these rules. It is around these type of approaches the theoretical framework and thesis will discuss.

(16)

2.2 Introduction to pattern recognition

Pattern recognition is a branch within the area of artificial intelligence. It is used for inductive inference where predictions are based on observations. Or as Bezdek wrote, ”pattern recognition is a search for structure in data” [3]. The importance of pattern recognition have increased over the last few decades. Today is extraction of information from large databases required in most areas, it can be used for stock market analysis, machine vision, character recognition and labeling of driver maneuvers in vehicles.

Pattern recognition can be divided into different categories. There are syntactic and numeri-cal pattern recognition, the first one uses symbolic features allowing more complex relationships between features while the second one use standard numerical feature vectors of fixed dimension. Another subcategory involves generative and discriminative models. Generative models require an estimation of the prior probability P (a) and the conditional probability P (b|a), compared to discriminative models that only require an estimation of the conditional probability P (a|b), see Figure 2.

Figure 2: Generative versus discriminative models.

For numerical pattern recognition there are four important elements as described by Bezdek [4], • Process description: the overall description of which model to use, features to measure

and sensors to gather this data.

• Feature analysis: the part of extraction, preprocessing and selection of data to use. • Classifier design: how to classify, estimate, predict and control.

• Classification analysis: tendencies, validation and labeling of data.

These elements can be schematically described with Figure 3. The process description includes selecting how to model reality, which features to measure and which sensors to use for these measurements. The sensor measurements, the inputs, are then analyzed and preprocessed in the feature analysis block. This block assures that all adequate features are selected. This part is crucial because it helps to lower the amount of data to be processed in later stages. Analysis of the feature vectors also assure that the correct mathematical approach is chosen for the classifier design. Since there are no general optimal approaches for a set of data should each problem be treated uniquely. A feature analysis of Figure 4 could conclude that; due to the crisp separation of data could a non-probabilistic method be used for Figure 4a, linear and shell-clustering for Figure 4b and a fuzzy approach due to the fuzzy relation between data for Figure 4c [5].

The classifier design aims to tune the parameters of the algorithm for optimal classification. Finally, a classification analysis is performed with validation of given results. For successful pattern recognition are these four parts revisited until the system satisfies specified requirements.

(17)

Figure 3: Schematic view of the pattern recognition process. [6]

(a) Pattern type 1. (b) Pattern type 2 (c) Pattern type 3

Figure 4: Appearance for three different types of data sets.

The classification process can be designed by either using data with known class belongings or by learning the classes based on similarities in the data. Common for both methods is that it is first when the classifier is fully designed and tuned that it can be used for classifying new sets of data. These two methods are called supervised- and unsupervised learning. Supervised learning is the method of using data whose targeting classes are already known, a method referred to as classification in literature. Unsupervised learning, referred to as clustering, is used when inputs are of unknown classes and also when the number of classes is unknown.

Pattern recognition aims to find tendencies and structures in a set of data. In comparison, there is also pattern matching that aims to find strict patterns of reference in a given set of data. For example find the string A in text document B. Due to the stochastic variation in real-world situations and in sensor measurements strict pattern matching is not applicable for the problem presented in this thesis.

2.2.1 Mathematical description of pattern recognition

Pattern recognition aims to separate a given set of data X consisting of n feature vectors, x1, ..., xn, of real p-dimensional space Rp into a number of classes or clusters. In classical

analysis these form a partition of X, data with a strong degree of membership in one subset while weak against data in other subsets [7]. A set of data X is classified into m classes (or clusters) C1, ..., Cm such that the following conditions are fulfilled:

(18)

Ci6= ∅, i= 1, ..., m (1)

∪m_i=1Ci= X (2)

Ci∩ Cj = ∅, i 6= j and i, j = 1, ..., m (3)

Pattern recognition assert that similar feature vectors are contained in the same class Ci

while other much less similar feature vectors are contained in others. There are different meth-ods for computing this dissimilarity or similarity (also known as proximity measures) between classes. The definition of similarity is equivalent to one determining dissimilarity between two feature vectors [8]. With data points placed in topological space, the nearness between two objects is given by the dissimilarity, d(x, y) with x, y ∈ X. A common dissimilarity (or metric) is the Euclidean distance,

d(x, y) = v u u t p X j=1 (xj− yj)2=k x − y k (4)

The Euclidean distance is popular but the dissimilarity in Equation 5 is also common. d(x, y) =

p

X

j=1

(xj− yj)2 (5)

2.2.2 Algorithms for pattern recognition

There are several methods for pattern recognition. Since there is no optimal generic method, should the method be chosen depending on the task. Common techniques are described below and summarized in Table 1 at the end of this subsection.

Classification techniques are characterized by its usage of allocating known labels (or targets) to data points, so called supervised learning. From training samples a decision rule can be constructed that accurately assigns labels for feature vectors in new sets of data, e.g. Support Vector Machine (SVM) and perceptrons. A visualization of a typical linear decision rule between two separated sets of data can be seen in Figure 5. Other common classification approaches are decision trees and Relevance Vector Machine (RVM).

Figure 5: Visualization of a typical linear decision rule for classification.

Clustering techniques are used for sets of data with unknown labels and unknown number of labels, so called unsupervised learning. For example, in Figure 6 a set of data is separated into three clusters since the data is concentrated in these three areas. Some clustering techniques

(19)

search for local extremes using optimization to reduce a cost function. The local extremes then define optimal clusterings [8]. Hierarchical methods creates a hierarchy of clusters that are either merged (agglomerative) or divided (divisive) at each iteration. Another method is genetic clustering that involves creating initial populations of clusters that are allowed to evolve and improve.

Figure 6: Visualization of how clusters are assigned to data.

Regression algorithms are used when real-valued outputs are required compared to the discrete outputs for classification algorithms. They are mainly used for prediction (forecasting) with a predictive model being applied to observed data, see gray line in Figure 7. For additional values of x1 can predictive x2 values be determined using this model. Regression is also used

to find the strength of relationship between dependent and independent data. There are linear regression approaches, see Figure 7, and others based on for example SVM and perceptrons.

Figure 7: Visualization of linear regression for a set of data.

Sequenced labeling is often used in combination with classification or regression. This ap-proach assigns a categorical label or real-valued output to each member of a sequence of ob-served values. One example is speech recognition when the labeling aims to assign a label to each word in a sentence. Accuracy is achieved by making labels dependent on nearby labels in the sequence, finding the best set of labels. These methods are mostly based on probabilistic approaches, such as Markov models. Similar sequential techniques that aims to find real values are Kalman and particle filters.

(20)

Table 1: Common algorithms for pattern recognition.

Classification Clustering Regression Sequence labeling

Support vector machine Decision trees

Perceptrons

Relevance vector machine

Fuzzy clustering Hierarchical clustering Agglomerative clustering Genetic clustering Linear regression Perceptrons

Support vector machine

Hidden Markov models Kalman filters

Particle filters

2.2.3 Related research for inferring driver intent and actions

Early research within this area focused on motion-based approaches with kinematic or dynamic models to describe a vehicle and predict its state evolution (speed, position, heading). As research continued the importance of contextual information gained notice resulting in the behavior based techniques most commonly used today. Behavior based techniques relies on the fact that drivers and vehicles exhibit typical patterns in specific maneuvers. To obtain these patterns are more sensors used to gather data from the driver, vehicle and environment yielding a more holistic solution. [9] [10] [11]

One of the first approaches by Liu and Pentland used Hidden Markov Models (HMM) to classify patterns in driver actions. This method modeled human behavior with a set of dynamic models sequenced together with a Markov network of probabilistic transitions [12] [13]. The first versions developed did not have continuous recognition as would be required in a real vehicle, but they were able to distinguish 7 maneuvers using coupled HHM [11] [12] [13]. Work by Kuge et al. extended this work with continuous recognition [14]. Other research using HHM for continuous recognition was made by Berndt et al. for prediction of driver maneuvers [15].

In recent work, the binary classification approaches SVM and RVM has gained popularity over the previously more used HMM. Mandalia et al. used SVM to create a continuous classifier for lane-keep and lane-change actions [16] [17]. Mabuchi et al. used SVM to classify if a driver decides to stop at yellow light in traffic intersections [18]. Research by Morris et al. used a discriminative real-time RVM classifier to detect the intent to change lanes [19]. McCall et al. also used RVM to predict driver intents to change lanes using real-world data [20].

Salvucci used a ”mind-tracking architecture” for simulating driver intents using a cognitive model of driver behavior implemented in the architecture for Adaptive Control of Thought -Rational (ACT-R). This system compared simulations with off-line recordings of driver’s actual behavior to infer unobservable intents [9]. An approach by Lefevre et al. used a occupancy grid and Bayesian occupancy filter for estimating and predicting vehicles position [10]. Henning et al. used a ”knowledge discovery framework” to study behavioral and environmental indicators to predict lane-changes [21]. A driver behavior and aware brake assistance system by Trivedi et al. used a Bayesian classification framework [22].

2.3 Introduction to SVM, RVM and HMM

When combining the presented problem in this thesis with common methods used for this type of problem in related research, it was determined to continue with the study of SVM, RVM and HMM for the concept evaluation. These methods have proved to work well for finding patterns in sensor data. Some of the results obtained in related research are presented in Section 2.6.

Each of these three techniques will be briefly described in the following sections. In com-bination with information about available sensors and common scenarios on motorways their advantages and disadvantages are evaluated in the discussion for choice of algorithm for the implementation in Section 2.6.

(21)

2.3.1 Support vector machine

Support vector machine is a method using supervised learning for analyzing and recognizing patterns in data both for classification and regression. SVM emerged in mid 1990’s from the area of statistical learning theory developed by Vapnik in the late 1970’s. It can be viewed as a new way of training multilayer perceptrons (used in neural nets) by solving a quadratic problem [23]. A quadratic problem is an optimization problem of minimizing or maximizing a function subject to constraints on its variables. SVM are today widely used in many areas, for handwritten digit recognition [24], object recognition [25] and many others.

SVM finds a hyperplane or set of hyperplanes of Rp−1_{dimension in a R}p_{-dimensional space}

that separates two classes of interest. These hyperplanes may also be called decision surfaces, optimal separating hyperplanes or optimal margin hyperplanes [23]. A good classification is achieved when the functional margin to the nearest data points is large. This is because a larger margin lowers the generalization error of the classifier, see Figure 8.

Figure 8: A hyperplane separating two sets of data (dots and boxes).

Many problems may not be linearly separable in defined finite dimensional space. Therefore is a kernel function, K(x, xi), used to transform original feature space to a higher dimensional

space to find a separable linear hyperplane there. This yields the general solution,

f(x) =

k

X

i=1

ciK(x, xi) (6)

with xibeing the inputs (feature vectors), K the kernel and ci a set of parameters to be decided

by the inputs [16].

Mathematically can a linear SVM be described using a set of training data (xi, yi) for

i= 1, ..., n with yi ∈ {−1, 1} and xi ∈ Rp. With yi indicating the class the feature vector xi

belongs to. In other words, yi shows on which side of the decision rule the data point should

reside on. For a given set of data, there can be several hyperplanes that separates two classes of data, see Figure 9. The goal is to find a hyperplane that maximizes the margin between these two classes. The margin is defined as the sum of distances from the closest data points of both classes to the hyperplane. A larger margin is desired because it reduces the problem of overfitting. Overfitting occurs when the solution is too customized for the training data and not generalized for unknown test data.

(22)

Figure 9: Several hyperplanes separating two sets of data.

An arbitrary hyperplane that completely separates points of data into two classes is defined by the equation

f(x) ≡ w · x + b = 0 (7)

with w being the normal to the hyperplane, x a set of feature vectors and b the perpendicular distance from the origin to the plane. Given an initial hyperplane with the target classes yi = {1, −1}, the parameters w and b can be adjusted to create the following constraints,

xi· w + b ≥ +1 when yi = +1 (8)

xi· w + b ≤ −1 when yi = −1 (9)

In Figure 8 these constraints represents the two parallel bounding hyperplanes (dotted lines). All points of data on these hyperplanes are the support vectors, hence the name of the method [26]. The perpendicular distance between these two bounding hyperplanes is _||w||2 . Summarized, both Equation 8 and Equation 9 can be combined to one inequality,

yi(xi· w + b) − 1 ≥ 0 ∀i (10)

Research within this area has investigated different approaches for multi-class problems. Common approaches are to solve multiple one-versus-one problems or one-versus-rest. Methods such as directed acyclic graphs can combine several binary classifiers to determine one resulting class label. Other approaches combine many binary objective functions and then solves the resulting optimization problem. A comparative work found that for multi-class problems the one versus one method is optimal in terms of accuracy and computational cost [23].

2.3.2 Relevance vector machine

SVM has become a well-known method for pattern recognition and machine learning by ex-pressing predictions based on training data. However, SVM is limited because it makes point predictions rather than predictive distributions. RVM was formulated to address this problem by adding class membership probability to its predictions. [27]

RVM has been used for several different pattern recognition tasks, e.g. image analysis and facial expression recognition. The technique is based on a Bayesian formulation (degrees of plausibility given the information) using a prior (initial degree of belief) that results in a sparse representation. A representation that in a compact way describe features, similarly as the Fast Fourier Transform decomposes a signal into its building blocks.

(23)

The method is similar to SVM and can be viewed as a Bayesian extension of it, useful for both regression and classification. Like SVM, is RVM used for binary classification with the aim to predict the posterior probability of membership for a given input feature vector x. A label may then be allocated to the class with which it has the greatest likelihood of belonging to. [28]

The training of a RVM classifier consists of two steps. The first step is to collect training data with given targets (xi, yi) where y ∈ {−1, 1}, xi∈ Rp for i = 1, ..., n. The second step is to

find optimal parameters for the classifier given the training data. The process is as follows, for a given input vector x, RVM models the probability distribution of its label y using a logistic function. An useful function since it maps inputs with values from negative infinity to positive infinity to between 0 and 1,

σ(fRV M(xi)) =

1

1 + e−fRV M(xi) (11)

In Equation 11 the classifier function fRV M(x) is given by,

fRV M(x) = N

X

i=1

wiK(x, xi) (12)

with K(x, xi) being a kernel function and w are adjustable weights. Compared to SVM it is

not required for the kernel to satisfy Mercer’s condition. A condition that requires that the kernel is continuous, symmetric and have a positive definite gram matrix. The gram matrix is a matrix of all possible inner products a set of vectors. [27]

Based on the discrete Bernoulli distribution (only having two possible outcomes) the likeli-hood of label y given the adjustable weights w is expressed according to Equation 13.

P(y|w) =

N

Y

i=1

σ(fRV M(xi))yi(1 − σ(fRV M(xi)))1−yi (13)

The marginal likelihood P (y|α), where α denotes the prior on weight w, can not be obtained analytically resulting in the need of a iterative process. With α∗

i being the maximum a posteriori

(MAP) estimate of the parameter αi it is possible to estimate the MAP for the weights, wM AP.

This is obtained by maximizing the posterior distribution of the class labels given the input vectors. This is equivalent to maximizing the objective function J in Equation 14.

J(w1, w2, ..., wN) = n X i=1 logP(yi|wi) + n X i=1 logP(wi|α∗i) (14)

In this equation the first summation term is corresponding to the likelihood of the class labels and the second term to the prior on the parameters wi. In the resulting solution it is only

training data points with nonzero weights wi (called relevance vectors) that will contribute to

the decision function. [23] [27] 2.3.3 Hidden Markov models

A Hidden Markov Model is a statistical tool for modeling generative sequences developed in the 1960s by Baum et al. It is the Russian mathematician Andrei Markov that gave the approach its name. The technique has been applied to several tasks including signal processing, speech recognition and extraction of information from documents [29].

(24)

probabilistic transitions between states. Given a sequence of observations: red, red, green and blue it can be verified that the states Urn 2, Urn 2, Urn 1 and Urn 3 generated this sequence. The probability for this sequence would then be 0, 3 · 0, 3 · 0, 5 · 0, 2 = 0, 009.

Figure 10: Example of Markov process with three states and three outputs red, blue and green. To extend this model to be a HMM the new model allows all observations to be emitted at each state with a specific probability, see Figure 11. The main difference now is that it is not possible to exactly know which sequence that will generate a specific sequence of observations. The state sequence is hidden hence the name of the method. What can be done is to calculate the probability that the model will generate the sequence of observations and the most likely sequence of states to produce them. Therefore are the individual bias of internal states, transi-tion probabilities between states and set of initial probabilities the properties that characterize a HMM [30].

(25)

The compact mathematical notation for HMM is λ = (A, B, π) with the following elements: A: The state transition probabilities of state j following state i.

B: The emission probability of observing vk when at state j.

πt: The probability of being in state q at time t=0 (initial state).

O: The observation sequence of fixed length T, o1, ..., oT, at time t.

Q: The state sequence of fixed length T, q1, ..., qT, at time t.

The HMM includes two assumptions, the first one called a Markov assumption describes a model when the Markov property is assumed to hold. A Markov property, is a property of a state signal that contains all relevant past information compactly [31]. One example is a golf ball in flight, all that matters is the current position and velocity for its future states (independently how it was achieved), P (qt|qt−1). The second assumption states that the output (observation)

at time t is dependent only on the current state, i.e. independent of previous observations, P(ot|qt).

When implementing a HMM method there are three main problems to solve.

1. Given the model λ = (A, B, π) and the observation sequence O = O1, ..., OT, how is the

probability of the the sequence given the model, P (O|λ), computed efficiently?

2. Given the model λ = (A, B, π) and the observation sequence O = O1, ..., OT, how is the

optimal state sequence Q = q1, .., qN computed to explain the observations?.

3. How should the parameters of λ = (A, B, π) be tuned so that P (O|λ) is maximized? The first two problems can be solved analytically while the third problem is adjusted with model training.

2.4 Common scenarios on motorways

The benefits of the limitation (see Section 1.3) to only look at scenarios occurring on limited-access motorways including entrance and exit ramps are many. It limits the number of possible scenarios and they are fairly easy to recreate for own measurements for data collection. Another advantage is that only motor vehicles with allowed speeds of 40 km/h or higher are allowed thereby excluding pedestrians, mopeds, bicycles and vehicles for agriculture. There are also no intersections nor traffic lights which also decrease the number of scenarios. As mentioned earlier, in the scenarios described below are drivers assumed to follow existing laws and norms excluding the usage of turn signals.

Limited-access motorways have opposing lanes of traffic separated by a median strip. This median strip can be a traffic barrier, boulder or simply a strip of grass. In Europe there are more or less the same rules for these motorways with speed limits being the main difference. Maximum speed limits are often varying between 100 km/h and 130 km/h with the exception Germany that in general have no speed limits. With the research and collection of data for this thesis being made in Sweden are all results discussed in reference to right hand traffic. Since it also involves pattern recognition of driver intents and actions with heavy vehicles and a connected trailer, the maximum allowed speed for these vehicles is 80 km/h according to Swedish law [32].

(26)

middle or outer lane but this contradicts existing laws and regulations. For illustration are these scenarios represented using a motorway with four lanes but the scenarios are applicable to a motorway of arbitrary size.

(a) Scenario 1 (b) Scenario 2 (c) Scenario 3 (d) Scenario 4

(e) Scenario 5 (f) Scenario 6 (g) Scenario 7 (h) Scenario 8

(i) Scenario 9 (j) Scenario 10 (k) Scenario 11 (l) Scenario 12

(m) Scenario 13 (n) Scenario 14

Figure 12: Common scenarios occurring on a motorway. Explanation of each sub-figure are described below.

(27)

Explanations for each subfigure in Figure 12:

1. Ideal situation, the ego-vehicle driving in the inner lane with desired speed (80 km/h) and no surrounding traffic.

2. Situation with a leading vehicle in-front of the ego-vehicle and no other surrounding traffic. Depending on the leading vehicle’s relative speed the ego-driver will intend to either stay in current lane at desired speed or change lane to obtain desired speed.

3. Obstacle in-front of ego-vehicle and no surrounding traffic. The driver intends to change lane to maintain desired speed.

4. A leading vehicle in-front of the ego-vehicle and surrounding traffic. Depending on the relative speed of the leading and surrounding vehicles, the driver will intend to either stay in current lane (at optimal speed or lowered speed) or change lane to obtain optimal speed.

5. Obstacle in-front of ego-vehicle. Depending on amount of traffic, the driver will intend to either change lane to avoid the obstacle and maintain desired speed, or decelerate until full stop in current lane.

6. Driver intends to change lane and enter motorway.

7. No surrounding traffic and the driver intends to change lane to aid vehicle on entrance-ramp.

8. Driver intends to stay in current lane due to surrounding traffic, cannot aid the vehicle on the entrance-ramp.

9. Driver intends to stay in current lane since there is no intent to exit the motorway. 10. Driver intends to change lane to enter exit-ramp and leave motorway.

11. Driver intends to change lane to an inner lane for slower vehicles. This could be due to either additional lanes on the motorway, previous lane-change for a passing, prepare for a oncoming exit-ramp, or vehicles with a higher speed in the current lane.

12. Traffic congestion, driver intends to stay in current lane. Alternatively the lanes have different end paths.

13. Entrance ramp becomes a new lane on the motorway. Driver aims to change lane. 14. Current lane turns into a exit lane, driver aims to change lane.

Conclusions from these common scenarios shows that it is important when classifying driver intents and actions to determine the current scenario. Because it is clearly the scenario that determines any intent, successively determines any actions. These scenarios also verify that during motorway driving there are a limited amount of maneuvers to predict and detect. There are two lateral maneuvers (as handled by this thesis), lane-keep and lane-change, and two longitudinal maneuvers, keep velocity or change velocity.

With the assumption that drivers will always try to obtain the maximum allowed speed, the scenarios in Figure 12 can compactly be described using a single flowchart, see Appendix C. This flowchart shows the required decisions the driver have to make, but also necessary sensor data for an algorithm to estimate which maneuver the driver most likely will do in the near

(28)

2.5 Sensors

To classify a driver’s lateral intents and actions there are sensors to quantify behavior and scenarios into digital values. These digital values corresponds to the feature values used for pattern recognition as explained earlier. It is important to only use sensors that measure significant changes and avoid those that do not show enough predictability or varies randomly. This requires that the sensor output change adequately to changes in the object measured and that the sensor have limited effect on the actual measurements. For the purpose of obtaining vehicle and driver data there are in general no constraints. It could be from ego-vehicle sensors, external stationary sensors or from V2X communication (e.g. vehicle to vehicle).

Sensors used for this type of problem can be separated into three categories; driver behavior, vehicle dynamics and environmental data. Sensors for driver behavior display how the driver assess the current situation and prepares for oncoming lateral actions with head, eye and feet movements. These inputs also show how the driver operates the vehicle by changing inputs such as steering wheel and accelerator pedal. Sensors for vehicle dynamics gives information about accelerations, velocities and yaw rates for the vehicle. Sensors for environmental data gives information about surrounding traffic and position of ego-vehicle in relation to reference systems (e.g. lane markings or position using GPS).

In the two following subsections are first sensors used in related research summarized followed by a summary of available sensors in the truck that will be used for testing. A combination of these will later be used to select appropriate sensors for the implementation phase. After these subsections is also a discussion of how available sensors in the test vehicle in theory will affect the results for predicting driver intent and driver actions.

2.5.1 Sensors used in related research

For Salvucci’s mind tracking technique for predicting lane-changes were steering wheel angle, accelerator pedal position, future lane position (calculated from current heading) and time to leading vehicle used [9]. Pentland and Liu used longitudinal velocity, longitudinal acceleration, current gear and steering wheel angle for their HMM approach to detect lane changes [12]. McCall et al. used accelerator pedal position, brake pedal position, longitudinal acceleration, longitudinal velocity, steering wheel angle, yaw rate, lateral acceleration, road curvature, head-ing, lateral lane position, side-to-side and up-down head movements using RVM to detect intents to change lanes [20]. Doshi et al. tried to predict lane changes with RVM by using steering wheel angle, yaw rate, turn signals (and length of time activated), distance to leading vehicle, relative speed of leading vehicle, gap in time to leading vehicle, difference between current and desired speed, lane position and road curvature. They also used eye- and head-tracking to measure head and eye movements [19]. The SVM approach by Salvucci and Mandalia used only seven features for their lane-change prediction. Lane positions at different distances (calculated from current angle towards lane markings) at 0, 10, 20 and 30 meters ahead, angle towards lane mark-ings, lateral- and longitudinal acceleration [16]. Berndt et al. used accelerator pedal position, brake pedal position, longitudinal acceleration, steering wheel angle, steering wheel velocity, yaw rate, curvature and type of road [15]. Trivedi et al. used steering wheel angle, wheel speed, longitudinal acceleration, lateral acceleration, yaw rate, brake pedal pressure, accelerator pedal position, head movements, foot movements and distance to leading vehicle to detect intent to brake [22]. Mabuchi et al. used distance from intersection, velocity and acceleration to detect intent to stop at an intersection with yellow light [18].

Map data has mostly been applied to stationary safety systems at intersections. Maps that have been used for constraints on reachable points. However recent work have shown that information in these road network maps are important indicators to predict vehicle behaviors

(29)

[10]. To meet this increasing demand of using map data the ADAS Horizon project was initiated. This project uses the ADASIS protocol for standardizing the communication of map data to ADAS using CAN in vehicles [33].

The most recent studies has particularly shown the importance of measuring eye- (gaze) and head movements for predicting driver intents. Results by Henning et al. showed that a glance to the left mirror happened in 99% of all lane changes to the left and in 87% of the cases did a glance to the left mirror precede a turn signal [21]. Related research by Tijerina et al. found that the probability was between 65-92% that a driver glances at the mirror prior to a lane change and Olsen et al. concluded that these visual mirror searches last on average for 1,1 seconds [1] [34]. Research by Boer et al. showed that approximately three seconds prior to a lane-change did drivers exhibit a significantly different context scanning behavior with more frequent gazes to the rear view mirror at the expense of views at current lane. When the lane-change is initiated the gaze shifts from guiding features in the current lane to the destination lane. The drivers also increased gaze at surrounding vehicles (front and back) before and during the lane change for situation awareness [35]. Studies by Doshi et al. also showed that head motions were significant 4 to 1.5 seconds before a lane-change while lane deviation is more important 2 to 0 seconds prior to the actual maneuver. Their resulting classifier was able to predict lane-changes up to 3 seconds prior to actual maneuver due to these head movements [19].

Another but not as convenient approach for measuring intent could be using electroen-cephalography (EEG) for measuring electric responses in the brain. Studies by Haufe et al. showed that an driver assistance system using EEG to detect intent for emergency braking occurs 130 ms earlier than systems only relying on pedal responses. At 100 km/h driving speed this would yield a shortened braking distance of 3,66 meters [36]. The question whether this type of sensor to detect electric responses in the brain is implementable in commercial vehicles is another question.

In Table 2 the sensors used in related research are summarized and separated into the three categories; driver, vehicle dynamics and environmental data. The longitudinal, lateral and yaw axis are defined in Figure 13.

Yaw Axis

Longitudinal Axis

Lateral Axis

(30)

Table 2: Sensor data used for related research.

Category Sensor type

Vehicle dynamics Longitudinal and lateral velocity Longitudinal and lateral acceleration Yaw rate

Driver behavior Eye movements Head movements Feet movements Steering wheel angle Steering wheel velocity Accelerator pedal position Brake pedal position Turn signals

Gear EEG

Environmental data Lateral lane position (current and future) Distance to leading vehicle

Distance to following vehicles (current and adjacent lanes) Relative velocity of leading vehicle

Relative velocity of following vehicle Road curvature

Map data

2.5.2 Available sensors in the test vehicle

A majority of the sensors in the categories for car dynamics and environmental data used in related research also exist in truck that will be used for testing. There is a distance radar to track distances and velocities to leading vehicles in both current and adjacent lanes (part of ACC system) and an LDW camera to detect distances and angle towards lane markings. The only exception is that there are no sensors to track vehicles behind and to the side of the trucks. Looking at sensors for driver behavior, unfortunately there are no trackers for eye, head and feet movements in the test vehicle. It should be noted, that these trackers were only available in related research because specially equipped vehicles were used for the testing. They did not use commercially available vehicles as the objective is for this thesis. As expected are also EEG technologies not available. Otherwise all other signals are present including retarder lever position and reference speed for cruise control system. In Table 3 available systems and sensor data at Scania are summarized.

(31)

Table 3: Sensor data available in the test vehicle.

Category Sensor type

Car dynamics Longitudinal and lateral velocity Longitudinal and lateral acceleration Yaw rate

Driver behavior Steering wheel angle Accelerator pedal position Brake pedal position Turn signals

Cruise control reference speed Retarder position

Environmental data Distance to right and left lane marking Angle to right and left lane marking Distance to leading vehicle

Relative speed of leading vehicle

Distance to leading vehicles in adjacent lanes Relative speed of vehicles in adjacent lanes Curvature

2.5.3 Conclusions from related research and available sensors

After studying related research it is clear that the terms intent and predict has not been used consistently. Most of the studies have tried to detect initial stages of the lane-change maneuver (the action). Whereas only a few actually aimed to predict the lane-change (the intent) by looking at the patterns in driver behavior prior to the lane-change action.

The objective for this thesis is to investigate how a driver’s lateral intents and actions mani-fest in sensor data. In other words, construct a system that can detect driver related patterns for lane-keep and lane-change actions and to predict lane-changes by detecting preparatory driver actions prior to the lane-change (the intent).

Results from related research have shown that with the current set-up of sensors in the test vehicle there are good possibilities to test and classify between lane-keep and lane-change actions. A task that requires sensor data from all three sensor categories, environmental data, vehicle dynamics and driver behavior.

The possibilities to predict a lane-change by looking at drivers preparatory actions, prior to the actual lane change will be a more cumbersome task. For these actions, referred to as the lateral intent, has only significant patterns been found when using eye- or head-trackers. No significant patterns were found in how the driver operates the vehicle prior to a lane-change, for example, the driver positions the vehicle closer to the lane marking. Therefore, for this thesis and testing it will be required to infer the lateral intent using other cues. This implies that there will be less focus on the driver and more focus on the surroundings and current scenario. This introduces a natural uncertainty when decisions are being made on what the driver should prefer to do given the circumstances rather than using more direct cues of what the driver intends. This third approach will be referred to as inferred environmental intent.

Visualizing this trisect approach is made in Figure 14. In this approach and for the imple-mentation there will initially be three separate tasks. Beginning from the end is the first task to classify between lane-change and lane-keep actions. Preceding lateral actions is the task to detect the lateral intent, in other words predict the lane-change. Even though the absence of

(32)

patterns found in related research without using eye- and head-trackers it is still worth verifying those results in this work. There could be differences since heavy vehicles are used for this the-sis compared to standard vehicles in previous research. The final task is then to determine the possibilities and accuracies when predicting actions using only environmental data, the inferred environmental intent.

Figure 14: Dependency between inferred environmental intent, lateral intent and lateral actions. The suggested approach might also be useful in the quest to solve the problem of detect-ing drowsy or distracted drivers. Because if the inferred environmental intent contradicts the detected lateral actions it may imply distracted or drowsy actions. Secondly, by incorporating driver inputs such as steering wheel angle into the solution it may also be possible to exclude lane-changes without the appropriate steering wheel patterns.

2.6 Choice of algorithm for the implementation

Results from related research have proven the strength of binary classification algorithms for the tasks of detecting and predicting driver actions with adequate sensors.

The SVM algorithm has worked very well offering a robust and efficient approach for both lateral and longitudinal studies. For detecting lateral actions using available sensors in the test vehicle has true positive rates up to 97.9% been achieved at 5% false positive rates using real vehicle data. The approach also proved to be quick with true positive rate of 87% within 0.3 seconds from start of the maneuver. In a comparative work did an HMM approach only obtain true positive rates up to 80% at 5% false positive rates [17]. Other work using HMM with real-world data has achieved 71% and 74% accuracy for detecting left and right lane changes also using sensors available in the test vehicle [15]. An RVM approach obtained 80% accuracy but had problems with false positives [19]. This approach also used sensor information not available in the test truck.

The advantages with SVM are many. The method works well with small sets of training data and a large number of inputs as will be beneficial for this thesis [24]. SVM is also robust to the over-fitting problem by using a cost function for finding a large classifier margin. This approach also allows other mathematical approaches to be incorporated into the method. For example, it can be extended to include fuzzy sets thereby capturing real-world uncertainty with each point belonging to different classes to some degree [37].

There are some drawbacks using SVM. One issue is that the method does not incorporate temporal variations like HMM do. For example, a lane-change is not of fixed time length with longer lane-changes having smoother transitions in sensor-values. However, this problem can be solved by looking at windows of sample data. The continuous data stream is broken down to a window of fixed sample size. This window is then shifting for each addition of new data yielding a sample by sample (continuous) prediction. The best indication is then given by the label of the last window [16]. Another drawback is the difficult choice of kernel, kernel

(33)

parameters and regularization parameters. Using results by Mandalia et al. this problem can be simplified because simple kernels proved to be sufficient enough and appropriate parameters can be found using grid- or pattern search [16]. Compared to RVM there is also no probabilistic distribution for the output, only two discrete values, one for each class. SVM is also not optimal for large amounts of feature vectors due to the transformation process to obtain the dual form as described in Appendix A [38]. When using a fixed window of data this problem can be reduced. SVM also require more computational power than RVM, but with the most time-consuming operations being made off-line this should not affect the performance when implemented in a vehicle.

The advantages with RVM compared to SVM is that RVM have a reduced sensitivity to parameter settings, yields a probabilistic output and does not need a cost function as for SVM. The method can also sift through large inputs of data and still obtain sparse representation. This results in a less computationally intensive method that requires less relevance vectors compared to number of support vectors for SVM. However when it comes to results has SVM actually proved to be slightly superior to RVM [23].

Combining available sensors, tasks to solve and results from previous research it was decided to choose SVM for the implementation. Even though both HMM and RVM looks promising has SVM yielded higher accuracy in comparative work which is important to reduce the number of false positives. There is also more information and toolboxes available for SVM simplifying the testing and construction of the desired system. Summarized the reasons for selecting and continue with SVM are:

1. Method is suitable for binary classification problems. 2. Method works well with small sets of training data.

3. Method has yielded highest accuracy in comparative work with other methods. 4. Suitable for large sets of inputs.

5. Method have proven to quickly classify data.

6. There exist toolboxes for Matlab for fast prototyping.

Following this selection is a more extensive mathematical description of SVM explained in Appendix A.

(34)

3 Implementation

This chapter describes the implementation of the SVM algorithm for the off-line testing using recorded lane-keep and lane-change data. The first sections describe how sensor data was acquired followed by a feature analysis. This chapter also describes the process of tuning the SVM algorithm to obtain highest possible accuracy.

3.1 Aquiring sensor data

All sensors in a Scania truck are connected to the built-in controller area network (CAN). There are three buses (red, yellow and green) which these systems are distributed on with main control unit COO as connection node, see Figure 15.

Yellow Green Red External COO System System System System System

Figure 15: CAN-network and main control unit COO.

To acquire necessary sensor data for training of the algorithm, field trips were arranged using a Scania R440 truck and five different drivers, see Appendix B for vehicle specifications. A laptop with the software CANalyzer was connected to the external port to listen and log all CAN communication. Together with CAN data were videos of the scenario in-front of the vehicle recorded for all trips. The intent of these videos was to easily find the position of a specific maneuver in the CAN data.

When driving a semi-trailer truck on motorways, events of interest (lane-changes) rarely occurs. This is because the relatively low speed compared to other vehicles. To increase the probability of natural lane-changes it was decided to collect data using only the truck without trailer. This change allowed the drivers to drive at 90 km/h compared to 80 km/h when driving with a trailer. Descriptions of the five trips can be viewed in Table 4.

Table 4: Summary of field trips for recording sensor data. Trip Driver, Age Distance Duration Road, Route

1 Male, 29 26,4 km 25:53 E4, S¨odert¨alje to Telefonplan 2 Male, 29 11,8 km 08:08 E4, Telefonplan to Botkyrka

3 Male, 32 53,0 km 34:35 E4, Södertälje to Vagnhärad (with return) 4 Male, 50 26,2 km 18:11 E4, Södertälje to Telefonplan

5 Male, 42 26,5 km 23:34 E4, Södertälje to Vagnhärad

This resulted in 143,9 km and 110 minutes of effective motorway driving. It yielded in 7 lane-changes due to entrance ramps, 7 lane-changes due to exit ramps and 29 lane-changes during normal motorway driving.

(35)

3.1.1 Defining test and training data

To train and test the SVM algorithm it is required to have specific data for this purpose. From the large CAN logs were smaller subsets of data extracted defining lane-change and lane-keep samples for lateral actions; preparatory lane-change and lane-keep samples for lateral intent; and subsets defining scenarios prior to lane-change and lane-keep actions. These subsets are defined by using timestamps in the CAN logs. Each subset was also given a label for the classification process, -1 for lane-keep and +1 for lane-changes.

Looking at lateral actions it is required to define the lane-keep and lane-change maneuvers. A lane-change action is defined as the movement within the current lane when the ego-vehicle starts to steer towards the destination lane until the first wheel crosses the lane-marking. This is independent of the initial position in the lane before the maneuver. Given this definition the 43 lane-changes did result in an average lane-change of 2.3 seconds. The longest was 6.0 seconds and the shortest 0.6 seconds. Lane-keep data is defined as the data not resulting in a lane-change. This is the majority of all collected data and resulted in approximately 108 minutes of lane-keep driving.

For lane-changes, the end of preparatory actions is when the driver commence the actual lane-change. Determining the start of preparatory actions without access to eye- or head-trackers is not as trivial. Based on results from research using these head-trackers it was decided to use a fixed start-time, 5.0 seconds prior to a lane-change [19]. Similarly as for lateral actions are ”preparatory” actions not resulting in lane-changes selected for the lane-keep samples.

Defining data for inferred environmental intent is similar to how data was selected for lateral intent. For this task it is of interest to determine the scenario prior to a change or lane-keep action. Therefore are scenarios for lane-changes defined as the time between the start of a lane-change and 5 seconds prior it. Scenarios for lane-keep are selected as all other.

3.2 Analysis of sensor data

The sensor signals of interest, from Section 2.5.2, for each training and test sample of data were exported from CANalyzer to Matlab for analysis and processing before the classification. Depending on the origin of a sensor signal there are some important properties. Sensor data from ACC radar and LDW camera sensor are sometimes discontinuous, see example for LDW camera in Figure 16. This discontinuity occurs when these sensors for some reason lose tracking of their targets, e.g. ACC radar can not see any target in front of the ego-vehicle or the LDW camera can not distinguish any lane markings. When this happens these sensors sends out CAN messages as usual but with values defined by an upper-boundary. These values are therefore much larger than normal values making the signal resemble a discontinuity. Except for signals from these systems can all other sensor data be considered as continuous.

(36)

330 335 340 345 350 355 0 100 200 300 400 500 600 Time [s] D is tan ce to Le ft Lan e M ar k in g [c m ]

Distance to Left Lane Marking

(a) Distance to left lane marking with discontinuities.

330 335 340 345 350 355 −0.2 −0.1 0 0.1 0.2 0.3 Time [s] S te er in g W h ee l An gl e [r ad ]

Steering Wheel Angle

(b) Steering wheel angle without discontinuities.

Figure 16: Two signals showing the necessity to handle discontinuities.

When defining lane-keep and lane-change data for the training samples it is of importance to avoid these discontinuities. This is to guarantee that impracticable data will not affect the performance of the classifier. For a real implementation it will be necessary to handle these discontinuities on the run before data is sent to be classified. One approach is to introduce status flags for each sensor. This is to control at each new classification which sensor data is available. For a continuous classification process this implies that different decision surfaces will be required depending on the status flags. Thus, different decision surfaces will yield different accuracies. It is also possible to transform these status flags and think of them as inputs to the system, e.g. a discontinuity for the ACC system denotes that there is no vehicle in-front of the ego-vehicle.

Preceding the classification process all inputs are also scaled down to between -1 and 1 to simplify the calculations. An operation that will be necessary both during training and testing since it is required that the inputs have the same properties.

The signal for longitudinal acceleration with a sampling rate of 50 Hz had a lot of high frequency components (noise), components that were filtered out to highlight data of interest. The noise comes from vehicle motions, vibrations and the sensor itself. A Butterworth low-pass filter was added to dampen frequencies above the chosen cutoff frequency of 5 Hz. This introduced a delay for the filtered signal (of approximately 0.1 seconds) but the gain of having a smooth signal was considered more important in order to capture significant changes rather

(37)

than small instantaneous ones.

3.3 Enhancement of sensor data

The original signals presented in Section 2.5.2 can be combined and extended using expert knowledge to better and more compactly present the sensor data. Three new signals are the results from this transformation; current lane position, future lane position and difference from reference speed. Each of these will be described in the following subsections.

3.3.1 Current lane position

To calculate the current lane position are three signals from the LDW camera used; distance from left and right lane marking and lane width. The distances to lane markings are calculated from the center point of the truck. A schematic view of these signals is seen in Figure 17. In this figure is also the angle towards left lane marking showed. A signal required for calculating the future lane position being described in Section 3.3.2

αLANE

dRIGHT

dLEFT

w

Figure 17: Schematic view of the lane width, distance to right and left lane marking and angle to left lane marking.

If only one lane marking is visible it is not possible for the LDW camera to determine the lane width. For these situations it is required to use a representative value. From collected field trip data could this representative value be calculated to 363 centimeters for a normal motorway lane in Sweden. Using this information the current lane position can be calculated as the quotient of the distance from right lane marking and lane width,

posc=

dright

w (15)

with w being the measured or mean lane width and dright being the distance from right lane

marking. Depending on if only one or both lane markings are visible there are three cases for calculating the distance from right lane marking,

dright =     

dright if only right lane marking is visible

wmean− dlef t, if only left lane marking is visible

dright if both lane markings are visible

(16) with dlef t and dright denoting the distance from lane markings and wmeanbeing the mean lane