Test method development by use of SOM-GRNN

(1)

IN

DEGREE PROJECT

VEHICLE ENGINEERING,

SECOND CYCLE, 30 CREDITS

,

STOCKHOLM SWEDEN 2018

Test method development by

use of SOM-GRNN

YIHAO TANG

HUI ZHU

(2)

Abstract

The relationship between Objective Metrics (OM) and Subjective Assessments (SA) has been analyzed by people using different methods. This paper continues Gaspar Gil Gómez’s research over test method development by use of SOM-GRNN, aiming to find correlations between OM and SA. In this paper, CAE simulation is performed to analyze the relationship between OM and vehicle parameters. First impression test is refined and one more dataset has been added in order to populate SOM-GRNN map. This paper also conducts analysis over SOM and GRNN algorithms, and explores several possible applications using SOM-GRNN map. Finally the whole SOM-GRNN system is integrated and a User Interface is built in GUI for future research and application. The system can still be improved by populating database, refining SA evaluation method and optimizing SOM-GRNN training algorithms.

(3)

Sammanfattning

Relationen mellan målmetri (OM) och subjektiv bedömning (SA) har analyserats av per-soner som använder olika metoder. I detta dokument fortsätter Gaspar Gil Gómezs forskning kring testmetodutveckling med hjälp av SOM-GRNN, som syftar till att finna korrelationer mellan OM och SA. I detta papper utförs CAE-simulering för att analysera förhållandet mel-lan OM och fordonsparametrar. Första intryckstestet är raffinerat och ytterligare en dataset har lagts till för att fylla i SOM-GRNN-kartan. I detta dokument analyseras även SOM- och GRNN-algoritmer, och undersöker flera möjliga tillämpningar med hjälp av SOM-GRNN-kartan. Slutligen är hela SOM-GRNN-systemet integrerat och ett användargränssnitt är byggt i GUI för framtida forskning och tillämpning. Systemet kan fortfarande förbättras genom att fylla i databasen, förädla SA utvärderingsmetod och optimera SOM-GRNN träningsalgoritmer.

(4)

Acknowledgments

This master thesis has been performed at the Driving Dynamics department at Volvo Car Corporation, in collaboration with the Vehicle Dynamics division at the department of Aeronautical and Vehicle Engineering from KTH Royal Institute of Technology. The general structure of the main idea in this master thesis is built on the PhD thesis by Gaspar Gil Gómez that is also performed at the Driving Dynamics department at Volvo Car Corporation. Doctor Gaspar layed a solid theoretical foundation for building SOM + GRNN model, and he also proposed a general idea for experimentation to collect OM and SA sample data. This master thesis made a step forward based on Doctor Gaspar’s work by refining test plan and perform more database to generalize the model, unifying the whole SOM+GRNN model with further application and improving the model performance by analyzing model training algorithm.

We would also like to express our gratitudes to our manager and supervisors: Hans Bäckström, Egbert Bakker, Kenneth Ekström at Volvo, Prof. Mikael Nybacka and Prof. Lars Drugge at KTH. Hans Bäckström, Egbert Bakker and Kenneth Ekström not only helped us with experimentation work performed at Volvo, but also made nice arrangement for us to work at Volvo and transfered Gaspar’s work to us smoothly. Prof. Mikael Nybacka and Prof. Lars Drugge gave us valuable instructions and advice during the whole thesis working process.

We would also like to express our gratitudes to Henrik Hvitfeldt, Marcus Ljungberg, Suraj Sivaramakrishnan and Shounak Bhattacharyya for helping with the test.

(5)

1. Nomenclature

A. List of Symbols

d = Euclidean distance a = example data a (vector) b = example data b (vector) T = SOM training iteration No.

Tmax = the maximum SOM training iteration p = sample data No.

P = sample size xp = sample data

i = horizontal coordinate of the node j = vertical coordinate of the node = learning rate of SOM

t = the current state (before SOM training)

N = neighbor range (a set of node position coordinates) w = weight of node

nnode = node size

h = half side-length of the neighbor square σSO M = the decay rate of SOM neighborhood function σGR N N = GRNN spread coefficient

dp = Euclidean distance between test data and sample data x = test data

Yp = output of the sample data Y (x) = output of the test data

B. List of Abbreviations

OM = Objective Metrics S A = Subjective Assessments SOM = Self Organizing Map

GRN N = General Regression Neuron Network C AE = Computer Aided Engineering E P AS = Electric Power Assisted Steering ARB = Anti-roll Bar

M E R = Mean Error Rate M SE = Mean Square Error DS = Dataset

(8)

2. Introduction

Vehicles are driven and felt by people. And for an automotive company, vehicles are designed for its customers. Thus the target of the company and engineers is to design and manufacture vehicles which meet customers’ expectations and give them good feelings. Vehicles can be evaluated by objective test with numerical indexes (objective metrics). It can also be assessed by expert drivers with numerical ratings (subjective assessments). The former objective metrics can not only be obtained by field tests but also from virtual simulations, which means during the design phase, the objective evaluation of the vehicle is already known. The latter, however, can only be obtained when the vehicle can be physically felt by drivers.

Thus, there is a strong demand to build up a correlation mapping between objective metrics (OM) and subjective assessments (SA). With this correlation, engineers can easily estimate how the vehicle will be felt according to the results from CAE simulation before the vehicle is manufactured. Engineers can then check if that new model is a desired design, and also get guidelines about how to modify to get desired vehicles.

The relationships between objective metrics and subjective assessments have been analyzed by researchers using different methods. This thesis work continues the existing PhD research∗_{over test method development by use of} SOM-GRNN (self-organizing map, general regression neural network), aiming to find correlations between OM and SA. In this paper, CAE simulation is performed to analyze the relationships between OMs and vehicle parameters. First impression test is refined to better collect test data and one more dataset has been added in order to populate the SOM-GRNN map. This paper also conducts analysis over SOM and GRNN algorithms, and explores several possible applications using the SOM-GRNN map. Finally the whole SOM-GRNN system is integrated and a user interface is built in GUI for future research and application.

A lot of researches have been done to investigate the correlations between objective metrics and subjective assessments of the vehicle dynamic behaviors. The existing PhD thesis [1] (the base of this thesis work) utilizes neural networks method (self organizing map and general regression neural network) to do the study, which manages to match multiple dimensions of objective metrics to each single dimension of subjective assessments. The correlation map is trained from sample vehicle data and generalized to normal vehicle database. The aim of this thesis work is to continue and extend the existing work to make it more generalized and applicable.

The objectives of this paper are the following items:

1. Perform CAE simulation to get relationships between OMs and vehicle parameters; 2. Refine first impression test to better collect SA data;

3. Populate OM-SA based SOM-GRNN database to strengthen the neural networks method; 4. Analyze and improve the SOM and GRNN algorithms to make it more robust;

5. Explore applications from the SOM-GRNN map; 6. Build user interface for OM-SA based SOM-GRNN map.

∗_{Gaspar Gil Gómez, with the PhD thesis: Towards efficient vehicle dynamics development (From subjective assessments to objective metrics,}

(9)

3. Delimitations

The performance of this OM-SA based SOM+GRNN system depends mainly on three aspects, which are OM selection, SA grading and SOM+GRNN model accuracy.

Regarding the certain OMs to be selected, theoretically speaking, for the SOM-GRNN method we have to include all the OMs with no inter-correlations. But the OM correlation results are a bit different between the simulation-based data and the test-based data. And Not all the OMs with inter-correlations can be strictly figured out, which will be limited by the sample size and test accuracy (consistency), etc.

Regarding the SA grading, the measured SA results will be affected by factors that are difficult to control, such as human factor for different drivers, road condition and weather, etc. The current SA grading method still needs to be improved to make SA results more comparable and compatible between different test sessions.

Regarding the SOM+GRNN model, although the training algorithms are improved, the predicted SA results are still not accurate enough. We need to develop an optimization loop which can systematically find optimum parameters for SOM and GRNN training algorithms. Also training data size is not large enough. More data samples need to be added to popularize the model and make it more accurate. On the other hand, a very important step or concept in the SOM method is to compare similarities and find "winning node" (the node which has the highest similarity). And in this method, "the most similar" is regarded as "equal". It is just a relative concept and not absolutely true, which will result into systematic error.

Last but not the least, the user interface to connect our system to OM optimization system still needs to be finished, so that engineers can compact the whole OM-SA optimization and prediction loop for vehicle dynamic performance analysis and vehicle dynamic design & optimization.

4. Theoretical background

To better understand the principle behind this test method, there are several critical concepts needed to be introduced first. In this chapter, the concepts of objective metrics (OM), subjective assessments (SA), self-organizing map (SOM) and general regression neural network (GRNN) will be introduced.

A. Objective metrics (OM)

In this paper, objective metrics, or OM for short, are certain objective evaluation indexes describing vehicle dynamic behaviors, and they can be extracted from objective measurements by open-loop manoeuvres like swept steer, on-center weave test and frequency response test, etc.

Below are some OM examples: OM-1 (window), it can be extracted from the swept steer test (120 km/h) and it describes the value of the steering wheel angle when the lateral acceleration is 0.05 g. OM-10 (friction feel), it can be extracted from the on-center weave test (120 km/h, 0.2 g) and it describes the value of the steering wheel torque when the lateral acceleration is 0 g.

(10)

Table 1 List of the 27 objective steering-feel metrics (OM) used in this research. [2]

Level 3 Level 4 Level 5 Unit Measure

S

traight-ahead

Controllability

Response

Window (SWA at 0.05g) [◦] OM-1

Yaw response gain on-center [◦/s/100◦SW A] OM-2 Lateral acceleration response gain low speed [g/100◦_{SW A]} _OM-3 Lateral acceleration response gain high speed [g/100◦SW A] OM-4 Gain linearity (Steering sensitivity ratio) [-] OM-5

Response time delay [ms] OM-6

Roll Control Total roll-rate gradient at 1Hz (on-center) [◦/s/g] OM-7 Torque Feedback

Torque dead-band (SWA at 1.3 Nm) [◦] OM-8 Torque build-up (SWA Torsional rate) [Nm/100◦SW A] OM-9 Friction feel (Torque at 0 g) [Nm] OM-10

Cor

ner

ing

Controllability

Response

Yaw response gain off-center [◦/s/100◦SW A] OM-11 Linear range understeer gradient [◦/g] OM-12

Yaw gain linearity [%] OM-13

Yaw gain at maximum lateral acceleration

/ maximum yaw gain [

◦

/s/100◦SW A] OM-14 Yaw - SWA phase time lag at 4m/s2 [ms] OM-15 Ay - SWA phase time lag at 4m/s2 [ms] OM-16 Ay - Yaw phase time lag at 4m/s2 [ms] OM-17 Roll control Total roll-rate gradient during cornering [◦/g] OM-18

Torque Feedback

SWA torque build-up into the corner [Nm/100◦SW A] OM-19 SWA torque build-up cornering [Nm/g] OM-20 On-center hysteresis (Torque dead-band in degrees) [◦] OM-21 Off-center hysteresis (Torque hysteresis at 0.3g) [Nm] OM-22 Effort level (Torque at 0.3g) [Nm] OM-23

F

irs

t

impression

-Yaw low speed response gain on-center [◦/s/100◦SW A] OM-24 Low speed torque build-up (maximum SWA

torsional rate) [Nm/s/100

◦

SW A] OM-25 Parking effort standstill [Nm] OM-26

Parking effort rolling [Nm] OM-27

B. Subjective assessments (SA)

In this paper, subjective assessments, or SA for short, are subjective evaluations for vehicle dynamic performances rated by test drivers during subjective tests. SA plays an essential role in the development of vehicle dynamics. Since it straightly tells about how the vehicle feels by the people. And it is highly dependent on test driver’s personal feeling and thinking from practical tests, it may vary from time to time and different people may have different preferences for certain SAs.

(11)

Table 2 Overview of the 36 subjective assessments (SA) used in this research. [2]

Level 2 Level 3 Level 4 Level 5 SA Number

Steering Feel SA-1

First impression SA-2

Response SA-3

Toque Feedback SA-4

Manoeuvrability SA-5

Compliance Feel SA-6

Friction Feel SA-7

Effort SA-8

Parking/Manoeuvring SA-9

Effort SA-10

Returnability SA-11

Manoeuvrability SA-12 Straight-ahead Controllability SA-13

Response SA-14

Window SA-15

Roll control SA-16

Torque Feedback SA-17 Torque Deadband SA-18 Torque buildup SA-19 Friction Feel SA-20 Damping SA-21

Modulation SA-22

Cornering Controllability SA-23

Response SA-24

Roll Control SA-25

Torque Feedback SA-26 Torque Buildup SA-27

Returnability SA-28

Modulation SA-28

Handling Feel SA-30

Straight Stability SA-31

Straight Running SA-32

Transitional Stability SA-33

Stability SA-34

Controllability SA-35

Capacity Feel SA-36

C. Self-organizing map (SOM)

Self-organizing map, also called Kohonen map, is widely used in data visualization, dimensionality reduction and cluster analysis. Ricco Rakotomalala explained this technic as follows, and the SOM concept diagram is shown in Figure 1:

(12)

(often rectangular) space with the following principle: similar individuals in the initial space will be projected into the same neuron or, at least, in neighboring neurons in the output space (preservation of proximity)."[3]

Figure 1. SOM concept diagram. [3]

In this paper, since each vehicle sample data has multiple OM measurements and SA evaluations, in order to generate a map for visualization, SOM method will be used to reduce sample data dimension into two (which represents the coordinates on the map), and group similar vehicle samples together to prepare for future regression using GRNN. SOM training only deals with OM data, and GRNN regression will introduce the SA data.

Unlike the conventional SOM algorithm with a decaying learning rate. In Gaspar Gil Gómez’s research, a constant learning rate is used. [4] The whole procedure consists of 3 stages as sample data preprocessing, SOM map initialization and SOM training. During the stage of SOM training, a maximum value of training iteration is predefined. And within each training iteration, all sets of the sample data will be trained in turn. Thus, if for example there are 50 sets of sample data and 1000 maximum training iterations, then the total training times are 50000. Here the learning algorithm for SOM will be introduced step by step.

Step 1: Normalize the sample data to be as mean value 0 and standard deviation value 1. Each set of sample data consists of multiple OM measurements with different units and different orders of magnitude. After normalization, all the different OMs will vary within a similar span, which will facilitate SOM training.

Step 2: Predefine a base for the SOM map (e.g. a square with 100 nodes*100 nodes), and randomly generate initial values (or called weights) for all these nodes on the map with the same dimension as the sample data. But the randomly generated weights also have prerequisites as mean value 0 and standard deviation value 1. Then the initial weights on the map shall be on the same order of magnitude as the sample data, which will also facilitate SOM training.

Step 3: Start the training process and to find out the winning node of one set of sample data. Winning node is the node on the map who is the most similar to the sample data. The similarities between two sets of data can be quantified by calculating the Euclidean distance between them. The Euclidean distance is defined as:

d =p(a − b)T_{(a − b)} ₍₁₎

Here d is the Euclidean distance, a and b are two sets of data (two vectors with the same dimension).

The node on the map who has the minimum Euclidean distance is defined as the winning node to the corresponding sample data.

Step 4: Update the weights of the nodes covered in the neighbor range. Neighbor range is a square centering on the winning node. And the covered nodes will be updated according to the same update function:

(13)

T = 1, 2, ..., Tmax (3)

p= 1, 2, ..., P (4)

(i, j) ∈ N(T, p) (5)

Here T represents different steps of training iteration and Tmax represents the maximum training iteration. p represents the index of different sets of sample data, P represents the total amount of the sample data (or called sample size) and xprepresents one specific set of sample data. N represents the neighbor range (or can be regarded as neighbor square), which is a set of position coordinates for all the covered nodes on the map. The size of the neighbor range is relevant to the specific training iteration and the position of the neighbor range is relevant to the corresponding sample data. i and j represent the position coordinates of the covered nodes on the map. represents the learning rate, which is constant with a very small value (e.g. 0.01). t represents the current state (before training) and t+1 represents the updated state (after training). Then wi j(T,t) represents the original weights of one specific node in the neighbor range (centering on the winning node of sample data p) at training iteration T, wi j(T,t+1) represents the corresponding updated weights.

Step 5: Repeat the same training (finding out winning node and updating weights) for all the other sets of sample data, which constitutes 1 complete training iteration. During each training iteration, the size of the neighbor range remains the same, but the position of the neighbor range will change as the position of the winning node changes.

Step 6: Repeat the same training (1 complete training iteration) for a large number of iterations (e.g. 1000). During this process, the size of the neighbor range will decay according to its neighborhood decay function:

h(T )=nnode

2 × exp(− T σSO M

) (6)

Here nnoderepresents the amount of node constituting the side-length of the map (or called node size, e.g. 100). σSO Mrepresents the decay rate which will influence how quick the neighbor range will shrink. h(T) represents the half side-length of the neighbor square at training iteration T, which determines the size of the neighbor range. The neighborhood decay function with the node size as 100, decay rate as 230 and maximum training iteration as 1000 is plotted in Figure 2 (a). Since the side-length of the neighbor range is basically the amount of node, thus the corresponding neighbor ranges shall be processed into integers, which is plotted in Figure 2 (b).

(a) Neighborhood decay function (smooth) (b) Neighborhood decay function (integer) Figure 2. Neighborhood decay function.

The whole training is done when the final iteration is finished. Ideally, after this procedure of SOM training, the similar sample data will be grouped together and the neighbor nodes on the trained map are similar to each other. But the quality of the map also depends on the parameter setting of the algorithm, which will be discussed further in the algorithm analysis section.

D. General regression neuron network (GRNN)

(14)

leads to smaller weight. In this section, detailed training steps and the principle of GRNN will be introduced, mainly referred from the blog called "Basic Neural Network: Algorithm and Example", written by Gopinath Bej on Blogger in 2013. [5]

GRNN contains four basic layers, which are input layer, pattern layer, summation layer and output layer. Input layer feeds the input to pattern layer, which calculates the Euclidean distance and activation function. The summation layer will sum the multiplication of training output data and activation function to the numerator, and sum all the activation to the denominator. Finally the output layer will take the numerator and denominator parts and calculate the ratio of them. The GRNN concept diagram is shown in Figure 3.

Figure 3. GRNN concept diagram.

The governing function of GRNN is:

Y (x)= ΣYpex p(−( d2 p 2σ2 G R N N )) Σex p(− d 2 p 2σ2 G R N N ) (7)

where σGR N N is the GRNN spread coefficient, Y(x) is the output of the test data, Ypis the output of the sample data, dpis the Euclidean distance between the test data and sample data, calculated as

dp = q

(x − xp)T_{(x − xp}₎ ₍₈₎

Here x is the test data and xpis the sample data. In the governing function, the activation function is calculated as ex p(− d

2 p 2σ2

G R N N

). From the activation function we can see that small Euclidean distance leads to huge contribution to the output and larger Euclidean distance leads less contribution. If the Euclidean distance is zero, that means the test data equals to the sample data. The only adjustable parameter here is the spread coefficient σGR N N. Usually σGR N N is tunned so that the mean square error reaches small values and the fitted curve (or surface) tends to be smoothly changed.

(15)

5. SOM-GRNN Procedure

In this chapter, the general procedure for this test method will be introduced briefly. It will be mainly divided into two parts: 1) OM-SA map generation section using SOM-GRNN, 2) OM extraction & SA prediction sections from the map. The method procedure flow is shown in Figure 4.

Figure 4. Test method procedure flow chart.

Here each step will be introduced into detail.

SOM-steering feel OM: Some OM types are ruled out due to high liner-correlations between each other. So the

number of OM types for SOM training are reduced, which means each sample point has a certain dimension to describe its feature. Then using SOM training algorithm to form a two-dimensional map according to those multi-dimensional OM features, which will result similar sample points grouped together.

Assigning mean SA: During test, each sample point has also been assigned with certain SA values to evaluate its

steering performance. After averaging out those SA values from different test drivers, the mean SA is assigned to those sample points, which makes the overall OM-SA map transformed from 2-D to 3-D, and the third dimension is the mean SA.

GRNN between OM-SA: Then using GRNN algorithm to complete regression between the OM-based SOM 2-D

map and mean SA values, so that there will be continuous curve (surface) between nodes in the map.

SOM-GRNN contour: Finally generate contour map based on SA values for different nodes and using color level

to represent SA level in 2-D space. Until now the basic OM-SA regression map based on SOM-GRNN method is completed.

SA & group prediction: After generating OM-SA regression map, we can input a certain OM-DNA (DNA is a

vivid description of a set of OMs with certain values) and find the winning node which has minimum Euclidean distance with it. In this way we can know which OM combination is that DNA most similar to, and use the SA value assigned to that winning node as a SA prediction. Also since similar OM-DNAs are grouped together, we can know what kind of vehicle configuration is this inputted OM-DNA similar to.

OM extraction: When developing certain vehicle configuration, we can start by selecting certain nodes from the

(16)

(17)

6. CAE simulation

Objective metrics can be obtained from field tests as well as CAE simulations. CAE simulation can investigate the influences of different vehicle parameters on the vehicle dynamic performances in the early design phase. Here in this case, CAE simulation can help to decide how to modify vehicle parameters to get desired vehicle configurations to get a spread of OM data, which is helpful to get a generalized SOM-GRNN map. On the other hand, doing the parameter study can also get a guideline about how to choose and tune vehicle parameters to push OMs into targeted ranges in the design phase.

A. Simulation set-up

The simulation environment consists of three parts: manoeuvre input, vehicle model and signal extraction, which is indicated in Figure 5.

Figure 5. Simulation set-up.

The selected 27 OMs in this paper are measured from several different types of test manoeuvres like swept steer, on-center weave test, frequency response test and sine with dwell test, etc, with different velocities or steering wheel angle amplitudes. Figure 6 shows some examples of the test manoeuvres with velocity as 120 km/h.

(a) Swept steer test (b) On-center weave test (c) Frequency response test

Figure 6. Examples of test manoeuvres.

Vehicle model is a multi-body system with K&C test data in CarMaker†. There is also an electric power assisted steering (EPAS) control system co-simulated with the vehicle model. Figure 7 shows the visualization of the vehicle model in the simulation environment.

(18)

Figure 7. Vehicle model.

Raw data as steering wheel torque, yaw rate, roll rate and lateral acceleration, etc are recorded from the simulation to extract OMs. OM extraction is actually a process of data processing of the raw data. Some examples of the data processing are steering wheel torque at 0 lateral acceleration, yaw rate to steering wheel angle changing rate and metrics from frequency-response performances, etc. It is shown in Figure 8 with some examples of OM extraction.

(a) Friction feel (b) Yaw response gain

(c) Steering wheel angle-torsional rate (d) Frequency response

Figure 8. Examples of OM extraction.

B. Multicollinearity check

1. Based on test data

(19)

Figure 9. Multicollinearity check based on test data. [4]

Thus in the previous research, OMs-1, 2, 3, 4, 9, 11, 12 and 19 are represented only by OM-4. OMs-15, 16 and 17 are represented only by OM-16. Consequently, only OMs-4, 5, 6, 7, 8, 9, 10, 13, 16, 18, 19, 20, 21, 22, 23, 26 and 27 are used in the SOM-GRNN method (OMs-14, 24 and 25 are not considered for presenting wrong or missing values). [4] 2. Based on simulation

Objective tests were performed to 13 different virtual vehicle models in CAE simulation. Thus 13 sets of OM data can be extracted from the simulation. The same multicollinearity check was performed on the simulation data. The inter-correlation relations between the OMs based on simulation are shown in Figure 10. The inter-correlations

Figure 10. Multicollinearity check based on simulation.

between OM-4 and OMs-1, 2, 3, 11 and inter-correlations between OM-16 and OMs-15, 17 are further confirmed by the simulation. But there are no obvious correlations between OM-4 and OMs-9, 12, 19. On the other hand, there are more other correlations found from the simulation, among which the correlations between OM-19 and OM-9 and correlations between OM-23 and OM-19 have the highest potential to be confirmed. Since the correlations between them can also be seen from the testing data. Regarding OMs-14, 24 and 25, correlations between OM-14 and OM-24, OM-4 and OM-25 can be found from the simulation. Correlation between OM-9 and OM-24 can be seen from the testing data. But no correlations for OM-14 and OM-25 can be found from the testing data. Combining the findings from simulation and testing, we can conclude that OM-14 is more independent and shall be taken into consideration as well. Consequently, OMs-4, 5, 6, 7, 8, 9, 10, 13, 14, 16, 18, 19, 20, 21, 22, and 23 were included in the parameter study. (Because of unreliable tire models in parking manoeuvre, OM-26 and OM-27 were not included.)

C. Parameter study

(20)

The parameter study was done by varying one parameter at a time and comparing with the standard set-up to figure out the change of OMs. The detailed correlations between OMs and parameters are listed in the appendix.

The summary of the influences of different vehicle parameters on OMs is shown in Table 3, where "+" represents positive correlation‡_{, "-" represents negative correlation}§_{, "*" represents unfixed positive or negative correlation, more} signs represent stronger correlation (e.g. "+++" is a stronger positive correlation than "++").

Table 3 Summary of the vehicle parameter study.

OM No. OM description Unit Most sensitive vehicle parameters in order

OM-4 Lateral acceleration

re-sponse gain high speed [g⁄(100°SWA)]

mass distribution(- - -), toe-in front(- -), toe-in rear(++), tyre pressure front(+), tyre pressure rear(-)

OM-5 Gain linearity (Steering

sensitivity ratio) [-]

mass distribution(- - -), toe-in front(- -), toe-in rear(++), tyre pressure front(+), tyre pressure rear(-)

OM-6 Response time delay [ms] mass distribution(- - -), toe-in front(- -), toe-in rear(++), tire pressure rear(-)

OM-7 Total roll-rate gradient at

1 Hz (on-centre) [(°⁄s)⁄g]

ARB front(- - -), ARB rear(- - -), front spring(- -), rear spring(- -), front damper(- -), rear damper(- -), camber front(-), camber rear(-)

OM-8 Torque dead-band (SWA

at 1.3 Nm) [°] toe-in front(++) OM-9 Torque build-up (SWA

Torsional rate) [Nm⁄(100°SWA)] toe-in front(**), camber rear(**), tire pressure rear(-) OM-10 Friction feel (Torque at 0

g) [Nm] N.A.

OM-13 Yaw gain linearity [%] ARB front(+++), ARB rear(- - -), front spring(++), rear spring(- -)

OM-14

Yaw gain at maximum lat-eral acceleration / maxi-mum yaw gain

[(°⁄s)⁄(100°SWA)]

mass distribution(- - -), ARB front(- -), front spring(- -), rear spring(++), camber front(-), camber rear(+), ARB rear(+)

OM-16 Ay-SWA phase time lag

at 4m/s2 [ms]

mass distribution(- - -), toe-in front(- -), toe-in rear(++), ARB front(- -), ARB rear(++), front spring(-), rear spring(+), tire pressure rear(-), camber rear(+)

OM-18 Total roll-rate gradient

during cornering [°⁄g]

ARB front(- - -), ARB rear(- - -), front spring(- -), rear spring(- -), camber front(-), camber rear(-)

OM-19 SWA torque build-up into

the corner [Nm⁄(100°SWA)] toe-in front(- -) OM-20 SWA torque build-up

cor-nering [Nm⁄g]

mass distribution(+++), ARB front(++), ARB rear(- -), rear spring(-), toe-in front(+), toe-in rear(-)

OM-21

On-centre hysteresis (Torque dead-band in degrees)

[°]

ARB front(- - -), ARB rear(- - -), mass distribution(- -), rear spring(- -), toe-in front(+), toe-in rear(-), tire pressure rear(+), rear damper(+)

OM-22

Off-centre hysteresis (Torque hysteresis at 0.3 g)

[Nm] N.A.

OM-23 Effort level (Torque at 0.3

g) [Nm] mass distribution(++)

(21)

7. Test

A. Test process

Tests consist of objective test and subjective test. Some typical vehicles from different classes and brands are chosen to get a spread of OM and SA data.

The strategy of the objective test is the same as CAE simulation. Different open-loop manoeuvres are inputted to the vehicle, and the vehicle’ states are logged to extract all kinds of OMs. There are only two parameters defining different manoeuvres: steering wheel angle and vehicle velocity, which are controlled by steering wheel control robot and gas pedal control robot in the objective test.

Subjective test is to quantify drivers’ feelings towards the tested vehicles. Several closed-loop manoeuvres are predefined to constitute the subjective test, which is called first impression test. Based on the previous research, the first impression test was further refined to more comprehensively evaluate the vehicle dynamic behaviors and let drivers get a more complete picture of the vehicle. The predefined manoeuvres include low-speed slalom, low-speed parking manoeuvrability, medium-speed skid-pad, medium-speed short handling track and high-speed oval track, which is shown in Figure 11.

Figure 11. Test track.

Thus, all aspects of vehicle dynamics like parking, cornering, straight-running and the whole range of speed are covered altogether. After or during each test, drivers are expected to give ratings to all the SAs and also have interview to describe the characters of the vehicle. There are two rating criteria for SAs. And the rating criteria are kept the same as the previous PhD research. [1] For the SAs in level 2-4, 1 to 10 (interval 1) SAE scale is used, which is shown in Table 4. For the SAs in level 5, 1 to 5 (interval 0.25) rating scale with anchored antonym adjective is used, which is shown in Table 5. And Table 5 also shows an example of rating as 2.75, which is a medium-level feeling.

Table 4 Subjective rating scale for SA levels 2-4. SAE J1441. [1]

Very poor Poor Fair Good Excellent

1 2 3 4 5 6 7 8 9 10

Undesirable Borderline Desirable

Table 5 Subjective rating scale for SA level 5. [1]

Small Medium Large

(22)

The detailed test procedure is:

1) Start the car from parking lot to the manoeuvrability area. 2) Drive the car to slalom.

3) Drive the car to the skid-pad area for half lap. 4) Drive the car to the short handling track for 1 lap. 5) Drive the car to the high speed track for 2 laps. 6) Drive back and park the car.

7) Rating and interview inside the vehicle. (During the interview, drivers are expected to describe steering and handling performances of the car, also the character conveyed by each vehicle.)

8) Test next vehicle, until all the vehicles are tested.

B. Test result

The results from the new test will be called as "DS4". Some lower levels and higher levels of SAs are chosen to show the test results, which is detailedly listed in the appendix. Based on the mean values of some selected SAs in level 2 and 3 by all the drivers, one can see the overall SA performances of different vehicles by radar plot. Besides SA performance, we also have OM measurements of the test vehicles, and driving-experience interviews from the test drivers. The OM measurements can be presented using OM-DNA. And the results from the driving-experience interview can be presented using word cloud (produced by the online software Wordle™ [6]), where more often mentioned words appear in larger fonts.

With the SA radar plot, combined with the measured OM-DNA plot and the word cloud based on interviews from test drivers, we can conclude the overall performances and characters of the tested vehicles. The results are shown in Figure 12 to Figure 16. The objective test data (OM-DNA) and subjective test data (mean values of SAs by all the drivers) will then be further used in the SOM and GRNN algorithms respectively.

(a) Vehicle 1 OM-DNA (b) Vehicle 1 SA radar

(c) Vehicle 1 word cloud

(23)

Figure 13. Overall performances of vehicle 2.

(24)

Figure 15. Overall performances of vehicle 4.

(25)

8. SOM-GRNN algorithm analysis

In this chapter, two types of neural networks (SOM and GRNN) used in this study are analyzed into detail. First for the SOM part, the quality check is done for the SOM algorithm with the original parameter setting from the previous research. Then a parameter study is done for the SOM algorithm, showing the comparisons of SOM quality between different parameter settings. Second for the GRNN part, apart from the original GRNN regression from the previous research, a new type of GRNN regression with a different strategy of data-input is analyzed and compared.

In order to make it compatible with the previous research, the same 17 OMs (OM-4, 5, 6, 7, 8, 9, 10, 13, 16, 18, 19, 20, 21, 22, 23, 26 and 27) as the previous research are used to constitute the SOM-GRNN map in this paper.

A. SOM algorithm Analysis

1. SOM quality check

As can be seen from the theoretical-background section, there is no convergence criterion in the SOM algorithm to terminate the training. And the training is naturally done when the predefined iterations are fulfilled. Thus in order for the SOM-GRNN method to be valid, we have to extra check and ensure that the SOM map has high quality of achieving its 2 targets: 1) To cluster similar data (vehicles); 2) To accurately reflect sample data (sample vehicles) on the map.

Correspondingly, we can introduce two indicators to check these two targets. The first target can be indicated by the Euclidean distances between neighbor nodes on the map, defined as neighbor distance. Shorter distance means more similarities between neighbor nodes, which further means similar vehicles are grouped together and vice versa. The second target can be indicated by the Euclidean distances between sample data (sample vehicles) and the winning nodes on the map, defined as sample data-winning node distance. Shorter distance means sample data (sample vehicles) are more accurately reflected on the map, which further means sample data are not distorted after training and vice versa. How short the distance can indicate two nodes or two sets of data are similar to each other? To get a rough idea of this we can do a simplified analysis: After normalization of the OM data to be as mean value 0 and standard deviation value 1, the spread of each OM can be conservatively estimated as from -1.5 to 1.5. Then for example if for two nodes, the difference of each OM (17 OMs in total) is 0.01, the overall Euclidean distance between these two nodes can be calculated as 0.04. The difference ratio for each OM can be defined as the difference value divided by the spread width, calculated as 0.33% in this case. With the same method, the interpretation of different distances is shown in Table 6. From the table we can say that if the distance is smaller than 0.2, these two nodes are quite similar. And if the distance is larger than 1, the differences between two nodes are quite noticeable.

Table 6 Interpretation of the euclidean distances.

Average difference value of each OM Average difference ratio of each OM Euclidean distance

0.01 0.33% 0.04 0.02 0.67% 0.08 0.05 1.67% 0.21 0.1 3.33% 0.41 0.5 16.7% 2.06 1 33.3% 4.12

(26)

(a) Interpretation of neighbor distances (b) Visualization of neighbor distances (learning rate: 0.01) Figure 17. Neighbor distance map.

(a) Before training (b) Training iteration 5 (c) Training iteration 10

(d) Training iteration 30 (e) Training iteration 50 (f) Training iteration 100

(g) Training iteration 300 (h) Training iteration 700 (i) Training iteration 1000

Figure 18. SOM training process.

(27)

Figure 19. SOM training process (learning rate: 0.01).

From Figure 17 (b) and Figure 19 we can see that after training, the sample data-winning node distances are kept at a extremely low level, which means the sample data are accurately reflected on the trained map (or too accurately reflected). On the other hand, the neighbor distances are much larger and unevenly distributed on the map. Thus for this trained SOM map, the second target is achieved while the first target is not. Then we can conclude that the overall quality of the SOM map is not good enough, the similarities in some areas are too low or the clustering performances in these areas are too poor, so the further application of SOM-GRNN cannot be guaranteed.

In order to improve the quality of the SOM map we need to study and understand the training process first. From Figure 19, we can discover that over the whole training process, the two targets of SOM contradict with each other. Until around 30th training iteration, neighbor distances reduce while the sample data-winning node distances increase, from 30th training iteration to the end, the situation is the other way around. This phenomenon can also be seen from Figure 18. Based on neighborhood decay function shown in Figure 2 and SOM training process shown in Figure 19, we can get an idea about how the SOM training is going on. The learning rate is constant (0.01 in this case), and the covered neighbor range decays according to Figure 2. From Figure 19 we can see that from the initial state to around 30th training iteration, the neighbor distances reduce significantly to minimum while the sample data-winning node distances increase to maximum. It is because during these training iterations, the covered neighbor ranges are quite large, from 100 to 90 (2*50 to 2*45, get from Figure 2) , which covers a large portion of the map (if the winning node is at the center of the map, then it covers almost the whole map). So all the nodes on the map can be trained altogether for a large number of times. This is illustrated in Figure 20 (exemplified with node size as 10*10 and 5 winning nodes), where the red squares represent the neighbor squares (neighbor ranges).

Figure 20. Neighbor ranges in early stage of training.

(28)

of magnitude. The update function for each node after each time of training is shown in Equation 2. For this update function (learning rate 0.01), if the target value is one fixed data, then after around 600 times of training, the trained value will be closely converged to the target value, which is exemplified in Figure 21 (a). If the target value is a vector consisting of several elements, then still after around 600 times of training (all the elements in the target vector will be defined as the target value in turn and repeat for training), the trained value will be closely converged to the mean value of all the elements in the target vector. It is indicated in Figure 21 (b), where in this example, the initial trained value is 0 and the target vector is [1 3 5 7 9].

(a) Target value as one fixed data (b) Target value as a vector

Figure 21. Interpretation of the update function.

It explains that until around 30th training iteration, the neighbor distances reduce to minimum, all the nodes on the map are closely converged to the mean value of all sets of sample data (a mean vector of all the sample vectors). On the other hand, during this process, because of large neighbor ranges, winning nodes are covered in the neighbor ranges of other winning nodes, which means winning nodes are influenced by each other and the positions of the winning nodes keep changing. Thus all the nodes on the map are not trained to any fixed sample data, which results into large sample data-winning node distances. But afterwards when the neighbor range becomes smaller and smaller, winning nodes will be less influenced by other winning nodes and their positions will tend to be fixed. The change of the positions of the winning nodes can be clearly seen from Figure 18. So the sample data-winning node distances will then be reduced. In the mean time, the large neighbor distances take place in the process when the winning nodes are fixed and the neighbor ranges shrink. And the large neighbor distances emerge between the nodes on the outer ring and the nodes on the inner ring of the neighbor square. This is illustrated in Figure 22 (exemplified with node size as 10*10 and 3 winning nodes), where the red squares represent the neighbor range in the current step and the green square represents the neighbor range of one winning node in the previous step, and the nodes covered by the dotted line (green and red) represent the outer-ring nodes and inner-ring nodes respectively.

Figure 22. Neighbor ranges in late stage of training.

(29)

neighbor range, which can be seen from the update function shown in Equation 2. Obviously, the values of these neighbor distances are relevant to the update function (learning rate), neighborhood decay function, total number of training iteration and also the sample size.

2. SOM parameter study

To improve the quality of the current SOM map, in this case it is especially needed to reduce the neighbor distances. Several methods have been studied as: varying the decay rate of the neighborhood decay function, letting the learning rate decay or extending the number of training iteration, etc. But all these methods cannot achieve both targets of SOM altogether, often it appears the situation is that one target is well satisfied while the other is not met. Solutions proven to be the most efficient are lowering the constant learning rate and extending the node size. Also, changing the sample size will lead to an obvious influence as well.

a. Influence of the learning rate

Keeping using constant learning rate but with smaller values can balance the neighbor distances and the sample data-winning node distances. Since as mentioned before, the sample data are over accurately reflected on the trained map, then it shall be applicable to use smaller learning rate to reduce the neighbor distances, in the meantime, increase the sample data-winning node distances but still keep it within an acceptable low level. Figure 23 shows the comparison between different learning rates for the final neighbor-distance map. Figure 24 shows the comparison between different learning rates for the training process. The final trained distances are more clearly indicated in Table 7.

(a) Learning rate: 0.01 (b) Learning rate: 0.005

(c) Learning rate: 0.0035 (d) Learning rate: 0.002

(30)

(a) Learning rate: 0.01 (b) Learning rate: 0.005

(c) Learning rate: 0.0035 (d) Learning rate: 0.002

Figure 24. Comparison between different learning rates for training process.

Table 7 Comparison between different learning rates for final trained distances.

Learning rate Maximum neighbor distance Mean neighbor distance Maximum sample data-winning node distance

Mean sample data-winning node distance

0.01 1.644 0.2354 0.0034 0.0015

0.005 1.055 0.2112 0.126 0.056

0.0035 0.811 0.187 0.293 0.1687

0.002 0.6914 0.1646 1.126 0.5475

Among these alternatives of learning rate, we can find that the original value 0.01 results into the sample data-winning node distances to be overly small and the neighbor distances to be too large and unevenly distributed. While the value 0.002 results into too large sample data-winning node distances, which will make the sample data distorted and incorrectly reflected after training. Value 0.0035 seems to be a more suitable choice, which balances the neighbor distances and sample data-winning node distances to a similar scale with acceptable small magnitudes according to the judging standard shown in Table 6. And the neighbor distances are more evenly distributed between winning nodes on the map.

b. Influence of the node size

(31)

(a) Neighbor function, 100 nodes (b) Training process, 100 nodes (c) Neighbor distance map, 100 nodes

(d) Neighbor function, 150 nodes (e) Training process, 150 nodes (f) Neighbor distance map, 150 nodes

(g) Neighbor function, 200 nodes (h) Training process, 200 nodes (i) Neighbor distance map, 200 nodes

(j) Neighbor function, 250 nodes (k) Training process, 250 nodes (l) Neighbor distance map, 250 nodes Figure 25. Comparison of SOM quality between different node sizes.

From Figure 25 we can find that as the node size increases, the neighborhood decay function demonstrates the same decaying trend, but the width of each step decreases (the curve tends to be more smooth), which means at each step of neighbor range, its number of training iteration decreases. From the figures of training process and neighbor distance map we can see the neighbor distances decrease a lot, which is more clearly indicated in Table 8.

Table 8 Comparison of SOM quality between different node sizes.

Node size Maximum neighbor distance

Mean neighbor distance

Maximum sample data-winning node distance

100 0.811 0.187 0.293 0.168

150 0.651 0.122 0.299 0.156

200 0.429 0.092 0.330 0.153

250 0.332 0.075 0.270 0.146

From Table 8 we can see an obvious improvement of the neighbor distances. But there is less influence on the sample data-winning node distances, which is reasonable since the learning rate and the total number of training iteration remain the same.

(32)

(a) Training process (b) Neighbor distance map Figure 26. SOM quality of learning rate 0.004 and node size 200.

Table 9 SOM quality of learning rate 0.004 and node size 200.

Maximum neighbor distance

Mean neighbor distance

Maximum sample data-winning node distance

0.383 0.091 0.204 0.102

c. Influence of the sample size

(33)

(a) Training process, 45 sample data (b) Neighbor distance map, 45 sample data

(c) Training process, 35 sample data (d) Neighbor distance map, 35 sample data

(e) Training process, 25 sample data (f) Neighbor distance map, 25 sample data

(g) Training process, 15 sample data (h) Neighbor distance map, 15 sample data

(34)

Table 10 Comparison of SOM quality between different sample sizes. Sample size Maximum neighbor distance Mean neighbor distance Maximum sample data-winning node distance

Mean sample data-winning node distance 45 0.811 0.187 0.293 0.168 35 0.73 0.175 0.238 0.161 25 0.69 0.166 0.234 0.153 15 1.75 0.31 0.196 0.143 5 7.88 1.61 0.104 0.097

From Figure 27 and Table 10 we can find:

1) As the sample size decreases, the sample data-winning node distances decrease. It’s because that as the sample size decreases, the winning nodes become less and they are less influenced by others. So they can be more directly trained towards the sample data, which results into smaller sample data-winning node distances.

2) When the sample size is lower to a specific level (e.g. lower than 25 as 15 and 5), the winning nodes are less influenced by others. And they tend to be fixed at very early training stages, so some regions on the map don’t even have the opportunities to be trained, which results into very huge neighbor distances. This situation is obviously shown in the case when the sample size is 5, indicated in Figure27 (i) and (j).

3) When the sample size is larger to a specific level (e.g. larger than 25), the winning nodes are severely influenced by others. And all the nodes on the map can be thoroughly trained, which results into very small neighbor distances. But as the sample size further goes up, the neighbor distances also slightly increase. This phenomenon can be similarly interpreted as the influence of the node size in the previous section. The large neighbor distances take place between the outer-ring nodes and inner-ring nodes in the process when the winning nodes tend to be fixed and neighbor ranges shrink. More sample data means more training times during each training iteration, which will enlarge the distances between the outer-ring nodes and the inner-ring nodes.

(35)

B. GRNN algorithm analysis

The main purpose for this paper is to find proper correlation between OMs and SAs, and the method here to connect OMs and SAs is using GRNN algorithm. The reason to use GRNN is that it has high calculation efficiency and it can always reach convergence.

In the previous research done by Gasper [1], node coordinates (x,y) after SOM training are used as the input for GRNN regression to predict SAs. Here the node coordinates (x,y) can reflect data topology to some extent, so in this way we can reduce the data input used for GRNN regression from 17 dimensions to 2 dimensions. The node coordinates (x,y) can be viewed as media to connect OMs and SAs, and the principle is shown by Figure 28.

Figure 28. GRNN method I concept diagram.

Since in this case the sample size is small and sample dimension is not high, thus the calculation cost is trivial for GRNN regression step. Instead of using node coordinates (x,y) for GRNN regression, we can use the original OMs sets generated by SOM training for GRNN regression, which has dimension of 17 instead of 2. By principle, using original OM dataset for GRNN regression, the predicted SAs should be more accurate, and it will not be affected by the node coordinates (x,y) since the (x,y) coordinates cannot completely reflect the original OMs sets’ topology. The concept diagram for GRNN method II is shown in Figure 29.

Figure 29. GRNN method II concept diagram.

(36)

previous SOM analysis section. The SA data that is used for analysis here is SA-2, which is the general steering feel based on first impression test.

1. GRNN quality measurement

Here we mainly use two quantities to describe the quality of the regression model after using GRNN. One is the Mean Square Error(MSE). The calculation method is

M SE = 1 P P Õ p=1 (S A0_p− S Ap)2 (9) Here P is the validation sample size, p is the sample data No., S A is the real sample SA value, and S A0is the predicted SA value.

The other is the mean error rate between the real sample SA value and the predicted SA value after GRNN regression. The calculation method is

M E R= 1 P P Õ p=1 |S A0_p− S Ap| S Ap × 100% (10)

(37)

2. Influence of GRNN spreadσGR N N

In this section, we first set the database to be DS123, which includes Dataset 1, Dataset 2 and Dataset 3. The sample size for DS123 is 48.

We start from tuning GRNN spread using 2-D GRNN method. The results for SOM-GRNN map, the corresponding 3D plots and the corresponding error analysis is shown in Figure 30.

(a) SOM-GRNN map (sp=1) (b) SOM-GRNN 3D plot (sp=1) (c) GRNN error analysis (sp=1)

(d) SOM-GRNN map (sp=10) (e) SOM-GRNN 3D plot (sp=10) (f) GRNN error analysis (sp=10)

(g) SOM-GRNN map (sp=20) (h) SOM-GRNN 3D plot (sp=20) (i) GRNN error analysis (sp=20)

Figure 30. 2-D GRNN results with different GRNN spread(DS123, node10000).

In the error analysis figures, x-axis sample number refers to certain sample, eg. 23 refers to sample No.23. Here we can see that error will increase with the increase of GRNN spread. When the GRNN spread is too large, the regression model will be completely distorted. On the other hand, from SOM-GRNN map and 3D plot we can see that when the GRNN spread is too small, the regression model will be too complex, which will cause problem of overfitting. For 2-D GRNN method with DS123 and node size of 100*100, the optimum GRNN spread is roughly about 10.

(38)

GRNN regression quality, GRNN spread in full-D GRNN method needs to be reduced. After experimentation, for full-D GRNN, similar GRNN regression quality requires GRNN spread of 1.85, and the results are shown in Figure 31.

(a) SOM-GRNN(2-D) map (sp=10) (b) SOM-GRNN(full-D) map (sp=1.85)

(c) SOM-GRNN(2-D) 3D plot (sp=10) (d) SOM-GRNN(full-D) 3D plot (sp=1.85)

(e) GRNN error comparison

(39)

From Figure 31 we can see that the regression models are roughly similar. Compared with 2-D GRNN regression model, full-D GRNN regression model is more complex and more similar to the database’s original topology. Unlike figure (a) and (b) in Figure 30 whose predicted SA values are almost exactly the same as real SA values and there are always sudden value-jump around those sample nodes, which will inevitably cause problem of overfitting, in the case of full-D GRNN regression model, it not only describes the database’s original topology properly, but also generates proper trend prediction in the intervals among sample nodes, and there are no sudden value-jump around those sample nodes. Based on this result, one can anticipate that for database which has larger sample size and more complex database topology, full-D GRNN method should generate regression model more similar to the original database topology.

On the other side, for 2-D GRNN regression model, since (x,y) coordinates are always continuous and has uniform change rate, so generally the map has uniform change rate with respect to SA values. But actually although (x,y) has uniform change rate, the OM-space behind (x,y) coordinate space is not necessarily continuous and it does not necessarily have uniform change rate, which can be reflected in full-D GRNN regression model. This (x,y) coordinate space is generated by SOM training algorithm, so this coordinate space quality depends on SOM training quality, and SOM training quality is largely influenced by node size. One can anticipate that if the node size is increased, (x,y) space will have better quality and the rapid changes in OM high-dimension space can be represented in (x,y) 2-dimension space. But increasing node size will significantly increase calculation cost for SOM training, thus with relatively reasonable calculation cost, full-D GRNN method should have better regression model.

(40)

3. OMs extraction analysis

One of the major applications for this OM-SA related SOM-GRNN map is to perform OMs extraction from the map. The principle is that after SOM training based on OM combinations for the sample data, all nodes in the map will be assigned one certain OM combination. So by picking up certain nodes from the map, one can extract the OM combinations behind those nodes, which can be used for vehicle dynamic design and optimization.

In this section, we will still use DS123 as database which has 48 sample data, node size is fixed to be 100*100 and GRNN spread will be fixed to 10 for 2-D GRNN regression model and 1.85 for full-D GRNN regression model, so that both models will have similar error rate.

After SOM-GRNN training, the SOM-GRNN map we get from both GRNN methods is shown in Figure 32.

(a) SOM-GRNN(2-D) map (sp=10) (b) SOM-GRNN(full-D) map (sp=1.85)

(c) SOM-GRNN(2-D) 3D plot (sp=10) (d) SOM-GRNN(full-D) 3D plot (sp=1.85) Figure 32. 2-D and full-D GRNN results for OM extraction analysis (DS123, node 100*100).

Based on those two SOM-GRNN maps, we extract certain OM ranges based on the same SA-threshold. In this case, the SA-threshold is set to be 7.5-7.7, and the resulted nodes-extraction and OMs ranges extraction are shown in Figure 33.

(41)

(a) SOM-GRNN(2-D) picked nodes (SA = 7.5-7.7) (b) SOM-GRNN(2-D) OMs extraction (SA = 7.5-7.7)

(c) SOM-GRNN(full-D) picked nodes (SA = 7.5-7.7) (d) SOM-GRNN(full-D) OMs extraction (SA = 7.5-7.7) Figure 33. 2-D and full-D OM extraction analysis(SA = 7.5-7.7).

full-D GRNN method to build SOM-GRNN map.

(42)

4. Influence of sample size P and node size nnodes

In this section, we will tune the sample size P and node size nnodeto see its effect on full-D GRNN method. First we fix the GRNN spread to be 1.85, and fix the database to be the same as before, DS123, which has sample size of 48, and tune the node size nnode. The node size are 50*50, 100*100 and 200*200 accordingly. Because SOM training needs to randomize weights for the nodes during initialization every time, so the resulted SOM-GRNN maps are different from each other. The focus here is to analyze the quality of the generated map. The results are shown in Figure 34.

(a) SOM-GRNN map (node2500) (b) SOM-GRNN 3D plot (node2500) (c) SOM-GRNN error (node2500)

(d) SOM-GRNN map (node10000) (e) SOM-GRNN 3D plot (node10000) (f) SOM-GRNN error (node10000)

(g) SOM-GRNN map (node40000) (h) SOM-GRNN 3D plot (node40000) (i) SOM-GRNN error (node40000) Figure 34. full-D sample size and node size analysis(DS123, sp = 1.85).

(43)

minima.

While we can see that when node size is too large, in this case node size 200*200 with only 48 samples, there will be minor errors at the edges where SA value generally changes. This means at those places, SOM training generates many nodes with OM combinations that are generally similar to each other, so that when calculating Euclidean distance during GRNN process, the resulted SA values are close but not steadily changing. This is an overfitting phenomenon, which is caused by too small sample size compared with node size. This problem cannot be fixed by tuning GRNN spread.

Next, we set the sample size to be smaller by choosing solely DS2 with sample size of 15. The results are shown in Figure 35.

(a) SOM-GRNN map (node10000) (b) SOM-GRNN 3D plot (node10000)

(c) SOM-GRNN map (node2500) (d) SOM-GRNN 3D plot (node2500)

Figure 35. full-D sample size and node size analysis(DS2, sp = 1.85).

From Figure 35 we can see that when sample size is reduced from 48 to 15, node size of 100*100 has already caused problem of overfitting. And this problem can be fixed by reducing node size from 100*100 to 50*50.

(44)

9. SOM-GRNN map validation & application

In this chapter, the SOM-GRNN map is first validated using DS4. Then some applications using this OM-SA related SOM-GRNN map are presented as OMs extraction, SAs extraction & localization by certain OM combinations, OM sensitivity test and extracted OM ranges for CAE optimization.

A. Validation of SOM-GRNN map using DS4

In this section, we will validate the accuracy of the generated SOM-GRNN map using test results from DS4. Thus, here the SOM-GRNN map will be generated using DS123, with sample size as 48. And the validation will be performed using DS4, with sample size as 5. The validation will be mainly regarding the results of SA values and vehicle character grouping result from SOM-GRNN map based on the OM-DNA input from DS4.

1. SA validation

Using OM-DNA input from DS4 test result, we can obtain SA prediction values from SOM-GRNN map generated by DS123. Then with the comparison between the predicted SA values and real SA values from DS4, we performed SA error analysis accordingly and calculated the corresponding MSE and MER. The results of SA error analysis for five test vehicles in DS4 are shown in Figure 36.

(45)

(a) Vehicle - 1 (b) Vehicle - 2

(c) Vehicle - 3 (d) Vehicle - 4

(e) Vehicle - 5

(46)

2. Vehicle character grouping validation

As introduced in the previous sections, when inputting a certain OM-DNA into the SOM-GRNN map, it will be assigned to a winning node whose OM-DNA from SOM training is the most similar to this inputted OM-DNA. By principle, similar vehicles will be assigned to neighboring nodes. Thus this section will perform the validation of this assumption by the combination of SOM-GRNN winning node location of DS4 and their corresponding word clouds. The SOM-GRNN map with DS4 winning node location and the corresponding word clouds is shown in Figure 37.

Figure 37. DS4 winning node and Word Cloud.

This SOM-GRNN map is generated based on SA-2 values. Since SOM grouping does not rely on SA values, so it is the same with other SA related maps, and only the node location here matters with vehicle character grouping analysis. Here we can see that Vehicle - 1, Vehicle - 4 and Vehicle - 5 are grouped together, and they are all defined as sporty cars by most drivers. Vehicle - 3 is quite far away from the other test vehicles, which is reasonable because it is the only vehicle that belongs to SUV.

(47)

B. SOM-GRNN map application

In this section, the OM-SA related SOM-GRNN map that will be used to perform applications 1, 2 and 3 is the map shown in Figure 38. It has node size as 100*100, which means the x- and y- coordinates range from 1-100. The database is DS123 with sample size as 48.

Figure 38. OM-SA related SOM-GRNN map for application demonstration.

1. OMs extraction

(48)

(a) OM extraction on the map(single node -(50,30))

(b) OM extraction result(single node - (50,30))

(c) OM extraction on the map(rectangular area) (d) OM extraction result(rectangular area)

(e) OM extraction on the map(SA 7.4 - 7.8) (f) OM extraction result(SA 7.4 - 7.8)

(g) OM extraction on the map(Area + SA 7.4 -7.8)

(h) OM extraction result(Area + SA 7.4 - 7.8)

(49)

2. SAs extraction and localization by certain OM-DNA

Based on this map, one can manually input certain OM-DNA, and find the node that is closest to that OM-DNA to localize it on the map. Then the predicted SA values for that OM combination can be extracted, and we can also see what types of vehicle setting is that OM-DNA similar to. The result is shown in Figure 40.

(a) winning node localization on the map (b) Manual input OM-DNA Figure 40. winning node localization by certain OM-DNA.

(50)

(a) SA radar plot (three SAs) (b) SA radar plot (four SAs)

(c) SA radar plot (five SAs) (d) SA radar plot (six SAs)

(e) SA radar plot (seven SAs) (f) SA radar plot (eight SAs)

(51)

Also we can modify certain OM value among this OM-DNA to see how it influence SA results and the general vehicle type of this OM-DNA. The example is shown in Figure 42.

(a) winning node localization on the map(OM-19 changed)

(b) Manual input OM-DNA(OM-19 changed)

Figure 42. winning localization by varying OM-19.

(52)

3. OM sensitivity test

OM sensitivity is performed by tuning one certain OM value from minimum to maximum, while at the same time fixing the rest of the OMs, and see how the winning nodes vary in the map. In this example we perform the OM-4 sensitivity test, and set the tuning interval to be 100 steps. The result is shown in Figure 43.

Figure 43. OM sensitivity test (OM-4, 100 steps).

The red stars on the map are the winning nodes during the test, and the number shows how many times the tuned OM combination fit that certain winning node.

We can also analyze OM sensitivity by simply looking at the size of the extracted OM ranges during OM extraction application, like the example shown in Figure 44.

(a) OM extraction on the map(SA > 7.8) (b) OM extraction result(SA > 7.8) Figure 44. OM extraction result (SA > 7.8).

(53)

4. Extracted OM ranges for CAE optimization

The ultimate target for CAE optimization is to get a design setting which gives satisfying driving feedbacks, or high levels of SAs. But the CAE simulation only deals with the OM data instead of SA data. And the typical way to do CAE optimization is to define OM values (or ranges) as design targets and the vehicle parameters as design variables. Thus, one of the important applications for this SOM-GRNN map tool is to extract OM ranges for CAE optimization, and the way to extract OM ranges has already been introduced in the previous section "OMs extraction".

But one important issue that needs to be emphasized is that the extracted OM ranges by picking up neighboring nodes from the map are actually gathering of a group of OM-DNAs, which means that the values for different OM types are inter-connected. Figure 45. shows how the extracted OM-DNAs are inter-connected within the extracted OM ranges.

Figure 45. Extracted OMs with top 3% of SA.

This means that by theory, when applying this extracted OM-ranges for CAE optimization, we cannot pick values from each OM ranges freely to form OM-DNA since they are inter-connected when extracting those OM-ranges.

So in this section, we tested the validity of this application by randomly forming a large number of OM-DNAs within the extracted OM ranges without considering their original inter-connections. And then assigning those formed OM-DNAs back to the map to find their winning nodes, so that we can check whether they are still within the desired area. Figure 46. shows the randomly formed OM-DNAs based on extracted OM ranges.

Figure 46. Randomly generated OMs with top 3% of SA.

If the winning nodes for those randomly formed OM-DNAs still stay within the desired area, then it means that those OM-DNAs still satisfy the SA target, and this application is valid.