Master’s Thesis Computer Science September 2012
School of Computing
Blekinge Institute of Technology SE – 371 79 Karlskrona
An Empirical Study of Machine Learning Techniques for Classifying Emotional
States from EEG Data
Ahmad Tauseef Sohaib
&
Shahnawaz Qureshi
This thesis is submitted to the School of Computing at Blekinge Institute of Technology in partial fulfillment of the requirements for the degree of Master of Science in Computer Science.
The thesis is equivalent to 2 x 20 weeks of full time studies.
Contact Information:
Authors:
Ahmad Tauseef Sohaib E-mail: sohaib@sohaib.me Shahnawaz Qureshi
E-mail: pioneer_taraus@yahoo.com
University Advisors:
Prof. Craig Lindley E-mail: craig.lindley@bth.se Assist. Prof. Johan Hagelbäck E-mail: johan.hagelback@bth.se
Game Systems and Interaction Research Laboratory (GSIL)
School of Computing, Blekinge Institute of Technology Karlskrona, Sweden
School of Computing
Blekinge Institute of Technology SE – 371 79 Karlskrona
Internet : www.bth.se/com Phone : +46 455 38 50 00 Fax : +46 455 38 50 57
A BSTRACT
With the great advancement in robot technology, smart human-robot interaction is considered to be the most wanted success by the researchers these days. If a robot can identify emotions and intentions of a human interacting with it, that would make robots more useful. Electroencephalography (EEG) is considered one effective way of recording emotions and motivations of a human using brain. Various machine learning techniques are used successfully to classify EEG data accurately. K-Nearest Neighbor, Bayesian Network, Artificial Neural Networks and Support Vector Machine are among the suitable machine learning techniques to classify EEG data.
The aim of this thesis is to evaluate different machine learning techniques to classify EEG data associated with specific affective/emotional states. Different methods based on different signal processing techniques are studied to find a suitable method to process the EEG data.
Various number of EEG data features are used to identify those which give best results for different classification techniques. Different methods are designed to format the dataset for EEG data. Formatted datasets are then evaluated on various machine learning techniques to find out which technique can accurately classify EEG data according to associated affective/emotional states.
Research method includes conducting an experiment. The aim of the experiment was to find the various emotional states in subjects as they look on different pictures and record the EEG data. The obtained EEG data is processed, formatted and evaluated on various machine learning techniques to find out which technique can accurately classify EEG data according to associated affective/emotional states. The experiment confirms the choice of a technique for improving the accuracy of results.
According to the results, Support Vector Machine is the first and Regression Tree is the second best to classify EEG data associated with specific affective/emotional states with accuracies up to 70.00% and 60.00% respectively. SVM is better in performance than RT.
However, RT is famous for providing better accuracies for diverse EEG data.
Keywords: Human Robot Interaction (HRI), EEG Data Classification, Machine Learning Techniques
A CKNOWLEDGEMENT
At first, we wish to thank Almighty ALLAH who is the most beneficent and merciful for His blessings to let us complete our thesis work. Then we like to thank everyone who helped and advised us for our thesis work.
We like to express our special thanks to our supervisors Prof. Dr. Craig Lindley and Assist.
Prof. Dr. Johan Hagelbäck for their continuous assistance and support through our thesis work. Their kind comments and detailed feedback let us complete this study in an improved manner.
We wish to praise the efforts of the members for Game Systems and Interaction Research Laboratory (GSIL) for their unconditional help in carrying out this study at each stage.
We also like to appreciate the contribution of the participants who took their time out for the experiment and let us collect the data for our thesis work. We would not be able to complete our thesis work without it.
At end, we like to thank our family and friends who have been supporting us morally to keep going with this study and inspired us to come up with expected results.
Sohaib & Shahnawaz
C ONTENTS
AN EMPIRICAL STUDY OF MACHINE LEARNING TECHNIQUES FOR
CLASSIFYING EEG DATA IN HUMAN-ROBOT INTERACTION ... I ABSTRACT ... I ACKNOWLEDGEMENT ... II CONTENTS ... III LIST OF FIGURES ... VI LIST OF TABLES ... VII LIST OF GRAPHS ... VIII LIST OF ABBREVIATIONS ... IX
1 INTRODUCTION ... 1
1.1 BRAIN ... 2
1.1.1 Human Brain ... 2
1.1.2 Structure of Brain ... 2
1.2 ELECTROENCEPHALOGRAPHY (EEG) ... 3
1.2.1 Types of Signals ... 3
1.2.2 Delta Waves (δ) ... 4
1.2.3 Theta Waves (θ) ... 4
1.2.4 Alpha Waves (α) ... 4
1.2.5 Beta Waves (β) ... 5
1.2.6 Gamma Waves (γ) ... 5
1.3 ACQUIREMENT OF EEGSIGNALS ... 5
1.3.1 Artifacts ... 6
1.3.2 10-20 System of Electrodes Placement ... 7
1.4 CLASSIFICATION OF EMOTIONS ... 9
1.4.1 Emotions Recognition Algorithms ... 9
1.5 MACHINE LEARNING TECHNIQUES TO CLASSIFY EEGDATA ... 10
1.5.1 K-Nearest Neighbor (KNN) ... 10
1.5.2 Regression Tree (RT) ... 11
1.5.3 Bayesian Network (BNT) ... 11
1.5.4 Support Vector Machine (SVM) ... 12
1.5.5 Artificial Neural Networks (ANN) ... 12
1.6 PSYCHOPHYSIOLOGICAL INTERACTION AND EMPATHIC COGNITION FOR HUMAN- ROBOT COOPERATIVE WORK (PSYINTEC) ... 13
2 BACKGROUND AND PROBLEM DEFINITION ... 15
2.1 PROBLEM FOCUSED ... 16
2.2 AIMS AND OBJECTIVES ... 16
2.3 RESEARCH QUESTIONS ... 16
2.4 EXPECTED OUTCOMES ... 17
3 RESEARCH METHODOLOGY ... 18
3.1 CONSTRUCTIVE RESEARCH ... 18
3.2 QUANTITATIVE RESEARCH ... 19
3.3 APPLICATION OF METHODS ... 19
3.3.1 Research Question 1 ... 19
3.3.2 Research Question 2 ... 19
3.3.3 Research Question 3 ... 19
4 THEORETICAL WORK ... 21
4.1 LITERATURE STUDY ... 21
4.1.1 BioSemi ActiveTwo System ... 21
4.1.2 EDF Browser ... 27
4.1.3 EEGLAB Toolbox for MATLAB ... 28
4.1.4 Waikato Environment for Knowledge Analysis ... 29
5 EMPIRICAL EVALUATION ... 31
5.1 EXPERIMENT CONTEXT AND OPERATIONS ... 31
5.1.1 Experiment Planning ... 31
5.1.2 Subjects Demographics ... 32
5.1.3 Experiment Preparation ... 32
5.1.4 Experiment Design and Execution ... 32
5.1.5 Experiment Limitation ... 33
5.2 THREATS TO VALIDITY ... 33
5.2.1 Conclusion Validity ... 34
5.2.2 Internal Validity ... 34
5.2.3 Construct Validity ... 34
5.2.4 External Validity ... 34
6 DATA ANALYSIS AND INTERPRETATION ... 35
6.1 EEGDATA COLLECTION ... 35
6.2 EEGDATA SCREENING ... 35
6.3 EEGDATA PROCESSING ... 35
6.4 EEGDATA FORMATTING ... 36
6.5 EEGDATA CLASSIFICATION ... 38
7 DISCUSSION ... 39
7.1 DISCUSSION FOR RESEARCH QUESTIONS ... 39
7.1.1 Research Question 1 ... 39
7.1.2 Research Question 2 ... 41
7.1.3 Research Question 3 ... 42
8 THE TOWER OF HANOI - PILOT STUDY ... 48
8.1 INTRODUCTION ... 48
8.2 AIMS AND OBJECTIVES ... 49
8.3 EXPECTED OUTCOMES ... 49
8.4 RESEARCH OPERATIONS ... 49
8.5 DISCUSSION ... 50
8.6 SUMMARY AND CONCLUSION ... 52
9 SUMMARY AND CONCLUSION ... 54
9.1 ANSWERS TO RESEARCH QUESTIONS ... 54
9.1.1 Research Question 1 ... 54
9.1.2 Research Question 2 ... 55
9.1.3 Research Question 3 ... 55 9.2 FUTURE WORK ... 55 10 REFERENCES ... 57 APPENDIX A: SELF-ASSESSMENT MANIKIN (SAM) AND INTERNATIONAL AFFECTIVE PICTURE SYSTEM (IAPS) ... 62 APPENDIX B: QUESTIONNAIRE ... 63
L IST OF F IGURES
Figure 1.1 – Different Parts of Human Brain ... 2
Figure 1.2 – An EEG Cap being used on a Participant to Record Brain Activity ... 3
Figure 1.3 – EEG Signals ... 3
Figure 1.4 – Delta Wave ... 4
Figure 1.5 – Theta Wave ... 4
Figure 1.6 – Alpha Wave ... 5
Figure 1.7 – Beta Waves ... 5
Figure 1.8 – Gamma Waves ... 5
Figure 1.9 – Acquirement of EEG Signals using Headcap and Active-Electrodes ... 6
Figure 1.10 – Schematic of the EEG Recording System ... 6
Figure 1.11 – Removal of Artifacts from EEG Signals ... 7
Figure 1.12 – 10-20 System of Electrodes Placement ... 8
Figure 1.13 – Location and Nomenclature of the Intermediate 10% Electrodes ... 8
Figure 1.14 – Emotion Labels in Arousal-Valence Dimension ... 9
Figure 1.15 – Classification using KNN ... 10
Figure 1.16 – Example of Regression Tree ... 11
Figure 1.17 – Structure of BNT ... 12
Figure 1.18 – Maximum-Margin Hyperplane and Margins for an SVM ... 12
Figure 1.19 – An Artificial Neural Network ... 13
Figure 1.19 – Functional Architecture of the PsyIntEC System ... 13
Figure 3.1 – Constructive Research Diagram ... 18
Figure 4.1 – BioSemi ActiveTwo System ... 21
Figure 4.2 – Flat-Type Active-Electrodes ... 22
Figure 4.3 – Pin-Type Active-Electrodes ... 22
Figure 4.4 – Pin-Type Active-Electrodes ... 23
Figure 4.5 – BioSemi Headcap with Electrode Holders and Active-Electrodes ... 23
Figure 4.6 – Filling of Electrode Gel into Electrode Holders by Syringe ... 24
Figure 4.7 – Front of USB2 Receiver ... 24
Figure 4.8 – Back of USB2 Receiver ... 25
Figure 4.9 – Analog Input Box (AIB) ... 25
Figure 4.10 – 8 Touchproofs for EOG, EMG, ECG ... 25
Figure 4.11 – ActiveTwo AD-box with Battery ... 26
Figure 4.12 – ActiView BioSemi Acquisition Software ... 26
Figure 4.13 – EEG Data Acquisition using ActiView ... 27
Figure 4.14 – EDF Browser ... 27
Figure 4.15 – Manipulating EEG Signals and their Annotations ... 28
Figure 4.16 – EEGLAB Toolbox ... 28
Figure 4.17 – Waikato GUI Chooser ... 29
Figure 4.18 – Weka Explorer ... 29
Figure 5.1 – Experiment Stages ... 31
Figure 6.1 – EEG Data Collection ... 35
Figure 6.2 – EEG Data Screening ... 35
Figure 6.3 – EEG Data Processing ... 36
Figure 6.4 – EEG Data Formatting ... 38
Figure 6.5 – EEG Data Classification ... 38
Figure 8.1 – The Tower of Hanoi Puzzle ... 48
Figure 8.2 – Research Operations ... 50
L IST OF T ABLES
Table 1.1 – Frequency and Mental States of Waves ... 4
Table 6.1 – Formatting Dataset for Different Emotions in Subject 1 using Model A ... 37
Table 6.2 – Formatting Dataset for Different Emotions in Subject 1 using Model B ... 37
Table 7.1 – Contrast of Selected Machine Learning Techniques ... 41
Table 7.2 – Datasets and their Description while Passing to WEKA ... 42
Table 7.3 – Classification Accuracies by Selected Techniques for Dataset A ... 42
Table 7.4 – Classification Accuracies by Selected Techniques for Dataset B ... 43
Table 7.5 – Comparison of Model A and Model B Based on Findings ... 45
Table 7.6 – Division of Dataset B ... 45
Table 7.7 – Datasets and their Description while Passing to WEKA ... 45
Table 7.8 – Classification Accuracies by Selected Techniques for Dataset 1B, 2B and 3B .. 45
Table 7.9 – Passing Datasets to WEKA for Each Subject Individually ... 46
Table 7.10 – Classification Accuracies by Selected Techniques for Subject 1, 2 and 3 ... 46
Table 8.1 – Dataset and its Description while Passing to WEKA ... 50
Table 8.2 – Classification Accuracies by Selected Techniques for Dataset TB ... 50
Table 8.3 – Comparison of Classification Accuracies by Selected Techniques ... 51
Table 9.1 – Classification Accuracies by Selected Techniques ... 55
L IST OF G RAPHS
Graph 7.1 – Contrast of Selected Machine Learning Techniques ... 40
Graph 7.2 – Classification Accuracies by Selected Techniques for Dataset A ... 43
Graph 7.3 – Classification Accuracies by Selected Techniques for Dataset B ... 43
Graph 7.4 – Comparison of Results Obtained Due to Dataset Formatting ... 44
Graph 7.5 – Classification Accuracies by Selected Techniques for Dataset 1B, 2B and 3B . 46 Graph 7.6 – Classification Accuracies by Selected Techniques for Subject 1, 2 and 3 ... 47
Graph 8.1 – Classification Accuracies by Selected Techniques for Dataset TB ... 51
Graph 8.2 – Comparison of Classification Accuracies by Selected Techniques ... 52
L IST OF A BBREVIATIONS
AIB Analog Input Box
ANN Artificial Neural Networks
ARFF Attribute-Relation File Format
BNT Bayesian Network
BDF BioSemi Data Format
EKG Electrocardiography
EEG Electroencephalography
EMG Electromyography
ECHORD European Clearing House for
Open Robotics Development
GSIL Game Systems and Interaction
Research Laboratory
HRI Human-Robot Interaction
HCI Human-Computer interaction
ICA Independent Component Analysis
IAPS International Affective Picture
System
KNN K-Nearest Neighbor
LabVIEW Laboratory Virtual Instrument
Engineering Workbench PsyIntEC
Psychophysiological Interaction and Empathic Cognition for Human-Robot Cooperative Work
RT Regression Tree
SAM Self-Assessment Manikin
SVM Support Vector Machine
ToH The Tower of Hanoi
WEKA Waikato Environment for
Knowledge Analysis
1 I NTRODUCTION
Human-computer interaction (HCI) is the study of interaction among people and computers.
HCI aims to produce a design that should produce a good fit among the user, the machine and the required services in order to achieve a certain performance in terms of both quality and optimality of the services [1]. HCI is one of the most active research areas in computer science presently [2]. Modern types of human-driven and human-centric interaction with digital media have opened new possibilities to modernize the different areas of human life such as learning, working etc. Human-computer interface applications have increased the significance of emotion recognition as emotions are vital in daily life of humans [3] [4].
Human-robot interaction (HRI), a subfield of HCI, is of increasing importance as people and robots perform various works together [5]. It is now becoming feasible to integrate this technology into real-world, real-time systems to enhance human-machine interaction across a wide range of application domains including clinical, industrial, military and gaming applications [6] [7] [8] [9]. HRI is one of the significant fields under robots community. To achieve robust interaction of robots with human, robots must have proficient components of HRI [2]. It is vital for a service robot to deduce the co-worker for his/her needs in order to achieve efficient results.
Electroencephalography (EEG) is one effective way of recording and analyzing the brain activity due to its ease of use, affordable cost and fine resolution. Due to brain activity neurons get fired producing higher electrical potential which is recorded by the electrodes attached to the scalp of a human being. The measures differ due to varying levels of cognitive stimuli [10]. EEG is beneficial due to high temporal resolution which helps to record variations in cognitive activity based on millisecond gauge. Hence, EEG measurements are insight of the cognitive situations of the participant [11]. However, EEG is sensitive to noise from electrical equipment, mostly useable in controlled lab environments and not so effective in real-world situations.
In recent years, there has been great advancement in robot technology which has introduced different fields of applications such as space, war field and assistance in work etc. However, current robots are not completely dependable or autonomous to perform a certain tasks especially when it comes to a robot working with a human co-worker. Hence there is a need of robotic systems that are capable of understanding human emotions, desires and intentions which will ultimately enhance the performance of certain tasks. This research has been motivated by these needs and is an attempt to take a step in this direction.
Psychophysiological Interaction and Empathic Cognition for Human-Robot Cooperative Work (PsyIntEC) is “a feasibility demonstration project targeting advances that address safe ergonomic and empathetic adaptation by a robotic system to the needs and characteristics of a human co-worker during collaborative work in a joint human-robot work cell” [12]. The human co-worker is the source of psychophysiological or biometric data that is input to a robot control system to provide the basis for affective and cognitive modeling of the human by the robot as a basis for behavioral adaption [12].
Different techniques are needed for the classification of psychophysiological data, i.e., taking signals from a human, including EEG and creating interpretations of these in terms of emotional and attention states. This involves acquiring EEG data and processing it, formatting the dataset and evaluating it using different technical approaches of machine learning. The results of the evaluation are presented which can be used to develop the
classifier system for the PsyIntEC project. The developed classifier will be part of the robot software.
1.1 Brain
To help understanding the further discussion by the readers, basic structure and function of human brain are explained under this section.
1.1.1 Human Brain
Brain is the most complex organ of human. All kind of physical tasks are led by it. It contains a neural network of 10 billion nerve cells, neurons. It handles feelings, hunger, thirst, body movements and sleep functions of human. It controls almost all the core activities required for the survival of a human. It communicates with body parts environment by sending and receiving signals. It is the core of central nervous system having brainstem, spinal cord and large brain as described in the Figure 1.1. Spinal cord and large brain are connected through brainstem. It is divided into three different parts based on its anatomy and functionality.
Figure 1.1 – Different Parts of Human Brain [13]
1.1.2 Structure of Brain
Based on anatomy, brain is divided into three parts as hind brain, mid brain and forebrain. At first part, myelencephalon exists in hind brain whereas cerebellum and fourth ventricle are located above it with spinal cord. At second part, mesencephalon exits in mid brain and contains tectum, tegmentum and cerebral aqueduct. At third part, diencephalon and telecephalon exits in fore brain [13].
Based on functionality, brain is divided into three parts. First part is called as forebrain; also known as cerebrum or large brain and handles high level mental tasks such as computational thinking etc. Second part is called as brainstem and controls the visual functions. Third part is called as cerebellum and is responsible for body movements.
1.2 Electroencephalography (EEG)
Electroencephalography (EEG) is a signal representation of brain activity. The signal waves hold the valuable information about the state of brain. It is one of the non-invasive techniques for brain imaging which provides electrical potential recording for the surface of the scalp due to the electrical activity of the large collections of neurons in the brain [14].
Non-invasive is a technique in which the body is not invaded or cut open as during surgical investigations or therapeutic surgery. Invasive technique is opposite to non-invasive.
Figure 1.2 – An EEG Cap being used on a Participant to Record Brain Activity [15]
1.2.1 Types of Signals
EEG signals are defined in term of rhythmic and transient and are complex signals as shown in Figure 1.3. The rhythmic activity is distributed into different frequency bands. Different people of different ages may have different amplitude and frequency of EEG signals while they are recorded in different states such as performing a task or relaxing.
Figure 1.3 – EEG Signals [16]
Based on frequency ranges, five types of waves can be identified. They are alpha (α), theta (θ), beta, (β), delta (δ) and gamma (γ) from low to high frequency respectively.
A specific wave is mostly available in specific lobe of cerebral cortex however this is not always true. Different mental states are associated with different waves which is helpful to define one’s situation at a specific time as described in the Table 1:
Wave Frequency (Hz) Mental State
Delta (δ) 0 – 4 Deep Sleep
Theta (θ) 4 – 8 Drifting Thoughts, Dreams, Creativity Alpha (α) 8 – 13 Calmness, Relaxation, Abstract Thinking
Beta (β) 13 – 30 Highly Focused, Highly Alertness Gamma (γ) > 30 Simultaneous Process, Multi-Tasking
Table 1.1 – Frequency and Mental States of Waves
1.2.2 Delta Waves (δ)
Delta waves are under the frequency range of 0 – 4 Hz. Mental states associated with these waves are deep sleep, coma or hypnosis and sometimes awake. In awake state, it is always considered to be pathological phenomenon. The higher is the amplitude, higher serious is the problem considered. These waves are decreased by the age and are normally present in healthy people in their awake state.
Figure 1.4 – Delta Wave [16]
1.2.3 Theta Waves (θ)
Theta waves are under the frequency range of 4 – 8 Hz. Mental states associated with these waves are drifting thoughts, creative thinking and unconscious materials. These waves appear in central, temporal and parietal parts of head. These waves are normally present in healthy people while they are in deep sleep.
Figure 1.5 – Theta Wave [16]
1.2.4 Alpha Waves (α)
Alpha waves are under the frequency range of 8 – 13 Hz. Mental states associated with these waves are relaxed and calm states. These waves appear on back side of head and occipital area of brain. These waves are of high amplitude as compared to others. This can be observed while subject is awake and clam. Sometimes, these waves interfere with µ-rhythm.
These waves are normally present in people while they are calm and relax being in awake state.
Figure 1.6 – Alpha Wave [16]
1.2.5 Beta Waves (β)
Beta waves are under the frequency range of 13 – 30 Hz. Mental states associated with these waves are highly focused and alertness, such as during deep thinking and concentration. Beta waves are having large band of frequency as compared to others. These waves appear at front side of head and central area of brain.
Figure 1.7 – Beta Waves [16]
1.2.6 Gamma Waves (γ)
Gamma waves are under the frequency range of 30 Hz. Mental states associated with these are simultaneous work and multi-tasking. These waves are hard to notice due to their very low amplitude. These waves appear in each part of brain.
Figure 1.8 – Gamma Waves [16]
1.3 Acquirement of EEG Signals
EEG signals are acquired from sculp. Signals are measured using electrodes stick to head.
Calculation of potential difference between two electrodes is the basic principle of EEG. One or more electrodes are attached either to mastoids or ear lobes. They are called reference electrodes. These electrodes help to find the background electric field of skull. The location of reference electrodes is very important. They should not be located neither very close to brain nor at any other part of body as signals can possibly be affected by the electrical activity of muscles or heart [17].
Figure 1.9 – Acquirement of EEG Signals using Headcap and Active-Electrodes [15]
Different EEG devices are available with certain number of electrodes and filters for them.
These devices help to acquire analog EEG signals which are then altered to digital data and sampling frequency. Filters help to remove the artifacts. Unwanted frequencies are ignored using low pass and high pass filters by removing signals such as Electromyography (EMG) and Electrocardiography (EKG) [18]. The data resolution is also important during data processing. Hence, sampling frequency, sampling rate and number of electrodes are important factors while recording EEG signals.
Figure 1.10 – Schematic of the EEG Recording System [19]
Different electrodes acquire EEG data using different methods. Following electrodes are normally used to record EEG data:
• Disposable Electrodes (Gelled/Pre-Gelled)
• Reusable Electrodes
• Electrodes Caps (Headbands)
• Saline-Based Electrodes
• Needle Electrodes
1.3.1 Artifacts
Artifacts are unwanted signals due to noise from; for example electric circuits. These are not due to brain activity rather affecting the signal measurement making it difficult for analysis.
There are different types of artifacts. One of the main artifacts is due to impedance of system and another is sampling frequency artifact which is 50 Hz and 60 Hz caused due to ground loop. The importance of artifacts and their removal can be explained better by the Figure 1.11 as:
Figure 1.11 – Removal of Artifacts from EEG Signals [19]
(A) The raw EEG signal having large artifacts.
(B) The averaged imaging artifact.
(C) The result of subtracting the averaged imaging artifact in B from the EEG in A, followed by down-sampling and showing Pulse artifact.
(D) The averaged pulse artifact from trace C (not to scale).
(E) Result of subtracting the averaged pulse artifact in D from the EEG in C.
(F) The EEG from the same subject, recorded outside the scanner, i.e., free of imaging and pulse artifact. The character of this EEG appears to match closely the artifact corrected trace in E.
However, some of the artifacts are useful. Biological signals such as EMG and EKG can help to predict different mental states. Such as, EMG artifact which is due to eye blinking can provide information about sleep or awake states.
1.3.2 10-20 System of Electrodes Placement
One of the commonly used methods of electrodes placement is 10-20 System for recording EEG signals which is standardized by the American Electroencephalographic Society. Using this system, a total of 21 electrodes are placed on scalp as shown in A of Figure 1.9.
Location of placement for electrodes is as:
Nasion: Electrodes placed at level of eyes and top of nose are Reference Points, Nasion.
Inion: Electrodes placed on midline at backside of head and base of skull are Inion.
Parameters of skull are measured from above points. Locations for electrodes are determined by division of parameters to intervals of 10% and 20%. Three electrodes are placed on middle of adjacent points as shown in B of Figure 1.9 [20]. Relationship between the location of an electrode and cerebral cortex is the basis of 10-20 system. Electrode letters are used to determine the placements such as:
A – Ear Lobe
C – Central Lobe F – Frontal Lobe Fp = Frontal Polar O – Occipital Lobe P – Parietal Lobe Pg = Nasopharyngeal T – Temporal Lobe
A combination of letter(s) and an integer number is used to determine the placement.
Figure 1.12 – 10-20 System of Electrodes Placement [20]
Seen from (A) Left and (B) Above the Head
Figure 1.13 – Location and Nomenclature of the Intermediate 10% Electrodes [20]
In addition to 10-20 system, there are many other systems available to record EEG signals.
The Queen Square system of electrode placement was proposed as standard for recording electric potentials on the scalp [20]. EEG measurement can also be carried out using bipolar
or unipolar electrodes. In case of bipolar, potential different among pair of electrodes are measured whereas in case of unipolar, average of all electrodes is compared with potential of every electrode [20].
1.4 Classification of Emotions
Various systems for classification of emotions exist. This classification can be observed in two aspects as dimensional and discrete [21]. According to Plutchik, there are eight basic states of emotion as acceptance, anger, anticipation, disgust, fear, joy, sadness and surprise.
Rest of the emotion states can be model using the basic states such as sadness and surprise make disappointment [22].
Considering dimensional aspects, the commonly used classification system is bipolar model, proposed by Russells [23] which considers arousal and valence dimensions. In this case, valance dimensions are from negative to positive whereas arousal dimensions are from not aroused to excited. The dimensional model is advantageous for emotion recognition because it can determine discrete emotions in its space even if no certain label can be used to determine a specific feeling [21] [24]. The dimensional model is the most commonly used model for classification of emotions [23].
Figure 1.14 – Emotion Labels in Arousal-Valence Dimension [25]
(Circumplex Model of Russell)
1.4.1 Emotions Recognition Algorithms
There are many algorithms available for emotions recognition. There is much ongoing research on EEG-based emotion recognition algorithms. Different methods are employed to extract features and classify data into different emotion modes such as joy, sadness, anger and pleasure etc. However, none of the algorithm resulted in good accuracy [26].
Emotion recognition is an immature field without standards availability for different EEG signals for emotions. Limited types of emotions are identified until yet and emotion algorithms lack to classify them accurately [26].
1.5 Machine Learning Techniques to Classify EEG Data
Machine Learning is a branch of artificial intelligence and aims to identify unknown samples by learning from known sample. There are different machine learning techniques available with their own advantages and disadvantages for use. Currently, Artificial Neural Network, Genetic Algorithm and Support Vector Machine are the most commonly used to classify EEG data [27].
Various machine learning techniques are used to classify EEG data and some of them are successful but lacks in accuracy. However, the following machine learning techniques are considered in most of empirical studies:
• K-Nearest Neighbor (KNN)
• Regression Tree (RT)
• Bayesian Network (BNT)
• Support Vector Machine (SVM)
• Artificial Neural Networks (ANN)
The first four techniques are the most popular techniques [5] and are most widely used.
1.5.1 K-Nearest Neighbor (KNN)
K-Nearest Neighbor (KNN) is one of the most basic and simple classification techniques [28]. KNN is usually considered when there is no or very little knowledge available for the distribution of data. It is a potential non-parametric classification technique which completely bypasses the problem of probability densities [29].
While classifying using KNN, X is assigned with a label of most frequently represented between K-nearest samples. This determines that labels on KNN are examined; voting has been carried out and a decision is made. KNN classification technique was developed for the need to carry out discriminant analysis while reliable parametric estimates of probability densities are either unknown or difficult to find out [28].
Figure 1.15 – Classification using KNN [30]
Figure 1.15 shows classification performed using KNN; ‘.’ denotes the prototypes of the same class as y, and ‘x’ denotes the prototypes of the class different from y [30] .
1.5.2 Regression Tree (RT)
Regression Tree (RT) takes an object or situation as input which is categorized a number of properties and provides a decision as an output [31]. One input feature is represented by every node and possible test result values are represented by the branches of node. The positive test values can either be positive or negative. If that node is reached, leaf nodes, also called as terminal nodes; represent the decision value.
Figure 1.16 – Example of Regression Tree [32]
Regression tree is widely used in medical field for classification such as speech recognition, heart attack and cancer diagnosis [33] [34].
1.5.3 Bayesian Network (BNT)
Bayesian Network (BNT) is the most effective classifier in terms of predictive performance and state-of-the-art classifiers [35]. BNT is a graph that contains a network structure N which put a set of conditional independence relations between a set of variables C = {A1, A2
… An} and a set of Tables T of local probability distributions associated with every variable.
The joint probability distribution of A is determined by N and T. Network nodes have one-
to-one contact with A variables. Therefore in N, single nodes are denoted by Ai whereas parent nodes are indicated by Pi.
Figure 1.17 – Structure of BNT [35]
1.5.4 Support Vector Machine (SVM)
Support Vector Machine (SVM) is a linear machine that operates in k-dimensional space created due to n-dimensional input data X into k-dimensional space using non-linear mapping ȹ(X). This helps to isolate data normally by geometry and linear algebra. To find a linear classifier for any data points with known class labels, it can be done by identifying a separating hyper plane.
Figure 1.18 – Maximum-Margin Hyperplane and Margins for an SVM Trained with Samples from Two Classes [36]. Samples on the margin are called the Support Vectors.
SVM was proposed by Vapnik [36] and can deal with over fitting by having misclassified instances on training data. Due to this, SVM is more appropriate for affect recognition because physiology data is noisy [5].
1.5.5 Artificial Neural Networks (ANN)
Artificial Neural Networks (ANN) has the capability of finding a nonlinear transformation of the pattern in order to classify more accurately [37].
Figure 1.19 – An Artificial Neural Network [37]
Artificial neurons are interconnected in a neural network and process information with connected neurons. ANN molds itself and changes its structure based on the internal and external information of network while learning.
1.6 Psychophysiological Interaction and Empathic Cognition for Human-Robot Cooperative Work (PsyIntEC)
Psychophysiological Interaction and Empathic Cognition for Human-Robot Cooperative Work (PsyIntEC) is “a feasibility demonstration project targeting advances that address safe ergonomic and empathetic adaptation by a robotic system to the needs and characteristics of a human co-worker during collaborative work in a joint human-robot work cell” [12]. The human co-worker is the source of psychophysiological or biometric data that is input to a robot control system to provide the basis for affective and cognitive modeling of the human by the robot as a basis for behavioral adaption [12].
Figure 1.19 – Functional Architecture of the PsyIntEC System [12]
PsyIntEC extends the European Clearing House for Open Robotics Development (ECHORD) state of human-robot co-worker and hyper-flexible cells with focus on robot hands and complex management in cooperative human-robot tasks [12]. The industry targeted is a small medium enterprise involved in prototyping of novel devices.
PsyIntEC is aimed to focus on demonstration of robots feasibility to guide, support and facilitate the production of human-robot co-worker prototype. It also focuses on human emotions to assess its intentions in order to proceed the required task accordingly. This is helpful to maintain optimum level of attention among robot and human co-worker by using biometric data to find human emotional states [12].
PsyIntEC is an ongoing EU funded research project at Blekinge Institute of Technology. It was required to develop a classifier system for PsyIntEC. The developed classifier will be part of the robot software. This study has evaluated different technical approaches of machine learning and presented the results of evaluation to help the selection of most suitable technique for the classifier system for PsyIntEC.
2 B ACKGROUND AND P ROBLEM D EFINITION
In last few years, robots have been developed to work together with humans in various environments. For example, entertainment is provided by the humanoid robots while patient assistance is provided by the haptic robots. However, it is hard to evaluate and interpret the interaction among human and robot [38].
Human interaction with computer applications is part of everyday life. Similarly, emotions are vital in everyday life of humans. Hence there is a growing need of emotion recognition to help human interacting with computer interfaces quickly and easily. Researchers have made successful emotion recognition using text, speech and facial expressions or gesture [26].
Emotions accompany everyone in the daily life, playing a key role in non-verbal communication, and they are essential to the understanding of human behavior. Emotion recognition could be done from the text, speech, facial expression or gesture. However, most of the researches have been carried out on emotion recognition from EEG [3] [39] [40] [41]
[42] [4].
Human machine interaction on the base of physiological signals has been investigated by much past and recent research. Of particular interest are systems that can make interpretations about psychological states based upon physiological data. Linear classifiers [43][44][45] are considered to be the most appropriate classification technology due to their simplicity, speed and interpretability. However, non-linear classifiers are considered to be the most appropriate when it comes to signal features and cognitive state according to [46]
[47].
Sequential Floating Forward Search and Fisher Projection methods are used by Picard and colleagues to classify eight emotions with 81% accuracy [48]. Lisetti and Noasoz used Marquardt Back propagation, Discriminant Function Analysis and K-Nearest Neighbor to distinguish between six emotions and acquired the correct classification in 83%, 74% and 71% [49]. According to [50], probabilistic models can be developed using a methodology provided which uses various body expressions of the user, personality of user and context of the interaction. Mental workload has been evaluated using Artificial Neural Networks providing mean classification accuracies of 85%, 82% and 86% for the baseline, low task difficulty and high task difficulty states respectively [51]. In [43] an emotion-recognizer based on Support Vector Machines has been analyzed which provided accuracies of 78.4%
and 61.8%, 41.7% for recognition of three, four and five emotions categories respectively.
According to [5], if the same physiological data is used then Support Vector Machines with a classification accuracy of 85.81% perform the best, closely followed by the Regression Tree at 83.5%, K-Nearest Neighbor at 75.16% and Bayesian Network at 74.03%. Performance of K-Nearest Neighbor and Bayesian Network algorithms can be improved using informative features [5]. According to [5], Support Vector Machine shows 33.3% and 25% accuracy for three and four emotion categories respectively when it comes to physiological signal databases acquired from ten to hundreds of users.
Due to different experiment environment, data and pre-processing techniques used in different studies, it is not easy to compare results concluded by each study. However, studies show that various factors such as pre-processing and classification techniques can strongly affect the results and improve their accuracy. This means, a certain tradeoff can lead to desired results however a strong decision is required to make the best one.
However several methods are used successfully to develop affect recognizers from physiological indices; it is still required to select an appropriate method for the classification
of EEG data to attain uniformity in various aspects of emotion selection, data collection, data processing, feature extraction, base lining, and data formatting procedures.
Looking at the current research, it has been found that, various methods are successfully applied to develop different classifiers for the correct classification of physiological data.
However, the classifiers are generally not available and there is always both room for improvement and the need to develop classifiers suitable for the needs of specific applications.
In this case, for the PsyIntEC project, we needed to choose an appropriate method to classify EEG data. This involved taking EEG data, processing it to extract features and formatting the dataset to evaluate the dataset using different technical approaches of machine learning.
The results of the evaluation can be used to develop the classifier system for PsyIntEC project. The developed classifier will be part of the robot software.
2.1 Problem Focused
The study focuses to find a suitable technique for classifying EEG data. To deal with the problem focused, standard methods are used for different steps involved such as recording EEG data, processing it to extract features and formatting dataset. Different parameters have been modified in order to achieve better results. The formatted dataset is then evaluated for different machine learning techniques to achieve the desired results.
2.2 Aims and Objectives
The main aim of this research work was to evaluate different machine learning techniques to classify EEG data associated with affective/emotional states along with the followings:
• Find a suitable method to process the EEG data based on different signal processing techniques.
• Identify key features that can represent emotional states better than raw EEG data.
• Find a method to shape the dataset for EEG data to classify it.
• Identify and evaluate most commonly practiced techniques used to classify EEG data associated with specific affective/emotional states.
2.3 Research Questions
In contribution to this research work, we have developed the following research questions based on the aims and objectives:
RQ1: Which techniques are available in current literature for classifying EEG data associated with specific affective/emotional states?
RQ2: What EEG data features give the best results for different classification techniques?
RQ3: How accurately can EEG data be classified according to associated affective/emotional states using the selected techniques?
2.4 Expected Outcomes
Expected outcome of this research work was identification of the most appropriate machine learning technique to classify EEG data according to associated affective/emotional states.
This included:
• List of most commonly practices techniques used to classify EEG data.
• List of EEG data features which helps to classify EEG data more accurately.
• Analysis of challenges and issues based on software and hardware used.
• Identification of the most appropriate technique for the correct and most accurate classification of EEG data.
Discussion and conclusion about how the proposed technique can be used to classify EEG data associated with specific affective/emotional states.
3 R ESEARCH M ETHODOLOGY
3.1 Constructive Research
Constructive Research is a common research methodology of computer science. As compared to other types of research, this one is not required to be validated quite as empirically as others [52]. Using this methodology, a system is developed and then evaluated. The aim is to construct artifacts and knowledge for them using practical potential value [53].
According to Crnkovic [54], constructive research method aims to construct an artifact (practical, theoretical or both) which solves a domain specific problem to build knowledge about how the problem can be solved (understood or explained) and provides with results in relevant to practical and theoretical.
“Construct” as a term means a new constructed contribution. It can be a new algorithm, technique, framework or theory. From the case study by Lukka [55], constructive research can be considered as a form of conducting case research parallel to ethnography, grounded theory, theory illustration, theory testing and action research.
Figure 3.1 – Constructive Research Diagram [52]
In constructive research, information is collected from various sources such as articles, literature, tutorials etc. Such sources help to acquire theoretical knowledge. Using this theoretical knowledge, solution to a problem is derived. This derived solution derives new knowledge as explained in the Figure 3.1.
Our research work involved recording EEG signals using EEG headcap and BioSemi ActiveTwo recording device. The artifacts/innovative constructs developed and used in this setup are electrodes and EEG interface. Target knowledge of setup is human emotions as positive/negative arousal/valence. Hardware and software components used to acquire the assistance in completion of required tasks from the setup. Hardware and software
components have their complete documentation available and that is summarized in coming chapter.
3.2 Quantitative Research
In order to evaluate the validity of the developed construct, we have conducted an experiment. The theory obtained by constructive research was verified by quantitative methods. A standard experiment design was used to conduct experiment and collect the data from it. Collected data was then processed to extract features; format the dataset then evaluated using different machine learning techniques in order to test the theory. During the whole process, different factors affecting the process were identified and improvements are determined. This has motivated for the use of quantitative research [56].
3.3 Application of Methods
This section clarifies the research operations and how the application of selected methods helped to answer all the research questions.
3.3.1 Research Question 1
Which techniques are available in current literature for classifying EEG data associated with specific affective/emotional states?
Literature has been studied in order to find the answer to this question. Research on classification of EEG data is explored. The literature collected was filtered in order to have research which work with classification of data associated with specific affective/emotional states. The filtered literature has been used to extract techniques used in them. While extracting the techniques, researches using quantitative methods were preferred keeping the validity constraints in consideration.
3.3.2 Research Question 2
What EEG data features give the best results for different classification techniques?
Literature has been studied to acquire the answer to this question. Research based on EEG data are being searched specially those which involved recording of EEG data and processing it later. The collected literature was filtered to look into the research which worked with either 4 or 6 emotions. This has been considered in order to narrow down the problem domain and achieve better results. The features reporting best results for the selected techniques were extracted from the filtered literature. Different parameters used for those features and classification techniques applied on them were also extracted.
3.3.3 Research Question 3
How accurately can EEG data be classified according to associated affective/emotional states using the selected techniques?
An experiment has been conducted in order to get the answer to this question. The experiment confirms the choice of a technique and selection of EEG data features for improving the accuracy of results. Hence, experiment actually validates the answer of both
RQ1 and RQ2; and answers RQ3. The experiment involved steps for planning, preparation, execution and reporting of results.
Data collection was performed during the experiment using the device, BioSemi ActiveTwo System [57] and software, ActiView [57]. Data analysis was performed using the software, EDF Browser [58] and EEGLAB Toolbox for MATLAB [59]. EEG data classification is performed using the software, Waikato Environment for Knowledge Analysis (WEKA).
Software components used during the experiment and analysis are described in the next chapter.
4 T HEORETICAL W ORK
4.1 Literature Study
4.1.1 BioSemi ActiveTwo System
BioSemi ActiveTwo System is a widely used system to achieve advanced EEG data and uses an active-electrode technology with open architecture [60]. The system is an improved version of ActiveOne and purely designed for research purposes. It features multi-channel and supports high resolution measurements for bio-potentials. With the help of active- electrode technology, requirements like impedance measurement, skin preparation and faraday cage are not required to be used [57].
Figure 4.1 – BioSemi ActiveTwo System [61]
Pin-Type Active Electrodes and Headcap on Head of Subject
ActiveTwo AD-box with Battery on Back of Subject and Connected to Computer The engineers Robert Honsbeek, Ton Kuiper and physicist Coen Metting van Rijn have introduced BioSemi system for researches in 1998 at Medical Physics department of the University of Amsterdam. Different projects such as AGN1667, AGN3416 and AGN4098, funded by Technology Foundation STW [62]; are using BioSemi system as well as various researches are published using it. Foundation Vision Research Amsterdam [63] has tested various small prototypes using this system and made results available to more than 40 scientific and clinical workers in 9 different countries.
BioSemi ActiveTwo System used in the experiment involved the following components:
• BioSemi Active-Electrodes
• BioSemi Headcaps
• ActiveTwo AD-box
• USB2 Receiver
• ActiView
4.1.1.1 BioSemi Active-Electrodes
BioSemi active-electrode is a low output impedance sensor that discards any coupling of sources or interferences and artifacts caused by any medium. It helps to lower the level of noise to thermal noise level due to electrode material based on sintered Ag-AgCl [57].
Figure 4.2 – Flat-Type Active-Electrodes [57]
It is alcohol and water resistant. It has an input protection circuit that shields electronic amplifier from static discharge and defibrillator pulses. It can fulfill the needs for EEG, Electrocardiography (ECG) and Electromyography (EMG). BioSemi active-electrodes are either flat-type active-electrodes or pin-type active-electrodes [57].
Figure 4.3 – Pin-Type Active-Electrodes [57]
BioSemi pin-type active-electrode is specialized to be used with BioSemi headcap. It is capable of fitting into headcap having BioSemi electrode holders. It has sintered Ag-AgCl electrode on tip that provides low noise and offset voltages along with stable DC performance [57]. It is alcohol and water resistant. These electrodes are labeled with water resistant numbers to identify each channel. Standard set of these have 32 electrodes and 140 centimeter of cable length with a common connector. These electrodes hold fast application time [57].
Figure 4.4 – Pin-Type Active-Electrodes [57]
4.1.1.2 BioSemi Headcaps
BioSemi headcap is a plain elastic cap with plastic electrode holders and without any electrodes or wires attached. The electrode holders allow easy and fast placement of electrodes into them; making it 30 minutes for a 128 channel EEG headcap to be ready for EEG measurement. No skin preparation is required to use this headcap because high impedances do not influence the signal quality due to active-electrode principal [57].
Standard BioSemi headcaps are available in different sizes and have ear-slits for easy adjustment of headcaps over the ears area.
Figure 4.5 – BioSemi Headcap with Electrode Holders and Active-Electrodes into them [61]
While setting up the headcap for EEG measurement, the headcap is sited at the head of subject; electrode gel is filled into electrode holders using a syringe and active-electrodes are inserted into the electrode holders.
Figure 4.6 – Filling of Electrode Gel into Electrode Holders by Syringe [61]
Dr. Peter Praamstra has developed the BioSemi headcap at Behavioral Brain Science Center, University of Birmingham, United Kingdom [57].
4.1.1.3 USB2 Receiver
USB2 receiver takes the optical data from the AD-box and converts it to USB2 output. It has a trigger port with 16 independent triggers input and outputs. This keeps the subject separate from the stimulation setup. The triggered output signals are manipulated using BioSemi acquisition software, ActiView.
Figure 4.7 – Front of USB2 Receiver [61]
USB2 receiver is a plug and play device with complete reliable data throughput. A 32 channels analog input box with 24 bits ActiveTwo system having 256 channels at 4096 kHz has a total data throughput of 3.54 megabyte per second [57]. Using USB2 receiver, ActiveTwo system can be used with either desktops or notebooks. LED indicator with USB2 receiver helps to identify the incoming data.
Figure 4.8 – Back of USB2 Receiver [61]
Figure 4.9 – Analog Input Box (AIB) [61]
4.1.1.4 ActiveTwo AD-Box
ActiveTwo AD-box is the front-end of BioSemi system. It is ultra-compact, low power and can digitize 256 sensor-signals at 24 bits resolution. These sensors can be of various types such as active electrodes or bufferboxes with normal passive electrodes etc.
Figure 4.10 – 8 Touchproofs for EOG, EMG, ECG [57]
Every AD-box channel contains a low noise DC coupled post amplifier, first order anti- aliasing filter, steep fifth order sinc response and output of 24 bits high resolution. Digital output from all AD converters are digitally multiplexed and sent uncompressed and unreduced data is sent to computer through a single optical fiber [57].
Figure 4.11 – ActiveTwo AD-box with Battery [57]
4.1.1.5 ActiView
ActiView is a free open source program written in Laboratory Virtual Instrument Engineering Workbench (LabVIEW) [64]. It is BioSemi acquisition software that shows all ActiveTwo channels over the screen and allows saving data over the disk in BioSemi Data File (BDF) format. EEG/ECG/EMG signals can be acquired using ActiView. Data from extra sensors such as additional sensors connected to AD-box, AnalogInputBox (AIB) and digital triggers through USB2 receiver can also be acquired using ActiView [57].
Figure 4.12 – ActiView BioSemi Acquisition Software [61]
Layout of ActiView is designed to provide user with easy and quick check of data quality.
ActiView have various options to select, filter and reference data for down sampling. It allows completely stable and reliable acquisition of data using single buffered method. Being an open source program, it allows to molds it according to any requirements making it a versatile program [57].
Figure 4.13 – EEG Data Acquisition using ActiView [61]
4.1.2 EDF Browser
EDF browser is an open source, free viewer and toolbox for EEG data files [58]. EDF is an abbreviation of European Data Format which is a file format for storage of multichannel biological and physical signals. It was developed by a few medical engineers which later become de-facto standard for EEG and PSG recording in commercial equipment and multicenter research projects [65].
Figure 4.14 – EDF Browser [58]
EDF browser allows analyzing and manipulating BDF files containing EEG data. It has various options such as converting one file format to another, annotation/events manipulation, header editing, heart rate detection, data reduction and cropping, down sampling signals, combining multiple BDF files to one, averaging using triggers, events or annotations etc.
Figure 4.15 – Manipulating EEG Signals and their Annotations [58]
4.1.3 EEGLAB Toolbox for MATLAB
EEGLAB is a toolbox and graphical user interface that runs under MATLAB (The Mathworks, Inc.) [66]. It can process collection of EEG data for any number of channels. It support various functions of EEG data processing such as importing channel and event information, visualization of data (plus multi-trial ERP-image plots, scalp map and dipole model plotting, scrolling), data pre-processing (including filtering, epoch selection, averaging and artifact rejection), independent component analysis (ICA) and time/frequency decompositions including channel and component cross-coherence supported by bootstrap statistical methods based on data resampling [59].
Figure 4.16 – EEGLAB Toolbox [67]
Three layers are designed to divide the functionalities of EEGLAB. First layer lets the user interact with data using graphical interface without using any syntax of MATLAB. User can also mold the operation of MATLAB according to memory available. Data processing, command history and interactive pop functions can be performed under middle layer.
EEGLAB data structures and individual signal processing function can be used to write custom scripts by experienced users of MATLAB. Plug-in feature of EEGLAB allows easy integration of any modules to its main menu. EEGLAB is open source; available under GNU public license for noncommercial use and development, and can be downloaded for free from web1.
4.1.4 Waikato Environment for Knowledge Analysis
Waikato Environment for Knowledge Analysis (WEKA) [68] is used to apply classification algorithms and is most suited for developing new machine learning schemes. WEKA is a wide collection of machine learning algorithms and methods for data pre-processing with graphical user interface for manipulating data and comparison of various machine learning techniques for a problem [69].
Figure 4.17 – Waikato GUI Chooser
Figure 4.18 – Weka Explorer
1 http://www.sccn.ucsd.edu/eeglab/
As shown in the figure 4.17, WEKA has four graphical user interfaces as Explorer, Experimenter, KnowledgeFlow and Simple CLI. Explorer is the main interface of WEKA which allows inputting data for pre-processing and their results. However, the rest of interfaces are used for making new methods and combinations of methods for classification or clustering.
The basic purpose of WEKA is to help user to extract useful information from data and determine an appropriate algorithm for producing accurate prediction model using it. Weka is written in Java and developed by University of Waikato, New Zealand [68]. WEKA is a free software and available under the GNU General Public License.
5 E MPIRICAL E VALUATION
5.1 Experiment Context and Operations
The aim of the experiment was to find the various emotional states in subjects as they look on different pictures that are inducing strong emotions in most people. International Affective Picture System (IAPS) [70] are used for this purpose which is a general picture database and especially designed for experiments conducted for researches. During the experiment, EEG data was recorded in order to process, format and evaluate against different machine learning techniques for the classification of EEG data. The experiment helped to confirm the selection of right features and technique, i.e., that providing the most accurate results. The experiment involved planning, preparation, execution and reporting the results as explained in Figure 5.1:
Figure 5.1 – Experiment Stages
5.1.1 Experiment Planning
Planning was the initial stage of experiment. In this stage, we arranged the equipment for the experiment and the subjects who helped us to perform this experiment. The equipment for experiment was arranged through the Game Systems and Interaction Research Laboratory (GSIL) [61] available at Blekinge Institute of Technology. The subjects were students. To reduce the risk factor of resource unavailability, we had a backup plan for the experiment by having more than one pre-booking of the equipment and more number of subjects than required. Hence, 30 subjects were invited, 20 out of them participated in the experiment and EEG data for them was recorded.
The following equipment was involved in the experiment and booked under GSIL:
• BioSemi ActiveTwo System
o BioSemi Pin-Type Active-Electrodes o BioSemi Headcaps
o USB2 Receiver o ActiveTwo AD-box o Two Batteries for AD-box
• Electrode Gel
• Syringe
• Lab Computer (High Specification) Preparation Design Execution Planning
Setting up Experiment Guide
for Subjects and Equipment for
Experiment
Arranging Equipment Using
Standards and Design Instructions
Conducting Experiment following Instructions and Recording Data Booking
Equipment and Place, Arranging
Subjects for Experiment
5.1.2 Subjects Demographics
A total of 20 subjects (15 men and 5 women) participated in the experiment. All the subjects were students of Blekinge Institute of Technology and aged from 21 to 35 years. All the subjects were having different cultural background, nationalities and field of studies. None of subjects were professionals.
5.1.3 Experiment Preparation
At this stage, we prepared the equipment for the experiment and invited the subjects. A chart of observations was prepared that we used to note down issues during the experiment. This let us identify the limitations and possible improvements in the experiment. An experiment guide for the subjects was also prepared to help them understanding the experiment and performing it with ease.
Equipment was prepared for the experiment following the steps as below:
• Lab computer was powered on with ActiView running on it.
• Lab computer was connected to USB2 receiver though a USB cable.
• USB2 receiver was connected to AD-box using optical fiber.
• Pin-type active-electrodes were connected through their connector to AD-box.
• Battery was connected to AD-box
• AD-box was powered on.
Each subject was prepared for the experiment following the steps as below:
• Subject was asked to sit over the chair calmly in front of lab computer.
• Suitable BioSemi headcap was chosen to place on the head of subject and adjusted well to fit over the head.
• Electrode holders and pin-type active-electrodes were chosen according to the experiment design.
• Electrode gel was filled into electrode holders using syringe and pin-type active- electrodes were inserted into the electrode holders.
5.1.4 Experiment Design and Execution
The asymmetry among left and right brain hemispheres are the major area where the emotion signals can be captured [71]. According to a model developed by Davidson et al. [72], two core dimensions i.e., arousal and valence are related to asymmetric behavior of emotions. A judgment about a state as positive or negative lies under valence whereas area among calmness, excitement, expressing the level of excitation lies under arousal. Davidson et al [10] captured the EEG signals from left and right frontal, central, anterior temporal and parietal regions (F3, F4, C3, C4, T3, T4, P3, P4 positions according to the 10-20 system [73]
and referenced to Cz) in order to distinguish the happy and hatred emotions. Based on these findings, the experiment was executed with the instructions as [72] [74]:
• An appropriate interface was applied for the automated projection of the IAPS [70], emotion-related pictures.
• To compensate opening/closing of eyes, 30 seconds gap was maintained before starting the experiment.
• IAPS, 30 pictures (6 pictures for each emotion cluster as neutral, positive arousing/calm, negative arousing/calm) were displayed randomly for the duration of
