Individualized Motion Monitoring by Wearable Sensors
PRE-IMPACT FALL DETECTION USING SVM AND SENSOR FUSION
TOR CARLSSON
KTH ROYAL INSTITUTE OF TECHNOLOGY
MSc. Thesis Student Department of Machine Design, KTH
ÅF Industrial supervisor: Roger Ericsson
Individualized Motion Monitoring by Wearable Sensors
Individanpassad rörelsemonitorering med hjälp av bärbara sensorer
TOR CARLSSON Master of Science Thesis in Medical Engineering Advanced level (second cycle), 30 credits Supervisor at KTH: Farhad Abtahi Reviewer: Kaj Lindecrantz School of Technology and Health HL202X 2015:073
Royal Institute of Technology KTH STH SE-141 86 Flemingsberg, Sweden http://www.kth.se/sth
KTH Technology and Health
Individualized Motion Monitoring by Wear- able Sensors
Tor Carlsson
Approved Reviewer Supervisor STH
2015-07-09 Kaj Lindecrantz Farhad Abtahi
Commissioner Supervisor
ÅF Roger Ericsson
Abstract
Among the elderly, falling represents a major threat to the individual health, and is considered as a major source of morbidity and mortality. In Sweden alone, three elderly are lost each day in accidents related to falling. The elderly who survive the fall are likely to be suffering from decreased quality of life. As the percentage of elderly increase in the population worldwide, the need for preventive methods and tools will grow drastically in order to deal with the increasing health-care costs.
This report is the result of a conceptual study where an algorithm for individual- ized motion monitoring and pre-impact fall detection is developed. The algorithm learns the normal state of the wearer in order to detect anomalous events such as a fall. Furthermore, this report presents the requirements and issues related to the implementation of such a system. The result of the study is presented as a compari- son between the individualized system and a more generalized fall detection system.
The conclusion is that the presented type of algorithm is capable of learning the user behaviour and is able to detect a fall before the user impacts the ground, with a mean lead time of 301ms.
Keywords: support vector machine (SVM), pre-impact fall detection, elderly, ac- celerometer, gyroscope, MEMS
KTH Technology and Health
Individanpassad rörelsemonitorering med hjälp av bärbara sensorer
Tor Carlsson
Godkänt Granskare Handledare
2015-07-09 Kaj Lindecrantz Farhad Abtahi
Uppdragsgivare Kontaktperson
ÅF Roger Ericsson
Sammanfattning
Bland äldre är risken för att drabbas av fallrelaterade skador överhängande, ofta med svåra fysiska skador och psykiska effekter som följd. Med en ökande andel äldre i befolkningsmängden beräknas även samhällets kostnad för vård att stiga.
Genom aktiva samt preventiva åtgärder kan graden av personligt lidande och fallre- laterade samhällskostnader reduceras. Denna rapport är resultatet av en konceptuell studie där en algoritm för aktiv, individanpassad falldetektion utvecklats. Algorit- men lär sig användarens normala rörelsemönster och skall därefter särskilja dessa från onormala rörelsemönster. Rapporten beskriver de krav och frågeställningar som är relevanta för utvecklingen av ett sådant system. Vidare presenteras resul- tatet av studien i form av en jämförelse mellan ett individanpassat och generellt system. Resultatet av studien visar att algoritmen kan lära sig användarens vanliga rörelsemönster och därefer särskilja dessa från ett fall, i medelvärde 301ms innan användaren träffar marken.
Nyckelord: support vektor maskin (SVM), falldetektion, äldrevård, accelerometer, gyroskop, MEMS
To the subjects; please accept my utmost gratitude for voluntarily participating in this study and for your outstanding patience. Without your participation, the suc- cess of this thesis would have been impossible.
To my supervisor Farhad Abtahi; thank you for accepting this thesis, providing mentorship and for your invaluable feedback during the past months.
To my reviewer Kaj Lindecrantz; thank you for accepting and reviewing my thesis.
I would like to thank my industrial supervisor at ÅF, Roger Ericsson, for developing the subject of the thesis. Furthermore, to my fellow colleagues at ÅF, for their work on the hardware implementation and general ideas.
To my thesis colleague Simon Johansson; thank you for being such a skillful and professionally competent engineer and for the newfound friendship.
Tor Carlsson, Stockholm, May 2015
αxy IMU z-axis tilt angle relative to g rad
ωO Calibrated gyroscope reading rad/s
ωS Raw gyroscope reading LSB
aO Calibrated accelerometer reading m/s2
aS Raw accelerometer reading LSB
F(t) Feature vector at time t xs(t) Observation at time t
xtw(t) Observation at time t with observation history κ Level of truth-classifier agreement
θ Pitch, rotation about the IMU y-axis rad
ϕ Roll, rotation about the IMU x-axis rad
F P R False Positive Rate F P R= F P/(T N + F P )
g Gravitational force m/s2
SE Sensitivity T P/(T P + F N)
SP Specificity T N/(T N + F P )
TP LT Pre-impact lead time s
ADL Activities of Daily Living. 8
ARIMA Autoregressive Integrated Moving Average. 12 DFT Discrete Fourier transform. 15
FFT Fast Fourier Transform. 14
FLOPS Floating-point Operations Per Second, measure of MCU speed. 43 FN False negative. 11
FP False positive. 11
IMU Inertial Measurement Unit.. 16 IoT Internet of Things. 2
IPS Instructions Per Second, measure of MCU speed. 43 LOOCV Leave-One-Out cross-validation. 61
MCU Microcontroller Unit. 43
MEMS Micro-Electro-Mechanical System. 3 MLM Machine Learning Method. 12
PCA Principal Component Analysis. 14 RMS Root mean square. 15
SMA Signal magnitude area. 15 SNR Signal-to-Noise Ratio. 46 SVM Support Vector Machine.. 12
TBM Threshold based method. 11, 12 TN True negative. 11
TP True positive. 11
WHO World Health Organization. 1
Abstract v
Sammanfattning vii
Acknowledgements ix
Nomenclature x
Glossary xi
List of Figures xvii
List of Tables xx
1 Introduction 1
1.1 Background . . . 1
1.2 Purpose . . . 3
1.3 Problem Description . . . 3
1.4 Delimitations and Assumptions . . . 4
1.4.1 Hardware . . . 4
1.4.2 Time limit . . . 4
1.4.3 Subject restrictions . . . 5
2 Theoretical Framework 7 2.1 Dynamics of Human Motion . . . 7
2.1.1 Body Accelerations . . . 7
2.1.2 Body Angular Velocities . . . 8
2.1.3 Differences Between Young and Elderly Subjects . . . 9
2.1.4 Properties of Human Falls . . . 9
2.2 Fall Detection and Motion Monitoring Systems . . . 10
2.2.1 State of the Art . . . 10
2.2.2 Principles of Feature Construction . . . 15
2.2.2.1 Time Domain Features . . . 15
2.2.2.2 Frequency Domain Features . . . 15
2.2.2.3 Time Frequency Domain Features . . . 16
2.2.2.4 Heuristic Features . . . 16
2.3 Micro-Electro-Mechanical Systems Sensors . . . 19
2.3.1 Triaxial Accelerometers . . . 19
2.3.2 Triaxial Gyroscopes . . . 20
2.3.3 Sensor Error Model . . . 21
2.3.4 Sensor Calibration Using Sensor Error Model . . . 23
2.4 Algorithm Theoretical Framework . . . 24
2.4.1 Principles of Sensor Fusion . . . 24
2.4.2 Sliding Window Discrete Fourier Transform . . . 26
2.4.3 Principal Component Analysis . . . 27
2.4.4 The Support Vector Machine . . . 31
2.4.4.1 The Gaussian Radial Basis Function Kernel . . . 32
2.4.4.2 One-class SVM . . . 33
2.4.4.3 Selecting the Proper SVM Training Parameters . . . 34
2.4.4.4 General Properties of the Support Vector Machine . 35 3 Methodology 37 3.1 Thesis Methodology . . . 37
3.2 Experimental Setup . . . 40
3.3 Implications . . . 42
3.3.1 Types of Demonstrators . . . 42
3.3.2 Hardware Requirements . . . 43
3.3.2.1 Sensor Requirements . . . 43
3.3.2.2 Processor Speed Requirements . . . 43
3.3.2.3 Communication Link Requirements . . . 44
3.3.2.4 Memory Requirements . . . 44
3.4 System Evaluation . . . 45
3.5 Sensor Evaluation . . . 45
3.5.1 Sensor Characteristics . . . 45
3.5.2 Signal-to-Noise Ratio . . . 46
3.6 General System Structure . . . 47
3.6.1 Timing Constraints . . . 48
3.7 Sensor Calibration . . . 49
3.7.1 Sensor Model Calibration . . . 49
3.7.2 Sensor Orientational Calibration . . . 52
3.8 SVM Implementation . . . 53
3.8.1 Feature Construction . . . 53
3.8.2 Dimensionality Reduction . . . 55
3.8.3 SVM Parameter Optimization . . . 56
3.9 TBM Implementation . . . 58
3.10 Method of Evaluation . . . 60
3.10.1 SVM Method of Evaluation . . . 60
3.10.2 TBM Method of Evaluation . . . 62
4 Results 63
4.1 System . . . 63
4.2 Sensors . . . 65
4.3 Sensor Attachment Calibration . . . 66
4.4 SVM . . . 66
4.5 TBM . . . 69
4.6 Results Summary . . . 71
5 Discussion 73 5.1 System Realization . . . 73
5.2 SVM Outlier Rate . . . 73
5.3 Impact-point Labeling . . . 75
5.4 Method of Evaluation . . . 75
5.5 SVM and TBM Performance . . . 76
5.6 Further Improvements . . . 77
5.7 Ethical and Economical Considerations . . . 78
6 Conclusion 81
7 Future Work 83
Bibliography 83
A Appendix - Recorded data I
B Appendix - Planning report III
C Appendix - Keywords XV
D Appendix - Wolk Interview XVII
2.1 Directions of the human body commonly used in the literature. . . . 7
2.2 Axes of IMU rotation (also referred to as Euler angles): αxy is the angle between the gravitational component and IMU z-axis, ϕ and θ the rotation about the x- and y-axis respectively. . . 17
2.3 Non-orthognal sensor axes (xS, yS, zS) and body fram axes (xB, yB, zB), original picture taken from [1]. . . 22
2.4 Accelerometer and gyroscope tilt data relative to z-axis fused into a more reliable estimate using a complementary filter. . . 25
2.5 Model of the complementary filter that estimates the rotational angle by data fusion. . . 25
2.6 Sliding window on a discrete signal. . . 26
2.7 A set of observations in the original feature space x1, x2. . . 28
2.8 Principal components, y1, y2, of the set of observations. . . 29
2.9 Projected observations on the first principal component y1. . . 30
2.10 Hyperplane separate. . . 31
2.11 The gaussian kernel with σ1 = 0.1 (left) and σ2 = 0.37 (right) that transforms R2 7→ R3, where the spikes correspond to some SVs xj. . . 33
3.1 General thesis process workflow. . . 37
3.2 The general building block that creates the thesis workflow I/O- structure. . . 38
3.3 Examples of recorded fall data, top left clockwise: (1) sideway fall, (2) falling from walking, (3) falling during transitioning into standing posture, (4) stumble induced fall. . . 41
3.4 System overview. . . 47
3.5 Flow of data and MATLAB feedback packets. . . 47
3.6 System timing diagram. . . 48
3.7 Coordinate system of IMU and human body. . . 49
3.8 aSis the calculated norm of non-calibrated sensor readings, here com- pared to the local gravity constant g. Scaling factors calculated from ADC and dynamic range are applied to the data. . . 49
3.9 Norm of calibrated sensor readings, aO, during static positions differs less that ±0.5% from gravity g after calibration. . . 50
3.10 Comparison of gyroscope calibrated and non-calibrated integrated readings to the known accelerometer reference. . . 51 3.11 Difference between integrated and calculated angle using non cali-
brated gyroscope. . . 51 3.12 Measured difference between the calculated and calibrated gyroscope
angle. . . 51 3.13 Accelerometer and gyroscope data for normal walking (triangle) and
limping (square). . . 53 3.14 Features from normal walking (triangle) and limping (square) pro-
jected on the first three principal components from the combined data set. . . 55 3.15 Schematic illustration where the elliptic data reduction method pre-
serves the contour of the data while removing redundant data points. 56 3.16 Optimization routine flow. . . 57 3.17 The SVM parameter optimization objective for subject 6 as traversed
by the Multistart-function in the region of feasible ν and σ. . . 58 3.18 Increasing the thresholds in a sequential order starting with a,b and
continuing with c,d and finally e,f to increase the SP at the cost of TP LT. The circle indicates the selected threshold. . . 59 3.19 The entire process of creating the personalized SVM. Separate data
is used for training and evaluation of the classifier. . . 61 3.20 The process of creating the TBM classifier. . . 62 4.1 Histogram plot of system response time (feedback time Tf eedback as
defined in figure 3.5). Mean TSV M = 59.38ms, TT BM = 49.38ms.
Max TSV M = 187.5ms, TT BM = 137.5ms. . . 64 4.2 Distribution of the spectral components (sum of sensor FFT data) for
ADL and during a fall from walking. The majority of the frequencies are located below 30Hz. . . 65 4.3 Raw accelerometer and gyroscope data as measured by the sensors,
during falling from walking. . . 66 4.4 Prior to and after sensor attachment calibration. Data is heavily
filtered to better illustrate the calibration result. . . 66 4.5 Plot of outlier rate per minute during re-training session, as well as
the number of support vectors in the model. . . 67 4.6 Falling from walking as recognized by the SVM, trained on subject 5
ADL. . . 68 4.7 Feature extraction execution time on the personal computer. . . 69 4.8 Variance covered by the 10 first principal components. . . 69 4.9 SVM classification (left) and TBM (right) classifying a test data set
containing a walking from falling motion. . . 70 5.1 The effects of labeling error. . . 75 5.2 Data is stored in the Cloud and protected by different levels of en-
cryption. . . 78
2.1 Possible outputs from a fall detection system. . . 11
2.2 Interpretation of κ. . . 15
2.3 List of features used in motion monitoring systems. . . 18
3.1 List of recorded fall data, F: forward, L:left, R:right . . . 41
3.2 MPU9150 Sensor performance data. . . 46
3.3 Formulas used for calculating the angle of rotation. . . 50
3.4 Final calibration parameters for accelerometer and gyroscope sensor error models. . . 52
3.5 Contents of feature vector F (t). . . 55
3.6 Percentage of the original ADL data used as input to the SVM pa- rameter optimization process. . . 56
3.7 Final threshold values implemented in the TBM classifier. . . 60
3.8 Subject participation. . . 60
4.1 Feature vector memory size. . . 64
4.2 Buffer size required to hold time window in order to compute the feature vector. . . 64
4.3 Memory required for a re-trained SVM classifier for 500 support vectors. 65 4.4 Floating points operations required for SVM classification. . . 65
4.5 SVM training parameters as selected by the optimization routine. . . 67
4.6 Parameters that defines the re-trained SVM object. . . 67
4.7 Performance data for the SVM on assorted fall types. . . 68
4.8 ADL test set fall-out, F P -rate on ≈ 8 minutes of ADL data. Required training time can be seen in table 4.6. . . 68
4.9 Performance data for the TBM implementation. . . 69
4.10 ADL test set fall-out, F P -rate on ≈ 8 minutes of ADL data. . . 69
4.11 Memory needed to hold the sample-by-sample feature data. . . 70
4.12 Buffer size required compute the TBM feature vector. . . 70 A.1 List that presents the usage of data . . . I A.2 List that presents the usage of data, cont. table A.1 . . . II C.1 Keywords . . . XV
1.1 Background
The World Health Organization (WHO) [2] concludes that approximately one third of all people worldwide above the age of sixty-five have fallen at least once during the past year. Among the elderly, falling represents a major threat to the individual health, and is considered as a major source of morbidity and mortality. In Sweden alone, three elderly are lost each day in accidents related to falling. The elderly who survive are likely to be suffering from decreased quality of life. The effects of a fall that decrease quality of life may include increased fear of falling, loss of self- confidence and self-esteem [3]. These effects might lead to social isolation, further decreasing the overall quality of life of the elderly.
The annual cost (direct and indirect) concerned with fall-related injuries in Sweden is calculated to continue increasing from 14 billion SEK to 22 billion SEK in year 2050 if the development is left unattended [4]. These numbers does not only ac- count for direct costs related to health-care but also to costs related to decreased quality of life for the elderly. As the percentage of elderly increase in the population worldwide, the need for preventive methods and tools will grow drastically in order to deal with the increasing health-care costs.
WHO [2] lists challenges related to fall-prevention (excerpt):
• Encouraging elderly to adopt a more active lifestyle (i.e. joining a exercise class)
• Rise awareness in the general population of basic, efficient fall prevention meth- ods
• Promote benefits related to interventions, and fit them to a positive self- identity
• Ensure interventions is designed to meet the needs, capabilities and preferences of the individual
The applied method used for fall-prevention should implement a broad set of actions targeted to different areas that effect the risk of falling and/or the severity of the injury. Preventive measures have been shown to have positive effects on the falling rate [4]. In general, there exists two main approaches for minimizing the rate of falling among the elderly: pro-active measures and direct measures. Pro-active measures may consist of the following [4, 3]:
• Extrinsic factors: supplying well-suited personal equipment (cane, shoes) and medication.
• Environmental factors: removing slippery floors and improving lighting con- ditions.
• Reducing high-risk activities: let the home-care assist in switching light-bulbs.
• Physical factors: physiotherapy to improve stability, balance and strength.
Direct measures are intended to decrease the severity of the impact in connection to, or other effects related to the fall such as prolonged lying on the floor associated with the injury. Such systems may consist of wearable hip-protection trousers or wearable fall-alarm systems. Current available systems display significant shortcom- ings in the areas of acceptability and usability. One such example is the widely used wearable hip-protection trousers. The current design is shown to exhibit strong pos- itive clinical impact on the outcome on the severity of a fall but the system displays signs of user rejection since they are uncomfortable and difficult to integrate into recurrent nursing activities and in the everyday living of the elderly [5, 4].
It has been shown that the medical outcome of a fall is highly dependant upon the response and rescue time [6]. Therefore, developing and refining methods that allow for fast and efficient detection and communication of falls are of interest. Further, WHO [2] states that in order to cope with the effects of ageing and ultimately the risk of falling, adopting a healthy and active lifestyle is critical for success. There- fore, it is desirable to provide enabling tools and systems that allow for maintaining such active lifestyle and that also at the same time alleviates the fear of falling.
Motion monitoring of the elderly is considered as a plausible method for reducing number of falls by actively preventing fall-related injuries or used for evaluating clinical factors that can be used in pro-active health-care [7].
The overall trend within Swedish health-care sector is to encourage a high level of patient participation, in order to shift the aim of the health-care system from delivering retroactive care towards a more pro-active system. This is believed to reduce future health-care costs. An example of this is the swedish eHälsa-platform [8] where patients can access their medical journals online. Furthermore, Internet of Things (IoT) and cloud-based computing is a significant trend that together with the idea of a pro-active health-care system allows for incorporation of wearable med- ical systems into the everyday life of the elderly. Such systems can be used for fall risk assesment and fall identification.
In state of the art research related to wearable medical monitoring systems [6], Micro-Electro-Mechanical System (MEMS)-sensors are widely used mainly due to their small size and low energy consumption. This allows for long-time monitoring of the elderly that enables the collection of large data sets that can be stored in the patient’s journal (the Cloud) and clinically evaluated, either by a physician or by a machine-learning algorithm [9].
1.2 Purpose
The purpose of this thesis work was originally presented by ÅF with the following motivation:
" In today’s society the demand for sensor information is increasing. In the industry the use cases can vary and are often different kinds of detection of motions of robots, cars and humans. Also in the area of Medicare there are huge benefits by using a system of sensors to supervise and to create a better safety for the patients. At the same time, we use our mobile devices like smartphones to supervise our health by monitor our motions, just to mention a few.
The usage of sensors is an area that develops rapidly and ÅF is interested to study, develop techniques and to create a broad knowledge base at ÅF. The area of usage is many. To implement tomorrows advanced active systems there are a demand for sensors that read present situation information and motion within an object. The sensors can be of the type accelerometers, cameras, gyros, GPS or likewise. Further there are needs for advanced signal processing and sensor fusion to enhance and combine information from the various sensors."
Furthermore, fall prevention and motion monitoring systems for clinical applications have been widely studied in academic research. By introducing this body of knowl- edge to the industry, the speed of development could be further increased, so that useful products may be manufactured and introduced into clinical care even faster.
This report is the result of the author’s MSc-thesis, performed in collaboration with ÅF.
1.3 Problem Description
A tri-axial accelerometer and gyroscope MEMS sensor shall be used to detect a fall of an elderly human. The MEMS sensors shall be attached to the hips in order to detect changes in the dynamic features of the body during the initiation of a fall from walking activity.
The requirement is that the sensor accuracy shall be high enough to activate an active fall protection system before the human impacts the ground when falling (i.e.
pre-impact detection). Two systems (demonstrators, algorithms) shall be developed.
The following questions shall be answered by the thesis:
• What are the implications of this requirement on software and hardware, given the two different demonstrators used?
• Are different sensor fusion algorithm optimal based on the differences between the demonstrators?
• What are the implications of this requirement with regard to overall system response time, given the two different demonstrators used?
• Are different learning systems that can distinguish a fall from a walking motion optimal based on the differences between the demonstrators?
1.4 Delimitations and Assumptions
1.4.1 Hardware
The following hardware was initially specified for this thesis:
• TivaTM C Series TM4C123G [10]
• SimpleLink Wi-Fi CC3100 BoosterPack, 802.11b/g/n [11]
• Fuel Tank BoosterPack [12]
• Sensor Hub BoosterPack [13]
• Additional sensors if needed (mounted on the TIVA board) The Sensor Hub board contains the following sensors:
• Triaxial accelerometer
• Triaxial gyroscope
• Magnetometer
• Ambient, atmospheric pressure sensor
• Non-contact temperature sensors
1.4.2 Time limit
The thesis shall be performed within a time span of twenty weeks.
1.4.3 Subject restrictions
This thesis is limited to young subjects for evaluating the algorithms. The young subjects used in the testing phase will try to mimic the motion of an elderly person.
Furthermore, due to time constraints, the number of subjects included in this study is limited to eight.
2.1 Dynamics of Human Motion
This section introduces the reader to the properties of human motion, important terminology and measurands used for describing and capturing the dynamic behaviour of the human body. The definition of a fall is presented, as well as the
difference between young and elderly people that is significant to the research question.
The human motion is highly complex, non-linear and varies with time. Much re- search has been performed on the dynamics of human locomotion and falls.
vertical
antero-posterior medio-lateral
trunk roll trunk pitch
Figure 2.1: Directions of the human body commonly used in the literature.
2.1.1 Body Accelerations
The acceleration generated by the human body is mainly made up of low frequency components and the exact composition depends on the type of movement. The ver- tical acceleration is generally of a higher frequency than the antero-posterior and medio-lateral accelerations (see figure 2.1). During walking, acceleration frequencies are in the range between 0.8 to 15Hz if measured in the upper body region. A heel- strike can create frequency components as high as 60Hz in the vertical direction,
but is not generated by voluntary muscoskeletal work. In general, the spectrum shifts to higher frequencies from upper body positions to lower caudal positions.
For activities of daily living (ADL), the main part of the spectrum energy lie in the range between 0.3 to 3.5Hz.
Walking acceleration amplitudes in the upper body varies between -0.3 to 0.8g in the vertical direction and in the back -0.3 to 0.4g in the horizontal direction. Running creates significantly larger amplitudes, ranging from 0.9 to 5.0g in the back and 3.0 to 12.0g at the ankle in the vertical direction.
Bouten et al. [14] concludes that in order to accurately capture ADL such as walk- ing, the accelerometer should sample with a frequency of at least 20Hz and posses a dynamic range of ±6g. Similar work also shows that the acceleration frequencies for ADL are concentrated below 15Hz [15, 16].
By increasing the sampling frequency, additional features can be extracted from the movements of the body. Bonnet et al. [17] sampled at 150Hz in order to acquire high frequency micro-oscillations that can be used to determine postural control in the elderly. Previous published articles in the area of human motion monitoring and pre-fall detection have used sampling frequencies in the range 50Hz to 1kHz [18, 19, 20, 14].
2.1.2 Body Angular Velocities
Liu et al. [21] used a video monitoring system and a gyroscope attached to the chest to capture upper trunk angular velocities during slip-induced backward falls and ADLs for ten elderly people. Their results are based both on previous studies and experimental data. The authors concluded that the average peak angular ve- locities during walking were 7 to 9.5°/s, performing a sitting down transition were 112.1°/s and bending over 118.7°/s. During slip induced backward falls, the upper trunk average peak angular velocity was measured to 139.7°/s. During evaluation of a pre-impact fall detection algorithm, the same authors [22] concluded that trunk angular velocities of ADLs are approximately up to 180°/s and that slip induced backward fall angular velocities are not greater than 250°/s.
Furthermore, Bourke et al. [18] gathered angular velocity data for ten elderly, community-dwelling subjects (age = 77.2 ± 4.3years) during scripted ADL activ- ities. Data was collected using a bi-axial gyroscope (trunk roll and trunk pitch) attached to the chest. Activities performed were the following:
• Sitting down, standing up from five different type of chairs (armchair, kitchen chair, toilet seat, car seat, low stool).
• Sitting down, lying down and standing up from bed.
• Walking 10m.
Bourke et al. reported that maximum peak angular velocity measured during ADLs did not exceed 220°/s.
In a survey performed by Rodriguez-Martin et al. [23], the most common sam- pling frequencies set in motion monitoring systems that perform measurements on angular velocity is in between 50Hz to 350Hz.
2.1.3 Differences Between Young and Elderly Subjects
There’s a significant difference in the dynamic properties of locomotion between young and elderly subjects. The following differences in locomotion can be observed in the elderly [24]
• Push-off peak positive trunk anterior-posterior acceleration is significantly lower than for younger subjects (elderlymean = 0.26g, youngmean= 0.40g).
• Heel-contact, peak negative trunk acceleration is significantly higher for the elderly (elderlymean= −0.20g, youngmean = −0.09g).
• Time delay between head-trunk acceleration is lower for the elderly, compared to young subjects (elderlymean= −31.3ms, youngmean= −96.7ms).
In general, the elderly seem to apply a more cautions gait strategy. As [24] discusses, the differences in locomotion between young and elderly is probably due to the need for the elderly to maximise the dynamic stability during critical parts of the gait cycle.
2.1.4 Properties of Human Falls
The definition of a fall is the following: "an event which results in a person coming to rest inadvertently on the ground or other lower level". A fall can be divided into the following phases [25]
• Pre-fall: the phase during ADL in which an event leads to the fall.
• Critical phase: the uncontrolled free fall and impact.
• Post-fall: the phase when the person is lying on the ground after impact.
• Recovery: the phase where the person regains standing position, independently or by the help from another person.
A fall can be divided into the following classes [26]:
• Falling from lying (sleeping).
• Falling from sitting.
• Falling from walking or standing.
Abbate et al. [3] performed a survey of motion monitoring techniques, and presented the most important fall scenarios for the elderly:
• Falling from standing or walking.
• Falling from chair.
• Falling from bed.
Vlaeyen et al. [27] evaluated real-world fall incidents of elderly people using a camera surveillance system. The conclusion is that the critical phase will last for tmean = 1496ms(SD = 639ms, trange = [1075 − 3790ms]). The results are gener- ated from a broad spectrum of fall directions, but the most common fall type was identified to be the backwards fall. Most of the falls occurred during walking or transitioning. The most common impact site on the body was determined to be the pelvis, torso, head and elbow or forearm.
2.2 Fall Detection and Motion Monitoring Sys- tems
This section introduces the reader to various forms of wearable motion and fall detection systems that is present. Advantages and disadvantages of each type of
system is discussed. A comprehensive list of equations useful for calculating properties related to human motion is presented.
2.2.1 State of the Art
Note: Since this thesis is focused on pre-impact fall detection, this chapter is biased towards abnormal motion pattern detection using wearable systems. A fall can be regarded as a set of abnormal motion patterns when compared to ADL.
The research area of long-time motion monitoring and fall detection systems has been accelerated by the introduction of miniaturized sensors and energy-efficient micro-computers. Extensive research has been conducted within this field during the last decades, and there is a multitude of different approaches. In general, fall detection systems can be regarded as a subset of the more general wearable motion monitoring systems. The features and functions of the motion monitoring system is dependent upon the application, that is often related to activity monitoring. Ex- amples of such applications may be energy expenditure analysis where the duration and type of movement of a human is analysed. This can be useful for evaluation of the amount of physical activity performed by the elderly. Other applications may include posture stability and balance assessment in fall risk analysis. Howcroft et al. [7] showed that inertial MEMS sensors can be used to asses the risk of falling in the elderly population.
The aim of a fall detection system is to reduce the injury to the human caused by a fall. The first fall detection system was developed in the 1970’s. When the user pressed a remote button, an alert message was transmitted [26]. Since then, auto- matic fall detection systems and motion monitoring systems have been developed.
The following table 2.1 presents measurements that is commonly used with fall de- tection systems in order to state the performance.
Table 2.1: Possible outputs from a fall detection system.
Event A fall occurs A fall does not occur A fall is detected True positive, TP False positive, FP A fall is not detected False negative, FN True negative, TN
• Sensitivity: The capacity to detect a fall. Given by the number of detected falls divided by the numbers of total falls.
SE = T P
T P + F N (2.1)
• Specificity: The capacity to detect a fall only if it occurs, i.e. avoid false alarms.
Can also be seen as the capability of the algorithm to correctly classify an ADL as an ADL.
SP = T N
T N + F P (2.2)
Note:Performance in this text is referred to SE, SP and pre-impact lead time (see eq. 2.3) compared to current state of the art.
A fall can be identified by studying the acceleration that is created during impact of the body with the ground. Such system can be referred to as a post-fall identi- fication system. In order to enhance the specificity and sensitivity of the detection algorithm, body posture is often evaluated in the post-fall phase. The least complex fall detection algorithms are based upon a threshold based method (TBM). The threshold is set by evaluating experimental results. The benefit of using a threshold is the speed and ease of evaluation. Such systems can be realized on hardware plat- forms with limited computing power and can potentially be very energy efficient.
According to Igual et al. [28], accelerometer TBM fall-detection systems presents a performance of SP =92 − 100%, SE=90 − 100% in a supervised laboratory setting.
Bourke et al. [18] demonstrated that a multi-leveled TBM employing several thresh- olds improved the system performance, compared to a single-TBMs. However, it is difficult to set an appropriate threshold for all possible types of falls while reduc- ing false positive events created by ADL. Inter-individual variation may also affect the threshold level. In a study performed by Bagala et al. [20], the performance of thirteen TBM systems were evaluated (among them several earlier algorithms by Bourke et al. but not [18]) by applying the algorithms to real-world fall and
ADL data. They concludes that the evaluated systems performed worse than pre- viously stated. The performance of all of the included systems were measured to be SPmean= 83.0% ± 30.3%SD and SEmean= 57.0% ± 27.3%SD.
Recent development on fall detection systems has focused on identifying falls during the critical phase (during descent, prior to impact). The purpose of such systems is to deploy counter-measures to reduce the severity of an possible injury. A common counter-measure is an airbag deployed to protect the hip from injury. The require- ment for such systems are more stringent, due to the real-time constraints imposed on the system. The performance of such systems is often evaluated in terms of pre-impact lead time (TP LT)(eq. 2.3) as well as SE and SP [29].
TP LT = Timpact− Talarm (2.3)
False Positive Rate (F P R) describes the amount of F P relative to the overall neg- atively labeled data size
F P R= F P
T N + F P (2.4)
Among the reviewed articles, such systems presents TP LT in the range of 0.1857s to 0.0625s [30]. There are some available pre-impact systems available or is "soon-to- be-released" on the commercial market, but the implemented algorithm remains a corporate secret [31, 32]. However, [31] state an airbag inflation time of ≈ 80ms, see appendix D.
A novel detection method was recently described by Hu et al. [33] where auto- regressive integrated moving average (ARIMA) modelling was used to forecast in- dividual time series motion patters, and where a statistical process control chart were used to set an individualized Shewhart’s 3-sigma control theory-threshold for detection. ARIMA-modelling is widely used in economics and meteorology. Hu et al.’s report was limited to slips from steady state walking, not considering how to model falls from a set of differently mixed ADL motion patterns. However, Hu et al.
showed that a slip could be detected with a SE = 94.7% and SP = 99.2% using this approach. Minimum detection time from heel-strike to alert was measured to 0.540s.
In an effort to overcome the limitations associated with TBM systems, fall detection systems with integrated smart machine-learning algorithms have been developed that is able to distinguish fall events from ADL activity with high sensitivity and specificity [28]. Machine learning methods (MLM) tries to classify movement data into ADL or a fall-event and have been implemented with varying success, depending on the algorithm used. Classification or probability analysis is commonly performed in the following way [34, 35]:
• Support Vector Machines (SVM) (optimization)
• Naïve Bayes (Probabilistic)
• K-Nearest Neighbour (k-NN) (brute force)
• Decision Trees (divide and conquer)
• Multi-layered Neural Networks (optimization)
• Markov Models (probability)
SVM is a popular machine learning algorithm that has proven to perform similarly to TBM systems, with regards to TP LT, SE and SP in pre-impact fall-detection systems. Furthermore, SVM outperforms many of the classical MLMs according to Delahoz et al. [35]. The SVM machine training data can be set up in different ways:
1. Training set contains fall data (one class SVM):
Movement is classified as fall or non-fall.
2. Training set contains ADL data (one class SVM):
Movement is classified as ADL or non-ADL.
3. Training set contains fall and ADL data (two class SVM):
Movement is classified as ADL or fall.
Depending on the training set provided for training, the SVM algorithm is only able to classify a specific set of motions. By simply providing fall data for training, the classifier can only output a statement fall or non-fall. SVM is thus a (semi) super- vised machine learning method, i.e. the algorithm is provided with the necessary data labels used to infer the human state.
The optimal training set for SVM-classifiers should incorporate all commonly per- formed, significant ADL data as well as fall data for all types of falls and patient groups. As it is today unlikely to create a fall training set without performing scripted real world fall studies on elderly people, this type of SVM-classifier is lim- ited in the sense of real world accuracy. The commonly applied solution to this problem is to gather fall data from young people as the risk of injury during data gathering is less. However, the European project FARSEEING [36] is setting up a database for the collection of two hundred real world falls of elderly people. The purpose of this database is to collect information on real world data so that a dy- namic user-specific fall risk model can be constructed.
Shan et al. [37] implemented a two-class SVM classifier on a PC-based system with a reported SE = 100% and SP = 100% with a minimum pre-impact lead time of TP LT min = 100ms and average TP LT mean = 203ms in a laboratory setting. An accelerometer (±6g) was used for data collection, sampled at 160Hz. Shan et al.
used both ADL and fall-data training sets to increase the performance of the sys- tem. Time domain features such as mean, standard deviation and slope (among others) were used as input for creating the training set, in total 25 features formed the feature vector. The kernel used for training the SVM was not reported.
Shi et al. [38] implemented a two-class SVM in a stand-alone embedded pre-impact fall detection system. Accelerometer (±1.7g) and gyroscope (±300°/s) data was sampled at 200Hz. Both ADL and fall data from non-elderly people was used for
training the classifier. Fast Fourier Transform (FFT)-features were extracted using Principal Component Analysis (PCA) in order to select high-quality combinations of the original data, achieving better SVM recognition performance. PCA is a widely used technique for motion feature dimension reduction and data fusion [34]. The authors reduced the number of dimensions to three with while maintaining perfor- mance. Polynomial kernel was used for separating the two classes. The classifier output was used to inflate an airbag to protect the hips. The airbag inflation time was measured to 0.333s.
Aziz et al. [30] evaluated two-class SVM performance by studying the effects of different TP LT and window sizes on tri-axial accelerometer (±6g) and gyroscope (±26.18rad/s) offline-data sampled at 128Hz. Data features extracted contained time-domain mean and variance of the windowed data, velocities and angular veloc- ities from each sensor axis, 18 in total. The Gaussian RBF kernel was used. From Aziz et al. results, it is apparent that the TP LT and window length have a direct impact on the SE and SP. The classifier achieved optimum performance for data samples gathered t = 0.1875s before impact with a window size of 1s or less. They concluded that a SVM machine can be used for pre-impact fall detection with at least a SE = 95% and SP = 90%, for a data set of seven fall and eight non-fall events. The results from Aziz et al. contradicts those of Shan et al. and Shi et al. [37, 38] with regards to the maximum achievable SE and SP for a fixed TP LT. A visible difference between the systems implemented are the chosen features and kernel used for SVM-training.
Furthermore, Rodriguez et al. [23] used a embedded two-class SVM for the imple- mentation of a gait detector. Gaussian RBF kernel was used. Walking patterns were gathered from elderly patients. Data was sampled with different frequencies, ranging from 40Hz to 200Hz using an accelerometer (±6g, 340LSB/g), and by extracting two power spectral density features (window=3.2s long) from the FFT- spectrum they were able to perform classification within 44.86ms.
The Kappa statistic can be used to quantify the level of agreement between the truth and classifier output, and adjusting the quantified level of agreement to the chance of correct classification [39]. The maximum value of κ is 100% and for a random predictor κ = 0%. A κ ≤ 0 indicates systematic disagreement. Calculation is based on the observed agreement po constituting the actual classifier output and the expected agreement pe as provided by chance (a random predictor)
po = T P + T N
T P + F P + T N + F N pe =T P + F N
n · T P + F P n
+F P + T N
n · F N + T N n
(2.5)
where n is the sum of all the labels. Kappa is then calculated as ˆκ = po− pe
1 − pe (2.6)
As suggestion on how to interpret the performance using κ is presented in table 2.2.
Table 2.2: Interpretation of κ.
Poor Slight Fair Moderate Substantial Perfect
≤0 0.01 − 0.20 0.21 − 0.40 0.41 − 0.60 0.61 − 0.80 0.81 − 0.99
2.2.2 Principles of Feature Construction
A feature is a set of data that contains information on the property investigated.
The feature data is used for making higher level decisions, and thus a proper feature selection is critical to the performance of any motion monitoring system. Depending on the property investigated, one specific type of feature might deliver more or less accurate data. A well chosen feature expresses not only minimal variance for an identical set of motion activities, but is also able to clearly separate different set of motion activities [34].
At a given time t, a number n observations (sensor readings) is gathered, so that an observation vector xs can be created
xs(t) = [x1(t), x2(t), ..., xn(t)] ∈ Rn (2.7) n are the numbers of sensor outputs. Motion history is saved by storing the k number of observations in a matrix xtw such that
xtw(t) = [xs(t), xs(t − 1 · Ts), . . . , xs(t − k · Ts)]T ∈ Rn×k (2.8) This allows for trend monitoring since a feature vector F is created, for every new observation of xs(t)
F(t) = f(xtw(t)) = [F1, . . . , Fm] ∈ Rm (2.9) Fiare the features derived from the sensor data. Depending on the technique used for calculating the feature (feature construction), different features can be divided into the following four classes: time-domain, frequency-domain, time-frequency-domain and finally heuristic features.
2.2.2.1 Time Domain Features
Features derived directly from the time windowed data is referred to as time domain features, and are often statistical measures [34]. Examples of such measures are variance, mean and standard deviation. Further, non statistical measures include root mean square (RMS), signal magnitude area (SMA) and peak-to-peak (Ap−p) [26, 34].
2.2.2.2 Frequency Domain Features
The frequency-domain features are commonly extracted using the discrete Fourier transform (DFT) and the signal content is received as a distribution of amplitudes
for each frequency component in the signal. The individual amplitudes can then be used as features for describing the current time window.
Shi et al. [38] used the DFT to aquire the 10 first spectral coefficients and used them as features in pre-impact fall detection. Furthermore, it has been shown that frequency domain features are very well suited for periodic behaviour detection in for example gait analysis systems [14, 23].
2.2.2.3 Time Frequency Domain Features
Frequency domain features are useful for investigating the individual components of a time domain signal. Fourier transformation is widely used for acquiring the signal spectrum, but due to the intrinsic properties of the Fourier transformation, temporal information is lost. However, the discrete wavelet transform (DWT) is a technique that preserves this spatial information by decomposing the signal using high-pass and low-pass filters in an iterative fashion, the process often illustrated as large tree-shaped filter bank. This enables the frequency content to be studied in great detail for non-stationary signals. Depending on signal characteristics and the type of and DWT selected (for example the Haar, Coiflet or Daubachies) the resulting transformation will provide different output. Preece et al. [34] reports that DWT has been used for the classification of a wide range of motion activities. For example, in 2010, Karely et al. [40] used wavelets to identify stumbles during treadmill walking.
However, the DWT features seems to be outperformed by the more specialized time- and frequency domain counterparts, suggesting that the DWT might not be the best selection for short time windows [34]. Furthermore, it seems that DWT has not yet been applied to pre-impact fall detection (author note).
2.2.2.4 Heuristic Features
Preece et al. [34] defined heuristic features as information that provides data on a specific type of attribute. The sensor-, or Inertial Measurement Unit (IMU) axis information can thus be used for calculation of qualitative features such as walking speed, body posture or free fall-velocity.
At rest, accelerometer data can be directly used for calculation of body posture, by applying trigonometric formulas. However, this requires that an initial assumption is made on the orientation of the IMU with regards to the subject.
The subscript notation G is hereinafter used for angles derived using gyroscope data and A from accelerometer data.
The pitch θ and roll ϕ can be calculated using the accelerometer data, using g as the reference.
ϕ= arctanay
az
, θ = arctanax
az
(2.10)
The total IMU tilt angle αxy can then be calculated as the norm of ϕ and θ, or as in eq. 2.11
x
z
y
g
x
y z
g αxy
θ
φ
Figure 2.2: Axes of IMU rotation (also referred to as Euler angles): αxy is the angle between the gravitational component and IMU z-axis, ϕ and θ the rotation about the x- and y-axis respectively.
αxy = arccos
az
q(a2x+ a2y + a2z)
(2.11)
Gyroscope signals can also be used for calculation of IMU tilt and data is gathered as pitch and roll by integrating the angular rate
ϕ=Z ωxdt, θ =Z ωydt (2.12)
Table 2.3: List of features used in motion monitoring systems.
Feature Equation No.
Acceleration a= [ax, ay, az] F1
Angular velocity ω= [ωx, ωy, ωz] F2
Peak-to-peak A= max(x) − min(x) F3
Root mean square xrms=q
1 N
PN −1
i=0 x2i F4
Mean µ= N1 PN −1
i=0 xi F5
Standard deviation σ=q
1 N
PN −1
i=0 (xi− µ)2 F6
Variance σ2=N1 PN −1
i=0 (xi− µ)2 F7
Sum vector magnitude a=q
a2x+ a2y+ a2z F8 Discrete fourier transform Xk = PN −1n=0 xne−i2πkNn F9 Angle between sensor and gravity θx= arcsina
y
g , θy= arcsin
ax g
F10
Skewness N σ13
PN −1
i=0 (xi− µ)3 F11
Kurtosis N σ14
PN −1
i=0 (xi− µ)4 F12
Angular acceleration α= dtdω(t) F13
Angle θ= R0tω(t)dt F14
Velocity v= R0ta(t)dt F15
Signal magnitude area SM A= 1tRt
0|x(t)|dt F16
Note: x = observation, N = number of samples, i = index of sample, g ≈ 9.82m/s2, ax,y,z
= acceleration along the x-, y- and z-axis of the IMU, ωx,y,z = angular velocity around the x-, y- and z-axis of the IMU.
2.3 Micro-Electro-Mechanical Systems Sensors
In this section, the reader is presented to the commonly used inertial MEMS-sensor accelerometer and gyroscope.
A Micro-Electro-Mechanical System is a micro-meter sized mechanical set of com- ponents that are capable of converting energy (radiation, temperature, electrical, mechanical etc) from one type to another. Such a system is referred to as a trans- ducer. Transducers are divided into two subsets; sensors and actuators.
MEMS inertial sensors utilize micro-meter sized mechanical elements for determin- ing the magnitude of the measurable property of interest. Inertial sensors are based on a classical electromechanical mass-spring system. Inertial measurement units (IMU) are small, System-On-Chip units that measure movement using accelerome- ters and gyroscopes.
MEMS-sensors are mainly fabricated in silicon (Si) using micro-machining processes similar to the semiconductor industry. The properties of silicon makes it a suitable material for MEMS products since
• Si is manufactured from quartzite (SiO2) sand, which is abundant in nature, making it a cheap source of raw material.
• Pure Si arranges in a crystalline cubic-lattice structure and can be refined to extremely high purity, with well defined mechanical properties needed in MEMS manufacturing.
• Si and its derivatives can be processed with the high accuracy and selectivity needed for creating such delicate structures.
• MEMS structures manufactured in Si allows for integration with integrated circuitry using standardized process flows.
Inertial MEMS sensors are widely used in the automotive industry, in navigation applications, structural/vibrational monitoring, consumer products and medical ap- plications. However, applicable to the semiconductor industry in general, silicon processing and the fabrication of MEMS sensors in clean-room environment require significant amounts of energy [41].
2.3.1 Triaxial Accelerometers
Triaxial accelerometers measure acceleration in three axes (ax, ay, az) by measuring the force exerted on a suspended proof mass, according to Newton’s Second Law of Motion:
F= m · a
where m is the suspended mass, constituting a damped mass-spring system. The proof-mass remains in resting position unless a force is applied. The force applied to the mass is created by two types of acceleration:
• Gravitational acceleration:
Force experienced by mass in Earth’s gravitational field, g ≈ 9.82m/s2
• Inertial acceleration:
A force on the proof-mass is created when a change in velocity occurs, where the force is counteracting the change in velocity.
atot = g + amotion
In this way, acceleration is measured as a displacement x of the proof mass. In a damped spring-mass system the resonance frequency (system bandwidth) is equal to
BW = ωresonance =
sk m
where k is the mechanical spring constant. However, there exists a trade-off between sensitivity and bandwidth, such that
Sensitivity= x a = k
m = 1
ωresonance2
Sensitivity is limited by the Brownian equivalent acceleration noise (NEA) that is partly dependant on temperature T and size of the proof-mass m3.
The displacement of the proof mass can be derived by distance sensing, where com- mon techniques used include capacitive sensing using parallel-plates or comb-drives.
Piezoresistive sensing is also regurarely applied.
Basic parameters used for characterizing accelerometers include full-scale swing in
±g, sensitivity LSB/g and bandwidth fres. Errors include noise g/√
Hz, 0g-offset, temperature- and time drift [42].
2.3.2 Triaxial Gyroscopes
Gyroscopes are used for sensing the angular rotation. MEMS gyroscopes differ from traditional gyroscopes in the sense that no rotating disk is used. In the micro- domain, surface effects dominate over gravitational and inertial effects because of scaling phenomena. This can be understood by investigating the relationship be- tween surface and volume, as they scale to smaller dimensions. Surfaces scales according to L2, while volume scale according to L3 so that at a certain point, physical effects created between surfaces will be of equal magnitude to that of gravi- tational and inertial forces. Therefore, surface effects have a large impact on MEMS structures. Issues common in this domain are surface-to-surface sticking, friction forces and increased wear in moving parts that makes it difficult to construct reli- able devices.
In order to measure angular velocity ω0 without applying a rotating disk, MEMS gyroscopes are based on the Coriolis effect. This effect is visible when a proof mass m is oscillated in the rotational plane along a radius r. As the radius r from the center of rotational motion increases, so does the rotational energy Erot
Erot= 1
2mr2ω02
The inertia of m will counteract this increase in rotational energy when r is changing, such that a force FCor is created that is proportional to ω0, orthogonal to the in- plane movement of m and to the plane of rotation. The acceleration experienced by m is
aCor = 2v × ω0
where V is the speed (frequency) of the mass. Thus, the energy used for driving the proof mass will be transferred to the sensing axis, and the angular velocity w0 is then measured as the magnitude of the oscillation in this axis. MEMS gyroscopes are much more complex in comparison to accelerometers, since a stable drive frequency is needed. This working principle is referred to as resonance mode coupling.
Basic parameters used for characterizing MEMS-gyroscopes include resolution in deg−1Hz−1/2, angle random walk (ARW) deg hr−1/2, sensitivity V deg−1 s [42].
2.3.3 Sensor Error Model
Each sensor has it’s own individual characteristic which will vary within the fab- rication tolerance as an effect of MEMS fabrication process variance. Therefore, MEMS sensors are affected by measurement errors, such as axis misalignment, im- precise scaling factors and non-linearity. In order to acquire accurate readings from the sensor, initial sensor calibration is needed. Tedaldi et al. [1] presents a sensor model that can be used for IMU sensor calibration, without the need for specialized reference equipment. The method as presented by Tedaldi et al. is now described;
In the ideal case, the orthogonal accelerometer and gyroscope frames (AF,GF) are orthogonal with respect to their own axes and no crosstalk is present. If that is the case, each of the sensors make up a orthogonal frame (AOF, GOF) which are perfectly aligned with respect to the other sensor frame. However, this is not the case due to assembly inaccuracy. Instead, the sensors form a set of sensor frames with an internal and external offset such that overlap between the sensor axes and frames are present. The AOF and GOF frames might also be biased with respect to the IMU chassi and the PCB board (the body frame BF). Furthermore, the sensor output might be corrupted by non-zero axis dependant bias.
A sensor measurement sS in the AF and GF can be transformed into the IMU chassi body frame BF
