Design Proposal of a Fall and Step Pattern Recognition System

(1)

M A S T E R’S T H E S I S

MASTER OF SCIENCE PROGRAMME Computer Science and Engineering

Luleå University of Technology

Department of Computer Science and Electrical Engineering EISLAB

Design Proposal of a Fall and Step Pattern Recognition System

Jimmie Wiklander

(2)

Abstract

The aim of this thesis was to make a design proposal of a pattern recognition system. The system should be capable to detect falls, but also steps of elderly persons with possible walking limitations. However, in order to evaluate the step detection of elderly, it was considered necessary to let the system detect steps of non-elderly persons with no walking disorders as well, allowing for recognition performance comparisons between the two step pattern types.

Two wireless sensors were developed, allowing for collection of syn- chronized signal data from test persons performing fall and step movements. The author, equipped with a sensor at the hip, was the participant in the fall data collection. The step data collection was carried out in two tests where both sensors, positioned at the hip and ankle, were used. The participants in the ﬁrst test were non-elderly with no walking disorders, and the participants in the second test were elderly persons with varying walking capabilities.

The multivariate normal density was chosen as a model for the recognition algorithm. The combinations of features considered appropriate for input to the algorithm were evaluated by the comparison of probability density plots. Running test ﬁles from the data collection with known structures through the feature extractor and algorithm generated the plots. The model parameters used in the evaluation were calculated from training samples, extracted with the feature-pair under test from the collected ﬁles.

(3)

The system was compiled allowing for diﬀerent system conﬁgurations.

The conﬁguration parameters required for each of the three pattern types were: the optimal feature-pair (for the type of pattern to be recognized) and the model parameters (calculated in the feature evaluation for this feature-pair). Threshold conﬁgurations for each pattern type were used to get a Boolean output from the algorithm.

The resulting system was evaluated by using test ﬁles with known structure (as documented by video recordings) from the data collection.

According to the evaluation performed, the system detects both fall and step patterns of elderly satisfactory. In the evaluation, the detection of steps was signiﬁcantly lower for some of the test persons, compared to the group as a whole. One person also generated more detection of non-existing steps. It is possible that the walking patterns of these persons diﬀer from the rest of the group, which could explain the results. Excluding the outliers from the step evaluation when calculating the detection percentage shows that the performance of the system is pleasing. The detection of steps is lower when detecting steps of elderly compared to non-elderly. This is however expected, since it is usual that elderly have limited walking capabilities compared to non-elderly.

(4)

Preface

This thesis is the graduate thesis of the Master of Science program in Computer Science and Engineering at Luleå University of Technology. The work has been performed at the Embedded Internet Systems Laboratory (EISLAB) within the Research Trainee program at the university.

First of all, I would like to thank my supervisor, Associate Professor Per Lindgren at EISLAB, for the help and guidance he has given me. Thanks also to Associate Professor Johan Carlson at EISLAB, for reviewing this thesis.

I would also like to thank Ph.D. students Johan Borg and Jens Eliasson, and Research Engineer Kennet Hartvik at EISLAB, which have been a great source of information and help. A special thanks goes to Professor Lars Nyberg and Ph.D. student Irene Vikman at the Department of Health Sci- ences at Luleå University of Technology, for arranging the data collection tests.

I am also grateful to have been part of the Research Trainee group of 2005/2006 and for all the good memories we share from this year. Finally, I would like to thank my family and friends for their patience and support.

Luleå, May 2006 Jimmie Wiklander

(5)

(6)

List of Acronyms

EIS Embedded Internet System GUI Graphical User Interface

IDE Integrated Development Environment LAP LAN Access Proﬁle

MVNPDF Multivariate Normal Probability Density Function PAN Personal Area Network

PD Probability Density

PRS Pattern Recognition System SMC Surface Mount Connector USB Universal Serial Bus

(11)

(12)

List of Figures

2.1 Typical PRS components . . . . 5

3.1 Picture of the Mulle platform . . . 10

3.2 Picture of the sensor interior . . . 12

3.3 Picture of the sensor developed . . . 12

4.1 Screenshot of the SensorApp program . . . 17

6.1 Plot of typical fall patterns from data collection . . . 25

6.2 Plot of typical step patterns from data collection . . . 27

6.3 Output from MVPDF, evaluating the eight possible fall feature- pairs . . . 31

6.4 Output from MVNPDF, evaluating the eight possible step (non-elderly) feature-pairs . . . 35

6.5 Output from MVNPDF, evaluating the eight possible step (elderly) feature-pairs . . . 39

7.1 Plot of diﬀerent threshold levels . . . 46

7.2 Plot of diﬀerent threshold levels magniﬁed . . . 47

(13)

(14)

List of Tables

3.1 Accelerometer specifications . . . 11 4.1 Fall types performed in the fall data collection . . . 19 4.2 Number of complete/incomplete files from first test of step

data collection . . . 19 4.3 Number of complete/incomplete ﬁles from second test of

step data collection . . . 19 6.1 Possible feature combinations . . . 26 6.2 Feature-pairs chosen for each pattern type in feature evalu-

ation . . . 30 7.1 Feature-pair configuration for different pattern types . . . . 43 7.2 Recognition algorithm configuration for different pattern

types . . . 44 7.3 Pattern detection result from simple threshold implementation 46 7.4 Pattern detection result from threshold implementation us-

ing mdac. . . 47 7.5 Threshold conﬁguration for diﬀerent pattern types . . . 48 8.1 System evaluation results when using the fall parameters . . 50 8.2 System evaluation results when using the step parameters

of non-elderly . . . 51

(15)

A.1 Computer speciﬁcations . . . 58

A.2 Bluetooth dongle speciﬁcations . . . 58

B.1 BlueSoleil speciﬁcations . . . 62

B.2 IAR Embedded Workbench speciﬁcations . . . 63

B.3 M16C-Flasher speciﬁcations . . . 63

B.4 MATLAB speciﬁcations . . . 64

B.5 MinGW Developer Studio speciﬁcations . . . 64

C.1 Files acquired in fall data collection : 2006-04-17 . . . 69

C.2 Files acquired in step data collection, test 1 : 2005-11-15 . . . 70

C.3 Files acquired in step data collection, test 2 : 2006-04-20 . . . 72

C.4 Files constructed by merging samples from ﬁles acquired in data collection . . . 75

C.5 Calculated parameters when extracting feature-pairs from random falls, acquired in the fall data collection test . . . 79 C.6 Calculated parameters when extracting feature-pairs from

random steps, acquired in the ﬁrst step data collection test . 87 C.7 Calculated parameters when extracting feature-pairs from

random steps, acquired in the second step data collection test 95

(16)

Chapter 1 Introduction

In [1], the following deﬁnition of pattern recognition can be found:

Pattern recognition is deﬁned as the process of identifying structure in data by comparison to a known structure. Patterns are typically described in terms of multidimensional data vectors, where each component is called a feature. The aim of a pattern recognition system is to associate each pattern with one of the possible pattern classes (or simply classes).

The patterns sought in this thesis are fall patterns, but also step patterns of elderly persons.

1.1 Background

It is probable that the growing elderly citizen population in the industrial countries will create strain on the current health care systems. Therefore there is an increasing emphasis on allowing for elderly people to remain their independent living at home as long as possible. Falls and fall injuries are common among old people [2], the fear of falling and the worry about not being able to call for help is therefore a major concern. The Centre for

(17)

Distance-Spanning Healthcare (CDH) at Luleå University of Technology (LTU) focuses on applied R&D projects, company development/ spin-oﬀ companies and international excellence in the ﬁeld of distance-spanning healthcare. In the CDH project, Mobile Safety Alarm; technical solutions for a mobile safety alarm, intended for elderly people, have been studied.

One objective of the project, which is also a part of this thesis, has been to design a fall pattern recognition system.

Fall incidence and fall risk factors have been studied extensively. Still, the relation between fall incidence and the amount of physical activity being performed has not been established. This can be explained by the lack of instruments accurate enough to provide valid measurements of physical activity among old and disabled persons. Therefore, the development of accurate and feasible methods to monitor physical activity is an important issue in the care of old people. These methods are important both for studying the activity rate and its relation to different health parameters and accidents, and to evaluate activity and exercise interventions. The most basic performance of physical activity is walking on one’s legs, and, for most people, the number of steps taken during a certain time interval has a strong correlation to general physical activity and energy expendi- ture [3]. Plain, waist-worn pedometers have been demonstrated as very feasible for use among children, adults and vigorous old individuals. Con- versely, among old people with altered walking patterns, such as lower gait speed or shorter step-length and reduced acceleration during toe-off and heel-strike phases, there is a significant risk of underestimating the number of steps taken [3]. This is particularly obvious in residential care populations where ongoing work indicates that a small percentage of ac- tual steps are registered by the apparatus [4]. A better precision, however not adequate, has been shown for accelerometers worn at the ankle [4].

At the Department of Health Sciences at Luleå University of Technology research is carried out where the physical activity of elderly needs to be

(18)

1.2. OBJECTIVE

monitored. Therefore, one objective of this thesis has been to design a step pattern recognition system for elderly people.

1.2 Objective

The objective of this thesis was to make a design proposal of a pattern recognition system. The system should be capable to detect falls, but also steps of elderly persons with possible walking limitations. However, in order to evaluate the step detection of elderly, it was considered necessary to let the system detect steps of non-elderly persons with no walking disorders as well, allowing for comparisons of the performance between the two step pattern types.

When designing a pattern recognition system, a sensor is needed. It was required that the sensor should be small, user-friendly and allow for wireless transfer of signal data. EISLAB, at Luleå University of Technology has developed an Embedded Internet System (EIS) platform, suitable for these types of applications. Therefore, it was decided that the development of a sensor built on this platform, should be included.

1.3 Method

The method for developing the fall and step pattern recognition system in this thesis was to ﬁrst develop a sensor consisting of a EIS platform and an accelerometer. The sensor was used to collect signal data of fall patterns, step patterns of elderly and step patterns of non-elderly. A recognition algorithm was implemented. Appropriate features were evaluated by looking at the output of the recognition algorithm when testing diﬀerent feature combinations. This was performed by using the implementation of the feature extractor together with the implementation of the recognition algorithm. The system was then compiled in MATLAB, allowing for dif-

(19)

ferent system conﬁgurations determined by the sought pattern type. The conﬁguration parameters chosen for each of the three pattern types were:

the optimal feature-pair (for the type of pattern to be recognized) and the model parameters (calculated in the feature evaluation for this feature- pair). To get a Boolean output from the system, threshold configurations for the different pattern types where chosen. Finally, the system was evaluated by using the files with known structure from the data collection.

1.4 Thesis Structure

The thesis starts with an introduction and background of the work performed. In chapter 2, the design actions used are covered and in chapter 3, the development of the data collection sensor is described. The data collection tests and the results of the data collection can be found in chapter 4. In chapter 5, the model used as the recognition algorithm is described and in chapter 6, the features needed to implement the feature extractor are chosen. The compilation of the system is covered in chapter 7, and in chapter 8 the implemented system is evaluated. Finally, in chapter 9, the resulting system and its performance is discussed.

(20)

Chapter 2 Method

This section describes the method used to design the Pattern Recognition System (PRS).

2.1 Typical Pattern Recognition System Compo- nents

In ﬁgure 2.1, typical PRS components are shown. A sensor converts

Figure 2.1: Typical PRS components

physical inputs into signal data. A feature extractor measures properties of the signal data useful for the recognition algorithm. The recognition algorithm uses these features to associate the current state of the signal to a speciﬁc class.

(21)

2.2 Design Actions

The method used to design the PRS was to ﬁrst design the components of the system and later perform a system compilation, combining the components into a system. In order to design the components, the design actions described next were performed.

2.2.1 Sensor Development

The sensor was developed by connecting a two-axis accelerometer to an Embedded Internet System platform. Software allowing for sampling of accelerometer data and wireless transferring of the data was developed.

2.2.2 Data Collection

The purpose of the data collection was to get sample data, later used when training and testing the system. Data representing three pattern types were collected: fall, step elderly and step non-elderly.

2.2.3 Recognition Algorithm

By looking through relevant literature, a recognition algorithm was chosen. A model describing the recognition algorithm was implemented in MATLAB.

2.2.4 Features

In order to design the feature extractor, appropriate features for each of the three pattern types were chosen. The method to choose the most appropriate features was to first implement a feature extractor in MATLAB, capable of using different features. Then, the output of the recognition algorithm when training and testing the algorithm using different feature

(22)

2.2. DESIGN ACTIONS

combinations was analyzed. The feature combination producing the best result for each pattern was chosen.

2.2.5 System Compilation

The PRS was compiled into a complete system in MATLAB by using the feature extractor and the recognition algorithm previously implemented.

The system allows for different configurations (determined by the sought pattern type) by using different system parameters and features. Thresh- olds for the different configurations, allowing for a Boolean output, were also added to the recognition algorithm. The system was evaluated by using data files with known structures, acquired in the data collection, as input.

(23)

(24)

Chapter 3 Sensor Development

The development of a pattern recognition system often contains some kind of data collection. In order to perform the data collection, one or several sensors are needed. Since a EIS platform suitable for these types of applications have been developed at EISLAB at Luleå University of Technology, it was decided that the development of a sensor should be included as a part of the thesis.

3.1 Requirements

In order to make the collection of data eﬀective, it was desirable that the sensor would be small, user-friendly and be capable to perform wireless transfers of data. It was also considered that simultaneous acceleration measurements on diﬀerent locations of the body would be needed in the data collection.

3.2 Components Chosen

This section describes the components chosen for the sensor development.

(25)

3.2.1 EIS Platform: Mulle

Mulle, ﬁgure 3.1, is a small Embedded Internet Systems platform, based on the Renesas M16C microprocessor. It is developed by EISLAB at Luleå

Figure 3.1: Picture of the Mulle platform and a Swedish coin.

University of Technology. Mulle has both analog and digital interfaces. The analog interface can be used to connect an accelerometer. Mulle allows for wireless communication by the integrated bluetooth module. The platform also has signal processing capabilities and the power consumption is low.

The size of the platform is only 25 x 23 x 5 mm. More details about Mulle can be found in [5].

3.2.2 Accelerometer

The accelerometer chosen for the sensor development is the ADXL322, manufactured by Analog Devices. The speciﬁcations can be found in table 3.1.

(26)

3.3. REALIZATION

Table 3.1: Accelerometer, ADXL322, speciﬁcations.

Product Name ADXL322

# of Axes 2

Range +/- 2g

Sensitivity 420 mV/g

Sensitivity Accuracy (%) +/- 10

Output Type Analog

Max Bandwidth (kHz) 2.5 Noise Density (μg/rtHz) 220 Voltage Supply (V) 2.4 to 6 Supply Current (mA) 0.5

Temperature Range -20 to 70^◦C

3.3 Realization

The sensor was realized by using a Mulle together with an accelerometer, both described in section 3.2. The small size of the Mulle made it possible to fit the platform, accelerometer and a battery in a small box, 18 x 56 x 71 mm, thereby fulfilling the size requirements. The wireless communication was achieved by using the Bluetooth module of the Mulle. In order to make the sensor user-friendly, the only interfaces added was a power-switch and a dc-jack (used to charge the battery). The sensor was configured and programmed enabling the sensor to perform the data collection. The source code however, is not described or included in this thesis. In order to perform simultaneous measurements on different locations, two identical sensors were constructed. The interior of one of the sensors can be seen in figure 3.2.

3.4 Functionality

The sensor constructed can be seen in ﬁgure 3.3. It has a power-switch and a dc-jack, used for charging the battery. When turning on the sensor,

(27)

Figure 3.2: Box containing the Mulle (middle), a battery (right) and a dc-jack and power-switch (left). The chip above the box contains the accelerometer and can be attached to the Mulle.

Figure 3.3: The sensor resulting from the sensor development.

(28)

3.4. FUNCTIONALITY

it ﬁrst makes a DHCP request. If the request is successful an IP address is assigned and the sensor enters idle mode, waiting for a connection from a client. The client can send a start command to the sensor when connected.

Upon reception of the start command, the sensor starts to sample data from the two-axis accelerometer at a rate of 50 Hz. The horizontal and vertical acceleration samples are two bytes each. The total amount of data buﬀered is therefore 200 bytes/second. Each second, a TCP packet containing the buﬀered 200 bytes is sent to the client. The client can end the sampling of data by invoking the stop command. It is then possible to either send a new start command, or to terminate the connection; the latter causes the sensor to enter idle mode again.

(29)

(30)

Chapter 4 Data Collection

In order to perform the data collection, two sensors were developed as described in chapter 3. The data was collected in diﬀerent tests. When per- forming the step data collection tests, voluntary test persons were used.

However, due to the risk of injuries, the author was the only participant in the fall data collection. The step data collection tests were recorded on video, which makes it possible to later evaluate the performance of the system by comparing the output with the recordings. The recordings can be found on the CD-ROM, appended to the copy of this thesis, which is archived at the Department of Computer Science and Electrical Engineer- ing at Luleå University of Technology.

4.1 Technical Setup

In order to collect data from the two sensors, a Bluetooth access point and a laptop computer equipped with a Bluetooth dongle were used. The laptop and the Bluetooth access point were placed right next to the course. By using the LAN Access Proﬁle (LAP) service of the Bluetooth access point it is possible for the computer to access the LAN, consisting of the sensors connected to the Bluetooth access point. The sensors automatically connect

(31)

to the Bluetooth access point when switched on, which assigns pre speciﬁed IP addresses to the sensors. The SensorApp program, described in appendix B, were run on the computer. From the Graphical User Interface (GUI) of this program, ﬁgure 4.1, it is possible to establish a connection and send a start command to the sensor.

4.2 Acquiring Data

The sensor will on reception of the start command begin to sample data from the two-axis accelerometer. The data is sent to the SensorApp program, ﬁgure 4.1, which stores the data on the hard drive of the computer. The data is stored in one ﬁle for each sensor containing the sampled vertical and horizontal accelerations. Details about the equipment used can be found in appendix A.

(32)

4.3. COLLECTING THE DATA

Figure 4.1: Screenshot of the SensorApp program.

4.3 Collecting the Data

Both the fall and step data was collected in tests using the sensor described earlier. With the sensor attached to the body, the person can perform appropriate movements to generate the data needed. The results of the data collection are described in section 4.4.

4.3.1 Fall Data

The fall data were collected in a test using one sensor. The author was attached with a sensor at the hip where after diﬀerent kinds of falls, such as forward and sideways, was performed. A thin mattress was placed

(33)

on the ground to avoid injuring the author when making the falls. When falling forward the sensor where located at the left hip. To avoid breaking the sensor when performing the sideway falls, the sensor was attached to the hip in opposite to the falling direction.

4.3.2 Step Data

The step data was collected in two tests. The tests were planned and performed in cooperation with the Department of Health Sciences, Luleå University of Technology. The participants in the first test were 19 voluntary test persons with no walking disorders and the participants in the second test were 18 voluntary elderly persons with varying walking capabilities. In both tests the participants where in turn attached with two sensors, one at the hip and one at the ankle. They were instructed to walk around cones and go through a passage in a pre specified course on a plane surface. The course in the first test was designed to generate data from different types of motion, such as walking forward, backward and sideways. The second test was designed to generate data from motions such as walking forward and getting down and up from a chair, i.e. examples of normal daily life activities.

4.4 Results

The ﬁles resulting from the data collection tests described earlier contain sampled accelerometer data in horizontal and vertical direction representing the movements performed by the test persons when performing the tests. Lists over the collected ﬁles containing information about size and completeness can be found in appendix C.

(34)

4.4. RESULTS

4.4.1 Fall Data

The types of falls tested and the number of falls collected on each type can be seen in table 4.1.

Table 4.1: Fall types performed in the fall data collection.

Fall type Number of falls

Forward 24

Sideways Left 21 Sideways Right 20

4.4.2 Step Data

Some of the ﬁles acquired in the tests of the step data collection were not complete due to connectivity problems. The number of complete/incomplete ﬁles acquired in the two tests can be seen in table 4.2 and table 4.3 respec- tively.

Table 4.2: Number of complete/incomplete ﬁles from ﬁrst test of step data collection.

Ankle Hip Complete ﬁles 14 18 Incomplete ﬁles 5 1

Total ﬁles 19 19

Table 4.3: Number of complete/incomplete ﬁles from second test of step data collection.

Ankle Hip Complete ﬁles 15 13 Incomplete ﬁles 3 5

Total ﬁles 18 18

(35)

(36)

Chapter 5 Recognition Algorithm

The multivariate normal density is an appropriate model in the case where the feature vectors for a given class are continuous-valued, randomly cor- rupted versions of a single typical or prototype vector [6]. Therefore, it was chosen as the model for the recognition algorithm in this thesis.

5.1 Multivariate Normal Density

The general multivariate normal density in d dimensions is written as

p(x)= 1 (2π)^d/2√

|Σ|e⁻¹²^(x^−μ)^t^Σ⁻¹^(x^−μ), (5.1) where x is a d-component column vector,μ is the d-component mean vector, Σ is the d-by-d covariance matrix, and |Σ| and Σ⁻¹are its determinant and in- verse, respectively [6]. In order to use this model the unknown parameters μ and Σ need to be estimated and the method for doing this is described in section 6.4.

(37)

5.2 Implementation

The multivariate normal density is represented in MATLAB by the function, mvnpdf¹. The syntax used in this thesis was y = mvnpdf(X,mu,SIGMA), which returns the density of the multivariate normal distribution with mean mu and covariance SIGMA, evaluated at each row of X.

1Found in MATLAB Statistics Toolbox.

(38)

Chapter 6 Features

In [7], a wearable sensing platform was constructed to recognize everyday postures and activities of a person. The features used there as input to the recognition algorithm were the running mean, μ, and variance, σ², computed over a window of 50 samples. Detection of a fall or a step is quite similar to detecting diﬀerent activities of a person. Therefore it was considered that the same features would be appropriate to use in this work.

6.1 Choosing Window Size

The features in [7] was computed over a sliding window of 50 samples, sampled at a rate of 92 Hz, which corresponds to roughly 0.5 sec. In this work the sample rate was 50 Hz. Therefore, to get the same window size with respect to time as in [7], the window was chosen to be 25 samples wide.

(39)

6.2 Possible Features

It was considered that one feature in each of the two acceleration directions, a feature-pair, would be sufficient in this work. In each direction, either the mean or variance can be chosen, producing four different feature-pair combinations. Comparing the characteristics of the fall patterns, figure 6.1, with the characteristics of the step patterns, figure 6.2, gives an indication that the optimal feature-pair, expressing the characteristics of the two patterns, might differ. Correspondingly, when comparing the characteristics of the step patterns of elderly with the characteristics of the step patterns of non-elderly, figure 6.2a and figure 6.2b respectively, indicates that the op- timal feature-pair for the two different step pattern types might also differ.

Therefore the different combinations of feature-pairs must be evaluated for each of the three patterns types: fall, step elderly and step non-elderly, in order to choose the best possible. However, to perform the evaluation, feature-pair values, or training samples, must be extracted from training files, section 6.4.1. This requires for a priority to be put on one of the features in the feature-pair. The result of the evaluation becomes different depending on which of the features in the pair that have the priority. Hence, the four possible feature-pair combinations, doubles to eight, all listed in table 6.1.

6.3 Feature Extractor

The feature extractor was implemented in MATLAB. It is possible to choose which one of the eight possible feature-pair combination it should use.

When run on a signal data ﬁle, it traverses a sliding window over the data and extracts one feature-pair value for each window position, using the window size of 50 samples as discussed earlier.

(40)

6.4. FEATURE EVALUATION METHOD

6.4 Feature Evaluation Method

The evaluation of the possible feature-pairs was performed by running appropriate ﬁles from the data collection through the implemented feature extractor. The features are then fed to the recognition algorithm described in chapter 5. Because the movements generating the data were documented

1 2 3

0 200 400 600 800

Horizontal acceleration

Time, s

Accelerometer values

1 2 3

0 200 400 600 800

Vertical acceleration

Time, s

(a) Fall forward (typical fall f.dat).

1 2 3

0 200 400 600 800

Time, s

1 2 3

0 200 400 600 800

Time, s

(b) Fall sideways, left (typical fall side l.dat).

Figure 6.1: Typical fall patterns from data collection. Name of sourceﬁle printed in caption parenthesis.

(41)

1 2 0

200 400 600 800

Time, s

1 2

0 200 400 600 800

Time, s

(c) Fall sideways, right (typical fall side r.dat).

Table 6.1: Possible feature combinations for each of three pattern types: fall, step elderly and step non-elderly. Bold features indicates a priority feature.

Horizontal acc. Vertical acc.

μ μ

μ σ²

σ² μ

σ² σ²

μ μ

μ σ²

σ² μ

σ² σ²

when collecting the data, the output from the algorithm can be compared with “reality”, and thereby the most suitable feature-pair for each of the three pattern types sought can be found. However, ﬁrst of all, in order to use the algorithm, the unknown parametersμ and Σ, cf. equation 5.1, must be determined. The method to ﬁnd these parameters in this work is to use a labeled set of extracted feature-pairs, or training samples, with known class to train the recognition algorithm.

(42)

6.4. FEATURE EVALUATION METHOD

0.5 1

0 100 200 300 400 500 600 700 800 900

Acceleration

Time, s

horizontal acceleration vertical acceleration

(a) Step, non-elderly (typical step test1.dat).

0.5 1 1.5

0 100 200 300 400 500 600 700 800 900

Acceleration

Time, s

horizontal acceleration vertical acceleration

(b) Step, elderly (typical step test2.dat).

Figure 6.2: Typical step patterns from data collection. Name of sourceﬁle printed in caption parenthesis.

(43)

6.4.1 Extracting Training Samples

The method to extract the training samples in this work was to use MATLAB- scripts implemented by the author. From the scripts it is possible to ﬁrst manually traverse appropriate signal data from the data collection and mark the parts containing the patterns sought. The marked data can then be traversed with a sliding window again automatically. When the sliding window passes a marked part, the feature-pair corresponding to the highest value of the priority feature in that part, is extracted into a training samples matrix, which is returned. Each row in this 2-column matrix represents the values of the feature-pair extracted where the ﬁrst column contains the value of the horizontal direction feature. From this matrix,μ and Σ, can be determined.

6.4.2 Determining Parameters

The recognition algorithm parameters, denotedμ and Σ, can both be cal- culated from the training samples matrix. The 2-component mean vector,μ, is equivalent to the mean of the values in each column of the 2-column training samples matrix. The 2-by-2 covariance matrix,Σ, is equivalent to the covariance of the values in the training samples matrix.

6.4.3 Evaluating Features

The output from the recognition algorithm is a probability plot, representing the resemblance between the feature-pairs of each position of the sliding window traversing the test file; and the feature-pair representing the sought pattern, as expressed by the recognition algorithm parameters. By comparing the output for each pattern type when using identical test files, to the real movements used to generate the test files, the most advantageous feature-pair for each pattern type, can be chosen.

(44)

6.5. FEATURE EVALUATION RESULTS

6.5 Feature Evaluation Results

In order to perform the feature evaluation some files were constructed, merging samples from the data collection into new files. The data collec- tion is described in chapter 4. In appendix C, lists, containing the filenames and sizes of the created files, and also references to the samples used to create the files, can be found. Appendix C also contains the recognition algo- rithm parameters used when creating the probability plots in this section, together with references to the samples used to estimate the parameters.

6.5.1 Fall Features

In the fall feature-pair evaluation, a file containing accelerometer data for 20 falls, randomly selected from the fall data collection files, were used to estimate the recognition algorithm parameters, but also as the test file. In figure 6.3, the probabilities, used to evaluate the eight possible feature-pairs, are plotted.

6.5.2 Step Features

In the step feature-pair evaluation, the data obtained in the ﬁrst and second step data collection tests, were evaluated separately. In the ﬁrst test, the data was generated by voluntary test persons with no walking disorders.

In the second test, the data was generated by voluntary elderly persons with varying walking capabilities.

Non-elderly

In the evaluation, a file containing accelerometer data for 20 steps, randomly selected from the first step data collection test, where used to esti- mate the parameters, but also as the test file. In figure 6.4, the probabilities, used to evaluate the eight possible feature-pairs, are plotted.

(45)

Elderly

In the evaluation, a file containing accelerometer data for 20 steps, randomly selected from the second step data collection test, where used to estimate the parameters, but also as the test file. In figure 6.5, the probabil- ities, used to evaluate the eight possible feature-pairs, are plotted.

6.6 Feature Evaluation Discussion

By inspecting the eight probability plots generated for each of the three pattern types, figure 6.3 to figure 6.5, one feature-pair producing the best result for each type must be chosen. In all the three cases there should only be a high probability where a movement corresponding to the pattern sought have occurred. Because the test files contains 20 sought patterns, the optimal result should generate a graph containing 20 narrow peaks. In table 6.2, the graphs considered best, and the corresponding feature-pair chosen, is shown.

Table 6.2: Feature-pairs chosen for each pattern type based on the feature evaluation graphs. Priority feature in bold.

Pattern type Plot considered best Feature-pair chosen

fall ﬁgure 6.3g σ²,μ

step (non-elderly) ﬁgure 6.4b μ,σ²

step (elderly) ﬁgure 6.5f μ,σ²

(46)

6.6. FEATURE EVALUATION DISCUSSION

Figure 6.3: Output from MVNPDF, evaluating the eight possible fall feature-pairs, a-h. Priority feature in bold. Testﬁle: random falls.dat.

40 80 120 160 200

0 1 2 3 4 5 6 7

x 10⁻⁵ Output from MVNPDF

Time, s

Probability

(a) μ, μ

40 80 120 160 200

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6

x 10⁻⁶ Output from MVNPDF

Time, s

Probability

(b) μ, σ²

(47)

40 80 120 160 200 0

1 2 3 4 5 6 7 8

x 10⁻⁷ Output from MVNPDF

Time, s

Probability

(c)σ²,μ

40 80 120 160 200

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6

1.8 x 10⁻⁸ Output from MVNPDF

Time, s

Probability

(d) σ²,σ²

(48)

40 80 120 160 200

0 0.2 0.4 0.6 0.8 1 1.2

x 10⁻⁴ Output from MVNPDF

Time, s

Probability

(e)μ, μ

40 80 120 160 200

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6

Time, s

Probability

(f)μ, σ²

(49)

40 80 120 160 200 0

0.5 1 1.5 2 2.5

Time, s

Probability

(g) σ²,μ

40 80 120 160 200

0 0.2 0.4 0.6 0.8 1 1.2

x 10⁻⁸ Output from MVNPDF

Time, s

Probability

(h) σ²,σ²

(50)

Figure 6.4: Output from MVNPDF, evaluating the eight possible step (non-elderly) feature-pairs, a-h. Priority feature in bold. Testﬁle: random steps test1.dat.

2 4 6 8 10 12 14 16 18 20 22 0

0.5 1 1.5 2 2.5

Time, s

Probability

(a) μ, μ

2 4 6 8 10 12 14 16 18 20 22 0

0.5 1 1.5 2 2.5

Time, s

Probability

(b) μ, σ²

(51)

2 4 6 8 10 12 14 16 18 20 22 0

0.5 1

1.5 x 10⁻⁶ Output from MVNPDF

Time, s

Probability

(c)σ²,μ

2 4 6 8 10 12 14 16 18 20 22 0

1 2 3 4 5 6 7 8

x 10⁻⁹ Output from MVNPDF

Time, s

Probability

(d) σ²,σ²

(52)

2 4 6 8 10 12 14 16 18 20 22 0

0.2 0.4 0.6 0.8 1 1.2 1.4 1.6

Time, s

Probability

(e)μ, μ

2 4 6 8 10 12 14 16 18 20 22 0

0.5 1 1.5 2

Time, s

Probability

(f)μ, σ²

(53)

2 4 6 8 10 12 14 16 18 20 22 0

0.5 1 1.5 2

Time, s

Probability

(g) σ²,μ

2 4 6 8 10 12 14 16 18 20 22 0

1 2 3 4 5 6 7

x 10⁻⁹ Output from MVNPDF

Time, s

Probability

(h) σ²,σ²

(54)

Figure 6.5: Output from MVNPDF, evaluating the eight possible step (elderly) feature-pairs, a-h. Priority feature in bold. Testﬁle: random steps test2.dat.

4 8 12 16 20 24 28 32

0 1 2 3 4

Time, s

Probability

(a) μ, μ

4 8 12 16 20 24 28 32

0 0.2 0.4 0.6 0.8 1

x 10⁻⁵ Output from MVNPDF

Time, s

Probability

(b) μ, σ²

(55)

4 8 12 16 20 24 28 32 0

1 2 3 4 5 6

Time, s

Probability

(c)σ²,μ

4 8 12 16 20 24 28 32

0 1 2 3 4 5 6 7

Time, s

Probability

(d) σ²,σ²

(56)

4 8 12 16 20 24 28 32

0 1 2 3 4 5 6 7

Time, s

Probability

(e)μ, μ

4 8 12 16 20 24 28 32

0 1 2 3 4 5 6

Time, s

Probability

(f)μ, σ²

(57)

4 8 12 16 20 24 28 32 0

1 2 3 4 5 6 7 8

Time, s

Probability

(g) σ²,μ

4 8 12 16 20 24 28 32

0 1 2 3 4 5 6

Time, s

Probability

(h) σ²,σ²

(58)

Chapter 7 System Compilation

The system was compiled using the feature extractor described earlier together with the existing recognition algorithm. However, the output from the recognition algorithm must be further processed in order to be able to count the number of patterns detected.

7.1 Feature Extractor

The implementation of the feature extractor, described in section 6.3, was used in the system compilation. The feature-pair conﬁguration to use for the diﬀerent pattern types can be found in table 7.1.

Table 7.1: Feature-pair conﬁguration for diﬀerent pattern types. Priority feature in bold.

Pattern type Feature-pair conﬁguration

fall σ²,μ

step (non-elderly) μ,σ²

step (elderly) μ,σ²

(59)

7.2 Recognition Algorithm

The recognition algorithm implementation used earlier was used in the system compilation as well. However, the output must be further processed to get a Boolean output and thereby enable pattern-counting capabilities, instead of just returning the probability. This can be accomplished by the use of a threshold. The threshold implementation is described in section 7.3. The recognition algorithm also needs different configuration parameters for the three different pattern types. As recognition parameters, the parameters corresponding to the feature-pairs considered best in the feature evaluation were chosen. In table 7.2, the recognition algorithm configuration to use for the different pattern types can be found.

Table 7.2: Recognition algorithm conﬁguration for diﬀerent pattern types.

Pattern type Recognition algorithm conﬁguration

μ^t Σ

fall

574 6583 297 −35268

−35268 14154173

step (non-elderly)

536 6101

932 24353 24353 4781228

step (elderly)

1907 523

1058539 −5966

−5966 536

7.3 Threshold

To get a Boolean output from the system, using the output from the recognition algorithm, a threshold can be used. If the probability exceeds the threshold value, a detection has occurred. Because the values of the probability density function depend on the covariance parameter, it is not possible to have a constant threshold for the system. Instead, the threshold can

Design Proposal of a Fall and Step Pattern Recognition System

M A S T E R’S T H E S I S

Design Proposal of a Fall and Step Pattern Recognition System

Jimmie Wiklander

Abstract

Preface

Contents

List of Acronyms

List of Figures

List of Tables

Chapter 1 Introduction

1.1 Background

1.2 Objective

1.3 Method

1.4 Thesis Structure

Chapter 2 Method

2.1 Typical Pattern Recognition System Compo- nents

2.2 Design Actions

Chapter 3

Sensor Development

3.1 Requirements

3.2 Components Chosen

3.3 Realization

3.4 Functionality

Chapter 4

Data Collection

4.1 Technical Setup

4.2 Acquiring Data

4.3 Collecting the Data

4.4 Results

Chapter 5

Recognition Algorithm

5.1 Multivariate Normal Density

5.2 Implementation

Chapter 6 Features

6.1 Choosing Window Size

6.2 Possible Features

6.3 Feature Extractor

6.4 Feature Evaluation Method

6.5 Feature Evaluation Results

6.6 Feature Evaluation Discussion

Chapter 7

System Compilation

7.1 Feature Extractor

7.2 Recognition Algorithm

7.3 Threshold