Classification of the different

Juni 2021

Classification of the different

movements (walk/trot/canter) and data collection of pose estimation

Moa Sjöström

Teknisk- naturvetenskaplig fakultet UTH-enheten

Besöksadress:

Ångströmlaboratoriet Lägerhyddsvägen 1 Hus 4, Plan 0

Postadress:

Box 536 751 21 Uppsala

Telefon:

018 – 471 30 03

Telefax:

018 – 471 30 00

Hemsida:

http://www.teknat.uu.se/student

Classification of the different movements (walk/trot/canter) and data collection of pose estimation

Sjöström

Pose estimation uses computer vision to predict how a body moves. The likeliness off different movements is predicted with a neural network and the most likely pose is predicted. With DeepLabCut, an open source

software package for 3D animal pose estimation, information about animals behaviour and movement can be extracted. In this report the pose estimation of horses four hooves is used. By looking at the position of the hooves different gaits can be identified. Horses used for riding in the major

disciplines in Sweden have three different gaits, walk, trot and canter. Walk is a four-stoke gait, trot is two-stoke and canter is three-stoke. This can be used to classify the different gaits. By looking at the hooves movement in vertical position over time and fitting a sinewave to the data it is possible to see the phase difference in the hooves movement. For walk and trot there was a significant pattern which was easy to identify and corresponded well to the theory of horses movement. For canter our pre-trained model lacked in accuracy, so the output data were insufficient. Therefore it was not possible to find a significant pattern for canter which corresponds to the theory of horses movements. The Fourier Transform were also tested to classify the gaits and when plotted it was possible to detect the different gaits, but not significant enough to be reliable for different horses in different sizes running in different paces. It was also possible to add the data for all four hooves together and fit a sinewave to the added data, and then compare it with the sinewaves for each hoof separately. Depending on the gait the frequency of the sinewaves differed between the hooves separately and added together and the gaits could be identified.

Handledare: Elin Lorin Ämnesgranskare: Chao Xu Examinator: Martin Sjödin

ISSN:1401-5757, MAT-VET-F-21026

Contents

1 Introduction 4

1.1 Introduction to the thesis . . . 4

1.2 Problem description . . . 4

2 Theory 5 2.1 Gaits of the horse . . . 5

2.2 Pose Estimation . . . 5

2.3 DeepLabCut . . . 6

2.4 Sine curves . . . 6

2.5 Fourier transform . . . 7

3 Method 9 3.1 Fourier transform . . . 11

3.2 Fitted sine curve . . . 11

3.3 Different phases . . . 12

4 Results and discussion 12 4.1 Fourier transform . . . 12

4.2 Different phases . . . 13

4.3 Added curves . . . 15

4.4 Other possible solutions . . . 16

5 Conclusions 17

6 References 18

7 Popul¨arvetenskaplig sammanfattning 19

8 Appendix 19

1 Introduction

1.1 Introduction to the thesis

Machine learning is considered to be a part of artificial intelligence. Machine learning algorithms improve automatically when being used through experience [1]. The machine learning algorithms build on a set of training data where the algorithm makes decisions or predictions without being explicitly programmed to make these decisions. The primary aim is to make the computers to learn automatically. Machine learning algorithms are often categorized as supervised or unsupervised. Supervised algorithms have a training data set that contains information on how a specific input relates to a specific output, the data is labeled [1]. Unsupervised algorithms only have inputs and then finds a structure in the data and then categorizes it. Machine learning is used widely in many different areas of application, such as medicine and email filtering, and many more areas where it is difficult to use explicit programming to solve tasks[2].

One underline of machine learning is pose estimation, where you basically find joints on a human, or in this case, a horse, from a video and then predicts its movement. Pose estimation can be done in both 2D, with only x and y coordinates, and in 3D with a z coordinate added. Pose estimations on humans is widely used in for example animation, gaming etc. To do a pose estimation the object is first tracked and the joints are located.

Then the model predicts the joints movement and therefor also a pose. The pose goes through a neural network which scores the likeliness of different poses. From here it is possible to sample a great amount of data to use to describe different movements. This can be done for both humans and animals, for example horses.

According to the Swedish Board of Agriculture there is around 355 500 horses in Sweden [3]. Many of those horses are used for riding and competition, for example dressage and showjumping. The horse business in Sweden capitalises around 72 billion swedish crowns every year. The sport is not only big in Sweden, horseback riding is enjoyed all over the world, but sometimes for different purposes. Horses moves in some different gaits, most common is walk, trot and canter. Each gait has its own moving pattern and are used in different ways.

Out in nature horses moves long distances to find food and their body are build to be moving most of the time, and to stay healthy it is important for horses kept in stables to exercise every day and have the ability to move around. This benefits both their blood circulation as well as their tendons and joints.

Many riders are interested in their horses movements, both for the horses body strength but also for progress in their training. In showjumping for example the jump curve of the horse is very beneficial to optimize for increased progress and the possibility to jump bigger fences. It is also of advantage to know how much you ride in each gait and in which lap, then it is possible to make sure the horse gets equilateral training, which is important for the horses durability and well being. It is also important for most riders to be able to follow their horses development, for example to know how much more they can longer the horse’s steps in the rising trot now compared to three months earlier.

EquiEye is an app that tracks the movement of the horse [4]. A lot of riders base their training on their intuition instead of hard data. EquiEye wants to extract information from a video and deliver concrete data and a tool for riders to follow their horses development. When riding there is quite a lot of equipment, therefor it is beneficial not to add any extra gear. EquiEye uses computer vision and pose estimation to collect data about a horse’s movement, and from there extract information needed.

1.2 Problem description

There already is a trained model for pose estimation of the horse for this project, which gives 22 different joints to look at. The model gives information about the joints estimated position in 2D and also the likeliness of this position. This output information is used to find a pattern and classify which gait the horse is moving in.

The pose estimation model have some limitations at the moment. As an example the video analyzed must be taped with the camera hold still, and the horse passing the camera from the left side to the right side. The pose estimation are lacking in accuracy when the horse is cantering, which gives unstable data to analyze.

The goal of this thesis is to find a way to classify the three most common gaits of the horse from a movie, only using raw data. The gaits are walk, trot and canter. To classify the different gaits a unique pattern need to be found for each gait, or a neural network need to be set up and trained to identify the different gaits.

2 Theory

2.1 Gaits of the horse

A horse can move in many different gaits depending on the breed of the horse. The three most common natural gaits is however walk, trot and canter. Islandic horses have as an example two additional gaits, called t¨olt and flying pace, as well as other breeds also has some additional gaits. This thesis will however only cover the three main gaits for riding horses in Sweden. There is some characteristics for every gait which decides which gait it is, this is for example in which order the horse moves its hooves, if there is any flying moment between moving the hooves and so on. The pace of the gaits is also relevant. Most common is that walk is the slowest gait, followed by trot which is a bit faster. The fastest gait of these three is canter. When riding it is common that you change the pace of the gaits, which means canter can sometimes be slower than trot, which means that just looking at the pace is a poor way to classify gaits. However, there is more ways to separate the gaits from each other.

Walk is a 4-stoke gait, which means that the horse walk with one foot at the time until all hooves have been moved[5]. The pattern goes as follows, outer-hind-hoof, outer-front-hoof, inner-hind-hoof, inner-front-hoof, and then repeats that sequence over and over again.

Trot on the other hand is a equilateral 2-stoke gait. The horse trots with diagonal hoof movement. The pattern for the moving hooves goes as follows, outer-hind-hoof and inner-front-hoof at the same time, then a short floating moment, followed by the other diagonal, inner-hind-hoof and outer-front-leg at the same time, then a short floating moment again.

Canter is a bit more complex. The horse have a left-canter and a right-canter, this depends on which hoof the horse puts down first. For a horse running with its right side to the camera, as the case is in this thesis, the pattern goes as follows, outer-hind-hoof followed by inner-hind-hoof and outer-front-hoof, then inner-front-leg, followed by a floating moment.

The movement of each hoof is the same for each gait, independent of the pace. There is also other body parts of the horse that moves in a different pattern depending on which gait the horse is moving in, but they are not as significant as the hoof movement, and changes depending on the pace.

2.2 Pose Estimation

Pose estimation uses computer vision to predict how a body part or joint moves from a simple picture or a movie. Actually there is not a big difference between pose estimation from a picture or a video - a video is simply many pictures in a row. There is both single person pose estimation (SPPE), where there is only one person visible in the frame and multiple person pose estimation (MPPE). SPPE is of course much easier than MPPE, which also have to handle the event of two, or more, people’s bodys interacting with each other [7].

For MPPE each person first has to be detected and separated for the computer before a pose estimation can be preformed for each person separately. To do a pose estimation a object is tracked, this can be a human or some animal. A neural network is used to predict the likeliness of different poses, and the most likely pose is the one to be predicted. For a good pose estimation mostly deep neural networks is used.

A neural network is a non-linear function that describes a prediction of the output pose as a nonlinear function of the input pose[6]. The idea of a neural network is to mimic the way the human brain works. A neural network consists of a serie of algorithms that aims to recognize underlying relationships in a set of data. The nonlinear function, which is the neural network, can be obtained in many ways. The idea for neural networks is to use linear regression in several of layers and a activation function, which is a nonlinear scalar function. For one layer several of generalized linear regression models is constructed. With only one layer of those linear regression models a neural network is constructed. To create a deep neural network several of layers are built and stacked on top of each other. Deep neural networks can model complicated relationships, for example a deep neural network can be trained to recognize pictures and whats in the picture on a high level.

Deep neural networks are widely used for pose estimation.

2.3 DeepLabCut

DeepLabCut is a open source software package for 3D animal pose estimation with python [8],[9]. DeepLabCut uses deep neural network and can be used to do pose estimation of both humans and several of different animals.

This makes it possible to extract information about animals behaviour without attach any markers that could potentially affect the animal or its behaviour. The model used in this thesis recognizes 22 joints on the horse and predicts how the horse will move. The model is trained on horses moving in walk, hence the pose estimation works best for horses in walk. The joints detected by the model can be seen in Figure 1, together with lines connecting them, creating a skeleton. The output from the model is a DLC file with the predicted x and y coordinates for each joint, as well as the likelihood of this. DeepLabCut also contains a toolbox where it is possible to analyze and the data in different ways, such as get the movie with a skeleton on and plot the predicted position of the joints.

Figure 1: Pose estimation of a horse in trot using DeepLabCut

2.4 Sine curves

Sine curves occurs in many different places such as mathematics, physics, engineering and signal processing.

The wave pattern also occurs in nature, such as sound waves, waves on the sea etc. A mathematical sine curve describes a smooth, periodic oscillation and its most basic form is

A ∗ sin(ω ∗ t + φ)

where A is the amplitude ω is the angular frequency and φ is the phase in radians [10].

Figure 2: A simple sine curve with A=2, ω = 2π and φ=0

It is possible to add sine waves with each other. When a sine wave is added to another sine wave of the same frequency and phase it keeps its shape, it is only the magnitude that differs. This is of highest importance

in for example Fourier analysis. Sine waves are easy to analyze since they are periodic and repeats the same pattern over and over again.

Sine waves can be used to represent different data sets, as an example one can fit a sine curve to the data, if the data it self is similar to a wave.

Figure 3: A sine curve fitted to example data

As seen in figure 3, a sine wave sometimes is a good estimation of some data. By using a sine wave it is easier to do some calculations and to get an overview of the data.

2.5 Fourier transform

Fourier transform is a mathematical transform used to transform functions depending on time to depending on frequency[11]. This is widely used in for example signal processing. If two sine curves of different frequencies are added and the Fourier transform is taken to plot frequency versus amplitude there will be two peaks on the plot, one for each frequency. The amplitude of the peaks tells how much there is of each frequency.

Figure 4: The blue curve consists of two different sine curves added together and the red curve is the Fourier transform of it

The Fourier transform function is a complex-valued function of frequency and is used to describe the same function as the time-domain, but in a different way. The amplitude of the Fourier transform represents the amount of the frequency present in the function in time-domain. Sometimes the Fourier transform is used to ease calculations, since calculation in one domain is sometimes easier to preform than in the other domain.

Linear operations preformed in one domain have corresponding operations in the other domain, and it is possible to convert between time-domain and frequency-domain to preform calculations without affecting the original function.

The simplest form, and most common, of Fourier transform is the Continuous Time Fourier Transform (CTFT).

F(ω) =R∞

−∞f (t)exp(−jω0t)dt (1)

In order to be used in computer science the CTFT can not be used, instead Discrete Time Fourier Transform (DTFT) is used, which uses discrete data values instead of a continuous function. Assume that the finite data sequence x(n) consists on N elements and is equal to zero outside the interval [0,N-1], then the DTFT can be expressed as

X(k) =

x(n)exp(−j ∗ 2π ∗ kn/N ), k = 0, 1, ...N − 1

On large data samples the DTFT will have very long computation time, which have lead to the Fast Fourier Transform (FFT) that is based on the DTFT. The FFT is commonly used so both numpy and scipy have FFT integrated in their packages.

The FFT is a widely used tool in signal processing due to its ability to process large amounts of data. The FFT builds on the DTFT, but works much faster and are therefore good when large amount of data is handled.

3 Method

To start with some video material were sampled. At first only one horse were filmed in all three different gaits, but later on one additional horse were also filmed to compare the plots of the different gaits. Both horses were fully grown and were around 165 to 175 cm. The camera were held as still as possible by hand and the horse passed from left to right. The videos then were analyzed by the pre-trained model. The model is trained to detect 22 different joints and give data about how the horse moves by pose estimation of the horse. The data used is the pre-trained models prediction of the different joints position, in pixels, on the video. Not all joints were analyzed in this project, but some joints considered to be of higher interest were. Joints available were:

• Nose

• Eye

• Near knee

• Near front fetlock

• Near front hoof

• Off knee

• Off front fetlock

• Off front hoof

• Shoulder

• Midshoulder

• Elbow

• Girth

• Wither

• Near hind hock

• Near hind fetlock

• Near hind hoof

• Hip

• Stifle

• Off hind hook

• Off hind fetlock

• Off hind hoof

• Ischium

Figure 5: Horse with all joints marked and a ”skeleton” connecting all joints.

For classifying the gaits a unique pattern or quality need to be found for each gait. When deciding which joints to use as markers for classifying the gaits there where only four valid options. Those were the four hooves.

Other joints such as ”Girth”, ”Nose” and ”Hip” where tried, but without any distinct qualities. Since the three different gaits to be classified all have different stokes, which are detected by the movement of the hooves, a natural measuring point is at each hoof. The walk should be 4-stoke, the trot 2-stoke and the canter 3-stoke.

The movement of each hoof represented as the x and y position in pixels were plotted, see figure 6.

Figure 6: x and y coordinate for the four hooves of a horse in walk

When the camera failed to be held still the y position of the hoof differs more than just oscillating around the same height, as seen in figure 6 this gives some deviation in the data. This also occurs when the horse is not passing in a parallel line to the camera, which means the distance along the y axis will differ with a linear difference, see figure 7 and figure 8.

Figure 7: x and y coordinate for the four hooves of a horse in trot

For all gaits it is possible to detect some sort of wave form from the plots. This waveform is used to find a pattern for the different gaits.

Figure 8: x and y coordinate for the four hooves of a horse in canter

Since only the stokes of the gaits and the hooves movement in y direction is of interest, only the y position of the hoof is used further on. Two different methods to classify the gaits were tried out, this was using the Fourier Transform and to fit a sine curve to the data and then use the phases between the curves to recognize the gaits. To analyze the data Python were used. A script for every gait were constructed and the data from the pose estimation were putted in every script, see an example in Appendix. Some other possible solutions were brought up for discussion but neglected for different reasons, more about that in the section ”Results and Discussion”.

3.1 Fourier transform

One way to classify the different gaits could be to use some sort of Fourier Transform. Since our data is in discrete form DTFT or FFT should be used.

Here the FFT is used, since it builds on DTFT but works much faster. For our data where each script consists of no more than 400 data points it is not necessary to use the FFT instead of DTFT, but for grater amount of data the FFT would be preferred. The function in NumPy for computing FFT were used, both for one hoof at the time, and all positions of the hooves added together.

3.2 Fitted sine curve

To find a pattern in the data it were structured in a more perspicuous way. As mentioned earlier sine waves contains a lot information, and they are also easy to read. To fit a sine wave to the data without putting any input parameters manually and to optimize the sine curves accuracy, uniform spacing was assumed and the standard derivation was computed and used as a first guess of amplitude. The offset was guessed to be the mean of the measured y position as a starter. Before it was possible to fit a sine curve the data needed to be processed a bit, such as tilt the y-coordinates to oscillate around an axis parallel to the x-axis. The displacement of the y-coordinates comes from how the video was filmed and the adjustment only affect the visualisation of the different gaits and the hooves relationship.

Figure 9: A sine curve fitted to one hoof, in this case the off front hoof in walk

Every time the sine curve is at its lowest it represents the horse putting the hoof on the ground, as well as the top of the sine curve represents the highest position of the hoof. Since it is clear how the hooves in each gait moves related to each other, it is possible to make this visible using sine waves. A sine wave were fitted to the data of each hoof.

3.3 Different phases

This builds on the previous section where a sine curve were fitted to our data. All the sine waves representing the y-coordinate of the four different hooves were plotted together. This way there is easy to get an overview of how the hooves moves in relation to each other. Those plots then were compared to how the horse actually moves and which legs the horse moves at the same time and in which order.

4 Results and discussion

4.1 Fourier transform

Using the Fourier transform to determine which gait a horse is moving in seems like an easy way, and it is in some specific cases. The Fourier transform relives which frequency something appears in, and how much of this frequency that is present in the data. When only one horse is used, and the horse only moves with one frequency for each gait it seems possible to use the Fourier transform without any heavy work.

Figure 10: Fourier transform of walk

Figure XX shows the Fourier transform and its amplitude for the horse in walk. There is mainly one frequency present and some sub frequencies.

Figure 11: Fourier transform of trot

The same yields for trot, shown in Figure XX. There is detection of around the same frequencies, but the magnitude of detected frequencies is greater for walk than trot. In this case it is possible to separate walk and trot from each other by using the magnitude.

Figure 12: Fourier transform of canter

Canter, seen in Figure XX, also have frequencies in the same spectra, but the difference in magnitude is not as significant.

The FFT could be used to detect where there is translation between the different gaits, but to determinate which gait a horse moves in is more problematic. One main reason for this is that there are many different horses, which runs in different frequencies. Small ponies will take shorter steps and also more steps over time than a big horse. When riding you also uses the the gaits in different paces, depending on the aim. This means that one horse can run both fast and slow in the same gait at the same video, and therefore also the frequency will differ. This makes using the Fourier transform problematic to use if applied on different horses and more complicated videos, such as when changes in pace occurs.

4.2 Different phases

Using the different phases of the fitted sine curve where a successful way determine which gait a horse is moving for walk and trot, but for canter there is some obscurity. The model which do the pose estimation is trained on horses moving in walk, which leads to poor quality for pose estimation in canter. This can also be seen on the videos where the skeleton is put on the horse, for walk and trot the pose estimation is working without any major errors, but for canter the skeleton loses track of the horse. This will of course show also in the estimated position of the hooves. As mentioned in the theory, walk is a 4-stoke gait in which the horse first moves near- hind-hoof then near-front-hoof followed by off-hind-hoof and then off-front-hoof. This can also clearly be seen in figure 13.

Figure 13: Fitted curves representing all hooves for a horse moving in walk

From the script where the sine curve were fitted to the data it is easy to extract some information about the sine curve. For walk the mean-period for all four hooves were 4.23 s. The extracted data about the different sine curves are plotted in table 1. From the table it is also possible to see the difference in phase and the order of the hooves.

Amplitude Omega Phase

Near hind 64 1.47 1.49

Near front 55 1.49 3.45

Off hind 61 1.47 0.6

Off front 62 1.50 1.17

Trot is a diagonal 2-stoke gait, which means that near-hind-hoof and off-front-hoof moves at the same time, followed by off-hind-hoof and near-front-hoof. This is also clearly visualized in figure 14. Walk and trot is quite similar, which should be the reason why the skeleton follows the horse good in trot.

Figure 14: Fitted curves representing all hooves for a horse moving in trot

The mean-period for the four hooves in trot were 2.75 s. The extracted information is found in table 2.

Amplitude Omega Phase

Near hind -45 2.18 1.23

Off front -39 2.22 1.06

Off hind 43 2.32 0.30

Near front 36 2.42 -0.09

As seen in tabular XX the sine waves for trot follows 2 and 2, as they are expected to do, and have a phase differ of about half the period time. For canter this method works poorly. As mentioned earlier the main reason for this is that the model is a poor predictor of the horse movement when in canter. The legs of the skeleton is often after the real horse legs, especially on the front legs. This of course shows of in the plot, see figure 15.

Figure 15: Fitted curves representing all hooves for a horse moving in canter

Because of the lack of accuracy in the data points the extracted information about the different sine waves did not make any sense. Since it is possible to detect both walk and trot it is possible to draw the conclusion that if the horse is moving, and the gait is not walk nor trot, then it is canter. The complexion between the different gaits is sufficient and it is enough to see one plot to tell which gait it is.

4.3 Added curves

Another possible way to classify the different gait is to look at the behaviour when they are added together.

As known, the walk is a 4-stoke gait, and from a look at figure 13, one can imagine that if we add all hooves together and den estimates a sine wave the estimated sine wave will have a lower frequency, as well as higher amplitude.

Figure 16: A sine curve fitted to all y-coordinates added together for walk

As seen in figure 16, the sine curve for all hooves added in walk decreases its frequency significant to omega=0.25. If we do the same for trot the sine curve for all hooves added should look completely different.

Figure 17: A sine curve fitted to all y-coordinates added together for trot

In figure 17 the fitted sine curve to all hooves added for trot is seen. Here the frequency increases instead to around the double, which corresponds to what could be expected for the 2-stoked trot. Since the major problem with classifying canter originates from the model and the data used, that is a potential problem here.

Figure 18: A sine curve fitted to all y-coordinates added together for canter

In figure 18 it is seen that the frequency from all hooves together does not differ from the frequency from each hoof on its own. For walk the frequency decreased significant, for trot the frequency doubled and for canter it stayed around the same. From this it is possible to determine which gait the horse is moving in. The easiest way to detect different frequencies is to use some sort of Fourier transform. One way could be to take the Fourier transform of a single hoof, and then of all four hooves added together, but possible issues are the same as discussed in chapter 4.1. This have not been tried out in this project.

4.4 Other possible solutions

Machine Learning is often used to solve classification problems. There is several different algorithms to use to solve the problem this way, such as logistic regression and k-nearest-neighbour, knn. The knn method were brought up for discussion, but more data from different horses would have been needed. Knn is a supervised learning algorithm, meaning it uses labeled data as an input to give appropriate result as an output. Knn assume that similar data are close to each other, meaning it compares the data point to be classified with k other points nearby, and then assumes that it is similar to its neighbours. For knn to work as a classifier for the gaits of horses more video material would have been needed. It would also be preferably if different horses of different sizes were used to train the model.

5 Conclusions

For this project looking to the phase difference is the most beneficial. With this method there is a significant difference between the different gaits, which also corresponds well to the theory about horses gaits. The goal of this project was to be able to classify the different gaits and the method of looking at the hooves movement and different phases solves this no matter of the size of the horse or which pace the horse move in.

To use the Fourier transform appeared to deal with some limitations, mainly due to later application of the product where the horse will run in other frequencies and paces. It was possible to see some difference between the gaits when one specific horse were on the video, but for different horses in different sizes running with varying frequencies this method seems unstable.

6 References

1. Mitchell, Tom (1997). Machine Learning.

2. Hu, J.; Niu, H.; Carrasco, J.; Lennox, B.; Arvin, F. (2020) Voronoi-Based Multi-Robot Autonomous Explo- ration in Unknown Environments via Deep Reinforcement Learning

3. Jordbruksverket (2016), [online] available: http://statistik.sjv.se/ (visited 2021-05-19) 4. EquiEye (2021), [online] Available: https://www.equieye.co/ (visited 2021-05-18)

5. Miesner, S.; Putz,M.; Plewa,M. and Fr¨omming, A. (2011) Ridhandboken 1 grundutbildning f¨or ryttare och h¨aast, 5th ed. Svenska Ridsportsf¨orbundet

6. Lindholm, A; Wahlstr¨om, N; Lindsten, F; Sch¨on, Thomas B. (2020) Supervised Machine Learning

7. Nanonets (2019), [online] Available: https://nanonets.com/blog/human-pose-estimation-2d-guide/ (visited (2021-06-10)

8. Mathis, A.; Mamidanna, P.; Cury, K.; Abe, T. (2018) DeepLabCut: markerless pose estimation of user- defined body parts with deep learning

9. Nath, T.; Mathis, A.; Chi Chen, A.; Patel, A.; Bethge, M.; Weygandt, M. (2019) Using DeepLabCut for 3D markerless pose estimation across species and behaviors

10. Adams, R.; Essex, C. (2013) Calculus- A complete course, 8th edition 11. Sollervall, H.; Styf, B. (2007) Transformteori f¨or ingenj¨orer

7 Popul¨ arvetenskaplig sammanfattning

Ryttare ¨ar ofta intresserade av att kunna f¨olja sin h¨asts utveckling och sin egen ridning, b˚ade f¨or att kunna f¨orb¨attra h¨astars prestationer, men ¨aven f¨or h¨astarnas h˚allbarhet. H¨astar kan r¨ora sig p˚a flera olika s¨att, i olika g˚angarter. De vanligaste g˚angarterna f¨or ridh¨astar i Sverige ¨ar skritt, trav och galopp. De olika g˚angarterna har olika r¨orelsem¨onster, och man s¨ager att skritt ¨ar en fyrtaktig g˚angart, trav ¨ar tv˚ataktig och galopp ¨ar tretaktig.

Med hj¨alp av ”pose estimation”, det vill s¨aga att f¨oruts¨aga en kroppsdels r¨orelsebana, kan man identifiera hur en kropp r¨or sig och f¨orutse framtida r¨orelse. I den h¨ar rapporten har pose estimation gjorts p˚a h¨ast och sedan har h¨astens g˚angarter klassificerats utifr˚an data som samlats in genom pose estimation och j¨amf¨ort den datan med redan k¨and information om h¨astars r¨orelsem¨onster. Datan som f˚as vid pose estimation ¨ar sannolikheten f¨or att en kroppsdel ¨ar p˚a en viss punkt, i detta fall en pixel p˚a en bild. Programmet som utf¨orde pose estimation var tr¨anad p˚a h¨astar som r¨orde sig i g˚angarten skritt. Travens r¨orelsem¨onster ¨ar relativt likt det i skritt, s˚a f¨or de b˚ada g˚angarterna fanns bra data att tillg˚a. Galoppen skiljer sig mycket fr˚an de b˚adra andra g˚angarterna, d¨arav erh¨olls data med st¨orre fel.

F¨or att klassificera g˚angarterna anv¨andes h¨astens hovar som m¨atpunkt. Genom att anpassa en sinuskurva till datan kunde ett tydligt m¨onster ses som st¨amde v¨al ¨overens med hur h¨astar r¨or sig. F¨or skritt kunde tillexempel fyra stycken, en f¨or varje hov, sinuskurvor ses som hade en lika stor fasf¨orskjutning mellan sig, vilket indikerar att skritten ¨ar fyrtaktig. I trav f¨oljde de fyra sinuskurvorna tv˚a och tv˚a, vilket de b¨or i och med att traven ¨ar tv˚ataktig. F¨or galopp gick det inte att identifiera ett tydligt m¨onster i och med problemen med pose estimation.

F¨ors¨ok att klassificera g˚angarterna gjordes p˚a flera andra s¨att, bland annat utf¨ordes en fouriertransform, vilken kan identifiera vilka frekvenser samt hur mycket av dem som finns n¨arvarande i ett dataset. Det var m¨ojligt att urskilja g˚angarterna beroende av dess fouriertransform, men f¨or ¨andam˚alet visade det sig vara b¨attre att titta p˚a fasf¨orskjutningen. Ut¨over tidigare n¨amnda s¨att testades ¨aven att addera samman datan f¨or alla fyra hovar och anpassa en sinuskurva till den adderade datan och sedan j¨amf¨ora den med sinuskurvorna f¨or hovarna var f¨or sig. F¨or denna metod visade det sig ocks˚a m¨ojligt att identifiera de olika g˚angarterna genom att titta p˚a hur frekvensen f¨or den adderade sinuskurvan skiljde sig j¨amf¨ort med frekvensen f¨or en enskild hov i samma g˚angart.

8 Appendix

Example python script for a horse in trot.

import numpy, scipy.optimize import pylab as plt

import numpy as np

n=100-8 #number of datapionts

t=np.linspace(0,4*np.pi,n) #creating a time-axis

#trot

nearfrontfoot=[126.7450333,127.2896118,129.3950043,166.3801575,171.0973663,309.9075012,30 9.9075012,311.8971252,315.6721191,315.6721191,318.8739319,328.4506226,333.5168152,340.02 20642,359.9265442,359.9265442,379.7251587,496.8885193,496.8885193,501.6280823,505.338775 6,506.391571,512.5376587,530.117981,530.117981,596.5184937,623.9015503,665.8093872,682.65 80811,699.5068359,711.8512573,712.4367065,713.4865112,715.9358521,715.9358521,716.134399 4,719.9634399,721.7080688,722.1929932,732.1646729,755.6590576,773.1603394,799.4446411,81 3.8962402,848.5762939,890.5875854,910.6416626,910.7049561,913.6793213,913.6793213,925.60 34546,1000.030823,1065.000244,1084.703125,1106.98877,1119.769897,1119.769897,1125.953247 ,1125.953247,1124.355225,1124.355225,1124.355225,1124.355225,1128.295288,1151.325439,115 1.325439,1175.282593,1200.841309,1309.663208,1326.435547,1336.252441,1333.329224,1333.32 9224,1340.783447,1417.786133,1417.786133,1450.576782,1466.256714,1499.239258,1525.499268 ,1525.499268,1525.499268,1531.080566,1531.080566,1546.645752,1546.645752,1546.645752,153 9.9646,1539.9646,1539.9646,1590.963989,1615.752197]

offfrontfoot=[127.1527786,127.4393921,141.1080475,141.1080475,191.9742737,208.5990906,208.

5990906,299.1352844,315.7524109,316.0012512,316.0012512,327.103302,330.9394836,333.90982 06,344.7781677,364.3692932,399.5624084,399.5624084,417.7330627,497.8653564,501.4336853,5 03.3256226,505.8273315,511.2800293,511.5257568,512.1624146,517.2134399,517.2134399,534.1 696167,565.7367554,586.977478,621.7000732,637.2003784,675.22229,699.9378052,714.5094604, 748.4081421,781.3754883,809.1049194,832.3676147,832.3676147,849.4472656,866.4176636,901.

973999,905.5950928,905.9708862,906.8264771,907.2521362,910.6845093,910.7939453,934.47613 53,932.7827759,932.7827759,962.1071167,987.7443848,987.7443848,1008.939148,1040.608032,1 061.951904,1108.499878,1125.168945,1129.123413,1150.180054,1191.811523,1245.79541,1264.6 01807,1282.886841,1282.886841,1304.343018,1319.374512,1325.446411,1329.184814,1332.64355 5,1332.643555,1338.043457,1338.043457,1338.921265,1338.921265,1346.592041,1377.798218,14 02.142334,1426.119263,1460.61792,1487.574951,1519.488647,1534.987915,1534.987915,1539.28 4302,1556.688965,1583.453003,1662.649536,1662.649536]

nearhindfoot=[45.69850159,45.69850159,65.0431366,86.44486237,109.3317719,137.250824,156.0 782471,162.5395203,165.4179535,166.1631927,185.5826721,200.4707794,223.6174164,254.17106 63,278.3424072,302.4686279,329.3612061,349.7853699,367.9523926,367.9523926,367.9523926,3 69.983551,369.983551,371.2481079,381.6588135,382.9390869,385.7789917,401.6919556,401.691 9556,423.250824,446.1056519,488.6266785,538.2192383,555.265686,555.265686,563.1500854,57 0.5247192,573.1140747,641.8400879,669.8328857,696.2548218,726.2439575,742.598877,762.855 5908,762.8555908,762.8555908,766.2052002,766.2052002,766.2052002,774.2674561,776.2241821 ,776.3147583,776.3147583,803.3806152,810.0163574,828.4128418,850.977478,879.8375854,915.0 628052,948.0029907,964.3643188,981.4545288,998.5015259,1042.561523,1065.313477,1093.7637 94,1123.843018,1141.444458,1163.726685,1163.726685,1163.726685,1168.583374,1168.583374,1 168.583374,1173.281616,1173.281616,1177.922485,1177.922485,1183.776367,1207.31311,1229.0 75195,1251.709473,1268.555542,1290.828857,1321.488037,1360.540649,1384.280029,1410.01293 9,1443.06604,1473.916382,1506.024658,1534.450317]

offhindfoot=[29.61894989,29.61894989,29.61894989,37.67149353,53.28766251,76.75076294,116.4 201279,116.4201279,155.7439117,157.0192566,166.0243073,172.078125,173.9486542,176.813598 6,176.8135986,305.2750854,305.2750854,305.2750854,282.0390015,282.0390015,282.0390015,31 0.0982971,348.0375061,361.2974243,381.1708069,384.5845337,451.6880798,477.7702332,508.26 89819,508.2689819,523.8101807,538.1964111,547.3257446,547.3257446,555.4794312,555.479431 2,567.6800537,570.0336304,570.0336304,570.0336304,570.0336304,596.6876831,614.3797607,63 8.809021,654.8778076,680.5157471,755.8041382,757.0119019,761.6794434,768.8512573,802.333 9844,849.1653442,883.8380127,908.451416,926.7828369,953.5439453,968.9464111,968.9464111, 968.9464111,962.352356,962.352356,962.352356,969.6027832,969.734314,970.6696167,974.3692 627,981.3712769,1014.322449,1036.329834,1060.714722,1086.85376,1160.384277,1164.341064,1 167.373291,1175.242554,1215.124268,1252.754028,1279.052856,1310.173096,1336.520508,1368.

404297,1368.404297,1368.404297,1367.258911,1367.258911,1367.258911,1368.896606,1370.7449 95,1371.943481,1378.749512,1380.740234,1403.632446]

nearfrontfoot_tilted=[]

offfrontfoot_tilted=[]

nearhindfoot_tilted=[]

offhindfoot_tilted=[]

for i in range(n):

nearfrontfoot_tilted.append(nearfrontfoot[i]-i*16) #-130 offfrontfoot_tilted.append(offfrontfoot[i]-i*16) #-120 nearhindfoot_tilted.append(nearhindfoot[i]-i*16) #+20 offhindfoot_tilted.append(offhindfoot[i]-i*16) #+30

#Add all data to one list data_added=[]

for i in range(n):

data_added.append(nearfrontfoot_tilted[i]+offfrontfoot_tilted[i]+nearhindfoot_tilted[i]+offhindfoot _tilted[i])

def fit_sin(tt,yy):

tt=np.array(tt) yy=np.array(yy)

ff=np.fft.fftfreq(len(tt), (tt[1]-tt[0])) #assume uiform spacing Fyy=abs(np.fft.fft(yy))

guess_freq=abs(ff[np.argmax(Fyy[1:])+1]) # excluding the zero frequency "peak", which is related to offset

guess_amp = np.std(yy)*2.**0.5 #compute standard derivation of values y guess_offset=np.mean(yy)

guess=np.array([guess_amp,2.*np.pi*guess_freq,0.,guess_offset])

def sinfunc(t,A,w,p,c): return A*np.sin(w*t+p)+c

popt, pcov = scipy.optimize.curve_fit(sinfunc, tt, yy, p0=guess) A, w, p, c = popt

#f = w/2.*np.pi

fitfunc = lambda t: A* np.sin(w*t + p) + c

return {"Amplitud":A, "omega": w, "phase":p, "offset": c, "frequency": w/(2.*np.pi),

"period":1./(w/(2.*np.pi)), "fitfunc": fitfunc, "maxcov": np.max(pcov), "rawres":(guess, popt, pcov)}

tt=np.linspace(0,10,n) tt2=np.linspace(0,10,10*n) freq_data=np.fft.fftfreq(n,d=0.1)

#Fit sinecurve to data

fit_nearfrontfoot=fit_sin(tt,nearfrontfoot_tilted) fit_offfrontfoot=fit_sin(tt,offfrontfoot_tilted) fit_nearhindfoot=fit_sin(tt,nearhindfoot_tilted) fit_offhindfoot=fit_sin(tt,offhindfoot_tilted) fit_added=fit_sin(tt,data_added)

print(fit_nearhindfoot) #1 print(fit_offfrontfoot) #1 print(fit_offhindfoot) #2 print(fit_nearfrontfoot) #2 print(fit_added)

#Fouriertransform

#freq = np.fft.fftfreq(tt.shape[-1])

#fourier_added=np.fft.fft(56*np.sin(4.6*tt-1.87)+346)

#plt.plot(freq, fourier_added)

#plt.plot(tt, nearfrontfoot_tilted, "-k", label="y") #create plot with our datapoints

#plt.plot(tt2,fit_nearfrontfoot["fitfunc"](tt2),"r-", label="#2 near front", linewidth=2)

#plt.plot(tt,offfrontfoot_tilted, "-k", label="y")

#plt.plot(tt2,fit_offfrontfoot["fitfunc"](tt2),"r--", label="#1 off front", linewidth=2)

#plt.plot(tt, nearhindfoot_tilted, "-k", label="y")

#plt.plot(tt2,fit_nearhindfoot["fitfunc"](tt2),"b-", label="#1 near hind", linewidth=2)

#plt.plot(tt, offhindfoot_tilted, "-k", label="y")

#plt.plot(tt2,fit_offhindfoot["fitfunc"](tt2),"b--", label="#2 off hind", linewidth=2)

plt.plot(tt, data_added, "k--", label="Data")

plt.plot(tt2,fit_added["fitfunc"](tt2),"r-", label="Fitted sinewave", linewidth=2)

plt.legend(loc="best") plt.xlabel("time (s)")

plt.ylabel("y position in pixels") plt.show()