INOM
EXAMENSARBETE TEKNIK, GRUNDNIVÅ, 15 HP
,
STOCKHOLM SVERIGE 2017
Quadcopter formation simulated in a choreographed dance to music
JACOB FRIMAN
ROSMARIE LEIJEL
Abstract
The purpose of this project is to simulate a dance formation for a group of quadcopters, moving in rhythm to music. Focus has lied on getting the dance to move in synchronization to the music. A description of the quadcopter and its dynamics is made. After simplifications was made, the dance was designed by controlling the velocities of the quadcopters.
A song was chosen for the music and a beat detection analysis was made to calculate the number of beats per minute (BPM) of the song. The BPM was used when creating the dance formation. Velocity restrictions and the composition of the music was the governing factors of how the dance was designed. A feedback control was implemented into the dance formation. The simulation was performed in MATLAB and a video was made of the resulting dance. The idea is that the simulation can be tested on real quadcopters, additions to the project can be made for future work to implement this.
Sammanfattning
Syftet med detta projekt är simulera en dansformation med en grupp quadcopters, som rör sig i takt till musik. Fokus har legat på att få till rörelserna i takt med musiken. Först gjordes en efterforskning av hur quadcopternas dynamik fungerar. Efter förenklingar så designades dans- koreografin genom att kontrollera hastigheten hos quadcopterna, för att komma till önskade positioner genom önskad rörelse. En sång för ändamå- let valdes och en beat detection gjordes för att hitta tiden mellan två beat, och lägga in denna i dansen. Såväl hastighetsbegränsningar som musikens komposition var avvägande faktorer att förhålla sig till vid skapandet av dansen. En reglerteknisk feedback kontroll implementerades till dansen.
Själva simuleringen gjordes i MATLAB och en video gjordes senare som visade upp resultatet av dansen. Iden med simulationen är att den ska kunna testas på riktiga quadcopters, tillägg till projekter för framtida ar- bete kan läggas till för att implementera detta.
1 Acknowledgements
We would like to thank our supervisors Xiaoming Hu and Han Zhang for guiding and helping us in this project.
Contents
1 Acknowledgements 4
2 Introduction 6
3 Theory 7
3.1 Quadcopter dynamics . . . 7
3.2 Quadcopter measuring instruments and hardware . . . 10
3.3 High and lower layer controller . . . 10
3.4 Music and beat detection methods . . . 11
4 Method 12 4.1 Simplifications and Limitations . . . 12
4.2 Design of the dance formation . . . 13
4.3 Feedback control . . . 14
4.4 Beat detection method . . . 15
5 Simulation 18 6 Discussion and conclusion 22 7 References 23 7.1 Figures . . . 24
*
2 Introduction
A quadcoptor is an unmanned helicopter with four rotors; it is therefore an UAV (unmanned aerial vehicle). Quadcopters have a simpler mechanical design than conventional helicopters and because of recent advances in electronics this has allowed the production of cheap commercial quadcopters. This has made them quite popular and there are now many hobbyists who fly quadcopters for personal use. Due to their small size and flexibility they also have many practical purposes such as aerial photography, surveillance and reconnaissance as well as search and rescue missions. They can be controlled by a pilot on the ground with a joystick, but they can also autonomously control themselves as a robot. A quadcopter has four rotors with four motors placed in a cross like pattern. Two of the rotors (that are placed opposite of each other) both spin clockwise while the other two spin counter clockwise. This is so that the quadcopter doesn’t spin around its own z-axis while applying equal thrust to all rotors. The quadcopter moves by increasing the angular velocity on some rotors and decreasing on others. This causes the quadcopter to tilt and since some of the thrust now is directed sideways the quadcopter starts to move horizontally.
It can also spin around its own z-axis (changing the yaw-angle) like a pirouette by increasing the rotors that spin in one direction while decreasing the other two. To ascend the quadcopter naturally increases the angular velocity on all four rotors. A pilot user or a robot only gives four inputs to the quadcopter;
thrust, forward and sideways motion and yaw angular velocity [1].
The purpose of this thesis is to design a formation for a group of quadcopters which they will follow in synchronization with music. We will create a dance for quadcopters. Dance is to move rhythmically to music using prescribed or improvised steps and gestures. This is something that humans do best and making a single quadcopter dance will not look graceful since it is a rigid body.
However, if we control several quadcopters at once we can control them so that the formation as a whole is dancing to the music. The movement of several quadcopters can emulate a dance. This gives us more options in how to design dancing maneuvers. To do this we will design a dance formation so that the quadcopters move in synchronization to the beat of the music. We will choose a song for our dance and analyze it and sample it to obtain the beat time. We will design trajectories for each quadcopter adapted to the song and specify the velocities required to follow the trajectories. The velocities designed will be bounded to not exceed maximum velocities. The maximum velocities was chosen from the quadcopter model DJI Phantom 4. A picture of this quadcopter can be seen in figure 1. To ensure that the quadcopters follow the given trajectories we will also implement a feedback control to handle disturbances and errors.
This will all be simulated using MATLAB.
Section 2 goes into detail on how quadcopters operate and introduce their dynamics as well as explaining the basic principle of how music is composed and and introduce concepts about beat detection. Section 3 contains simplifications and limitations that are used in this project and our motivations and proceedings for our method. The simulation is presented in section 4 and finally discussions and necessary additions for real life implementation can be viewed in section 5.
Figure 1: A picture of a quadcopter, the model is DJI Phantom 4 [[1]].
3 Theory
3.1 Quadcopter dynamics
The quadcopter’s dynamics will be explained and it will be shown that one can control the quadcopter’s movements by controlling the four rotor blades. First two coordinate systems need to be defined. One is the Cartesian inertial system fixed to the ground; the other is the non-inertial body frame of the quadcopter.
The quadcopter can be at different positions in space, have different coordinates X,Y,Z in the inertial frame, but it can also have different orientations. A quad- copter’s orientation or pose is described by its Euler angles. The quadcopter can rotate in three different ways. If it rotates around its own z-axis it changes its yaw angle (ψ), roll (φ) is the angle around its x-axis and pitch (θ) around its y-axis. The quadcopter moves in horizontally by adjusting either it’s pitch or roll angles. It does this by increasing the angular velocity of one rotor while decreasing it on the rotor that is 180 degrees away from it [2][3]. It can yaw by increasing the rotor speed of all rotors spinning in one direction while decreas- ing the other two. Quadcopters can either operate this way or they can be an X-type quadcopter. X-types pitch and roll by increasing rotor speed on the two rotors closest to each other instead. Basically they fly forward by pitching and rolling at the same time while the first quadcopter would fly forward by only pitching or rolling. The X-type quadcopter will not be used in this mathemati- cal model although the dynamics are very similar [4]. The Euler angles can be seen in the figure 2. Forces and torques around the z-axis of a quadcopter can be seen in figure 3. The rotation matrix to convert coordinates from the body frame to the inertial frame is defined as:
R=
cψcθ cψsθsφ− sψcφ cψsθcφ+ sψsφ
sψcθ sψsθsφ+ cψcφ sψsθcφ− cψsφ
−sθ cθsφ cθcφ
Where cx = cos(x) and sx = sin(x). For example a vector ~a in the body frame
Figure 2: A picture showing the three Euler angles yaw, pitch and roll and how they are defined in the body frame [[2]].
is R~a in the inertial frame. The total angular velocity of the whole quadcopter is given by:
~ w =
1 0 −sθ
0 cφ cθsφ
0 −sφ cθsφ
φ˙
˙θ ψ˙
The angular velocity of the rotor blades create thrust and also torque in the z-axis in the body frame. The force and the torque from one rotor can be modeled as:
fi= kω2i, τMi= bω2i ,
where k and b are constants due to the air drag and ωiis the angular velocity of the rotor blade. The total thrust produced from the quadcopter is the sum of the thrust from all rotors. Thus the total thrust in the body frame is:
T~B =Xf~i= k
0 0 P ωi2
The numbers on the rotors are chosen according to figure 3. Roll is achieved when decreasing the angular velocity of the 2nd rotor while increasing it on the 4th. Pitch is achieved when decreasing the angular velocity of the first rotor while increasing it on the third. Yaw is achieved when increasing the speed on rotors two and four and decreasing it on rotors 1 and three. The yaw torque around the body z-axis is:
τψ = b(−ω21+ ω22− ω32+ ω42)
Figure 3: The body frame of the quadcopter is shown together with angular velocities, torques and forces from the rotors [[3]].
The different signs on the angular velocities are due to the rotors spinning in different directions. The roll torque is:
τφ=X
r × fi= Lk(−ω22+ ω42)
Where L is the distance from the center of the quadcopter to the rotor. And correspondingly the pitch torque is:
τθ= Lk(−ω12+ ω32) Thus the total torque in the body frame is:
~ τB =
τφ
τθ
τψ
=
Lk(−ω22+ ω24) Lk(−ω21+ ω23) b(−ω21+ ω22− ω23+ ω42)
Modeling the quadcopter as two crossed rods, by the symmetry the inertia matrix becomes diagonal.
I=
Ixx 0 0 0 Iyy 0 0 0 Izz
The dynamical equations can be derived using an Euler-Lagrange approach or using Newton-Euler equations. Using the latter, the torque can be written as:
I ˙w + ~~ w × I ~w = ~τB
This can be rewritten as:
˙~w = I−1( ~τB− ~w × I ~w) (1) And finally using Newton’s Second Law in the inertial frame one obtains:
m¨~r = −mg~z + R ~TB (2)
~r =¨
¨ x
¨ y
¨ z
where m is the mass of the quadcopter, g is the gravitational acceleration, ~z is the unit vector on the z-axis in the inertial frame and ~r is position vector of the quadcopter in the inertial frame. equation 1 and 2 describes the motions of the quadcopter and if one can control the angular velocity of the rotor blades one can also control the acceleration and angular acceleration of the quadcopter.
Thereby one is able to control the quadcopter fully. Air resistance is not taken into account but it is not necessary for this example[2][3].
3.2 Quadcopter measuring instruments and hardware
A quadcopter has several instruments available to measure and detects its posi- tion, orientation and other obstacles. A barometer measures air pressure which is used to measure the aircrafts altitude. The pressure sensors are so sensi- tive that they can detect change in air pressure for a few centimeters. The accelerometer measures the acceleration of the aircraft. The accelerometer can also calculate the orientation since the accelerometer has three axis sensors. The gyroscope measures the angular velocity in a three axis, the rate of change of the Euler angles. The angles and acceleration can then be used to calculate the velocity by integration of data and time. Ultra sonic sensor, cameras and a Global Positioning System (GPS) are also used as a complement. Ultrasonic sensors attached to the quadcopters, measures the distance to its nearby neigh- bors and other obstacles. GPS is unreliable on small scales and the barometer is unreliable in indoor environments [1]. It is useful to have a preprogrammed fixed coordinate system when designing formations of quadcopters. This can be achieved using a capture motion method. Capture motion is a method where cameras on three different surfaces together combine data to create signals in X,Y,Z directions. To be more independent from light conditions in the room, ac- tive bright markers mounted at each arm of the quadcopter reduce the exposure time of the cameras. It is more accurate than other methods, making it a good choice for implementation for our project. Since the dance formation designed needs to follow the trajectory accurately and following it on time is essential.
The input given from these instruments are given to the inertial measurement unit (IMU). The data is processed in a micro control unit (MCU). Each rotor blade has its own motor and electronic speed controller, the last one translating signals from the MCU to the motor. The MCU sends the output to all four rotors and tells them what angular velocity they should have [1].
3.3 High and lower layer controller
To create a clear separation of hardware-dependent software from hardware- independent software an abstraction layer is placed between the micro controller hardware and the application software. The top layer architecture is the only visible interface from the application point of view [5]. We divide these into the higher and lower layer control. Most quadcopters are designed with different layers of control. The hardware dependent lower layer controls what the angular velocity on the rotor blades should be in order to achieve desired movement and
position. It also stabilizes the quadcopter. The input given to the lower layer controller is the linear velocities for each axis X,Y,Z, the angular velocity of the quadcopters yaw angle and the current orientation of the quadcopter. If the orientation required for the desired velocities and the current orientation differ, they will produce an error. The lower layer produces the required angular velocities on the rotors so that this error converges to zero. Thus the output is the angular velocity of the rotor blades and therefore also the torque given from each rotor. The higher layer controller on the other hand is given input from a user (with a joystick controller) or computer software. Taking the joystick as an example, a user can specify in which direction the quadcopter should fly and how fast. The joystick can also control the angular velocity of the yaw, but it cannot control the pitch and roll. Pitch and roll are determined by the linear velocities in X,Y,Z directions, the higher the speed the higher the pitch or roll angles are. So a user with a joystick or computer software gives input to the higher layer controller and it outputs the desired velocities in all three axis directions as well as the yaw angular velocity. These are then fed to the lower layer controller which will attain the desired input[6][5].
3.4 Music and beat detection methods
Sound are vibrations, that propagates as an audible mechanical wave of local dynamical pressure that varies in time, through a transmission medium such as air. It can be measured by a microphone that converts sound into an electrical signal. The signals are collected by a defined procedure, and the set of data collection is called “sample”. The elements of a sample are called “sample units”
or “sample points”, the samples are proportional to the energy level of the sound. In stereo mode there are samples from two channels, corresponding to the left and right loudspeaker. The sound frequencies represent the different audible tones. The human ear hears frequencies from 20 Hz up to 20000 Hz.
The numeric algorithm Fast Fourier transform (FFT) is used nowadays to get signals from time domain to frequency domain. FFT needs 2M samples to give M number of tones, each tone for each integer frequency. The Nyquist- Shannon sampling theorem says sampling frequency must be greater than twice the maximum frequency one wishes to reproduce. The sampling rate for music therefore has to be greater than 40000 Hz since that is twice the frequency 20000 Hz. The most common sampling rate is around 44100 samples per second, used for MP3 and other audio file formats. The 44100 sampling rate is used because it allows a 2050 Hz transition band, that filters the signals [7].
The beat of a music sound is the fundamental frequency. It is also the frequency which reduplicates itself the most in a music piece, with short time expression and short lay-off. Modern songs often have a beat around 60-150 beats per minute (BPM), which gives 1 to 2.5 beats per second. The interval between two beats, that we chose to call “tau” in this report, must therefore be between 0.4 to 1 seconds long [8]. The sound will be heard as a beat if the current sound is louder (higher energy) than the previous average sound or if the frequencies of the tones are beneath 200 Hz and repeated regularly. Therefore there are two main ways to detect a beat correctly.
The first way is to compare sound energy level in a small amount of samples, to its previous average energy level. Where the current local sound energy is higher than the previous average level, then we might have a beat. The second
way is to detect the frequencies in a song using FFT for small intervals, then dividing the frequency spectrum into sub bands. Choose the lowest frequencies and detect its energy and compare the sound energy in each sub band to the recent average energy corresponding to these sub bands. When the energy level is superior to the average, then we might have a beat. Since drum and bass in musical instruments play the main role in expressing the beat in most types of modern music, it is enough to only study frequencies beneath 200 Hz. A combination of these methods is more precise, but if one is precise enough, it is unnecessary to go further with the other one too.
4 Method
4.1 Simplifications and Limitations
The task to fully control the quadcopter from the bottom layer control is a difficult task due to the complex flight dynamics. Controlling the rotor speeds to follow our desired dance trajectories would be a project of a different scope and level than this one. To simplify we have decided to only control the higher layer controller. Thus we only control the velocities of the quadcopter and its yaw velocity. This has the added benefit of being easy to implement on real quadcopters. If we had real quadcopters our simulation code would replace the inputs given from a joystick since all we input are velocities which is what the lower layer needs. Thus we only need to design what velocities the quadcopters should have in order to follow our desired trajectory from the dance formation.
All the velocities we will design will be of a form:
V =~
vx
vy
vz
ωψ
Where ωψ is the angular velocity of the yaw angle. To make sure that the simulation is possible to implement on real quadcopters, physical quadcopter limitations have been taken into account. Since real quadcopters are being simulated an upper limit on their velocity is necessary. Real quadcopters cannot fly infinitely fast. Since there are different types and models of quadcopters with different maximum speeds, we needed to choose a specific quadcopter and use its performance metrics for our simulation. Our choice of quadcopter was the DJI Phantom 4. This model was chosen since it is a quality quadcopter that has a good visual system. We did not want a budget quadcopter with too poor performance. This model was sufficient for this goal. The maximum velocities of this quadcopter can be seen in table 1 [9].
In addition, due to quadcopters being rotor aircraft, they are subject to a problematic condition called vortex ring state. The vortex ring state is when the rotor-craft descends into its own down wash, its own air it pushes down to create lift. A vortex of air is formed around the rotors and the air just re- circulates around in this vortex and further increasing the rotor blade speed only accelerates this vortex. The result being that the aircraft loses lift and starts descending into the ground. This condition usually occurs when the rotor-craft
maximum velocity
vertical velocity 6 m/s horizontal velocity 10 m/s
yaw rate 200 grad/s
Table 1: The maximum velocities of quadcopter DJI Phantom 4
descends too quickly. If the rotor-craft descends slower or if it descends at an angle, not straight down, the vortex ring state can be avoided. This means that the maximum vertical descent speed must be lower than maximum ascent speed [11][12]. Since the maximum vertical speed is the same for descent and ascent and we do not know whether the vortex ring state is taken into account in these numbers, we decided to lower the maximum descent speed to 4 m/s.
The limitations explained above restrict how the dance formation can be designed and must be taken into account when creating the dance formation.
The song that was chosen for the simulation was “Tokyo” by Danny Saucedo.
The song was chosen since it has a simple, easily recognizable beat and the composition of the song is divided into regular parts. The song belongs to the genre electronic dance music. This genre usually has songs which all have the same regular composition. This song is divided into parts that have a number of beats that are divisible by 8. It has different sections of music that are 16, 32 or 48 beats long that are repeated throughout the song. The regularity of the song and its division into these parts means that we can design a dance for each section separately. Except being easier to design the dance, this has the added benefit that we can recombine the order of these segments so the designed dance can be used with different songs.
4.2 Design of the dance formation
Knowing the limitations and the composition of the song the dance formation can be designed. The dance formation created was divided into sections that match the music sections. Dance sections 16, 32 or 48 beats long were created.
When a section of music is played with a specific number of beats in it, a corresponding dance formation will be performed with the same length. When a new music section begins, a new dance formation will be performed. Each dance section was designed independently of the other sections in order to recombine the order of the dance formations as we saw fit. To keep the formation as desired in each section, the velocities were designed so that the quadcopters’
starting and ending positions of each section were the same. This assures that the dance sections can be put in different order, which gives us the option of choosing another song with this composition as well. Since more than three dance sections were created, music sections of a specific length can have more than one dance section attached to it.
The dance formation was created using mostly periodic functions. Specifi- cally sinusoidal functions were used, often to create circular trajectories. The motivations behind this are purely aesthetic. Since music and dance is periodic with a regular structure, it was thought fitting to include sinusoidal functions.
Sinusoidal functions also have a smooth derivate which means that changes in velocity will be smooth too. The acceleration was not controlled in this thesis,
but in order to make the movements smooth and beautiful the changes in speed should not be discontinuous or too large. Choosing sinusoidal functions as ve- locities means that the acceleration will be bounded and not infinite, causing the velocity to be continuous and smooth. However due to simplicity of design and time constraints some movements were designed using constant velocities with discontinuous changes of velocity. To alleviate this, a short continuous change of velocity between two different velocities was introduced.
V = δ(t) ~~ V2+ (1 − δ(t)) ~V1 (3) Initially before equation 1 is implemented, for time t < T the quadcopter have the velocity ~V1(t) and for time t > T ~V2(t). At time T the change in velocity between ~V1(T ) and ~V2(T ) is discontinuous. Instead for time T − dt < t < T + dt we use the velocity from equation 1. The change in velocity from ~V1to ~V2will be continuous since the function δ(t) is continuous and δ(T −dt) = 0, δ(T +dt) = 1.
A sinusoidal function 0.5 + 0.5sin(2dtπ (t − (T − dt) − dt)) was chosen as δ(t) with these properties. However, replacing the desired velocities ~V1 and ~V2 by this transition will cause the quadcopters’ trajectories to be slightly off the desired trajectory. Since the dance formation was designed using the desired velocities V~1 and ~V2 during the time T − dt < t < T + dt, and not with the velocity transition. This transition was applied due to removing discontinuities and making the dance look smoother. Since the velocities from the transition differ from the desired ones, this means the quadcopters end up at different positions.
This is easily fixed if this transition only is enabled together with the feedback control as described below.
4.3 Feedback control
To control the quadcopters to follow their intended trajectories a feedback con- trol is implemented. A linear proportional feedback control is used for this purpose. The control equation is:
X = −k( ~˙~ X − ~Xdesired) + ~Vdesired (4) The desired velocities ~Vdesired are the ideal velocities which should be the velocities of the quadcopter. When the desired velocities for each quadcopter are designed we can integrate them to find out the desired positions ~Xdesiredof all four coordinates including yaw for each time. The desired positions are then used to determine how far off the quadcopter is from its intended trajectory.
This creates an error that will correct the velocity to return the quadcopter to its path. The error will converge to zero if the initial positions of the quad- copters are displaced. However this controller cannot converge to zero with a constant disturbance, only if the average of the disturbance is zero will the controller converge. For example with a Gaussian disturbance with expectation value zero the error would converge. If the dance is performed in an indoor environment the wind disturbance would be minimal and this controller would be satisfactory. When using the feedback control we include the velocity transi- tion above in the desired velocities in equation 3. The desired positions stay the same, being derived from the desired velocities excluding the velocity transition.
The result is that during the time T − dt < t < T + dt the quadcopter will follow
the continuous velocity transition but will still return to the desired trajectory afterwards. A proof that the feedback controller converges can be seen below.
X = −k( ~˙~ X − ~Xdesired) + ~Vdesired=>X − ~˙~ Vdesired= −k( ~X − ~Xdesired) Which is the same differential equation as ˙y = −ky, meaning that:
X − ~~ Xdesired= ( ~X0− ~X0,desired)e−kt
This means that as time passes the error becomes smaller and smaller. It converges to zero exponentially fast.
4.4 Beat detection method
The shortest amount of time the human eardrum can detect a sound in depends on its frequency and its power. If the frequency is below 200 Hz, which it is usually is for beats, this time is 1/200=5 ms. A sound that has a shorter duration than about 5 ms, will not be detected as a low level as a longer sound, because of different neuronal systems in our ears, might be integrating stimulus energy at shorter and longer durations. To make sure the hearing system detects every frequency correct, about 1 second is preferred at a normal music sound level (about 60 dB). [27]
Two short audio samples played just 20-30 ms apart, will be heard as a single entity. The brain spontaneously makes a single object of them. [14] Therefore, a part of music about 1024 samples (=23 ms) is instant for human ears.
Since not only the beat is playing in a song, it can be difficult to see what energy peaks correspond to the beat. Therefore it is a good idea to analyze parts when drums are the major part that is playing, with little disturbance from other sounds. The basic method performed to extract the beat is to compare the local current sound energy to the average sound energy of the previous one second, which is called the average sound energy history by us. While the samples we analyze are not values for energy, they are proportional to energy. The energy is converted to a voltage which the computer reads and measures, that measurement is what have been called a sample. In music other high energy level sounds than the ground beat can occur. And therefore it gives better results if some constant C strengthens the average sound energy history. This constant will be used. Because a song has many different acoustic energy levels, at least 5 seconds of the song has to be analyzed to give an exact value of the beat. However because computer memory cannot handle information without adding errors when lacking memory, at most 500 000 samples ( 11 s) can be analyzed[15][16].
First a part of the song where mostly drums were was chosen to be analyzed.
The song was sampled and uncompressed in MATLAB. The samples from the song are from two channels. One channel is the sound played by the right loudspeaker and the other from the left one. Five to eight seconds was analyzed to obtain the beat. At first it is impossible to see the beat as seen in figure 4.
Then the local current energy blocks were created. They were created by taking the square of each sample from both channels and then taking the sum of 1024 samples from both channels together. 1024 was chosen since it humans
Figure 4: The sampled part of the song read into the computer, showing that further analysis is needed to extract the beat.
cannot distinguish sounds shorter than that as mentioned above. The j:th local energy level is therefore defined as:
Ej=
1023+j
X
i=j
Si,lef t2 + Si,lef t2 (5)
Where j = 1024n, n is a positive integer including zero, Si,lef t2 is a sample measurement from the left loudspeaker channel and vice versa. Then the average sound energy history is formed by taking the average of 44032 squared samples from both channels. 44032 is chosen since it is 43*1024 samples which is about one second, one second was an adequate number for this. Then to create a sound which is 1024 samples long, the average was multiplied by 1024 for reasons explained above. Each local energy level is associated with a unique average sound energy history. The current local sound energy has its average sound energy history formed by taking the average of all the previous 44032 samples from the current local. The sound energy history is defined:
Ehj = 1024 44032
j
X
i=j−44032
Si,lef t2 + Si,lef t2 (6)
Where j is defined the same way as above. Then the constant C is calculated by calculating the variance from 43 local energy levels and then C is defined as[15]:
C = −1.5 × 10−6V ar(E) + 1.5142857 (7) V ar(E) = 1
43
43
X
j=0
(Average(E) − Ej)2
Average(E) = 1 43
43
X
j=0
Ej
Figure 5: Local energy level divided with average sound energy history and C is plotted against time. One can clearly see the beat in this picture.
After that the following equation is analyzed:
Lj= 10log( Ej
EhjC) (8)
If Lj is greater than zero the local energy level is greater than its history, which is required for a beat and Lj and Lj+1 cannot be equal, it should be larger or decrease. The result from this is bands of consecutive local energy levels (which each are 1024 samples long) for which this is true. In each band, the highest local energy level is taken and in each local energy level the maximum sample value is taken. The time between each maximum sample in each band should be the beat. Finally the average of all these values of time is taken to obtain the final value for the beat. A figure illustrating the beat can be seen in figure 5.
5 Simulation
The result of this thesis is the simulation of the created dance formation. The simulation was performed using MATLAB. The simulation was largely done using the MATLAB built in function ode45. Ode45 is a function that solves dif- ferential equations, and in our case it was used to integrate the velocities. Ode45 integrates different velocities for different times and different quadcopters. Since the feedback controller needs the desired positions to operate one ode45 was run without feedback control to obtain this. The data points returned from ode45 are discrete and to be able to obtain a value of the desired positions for all times, a spline interpolation was done. After that an ode45 could be run with the feedback loop implemented. The number of quadcopters that was used in the simulation was chosen to be four. Four is an even number which made it easier to design symmetries in the dance. A higher number was not chosen due to taking longer time to code and taking more computer power. Using a number lower than four would not have given the needed complexity to design a good dance. The dance formation was conceived with purpose of dancing to music. Therefore the visual simulation of the quadcopters moving needed to be in synchronization to the music. Doing the simulation of the quadcopters and playing the music simultaneously in MATLAB proved to be problematic.
The MATLAB plotting function was unable to synchronize the music and the dance due to not being able to plot in real time. Instead a video was created to visualize the dance. The video was a stop motion animation, with the time between each frame the same as the time difference between two data points.
The video shows that the quadcopters move in synchronization to the beat, which was the goal. Trajectories showing the movement of the quadcopters can be seen in figure 6-9.
Figure 6: Picture is showing a circular maneuver in the dance. While not seen here the quadcopters change direction while in the circle. After the quadcopters end up in a vertical line the adjust their positions for their next move. The velocities are of the form Awsin(k + w(t − d)) where the constants assure the phase is correct for each quadcopter and to make them able to change directions.
Figure 7: After going around in a circle for a time the quadcopters move in a circular pattern to form a vertical line. As an example, for the circular ascending part the velocities are ~V = [2wsin(w(t−2τ q)), −2wcos(w(t−2τ q)),1.5−(q−1)6τ , 0].
Where q is a number depending on which quadcopter it is and w is 4τπ. After the quadcopters end up in a vertical line they adjust their positions for their next move.
The beat detection analysis returned a value for the calculated to be:
τbeat= 0.455
τbeat is the time between each beat in the song. In other words the song was determined to have about 131 beats per minute. Observation of the sim- ulated dance verified that this was a value that was satisfactory for use in the simulation.
The feedback control proved to be effective as well. If the starting positions were displaced and a Gaussian disturbance was applied, the quadcopters were able to correct themselves to the correct path. When using the feedback con- troller in eq (nr), there is a constraint on k. The velocities cannot violate the velocity restrictions, the quadcopter cannot fly faster than its maximum veloc- ity. Therefore k cannot be too large so that the velocities exceed the bounds.
Since the velocity depends also on the error of position, the best way keep within the bounds was to only change k to a lower value if the velocity restriction was exceeded, and otherwise keep k as the same value. A figure showing the trajec-
Figure 8: In this part the quadcopters are moving up and down in a wavelike pattern, they are also yawing. The time between two quadcopters reaching the top of the wave is exactly the time between one beat.
Figure 9: Here the quadcopters move with linear velocities in a square like formation.
Figure 10: The blue trajectory is showing the desired trajectory and red is the actual trajectory of the simulated quadcopter. Here k = 1.
Figure 11: The rate of convergence for different values of k. The distance from the quadcopter to its desired position is on the y axis and time on the x-axis.
tories with initial displacement and disturbance can be seen in figure 10 and the rate of convergence with respect to different values of k can be seen in figure 11.
6 Discussion and conclusion
Since a simulation of quadcopters was done in this project, a relevant question is whether or not it can be implemented on real quadcopters. Since we have designed the dance from the higher layer controller existing in quadcopters, the simulation is possible to implement. The quadcopters would be given input from the higher layer controller from our simulation which the lower layer would use to produce the desired velocities. The velocities of the designed dance are within the bounds of max velocity for DJI phantom 4. It is possible to use Matlab compilers to use the program code to implement on real quadcopters.
However there are other things that are not taken into account if this sim- ulation would be implemented. Acceleration was not taken into account, only the velocities were designed. The dance was designed to be in sync using the velocities. It was not taken into account how fast the velocities could change.
If the velocity in the designed dances changes faster than the maximum allowed acceleration there will be some lag in the dance as they will not be able to follow the designed trajectories perfectly. The simulation would still be able to be im- plemented but the the desired velocities might not be achievable at all points.
Still, the feedback controller would be able to correct this so that the quad- copters return to the desired trajectory on time. The quadcopters would lag behind when the change in the desired velocities are greater than the maximum acceleration, but they would catch up afterwards when the desired velocity is reached. As long as the lag is not too great or too frequent the dance would still look presentable.
In the simulation the music was played along the dance in MATLAB. This works well since the computer can start the music and dance simultaneously and in the simulation environment the music and dance run ideally with any delays. In reality there would probably be some error between the music and dance. The dance itself could still run as intended but the music might be not be in synchronization to where the dance. The quadcopter would have to detect the music in real time and adjust its velocities accordingly. A feedback control for controlling that the dance would be aligned with the music in time could be implemented in addition to the position feedback. The time delay for the quadcopter to detect and sample the music in real time must be taken into account when designing this. In Matlab these issues do not appear. Other small time delays is the time needed for the angular velocity of the rotor blade to change, in the dynamics it is assumed that there is no angular acceleration of the rotor blades. Also the calculation of velocities that quadcopter does several times per second takes a small time too. These usually have a negligible effect, but they might be an issue in applications were time precision is important such as this.
When designing the dance absolute coordinates were used to design the velocities, in the feedback loop absolute coordinates was used for the desired position and the current position. Often this is not the case in real life appli- cations as the quadcopters only know their positions relative to each other and the direction of its velocities are given in its body frame. Before this simulation is applied on real quadcopters, a conversion to relative coordinates is needed.
Absolute coordinates can be achieved using GPS but this is rather inaccurate for the purpose of the designing a dance that must be synchronized.
The purpose of this project was to design and simulate a moving formation
of quadcopters that will dance to music, move in rhythm to the beat. The dance was designed by designing the velocities of the quadcopters while quad- copter velocity restrictions was taken into account. A song was chosen as the music for the dance and the song was analyzed to detect the beat so the dance would be able to move in synchronization to the music. A feedback control was implemented into the dance to control the quadcopters if they would be dis- placed. The dance formation was then simulated and coded using MATLAB.
The presentation of the results was made using a video of the simulated dance.
If additional work was to be done on this project in the future, several ad- ditions could be made. Maximum acceleration could also be taken into account when designing velocities. One step further is to design the lower layer directly, designing the rotor blade velocities instead of the quadcopters’ velocity, thus di- rectly taking quadcopter dynamics into account. An extended feedback control could be added, an integral and derivate part could be added to the controller, producing a PID controller. Naturally implementing on real quadcopters is the most logical follow up when doing a simulation. The only certain way of testing a whether a simulation is succesful or not is actually implementing it in reality.
7 References References
[1] Liang Oskar. June 25 2013. Build A Quadcopter From Scratch.
https://oscarliang.com/build-a-quadcopter-beginners-tutorial-1/ (viewed May 31 2017)
[2] Luukkonen Teppo.2011. Modelling and control of quadcopter. Aalto Uni- versity.
[3] Gibiansky Andrew. November 23 2012. Quadcopter Dynamics and Simu- lation. http://andrew.gibiansky.com/blog/physics/quadcopter-dynamics/.
(viewed May 2017)
[4] García Carrillo, L.R., Dzul López, A.E., Lozano, R., Pégard, C. 2013. Quad Rotorcraft Control. London, Springer
[5] Wolfhard L. 1997 CAN system engineering from theory to practical appli- cations. New York, Springer.
[6] Bresciani Tommaso. October 2008 Modelling, Identification and Control of a Quadrotor Helicopter. Lund University
[7] Watkinson, John. 2000 The art of Digital Audio. 2:nd edition, Seattle. Ama- zon books.
[8] Roberts, Gareth E. 2016 From Music to Mathematics, exploring the con- nections. Baltimore. John Hopkins University Press.
[9] DJI. 2017. Phantom 4 SPECS.DJI https://www.dji.com/phantom-4/info (viewed May 31 2017)
[10] Corrigan, Fintan. 2016. DJI Phantom 4 and 4 pro IMU calibration How to instructions.Dronezone. https://www.dronezon.com/diy-drone-repair- videos/dji-innovation-drones/how-to-calibrate-dji-phantom-4-and-4-pro- imu/ (viewed May 2017)
[11] Quadcopter 101. August 1 2014. Quadcopter "Wob- ble of Death": VRS Recovery and Avoidance.Youtube.
https://www.youtube.com/watch?v=LCret4rv0HE (viewed May 2017) [12] Dziubinski Adam, Stalewski Wienczyslaw. 2007 Vortex Ring State Simula-
tion Using Actuator Disc. Institute of Aviation, Warsaw.
[13] Picton T.W.2010 Human auditory evoked potentials. San Diego: Plural Publishing. E-book
[14] Deruty, Emanuel.2011 How we hear pitch.SOS Sound on Sound.
http://soundonsound.com/sound-advice/how-we-hear-pitch (viewed May 2017)
[15] Hawkins, Kevin. 2001. Beat detection algorithm.GameDev
http://archive.gamedev.net/archive/reference/programming/features/beatdetection/
(viewed May 2017)
[16] Ziccardi, Marco. 2015 Beat detection algorithms (part2).
http://mziccard.me/2015/05/28/beats-detection-algorithms-1 (viewed May 2017)
7.1 Figures
References
[1] Tested Hands-On with DJI’s Phantom 4 Quadcopter Drone! Youtube https : //i.ytimg.com/vi/LieIde5ByqU/maxresdef ault.jpg
[2] Juan C. Mendez Quadcopter dynamics: Reference frames The UAV Net http : //theuav.net/assets/articleimages/2014 − 11 − 19 − quadcopter − dynamics/angles.png
[3] Luukkonen Teppo.2011. Modelling and control of quadcopter. Aalto Univer- sity.
