A Survey on Methods for Time-Frequency Analysis

Master Thesis MEE-98-01

A SURVEY ON

METHODS FOR

TIME-FREQUENCY ANALYSIS

Jan Mark de Haan

_____________________________________________

Master of Science program in Electrical Engineering 1998 University of Karlskrona/Ronneby

Department of Signal Processing Examiner: Benny Lövström

Supervisor: Jens Augustsson, SSPA Maritime Consulting

Version nr Date Updates

1.0 24-06-1997 Start Chapter 1 and Chapter 2.

1.1 07-09-1997 Start Chapter 3 and 4.

1.9 10-10-1997 Towards version 2.0 (Deliverance of chapter 1, 2, 3 and 4).

2.0 16-10-1997 Deliverance of chapter 1, 2, 3 and 4.

2.9 19-12-1997 Chapter 4 rewritten. Addition of 5, 6, 7 and 8. Towards 3.0 3.0 26-01-1998 Deliverance of chapter 5, 6, 7 and 8.

4.0 13-03-1998 Final version.

Version 4.0 March 1998

Master Thesis MEE 98-01

Keywords:

Signal processing, Time-frequency analysis, Wavelets, Wavelet Packets.

$FNQRZOHGJHPHQWV

This Master's thesis is the result of my final project at the University of Karlskrona/Ronneby.

There are many people whom I'd like to thank for their assistance during the project.

First, I would like to thank Jens Augustsson and Erik Pavlica, working at SSPA Maritime Consulting, for their guidance and the possibility to gain knowledge in an exciting field of science. In this context I also would like to thank Martin Almgren.

Thanks also goes to Mathias Winberg for the useful discussions and Benny Lövström, both working at the Department of Signal Processing at the University of Karlskrona/Ronneby, for the final comments on my thesis and for the possibility to work on my project at the department.

Last but not least, I would like to thank Karl Thorén for reading my thesis thoroughly and Nedelko Grbic and Xiao-Jiao Tao for their inspiring suggestions.

Finally I would like to thank everyone who helped me during my project, and Lisa Lorentz for putting up with me during my studies in Sweden.

Jan Mark de Haan Ronneby, Sweden 13 March 1998

$EVWUDFW

Time analysis and frequency analysis are both well-established ways in engineering to gain more knowledge about a physical phenomena. Time and frequency analysis can be combined in a joint time and frequency distribution. A simple method to gain a joint distribution is to window segments of the data at different time locations and calculate its Fourier transform.

By doing this a set of ´local´ spectra are gained and joined to present a time-frequency distribution. This method is well known as the Short-Time Fourier Transform.

The Short-Time Fourier Transform has the disadvantage that is does not localize time and frequency phenomena very well. Instead the time-frequency information is scattered which depends on the length of the window. This can be attended to by altering the length of the window but a certain balance between good time and good frequency localization is unavoidable.

To cope with this disadvantage, the Wavelet Transform uses dilated and translated functions, which are local in time, and frequency, which results in good frequency resolution for low- frequency phenomena and good time resolution for high-frequency phenomena. The advantage of the Wavelet Transform is its efficient fast transform in discrete time. But still, there is no complete solution to the localization problem.

Adaptive Time-Frequency Analysis can be advantageous for solving the localization problem.

The functionality of methods is hereby adapted to the time-frequency content of the signal.

The Adaptive Wavelet Packets Transform is based upon the Wavelet Transform but is a more general way to gain a time-frequency distribution. It is even possible to gain a time-frequency distribution similar to the Short-Time Fourier Transform. The energy levels in the frequency bands determine the frequency resolution. Much energy located in a small frequency band will result in good frequency resolution for that specific band. Other frequency areas will be analyzed with as good time resolution as possible. Sine wave with constant frequency precedes time phenomena. The method is implemented using a fast Quadrature Mirror Filter bank tree which form determines resolution of the analysis.

In the Adaptive Window Short-Time Fourier Transform, the time phenomena precede sine waves in the analysis. Good time resolution is gained where the time-frequency concentration is highest for short windows. Other time intervals will be analyzed with a longer window, to gain better frequency resolution. The method is implemented using a set of Fast Fourier Transform calculations.

7DEOHRI&RQWHQWV

Acknowledgements...3

Abstract...5

Table of Contents...7

1 Introduction ...11

1.1 Representations ...11

1.2 Objectives of this work...12

1.3 Mathematical Notations ...13

1.4 Thesis Organization ...13

2 Uniform Resolution Time-Frequency Analysis...15

2.1 Introduction...15

2.2 The Fourier transform ...15

2.3 Towards Time-Frequency Analysis...17

2.4 The Short-Time Fourier transform ...17

2.4.1 Definition ...17

2.4.2 Windowing in the Time-domain ...19

2.4.3 Windowing in the Frequency-domain...20

2.4.4 The Time-Frequency Plane...21

2.4.5 Time and Frequency Uncertainty ...22

2.4.6 The Heisenberg Uncertainty Principle...26

2.4.7 The Spectrogram ...28

2.4.8 The disadvantages of the Short-Time Fourier Transform...28

2.5 Other Uniform Resolution Time-Frequency Representations...31

2.5.1 The Gabor transform ...31

3 Multiresolution Time-Frequency Analysis ...33

3.1 Introducing Scale...33

3.2 The Multiscale Short-Time Fourier transform...33

3.3 The Continuous Wavelet Transform ...36

3.3.1 Introduction ...36

3.3.2 Definition ...36

3.3.3 Properties of the Continuous Wavelet Transform...38

3.3.4 The Continuous Wavelet Transform and the Time-Frequency Plane ...39

3.3.5 Profile of the Continuous Wavelet Transform ...40

3.3.6 Wavelet radius and bandwidth ...41

3.3.7 The Scalogram...41

4 Adaptive Time-Frequency Analysis...43

4.1 Introduction...43

4.2 The Multiresolution Fourier Transform...44

4.2.1 Profile of the Multiresolution Fourier Transform...44

4.2.2 Wavelet Duration and Bandwidth...46

4.3 Adaptive Window Short-Time Fourier Transform ...46

4.3.1 Short-Time Time-Frequency Concentration ...47

4.3.2 Optimal Window Time-Frequency Representation ...47

5 Discrete Short-Time Fourier Transform and Implementations...49

5.1 Introduction...49

5.2 Parameter Discretization ...49

5.3 Discrete Short-Time Fourier Transform... 51

5.3.1 Discrete Fourier Transform... 51

5.3.2 Windowing in discrete time ... 52

5.4 Time domain based implementations... 53

5.4.1 Redundant discrete STFT ... 54

5.4.2 Non-redundant discrete STFT... 56

5.5 Frequency domain based implementations... 58

5.5.1 Redundant discrete STFT ... 58

5.5.2 Non-redundant discrete STFT... 60

6 Wavelet Transforms and Implementations... 63

6.1 Introduction ... 63

6.2 Parameter Discretization: Discrete Wavelet Transform ... 63

6.3 Multiresolution Analysis ... 65

6.3.1 Subspaces, Scaling functions and Wavelet functions ... 65

6.3.2 Wavelet composition and decomposition ... 67

6.4 Fast Wavelet Transform ... 68

6.4.1 Quadrature Mirror Filter Bank... 68

6.4.2 Discrete Wavelet Transform implemented by a QMF-tree ... 69

6.5 Implementations... 70

6.5.1 Continuous Wavelet Transform... 70

6.5.2 Fast Wavelet Transform... 71

7 Adaptive Transforms and Implementations ... 73

7.1 Introduction ... 73

7.2 Wavelet Packets ... 74

7.2.1 Expansion of the MRA... 74

7.2.2 Wavelet Packets composition and decomposition ... 75

7.2.3 FWT based upon the expanded MRA ... 76

7.2.4 Energy Prediction and Tree Pruning ... 77

7.2.5 Calculation of Filter Coefficients ... 77

7.2.6 Implementation... 78

7.3 Adaptive Window Short-Time Fourier Transform... 80

7.3.1 Optimal adaptive window ... 80

7.3.2 Towards an efficient implementation... 80

8 Conclusions... 85

8.1 Introduction ... 85

8.2 Comparison of Results... 85

8.3 Further Reading ... 88

References... 91

Index... 95

Appendix... 99

Appendix A: Symbol List... 101

Appendix B: Abbreviations ... 103

Appendix C: Mathematical Proof... 105

1 Introduction

THIS PAGE IS NOT PART OF THE THESIS DOCUMENT COVERS PAGES 1-14

Organization of this chapter:

1 Introduction _________________________________________________________ 10 1.1 Representations___________________________________________________________ 11 1.2 Objectives of this work ____________________________________________________ 13 1.3 Mathematical Notations ____________________________________________________ 13 1.4 Thesis Organization _______________________________________________________ 13

&KDSWHU

,QWURGXFWLRQ

 5HSUHVHQWDWLRQV

When dealing with physical objects, many of its different faces, or, representations are encountered. For example numbers in various systems can be represented depending on the application; in real life we use the decimal system, while for use in computers we employ the binary representation of numbers. Consequently in many fields, such as signal processing, a preliminary task is to find an adapted representation of the data that may be particularly suitable for a problem. A certain representation can reveal certain aspects; other representations preserve integrity while reducing the amount of data.

A way to obtain a specific representation is to decompose a datasequence or signal into elementary building blocks, which have certain importance. This is often achieved with a technique that is called transformation. Moreover, these building blocks have to reveal unique information. Then the question raises how to decompose a signal. A fast algorithm is desired, to perform the decomposition, since otherwise a particular decomposition might be only of theoretical importance. In a practical situation, the possibilities concerning speed of computations and the data storage space are limited.

Once these building blocks are collected, the data sequence might be composed of the blocks to approximate the original sequence as good as possible, usually with an inverse transform.

A goal of a transform might be the reduction of size of the signal; therefore we need to be able to compose the signal with only the few building blocks we gained from the transform, so that it is a desirable approximation of the original signal.

One of the classic sets of tools to achieve a different representation of a signal is the Fourier theory. The Fourier integral gives a continuous-time decomposition while its discrete variant gives a discrete-time decomposition, which can be implemented using a fast algorithm.

Although the algorithms differ, the underlying mathematical ideas are the same for these representations.

The representation of a signal by means of the Fourier theory is essential to solve many problems in pure mathematics and in applied science. However, it is in some instances not the most natural or useful way of representing a signal.

For example, music or speech are signals in which the spectrum evolves over time in a significant way. At each moment in time the ear hears a certain combination of frequencies, and that these frequencies are constantly changing. This time evolution of frequencies is not reflected in the Fourier transform¹, because it decomposes over the infinite time interval. In theory a signal can be reconstructed from its Fourier transform, but the transform contains information about the frequencies of the signal over all times instead of showing how the frequencies vary with time.

It is desired to see how the frequency content of a signal varies with time in the way musical score serves the musician see figure 1.1. An analysis is wanted comparable to an exercise called musical dictation which is writing a note, or a set of notes, at certain pitch levels (frequencies) at the proper horizontal position (time) on the bars.

Figure 1.1: Musical score can be seen as a time-frequency representation.

A representation, which combines time and frequency, is gained by performing time- frequency analysis. Several methods exist to decompose a signal in time-frequency components. The most common methods can be divided in two groups:

• Uniform time-frequency analysis

• Multiresolution time-frequency analysis.

1 The Fourier Transform is usually split up in an amplitude spectrum and a phase spectrum. The amplitude spectrum representation, which is meant in the text, does not show time information of the original signal. The time

 2EMHFWLYHVRIWKLVZRUN

The main objective of this work is to describe the properties and limitations of time-frequency analysis, by presenting basic methods to give background to its problems and an introduction to more advanced methods, dealing with these limitations. The aspects of wavelets in general are investigated, and more thorough, their benefits in time-frequency analysis. Simply how wavelets can be used to workaround the limitations of basic methods.

Adaptive time-frequency analysis is investigated, and how it can increase the value of a multiresolution time-frequency analysis with wavelets.

The discussed methods will be implemented in MATLAB, to show advantages, disadvantages and above all the distinctions between the methods.

 0DWKHPDWLFDO1RWDWLRQV

The mathematical notations in this thesis make use of many superscripts and subscripts. In order to make this thesis readable for a relatively large audience; an attempt has been made to use this notation in a consistent manner. A large letter indicates a continuous variable, which can adopt a value out of an infinite amount of numbers. A small letter indicates the discrete version, which can only adopt a number out of a finite amount of numbers. If they have a subscript attached they are a constant and do not vary within a range of values.

Functions, however, are written with small letters for continuous functions as well as discrete functions. The large letters are used to refer to other representations than time-domain representation, e.g. the Fourier Transform.

 7KHVLV2UJDQL]DWLRQ

This thesis is divided into two main parts. The first part describes the fundamental theories and the continuous cases of the time-frequency analysis. The second part describes the discrete cases and implementations. Each part is divided in three chapters, see table 1.1.

PART 1, Chapter 2-4 PART 2, Chapter 5-7

1. Uniform 2. Multires 3. Adaptive 1. Uniform 2. Multires 3. Adaptive

Fundamental theory Fundamental theory Fundamental theory Discrete parameters Discrete parameters Wavelet Packetes Continuous STFT Continuous WT Continuous MRFT Full discrete STFT Full discrete WT Adap. window STFT

Adap. window STFT Implementations Implementations Implementations

Table 1.1: Thesis organization.

&KDSWHU

8QLIRUP5HVROXWLRQ7LPH)UHTXHQF\

$QDO\VLV

 ,QWURGXFWLRQ

This chapter is divided in four parts. The first part describes one of the most important transforms in signal processing, which is the Fourier transform. The Fourier transform reveals the notion of frequency in a signal. The time-frequency analysis introduced in this chapter is based on this transform.

After a short introduction towards time-frequency analysis, the easiest method for uniform resolution time-frequency analysis is introduced: the short-time Fourier transform, and its spectrogram representation. The time-frequency plane is described which is the basis for a representation of a signal in both time and frequency. The windowing-in-time technique and its filterbank interpretation are discussed. Also the name for this chapter is justified, resulting in a general limitation of combined time-frequency analysis; the Heisenberg uncertainty principle. Finally the disadvantages of uniform resolution time-frequency analysis are depicted.

The last section gives a short overview of other, to the short-time Fourier transform related time-frequency representations.

 7KH)RXULHUWUDQVIRUP

Time-frequency analysis exploits ordinary frequency analysis with the Fourier transform. The latter of the time-frequency methods use the time wise decomposition of a signal in its frequencies. With the Fourier transform, a time representation of a signal, x T( ) , is transformed into a new function X_FT(F) of which the absolute value gives a frequency representation.

First in 1807, Jean-Baptiste Fourier discovered, partly based on work done by Daniël Bernoulli, that any periodic function could be decomposed into only sinewaves. The Fourier series of any periodic function (Theorem of Euler-Fourier) shows how it is composed of elementary infinite sines and cosines. This theorem concerns only those sines and cosines that are integer multiplies of a base frequency ²

0

π

T [CM90]:

x T a a kT

T b kT

k k T

k

( )= +  cos + sin

 



=

∑

∞ 1

2 0

0 0

1

2π 2π

The coefficients a and b rely on the function being analyzed and are called its Fourier coefficients. The term ¹₂a is the coefficient of the constant function cos2 0₀ π T=1.

It is in many aspects easier to work with the complex exponential function e^jθ instead of the trigonometric functions cosθ and sinθ because it reduces computational efforts, for example with differentiation and addition¹. Using the relations ^{cosθ =21}

(

^{ejθ +e−jθ}

)

^and

( )

sinθ = _{2 j}¹ e^jθ −e^−jθ , the Fourier series expansion can be rewritten to:

x T c ek k

j kT T

( )=

=−∞

∑

∞ ^{2π 0}

where c0 a

1

2 0

= , ^{ck = 12}

(

^{ak − jbk}

)

^{and c}−^k = ¹2

(

^ak + ^jbk

)

for k =1 2 3, , ,... The coefficients ck

can be calculated with the following formula [CM90]:

ck T x T e dT

j kT T T

T

= ⁻

−

∫

1

0

2

2 2

0

0 0

( ) ^π

The Fourier transform calculates the Fourier coefficients X_FT(F) of a Fourier series expansion of the non-periodic function x T( ) on the whole real line R= −∞,∞ . The expansion is defined as continuous superpositions of the analyzing function e^−j2πFT:

dT e

T x F

X_FT ^∞

∫

^{j FT}

∞

−

= ( ) − ^2π

)

( (2-1)

or, if using the angular frequency Ω =2πF :

X( )Ω = x T e( ) ^{−j TΩ} dT

−∞

∞

∫

The analysis coefficients X_FT(F) in (2-1) define the notion of frequency F in the signal. The inverse Fourier transform is defined by:

1 d j j

e ^θ = je ^θ, e^{j(θ ξ+ )} =e e^{jθ jξ}

dF e

F X T

x ^∞

∫

_FT ^{j FT}

∞

−

= ^π

π

) 2

2 ( ) 1 (

The analysis function is a complex periodic function. As a result, Fourier analysis works well if x T( ) is composed of stationary (wave) components². However, any abrupt change in time in a non-stationary signal is spread out over the whole frequency domain in X_FT(F). Therefore, an analysis adapted to non-stationary signals requires more than the Fourier transform.

 7RZDUGV7LPH)UHTXHQF\$QDO\VLV

The usual approach is to introduce time dependency into the Fourier analysis while preserving linearity. In the next section, a translation-parameter is introduced which refers to the specific time interval or window wherein the signal is approximately stationary.

A function of two variables, time and frequency, is introduced, meant to be a joint distribution representing the time-varying spectral properties of the signal. A time-frequency distribution should show which frequencies are present at a given time. A simple and intuitive way to achieve such a distribution is through the use of the short-time Fourier transform (STFT).

In the next section the STFT will be introduced, a transform that results in a local spectrum with the windowing-in-time technique. Then a relationship with the ordinary Fourier transform is established, which shows that windowing in the time-domain is equal to windowing in the frequency-domain. The result of the transform can be mapped into a two- dimensional plane, which is called the time-frequency plane. The following section then shows by examples that windowing in time and frequency involves the loss of measuring accuracy and thus introduces uncertainty, which is known as the Heisenberg uncertainty principle.

 7KH6KRUW7LPH)RXULHUWUDQVIRUP

 'HILQLWLRQ

The Short-Time Fourier Transform (STFT) or windowed Fourier transform is a transform that yields a representation of sequences of any length by breaking them into shorter sections, and applying the Fourier transform to each section. This is a time-frequency localization technique in that it computes the frequencies associated with small segments of the signal. For each section, the STFT calculates its Fourier transform. The continuous STFTis defined as [CC92]:

2

∑

∀

= Ω n

T j _n

e T x( )

∞

∫

∞

−

− −

= xT wT T e dT F

T

X_STFT(ˆ, ˆ) ( ) ( ˆ) ^j2πFˆT (2-2)

The window w T( ) that is used is local in time, it is zero for values of T outside the window.

The line above the window function means complex conjugate. The choice of the window function is important. Section 2.4.5 will clarify the need to consider carefully the choice of window duration and the shape.

If we compare the transform to the ordinary Fourier transform, it can be recognized that the analyzing function e^−j2πFT is replaced by a variant, which is local in time, since it's multiplied with the window function. The new analyzing function is therefore local in time and has a fixed duration for all frequencies. The frequency inside this window varies, just like the frequency of the analyzing function in the ordinary Fourier transform does indefinitely over all times. The analyzing function of the STFT is defined as [CC92]:

T F j F

T T wT T e

g_ˆ,ˆ( )= ( − ˆ) ^2πˆ (2-3)

It contains the variables Tˆ and Fˆ that represent time and frequency in the so-called time- frequency plane which will be discussed in section 2.4.4

Figure 2.1 shows the analyzing function with a specified length and for a specific value of the frequency Fˆ .

a)

-50 0 50

-1.5 -1 -0.5

0 0.5

1

T

g(T)

b)

-50 0 50

-1.5 -1 -0.5

0 0.5

1

T

g(T)

Figure 2.1: The analyzing function of the STFT g_0,4(T). a) Real part, b) Imaginary part.

Fˆ =4 and the window used is a Hamming window with duration 100.

The analyzing function has instances, each connected to specific values for Tˆ and Fˆ . The instances are translated across the time-domain due to parameter Tˆ . This translation is shown in figure 2.2. Three instances of the analyzing function are given at different locations Tˆ , and have different values for Fˆ . Notice that the analyzing function can be located at any Tˆ ,

1 0 -1Amplitude

-50 0 50 Time

1 0 -1Amplitude

-50 0 50 Time

contain any frequency Fˆ and has uniform duration which means that the analyzing function has the same duration for all Tˆ and Fˆ .

-50 0 50 100 150

-1.5 -1 -0.5

0 0.5

1

T

g(T)

Figure 2.2: Three instances of the analyzing function of the STFT. The instances are from left to right: ( )

50

,1

0 T

g , ( )

25

,1

75 T

g and ( )

25

,2

150 T

g .

 :LQGRZLQJLQWKH7LPHGRPDLQ

Two important properties of the analysis function g(T) are its center T₀ and its radius ∆_{g( )T} . They are defined by [CM90]:

( ) ( )

²₂

2

0 g T

dT T g T T

∫

∞

∞

= − (2-4)

( )

( ) ( )

( )

²₂

2 2 0 2

T g

dT T g T T

T g

∞

∫

∞

−

−

=

∆ (2-5)

Where g

( )

T ²₂ =

∫

−^∞∞ g

( )

T ²dT is the energy of the analysis function. (2-4) And (2-5) are also well known properties in statistics, namely mean and variance [HB96]. The duration of the analysis function can be calculated with 2∆_{g( )T} . The STFT gives local information of x T( ) in the time-window:

( )^{T g}( )^T

g T T

T

T + ˆ−∆ , + ˆ+∆

0 0

1 0 -1Amplitude

0 50 100 Time

 :LQGRZLQJLQWKH)UHTXHQF\GRPDLQ

A relation can be established between the STFT X_STFT(Tˆ,Fˆ), and the Fourier transform (F)

X_FT of the original signal x T( ) . First, an analysis function in the frequency domain needs to be determined, which is the Fourier transform of the analysis function in the time domain ^GFT

( )

^F =^FT

[ ]

^g

( )

^T . The Fourier transform of the analyzing function g T( ) is:

( )

^{F e 2 ( ˆ)ˆW (F Fˆ)}

G_FT = ^{−j π F−F T} _FT − Using (2-2), the definition of the STFT can be rewritten to:

∫

∞

∞

−

= xT g T dT F

T

X_STFT(ˆ, ˆ) ( ) ( )

The Parseval Identity

∫

^x

( ) ( )

^{T g T dT}

∫

−^∞∞^XFT

( ) ( )

^{F G}FT ^FdF

∞

∞

− = ₂¹_π is used to rewrite (2-2) to:

∫

∫

∫

∫

∞

∞

−

−

∞

∞

−

−

∞

∞

−

−

∞

∞

−

−

⋅

=

−

=

−

=

=

dF e

F F W F X e

dF e

F F W e

F X

dF F F W e

F X

dF F G F X F

T X

T F j FT

FT T

F j

T F j FT

T F j FT

FT T F F j FT

FT FT

STFT

2 ˆ ˆ

2 ˆ

2 ˆ ˆ

2 ˆ )ˆ ( ˆ 2

ˆ) ( ) 2 (

) 1 (

ˆ) ( )

2 ( 1

ˆ) ( )

2 ( 1

) ( ) 2 (

) 1 , ˆ (ˆ

π π

π π

π

π π

π π

(2-6)

where a relationship is given between the STFT and the Fourier transform of a signal x T( ) . The result of (2-6) and the definition (2-2) shows the relationship between windowing in the time-domain and windowing in the frequency-domain:

( )

∫

∫

∞

∞

−

−

∞

∞

−

−

−

⋅

=

−

dF e

F F W F X e

dT e

T T w T x

T F FT j

T FT F j T

F j

2 ˆ ˆ

2 ˆ 2 ˆ

ˆ) ( ) 2 (

) 1 (

ˆ) (

π π

π

π

(2-7)

With the exception of the factor (e^{−j2πFˆTˆ}) one can see that ”the windowed-in-time Fourier transform of x(T) is similar to ”the windowed-in-frequency inverse Fourier transform

(F) X_FT ”.

We can also interpret W_FT(F) being the transfer function, and g(T)=w(T)e^j2πFˆTthe impulse response of an infinite set of bandpass filters. X_STFT(Tˆ,Fˆ) is then an infinite set of signals, associated with the outputs of these bandpass filters. The outputs of these filters can be

calculated using equation (2-6), which can be interpreted as an inverse Fourier transform of the signals Fourier transform multiplied with the transfer function of each bandpass filter. Or with equation (2-7), which can be interpreted as a convolution integral, which calculates the convolution of the signal in the time-domain with all the impulse responses.

Similar as done in the time-domain we can calculate the center frequency F and the radius₀

( )^F

GFT

∆ of the window function G_FT(F) [CC92]:

( ) ( )

²₂

2

0 G F

dF F G F F

FT

∫

FT

∞

∞

= −

( )

( ) ( )

( )

²2 2 2 0 2

F G

dF F G F F

FT FT F

GFT

∞

∫

∞

−

−

=

∆ ^(2-8)

Where G_FT

( )

F ²₂ =

∫

−^∞∞G_FT

( )

F ²dT is the energy of the analysis function. The width of the analysis function, the bandwidth, can be calculated with _{G ( )F}

∆ FT

2 . The result in (2.14) gives local spectrum information of x T( ) in the frequency-window [CC92]:

( )^{F G} ( )^F

G_FT F F _FT

F

F + ˆ −∆ , + ˆ+∆

0 0

The choice of window function has to be taken carefully, since windowing of the signal in the time domain means filtering of the signal in the frequency domain. A window that is local in time and local in frequency is required. The choice of the window depends on the requirements on accurate amplitude measurement weighted against spectral leakage [PJ96].

 7KH7LPH)UHTXHQF\3ODQH

The absolute value of the short-time Fourier transform X_STFT(Tˆ,Fˆ) can be mapped into a two-dimensional plane with a time axis and a frequency axis. This is called an amplitude time-frequency distribution. The two previous sections defined the windows in respectively time and windows in frequency. These windows generate rectangular time-frequency frames or sometimes called Heisenberg frames. They are defined by [CC92]:

( )^{T g}( )^{T G} ( )^{F G} ( )^F

g T T F F FT F F FT

T

T + ˆ−∆ , + ˆ+∆ × + ˆ −∆ , + ˆ +∆

0 0

0 0

Figure 2.3: Location of time-frequency frames in the Time-Frequency Plane. The related analysis functions are

T F j F

T T w T T e

g ¹

1 1

2 ˆ ˆ 1

ˆ, ( )= ( − ˆ ) ^π and ^{j FT}

F

T T w T T e

g ²

2 2

2 ˆ ˆ 2

ˆ , ( )= ( − ˆ ) ^π

The shape of the frames depends on the duration of the window in time and the bandwidth of its Fourier transform. Due to the scaling property of the Fourier transform, the areas of the frames are the same for any window length [CM90]:

[ ]





 



= 

a X F aT a

x

FT 1 _FT

)

( (2-9)

If the duration of the analysis function is increased, the bandwidth decreases proportionally.

This means that only the shape of the window influences the area of the time-frequency frame.

 7LPHDQG)UHTXHQF\8QFHUWDLQW\

A simple example showing an important property of the STFT is that for windows with infinite size in the time- and -frequency-domain [PE91]. The STFT of the function x T( ) can be calculated using the Dirac impulse as the window function: w T( )=δ( )T , so that its Fourier transform is W_FT(F)=1. This is rather strange and it does not reflect a very practical situation but it shows what one might expect. Using (2-6):

( )

ˆ) (

2 1 ) 1 (

ˆ) ˆ,

( ^{2 ˆˆ 2 ˆ}

T x

dF e

F X e

F T

X_STFT ^{j FT} _FT ^{j FT}

=

⋅

⋅

⋅

= ^∞

∫

∞

−

− π π

π (2-10)

If the result is mapped into the time-frequency plane we gain the time-domain representation of the original function x T( ) which is not very surprising. If a window w T( ) is used, that is infinitely small, and therefore a window WFT(F) that is infinitely long, the time-domain representation of the function x T( ) is gained.

A similar calculation, can be done by setting the window w T( )=1 and therefore its Fourier transform ^WFT(^F)=δ( )^F , using (2-7):

ˆ1

F ˆ2

F 2∆^G(F)

2∆^g(T) Fˆ

0 Tˆ

ˆ1

T Tˆ2

ˆ) (

1 ) ( ˆ)

ˆ,

( ^{2 ˆ}

F X

dT e

T x F

T X

FT

T F j STFT

=

⋅

⋅

= ^∞

∫

∞

−

− π

(2-11)

If this result is mapped into the time-frequency plane, the frequency-domain representation of the original function x T( ) is gained. If a window w T( ) is used that is infinitely long, and therefore a window WFT(F) that is infinitely small, the normal time-domain representation of the function x T( ) is gained.

The calculations (2-10) and (2-11) do not reflect a practical situation. The results however are important. They implicate that the smaller the window w(T), the sooner a representation is obtained which is similar to the time-domain representation. The smaller the window W_FT(F), the sooner a frequency-domain similar representation is obtained. The STFT is thus a representation that lies between the time-domain representation and the frequency-domain representation.

The following examples show that it is not possible to get an ideal time-frequency representation. A simple example signal consists of a single complex sinusoid [PE91]:

T F j _x

e T

x( )= ^2π

Using (2-2) and any window w T( ) to calculate its STFT:

ˆ ) (

ˆ) (

ˆ) ( ) ( ˆ)

ˆ, (

2 ˆ 2

2 ˆ

x FT

T F j T

F j

T F j STFT

F F W

dT e

T T w e

dT e

T T w T x F

T X

x

−

=

−

=

−

=

∫

∫

∞

∞

−

−

∞

∞

−

−

π π

π

(2-12)

The idealized representation would be a Dirac function located at F for all times Tˆ , since_x x T( ) only consists of one frequency. The STFT though yields the window function with its center located at F . The relation between the idealized representation is clear but there is a_x loss of accuracy, a loss of resolution.

Figure 2.4.a shows the idealized representation with infinite resolution and accuracy, which is a straight line at the oscillation frequency F . Figure 2.4.b shows the _x STFT representation of the complex sinusoid, which is the Fourier transform of the window function.

a) b)

Figure 2.4: XSTFT(Tˆ,Fˆ) representation of the ^STFT of a single complex sinusoid with F =50. a) Idealizedx

representation, b) STFT representation using Hamming window.

Another example is the calculation of the STFT of a Dirac impulse located at time T [PE91]:_x )

( )

(T T T_x

x =δ − (2-13)

Using (2-6) and the window w T( ) , with its Fourier transform W_FT(F) and

FTx

j

FT F e

X ( ) = ^{− 2π} to calculate the STFT yields:

( )

π π π π

π π

π π

π

π π

2 ˆ ) (

ˆ ) 2 ( ) 1 (

ˆ) 2 (

) 1 (

ˆ) 2 (

) 1 (

ˆ) ˆ, (

ˆ 2 ˆ ˆ

2 ˆ

2 ˆ ˆ 2

2 ˆ

2 ˆ ˆ

2 ˆ

x

T F j x T

F j

T F j FT

FT j T

F j

T F j FT

FT T

F j STFT

T T w

e T T w e

dF e

F F W e

e

dF e

F F W F X e

F T X

x

= −

−

=

−

=

−

=

−

∞

∞

−

−

−

∞

∞

−

−

∫

∫

(2-14)

This case is more or less similar to that of the sinusoid in the example above. One might expect a Dirac impulse at T =Tˆ_x for all frequencies Fˆ , but instead the STFT yields the time window located at T =Tˆ_x for all frequencies Fˆ . Even with this example, a loss of resolution occurs. But even here, the relation between the idealized representation and the one that is yielded by the STFT is still clear.

Figure 2.5 shows the idealized time-frequency representation of a Dirac impulse located at

=50

T . The STFT representation of the Dirac impulse shows the result of (2-14), the time window with its center located at T =50.

a) b)

Figure 2.5: X_STFT(Tˆ,Fˆ) representation of the STFT of a Dirac impulse located at T =50. a) Idealized representation, b) STFT representation using the same Hamming window as in figure 2.4.

This final example of two complex sinusoids and two Dirac impulses will illustrate the need for larger or smaller time window [PE91]:

T F j T F

j e

e T

x( )= ^2π1 + ^{2π 2} ) ( ) ( )

(T T₁ T₂

y =δ +δ

Using the results of (2-24) and (2-26) the STFTs of x(T) and y(T) are:

ˆ ) ( ˆ )

( ˆ)

ˆ,

(T F W F F₁ W F F₂

X_STFT = _FT − + _FT −

{

^{(ˆ) (ˆ )}

}

ˆ) ˆ,

(T F ₂¹ wT₁ wT₂

Y_STFT = _π +

The following three figures will show the STFT representations of x(T) and y(T)for three different radii ∆_{g( )T} :

a) b)

Dirac impulse at T=35 and T=65 Window width = 50

Sinusoid at F=45 and F=55 Window width = 50

c) d)

Dirac impulse at T=35 and T=65 Window width = 20

Sinusoid at F=45 and F=55 Window width = 20

e) f)

Dirac impulse at T=35 and T=65 Window width = 14

Sinusoid at F=45 and F=55 Window width = 14

Figure 2.6: STFT representation of two Dirac impulses (a,c,e) and two sinusoids (b,d,f). It clearly shows the Fourier scaling theorem and inaccuracy that occurs when an analyzed signal would consist of both long duration frequencies and transients. It shows the need of longer windows for long duration frequencies and shorter windows for transients.

 7KH+HLVHQEHUJ8QFHUWDLQW\3ULQFLSOH

A signal cannot be concentrated in both time and frequency. It simply takes time for a frequency to exist. The areas of the time-frequency frames are lower bounded, which limits the choice of resolution in the time-frequency plane [CC92].

The area of a frame is calculated with (2-5) and (2-8), where is assumed that T₀ =0 and

0 =0

F :

( ) ( )

( )

( ) ( )

( ) ( )

_^





















= 

∆

∆

∫ ∫

^∞

∞

−

∞

∞

−

dF F G F F

G dT T g T T

g _FT ^FT

F G t

g _FT

2 2 2

2 2 2

2 2

2 16 1

4 (2-15)

By using

( )

² ₂

[ ( ) ]

²

2 '

4

1 FT g T F

G F _FT

= π (A-3)

and the Parseval Identity, (2-15) can be rewritten:

( ) ( )

( )

( ) ( ) ( ) ( )

_^





















⋅ 

=

∆

∆

∫

^∞

∫

∞

−

∞

∞

−

dT T g dT

T g T F

G T

g _FT

F G t

g _FT

2 2

2 2 2 2 2

2

2 '

4 1 2

4 16

π

π (2-16)

We can now use the Cauchy-Schwarz inequality, which defines the lower boundary of the product of the length of two vectors v and w by the scalar product v w, . If v and w are vectors in an infinite dimensional space they can be seen as functions. The length of v can then be calculated with _^

∫

^−∞∞

[ ]

^v

( )

^{T 2dT}_^12. The scalar product v w, can be calculated with

( ) ( ) v T w T dT

−∞

∫

∞ ^{. Setting v T( )=T g T( ) and w T( )= g T'( ) ,} the Cauchy-Schwarz inequality can be written as [CC92]:

( )

²

( )

²

( ) ( )

²

2

∫

' Re

∫

'

∫

^∞

∞

−

∞

∞

−

∞

∞

−

≥

 



 

 T g T dT g T dT Tg T g T dT

If this is substituted in (2-16):

( ) ( )

( ) _{( ) ( )} ( ) ( )

( ) ( ) ( )

( ) ( ) ( )

²₂ ²

2 2 2

2 2 2

2

2 2 2

2 2 2

2

2

2 2 2 2

2 2

1 1

) 16 (

1 2

16

2 1 4

1 2

16

' 4 Re

1 2

4 16

π

π π

π π

π π

⋅

≥

⋅ ⋅

≥



 



⋅ 

≥ ⋅

⋅ ⋅

≥

∆

∆

∫

∫

∞

∞

−

∞

∞

−

T g

T g F

G T g

dT T dT g T d F

G T g

dT T g T Tg F

G T g

FT FT FT F

G t g

If )g(T has unit energy the above yields that the area of any time-frequency frame is at least

π1

, so that the product of the radii ∆_{g( )T} and _{G ( )F}

∆ FT is bounded by:

( ) ( ) 4π

≥ 1

∆

∆gT G_FT F

This is known as the Heisenberg uncertainty principle. The Heisenberg uncertainty principle applies with localizing frequency in time with any time-frequency analysis. It is not possible to have arbitrarily good time resolution simultaneously with good frequency resolution. A long time window gives poor time resolution but relatively good frequency resolution. A large bandwidth window gives poor frequency resolution but relatively good time resolution. A certain uncertainty is unavoidable.

The variables Tˆ and Fˆ are the centers of the uncertainties and time-frequency information is spread out within a range ±∆_{g( )t} and _{G ( )F}

∆ FT

± around Tˆ and. Note that the area of a time- frequency window only depends on the shape of the window and the duration of the window.

 7KH6SHFWURJUDP

The spectrogram is a way to visualize the STFT. It is defined by [RO91]:

2 2 ˆ

2 2 ˆ

2 ( ) ( ˆ)

2 ) 1

( ˆ ) ( ˆ)

ˆ,

(

∫

^∞

∫

∞

−

∞

∞

−

− = −

−

= x T wT T e dT X F W F F e dF

F T

X_STFT ^{j πFT} _{FT FT} ^{j πFT}

π

It shows how the energy of the signal is distributed over the time-frequency plane. The spectrogram is often presented as an image of which the colors or gray-scales indicate the energy. Two examples are given in figure 2.7.

 7KHGLVDGYDQWDJHVRIWKH6KRUW7LPH)RXULHU7UDQVIRUP

A disadvantage with the STFT is that it poorly resolves phenomena which have a duration shorter than the duration of time window. Moreover, shortening the window to increase time resolution may result in unacceptable increases in computational effort of an implementation, especially if the short-duration phenomena being investigated do not occur very often. One of the biggest disadvantages however, due the Fourier scaling property described in section 2.4.4, is that it also involves less good frequency resolution. The limitation described in 2.4.6, the Heisenberg uncertainty principle is also a disadvantage, but this regards time-frequency analysis in general.

The limitation described by the Fourier scaling property makes us choose between relatively good time or frequency resolution. This implies that in the case of the STFT, we are forced to choose whether we require good time or good frequency resolution due the window that has a fixed length over the entire time-frequency plane.

A signal that contains both low frequency components with long duration and high frequency transients of short duration cannot be analyzed simultaneously using the STFT.

A way to overcome this limitation is to introduce scale. This technique will be introduced in chapter 3.