Change Detection in Telecommunication Data using Time Series Analysis and Statistical Hypothesis Testing

(1)

Master Thesis

Change Detection in Telecommunication Data using Time

Series Analysis and Statistical Hypothesis Testing

Tilda Eriksson

LiTH-MAT-EX–2013/04–SE

(2)

(3)

Change Detection in Telecommunication Data using Time

Series Analysis and Statistical Hypothesis Testing

Applied Mathematics, Link¨oping University, Institute of Technology Tilda Eriksson

LiTH-MAT-EX–2013/04–SE

Master Thesis: 30 hp Level: A

Supervisors: Lars-Olof Bj¨orketun, Ericsson GSM RAN Torkel Erhardsson,

Mathematical Statistics, Link¨oping University Examiner: Torkel Erhardsson,

Mathematical Statistics, Link¨oping University Link¨oping: June 2013

(4)

(5)

Abstract

In the base station system of the GSM mobile network there are a large number of counters tracking the behaviour of the system. When the software of the system is updated, we wish to find out which of the counters that have changed their behaviour.

This thesis work has shown that the counter data can be modelled as a stochastic time series with a daily profile and a noise term. The change detection can be done by estimating the daily profile and the variance of the noise term and perform statistical hypothesis tests of whether the mean value and/or the daily profile of the counter data before and after the software update can be considered equal.

When the chosen counter data has been analysed, it seems to be reasonable in most cases to assume that the noise terms are approximately independent and normally distributed, which justifies the hypothesis tests. When the change detection is tested on data where the software is unchanged and on data with known software updates, the results are as expected in most cases. Thus the method seems to be applicable under the conditions studied.

(6)

vi

Keywords: counter data, software update, change detection, stochastic, time series, daily profile, noise, mean value, statistical hypothesis tests

URL for electronic version:

(7)

Acknowledgements

I would like to thank Helene and Lars-Olof at Ericsson, for making me feel welcome and for giving me the opportunity to grow, both personally and in my field of study.

I would also like to thank Torkel at Link¨oping University, for helping me when my knowledge in statistics was insufficient.

Thanks to my parents, for always letting me know that I could study what-ever I wanted. And to my fellow students at Yi, who made my initial years at the university very pleasant.

Special thanks to Richard and Maud, for being true friends through difficult times. And to my husband. Thank you for your unconditional love and for always believing in me.

Finally, to myself, for never giving up.

Link¨oping, June 2013 Tilda Eriksson

(8)

(9)

Nomenclature

Most of the reoccurring abbreviations and symbols are described here.

Symbols

X Random Variable

x Observation of a Random Variable µ Population Mean

σ2 _{Population Variance}

σ Population Standard Deviation ¯

x Sample Mean

s2 _{Sample Variance}

s Sample Standard Deviation H0 Null Hypothesis

H1 Alternative Hypothesis

T Test Statistic

tobs Observation of the Test Statistic

C Critical Region, Rejection Region α Level of Significance

λα/2 α-Quantile

P P-value

df Degrees of Freedom n Number of Observations Xt Stochastic Time Series

st Seasonal Component, Daily Profile

p Period

mt Trend Component

Yt Noise Term

h Lag

(10)

x

Abbreviations

acf Auto Correlation Function cdf Cumulative Distribution Function iid Independent and Identically Distributed pdf Probability Density Function

BRP Basic Recording Period BSC Base Station Controller BSS Base Station System BTS Base Transceiver Station CN Core Network

DL Downlink

EGPRS Enhanced General Packet Radio Service GSM Global System for Mobile communications KPI Key Performance Indicator

MS Mobile Station

OSS Operation and Support System PDCH Packet Data CHannel

PI Performance Indicator RAN Radio Access Network ROP Result Output Period

RPI Resource Performance Indicator

STS Statistics and Traffic Measurement Subsystem TBF Temporary Block Flow

(11)

List of Figures

2.1 The GSM system model . . . 5

2.2 The Counters Divided Into Object Types In the STS . . . 6

2.3 The KPI-Pyramid . . . 7

3.1 An example of a pdf. . . 10

3.2 An example of a cdf. . . 10

3.3 The pdf of normal distributions with standard deviation σ. . . . 12

3.4 The α-quantile. . . 12

3.5 Two-sided quantile for a standard normal distribution. . . 13

3.6 The null distribution and rejection of H0. . . 15

3.7 One-sided P-value. . . 16

3.8 Two-sided P-value. . . 16

3.9 The t-distribution. . . 17

3.10 The F-distribution. . . 19

3.11 The pdf of an F-distributed test statistic and its two-sided rejec-tion region. . . 20

3.12 Histogram of random samples from a normal distribution. . . 21

3.13 Histogram of a large number of random samples from a normal distribution with a fitted normal pdf. . . 21

3.14 Definition of the test statistic for the Kolmogorov-Smirnov and Lilliefors test. . . 22

4.1 A time series and its estimated seasonal component and trend. . 26

4.2 The estimated noise term of figure 4.1. . . 28

4.3 Normally distributed iid noise. . . 29

4.4 The sample acf of a time series with a trend and seasonal behaviour. 30 4.5 The sample acf of an iid-sequence. . . 30

5.1 An example of counter data. . . 33

5.2 An example of counter data with missing values. . . 34

5.3 An example of counter data with an outlier. . . 34

5.4 An example of estimated trend and daily profile for counter data. 36 5.5 An example of estimated trend and daily profile for counter data. 37 5.6 An example of the sample acf for counter data with a daily profile. 38 5.7 Counter data with a removed outlier. . . 38

5.8 An example of the sample acf for the noise. . . 39

5.9 An example of the sample acf for the noise. . . 40

5.10 Testing the normality assumption of the noise. . . 40

(14)

xiv List of Figures

7.1 Relations between the KPI, PI:s and counters used. . . 46

7.2 No significant change in User Data Volume. . . 48

7.3 No significant change on KPI-level. . . 49

7.4 No significant change on PI level. . . 50

7.5 No significant change on counter level. . . 51

7.6 Change detected. . . 52

7.7 Change still detected when the possible outliers are removed. . . 53

7.8 Change detection of Throughput. . . 54

7.9 Change detection of Simultaneous TBFs per PDCH. . . 54

7.10 Change detection of DLTBFPEPDCH. . . 55

7.11 Change detection of DLEPDCH. . . 55

7.12 Change detection of Radio Link Bitrate. . . 56

7.13 Change detection of MC19DLACK. . . 57

7.14 Change detection of MC19DLSCHED. . . 57

7.15 Change detection of Multislot Utilization. . . 58

7.16 Change detection of Average Maximum Number of TS reservable by MSs. . . 58

7.17 Change detection of MAXEGTSDL. . . 59

7.18 Change detection of TRAFF2ETBFSCAN. . . 59

7.19 Change Detection of User Data Volume. . . 60

7.20 Change Detection of Daily Profile. . . 60

7.21 Change Detection of Daily Profile. . . 61

(15)

Chapter 1

Introduction

This first chapter will present some background to the problem formulation, fomu-late the questions to be answered in the forthcoming chapters and describe what topics are covered.

1.1 Background

The mobile telecommunication system is a very large infrastructure with more than 4,5 Billion GSM only subscribers around the world. The GSM system has been developed over 20 years and is continuously enhanced. This has resulted in a complex system. One way of monitoring that the GSM system behaves as expected is to use statistics and counters. In the Ericsson BSS system (the radio part of the GSM system) there are over 2000 different types of counters, often with instances in every cell, that track the behaviour of the system. Due to new software releases the behaviour of the system changes. Often these changes are visible in the counter data. Today the changes in the counter data is detected by hand. This is a very time consuming process and many changes are undetected due to the large number of counters. The purpose of this master thesis is to investigate if the changes in the counters due to new software releases can be detected using mathematical methods and computer calculations. If this is achieved, it can reduce the number of counters that has to be analysed by hand.

An earlier attempt was made to solve the problem, but the change detection failed because of assumptions that the counter data behaved in a deterministic way. Due to the stochastic behaviour of the system, probability theory and statistics seems to be a better approach.

1.2 Problem Statement

The thesis problem statement is: Can any mathematical method/methods to detect changes in counter data after new software releases be found? And if a method is found, how can it be implemented and tested on counter data?

To answer the question a couple of other questions arises. How do the counters behave? What type of changes are we looking for? Is the detection to be made on unprocessed data or data that has been processed in some way?

(16)

2 Chapter 1. Introduction

Which data is to be used to perform the analysis? Should knowledge of the system be used or should the data be treated as unknown? How do we decide if the method is working or not?

1.3 Methods used To Perform the Thesis Work

Methods used to perform the thesis work was to gain knowledge about the counter data to work with and to gain some knowledge about the system, to put the counters into a broader context. Literature on different mathematical methods, mainly in statistics and also some literature in machine learning and signal processing was also studied. And a set of data to work with was chosen. When a possible method was found, the assumptions that has to be made on the data was tested. Then the method was implemented in Matlab and tested on parts of the counter data. Finally, an analysis of the method was performed.

1.4 Scope

The purpose of the method searched for in this thesis work is only to detect changes, not to determine whether the changes should or should not occur. The method is also limited to the use of a reference week, a fixed time for software update and a test week after the update. Consideration is therefore not taken to long term trends and seasonal behaviour of the data over longer time periods. The period of the seasonal component of the data is considered as 24 hours for all data and every day of the week is treated as equal. When the data is plotted, the x-axis shows the number of hours elapsed.

1.5 Topics Covered

There are eight chapters (including this introduction). Main topics dealt with are:

Chapter 2: We get to know the counters, the system that they are part of, the behaviour of the counters and how the counter data is used and processed. Chapter 3: This chapter is devoted to mathematical concepts used in this thesis work regarding statistics and probability theory, mainly about sta-tistical hypothesis testing and the underlying assumptions.

Chapter 4: This chapter presents some theory of time series and how to study the properties of the noise term.

Chapter 5: In this chapter the results from the analysis of the counter data is presented.

Chapter 6: Describes the method for change detection in counter data. Chapter 7: Here the results from applying the change detection method to

(17)

1.5. Topics Covered 3

Chapter 8: Here we find a discussion about the method and the results, exam-ples of how the method could be used and thoughts about future work on the subject. Here we also find conclusions and motivations for the choice of method.

(18)

(19)

Chapter 2

Counter Data

There are about 3000 different types of counters in the Ericsson Base Station System of the GSM-network. They track the behaviour of the system. In this chapter we will get an overview of the counters and how the counter data is processed and used.

2.1 Overview of the Ericsson GSM System

The Ericsson GSM system is divided into three parts. The Core Network (CN), the Operation and Support System (OSS), and the Base Station System (BSS). The CN includes control servers that sets up calls, handle subscription informa-tion, location of the user, authenticainforma-tion, ciphering, etc. The OSS is a support node and the BSS is the radio part of the system that contains Base Transceiver Stations (BTS), which is the radio equipment with antenna, transmitters and receivers, and a control unit, the Base Station Controller (BSC). See Figure 2.1.

Figure 2.1: The GSM system model

(20)

6 Chapter 2. Counter Data

In different parts of the BSC there are counters tracking the behaviour of the system and the traffic that it handles. The counters considered in this report are called PEG-counters and can only be incremented. They count system- or user events, resource usage and traffic volumes.

The geographical area over which the mobile users are distributed is divided into cells and one BSC handles many cells. The counters often have instances in every cell, so one counter can have many instances since the number of cells is large. The counters track the behaviour of the system on system level, BSC-level and cell BSC-level and there is a subsystem in the BSC that collects and handles the counter values. This subsystem is called Static and Traffic Measurement Subsystem (STS).

2.2 The STS Counters

The Static and Traffic Measurement Subsystem collects counter values from the counters in the BSC. The STS collects the counter values with a fixed time interval of 5 or 15 minutes, the Basic Recording Period (BRP), saving a value that is the difference between the current value and the last value. The counter values in the STS is thus a measure of what has happened during the latest time interval, the BRP-interval. Since the counters in the BSC are only incremented, the values in the STS will be positive numbers or zero.

Figure 2.2: The Counters Divided Into Object Types In the STS

In the STS, the counters are divided into object types. There are up to 30 counters in every object type, they are mostly linked to each other in system behaviour. Figure 2.2 shows how the counters are divided into object types in the STS.

(21)

2.3. Processing of Counter Data 7

2.2.1 Collection and Behaviour of the Counter Values

When the counter values are collected from the STS it is done at fixed time intervals called ROP (Result Output Period). Since the counter values are collected every 5 or 15 minutes in the STS, the values collected between ROP-intervals are accumulated. Thus the value collected from the STS will still be a measure of what has happened during the latest time interval, but now the time interval is ROP instead of BRP. The ROP-interval is often an hour, and thus the STS counters consists of observations of the number of events during an hour, recorded every hour.

2.3 Processing of Counter Data

The counter data is used to observe the system behaviour. It can be used to analyse trends in the traffic behaviour, fault indication and as a measure of end user performance. Since the data is collected from a large number of cells, it can be relevant to aggregate the collected data to BSC level, to get an overview of the system behaviour in all cells. This is done by a sum over all the cells. The data can also be aggregated in time. For example the data on BSC level hour is a summation of all the collected values from every cell during the previous hour.

2.3.1 Key Performance Indicators and Change

Detection

The Key Performance Indicators (KPI) describes the end-users perception of the performance of the BSS. The KPI:s are calculated from counter data. Over 100 counters are used.

Figure 2.3: The KPI-Pyramid

There are different levels of the KPI:s. On the highest level are the KPI:values that describe the end-user perception. The next level shows the supporting

(22)

Per-8 Chapter 2. Counter Data

formance Indicators (PI), which describe system performance. After that comes the Resource Performance Indicators (RPI), which are not dealt with in this re-port but could be treated the same way. See figure 2.3. The KPI:s and PI:s are calculated on cell level and can be aggregated on BSC- or system level.

When the system performance is evaluated, the K and supporting PI-values are used. To see if the system behaviour has changed after a software update, the KPI-values are analysed and when a KPI-value is found different the analysis continues further down the pyramid of figure 2.3 to see where the system change takes place. Since focus are on KPI- and PI-values, counters that are not involved in the KPI-evaluations are often missed in the change detection.

(23)

Chapter 3

Statistical Hypothesis

Testing

In this chapter the concept of random variables will be introduced, together with their distributions and how to perform statistical hypothesis tests.

3.1 Random Variables And Their Probability

Distributions

A deterministic function shows the same value over and over again if you have the same input, but a random variable can change value every time you observe it, even if the input is unchanged. The value varies in an uncontrollable way, but the probability of its behaviour can often be determined.

The probability of a certain outcome x of a random variable X is denoted P (X = x) and is specified by the probability distribution of that random vari-able. The probability distribution is a kind of weight function, with higher values for the outcomes that occur more often and much smaller values for the outcomes that occur very seldom. If the variable is continuous, the probability distribution is called probability density function, pdf, and is denoted fX(x).

When the pdf is integrated over all possible values of X, its integral equals one. An area that equals one is thus distributed over all the possible outcomes and the most probable outcome has the largest piece of the area. Figure 3.1 is showing an example of a pdf. Any function f (x) ≥ 0, ∀x ∈ < that satisfies R

<f (x)dx = 1 can be used as a probability density function. [2]

For a continuous random variabe X, the probability that X take values in A is defined as:

P (X ∈ A) = Z

A

f (x)dx

If A = (−∞, x], P (X ∈ A) is called the cumulative distribution function, cdf, denoted FX(x):

FX(x) = P (X 6 x) =

Z x

−∞

f (t)dt

The cumulative distribution function thus accumulates the probability from −∞ up to a value x. [7] Figure 3.2 is showing an example of a cdf.

(24)

10 Chapter 3. Statistical Hypothesis Testing

Figure 3.1: An example of a pdf.

Figure 3.2: An example of a cdf.

To describe the behaviour of a random variable, a measure of the central tendency of its values and the dispersion around that value can be useful. The expected value, also called the mean value or average value, is a measure of that central tendency. It is the average of many independent observations of that

(25)

3.1. Random Variables And Their Probability Distributions 11

particular random variable.[2] The expected value is defined as follows: E(X) =

Z

<

xfX(x)dx = µ

The measure of dispersion around the central tendency is called the variance. It is defined as the average squared deviations from the mean: [7]

V (X) = E[(X − µ)2] = Z

<

(x − µ)2fX(x)dx = σ2

To get the dispersion in the same unit as X itself the square root of the variance is often used, called the standard deviation, σ.

3.1.1 Independent and Identically Distributed Random

Variables

Random variables X1, ..., Xn are called iid random variables if they are

inde-pendent and identically distributed. The random variables are identically dis-tributed if they have the same probability distribution and they are independent if:

fX1,...,Xn(x1, ..., xn) = fX1(x1)· · · fXn(xn)

This means that the knowledge of the value of one random variable does not affect the knowledge of the other.

3.1.2 The Normal Distribution and the Central Limit

Theorem

A commonly known distribution is the Normal Distribution, also called Gaus-sian Distribution. Many phenomena around us is approximately normally dis-tributed and it is used throughout much of the statistical theory. The math-ematical properties of the normal distribution also make it very useful. The probability density function of the normal distribution is calculated as follows:

fX(x) =

1 σ√2πe

−(x−µ)2

2σ2

And the cumulative distribution function: FX(x) = 1 σ√2π Z x −∞ e−(t−µ)22σ2 dt

Figure 3.3 shows the probability density function of normal random variables with different standard deviation.

A random variable that is normally distributed with expected value µ and standard deviation σ is denoted X ∈ N (µ, σ). When µ = 0 and σ = 1 you get the standardized normal distribution with probability density function:

ϕ(x) = √1 2πe

−x2 2

and cumulative distribution function: Φ(x) = √1

2π Z x

−∞

(26)

Figure 3.3: The pdf of normal distributions with standard deviation σ.

Figure 3.4: The α-quantile.

The x-value where the area of the probability density function above x equals α is called the α-quantile. See figure 3.4. For a standardized normal distribution the α-quantile is denoted λα.

(27)

3.2. Statistical Description of Data 13

ϕ(−x) = ϕ(x) and the area of ϕ(x) from x to ∞ is equal to Φ(−x). Since the whole area is one Φ(−x) = 1 − Φ(x). [2] The total area below −x and above x is thus 2 ∗ Φ(−x). If that area is α, then x = λα/2. See figure 3.5. This fact is

used in two-sided hypothesis tests.

Figure 3.5: Two-sided quantile for a standard normal distribution. The normal distribution has many applications. One reason for that is that the sum of many iid random variables are approximately normally distributed with mean value nµ and standard deviation σ√n, where n is the number of random variables. This is called the central limit theorem. The sample mean:

¯ x = 1 n n X i=1 xi

of a sequence of n iid random variables is thus approximately normally dis-tributed with mean µ and standard deviation σ/√n, another fact used in the hypothesis testing.

Linear combinations of independent and normally distributed random vari-ables are also normally distributed. If X1 ∈ N (µ1, σ1) and X2 ∈ N (µ2, σ2),

then X1− X2∈ N (µ1− µ2,pσ21+ σ22). [2]

3.2 Statistical Description of Data

A set of data, or sample, is treated as observations of a larger population. When the data is described, the purpose is to describe the behaviour of the population as accurately as possible. Even though the population does not have to be real, the name arises from the fact that in statistics, the object of interest is often a group of individuals, a population.

(28)

We wish the sample to reflect the characteristics of the whole population. Thus it has to be drawn at random. A random sample of observations of the random variable X is a set of iid random variables X1, ..., Xn each with the

distribution of X. [7]

The probability model for the population of the data is usually not known. It has to be estimated using the sample. The estimator is a function of the sample observations and such a function is called a statistic. It is a random variable since it is a function of random variables:

T = t(X1, ..., Xn)

Since it is a random variable, a distribution function describes its behaviour. It is called the sampling distribution and it is derived from the distribution of the population. For example the population mean, µ, is estimated using the sample mean: ¯ X = 1 n n X i=1 Xi

which is a statistic with a sample distribution and corresponding expected value and variance. The expected value of the sample mean is the mean of the whole population.

The sample variance, S2 _{is calculated:}

S2= 1 n − 1 n X i=1 (Xi− ¯X)2

S is the sample standard deviation and the expected value of the sample variance is the population variance, σ2.

3.3 How to Perform a Hypothesis Test

When performing a hypothesis test we want to test an assumption about a random sample, x = (x1, ..., xn). The assumption could be that the sample

observations come from a normal distribution, that the distribution mean has a certain value etc. The assumption that you want to test is called the null hypothesis, H0. The null hypothesis is tested against an alternative hypothesis

H1that is considered true if H0is rejected. If H1is a single interval, for example

if the assumption is µ = 0 and H1 : µ > 0, the test is called one-sided and if

instead H1: µ 6= 0 (µ > 0 or µ < 0) the test is called two sided.

First you have to find a so called test statistic: T = t(X), X = (X1, ..., Xn)

The test statistic is a function of the random variables that the sample observa-tions are observaobserva-tions of. The test statistic tells us how those random variables behave if the null hypothesis is true. It is based on knowledge of the distribu-tion under the null hypothesis, the null distribudistribu-tion, and different assumpdistribu-tions requires different test statistics with different null distributions. There are some well specified test statistics to use for different kinds of tests.

H0is rejected if the observation of the test statistic, tobs= t(x), belongs to a

(29)

3.3. How to Perform a Hypothesis Test 15

Figure 3.6: The null distribution and rejection of H0.

if H0 is true. If the test is one-sided the critical region is the values above the

α-quantile of the null distribution, see section 3.1.2. If the test is two sided the critical area is below −tα/2and above tα/2. See figure 3.6. The null distribution

then has its most probable values between −tα/2 and tα/2 and if tobs takes

values there the assumption of H0 cannot be rejected. α is called the level of

significance of the hypothesis test and the test is called significant at level α if the hypothesis is rejected and the assumption is considered false. Usual values of α are 0.05, 0.01 and 0.001. [2]

Suppose that the value of tobs ∈ C. Then H0 is rejected, thus we assume

that the assumption we want to test does not hold. But it could also be the case that the hypothesis is true and that tobstakes a value that is less probable

but still probable under the null hypothesis. Therefore α is the probability that H0 is rejected if H0 is true. This is called type I error. Another kind of error

is the so called type II error, which is the probability of not rejecting H0when

H0 is not true.

If H0is not rejected this does not necessarily mean that H0is true.

3.3.1 Using P-values

If the hypothesis test is one-sided, the P-value is the area of the null distribution above the observed value of the test statistic, tobs, and is defined as follows:

P = P (T ≥ tobs)

This is shown in figure 3.7. If the P-value is smaller than the level of significance, α, the null hypothesis is rejected. This means that we do not believe in H0 if

the result is unlikely when H0is true. [2]

When the hypothesis test is two-sided a two-sided P-value is calculated: P = P (|T | > tobs)

(30)

Figure 3.7: One-sided P-value.

Figure 3.8: Two-sided P-value.

The null hypothesis is rejected if P < α, and as seen in figure 3.8 the rejection of H0 is more accurate for smaller P-values.

3.4 Comparing the Means of Two Samples

When comparing the means of two samples from normal distributions, a t-distributed test statistic is used. The density function of the t-distribution is shown in figure 3.9 with different degrees of freedom, df . The degrees of freedom is the number of samples minus the number of estimated parameters. [9] If df = ∞, the t-distribution becomes the standard normal distribution.

(31)

3.4. Comparing the Means of Two Samples 17

Figure 3.9: The t-distribution.

where the variance of the two samples are considered equal and one where the variances are not equal. In both cases a t-test is used, but with different appearance of the test statistic.

If the sample standard deviation of the two samples are considered equal, the test statistic is defined as follows:

T = ¯ X1− ¯X2 Sp q 1 n1 + 1 n2

where Sp is called the pooled standard deviation, and Sp2 is an estimate of the

common variance: [8]

S_p2= (n1− 1)S

2

1+ (n2− 1)S22

n1+ n2− 2

The term df = n1+ n2− 2 is the number of degrees of freedom and S12 and S22

are the respective sample variances.

If the variances are not considered equal, σ₁26= σ2

2, the test statistic is defined

as: T = ¯ X1− ¯X2 q S2 1 n1 + S2 2 n2

where S1 and S2are the sample standard deviations. This test statistic is only

approximately t-distributed. The number of degrees of freedom is calculated as: [9] df = ( S₁2 n1 + S2₂ n2) 2 (S2 1/n1)2 n1−1 + (S2 2/n2)2 n2−1

(32)

We want to test the null hypothesis, H0: µ1= µ2 (or µ1− µ2 = 0) against

the alternative hypothesis H1 : µ1 6= µ2. The test is thus two-sided. If the

null hypothesis is true, the test statistic is t-distributed with the given degree of freedom according to equal variance or not. The observation of the test statistic is: tobs= ¯ x1− ¯x2 sp q 1 n1 + 1 n2

if the variances are considered equal and: tobs= ¯ x1− ¯x2 q s2 1 n1 + s2 2 n2

if the variances are unequal. If |tobs| > tα/2,df the null hypothesis is rejected.

tα/2,df means that the area above tα/2 of a t-distributed probability density

fuction with df degrees of freedom equals α/2. α is the pre-defined level of significance. See figure 3.6. You can also calculate the P-value:

P = P (|T | > tobs)

and reject H0if P > α.

The test is based on an assumption that the samples are random and come from normal distributions. Even if the distributions are not normal this kind of test can still be used, since the test is based on the sample mean and the sample mean is approximately normally distributed if the number of observations is large enough, according to the central limit theorem.

3.4.1 Why The Test Statistic Is t-Distributed

When we compare the mean of two different samples we want to say something about the mean of the entire population, not just the observations drawn from it. But since only samples of the two populations are available to us, we have to estimate the mean of the population using the sample mean. This gives rise to uncertainty in the estimated mean and the sample mean has its own probability distribution, expected value and standard deviation.

If the standard deviation is not known, it has to be estimated as well. This gives rise to more uncertainty. If the standard deviation is known: [2]

¯ X − µ

σ/√n ∈ N (0, 1)

Since s, the estimate of σ, has to be used instead of σ, _s/X−µ¯√

n has to be used

instead of _σ/X−µ¯√

n, where s is the estimate of the sample standard deviation:

s = v u u t 1 n − 1 n X i=1 (xi− ¯x)2

(33)

3.5. Comparing the Variance of Two Samples 19

T = ¯ X − µ

S/√n, T ∈ t(n − 1) T is t-distributed with n − 1 degrees of freedom.

If you exchange ¯X with ¯X1− ¯X2and use the fact that:

¯ X1− ¯X2∈ N (µ1− µ2, s σ2 1 n1 +σ 2 2 n2 )

you get the test statistics for the two sample t-test, see section 3.1.2.

3.5 Comparing the Variance of Two Samples

Figure 3.10: The F-distribution.

The ratio of the sample variances of two independent samples from normal distributions is F-distributed with n1− 1 numerator and n2− 1 denominator

degrees of freedom, where n1 and n2 are the number of observations of the

samples. [8] The F-distribution is shown in figure 3.10. This fact can be used when testing if two samples from normal distributions have equal variance. The test statistic for the test is

F = S

2 1

S₂2

The null hypothesis H0 : σ12 = σ22 is rejected if fobs > Fα/2,n1−1,n2−1 or if

fobs< F1−α/2,n1−1,n2−1 for a two sided test. See figure 3.11.

When calculating a two-sided P-value for an F-distributed test statistic, care must be taken as to whether the value of fobsis in the upper or lower tail of the

(34)

Figure 3.11: The pdf of an F-distributed test statistic and its two-sided rejection region.

F-distribution. If fobsis below the median of the null F-distribution it is in the

lower tail and if it is greater than the median it is in the upper tail. The two sided P-value is:

P = 2 ∗ (1 − P (Fn1−1,n2−1> fobs))

if it is in the lower tail and:

P = 2 ∗ P (Fn1−1,n2−1> fobs)

if it is in the upper tail.[5]

The F-test is more sensitive to the normality assumption than the t-test, since the central limit theorem can not be used.

3.6 Testing Normality

The assumptions that has to be made to use a statistical hypothesis test of the kind described in section 3.4 and 3.5 is that the samples are independent and identically distributed, hopefully with a normal distribution. To test the normality assumption two different methods are considered in this report, one of them visual.

3.6.1 Histogram

A histogram is a graphical representation of the distribution of a set of observa-tions and it can be used for density estimation. The values of the observaobserva-tions are divided into bins and the height of each bin represents the number of obser-vations in that bin. See figure 3.12. One rule of thumb for the number of bins is the square root of the number of observations. [10]

(35)

3.6. Testing Normality 21

Figure 3.12: Histogram of random samples from a normal distribution.

Figure 3.13: Histogram of a large number of random samples from a normal distribution with a fitted normal pdf.

Since the histogram is an estimate of the probability distribution for a set of data, a probability density function can be fitted to the data if the population distribution is known. If we want to test if the observations come from a specific

(36)

distribution we can compare the shape of the histogram to the fitted pdf. See figure 3.13. If we compare the shape of figure 3.12 with the shape of figure 3.13, with the same probability distribution but a much larger number of observations, we see that the shape of the histogram approaches the shape of the population pdf when the number of observations gets larger.

3.6.2 Lilliefors Test

The Kolmogorov-Smirnov test is a statistical test to determine if a set of obser-vations comes from a normal distribution. The test statistic for the Kolmogorov-Smirnov test, D, is the largest absolute difference between the sample cdf, Fn(x),

and the population cdf, F (x). [7] See figure 3.14. D = supx|F (x) − Fn(x)|

This test statistic is distribution free and a table must be used for the critical values.

Figure 3.14: Definition of the test statistic for the Kolmogorov-Smirnov and Lilliefors test.

When the mean and variance are not specified and have to be estimated from the sample, the Kolmogorov-Smirnov test has to be modified. This is done by using the Lilliefors approach of the Kolmogorov-Smirnov test, where the tables for the test statistic is changed to fit a normal distribution with estimated mean and variance. [6] The test statistc is then:

D = supx|F∗(x) − Fn(x)|

where Fn(x) is sample cdf and F∗(x) is the normal population cdf with estimated

(37)

3.6. Testing Normality 23

Lilliefors table, the null hypothesis that the observations come from a normal distribution is rejected.

(38)

(39)

Chapter 4

Time Series Analysis

In this chapter we will learn how to estimate trend and seasonal components in a time series and how that can be used for testing assumptions on the remaining noise term.

4.1 Time Series and Hypothesis Tests

A time series is a set of observations {xt}nt=1, each one being recorded at a

specific time t. [3] If the time series consists of observations of random variables, i.e. if xtis an observation of Xt, the time series is a realization of a stochastic

process. Like in the case of random variables, every time series is one of many possible realizations of the same underlying stochastic process. Thus two time series that are observations of the same process are not necessarily equal.

When the set of times is a discrete set, the time series is a discrete-time series. This is the case when observations are made on fixed time intervals. Another property of a time series is stationarity. A time series is said to be stationary if it has the same behaviour regardless of when you observe it. More precisely, Xt, t = 0, 1, ... and the time shifted series Xt+h, t = 0, 1, ... has to have

the have the same statistical behaviour.

In many applications the behaviour of a given time series is not specified and has to be analysed. You have to look for dependence among the observations, periodic behaviour, long-term trends, etc. A suitable model for the time series has to be found and often the signal has to be separated from the noise term. A probability model for the remaining noise term has to be found and this can be used as a diagnostic tool for the accuracy of the model of the signal.

When statistical hypothesis tests are to be performed on a time series, we wish its behaviour to be like that of iid random variables. If a sequence of random variables is iid and the mean of the series is zero the stochastic process is called an iid-process and is denoted IID(0, σ2). If there is a seasonal be-haviour in the time series there is a dependence among the observations. This dependence has to be eliminated for the hypothesis test to be used.

(40)

26 Chapter 4. Time Series Analysis

4.2 Estimation of Trend, Seasonality and the

Remaining Noise Term

In a time series we can often find an underlying trend and a seasonal variation, a repeated pattern over a shorter time interval. The seasonal variation can be modelled as a periodic component, st, with a fixed period p, such that st= st+p.

A model can then be created for the time series by dividing the series into a trend component mt, the seasonal component stand a noise term Yt. The noise

term is a random variable that describes the deviations from the model. This kind of model is called the classical decomposition:

Xt= mt+ st+ Yt

Since the trend is described in the trend component, the mean value of the noise term is zero. The sum of the periodic component over a whole period is also zero: [3]

p

X

k=1

sk = 0

Figure 4.1: A time series and its estimated seasonal component and trend. Figure 4.1 shows a time series with estimated trend and seasonal behaviour. The trend is a deterministic component that describes the level of the signal. The seasonal component is also a deterministic component that describes the seasonal behaviour around the trend. The remaining stochastic noise term is the deviations of the signal from those components.

To estimate the seasonal component, a suitable period is choosen, with p = 2q if p is even. Then a trend is estimated using a moving average filter, which is a weighted average of neighbouring points that is used to flatten the

(41)

4.2. Estimation of Trend, Seasonality and the

Remaining Noise Term 27

fast variations in the data. This leaves the slow components, the trend. A special kind of moving average filter is used that is suitable for estimation of seasonal components: [3]

ˆ

mt= (0.5xt−q+ xt−q+1+ ... + xt+q−1+ 0.5xt+q)/p, q < t ≤ n − q

If p instead is odd, then p = 2q + 1 and:

ˆ mt= 1 2q + 1 q X j=−q xt−j, q + 1 ≤ t ≤ n − q

After the trend is estimated, then for every k = 1, ...p, the average of the deviations from the trend is calculated, wk. How many points that can be used

for calculating the average of the deviations is decided by the number of periods in the time series when the first and last q values is eliminated in the trend-estimation. The deviations from the trend is calculated for every k = 1, ...p as:

{(xk+jp− ˆmk+jp), q < k + jp ≤ n − q}

and wk is then calculated by taking the average of those deviations over the

number of periods fitted in the data available. To make sure that the seasonal component sum to zero, the seasonal component is estimated as:

sk = wk− 1 p p X i=1 wi, k = 1, ..., p since p X k=1 sk= p X k=1 (wk− 1 p p X i=1 wi) = p X k=1 wk− p ∗ 1 p p X i=1 wi= p X k=1 wk− p X i=1 wi= 0

and since s is periodic with period p, sk = sk−p for k > p.

Now the seasonal component has been estimated. To estimate the trend component the seasonal component is first removed from the original data. The deseasonalized component, dt, is defined as:

dt= xt− st, t = 1, ..., n

Then another moving average filter is used to estimate the trend since the earlier trend estimation we did was just for calculating the seasonal component. This trend estimation uses a moving average filter of the type:

mt= 1 2q + 1 q X j=−q dt−j, q + 1 ≤ t ≤ n − q

It also eliminates the components of fast variations and is thus a low-pass filter since the components of low frequencies passes the filter. This is not the only way of estimating the trend. Instead of this trend estimation, a constant representing the average value could be used.

(42)

To get the noise term the trend is eliminated from the deseasonalized time series and we get:

Yt= xt− st− mt

The estimated noise term of the time series in figure 4.1 is shown in figure 4.2.

Figure 4.2: The estimated noise term of figure 4.1.

4.3 Testing the Independence Assumption of the

Noise

As discussed throughout this report, for many hypothesis tests we need the samples to be iid. When the mean of two samples is to be compared, the data is averaged over the number of observations. If the number of observations is a whole number of periods, the seasonal behaviour is extinguished. Thus the behaviour of the noise term can be used to test the assumptions of the hypoth-esis test. We want to check if the noise term is approximately independent IID(0, σ2_{) and if it is normally distributed N (0, σ}2_).

To see if the noise sequence is independent a first test is to plot the sequence. A typical behaviour if there are dependence among the observations is that many observations in a row are having the same sign. Figure 4.3 shows an independent noise sequence that is normally distributed.

Another more distinct method to see if the estimated noise sequence is in-dependent is to use the so called sample autocorrelation function, sample acf. The autocorrelation of a sequence at lag h is the correlation between the ob-servations in the sequence. The autocorrelation function, acf, for a stationary

(43)

4.3. Testing the Independence Assumption of the Noise 29

Figure 4.3: Normally distributed iid noise.

time series is defined as: ρX(h) =

γX(h)

γX(0)

= Corr(Xt+h, Xt)

where γX(h) = Cov(Xt+h, Xt) is the covariance defined as:

γX(r, s) = Cov(Xr, Xs) = E[(Xr− µX(r))(Xs− µX(s))]

To test the independence of the noise sequence the sample acf is used as an estimate of the acf, since we do not have a model for the noise sequence but only observed data. The sample autocorrelation function is:

ˆ

ρ(h) =γ(h)ˆ ˆ

γ(0), −n < h < n where the sample autocovariance function is:

ˆ γ(h) = 1 n n−|h| X t=1 (xt+|h|− ¯x)(xt− ¯x), −n < h < n

and the sample mean is calculated as:

¯ x = 1 n n X i=1 xi

If the data is containting a trend the sample acf will exhibit slow decay when h increases and if there is a periodic component it will exhibit a similar

(44)

Figure 4.4: The sample acf of a time series with a trend and seasonal behaviour.

Figure 4.5: The sample acf of an iid-sequence.

behaviour with the same periodicity. [3] Figure 4.4 shows the sample acf for a sequence with trend and seasonal behaviour.

For an iid sequence the autocorrelation is ρX(0) = 1 and ρX(h) = 0, |h| > 0,

(45)

4.3. Testing the Independence Assumption of the Noise 31

be regarded as iid-noise. See figure 4.5. For a formal test, one can use the fact that for large n the sample autocorrelations of an iid sequence with finite variance are approximately iid with distribution N (0,1

n). We can therefore

check if the observed noise sequence is consistent with iid noise by examine the sample autocorrelations of the residuals. If the sequence is iid, 95% of the sample autocorrelations should fall between the bounds ±1.96/√n. [3]

To check if the noise sequence is normally distributed a histogram or the Lilliefors Test can be used, see 3.6.1 and 3.6.2.

(46)

(47)

Chapter 5

Analysis of Counter Data

In this chapter we will get to know the behaviour of the counter data.

5.1 Selection of Data to Work With

When performing an analysis of counter data, a set of data that represents a usual behaviour of the system is preferred. Therefore data associated with a commonly used KPI-value and its underlying PI- and counter-values is used. The data is aggregated to BSC-level and one sample represents the latest hour.

Figure 5.1: An example of counter data.

Figure 5.1, figure 5.2 and figure 5.3 show some examples of counter data. In figure 5.2 we see that some data points are missing. Missing values are handled in section 5.4.

(48)

34 Chapter 5. Analysis of Counter Data

Figure 5.2: An example of counter data with missing values.

(49)

5.2. The Daily Profile 35

In figure 5.3 we see a data point that lies far outside the other data points. This can be an indication that there is something wrong with that particular collection of data and that point needs to be removed for the analysis to be as accurate as possible. The so called outliers are handled in section 5.3.

As seen in all the figures shown here, the data exhibits a seasonal behaviour. This seasonal behaviour is the variations over day and comes from the traffic pattern with peak hour or hours, called busy hour(s) since the traffic is high.

5.2 The Daily Profile

The data exhibit a similar behaviour every 24 hours. This is the daily profile and it is estimated using the technique in section 4.2.

Figures 5.4 and 5.5 show the estimation of the trend and daily profile for two counters and their remaining noise term.

The daily profile can also be seen in the autocorrelation function. In figure 5.6 the sample acf, ρX(h) exhibits a slow decay for larger values of h and the

periodicity p = 24 is clearly seen. This is a typical behaviour for a time series with a periodic component, see section 4.3.

5.3 Outliers

First the classical decomposition model of the data, see section 4.2, is estimated ignoring values that are not numbers. The variance of the data is estimated and then the deviations from the model are calculated. The data point where the deviation is largest is eliminated and the variance is estimated again. If the new variance is significantly smaller than the previous variance, the removed point is regarded as an outlier. If the variance is not significantly smaller it means that the point with the largest deviation from the model is still close to the other data points and it is put back since it was not regarded as an outlier. Figure 5.7 shows data with a removed outlier.

This method is a heuristic, since the one-sided hypothesis test applied is used under violated assumptions. There is a strong correlation between the first and second sample. But when tested on the data studied, it seems that the data points that lie far outside the other data points are removed.

After the outliers have been removed a new estimation of daily profile and trend has to be made, since the outliers could have affected the previous result.

5.4 Missing Values

For the estimation of the daily profile, the time series has to have 24 samples a day. Therefore a sample missing has to be regarded as a missing value for the period to be correct. If there is a missing value at t = i, the value of sk where

sk= sk+jp = si is estimated using the average of the deviations:

{(xk+jp− ˆmk+jp), q < k + jp ≤ n − q}

but with the addition of the criteria k + jp 6= i, see section 4.2.

The missing value xi is then replaced with the estimated value for that

(50)

(51)

5.4. Missing Values 37

(52)

Figure 5.6: An example of the sample acf for counter data with a daily profile.

(53)

5.5. Testing Independence and Normality of the Noise 39

5.5 Testing Independence and Normality of the

Noise

The noise term is what is left when the daily profile and the trend are eliminated from the data. In this report it is seen as the deviations around the mean value, when the data is averaged over a whole number of days. This is due to the fact that the seasonal components averaged over a whole period is zero, see section 4.2, and that the mean is approximately constant, see figure 5.4 and 5.5. Thus the variance of the noise term is used in the t-test for equal mean in the change detection method. To see if the noise term is iid, we use the autocorrelation function as seen in section 4.3.

Figure 5.8: An example of the sample acf for the noise.

For most of the counters there are still dependence in the noise term, since the autocorrelation has more than small deviations from zero for non-zero lags. In most cases the sample acf looks like something between figure 5.8 and figure 5.9. This looks reasonaly good if we compare with the original signal, an example shown in figure 5.6.

For the normality assumption of the noise term, a histogram and the Lil-liefors test is used, see section 3.6.1 and section 3.6.2. Most of the counters exhibit a behaviour close to that of a normally distributed population, but since the number of points is less than 200 it is not always easy to see in a histogram. The Lilliefors test with a 0.01 level of significance is used and then the hypothesis of normality cannot be rejected for most of the counters. See figure 5.10.

Sometimes the analysis of the whole week shows that the noise is not nor-mally distributed. When changing the data to only weekdays the analysis looks different. Then the hypothesis of normality can not be rejected.

(54)

Figure 5.9: An example of the sample acf for the noise.

(55)

Chapter 6

The Change Detection

Method

This chapter presents the change detection method and the theory behind it. In the next chapter we will see the results from applying the change detection to counter data.

6.1 The Change Detection Method and Its

Limitations

A shown in chapter 5, the counter data can be modelled as a stochastic time series with a daily profile. To detect changes in the data we cannot just compare if the values are the same, due to the stochastic behaviour. We also need to know which changes to detect. In this thesis work focus are on changes in the mean value and in the daily profile, since it is two factors that characterize the data.

The change detection method uses about two weeks of data. The first week, or set of data, is a reference week or reference data. For this week we want to freeze as many factors as possible to be able to detect changes due to software changes and not to changes in the environment. The second week, or set of data, is the test week or test data.

Different days of the week have different traffic patterns. Since only a limited amount of data is used, there will be too few data-points to estimate a model for each day, and therefore different days has to be treated as equal. The test data has to be from the same weekdays as the reference data to eliminate the risk that changes due to difference between weekdays will appear.

No consideration is taken to long term trends and seasonal patterns over longer periods than 24 hours, and this can affect the result. Therefore the change detection can only answer the question if the test week is significantly changed from the reference week. Care must be taken when choosing the data for the reference week since it has to behave as a typical week if we want to say something about whether the test week behaves as a typical week and not just as the reference week.

One thing to be done to get a more correct model, and at the same time use

(56)

42 Chapter 6. The Change Detection Method

the amount of data provided, is to use only weekdays since there is a greater dif-ference in traffic pattern between weekdays and weekends than there is between different weekdays. (Note that differences in this pattern can occur in different countries.) Since less data can be used, a trade-off must be made between using more data or a better model.

One more limitation of the change detection method is that it can show different results when performing it on different number of days. Care must be taken to what you want to achieve by using the method. If the data changes in the end of the week, a change will occur according to the change detection method if you test against the whole week, but if you only choose the beginning of the week, the result will be different since the change was in the end of the week.

Before the change detection takes place, missing values and outliers are han-dled, see section 5.4 and section 5.3. Then the daily profile is estimated, the noise variance is calculated and statistical hypothesis tests are performed to test whether the mean value and/or the daily profile of the data has changed.

6.1.1 Change Detection of Mean Value

The change detection focuses on the mean value of the counter data. This is a change easy to understand and to calculate. As discussed before, comparison of the mean of the two data-sets cannot just be performed by comparing if the two values are equal. Due to the stochastic behaviour, the two means can differ but still be considered equal in a probabilistic sense.

When performing the change detection of the mean value, the data is aver-aged over day and thus the daily profile is eliminated. Since the data is also averaged over the same number of days and the same weekdays before and after the software change, we judge that also variations due to differences between weekdays are eliminated reasonably well.

The noise term describes the deviations from the mean under the same probability model. Therefore the variances of the noise terms is used in the hypothesis test.

For the hypothesis test of equal mean we use a two-sided t-test, see section 3.4. Since we have to know if the variance is equal or not, a two-sided hypothesis test of equal variance has to be performed on the noise variance prior to the t-test. The F-test of equal variance is described in section 3.5.

If the hypothesis of equal mean is rejected at the pre-defined level of signif-icance, α, the test result is considered significant and a change has occurred in the test data. To tune the parameter α, data without a software change can be used. If a change is detected, the P-value can be used as a measure of how accurate the rejection of the hypothesis that there is no change is. If the P-value is less than α, the hypothesis of no change is rejected and then the closer to zero the P-value is, the more can we trust the detection of a change, see P-value section 3.3.1.

6.1.2 Change Detection of Daily Profile

The change detection is also done on the daily profile. The daily profile looks different over the week and is affected by changes in behaviour of the mobile users, mobile subscription conditions, weekends, larger events in the area, etc. A

(57)

6.2. Normality Assumption 43

model of the daily profile is estimated using the reference week and the variance of the noise term is calculated. Then the test data after software update is used. The deviations from the daily profile of the reference week is calculated and de-trended since the trend change is not considered here. The remaining noise term should be of the same size as the reference noise if the model is well suited also for this data. If the variance of the noise term is significant greater than the variance of the reference week noise the model for the daily profile is not suited for this data and thus the daily profile has changed. Since it is very unlikely that the model is better fitted for the test week, it is only relevant to know if the variance is significantly greater than the variance of the reference week. Thus a one-sided hypothesis test is used.

6.2 Normality Assumption

If the normality assumption of the counter data is violated, the result from the change detection of daily profile may or may not be correct since the test is based on the fact that the data is normally distributed. Therefore a warning occurs if the normality assumption is rejected at the 0.01-level of significance.

The change detection of the mean is not so sensitive to the normality as-sumption, see section 3.4.

6.3 Level of Significance

If the observed value of the test statistic is close to the rejection region, the result of the change detection method can change when the level of significance is changed. For smaller values of α, a rejection of the null hypothesis of no change is a stronger indication that a change has occurred. Common values of α are 0.05, 0.01 and 0.001, see section 3.3. This can also be seen in the P-value since a P-value around the level of significance can show which value of α that changes the outcome of the hypothesis test.

A better scenario would be that of two reference weeks instead of one, where one of the reference weeks could be used for choosing an appropriate value of α for that particular environment. You could also change the value of α depending on the use of the method. If you would like all changes to be detected, a larger value of α is suitable, for example 0.05, and if you do not want any false alarm, a smaller value of α is suitable, for example 0.001.

(58)

(59)

Chapter 7

Applying the Change

Detection Method To

Counter Data

In this chapter we will see the results of the change detection method on counter data with known changes and on counter data without changes.

7.1 Selection of Data

When performing change detection on counter data, it is preferable to use a set of data on which the change detection method can be analysed. Therefore a set of data with a feature change that affects a KPI called Throughput and its underlying PI:s and counters is used. A measure of user data volume is also used, since it could affect the other results. All data comes from EGPRS Downlink (DL) transfers, which means that it is data and not speech and that it is directed from the network to the mobile.

To know if the change detection method is working correctly, also data with-out any known changes is used.

7.1.1 Data Volume

The User Data Volume is the data that the end-user, the subscriber, pays for. The User Data Volume is not an end-user KPI, but in general if the performance of the network improves then the users will be able to transfer a greater volume of data within the same time. Thus, this measure is important to monitor, since an unexpected change in data volume may indicate performance problems.

If the data volume is too low, this indicates that the traffic is very low and deviations found in other KPIs and PIs may be due to too scarce data rather than ”true” system effects. The user data volumes used in this thesis work are high and should not affect the result in that way.

(60)

46 Chapter 7. Applying the Change Detection Method To Counter Data

7.1.2 The Throughput KPI and Underlying PI:s and

Counters

The Throughput is a KPI that measures the total percieved transfer velocity of the data volume that is going from the network out to the user on all data channels. It is thus a measure of end-user perceived performance, which is the definition of a KPI, see section 2.3.1. The Throughput is measured in [kbit/s]. Figure 7.1 is showing some underlying PI:s and counters associated with the Throughput.

Figure 7.1: Relations between the KPI, PI:s and counters used. What the Simultaneous TBFs per E-PDCH shows is the number of ”users” carried on each individual Packet Data CHannel (PDCH). It is a measure of the traffic load in the system. This is also the case for the PI Average number of simultaneous EGPRS DL TBFs in Cell. TBF stands for Temporary Block Flow and it is the logical identifier for the data to be sent to one Mobile Station (MS). Simultaneous TBFs per E-PDCH is dependent on two counters called DLTBFPEPDCH and DLEPDCH. The formula for calculation is:

Simultaneous TBFs per E-PDCH =DLT BF P EP DCH DLEP DCH

The Radio Link Bitrate per PDCH measures the performance of the radio interface between mobile and network per channel. If each PDCH is thought of as a ”radio pipe” through which data can be transferred then the measured radio link bit rate represents the average size of each such ”radio pipe”. There are many different influences on the quality of a radio link: signal strength; interference; time dispersion and more. The formula for the Radio Link Bit Rate (DL) per PDCH is:

Radio Link Bit Rate = M C19DLACK

(61)

7.2. Interpreting the Figures 47

The Average Maximum Number of Time Slots (TS) reservable by MSs is a measure of how much of the data channels that the MSs can reserve. The formula for the Average Maximum Number of TS reservable by MSs is:

Average Max Nr of TS Reservable = M AXEGT SDL T RAF F 2ET BF SCAN The Multislot Utilization shows, in percentage, the amount of mobiles that get as many packet data channels as they can handle. Typically the Multislot Utilization is high at low traffic hours while it decreases at high traffic hours.

7.2 Interpreting the Figures

In the figures the test data is plotted together with the reference data, with the corresponding mean values. The header tells us which KPI, PI or counter that is studied and the question ”Significant change of mean” is answered using the change detection method at the level of significance given by α. The P-value is defined in section 3.3.1.

When plotting the daily profile, the daily profile of the reference data is plotted together with the test data. The question ”Significant change of daily profile” is answered using the method described in section 6.1.2 at the level of significance given by α.

7.3 Counter Data Without Known Changes

The data is taken from the same BSC, the same software release and two weeks in a row. The KPI:s and counters are the ones described in section 7.1. Worth noticing is that there is no change in the User Data Volume, see figure 7.2. This could be an indication that the end-user behaviour is similar and should therefore not affect the result.

Detection of changes in the KPI, in 4 of the 5 PI:s and in the 6 counters do not occur. Figure 7.3, figure 7.4 and figure 7.5 are showing some examples of the result from the change detection. The PI that is considered changed is shown in figure 7.6. The reason for the change is not known. As seen around sample 150, the change could be due to outliers, but when the 3 samples creating the outlying values are removed there is still a significant change of mean value. This is seen in figure 7.7. Another reason could be that there is something wrong with the underlying counters for a day around sample 100 or that something in the environment changed that day, that only had an impact on this PI.

When performing an hypothesis test of equal mean, an hypothesis test of equal noise variance is first performed. On the 0.01-level of significance, the hypothesis of equal noise variance can not be rejected, except in one case. This is in the PI Multislot Utilization, discussed earlier and shown in figure 7.6.

7.3.1 Level of Significance

Since the data used is unchanged, the method should not find any changes except in special cases. Therefore this data can be used for tuning the parameter α. There is no detection of a change for α = 0.01, both when using a whole week

(62)

(63)

7.3. Counter Data Without Known Changes 49

(64)

(65)

7.3. Counter Data Without Known Changes 51

(66)

Change Detection in Telecommunication Data using Time Series Analysis and Statistical Hypothesis Testing

Master Thesis

Change Detection in Telecommunication Data using Time

Series Analysis and Statistical Hypothesis Testing

Tilda Eriksson

Change Detection in Telecommunication Data using Time

Series Analysis and Statistical Hypothesis Testing

Abstract

Acknowledgements

Nomenclature

Symbols

Abbreviations

Contents

List of Figures

Chapter 1

Introduction

1.1

Background

1.2

Problem Statement

1.3

Methods used To Perform the Thesis Work

1.4

Scope

1.5

Topics Covered

Chapter 2

Counter Data

2.1

Overview of the Ericsson GSM System

2.2

The STS Counters

2.2.1

Collection and Behaviour of the Counter Values

2.3

Processing of Counter Data

2.3.1

Key Performance Indicators and Change

Detection

Chapter 3

Statistical Hypothesis

Testing

3.1

Random Variables And Their Probability

Distributions

3.1.1

Independent and Identically Distributed Random

Variables

3.1.2

The Normal Distribution and the Central Limit

Theorem

3.2

Statistical Description of Data

3.3

How to Perform a Hypothesis Test

3.3.1

Using P-values

3.4

Comparing the Means of Two Samples

3.4.1

Why The Test Statistic Is t-Distributed

3.5

Comparing the Variance of Two Samples

3.6

Testing Normality

3.6.1

Histogram

3.6.2

Lilliefors Test

Chapter 4

Time Series Analysis

4.1

Time Series and Hypothesis Tests

4.2

Estimation of Trend, Seasonality and the

Remaining Noise Term

4.3

Testing the Independence Assumption of the

Noise

Chapter 5