Forecasting in contact centers: A step by step method to get an accurate forecast

DEGREE PROJECT, IN SYSTEMS ENGINEERING , SECOND LEVEL STOCKHOLM, SWEDEN 2015

Forecasting in contact centers

A STEP BY STEP METHOD TO GET AN ACCURATE FORECAST

OSKAR EKELIUS

Forecasting in contact centers

A step by step method to get an accurate forecast

O S K A R E K E L I U S

Master’s Thesis in Systems Engineering (30 ECTS credits) Master Programme in Mathematics (120 credits) Royal Institute of Technology year 2015

Supervisor at Teleopti was Maria Stein Supervisor at KTH was Per Engvist

Examiner was Per Engvist

TRITA-MAT-E 2015:47 ISRN-KTH/MAT/E--15/47--SE

Royal Institute of Technology

School of Engineering Sciences

KTH SCI

SE- 100 44 Stockholm, Sweden

URL: www.kth.se/sci

Abstract

Teleopti WFM Forecasts is a tool that can be used in order to predict future contact volumes in contact centers and stang requirements, both in the short and the long term. This tool uses his- torical data of incoming contact volumes to perform a forecast on a given forecasting period. Today this tool uses a very simple algorithm which is not always very accurate. It also requires inputs from the customer in some of the steps, in order to generate the forecast. The task of this thesis is to improve this algorithm to get a more accurate forecast that can be generated automatically, without any input from the customer. Since Teleopti has more than 730 customers in more than 70 countries worldwide [3] the most challenging part of this project has been to nd an algorithm that works for a lot of dierent historical data. Since dierent data contains dierent patterns there is not a single method that works best for all types of data. To investigate what method that is best to use for some specic data, and to perform a forecast according to this method, a step by step method was produced. A shortened version of this method is presented below.

• Remove irrelevant data that diers too much from the latest data.

• Use the autocorrelation function to nd out what seasonal variations that are present in the data.

• Estimate and remove the trend.

• Split the data, with the estimated trend removed, into two parts. Use the rst part of the data to t dierent models. Compare the dierent models with the other part of the data. The one that ts the second part best in least square sense is the one that is going to be used.

• Estimate the chosen model again, using all the data, and remove it from the full sample of data.

• Forecast the trend with Holts method.

• Combine the estimated trend with the estimated seasonal variations to perform the forecast.

There are a lot of factors that aect the accuracy of the forecast generated by using this step by step method. By analysing a lot of data and the corresponding forecasts, the following three factors seem to have most impact on the forecasting result. First of all, if the data contains a lot of randomness it is dicult to forecast it, no matter how good the forecasting methods are. Also, if there are small volumes of historical data it will aect the forecasting result in a bad way, since estimating each seasonal variation requires a certain volume of data. And nally, if the trend tends to often change direction considerably in the data it is quite dicult to forecast it, since this means that it could probably change a lot in the future as well.

This step by step method has been tested on plenty of data from a lot of dierent contact centers

in order to get it as good as possible for as many customers as possible. However, even though it has

exhibited a good forecast of these data there is no guarantee that it will perform a good forecast for

all possible data amongst Teleopti's customers. Hence, in the future, if this step by step method will

be used by Teleopti, it will probably be updated continuously in order to satisfy as many customers

as possible.

Prognostisering i kontaktcenter

En steg-för-steg-metod för en exakt och noggrann prognos

Teleopti WFM Forecast är ett verktyg som kan användas för att förutsäga framtida kontaktvoly- mer och personalbehov, både på kort och lång sikt. Detta görs genom att använda historisk data för inkommande kontaktvolymer för att utföra en prognos av en given prognostiseringsperiod. Idag använder sig detta program av en ganska enkel algoritm som ibland kan ge en ganska dålig prog- nos. Den kräver också inputs från kunden i vissa steg för att prognosen ska kunna genomföras.

Huvudmålet med detta examensarbete har varit att förbättra den nuvarande algoritmen för att få mer noggranna prognoser som kan fås fram automatiskt, utan några inputs från kunden. Eftersom Teleopti har er än 730 kunder i er än 70 länder [3], utspridda över hela världen, så har en av de mest utmanande delarna med det här projektet varit att hitta en algoritm som fungerar för många olika sorters data. För att undersöka vilken metod som är bäst lämpad för en specik datamängd, och för att utföra en prognos med denna metod, togs en steg-för-steg-metod fram. En förkortad variant av denna visas här nedan.

• Ta bort icke relevant data som skiljer sig för mycket från övrig data.

• Använd autokorrelationsfunktionen för att identiera vilka säsongsvariationer som nns.

• Beräkna trenden och ta bort den.

• Dela upp historisk data med borttagen trend i två delar. Använd den första delen till att bestämma parametrarna för olika modeller. Jämför sedan de olika modellerna, med de upp- mätta parametrarna, med den andra delen. Den model som skiljer sig minst från den andra delen av data är den modellen som kommer att användas.

• Beräkna parametrarna till den valda modellen igen, men använd nu all data.

• Prognostisera hur trenden kommer att fortsätta genom att använda Holts metod.

• Kombinera den beräknade trenden med de beräknade säsongsvariationerna för att få fram de prognostiserade värdena.

Det nns många faktorer som avgör hur exakt prognosen blir genom att använda denna steg-för- steg-metod. Genom att analysera mycket data och motsvarande prognos, har följande tre faktorer tagits fram som de som verkar ha mest påverkan på prognostiseringsresultatet. Till att börja med, om data innehåller en hög slumpfaktor är det svårt att utföra en bra prognos, oavsett vilka metoder som används. Om volymen data är väldigt begränsad påverkar det också prognostiseringsresultatet på ett negativt sätt, i och med att beräkningarna för de olika säsongsvariationerna kräver en viss mängd data. Och till sist, om trenden ofta ändrar riktning kraftigt i data är det ganska svårt att prognostisera trenden, i och med att detta betyder att den förmodligen kommer att ändra riktning ofta i framtiden också.

Den här steg-för-steg-metoden har testats på mycket olika data, tillhörande många olika kontak-

tceter, för att få den så bra som möjligt för så många kunder som möjligt. Men, även om den har

uppvisat ett bra resultat på de esta av dessa data så nns det inga garantier för att den kommer

att uppvisa bra prognostiseringar hos alla Teleoptis kunder. Därför, om Teleopti väljer att använda

denna steg-för-steg-metod i framtiden, så kommer den förmodligen att uppdateras kontinuerligt för

att tillfredsställa så många av kunderna som möjligt.

Contents

1 Introduction 7

1.1 Time series forecasting . . . . 7

1.2 The Teleopti forecast . . . . 7

2 Assumptions 9 2.1 Basic models . . . . 9

2.2 Available historical data . . . . 9

2.3 Days when the contact centre is closed . . . . 9

3 Remove irrelevant data 11 4 The trend 14 4.1 Main trend methods . . . 14

4.2 Seasonal variations in contact centers . . . 20

4.3 Trend methods for dierent cases of seasonal variations . . . 21

5 Seasonal variations 25 5.1 Month of the year variation . . . 25

5.2 Week of the year variation . . . 27

5.3 Week of the month variation . . . 28

5.4 Day of the month variation . . . 28

5.5 Day of the week variation . . . 28

6 Autocorrelation function 30 7 Choosing the best model 33 8 Bank holidays 39 9 Improve the estimates 42 10 Outliers 43 11 Irregular uctuations and ARMA-model 49 11.1 ARMA-model . . . 49

11.2 The variance of the irregular uctuations . . . 50

12 Extrapolate the trend. 52 12.1 Assuming small trend dierences . . . 52

12.2 Larger trend dierences . . . 56

13 The accuracy of the forecast 57

14 Final step by step method 59

15 Summarizing discussion 60

List of Figures

2.1 Historical data and the average number of calls calculated with and without the days when

the call center was closed. . . 10

3.1 120 weeks of historical data. . . 11

3.2 The Fourier Transform of both of the parts of the data in Figure 3.1. . . 12

3.3 The spectrogram of the data in Figure 3.1. . . 13

4.1 Two examples of historical data with an upgoing trend and a seasonal variation with time period 10 days. . . 14

4.2 Two examples of historical data and the trend, calculated by the Small Trend Method. The left one assumed a linear model and the right one assumed a non-linear model. . . 15

4.3 Two examples of historical data when the trend, calculated by the Small Trend Method, has been removed. The left one assumed a linear model and the right one assumed a non-linear model. . . 16

4.4 Two examples of historical data with the trend, estimated with the Small Trend Method, removed. The estimated seasonal variation is also present. The left one assumed a linear model and the right one assumed a non-linear model. . . 17

4.5 Two examples of historical data when trend and seasonal variations, calculated with Small Trend Method, have been removed. The left one assumed a linear model and the right one assumed a non-linear model. . . 17

4.6 Two examples of historical data and the trend, calculated with the Moving Average Method. The left one assumed a linear model and the right one assumed a non-linear model. . . 18

4.7 Two examples of historical data without the trend, calculated with The Moving Average Method. The estimated seasonal variation is also shown. The left one assumed a linear model and the right one assumed a non-linear model. . . 19

4.8 Two examples of historical data and the trend, calculated by tting a polynomial of the rst degree to the data. The left one assumed a linear model and the right one assumed a non-linear model. . . 19

4.9 Historical data and the trend calculated with the Moving Average Filter and the Small Trend Method. . . 23

5.1 Historical data with the trend removed. . . 25

5.2 Left plot: Histogram over all the days that belong to March. Right plot: Histogram over all the days that belong to May. . . 26

5.3 Historical data with the trend removed, and the estimated month of the year variation. . 26

5.4 Historical data with the trend and the estimated month of the year variation removed. . . 27

5.5 Historical data with the trend removed, and the estimated week of the year variation. . . 27

5.6 Historical data with the trend and the estimated week of the year variation removed. . . . 28

6.1 The left plot shows some historical data and the right plot the corresponding autocorre- lation function. . . 30

6.2 The left plot shows some historical data and the right plot the corresponding autocorre- lation function. . . 31

6.3 The left plot shows the autocorrelation function for some historical data containing both a monthly and a weekly variation. The right plot shows the autocorrelation function for the same data but with a Moving Average Filter with d = 7. . . 31

6.4 The autocorrelation function for some data, rst the original data and then with a Moving Average Filter with d = 7 and then d = 30 applied. . . 32

7.1 Historical data with the trend removed, using Moving Average Method and linear model. The left part is the one used to estimate the indexes in the models and the right part is the one that is used to test the models. . . 34

7.2 The test set compared with two dierent models, Model 2.1 and 2.2. The left plot is the linear case and the right plot is the non-linear case . . . 35

7.3 The original data in the test set compared with two dierent models, the linear and the

non-linear model. . . 35

7.4 The errors calculated by taking the real values minus the values according to the model.

The linear model is shown in the left plot and the non-linear in the right plot. . . 36 7.5 The absolute errors calculated by taking the real values minus the values according to the

model. The linear model is shown in the left plot and the non-linear in the right plot. . . 36 7.6 Two dierent models compared with the correct values. . . 37 8.1 Historical data with estimated trend and seasonal variations removed. The estimate part

and the test part is marked out. . . 39 8.2 Historical data with estimated trend and seasonal variations removed. The bank holiday,

estimated from the estimate part is also shown. . . 40 8.3 Historical data with estimated trend and seasonal variations removed. The bank holiday,

estimated from both the parts is also shown. . . 40 8.4 Historical data with estimated trend and seasonal variations removed and bank holidays

removed. . . 41 9.1 Historical data with estimated trend estimated in the rst and the second iteration. . . . 42 10.1 Some historical data. . . 44 10.2 Some historical data with the estimated seasonal variations and the bank holidays removed. 44 10.3 Some historical data with the estimated seasonal variations and the bank holidays re-

moved. The Moving Average is also shown and some upper and lower limit in order to classify outliers. . . 45 10.4 Some historical data with the estimated seasonal variations and the bank holidays re-

moved. The Moving Average is also shown and some upper and lower limit. The upper and lower limits are calculated with dierent p:s. The upper plot with a large p and the lower plot with a small p. . . 46 10.5 Some historical data with the estimated seasonal variations and the bank holidays re-

moved. The Moving Average is also shown and some upper and lower limit. The Moving Average is calculated with dierent q:s. The upper plot with a small q and the lower plot with a large q. . . 47 10.6 The histogram of some historical data with the estimated trend, seasonal variations and

the bank holidays removed. 3 times the standard deviation of these values are also shown. 48 11.1 Some historical data with the estimated trend, seasonal variations and bank holidays

removed. Dierent ARMA-models have been used to do a forecast. . . 49 11.2 The same as in Figure 11.1 but zoomed in on the forecasts. . . 50 11.3 An example of what the distribution of the irregular uctuations may look like. . . 51 12.1 Left plot: Some historical data and the forecast period. Right plot: Same historical data

as in the right plot but with the seasonal variations and bank holidays removed. . . 53 12.2 Piecewise trend estimates and Holts extrapolation of the trend with α = 0.5 and β = 0.5. 53 12.3 Holts forecast with α and β that dier between 0.3 and 0.9. . . 54 12.4 The sum of the squares of the errors for dierent combinations of α and β in Holts method. 55 12.5 The Holts Method with the best combination of α and β in least square sense is highlighted. 56

List of Tables

4.1 7 cases of dierent combinations of seasonal variations. . . 21

1 Introduction

1.1 Time series forecasting

Good time series forecasts are important in a lot of areas such as economic planning, sales forecasting, inventory control and production and capacity planning [7]. When dealing with time series forecasts, the target is to forecast the future value x N +h (or values x N +h , x N +h+1 ... ), given an observed time series x 1 , x 2 , ..., x N . There are a lot of methods that can be used to forecast this value. These methods may be broadly classied into three types [9]:

1. Judgemental forecasts based on subjective judgement, intuition, inside commercial knowledge, and any other relevant information.

2. Univariate methods where forecasts depend only on present and past values of the single series being forecasted.

3. Multivariate methods where forecasts of a given variable depend, at least partly, on values of one or more additional time series variables.

Another way to classify these methods could be [7]: automatic methods, which does not require any human input, and non-automatic methods which requires some human input. For example, in inventory control there may be a lot of items to monitor. So instead of tting separable models to each individual time series of sale, an automatic method could be used for the whole range of items. The non-automatic methods are more often used in economic planning which requires that the analyst carefully builds appropriate model describing the relationship between relevant economic variables, after which forecasts can be produced from the model. In this report, automatic univariate methods will be used. The reason for these choices will be explained in the end of Section 1.2.

Forecasting is sometimes confused with planning [8]. These are two dierent things even though they are related to each other. The forecast is a prediction of how the future will look like, not how it should look like. The forecast can later be used as an input to the planning model. A forecasting model can be used to nd out what the future will look like if not doing any changes, while the planning can be used in order to try to change the future.

When performing a forecast it is important to have some knowledges of how the forecast will be used.

Sometimes it is used as a basis for planning and sometimes it is used as a target value. This is important since dierent uses have dierent requirement of the forecast. In some cases it is much worse that the forecasted value is too large than too low and vice versa.

Time series forecasting is mostly a form of extrapolation since it involves tting a model to a set of data and then use the model outside the range of data to perform the forecast. Hence, it depends on the assumption that the future should be as the past. So, when performing the forecast it is important for the forecaster to consider if he/she believes in this assumption or not.

1.2 The Teleopti forecast

Teleopti is a provider of solutions for strategic optimization. One part of Teleopti is Teleopti WFM which

is a solution for strategic Workforce Management that optimizes contact center, back-oce, - as well as

branch oces and stores. One important part in Teleopti WFM is the Teleopti WFM Forecasts which

is a tool that can be used to predict future contact volumes and stang requirements, both in the short

and the long term. This forecast will later be used as a basis for nding an optimal schedule for the

sta in the contact center, in order to satisfy some service requirement without getting overstaed. The

service requirement could for example be: Answer p% of the incoming calls within t seconds. Teleopti

WFM Forecasts uses historical data to do a forecast on a given forecasting period. The forecasting

period usually starts at least two weeks ahead in the future and the length of the forecasting period

is often one or two months. However, sometimes the customers also want to do a long term forecast

over a full year. Teleopti has more than 730 customers in more than 70 countries worldwide [3] which

means that this forecasting tool needs to use a method that works for a lot of dierent companies in dierent countries. The algorithm that Teleopi uses today is quite simple and is not always very accu- rate. It also involves the customer in some of the steps. For example, the customer needs to select what parts of the data that should be used in the forecast and what parts that should be left out. The cus- tomer also needs to remove outliers and decide how he/she thinks that a trend will look like in the future.

The task of this master's thesis is to improve this forecasting method with better mathematical methods.

Dierent customers want to use the forecasting tool in dierent ways. One group of customers have quite limited knowledge about time series analysis and forecasting and just want the forecast to run in the background without getting involved in it. Another group of customers have analysts that want to get involved in every step in the forecasting procedure in order to get the forecast as accurate as possible.

And of course, there are a lot of costumers in between these groups. The objective is that in the future, there should be a possibility to run the forecast automatically, without any input from the customer.

Below, some improvement areas are listed, in order to satisfy a better forecast that could be generated automatically:

• Sort out irrelevant data automatically without involving the customer.

• Remove outliers automatically.

• Use appropriate trend methods to estimate the trend.

• Identify what seasonal variations are present in the data.

• Find better ways to estimate the seasonal variations without getting aected by the trend, other seasonal variations or bank holidays.

• Use special models when the volume of the historical data is limited.

• Estimate the eects of the bank holidays and use it to forecast upcoming holidays.

• Find a way to measure how accurate the forecasting method is.

All of the above improvement areas will be handled in this report. How to sort out data will be covered in Section 3 and how to remove outliers in Section 7. Dierent trend methods will be presented in Section 4 and 12 and identifying seasonal variations will be covered in Section 6 and 7. Estimation of the seasonal variations will be presented in Section 5 and bank holidays in 8. The accuracy of the forecast will be covered in Section 13. In Section 14, a nal step by step method, that automatically will perform a forecast given any historical data, will be presented.

In forecasting, the human ability to detect patterns, deviations and discontinuities in a time plot is usually of great importance. The human eye is also a great tool to evaluate if a forecast seems reasonable or not.

Since one of the objectives with the new forecasting tool is to be able to run the forecast automatically, without any input from the customer, the human ability to detect these things will be lost. Hence, this complicates the forecasting process and places great demands on the models and methods that are used to perform the forecast. For the reader, it is important to have in mind that all of the methods that will be described are produced such that it will be possible to include them in an automatic way. The

nal step by step method will only have two inputs, namely the historical data and a forecasting period.

With only these two inputs, the goal is that the step by step method should perform an accurate forecast

automatically.

2 Assumptions

2.1 Basic models

To be able to do a forecast the following assumptions were made. Either the number of calls X t of a certain day t can be expressed as a linear model

X _t = m _t + s _t + Y _t (2.1)

or as a non-linear model

X _t = m _t · s t · (1 + Y t ), (2.2)

where m t is the trend, which loosely can be dened as long-term change in the mean level (see Section 4). s t is the seasonal variation (or a combination of seasonal variations) which is when similar patterns of behaviour are observed at particular times of the period (see Section 5). In the linear case the assumption

∑ d

j=1 s j = 0 is made and in the non-linear case the assumption ∏ d

j=1 s j = 1 is made, where d is the period for the seasonal variation. Y t is an irregular uctuation with the assumption E(Y t ) = 0 , which in some case can be completely random, in which case it cannot be forecasted. However it might have some short-term correlation which will be discussed in Section 11. When doing the forecast, it is assumed that the future values can be expressed by the same basic model as the historical data. Hence, it is assumed that the future will approximately follow the same pattern as the past. If the forecaster doesn't believe in this assumption and believes that the past historical data has nothing in common with the upcoming ones, then the following models and methods described in this report will be useless. However, this assumption doesn't mean that all the historical data have to follow the same pattern as the future. This will be discussed more in Section 3 and 10. All of the methods that are presented in this report have been analysed both for the linear and the non-linear model. But in some of the examples and gures in this report, only the linear model is included. In these cases, the reasoning and results are very similar in the non-linear case wherefore they have been left out.

2.2 Available historical data

The volume of historical data that is available diers a lot between dierent contact centers. Some of them have several years of data while others only have a few weeks. A lot of the methods that will be described in this report require a certain volume of data. If the given volume of historical data is not enough to perform the method, alternative methods have to be used. This will be explained more in detail in Section 5. In most cases, the forecast will be better the more data available, given that the data doesn't change markedly through the historical period (see Section 3). For example, in the cases when only a few weeks are available it is impossible to know if there will be some yearly variation in the future or not. This is for example a problem if the forecaster wants to do a forecast for the upcoming month, since he has no idea of how the same month last year looked like. More data will also improve the estimates of the seasonal variations since it decreases the risk that the randomness in the data aects the estimates. Hence, larger volumes of historical data will usually give a more accurate forecast.

2.3 Days when the contact centre is closed

In a lot of historical data in contact centers the data is equal to zero for some of the days. For example,

a common case is that it is equal to zero every Saturday and Sunday. This is due to the simple reason

that it is closed these days. Hence, when dealing with the data, the forecaster needs to decide if these

days should be included or not, when estimating trends and seasonal variations. In the examples in this

report those days will not be included. This since they don't give any relevant information about the future. If a call center is closed one day, then it is obvious that there will be zero calls that day. It is assumed that these days can not be described by the basic models that were presented in equation (2.1) and (2.2) and therefore they should not be included when estimating trends and seasonal variations.

Concluding them might give some misleading information. For example, consider the historical data in Figure 2.1 and the two dierent measures of the average number of calls per day. One calculated using all the days and one calculated by only considering Monday to Friday every week, i.e. the ones when the call center was open.

Day

Sat Sun Mon Tue Wed Thu Fri Sat Sun Mon Tue Wed Thu

Calls

0 500 1000 1500 2000 2500 3000 3500 4000

Historical data and the average number of calls

Historical data Mean using all days Mean using open days

Figure 2.1: Historical data and the average number of calls calculated with and without the days when the call center was closed.

In Figure 2.1 it can be seen that the one that includes the zeros doesn't seem like a good measurement.

This since, for all of the days when the call center has been open, the number of calls have been larger than the calculated average number of calls, which indicates that this measure is not very good. When not concluding the zeros on the other hand, half of the number of calls when it was opened are above the average and half of them are below the average, which seems more reasonable. With this in mind, only the days when the contact center has been open will be included when calculating the trends, seasonal variations etc. However, if a contact center has very low volumes in the historical data, with just a few calls per day and sometimes even zero calls per day, the zeros should not be removed. This since the zeros give some relevant information in this case.

Another question in the same category is: Should all historical data points, no matter how old they are,

be used when doing the forecast? The answer to this question depends on if there have been some great

changes in the data during the historical period. What's meant by great changes will be discussed in

Section 3.

3 Remove irrelevant data

An important part when performing a forecast is to use relevant historical data. Sometimes the historical data changes signicantly during the historical period, which means that it is not very wise to assume that all of the data points can be expressed by the same model. In cases like this, it is important to remove the parts that dier too much from the rest of the data. There are mainly two things that can change in data. Either the seasonal variations may have diered. Maybe some weekly variation was present in the rst part but not in the last part. Then it would be foolish to assume that both parts can be expressed by the same linear (2.1) or non-linear (2.2) model. There could also have been some drastic changes in the mean level of the volumes between the rst part and the last part. Then it would be preferable to deal with the rst part separately when estimating the trend and the seasonal variations.

As an example, consider the data in Figure 3.1.

Weeks

0 20 40 60 80 100 120

Calls

0 2000 4000 6000 8000 10000

12000 Historical data

Figure 3.1: 120 weeks of historical data.

By looking at the rst 60 weeks in Figure 3.1 it is clear that these look completely dierent from the

last 60 weeks. They dier signicantly in the mean level and if looking carefully, it also seems like they

have dierent patterns. To investigate if there has been some markedly change in the data the Fourier

Transform may be used. Performing the Fourier Transform on the two parts of the data in Figure 3.1

gives the result in Figure 3.2.

0 0.1 0.2 0.3 0.4 0.5

|x(f)|

0 5000 10000 15000

Fourier Transform of the second part of the historical data

Frequency (Occurrences per day)

0 0.1 0.2 0.3 0.4 0.5

|x(f)|

0 5000 10000 15000

Fourier Transform of the first part of the historical data

Figure 3.2: The Fourier Transform of both of the parts of the data in Figure 3.1.

As can be seen in Figure 3.2 the mean value of the rst part is much lower than the other part. It can also be seen that the rst part contains some pattern that repeats itself 0.14 times a day ≈ once a week, i.e. contains a weekly variation, but the second part does not contain this pattern. Hence, the Fourier Transform may be used both to determine changes in the mean level and changes in seasonal variations.

However, the change in the data may appear at any time, not just in the middle of the data as in this

case. To determine if the data changes markedly and at what time it changes, the spectrogram may be

used. The spectrogram is a graph that shows how the frequency and the intensity changes over time

in a time series. This is produced by rst dividing the data into smaller parts. In most cases these

parts overlaps each other. Then the Fourier Transform of each of the parts is performed in order to

investigate the frequencies and intensities in each time period. The spectrogram of the data in Figure

3.1 is illustrated in Figure 3.3.

Frequency (Occurrences per day)

0 0.1 0.2 0.3 0.4 0.5

Weeks

100 200 300 400 500 600 700

Spectogram of the historical data

20 30 40 50 60 70 80 90

Figure 3.3: The spectrogram of the data in Figure 3.1.

In Figure 3.3, changes somewhere between week 55 and 65 can be seen. Both in the mean level and in

the seasonal variation. However, since the dierent parts overlap each other it is dicult to get an exact

week where the data changes. Hence, the spectrogram only gives some approximate information about

where the data changes. When the program has identied a great change in the data, the next thing is

to decide which part of the data that should be used. The oldest part or the newest part. In most cases

the newest part will be the most relevant one since it lies closest to the forecasting period. Therefore, if

no other information is given, this one will be used for the forecast. However, in some cases the oldest

part is the most relevant one. In these cases the newest part was just a temporal change and the old

one is therefore more relevant. But, if just the historical data is given, without any more information, it

is impossible to know which one of the mentioned cases that is, and therefore the newest part is always

used. Another important question is: How much should the data change, in order to be classied as

a "great" change. It is important to not remove data just because the mean level is just a little bit

lower in some old data. This might be due to trends and seasonal variations and by removing this, some

relevant information will be lost. Hence, if comparing the intensities at each frequency for the dierent

time intervals, a big change should be required in order to classify it as a great change in data. Finding a

specic number that species a limit where the data can be classied as a great change is quite dicult,

but by investigating a lot of data, the following has been set as a benchmark: If each time interval is

100 days, the intensities should change at least 100% between two intervals in order to be classied as a

great change.

4 The trend

The trend can loosely be dened as long-term change in the mean level and should not be confused with the seasonal variations. For example, if the volumes increase for the months June and July compared to the months April and May for an ice cream company it would be questioned to conclude that this is due to an upgoing trend. This is rather due to the fact that people usually eat more ice creams in the summer because of the warmer weather. This is an example of a seasonal variation. However, if the volumes increases from one year to another, then it would be reasonable to conclude that there is an upgoing trend. In some cases, it is quite tricky to estimate the trend, without letting the seasonal variations aect it. However, there are some well known methods. These will be illustrated with a simple example.

4.1 Main trend methods

Day

0 20 40 60 80

Calls

980 1000 1020 1040 1060 1080

1100 Calls per day for a 80 day period

Day

0 20 40 60 80

Calls

0 2000 4000 6000 8000 10000

12000 Calls per day for a 80 day period

Figure 4.1: Two examples of historical data with an upgoing trend and a seasonal variation with time period 10 days.

Example 4.1. Consider the number of calls each day (x 1 , ..., x 80 ) illustrated in Figure 4.1. In both cases, the data seems to contain an upgoing trend and some seasonal variation that repeats itself with the seasonal period d = 10 day. This pattern repeats itself N = 8 times during the time series, hence it contains 8 seasonal periods. To be able to nd the trend there are some well-known methods [1] which will be explained here. These will look a little bit dierent, depending on if the model is assumed to be linear or not.

1. The Small Trend Method. If the trend is small it can be assumed that the trend is constant m _j over the seasonal period j. If renaming the calls each day (x 1 , ..., x ₈₀ ) with

(x 1,1 , ..., x _1,10 , x _2,1 ..., x _8,1 , ..., x _8,10 ) instead the estimates can be expressed as

ˆ m j = 1

d

∑ d k=1

x j,k .

Since E(y j,k ) = 0 and ∑ d

k=1 s k = 0 one gets

E( ˆ m _j ) = E ( 1

d

∑ d k=1

x _j,k )

= E ( 1

d

∑ d k=1

m j + s k + y j,k

)

= E(m j ) + E ( 1

d

∑ d k=1

s k

) + E

( 1 d

∑ d k=1

y j,k

)

= E(m j )

which means that this estimate is unbiased. The non-linear case gives the estimate

ˆ

m j = exp( 1 d

∑ d k=1

log(x j,k )).

If assuming E(y j,k ) = 0 and ∏ d

k=1 s k = 1 ⇒ ∑ d

k=1 log(s k ) = 0 one gets

E( ˆ m j ) = E [

exp ( 1

d

∑ d k=1

log(x j,k ) )]

= E [

exp ( 1

d

∑ d k=1

log(m _j · s k · y j,k ) )]

= E [

exp ( 1

d

∑ d k=1

log(m _j ) + 1 d

∑ d k=1

log(s _k ) + 1 d

∑ d k=1

log(1 + y _j,k ) )]

= E[exp(log(m _j )] · E [

exp ( 1

d

∑ d k=1

log(s _k ) )]

· E [

exp ( 1

d

∑ d k=1

log((1 + y _j,k ) )]

= E(m _j ).

If looking at the plots in Figure 4.1 it seems that in the right one, the amplitudes of the seasonal variation increase when the trend increases, but in the left one they remain the same. This means that it is reasonable to assume that the left one is best described by a linear model and the right one with a non-linear model. Applying this trend method to the data in the plots gives the result in Figure 4.2.

Day

0 20 40 60 80

Calls

980 1000 1020 1040 1060 1080

1100 Calls per day and the trend

Historical data Trend

Day

0 20 40 60 80

Calls

0 2000 4000 6000 8000 10000

12000 Calls per day and the trend

Historical data Trend

Figure 4.2: Two examples of historical data and the trend, calculated by the Small Trend Method. The

left one assumed a linear model and the right one assumed a non-linear model.

Now, when an estimate of the trend, ˆ m t , is found, one wants to remove it from the historical data to be able to estimate the seasonal variations. Lets call the data with the trend removed (ˆ x _1,1 ..., ˆ x _8,10 ) . In the linear case it is generated by subtracting the trend from the historical data and in the non-linear case it is generated by dividing the historical data by the trend i.e.

ˆ

x ^lin. _j,k = x j,k − ˆ m j = m j − ˆ m j + s k + y j,k

ˆ

x ^non.lin. _j,k = x _j,k ˆ m j

= m _j ˆ

mj · s k · (1 + y j,k ).

If the estimated trend is close to the real trend, then this new sample will only contain a seasonal variation and some noise. Figure 4.3 illustrates this new sample.

Day

0 20 40 60 80

-30 -20 -10 0 10 20

30 Historical data without the trend

Day

0 20 40 60 80

0 0.5 1 1.5 2

2.5 Historical data without the trend

Figure 4.3: Two examples of historical data when the trend, calculated by the Small Trend Method, has been removed. The left one assumed a linear model and the right one assumed a non-linear model.

Now the seasonal variation can be estimated using

ˆ s k = 1

N

∑ N j=1

ˆ x ^lin. _j,k

in the linear case and in the non-linear case

ˆ s _k = exp

[ 1 N

∑ N j=1

log ( ˆ

x ^non.lin. _j,k )]

.

The result when applying this to the data without trend in Figure 4.3 is illustrated in Figure 4.4.

Days

0 20 40 60 80

-30 -20 -10 0 10 20 30

40 Seasonal variation

Hist. data without trend Seasonal variation

Day

0 20 40 60 80

0 0.5 1 1.5 2 2.5

3 Seasonal variation

Hist. data without trend Seasonal variation

Figure 4.4: Two examples of historical data with the trend, estimated with the Small Trend Method, removed. The estimated seasonal variation is also present. The left one assumed a linear model and the

right one assumed a non-linear model.

Finally the estimated seasonal variations are also removed in the same way as the trend. What's left is illustrated in Figure 4.5.

Day

0 20 40 60 80

-10 -5 0 5

10 Hist. data without trend and seas.

Day

0 20 40 60 80

0.8 0.9 1 1.1 1.2

1.3 Hist. data without trend and seas.

Figure 4.5: Two examples of historical data when trend and seasonal variations, calculated with Small Trend Method, have been removed. The left one assumed a linear model and the right one assumed a

non-linear model.

What is left in Figure 4.5 is, if the correct trend and seasonal variations have been found, the irregular uctuations. The analyse of these will be handled in Section 11.

2. Moving average estimation. This method is often preferable to the Small Trend Method since it does not assume that the trend is constant over each cycle. If the period is even, say d = 2q, then

ˆ

m _t = 0.5x _{t −q} + x _{t −q+1} + ... + x _{t+q −1} + 0.5x _t+q d

q < t ≤ n − q

is used to estimate the trend, and if the period is odd, say d = 2q + 1 then

ˆ

m t = x t −q + x t −q+1 + ... + x t+q −1 + x t+q

d

q < t ≤ n − q

is used. If a non-linear model is assumed, the trend will instead be calculated as

ˆ

m t = exp

( 0.5log(x t −q ) + log(x t −q+1 ) + ... + log(x t+q −1 ) + 0.5log(x t+q ) d

)

q < t ≤ n − q or

ˆ

m t = exp

( log(x t −q ) + log(x t −q+1 ) + ... + log(x t+q −1 ) + log(x t+q ) d

)

q < t ≤ n − q Using this on the data in Figure 4.1 gives the result in Figure 4.6.

Days

0 20 40 60 80

Calls

980 1000 1020 1040 1060 1080

1100 Historical data and the trend

Historical data Trend

Days

0 20 40 60 80

Calls

0 2000 4000 6000 8000 10000

12000 Historical data and the trend

Historical data Trend

Figure 4.6: Two examples of historical data and the trend, calculated with the Moving Average Method. The left one assumed a linear model and the right one assumed a non-linear model.

From Figure 4.6 it can be seen that a disadvantage with the Moving Average Trend Method is that it gives no information about the trend for t ≤ d/2 or t > t n − d/2. This will be a problem when the fraction d/n is large since then a great part of the data will be removed. The next step is to compute the average w k of the deviations {(x k+jd − ˆ m _k+jd ) : q < k + jd ≤ n − q}. Since these average deviations do not necessarily sum up to zeros the seasonal components, s k will be calculated as

ˆ

s k = w k −

∑ d i=1 w _i

d , k = 1, ..., d.

ˆ s k = exp

(

log(w k ) − log (∑ d

i=1 w i

d ))

, k = 1, ..., d.

Doing this on the given data gives the result in Figure 4.7.

Days

0 20 40 60 80

-30 -20 -10 0 10 20 30

40 Seasonal variation

Hist. data without trend Seasonal variation

Days

0 20 40 60 80

0 0.5 1 1.5 2 2.5

3 Seasonal variation

Hist. data without trend Seasonal variation

Figure 4.7: Two examples of historical data without the trend, calculated with The Moving Average Method. The estimated seasonal variation is also shown. The left one assumed a linear model and the

right one assumed a non-linear model.

Notice in Figure 4.7 that the data without trend starts at day 6 and continues to day 75 since the Moving Average Trend Method does not give any information about the trend the other days. The data with removed trend and seasonal variations will look similar to Figure 4.5 and is therefore not shown here.

3. Fit a polynomial. In this case a polynomial of some degree d is tted in least square sense to the data in the linear case and to the logarithm of the data in the non-linear case. In the non-linear case, the trend will be estimated as e to the power of the estimated polynomial. For the data in the plots in Figure 4.1 a linear polynomial seems reasonable.

Day

0 20 40 60 80

Calls

980 1000 1020 1040 1060 1080 1100

Calls per day and the trend

Historical data Trend

Day

0 20 40 60 80

Calls

0 2000 4000 6000 8000 10000 12000

Calls per day and the trend

Historical data Trend

Figure 4.8: Two examples of historical data and the trend, calculated by tting a polynomial of the rst

degree to the data. The left one assumed a linear model and the right one assumed a non-linear model.

If looking at the right plot in Figure 4.8, the trend doesn't seem fully linear in this case. This is due to the fact that the line was tted to the logarithm of the data and not the data. The calculations of the seasonal indexes will then be done in the same way as for the Moving Average Method. Even in this case, the results will look pretty similar and are therefore not shown here.

4. Dierencing with lag. If using this method ∇ d is dened as

∇ d X t = X t − X t −d

in the linear case and

∇ d X t = X _t X t −d

in the non-linear case. Applying this to the linear model (2.1) and the non-linear (2.2) gives

∇ d X t = m t − m t −d + Y t − Y t −d

in the linear case and

∇ d X _t = m t

m _{t −d}

1 + Y t

1 + Y _{t −d}

in the non-linear case. Now the trend component m t − m t −d and _m ^m

can be eliminated using any of the methods already described. The dierence is that now the seasonal variation will not aect the trend. Therefore, if using Small Trend Method or Moving Average Method, d can take other values than 10. The next steps will look similar to the other methods and are therefore not shown here.

4.2 Seasonal variations in contact centers

Example 4.1 is a very simplied example of some data containing a trend and one seasonal variation. In contact centers historical data often have more than one seasonal variation. Hence, in the models (2.1) and (2.2) s t may include combinations of more than one seasonal variation. These are the ones that might appear.

1. Weekly variation. If a weekly variation exists, a similar pattern repeats itself with a seasonal period of d = 7 days. For example, the average number of calls on Mondays might be higher than the average number of calls on Saturdays. This variation, s ^(D,W) = (s ^{(D,W )} ₁ , ..., s ^{(D,W )} ₇ ) ^⊤ , will further be referred as the 'day of the week variation' and will satisfy ∑ 7

k=1 s ^{(D,W )} _k = 0

2. Monthly variation. If a monthly variation exists, a similar pattern repeats itself every month i.e. the seasonal period d diers between 28, 29, 30 and 31 days. The seasonal variation for a month containing d days can be represented by either a 'day of the month variation' s ^(D,M,d) = (

s ^(D,M,d) ₁ , ..., s ^(D,M,d) _d ) _⊤

, or a 'week of the month variation' s ^(W,M,d) = (

s ^(W,M,d) ₁ , ..., s ^(W,M,d) ₅ ) _⊤

.

The dierence between these ones is that the day of the month variation will contain a seasonal

index for each day of the month but the week of the month variation will contain one index for

each week of the month. What is meant by the week of the month can be interpreted in two ways.

rst week refers to the 1st day of the month and continues until the rst Sunday appears, then the second week will begin the upcoming Monday etc. However, in this report only the rst version of the week of the month variation will be considered. Since d is dierent for dierent months the seasonal variation for a month containing 30 days will not be exactly the same as for the ones with 31 days. If using the day of the month variation the assumption will be ∑ d

k=1 s ^(D,M,d) _k = 0 and if using week of the month variation the assumption will be 7 · ∑ 4

k=1 s ^(W,M,d) _k + (d − 28) · s 5 = 0 . I.e, if summing up data for a full month, the monthly variations will not aect the result, no matter which month it is.

3. Yearly variation. If a yearly variation exists, a similar pattern repeats itself every year, i.e. the seasonal period d is 365 or 366 days. This can be represented by either 'month of the year variation' s ^(m,y) =

(

s ^(m,y) ₁ , ..., s ^(m,y) ₁₂ ) _⊤

or a 'week of the year variation' s ^(w,y) = (

s ^(w,y) ₁ , ..., s ^(w,y) ₅₂ ) _⊤

. Similar to the week of the month, the week of the year can be interpreted in two ways. Either the rst week contains the 1st to the 7th of January, second week the 8th to the 14th of January,..., fth week the 29th of January to the 4th of February etc. Or the rst week refers to the 1st of January and continues until the rst Sunday appears, then the second week will begin the upcoming Monday.

However, in this case only the rst version of week of the year will be considered. When using this one, there will be 52 full weeks and one week, the 53rd, that only will contain one day in the ordinary years and two days in leap years. Since the 53rd week is not a full week, the assumption that this week is very close to the 52nd week will be made. Hence, the days in the 53rd week will have the same seasonal component as the ones in the 52nd, i.e. s ^(w,y) 53 = s ^(w,y) ₅₂ . In this case the assumptions will be 28s ^{(M,Y )} 2 + 30

(

s ^{(M,Y )} ₄ + s ^{(M,Y )} ₆ + s ^{(M,Y )} ₉ + s ^{(M,Y )} ₁₁ )

+ 31 (

s ^{(M,Y )} ₁ + s ^{(M,Y )} ₃ + s ^{(M,Y )} ₅ + s ^{(M,Y )} ₇ + s ^{(M,Y )} ₈ + s ^{(M,Y )} ₁₀ + s ^{(M,Y )} ₁₂

)

= 0 and 7 ∑ 51

k=1 s k + 8s 52 for a normal year, i.e., when summing up a full year, the yearly variation will not aect the result. Since a leap year only diers with one day, there will be the same assumptions for leap years.

4.3 Trend methods for dierent cases of seasonal variations

The method that is used to calculate the trend should not be aected by any of the seasonal variations that may appear in the historical data. Therefore, the best way to estimate the trend may dier a bit, depending on what seasonal variations are present. Here, seven dierent cases may appear.

Case 1 Only weekly variation is present Case 2 Only monthly variation is present Case 3 Only yearly variation is present

Case 4 Monthly and weekly variations are present Case 5 Yearly and weekly variations are present Case 6 Yearly and monthly variations are present

Case 7 Yearly, monthly and weekly variations are present Table 4.1: 7 cases of dierent combinations of seasonal variations.

For almost all of the dierent cases the Moving Average Method is used. This since it is preferable to

the Small Trend Method, since the latter one assumes that the trend is nearly constant over the seasonal

period d which is not always the case. If choosing to t a polynomial, the degree of the polynomial must

be decided before. Since dierent data might have dierent types of trends, this doesn't seem to be a

good method either. Using the "Dierencing at Lag d Method" is not preferable either since d changes

in a lot of the cases which means that ∇ d X t will contain the dierences of trends with dierent lags.

Therefore, the best trend method to use seems to be the Moving Average Method. However, the period d will dier a bit for each of the dierent cases which will be explained below.

• Case 1 Weekly variation. This is the most simple case since it only contains one seasonal variation with constant period d = 7 and q = (d − 1)/2 = 3. This is comparable with Example 4.1 and the trend estimates become

ˆ

m t = x _{t −q} + x _{t −q+1} + ... + x _{t+q −1} + x _t+q d

q < t ≤ n − q in the linear case and

ˆ

m _t = exp

( log(x _{t −q} ) + log(x _{t −q+1} ) + ... + log(x _{t+q −1} ) + log(x _t+q ) d

)

q < t ≤ n − q

in the non-linear case. This method requires at least 13 days of data since at least 7 days will be needed to calculate the seasonal variation in the next step.

• Case 2 Monthly variation. This case is a little bit trickier since the period diers between d = 28, d = 29 , d = 30 and d = 31. This since dierent months contain dierent numbers of days and February contains dierent numbers of days depending on if it is a leap year or not. This means that, depending on what year and month the day t belongs to when estimating m t , dierent d:s will be used. In this case it is not obvious if the Moving Average Method is better than the Small Trend Method. If, for example, the day t is the 19th of May 2015 the Moving Average would be

ˆ

m t = x M ay,4 + x M ay,5 + ... + x J une,2 + x J une,3

31 (4.1)

in the linear case and

ˆ

m _t = exp

( log(x M ay,4 ) + log(x M ay,5 ) + ... + log(x J une,2 ) + log(x J une,3 ) 31

)

(4.2)

in the non-linear case. This means that the average of this day will contain the indexes (s ^(D,M,31) ₄ , ..., s ^(D,M,31) ₃₁ , s ^(D,M,30) ₁ , ..., s ^(D,M,30) ₃ ) . In the previous section the assumptions ∑ 30

i=1 s ^(D,M,30) _i = 0 and ∑ 31

i=1 s ^(D,M,31) _i = 0 were made in the linear case. This means that ∑ 3

i=1 s ^(D,M,30) _i +

∑ 31

i=4 s ^(D,M,31) _i is probably not equal to zero even though it might be close. The same yields for the non-linear case and for the indexes (s ^{(W,M )} 1 , ..., s ^{(W,M )} ₅ ) if a week of the month variation exists. This means that the seasonal variation will aect the trend a little bit. If instead using the Small Trend Method the estimate for the day t would be

ˆ

m _t = x M ay,1 + x M ay,2 + ... + x M ay,30 + x M ay,31

31 (4.3)

in the linear case and

( )

in the non-linear case. This means that the average of this day will contain the indexes (s ^(D,M,31) ₁ , ..., s ^(D,M,31) ₃₁ ) and since ∑ 31

i=1 s ^(D,M,31) _i = 0 the monthly variation will not aect the trend. An example of the resulting trends when using these two methods on some historical data with a monthly variation, are shown in Figure 4.9 in the linear case. The non-linear case will look very similar and is therefore not shown here.

Date

2010-Jan 2010-Apr 2010-Jul 2010-Oct 2011-Jan 2011-Apr

Calls

×10 ⁹

0 0.5 1 1.5

2 Historical data and the trend

Historical data Moving ave.

Small trend.

Figure 4.9: Historical data and the trend calculated with the Moving Average Filter and the Small Trend Method.

As seen in Figure 4.9 and as mentioned before, the Small Trend Method assumes that the trend is nearly constant over the whole month which is not always the case. Hence, it is not obvious which one of these methods that is the best. However, recall the arguments in Section 2.3 for not using the days when the call center was closed. Even though it doesn't seem to be any closed days in the particular data in Figure 4.9, it is quite common in other data. This means that only the days when the call center was open should be included in the equations (4.1)-(4.4) and d = 31 should be modied as the total number of opened days of the period and not all the days in the period. This means that, it is not always the case that all of the indexes (s ^(d,m,31) 1 , ..., s ^(d,m,31) ₃₁ ) will be contained when calculating the trend for May and therefore the monthly variation may aect the trend anyway. Hence, since the monthly variation may aect the trend even when using Small Trend Method it doesn't seem preferable to use this instead of the Moving Average Method.

Therefore, the Moving Average will be used further on when data only has a monthly variation.

• Case 3 Yearly variation. In this case, d will dier between 365 and 366 depending on if it is a leap year or not. However, since the number of days only dier with (366 − 365)/365 ≈ 0.3%, the approximation that (

s (m,y,ord.year)

1 , ..., s (m,y,ord.year) 12

) ≈ (

s (m,y,leap year)

1 , ..., s (m,y,leap year) 12

) is made and similar for the week of the year variation. In this case, when the seasonal time period is one year, the assumption that the trend is almost constant over the year doesn't seem very realistic.

Therefore the Small Trend Method is not used. Instead the Moving Average Method with d = 365 for ordinary years and d = 366 for leap years is used. Since the yearly variation doesn't aect the result when summing up a full year, it will not aect the trend.

• Case 4 Monthly and weekly variation. This one is trickier than the ones before since it contains

two dierent seasonal variations that overlap each other. Here there are some dierent ways to do

it. Either use the Moving Average Method with d = 28. This will contain 4 full weeks which means

that the weekly variation will not aect the trend. However, the monthly variation will aect it

a bit, since 28 days is in most cases not a full month. If instead using any of the two methods

described in Case 2, the weekly variation will aect the trend. This since the trend will almost

always be calculated using 29, 30 or 31 historical data points, which are not full weeks. A fourth way of doing it, would be to use the Small Trend Method but only consider the 28 rst days in every month. In this case, the weekly variation would not aect the trend but the monthly would.

To decide which ones of these that are the best depends on whether the weekly variation or the monthly variation is the dominant one. Amongst the data that has been analysed, the weekly often seems to be the dominant one, when both of them exist. Therefore, the Moving Average with d = 28 is suggested for these cases.

• Case 5 Yearly and weekly variation. In this case the best way to calculate the trend also depends on which seasonal variation that is the most dominating one. If it is the weekly, then a Moving Average Method with d = 364 should be used, which contains 52 full weeks. If it's the yearly, then any of the methods described in Case 3 could be used. Even in this case, amongst the data that have been analysed, the weekly often seems to be the dominant one, when both of them exist.

Therefore, the Moving Average with d = 364 is suggested for these cases.

• Case 6 Yearly and monthly variation. In this case, the Moving Average with d = 365 should be used, since this will sum up 12 full months, which means that neither the yearly variation or the monthly variation will aect the trend.

• Case 7 Yearly, monthly and weekly variation. Even in this case it depends which seasonal variation that is the most dominant one in order to estimate the trend in the best way. Even here, the weekly variation often seems to be the dominant one. Therefore, the Moving Average Trend Method with d = 364 is suggested.

To sum up, it is quite dicult to estimate the trend, without letting it getting aected by the seasonal

variations. Especially in data from a contact center that has been closed some of the days. Except for

the seasonal variations there are also outliers and bank holidays that will aect the estimates of the

trend. How to deal with this, and to get the estimate as accurate as possible will be discussed in Section

9.

5 Seasonal variations

When the trend has been removed, the next thing to do is to estimate the seasonal variations. As mentioned in Section 4.2, there are ve types of seasonal variances that may appear: Month of the year, week of the year, week of the month, day of the month and day of the week. In this section, methods will be described to estimate each of them. When estimating the seasonal variations, it is preferable to start with the yearly (if it exists), continue with the monthly(if it exists) and at last the weekly(if it exists).

When estimating each of these in the upcoming sections, the sample ˆx = (ˆx 1 , .., ˆ x n ) represents the historical data where the trend and the earlier seasonal variations have been removed. When estimating the seasonal variation it is important that the other seasonal variations, that haven't yet been estimated, do not aect the estimates. All of the upcoming methods assume a linear model. However they can all be used even in the non-linear case if starting with the logarithm of the sample.

5.1 Month of the year variation

The best way to estimate the month of the year variation depends on whether a weekly and/or a monthly variations are present or not. If neither of them are, the best way is simply to take the average of all the days in January in the sample, i.e. s ^(m,y) 1 = mean {(ˆx 1 , ..., ˆ x n ) : ˆ x j belongs to J anuary } and then do the same for the rest of the months. If weekly is present, then the vector {(ˆx 1 , ..., ˆ x _n ) : ˆ

x _j belongs to J anuary } should include the same number of Mondays, Tuesdays etc. so that the weekly variation doesn't aect the trend. If a monthly variation is present, then it is important that the vector {(ˆx 1 , ..., ˆ x _n ) : ˆ x _j belongs to J anuary } concludes full months so that the monthly variation does not aect the yearly variation. If both of them are present, a compromise of the above methods is preferable.

Again, recall from Section 2.3 that the closed days shouldn't be included in the calculations. This means that, even though a full month is considered, some of the days that month might have been closed, which means that they were not included in the calculations. Hence, the monthly variations might aect the estimate of the month of the year variation anyway. The same yields for the weekly variation. Hence, in data that concludes closed days, it is quite dicult to nd a way to estimate the seasonal variations in a 'perfect mathematical way', since the estimates will be aected by other seasonal variations. A way to deal with this issue will be discussed in Section 9.

Date

2011 2012 2013 2014 2015

×10 ⁴

-1 0 1 2 3

4 Data with the estimated trend removed

Figure 5.1: Historical data with the trend removed.

As an example, consider the historical data in Figure 5.1 with the trend removed. In Figure 5.2 all the

data points that belong to March and May have been gathered in two histograms. The mean and the