VOLATILITY FORECASTING PERFORMANCE

(1)

Master Thesis, 15 hp Economics

VOLATILITY FORECASTING PERFORMANCE

An evaluation of GARCH-class models

Marcus Ryhage

(2)

(3)

ACKNOWLEDGMENT

I would like to thank Tomas Sjögren at Umeå University, Department of Economics, for his guidance and expertise in the field of econometrics.

(4)

(5)

ABSTRACT

Volatility is considered among the most vital concepts of the financial market and is frequently used as a rough measure of the total risk of financial assets. Volatility is however not directly observable in practice; it must be estimated. The procedure in estimating and modeling volatility can be performed in numerous ways. However, the GARCH-class models have historically proven to be successful in these estimations while also being rather simple and practically applicable. The purpose of this study is to evaluate which of the included six GARCH models that produce the most accurate forecasts of future volatility during a “normal”

trading year. Our data sample consists of 9 stock market indices and the forecast is performed on the trading year of 2019. The out-of-sample forecasts have been analyzed against the proxy of realized volatility which is based on high-frequency data and proven to be successful in previous studies. To evaluate the correlation between the forecasts and the realized volatility proxy, four different evaluation measures (MAE, MAPE, MSE and QLIKE) are executed.

Findings suggest that the asymmetric GARCH-class and particularly the EGARCH volatility model(s) tend to be superior.

(6)

1 INTRODUCTION

Estimating and forecasting stock market volatility has been researched extensively in empirical and theoretical studies by both academics as well as practitioners, with the motivations for this extensive work being numerous. Volatility is considered among the most vital concepts of the financial market and is frequently used as a rough measure of the total risk of financial assets.

Several value-at-risk models, used for measuring market risk, require an estimated volatility parameter. Volatility is also one of the variables which enter directly into the Black-Scholes formula used to derive option prices (Brooks 2014). More recently, financial instruments have been introduced that are based solely on the speculation of volatility, with the VIX index as an example that was introduced in 2004.

Even though asset volatility is well defined, measured by the variance or standard deviation of asset returns, it is not directly observable in practice. However, using observed prices of assets and derivatives enables us to estimate the volatility of the underlying prices. The procedure in estimating and modeling volatility, which can be performed in countless ways, is what has encouraged the vast research in finding methods to estimate volatility as accurately as possible (Tsay 2013).

While volatility is not directly observable on the market, it still possesses certain characteristics that are commonly seen in asset returns. First, volatility does not tend to diverge infinitely but is considered fixed within a certain range, statistically known as being stationary. Second, volatility is continuous in its evolvement and large sudden jumps are unusual. Third, volatility tends to increase/decrease in clusters. This is known as volatility clustering. Volatility clustering implies that if volatility was high (low) today, there is a higher probability that volatility will be high (low) tomorrow. Fourth, volatility tends to react differently to large price drops in comparison to a price increase of the same magnitude. This is known as the leverage effect. The leverage effect implies that a large price drop will have a greater effect on future volatility than a price increase of the same magnitude. These four characteristics are important to consider in the research and development of volatility estimation procedures (Tsay 2013).

Of the countless numbers of non-linear models, there are only a few that have been found useful when modeling the volatility of financial data. The most popular models are the autoregressive

(8)

ARCH and GARCH models that both allow for conditional heteroscedasticity, implying that the estimations can follow different processes at different periods of time (Brooks 2014).

With the ARCH model being condemned as inferior to the GARCH models by historical volatility forecasting literature, only GARCH models will be featured in this thesis. Starting with the symmetric GARCH-class models, developed to account for aforementioned volatility characteristics one to three. The symmetric GARCH-class models produce forecasts that converge to the unconditional variance of the series as the forecast horizon increases. This mean-reverting property implies that if volatility is currently at a level considered high (low) relative to the historic data, it will tend to converge back down (up) the historic average. (Brooks 2014).

The asymmetric GARCH-class models are extensions to the symmetric GARCH-class models, developed to furthermore incorporate the fourth volatility characteristic, leverage effects. These models can capture the asymmetry in volatility that is brought on by large price drops or price increases (Tsay 2013).

The extensive literature on the subject of volatility forecasting is not without critique. As pointed out by Poon and Granger (2003), compiling previous individual studies to find an optimal recipe for success in volatility forecasting is hard because of the many discrepancies.

For example, different studies are analyzed from different viewpoints and reasons, studies are performed on different assets during different periods of time. In addition, the forecasted performance is calculated with many different kinds of evaluation measures.

This thesis will feature a comparison of the most widely acclaimed GARCH models, two symmetric and four asymmetric. The forecasted results will be evaluated against the realized volatility proxy based on high-frequency intraday data of returns, which has been proven to be very effective in more recent volatility literature. The thesis will also feature four different methods of measuring the performance of the forecasts against the realized volatility proxy.

The purpose of this study is to evaluate which of the included six GARCH models that produce the most accurate forecasts of future volatility during a “normal” trading year.

(9)

latest trading year not under the heavy influence of extreme market conditions, such as the corona crisis. This time period is chosen since such extreme market conditions tend to be difficult if not impossible to predict using time-series models. In addition, only a few such trading days have the potential to influence the ranking of the GARCH models (Lyócsa, Molnár and Výrost 2020).

The thesis is organized as follows: In section two, the literature review and model selection are presented. Section three presents the theoretical approach of the examined GARCH-class models. In section four, the data set and methodology are introduced. Section five exhibits the out-of-sample forecast result. Finally, the discussion and conclusion are presented in sections six and seven.

(10)

2 LITERATURE REVIEW

The review by Poon and Granger (2003) examines the results from 93 volatility forecasting studies. The review is very extensive but considering its large base of content, still very succinctly presented and a good starting point for novices to volatility forecasting literature.

The survey is focused to answer the two questions: is volatility forecastable? If it is, which method will provide the best forecasts?

The different methods of volatility forecasting that are reviewed by Poon and Granger are categorized into the four groups:

- Historical volatility (HISVOL) models – Such as random walk, historical averages of squared or absolute returns, exponentially weighted moving averages and autoregressive models, etc.

- Autoregressive conditional heteroscedasticity (ARCH) models – Such as GARCH (generalized ARCH) and all extended GARCH-class models. The group that will be evaluated in this thesis.

- Implied standard deviation (ISD) models – Such as the Black-Scholes model and its various generalizations, based on the concept of implied volatility.

- Stochastic volatility (SV) models – All stochastic volatility model forecasts.

The presented findings by Poon and Granger suggest that the group of ISD models, based on the concept of implied volatility, performed best in volatility forecasting, followed by the ARCH/GARCH models and HISVOL models. However, the result of the ISD models being superior is not surprising since their forecasts are based on a more relevant but less practical data set (option prices) than the ARCH/GARCH and HISVOL models.

More specifically for the group of ARCH/GARCH models, it was clear that GARCH dominate ARCH. Moreover, it is suggested that the asymmetric GARCH models, incorporating volatility asymmetry, were found superior to the symmetric GARCH models. It is however worth noting that the comparison of different methods is problematic since the results depend heavily on the investigated market and time period.

(11)

Alberg, Shalit and Yosef (2008) compare the forecasting performance on several GARCH-class models. The study is performed on two market indices on the Israeli, Tel Aviv Stock Market using daily data over 13 years, ranging from the year 1992 to 2005. The analysis implemented five loss functions for forecast evaluations: MSE, MedSE, MAE, AMPAPA and TIC. Their findings suggest that the asymmetric GARCH-class models with Student’s T distribution, generally improve the forecasting performance in comparison to the symmetric GARCH-class models. Further, for the in-class asymmetric GARCH models, the EGARCH model outperforms both the APARCH and GJR-GARCH (very similar to later presented TGARCH).

Lim and Sek (2013) compare the forecasting performance on several GARCH-class models.

The study is performed on one market index on the Malaysian Stock Market to compare the different GARCH model's performance pre-, post- and during the 1997 financial crisis. The analysis implemented three loss functions for forecast evaluations: MSE, RMSE and MAPE.

Their findings concluded that the symmetric GARCH-class models outperform the asymmetric models during the crisis and vice versa for the pre- and post-crisis scenarios.

Except for the model selection, there are many other aspects to consider within the volatility forecasting procedure. A very recent study by Lyócsa, Molnár and Výrost (2020) investigate the benefit of using high-frequency (returns) data as the underlying data set. The study is performed on 18 developed and emerging market indices using daily data over 20 years, ranging from the year 2000 to 2020. The analysis implemented two loss functions for forecast evaluations: QLIKE and MSE. Their findings suggest that high-frequency volatility models can be slightly superior for short-term, 1-5 day ahead, forecasts. However, for most market indices these differences are statistically insignificant and with increasing forecast horizons, both low- and high-frequency lead to similar forecast errors.

Further, the study by Lyócsa, Molnár and Výrost (2020) reveals that almost all market indices included in their data set suffer from larger forecast errors in periods of higher market volatility.

In these periods none of the models achieves well, making it difficult to statistically distinguish between the forecast errors. The overall conclusions suggest that both low- and high-frequency volatility models provide statistically comparable forecast accuracy.

Similarly, to the above study by Lyócsa, Molnár and Výrost (2020), Andersen and Bollerslev (1998) examine the benefits of using high-frequency intraday data as the underlying data for

(12)

the volatility proxy. The volatility proxy aims to represent the true, unobservable, volatility process which the generated forecasts will be evaluated against. Their analysis suggests that high-frequency intraday returns (known as realized volatility) are superior to the previously used, low-frequency daily squared returns when acting as a proxy to true volatility.

Generous efforts have been made in researching the optimal construction of forecasting models.

However, not much consideration has been applied to the evaluation measures and there is no generally applicable way of evaluating the forecast. Different measures that are frequently used in historical volatility forecasting literature are MAE (Mean Absolute Error), MAPE (Mean Absolute Percentile Error), MSE (Mean Square Error) and RMSE (Root Mean Square Error) (Poon and Granger 2003). This can also be suggested by the above literature review. Patton (2011) promotes the use of two forecast evaluation measures that are considered robust to noisy volatility proxies. The QLIKE (Quasi-Likelihood) and the MSE (Mean Square Error) both propose a consistent ranking between competing models even with the occurrence of noisy volatility proxies.

As highlighted by Poon and Granger (2003), the comparison of different methods is problematic since the results depend heavily on the investigated market and time period.

Nevertheless, from what can be obtained by the literature review, asymmetric GARCH-class models generally seem to outperform the symmetric GARCH-class models. Particularly during periods of normal/less volatility. The importance of high-frequency (returns) data seems to apply more to the volatility proxy than to the underlying sets of time series. In addition, many different evaluation measures have historically been employed. Patton (2011) suggests implementing robust evaluation measures such as the MSE and QLIKE.

(13)

3 THEORETICAL FRAMEWORK

3.1 ARCH MODEL

The non-linear ARCH (autoregressive conditionally heteroscedastic) model has gained widespread use among researchers. The ARCH model estimates the conditional variance of return by a maximum likelihood function. In this way, it also accounts for heteroscedasticity, very usually seen in the context of returns in financial time series data.

The ARCH process introduced by Engle (1982) recognizes the difference between unconditional and conditional variance, allowing the conditional variance to change at any point in time. This is accomplished by allowing the conditional variance (𝜎_𝑡²) to depend on the previous values of the squared errors (𝑢_𝑡²) rather than previous values of standard deviation (Brooks 2014). The general ARCH(q) is expressed in Equation 1 below:

𝜎_𝑡² = 𝛼₀+ 𝛼₁𝑢_𝑡−1² + 𝛼₂𝑢_𝑡−2² + ⋯ + 𝛼_𝑞𝑢_𝑡−𝑞² (1) where

𝑢_𝑡 = 𝜖_𝑡𝜎_𝑡

Where 𝜎_𝑡² is the conditional variance and 𝑎₀ is the long-run average variance rate. 𝑢_𝑡−1² … 𝑢_𝑡−𝑞² are the lagged values of the squared errors and 𝛼₁… 𝑎_𝑞 its corresponding coefficients. 𝜖_𝑡 is a sequence of i.i.d. random variables with zero mean and zero unit variance. The distribution of 𝜖_𝑡 is assumed to take the form of a standard normal, standardized student t or generalized error distribution (Tsay 2013). Since 𝜎_𝑡² is a conditional variance, the value must be strictly positive.

The variables on the right-hand side are all squared and can therefore not take a negative value.

However, to further impose the non-negativity constraint on the fitted value of 𝜎_𝑡², below constraints are added to the coefficients:

𝛼₀ ≥ 0 𝛼₁… 𝑎_𝑞 ≥ 0

(14)

3.2 CONDITIONAL MEAN

Further, the dynamics of the conditional mean need to be specified through an ARMA (autoregressive moving average) process. The general ARMA(p,q) is expressed in Equation 2 below:

𝑋_𝑡 = 𝛼₀ + ∑ 𝛼_𝑖

𝑝

𝑖=1

𝑋_𝑡−𝑖 + ∑ 𝛽_𝑗

𝑞

𝑗=1

ℯ_𝑡−𝑗 (2)

Where the time-series of data (𝑋_𝑡) is regressed on the AR part involving lagged values of the time series (𝑋_𝑡−1) and the MA part involving a linear combination of past errors (ℯ_𝑡−𝑗). The combination of past errors (ℯ_𝑡−𝑗) is assumed to be i.i.d. with a mean zero and variance one (Alexander 2008).

To identify the optimal order for our AR(p) and MA(q) processes, we fit all different (p,q) combinations to the ARMA model to establish the best fit. The goodness of fit is assessed by the use of the BIC (Bayesian Information Criterion) criterion. For eight out of nine market indices, the ARMA(0,0) was chosen as the optimal model. For the DJI market index, the ARMA(0,1) was indicated as the preferred model.

(15)

3.3 GARCH-CLASS MODELS – SYMMETRIC

There are however a couple of weaknesses in the ARCH model. The model is likely to overpredict the volatility and also frequently requires several parameters to describe the volatility process of returns. The GARCH (generalized autoregressive conditionally heteroscedastic) models are considered advantageous and more widely used than ARCH is because it is more parsimonious and avoids overfitting while it still not breaching the non- negativity constraints (Tsay 2013).

The GARCH-class models are also better constructed to incorporate an almost universal characteristic of asset returns in financial data, volatility clustering. Volatility clustering implies that the rate of volatility at time 𝑡 is dependent on previous time periods. Large returns (positive or negative) are expected to follow large returns and small returns are expected to follow small returns (positive or negative). This phenomenon can partly be explained by the asymmetric arrivals of price-driving information, the information reaches the market in bunches (or clusters) rather than constantly paced. The existence of volatility clustering promotes the use of GARCH-class models since it can parameterize (model) this existence (Brooks 2014).

Followingly we will introduce six extended GARCH models which will be separated into two subgroups, symmetric and asymmetric. The symmetric models require the placement of non- negativity constraints to avoid estimating a negative conditional variance. The asymmetric models can avoid these restrictions, making them more dynamic and therefore able to capture the leverage effects which are discussed later in this section.

(16)

3.3.1 GARCH MODEL

The GARCH model, developed (independently) by Bollerslev (1986) and Taylor (1986), is an extension of the previously introduced ARCH model. In the GARCH structure, the fitted conditional variance (𝜎_𝑡²), is now allowed to also be dependent on its own lagged values. This implies that, if volatility was high in the preceding period(s), the estimated volatility will remain more volatile until it gradually returns to its average.

In the academic finance literature, higher orders than GARCH(1,1) are rarely estimated. The reason being that, in general, the GARCH(1,1) model will be sufficient to capture the volatility clustering in financial time series data (Poon and Granger 2003). The following GARCH model and all later introduced GARCH-class extensions will be expressed in the (1,1) formulation.

The GARCH(1,1) is expressed in Equation 3 below:

𝜎_𝑡² = 𝛼₀ + 𝛽₁𝜎_𝑡−1² + 𝛼₁𝑢_𝑡−1² (3)

With similar notations as from the previously introduced ARCH model. For the GARCH model, the lagged fitted conditional variance (𝜎_𝑡−1² ) and its corresponding coefficient (𝛽₁) are introduced. Intuitively for the coefficients, 𝛼₁ measures the effect of a shock while 𝛽₁ measures the persistence of the effect. The current fitted conditional variance (𝜎_𝑡²) can be interpreted as a weighted function of the long-run average variance rate function (𝛼₀), information about volatility during the previous period (𝛼₁𝑢_𝑡−1² ) and the fitted conditional variance from the model during the previous period (𝛽₁𝜎_𝑡−1² ).

Building upon the non-negativity constraints from the ARCH model, we now include the constraint 𝛼₁ + 𝛽₁ < 1 ensuring that the unconditional variance is held constant. Implying that the volatility will converge to long-run average variance rate over time (Tsay 2013).

𝛼₀ ≥ 0 𝛼₁, 𝛽₁ ≥ 0 𝛼₁ + 𝛽₁ < 1

(17)

3.3.2 IGARCH MODEL

The integrated GARCH (IGARCH) model by Engle and Bollerslev (1986) is very similar in construction to the GARCH model. The previously presented GARCH model, including the constraint 𝛼₁ + 𝛽₁ < 1, is considered stationary in variance and therefore conditional variance forecasts converge to the long-run average variance rate over time. In the IGARCH model, this convergence will not occur (Brooks 2014).

Consider that the GARCH constraint would instead be constructed 𝛼₁ + 𝛽₁ = 1, then it would imply a unit-root in variance and result in the non-stationary IGARCH model. The IGARCH model captures the volatility clustering effect but a shock in this model impacts the future volatility over an infinite horizon, implying that the unconditional variance is none existing for this model (Poon and Granger 2003). The IGARCH(1,1) is expressed in Equation 4 below:

𝜎_𝑡² = 𝛼₀+ 𝛽₁𝜎_𝑡−1² + (1 − 𝛽₁)𝑢_𝑡−1² (4)

With similar notations as from the previously introduced standard GARCH model and with non-negativity constraints imposed on parameters 𝛼₀ and 𝛽₁. For the IGARCH model, 𝛼₁ is replaced with (1 − 𝛽₁), this is done to fulfill the previously hypothesized condition of 𝛼₁ + 𝛽₁ = 1 without the need to impose any further restrictions (Brooks 2014).

3.4 GARCH-CLASS MODELS – ASYMMETRIC

The previously presented symmetric GARCH models manage to capture leptokurtosis and volatility clustering, but since their distributions are symmetric, they fail to model the leverage effect. The leverage effects refer to the phenomena that volatility tends to increase more after a large price fall in comparison to a price rise of the same scale. Due to the non-negativity constraints, restricting the dynamics of the ARCH model and the symmetric GARCH models, these models assume that both negative and positive shocks have the same effect on volatility.

In attempts to capture these leverage effects, many nonlinear extensions of the symmetric GARCH-class models have been proposed (Nelson, 1991). Followingly we will introduce four of these extensions which are all categorized as asymmetric GARCH-class models.

(18)

3.4.1 EGARCH MODEL

The asymmetric EGARCH (exponential GARCH) model developed by Nelson in 1991, specifies the conditional variance in logarithmic form. This implies that we do not need to impose the non-negativity constraints to avoid negative conditional variance. The logarithmic form specification, with no imposed constraints, captures the aforementioned leverage effects, meaning that a negative shock would lead to a higher conditional variance in the following period than a positive shock would (Poon and Granger 2003). The EGARCH(1,1) is expressed in Equation 5 below:

𝑙𝑛(𝜎_𝑡²) = 𝛼₀+ 𝛽₁𝑙𝑛(𝜎_𝑡−1² ) + 𝑔(𝑧_𝑡) (5) where

𝑔(𝓏_𝑡) = 𝜃𝓏_𝑡+ 𝛾(⌈𝓏_𝑡⌉ − √2 𝜋⁄ )

With similar notations as from the previously introduced standard GARCH model and with non-negativity constraints imposed on parameters 𝛼₀ and 𝛽₁. For the EGARCH model, the log fitted conditional variance (𝑙𝑛(𝜎_𝑡²)) is determined by the long-run average variance rate (𝛼₀), the asymmetric response function (𝑔(𝑧_𝑡)) and the log fitted conditional variance in the previous period (𝛽₁𝑙𝑛(𝜎_𝑡−1² )). Further, the asymmetric response function (𝑔(𝑧_𝑡)), capturing the leverage effects, is derived from 𝜃𝓏_𝑡 which determines the sign of the effect and 𝛾(⌈𝓏_𝑡⌉ − √2 𝜋⁄ ) which determines the size of the effect (Alexander 2001).

(19)

3.4.2 TGARCH MODEL

The asymmetric TGARCH (threshold GARCH) was developed by Glosten, Jagannathan, and Runkle in 1993. In difference to the previously presented EGARCH, the TGARCH instead uses a multiplicative dummy variable to capture the leverage effects. The dummy variable is incorporated into the specification to control if a negative shock would indicate a statistically significant difference (compared to a positive shock). The TGARCH(1,1) is expressed in Equation 6 below:

𝜎_𝑡² = 𝛼₀+ 𝛽₁𝜎_𝑡−1² + (𝛼₁+ 𝛾₁𝑁_𝑡−1)𝑢_𝑡−1² (6) where

𝑁_𝑡−1 = {1 𝑖𝑓 𝑢_𝑡−1 < 0 0 𝑖𝑓 𝑢_𝑡−1 ≥ 0

With similar notations as from the previously introduced standard GARCH model and with non-negativity constraints imposed on parameters 𝛼₀, 𝛽₁, 𝛼₁ and 𝛾₁. For the TGARCH model, the dummy variable (𝑁_𝑡−1) takes value depending on lagged values of the errors (𝑢_𝑡−1).

Thereby, implying that a negative shock has a larger impact with 𝛾₁ > 0 than that of a positive shock and using zero as the threshold to divide the effects of past shocks (Tsay 2013).

3.4.3 NGARCH MODEL

The asymmetric NGARCH (non-symmetric GARCH) was developed by Engle and Ng in 1993.

The NGARCH model also features the capability to capture leverage effects. The NGARCH(1,1) is expressed in Equation 7 below:

𝜎_𝑡² = 𝛼₀ + 𝛽₁𝜎_𝑡−1² + 𝛼₁(𝑢_𝑡−1− 𝜃𝜎_𝑡−1)² (7)

With similar notations as from the previously introduced standard GARCH model and with non-negativity constraints imposed on parameters 𝛼₀, 𝛽₁, 𝑎₁ and 𝜃. For the NGARCH model, if the leverage parameter (𝜃) > 0, the exhibited specification indicates that a negative shock will have a greater effect on future volatility than that of a positive shock. Interestingly to note is also that if the leverage parameter (𝜃) = 0, the model is reduced to a standard GARCH model (Tsay 2013).

(20)

3.4.4 APARCH MODEL

The asymmetric APARCH (asymmetric power ARCH) was developed by Ding, Granger and Engle in 1993. The APARCH model is among the most used models in the volatility forecasting literature because of its ability to take the shape of several of the previously presented GARCH- class extensions. The APARCH(1,1) is expressed in Equation 8 below:

𝜎_𝑡^𝜌 = 𝛼₀ + 𝛽₁𝜎_𝑡−1^𝜌 + 𝛼₁(|𝑢_𝑡−1| − 𝛾₁𝑢_𝑡−1)^𝜌 (8)

With similar notations as from the previously introduced GARCH model and with non- negativity constraints imposed on parameters 𝛼₀, 𝛽₁, 𝑎₁ and 𝛾₁. For the APARCH, the power function (𝜌) is difficult to interpret, there are however few special cases worth noting. If the first special case of the power function (𝜌) = 0, the parameter limits to 𝜌 → 0 and the model reduces to an EGARCH. In the second special case of the power function (𝜌) = 1, the model inherits the volatility straight from the equation. In the third and final special case of the power function (𝜌) = 2, the parameter can be fixed and the model reduces to a TGARCH. The power function (𝜌) is a method of transformation designed to improve the model’s goodness of fit.

This can explain the popularity of this model if the intention is to predict volatility.

(21)

4 DATA AND METHODOLOGY

4.1 DATA

To investigate the forecasting ability of our previously presented models, we use data from nine different developed market indices. The gathering of data has been done through two data sources, Thomson Reuters Datastream and Oxford-Man Institute's Realized Library. The full statistical analysis has been performed in the software R-Studio.

Our sample period covers 14 years and ranges from the beginning of January 2006 to the end of December 2019. To execute our out-of-sample analysis, we define an in-sample period ranging from the beginning of January 2006 to the end of December 2018. This leaves the beginning of January 2019 to the end of December 2019, for out-of-sample analysis. This period was chosen to execute the forecasting on the most present trading year that was not under the heavy influence of major exogenous events, such as the Corona crisis. As in line with previous volatility forecasting literature, weekends and holidays are excluded and trading days are treated as a consecutive time series.

To enhance the validity of our later presented conclusions, the ambition has been to study as many market indices as possible. However, with the restriction of time and available data for long time series, below nine market indices have been included.

Table 1. Overview of market indices

Index ticker Index name Region Observations

DJI Dow Jones Industrial Average USA 3516

FTSE Financial Times Stock Exchange 100 United Kingdom 3537

GDAXI Deutscher Aktien Index Germany 3547

GSPTSE Toronto Stock Exchange Composite Canada 3502

IXIC NASDAQ Composite USA 3517

N225 Nikkei Stock Average Japan 3426

OMXSPI OMX Stockholm All Share Sweden 3509

SSEC Shanghai Stock Exchange Composite China 3400

STOXX50E Euro STOXX 50 Index Eurozone 3574

Note: The number of observations varies due to the variation in trading days per year (between 239 and 253) depending on the market index.

(22)

The presented models are estimated using the daily rate of returns. For a better estimation of the daily rate of return, one needs to account not only for the intraday price variation but also for price changes during the non-trading, overnight period. Prices for the non-trading periods are generally hard to come across, which is why we use close-to-close prices to calculate our daily rate of return. Given the closing price from the current day 𝑝_𝑡^𝑐 and closing price for the preceding day 𝑝_𝑡−1^𝑐 , our daily rate of return is expressed in Equation 9 below:

𝑟_𝑡 = 100(𝑙𝑛(𝑝_𝑡^𝑐) − 𝑙𝑛(𝑝_𝑡−1^𝑐 )) (9)

All volatility estimators are occasionally subjected to rare and extreme observations (outliers).

These extreme occasions are generally difficult if not impossible to predict using time-series models. In addition, the outliers could affect the estimations of our volatility models in a damaging way (Lyócsa, Molnár and Výrost 2020). To decrease the effect of these extreme observations, we executed a winsorization by a rolling window filtering procedure on our daily rate of return. The procedure was executed so that values above the 99.5 percentile were replaced by the 99.5 percentile value and the rolling window was set to 1000 observations.

Table 2. Summary statistics of daily returns

Index ticker Min. Max. Mean Std. deviation Skewness Kurtosis

DJI -5.5429 5.2709 0.0256 1.0440 -0.4533 4.8682

FTSE -5.5902 5.3262 0.0073 1.0734 -0.2669 3.9834

GDAXI -6.5499 5.9590 0.0229 1.2826 -0.3294 3.2634

GSPTSE -6.4281 5.1449 0,0113 1.0122 -0,6715 6.3971

IXIC -5.7029 5.9930 0.0400 1.2270 -0.3117 3.2947

N225 -7.0276 5.6448 0.0120 1.4128 -0.4339 2.7065

OMXSPI -5.7894 5.8201 0.0222 1.2453 -0.2598 3.1958

SSEC -7.6272 7.0934 0.0289 1.6047 -0.6132 3.6320

STOXX50E -6.2949 5.6401 0.0014 1.2867 -0.3074 3.0638

Notes: Min., max., mean and standard deviation are multiplied by 100 (% format).

(23)

4.1.1 DISTRIBUTION

Depending on the underlying stochastic process and whether the parameters are time-varying, the distribution of returns may follow different shapes. When modeling returns, three frequently used distributions are Standard Normal (Gaussian), Student's T, and Generalized Error distributions (Poon and Granger 2003). To ensure the optimal goodness-of-fit for our GARCH models, we must establish the appropriate distribution of the error term (𝑢_𝑡). This was done by performing QQ (quantile-quantile) plots on all market indices, which can be seen in Figure A1 in the appendix. The QQ plots use the quantiles of the in-sample data and visually plots them against the quantiles from the aforementioned distributions, thereby allowing us to establish the distribution with the best goodness-of-fit.

As can be seen in Figure A1, the Student’s T distribution is declared as the best fit for all our market indices, implying that our data has relatively fat tails. This finding is very common since returns in financial data often exhibit Leptokurtosis, which implies a distribution with fat tails and increase peakedness at the mean (Brooks 2014). This is also in line with the findings of historical volatility forecasting literature. Consequently, the Student’s T distribution is fitted for all GARCH models.

(24)

4.2 METHODOLOGY

4.2.1 DIAGNOSTICS

Developed by Ljung and Box in 1978, the Box-Ljung test is performed on the lagged residuals from the fitted ARMA(p,q). The test checks for serial correlation in the residuals, the null hypothesis implying no serial correlation against the alternative that the residuals are serially correlated. Under the GARCH framework, the rate of return should be serially uncorrelated but dependent. Looking below at the 𝑄₂ test statistic in Table 3, we can see that we can accept the null hypothesis of no serial correlation for seven out of nine market indices. For the two indices IXIC and N225, we reject the null hypothesis at a 1% and 5% significance level respectively, indicating that these time series suffer from serial correlation.

Followingly, the ARCHLM (ARCH Lagrange Multiplier) test for ARCH effects is carried out.

Similar to the Box-Ljung test, the ARCHLM test is also performed on the residuals from the fitted ARMA(p,q). This test checks for the presence of volatility clustering at a different order of lags with the null hypothesis of no ARCH-effects (volatility clustering) against the alternative hypothesis of ARCH-effects existing. Looking below at the ARCHLM (12) test statistic in Table 3, we can see the positive result that we can reject the null hypothesis of no ARCH-effects for all our market indices on lag twelve at a 1% level of significance. The same results were also found on the lower order of lags (one through twelve) for all market indices.

Table 3. Test statistics - Box-Ljung test and ARCHLM test.

Index ticker Q2 ARCHLM (12)

DJI 0.0102 309***

FTSE 0.5911 373***

GDAXI 0.5967 424***

GSPTSE 0.0001 255***

IXIC 7.7462*** 305***

N225 4.8596** 309***

OMXSPI 0.0411 395***

SSEC 1.1672 635***

STOXX50E 0.1391 408***

(25)

4.3 OUT-OF-SAMPLE FORECASTING

The method of use for our out-of-sample forecasting of volatility is a one-day-ahead rolling forecast with recursive refitting for every trading day. As mentioned early in this section, the out-of-sample analysis will be performed on the full trading year of 2019. Depending on the number of trading days for the different market indices in 2019, the rolling forecast window will vary between 239 to 253 trading days. The recursive refitting with the true rate of return for each period, 𝑡, enable us to make as accurate predictions as possible. Intuitively, the first day (𝑡 + 1) forecast is calculated from the true rate of return of the in-sample period until 𝑡 = 0. For the second day (𝑡 + 2) forecast, the model is then refitted with the true rate of return at time 𝑡 = 1. This process is repeated for the full forecasted trading year. The first day (𝑡 + 1) ahead rolling forecast method of GARCH (1,1) is expressed in Equation 10 below:

𝜎̂_𝑡+1² = 𝛼̂₀ + 𝛽̂₁𝜎_𝑡² + 𝛼̂₁𝑢_𝑡² (10)

With similar notations as presented in the previous section, 𝜎̂_𝑡+1² is the predicted volatility at time 𝑡 + 1 (Alexander 2008). In the same way as the above GARCH(1,1), the other GARCH- class models use the same method of forecasting, built upon their respective specifications.

4.3.1 REALIZED VOLATILITY

To evaluate the forecasting performance of each model, the forecasts must be compared to a

“true” value or proxy. Measuring the exact volatility of a market index for one day is a complex task since prices fluctuate within micro-seconds. In addition, sub-sampling in very close intervals may lead to a proxy that in addition to the deterministic data, contains a large portion of noise.

Historically, a frequently used measure in the volatility forecasting literature has been the daily squared return. It has however been proven that this is a noisy volatility proxy that does not capture reliable inference regarding the underlying “true” volatility in daily samples (Andersen and Bollerslev 1998).

(26)

The realized variance measure has however been proven to provide an accurate proxy in previous volatility forecasting literature (Poon and Granger 2003). The measure is constructed of high-frequency, intraday data of price movement. The realized variance estimates volatility by summing the squared, high-frequency, intraday returns. A relatively simple realized variance measure is expressed in Equation 11 below.

𝑅𝑉_𝑡^(𝑛)= ∑ 𝑟_𝑖,𝑛²

𝑛

𝑖=1

(11)

Where the interval of time, for example 𝑇₀ and 𝑇₁, is divided into 𝑛 subintervals

𝑇₀ = 𝑡_0,𝑛 < 𝑡_1,𝑛 < ⋯ < 𝑡_𝑛,𝑛 = 𝑇₁

And where intraday returns, 𝑟_𝑖,𝑛, are defined as

𝑟_𝑖,𝑛= 𝑝_𝑡₁_,𝑛− 𝑝_𝑡_𝑡−1_,𝑛

With optimal circumstances, the realized variance measure is consistent for the quadratic variation. However, it is well known by historical volatility forecasting literature that the market microstructure noise becomes increasingly problematic as 𝑛 → ∞. This makes the realized volatility unreliable when subsampling is performed at too close intervals of time. The study by Hansen and Huang (2012) researches the performance of eight different realized volatility measures, including six different realized variance measures, subsampled at different (15s, 2m, 5m, 10m, 15m and 20m) intervals of time. Their findings suggest that the out-of-sample log- likelihood for the two-minute and five-minute subsampling, on average, has the best empirical fit. For our research, the daily realized volatility proxy is constructed by subsampled five- minute intraday returns.

(27)

4.3.2 FORECAST EVALUATION

To quantify how the competing models’ respective forecasts relate to our realized volatility proxy, we use a set of evaluation measures. Simply put, the evaluation measures quantify the deviation (error) between the forecast and the realized volatility at time 𝑡 and then produces an averaged deviation (error) value for the whole forecast period.

In contradiction to the generous efforts made in the construction of forecasting models, not much consideration has been applied to the evaluation measures and there is no generic way of determining the best forecast (Reschenhofer, Mangat and Stark 2020). In this study, we execute three different measures that are frequently used in historical volatility forecasting literature (MAE, MAPE and MSE) and one more recently developed asymmetric loss function (QLIKE).

The MAE (Mean Absolute Error) measures the average magnitude of the error (or loss) between the forecast and the proxy. The error is calculated by taking the median of all absolute differences between the realized volatility proxy and the forecast. The measure is particularly useful since it is robust to outliers. However, a drawback to this measure is that it measures the error in units of the variable of interest. This makes it less useful when comparing across different sets of time series. The MAE measure is symmetric, meaning that it assigns equally large penalties to under-predicting as it does to over-predicting the proxy (Hyndman and Koehler 2006).

The MAPE (Mean Absolute Percentile Error) measures the average percentage error between the forecast and the proxy. Building upon the MEA measure, the idea of this metric is to be sensitive to relative errors and therefore introduces the advantage of being unit-free. This means that it for example will not be changed by a global scaling in the variable of interest which makes it favorable when comparing across different sets of time series. However, this also causes the MAPE to be scale sensitive where values close to zero can cause extreme values of the error measurement. The MAPE measure is symmetric when forecasted values and proxy values are within close proximity, meaning that it assigns equally large penalties to under- predicting as it does to over-predicting the proxy. However, it can be considered asymmetric when big deviations occur. This is due to the fact that under-predictions are limited to the percentage error of 100% while over-predictions have no upper limit. As a result, the MAPE

(28)

measure will favor models that under-forecast rather than over-forecast (Hyndman and Athanasopoulos 2021).

The MSE (Mean Square Error) measure is a quadratic loss function that assigns larger weights to big deviations between the forecast and proxy. It is therefore considered favorable when the differences between forecasted and realized volatility are large (Hansen and Lunde 2005). The measure is considered to propose a consistent ranking between competing models even with the occurrence of a noisy proxy. The MAE measure is symmetric, meaning that it assigns equally large penalties to under-predicting as it does to over-predicting the proxy (Patton 2011).

The QLIKE (Quasi-Likelihood) measure is an asymmetric loss function that severely penalizes large under-forecast. Similarly, to the MSE, the QLIKE measure is robust to outliers and considered to propose a consistent ranking between competing models even with the occurrence of a noisy proxy (Patton 2011).

Figure 1. General display of loss function for respective evaluation measure.

Error

(-) (+)

Loss

MSE

MAPE

MAE

QLIKE RV

(29)

The general evaluation measures are expressed in Equations 12-15 below:

𝑀𝐴𝐸 = 𝑛⁻¹∑⌈𝑅𝑉_𝑡− 𝜎̂ ⌉_𝑡

𝑛

𝑡=1

(12)

𝑀𝐴𝑃𝐸 = 100𝑛⁻¹∑ |(𝑅𝑉_𝑡²− 𝜎̂_𝑡²) 𝑅𝑉_𝑡² |

𝑛

𝑡=1

(13)

𝑀𝑆𝐸 = 𝑛⁻¹∑(𝑅𝑉_𝑡²− 𝜎̂_𝑡²)²

𝑛

𝑡=1

(14)

𝑄𝐿𝐼𝐾𝐸 =𝑅𝑉_𝑡

𝜎̂_𝑡² − 𝑙𝑛 (𝑅𝑉_𝑡

𝜎̂_𝑡²) − 1 (15)

Where 𝜎̂ is the forecasted volatility at time 𝑡, 𝑅𝑉_𝑡 _𝑡 is the realized volatility at time 𝑡 and 𝑛 representing the number of total steps (days) forecasted. The presented four evaluation measures individual result and ranking will followingly be presented for all competing GARCH-class models.

(30)

5 RESULTS

All forecasts are graphically displayed against their respective realized volatility proxy in Figure A2 in the appendix. In below example, the out-of-sample forecast for the DJI market index is displayed.

Figure 2. Out-of-sample forecast – DJI market index

Note: RV denotes the realized volatility proxy.

From the graphical displays of the out-of-sample forecasts in Figure 2 and A2, we can see that we are able to capture the underlying volatility process and all forecasts seem to follow the trends of our realized volatility proxy rather well. As expected, the models are however incapable of capturing the sudden, large magnitude, spikes of volatility. As a result of this, the models are generally seen to be underperforming during times of high volatility, a very clear example is given in the SSEC forecast in Figure A2 in the appendix. All models also display delays in their responses given a spike in the realized volatility proxy.

After more thorough auditing of Figure 2 and A2, it can also be seen that the models tend to separate into two groups, producing very visually similar forecasts. The symmetric GARCH and IGARCH as well as the asymmetric NGARCH all seem to follow similar trends and magnitude in their variance. Similarly, the asymmetric GARCH, TGARCH and APARCH forecasts tend to follow each other very closely and exhibit a more reactive movement, with

0.00 0.01 0.02 0.03

1/2/19 3/2/19 5/2/19 7/2/19 9/2/19 11/2/19

DJI

RV GARCH IGARCH EGARCH TGARCH NGARCH APARCH

(31)

The forecast performance results of all GARCH-class models will followingly be presented. As mentioned in the previous section, the forecast performance is determined by the four evaluation measures MAE, MAPE, MSE and QLIKE and their respective evaluation results will be presented in sections, following that order. The evaluation measures estimate their results by measuring the deviation (errors) between the forecasted daily volatility against the realized volatility proxy. For all evaluation measures, smaller values imply smaller deviations between the forecast and the realized volatility proxy and are therefore better.

5.1 FORECAST EVALUATION – MAE

From the MAE forecast evaluation results in Table 4, we can see that the asymmetric EGARCH model is most often preferred, forecasting closest to the proxy four out of nine times.

Followingly, the asymmetric TGARCH comes in second, forecasting closest to the proxy in three out of nine times each. In addition, the asymmetric APARCH is only preferred one time even though it presents almost identical loss values as the TGARCH. For the symmetric models, the IGARCH is preferred one time, and this is in the case of the more volatile market index SSEC.

Table 4. MAE forecast evaluation.

Model DJI FTSE GDAXI GSPTSE IXIC N225 OMXSPI SSEC STOXX50E Total GARCH 0,0050005 0,0054590 0,0048745 0,0025470 0,0059868 0,0049618 0,0035857 0,0085958 0,0062216 0 IGARCH 0,0050062 0,0053953 0,0048608 0,0025466 0,0060221 0,0049915 0,0035966 0,0085798 0,0061854 1 EGARCH _0,0048160 _0,0051984 _0,0045290 _0,0024770 _0,0058107 _0,0047583 _0,0032732 _0,0086882 _0,0059388 4 TGARCH 0,0047776 0,0052366 0,0045618 0,0025087 0,0057432 0,0047543 0,0032754 0,0087172 0,0060181 3 NGARCH 0,0050240 0,0054486 0,0048916 0,0025262 0,0060016 0,0049259 0,0035575 0,0086480 0,0062694 0 APARCH 0,0047790 0,0052724 0,0045293 0,0025059 0,0057532 0,0047550 0,0032540 0,0086515 0,0060344 1 Note: Shaded values indicate the lowest loss value (preferred model) for each market index. The “Total” column summarizes the total number of times that each model was preferred.

(32)

5.2 FORECAST EVALUATION – MAPE

From the MAPE forecast evaluation results in Table 5, we can see that the asymmetric EGARCH and TGARCH model is most often preferred, forecasting closest to the proxy three out of nine times respectively. Followingly, the asymmetric APARCH comes in second, forecasting closest to the proxy two out of nine times, once again displaying very similar loss values as the TGARCH. For the symmetric models, the GARCH is preferred only one time, and this is for the case for more volatile market index SSEC.

Table 5. MAPE forecast evaluation.

Model _DJI _FTSE _GDAXI _GSPTSE _IXIC _N225 _OMXSPI _SSEC _STOXX50E Total GARCH _227,35 _97,04 _119,99 _476,76 _325,02 _608,21 _164,14 _115,04 _311,21 1 IGARCH 229,38 98,60 120,80 473,01 328,33 599,93 166,01 115,60 309,20 0 EGARCH 195,69 74,77 102,05 468,85 258,39 532,80 143,02 126,66 216,56 3 TGARCH 188,09 73,38 100,15 482,56 254,41 543,50 142,14 125,21 219,03 3 NGARCH 227,28 95,30 118,74 473,77 324,03 591,14 161,38 121,09 304,29 0 APARCH 189,23 74,80 99,97 484,91 253,21 543,43 144,01 121,58 217,23 2 Note: Shaded values indicate the lowest loss value (preferred model) for each market index. The “Total” column summarizes the total number of times that each model was preferred.

5.3 FORECAST EVALUATION – MSE

From the MSE forecast evaluation results in Table 6, we can see that the asymmetric TGARCH model is most often preferred, forecasting closest to the proxy three out of nine times.

Followingly, the asymmetric TGARCH and APARCH and the symmetric IGARCH come in second, forecasting closest to the proxy two out of nine times respectively.

Table 6. MSE forecast evaluation.

Model DJI FTSE GDAXI GSPTSE IXIC N225 OMXSPI SSEC STOXX50E Total GARCH 2,667E-07 5,673E-07 3,540E-07 1,382E-09 3,478E-07 9,815E-08 2,880E-08 4,462E-06 8,020E-07 0 IGARCH _2,663E-07 _5,656E-07 _3,529E-07 _1,388E-09 _3,455E-07 _9,762E-08 _2,857E-08 _4,459E-06 _8,010E-07 2 EGARCH 2,710E-07 5,660E-07 3,433E-07 1,279E-09 3,410E-07 9,550E-08 2,650E-08 4,463E-06 7,949E-07 2 TGARCH 2,695E-07 5,665E-07 3,453E-07 1,268E-09 3,403E-07 9,476E-08 2,681E-08 4,455E-06 7,979E-07 3 NGARCH 2,693E-07 5,668E-07 3,548E-07 1,370E-09 3,483E-07 9,842E-08 2,862E-08 4,464E-06 8,042E-07 0

APARCH 2

(33)

5.4 FORECAST EVALUATION – QLIKE

From the QLIKE forecast evaluation results in Table 7, we can see that the asymmetric APARCH model is most often preferred, forecasting closest to the proxy four out of nine times.

Followingly, the asymmetric EGARCH comes in second, forecasting closest to the proxy three out of nine times. In addition, the asymmetric TGARCH is not preferred in any case even though it presents almost identical loss values as the APARCH. For the symmetric models, the IGARCH is preferred two times, and this is for the cases of the more volatile market indices SSEC and STOXX50E.

Table 7. QLIKE forecast evaluation.

Model _DJI _FTSE _GDAXI _GSPTSE _IXIC _N225 _OMXSPI _SSEC _STOXX50E Total GARCH _0,253161 _0,265133 _0,175734 _0,237020 _0,267873 _0,249938 _0,136939 _0,280673 _0,315808 0 IGARCH 0,252058 0,258591 0,174062 0,236241 0,266304 0,249532 0,136907 0,278435 0,315608 2 EGARCH 0,226311 0,248904 0,152373 0,227660 0,250489 0,234426 0,122692 0,282773 0,317404 3 TGARCH 0,225362 0,251675 0,155303 0,231506 0,245530 0,234043 0,123246 0,283603 0,322972 0 NGARCH _0,257553 _0,265686 _0,178372 _0,236318 _0,269136 _0,247889 _0,135744 _0,282057 _0,329338 0 APARCH 0,225353 0,252521 0,153358 0,231779 0,244972 0,234029 0,121138 0,281658 0,326645 4 Note: Shaded values indicate the lowest loss value (preferred model) for each market index. The “Total” column summarizes the total number of times that each model was preferred.

(34)

5.5 FORECAST EVALUATION – SUMMARY

The results from Table 8 suggest that the asymmetric EGARCH model is most often preferred, forecasting closest to the proxy in 12 out of 36 times according to our four evaluation measures.

Followingly, the asymmetric TGARCH and APARCH come in jointly second, forecasting closest to the proxy in 9 out of 36 times each while displaying almost identical loss values under all evaluation measures and across all market indices. The symmetric GARCH and IGARCH are collectively only preferred 6 times out of 36, with the majority of their success coming from the more volatile market indices SSEC and STOXX50E.

Table 8. Summary – Total number of times model preferred - All evaluation measures.

Model Total times preferred GARCH 1 (out of 36) IGARCH 5 (out of 36) EGARCH 12 (out of 36) TGARCH 9 (out of 36) NGARCH 0 (out of 36) APARCH 9 (out of 36)

(35)

6 DISCUSSION

As seen in Tables 4-7, the differences of the model's computed loss values are very small for all market indices and evaluation measures, suggesting that all models produce forecasts with relatively similar accuracy. This might not be unexpected since the out-of-sample forecasting is performed with one day ahead rolling forecast with recursive refitting of true observations for every trading day. The small differences of the model's computed loss values also create a struggle in the ranking within the respective GARCH-classes. Differences within the asymmetric GARCH-class models EGARCH, TGARCH and APARCH are so small that one should cautiously draw any conclusion from the ranking of these three models.

Slightly bigger differences can be seen when comparing the symmetric and asymmetric GARCH-classes. The asymmetric models, incorporating leverage effects, are visually seen in Figure A2 as more responsive to shocks in the volatility which may reflect the smaller computed loss values and better ranking. The findings suggesting the asymmetric GARCH-class models to be superior is in line with previous studies by Alberg, Shalit and Yosef (2008) and Poon and Granger (2003) and underlines the existence of leverage effects in stock market volatility.

The out-of-sample forecast period may not be seen as under the heavy influence of major exogenous events or crises and may be considered similar to the pre- and post-crisis time category of Lim and Sek (2013). For the pre- and post-crisis time periods, Lim and Sek’s findings similarly suggest the superiority of asymmetric GARCH-class models. However, for the crisis period, Lim and Sek find the symmetric models to outperform the asymmetric models.

This may act as an explanation for the success of our symmetric models when forecasting the more volatile market indices SSEC and STOXX50E.

Interestingly, the asymmetric NGARCH model is not preferred in any of our cases. Seen by the computed loss values and by visual inspection, the NGARCH predictions seem to perform more similarly to the symmetric GARCH-class models even though the NGARCH model is considered asymmetric. As stated in the presentation of the NGARCH framework, in section three, the NGARCH loses its asymmetric response to volatility shocks if the leverage parameter (𝜃) = 0 and is in that case reduced to a standard GARCH model. However, the leverage

(36)

parameter for all NGARCH models ranges between 1.2 and 2.6 and is significantly larger than zero.

Considering the drawbacks and benefits of the individual evaluation measure, it is still found difficult to determine which evaluation measure is more suitable. The symmetric model's out- of-sample forecasts move almost identically, with the asymmetric models fluctuating around the symmetric models. This implies that the drawbacks and benefits of the evaluation measures punish and reward all models very similarly. This is further insinuated by the outcome that the asymmetric models we preferred, at minimum, on seven out of nine market indices, for all evaluation measures.

VOLATILITY FORECASTING PERFORMANCE