Asymptotic distribution of the estimators of a
harmonic component in a multivariate time series
under m-dependence
Zhanna Andrushchenko† Dietrich von Rosen†∗ LiTH-MAT-R-2011/10-SE
† Energy and Technology, Swedish University of Agricultural Sciences, SE–
750 07 Uppsala, Sweden. Corresponding author: Zhanna Andrushchenko. E-mail address: Zhanna.Andrushchenko@yahoo.com
∗ Department of Mathematics, Link¨oping University, SE–581 83 Link¨oping,
Asymptotic distribution of the estimators
of a harmonic component in a multivariate
time series under m-dependence
Zhanna Andrushchenko†and Dietrich von Rosen† ∗
†Energy and Technology,
Swedish University of Agricultural Sciences, SE–750 07 Uppsala, Sweden.
E-mail: Zhanna.Andrushchenko@yahoo.com, dietrich.von.rosen@slu.se.
∗Department of Mathematics,
Link¨oping University, SE–581 83 Link¨oping, Sweden.
Abstract
Multivariate time series with definite harmonic structure is considered, in the special case when the marginal univariate time series are inde-pendent and asymptotically stationary to second order. The asymp-totic distribution of the estimators of a harmonic component under m-dependence is found.
Keywords: Asymptotic distribution; Banded covariance matrix; Es-timators; Harmonic regression; Multivariate normal distribution; Mul-tivariate time series
1
Introduction
One may think of a stationary time series as being formed by a general mean, a harmonic component consisting of a sum of finite number of sinusoids with different angular frequencies and a noise component (see, for example, Brockwell and Davis (1991)). In the case when all the frequencies are known, the model becomes linear in the parameters. If the underlying frequencies are unknown, Schuster’s periodogram can be used to estimate them. Whit-tle (1952) was the first dealing with estimation. He limited the consideration to a zero-mean time series with one harmonic component, and a noise com-ponent being a purely random process with E(ε) = 0, E(ε2) = v < ∞:
xt= A cos(ωt) + B sin(ωt) + ε. (1.1)
Here A and B are unknown parameters, and ω is unknown angular frequency. Walker (1971) formalized and extended Whittle’s results, using a method of estimation which was approximately equivalent to least squares (LS), and approximately equal to maximum likelihood (ML) estimation when ε has a normal distribution so that xt becomes a normal / Gaussian process. He
started from the fact that for the errors being normally distributed, the log-likelihood function of the observation x1, x2, . . . , xn is
L(A, B, ω, σ2) = −n
2log(2πv) − 1
2v S(A, B, ω), (1.2)
where S is the residual sum of squares: S(A, B, ω) = n X t=1 xt− A cos(ωt) − B sin(ωt) 2 , (1.3)
The estimators of A, B and ω are then obtained by minimizing S. Walker (1971) showed that it is easier to proceed by minimizing the function
U (A, B, ω) = n X t=1 x2t− 2 n X t=1 xt A cos(ωt) + B sin(ωt) + n 2(A 2+ B2). (1.4) Since S(A, B, ω) − U (A, B, ω) = 1 2 n X t=1
(A2− B2) cos(2ωt) + 2AB sin(2ωt), (1.5) which is O(1) as n → ∞ (provided ω 6= 0, π), minimizing U instead of S will have negligible effect on the estimators for large n provided that ω0,
Walker (1971) showed that in the case when errors in the model (1.1) are independent and normally distributed, the estimators ˆA, ˆB and ˆω are all consistent as n → ∞. He also found the joint asymptotic distribution of
ˆ
A, ˆB and ˆω .
While in many statistical problems the observations are indeed indepen-dent and iindepen-dentically distributed (i.i.d.), there are a lot of situations when this is not true. In many models, it is too restrictive to assume that the covariance matrix of the errors is diagonal, in which case the observations are neither independent nor identically distributed.
One says that a sequence of random variables (r.v.’s) (X1, X2, . . . , Xn)
is m-dependent if two sets (X1, X2 . . . , Xr), (Xs, Xs+1 . . . , Xn) are
independent, provided s − r > m (Hoeffding and Robbins, 1948). This implies a banded covariance structure obtained by setting all covariances more than m steps apart equal to zero. The m-dependent case includes moving-average time series data, matched-pair data, repeated-measurements data, etc.
Different aspects related to m-dependence were actively studied. A cen-tral limit theorem (CLT) for m-dependent r.v.’s was presented in Hoeffding and Robbins (1948), Diananda (1954), Cocke (1972), Berk (1973), Ferguson (1996), etc. Moricz (1987) presented a law of large numbers, Chen (1997) gave a law of the iterated logarithm. Explicit estimators of the mean and the banded covariance matrix were presented in Andrushchenko et al. (2008) (for m = 1) and Ohlson et al. (2011) (for arbitrary m).
Andrushchenko (2010) dealt with a multivariate time series with m-dependent errors that show a harmonic structure. The consideration was limited to time series that is asymptotically stationary to second order (see Definition A.2 in Appendix A). The model was based on a spectral approach, under the assumption that the observations are multivariate normally dis-tributed. The case when the univariate time series are independent and the covariance matrix of errors is banded of order m was investigated in details. Each univariate time series was analyzed with a spectral method, and esti-mators for the harmonic structure were calculated via a modified LS method. It was also shown that estimators obtained are consistent. Estimation of the banded covariance structure was presented in details as well.
The main aim here is to find asymptotic distribution of explicit estima-tors, obtained in Andrushchenko (2010). This can be done in the same way as in the classical asymptotic theory of estimation. As far as consistency of the estimators is established, the mean value theorem can be applied to yield asymptotic normality with the aid of a central limit theorem.
The paper is organized as follows. Section 2 describes the model and reviews the estimation procedure used. The theorem about consistency of estimators is presented as well. In Section 3, the main theorem, about asymptotic distribution of the estimators, is formulated and proved. Finally, Section 4 summarizes the paper. Some auxiliary results are presented in the
Appendices: Appendix A contains a review of the CLT, both the classical ones and those for the m-dependent case; Appendix B gives details of the proof of Theorem 3.1.
2
Preliminaries
Unless otherwise stated, scalars and matrix elements will be denoted by ordinary letters, vectors by small bold letters, and matrices by capital bold letters.
We assume that the data form a matrix X = (xit) : p × n,
X = (xit) = x11 . . . x1n .. . . .. ... xp1 . . . xpn = x01 .. . x0p , xi= xi1 . . . xin 0 , (2.1)
where i = 1, . . . , p, p is the number of univariate time series and n is the number of observations in each univariate time series. Next, we assume that X follows the matrix normal distribution (Kollo and von Rosen, 2005), namely, X ∼ Np,n(M, Σ, Ψ), where M = (mit) : p × n describes the mean
structure, Σ = (σij) : p × p describes the covariance between rows, and
Ψ = (ψij) : n × n describes the covariance between columns. As usual, we
assume that both covariance matrices are positive definite.
Next, we assume that each univariate time series consists of one har-monic component and a noise component. We allow the amplitudes of all fundamental oscillations and the fundamental frequencies to be different for different univariate time series. If the sampling times are taken to be equally spaced so that t = 1, . . . , n, the model can be written in a matrix form as
X = M + E, (2.2)
where X = (xit) : p × n, M = (mit) : p × n, E = (εit) : p × n, and
mit= Aicos(ωit) + Bisin(ωit), i = 1, . . . , p, t = 1, . . . , n. (2.3)
Here Ai’s and Bi’s are unknown parameters, and ωi are unknown different
frequencies. It is also assumed that errors, εit, are normally distributed, but
are not necessary white noise. As far as we deal with a time series, where nearby observations are correlated, it is reasonably to assume that Ψ has a banded covariance structure of order m, i.e. all covariances more than m steps apart the main diagonal are equal to zero, ψij = 0 for |i − j| > m.
Next, we assume that each row or each univariate time series is independent from others, therefore Σ is I. With such assumptions about the covariance matrices, X ∼ Np,n(M, I, Ψ).
As far as Ai, Bi, ωi are all unknown, the model (2.2) is nonlinear in the
errors has been studied in Andrushchenko (2010). It has been shown that the modified LS techniques can be applied to estimate parameters in each row in model (2.2). As far as the covariance matrix is unknown, the consideration was limited to an unweighted procedure.
For each row (index, indicating the row number, is omitted for conve-nience), Schuster’s periodogram, I(ω), is defined as
I(ω) = 2 n n X t=1 xtexp(iωt) 2 . (2.4)
Here and below i under the expression containing exp(. . .) is the imaginary unit, and for a complex variable z, |z|2= zz∗, where ∗ means conjugate.
The estimators of A and B are obtained by minimizing a modified resid-ual sum of squares, U (A, B, ω), given by (1.4):
ˆ A(ˆω) = 2 n n X t=1 xtcos(ˆωt), B(ˆˆ ω) = 2 n n X t=1 xtsin(ˆωt), (2.5)
with ˆω being estimated by maximizing the periodogram I(ω) (see (2.4)), i.e.,
ˆ
ω = arg max
ω I(ω). (2.6)
Thereafter, the matrix M is estimated via (2.3) as ˆM = (mit( ˆAi, ˆBi, ˆωi)).
Estimation of the banded covariance structure is presented in details in Andrushchenko (2010), and will not be repeated here. The main idea is that, first, the mean (harmonic structure) is estimated, then the estimators ob-tained are inserted in the likelihood function. Next, the likelihood function is presented as the product of marginal and conditional distributions. First, we maximize the first factor in the likelihood function (marginal distribution with unstructured covariance matrix) and estimate the parameters. Those parameters which also appear in the next factor of the likelihood function (conditional part), are replaced by the estimators from the previous part. The estimation proceeds in the same manner until the parameters of the last factor have been estimated. Finally, the following theorem about the consistency of estimators is formulated and proven:
Theorem 2.1. For each row in X, let
xt= A0cos(ω0t) + B0sin(ω0t) + εt (0 < ω0 < π, t = 1, . . . , n), (2.7)
where the index 0 is used to indicate the true value, the εt are m-dependent
normally distributed with E(εt) = 0 and V ar(εt) = v < ∞ , whereas
Cov(εtεt+k) < v < ∞ for |k| ≤ m and Cov(εtεt+k) = 0 for |k| > m. Then
the estimators ˆA, ˆB, ˆω are given by (2.5)-(2.6), and are all consistent as n → ∞, namely:
ˆ
Here, Zn= op(f (n)) means a r.v. such that for any > 0 P |Zn|/f (n) > = 0 as n → ∞. (2.9)
Similarly, Zn= Op(f (n)) if for all > 0 there exist C > 0 such that
P
|Zn| ≤ Cf (n)
≥ 1 − f or all n. (2.10)
It is worth remembering that the index i, indicating the row number, is omitted for convenience. The generic notation ”v” is different for each row i and, formally, v = V ar(εit) ≡ ψii, Cov(εtεt+k) = Cov(εitεi,t+k) ≡ ψi,i+k.
3
Asymptotic distribution
Let us remind that the univariate time series are assumed to be independent from each other, and each univariate time series (each row in X) is analyzed separately. The consideration below is limited to one row at a time, and index i, indicating the row number, is omitted for convenience.
Theorem 3.1. Under the same conditions as in Theorem 2.1,
n1/2( ˆA − A0), n1/2( ˆB − B0), n3/2(ˆω − ω0)
d
→ N (0, 2V W−1), (3.1) as n → ∞, i.e., a multivariate normal distribution with mean (0, 0, 0) and covariance matrix 2V W−1, where
V = X |t−k|≤m Cov(εtεk) = v + 2 m X k=1 Cov(εtεt+k), (3.2) W = 1 0 12B0 0 1 −1 2A0 1 2B0 − 1 2A0 1 3(A20+ B02) . (3.3)
It is worth noting that W12= 0 and therefore ˆA and ˆB are conditionally
independent, given ˆω.
Proof. In general, for U (A, B, ω) defined by (1.4), ∂U ∂A = nA − 2 n X t=1 xtcos(ωt), ∂U ∂B = nB − 2 n X t=1 xtsin(ωt), ∂U ∂ω = 2 n X t=1 xtt A sin(ωt) − B cos(ωt), (3.4)
and U (A, B, ω) has a minimum when A = ˆA, B = ˆB, ω = ˆω.
An application of the mean value theorem to f = ∂U /∂(A, B, ω) gives ∂U ∂A0 = ∂ 2U ∂A2 ∗ A0− ˆA + ∂2U ∂A∗∂B∗ B0− ˆB + ∂2U ∂A∗∂ω∗ ω0− ˆω, ∂U ∂B0 = ∂ 2U ∂A∗∂B∗ A0− ˆA + ∂2U ∂B2 ∗ B0− ˆB + ∂2U ∂B∗∂ω∗ ω0− ˆω, ∂U ∂ω0 = ∂ 2U ∂A∗∂ω∗ A0− ˆA + ∂2U ∂B∗∂ω∗ B0− ˆB + ∂2U ∂ω2 ∗ ω0− ˆω. (3.5)
Here (A∗, B∗, ω∗) denotes a point on the line joining (A0, B0, ω0) and
( ˆA, ˆB, ˆω), such that
(A∗, B∗, ω∗) = λ(A0, B0, ω0) + (1 − λ)( ˆA, ˆB, ˆω) (0 < λ < 1). (3.6)
For the sake of convenience, we denoted ∂U ∂A0 = ∂U ∂A A=A0 , ∂U ∂A∗ = ∂U ∂A A=A∗ , (3.7)
and in the same way for B(0,∗) and ω(0,∗).
The LHS of (3.5) can be transformed to (see Appendix B.1 for details) ∂U ∂A0 = −2 n X t=1 εtcos(ω0t) + Op(1), ∂U ∂B0 = −2 n X t=1 εtsin(ω0t) + Op(1), ∂U ∂ω0 = 2 n X t=1 tεt A0sin(ω0t) − B0cos(ω0t) + Op(n). (3.8)
All the sums in (3.8) are of the formPn
t=1εtkt, with, lim n→∞ 1≤t≤nmax |kt|/ n X t=1 k2t1/2 ! = 0. (3.9)
The condition (3.9) implies the Lindeberg-Feller condition (see Appendix B.1 for details). Therefore the central limit theorem for m-dependent r.v.’s can be applied here (see, for example, Theorem A.5 in Appendix A, adopted from Diananda (1954)), resulting in:
n−1/2 ∂U ∂A0 d → N 0, 2V, n−1/2 ∂U ∂B0 d → N 0, 2V, n−3/2 ∂U ∂ω0 d → N0,2 3V A 2 0+ B02 , (3.10)
as n → ∞. Here V is given by (3.2).
It is easy to show (see Appendix B.2 for details) that the limiting joint distribution is asymptotically normal, namely,
n−1/2 ∂U ∂A0 , n−1/2 ∂U ∂B0 , n−3/2∂U ∂ω0 ! d → N(0, 0, 0), 2V W, (3.11) with W given by (3.3).
Now, let us look at the behavior of the second order partial derivatives occurring on the RHS of (3.5). Using (3.4), it is easy to show that
∂2U ∂A2 ∗ = ∂ 2U ∂B2 ∗ = n, ∂ 2U ∂A∗∂B∗ = 0. (3.12)
The other terms on the RHS of (3.5), after straightforward but tedious calculations (see Appendix B.3 for detail), can be transformed to
1 n2 ∂2U ∂A∗∂ω∗ p → 1 2B0, 1 n2 ∂2U ∂B∗∂ω∗ p → −1 2A0, 1 n3 ∂2U ∂ω2 ∗ p → 1 3(A 2 0+ B02). (3.13) Thus, if W∗ = 1 n ∂2U ∂A2 ∗ 1 n ∂2U ∂A∗∂B∗ 1 n2 ∂ 2U ∂A∗∂ω∗ 1 n ∂2U ∂A∗∂B∗ 1 n ∂2U ∂B2 ∗ 1 n2 ∂ 2U ∂B∗∂ω∗ 1 n2 ∂ 2U ∂A∗∂ω∗ 1 n2 ∂ 2U ∂B∗∂ω∗ 1 n3∂ 2U ∂ω2 ∗ , (3.14) then W∗ p → W, (3.15)
with W given by (3.3). It is easy to verify that |W| = 121 (A20+ B02) 6= 0, therefore W is nonsingular. As far as (3.15) implies that g(W∗)
p
→ g(W) for any continuous function g, then |W∗|
p
→ |W| 6= 0. Hence, with the probability tending to 1 as n → ∞, W∗ will be nonsingular too (by (3.15)).
From (3.5), we see that n−1/2∂U ∂A0 , n−1/2 ∂U ∂B0 , n−3/2∂U ∂ω0 ! = − n1/2( ˆA − A0), n1/2( ˆB − B0), n3/2(ˆω − ω0) W∗. (3.16)
Therefore n1/2( ˆA − A0), n1/2( ˆB − B0), n3/2(ˆω − ω0) = − n−1/2 ∂U ∂A0 , n−1/2 ∂U ∂B0 , n−3/2∂U ∂ω0 ! W∗−1. (3.17)
Using an generalization of the Cram´er-Slutsky theorems (see, for example, Rao (1973), p.122 (xb)), if Yn and Y are row vector-valued r.v. and Zn is
a matrix-valued r.v., such that Yn d → Y and Zn→ C,p (3.18) then YnZn d → YC. (3.19)
We have shown above (see (3.11)) that the row vector on the RHS of (3.17) converges in distribution to N (0, 0, 0), 2V W when n → ∞. Us-ing (3.15), (3.19) implies that the RHS of (3.17) converges in distribution to N (0, 0, 0), 2V W−1 when n → ∞. Therefore the row vector on the LHS of (3.17) converges in distribution to N (0, 0, 0), 2V W−1.
The asymptotic distribution of A, ˆˆ B, ˆω can be found by calculating the inverse of W explicitly. Namely,
W−1 = 1 A20+ B20 A20+ 4B02 −3A0B0 −6B0 −3A0B0 4A20+ B02 6A0 −6B0 6A0 12 . (3.20)
Corollary 3.1. The asymptotic distribution of A, ˆˆ B, ˆω is asymptotically equivalent to the multivariate normal distribution with mean (A0, B0, ω0)
and covariance matrix 2V A2 0+ B02 n−1(A2 0+ 4B02) −3n−1A0B0 −6n−2B0 −3n−1A0B0 n−1(4A20+ B02) 6n−2A0 −6n−2B 0 6n−2A0 12n−3 , (3.21) with V given by (3.2).
4
Conclusions
In this paper, we have presented the estimators of the parameters of har-monic structure in a multivariate time series with m-dependent errors. We have shown that for each univariate time series, the asymptotic distribution of the estimators is asymptotically multivariate normal with the mean equal
to the true values of estimators. The covariance matrix of the estimators is found to be a function of both the true values of harmonic structure and covariance matrix of observations.
Acknowledgement
Z.A. will acknowledge the hospitality of Prof. J.W.Silverstein at the De-partment of Mathematics, North Carolina State University, USA, where the work has been partly performed.
Z.A. will also acknowledge the support provided by the Royal Swedish Academy of Sciences (travel grant from Stiftelsen G.S.Magnusons fond).
A
Central Limit Theorems and m-dependance
Theorem A.1 (Theorem 9.7.1 (CLT for iid r.v.’s), Resnick (2001)). Let {Xn, n > 1} be iid r.v.’s with E(Xn) = µ and V ar(Xn) = σ2. If
Sn= X1+ . . . + Xn, then
Sn− nµ
σ√n
d
→ N (0, 1), (A.1)
i.e., standard normal distribution.
Theorem A.2 (Theorem 9.8.1 (Lindeberg-Feller CLT),
Resnick (2001)). Let {Xn, n > 1} be independent (but not necessarily
identically distributed) and suppose Xk has distribution Fk, and E(Xk) = 0
and V ar(Xk) = σk2. Define Sn = X1+ . . . + Xn and s2n = σ21 + . . . + σn2 =
V ar Pn
i=1Xi. One says that {Xk} satisfies the Lindeberg condition, if for
all δ > 0 as n → ∞ one has 1 s2 n n X i=1 E Xk2 1[ |Xk/sn|>δ] = 1 s2 n n X i=1 Z |x|>δsn x2Fi(dx) → 0. (A.2)
1[...] here denotes the indicator function. The Lindeberg condition (A.2) implies
Sn
sn d
→ N (0, 1). (A.3)
Definition A.1 (Hoeffding and Robbins (1948)). Let X1, X2 . . . be
a sequence of random variables. If for some function f (n) the inequality s − r > f (n) implies that the two sets
(X1, X2 . . . , Xr), (Xs, Xs+1 . . . , Xn) (A.4)
are independent, then the sequence (A.4) is said to be f (n)-dependent. An important special case, m-dependance, occurs when f (n) = m is independent of n. In particular, 0-dependance is equivalent to independence.
Theorem A.3 (Hoeffding and Robbins (1948)). Let X1, X2 . . . be an
m-dependent sequence of random variables such that
EXi=0, EXi2< ∞, E|Xi|3 ≤ R3 < ∞, (A.5) lim p→∞p −1 p X h=1
Ai+h=A exists, unif ormly f or all i = 0, 1, . . . , (A.6)
where
Ai = EXi+m2 + 2 m
X
j=1
EXi+m−jXi+m (i = 1, 2, . . .). (A.7)
Then, as n → ∞ the random variable n−1/2(X1+ . . . + Xn) has a limiting
Definition A.2 (Diananda (1954)). A sequence of r.v.’s {Xi}, (i =
1, 2, 3, . . .) is said to be
a) stationary to second order if, for every integral k, the expectations E(Xi),
E(XiXi+k) (i > 0, i + k > 0) exist and are independent of i;
b) asymptotically stationary to second order if, for every integer k, the expectations E(Xi), E(XiXi+k) (i > 0, i + k > 0) exist and have finite
limits as i → ∞;
c) stationary if the distribution function of (Xi+1, Xi+2, . . . , Xi+k) is
inde-pendent of i, for every positive integer k.
Theorem A.4 (Theorem 3, Diananda (1954)). Let {Xi}, i = 1, 2, 3, . . .
be a sequence of m-dependent r.v.’s stationary to second order with zero means and finite variances.
Define Sn = X1+ . . . + Xn and s2n = V ar
Pn
i=1Xi. Suppose that Xi
has distribution Fi, and for every integer p,
E(XiXi+p) = Cp (i > 0, i + p > 0) (A.8)
and that, for every positive δ, 1 n n X i=1 Z |x|>δ√n x2dFi→ 0, (A.9) or 1 s2 n n X i=1 Z |x|>δsn x2dFi→ 0. (A.10) Then the d.f. of Sn/ √
n converges in distribution to the normal d.f. with mean 0 and variancePm
−mCp.
Theorem A.5 (Theorem 4, Diananda (1954)). Let {Xi}, i = 1, 2, 3, . . .
be a sequence of m-dependent r.v.’s asymptotically stationary to second or-der with zero means and finite variances. Then Theorem A.4, with (A.8) replaced by
E(XiXi+p) → Cp, (A.11)
B
Details of the proof of Theorem 3.1
B.1 LHS of (3.5)
Using (3.4), the LHS of (3.5) can be transformed to ∂U ∂A0 = nA0− 2 n X t=1 xtcos(ω0t) = nA0− 2 n X t=1 A0cos(ω0t) + B0sin(ω0t) + εt cos(ω0t) = nA0− A0 n X t=1 1 + cos(2ω0t) + B0 n X t=1 sin(2ω0t) + 2 n X t=1 εtcos(ω0t) = −2 n X t=1 εtcos(ω0t) + O(1), (B.1) ∂U ∂B0 = nB0− 2 n X t=1 xtsin(ω0t) = nB0− 2 n X t=1 A0cos(ω0t) + B0sin(ω0t) + εt sin(ω0t) = nB0− A0 n X t=1 sin(2ω0t) + B0 n X t=1 1 − cos(2ω0t) + 2 n X t=1 εtsin(ω0t) = −2 n X t=1 εtsin(ω0t) + O(1), (B.2) ∂U ∂ω0 = 2 n X t=1 xtt A0sin(ω0t) − B0cos(ω0t) = 2 n X t=1 tA0cos(ω0t) + B0sin(ω0t) + εt A0sin(ω0t) − B0cos(ω0t) = n X t=1 t(A20− B2 0) sin(2ω0t) − 2A0B0cos(2ω0t) + 2 n X t=1 tεt A0sin(ω0t) − B0cos(ω0t) = 2 n X t=1 tεt A0sin(ω0t) − B0cos(ω0t) + O(n). (B.3)
All the sums in (B.1)-(B.3) are of the formPn t=1εtkt, with, correspondingly, n X t=1 k2t (B.1) = n X t=1 2 cos(ω0t) 2 = 2 n X t=1 1 + cos(2ω0t) = 2n + O(1), (B.4) n X t=1 k2t (B.2) = n X t=1 2 sin(ω0t) 2 = 2 n X t=1 1 − cos(2ω0t) = 2n − O(1), (B.5) n X t=1 k2t (B.3) = n X t=1 2t A0sin(ω0t) − B0cos(ω0t) 2 = 4 n X t=1 t2
A20sin2(ω0t) + B20cos2(ω0t) − A0B0sin(2ω0t)
= 2 A20+ B02 n X t=1 t2+ O(n2) = 2 3n 3 A2 0+ B02 + O(n2). (B.6)
It is easy to see that for all the sums the following condition holds: lim n→∞ 1≤t≤nmax |kt|/ n X t=1 k2t1/2 ! = 0. (B.7)
This condition implies the Lindeberg-Feller condition. To show this, let ηt=
ktεtand Gtbe the distribution functions of ηt. Then E(ηt) = 0, V ar(ηt) =
E(ηt2) = k2t V ar(εt) = v k2t. Let s2n= V ar
Pn t=1ηt = v Pnt=1k2t. Then |kt| Pn t=1kt2 1/2 !2 = vk 2 t vPn t=1k2t = 1 s2 n E(ηt2) = 1 s2 n E ηt21[ |ηt/sn|≤δ] + 1 s2 n E η2t 1[ |ηt/sn|>δ] ≤ δ2+ 1 s2 n n X t=1 Z |η|>δsn η2dGt(η). (B.8)
As far as (B.7) holds, (B.8) implies the Lindeberg-Feller condition (A.10). Therefore the central limit theorem for m-dependent r.v.’s can be applied here (see, for example, Theorem A.5 in Appendix A), resulting in the marginal distributions given by (3.10).
B.2 The joint limiting distribution
To find the joint limiting distribution, let us consider a r.v. Z(λ1, λ2, λ3) = λ1n−1/2 ∂U ∂A0 ! + λ2n−1/2 ∂U ∂B0 ! + λ3n−3/2 ∂U ∂ω0 ! , (B.9)
where λ1, λ2, λ3 are arbitrary real numbers. Then Z(λ1, λ2, λ3) = n X t=1 2εt λ3n−3/2t A0sin(ω0t) − B0cos(ω0t)
− n−1/2 λ1cos(ω0t) + λ2sin(ω0t) + O(n−1/2)
. (B.10) The sum here is of the formPn
t=1εtkn,t, with lim n→∞ Xn t=1 kn,t2 = lim n→∞ 4 λ3n−3/2t A0sin(ω0t) − B0cos(ω0t) − n−1/2 λ1cos(ω0t) + λ2sin(ω0t) 2 ! = 2 λ21+ λ22 +2 3 A 2 0+ B02λ23+ 2B0λ1λ3− 2A0λ2λ3. (B.11) Therefore, lim n→∞ 1≤t≤nmax |kn,t|/ n X t=1 kn,t2 1/2 ! = lim n→∞ O(n−1/2) O(1) ! = 0. (B.12)
The condition (B.12) means that by the Lindeberg-Feller condition, the central limit theorem can be applied here. Thus, Z(λ1, λ2, λ3) converges in
distribution to a normal distribution with mean zero and variance given by the RHS of (B.11).
By a definition of multivariate normal distribution (see, for example, Srivastava (2002), Definition 2.5.1), a p-dimensional r.v. x has a multivariate normal distribution iff every linear combination of x has a univariate normal distribution, namely, if for any given vector λ, λ0x ∼ Np(λ0µ, λ0Σλ), then
x must be Np(µ, Σ). This was first proved by Cram´er (1937). Applying
this argument here, it is easy to show that the joint limiting distribution is asymptotically normal, namely,
n−1/2 ∂U ∂A0 , n−1/2 ∂U ∂B0 , n−3/2∂U ∂ω0 ! d → N(0, 0, 0), 2V W0 , (B.13)
with V given by (3.2) and W0 by (3.3).
B.3 RHS of (3.5)
Now, let us look at the behavior of the second order partial derivatives occurring on the RHS of (3.5). It is obvious that
∂2U ∂A2 ∗ = ∂ 2U ∂B2 ∗ = n, ∂ 2U ∂A∗∂B∗ = 0, (B.14)
with A∗, B∗, ω∗ being given by (3.6). Let us denote
u±= ω∗± ω0, (B.15)
and use (2.7). Then ∂2U ∂A∗∂ω∗ = 2 n X t=1 txtsin(ω∗t) (B.16) = 2 n X t=1 tA0cos(ω0t) + B0sin(ω0t) + εt sin(ω∗t) = A0 n X t=1 t sin(u−t) + sin(u+t) + B0 n X t=1 t cos(u−t) − cos(u+t) + 2 n X t=1 tεtsin(ω∗t), (B.17) ∂2U ∂B∗∂ω∗ = −2 n X t=1 txtcos(ω∗t) = −A0 n X t=1 t cos(u+t) + cos(u−t) − B0 n X t=1 t sin(u+t) − sin(u−t) − 2 n X t=1 tεtcos(ω∗t), (B.18) ∂2U ∂ω2 ∗ = 2 n X t=1 t2xt A∗cos(ω∗t) + B∗sin(ω∗t) = 2 n X t=1 t2 A0cos(ω0t) + B0sin(ω0t) + εt A∗cos(ω∗t) + B∗sin(ω∗t) = A0A∗+ B0B∗ n X t=1 t2cos(u−t) + A0B∗− A∗B0 n X t=1 t2sin(u−t) + A0A∗− B0B∗ n X t=1 t2cos(u+t) + A0B∗+ A∗B0 n X t=1 t2sin(u+t) + 2A∗ n X t=1 t2εtcos(ω∗t) + 2B∗ n X t=1 t2εtsin(ω∗t). (B.19)
To proceed with the estimation, the theorem about consistency (Theorem 2.1) and (3.6) will be taken into account. As result,
Next, let us introduce a new function, h(u), h(u) = n X t=1 exp(iut) = ( sin(nu/2) sin(u/2) exp i(n+1)u 2 (0 < u < 2π), n (u = 0 or 2π). (B.21)
According to (B.21) and since 0 < ω0 < π,
max
0≤ω≤π|h(u−)| = n, 0≤ω≤πmax |h(u+)| = O(1), (B.22)
To estimate n X t=1 t sin(u±t) and n X t=1 t cos(u±t),
let us look on the < and = parts of Pn
t=1t exp(iut). It is clear that n X t=1 t exp(iut) = −i ∂ ∂u n X t=1 exp(iut) = −i ∂
∂uh(u) = −ih
0(u). (B.23)
Applying the mean value theorem to h0(u−) yields h0(u−)−h0(0) = h00(˜u)u−,
where ˜u is some value, 0 ≤ ˜u ≤ u−. We know that h0(0) = iPnt=1t =
in(n+1)2 , and |h00(˜u)| = | −Pn
t=1t2exp(i˜ut)| ≤
Pn
t=1t2 = O(n3) for all ˜u.
Therefore, h0(u−) = h0(0) + h00(˜u)u−= i n(n + 1) 2 − op(n 2), h0(u +) = O(n). (B.24)
Here we took into account (B.20) and (B.22). Finally,
n X t=1 t sin u−t = op(n2), n X t=1 t cos u−t = n(n + 1) 2 , (B.25) n X t=1 t sin u+t = O(n), n X t=1 t cos u+t = O(n). (B.26) To estimate n X t=1 t2sin(u±t) and n X t=1 t2cos(u±t),
let us look on the < and = parts of Pn
t=1t2exp(ut). Again, n X t=1 t2exp(iut) = − ∂ 2 ∂u2h(u) = −h 00(u), (B.27)
Applying the mean value theorem to h00(u−) yields h00(u−)−h00(0) = h000(˜u)u−,
with h00(0) = iPn t=1t2= − n(n+1)(2n+1) 6 , and |h 000(˜u)| = |−Pn t=1t3exp(i˜ut)| ≤ Pn
t=1t3 = O(n4) for all ˜u. Therefore,
h00(u−) = h00(0) + h000(˜u)u−
= −n(n + 1)(2n + 1)
6 + iop(n
3), h00(u
Finally, n X t=1 t2sin u−t = op(n3), n X t=1 t2cos u−t = n(n + 1)(2n + 1) 6 , (B.29) n X t=1 t2sin u+t = O(n2), n X t=1 t2cos u+t = O(n2). (B.30)
Next, let us look onPn
t=1tεtexp(iω∗t). The same way as in Andrushchenko
(2010, p.10), n X t=1 tεtexp(iωt) 2 = n X t=1 tεtexp(iωt) n X τ =1 τ ε∗τexp(−iωτ ) = n X t=1 n X τ =1 tτ εtετexp(iω(t − τ )) = n X t=1 t2ε2t+ 2 n−s X t=1 n−1 X s=1 t(t + s)εtεt+sexp(−iωs) ≤ n X t=1 t2ε2t+ 2 n−1 X s=1 n−s X t=1 t(t + s)εtεt+s , (B.31)
with the expectation being E n X t=1 tεtexp(iωt) 2 ≤ E n X t=1 t2ε2t + 2 n−1 X s=1 E n−s X t=1 t(t + s)εtεt+s C−S ≤ E n X t=1 t2ε2t + 2 n−1 X s=1 E n−sX t=1 t(t + s)εtεt+s 21/2 m−dep. = E n X t=1 t2ε2t+ 2 m X s=1 E n−s X t,τ =1 t(t + s)εtεt+sτ (τ + s)ετετ +s 1/2 ≤ vn 3 3 + 2 m X s=1 Xn−s t=1 t2(t + s)2 1/2 . (B.32)
Here, C−S means ”by the Cauchy-Schwarz inequality”, and m−dep. means
”taking into account m-dependence”. As far as
m X s=1 n−sX t=1 t2(t + s)2 1/2 ≤ m X s=1 (n − s)5/2≤ Z n n−m y5/2dy ≤ mn5/2, (B.33) then E n X t=1 tεtexp(iω∗t) 2 ≤ vn3+ m O(n5/2). (B.34)
Assuming large n and reasonable m, applying the Markov inequality and using the definition of Op(n), this implies that
n X t=1 tεtexp(iω∗t) = Op(n 3/2). (B.35) For large m, Pn t=1tεtexp(iω∗t) = Op(n
7/4). Clearly, that in both cases,
Pn t=1tεtexp(iω∗t) < Op(n 2).
Next, let us look on Pn
t=1t2εtexp(iω∗t). The same way as before,
n X t=1 t2εtexp(iωt) 2 = n X t=1 t2εtexp(iωt) n X τ =1 τ2ε∗τexp(−iωτ ) = . . . ≤ n X t=1 t4ε2t + 2 n−1 X s=1 n−s X t=1 t2(t + s)2εtεt+s , (B.36)
with the expectation being
E n X t=1 t2εtexp(iωt) 2 ≤ E n X t=1 t4ε2t+ 2 n−1 X s=1 E n−s X t=1 t(t + s)εtεt+s ≤ . . . ≤ vn 5 5 + 2 m X s=1 n−sX t=1 t4(t + s)4 1/2 . (B.37) As far as m X s=1 n−sX t=1 t4(t + s)41/2≤ m X s=1 (n − s)9/2≤ Z n n−m y9/2dy ≤ mn9/2, (B.38) then E n X t=1 t2εtexp(iω∗t) 2 ≤ vn5+ m O(n9/2) . (B.39)
Assuming large n and reasonable m, applying the Markov inequality and using the definition of Op(n) implies that
n X t=1 t2εtexp(iω∗t) = Op(n 5/2). (B.40) For large m, Pn t=1t2εtexp(iω∗t) = Op(n
11/4). Clearly, in both cases
Pn t=1t2εtexp(iω∗t) < Op(n 3).
Therefore, after combining all the terms, (B.16)-(B.19) are transformed to 1 n2 ∂2U ∂A∗∂ω∗ p → 1 2B0, (B.41) 1 n2 ∂2U ∂B∗∂ω∗ p → −1 2A0, (B.42) 1 n3 ∂2U ∂ω2 ∗ p → 1 3(A 2 0+ B02). (B.43)
References
Andrushchenko, Z. (2010). Estimation of harmonic component and banded covariance matrices in a multivariate time series. Centre of Biostochastics, Swedish University of Agricultural Sciences. Report 2010:5.
Andrushchenko, Z., Ohlson, M., and von Rosen, D. (2008). Estimation of banded covariance matrices in a multivariate normal distribution. Centre of Biostochastics, Swedish University of Agricultural Sciences. Report 2008:2.
Berk, K. (1973). A central limit theorem for m-dependent random variables with unbounded m. The Annals of Probability, 1:352–354.
Brockwell, P. J. and Davis, R. A. (1991). Time Series: Theory and Methods. Springer, New York, 2nd edition.
Chen, X. (1997). The law of the iterated logarithm for m-dependent banach space valued random variables. Journal of Theoretical Probability, 10:695– 732.
Cocke, W. J. (1972). Central limit theorems for sums of dependent vector variables. The Annals of Mathematical Statistics, 43:968–976.
Cram´er, H. (1937). Random Variables and Probability Distributions. bridge Tracts in Mathematics, No 36. Cambridge University Press, Cam-bridge, England.
Diananda, P. H. (1954). The central limit theorems for m-dependent vari-ables asymptotically stationary to second order. Proc. Cambridge Philos. Soc., 50:287–292.
Ferguson, T. S. (1996). A course in large sample theory. Chapman and Hall, London.
Hoeffding, W. and Robbins, H. (1948). The central limit theorem for de-pendent random variables. Duke Math. J., 15:773–780.
Kollo, T. and von Rosen, D. (2005). Advanced Multivariate Statistics with Matrices. Springer, Dordrecht.
Moricz, F. (1987). Strong limit theorems for blockwise m-dependence and blockwise quasiorthogonal sequences of random variables. Proc. Amer. Math. Society, 101:709–715.
Ohlson, M., Andrushchenko, Z., and von Rosen, D. (2011). Explicit estima-tors under m-dependence for a multivariate normal distribution. Annals of the Institute of Statistical Mathematics, 63(1):29–42.
Rao, C. R. (1973). Linear statistical inference and its applications. Wiley, New York, USA, 2nd edition.
Resnick, S. I. (2001). A probability pass. Birkh¨auser, Boston, USA, 2nd edition.
Srivastava, M. S. (2002). Methods of Multivariate Statistics. Wiley-Interscience, New York, USA.
Walker, A. M. (1971). On the estimation of a harmonic component in a time series with stationary independent residuals. Biometrika, 58:21–36. Whittle, P. (1952). The simulttaneous estimation of a time series harmonic
