On the Saddlepoint Approximation for the Residual Sum of Squares in Special Linear Heteroscedastic Models

(1)

On the Saddlepoint Approximation for the Residual Sum of Squares in Special Linear Heteroscedastic Models

Viktor Sn´ıˇzek

Examensarbete i matematisk statistik, 20 po¨ang Handledare: Silvelyn Zwanzig

Maj 2003

Matematiska institutionen

Uppsala universitet

(2)

Abstract

There are several aims of this work. First some simplifications of the general saddlepoint approximation have to be done. Secondly the conditions for the explicit solution of the saddlepoint equations are discussed and finally the simulations are performed and compared to the approximations.

4

(5)

Chapter 2

Introduction

This paper is based on results of Saddlepoint approximation (SA) of RSS published in Journal of Multivariate Analysis by Alexander V. Ivanov and Silvelyn Zwanzig under the title Saddlepoint Expansions in Linear Regression.

The main aim is to check out the saddlepoint approximation. It is necessary to find simplifications of the theoretical SA for some special models and to proof the theoretical formulas by simulations.

The solution of the saddlepoint equations is explicitly known only for the homoscedastic model. In this case it’s also known that the RSS is χ²(n − p) distributed. We are interested in finding the solution of the sad- dlepoint equations in the heteroscedastic model. The results are summed up in Theorem 2 where necessary conditions are presented. Under these conditions the solution of the saddlepoint equations can be explicitly solved for p components, only one component is expressed implicitly. This essen- tially simplify the approximation formulas in the paper of A. V. Ivanov and S. Zwanzig. In Theorem 3 is shown that p-sample model fulfills conditions of Theorem 2 if the weights and variances in each sample are equal. But weights and variances in different samples can be different.

The second part of the paper will give us the answer if the theoretical research is on the correct way. The simulations use the 2-sample regression linear model with different starting conditions. After the series of simulations we discovered some practical problems with the saddlepoint approximation that we should be aware of. Such problems can appear when we have just a small sample size of the data. The SA function gives big functional values for increasing negative values. Nevertheless this is not the problem of the theory but the computation. The simulation chapter shows ways how to solve it. Summarizing, I found the saddlepoint approximation as a really good approximation for the 2-sample case.

5

(6)

Thanks

This paper was written during my study stay at the Uppsala university at the Mathematical institution. I would like to express my warmest thanks to Silvelyn Zwanzig who advised me the whole year and the co-operation with her has always taken place in a very friendly and nice atmosphere.

The stay was only possible thanks to the Svenska institutet’s scholarship and the co-operation between it and the Mathematical institution namely Allan Gut who become my contact professor.

Thanks to J¨org Polzehl, Weierstraß Institute for Applied Analysis and Stochastics, Berlin, for the R procedure convolute and for advice with the kernel density estimator.

6

(7)

Chapter 4

Linear model

4.1 Heteroscedastic linear model

Consider the heteroscedastic linear regression model as:

y_i,n= x^T_i,nβ + ε_i,n, (4.1) where

ε_i,n= σ_i,n²_i, ²_i ∼ N (0, 1) i.i.d. (4.2)

y_i,n are observations. The n × p design matrix is given by X =



 x^T₁

... x^T_n



.

β is a p × 1 vector of fixed unknown parameters.

Further we suppose W = (w_ij) to be a known symmetric diagonal n × n matrix of weights, where 0 < w_min< w_i,n< w_max.

We should also have the variances bounded using the condition:

w_i,nσ_i,n² = 1 + ∆²_i,n, (4.3) where max

1≤i≤n∆²_i,n≤ ∆² < ∞.

In the homoscedastic case we are using the simplified form:

w_i,nσ_i,n² = 1 + ∆²_n, (4.4) with ∆²_n≤ ∆²< ∞.

Let ¯σ_n² = _n¹ Pⁿ

i=1

w_i,nσ²_i,n.

7

(8)

Let’s present two regularity conditions:

There exist constants λ_x> 0 and c_x< ∞ such that lim_n→∞λ_min

Ã 1 n

Xn i=1

x_i,nx^T_i,n

!

> λ_x, (4.5)

and lim_n→∞max

i≤n kx_ik_max< c_x. (4.6) Under (4.5) and (4.6) there exist a positive constant λ₀ = λ_xw_min and a bounded constant c₀= c_xw_max such that

λ_minI_(n)> λ₀ and tr(I_(n)) < c₀, where

I_(n)= 1

nX^TW X = 1 n

Xn i=1

w_i,nx_i,nx^T_i,n (4.7) is a weighted Fisher matrix.

The weighted residual sum of squares is:

RSS = min

β

Xn i=1

w_i,n(y_i,n− x^T_i,nβ)².

Theorem 1. Under assumptions of (4.2), (4.5), (4.6) it holds

P µ√

n µ1

nRSS − ¯σ²_n

¶

> u

¶

= Z∞ u

Z

f_het(y)(1 + R_n(y))dy₂dy₁

with y = µ y₁

y₂

¶

∈ R^p+1, y₁∈ R, y₂ ∈ R^p and with f_het(y) given in (4.11);

sup_y∈Y|R_n(y)| = O(n⁻¹) for Y = n

y : kˆθ(y)k ≤ c o

, where ˆθ(y) ∈ R^p+1 is the exact solution of (4.18)

Proof. The proof of this theorem is shown in [1] (p. 203). ¥ Lemma 1. Let

S_i(θ₁) = −1

2ln(1 − 2w_i,nσ_i,n² θ₁).

Then

s_i(θ₁) = w_i,nσ_i,n²

1 − 2w_i,nσ²_i,nθ₁ = S_i⁰(θ₁), (4.8) s⁰_i(θ₁) = 2s²_i(θ₁), (4.9) s⁰⁰_i(θ₁) = 8s³_i(θ₁). (4.10) The prime denotes the derivation with respect to θ₁.

(9)

4.1. HETEROSCEDASTIC LINEAR MODEL 9

Proof.

S_i⁰(θ₁) = ∂

∂θ₁ µ

−1

2ln(1 − 2w_i,nσ_i,n² θ₁)

¶

=

= −1 2

1

1 − 2w_i,nσ_i,n² θ₁(−2w_i,nσ²_i,n) = s_i(θ₁), s⁰_i(θ₁) = ∂

∂θ₁

Ã w_i,nσ²_i,n 1 − 2w_i,nσ²_i,nθ₁

!

=

= −w_i,nσ_i,n²

(1 − 2w_i,nσ_i,n² θ₁)²(−2w_i,nσ_i,n² ) = 2s²_i(θ₁).

¥ The saddlepoint approximation density looks like:

f_het(y) = exp (κ(ˆθ) −√

nˆθ^TF⁻¹(y))

(2π)^p+1² |C_het(ˆθ)|¹² , (4.11) where

κ_het(θ) = −n¯σ²_nθ₁+ Xn i=1

S_i(θ₁) + nθ₂^TI_(n)⁻¹B(θ₁)I_(n)⁻¹θ₂, (4.12)

with θ = µ θ₁

θ₂

¶

∈ R^p+1, θ₁ ∈ R, θ₂ ∈ R^p.

S_i(θ₁), s_i(θ₁) and s⁰_i(θ₁) are defined in Lemma 1.

B(θ₁) = 1 n

Xn i=1

s_i(θ₁)

2 w_i,nx_i,nx^T_i,n, (4.13)

s(θ₁) = 1 n

Xn i=1

s_i(θ₁). (4.14)

F⁻¹(y) = Ã

y₁+ n⁻¹²y₂^TI_(n)y₂ y₂

!

. (4.15)

C_het(θ) =

Ã s⁰(θ₁) + θ₂^TI_(n)⁻¹B⁰⁰(θ₁)I_(n)⁻¹θ₂ 2θ^T₂I_(n)⁻¹B⁰(θ₁)I_(n)⁻¹ 2I_(n)⁻¹B⁰(θ₁)I_(n)⁻¹θ₂ 2I_(n)⁻¹B(θ₁)I_(n)⁻¹

!

. (4.16)

The saddlepoint ˆθ = ˆθ(y) ∈ R^p+1 is defined as the exact solution of the equation system

m_het(ˆθ) = F⁻¹(y), (4.17)

(10)

where

m_het(ˆθ) =√ n

Ã −¯σ²_n+ s(ˆθ₁) + ˆθ₂^TI_(n)⁻¹B⁰(ˆθ₁)I_(n)⁻¹θˆ₂ 2I_(n)⁻¹B(ˆθ₁)I_(n)⁻¹θˆ₂.

!

. (4.18)

To summarize all the previous formulas connected to the saddlepoint density in (4.11) we have:

exp (κ(ˆθ) −√

nˆθ^TF⁻¹(y)) = exp(−n¯σ_n²θˆ₁+Pⁿ

i=1

S_i(ˆθ₁) + nˆθ^T₂I_(n)⁻¹B(ˆθ₁)I_(n)⁻¹θˆ₂

−n

³θˆ₁(−¯σ_n²+ s(ˆθ₁) + ˆθ₂^TI_(n)⁻¹B⁰(ˆθ₁)I_(n)⁻¹θˆ₂) + 2ˆθ^T₂I_(n)⁻¹B(ˆθ₁)I_(n)⁻¹θˆ₂

´ ) =

= exp(Pⁿ

i=1

S_i(ˆθ₁) − nˆθ₁s(ˆθ₁) − nˆθ₂^TI_(n)⁻¹

³

B(ˆθ₁) + ˆθ₁B⁰(ˆθ₁)

´

I_(n)⁻¹θˆ₂) (4.19) and using the following formula for the determinant calculation of the matrix divided into segments

|C| = |C₂₂||C₁₁− C₁₂C₂₂⁻¹C₁₂^T| (4.20) for a C matrix C =

µ C₁₁ C₁₂ C₁₂^T C₂₂

¶ , we have:

|C_het(ˆθ)| = |2I_(n)⁻¹B(ˆθ₁)I_(n)⁻¹|(s⁰(ˆθ₁) + ˆθ^T₂I_(n)⁻¹B⁰⁰(ˆθ₁)I_(n)⁻¹θˆ₂

−4

2θˆ^T₂I_(n)⁻¹B⁰(ˆθ₁)I_(n)⁻¹I_(n)B⁻¹(ˆθ₁)I_(n)I_(n)⁻¹B⁰(ˆθ₁)I_(n)⁻¹θˆ₂) =

= |2I_(n)⁻¹B(ˆθ₁)I_(n)⁻¹|(s⁰(ˆθ₁) + ˆθ^T₂I_(n)⁻¹

³

B⁰⁰(ˆθ₁) − 2B⁰(ˆθ₁)B⁻¹(ˆθ₁)B⁰(ˆθ₁)

´

I_(n)⁻¹θˆ₂).

(4.21)

4.2 Saddlepoint computation in a special case

In this section we try to solve the system of the saddlepoint equation in (4.17) for the heteroscedastic model and to find out the conditions for finding the explicit solution ˆθ.

Theorem 2. Provided that following conditions are fulfilled 1. B⁰(θ₁) = 4B(θ₁)I_(n)⁻¹B(θ₁) ∀θ₁ ∈ Θ,

2. B⁻¹(ˆθ₁) = 2I_(n)⁻¹ΣI_(n)⁻¹− 4ˆθ₁I_(n)⁻¹, for ˆθ₁ solved in (4.17) and Σ inde- pendent of ˆθ

it follows that

(11)

4.2. SADDLEPOINT COMPUTATION IN A SPECIAL CASE 11 1. θˆ₁: s(ˆθ₁) = y₁

√n + ¯σ_n² (4.22) is the first component (implicit),

2. and

θˆ₂ = 1 2√

nI_(n)B⁻¹(ˆθ₁)I_(n)y₂ (4.23) is the second, p-dimensional component (explicit) of the solution of (4.17)

3. the saddlepoint density in (4.11) can be split into

f_spec(y₁, y₂) = g(ˆθ₁(y₁))ϕ_0,Σ(y₂) (4.24) with

ϕ_0,Σ(y₂) = exp(−¹₂y₂^TΣy₂)

(2π)^p²|Σ|⁻¹² , the density of N_p(0, Σ), (4.25) and

g(ˆθ₁(y₁)) =

exp(Pⁿ

i=1

S_i(ˆθ₁) − nˆθ₁˜s)

(2π|C_specΣ|)¹² , (4.26)

where

|C_spec| = s⁰(ˆθ₁)|2I_(n)⁻¹B(ˆθ₁)I_(n)⁻¹|. (4.27) Proof. We will compare left and right side of the system

Ã

y₁+ n⁻¹²y₂^TI_(n)y₂ y₂

!

=√ n

Ã −¯σ_n²+ s(ˆθ₁) + ˆθ₂^TI_(n)⁻¹B⁰(ˆθ₁)I_(n)⁻¹θˆ₂ 2I_(n)⁻¹B(ˆθ₁)I_(n)⁻¹θˆ₂.

!

(4.28) and we directly get that

y₂ = 2√

nI_(n)⁻¹B(ˆθ₁)I_(n)⁻¹θˆ₂, (4.29) equivalently: ˆθ₂ = 1

2√

nI_(n)B⁻¹(ˆθ₁)I_(n)y₂. (4.30) We’ll put the result in (4.29) into the first term on the left in (4.28) and compare to the first term on the right side:

y₁+ 4√

nˆθ^T₂I_(n)⁻¹B(ˆθ₁)I_(n)⁻¹B(ˆθ₁)I_(n)⁻¹θˆ₂ = −√

n¯σ_n²+√ ns(ˆθ₁)

+ √

nˆθ^T₂I_(n)⁻¹B⁰(ˆθ₁)I_(n)⁻¹θˆ₂.(4.31)

(12)

Comparison shows that:

y₁ = −√

n¯σ_n²+√

ns(ˆθ₁), (4.32) equivalently: s(ˆθ₁) = y₁

√n+ ¯σ_n² = ˜s. (4.33)

Comparing the terms with ˆθ₂ inside in (4.31) we get the condition (1) for the solution of this equation system:

B⁰(ˆθ₁) = 4B(ˆθ₁)I_(n)⁻¹B(ˆθ₁). (4.34)

We’ll try to insert this result into the density calculation in (4.19) but we’ll concentrate on the quadratic form with ˆθ₂:

nˆθ^T₂I_(n)⁻¹

³

B(ˆθ₁) + ˆθ₁B⁰(ˆθ₁)

´

I_(n)⁻¹θˆ₂ = nˆθ^T₂I_(n)⁻¹

³

B(ˆθ₁) + 4ˆθ₁B(ˆθ₁)I_(n)⁻¹B(ˆθ₁)

´

I_(n)⁻¹θˆ₂ =

= 1

4y₂^TI_(n)B⁻¹(ˆθ₁)

³

B(ˆθ₁) + 4ˆθ₁B(ˆθ₁)I_(n)⁻¹B(ˆθ₁)

´

B⁻¹(ˆθ₁)I_(n)y₂ =

= 1 4y₂^T

³

I_(n)B⁻¹(ˆθ₁)I_(n)+ 4ˆθ₁I_(n)

´

y₂. (4.35)

The computation in (4.35) with the condition (2) will give the following result:

1 4y^T₂

³

I_(n)B⁻¹(ˆθ₁)I_(n)+ 4ˆθ₁I_(n)

´ y₂= 1

2y₂^TΣy₂.

Now let’s try to compute the determinant |C_spec| in (4.21) using the condition (1) and the result of ˆθ₂ in (4.30):

|C_spec| = |2I_(n)⁻¹B(ˆθ₁)I_(n)⁻¹|(s⁰(ˆθ₁) + ˆθ₂^TI_(n)⁻¹

³

B⁰⁰(ˆθ₁) − 2B⁰(ˆθ₁)B⁻¹(ˆθ₁)B⁰(ˆθ₁)

´

I_(n)⁻¹θˆ₂) =

= |2I_(n)⁻¹B(ˆθ₁)I_(n)⁻¹|(s⁰(ˆθ₁) + 1

4ny₂^TI_(n)B⁻¹(ˆθ₁)I_(n)I_(n)⁻¹

³

B⁰⁰(ˆθ₁) − 32B(ˆθ₁)I_(n)⁻¹B(ˆθ₁)B⁻¹(ˆθ₁)B(ˆθ₁)I_(n)⁻¹B(ˆθ₁)

´

I_(n)⁻¹I_(n)B⁻¹(ˆθ₁)I_(n)y₂) =

= |2I_(n)⁻¹B(ˆθ₁)I_(n)⁻¹|(s⁰(ˆθ₁) + 1

4ny₂^TI_(n)B⁻¹(ˆθ₁) (4.36)

³

B⁰⁰(ˆθ₁) − 32B(ˆθ₁)I_(n)⁻¹B(ˆθ₁)I_(n)⁻¹B(ˆθ₁)

´

B⁻¹(ˆθ₁)I_(n)y₂).

We will compute B⁰⁰(θ₁) for any θ₁∈ Θ by the per partes method (product

(13)

4.2. SADDLEPOINT COMPUTATION IN A SPECIAL CASE 13 rule) and under the condition (1):

B⁰⁰(θ₁) = (B⁰(θ₁))⁰= [4

³

B(θ₁)I_(n)⁻¹

´

B(θ₁)]⁰ = 4(B⁰(θ₁)I_(n)⁻¹B(θ₁) + +B(θ₁)I_(n)⁻¹B⁰(θ₁)) = 4(4B(θ₁)I_(n)⁻¹B(θ₁)I_(n)⁻¹B(θ₁) +

+4B(θ₁)I_(n)⁻¹B(θ₁)I_(n)⁻¹B(θ₁)) = 32B(θ₁)I_(n)⁻¹B(θ₁)I_(n)⁻¹B(θ₁).

And the C_spec determinant will be:

|C_spec| = s⁰(ˆθ₁)|2I_(n)⁻¹B(ˆθ₁)I_(n)⁻¹|. (4.37)

Summarizing these results we get this equation for the density:

f_spec(y) = exp

µ _n P

i=1

S_i(ˆθ₁) − nˆθ₁s −˜ ¹₂y^T₂Σy₂

¶

(2π)^p+1² |C_spec|¹² . (4.38)

We are able to split the density into two parts: first one depending only on ˆθ₁ – (4.26) and the second part depending only on y₂. The second part has p-dimensional Normal distribution with Σ variance matrix as shown in

(4.25). ¥

Corollary 1. Under the conditions of Theorem 2 we have

P µ√

n µ1

nRSS − ¯σ_n²

¶

> u

¶

= Z∞ u

g(z) dz + Rest,

where g(z) = g(ˆθ₁(y₁)) and Rest = R^∞

u

Rf_spec(y)R_n(y)dy₂dy₁.

Proof. Hence Theorem 1 we have:

Z

f_spec(y₁, y₂) dy₂= g(ˆθ₁(y₁)) Z

ϕ_0,Σ(y₂) dy₂

| {z }

=1

= g(ˆθ₁(y₁)).

¥

(14)

4.3 Homoscedastic linear model

Now let’s consider the homoscedastic linear model. It means that the model holds the condition in (4.4) and that is why the formulas can be simplified:

¯

σ²_n = w_i,nσ_i,n² , (4.39) s(θ₁) = s_i(θ₁) = σ¯²_n

1 − 2¯σ_n²θ₁ ∀i = 1, . . . , n, (4.40) B(θ₁) = 1

2s(θ₁)I_(n), (4.41)

B⁰(θ₁) = s²(θ₁)I_(n), (4.42) B⁰⁰(θ₁) = 4s³(θ₁)I_(n). (4.43) All the necessary conditions in Theorem 2 should be fulfilled and we’ll prove it:

• Condition (1):

4B(θ₁)I_(n)⁻¹B(θ₁) = 41

4s²(θ₁)I_(n)I_(n)⁻¹I_(n)= s²(θ₁)I_(n)= B⁰(θ₁).

B⁻¹(˜θ₁) = 2 1

s(˜θ₁)I_(n)⁻¹= 21 − 2¯σ_n²θ˜₁

¯

σ²_n I_(n)⁻¹= 2 1

¯

σ_n²I_(n)⁻¹− 4˜θ₁I_(n)⁻¹ =

= 2I_(n)⁻¹ΣI_(n)⁻¹− 4˜θ₁I_(n)⁻¹ with Σ = _σ_¯¹2

nI_(n).

From Theorem 2 it follows that the saddlepoint equations can be solved and, as we shall see, even explicitly for the first component ˆθ₁.

Using (4.40) and the results from Theorem 2 (4.23) and (4.22) we get:

¯ σ_n²

1 − 2¯σ²_nθ˜₁ = s(˜θ₁) = ˜s = ¯σ_n²+ 1

√ny₁, (4.44)

θ˜₁ = 1 2

µ 1

¯ σ_n² −1

˜ s

¶

= 1 2¯σ_n²

y₁

√n¯σ_n²+ y₁, (4.45) θ˜₂ = 1

2√ n

2

˜

sI_(n)I_(n)⁻¹I_(n)y₂ = 1

√n˜sI_(n)y₂. (4.46)

From (4.27) we have:

|C_hom(˜θ)| = 2˜s²|21

2˜sI_(n)⁻¹I_(n)I_(n)⁻¹| = 2˜s^p+2|I_(n)⁻¹|. (4.47)

(15)

4.3. HOMOSCEDASTIC LINEAR MODEL 15 Because of Theorem 2 we are able to split the density

f_hom(y) = exp

³

−ⁿ₂ ln^σ^¯_˜_sⁿ² − ⁿ₂

³ ˜s

¯ σ²n − 1

´

−_2¯_σ¹2

ny^T₂I_(n)y₂

´

((2˜s)^p+2π^p+1|I_(n)⁻¹|)¹² into two parts:

ϕ(y₂) = exp¡

−¹₂y^T₂Σy₂¢

³

(2π¯σ²_n)^p|I_(n)⁻¹|

´¹

2

and

g(y₁) = exp

³

−ⁿ₂

³ s˜

¯ σ²_n − 1

´´

³ 4π˜s²

³ s˜

¯ σ²n

´_p´¹

2 ³

¯ σ_n²

˜ s

´ⁿ

2

= exp

³

−ⁿ₂

³ s˜

¯ σ_n² − 1

´´

³ 4π¯σ⁴_n_¯_σ^s^˜²4

n

³ s˜

¯ σ²n

´_p´¹

2 ³

¯ σ²_n

˜ s

´ⁿ

2

=

= 1

2¯σ_n²√ π

µ ˜s

¯ σ_n²

¶^n−p

2 −1

exp µ

−n 2

µ s˜

¯ σ²_n − 1

¶¶

. (4.48)

Finally after inserting ˜s in (4.33) into (4.48) we have:

g(y₁) = 1 2¯σ_n²√

π µ

1 + y₁

√n¯σ²_n

¶^n−p

2 −1

exp µ

−

√ny₁ 2¯σ²_n

¶

. (4.49) Let’s compare this formula with the exact distribution.

We know that Y = ^RSS_σ2 ∼ χ² with n − p degrees of freedom. The linear transformation has to be used.

Y = aX + b.

In this case we have X =√

n µRSS

n − ¯σ²_n

¶

= RSS

√n −√

n¯σ_n² = σ¯_n²

√n RSS

¯ σ_n²

| {z }

Y

−√ n¯σ_n²,

Y =

√n

¯ σ²_n

|{z}

a

X + n|{z}

b

.

The density will be computed using this formula:

f_Y(x) = 1

|a||{z}f_X

=g

(x − b a ).

(16)

It means:

f_Y(x) = σ¯_n²

√n 1 2¯σ²_n√

π µ

1 + x − n

√n¯σ_n²

¯ σ_n²

√n

¶^n−p

2 −1

exp µ

−

√n(x − n) 2¯σ²_n

¯ σ_n²

√n

¶

=

= 1

2√ nπ

³ x n

´^n−p

2 −1

exp µ

−1

2(x − n)

¶

=

= 1

2√

πn^n−p−1² e⁻ⁿ²x^n−p² ⁻¹exp

³

−x 2

´

. (4.50)

Remind the Stirling formula: Γ(x) = q2π

x

¡_x

e

¢_x¡

1 + O(x⁻¹)¢ . The χ² distribution with N degrees of freedom has the density:

f_N(y) = 1 2^N²Γ¡_N

2

¢ y^N²⁻¹exp⁻^y² = 1 2^N²

q4π N

¡_N

2e

¢^N

2

y^N²⁻¹exp⁻^y² ¡

1 + O(N⁻¹)¢

=

= 1

2√

πN^{N −1}² e⁻^N² y^N²⁻¹exp⁻^y² ¡

1 + O(N⁻¹)¢

. (4.51)

We also know that

(n − p)⁻^n−p−1² = n⁻^n−p−1² (1 + O(n⁻ⁿ)). (4.52) Letting N = n − p we got the same result as in (4.50).

4.4 General p-sample regression model

We assume a p-sample model and comparing to the general model in (4.1) we can use even simpler formula:

y_i= µ_j + ε_i,j, (4.53)

where j = 1 . . . p and i = 1 . . . n.

Definition 1. Let’s define an index set J_(j) : i ∈ J_(j) ⇔ x_i,j = 1, for i = 1 . . . n and j = 1 . . . p. |J_(j)| = n_j corresponds to n_j observations in the j^th sample,P

n_j = n.

We further assume σ_(j)= σ_i for ∀i ∈ J_(j), j = 1 . . . p. It means that all the observations of the same sample have the same variance.

(17)

4.4. GENERAL P -SAMPLE REGRESSION MODEL 17 The model in (4.53) has a n × p dimensional design matrix

X =







1 0 0

... ...

1 0 ... 0 1

... · · · ... 1 0

0 1

... ...

0 0 1





 .

The p-dimensional matrix I_(n) is diagonal:

I_(n)= 1

nX^TW X = 1 n





w₍₁₎ 0

. ..

0 w_(p)



 ,

where w_(j)= P

i∈J(j)

w_i.

Some simplifications as in the homoscedastic case can be made even here:

Matrix B(ˆθ₁):

B(ˆθ₁) = 1 2n





 P

i∈J(1)

w_is_i(ˆθ₁) 0 . ..

0 P

i∈J(p)

w_is_i(ˆθ₁)





. (4.54)

All its derivations are diagonal with the following terms B⁰(ˆθ₁) = diag(b⁰₁(ˆθ₁), . . . , b⁰_p(ˆθ₁))

with b⁰_j(ˆθ₁) = 1 n

X

i∈J(j)

w_is²_i(ˆθ₁), (4.55) B⁰⁰(ˆθ₁) = diag(b⁰⁰₁(ˆθ₁), . . . , b⁰⁰_p(ˆθ₁))

with b⁰⁰_j(ˆθ₁) = 4 n

X

i∈J(j)

w_is³_i(ˆθ₁). (4.56)

κ_sam(ˆθ) = −n¯σ_n²θˆ₁+ Xn i=1

S_i(ˆθ₁) + n³ Xp j=1

θˆ_2,j² b_jw⁻²_(j), (4.57)

(18)

C_sam(ˆθ) =







s⁰(ˆθ₁) + n² P^p

j=1

θˆ_2,j² _w^b2⁰⁰^j

(j) 2n²θˆ_2,1_w^b2⁰¹

(1) · · · 2n²θˆ_2,p_w^b2⁰^p (p)

2n^{2 b}_w2¹ (1)

0 . ..

2n^{2 b}_w2^p (p)







(4.58) is a symmetric matrix. Using the formula in (4.21) for the determinant calculation we get:

|C_sam(ˆθ)| = n^2p Yp j=1

2b_j w²_(j)



s⁰(ˆθ₁) + n² Xp j=1

θˆ_2,j² b⁰⁰_j

w_(j)² − 2n² Xp j=1

θˆ_2,j² b⁰²_j b_jw²_(j)



 .(4.59)

m_sam(ˆθ) =√ n







−¯σ_n²+ s(ˆθ₁) + n² P^p

j=1

θˆ²_2,j_w^b2⁰^j (j)

2n²θˆ_2,1b₁w⁻²₍₁₎ ... 2n²θˆ_2,pb_pw⁻²_(p)







. (4.60)

Using these results we can write (4.11) as:

f_sam(y) = exp

ÃPn i=1

S_i(ˆθ₁) − nˆθ₁s(ˆθ₁) − n³ P^p

j=1

θˆ²_2,jb_jw_(j)⁻²− n³θˆ₁ P^p

j=1

θˆ²_2,jb⁰_jw⁻²_(j)

!

(2π)^p+1² |C_sam(ˆθ)|¹² . (4.61)

4.4.1 Special p-sample case

Theorem 3. Let’s suppose the p-sample regression model. The condition on weights:

w_k= w_l ∀k, l ∈ J_(j), j = 1 . . . p (4.62) is the necessary condition for fulfilling the prerequisites of Theorem 2.

Proof. The necessary conditions from Theorem 2 will be proofed:

b⁰_j = 1 n

X

i∈J(j)

w_i

Ã w_iσ_(j)² 1 − 2w_iσ²_(j)θˆ₁

!₂

= 4 1 4n²



 X

i∈J(j)

w_i w_iσ²_(j) 1 − 2w_iσ²_(j)θˆ₁





2

n w_(j),

On the Saddlepoint Approximation for the Residual Sum of Squares in Special Linear Heteroscedastic Models