Supplementary Material for “Bobrovsky-Zakai Bound for Filtering, Prediction and Smoothing of Nonlinear Dynamic Systems”

(1)

Supplementary Material for “Bobrovsky-Zakai Bound for

Filtering, Prediction and Smoothing of Nonlinear Dynamic

Systems”

Carsten Fritsche, Umut Orguner

Division of Automatic Control

E-mail: carsten@isy.liu.se, umut@metu.edu.tr

1st June 2018

Report no.: LiTH-ISY-R-3105

Address:

Department of Electrical Engineering Linköpings universitet

SE-581 83 Linköping, Sweden

WWW: http://www.control.isy.liu.se

AUTOMATIC CONTROL REGLERTEKNIK LINKÖPINGS UNIVERSITET

Technical reports from the Automatic Control group in Linköping are available from http://www.control.isy.liu.se/publications.

(2)

theorems that could not be included into the paper due to space limitations. The notation is adapted from the paper.

(3)

Supplementary Material for “Bobrovsky-Zakai Bound

for Filtering, Prediction and Smoothing of

Discrete-Time Nonlinear Dynamic Systems”

Carsten Fritsche and Umut Orguner

†

Department of Electrical & Electronics Engineering, Middle East Technical University, 06800, Ankara, Turkey

1 Recursive Computation of Filtering Bound

Lemma 1. The following identities hold (0 < ` < k):

L(Yk; Xk+ hi, Xk) = M0(x1; x0+ h (i) 0 , x0), (1a) L(Yk; Xk+ hnx`+i, Xk) = K`(x`+1, y`; x`+ h (i) ` , x`; x`−1), (1b) L(Yk; Xk+ hnxk+i, Xk) = Lk(yk; xk+ h (i) k , xk; xk−1). (1c)

(4)

Lemma 2. For ξ ≤ k − 2 it holds that

Jk|k(ξ, k) = 0. (3)

Proof. We make use of the fact that due to the Markov property, the product terms inside the expectation in

Jk|k(ξ, k) become independent. In order to see this, let ξ = k − 2 and ξ > 0. Then, the (i, j)-th element of the matrix Jk|k(k − 2, k) can be written as

h

Jk|k(k − 2, k)i

i,j= EL(Yk; Xk+ hnx(k−2)+i, Xk)L(Yk; Xk+ hnxk+j, Xk) − 1 = EnKk−2(xk−1, yk−2; xk−2+ h (i) k−2, xk−2; xk−3)Lk(yk; xk+ h (j) k , xk; xk−1) o − 1. (4) With the tower property of conditional expectations, the above expectation may be rewritten as

This can be generalized for 0 < ξ < k − 2 as follows: h Jk|k(ξ, k)i i,j= E {L(Yk ; Xk+ hnxξ+i, Xk)L(Yk; Xk+ hnxk+j, Xk)} − 1 = EnKξ(xξ+1, yξ; xξ+ h (i) ξ , xξ; xξ−1)Lk(yk; xk+ h (j) k , xk; xk−1) o − 1 = Exξ+1,yξ,xξ,xξ−1 n Kξ(xξ+1, yξ; xξ+ h (i) ξ , xξ; xξ−1) o Eyk,xk,xk−1 n Lk(yk; xk+ h (j) k , xk; xk−1) o − 1. (8)

(5)

Now, since Exξ+1,yξ,xξ,xξ−1 n Kξ(xξ+1, yξ; xξ+ h (i) ξ , xξ; xξ−1) o = Z _p(x_ξ+1|x_ξ_{+ h}(i) ξ ) p(xξ+1|xξ) p(yξ|xξ+ h (i) ξ )p(xξ+ h (i) ξ |xξ−1) p(yξ|xξ)p(xξ|xξ−1) p(xξ+1, yξ, xξ, xξ−1) dxξ+1dyξdxξdxξ−1 = Z p(xξ+1|xξ+ h (i) ξ )p(yξ|xξ+ h (i) ξ )p(xξ+ h (i) ξ |xξ−1)p(xξ−1) dxξ+1dyξdxξdxξ−1 = Z Z p(xξ+1|xξ+ h (i) ξ ) dxξ+1 Z p(yξ|xξ+ h (i) ξ ) dyξ p(xξ+ h (i) ξ , xξ−1) dxξdxξ−1 = Z p(xξ+ h (i) ξ , xξ−1) dxξdxξ−1 = 1 (9a) and Eyk,xk,xk−1 n Lk(yk; xk+ h (j) k , xk; xk−1) o = Z _p(y k|xk+ h (j) k )p(xk+ h (j) k |xk−1) p(yk|xk)p(xk|xk−1) p(yk, xk, xk−1) dykdxkdxk−1 = Z p(yk|xk+ h (j) k )p(xk+ h (j) k |xk−1)p(xk−1) dykdxkdxk−1 = Z Z p(yk|xk+ h (j) k ) dyk p(xk+ h (j) k , xk−1)dxkdxk−1 = 1, (9b) holds, we arrive at h Jk|k(ξ, k)i i,j = 0. (10)

Finally, for ξ = 0 we have h Jk|k(0, k)i i,j= E {L(Yk ; Xk+ hi, Xk)L(Yk; Xk+ hnxk+j, Xk)} − 1 = EnM0(x1; x0+ h (i) 0 , x0)Lk(yk; xk+ h (j) k , xk; xk−1) o − 1 =    Ex1 n Ex0|x1 n M0(x1; x0+ h (i) 0 , x0) o Ey2,x2|x1 n L2(y2; x2+ h (j) 2 , x2; x1) oo − 1, k = 2, Ex1,x0 n M0(x1; x0+ h (i) 0 , x0) o Eyk,xk,xk−1 n Lk(yk; xk+ h (j) k , xk; xk−1) o − 1, k > 2. (11) Since Ex0|x1 n M0(x1; x0+ h (i) 0 , x0) o = Z _p(x 1|x0+ h (i) 0 )p(x0+ h (i) 0 ) p(x1|x0)p(x0) p(x0|x1) dx0 = Z _p(x 1, x0+ h (i) 0 ) p(x1|x0)p(x0) p(x1|x0)p(x0) p(x1) dx0 = 1, (12a) Ey2,x2|x1 n L2(y2; x2+ h (j) 2 , x2; x1) o = 1, (12b) Ex1,x0 n M0(x1; x0+ h (i) 0 , x0) o = Z _p(x 1|x0+ h (i) 0 )p(x0+ h (i) 0 ) p(x1|x0)p(x0) p(x1|x0)p(x0) dx1dx0 = 1, (12c) Eyk,xk,xk−1 n Lk(yk; xk+ h (j) k , xk; xk−1) o = 1 (12d) holds, we arrive at h Jk|k(0, k)i i,j = 0. (13)

(6)

Jk|k(ξ, k − 1) = 0. (14)

Proof. Let ξ = k − 3 and ξ > 0. Then, we obtain

h

Jk|k(k − 3, k − 1)i

i,j= EL(Yk; Xk+ hnx(k−3)+i, Xk)L(Yk; Xk+ hnx(k−1)+j, Xk) − 1 = EnKk−3(xk−2, yk−3; xk−3+ h (i) k−3, xk−3; xk−4)Kk−1(xk, yk−1; xk−1+ h (i) k−1, xk−1; xk−2) o − 1 = Exk−2 n Eyk−3,xk−3,xk−4|xk−2 n Kk−3(xk−2, yk−3; xk−3+ h (i) k−3, xk−3; xk−4) o × Exk,yk−1,xk−1|xk−2 n Kk−1(xk, yk−1; xk−1+ h (j) k−1, xk−1; xk−2) o o − 1 = 0, (15)

This can be generalized for 0 < ξ < k − 3 as follows: h Jk|k(ξ, k − 1)i i,j= EL(Yk ; Xk+ hnxξ+i, Xk)L(Yk; Xk+ hnx(k−1)+j, Xk) − 1 = EnKξ(xξ+1, yξ; xξ+ h (i) ξ , xξ; xξ−1)Kk−1(xk, yk−1; xk−1+ h (j) k−1, xk−1; xk−2) o − 1 = Exξ+1,yξ,xξ,xξ−1 n Kξ(xξ+1, yξ; xξ+ h (i) ξ , xξ; xξ−1) o × Exk,yk−1,xk−1,xk−2 n Kk−1(xk, yk−1; xk−1+ h (j) k−1, xk−1; xk−2) o − 1 = 0, (17)

(7)

where the last equality follows from (9a) and Exk,yk−1,xk−1,xk−2 n Kk−1(xk, yk−1; xk−1+ h (j) k−1, xk−1; xk−2) o = Z _p(x k|xk−1+ h (j) k−1) p(xk|xk−1) p(yk−1|xk−1+ h (j) k−1)p(xk−1+ h (j) k−1|xk−2) p(yk−1|xk−1)p(xk−1|xk−2) × p(xk, yk−1, xk−1, xk−2) dxkdyk−1dxk−1dxk−2 = Z p(xk|xk−1+ h (j) k−1) Z p(yk−1|xk−1+ h (j) k−1)dyk−1 p(xk−1+ h (j) k−1|xk−2)p(xk−2) dxkdxk−1dxk−2 = Z Z p(xk|xk−1+ h (j) k−1) dxk p(xk−1+ h (j) k−1, xk−2) dxk−1dxk−2 = 1. (18)

Finally, for ξ = 0 we have h Jk|k(0, k − 1)i i,j= EL(Yk ; Xk+ hi, Xk)L(Yk; Xk+ hnx(k−1)+j, Xk) − 1 = EnM0(x1; x0+ h (i) 0 , x0)Kk−1(xk, yk−1; xk−1+ h (j) k−1, xk−1; xk−2) o − 1 =    Ex1 n Ex0|x1 n M0(x1; x0+ h (i) 0 , x0) o Ex3,y2,x2|x1 n K2(x3, y2; x2+ h (j) 2 , x2; x1) oo − 1, k = 3, Ex1,x0 n M0(x1; x0+ h (i) 0 , x0) o Exk,yk−1,xk−1,xk−2 n Kk−1(xk, yk−1; xk−1+ h (j) k−1, xk−1; xk−2) o − 1, k > 3. (19) Since Ex0|x1 n M0(x1; x0+ h (i) 0 , x0) o = 1, (20a) Ex3,y2,x2|x1 n K2(x3, y2; x2+ h (j) 2 , x2; x1) o = 1, (20b) Ex1,x0 n M0(x1; x0+ h (i) 0 , x0) o = 1, (20c) Exk,yk−1,xk−1,xk−2 n Kk−1(xk, yk−1; xk−1+ h (j) k−1, xk−1; xk−2) o = 1 (20d) holds, we arrive at h Jk|k(0, k − 1)i i,j = 0. (21)

This concludes the proof of Lemma 3.

Corollary 1. For ξ ≤ k − 3 it holds that

Jk−1|k−1(ξ, k − 1) = 0. (22)

Proof. Follows from Lemma 2.

Lemma 4. The following identity holds

Jk|k(k − 2, k − 1) = Jk−1|k−1(k − 2, k − 1). (23)

Proof. For k > 2 it holds

h Jk|k(k − 2, k − 1)i i,j= EL(Yk ; Xk+ hnx(k−2)+i, Xk)L(Yk; Xk+ hnx(k−1)+j, Xk) − 1 = EKk−2(xk−1, yk−2; xk−2+ h (i) k−2, xk−2; xk−3)Kk−1(xk, yk−1; xk−1+ h (j) k−1, xk−1; xk−2) − 1 = EnKk−2(xk−1, yk−2; xk−2+ h (i) k−2, xk−2; xk−3)Lk−1(yk−1; xk−1+ h (j) k−1, xk−1; xk−2) o − 1, (24)

(8)

where the last equality follows from E n Kk−2(xk−1, yk−2; xk−2+ h (i) k−2, xk−2; xk−3)Kk−1(xk, yk−1; xk−1+ h (j) k−1, xk−1; xk−2) o = Z Kk−2(xk−1, yk−2; xk−2+ h (i) k−2, xk−2; xk−3) p(xk|xk−1+ h (j) k−1) p(xk|xk−1) p(yk−1|xk−1+ h (j) k−1)p(xk−1+ h (j) k−1|xk−2) p(yk−1|xk−1)p(xk−1|xk−2) × p(xk, xk−1, yk−1, yk−2, xk−2, xk−3)dxkdxk−1dyk−1dyk−2dxk−2dxk−3 = Z Kk−2(xk−1, yk−2; xk−2+ h (i) k−2, xk−2; xk−3) Z p(xk|xk−1+ h (j) k−1)dxk _p(y k−1|xk−1+ h (j) k−1) p(yk−1|xk−1) ×p(xk−1+ h (j) k−1|xk−2) p(xk−1|xk−2) p(xk−1, yk−1, yk−2, xk−2, xk−3)dxk−1dyk−1dyk−2dxk−2dxk−3 = Z Kk−2(xk−1, yk−2; xk−2+ h (i) k−2, xk−2; xk−3) p(yk−1|xk−1+ h (j) k−1)p(xk−1+ h (j) k−1|xk−2) p(yk−1|xk−1)p(xk−1|xk−2) × p(xk−1, yk−1, yk−2, xk−2, xk−3)dxk−1dyk−1dyk−2dxk−2dxk−3 = EnKk−2(xk−1, yk−2; xk−2+ h (i) k−2, xk−2; xk−3)Lk−1(yk−1; xk−1+ h (j) k−1, xk−1; xk−2) o . (25) Now, since h Jk−1|k−1(k − 2, k − 1)i i,j= EL(Yk−1 ; Xk−1+ hnx(k−2)+i, Xk−1)L(Yk−1; Xk−1+ hnx(k−1)+j, Xk−1) − 1 = EKk−2(xk−1, yk−2; xk−2+ h (i) k−2, xk−2; xk−3)Lk−1(yk−1; xk−1+ h (j) k−1, xk−1; xk−2) − 1 (26) holds, we have proved the first part. In case k = 2, we have

h

J2|2(0, 1)i

i,j= E {L(Y2; X2+ hi, X2)L(Y2; X2+ hnx+j, X2)} − 1 = EM0(x1; x0+ h (i) 0 , x0)K1(x2, y1; x1+ h (j) 1 , x1; x0) − 1 = Z M0(x1; x0+ h (i) 0 , x0) p(x2|x1+ h (j) 1 ) p(x2|x1) p(y1|x1+ h (j) 1 )p(x1+ h (j) 1 |x0) p(y1|x1)p(x1|x0) × p(x2, x1, y1, x0)dx2dx1dy1dx0 = Z M0(x1; x0+ h (i) 0 , x0) Z p(x2|x1+ h (j) 1 ) dx2  p(y1|x1+ h (j) 1 )p(x1+ h (j) 1 |x0) p(y1|x1)p(x1|x0) × p(x1, y1, x0)dx1dy1dx0 = EM0(x1; x0+ h (i) 0 , x0)L1(y1; x1+ h (j) 1 , x1; x0) − 1. (27)

But this is equal to h J1|1(0, 1)i i,j = E {L(Y1 ; X1+ hi, X1)L(Y1; X1+ hnx+j, X1)} − 1 = EM0(x1; x0+ h (i) 0 , x0)L1(y1; x1+ h (j) 1 , x1; x0) − 1. (28)

Lemma 5. For ξ, η ≤ k − 2 it holds that

Jk|k(ξ, η) = Jk−1|k−1(ξ, η). (29)

Proof. For 0 < ξ, η ≤ k − 2 it follows

h

Jk|k(ξ, η)i

i,j= EYk,Xk

{L(Yk; Xk+ hnxξ+i, Xk)L(Yk; Xk+ hnxη+j, Xk)} − 1 = EYk,Xk n Kξ(xξ+1, yξ; xξ+ h (i) ξ , xξ; xξ−1)Kη(xη+1, yη; xη+ h(j)η , xη; xη−1) o − 1. (30) The expression inside the expectation depends neither on yk nor xk. Hence, these variables can be integrated out yielding h Jk|k(ξ, η)i i,j= EYk−1,Xk−1 n Kξ(xξ+1, yξ; xξ+ h (i) ξ , xξ; xξ−1)Kη(xη+1, yη; xη+ h(j)η , xη; xη−1) o − 1 = hJk−1|k−1(ξ, η)i i,j (31)

(9)

For ξ = η = 0 (and k ≥ 2) it follows h

Jk|k(0, 0)i

i,j = EYk,Xk{L(Yk; Xk+ hi, Xk)L(Yk; Xk+ hj, Xk)} − 1 = Ex1,x0 n M0(x1; x0+ h (i) 0 , x0)M0(x1; x0+ h (j) 0 , x0) o − 1. (32) This is equal to h Jk−1|k−1(0, 0)i i,j= EYk−1,Xk−1{L(Yk−1; Xk−1+ hi , Xk−1)L(Yk−1; Xk−1+ hj, Xk−1)} − 1 = Ex1,x0 n M0(x1; x0+ h (i) 0 , x0)M0(x1; x0+ h (j) 0 , x0) o − 1. (33)

Similarly, for ξ = 0 and 0 < η ≤ k − 2 we obtain h Jk|k(0, η)i i,j = EYk,Xk {L(Yk; Xk+ hi, Xk)L(Yk; Xk+ hnxη+j, Xk)} − 1 = EYk,Xk n M0(x1; x0+ h (i) 0 , x0)Kη(xη+1, yη; xη+ h(j)η , xη; xη−1) o − 1. (34)

The expression inside the expectation depends neither on yk nor xk. Hence, we can write h Jk|k(0, η)i i,j = EYk−1,Xk−1 n M0(x1; x0+ h (i) 0 , x0)Kη(xη+1, yη; xη+ h(j)η , xη; xη−1) o − 1 = hJk−1|k−1(0, η)i i,j . (35)

Essentially the same reasoning can be used to show that h Jk|k(ξ, 0)i i,j= h Jk−1|k−1(ξ, 0)i i,j, (36)

where 0 < ξ ≤ k − 2. This concludes the proof of Lemma 5.

Theorem 1. The MSE matrix for the filtering problem is bounded below by

Exk,Yk(xk−xbk(Yk))(·)

T_{H(k, k) J}

k|k −1

HT(k, k), (37)

where the information matrix Jk|k can be computed recursively as

with initialization J0|0 = J0|0(0, 0). The (i, j)-th elements of the matrices Jk|k(ξ, η) are given by

h Jk|k(k, k)i i,j= E n Lk(yk; xk+ h (i) k , xk; xk−1)Lk(yk; xk+ h (j) k , xk; xk−1) o − 1, (39a) h Jk|k(k, k − 1)i i,j = hJk|k(k − 1, k)i j,i =    E n L1(y1; x1+ h (i) 1 , x1; x0)M0(x1; x0+ h (j) 0 , x0) o − 1, k = 1, E n Lk(yk; xk+ h (i) k , xk; xk−1)Kk−1(xk, yk−1; xk−1+ h (j) k−1, xk−1; xk−2) o − 1, k > 1, (39b) h Jk|k(k − 1, k − 1)i i,j =    E n M0(x1; x0+ h (i) 0 , x0)M0(x1; x0+ h (j) 0 , x0) o − 1, k = 1, E n Kk−1(xk, yk−1; xk−1+ h(i)_k−1, xk−1; xk−2)Kk−1(xk, yk−1; xk−1+ h(j)_k−1, xk−1; xk−2) o − 1, k > 1. (39c)

Proof. We partition the information matrix Jk−1|k−1 at time instance k − 1 as follows:

Jk−1|k−1= _A k−1|k−1 Bk−1|k−1 BT k−1|k−1 J k−1|k−1_{(k − 1, k − 1)} , (40)

(10)

where Ak−1|k−1 =    Jk−1|k−1_{(0, 0)} _{· · ·} _Jk−1|k−1_{(0, k − 2)} .. . . .. ... Jk−1|k−1_{(k − 2, 0)} _{· · ·} _Jk−1|k−1_{(k − 2, k − 2)}   , (41a) Bk−1|k−1=      Jk−1|k−1_{(0, k − 1)} .. . Jk−1|k−1(k − 3, k − 1) Jk−1|k−1_{(k − 2, k − 1)}      Corollary 1 =      0 .. . 0 Jk−1|k−1_{(k − 2, k − 1)}      . (41b)

The information sub-matrix Jk−1|k−1 for estimating xk−1 is the inverse of the (nx× nx) lower-right sub-block ofJk−1|k−1−1

and can be obtained from block matrix inversion as follows:

Jk−1|k−1 = Jk−1|k−1(k − 1, k − 1) − BTk−1|k−1Ak−1|k−1 −1

Bk−1|k−1. (42)

Similarly, the information matrix Jk|k_{at time k can be partitioned according to}

Jk|k=   Ak|k Bk|k Ck|k BT_k|k Jk|k(k − 1, k − 1) Jk|k(k − 1, k) CT k|k J k|k_{(k, k − 1)} _Jk|k_{(k, k)}  , (43) where Ak|k=    Jk|k_{(0, 0)} _{· · ·} _Jk|k_{(0, k − 2)} .. . . .. ... Jk|k(k − 2, 0) · · · Jk|k(k − 2, k − 2)    Lemma 5 = Ak−1|k−1, (44a) Bk|k=      Jk|k(0, k − 1) .. . Jk|k_{(k − 3, k − 1)} Jk|k_{(k − 2, k − 1)}      Lemma 3,4 = Bk−1|k−1 (44b) and Ck|k=    Jk|k(0, k) .. . Jk|k_{(k − 2, k)}    Lemma 2 =    0 .. . 0   . (44c)

Hence, the information matrix Jk|k_{can be reduced to}

Jk|k=   _A k−1|k−1 Bk−1|k−1 BT k−1|k−1 J k|k_{(k − 1, k − 1)} 0 Jk|k_{(k − 1, k)} 0 Jk|k(k, k − 1) Jk|k(k, k)  . (45)

The (nx× nx) information sub-matrix Jk|kis again obtained from block-matrix inversion, yielding the recursion

Jk|k= Jk|k(k, k) − 0 Jk|k_{(k − 1, k)} T" ? ? ? hJk|k(k − 1, k − 1) − BT_k−1|k−1Ak−1|k−1 −1 Bk−1|k−1 i−1 # × 0 Jk|k(k − 1, k) = Jk|k(k, k) − Jk|k(k, k − 1)hJk|k(k − 1, k − 1) − Jk−1|k−1(k − 1, k − 1) + Jk−1|k−1 i−1 Jk|k(k − 1, k), (46)

(11)

where the last equality follows by using (42). The matrix elements Jk|k(η, ξ) constituting the above recursion can be obtained by making use of the results provided in Lemma 1. This yields

h Jk|k(k, k)i i,j= h Jk|ki nxk+i,nxk+j

= E {L(Yk; Xk+ hnxk+i, Xk)L(Yk; Xk+ hnxk+j, Xk)} − 1 = EnLk(yk; xk+ h (i) k , xk; xk−1)Lk(yk; xk+ h (j) k , xk; xk−1) o − 1 = E ( p(yk|xk+ h (i) k )p(yk|xk+ h (j) k )p(xk+ h (i) k |xk−1)p(xk+ h (j) k |xk−1) p2_(y k|xk)p2(xk|xk−1) ) − 1, (47a) h Jk|k(k, k − 1)i i,j =hJk|ki nxk+i,nx(k−1)+j =hJk|k(k − 1, k)i j,i

= EL(Yk; Xk+ hnxk+i, Xk)L(Yk; Xk+ hnx(k−1)+j, Xk) − 1 =    E n L1(y1; x1+ h (i) 1 , x1; x0)M0(x1; x0+ h (j) 0 , x0) o − 1, k = 1, E n Lk(yk; xk+ h (i) k , xk; xk−1)Kk−1(xk, yk−1; xk−1+ h (j) k−1, xk−1; xk−2) o − 1, k > 1, =        E p(x1+h(i)1 |x0)p(x1|x0+h(j)0 )p(x0+h(j)0 ) p2_(x₁_|x₀_)p(x₀₎ − 1, k = 1, E p(xk+h(i)_k |xk−1)p(xk|xk−1+h(j)_k−1)p(xk−1+h(j)_k−1|xk−2) p2_(x_k_|x_k−1_)p(x_k−1_|x_k−2₎ − 1, k > 1, (47b) h Jk|k(k − 1, k − 1)i i,j =hJ(k)i nx(k−1)+i,nx(k−1)+j

= EL(Yk; Xk+ hnx(k−1)+i, Xk)L(Yk; Xk+ hnx(k−1)+j, Xk) − 1 =    E n M0(x1; x0+ h (i) 0 , x0)M0(x1; x0+ h (j) 0 , x0) o − 1, k = 1, E n Kk−1(xk, yk−1; xk−1+ h (i) k−1, xk−1; xk−2)Kk−1(xk, yk−1; xk−1+ h (j) k−1, xk−1; xk−2) o − 1, k > 1, =                E p(x1|x0+h(i)0 )p(x1|x0+h(j)0 )p(x0+h(i)0 )p(x0+h(j)0 ) p2_(x 1|x0)p2(x0) − 1, k = 1, E

This concludes the proof of Theorem 1.

2 Recursive Computation of Prediction Bound

Theorem 2. The MSE matrix for the one-step ahead prediction problem is bounded below by

Exk+1,Yk(xk+1−bxk+1(Yk))(·)

T_{H(k + 1, k + 1) J}

k+1|k −1

HT(k + 1, k + 1), (48)

where the prediction information sub-matrix Jk+1|k can be computed from the recursion

Jk+1|k= Jk+1|k(k + 1, k + 1) − Jk+1|k(k + 1, k) h

Jk+1|k(k, k) − Jk|k(k, k) + Jk|k i−1

Jk+1|k(k, k + 1) (49)

with initialization J0|0. The (i, j)-th element of the matrix Jk+1|k(k + 1, k + 1) is given by

h Jk+1|k(k + 1, k + 1)i i,j = E ( p(xk+1+ h (i) k+1|xk)p(xk+1+ h (j) k+1|xk) p2_(x k+1|xk) ) − 1. (50)

Proof. The one-step ahead prediction information matrix Jk+1|k _{is partitioned into blocks J}k+1|k_{(ξ, η), ξ, η =} 0, . . . , k + 1 each of size (nx× nx) as follows

Jk+1|k ∆=        Jk+1|k_{(0, 0)} _Jk+1|k_{(0, 1)} _{· · ·} _Jk+1|k_{(0, k)} _Jk+1|k_{(0, k + 1)} Jk+1|k_{(1, 0)} _Jk+1|k_{(1, 1)} _{· · ·} _Jk+1|k_{(1, k)} _Jk+1|k_{(1, k + 1)} .. . ... . .. ... ... Jk+1|k(k, 0) Jk+1|k(k, 1) · · · Jk+1|k(k, k) Jk+1|k(k, k + 1) Jk+1|k_{(k + 1, 0)} _Jk+1|k_{(k + 1, 1)} _{· · ·} _Jk+1|k_{(k + 1, k)} _Jk+1|k_{(k + 1, k + 1)}        , (51)

(12)

where each block is given by h Jk+1|k(ξ, η)i i,j = hJk+1|ki nxξ+i,nxη+j

= EYk,Xk+1{L(Yk, Xk+1+ hnxξ+i, Xk+1)L(Yk, Xk+1+ hnxη+j, Xk+1)} − 1. (52) The idea is now, similarly to the proof for the filtering bound, to identify blocks in the matrix Jk+1|k_{that are} identical to the blocks in the matrix Jk|k_{, i.e. they do not change over time, such that a recursive computation} of the prediction information sub-matrix Jk+1|kin terms of the filtering information sub-matrix Jk|kis possible. In order to prove the above stated recursion, the following lemmas are required:

Lemma 6. The following identities hold (0 < ` < k + 1):

L(Yk; Xk+1+ hi, Xk+1) = M0(x1; x0+ h (i) 0 , x0), (53a) L(Yk; Xk+1+ hnx`+i, Xk+1) = K`(x`+1, y`; x`+ h (i) ` , x`; x`−1), (53b) L(Yk; Xk+1+ hnx(k+1)+i, Xk+1) = Nk+1(xk+1+ h (i) k , xk+1; xk). (53c)

Proof. We can simplify each ratio of joint densities as follows:

(13)

Proof. Let ξ = k − 1 and ξ > 0. Then, the (i, j)-th element of the matrix Jk+1|k(k − 1, k + 1) can be written as h

Jk+1|k(k − 1, k + 1)i

i,j= EL(Yk; Xk+1+ hnx(k−1)+i, Xk+1)L(Yk; Xk+1+ hnx(k+1)+j, Xk+1) − 1 = EnKk−1(xk, yk−1; xk−1+ h (i) k−1, xk−1; xk−2)Nk+1(xk+1+ h (j) k+1, xk+1; xk) o − 1. (56) The tower property of conditional expectations gives

E n Kk−1(xk, yk−1; xk−1+ h (i) k−1, xk−1; xk−2)Nk+1(xk+1+ h (j) k+1, xk+1; xk) o = Exk n Eyk−1,xk−1,xk−2|xk n Kk−1(xk, yk−1; xk−1+ h (i) k−1, xk−1; xk−2) o × Exk+1|xk n Nk+1(xk+1+ h (j) k+1, xk+1; xk) o o . (57) Since Eyk−1,xk−1,xk−2|xk n Kk−1(xk, yk−1; xk−1+ h (i) k−1, xk−1; xk−2) o = 1 (58a) and Exk+1|xk n Nk+1(xk+1+ h (j) k+1, xk+1; xk) o = Z _p(x k+1+ h (j) k+1|xk) p(xk+1|xk) p(xk+1|xk) dxk+1 = 1, (58b) we obtain h Jk+1|k(k − 1, k + 1)i i,j= 0. (59)

This can be generalized for 0 < ξ < k − 1 as follows: h Jk+1|k(ξ, k + 1)i i,j= EL(Yk ; Xk+1+ hnxξ+i, Xk+1)L(Yk; Xk+1+ hnx(k+1)+j, Xk+1) − 1 = EnKξ(xξ+1, yξ; xξ+ h (i) ξ , xξ; xξ−1)Nk+1(xk+1+ h (j) k+1, xk+1; xk) o − 1 = Exξ+1,yξ,xξ,xξ−1 n Kξ(xξ+1, yξ; xξ+ h (i) ξ , xξ; xξ−1) o Exk+1,xk n Nk+1(xk+1+ h (j) k+1, xk+1; xk) o − 1. (60) Now, since Exξ+1,yξ,xξ,xξ−1 n Kξ(xξ+1, yξ; xξ+ h (i) ξ , xξ; xξ−1) o = 1 (61a) and Exk+1,xk n Nk+1(xk+1+ h (j) k+1, xk+1; xk) o = Z "Z _p(x k+1+ h (j) k+1xk) p(xk+1|xk) p(xk+1|xk) dxk+1 # p(xk) dxk = 1, (61b) holds, we arrive at h Jk+1|k(ξ, k + 1)i i,j = 0. (62)

Finally, for ξ = 0 we have h Jk+1|k(0, k + 1)i i,j= EL(Yk ; Xk+1+ hi, Xk+1)L(Yk; Xk+1+ hnx(k+1)+j, Xk+1) − 1 = EnM0(x1; x0+ h (i) 0 , x0)Nk+1(xk+1+ h (j) k+1, xk+1; xk) o − 1 =    Ex1 n Ex0|x1 n M0(x1; x0+ h (i) 0 , x0) o Ex2|x1 n N2(x2+ h (j) 2 , x2; x1) oo − 1, k = 1, Ex1,x0 n M0(x1; x0+ h (i) 0 , x0) o Exk+1,xk n Nk+1(xk+1+ h (j) k+1, xk+1; xk) o − 1, k > 1. (63)

(14)

Since Ex0|x1 n M0(x1; x0+ h (i) 0 , x0) o = 1, (64a) Ex2|x1 n N2(x2+ h (j) 2 , x2; x1) o = 1, (64b) Ex1,x0 n M0(x1; x0+ h (i) 0 , x0) o = 1, (64c) Exk+1,xk n Nk+1(xk+1+ h (j) k+1, xk+1; xk) o = 1, (64d) holds, we arrive at h Jk+1|k(0, k + 1)i i,j = 0. (65)

Jk+1|k(ξ, k) = Jk|k(ξ, k) = 0. (66)

Proof. Let ξ = k − 2 and ξ > 0. Then, we obtain

h Jk+1|k(k − 2, k)i i,j = EL(Yk ; Xk+1+ hnx(k−2)+i, Xk+1)L(Yk; Xk+1+ hnxk+j, Xk+1) − 1 = EnKk−2(xk−1, yk−2; xk−2+ h (i) k−2, xk−2; xk−3)Kk(xk+1, yk; xk+ h (j) k , xk; xk−1) o − 1 = Exk−1 n Eyk−2,xk−2,xk−3|xk−1 n Kk−2(xk−1, yk−2; xk−2+ h (i) k−2, xk−2; xk−3) o × Exk+1,yk,xk|xk−1 n Kk(xk+1, yk; xk+ h (j) k , xk; xk−1) o o − 1 = 0, (67)

where the last equality follows from Eyk−2,xk−2,xk−3|xk−1 n Kk−2(xk−1, yk−2; xk−2+ h(i)_k−2, xk−2; xk−3) o = 1, (68a) and Exk+1,yk,xk|xk−1 n Kk(xk+1, yk; xk+ h (j) k , xk; xk−1) o = 1. (68b)

This can be generalized for 0 < ξ < k − 2 as follows: h Jk+1|k(ξ, k)i i,j= E {L(Yk ; Xk+1+ hnxξ+i, Xk+1)L(Yk; Xk+1+ hnxk+j, Xk+1)} − 1 = EnKξ(xξ+1, yξ; xξ+ h (i) ξ , xξ; xξ−1)Kk(xk+1, yk; xk+ h (j) k , xk; xk−1) o − 1 = Exξ+1,yξ,xξ,xξ−1 n Kξ(xξ+1, yξ; xξ+ h (i) ξ , xξ; xξ−1) o × Exk+1,yk,xk,xk−1 n Kk(xk+1, yk; xk+ h (j) k , xk; xk−1) o − 1 = 0, (69)

where the last equality follows from (61a) and Exk+1,yk,xk,xk−1 n Kk(xk+1, yk; xk+ h (j) k , xk; xk−1) o = 1. (70)

Finally, for ξ = 0 we have h Jk+1|k(0, k)i i,j = E {L(Yk ; Xk+1+ hi, Xk+1)L(Yk; Xk+1+ hnxk+j, Xk+1)} − 1 = EnM0(x1; x0+ h (i) 0 , x0)Kk(xk+1, yk; xk+ h (j) k , xk; xk−1) o − 1 =    Ex1 n Ex0|x1 n M0(x1; x0+ h (i) 0 , x0) o Ex3,y2,x2|x1 n K2(x3, y2; x2+ h (j) 2 , x2; x1) oo − 1, k = 2, Ex1,x0 n M0(x1; x0+ h (i) 0 , x0) o Exk+1,yk,xk,xk−1 n Kk(xk+1, yk; xk+ h (j) k , xk; xk−1) o − 1, k > 2. (71)

(15)

Since Ex0|x1 n M0(x1; x0+ h (i) 0 , x0) o = 1, (72a) Ex3,y2,x2|x1 n K2(x3, y2; x2+ h (j) 2 , x2; x1) o = 1, (72b) Ex1,x0 n M0(x1; x0+ h (i) 0 , x0) o = 1, (72c) Exk+1,yk,xk,xk−1 n Kk(xk+1, yk; xk+ h(j)_k , xk; xk−1) o = 1 (72d) holds, we arrive at h Jk+1|k(0, k)i i,j = 0. (73)

Now comparing this with the result stated in Lemma 2 concludes the proof.

Lemma 9. The following identity holds

Jk+1|k(k − 1, k) = Jk|k(k − 1, k) (74)

Proof. For k > 1 it holds

h

Jk+1|k(k − 1, k)i

i,j = EL(Yk; Xk+1+ hnx(k−1)+i

, Xk+1)L(Yk; Xk+1+ hnxk+j, Xk+1) − 1 = EKk−1(xk, yk−1; xk−1+ h (i) k−1, xk−1; xk−2)Kk(xk+1, yk; xk+ h (j) k , xk; xk−1) − 1 = E ( p(xk|xk−1+ h (i) k−1)p(xk−1+ h (i) k−1|xk−2)p(xk+ h (j) k |xk−1) p2_(x k|xk−1)p(xk−1|xk−2) ) − 1, (75)

where the last equality follows from integrating out the variables xk+1, yk and yk−1. Similarly, integrating out the variables yk and yk−1 in

h Jk|k(k − 1, k)i i,j= EL(Yk ; Xk+ hnx(k−1)+i, Xk)L(Yk; Xk+ hnxk+j, Xk) − 1 = EKk−1(xk, yk−1; xk−1+ h (i) k−1, xk−1; xk−2)Lk(yk; xk+ h (j) k , xk; xk−1) − 1 = E ( p(xk|xk−1+ h (i) k−1)p(xk−1+ h (i) k−1|xk−2)p(xk+ h (j) k |xk−1) p2_(x k|xk−1)p(xk−1|xk−2) ) − 1, (76)

yields the equality. For k = 1 we arrive at h J2|1(0, 1)i i,j = E {L(Y1 ; X2+ hi, X2)L(Y1; X2+ hnx+j, X2)} − 1 = EM0(x1; x0+ h (i) 0 , x0)K1(x2, y1; x1+ h (j) 1 , x1; x0) − 1 = E ( p(x1|x0+ h (i) 0 )p(x0+ h (i) 0 )p(x1+ h (j) 1 |x0) p2_(x 1|x0)p(x0) ) − 1, (77)

where the last equality follows from integrating out the variables x2 and y1. Similarly, integrating out the

variable y1 in h J1|1(0, 1)i i,j = E {L(Y1; X1+ hi , X1)L(Y1; X1+ hnx+j, X1)} − 1 = EM0(x1; x0+ h (i) 0 , x0)L1(y1; x1+ h (j) 1 , x1; x0) − 1 = E ( p(x1|x0+ h (i) 0 )p(x0+ h (i) 0 )p(x1+ h (j) 1 |x0) p2_(x 1|x0)p(x0) ) − 1 (78)

yields the equality. This concludes the proof of Lemma 9.

Lemma 10. For ξ, η ≤ k − 1 it holds that

(16)

Proof. For 0 < ξ, η ≤ k − 1 it follows

h

Jk+1|k(ξ, η)i

i,j = EYk,Xk+1

{L(Yk; Xk+1+ hnxξ+i, Xk+1)L(Yk; Xk+1+ hnxη+j, Xk+1)} − 1 = EYk,Xk n Kξ(xξ+1, yξ; xξ+ h (i) ξ , xξ; xξ−1)Kη(xη+1, yη; xη+ h(j)η , xη; xη−1) o − 1. (80) Equivalently, we can write

h

Jk|k(ξ, η)i

i,j= EYk,Xk

{L(Yk; Xk+ hnxξ+i, Xk)L(Yk; Xk+ hnxη+j, Xk)} − 1 = EYk,Xk n Kξ(xξ+1, yξ; xξ+ h (i) ξ , xξ; xξ−1)Kη(xη+1, yη; xη+ h(j)η , xη; xη−1) o − 1. (81) For ξ = η = 0 (and k ≥ 1) we have

h Jk+1|k(0, 0)i i,j= EYk,Xk+1{L(Yk; Xk+1+ hi , Xk+1)L(Yk; Xk+1+ hj, Xk+1)} − 1 = Ex1,x0 n M0(x1; x0+ h (i) 0 , x0)M0(x1; x0+ h (i) 0 , x0) o − 1. (82) This is equal to h Jk|k(0, 0)i i,j= EYk,Xk {L(Yk; Xk+ hi, Xk)L(Yk; Xk+ hj, Xk)} − 1 = Ex1,x0 n M0(x1; x0+ h (i) 0 , x0)M0(x1; x0+ h (i) 0 , x0) o − 1. (83)

Similarly, for ξ = 0 and 0 < η ≤ k − 1 we obtain h Jk+1|k(0, η)i i,j= EYk,Xk+1 {L(Yk; Xk+1+ hi, Xk+1)L(Yk; Xk+1+ hnxη+j, Xk+1)} − 1 = EYk,Xk n M0(x1; x0+ h (i) 0 , x0)Kη(xη+1, yη; xη+ h(j)η , xη; xη−1) o − 1, (84)

but this is equal to h

Jk|k(0, η)i

i,j = EYk,Xk{L(Yk; Xk+ hi, Xk)L(Yk; Xk+ hnxη+j, Xk)} − 1 = EYk,Xk n M0(x1; x0+ h (i) 0 , x0)Kη(xη+1, yη; xη+ h(j)η , xη; xη−1) o − 1. (85)

Essentially the same reasoning applies to show that h Jk+1|k(ξ, 0)i i,j = h Jk|k(ξ, 0)i i,j (86)

holds, where 0 < ξ ≤ k − 1. This concludes the proof of Lemma 10.

We partition the prediction information matrix Jk+1|k _{at time k + 1 as follows}

Jk+1|k=      Ak+1|k Bk+1|k Ck+1|k Dk+1|k BT k+1|k J k+1|k_{(k − 1, k − 1)} _Jk+1|k_{(k − 1, k)} _Jk+1|k_{(k − 1, k + 1)} CT k+1|k J k+1|k_{(k, k − 1)} _Jk+1|k_{(k, k)} _Jk+1|k_{(k, k + 1)} DT_k+1|k Jk+1|k(k + 1, k − 1) Jk+1|k(k + 1, k) Jk+1|k(k + 1, k + 1)      . (87)

The elements in this matrix can be simplified as follows Dk+1|k Jk+1|k(k − 1, k + 1) Lemma 7 =    0 .. . 0   , (88) Ck+1|k Jk+1|k_{(k − 1, k)} Lemma 8,9 = 0 Jk|k_{(k − 1, k)} , (89) and _A k+1|k Bk+1|k BT k+1|k J k+1|k_{(k − 1, k − 1)} Lemma 10 = _A k−1|k−1 Bk−1|k−1 BT k−1|k−1 J k|k_{(k − 1, k − 1)} , (90)

(17)

yielding Jk+1|k=       Ak−1|k−1 Bk−1|k−1 0 BT k−1|k−1 J k|k_{(k − 1, k − 1)} _Jk|k_{(k − 1, k)} 0 Jk|k_{(k, k − 1)} _Jk+1|k_{(k, k)}     0 0 Jk+1|k(k, k + 1)   0 0 Jk+1|k(k + 1, k) Jk+1|k(k + 1, k + 1)     . (91)

Block-matrix inversion gives

Jk+1|k= Jk+1|k(k + 1, k + 1) −   0 0 Jk+1|k(k, k + 1)   T  Ak−1|k−1 Bk−1|k−1 0 BT_k−1|k−1 Jk|k(k − 1, k − 1) Jk|k(k − 1, k) 0 Jk|k_{(k, k − 1)} _Jk+1|k_{(k, k)}   −1 ×   0 0 Jk+1|k(k, k + 1)   = Jk+1|k(k + 1, k + 1) − Jk+1|k(k + 1, k)hJk+1|k(k, k) − Jk|k(k, k − 1)hJk|k(k − 1, k − 1) −BT_k−1|k−1A−1_k−1|k−1Bk−1|k−1 i−1 Jk|k(k − 1, k) −1 Jk+1|k(k, k + 1) = Jk+1|k(k + 1, k + 1) − Jk+1|k(k + 1, k)hJk+1|k(k, k) − Jk|k(k, k − 1)hJk|k(k − 1, k − 1) −Jk−1|k−1_{(k − 1, k − 1) + J} k−1|k−1 i−1 Jk|k(k − 1, k) −1 Jk+1|k(k, k + 1) = Jk+1|k(k + 1, k + 1) − Jk+1|k(k + 1, k)hJk+1|k(k, k) − Jk|k(k, k) + Jk|k i−1 Jk+1|k(k, k + 1) (92) The (i, j)-th elements of the matrices Jk+1|k_{(k + 1, k + 1), J}k+1|k_{(k + 1, k) and J}k+1|k_{(k, k) can be simplified as} follows h Jk+1|k(k + 1, k + 1)i i,j= EYk,Xk+1L(Yk ; Xk+1+ hnx(k+1)+i, Xk+1)L(Yk; Xk+1+ hnx(k+1)+j, Xk+1) − 1 = EnNk+1(xk+1+ h (i) k+1, xk+1; xk)Nk+1(xk+1+ h (j) k+1, xk+1; xk) o − 1 = E ( p(xk+1+ h (i) k+1|xk)p(xk+1+ h (j) k+1|xk) p2_(x k+1|xk) ) − 1, (93a) h Jk+1|k(k + 1, k)i

i,j= EYk,Xk+1L(Yk; Xk+1+ hnx(k+1)+i, Xk+1)L(Yk; Xk+1+ hnxk+j, Xk+1) − 1 =    E n N1(x1+ h (i) 1 , x1; x0)M0(x1; x0+ h (j) 0 ; x0) o − 1, k = 0, E n Nk+1(xk+1+ h (i) k+1, xk+1; xk)Kk(xk+1, yk; xk+ h (j) k , xk; xk−1) o − 1, k > 0, =        E p(x1+h(i)1 |x0)p(x1|x0+h(j)0 )p(x0+h(j)0 ) p2_(x 1|x0)p(x0) − 1, k = 0, E p(xk+1+h(i)_k+1|xk)p(xk+1|xk+h(j)_k )p(xk+h(j)_k |xk−1) p2_(x_k+1_|x_k_)p(x_k_|x_k−1₎ − 1, k > 0, =hJk+1|k+1(k + 1, k)i i,j (93b) and h Jk+1|k(k, k)i i,j= EYk,Xk+1

{L(Yk; Xk+1+ hnxk+i, Xk+1)L(Yk; Xk+1+ hnxk+j, Xk+1)} − 1

=    E n M0(x1; x0+ h (i) 0 , x0)M0(x1; x0+ h (j) 0 , x0) o − 1, k = 0, E n Kk(xk+1, yk; xk+ h (i) k , xk; xk−1)Kk(xk+1, yk; xk+ h (j) k , xk; xk−1) o − 1, k > 0, = hJk+1|k+1(k, k)i i,j . (93c)

(18)

3 Recursive Computation of Smoothing Bound

Theorem 3. The MSE matrix for the smoothing problem is bounded below by

Ex`,YT (x`−xb`(YT))(·)

T_{H(`, `) J}

`|T −1

HT(`, `), (94)

where 0 ≤ ` < T and the information matrix J`|T can be computed recursively as

J`|T = h J`|`+ JT |T(`, `) − J`|`(`, `) i − JT |T_{(`, ` + 1)} × J`+1|T+ JT |T(` + 1, `) h J`|`+ JT |T(`, `) − J`|`(`, `) i−1 JT |T(`, ` + 1) −1 JT |T(` + 1, `), (95) with initialization JT |T. Proof. LetJT |T_t

1:t2,t3:t4 denote the sub-matrix of J

T |T _{that contains the rows that correspond to time instance}

t1 to t2 and the columns that correspond to time instance t3 to t4. We decompose JT |T into blocks which

correspond to time instants 0, 1, . . . , ` and ` + 1, ` + 2, . . . , T with ` < T , yielding

JT |T = " JT |T 0:`,0:` J T |T 0:`,`+1:T JT |T `+1:T ,0:` J T |T `+1:T ,`+1:T # , (96) where h JT |Ti 0:`,0:`= A B BT _JT |T_{(`, `)} , (97a) h JT |Ti 0:`,`+1:T = 0 0 JT |T_{(`, ` + 1)} ₀ (97b) and h JT |Ti `+1:T ,`+1:T =            JT |T(` + 1, ` + 1) JT |T(` + 1, ` + 2) 0 JT |T_{(` + 2, ` + 1)} . ._. . ._. . ._. JT |T(T − 1, T ) 0 JT |T_{(T, T − 1)} _JT |T_{(T, T )}            . (97c)

Similarly, the inverseJT |T−1 can be decomposed according to

We further note that the information matrix J`|`_{for ` < T can be partitioned as}

J`|`= " e A Be e BT J`|`(`, `) # , (99)

such that the information sub-matrix J`|`for estimating x` is given by

J`|`= J`|`(`, `) − eBTAe−1B.e (100) The matrices A = JT |T

0:`−1,0:`−1 and eA = J

`|`