Optimal Control of a Stochastic Heat Equation with Control and Noise on the Boundary

Degree project

Optimal Control of a

Stochastic Heat Equation with Control and Noise on the

Boundary

Author: Thavamani Govindaraj Supervisor: Astrid Hilbert Examiner: Roger Pettersson Date: 2018-06-16

Course Code: 5MA41E Subject: Thesis

Level: Master

Optimal Control of A Stochastic Heat Equation with Control and Noise on the

Boundary

by

Thavamani Govindaraj Supervisor: Prof. Astrid Hilbert

Linnaeus University

Sweden

June 16, 2018

Abstract

In this thesis, we give a mathematical background of solving a linear quadratic control problem for the heat equation, which involves noise on the boundary, in a concise way. We use the semigroup approach for the solvability of the problem. To obtain optimal controls, we use optimization techniques for convex functionals. Finally we give a feedback form for the optimal control. In order to enhance understanding of linear quadratic problem, we first present the methods in deterministic cases and then extend to noisy systems.

Acknowledgements

I would like to thank my supervisor Astrid Hilbert for her guidance through out the thesis work. I am very thankful to her for sharing her knowledge and for answering my questions at all times. I would also like to thank her for introducing to me some of the experts in stochastic control theory at the time of my thesis work. Finally, I would like to thank my family members for their constant support and encouragements.

Contents

1 Introduction 5

2 Preliminaries 6

2.1 Elements from Stochastic Calculus . . . 6 2.2 Elements from Semigroup theory . . . 9

3 Problem formulation 13

4 Linear Quadratic Optimal Control: Deterministic Cases 16 4.1 Introduction: A Motivating Example . . . 16 4.2 Linear Quadratic Optimal Control in Hilbert Spaces . . . 17 4.3 Linear Quadratic Optimal Control: Control on the Boundary . . . 23 5 Linear Quadratic Optimal Control for noisy systems 29 5.1 Linear Quadratic Optimal Control for a Stochastic Heat Equation 29 5.2 Existence and Uniqueness of Solutions . . . 30 5.3 Stochastic Optimal Control . . . 32 5.4 Stochastic Optimal Control as Feedback Control . . . 37

6 Conclusion 46

1 Introduction

The modeling of dynamical systems by differential equations requires the parame- ter values associated with the system to be completely known. Such models arise from problems in physics or economics which do not have sufficient information on parameter values. For instance, the parameter values may depend on the surroundings or medium where we consider the system. Moreover, due to internal or external noise the parameter values may fluctuate.

In situations like this, a common way of dealing with the problem is to use some kind of average values instead of the original values. Even though the corresponding system gives a good approximation to the original system, this approach is not satisfactory.

The following may be few reasons:

(i) We might be interested to understand the effect of fluctuations of parameters on the solution.

(ii) There is also a question, ’Is there any noise threshold such that if the noise exceeds a limit then the averaged model is not acceptable?’

Therefore, it is necessary to consider the fluctuations in order to get a good mathematical model. By taking the parameter with insufficient information as the stochastic quantity, we get a system model by stochastic differential equations.

Depending on the number of parameters it is classified as stochastic ordinary or stochastic partial differential equations [14].

Controls are introduced in a system in order to improve the performance of the system in order to achieve the desired goal. Combining the two phenomena, we obtain stochastic control problems.

There are two approaches that are commonly used in solving stochastic optimal control problems: Pontryagin’s maximum principle and Bellman’s dynamic pro- gramming. There are, however, some difficulties of applying finite dimensional techniques to stochastic partial differential equations in infinite dimensions. These are due to the invalidity of Peano’s theorem in the infinite dimensional case and the appearance of unbounded operators.

These difficulties lead us to adapt the methods from functional analysis. According to Mandrekar and Gawarecki ([13]) there are two approaches to deal these type of problems. The first method involves semigroups generated by unbounded operators and mild solutions. The second, a variational approach.

In this thesis, we give a mathematical background of solving a linear quadratic control problem for the heat equation, which involves noise on the boundary in a concise way. Here we use the semigroup approach in order to solve the problem.

2 Preliminaries

In this section we give preliminaries on stochastic calculus and semigroup theory which are required for this thesis. For more details, the reader may refer to [4],[6],[8],[13],[18],[19].

2.1 Elements from Stochastic Calculus

Definition: Let Ω⁰ be a set. Then a σ-algebra on Ω⁰ is a family F of subsets of Ω⁰ such that

• ∅ ∈ F ,

• A ∈ F ⇒ A^C∈ F ,

• A1, A2, · · · ∈ F ⇒ A :=S∞

i=1Ai∈ F .

Definition: Let Ω⁰ be a set and F be the σ-algebra generated by the subsets of Ω⁰. Then the pair (Ω⁰, F ) is called the measurable space.

Definition: A probability measure P on a measurable space (Ω⁰, F ) is a function P : F → [0, 1] such that

• P(∅) = 0,

• P(Ω⁰) = 1,

• P

 S∞

i=1Ai



=P∞

i=1P(Ai).

Definition: For a given set Ω⁰, σ-algebra F and probability measure P, the triple (Ω⁰, F , P) is called a probability space.

Definition: A complete probability space is a probability space (Ω⁰, F , P) if F contains all subsets A of Ω⁰ with P-outer measure zero. That is,

P(A) = inf{P(F ); F ∈ F , A ⊂ F } = 0.

Definition: Let (Ω⁰, F , P) be a given probability space. A function f : Ω⁰→ Rⁿ is called F -measurable if for all open sets U ∈ Rⁿ, we have,

f⁻¹(U ) = {ω ∈ Ω⁰; f (ω) ∈ U } ∈ F .

Definition: An F -measurable function X : Ω⁰→ Rⁿ is called a random variable.

Definition: A parameterized collection of random variables (X_t)_{t∈T =[0,∞)}defined on a probability space (Ω⁰, F , P) taking values in Rⁿ is called a stochastic process.

Definition: Let (F_t)_t≥0 be an increasing family of σ-algebras of subsets of Ω⁰ that is frequently named filtration. A process f (t, w) : [0, ∞) × Ω⁰ → Rⁿ is called Ft adapted if w → f (t, w) is Ft-measurable for each t ≥ 0.

Definition: A filtration Ft, t ∈ [0, T ] on a probability space (Ω⁰, F , P) is called normal if

(i) F₀contains all elements A ∈ F with P(A) = 0.

(ii) Ft= Ft+=T

s>tFsfor all t ∈ [0, T ].

Definition: A one-dimensional Wiener process is a continuous-time stochastic process such that

• W0= 0 a.s,

• Wthas independent increments for every t > 0,

• Wt+h− Wt∼ N (0, h),

• Wthas continuous paths.

Definition: Let H be a Hilbert space with the inner product h, i. An operator A ∈ L(H),the set of all bounded linear operators from H into H, is called sym- metric if hAh, gi_H = hh, Agi_H for all h, g ∈ H. Moreover, A ∈ L(H) is called nonnegative if hAh, hi ≥ 0 for all h ∈ H.

Proposition 2.1. Let H be a separable Hilbert space and let Q ∈ L(H) be nonneg- ative, symmetric and with finite trace. Then there exists a complete orthonormal system {e_k} in H and a bounded sequence of non negative real numbers λ_k such that

Qe_k= λ_ke_k, k = 1, 2, . . . .

Definition: Let H be a separable Hilbert space and let Q ∈ L(H) be nonnegative, symmetric and with finite trace. A stochastic process W (t), t ∈ [0, T ] with values in H on a probability space (Ω⁰, F , P) is called a Q-Wiener process if

• W (0) = 0,

• W (t) has P-a.s continuous trajectories,

• W (t) has independent increments,

• The increments have the following Gaussian laws:

L(W (t) − W (s)) = N (0, (t − s)Q) for all 0 ≤ s ≤ t ≤ T .

Representation of Q-Wiener process: Let H be a separable Hilbert space and let e_k, k ∈ N be an orthonormal basis of H consisting of eigenvectors of Q with corresponding eigenvalues λ_k, k ∈ N. Then a H-valued stochastic process W (t), t ∈ [0, T ], is a Q-Wiener process if and only if

W (t) =X

k∈N

pλkβk(t)ek, t ∈ [0, T ],

where βk, k ∈ N are independent real-valued Brownian motions on a probability space (Ω⁰, F , P) and the above series converges in L²(Ω⁰, F , P).

Definition: A Q-Wiener process W (t), t ∈ [0, T ] with respect to a filtration Ft, t ∈ [0, T ] is a Q-Wiener process such that

(i) W (t), t ∈ [0, T ] is F_t-adapted t ∈ [0, T ] and

(ii) W (t) − W (s) is independent of Fsfor all 0 ≤ s ≤ t ≤ T .

Proposition 2.2. Let W (t), t ∈ [0, T ] be an arbitrary H-valued Q-Wiener process on a probability space (Ω⁰, F , P). Then it is a Q-Wiener process with respect to a normal filtration Ft, t ∈ [0, T ] given by

Ft:=\

s>t

F˜_s⁰, t ∈ [0, T ],

where ˜F_t⁰:= σ( ˜Ft∪ N ), ˜Ft:= σ(W (s)/s ≤ t) and N := {A ∈ F /P(A) = 0}.

Stochastic Integral in Hilbert Spaces:

Let (H, h, i_H) and (K, h, i_K) be two real separable Hilbert spaces.

Definition: A process Φ(t), t ∈ [0, T ] with values in L(H, K) defined on a probability space (Ω⁰, F , P) with normal filtration F^t, t ∈ [0, T ] is said to be elementary if there exists 0 = t0< t1· · · < tk= T, k ∈ N such that

Φ(t) =

k−1

X

m=0

Φ_m1]t_m,t_m+1](t), t ∈ [0, T ]

where

(i) Φ_m: Ω⁰ → L(H, K) is Ftm-measurable with respect to the Borel σ-algebra on L(H, K), 0 ≤ m ≤ k − 1.

(ii) Φmtakes only a finite number of values in L(H, K), 1 ≤ m ≤ k − 1.

The class of all elementary processes with values in L(H, K) is denoted by E . Definition: The stochastic integral for an elementary process Φ(t), t ∈ [0, T ] is defined as

Z t 0

Φ(s) dW (s) :=

k−1

X

m=0

Φm(W (tm+1∧ t) − W (tm∧ t)), t ∈ [0, T ].

Proposition 2.3. Let Φ ∈ E . Then the stochastic integral Rt

0Φ(s) dW (s), t ∈ [0, T ], defined as in the above definition, is a continuous square integrable martin- gale with respect to Ft, t ∈ [0, T ].

A proof can be found in [19].

Definition: Let (H, h, i_H) and (K, h, i_K) be two separable Hilbert spaces. A bounded linear operator A : H → K is called Hilbert-Schmidt if

X

k∈N

kAekk²< ∞,

where ek, k ∈ N is an orthonormal basis of H. We denote the space of all Hilbert- Schmidt operators from H to K by L2(H, K).

Proposition 2.4. If Q ∈ L(H) is nonnegative and symmetric then there exists exactly one element Q^1/2 ∈ L(H) nonnegative and symmetric such that Q^1/2◦ Q^1/2= Q. Moreover, if tr Q < ∞, then Q^1/2∈ L2(H) where kQ^1/2k²= tr Q and L ◦ Q^1/2∈ L2(H, K) for all L ∈ L(H, K).

Definition: [13] Let W (t), t ∈ [0, T ] be a H-valued Q-Wiener process. Let us denote H0= Q^1/2(H) be a subspace of H with the inner product

hh, gi₀=D

Q^−1/2h, Q^−1/2gE

H

where h, g ∈ H₀and Q^−1/2 is the pseudo inverse of Q^1/2 in the case that Q is not one-to-one. Then (H₀, h, i₀) is a separable Hilbert space.

Let us denote the space of all Hilbert-Schmidt operators from H0 to K by L2(H0, K) which is also a separable Hilbert space and is called L⁰₂. We also denote

N_W(0, T ; K) =



Φ : [0, T ] × Ω⁰→ L⁰₂| Φ is predictable with P

 Z T 0

kΦ(s)k²_L0

2 ds < ∞



= 1

 .

We define the stochastic integralRt

0Φ(s) dW (s), 0 ≤ t ≤ T , for Φ ∈ NW(0, T ; K) by

Z t 0

Φ(s) dW (s) = Z T

0

Φ(s)1[0,t](s) dW (s).

Proposition 2.5. [Itˆo Isometry] For every Φ ∈ NW(0, T ; K), we have

E

Z t 0

Φ(s) dW (s)

2

= E Z t

0

kΦ(s)k²_L0 2ds.

2.2 Elements from Semigroup theory

Let us start this section by giving a short note on linear evolution equations and semigroups.

Let A : D(A) ⊂ X → X be a closed linear operator from a Banach space X into itself. Consider the initial value problem

x⁰(t) = Ax(t), t ≥ 0 , x(0) = x₀∈ X. (2.1) If A is bounded, then the solution to (2.1) is given by,

x(t) = e^tAx0. If we denote S(t) = e^tA, then we have,

S(t + s) = e^(t+s)A= e^tAe^sA= S(t)S(s), for all t, s ∈ R, and S(0) = I This implies that S(t) generates a group on X under composition.

If A is unbounded then the solution of (2.1) is still denoted by, x(t) = e^tAx₀= S(t)x₀.

Since S(t + s) = S(t)S(s), S(0) = I for t ≥ 0 and s ≥ 0, we say that S is a semigroup. If for fixed x ∈ H, t 7→ S(t)x is continuous then S is said to be strongly continuous, here denoted by C0.

In the case that if X is a finite dimensional space, the operator A is a matrix and hence it is identical with the derivative of S(·) at 0. That is,

dS dt = lim

h↓0

S(h) − I

h = A.

In general case, A is called the infinitesimal generator of S(t), t ≥ 0, D(A) contains all x ∈ X for which the limit

lim

h↓0

S(h)x − x h exists.

Now we have the following definitions.

Definition: Let X be a Banach space. A semigroup S(t), 0 ≤ t < ∞, is a one parameter family of bounded linear operators from X to X such that

(i) S(0) = I, the identity operator on X.

(ii) S(t + s) = S(t)S(s) for every t, s ≥ 0.

Definition: The infinitesimal generator of a semigroup S(t) is a linear operator A defined by

D(A) =



x ∈ X : lim

t↓0

S(t)x − x t



and

Ax = lim

t↓0

S(t)x − x

t ,

where D(A) is the domain of A.

Definition: A semigroup S(z), which is analytic in some sector ∆ containing the nonnegative real axis, that is ∆ = {z : ϕ1< arg z < ϕ2, −π/2 < ϕ1< 0 < ϕ2<

π/2}, is called an analytic semigroup.

Definition: Consider a linear operator A : H1→ H2 between two Hilbert spaces.

The adjoint operator of A is the operator A^∗: H₂→ H1 such that hAh1, h2i_H

2 = hh1, A^∗h2i_H

1.

Definition: A linear operator A with dense domain in a Hilbert space H is said to be self-adjoint if A = A^∗.

Fractional power operators:

Let A be a densely defined closed linear operator for which the resolvent set ρ(A) ⊃ Σ⁺= {λ : 0 < ω < |arg λ| ≤ π} ∪ V (2.2) where V is a neighborhood of zero and

kR(λ : A)k ≤ M

1 + |λ| for λ ∈ Σ⁺. (2.3) If M = 1 and ω = π/2, then −A is the infinitesimal generator of a C0 semigroup.

If ω < π/2, then −A is the infinitesimal generator of an analytic semigroup.

For an operator A satisfying the assumption (2.2) and α > 0, we define A^−α= 1

2πi Z

C

z^−α(A − zI)⁻¹ dz (2.4)

where the path runs in the resolvent set of A from ∞e^−iv to ∞e^iv, ω < v < π avoiding the negative real axis and the origin and z^−α is taken to be positive for real positive values of z.

For every α > 0, the integral (2.4) converges and defines a bounded linear operator A^−α. In addition, for 0 < α < 1, we have,

A^−α= sin πα π

Z ∞ 0

t^−α(tI + A)⁻¹ dt. (2.5) Let us give some properties of the fractional power operator.

Theorem 2.6. Let A satisfy the assumption (2.2) with ω < π/2. For every α > 0, we define A^α= (A^−α)⁻¹. Then

(i) A^α is a closed linear operator with domain D(A^α) = R(A^−α)=range of A^−α.

(ii) D(A^α) ⊂ D(A^β) for α ≥ β > 0.

(iii) D(A^α) = X for every α ≥ 0.

(iv) If α, β are real, then A^α+βx = A^αA^βx for every x ∈ D(A^γ), where γ = max (α, β, α + β).

A proof may be found in [18].

Theorem 2.7. Let 0 < α < 1. If x ∈ D(A) then A^αx = sin πα

π Z ∞

0

t^α−1A(tI + A)⁻¹x dt. (2.6) A proof may be found in [18].

Theorem 2.8. Let −A be the infinitesimal generator of an analytic semigroup S(t). If 0 ∈ ρ(A) then

(i) S(t) : X → D(A^α) for every t > 0 and α ≥ 0.

(ii) For every x ∈ D(A^α) we have S(t)A^αx = A^αS(t)x.

(iii) For every t > 0, the operator A^αS(t) is bounded and kA^αS(t)k ≤ Mαt^−αe^−δt, for some δ > 0.

(iv) Let 0 < α ≤ 1 and x ∈ D(A^α) then kS(t)x − xk ≤ Cαt^αkA^αxk.

A proof may be found in [18].

Definition: A bilinear form a : H × H → R, where H is a real Hilbert space, is called coercive if there exists a constant C > 0 such that a(x, x) ≥ Ckxk²for all x in H.

3 Problem formulation

We take a particular case of problem by Benner and Trautwein [3] to get a problem formulation.

Consider heat distribution of a system in a given region at a finite time interval.

The dynamic of the system can be described by a parabolic partial differential equation.

We consider the system in a Hilbert space H = L²(Ω), where Ω is an open bounded region in Rⁿ with a smooth boundary ∂Ω, and consider the time interval (0, T ).

Let x(t, ξ) denote the heat distribution of the system at a point ξ at time t. Let us assume that the initial value at t = 0 is given by x0(ξ) ∈ L²(Ω). Then the evolution system is given by

(_∂

∂tx(t, ξ) = ∆x(t, ξ), (t, ξ) ∈ (0, T ) × Ω,

x(0, ξ) = x0(ξ), ξ ∈ Ω. (3.1)

where ∆ is the Laplace operator in L²(Ω).

Let ν be the outward normal to the boundary ∂Ω. Then the system (3.1) with Neumann boundary conditions is given by







∂

∂tx(t, ξ) = ∆x(t, ξ), (t, ξ) ∈ (0, T ) × Ω, x(0, ξ) = x0(ξ), ξ ∈ Ω,

∂

∂νx(t, ξ) = 0, (t, ξ) ∈ (0, T ) × ∂Ω.

(3.2)

In order to optimize the system we introduce a control u ∈ L²([0, T ] × ∂Ω) on the boundary. Now the system (3.2) becomes,







∂

∂tx(t, ξ) = ∆x(t, ξ), (t, ξ) ∈ (0, T ) × Ω, x(0, ξ) = x0(ξ), ξ ∈ Ω,

∂

∂νx(t, ξ) = u(t, ξ), (t, ξ) ∈ (0, T ) × ∂Ω.

(3.3)

We define the Neumann realization A of the Laplace operator by Av = ∆v, for v ∈ D(A), where D(A) =



v ∈ H²(Ω) : ∂

∂νv = 0 on ∂Ω

 . and H²(Ω) denotes the Sobolev space restricted to L²(Ω).

From the properties of A, we have that A is a non-positive, self adjoint operator in L²(Ω) and D(A) is dense in L²(Ω). Thus A is the infinitesimal generator of an analytic semigroup e^At, t ≥ 0 [8], Section 4 and by Lumer-Phillips theorem, the analytic semigroup is a contraction semigroup. Moreover, it is possible to define the fractional power operator of A of order α for 0 < α < 1 [4], Part II, Chapter 1, Section 4 & 5. Therefore, we have

D((λ − A)^α) =

(H^2α if α ∈ (0, 3/4),

{h ∈ H^2α(Ω) :_∂ν^∂ h = 0 on ∂Ω} if α ∈ (3/4, 1),

for all λ > 0, that is, for λ in the resolvent of A, where (λ − A)^α denotes the fractional power operator of λ − A, [17].

Let us define the Neumann operator N : L²(∂Ω) → L²(Ω) by g = N v with (∆g(ξ) = λg(ξ) for ξ ∈ Ω,

∂

∂νg(ξ) = v(ξ) for ξ ∈ ∂Ω, where λ > 0, v ∈ H²(Ω), g ∈ L²(Ω) is an eigen vector of ∆.

According to [16], Chapter 1, N ∈ L(L²(∂Ω); H^3/2(Ω)) and hence N ∈ L(L²(∂Ω); D((λ− A)^α)) if α ∈ (0, 3/4). In addition, by the closed graph theorem, the operator (λ−A)^αN is

linear and bounded. Moreover, we have the following proposition for the properties of the fractional power operator of (λ − A).

Proposition 3.1. For any β ≥ 0, λ > 0 and any γ, δ ∈ R, we have a) (λ − A)^β is closed.

b) e^At: L²(Ω) → D((λ − A)^β) if t > 0.

c) D((λ − A)^β) is dense in L²(Ω).

d) (λ − A)^βe^Ath = e^At(λ − A)^βh if h ∈ D((λ − A)^β).

e) The operator (λ − A)^βe^At is bounded and for every h ∈ L²(Ω), we have, (λ − A)^βe^Ath

_L2(Ω)≤ ^M_tβ^βkhk_L2(Ω) for t > 0.

f ) (λ − A)^γ+δh = (λ − A)^γ(λ − A)^δh if h ∈ D((λ − A)^), where  = max{γ, δ, γ + δ}.

Proof can be found in [18], Section 2.6, Theorem 6.13.

From the above arguments, the abstract formulation of (3.3) is given by (dx(t) = [Ax(t) + (λ − A)N u(t)]dt,

x(0) = x0, (3.4)

or in the mild form as [4],

x(t) = e^Atx₀+ Z t

0

(λ − A)e^A(t−s)N u(s)ds. (3.5) Since N ∈ L(L²(∂Ω); D((λ − A)^α)) for α ∈ (0, 3/4), from Proposition 3.1(d) we have that equation (3.5) can be rewritten as,

x(t) = e^Atx₀+ Z t

0

(λ − A)^1−αe^A(t−s)(λ − A)^αN u(s)ds.

Let (Ω⁰, F , P) be a complete probability space associated with the filtration (Ft)_{t∈[0,T ]}. In a further step let us assume that the control u itself is affected by a noise which is given by Q-Wiener process. In distributional sense the stochastic boundary value problem reads







∂

∂tx(t, ξ) = ∆x(t, ξ), (t, ξ) ∈ (0, T ) × Ω, x(0, ξ) = x₀(ξ), ξ ∈ Ω,

∂

∂νx(t, ξ) = u(t, ξ) + w(t, ξ), (t, ξ) ∈ (0, T ) × ∂Ω.

(3.6)

where w(t, ξ) is white noise associated with the Q-Wiener noise W . The abstract formulation takes the linear form

(dx(t) = [Ax(t) + (λ − A)N u(t)]dt + (λ − A)N dW (t),

x(0) = x0, (3.7)

or the mild form as

x(t) = e^Atx₀+ Z t

0

(λ − A)^1−αe^A(t−s)(λ − A)^αN u(s) ds

+ Z t

0

(λ − A)^1−αe^A(t−s)(λ − A)^αN dW (s).

where (x(t))_{t∈[0,T ]} is an L²(Ω) valued stochastic process which satisfies the above stochastic partial differential equation(SPDE), x₀is an F₀-measurable random variable and W (t) is Q-Wiener process with values in L²(∂Ω).

Now we consider the quadratic cost functional in the form

J (x0, u) = 1

2E kx(T ) − ˆxk²_L2(Ω)+k

2E kuk²L²([0,T ];L²(∂Ω)),

where (x(t))_{t∈[0,T ]}is the mild solution of the system (3.7), ˆx ∈ L²(Ω) is the target function and k > 0 is parameter.

The aim of this thesis is to obtain an optimal control of the system (3.7) by minimizing the above cost functional. Let us start with linear regulators in finite dimensions. We follow [20] to present our arguments.

4 Linear Quadratic Optimal Control: Determin- istic Cases

4.1 Introduction: A Motivating Example

A linear quadratic regulator problem is a problem of minimizing a quadratic cost functional of the form

J (u) = 1 2

Z T 0

[x^T(t)Q(t)x(t) + u^T(t)R(t)u(t)]dt + x^T(T )Sx(T )

over all continuous functions u : [0, T ] → R^m defined on a fixed interval [0, T ] subject to a linear dynamics

˙

x(t) = A(t)x(t) + B(t)u(t), x(0) = x0,

where A(·), B(·), R(·) and Q(·) are continuous functions on the interval [0, T ] and the matrices R(·) and Q(·) are symmetric, R(·) is positive definite, Q(·) is positive semidefinite and S is a constant positive definite matrix.

For example, consider a linear system

˙

x = Ax + Bu, x(0) = x₀,

where x ∈ Rⁿ, u ∈ R^m, A is an n × n matrix, and B is an n × m matrix, with the quadratic cost function

J = Z T

0

[x^TQx + u^TRu]dt + x^T(T )Sx(T ),

where Q is an n × n symmetric positive semi-definite matrix, R is an m × m symmetric positive definite matrix and S is an n × n positive semi-definite matrix.

The objective is to minimize J over all measurable controls u : [0, T ] → R^m. We use Pontryagin’s maximum principle. For that we construct the Hamiltonian function H by,

H = x^TQx + u^TRu + λ(Ax + Bu),

where λ : [0, T ] → Rⁿ, a row vector which is referred as a co-state variable.

By the maximum principle, the necessary conditions for an optimal control are given by,

˙ x = ∂H

∂λ = Ax + Bu, x(0) = x0,

˙λ = −∂H

∂x = −x^TQ − λA, λ(T ) = ∂

∂x(x^T(T )Sx(T )) = x^T(T )S,

and

∂H

∂u = u^TR + λB = 0,

⇒ u = −R⁻¹B^Tλ^T.

If we substitute λ^T = µ, then we have to solve the following two point boundary value problem in order to solve for an optimal control.

˙

x = Ax − BR⁻¹B^Tµ, x(0) = x0,

˙

µ = −Qx − A^Tµ, µ(T ) = Sx(T ).

By setting µ(t) = P (t)x(t), where P (t) ∈ R^n×n, we have,

˙

µ = P ˙x + ˙P x,

= P (Ax − BR⁻¹B^Tµ) + ˙P x,

⇔ −Qx − A^TP x = P Ax − P BR⁻¹BP x + ˙P x,

⇔ P x = −P Ax + P BR˙ ⁻¹B^TP x − Qx − A^TP x.

This equation is satisfied if there exists a P (t) such that, ( ˙P = −P A + P BR⁻¹B^TP − Q − A^TP,

P (T ) = S.

The above equation is called the Riccati differential equation which can be solved backwards in time.

Thus the optimal control is given by,

u = −R⁻¹B^TP (t)x.

In this section we considered a finite dimensional system for which we obtained an optimal control. More details about this section may be found in [20]. However, in reality, there is large number of systems which are represented with infinite number of parameters. In the next section we will discuss the linear quadratic control of a system in Hilbert spaces. A Semi-group approach is used to deal with these systems. Here we follow J. Zabczyk, 1995 [21] and give the details more clearly.

4.2 Linear Quadratic Optimal Control in Hilbert Spaces

Let H and U be real, separable Hilbert spaces for the state and control parameters with norm and inner product denoted by | · | and h, i.

Consider a linear system

˙

x = Ax + Bu, x(0) = x0∈ H, (4.1)

where A is the infinitesimal generator of a semigroup S(t), t ≥ 0, u ∈ L²(0, T ; U ) and B is a continuous linear operator from U to H.

The linear quadratic problem is to find a control u which minimizes the cost functional

J = Z T

0

(hQx(s), x(s)i + hRu(s), u(s)i)ds + hP0x(T ), x(T )i , (4.2) where Q ∈ L(H, H), P0∈ L(H, H), R ∈ L(U, U ) be fixed self-adjoint continuous operators and in addition, R is invertible with continuous inverse R⁻¹.

A solution to equation (4.1) in a weak sense is given by, x(t) = S(t)x₀+

Z t 0

S(t − s)Bu(s) ds, t ≥ 0.

We consider the optimal control in the closed form given by, u(t) = K(t)x(t), t ≥ 0,

where K(·) is a function with values in L(H, U ).

Now the above solution can be written as x(t) = S(t)x₀+

Z t 0

S(t − s)BK(s)x(s) ds, t ≥ 0.

Lemma 4.1. A function K : [0, T ] → L(H, U ) is strongly continuous if for arbitrary h ∈ H the function K(t)h, t ∈ [0, T ] is continuous. If K(·) is strongly continuous on [0, T ] then equation

x(t) = S(t)x0+ Z t

0

S(t − s)BK(s)x(s) ds, t ≥ 0, has exactly one continuous solution x(t), t ∈ [0, T ].

In order to solve the optimal control problem, let us start from an analysis of an infinite dimensional version of the Riccati equation.

( ˙P = A^∗P + P A + Q − P BR⁻¹B^∗P,

P (0) = P0. (4.3)

Definition 4.1. A function P (t), t ≥ 0 with values in L(H, H), P (0) = P₀is a solution of (4.3) if for arbitrary g, h ∈ D(A), the function hP (t)h, gi , t ≥ 0 is absolutely continuous and

d

dthP (t)h, gi = hP (t)h, Agi + hP (t)Ah, gi + hQh, gi −P BR⁻¹B^∗P h, g (4.4) for almost all t ≥ 0.

Let us consider the equation

P = A˙ ^∗P + P A + Q(t), t ≥ 0, P (0) = P₀, (4.5) where Q(t), t ≥ 0 are continuous, self-adjoint operators.

Definition 4.2. An operator valued function P (t), t ≥ 0, P (0) = P₀is a solution of (4.5) if for arbitrary g, h ∈ D(A), the function hP (t)h, gi , t ≥ 0 is absolutely continuous and

d

dthP (t)h, gi = hP (t)h, Agi + hP (t)Ah, gi + hQh, gi (4.6) for almost all t ≥ 0.

Lemma 4.2. A strongly continuous operator valued function P (t), t ≥ 0 is a solution to (4.5) if and only if for arbitrary h ∈ H, P (t), t ≥ 0 is a solution of the integral equation

P (t)h = S^∗(t)P₀S(t)h + Z t

0

S^∗(t − s)Q(s)S(t − s)hds, t ≥ 0. (4.7)

Proof. Since the integralRt

0S^∗(t − s)Q(s)S(t − s)hds exists, the equation (4.7) is well-defined and P (t) is strongly continuous. Moreover,

hP (t)h, gi = hP₀S(t)h, S(t)gi + Z t

0

hQ(s)S(t − s)h, S(t − s)gi ds, t ≥ 0.

Since A is the infinitesimal generator of S(t), we have d

dthP₀S(t)h, S(t)gi = hP₀S(t)Ah, S(t)gi + hP₀S(t)h, S(t)Agi , t ≥ 0 and

d

dthQ(s)S(t − s)h, S(t − s)gi = hQ(s)S(t − s)Ah, S(t − s)gi + hQ(s)S(t − s)h, S(t − s)Agi , 0 ≤ s ≤ t.

Hence, d

dthP (t)h, gi = hP0S(t)Ah, S(t)gi + hP0S(t)h, S(t)Agi + hQ(t)h, gi +

Z t 0

[hQ(s)S(t − s)Ah, S(t − s)gi + hQ(s)S(t − s)h, S(t − s)Agi]ds

= hS^∗(t)P0S(t)Ah, gi + hS^∗(t)P0S(t)h, Agi + hQ(t)h, gi +

Z t 0

hS^∗(t − s)Q(s)S(t − s)Ah, gi ds + Z t

0

hS^∗(t − s)Q(s)S(t − s)h, Agi ds.

Using (4.7), we get d

dthP (t)h, gi = hP (t)Ah, gi + hP (t)h, Agi + hQ(t)h, gi . Conversely let us assume that (4.6) holds. That is,

d

dthP (t)h, gi = hP (t)Ah, gi + hP (t)h, Agi + hQ(t)h, gi .

Now for s ∈ [0, t], d

dshP (s)S(t − s)h, S(t − s)gi = hP (s)S(t − s)Ah, S(t − s)gi + hP (s)S(t − s)h, S(t − s)Agi + hQ(s)S(t − s)h, S(t − s)gi − hP (s)S(t − s)Ah, S(t − s)gi

− hP (s)S(t − s)h, S(t − s)Agi

= hQ(s)S(t − s)h, S(t − s)gi . Integrating from 0 to t, we obtain,

[hP (s)S(t − s)h, S(t − s)gi]^t₀= Z t

0

hQ(s)S(t − s)h, S(t − s)gi ds hP (t)h, gi − hP₀S(t)h, S(t)gi =

Z t 0

hQ(s)S(t − s)h, S(t − s)gi ds

⇔ hP (t)h, gi = hS^∗(t)P0S(t)h, gi + Z t

0

hS^∗(t − s)Q(s)S(t − s)h, gi ds.

Since D(A) is dense in H, the above equation holds for arbitrary h, g ∈ H.

Immediately we get the following theorem.

Theorem 4.3. A strongly continuous operator valued function P (t), t ≥ 0 is a solution to (4.3) if and only if, for arbitrary h ∈ H, P (t), t ≥ 0 is a solution to the following integral equation

P (t)h = S^∗(t)P0S(t)h + Z t

0

S^∗(t − s)(Q − P (s)BR⁻¹B^∗P (s))S(t − s)hds, t ≥ 0.

(4.8) Next we prove the existence and uniqueness of a global solution to the equation (4.3). We need the following lemmas.

Lemma 4.4. Let A be a transformation from a Banach space C into C, v an ele- ment of C and α a positive number. If A(0) = 0, kvk ≤ ¹₂α and kA(p1) − A(p2)k ≤

1

2kp1− p2k, if kp1k ≤ α, kp2k ≤ α then equation A(p) + v = p has exactly one solution p satisfying kpk ≤ α.

[Proof can be found in [21]].

Lemma 4.5. Let us assume that a function P (t), t ∈ [0, T0] is a solution to (4.8).

Then for arbitrary control u(·) and the corresponding output x(·), J_T0(x₀, u) = hP (T₀)x₀, x₀i +

Z T0

0

|R^1/2u(s) + R^−1/2B^∗P (T₀− s)x(s)|²ds. (4.9) [Proof can be found in [21]]

Theorem 4.6. (i) Equation (4.3) has exactly one global solution P (s), s ≥ 0. For arbitrary s ≥ 0 the operator P (s) is self-adjoint and nonnegative definite.

(ii) The minimal value of functional (4.2) is equal to hP (T )x₀, x₀i and the optimal control ˜u is given in the feedback form

˜

u(t) = ˜K(t)˜x(t),

K(t) = −R˜ ⁻¹B^∗P (T − t), t ∈ [0, T ].

Proof. First let us prove the existence of a local solution of equation (4.8).

Let CT be the Banach space of all strongly continuous functions P (t), t ∈ [0, T ] having values being self-adjoint operators on H, with the norm

kP (·)k_T = sup{|P (t)|; t ∈ [0, T ]}.

For P (·) ∈ CT, we define AT(P )(t)h = −

Z t 0

S^∗(t − s)P (s)BR⁻¹B^∗P (s)S(t − s)hds and

v(t)h = S^∗(t)P0S(t)h + Z t

0

S^∗(t − s)QS(t − s)hds, h ∈ H, t ∈ [0, T ].

Therefore (4.8) is equivalent to

P = v + AT(P ).

From the definitions of AT and v, we see that AT maps CT into CT and v(·) ∈ CT. Suppose kP1kT ≤ α, kP2kT ≤ α, then

|P1(s)BR⁻¹B^∗P1(s) − P2(s)BR⁻¹B^∗P2(s)| = |P1(s)BR⁻¹B^∗P1(s) − P2(s)BR⁻¹B^∗P2(s) + P₂(s)BR⁻¹B^∗P₁(s) − P₂(s)BR⁻¹B^∗P₁(s)|

≤ |(P1(s) − P₂(s))BR⁻¹B^∗P₁(s)|

+ |P2(s)BR⁻¹B^∗(P2(s) − P1(s))|

≤ 2α|BR⁻¹B^∗|kP1− P2kT. Thus,

kAT(P1) − AT(P2)k ≤ | Z t

0

S^∗(t − s)P1(s)BR⁻¹B^∗P1(s)S(t − s)hds

− Z t

0

S^∗(t − s)P₂(s)BR⁻¹B^∗P₂(s)S(t − s)hds|

≤ 2α|BR⁻¹B^∗|M² Z T

0

e^2wskP1− P2kTds

= αω⁻¹M²|BR⁻¹B^∗|e^{2ωT −1}kP1− P2kT. That is,

kAT(P1) − AT(P2)k ≤ αω⁻¹M²|BR⁻¹B^∗|e^{2ωT −1}kP1− P2kT. If for α > 0, T > 0,

αω⁻¹M²|BR⁻¹B^∗|e^{2ωT −1}| < 1

2, (4.10)

then we have,

kAT(P1) − AT(P2)k ≤ 1

2kP1− P2kT.

Also,

kvkT = kS^∗(t)P0S(t) + Z T

0

S^∗(t − s)QS(t − s)dsk

≤ M²e^2ωT|P₀| + M²|Q|e^{2ωT −1} 2ω . If for α > 0, T > 0,

M²e^2ωT|P0| + M²|Q|e^{2ωT −1} 2ω < α

2, (4.11)

we have

kvk < α 2.

Since the assumptions of Lemma 4.4 are satisfied, we have (4.8) has a unique solution in the set

{P (·) ∈ C_T; sup

t≤T

|P (t)| ≤ α}.

One can find α > 0 such that for given operators P0 and Q and for given numbers ω > 0 and M > 0,

M²|P0| + |Q|M² 2ω < α

2

and therefore one can find T = T (α) > 0 such that (4.10) and (4.11) are satisfied.

From Lemma 4.5, it follows that for arbitrary u(·) ∈ L²(0, T₀; U ), hP (T0)x0, x0i ≤ JT₀(x0, u).

Also from Lemma 4.1, equation

˜

x(t) = S(t)x0− Z t

0

S(t − s)BR⁻¹B^∗P (T0− s)˜x(s) ds, t ∈ [0, T0] has exactly one solution ˜x(t), t ∈ [0, T0]. If we define

˜

u(t) = R⁻¹B^∗P (T₀− t)˜x(t), t ∈ [0, T₀] then ˜x(·) is the corresponding output and by (4.9),

J_T0(x₀, ˜u(·)) = hP (T₀)x₀, x₀i . (4.12) Thus ˜u(·) is the optimal control.

From (4.12), it follows that P (T0) ≥ 0. Now setting u(·) = 0, we get 0 ≤ hP (T₀)x₀, x₀i ≤

Z T0

0

hQS(t)x0, S(t)x₀i dt,

0 ≤ hP (T₀)x₀, x₀i ≤

*Z T0

0

S^∗(t)QS(t)x₀dt, x₀ +

.

Consequently,

|P (T )| ≤ Z T0

0

|S^∗(t)QS(t)|dt for T ≤ T₀.

For a given positive number ˜T let α be a number such that

M² Z T^˜

0

|S^∗(t)QS(t)|dt + M²

2ω|Q| < α 2.

Now by assuming T1 < T (α), we can say there exists a solution to (4.8) on [0, T1]. Repeating the procedure on consecutive intervals [T1, 2T1], [2T1, 3T1], . . ., we obtain there exists a unique global solution to (4.3).

In this section, we gave a general method of dealing a linear quadratic problem in Hilbert spaces. More details about this section may be found in [21]. In the next section, we present a method to solve a linear quadratic problem in Hilbert spaces when the control acts particularly on the boundary. Here we follow [4] and give the details more clearly.

4.3 Linear Quadratic Optimal Control: Control on the Bound- ary

Let H, U and Y be the Hilbert spaces of states, controls, and observations with the inner product h, i and the norm | · |. Let us introduce the following notations.

We denote L(H) to be the set of all bounded linear operators from H into H and define

Σ(H) = {T ∈ H : T is Hermitian},

Σ⁺(H) = {T ∈ Σ(H) : hT x, xi ≥ 0, ∀x ∈ H}.

For any interval I in R, we denote C(I; L(H)) to be the set of all continuous map- pings from I into L(H) and C_s(I; L(H)) to be the set of all strongly continuous mappings from I into Σ(H).

We consider a deterministic dynamical system given by

˙

x(t) = Ax(t) + Bu(t), x(0) = x0∈ H, (4.13) where u ∈ L²(0, T ; U ) and A : D(A) ⊂ H → H generates an analytic semigroup in H. Let us assume that the linear operator B is not a bounded linear operator from U into H. We consider a situation where B maps U into the dual space of D(A^∗). Suppose B is of the form B = (λ − A)D, where D ∈ L(U, H) and λ is an element in the resolvent set ρ(A).

Now (4.13) can be written as,

˙

x(t) = Ax(t) + (λ − A)Du(t), x(0) = x₀

or in the mild form as [4]

x(t) = e^Atx0+ (λ − A) Z t

0

e^(t−s)ADu(s) ds. (4.14) We assume

HP







(i) A generates an analytic semigroup e^tAof type w and λ is a real number in ρ(A) such that w < λ,

(ii) There exists α ∈ (0, 1) such that D ∈ L(U, D(A^α)),

where D(A^α) = D((λ − A)^α) is the domain of the fractional power (λ − A)^α of the operator λ − A.

The above assumption is equivalent to

B = (λ − A)D = (λ − A)^1−α(λ − A)^αD ∈ L(U, (D(A^∗)^1−α)⁰).

where (D(A^∗)^1−α)⁰ is the dual space of D(A^∗)^1−α.

Moreover, by the closed graph theorem, we have (λ − A)^αD is bounded. Using Theorem 2.8, there exists a constant cα> 0 such that

|(λ − A)e^(t−s)ADu(s)| = |(λ − A)^1−αe^(t−s)A(λ − A)^αDu(s)|

≤ c_α(t − s)^α−1|u(s)|, s ∈ [0, t].

Hence equation (4.14) can be written as, x(t) = e^Atx₀+

Z t 0

(λ − A)e^(t−s)ADu(s) ds and

|x(t)| = |e^Atx₀+ Z t

0

(λ − A)e^(t−s)ADu(s) ds|,

≤ |e^tAx0| + cα

Z t 0

(t − s)^α−1|u(s)| ds.

For α > 1/2, from the H¨older estimate, it follows that x ∈ L^∞(0, T ; H). This implies that x ∈ C([0, T ], H).

If we set

xk(t) = e^Atx0+ Z t

0

(λ − A)e^(t−s)AkR(k, A)Du(s) ds,

where R(k, A) is the resolvent of A, then we have x_k∈ C([0, T ]; H) and xk(t) → x(t) uniformly in t.

If α ≤ 1/2, then x /∈ C([0, T ]; H). Hence we consider the case where α > 1/2.

Consider the optimal control problem that minimizes the cost function J (u) =

Z T 0

[C|x(t)|²+ |u(t)|²]dt + hP0x(T ), x(T )i (4.15)

overall u ∈ L²(0, T ; U ) , C ∈ L(H, Y ) and P₀∈ Σ⁺(H).

We study the Riccati equation in order to solve the optimal control problem, (P⁰= A^∗P + P A − P (λ − A)D((λ − A)D)^∗P + C^∗C,

P (0) = P0. (4.16)

If P ∈ Cs([a, b]; Σ⁺(H)), then setting E = (λ − A)^αD and V = (λ − A^∗)^1−αP , the equation (4.16) becomes,

(P⁰= A^∗P + P A − V^∗EE^∗V + C^∗C, P (0) = P0

(4.17)

or in the integral form

P (t)x = e^tA∗P₀e^tAx + Z t

0

e^(t−s)A∗C^∗Ce^(t−s)Ax ds

− Z t

0

e^(t−s)A∗V^∗(s)EE^∗V (s)e^(t−s)Ax ds, x ∈ H.

(4.18)

Let us assume that the assumptions HP are satisfied for α > 1/2 and we set λ = 0. We are interested in the solution of the Riccati equation (4.17) in the given functional space. For any set [a, b], we denote C_s,α([a, b]; Σ(H)) to be the set of all P ∈ Cs([a, b]; Σ(H)) such that:







(i)P (t)x ∈ D((−A^∗)^1−α), ∀x ∈ H, ∀t ∈ (a, b], (ii)(−A^∗)^1−αP ∈ C((a, b]; L(H)),

(iii) lim_t→a(t − a)^1−α(−A^∗)^1−αP (t)x = 0, ∀x ∈ H.

(4.19)

Let us define

kP k1= sup

t∈(a,b]

k(t − a)^1−α(−A^∗)^1−αP (t)k and Cs,α([a, b]; Σ(H)) be associated with the norm

kP kα= kP k + kP k1. We set

Cs,α([a, b]; Σ⁺(H)) = {P ∈ Cs,α([a, b]; Σ(H)) : P (t) ≥ 0, ∀t ∈ (a, b]}

and denote by Cs,α([0, ∞]; Σ(H)) the set of all P ∈ Cs([0, ∞]; Σ(H)) such that P ∈ Cs,α([0, T ]; Σ(H)) for all T > 0.

Definition:

A mild solution of problem (4.17) in [0, T ] is an operator valued function P ∈ C_s,α([0, T ]; Σ(H)) that satisfies the following integral equation

P (t)x = e^tA∗P0e^tAx + Z t

0

e^(t−s)A∗C^∗Ce^(t−s)Ax ds

− Z t

0

e^(t−s)A∗V^∗(s)EE^∗V (s)e^(t−s)Ax ds, x ∈ H.

(4.20)

A weak solution of problem (4.17) in [0, T ] is an operator value function P ∈ Cs,α([0, T ]; Σ(H)) such that for any x, y ∈ D(A), hP (·)x, yi is differentiable in [0,T] and satisfies

(_d

dthP (t)x, yi = hP (t)x, Ayi + hP (t)Ax, yi + hCx, Cyi − hE^∗V (t)x, E^∗V (t)yi , P (0) = P0.

(4.21) Proposition 4.7. Let P ∈ Cs,α([0, T ]). Then P is a mild solution of problem (4.17) if and only if P is a weak solution.

Generalization of the Contraction Mapping Principle:

Let T > 0 and let {γ_k} be a sequence of mappings from C_s,α([0, T ]; Σ(H)) into itself such that

kγk(P ) − γk(Q)kα≤ akP − Qkα

for some a ∈ [0, 1) and all P, Q ∈ Cs,α([0, T ]; Σ(H)). Moreover, assume that there exists a mapping γ from Cs,α([0, T ]; Σ(H)) into itself such that ∀P ∈ Cs,α([0, T ]; Σ(H)), ∀m ∈ N, ∀x ∈ H

(limk→∞γ_k^m(P )x = γ^m(P )x in C([0, T ]; H)

limk→∞t^1−α(−A^∗)^1−αγ^m_k (P )x = t^1−α(−A^∗)^1−αγ^m(P )x in C([0, T ]; H).

(4.22) Then by the contraction mapping principle, there exists unique Pk and P in Cs,α([0, T ]; Σ(H)) such that

γk(Pk) = Pk, γ(P ) = P.

Lemma 4.8. Under the above assumptions on the sequence of mappings {γk}, we have

(limk→∞Pk(·)x = P (·)x in C([0, T ]; H)

limk→∞t^1−α(−A^∗)^1−αPk(·)x = t^1−α(−A^∗)^1−αP (·)x in C([0, T ]; H), ∀x ∈ H, T > 0.

(4.23) We consider the approximating problem for k > λ:

(P_k⁰ = A^∗Pk+ PkA − V_k^∗EkE_k^∗Vk+ C^∗C,

P_k(0) = P₀ (4.24)

where E_k = k(k − A)⁻¹E, V_k = (λ − A^∗)^1−αP_k. Equation (4.24) can be written in the mild form as

P (t)x = e^tA∗P₀e^tAx + Z t

0

e^(t−s)A∗C^∗Ce^(t−s)Ax ds

− Z t

0

e^(t−s)A∗V_k^∗(s)EkE_k^∗Vk(s)e^(t−s)Ax ds, x ∈ H.

(4.25)

Theorem 4.9. Assume that the assumptions HP are satisfied for α > 1/2. Then the problem (4.17) has a unique mild solution P ∈ Cs,α([0, T ]; Σ(H)). Moreover, the solution Pk of problem (4.25) also belongs to Cs,α([0, T ]; Σ(H)) for all k ∈ N and (4.23) holds for all x ∈ H and T > 0.

Proposition 4.10. Let u ∈ L²(0, T ; U ) and let x be the solution to the state equation (4.14). Then

hP (t)x0, x0i + Z t

0

|u(s) + E^∗V (t − s)x(s)|² ds

= Z t

0

[|Cx(s)|²+ |u(s)|²] ds + hP0x(t), x(t)i .

(4.26)

Proposition 4.11. Let us assume that the assumptions HP hold. Then there exists a unique solution to the closed loop equation

x(t) = e^tAx0− Z t

0

(−A)^1−αe^(t−s)AEE^∗V (T − s)x(s) ds (4.27) where x ∈ C([0, T ]; H) and α > 1/2 .

Theorem 4.12. Assume that the assumptions HP are satisfied and let x0∈ H.

Then there exists a unique optimal pair (u^∗, x^∗). In addition, we have

(i) x^∗is the mild solution of the closed loop equation (4.27) and x^∗∈ C([0, T ]; H) if α > 1/2.

(ii) u^∗∈ C([0, T ]; U ) is given by the feedback formula:

u^∗(t) = −E^∗V (T − t)x^∗(t) (4.28) (iii) The optimal cost J (u^∗) is given by

J (u^∗) = hP (T )x0, x0i (4.29) Proof. Let u ∈ L²(0, T ; U ) and let x be the solution of the state equation (4.14).

Then by equation (4.26), we have

hP (T )x0, x0i + Z T

0

|u(s) + E^∗V (T − s)x(s)|² ds

= Z T

0

[|Cx(s)|²+ |u(s)|²] ds + hP0x(T ), x(T )i = J (u).

(4.30)

Therefore,

hP (T )x0, x₀i ≤ J (u) for all u ∈ L²(0, T ; U ).

Let x^∗ be the solution of the closed loop equation (4.27). Then by Proposition 4.11, we have x^∗∈ C([0, T ]; H) if α > 1/2.

Now let u^∗ be defined by the equation (4.28). By setting u = u^∗ and x = x^∗ in (4.30), we obtain,

hP (T )x0, x0i + Z T

0

|u^∗(s) + E^∗V (T − s)x^∗(s)|² ds = J (u^∗),

⇔ hP (T )x₀, x₀i = J (u^∗).

Hence u^∗is optimal.

Next we prove the uniqueness. Suppose that there exists another optimal pair (u, x). Then by setting u = u and x = x in (4.30), we obtain

hP (T )x0, x0i + Z T

0

|u(s) + E^∗V (T − s)x(s)|² ds = J (u),

= hP (T )x0, x0i . This implies that

Z T 0

|u(s) + E^∗V (T − s)x(s)|² ds = 0,

⇔ u(s) = −E^∗V (T − s)x(s), for every s ∈ [0, T ],

where x is the mild solution of the system (4.13). Since there exists a unique mild solution, we have x^∗= x and hence u^∗= u.

In this section, we gave a method of solving a linear regulator problem when the control acts on the boundary. More details about this section may be found in [4]. In the next section, we will discuss a particular problem of this type for noisy systems.

5 Linear Quadratic Optimal Control for noisy sys- tems

Let H, U and V denote real, separable Hilbert spaces for the state, control and noise of a dynamical system

(dx(t) = [Ax(t) + Bu(t)]dt + CdW (t), x(0) = x0 ∈ H,

where A : D(A) ⊂ H → H is an infinitesimal generator of an analytic semigroup (e^At)t≥0, W (t) is a noise on the space V , which is defined on a filtered proba- bility space (Ω⁰, F , (Ft)t≥0, P). The operators B : U → H and C : V → H are linear, which may be unbounded, x0is an F0- measurable random variable and u ∈ L²([0, T ], U ) is the control that influence the system to achieve the desired goal.

The linear quadratic problem is to minimize the quadratic cost functional J (u) = 1

2 Z T

0

(hQ(s)x(s), x(s)i + hR(s)u(s), u(s)i)ds + 1

2hM x(T ), x(T )i where Q ∈ L(H, H), R ∈ L(U, U ) and M ∈ L(H, H) are self-adjoint and continu- ous operators.

We continue this section with an example by considering a specific cost functional for which we derive an optimal control. We follow [3] to prove our results and give the details more clearly.

5.1 Linear Quadratic Optimal Control for a Stochastic Heat Equation

Consider the stochastic evolution system that we have formulated in the Section 3 . (dx(t) = [Ax(t) + (λ − A)N u(t)]dt + (λ − A)N dW (t),

x(0) = x0, (5.1)

where (x(t))_{t∈[0,T ]} is an L²(Ω) valued stochastic process which satisfies the above SPDE, x0 is an F0-measurable random variable, W (t) is a Q-Wiener processes with values in L²(∂Ω) and u ∈ U is the control, where

U = {u ∈ L²(Ω⁰; L²([0, T ]; L²(∂Ω))) : (u(t))_{t∈[0,T ]}is Ftadapted}.

Moreover, we assume that the process W (t) is Ftadapted.

Let L2(L²(∂Ω), L²(Ω)) be the space of Hilbert-Schmidt operators from the Hilbert space L²(∂Ω) into the Hilbert space L²(Ω). Let us assume that the norm in L2(L²(∂Ω), L²(Ω)) is denoted by k·k_L2(L²(∂Ω),L²(Ω)). Moreover, let Q ∈ L(L²(∂Ω)) be the kernel covariance operator of the process W (t), t ∈ [0, T ].

Since W is a Q-Wiener process, using Proposition 2.4, there exists a unique operator Q^1/2∈ L(L²(∂Ω)) such that

Q^1/2Q^1/2= Q.