An LQ problem for the heat equation on the halfline with Dirichlet boundary control and noise

# An LQ problem for the heat equation on the halfline with Dirichlet boundary control and noise

G. Fabbri111School of Mathematics and Statistics, UNSW, Sydney, e-mail: fabbri@maths.unsw.edu.au, supported by the ARC Discovery project DP0558539.,   B. Goldys222School of Mathematics and Statistics, UNSW, Sydney, e-mail: goldys@maths.unsw.edu.au, supported by the ARC Discovery project DP0558539.
###### Abstract

We study a linear quadratic problem for a system governed by the heat equation on a halfline with Dirichlet boundary control and Dirichlet boundary noise. We show that this problem can be reformulated as a stochastic evolution equation in a certain weighted space. An appropriate choice of weight allows us to prove a stronger regularity for the boundary terms appearing in the infinite dimensional state equation. The direct solution of the Riccati equation related to the associated non-stochastic problem is used to find the solution of the problem in feedback form and to write the value function of the problem.

Key words: heat equation, Dirichlet boundary conditions, boundary noise, boundary control, weighted space, analytic semigroup, stochastic convolution, linear quadratic control problem, Riccati equation.

MSC 2000: 35G15, 37L55, 49N10.

## 1 Introduction

In this paper we are concerned with a linear quadratic control problem for a heat equation on the halfline with Dirichlet boundary control and boundary noise. More precisely, for fixed , we deal with the equation

 ⎧⎪ ⎪ ⎪⎨⎪ ⎪ ⎪⎩∂∂ty(t,ξ)=∂2∂ξ2y(t,ξ)t∈[τ,T],ξ>0,y(t,0)=u(t)+˙W(t)t∈[τ,T],y(τ,ξ)=x0(ξ)ξ>0. (1)

where is a one dimensional Brownian motion and is a square-integrable control. Let us recall that a deterministic boundary control problem

 ⎧⎪ ⎪ ⎪⎨⎪ ⎪ ⎪⎩∂∂tz(t,ξ)=∂2∂ξ2z(t,ξ)t∈[τ,T],ξ>0,z(t,0)=u(t)t∈[τ,T],z(τ,ξ)=x0(ξ)ξ>0, (2)

is well understood, see for example [BDDM2], [Lasiecka80]). Denoting by the Dirichlet Laplacian in and by the Dirichlet map (defined as in (10) below), we can rewrite (2) in the form

 z(t)=etA0x0+(λ0−A0)∫t0e(t−s)A0Du(s)ds,

and it is easy to show that for all . Therefore, the process

 X(t)=etA0x0+(λ0−A0)∫t0e(t−s)A0Du(s)ds+(λ0−A0)∫t0e(t−s)A0DdW(s) (3)

seems to be a good candidate for a solution to (1). However, it was shown in [DaPratoZabczyk93] that the process is not -valued. More precisely, it was shown that the solution to (1) considered on a finite interval and for , when rewritten in the form (3), is well defined in a negative Sobolev space for only. It is easy to see that the same conclusion holds in the case of halfline. Then it was shown in [AlosBonaccorsi02], see also [BonaccorsiGuatteri02], that the process can be defined pointwise on and it takes values in a weighted space . This fact was used to study some properties of the process (in fact in the aforementioned papers more general nonlinear equations are studied) but the problem is not reformulated as a stochastic evolution equation in and therefore advantages of using the weighted space are somewhat limited.

Following the idea of Krylov [Krylov01] we introduce the weighted spaces , where for we have

 ρ(ξ)=ξ1+θorρ(ξ)=min(1,ξ1+θ),ξ≥0.

It was proved in [Krylov99] and [Krylov01] that the Dirichlet Laplacian defined on extends to a generator of an analytic semigroup on . We will show that the Dirichlet map takes values in for a certain and therefore equation (3), when considered in , can be given a form

 X(t)=etAx0+∫t0e(t−s)A(λ0−A)Du(s)ds+∫t0e(t−s)A(λ0−A0)DdW(s)

that is, we will study a controlled evolution equation

 (4)

for . This fact is a starting point for our analysis of the linear quadratic control problem (1). We will demonstrate that the control problem (4) when considered in the space can be solved using classical by now techniques presented, for example, in [BDDM2]. Let us emphasize that while focus of this paper is on the most interesting case of boundary control and boundary noise a more general control problem

 {dx(t)=(Ax(t)+Bu1(t)+v(t))dt+BdW(t)+dW1(t)x(τ)=x0∈Hρ (5)

with spatially distributed noise and control might be easily considered using the same technique.

Let us note that if the boundary conditions are of Neumann type then the analogue of equation (1) has a solution in and has been studied intensely (also for more general parabolic equations with boundary noise), see for example [DaPratoZabczyk93], [Maslowski95], [DebusscheFuhrmanTessitore07], [dunmas].

We study the linear quadratic problem characterized by the cost functional

 J(τ,x0,u)=E[∫Tτ|Cx(t)|2Y+|u(t)|2Rdt+⟨Gx(T),x(T)⟩Hρ] (6)

and governed by a state equation of the form (4). the operator that appears in (6) is in for a certain Hilbert space and is symmetric and positive. The direct solution of the Riccati equation related to a linear quadratic problem driven by a stochastic equation different from ours was studied in the Neumann case (non-weighted setting) in [Flandoli86] (see also [Ahmed81] and [DaPrato84] for the control inside the domain case ()). Our approach is different from the one used in the aforementioned works since we directly use the solution of the Riccati equation for the “associated” deterministic problem.

The deterministic linear quadratic problem associated to ours is that characterized by the state equation

 ˙x(t)=Ax(t)+Bu(t)

and the functional

 ∫Tτ(|Cx|2Y+|u|2R)dt+⟨Gx(T),x(T)⟩Hρ. (7)

It is well known, see [BDDM2] and Section 3 below, that the solution to the linear quadratic problem given above is determined by the operator-valued function which solves the so-called Riccati equation

 {P′(t)=−A∗P(t)−P(t)A∗−C∗C+P(t)ABB∗A∗P(t)P(T)=G. (8)

Such a problem has been intensely studied (see [BDDM2] and [LasieckaTriggiani00] and the references therein). We will refer in particular to the direct solution approach and we will use the formalism introduced in Section 2.2. of [BDDM2]. We show that the Riccati equation (8) has a unique solution in the space (see Definition 3.2). Let us note that in the deterministic case the minimum of the cost functional (7) is given by .

In the study of the problem with boundary noise some of the tools and the results of the deterministic case, as the properties of the elements of and the solution of (8), are still useful. It is possible to express the value function and the optimal feedback in terms of . A term due to the noise appears in the expression of the minimal cost and we have that (Theorem 3.7):

 (9)

## 2 The heat equation in Hρ

### 2.1 Notation

We will work in a weighted space , where either or for some and . All the results proved in the sequel are valid for both weights and therefore, in order to simplify notations we will use the same notation for both weights. Let us recall that if and only if

 ∫∞0f2(ξ)ρ(ξ)dξ<∞

and is a Hilbert space with the scalar product

 ⟨ϕ,ψ⟩H=∫∞0ϕ(ξ)ψ(ξ)ρ(ξ)dξfor all ϕ,ψ∈H.

Given , the Dirichlet map is defined as follows:

 Dλ(a)=ϕ⟺{(λ−∂2x)ϕ(ξ)=0for all ξ>0ϕ(0)=a (10)

so where

 {ψλ:R+→Rψλ:ξ↦e−√λξ (11)

Clearly .
It is well known that for every the solution to the heat equation with zero Dirichlet boundary condition

 ⎧⎪ ⎪ ⎪⎨⎪ ⎪ ⎪⎩∂∂ty(t,ξ)=∂2∂ξ2y(t,ξ)t>0,ξ>0,y(t,0)=0t≥0,y(0,ξ)=x0(ξ)ξ>0.

is given by the following well known expression

 y(t,ξ)=∫∞0k(t,ξ,η)x0(η)dη (12)

where

 k(t,ξ,η)=1√4πt(e−(ξ−η)24t−e−(ξ+η)24t),η,ξ≥0. (13)

This formula defines the corresponding heat semigroup in . It is also well known that is a symmetric -semigroup of contractions on .

### 2.2 Properties of the heat semigroup on H

###### Proposition 2.1.

For each of the weights considered above, the heat semigroup extends to a bounded semigroup on with generator . The semigroup is analytic.

###### Proof.

The case : .
Let . Then by Theorem 2.5 in [Krylov01] there exists independent of and such that

 ∣∣etAf∣∣H≤C|f|H,t≥0.

Since is dense in , can be extended to and the strong continuity follows by standard arguments. Let be the generator of in and let . Clearly

and is dense in . Therefore is a core for the generator of in . If then

 AetAf=∂2∂ξ2T(t)f

and again by Theorem 2.5 in [Krylov01] we have

 ∣∣∣∂2∂ξ2T(t)f∣∣∣H≤Ct|f|H.

Since is a core for the generator in , the above estimate can be extended to any and therefore

 ∣∣AetAf∣∣H≤Ct|f|Hf∈H.

The last inequality is equivalent to the analyticity of the semigroup in . follows.
The case : .
Let and . Then the functions and are in and for both weights . It follows that

 |(T(t)x|H ≤|T(t)(χ[0,1]x)|H+|T(t)(χ(1,+∞)x)|H (14) ≤|T(t)(χ[0,1]x)|L2ξ1+θ+|T(t)(χ(1,+∞)x)|L2(0,+∞) ≤C|x|H

for a certain . The fact that does not depend on is a consequence of the property of on (showed in the first part of the proof) and on . Therefore has an extension to a semigroup on and the -property follows by standard arguments. Similar arguments yield analyticity of . ∎

###### Lemma 2.2.

Assume that and . Then

 ψλ∈dom((r−A)α)forallα∈[0,12+θ4).

In particular for all .

###### Proof.

We consider the case of only. The other case may be proved by similar if somewhat simpler arguments.

Note first that if then for all 333 denotes the real interpolation space, see for example Theorem 11.5.1 in [MartinezAlix01]. Hence the claim will follow if we show that for

 12−θ4<σ<12. (15)

By Theorem 10.1 of [LionsMagenes72]) if and only if

 ∫∞0t2σ−3∣∣(etA−I)ψλ∣∣2Hdt<∞ (16)

and taking into account (15) it is enough to show that

 I:=∫10t2σ−3∣∣(etA−I)ψλ∣∣2H<∞. (17)

To show (17) we will use (12) and (13) and the definition of . Denoting by the cumulative distribution function of the standard normal distribution, we obtain

 I =∫10t2σ−3∫∞0ξ1+θ∣∣(etA−I)ψλ(ξ)∣∣2dξdt =∫10t2σ−3∫∞0ξ1+θ⎛⎜⎝∫∞0e−(ξ−η)24t√4pte−ληdη−∫∞0e−(ξ+η)24t√4pte−ληdη−eλξ⎞⎟⎠2dξdt =∫10t2σ−3∫∞0ξ1+θ(e−λξeλ2tN(ξ√2t−λ√t)−eλξeλ2t(1−N(ξ√2t+λ√t))−eλξ)2dξdt ≤2(I1+I2+I3)

where , and are respectively

 I1:=∫10t2σ−3∫+∞0ξ1+θ[e−λξ(eλ2t−1)N(ξ√2t−λ√2t)]2dξdt
 I2:=∫10t2σ−3∫+∞0ξ1+θ[e−λξ(N(ξ√2t−λ√2t)−1)]2dξdt
 I3:=∫10t2σ−3∫+∞0ξ1+θ[eλξeλ2t(1−N(ξ√2t+λ√2t))]2dξdt

Since for we have we find that converges for every . can be estimated, using that the standard estimate

 (1−N(s))≤1se−s2/2√2π

as follows:

 I3 ≤∫10t2σ−3∫+∞0ξ1+θe2λξe2λ2t2tξ2e−ξ2/(2t)dξdt ≤C1∫10t2σ−2∫+∞0ξ−1+θe2λξ−ξ2/(2t)dξdt =C1∫10t2σ−2∫+∞0y−1+θtθ−12e2λy√t−y2/2t1/2dξdt ≤C1(∫10t2σ−2+θ2dt)(∫+∞0y−1+θe2λy−y2/2dξ)<∞

where the finiteness of the first term follows from (15). The estimate for can be obtained in a similar way. ∎

### 2.3 Properties of the solution of the state equation

Let be a real Brownian motion on a probability space and let denote the natural filtration of . We need to give a rigorous meaning to equation (1). To this end we will assume in the sequel that

 λ0>0andα∈(12,12+θ4),

are fixed. We will denote by the operator and . By Proposition 2.1 the semigroup is analytic and therefore for any

 ∥(λ0−A)γetA∥H≤Mγt−γfor all t∈(0,T], (18)

see for example [Pazy83] (Theorem 6.13 page 75). By Lemma 2.2 the operator is bounded444For the space is defined as a completion of with respect to the norm . Moreover, for the operator

is bounded as well. We will write Now, we reformulate equation equation (1), still formally, as a stochastic evolution equation in :

 (19)

where the control is chosen in the set of progressively measurable processes endowed with the norm

 ∥u∥2M2W=E∫Tτ|u(t)|2dt<∞.

The next two results show that we can give a meaning to (19).

###### Theorem 2.3.

For all the following holds.
(i) The operator is bounded for each and the function

 t→etABa

is continuous for every .
(ii)

 ∫T0s−γ∥∥(λ0−A)esAψλ0∥∥2Hds<∞. (20)

(iii) For every the process

 WA(t)=∫tτe(t−s)ABdW(s),t∈[τ,T]

is well defined, belongs to and has continuous trajectories in .

###### Proof.

(i) It follows immediately from the definition of and Lemma 2.2 since

 etABa=a(λ0−A)1−αetA(λ0−A)αψλ0,a∈R (21)

(ii) (By 21) and (18) we have for

 ∥∥(λ0−A)esAψλ0∥∥2HS=∣∣(λ0−A)1−αesA(λ0−A)αψλ0∣∣2≤Cs2(1−α)∣∣(λ0−A)αψλ0∣∣2

and the estimate (20) follows immediately for a certain .
(iii) Using (20) with we find immediately that, for every , is well defined and (see for example [DaPratoZabczyk92] Proposition 4.5 page 91)

 E|WA(t)|2H=∫tτ|esA((λ−A)D)|2HSds<∞. (22)

Such an estimate gives also, through standard arguments, the mean square continuity. The continuity follows from (20) for using a factorization argument as in [DaPratoZabczyk93] Theorem 2.3 page 174. ∎

###### Lemma 2.4.

Let be fixed, and . Then the process

 I(t)=∫tτe(t−s)ABu(s)ds,t≤T,

is well defined, , and there exists such that

 E∥I∥2M2W≤C∥u∥2M2W.

Moreover, is in and has continuous trajectories.

###### Proof.

The first part of the Lemma follow from (21) by standard arguments. The mean-square continuity and continuity of follows from (18) and Hölder inequality (since ) in the expression

 I(t)=∫tτ[(λ0−A)1−αe(t−s)A][(λ0−A)αDu(s)]ds.

and the claim follows. ∎

###### Definition 2.5.

Let . An -valued predictable process , defined on is called a mild solution of (19) if

 P[∫Tτ|x(s)|2ds<∞]=1

and

 x(t)=e(t−τ)Ax0+∫tτe(t−s)ABu(s)ds+∫tτe(t−s)ABdW(s)
###### Theorem 2.6.

Equation (19) has a unique mild solution . Moreover, has continuous trajectories -a.s. If then equation (19) defines a Markov process in .

###### Proof.

The properties of the stochastic convolution term come from Lemma 2.3, those of from Lemma 2.4. The Markov property can be proved with standard arguments (see for example [DaPratoZabczyk92] Theorem 9.8 page 249). ∎

### 2.4 The approximating equation

Let . We will approximate using

 xn\lx@stackreldef=Inx. (23)

We have that

 xnC([τ,T];L2(Ω,H))−−−−−−−−−−→x. (24)

We use it to obtain more regularity and to guarantee the existence of a strong solution and then to be able to apply the Ito’s rule (Proposition 3.6). From Proposition 2.6 we know that . We have and then . Furthermore, satisfies the following stochastic differential equation:

 {dxn(t)=(Axn(t)+Bnu(t))dt+BndW(t)xn(τ)=Inx0 (25)

in strong (an then mild) sense (see [DaPratoZabczyk92] Section 6.1). So we have

 xn(t)=Inx0+∫tτAxn(s)ds+∫tτBnu(s)ds+∫tτBndW(s) (26)

## 3 The linear quadratic problem

Let us recall that we work under the assumption

 12<α<12+θ4.

We consider another Hilbert space , an operator and a symmetric and positive . For a fixed we define the set of the admissible controls as . We consider the linear quadratic optimal control problem governed by equation (19) and quadratic cost functional (to be minimized)

 J(τ,x0,u):=E[∫Tτ(|Cx(t)|2Y+|u(t)|2R)dt+⟨Gx(T),x(T)⟩]. (27)

The value function of the problem is

 V(τ,x0):=infu∈UτJ(τ,x0,u)

We consider now the “associated” deterministic linear quadratic problem. It is characterized by the state equation

 {˙x(t)=Ax(t)+Bu(t)x(τ)=x0, (28)

by the set of admissible controls and by the functional

 JDET(τ,x0,u):=∫Tτ(|Cx(t)|2Y+|u(t)|2R)dt+⟨Gx(T),x(T)⟩.

In what follows we we will use the following notations.

###### Notation 3.1.

Note that ([BDDM2] page 137) for every

 supt∈[0,T]∥P(t)∥<∞. (29)

The Riccati equation formally associated with the deterministic control problem (28) has the form

 {P′(t)=−A∗P(t)−P(t)A∗−C∗C+P(t)ABB∗A∗P(t)P(T)=G, (30)

but the concept of solution to this equation requires a rigorous definition. We start with some notations.

###### Definition 3.2.

We denote by the set of all such that

 (i)P(t)x∈D((λ0−A∗)1−α)∀x∈H,∀t∈[0,T)(ii)VP(t)\lx@stackreldef=(λ0−A∗)1−αP(t)∈C([0,T);L(H))(iii)limt→T−((T−t)1−αVP(t)x)=0∀x∈H

Given , the norm is defined as

 |P|α\lx@stackreldef=supt∈[0,T)∥P(t)∥+supt∈[0,T)(T−t)(1−α)∥(λ0−A∗)1−αP(t)∥

It can be proved (see [BDDM2] page 205) that , endowed with the norm , is a Banach space. We will use the notation .

Note that if then (since )

 |P|L2(0,T;L(H))<∞ (31)
###### Definition 3.3.

We say that is a weak solution of the Riccati equation (30) if for all and all

 {ddt⟨P(t)x,y⟩=−⟨P(t)x,Ay⟩−⟨P(t)Ax,y⟩−⟨Cx,Cy⟩+⟨E∗VP(t)x,E∗VP(t)y⟩P(T)=G. (32)

We recall now the existence and uniqueness theorem for the (32):

###### Theorem 3.4.

(i) The Riccati equation (32) has a unique weak solution in in
(ii) is a weak solution of (30) if and only if it solves the following mild equation:

###### Proof.

See [BDDM2] Theorem 2.1 page 207 for the proof of (i) and [BDDM2] Proposition 2.1 page 206 for (ii). ∎

### 3.1 Dynamic Programming

###### Lemma 3.5.

We have that

 ∫T0⟨((λ0−A)ψλ0),P(t)((λ0−A)ψλ0)⟩Hdt<∞
###### Proof.

We use the fact that satisfies the mild equation (33). We have that

 ∫T0⟨((λ0−A)ψλ0),P(t)((λ0−A)ψλ0)⟩Hdt=∫T0E∗(λ0−A∗)1−αP(t)(λ0−A)1−αE(1)dt=I1+I2+I3\lx@stackreldef=∫T0E∗(λ0−A∗)1−αe(T−t)A∗Ge(T−t)A(λ0−A)1−αE(1)dt+∫T0E∗(λ