A Multi-dimensional Stochastic Singular Control Problem Via Dynkin Game and Dirichlet Form

# A Multi-dimensional Stochastic Singular Control Problem Via Dynkin Game and Dirichlet Form

Yipeng Yang Department of Mathematics, University of Missouri-Columbia, Columbia, Missouri, 65211 (yangyip@missouri.edu)
###### Abstract

The traditional difficulty about stochastic singular control is to characterize the regularities of the value function and the optimal control policy. In this paper, a multi-dimensional singular control problem is considered. We found the optimal value function and the optimal control policy of this problem via Dynkin game, whose solution is given by the saddle point of the cost function. The existence and uniqueness of the solution to this Dynkin game are proved through an associated variational inequality problem involving Dirichlet form. As a consequence, the properties of the value function of this Dynkin game implies the smoothness of the value function of the stochastic singular control problem. In this way, we are able to show the existence of a classical solution to this multi-dimensional singular control problem, which was traditionally solved in the sense of viscosity solutions, and this enables the application of the verification theorem to prove optimality. 111The idea of this paper was proposed by Dr. Michael Taksar. Dr. Taksar passed away in February, 2012, however, his contributions should always be remembered.

Key words. Dynkin game, Dirichlet form, Multi-dimensional diffusion, Stochastic singular control

AMS subject classifications. 49J40, 60G40, 60H30, 93E20

## 1 Introduction and Problem Formulation

The characterization of the regularities of value function and optimal policy in stochastic singular control remains a big challenge in stochastic control theory, especially the higher dimensional case, see, e.g., [19]. The traditional approach is to use the viscosity solution technique, see [4] [3] [2], which usually yields a less regular solution. Another approach to solve singular control problems and characterize the regularity of value functions is through variational inequalities and optimal stopping or Dynkin game, see, e.g., Karatzas and Zamfirescu [14], Guo and Tomecek [9]. In [12] Karatzas and Shreve studied the connection between optimal stopping and singular stochastic control of one dimensional Brownian motion, and showed that the region of inaction in the control problem is the optimal continuation region for the stopping problem. In [1], the authors established and exploited the duality between the myopic investor’s problem (optimal stopping) and the social planning problem (stochastic singular control), where an integral form and change of variable formula were also presented on this connection. Ma [16] dealt with a one dimensional stochastic singular control problem where the drift term is assumed to be linear and the diffusion term is assumed to be smooth, and he showed that the value function is convex and and the controlled process is a reflected diffusion over an interval. Guo and Tomecek [10] solved a one dimensional singular control problem via a switching problem [9], and showed, using the smooth fit property [18], that under some conditions the value function is continuously differentiable ().

It is found that [6] through the approach via game theory and optimal stopping, it is possible to show the existence of a smooth solution. The connection is the following: given a symmetric Markov process on a locally compact separable metric space, it is well known that the solution of an optimal stopping problem admits its quasi continuous version of the solution to a variational inequality problem involving Dirichlet form, e.g., see Nagai [17]. Zabczyk [22] extended this result to a zero-sum game (Dynkin game). In the one dimensional case, the integrated form of the value function of the Dynkin game was identified to be the solution of an associated stochastic singular control problem, e.g., see Taksar [20], Fukushima and Taksar [6] where a more general one dimensional diffusion is assumed. As a result, the classical smooth solution () can be obtained for this singular control problem.

This paper extends the work by Fukushima and Taksar [6] to multi-dimensional stochastic singular control problem. There are many difficulties in this extension. In the one dimensional singular control problem, each point in the space has a positive capacity [6], hence the nonexistence of the proper exceptional set. However, this is no longer the case in multi-dimensional singular control problem. We overcome this difficulty using the absolute continuity of the transition function of the underlying process [7]. Under some conditions, the optimal control policy of the one dimensional case is proved to be the reflection of the diffusion at two boundary points, but the form of the optimal control policy and the conditions on the regularity of the value function in multi-dimensional case are much more complicated. For instance, in the two dimensional case, the boundary of the continuation region can have various formats, e.g., bounded curves, unbounded curves, singular points, disconnected curves, line segments, etc. The difficulty in characterizing the continuation region is due to the fact that its boundary is a free boundary, and this paper investigates such issues.

In this paper, we are concerned with a multi-dimensional diffusion on :

 dXt=μ(Xt)dt+σ(Xt)dBt, \hb@xt@.01(1.1)

where

 Xt=⎛⎜ ⎜⎝X1t⋮Xnt⎞⎟ ⎟⎠,μ=⎛⎜ ⎜⎝μ1⋮μn⎞⎟ ⎟⎠,σ=⎛⎜ ⎜⎝σ11 ⋯σ1m⋮⋮σn1 ⋯σnm⎞⎟ ⎟⎠,Bt=⎛⎜ ⎜⎝B1t⋮Bmt⎞⎟ ⎟⎠,

in which and () are continuous functions of , and is -dimensional Brownian motion with . Thus we are given a system , where is a measurable space, is a mapping of into , , and is a shift operator in such that . Here () is a family of measures under which is an -dimensional diffusion with initial state . We assume that and satisfy the usual Lipschitz growth condition.

A control policy is defined as a pair of adapted processes which are right continuous and nondecreasing in and we assume are nonnegative. Denote the set of all admissible policies, whose detailed definition will be given in Section LABEL:mdssc.

Given a policy we define the following controlled process:

 dX1t=μ1dt+σ11dB1t+⋯+σ1mdBmt,⋮⋮⋮dXnt=μndt+σn1dB1t+⋯+σnmdBmt+dA(1)t−dA(2)t,X0=x,

with the cost function

 kS(x)=Ex(∫∞0e−αth(Xt)dt+∫∞0e−αt(f1(Xt)dA(1)t+f2(Xt)dA(2)t)), \hb@xt@.01(1.2) f1(x),f2(x)>0, ∀x∈Rn.

Here we assume that is the minimal decomposition of a bounded variation process into a difference of two increasing processes.

###### Remark 1.1

A natural question is that why the control only applies on one dimension. The difficulty arises in the step where the value function of the zero-sum game is integrated (in one dimension) to obtain the value function of the singular control problem. If the control were applied to multi dimensions, no result so far is know on the choice of the direction of integration. This represents a traditional difficulty in multi-dimensional singular control problem. Interested readers are referred to [19] for a result on two dimensional singular control problem.

There are two types of costs associated with the process for each policy . The first one is the holding cost accumulated along time. The second one is the control cost associated with the processes , and this cost increases only when increase.

One looks for a control policy that minimizes , i.e.,

 W(x)=minS∈SkS(x). \hb@xt@.01(1.3)

As an application of this model, a decision maker observes the expenses of a company under a multi-factor situation but only has control over one factor, yet she still wants to minimize the total expected cost. Analogously, by studying the associated maximization problem, i.e., taking the negative of , this model can be used to find the optimal investment policy where an investor observes the prices of several assets in a portfolio and manages the portfolio by adjusting one of them. Notice that every time there is a control action, it yields a certain associated cost, e.g., the transaction cost.

The rest of this paper is organized as follows: we first introduce some preliminaries on Dirichlet form and a variational inequality problem in Section LABEL:DformDgame. In Section LABEL:DgameFBP we identify conditions for the value function as well as the optimal policy of the associated Dynkin game. The integrated form of the value function of this Dynkin game is shown in Section LABEL:mdssc to be the value of a multi-dimensional singular control problem, and the optimal control policy is also determined consequently. In the appendix we shall correct an error found in the paper by Fukushima and Taksar [6].

## 2 Dirichlet Form and a Variational Inequality Problem

Let be a locally compact separable metric space, be an everywhere dense positive Radon measure on , and denotes the space on . We assume that the Dirichlet form on is regular in the sense that is dense in and is uniformly dense in , where the norm is defined as follows:

 ∥u∥E1=(E(u,u)+∫Du(x)2m(dx))1/2.

Analogously we define as , where

 (u,v)=∫Du(x)v(x)m(dx).

For this Dirichlet form, there exists an associated Hunt process on , see [5], such that

 ptf(x):=Exf(Xt),x∈D

is a version of for all , where is the semigroup associated with the Dirichlet form . Furthermore, the -resolvent associated with this Dirichlet form satisfies

 Gαf∈F,Eα(Gαf,u)=(f,u),∀f∈L2(D;m), ∀u∈F, \hb@xt@.01(2.1)

and the resolvent of the Hunt process given by

 Rαf(x)=Ex(∫∞0e−αtf(Xt)dt),x∈D,

is a quasi-continuous modification of for any Borel function .

For , a measurable function on is called -excessive if and as for any . A function is said to be an -potential if for any with . For any -potential , define , then -a.e. and is -excessive (see Section 3 in [7]). is called the -excessive regularization of . Furthermore, any -excessive function is finely continuous (see Theorem A.2.7 in [5]).

As related literature, Nagai [17] considered an optimal stopping problem and showed that there exist a quasi continuous function which solves the variational inequality

 w≥g,Eα(w,u−w)≥0,∀u∈F with u≥g,

and a properly exceptional set such that for all ,

 w(x)=supσEx(e−ασ[g(Xσ)])=Ex(e−α^σ[g(X^σ)]),

where is a quasi continuous function in and

 ^σ=inf{t≥0;w(Xt)=g(Xt)}.

Moreover, is the smallest -potential dominating the function -a.e.

Zabczyk [22] then extended this result to the solution of the zero-sum game (Dynkin game) by showing that there exist a quasi continuous function which solves the variational inequality

 g≤V≤h  m a.e., Eα(V,u−V)≥0,∀u∈F, g≤u≤h  m a.e., \hb@xt@.01(2.2)

and a properly exceptional set such that for all ,

 V(x)=supσinfτJx(τ,σ)=infτsupσJx(τ,σ) \hb@xt@.01(2.3)

for any stopping times and , where

 Jx(τ,σ)=Ex(e−α(τ∧σ)(Iσ≤τg(Xσ)+Iτ<σh(Xτ))), \hb@xt@.01(2.4)

and -a.e. are quasi-continuous functions in .

In these works, there always existed an exceptional set . Fukushima and Menda [7] showed that, if the transition function of satisfies an absolute continuity condition, i.e.,

 pt(x,⋅)≪m(⋅), \hb@xt@.01(2.5)

for all and , and satisfy the following separability condition:
There exist finite -excessive functions such that, for all ,

 g(x)≤v1(x)−v2(x)≤h(x), \hb@xt@.01(2.6)

then Zabczyk’s result still holds and there does not exist the exceptional set . In what follows we shall introduce a version of Theorem 2 in [7], where we used in places of respectively for the convenience of later use.

Let be finely continuous functions such that for all

 −f1(x)≤f2(x), |f1(x)|≤ϕ(x), |f2(x)|≤ψ(x), \hb@xt@.01(2.7)

where are some finite -excessive functions, and are assumed to satisfy the following separability condition

 −f1(x)≤v1(x)−v2(x)≤f2(x). \hb@xt@.01(2.8)

We further define the set

 K={u∈F: −f1≤u≤f2, m−a.e.}. \hb@xt@.01(2.9)

Considering the variational inequality problem

 V∈K, Eα(V,u−V)≥0,∀u∈K, \hb@xt@.01(2.10)

we have:

###### Theorem 2.1

Assume conditions (LABEL:abscont), (LABEL:bdcond) and (LABEL:sepcondf). There exists a finite finely continuous function satisfying the variational inequality (LABEL:vineqH0f) and the identity

 V(x)=supσinfτJx(τ,σ)=infτsupσJx(τ,σ),∀x∈D,

where range over all stopping times and

 Jx(τ,σ)=Ex(e−α(τ∧σ)(Iσ≤τ(−f1(Xσ))+Iτ<σf2(Xτ))), \hb@xt@.01(2.11)

Moreover, the pair defined by

 ^τ=inf{t>0:V(Xt)=f2(Xt)},^σ=inf{t>0:V(Xt)=−f1(Xt)},

is the saddle point of the game in the sense that

 Jx(^τ,σ)≤Jx(^τ,^σ)≤Jx(τ,^σ),∀x∈D,

for all stopping times .

For a given function one looks for a solution to the following variational inequality problem

 Eα(V,u−V)≥(H,u−V),  ∀u∈K. \hb@xt@.01(2.12)

Then we have the following proposition:

###### Proposition 2.2

There exists a unique finite finely continuous function which solves (LABEL:vi).

Proof. The proof is essentially identical to the proof of Proposition 2.1 in [6] and is omitted here.

We assume further the following separability condition:

###### Assumption 2.1

There exist finite -excessive functions such that, for all ,

 −f1(x)−GαH(x)≤v1(x)−v2(x)≤f2(x)−GαH(x), \hb@xt@.01(2.13)

then the following result holds:

###### Theorem 2.3

For any function ) and any such that and are finely continuous and bounded by some finite -excessive functions, respectively. Assuming (LABEL:abscont)(LABEL:sepcondHneq0), we put

 Jx(τ,σ) = Ex(∫τ∧σ0e−αtH(Xt)dt) +Ex(e−α(τ∧σ)(−Iσ≤τf1(Xσ)+Iτ<σf2(Xτ)))

for any stopping times . Then the solution of (LABEL:vi) admits a finite finely continuous value function of the game

 V(x)=infτsupσJx(τ,σ)=supσinfτJx(τ,σ),∀x∈D. \hb@xt@.01(2.15)

Furthermore if we let

 E1={x∈D:V(x)=−f1(x)},E2={x∈D:V(x)=f2(x)},

then the hitting times , is the saddle point of the game

 Jx(^τ,σ)≤Jx(^τ,^σ)≤Jx(τ,^σ) \hb@xt@.01(2.16)

for any and any stopping times . In particular,

 \hb@xt@.01(2.17)

is the set of points where and is the set of points where . So and in Theorem LABEL:DgameHneq0 can be defined in the same way as in Theorem LABEL:fuku2. The proof of Theorem LABEL:DgameHneq0 is identical to Theorem 2.1 in [6].

## 3 The Dynkin Game and Its Value Function

Two players and observe a multi-dimensional underlying process in (LABEL:omodel) with accumulated income, discounted at present time, equalling for any stopping time . If stops the game at time , he pays the amount of the accumulated income plus the amount , which after been discounted equals . If the process is stopped by at time , he receives from the accumulated income less the amount , which after been discounted equals . tries to minimize his payment while tries to maximize his income. Let be two stopping times, the value of this game is thus given by

 V(x)=infτsupσJx(τ,σ),∀x∈Rn, \hb@xt@.01(3.1)

where is given by (LABEL:Jcost) on .

For the diffusion (LABEL:omodel), define its infinitesimal generator as

 L:=n∑i=1μi∂∂xi+n∑i,j=1Aij∂2∂xi∂xj, \hb@xt@.01(3.2)

where . We assume that is non-degenerate.

Define the measure , where satisfies the following condition:

 A∇ρ=ρ⋅(μ−b), \hb@xt@.01(3.3)

where in . (Notice that when and are constants, reduces to .) It can be seen that the absolute continuity condition (LABEL:abscont) is satisfied.

###### Remark 3.1

We are unable to solve the case with a general multidimensional diffusion. Even in the case of one dimensional diffusion, conditions on and should be made (see Appendix).

For the generator , its associated Dirichlet form densely embedded in is then given by

 E(u,v)=∫Rn∇u(x)⋅A∇v(x)m(dx),u,v∈F, \hb@xt@.01(3.4)

where

 F={u∈L2(Rn;m): u is continuous, ∫Rn∇u(x)T∇u(x)m(dx)<∞}.

For given functions satisfying the conditions of Theorem LABEL:DgameHneq0, and noticing that is a non-degenerate Ito diffusion, we can conclude that in Eq.(LABEL:vgame) is finite and continuous, and it solves (LABEL:vi). Furthermore if we let

 E1={x∈Rn:V(x)=−f1(x)},E2={x∈Rn:V(x)=f2(x)}, \hb@xt@.01(3.5)

then the hitting times , is the saddle point of the game

 Jx(^τ,σ)≤Jx(^τ,^σ)=V(x)≤Jx(τ,^σ) \hb@xt@.01(3.6)

for any and any stopping times .

In the next section we shall give conditions on and characterize the regularities of and the form of the optimal control policy.

### 3.1 Optimal Stopping Regions

In the one dimensional case, if the functions are defined over a bounded interval, a lot of properties are automatically satisfied [6]. But in multi-dimensional case, this is much harder.

It is obvious that the conditions on are critical on the form of optimal control policy. For example, if and , then no party would ever stop the game and there is no optimal control.

###### Assumption 3.1

are smooth functions, where is a constant, and is everywhere continuous, and the separability condition (LABEL:sepcondHneq0) holds. is strictly increasing in , is nondecreasing in , is nonincreasing in . Further more, is strictly increasing in and is strictly decreasing in . The (hyper)curves , such that

 (α−L)f1(¯x,a(¯x))+H(¯x,a(¯x)) = 0, (α−L)f2(¯x,b(¯x))−H(¯x,b(¯x)) = 0,

with , , are assumed to be bounded and uniformly Lipschitz continuous.

Then it is easy to see that

###### Proposition 3.1

Assume Assumption LABEL:assumpHf. For any with ,

 (α−L)f1(¯x,xn)+H(¯x,xn)<0,

and for any with ,

 (α−L)f1(¯x,xn)+H(¯x,xn)>0.

Similarly, for any with ,

 (α−L)f2(¯x,xn)−H(¯x,xn)>0,

and for any with ,

 (α−L)f2(¯x,xn)−H(¯x,xn)<0.

Define the set

 E={x∈Rn:−f1(x)

Since would stop the game once and the instant payoff is , while would stop the game once and the instant payoff is , we could write as a partition:

 Rn=E1∪E∪E2,

where were given in (LABEL:e12R).

###### Proposition 3.2

Assume Assumption LABEL:assumpHf. For each ,

 (α−L)f1(x)+H(x)≤0,

and for each ,

 (α−L)f2(x)−H(x)≤0.

Proof. We only give proof to the first half. We know at the point it must be true that , and it is optimal for to stop the game immediately. Suppose

 (α−L)f1(x)+H(x)>0, x∈E1,

then by the smoothness of and the continuity of , we can find a small ball containing the point , such that for each ,

 (α−L)f1(y)+H(y)>0.

Consider a policy for to stop the game at the first exit time of , denoted . Then by Dynkin’s formula, the payoff would be

 Jx = Ex∫τBr0e−αtH(Xt)dt+Ex(e−ατBr(−f1(XτBr))) = Ex∫τBr0e−αtH(Xt)dt−f1(x)+Ex∫τBr0e−αt(α−L)f1(Xt)dt = −f1(x)+Ex∫τBr0e−αt[(α−L)f1(Xt)+H(Xt)]dt > −f1(x).

This is a contradiction since tries to maximize his payoff but we assumed that the optimal policy at was to stop the game immediately.

###### Corollary 3.3

Assume Assumption LABEL:assumpHf. If , then for any point with ,

 (α−L)f1(¯x,y)+H(¯x,y)<0.

If , then for any point with ,

 (α−L)f2(¯x,y)−H(¯x,y)<0.

Furthermore, and .

Proof. This can be easily seen from Proposition LABEL:Jandf and the conditions on given in Assumption LABEL:assumpHf.

Further, noticing the conditions on the curves and , , we have the following:

###### Corollary 3.4

Assume Assumption LABEL:assumpHf. and hence is not empty. Furthermore, the value of this game is bounded by , where is given in Assumption LABEL:assumpHf.

Take any point , and denote the hitting time to the curve . Notice that the diffusion (LABEL:omodel) is a conservative process by the given conditions, and also by noticing the conditions given on , it can be concluded that goes to zero as goes to . Similarly , , goes to zero as goes to .

###### Assumption 3.2

There exist functions , that are uniformly bounded and such that for any ,

 Ex(∫σa0e−αtH(Xt)dt+e−ασaM)<−f1(x),

and for any ,

 Ex(∫σb0e−αtH(Xt)dt−e−ασbM)>f2(x).
###### Proposition 3.5

Assume Assumptions LABEL:assumpHf and LABEL:assumAB, then and , . Furthermore, on player would stop the game immediately and , and on player would stop the game immediately and .

Proof. Suppose there is a point with . Then by Dynkin’s formula,

 Ex(∫σa∧T0e−αtH(Xt)dt+e−α(σa∧T)M) >Ex(∫σa∧T0e−αtH(Xt)dt+e−α(σa∧T)(−f1(Xσa∧T))) =−f1(x)+Ex(∫σa∧T0e−αt((α−L)f1(Xt)+H(Xt))dt)>−f1(x)

by Proposition LABEL:f1f2Hsign. Taking we get a contradiction.

Now suppose . For any stopping time for player the payoff will be

 Ex∫σ0e−αtH(Xt)dt+Ex[e−ασ(−f1(Xσ))]

When , the following quantity

 =Ex∫σ∧T∧σa0e−αtH(Xt)dt+Ex[e−α(σ∧T∧σa)(−f1(Xσ∧T∧σa))] =−f1(x)+Ex∫σ∧T∧σa0e−αt((α−L)f1(Xt)+H(Xt))dt

is less than , by Proposition LABEL:f1f2Hsign.

When ,

 Ex∫σ0e−αtH(Xt)dt+Ex[e−ασ(−f1(Xσ))] ≤Ex(∫σa0e−αtH(Xt)dt+e−ασaM)

because is the bound of the payoff of each player. Hence by Assumption LABEL:assumAB,

 Ex∫σ0e−αtH(Xt)dt+Ex[e−ασ(−f1(Xσ))]<−f1(x).

As a summary, if , then for any stopping policy of , the expected payoff is less than , and the optimal strategy is to stop the game immediately. The other half of this proposition can be proved in a similar way.

By the properties of (or ), we can choose a bounded and continuous curve below (or a bounded and continuous curve above ) which also has the properties as given in Assumption LABEL:assumAB. Without loss of generality, we assume and are bounded and continuous.

Now it is easy to see that and . Since the functions are all continuous, the boundary of consists of continuous curves.

Let be the largest connected region in containing the set , and be the largest connected region in containing the set , then obviously , and . Furthermore has a continuous boundary curve that is bounded by and ,