Exponential Stabilization for Itô Stochastic Systems with Multiple Input Delays \thanksreffootnoteinfo

Exponential Stabilization for Itô Stochastic Systems with Multiple Input Delays \thanksreffootnoteinfo

[    [
Abstract

In this paper, we study the stabilization problem for the Itô systems with both multiplicative noise and multiple delays which exist widely in applications such as networked control systems. Sufficient and necessary conditions are obtained for the exponential stabilization problem of Itô stochastic systems with multiple delays. On one hand, we derive the solvability of the modified Riccati equation in case of the mean-square exponential stabilization. On the other hand, the mean-square exponential stabilization is guaranteed by the solvability of a modified Riccati equation. A novel stabilizing controller is shown in the feedback from of the conditional expectation in terms of the modified algebraic Riccati equation. The main technique is to reduce the original system with multiple delays to a pseudo delay-free system.

sdu]Juanjuan Xu, sdu]Huanshui Zhang

School of Control Science and Engineering, Shandong University, Jinan, Shandong, P.R. China 250061 


Key words:  Itô stochastic system, Multiple input delays, Stabilization, Riccati equation.

 

11footnotetext: This work is supported by the National Natural Science Foundation of China (61633014, 61573221) and the Qilu Youth Scholar Discipline Construction Funding from Shandong University. Corresponding author H. Zhang.

1 Introduction

The mathematical models described by delayed differential equations are ubiquitous and have wide applications in physics, engineering, communication, biology and so on [Kolmnovskii et al., 1999]. As is known, time delays usually degrade the system performance, and are the source of instability, and even lead to the occurrence of chaos phenomenon. So study on the stabilization problem of time-delay system is of great significance. Some essential progress has been made on the optimal control and stabilization problems for time delay systems, see [Richard, 2003], [Smith, 2003] and references therein. In particular, [Smith, 2003] designs a predictor-like controller which reduces the original delayed system to delay-free one. By virtue of the predictor-based technique, the problem for systems with more general delays has been studied in [Artstein, 1982]-[Manitius et al., 1979]. The linear quadratic regulation (LQR) problem for systems with multiple input delays was solved in [Zhang et al., 2006] by establishing a duality between the LQR problem and a smoothing problem. The optimal controller is presented using a Riccati equation. [Tadmor et al., 2005]-[Tadmor et al., 2005] studied the preview control problem and presented the necessary and sufficient solvability conditions in terms of a standard algebraic Riccati equation and a nonstandard -like algebraic Riccati equation. The aforementioned results are only related to the deterministic system and more details are referred to the survey paper [Richard, 2003].

Considering the accuracy requirement to the system in applications, it is necessary to take the uncertainty into consideration. One of the most popular models is the stochastic differential equation motivated by Brownian motion. When the stochastic system is delay-free, [Rami et al., 2000] presents some sufficient and necessary conditions for the mean-square stabilization. There have also been many important developments when both delay and uncertainty are considered, especially the noise is multiplicative, e.g., [Cao et al., 1999], [Zhang et al., 2009], [Wang et al., 2002] and references therein. Noting that most results in the literature depend on the linear matrix inequality (LMI) to characterize the sufficient conditions for the stabilization. For instance, [Wang et al., 2002] investigated the stochastic stabilization problem for a class of bilinear continuous time-delay uncertain systems with Markovian jumping parameters. Sufficient conditions were established to guarantee the existence of desired robust controllers, which are given in terms of the solutions to a set of LMIs, or coupled quadratic matrix inequalities. [Xie et al., 2000] considered a class of large-scale interconnected bilinear stochastic systems with time delays and time-varying parameter uncertainties and robust stability analysis was given in terms of a set of LMIs. In addition, some convergence theorems have been given in the literature. For example, [Mao, 1999]-[Mao, 2003] investigated the LaSalle-type asymptotic convergence theorems for the solutions of stochastic differential delay equations. More recently, some substantial progress for the optimal LQ control has been made by proposing the approach of solving the forward and backward differential/difference equations (FBDEs). See [Zhang et al., 2015] and [Zhang et al., 2017] for details. However, the stabilization problem for Itô stochastic systems with multiple delays have not yet been completely solved. The main obstacles are that the problem is in fact infinite dimensional and the classical controller such as current feedback form only leads to sufficient conditions which may be delay-dependent.

Inspired by the work [Zhang et al., 2017], we shall study the stochastic system with multiple delays. The main contribution is two-fold. Firstly, we derive the solvability of the modified Riccati equation in case of the mean-square exponential stabilization. Secondly, we obtain that the mean-square exponential stabilization can be guaranteed by the solvability of a modified Riccati equation. A novel stabilizing controller is shown in the feedback from of the conditional expectation in terms of the modified algebraic Riccati equation. The main technique is to reduce the original system with multiple delays to a pseudo delay-free system.

The rest of the paper is formulated as follows: Section 2 illustrates the studied problem. The system is reduced to a pseudo delay-free system and the optimization problems of the reduced system are studied in Section 3. Sufficient and necessary conditions are given in Section 4 for the exponential mean-square stabilization of the system. Some concluding remarks are shown in the last section.

Notation. denotes the family of -dimensional vectors; denotes the transpose of ; and a symmetric matrix is strictly positive-definite (positive semi-definite). is a complete stochastic basis so that contains all P-null elements of and the filtration is generated by the standard Brownian motion denotes the conditional expectation with respect to the filtration We simply denote and denotes the inner product in Hilbert space. The following sets are useful throughout the paper:

2 Problem Formulation

Consider the Itô stochastic systems with multiple input delays:

(1)

where is the state, is the control input, represent the input delays. is independent one-dimension standard Brownian motion. are constant matrices with compatible dimensions. The initial conditions are chosen as and

Remark 1. The system (1) has wide applications in network control systems. In particular, consider the continuous-time LTI system with both random input gains and multiple input delays as shown in Fig. 1:

Fig. 1: Continuous-time LTI system with both random input gains and multiple input delays
(2)

where is the state, is the th control input, represent the input delays. where is a real positive constant and is a zero-mean white noise with autocorrelation . By denoting and for (2) can be rewritten as

(3)

(3) can then be reformulated as a standard Itô form by using :

This is a special case of systems (1).

We now define the stabilization and exponential stabilization for system (1).

Definition 1

System (1) is mean-square stabilizable if there exists an -adapted controller in the form of

(4)

where is a constant matrix and is a time-varying matrix with compatible dimensions such that the closed-loop system satisfies

for any and any -adapted controller

Definition 2

System (1) is mean-square exponentially stabilizable if there exists an -adapted controller in the form of (4) and a positive constant such that the closed-loop system satisfies

for any and any -adapted controller

The aim of this paper is stated as follows.
Problem : Find the sufficient and necessary conditions for system (1) to be exponentially stabilized by a controller in the form of (4) following Definition 2.

The outline of the solvability to Problem is as follows: Firstly, we convert the original stochastic system with multiple input delays into a pseudo delay-free system where the delays are involved in the Brownian motions rather than the control input. Secondly, we solve finite-horizon optimization problems with a standard cost function and a discounted cost function subject to the pseudo delay-free system in terms of modified differential Riccati equations. Finally, the sufficient and necessary conditions for the exponential stabilization are characterized by the corresponding modified algebraic Riccati equation.

3 Reduction of the original system into a pseudo delay-free system

We firstly transform the original system (1) into a pseudo delay-free system. To this end, we define

(5)
Lemma 1

defined by (5) satisfies the dynamic

(6)

Proof. By taking Itô’s formula to and using (1), it is obtained that

This completes the proof.

Remark 2. Noting that there exists no delay in the control input However, the delays are involved in the Brownian motions Thus, we call the system (6) as a pseudo delay-free system.

Define a new -algebraic Then it holds that From (6), we have is -adapted. In addition, considering Definition 1 and (2), the controller is -adapted. For convenience of the future use, it is simply denoted that

3.1 Finite-horizon optimal control problem of pseudo delay-free system

We then study the finite-horizon optimization problem of minimizing the standard linear quadratic cost function subject to (6):

(7)

where is semi-positive definite matrix of compatible dimension.

Noting that the new state is -adapted rather than -adapted, we define the admissible control set as

(8)

where is time-varying matrices with compatible dimension and

Following [Wang et al., 2013], the stochastic maximum principle can be immediately obtained.

Lemma 2

The optimal solution to minimize (7) subject to (6) satisfies

(9)

where is the solution of the backward stochastic differential equation (BSDE):

(10)

while obeys (6) and is defined in (7).

Based on Lemma 2, the explicit solvability of forward and backward stochastic differential equations (6), (9) and (10) is the key to the derivation of the optimal solution. To this end, we define the modified differential Riccati equation:

(11)

and

(12)

where

(13)
(14)
(15)

with the terminal values for defined in (7).

Lemma 3

The equation (11)-(15) is equivalent to the following equations:

(16)

while is given by

(17)
(18)
(19)

with terminal values and for

Proof. The equivalence can be established by similar discussions to Remark 5 in [Zhang et al., 2017]. So we omit it.

We now present the optimal solution of the finite-horizon linear quadratic optimal control problem by using the solution to (11)-(15).

Lemma 4

Assume that the modified Riccati equation (11)-(15) admits a solution such that the matrix , then there exists a unique solution to the problem of minimizing (7) subject to the system (6) and the optimal controller is given by

(20)

The optimal cost is as

(21)

Proof. The proof is presented in Appendix A.

As a byproduct of Lemma 4 which is useful in the stabilization, we further state the following results.

Corollary 1

Under the same conditions in Lemma 4 and let the controller satisfy that for Then there exists a unique solution to the problem of minimizing (7) subject to the system (6). The optimal controller is given by (20) for and the optimal cost is as

(22)

Proof. Since for then Thus the optimal cost becomes from (21).

Corollary 2

Under the same conditions in Lemma 4 and let the controller satisfy that for Then there exists a unique solution to the problem of minimizing (7) subject to the system (6). The optimal controller is given by (20) for and the optimal cost is as

(23)

Proof. Since for then for By using for it is obtained that from (5). Combining with the proof of Lemma 4 and (12), the result follows. So we omit the details.

Next, we consider the optimization problem with respect to the admissible control set set (8).

Lemma 5

If a given linear feedback control is the unique optimal solution for the problem of minimizing s.t (6), then obeys the equations (16)-(19) with .

Proof. The proof is presented in Appendix B.

We now give the necessary and sufficient condition for the existence and uniqueness of the solution to the finite-horizon optimization problem.

Theorem 1

The problem of minimizing (7) subject to (6) within the admissible control set (8) has a unique solution if and only if (11)-(15) admits a solution such that the matrix is strictly positive definite. The optimal control is as (20) and the optimal cost is given by (21).

Proof. Combining with Lemmas 3-5, the result follows directly.

3.2 Finite-horizon optimal control problem of pseudo delay-free system with discounted cost function

In this subsection ,we study the finite-horizon optimization problem of minimizing the discounted cost function subject to (6):

(24)

The discounted setting is popular in many areas, such as in dynamic programming, reinforcement learning, and planning algorithms for optimal control. See [LaValle, 2006], [Sutton et al., 1998] and references therein.

To solve the discounted LQR problem, we define the modified Riccati equation:

(25)
(26)

where

with and Following similar discussions to Lemma 3 and Remark 5 in [Zhang et al., 2017], the following result is in force.

Lemma 6

The equation (25)-(26) is equivalent to the following equations:

(27)

while is given by

with terminal values and for

It is now in the position to give the solution to the discounted LQR problem.

Theorem 2

The problem of minimizing (24) subject to (6) within the admissible control set (8) has a unique solution if and only if (25)-(26) admits a solution such that the matrix is strictly positive definite. The optimal control is as

(28)

and the optimal cost is given by

(29)

Proof. The proof is presented in Appendix C.

4 Solution to the Problem

Based on the above results for the finite-horizon optimization problem, we discuss the mean-square stabilization problem. Sufficient and necessary conditions are to be derived for the exponential mean-square stabilization of system (1). The key is to investigate the properties of the modified Riccati equations (11)-(15) and (25)-(26) when the time tends to Firstly, we give the necessary condition for the mean-square stabilization for system (1).

Theorem 3

Assume that the system (1) is exponentially mean-square stabilizable in the sense of Definition 2, then the following modified algebraic Riccati equation (30)-(34) has a solution ,

(30)
(31)

where

(32)
(33)
(34)

Proof. The proof is put in Appendix D.

We then present the sufficient condition for the exponential mean-square stabilization by defining a new Lyapunov function.

Theorem 4

Assume that the following equation has a unique solution

(35)
(36)

where

(37)
(38)
(39)

then the system (1) is exponentially mean-square stable with the controller where is given by (39).

Proof. The proof is formulated in Appendix E.

5 Conclusions

This paper studied the stabilization problem for the Itô systems with both multiplicative noise and multiple delays. Sufficient and necessary conditions have been obtained for the exponential mean-square stabilization in terms of modified Riccati equations. The main technique is to reduce the original system with multiple delays to the pseudo delay-free one and study the finite-horizon optimization problems for the pseudo system with standard and discounted linear quadratic cost functions.

A Proof of Lemma 4

Using Lemma 3, the equations (16)-(19) admit a solution such that the matrix . Applying Itô’s formula to and combining with the equations (16)-(19), we have

(A.1)

Taking integral from to on both sides of (A.1) and then taking expectation, we have

(A.2)

where the fact of has been used in the derivation of the above equality. Note that , the optimal control exists uniquely. Furthermore, the optimal control (20) and cost function (21) follows from (A.2) directly combining with Lemma 3.

B Proof of Lemma 5

Consider the optimization problem for the controller set with respect to the matrix . The cost function is