Trade-Offs in Stochastic Event-Triggered Control

Trade-Offs in Stochastic Event-Triggered Control

Burak Demirel, Alex S. Leong, Vijay Gupta and Daniel E. Quevedo B. Demirel, A. S. Leong and D. E. Quevedo are with the Chair of Automatic Control (EIM-E), Paderborn University, Warburger Straße 100, 33098, Paderborn, Germany (e-mail: burak.demirel@protonmail.com, alex.leong@upb.de, dquevedo@ieee.org). V. Gupta is with the Department of Electrical Engineering, University of Notre Dame, South Bend, Indiana, 46556, USA (e-mail: vgupta2@nd.edu).
Abstract

This paper studies the optimal output-feedback control of a linear time-invariant system where a stochastic event-based scheduler triggers the communication between the sensor and the controller. The primary goal of the use of this type of scheduling strategy is to provide significant reductions in the usage of the sensor-to-controller communication and, in turn, improve energy expenditure in the network. In this paper, we aim to design an admissible control policy, which is a function of the observed output, to minimize a quadratic cost function while employing a stochastic event-triggered scheduler that preserves the Gaussian property of the plant state and the estimation error. For the infinite horizon case, we present analytical expressions that quantify the trade-off between the communication cost and control performance of such event-triggered control systems. This trade-off is confirmed quantitatively via numerical examples.

I Introduction

Over the past decade, distributed control and estimation over networks have been a major trend. Thanks to the forthcoming revolution of the Internet-of-Things (IoT) and resulting interconnectedness of smart technologies, the importance of decision making over communication networks grows ever larger in our modern society. These technological advances, however, bring new challenges regarding how to use the limited computation, communication, and energy resources efficiently. Consequently, event- and self-triggered algorithms have appeared as an alternative to traditional time-triggered algorithms in both estimation and control; see, e.g., [1].

A vast majority of the research in this area has mainly focused on proving the stability of the proposed control schemes, and demonstrating the effectiveness of such control systems, as compared to periodically sampled ones, through numerical simulations. However, an important stream of work in such schemes is analytically characterizing the trade-off between the control performance and communication rate achieved via these algorithms. Early works on event-triggered control, such as [2, 3, 4, 5], provided performance expressions but only for scalar systems. The authors of [6] later extended the work of [2] to a class of second-order systems. The work in [7] studies state estimation for multiple plants across a shared communication network, and quantified communication and estimation performance. Recently, the authors of [8] investigated the minimum-variance event-triggered output-feedback control problem; cf. [2]. They established a separation between the control strategy and the scheduling decision, and they also showed that scheduling decisions are determined by solving an optimal stopping problem. Our initial work in [9] considered a certain structure of controllers such as dead-beat controllers, and analytical expressions for the control performance and communication rate were obtained. Differing from [9], in the current work, we will focus on designing optimal controllers by establishing a separation between the controller and the scheduler.

Optimal event-triggered controller design requires the joint design of an optimal control law and an optimal event-based scheduler. The associated optimization problem becomes notoriously difficult [10] since, in general, the controller and the scheduler have different information. A vast majority of work in the literature focuses on the design of optimal feedback control laws for a predefined scheduling rule. It is important to note that designing an optimal control law might be very complicated even if one considers a fixed event-triggering policy. For instance, our recent work [11] shows that the optimal control problem, where a threshold-based event-triggered mechanism is used to decide if there is a need for transmission of new control actions based on knowledge of the plant state, leads to a non-convex optimization problem.

The selection of the event-triggering mechanism is essential for the computation of the control performance. As noted in [9], even in the case of Gauss-Markov plant models, due to the use of a (deterministic) threshold-based triggering mechanism, the plant state becomes a truncated Gaussian variable. As a result, computation of the control performance becomes challenging since it requires calculating the covariance of the plant state via numerical methods. One way to tackle this problem consists in employing a stochastic triggering mechanism, which preserves the Gaussianity of the plant state, as proposed in [12, 13, 14]. Our initial work in [15] used a deadbeat controller and a stochastic scheduler, which is similar to the ones mentioned above, to quantify the trade-off between the communication and the control cost for scalar systems. Similarly, the authors of [16] proposed an event-triggered control scheme that works under stochastic triggering rules. They also derived a control policy that always outperforms a periodic one. Differing from [16], in this work, we will focus on solving the optimal output event-triggered control problem.

Contributions. In this paper, we consider optimal output-feedback control of a linear time-invariant system where a stochastic event-based triggering algorithm dictates the communication between the sensor and the controller. The proposed scheduler decides at each time step whether or not to transmit new state estimates from the sensor to the controller based on state estimation errors. The main contributions of this manuscript are as follows:

  • We develop a framework for quantifying the closed-loop control performance and the communication rate in the channel between the sensor and the controller.

  • We confirm that the certainty-equivalent controller is optimal under the scheduling rule based on estimation errors. Our previous work [9] used a transmission strategy based on the plant state, and employed a sequence of deadbeat control actions to establish a resetting property, but this was not optimal since the separation principle between control and scheduling does not hold.

  • We derive analytical expressions for the (average) communication rate and control performance. Our analysis relies on a Markov chain characterization of the evolution of the state prediction error (cf. [9]) where the states of this Markov chain describe the time elapsed since the last transmission.

  • Due to the use of the stochastic triggering rule, we can compute the conditional covariance of the comparison error (i.e., the difference between the state estimation error at the sensor and the state estimation error at the controller) in a closed-form. Consequently, it becomes almost effortless to compute the closed-loop control performance; cf. [9].

Outline. The remainder of the paper is organized as follows: Section II describes the system architecture and formulates an optimal event-triggered control problem. In Section III, a control policy which minimizes a quadratic cost function under an event-triggered transmission constraint, is derived. This section also presents analytic expressions of the communication rate and the control performance for the infinite horizon problem. An illustrative example is presented to demonstrate the trade-off between communication and control performance in Section IV, while Section V finalizes the paper with concluding remarks. The Appendix provides detailed proofs of the main results.

Ii Problem Formulation

Control architecture. We consider the feedback control system depicted in Figure 1. A physical plant , whose dynamics can be represented by a linear time-invariant stochastic system, is being controlled. A sensor takes periodic samples of the plant output and transmits the estimate of the plant state to the controller over a resource-constrained communication channel. To tackle the resource constraint, the sensor employs an event-based scheduler, that makes a transmission decision by comparing its estimate of the plant state with the estimate at the controller. The controller computes new control actions based on the available information. Whenever the controller receives a new state estimate from the sensor, it calculates a control command based on this state estimate. Otherwise, it runs an estimator to predict the plant state, and it uses this information to calculate a new control action. In this context, we are interested in deriving analytical performance guarantees, both regarding the control performance and the number of transmissions between the sensor and the controller.

Figure 1: Block diagram of a feedback control system with plant , sensor/scheduler , and controller . The solid line represents an ideal channel, whereas the dashed line indicates a resource-constrained channel.

Process model. The system is modeled as a discrete-time, linear time-invariant (LTI) system,

(1)

driven by the control input (calculated by the controller ), and an unknown noise process . The state is available only indirectly through the noisy output measurement

(2)

The two noise sources and are assumed to be uncorrelated zero-mean Gaussian white-noise random processes with co-variance matrices and , respectively. We refer to as the process noise, and to as the measurement noise. The initial state is modeled as a Gaussian distributed random variable with mean and covariance . We assume that the pairs and are controllable while the pair is observable.

Sensor, pre-processor, and scheduler. Using a standard Kalman filter, the sensor locally computes minimum mean squared error (MMSE) estimates of the plant state based on the information available to the sensor at time , and transmits them to the controller. As noted in [17], sending local state estimates, in general, provides better performance than transmitting measurements. The sensor also employs a transmission scheduler, which decides whether or not to send the current state estimate to the controller at each time-step as determined by

(3)
Assumption

The sensor has precise knowledge of the control policy used to generate control actions, which are computed by the controller and applied by the actuator to the plant. Hence, the information set of the smart sensor contains all controls used up to time .

The information set available to the sensor at time is:

(4)

The minimum mean squared error estimate of the plant state can be computed recursively starting from the initial condition and using a Kalman filter [18]. At this point, it is worth reviewing the fundamental equations underlying the Kalman filter algorithm. The algorithm consists of two steps:

  • Prediction step: This step predicts the state, estimation error, and estimation error covariance at time dependent on information at time :

    (5)
    (6)
    (7)
  • Update step: This step updates the state, estimation error, and estimation error covariance using a blend of the predicted state and the observation :

    (8)
    (9)
    (10)

    where the gain matrix is given by

    (11)

It is worth noting that the estimation error at the sensor is Gaussian with zero-mean and co-variance , that evolves according to the standard Riccati recursion [19, Chapter 9]. Since the pair is observable and the pair is controllable, the matrices and converge exponentially to steady state values and , respectively. Similarly, the matrix also converges to a steady state value, i.e., .

The scheduler and the sensor are collocated, and the scheduler has access to all available information at the sensor. Moreover, the scheduler employs an event-based triggering mechanism to decide if there is a need for transmission of an updated state estimate from the sensor to the controller. The occurrence of information transmission is defined as

(12)

where is a (random) binary decision variable (which in this paper evolves according to (13)), is a non-negative integer variable introduced to describe the time elapsed since the last transmission, and is a time-out interval. Such a time-out mechanism is critical in event-triggered control systems to guard against faulty components; see, e.g., [9].

To maintain the Gaussianity of the comparison error

(note that is defined in (8) while is introduced in (17)) a variant of the stochastic triggering mechanism proposed in [12, 13, 14] is used. More specifically, the scheduler will decide to transmit a new sensor packet according to the following decision rule:

(13)

where the triggering parameter is a given positive scalar. As can be seen in (13), the probability of transmitting a new sensor packet (i.e., ) converges to one as goes to infinity. In other words, for large values of , the communication between the sensor and the controller is more likely to be triggered.

The integer-valued random process in (12) describes how many time instances ago the last transmission of a sensor packet occurred. Whenever a sensor packet is transmitted from the sensor to the controller, is reset to zero. Thus, the evolution of the random process is defined by

(14)

or equivalently,

(15)

where . Notice that the number of time steps between two consecutive transmissions is bounded by the time-out interval . If the number of samples since the last transmission exceeds a time-out value of , the sensor will attempt to transmit new data to the controller even if the comparison error does not satisfy the triggering condition (13). Thus, a transmission (i.e., ) will occur when either or there is a time-out.

Remark

It is worth noting that, as can be seen in (15), the events and are equivalent to each other.

At time instances when , the sensor transmits its local state estimate to the controller. As a result, the information set available to the controller at time (and before deciding upon ) can be defined as:

Under the event-based scheduling mechanism, (12) – (14), the controller runs an MMSE estimator to compute estimates of the plant state as follows:

(16)
(17)

where is the optimal estimate at the controller if the sensor did not transmit any information at time-step . Note that the optimality of this estimator can be shown by using a similar argument to that provided in [14, Lemma 4].

Assumption

In addition to computing , the sensor operates another estimator, which mimics the one at the controller, since transmission decisions rely on both and ; see (13). This can be done provided we make the following assumption.

Assumption

Both the smart sensor and the controller know the plant model (but not realizations of the noise processes).

Controller design and performance criterion. We aim at finding the control strategies , as a function of the admissible information set , to minimize a quadratic cost function of the form

(18)

where and . At time instances when (i.e., the controller has received sensor packets), the controller uses the state estimate which is transmitted by the sensor. However, at time instances when , the controller uses the outcome of the estimator at the controller side. As is well-known in related situations (see e.g., [10]), if the transmission decision is independent of the control strategy , then the certainty equivalent controller is optimal. In Section III, we will confirm that the certainty equivalent controller is optimal under the event-based scheduler proposed above.

Iii Main Results

We wish to quantify the communication rate and control performance of the feedback control system described by (1) and (2), where the event-based triggering mechanism (13) determines the communication between the sensor and the controller. We will first demonstrate that the time elapsed between two consecutive transmissions can be regarded as a discrete-time, finite state, time-homogeneous Markov chain. Then, using an ergodicity property, we will provide an analytical formula for the communication rate between the sensor and the controller. Subsequently, we will show that the certainty equivalent controller is still optimal with the event-triggering rule (13). Lastly, we will compute the control performance analytically for the infinite horizon case.

Assumption

In the rest of this paper, we will assume that the local Kalman filter at the sensor runs in steady state.

We first define the state prediction error at the controller

(19)

which evolves as

(20)

Then, we define the state estimation error at the controller

(21)

which evolves as

(22)

Define also the comparison errors:

(23)
(24)

Whenever a transmission occurs (i.e., ), the state estimation error at the controller is equal to , since the most recent sensor packet is available at the controller. It is then possible to write the stochastic recurrence equations (23) and (24) as

(25)

and

(26)

where . Notice that the comparison errors and propagate according to a linear system with open-loop dynamics , driven by the process .

Lemma

is a sequence of pairwise independent Gaussian random vectors such that with .

Remark

If the sensor has perfect state measurements (i.e., ), then is equal to .

Definition (Cumulative error)

We shall characterize the cumulative comparison error (i.e., the error that occurs in estimation at the controller over time due to intermittent transmissions) via

(27)

Using Definition III, the stochastic recurrence equations (25) and (26) can be then re-written as

(28)
(29)
Lemma (Markov process)

The random process is an ergodic, time-homogeneous Markov chain with a finite state space . Thus, it has a unique invariant distribution such that and for all .

Lemma (Augmented cumulative error vector)

Consider with as in (27). Then, is a random vector having a multivariate normal distribution with zero-mean and co-variance:

for any .

The next lemma computes the transition probabilities of the Markov chain defined in Lemma III.

Lemma (Transition probabilities)

The transition matrix of the Markov chain is given by

where the non-zero transition probabilities are computed as

The visit of the Markov chain to the state is analogous to a transmission (i.e., ) of the estimate of the plant state from the sensor to the controller. Using Remark II and the ergodic theorem for Markov chains [20, Theorem 5.3], we have:

(30)

where is the empirical frequency of transmissions. With the transition probabilities of this Markov chain, we can give an explicit characterization of the average communication rate of the event-triggered control system:

Theorem (Communication rate)

The average communication rate between the sensor and the controller under the stochastic event-based triggering mechanism, proposed in (12) – (14), for a fixed is given by

(31)

Remark

Note that, as goes to zero, the communication between the sensor and the controller becomes periodic.

The next theorem describes the optimal control law for the event-triggered control system at hand.

Theorem (Optimal control law)

Consider the system (1) and (2), and the problem of minimizing the cost function (18) under the event-based triggering mechanism (12) – (14) for a fixed . Then, there exists a unique admissible optimal control policy

(32)

where

(33)
(34)

with initial values . The minimum value of the cost function is obtained as

(35)

where .

Remark

Our result should be viewed in the light of the limited information available to the controller. At every time step , the controller computes an optimal control input based on the information set . Our result is akin to the one derived in [10], however here we can also provide the closed-form expression of the control cost for the infinite horizon case (see Theorem III).

Lemma (Gaussianity-preservation)

The conditional random variable, , has a Gaussian distribution with zero-mean and co-variance:

Using the previous theorems, we have the following result to calculate the average control performance measured by a linear-quadratic function.

Theorem (Infinite horizon control performance)

Suppose the pairs and are controllable, and the pairs and are observable. Moreover, suppose that . Then, we have the following:

  • The infinite horizon optimal controller gain is constant:

    (36)
  • The matrices and are the positive definite solutions of the following algebraic Riccati equations:

    (37)
    (38)
  • The expected minimum cost converges to the following value:

    (39)

    where , , and satisfies .

Iv Numerical Example

In this section, numerical simulations are provided to assess the performance of the stochastic event-triggering algorithm proposed in Section II, and verify the theoretical results presented in Section III. To this end, the system parameters are chosen as follows:

The matrix has one stable (i.e., ) and one unstable eigenvalue (i.e., ). The time-out interval is set to . Notice that the pairs and are controllable, the pairs and are observable, and , as required by the assumptions of the theorems presented in Section III.

Figure 2: A comparison of transmission rate (resp. control performance) derived from the analytic expression (31) (resp. (39)) and Monte Carlo simulations.

For various values of ranging from to , we evaluate the communication rate and the control performance as predicted by Theorems III and III, respectively. We compare the analytic results to Monte Carlo simulations of the closed-loop system. For each value of , we conduct Monte Carlo simulations for the horizon length of samples, and obtain the mean communication rate and the control performance. The comparison is shown in Fig. 2 for the communication rate and the control performance. It can be seen that the analytic results match the Monte Carlo simulations very closely.

We can also obtain results on when changing the scheduling parameter has the most effect as demonstrated in Fig. 3. There are two extreme cases: 1) as goes to infinity, the communication rate becomes one, and the control performance converges to ; 2) as goes to zero, the transmission rate converges to zero, and the control performance becomes unbounded. We observe, for instance, that changing the scheduling parameter from one to infinity has minimal effect on the control performance, but nearly doubles the communication frequency. As can be seen in Fig. 2, by setting , we can reduce the communication between the sensor and the controller by almost , while only slightly sacrificing the control performance of the closed-loop system.

Figure 3: The trade-off between the communication rate and the control performance (the scheduling parameter is illustrated by gray scale).

V Conclusions and Discussions

This paper has focused on the optimal control of a linear stochastic system, where a stochastic event-based scheduling mechanism governs the communication between the sensor and the controller. The scheduler is co-located at the sensor and employs a local Kalman filter. Based on the prediction error, the scheduler decides whether or not to send a new state estimate to the controller. The use of this transmission strategy reduces the communication burden in the channel. We showed that, in this setup, the optimal controller is the certainty-equivalent controller since the measurement quality is not affected by the control policy. We also provided analytical expressions to quantify the trade-off between the communication rate and the control performance.

Figure 4: Transition graph of the Markov chain .

Vi Appendix: Proofs

Proof of Lemma III: By Assumption III, the Kalman filter has reached its steady state. Consequently, the Kalman gain and the error co-variance matrices, and , become constant, i.e., , , and , respectively. Let us define the following random process:

Since , and are mutually independent Gaussian vectors with zero-mean and co-variances , , and , respectively, is Gaussian with zero-mean and co-variance:

(40)

where () is derived by writing while () is obtained by replacing with .

Since are Gaussian random vectors, pairwise independence is equivalent to

For , we have:

where () holds since and are independent of , and ; () is obtained by replacing with (9) iteratively from to and using the fact that and are independent of and ; () is obtained by writing ; and () follows from (40). This concludes the proof.

Proof of Lemma III: For simplicity, we will use a slight abuse of notation and write . We begin by proving that the process is a Markov chain. Using the total law of probabilities and the fact that , we have:

where () and () come from the definition of conditional probability, and () holds since depends stochastically only on as described in (28), and depends on and as described in (14). Bear in mind that knowing implies knowing . Consequently, the process is a Markov chain.

We now show the ergodicity of this Markov chain. Since the Markov chain , depicted in Fig. 4, has positive transition probabilities for any , the chain is evidently irreducible. The chain is also aperiodic because the state has a non-zero probability of being reached for any . By [20, Theorem 3.3], this irreducible chain with finite state space is positive recurrent. Since the process is irreducible, aperiodic and positive recurrent, it is also ergodic. As the process is an irreducible aperodic Markov chain with finitely many states, it has a unique invariant distribution such that and ; see [21, Corollary 2.11]. This concludes the proof.

Lemma

Suppose that is a sample of . Define the following events:

(41)

for all , with the convention that is a sure event. For any given , the probability of these events , for all , can be computed as:

(42)

Proof of Lemma VI: Assume that is a sample of . Since when , the stochastic triggering rule (13) can be rewritten as

For any given , we compute:

This concludes the proof.

Proof of Lemma III: We focus on the non-trivial cases where can take any value from and , as the remaining cases are evident from the structure of the Markov chain in Fig. 4. We first investigate the transition probability . Since corresponds to as a consequence of (14), we have: