TradeOffs in Stochastic EventTriggered Control
Abstract
This paper studies the optimal outputfeedback control of a linear timeinvariant system where a stochastic eventbased scheduler triggers the communication between the sensor and the controller. The primary goal of the use of this type of scheduling strategy is to provide significant reductions in the usage of the sensortocontroller communication and, in turn, improve energy expenditure in the network. In this paper, we aim to design an admissible control policy, which is a function of the observed output, to minimize a quadratic cost function while employing a stochastic eventtriggered scheduler that preserves the Gaussian property of the plant state and the estimation error. For the infinite horizon case, we present analytical expressions that quantify the tradeoff between the communication cost and control performance of such eventtriggered control systems. This tradeoff is confirmed quantitatively via numerical examples.
I Introduction
Over the past decade, distributed control and estimation over networks have been a major trend. Thanks to the forthcoming revolution of the InternetofThings (IoT) and resulting interconnectedness of smart technologies, the importance of decision making over communication networks grows ever larger in our modern society. These technological advances, however, bring new challenges regarding how to use the limited computation, communication, and energy resources efficiently. Consequently, event and selftriggered algorithms have appeared as an alternative to traditional timetriggered algorithms in both estimation and control; see, e.g., [1].
A vast majority of the research in this area has mainly focused on proving the stability of the proposed control schemes, and demonstrating the effectiveness of such control systems, as compared to periodically sampled ones, through numerical simulations. However, an important stream of work in such schemes is analytically characterizing the tradeoff between the control performance and communication rate achieved via these algorithms. Early works on eventtriggered control, such as [2, 3, 4, 5], provided performance expressions but only for scalar systems. The authors of [6] later extended the work of [2] to a class of secondorder systems. The work in [7] studies state estimation for multiple plants across a shared communication network, and quantified communication and estimation performance. Recently, the authors of [8] investigated the minimumvariance eventtriggered outputfeedback control problem; cf. [2]. They established a separation between the control strategy and the scheduling decision, and they also showed that scheduling decisions are determined by solving an optimal stopping problem. Our initial work in [9] considered a certain structure of controllers such as deadbeat controllers, and analytical expressions for the control performance and communication rate were obtained. Differing from [9], in the current work, we will focus on designing optimal controllers by establishing a separation between the controller and the scheduler.
Optimal eventtriggered controller design requires the joint design of an optimal control law and an optimal eventbased scheduler. The associated optimization problem becomes notoriously difficult [10] since, in general, the controller and the scheduler have different information. A vast majority of work in the literature focuses on the design of optimal feedback control laws for a predefined scheduling rule. It is important to note that designing an optimal control law might be very complicated even if one considers a fixed eventtriggering policy. For instance, our recent work [11] shows that the optimal control problem, where a thresholdbased eventtriggered mechanism is used to decide if there is a need for transmission of new control actions based on knowledge of the plant state, leads to a nonconvex optimization problem.
The selection of the eventtriggering mechanism is essential for the computation of the control performance. As noted in [9], even in the case of GaussMarkov plant models, due to the use of a (deterministic) thresholdbased triggering mechanism, the plant state becomes a truncated Gaussian variable. As a result, computation of the control performance becomes challenging since it requires calculating the covariance of the plant state via numerical methods. One way to tackle this problem consists in employing a stochastic triggering mechanism, which preserves the Gaussianity of the plant state, as proposed in [12, 13, 14]. Our initial work in [15] used a deadbeat controller and a stochastic scheduler, which is similar to the ones mentioned above, to quantify the tradeoff between the communication and the control cost for scalar systems. Similarly, the authors of [16] proposed an eventtriggered control scheme that works under stochastic triggering rules. They also derived a control policy that always outperforms a periodic one. Differing from [16], in this work, we will focus on solving the optimal output eventtriggered control problem.
Contributions. In this paper, we consider optimal outputfeedback control of a linear timeinvariant system where a stochastic eventbased triggering algorithm dictates the communication between the sensor and the controller. The proposed scheduler decides at each time step whether or not to transmit new state estimates from the sensor to the controller based on state estimation errors. The main contributions of this manuscript are as follows:

We develop a framework for quantifying the closedloop control performance and the communication rate in the channel between the sensor and the controller.

We confirm that the certaintyequivalent controller is optimal under the scheduling rule based on estimation errors. Our previous work [9] used a transmission strategy based on the plant state, and employed a sequence of deadbeat control actions to establish a resetting property, but this was not optimal since the separation principle between control and scheduling does not hold.

We derive analytical expressions for the (average) communication rate and control performance. Our analysis relies on a Markov chain characterization of the evolution of the state prediction error (cf. [9]) where the states of this Markov chain describe the time elapsed since the last transmission.

Due to the use of the stochastic triggering rule, we can compute the conditional covariance of the comparison error (i.e., the difference between the state estimation error at the sensor and the state estimation error at the controller) in a closedform. Consequently, it becomes almost effortless to compute the closedloop control performance; cf. [9].
Outline. The remainder of the paper is organized as follows: Section II describes the system architecture and formulates an optimal eventtriggered control problem. In Section III, a control policy which minimizes a quadratic cost function under an eventtriggered transmission constraint, is derived. This section also presents analytic expressions of the communication rate and the control performance for the infinite horizon problem. An illustrative example is presented to demonstrate the tradeoff between communication and control performance in Section IV, while Section V finalizes the paper with concluding remarks. The Appendix provides detailed proofs of the main results.
Ii Problem Formulation
Control architecture. We consider the feedback control system depicted in Figure 1. A physical plant , whose dynamics can be represented by a linear timeinvariant stochastic system, is being controlled. A sensor takes periodic samples of the plant output and transmits the estimate of the plant state to the controller over a resourceconstrained communication channel. To tackle the resource constraint, the sensor employs an eventbased scheduler, that makes a transmission decision by comparing its estimate of the plant state with the estimate at the controller. The controller computes new control actions based on the available information. Whenever the controller receives a new state estimate from the sensor, it calculates a control command based on this state estimate. Otherwise, it runs an estimator to predict the plant state, and it uses this information to calculate a new control action. In this context, we are interested in deriving analytical performance guarantees, both regarding the control performance and the number of transmissions between the sensor and the controller.
Process model. The system is modeled as a discretetime, linear timeinvariant (LTI) system,
(1) 
driven by the control input (calculated by the controller ), and an unknown noise process . The state is available only indirectly through the noisy output measurement
(2) 
The two noise sources and are assumed to be uncorrelated zeromean Gaussian whitenoise random processes with covariance matrices and , respectively. We refer to as the process noise, and to as the measurement noise. The initial state is modeled as a Gaussian distributed random variable with mean and covariance . We assume that the pairs and are controllable while the pair is observable.
Sensor, preprocessor, and scheduler. Using a standard Kalman filter, the sensor locally computes minimum mean squared error (MMSE) estimates of the plant state based on the information available to the sensor at time , and transmits them to the controller. As noted in [17], sending local state estimates, in general, provides better performance than transmitting measurements. The sensor also employs a transmission scheduler, which decides whether or not to send the current state estimate to the controller at each timestep as determined by
(3) 
Assumption
The sensor has precise knowledge of the control policy used to generate control actions, which are computed by the controller and applied by the actuator to the plant. Hence, the information set of the smart sensor contains all controls used up to time .
The information set available to the sensor at time is:
(4) 
The minimum mean squared error estimate of the plant state can be computed recursively starting from the initial condition and using a Kalman filter [18]. At this point, it is worth reviewing the fundamental equations underlying the Kalman filter algorithm. The algorithm consists of two steps:

Prediction step: This step predicts the state, estimation error, and estimation error covariance at time dependent on information at time :
(5) (6) (7) 
Update step: This step updates the state, estimation error, and estimation error covariance using a blend of the predicted state and the observation :
(8) (9) (10) where the gain matrix is given by
(11)
It is worth noting that the estimation error at the sensor is Gaussian with zeromean and covariance , that evolves according to the standard Riccati recursion [19, Chapter 9]. Since the pair is observable and the pair is controllable, the matrices and converge exponentially to steady state values and , respectively. Similarly, the matrix also converges to a steady state value, i.e., .
The scheduler and the sensor are collocated, and the scheduler has access to all available information at the sensor. Moreover, the scheduler employs an eventbased triggering mechanism to decide if there is a need for transmission of an updated state estimate from the sensor to the controller. The occurrence of information transmission is defined as
(12) 
where is a (random) binary decision variable (which in this paper evolves according to (13)), is a nonnegative integer variable introduced to describe the time elapsed since the last transmission, and is a timeout interval. Such a timeout mechanism is critical in eventtriggered control systems to guard against faulty components; see, e.g., [9].
To maintain the Gaussianity of the comparison error
(note that is defined in (8) while is introduced in (17)) a variant of the stochastic triggering mechanism proposed in [12, 13, 14] is used. More specifically, the scheduler will decide to transmit a new sensor packet according to the following decision rule:
(13) 
where the triggering parameter is a given positive scalar. As can be seen in (13), the probability of transmitting a new sensor packet (i.e., ) converges to one as goes to infinity. In other words, for large values of , the communication between the sensor and the controller is more likely to be triggered.
The integervalued random process in (12) describes how many time instances ago the last transmission of a sensor packet occurred. Whenever a sensor packet is transmitted from the sensor to the controller, is reset to zero. Thus, the evolution of the random process is defined by
(14) 
or equivalently,
(15) 
where . Notice that the number of time steps between two consecutive transmissions is bounded by the timeout interval . If the number of samples since the last transmission exceeds a timeout value of , the sensor will attempt to transmit new data to the controller even if the comparison error does not satisfy the triggering condition (13). Thus, a transmission (i.e., ) will occur when either or there is a timeout.
Remark
It is worth noting that, as can be seen in (15), the events and are equivalent to each other.
At time instances when , the sensor transmits its local state estimate to the controller. As a result, the information set available to the controller at time (and before deciding upon ) can be defined as:
Under the eventbased scheduling mechanism, (12) – (14), the controller runs an MMSE estimator to compute estimates of the plant state as follows:
(16)  
(17) 
where is the optimal estimate at the controller if the sensor did not transmit any information at timestep . Note that the optimality of this estimator can be shown by using a similar argument to that provided in [14, Lemma 4].
Assumption
In addition to computing , the sensor operates another estimator, which mimics the one at the controller, since transmission decisions rely on both and ; see (13). This can be done provided we make the following assumption.
Assumption
Both the smart sensor and the controller know the plant model (but not realizations of the noise processes).
Controller design and performance criterion. We aim at finding the control strategies , as a function of the admissible information set , to minimize a quadratic cost function of the form
(18) 
where and . At time instances when (i.e., the controller has received sensor packets), the controller uses the state estimate which is transmitted by the sensor. However, at time instances when , the controller uses the outcome of the estimator at the controller side. As is wellknown in related situations (see e.g., [10]), if the transmission decision is independent of the control strategy , then the certainty equivalent controller is optimal. In Section III, we will confirm that the certainty equivalent controller is optimal under the eventbased scheduler proposed above.
Iii Main Results
We wish to quantify the communication rate and control performance of the feedback control system described by (1) and (2), where the eventbased triggering mechanism (13) determines the communication between the sensor and the controller. We will first demonstrate that the time elapsed between two consecutive transmissions can be regarded as a discretetime, finite state, timehomogeneous Markov chain. Then, using an ergodicity property, we will provide an analytical formula for the communication rate between the sensor and the controller. Subsequently, we will show that the certainty equivalent controller is still optimal with the eventtriggering rule (13). Lastly, we will compute the control performance analytically for the infinite horizon case.
Assumption
In the rest of this paper, we will assume that the local Kalman filter at the sensor runs in steady state.
We first define the state prediction error at the controller
(19) 
which evolves as
(20) 
Then, we define the state estimation error at the controller
(21) 
which evolves as
(22) 
Define also the comparison errors:
(23)  
(24) 
Whenever a transmission occurs (i.e., ), the state estimation error at the controller is equal to , since the most recent sensor packet is available at the controller. It is then possible to write the stochastic recurrence equations (23) and (24) as
(25) 
and
(26) 
where . Notice that the comparison errors and propagate according to a linear system with openloop dynamics , driven by the process .
Lemma
is a sequence of pairwise independent Gaussian random vectors such that with .
Remark
If the sensor has perfect state measurements (i.e., ), then is equal to .
Definition (Cumulative error)
We shall characterize the cumulative comparison error (i.e., the error that occurs in estimation at the controller over time due to intermittent transmissions) via
(27) 
Using Definition III, the stochastic recurrence equations (25) and (26) can be then rewritten as
(28)  
(29) 
Lemma (Markov process)
The random process is an ergodic, timehomogeneous Markov chain with a finite state space . Thus, it has a unique invariant distribution such that and for all .
Lemma (Augmented cumulative error vector)
Consider with as in (27). Then, is a random vector having a multivariate normal distribution with zeromean and covariance:
for any .
The next lemma computes the transition probabilities of the Markov chain defined in Lemma III.
Lemma (Transition probabilities)
The transition matrix of the Markov chain is given by
where the nonzero transition probabilities are computed as
The visit of the Markov chain to the state is analogous to a transmission (i.e., ) of the estimate of the plant state from the sensor to the controller. Using Remark II and the ergodic theorem for Markov chains [20, Theorem 5.3], we have:
(30) 
where is the empirical frequency of transmissions. With the transition probabilities of this Markov chain, we can give an explicit characterization of the average communication rate of the eventtriggered control system:
Theorem (Communication rate)
Remark
Note that, as goes to zero, the communication between the sensor and the controller becomes periodic.
The next theorem describes the optimal control law for the eventtriggered control system at hand.
Theorem (Optimal control law)
Consider the system (1) and (2), and the problem of minimizing the cost function (18) under the eventbased triggering mechanism (12) – (14) for a fixed . Then, there exists a unique admissible optimal control policy
(32) 
where
(33)  
(34) 
with initial values . The minimum value of the cost function is obtained as
(35) 
where .
Remark
Our result should be viewed in the light of the limited information available to the controller. At every time step , the controller computes an optimal control input based on the information set . Our result is akin to the one derived in [10], however here we can also provide the closedform expression of the control cost for the infinite horizon case (see Theorem III).
Lemma (Gaussianitypreservation)
The conditional random variable, , has a Gaussian distribution with zeromean and covariance:
Using the previous theorems, we have the following result to calculate the average control performance measured by a linearquadratic function.
Theorem (Infinite horizon control performance)
Suppose the pairs and are controllable, and the pairs and are observable. Moreover, suppose that . Then, we have the following:

The infinite horizon optimal controller gain is constant:
(36) 
The matrices and are the positive definite solutions of the following algebraic Riccati equations:
(37) (38) 
The expected minimum cost converges to the following value:
(39) where , , and satisfies .
Iv Numerical Example
In this section, numerical simulations are provided to assess the performance of the stochastic eventtriggering algorithm proposed in Section II, and verify the theoretical results presented in Section III. To this end, the system parameters are chosen as follows:
The matrix has one stable (i.e., ) and one unstable eigenvalue (i.e., ). The timeout interval is set to . Notice that the pairs and are controllable, the pairs and are observable, and , as required by the assumptions of the theorems presented in Section III.
For various values of ranging from to , we evaluate the communication rate and the control performance as predicted by Theorems III and III, respectively. We compare the analytic results to Monte Carlo simulations of the closedloop system. For each value of , we conduct Monte Carlo simulations for the horizon length of samples, and obtain the mean communication rate and the control performance. The comparison is shown in Fig. 2 for the communication rate and the control performance. It can be seen that the analytic results match the Monte Carlo simulations very closely.
We can also obtain results on when changing the scheduling parameter has the most effect as demonstrated in Fig. 3. There are two extreme cases: 1) as goes to infinity, the communication rate becomes one, and the control performance converges to ; 2) as goes to zero, the transmission rate converges to zero, and the control performance becomes unbounded. We observe, for instance, that changing the scheduling parameter from one to infinity has minimal effect on the control performance, but nearly doubles the communication frequency. As can be seen in Fig. 2, by setting , we can reduce the communication between the sensor and the controller by almost , while only slightly sacrificing the control performance of the closedloop system.
V Conclusions and Discussions
This paper has focused on the optimal control of a linear stochastic system, where a stochastic eventbased scheduling mechanism governs the communication between the sensor and the controller. The scheduler is colocated at the sensor and employs a local Kalman filter. Based on the prediction error, the scheduler decides whether or not to send a new state estimate to the controller. The use of this transmission strategy reduces the communication burden in the channel. We showed that, in this setup, the optimal controller is the certaintyequivalent controller since the measurement quality is not affected by the control policy. We also provided analytical expressions to quantify the tradeoff between the communication rate and the control performance.
Vi Appendix: Proofs
Proof of Lemma III: By Assumption III, the Kalman filter has reached its steady state. Consequently, the Kalman gain and the error covariance matrices, and , become constant, i.e., , , and , respectively. Let us define the following random process:
Since , and are mutually independent Gaussian vectors with zeromean and covariances , , and , respectively, is Gaussian with zeromean and covariance:
(40) 
where () is derived by writing while () is obtained by replacing with .
Since are Gaussian random vectors, pairwise independence is equivalent to
For , we have:
where () holds since and are independent of , and ; () is obtained by replacing with (9) iteratively from to and using the fact that and are independent of and ; () is obtained by writing ; and () follows from (40). This concludes the proof.
Proof of Lemma III: For simplicity, we will use a slight abuse of notation and write . We begin by proving that the process is a Markov chain. Using the total law of probabilities and the fact that , we have:
where () and () come from the definition of conditional probability, and () holds since depends stochastically only on as described in (28), and depends on and as described in (14). Bear in mind that knowing implies knowing . Consequently, the process is a Markov chain.
We now show the ergodicity of this Markov chain. Since the Markov chain , depicted in Fig. 4, has positive transition probabilities for any , the chain is evidently irreducible. The chain is also aperiodic because the state has a nonzero probability of being reached for any . By [20, Theorem 3.3], this irreducible chain with finite state space is positive recurrent. Since the process is irreducible, aperiodic and positive recurrent, it is also ergodic. As the process is an irreducible aperodic Markov chain with finitely many states, it has a unique invariant distribution such that and ; see [21, Corollary 2.11]. This concludes the proof.
Lemma
Suppose that is a sample of . Define the following events:
(41) 
for all , with the convention that is a sure event. For any given , the probability of these events , for all , can be computed as:
(42) 