Semidefinite Programming Approach to Gaussian Sequential RateDistortion Tradeoffs
Abstract
Sequential ratedistortion (SRD) theory provides a framework for studying the fundamental tradeoff between datarate and dataquality in realtime communication systems. In this paper, we consider the SRD problem for multidimensional timevarying GaussMarkov processes under meansquare distortion criteria. We first revisit the sensorestimator separation principle, which asserts that considered SRD problem is equivalent to a joint sensor and estimator design problem in which datarate of the sensor output is minimized while the estimator’s performance satisfies the distortion criteria. We then show that the optimal joint design can be performed by semidefinite programming. A semidefinite representation of the corresponding SRD function is obtained. Implications of the obtained result in the context of zerodelay source coding theory and applications to networked control theory are also discussed.
I Introduction
In this paper, we study a fundamental performance limitation of zerodelay communication systems using the sequential ratedistortion (SRD) theory. Suppose that is an valued discrete time random process with known statistical properties. At every time step, the encoder observes a realization of the source and generates a binary sequence of length , which is transmitted to the decoder. The decoder produces an estimation of based on the messages received up to time . Both encoder and decoder have infinite memories of the past. A zerodelay communication system is determined by a selected encoderdecoder pair, whose performance is analyzed in the tradeoff between the rate (viz. the average number of bits that must be transmitted per time step) and the distortion (viz. the discrepancy between the source signal and the reproduced signal ). The region in the ratedistortion plane achievable by a zerodelay communication system is referred to as the zerodelay ratedistortion region.^{1}^{1}1Formal definition of the zerodelay ratedistortion region is given in Section VIA.
The standard ratedistortion region identified by Shannon only provides a conservative outer bound of the zerodelay ratedistortion region. This is because, in general, achieving the standard ratedistortion region requires the use of anticipative (noncausal) codes (e.g., [1, Theorem 10.2.1]). It is well known that the standard ratedistortion region can be expressed by the ratedistortion function^{2}^{2}2This quantity is defined by the infimum of the mutual information between the source and the reproduction subject to the distortion constraint [1, Theorem 10.2.1]. for general sources. In contrast, description of the zerodelay ratedistortion region requires more casedependent knowledge of the optimal source coding schemes. For scalar memoryless sources, it is shown that the optimal performance of zerodelay codes is achievable by a scalar quantizer [2]. Witsenhausen [3] showed that for the th order Markov sources, there exists an optimal zerodelay quantizer with memory structure of order . Neuhoff and Gilbert considered entropycoded quantizers within the class of causal source codes [4], and showed that for memoryless sources, the optimal performance is achievable by timesharing memoryless codes. This result is extended to sources with memory in [5]. An optimal memory structure of zerodelay quantizers for partially observable Markov processes on abstract (Polish) spaces is identified in [6]. The rate of finitedelay source codes for general sources and general distortion measures is analyzed in [7]. Zerodelay or finitedelay joint sourcechannel coding problems have also been studied in the literature; [8, 9, 10, 11] to name a few.
In [12, 13], Tatikonda et al. studied the zerodelay ratedistortion region using a quantity called sequential ratedistortion function,^{3}^{3}3Closely related or apparently equivalent notions to the sequential ratedistortion function have been given various names in the literature, including nonanticipatory entropy [14], constrained distortion rate function [15], causal ratedistortion function [16], and nonanticipative ratedistortion function [17]. which is defined as the infimum of the Massey’s directed information [18] from the source process to the reproduction process subject to the distortion constraint. Although the SRD function does not coincide with the boundary of the zerodelay ratedistortion region in general, it is recently shown that the SRD function provides a tight outer bound of the zerodelay ratedistortion region achievable by uniquely decodable codes [19, 16]. This observation shows an intimate connection between the SRD function and the fundamental performance limitations of realtime communication systems. For this reason, we consider the SRD function as the main object of interest in this paper.
Closely related quantity to the SRD function was studied by Gorbunov and Pinsker [14] in the early 1970’s. Bucy [15] derived the SRD function for GaussMarkov processes in a simple case. In his approach, the problem of deriving the SRD function for GaussMarkov processes under meansquare distortion criteria (which henceforth will be simply referred to as the Gaussian SRD problem) is viewed as a sensorestimator joint design problem to minimize the estimation error subject to the datarate constraint. This approach is justified by the “sensorestimator separation principle,” which asserts that an optimal solution (i.e., the optimal stochastic kernel, to be made precise in the sequel) to the Gaussian SRD problem is realizable by a twostage mechanism with a linearGaussian memoryless sensor and the Kalman filter. Although this fact is implicitly shown in [12, 13], for completeness, we reproduce a proof in this paper based on a technique used in [12, 13].
The sensorestimator separation principle gives us a structural understanding of the Gaussian SRD problem. In particular, based on this principle, we show that the Gaussian SRD problem can be formulated as a semidefinite programming problem (Theorem 1), which is the main contribution of this paper. We derive a computationally accessible form (namely a semidefinite representation^{4}^{4}4To be precise, we show that the exponentiated SRD function for multidimensional GaussMarkov source is semidefinite representable by (27). [20]) of the SRD function, and provide an efficient algorithm to solve Gaussian SRD problems numerically.
The semidefinite representation of the SRD function may be compared with an alternative analytical approach via Duncan’s theorem, which states that “twice the mutual information is merely the integration of the trace of the optimal mean square filtering error” [21]. Duncan’s result was significantly generalized as the “IMMSE” relationships in noncausal [22] and causal [23] estimation problems. Our SDPbased approaches are applicable to the cases with multidimensional and timevarying GaussMarkov sources to which the existing IMMSE formulas cannot be applied straightforwardly. Although we focus on the Gaussian SRD problems in this paper, we note that the standard RD and SRD problems for general sources and distortion measures in abstract (Polish) spaces are discussed in [24] and[17], respectively.
This paper is organized as follows. In Section II, we formally introduce the Gaussian SRD problem, which is the main problem considered in this paper. In Section III, we show that the Gaussian SRD problem is equivalent to what we call the linearGaussian sensor design problem, which formally establishes the sensorestimator separation principle. Then, in Section IV, we show that the linearGaussian sensor design problem can be reduced to an SDP problem, which thus provides us an SDPbased solution synthesis procedure for Gaussian SRD problems. Extensions to stationary and infinite horizon problems are given in Section V. In Section VI, we consider applications of SRD theory to realtime communication systems and networked control systems. Simple simulation results will be presented in Section VII. We conclude in Section VIII.
Notation: Let be an Euclidean space, and be the Borel algebra on . Let be a probability space, and be a random variable. Throughout the paper, we use lower case boldface symbols such as to denote random variables, while is a realization of . We denote by the probability measure of defined by for every . When no confusion occurs, this measure will be also denoted by or . For a Borel measurable function , we write . For a random vector, we write or depending on the initial index, and . Let be a real symmetric matrix of size . Notations or (resp. or ) mean that is a positive definite (resp. positive semidefinite) matrix. For a positive semidefinite matrix , we write .
Ii Problem Formulation
We begin our discussion with an estimationtheoretic interpretation of a simple ratedistortion tradeoff problem. Recall that a ratedistortion problem for a scalar Gaussian random variable with the mean square distortion constraint is an optimization problem of the following form:
(1)  
s.t. 
Here, is a reproduction of the source , and denotes the mutual information between and . The minimization is over the space of reproduction policies, i.e., stochastic kernels . The optimal value of (1) is known as the ratedistortion function, , and can be explicitly obtained [1] as
It is also possible to write the optimal reproduction policy explicitly. To this end, consider a linear sensor
(2) 
where is a Gaussian noise independent of . Also, let
(3) 
be the least mean square error estimator of given . Notice that the right hand side of (3) is given by . Then, it can be shown that an optimal solution to (1) is a composition of (2) and (3), provided that the signaltonoise ratio of the sensor (2) is chosen to be
(4) 
This gives us the following notable observations:
These facts can be significantly generalized, and serve as a guideline to develop a solution synthesis for Gaussian SRD problems in this paper.
Iia Gaussian SRD problem
The Gaussian SRD problem can be viewed as a generalization of (1). Let be an valued GaussMarkov process
(5) 
where and for are mutually independet Gaussian random variables. The Gaussian SRD problem is formulated as
(6a)  
s.t.  (6b) 
where (6b) is imposed for every . Here, is an valued reproduction of . The minimization (6a) is over the space of zerodelay reproduction policies of given and , i.e., the sequences of causal stochastic kernels^{5}^{5}5See Appendix A for a formal description of causal stochastic kernels. . The term is known as directed information, introduced by Massey [18] following Marko’s earlier work [25], and is defined by
(7) 
The Gaussian SRD problem is visualized in Fig. 1.
Remark 1
Directed information measures the amount of information flow from to and is not symmetric, i.e., in general. However, when the process is causally dependent on and is not affected by , it can be shown [26] that . By definition of our source process (5), there is no information feedback from to , and thus holds in our setup. Hence, can be equivalently used as an objective in (PSRD). However, we choose to use for the future considerations (e.g., [27]) in which is a controlled stochastic process and is dependent on . In such cases, and are not equal, and the latter is a more meaningful quantity in many applications.
Since (PSRD) is an infinite dimensional optimization problem, it is difficult to apply numerical methods directly. Hence, we first need to develop a structural understanding of its solution. It turns out that the sensorestimator separation principle still holds for (PSRD), and this observation plays an important role in the subsequent sections. We are going to establish the following facts:

Fact 1’: A sensorestimator separation principle holds for the Gaussian SRD problem. That is, an optimal policy for (PSRD) can be realized as a composition of a sensor mechanism
(8) where are mutually independent Gaussian random variables, and the least mean square error estimator (Kalman filter)
(9) 
Fact 2’: The original optimization problem (PSRD) over an infinitedimensional space is reduced to an optimization problem over a finitedimensional space of matrixvalued signaltonoise ratios of the sensor (8), defined by
(10) Moreover, the optimal , which depends on , can be obtained by SDP.
Unlike (4), an analytical expression of the optimal may not be available. Nevertheless, we will show that they can be easily obtained by SDP.
IiB LinearGaussian sensor design problem
In Section III, we establish the sensorestimator separation principle. To this end, we show that (PSRD) is equivalent to what we call the linearGaussian sensor design problem (PLGS) visualized in Fig. 2. Formally, (PLGS) is formulated as
(11a)  
s.t.  (11b) 
where (11b) is imposed for every . We assume that is produced by a linearGaussian sensor (8), and is produced by the Kalman filter (9). In other words, the optimization domain is the space of causal stochastic kernels with a separation structure (8) and (9), which is parameterized by a sequence of matrices . Intuitively, in (11a) can be understood as the amount of information acquired by the sensor (8) at time . We call this problem a “sensor design problem” because our focus is on choosing an optimal sensing gain in (8) and the noise covariance . Notice that perfect observation with and is trivially the best to minimize the estimation error in (11b) (in fact, is achieved), but it incurs significant information cost (i.e., ), and hence it is not an optimal solution to (PLGS).
Remark 2
In (PLGS), we search for the optimal and . However, the sensor dimension is not given a priori, and choosing it optimally is part of the problem. In particular, if making no observation is the optimal sensing at some specific time instance , we should be able to recover as an optimal solution.
Although the objective functions (6a) and (11a) appear differently, it will be shown in Section III that they coincide in the domain . Moreover, in the same section it will be shown that an optimal solution to (PSRD) can always be found in the domain . These observations imply that one can obtain an optimal solution to (PSRD) by solving (PLGS).
IiC Stationary cases
We will also consider a timeinvariant system
(12) 
where is an valued random variable with , and is a stationary white Gaussian noise. We assume and . Stationary and infinite horizon version of the Gaussian SRD problem is formulated as
(13a)  
s.t.  (13b) 
This is an optimization over the sequence of stochastic kernels . The optimal value of (13) as a function of the average distortion is referred to as the sequential ratedistortion function, and is denoted by .
Similarly, a stationary and infinite horizon version of the linearGaussian sensor design problem is formulated as
(14a)  
s.t.  (14b) 
Here, we assume where is a mutually independent Gaussian stochastic process and . Design variables in (14) are . Again, determining their dimensions is part of the problem.
IiD Soft vs. hardconstrained problems
Introducing Lagrange multipliers , one can also consider a softconstrained version of (PSRD):
(15) 
Similarly to the Lagrange multiplier theorem (e.g., Proposition 3.1.1 in [28]), it is possible to show that there exists a set of multipliers such that an optimal solution to (15) is also an optimal solution to (PSRD). We will prove this fact in Section IV after we establish that both (PSRD) and (15) can be transformed as finite dimensional convex optimization problems. For this reason, we refer to both (PSRD) and (15) as Gaussian SRD problems.
Iii Sensorestimator separation principle
Let and be the optimal values of (PSRD) and (PLGS) respectively. In this section, we show that , and an optimal solution to (PLGS) is also an optimal solution to (PSRD). This result establishes the sensorestimator separation principle (Fact 1’). We introduce another optimization problem (P1), which serves as an intermediate step to establish this fact.
(P1):  
The optimization is over the space of linearGaussian stochastic kernels , where each stochastic kernel is of the form
(16) 
where are some matrices with appropriate dimensions, and is a zeromean, possibly degenerate Gaussian random variable that is independent of . Notice that . The underlying GaussMarkov process is defined by (5). Let be the optimal value of (P1). The next lemma claims the equivalence between (PSRD) and (P1).
Lemma 1

If there exists attaining a value of the objective function in (PSRD), then there exists attaining a value of the objective function in (P1).

Every attaining in (P1) also attains in (PSRD).
Lemma 1 is the most significant result in this section, which essentially guarantees the linearity of an optimal solution to the Gaussian SRD problems. The proof of Lemma 1 can be found in Appendix B. The basic idea of proof relies on the wellknown fact that Gaussian distribution maximizes entropy when covariance is fixed. This proposition appears as Lemma 4.3 in [13], but we modified the proof using the RadonNikodym derivatives so that the proof does not require the existence of probability density functions. The next lemma establishes the equivalence between (P1) and (PLGS).
Lemma 2

If there exists attaining a value of the objective function in (P1), then there exists attaining a value of the objective function in (PLGS).

Every attaining in (PLGS) also attains in (P1).
Proof of Lemma 2 is in Appendix C. Combining the above two lemmas, we obtain the following consequence, which is the main proposition in this section. It guarantees that we can alternatively solve (PLGS) in order to solve (PSRD).
Proposition 1
Suppose . Then there exists an optimal solution to (PLGS). Moreover, an optimal solution to (PLGS) is also an optimal solution to (PSRD), and .
Iv SDPbased synthesis
In this section, we develop an efficient numerical algorithm to solve (PLGS). Due to the preceding discussion, this is equivalent to developing an algorithm to solve (PSRD). Let (5) be given. Assume temporarily that (8) is also fixed. The Kalman filtering formula for computing is
where is the covariance matrix of , which can be recursively computed as
(17a)  
(17b) 
for with . The variable is defined by (10). Using these quantities, mutual information terms in (11a) can be explicitly written as
Note that and guarantee that both differential entropy terms are finite. Hence, (PLGS) is equivalent to the following optimization problem in terms of the variables :
(18a)  
s.t.  (18b)  
(18c) 
Equality (18b) is obtained by eliminating from (17). At this point, one may note that (18) can be viewed as an optimal control problem with state and control input . Naturally, dynamic programming approach has been proposed in the literature in similar contexts [15, 12, 10, 11]. Alternatively, we next propose a method to transform (18) into an SDP problem. This allows us to solve (PSRD) using standard SDP solvers, which is now a mature technology.
Iva SRD optimization as maxdet problem
Now we show that (18) can be converted to a determinant maximization problem [29] subject to linear matrix inequality constraints. The first step is to transform (18) into an optimization problem in terms of only. This is possible by simply replacing the nonlinear equality constraint (18b) with a linear inequality constraint
This replacement eliminates from (18) giving us:
(19a)  
s.t.  (19b)  
(19c) 
Note that (18) and (19) are mathematically equivalent, since eliminated SNR variables can be easily constructed from through
(20) 
The second step is to rewrite the objective function (19a). Regrouping terms, (19a) can be written as a summation of the initial cost , the final cost , and stagewise costs
(21) 
for . Applying the matrix determinant lemma (e.g., Theorem 18.1.1 in [30]), (21) can be rewritten as
(22) 
Due to the monotonicity of the determinant function, (22) is equal to the optimal value of
(23a)  
s.t.  (23b) 
Applying the matrix inversion lemma, (23b) is equivalent to , which is further equivalent to
(24) 
Note that (24) is a linear matrix inequality (LMI) condition. The above discussion leads to the following conclusion.
Theorem 1
An optimal solution to (PLGS) can be constructed by solving the following determinant maximization problem with decision variables :
(25a)  
s.t.  (25b)  
(25c)  
(25d)  
(25e)  
(25f) 
where is a constant. The optimal sequence can be reconstructed from (20), from which satisfying (10) can be reconstructed via the singular value decomposition. An optimal solution to (PLGS) is obtained as a composition of (8) and (9).
Remark 3
Under the assumption that for every , the maxdet problem (25) is always strictly feasible and there exists an optimal solution.^{6}^{6}6To see the strict feasibility, consider for and for with sufficiently small . The constraint set defined by (25b)(25f) can be made compact by replacing (25b) with without altering the result. Thus the existence of an optimal solution is guaranteed by the Weierstrass theorem. Invoking Proposition 1, we have thus shown by construction that there always exists an optimal solution to (PSRD) under this assumption.
Remark 4
Using the same technique, the softconstrained version of the problem (15) can be formulated as:
(26a)  
s.t.  (26b)  
(26c)  
(26d)  
(26e) 
The next proposition claims that (25) and (26) admit the same optimal solution provided Lagrange multipliers , are chosen correctly. This further implies that, with the same choice of , two versions of the Gaussian SRD problems (PSRD) and (15) are equivalent.
Proposition 2
IvB Maxdet problem as SDP
Strictly speaking, optimization problems (25) and (26) are in the class of determinant maximization problems [29], but not in the standard form of the SDP.^{7}^{7}7In the standard form, SDP is an optimization problem of the form . However, they can be considered as SDPs in a broader sense for the following reasons. First, the hard constrained version (25) can be indeed transformed into a standard SDP problem. This conversion is possible by following the discussion in Chapter 4 of [32]. Second, sophisticated and efficient algorithms based on the interiorpoint method for SDP can almost directly be applied to maxdet problems as well. In fact, offtheshelf SDP solvers such as SDPT3 [33] have builtin functions to handle logdeterminant terms directly.
Recall that (PLGS) and (PSRD) have a common optimal solution. Hence, Proposition 1 shows that both (PLGS) and (PSRD) are essentially solvable via SDP, which is much stronger than merely saying that they are convex problems. Note that convexity alone does not guarantee the existence of an efficient optimization algorithm.
IvC Complexity analysis
In this section, we briefly consider the arithmetic complexity (i.e., the worst case number of arithmetic operations needed to obtain an optimal solution) of problem (25), and how it grows as the horizon length grows when the dimensions of the GaussMarkov process (5) are fixed to . For a preliminary analysis, it would be natural for us to resort to the existing interiorpoint method literature (e.g., [32, 34]). Interiorpoint methods for the determinant maximization problem are already considered in [29, 35, 36]. The most computationally expensive step in the interiorpoint method is the Cholesky factorization involved in the Newton steps, which requires operations in general. However, it is possible to exploit the sparsity of coefficient matrices in the SDPs to reduce operation counts [37, 38, 39]. By exploiting the structure of our SDP formulation (25), it is theoretically expected that there exists a specialized interiorpoint method algorithm for (25) whose arithmetic complexity is . However, more careful study and computational experiments are needed to verify this conjecture.
IvD Single stage problem
When , the result of Proposition 1 recovers the wellknown “reverse waterfilling” solution for the standard Gaussian ratedistortion problem [1]. To see this, notice that reduces problem (26) to
s.t. 
Here, we have already assumed and . This does not result in loss of generality, since otherwise a change of variables , where is an orthonormal matrix that makes diagonal, converts the problem into the above form. For any positive definite matrix , Hadamard’s inequality (e.g., [1]) states that and the equality holds if and only if the matrix is diagonal. Hence, if diagonal elements of are fixed, is maximized by setting all offdiagonal entries zero. Thus, the optimal solution to the above problem is diagonal. Writing , the problem is decomposed as independent optimization problems, each of which minimizes subject to . It is easy to see that the optimal solution is . This is the closedform solution to (PLGS) with , and its pictorial interpretation is shown in Fig. 3. This solution also indicates the optimal sensing formula is given by , where and satisfy
In particular, we have , indicating that the optimal dimension of monotonically decreases as the “price of information” increases.
V Stationary problems
Va Sequential ratedistortion function
We are often interested in infinitehorizon Gaussian SRD problems (13). Assuming that is a detectable pair, it can be shown that (13) is equivalent to the infinitehorizon linearGaussian sensor design problem (14) [40]. Moreover, [40] shows that (13) and (14) admit an optimal solution that can be realized as a composition of a timeinvariant sensor mechanism with i.i.d. process and a timeinvariant Kalman filter. Hence, it is enough to minimize the average cost per stage, which leads to the following simpler problem.
(27a)  
s.t.  (27b)  
(27c)  
(27d)  
(27e) 
To confirm (27) is compatible with the existing result, consider a scalar system with and . In this case, a closedform expression of the SRD function is known in the literature [12] [16], which is given by
(28) 
For a scalar system, (27) further simplifies to
(29a)  
s.t.  (29b) 
It is elementary to verify that the optimal value of (29) is if , while it is if . Hence, it can be compactly written as , and the result recovers (28). Alternative representations of the SRD function (27) for stationary multidimensional GaussMarkov processes when are reported in [13, Section IVB] and [17, Section VI].
VB Rank monotonicity
Using an optimal solution to (27) the optimal sensing matrices and are recovered from . In particular, determines the optimal dimension of the measurement vector. Similarly to the case of single stage problems, this rank has a tendency to decrease as increases. A typical numerical behavior is shown in Figure 4. We do not attempt to prove the rank monotonicity here.
Vi Applications and related works
Via Zerodelay source coding
SRD theory plays an important role in the rate analysis of zerodelay source coding schemes. For each , let
be a set of variablelength uniquely decodable codewords. Assume that for , and let be the length of . A zerodelay binary coder is a pair of a sequence of encoders , i.e., stochastic kernels on given , and a sequence of decoders , i.e., stochastic kernels on given . The zerodelay ratedistortion region for the GaussMarkov process (12) is the epigraph of the function
The SRD function is a lower bound of the achievable rate. Indeed, can be shown straightforwardly as
(30a)  
(30b)  
(30c)  
(30d)  
(30e)  
(30f) 
where (30a) holds since there is no feedback from the process to (Remark 1), (30b) follows from the data processing inequality, (30d) holds since conditional entropy is nonnegative, and (30e) is due to the chain rule for entropy. The final inequality (30f) holds since the expected length of a uniquely decodable code is lower bounded by its entropy [1, Theorem 5.3,1].
In general, and do not coincide. Nevertheless, by constructing an appropriate entropycoded dithered quantizer (ECDQ), it is shown in [16] that does not exceed more than a constant due to the “spacefilling loss” of the lattice quantizer and the loss of entropy coding.
ViB Networked control theory
Zerodelay source/channel coding technologies are crucial in networked control systems [41, 42, 43, 44]. Gaussian SRD theory plays an important role in the LQG control problems with information theoretic constraints [13]. It is shown in [19] that an LQG control problem in which observed data must be transmitted to the controller over a noiseless binary channel is closely related to the LQG control problem with directed information constraints. The latter problem is addressed in [27] using the SDPbased algorithm presented in this paper. In [27], the problem is viewed as a sensorcontroller joint design problem in which directed information from the state process to the control input is minimized.^{8}^{8}8The problem considered in [27] is different from the sensorcontroller joint design problems considered in [45] and [46].
ViC Experimental design/Sensor scheduling
In this subsection, we compare the linearGaussian sensor design problem (PLGS) with different types of sensor design/selection problems considered in the literature.
A problem of selecting the best subset of sensors to observe a random variable in order to minimize the estimation error and its convex relaxations are considered in [29]. A sensor selection problem for a linear dynamical system is considered in [47], where submodularity of the objective function is exploited. Dynamic sensor scheduling problems are also considered in the literature. In [48], an efficient algorithm to explore branches of the scheduling tree is proposed. In [49], a stochastic sensor selection strategy that minimizes the expected error covariance is considered.
The linearGaussian sensor design problem (PLGS) is different from these sensor selection/scheduling problems in that it is essentially a continuous optimization problem (since matrices can be freely chosen), and the objective is to minimize an informationtheoretic cost (11a).
Vii Numerical Simulations
In this section, we consider two numerical examples to demonstrate how the SDPbased formulation of the Gaussian SRD problem can be used to calculate the minimal communication bandwidth required for the realtime estimation with desired accuracy.
Viia Optimal sensor design for double pendulum
A linearized equation of motion of a double pendulum with friction and disturbance is given by
where is a Brownian motion. We consider a discrete time model of the above equation of motion obtained through the Tustin transformation. We are interested in designing a sensing model that optimally tradesoff information cost and distortion level.^{9}^{9}9In practice, it is often the case that is partially observable through a given sensor mechanism. In such cases, the framework discussed in this paper is not appropriate. Instead, one can formulate an SRD problem for partially observable GaussMarkov processes. See [50] for details. We solve the stationary optimization problem (27) for this example with various values of . The result is the sequential ratedistortion function shown in Figure 5. Finally, for every point on the tradeoff curve, the optimal sensing matrices and are reconstructed, and the Kalman filter is designed base on them. Figure 6 shows the tradeoff between the distortion level and the tracking performance of the Kalman filter. When the distortion constraint is strict (), the optimally designed sensor generates high rate information ( bits/sample) and the Kalman filter built on it tracks true state very well. When is large (), the optimal sensing strategy chooses “not to observe much”, and the resulting Kalman filter shows poor tracking performance.
ViiB Minimum downlink bandwidth for satellite attitude determination
The equation of motion of the angular velocity vector of a spinstabilized satellite linearized around the nominal angular velocity vector is