High-order maximum principles for the stability analysis of positive bilinear control systems
We consider a continuous-time positive bilinear control system (PBCS), i.e. a bilinear control system with Metzler matrices. The positive orthant is an invariant set of such a system, and the corresponding transition matrix is entrywise nonnegative for all time . Motivated by the stability analysis of positive linear switched systems (PLSSs) under arbitrary switching laws, we fix a final time and define a control as optimal if it maximizes the spectral radius of . A recent paper  developed a first-order necessary condition for optimality in the form of a maximum principle (MP). In this paper, we derive higher-order necessary conditions for optimality for both singular and bang-bang controls. Our approach is based on combining results on the second-order derivative of the spectral radius of a nonnegative matrix with the generalized Legendre-Clebsch condition and the Agrachev-Gamkrelidze second-order optimality condition.
Consider the continuous-time linear switched system
where is the state vector, and is a piecewise constant function referred to as the switching signal. This models a system that can switch between the two linear subsystems
Recall that (I) is said to be globally uniformly asymptotically stable (GUAS) if there exists a class function111A continuous function belongs to the class if it is strictly increasing and . A continuous function belongs to the class if for each fixed , belongs to , and for each fixed , the mapping is decreasing and as . such that for any initial condition and any switching law , the corresponding solution of (I) satisfies
This implies in particular that
and or linear switched systems, (2) is in fact equivalent to GUAS (see, e.g., ). Switched systems and, in particular, their stability analysis are attracting considerable interest in the last two decades; see e.g. the survey papers [4, 5, 6, 7] and the monographs [8, 9, 10, 11, 12, 13].
It is well-known that a necessary (but not sufficient) condition for GUAS of (I) is the following.
The matrix is Hurwitz for all .
Recall that a linear system
with , is called positive if the positive orthant
is an invariant set of the dynamics, i.e., implies that for all .
Positive systems play an important role in systems and control theory because in many physical systems the state-variables represent quantities that can never attain negative values (e.g. population sizes, probabilities, concentrations, buffer loads) [14, 15, 16]. A necessary and sufficient condition for (3) to be positive is that is a Metzler matrix, that is, for all . If is Metzler then is (entrywise) nonnegative for all . By the Perron–Frobenius theory, the spectral radius of (i.e., the eigenvalue with maximal absolute value) is real and nonnegative, and since is non-singular, it is in fact positive.
Had this conjecture been true, it would have implied that determining GUAS for a PLSS is relatively simple. (See  for analysis of the computational complexity of determining whether any matrix in a convex set of matrices is Hurwitz.) Gurvits, Shorten, and Mason  proved that Conjecture 1 is in general false (see also ), but that it does hold when (even when the number of subsystems is arbitrary). Their proof in the planar case is based on showing that the PLSS admits a common quadratic Lyapunov function (CQLF). (For more on the analysis of switched systems using CQLFs, see [5, 4, 21, 22, 23].) Margaliot and Branicky  derived a reachability–with–nice–controls–type result for planar bilinear control systems, and showed that the proof of Conjecture 1 when follows as a special case. Fainshil, Margaliot, and Chigansky  showed that Conjecture 1 is false already for the case . In general, it seems that as far as the GUAS problem is concerned, analyzing PLSSs is not simpler than analyzing linear switched systems.
There is a rich literature on sufficient conditions for GUAS, see, e.g., [5, 6, 8, 7, 12]. A more challenging problem is to determine a necessary and sufficient condition for GUAS. What makes this problem difficult is that the set of all possible switching laws is huge, so exhaustively checking the solution for each switching law is impossible.
A natural idea is to try and characterize a “most destabilizing” switching law of the switched system, and then analyze the behavior of the corresponding trajectory . If converges to the origin, then so does any trajectory of the switched system and this establishes GUAS. This idea was pioneered by E. S. Pyatntisky [26, 27], who studied the celebrated absolute stability problem (ASP). This variational approach was further developed by several scholars including N. E. Barabanov and L. B. Rapoport, and proved to be highly successful; see the survey papers [28, 29, 30], the related work in [31, 32], and the recent extensions to the stability analysis of discrete–time linear switched systems in [33, 34].
A first attempt to extend the variational approach to the stability analysis of PLSSs was taken in  using the classical Pontryagin maximum principle (PMP). Recently, Fainshil and Margaliot  developed an alternative approach that combines the Perron-Frobenius theory of nonnegative matrices with the standard needle variation used in the PMP.
The goal of this paper is to derive stronger, higher-order necessary conditions for optimality. We thus begin by reviewing the first-order MP in .
I-a Stability analysis of PLSSs: a Variational Approach
The variational approach to the stability analysis of a linear switched system includes several steps. The first step is relaxing (I) to the bilinear control system (BCS)
where , , and is the set of measurable controls taking values in . Note that for , Eq. (4) yields [, i.e., trajectories of the BCS corresponding to piecewise constant bang-bang controls are also trajectories of the original switched system.
The BCS (4) is said to be globally asymptotically stable (GAS) if for all and all . Since every trajectory of the switched system (I) is also a trajectory of (4), GAS of (4) implies GUAS of the linear switched system. It is not difficult to show that the converse implication also holds, so the BCS is GAS if and only if the linear switched system is GUAS. Thus, the GUAS problem for the switched linear system (I) is equivalent to the GAS problem for the BCS (4).
From here on we assume that the switched system is positive, i.e. is Metzler for all . For the BCS, this implies that if , then for all and all . Thus (4) becomes a positive bilinear control system (PBCS).
For , and , let denote the solution at time of the matrix differential equation
It is straightforward to verify that the solution of (4) satisfies for all and all . In other words, is the transition matrix from time to time of (4) corresponding to the control . To simplify the notation, we will sometimes omit the dependence on and just write .
When the initial time is we write (I-A) as
For a PBCS, is a non-negative matrix for all and all . Since it is also non-singular, the spectral radius is a real and positive eigenvalue of , called the Perron root. If this eigenvalue is simple then the corresponding eigenvector , called the Perron eigenvector, is unique (up to multiplication by a scalar). The next step in the variational approach is to relate to GAS of the PBCS.
Define the generalized spectral radius of the PBCS (4) by
Note that the maximum here is well-defined, as the reachable set of (I-A) corresponding to is compact . In fact, this is why we consider a bilinear control system with controls in rather than the original linear switched system with piecewise constant switching laws.
The next result relates the GAS of the PBCS to .
The PBCS (4) is GAS if and only if
Thm. 1 motivates the following optimal control problem.
Consider the PBCS (I-A). Fix an arbitrary final time . Find a control that maximizes .
The main result in  is a first-order necessary condition for optimality. Let denote the transpose of the matrix .
 Consider the PBCS (I-A). Suppose that is an optimal control for Problem 1. Let denote the corresponding solution of (I-A) at time , and let . Suppose that is a simple eigenvalue of . Let  be an eigenvector of  corresponding to , normalized such that . Let be the solution of
and let be the solution of
Define the switching function by
Then for almost all ,
This MP has some special properties.
First, note that (8) implies that
In particular, substituting yields
as is an eigenvector of corresponding to the eigenvalue . Since scaling by a positive constant has no effect on the sign of , this means that the final condition in (8) can be replaced by the initial condition . This leads to an MP in the form of a one-point boundary value problem (with the unknown as the initial conditions at time ).
Thus, the switching function is “periodic” in the sense that .
One difficulty in applying Theorem 2 is that both and are unknown. There are cases where this difficulty may be alleviated somewhat berceuse can be expressed in terms of . The next example demonstrates this.
Consider an optimal bang-bang control in the form
where . The corresponding transition matrix is
where and . Thus, and satisfy
Suppose that and are symmetric matrices. Then (13) becomes
and multiplying this on the left by yields
Since the Perron eigenvector of is unique (up to multiplication by a constant) this means that
for some .
The MP in Theorem 2 is a necessary, but not sufficient, condition for optimality and it is possible of course that a control satisfying this MP is not an optimal control. The next example demonstrates this.
Consider a PBCS satisfying the following properties:
The matrix is symmetric. Its maximal eigenvalue is simple with corresponding eigenvector , and
The matrices and are Metzler;
(A specific example is , and . Indeed, here , and it is straightforward to verify that all the properties above hold.)
Consider the possibility that the singular control is optimal. Then
Substituting this in (10) yields
Thus, (vacuously) satisfies Thm. 2. However, since the control yields
so clearly is not an optimal control.
The reason that in Example 2 cannot be ruled out is that Thm. 2 is a first-order MP. More specifically, its derivation is based the following idea. Suppose that is a candidate for an optimal control. Introduce a new control by adding a needle variation to , i.e.
where , is a Lebesgue point of , and is sufficiently small, and analyze the difference to first-order in . For ,
Combining this with known results on the derivative of a simple eigenvalue of a matrix (see, e.g. [37, Chapter 6]) yields
If then for all sufficiently small and thus is not optimal. However, in Example 2 the term multiplying in (15) is zero for all , , and , so a first-order analysis cannot rule out the possibility that is optimal.
Summarizing, Example 2 suggests that there is a need for a higher-order MP, i.e., an MP that takes into account higher-order terms in the Taylor expansion of with respect to , and can thus be used to rule out the optimality of a larger set of controls.
In the next section, we apply the generalized Legendre-Clebsch condition to derive a high-order necessary condition for a singular control to be optimal. We also combine known results on the second-order derivative of the Perron root  and the Agrachev-Gamkrelidze second-order variation for bang-bang controls (see, e.g., ) to derive a second-order MP for bang-bang controls. The proofs of these results are given in Section III.
Ii Main results
Our first result is a high-order necessary condition for singular optimal controls for Problem 1. Without loss of generality (see ), we assume that the singular control is . Let denote the Lie-bracket of the matrices .
Ii-a High-order MP for singular controls
Ii-B Second-order MP for bang-bang controls
In this section, we derive an Agrachev-Gamkrelidze-type second-order MP for optimal bang-bang controls for Problem 1. Note that for an optimal bang-bang we have
with and . Any cyclic shift of , e.g.,
also corresponds to an optimal control (as a product of matrices and its cyclic shift have the same spectral radius). This means that we can always assume that is a switching point of , and then (3) implies that is also a switching point of .
Let denote the set of all vectors satisfying
We can now state the main result in this section.
Suppose that is an optimal control for Problem 1, that the conditions of Thm. 2 hold, and that the switching function (10) admits a finite number of zeros at , with , , so that for , for , for , and so on, with . Denote , , and . Define matrices , , by
We refer to the control defined above as a control with bang arcs. As will be shown in the proof, condition (19) is a first-order condition (that can also be derived using the first-order MP). Condition (20) however is a second-order condition, and it is meaningful for values that make a certain first-order variation vanish, i.e. that belong to .
Note that the conditions in Thm. 4 are given in terms of and . It is possible of course to state them in terms of and , but this leads to slightly more cumbersome expressions.
The next example demonstrates the calculations for a control with two bang arcs.
Consider an optimal control in the form
where . In this case, (19) becomes
and the definition of yields
Of course, this is just the conclusion that we can get from the first-order MP, as at the switching point we must have
The second-order term is
so (20) becomes
Again, this provides information that can also be derived from the first-order MP, as the fact that and implies that
and differentiating (10) yields
Thus, , so (22) actually holds for all .
However, for a control with more than two bang arcs the second-order condition does provide new information. The next simple example demonstrates this.
Consider the PBCS (I-A) with
Note that is Metzler for all . Consider the control
with , , , and . The corresponding transition matrix is
Let . The spectral radius of is
and it is a simple eigenvalue. The Perron right and left eigenvectors of are
We now show that the second-order MP implies that is not an optimal control. Eq. (18) yields
Our goal is to find such that . Indeed, this will imply that is not optimal. It turns out that we can find such an satisfying and . Since must be zero, . Then
A tedious but straightforward calculation shows that
Summarizing, . The second-order term is
and a calculation yields
Clearly, , so the second-order MP implies that in (23) is not an optimal control. The reason that here satisfies the conditions of the first-order MP is that it actually minimizes the spectral radius at time . Thus, the second-order MP plays here a similar role to the second-derivative of a function: it allows to distinguish between a maximum point and a minimum point.
Iii-a Proof of Thm. 3
Assume that is an optimal control. The corresponding solution of (I-A) is . For , consider the control
If then for all sufficiently small , and this contradicts the optimality of . This proves (16).
Iii-B Proof of Theorem 4
The proof is based on introducing a new control defined by a perturbation of the switching times to , , …, . Here, and . Define by for , for , and so on. Note that (17) implies that the time length of the perturbed control is
Denote the corresponding transition matrix by . Note also that for any , so . Our goal is to derive an expression for the difference in the form
where denotes a function that satisfies .
Suppose for a moment that  for some