Highorder maximum principles for the stability analysis of positive bilinear control systems
Abstract
We consider a continuoustime positive bilinear control system (PBCS), i.e. a bilinear control system with Metzler matrices. The positive orthant is an invariant set of such a system, and the corresponding transition matrix is entrywise nonnegative for all time . Motivated by the stability analysis of positive linear switched systems (PLSSs) under arbitrary switching laws, we fix a final time and define a control as optimal if it maximizes the spectral radius of . A recent paper [2] developed a firstorder necessary condition for optimality in the form of a maximum principle (MP). In this paper, we derive higherorder necessary conditions for optimality for both singular and bangbang controls. Our approach is based on combining results on the secondorder derivative of the spectral radius of a nonnegative matrix with the generalized LegendreClebsch condition and the AgrachevGamkrelidze secondorder optimality condition.
I Introduction
Consider the continuoustime linear switched system
(1) 
where is the state vector, and is a piecewise constant function referred to as the switching signal. This models a system that can switch between the two linear subsystems
Recall that (I) is said to be globally uniformly asymptotically stable (GUAS) if there exists a class function^{1}^{1}1A continuous function belongs to the class if it is strictly increasing and . A continuous function belongs to the class if for each fixed , belongs to , and for each fixed , the mapping is decreasing and as . such that for any initial condition and any switching law , the corresponding solution of (I) satisfies
This implies in particular that
(2) 
and or linear switched systems, (2) is in fact equivalent to GUAS (see, e.g., [3]). Switched systems and, in particular, their stability analysis are attracting considerable interest in the last two decades; see e.g. the survey papers [4, 5, 6, 7] and the monographs [8, 9, 10, 11, 12, 13].
It is wellknown that a necessary (but not sufficient) condition for GUAS of (I) is the following.
Assumption 1
The matrix is Hurwitz for all .
Recall that a linear system
(3) 
with , is called positive if the positive orthant
is an invariant set of the dynamics, i.e., implies that for all .
Positive systems play an important role in systems and control theory because in many physical systems the statevariables represent quantities that can never attain negative values (e.g. population sizes, probabilities, concentrations, buffer loads) [14, 15, 16]. A necessary and sufficient condition for (3) to be positive is that is a Metzler matrix, that is, for all . If is Metzler then is (entrywise) nonnegative for all . By the Perron–Frobenius theory, the spectral radius of (i.e., the eigenvalue with maximal absolute value) is real and nonnegative, and since is nonsingular, it is in fact positive.
If both and are Metzler and then (I) is called a positive linear switched system (PLSS). Mason and Shorten [17], and independently David Angeli, posed the following.
Had this conjecture been true, it would have implied that determining GUAS for a PLSS is relatively simple. (See [18] for analysis of the computational complexity of determining whether any matrix in a convex set of matrices is Hurwitz.) Gurvits, Shorten, and Mason [19] proved that Conjecture 1 is in general false (see also [20]), but that it does hold when (even when the number of subsystems is arbitrary). Their proof in the planar case is based on showing that the PLSS admits a common quadratic Lyapunov function (CQLF). (For more on the analysis of switched systems using CQLFs, see [5, 4, 21, 22, 23].) Margaliot and Branicky [24] derived a reachability–with–nice–controls–type result for planar bilinear control systems, and showed that the proof of Conjecture 1 when follows as a special case. Fainshil, Margaliot, and Chigansky [25] showed that Conjecture 1 is false already for the case . In general, it seems that as far as the GUAS problem is concerned, analyzing PLSSs is not simpler than analyzing linear switched systems.
There is a rich literature on sufficient conditions for GUAS, see, e.g., [5, 6, 8, 7, 12]. A more challenging problem is to determine a necessary and sufficient condition for GUAS. What makes this problem difficult is that the set of all possible switching laws is huge, so exhaustively checking the solution for each switching law is impossible.
A natural idea is to try and characterize a “most destabilizing” switching law of the switched system, and then analyze the behavior of the corresponding trajectory . If converges to the origin, then so does any trajectory of the switched system and this establishes GUAS. This idea was pioneered by E. S. Pyatntisky [26, 27], who studied the celebrated absolute stability problem (ASP). This variational approach was further developed by several scholars including N. E. Barabanov and L. B. Rapoport, and proved to be highly successful; see the survey papers [28, 29, 30], the related work in [31, 32], and the recent extensions to the stability analysis of discrete–time linear switched systems in [33, 34].
A first attempt to extend the variational approach to the stability analysis of PLSSs was taken in [35] using the classical Pontryagin maximum principle (PMP). Recently, Fainshil and Margaliot [2] developed an alternative approach that combines the PerronFrobenius theory of nonnegative matrices with the standard needle variation used in the PMP.
The goal of this paper is to derive stronger, higherorder necessary conditions for optimality. We thus begin by reviewing the firstorder MP in [2].
Ia Stability analysis of PLSSs: a Variational Approach
The variational approach to the stability analysis of a linear switched system includes several steps. The first step is relaxing (I) to the bilinear control system (BCS)
(4)  
where , , and is the set of measurable controls taking values in . Note that for [], Eq. (4) yields [, i.e., trajectories of the BCS corresponding to piecewise constant bangbang controls are also trajectories of the original switched system.
The BCS (4) is said to be globally asymptotically stable (GAS) if for all and all . Since every trajectory of the switched system (I) is also a trajectory of (4), GAS of (4) implies GUAS of the linear switched system. It is not difficult to show that the converse implication also holds, so the BCS is GAS if and only if the linear switched system is GUAS. Thus, the GUAS problem for the switched linear system (I) is equivalent to the GAS problem for the BCS (4).
From here on we assume that the switched system is positive, i.e. is Metzler for all . For the BCS, this implies that if , then for all and all . Thus (4) becomes a positive bilinear control system (PBCS).
For , and , let denote the solution at time of the matrix differential equation
(5) 
It is straightforward to verify that the solution of (4) satisfies for all and all . In other words, is the transition matrix from time to time of (4) corresponding to the control . To simplify the notation, we will sometimes omit the dependence on and just write .
When the initial time is we write (IA) as
(6) 
For a PBCS, is a nonnegative matrix for all and all . Since it is also nonsingular, the spectral radius is a real and positive eigenvalue of , called the Perron root. If this eigenvalue is simple then the corresponding eigenvector , called the Perron eigenvector, is unique (up to multiplication by a scalar). The next step in the variational approach is to relate to GAS of the PBCS.
Define the generalized spectral radius of the PBCS (4) by
where
(7) 
Note that the maximum here is welldefined, as the reachable set of (IA) corresponding to is compact [36]. In fact, this is why we consider a bilinear control system with controls in rather than the original linear switched system with piecewise constant switching laws.
The next result relates the GAS of the PBCS to .
Theorem 1
The PBCS (4) is GAS if and only if
Thm. 1 already appeared in [2], but without a proof. For the sake of completeness we include its proof in the Appendix.
Remark 1
Thm. 1 motivates the following optimal control problem.
Problem 1
Consider the PBCS (IA). Fix an arbitrary final time . Find a control that maximizes .
The main result in [2] is a firstorder necessary condition for optimality. Let denote the transpose of the matrix .
Theorem 2
[2] Consider the PBCS (IA). Suppose that is an optimal control for Problem 1. Let denote the corresponding solution of (IA) at time , and let . Suppose that is a simple eigenvalue of . Let [] be an eigenvector of [] corresponding to , normalized such that . Let be the solution of
(8)  
and let be the solution of
(9)  
Define the switching function by
(10) 
Then for almost all ,
(11) 
This MP has some special properties.
Remark 2
First, note that (8) implies that
In particular, substituting yields
as is an eigenvector of corresponding to the eigenvalue . Since scaling by a positive constant has no effect on the sign of , this means that the final condition in (8) can be replaced by the initial condition . This leads to an MP in the form of a onepoint boundary value problem (with the unknown as the initial conditions at time ).
Remark 3
Note that
(12) 
Thus, the switching function is “periodic” in the sense that .
One difficulty in applying Theorem 2 is that both and are unknown. There are cases where this difficulty may be alleviated somewhat berceuse can be expressed in terms of . The next example demonstrates this.
Example 1
Consider an optimal bangbang control in the form
where . The corresponding transition matrix is
where and . Thus, and satisfy
and
(13) 
Suppose that and are symmetric matrices. Then (13) becomes
and multiplying this on the left by yields
Since the Perron eigenvector of is unique (up to multiplication by a constant) this means that
for some .
The MP in Theorem 2 is a necessary, but not sufficient, condition for optimality and it is possible of course that a control satisfying this MP is not an optimal control. The next example demonstrates this.
Example 2
Consider a PBCS satisfying the following properties:

The matrix is symmetric. Its maximal eigenvalue is simple with corresponding eigenvector , and
(14) 
The matrices and are Metzler;

.
(A specific example is , and . Indeed, here , and it is straightforward to verify that all the properties above hold.)
Consider the possibility that the singular control is optimal. Then
Since is symmetric, the corresponding right and left eigenvector is , so in the MP . Thus, (9) and (8) yield
and
Substituting this in (10) yields
Thus, (vacuously) satisfies Thm. 2. However, since the control yields
so clearly is not an optimal control.
The reason that in Example 2 cannot be ruled out is that Thm. 2 is a firstorder MP. More specifically, its derivation is based the following idea. Suppose that is a candidate for an optimal control. Introduce a new control by adding a needle variation to , i.e.
where , is a Lebesgue point of , and is sufficiently small, and analyze the difference to firstorder in . For ,
so
Combining this with known results on the derivative of a simple eigenvalue of a matrix (see, e.g. [37, Chapter 6]) yields
(15) 
If then for all sufficiently small and thus is not optimal. However, in Example 2 the term multiplying in (15) is zero for all , , and , so a firstorder analysis cannot rule out the possibility that is optimal.
Summarizing, Example 2 suggests that there is a need for a higherorder MP, i.e., an MP that takes into account higherorder terms in the Taylor expansion of with respect to , and can thus be used to rule out the optimality of a larger set of controls.
In the next section, we apply the generalized LegendreClebsch condition to derive a highorder necessary condition for a singular control to be optimal. We also combine known results on the secondorder derivative of the Perron root [38] and the AgrachevGamkrelidze secondorder variation for bangbang controls (see, e.g., [39]) to derive a secondorder MP for bangbang controls. The proofs of these results are given in Section III.
Ii Main results
Our first result is a highorder necessary condition for singular optimal controls for Problem 1. Without loss of generality (see [40]), we assume that the singular control is . Let denote the Liebracket of the matrices .
Iia Highorder MP for singular controls
Theorem 3
IiB Secondorder MP for bangbang controls
In this section, we derive an AgrachevGamkrelidzetype secondorder MP for optimal bangbang controls for Problem 1. Note that for an optimal bangbang we have
with and . Any cyclic shift of , e.g.,
also corresponds to an optimal control (as a product of matrices and its cyclic shift have the same spectral radius). This means that we can always assume that is a switching point of , and then (3) implies that is also a switching point of .
Let denote the set of all vectors satisfying
(17) 
We can now state the main result in this section.
Theorem 4
Suppose that is an optimal control for Problem 1, that the conditions of Thm. 2 hold, and that the switching function (10) admits a finite number of zeros at , with , , so that for , for , for , and so on, with . Denote , , and . Define matrices , , by
(18)  
Then
(19) 
Furthermore,
satisfies
(20) 
where
(21) 
We refer to the control defined above as a control with bang arcs. As will be shown in the proof, condition (19) is a firstorder condition (that can also be derived using the firstorder MP). Condition (20) however is a secondorder condition, and it is meaningful for values that make a certain firstorder variation vanish, i.e. that belong to .
Note that the conditions in Thm. 4 are given in terms of and . It is possible of course to state them in terms of and , but this leads to slightly more cumbersome expressions.
The next example demonstrates the calculations for a control with two bang arcs.
Example 4
Consider an optimal control in the form
where . In this case, (19) becomes
and the definition of yields
Of course, this is just the conclusion that we can get from the firstorder MP, as at the switching point we must have
The secondorder term is
so (20) becomes
(22) 
where
Again, this provides information that can also be derived from the firstorder MP, as the fact that and implies that
and differentiating (10) yields
Thus, , so (22) actually holds for all .
However, for a control with more than two bang arcs the secondorder condition does provide new information. The next simple example demonstrates this.
Example 5
Consider the PBCS (IA) with
Note that is Metzler for all . Consider the control
(23) 
with , , , and . The corresponding transition matrix is
Let . The spectral radius of is
and it is a simple eigenvalue. The Perron right and left eigenvectors of are
and
Calculating the switching function defined in (10) yields the behavior depicted in Fig. 1. Note that for , and for , so the control satisfies the firstorder MP.
We now show that the secondorder MP implies that is not an optimal control. Eq. (18) yields
Note that
(24) 
Our goal is to find such that . Indeed, this will imply that is not optimal. It turns out that we can find such an satisfying and . Since must be zero, . Then
so
A tedious but straightforward calculation shows that
so for
Summarizing, . The secondorder term is
and a calculation yields
Clearly, , so the secondorder MP implies that in (23) is not an optimal control. The reason that here satisfies the conditions of the firstorder MP is that it actually minimizes the spectral radius at time . Thus, the secondorder MP plays here a similar role to the secondderivative of a function: it allows to distinguish between a maximum point and a minimum point.
Iii Proofs
Iiia Proof of Thm. 3
Assume that is an optimal control. The corresponding solution of (IA) is . For , consider the control
(25) 
Then
and it follows from the computation in [40, p. 719] (see also [41]) that
(26) 
Note that this implies that any result derived using will be a highorder MP, as the width of the needle variations in (25) is of order yet the perturbation in with respect to is of order . By (26),
so
If then for all sufficiently small , and this contradicts the optimality of . This proves (16).
IiiB Proof of Theorem 4
The proof is based on introducing a new control defined by a perturbation of the switching times to , , …, . Here, and . Define by for , for , and so on. Note that (17) implies that the time length of the perturbed control is
Denote the corresponding transition matrix by . Note also that for any , so . Our goal is to derive an expression for the difference in the form
(27) 
where denotes a function that satisfies .
Suppose for a moment that [] for some