Safety Control of Positive Monotone Systems with Bounded Uncertainties
Abstract
Monotone systems are prevalent in models of engineering applications such as transportation and biological networks. In this paper, we investigate the problem of finding a control strategy for a discrete time positive monotone system with bounded uncertainties such that the evolution of the system is guaranteed to be confined to a safe set in the state space for all times. By exploiting monotonicity, we propose an approach to this problem which is based on constraint programming. We find control strategies that are based on repetitions of finite sequences of control actions. We show that, under assumptions made in the paper, safety control of cooperative systems does not require state measurement. We demonstrate the results on a signalized urban traffic network, where the safety objective is to keep the traffic flow free of congestion.
I Introduction
Designing control policies subject to safety constraints is a fundamental problem in the automation of complex systems. From a game theoretic perspective, the safety control problem, also known as safety game, is the problem of finding a control policy that guarantees that the evolution of the system is restricted to a safe region in the state space, regardless of the actions taken by the adversary. The solution to this problem involves finding a robust control invariant set [1]. Iterative computation of robust control invariant sets has been extensively studied for linear and piecewise affine systems [2][3], where intensive polyhedral operations are required to carry out set iterations.
In this work, we focus on a special class of systems that are monotone, or order preserving, and provide an alternative computational approach to the safety control problem. cooperative systems are common in models of biological, socioeconomical and transportation networks. Monotonicity, in general, is a mathematical property that indicates a type of order preserving law. Monotone autonomous systems are thoroughly studied in [4]. In [5], the authors introduced cooperative control systems and provided results on steady state responses and stability.
We consider discrete time uncertain control systems that are monotone with respect to positive orthant in the state and adversarial inputs space. In contrast to [5], we do not assume monotonicity with respect to controls. We do not even require the control space to be partially ordered. On the other hand, we assume a more restrictive form of the safety region in the problem formulation. Our consideration of such systems and specifications is motivated by the dynamics of urban traffic networks [6], which are described in more detail later in the paper. The key result of this work is to show that computing robust control invariant sets maps to computing finite sequences of control actions, which we call ssequences. We show that repeated executions of ssequences are safe control policies that do not require state feedback. We also show that, under some mild assumptions, the existence of ssequences is almost necessary. To the best of our knowledge, these fundamental insights were not established before.
Safety control of monotone systems has also been considered in [7] and [8]. However, in these papers, monotonicity with respect to the controls was also assumed. Therefore, the results of this paper are more general in this respect. Setinvariance theories are also closely related to stability analysis. In [4], [9] and [10], the authors studied the stability of monotone and mixed monotone deterministic systems with no control inputs. Extending these results to cooperative systems with partially ordered adversarial inputs is relatively straightforward, but it is not so obvious for systems with control inputs, specifically for discontinuous control admissible sets.
This work is also related to finite state abstraction based control of (mixed) monotone systems [11]. This approach enables control synthesis from rich temporal logic [12] specifications, of which safety is a special yet important class. However, discretization of the state space is computationally expensive and its complexity grows exponentially with respect to the size of the system. Furthermore, with particular focus on safety specifications of the form assumed in this paper, our results are stronger in the following ways. First, if our approach does not find a solution to the safety control problem, we are almost certain that a solution by any approach does not exist. This result is rarely achieved in finite state abstraction based control, unless a bisimulation quotient is constructed (see, e.g, [13]). Second, we find policies that do not require feedback, hence implementing the control loop does not require sensing. Third, our method is computationally more efficient.
This paper is organized as follows. We provide the necessary notation in Sec. II and formulate the problem in Sec. III. In Sec. IV, we show how to compute robust control invariant sets and ssequences. In Sec. V, we characterize the long term response of the system to repeated ssequences. In Sec. VI, we explain the underlying assumptions and formalize the notion of almost necessity for the existence of ssequences. Finally, we provide two case studies in Sec. VII.
Ii Preliminaries
We denote the positive orthant of an dimensional space by . For two vectors , we use the following notations:
(1) 
for all . We denote the dimensional vector of all ones by .
Definition 1
Given a vector , the set is defined as:
(2) 
Definition 2
[14] The set is a lowerset if we have .
A graphical illustration of a lowerset is depicted in Figure 1. Note that lowersets can be nonconvex.
Proposition 1
The set of lowersets is closed under union and intersection, i.e. if the sets and are lowersets, then and are also lowersets.
Iii Problem Statement and Approach
Iiia Motivating Application: Urban Traffic Networks
An urban traffic network is usually modeled as a directed graph, where its edges and vertices represent traffic links and junctions, respectively. An example of an urban traffic network is shown in Figure 2. We adopt the discrete time fluidlike vehicular flow model from [6], which is briefly explained in Sec. VIIB. The control input is the set of red/green light decisions at the junctions and the adversarial inputs are the numbers of exogenous vehicles arriving in each link in one time step. An upper bound for the adversarial input of each link is assumed to be known. From a game theoretical view, the aim of the adversary is to congest the network, while the winning condition for the player is to keep the links free of congestion.
Monotonicity in traffic networks indicates that given a fixed sequence of control actions, an increase in the vehicular occupancy of some link leads to subsequent higher or at least equal level of occupancy in the whole network at later times. However, traffic networks are not fully cooperative. It is shown in [9] that under a first in first out (FIFO) rule, monotonicity does not hold at diverging junctions. For instance, consider the flow in links in Figure 2. If the number of vehicles on link is near its capacity, then it limits the vehicular flow from link . On the other hand, under FIFO policy, the flow of the vehicles from link to is also impeded. Consequently, an increase in the occupancy of link may actually decrease the occupancy of link . The authors in [15] studied this phenomenon and showed that traffic networks are mixed monotone, which is a weaker property than monotonicity.
We desire that links do not impede the vehicular flow from their upstream links, i.e. the situation described above never happens. In other words, we desire the traffic network to behave as a cooperative system. The set of states that correspond to cooperative dynamics is called cooperative region, which is straightforward to show that is a lowerset in the state space, i.e. it always favors less amount of vehicles. Therefore, it is practically meaningful to design a control strategy that keeps the traffic dynamics cooperative, which literally means free of congestion. From safety control perspective, the safe set is defined as the cooperative region (or a subset of the cooperative region, as the whole cooperative region might require a large number of equations to characterize). In addition, since the model in [6] is a hybrid system, restriction to this type of safe sets discards a substantial amount of modes that are capturing the noncooperative behavior. As a result, the equations governing the evolution in the safe set (cooperative region) are much simpler than the dynamics of the system in the whole state space. This issue is discussed further in the case study at the end of the paper.
IiiB Problem Formulation
We consider discrete time systems in the form of
(3) 
where is the state, is the adversarial input and is the control input from an admissible set . We assume that the set is a rectangle in the form of:
(4) 
which is a reasonable assumption for many networked systems where the components of the adversarial inputs are stochastically independent. Note that any set can be overapproximated by a . We do not make any restrictive assumptions on . For instance, is an index set in an urban traffic network.
Definition 3
System (3) is cooperative if for all :
(5) 
We assume that system (3) is cooperative. Apart from this property, we do not further restrict the function . In particular, we are interested in hybrid systems. For example, the urban traffic model in [6] is a piecewise affine hybrid system. See Sec. VII or [6] for further details.
Remark 1
In this paper, monotonicity is defined with respect to the state and adversarial inputs, which is different from the definitions in [5], [7] and [8]. In the mentioned works ^{1}^{1}1In [5] only deterministic control systems are considered. , for all :
Such systems are also cooperative with respect to the control inputs. We have relaxed this condition in this paper. We do not even assume that the set is partially ordered.
We wish to restrict the evolution of the state of the system to a userdefined set, which is referred to as safe set in the rest of the paper. We assume that safe sets are lowersets. This is a restrictive assumption that is specifically motivated by the nature of the urban traffic networks and is also closely related to the stabilization of cooperative systems in the first orthant. The problems formulated in [7] and [8] consider a more general form of safe sets that are not necessarily lowersets. In this paper, we consider the following problem:
Problem 1
Given a cooperative system (3) and a lowerset safeset , find a set of initial conditions and a control strategy such that the evolution of the system, for any sequence of admissible adversarial inputs, is confined to for all times.
The solution to the problem above involves computation of a set and a control policy , such that the evolution of the system is restricted to . The set is a robust control invariant set (RCIS), which is formally defined in Sec. IV. We may also find the maximal robust control invariant set (MRCIS), which corresponds to the complete solution to Problem 1. However, finding MRCIS is not always computationally practical. Instead, we focus on a more tractable solution with some possible conservativeness. The main drawback of conservativeness is that if we can not find a RCIS, we can not claim that the MRCIS is nonexistent (empty). We investigate the limitations of our approach in Sec. VI. Informally, we show that if our approach is not able to find a RCIS (a solution to Problem 1), it is very likely that MRCIS is empty (there does not exist a solution to Problem 1).
Iv Robust Controlled Invariant Set
In this section, we explain how to find a RCIS inside the safe set . We begin with the definition of RCIS. Next, we focus on MRCIS and explain its geometrical features and computational limitations. Then the key method of this paper is presented.
Definition 4
Given system (3), the set is RCIS if and only if:
The following statements are well known results (see, e.g., [2]) that are stated without proof.
Proposition 2
If , are RCISs, then is also a RCIS.
Proposition 3
If there exist a RCIS , then there exist a unique MRCIS such that .
Implementing the MRCIS fixed point algorithm for a hybrid system is computationally intensive and is limited to very small systems subject to convex sets (see, e.g., [2] for discussion) . Specifically, computing the robust predecessor involves set projection that is computationally challenging for complex systems. Moreover, finite termination is not guaranteed and early termination does not result in a RCIS (a solution to Problem 1). Instead, we exploit monotonicity to introduce a new approach. The following lemma is the key idea of the paper.
Lemma 1
If there exist and a control sequence such that
(6) 
satisfies the following conditions:

,

,
then the set
(7) 
is a RCIS inside .
We show that for any point in , there exist a control such that for all adversarial inputs, the successor is in . For all . Now we apply . Monotonicity implies . Therefore, . But we know that for all , where follows from condition (2). Therefore, .
A graphical depiction of the assumptions in Lemma 1 is shown in Fig. 3. Lemma 1 motivates the following definition:
Definition 5
An ssequence is a finite length sequence of controls, denoted by:
(8) 
where there exist such that
(9) 
where is the length of the sequence and .
The conditions in the definition above can be formulated as the set of the following constraints:
(10) 
The theorem below immediately follows from Lemma 1.
Theorem 1
If , is a feasible solution to the set of constraints (10), then is an ssequence and the set
(11) 
is a RCIS inside .
We now explain how to use the theorem above and find an . If is fixed, finding a solution for (10) is a feasibility problem. One way to approach this problem is formulating (10) as an SMT (satisfiability modulo theories) problem. There exist powerful SMT solvers that are able to handle nonlinearities in the constraints [16]. An alternative approach is formulating (10) as the constraints of an optimization problem, where the cost function aims to maximize a notion of size for the set . For instance, the following optimization problem:
(12) 
provides a feasible solution to (10) where norm of is maximized. As opposed to the iterative procedure in [2], we are able to find a RCIS for system (3) by solving a single optimization problem.
The dynamics of a large class of systems can be written as mixed integer constraints. In particular, piecewise affine hybrid systems and safe sets that are unions of polyhedra (not necessarily convex) can be encoded using mixed integer linear constraints (see, e.g., [17]). Therefore, the optimization problem above can be written as a mixed integer linear programming (MILP) problem, which is solved using efficient state of the art solvers. If (3) is a linear system and is a polyhedron, then (12) is solved in polynomial time. Otherwise, the time required for solving (12) grows polynomially with respect to the size of system (3) and exponentially with respect to and the number of integer constraints (e.g., the number of modes of the hybrid system).
If the set of constraints (10) is infeasible, one has to change to search for feasibility. Algorithmically, we start from and implement until (10) becomes feasible and a solution to Problem 1 is obtained. Large values of makes finding a feasible solution for (10) impractical. In Sec. VI, we establish a relation for the necessity of the existence of ssequences.
Remark 2
As mentioned earlier, for any feasible solution, we may use (11) to find a RCIS. If multiple feasible solutions are available, we may find the union of all the RCISs provided by (11) to find a larger RCIS. Practically, RCIS are useful as terminal constraints of model predictive controllers (see [2]). Therefore, larger RCISs might be desirable. We do not yet have a proof that by taking the union of all RCISs, in the limit , we are able to get arbitrarily close to the MRCIS.
V Controlled Limit Cycles and Attractive Sets
In the last section, we provided a solution to Problem 1: is the set of initial conditions and the control strategy is based on ssequences. In this section, we characterize the infinite time system response to the repetitions of an ssequence and show its relation to controlled limit cycles and attractive sets.
Lemma 2
Let be the ssequence that corresponds to . Then the trajectory of the following system:
(13) 
converges to a limit cycle, i.e. exists.
It follows from the definition of ssequences that . Monotonicity implies:
(14) 
By continuing the argument above we draw the conclusion that:
(15) 
Therefore, each vector component of the following sequence is nonincreasing:
(16) 
and it is already known that is lower bounded (by the origin). As a result, it follows from the cooperative convergence theorem [18] that the limit exists. We denote:
(17) 
As a result, and the trajectory of (13) converges to .
We introduce the following repetitive sequence:
(18) 
The sequence above is basically the control strategy. Its applicability solely requires the initial condition to be in (it is straightforward to see from the proof of Lemma 1 that is reachable from any point in ). In other words, our solution to the control strategy in Problem 1 is unexpectedly a simple policy that does not require state feedback.
Theorem 2
(sketch) Let and , represent the trajectories of and , respectively. Monotonicity indicates that:
As , the right hand side approaches . Therefore, all the left hand side values also finally reach and remain there forever.
Vi Necessity of existence of ssequences
In the last sections, we showed that the existence of ssequences is sufficient for providing a solution to Problem 1. In this section we provide a fundamental result on the necessity conditions for the existence of ssequences. We show that, under some assumptions, the existence of ssequences is almost necessary.
Assumption 1
The safe set is bounded.
Assumption 2
(Strict monotonicity with respect to the adversarial inputs) There exist such that for all and such that:
(20) 
where is a ndimensional vector of all ones and , the following relation holds:
(21) 
We now use the assumptions above to provide the key idea of this section.
Lemma 3
Consider a uniform grid over the set with cube cells of length . The number of cells is proportional to , so we let , where depends on the shape of . Now consider a safe trajectory for system such that the trajectory does not meet the conditions in Lemma 1. By the virtue of the pigeonhole principle, after points obtained from the trajectory, there exist a cell that contains at least two points. In other words, without loss of generality, by redefining as the earlier point in the cell, there exist such that
(23) 
If the same control sequence, , is applied to the system , , it follows from Assumption 2 that
(24) 
By comparing (23) and (24), we obtain that , which indicates that is an ssequence for system (3) where and the following bound is obtained:
Theorem 3
The theorem above addresses the concern of searching for very long ssequences. Starting from and ending at some that is beyond our computational resources, without having an ssequence found, we know that the existence of a solution to Problem 1 is highly unlikely. Informally, such a policy, if exists, is fragile, in the sense that, a slight increase in the adversarial inputs makes the policy invalid.
We conclude this section by mentioning that the results of this section are still theoretical and preliminary. We did not explain how to determine for a cooperative system. Furthermore, the approach based on the number of cells in a uniform grid may lead to very wide bounds in Theorem 3 that seem conservative for practical use.
Vii Case studies
In this section, we provide two case studies. The first case study is an academic example in two dimensions hence it is convenient to graphically illustrate the results. The second case study is of practical interest, where we apply our methods to the urban traffic network shown in Fig. 2.
Viia Case Study 1: Twomode planar hybrid system
Consider (3) to be the following system in :
where , , , and
The system above represents a twomode hybrid (switched) system with additive disturbances where the control input set is . Note that if is fixed, trajectories grow unbounded. We wish to find a control policy that restricts the evolution of the system to the safe set
which is a triangular lowerset. We encode the system above as the set of the following mixedinteger constraints:
where is a sufficiently large number ( in our implementation). We setup the optimization problem (12) as a MILP.
Results
Using the Gurobi MILP solver [19], we find that the smallest that renders the MILP feasible is . The solution is found almost instantly on a personal computer. The following ssequence is obtained:
which corresponds to , . We find the RCIS using (11). As explained in Sec. V, by applying the control sequence to , we arrive at the limit cycle , where . The attractive set is found using (19). We also simulate a trajectory of system . The values of are drawn from a uniform distribution over . The results are illustrated in Fig. 4.
ViiB Case study 2: Urban traffic network
First, we explain the details of the model in [6]. Let and represent the set of links and junctions, respectively. Link is characterized by its tail junction and head junction , where indicates that link is an entry link to the network. We say that link is a downstream link for if . Similarly, link is an upstream link for . For simplicity, we consider networks in which all links are either in northsouth () or eastwest () directions. We denote the direction of link by . The traffic light at junction is denoted by . The control input is a dimensional tuple representing all the traffic lights in the network. The state is , where and is the number of vehicles on link . The number of vehicles that flow out of link in one time step, denoted by , is:
(25) 
where is the maximum outflow of vehicles from in one time step and is the supply available from downstream link to . The FIFObased model for supply is , where is the capacity ratio of dedicated to , is the ratio of flow turning from to and is the vehicular capacity of link . As mentioned in Sec. III, monotonicity does not hold when supply limits the flow at diverging junctions. Therefore, by restricting the state to the following rectangular safe set:
(26) 
where , we ensure that is never the minimizer in (25). As a result, (25) becomes:
(27) 
The discrete time evolution of is given by:
(28) 
where is the adversarial input corresponding to link . It is straightforward to check that , , and . Therefore, the evolution of each state component is cooperative with respect to the state and adversarial inputs. Finally, in a compact form, the evolution can be written in the form (3). We wish to find a control policy for the urban traffic network shown in Fig. 2 such that the state is always in . The network parameters are given in Table I.
, , 
, 
, , 
, 
, , 
, 
, , , 
, 
Results
We formulate (12) as a MILP. The smallest for which an ssequence is found is . The time required to solve the MILP using Gurobi is 79 seconds on a 3GHz Core i7 MacBook Pro. In comparison to finite statebased safety game implemented in[20], a problem of this size (12 links, 6 junctions) is intractable, unless a very coarse partitioning of the state space is considered.
Table II shows the traffic light at each junction for each control input in . We also find that:
We obtain a RCIS and an attractive set that lie in . As explained in Sec. VI, we can simulate the system to obtain the limit cycle, which is illustrated in Fig. 5. A trajectory of the system starting from with chosen from a uniform distribution over is also shown in Fig. 6. Note that all the components of the trajectory in Fig. 6 are upper bounded by their corresponding values in the trajectory in Fig. 5.
junction  
References
 [1] F. Blanchini, “Set invariance in controlâa survey,” Automatica, vol. 35, no. 11, pp. 1747–1767, 1999.
 [2] E. C. Kerrigan, “Robust Constraint Satisfaction: Invariant Sets and Predictive Control,” Ph.D. dissertation, University of Cambridge, 2000.
 [3] S. V. Raković, P. Grieder, M. Kvasnica, D. Q. Mayne, and M. Morari, “Computation of invariant sets for piecewise affine discrete time systems subject to bounded disturbances,” in Decision and Control, 2004. CDC. 43rd IEEE Conference on, vol. 2. IEEE, 2004, pp. 1418–1423.
 [4] H. Smith, Monotone dynamical systems: an introduction to the theory of competitive and cooperative systems. American Mathematical Soc., 2008, no. 41.
 [5] D. Angeli and E. D. Sontag, “Monotone control systems,” IEEE Transactions on Automatic Control, vol. 48, no. 10, pp. 1684–1698, 2003.
 [6] S. Coogan, E. A. Gol, M. Arcak, and C. Belta, “Controlling a network of signalized intersections from temporal logical specifications,” in American Control Conference (ACC), 2015. IEEE, 2015, pp. 3919–3924.
 [7] R. Ghaemi and D. Del Vecchio, “Safety control of piecewise continuous order preserving systems,” in Proceedings of the IEEE Conference on Decision and Control. IEEE, 2011, pp. 545–551.
 [8] P.J. Meyer, A. Girard, and E. Witrant, “Safety control with performance guarantees of cooperative systems using compositional abstractions,” in 5th IFAC Conference on Analysis and Design of Hybrid Systems (ADHS),, Atlanta, GA, 2015.
 [9] S. Coogan and M. Arcak, “Dynamical properties of a compartmental model for traffic networks,” in 2014 American Control Conference, 2014, pp. 2511–2516.
 [10] E. Lovisari, G. Como, and K. Savla, “Stability of monotone dynamical flow networks,” in Decision and Control (CDC), 2014 IEEE 53rd Annual Conference on. IEEE, 2014, pp. 2384–2389.
 [11] S. Coogan and M. Arcak, “Efficient finite abstraction of mixed monotone systems,” in Proceedings of the 18th International Conference …. ACM, 2015, pp. 58–67. [Online]. Available: http://dl.acm.org/citation.cfm?id=2728607
 [12] C. Baier, J.P. Katoen, and Others, Principles of model checking. MIT press Cambridge, 2008, vol. 26202649.
 [13] P. Tabuada, Verification and Control of Hybrid Systems . Springer Science & Business Media, 2008.
 [14] E. S. Kim, M. Arcak, and S. A. Seshia, “Directed Specifications and Assumption Mining for Monotone Dynamical Systems,” in 19th ACM International Conference on Hybrid Systems: Computation and Control (HSCC), Vienna, Austria, 2016.
 [15] S. Coogan, M. Arcak, and A. a. Kurzhanskiy, “On the Mixed Monotonicity of FIFO Traffic Flow Models,” arXiv preprint arXiv:1511.05081, 2015. [Online]. Available: http://arxiv.org/abs/1511.05081
 [16] S. Gao, S. Kong, and E. M. Clarke, “dReal : An SMT Solver for Nonlinear Theories over the Reals,” in Automated Deduction–CADE24. Springer, 2013, no. 1041377, pp. 208–214.
 [17] A. Bemporad and M. Morari, “Control of systems integrating logic, dynamics, and constraints,” Automatica, vol. 35, no. 3, pp. 407–427, 1999.
 [18] J. Yeh, Real analysis: theory of measure and integration. World Scientific, 2006.
 [19] G. O. Inc., “Gurobi Optimizer reference manual,” p. 572, 2014.
 [20] S. Sadraddini and C. Belta, “A Provably Correct MPC Approach to Safety Control of Urban Traffic Networks,” arXiv preprint arXiv:1602.01028, 2016.