Safety Control of Positive Monotone Systems with Bounded Uncertainties
Monotone systems are prevalent in models of engineering applications such as transportation and biological networks. In this paper, we investigate the problem of finding a control strategy for a discrete time positive monotone system with bounded uncertainties such that the evolution of the system is guaranteed to be confined to a safe set in the state space for all times. By exploiting monotonicity, we propose an approach to this problem which is based on constraint programming. We find control strategies that are based on repetitions of finite sequences of control actions. We show that, under assumptions made in the paper, safety control of cooperative systems does not require state measurement. We demonstrate the results on a signalized urban traffic network, where the safety objective is to keep the traffic flow free of congestion.
Designing control policies subject to safety constraints is a fundamental problem in the automation of complex systems. From a game theoretic perspective, the safety control problem, also known as safety game, is the problem of finding a control policy that guarantees that the evolution of the system is restricted to a safe region in the state space, regardless of the actions taken by the adversary. The solution to this problem involves finding a robust control invariant set . Iterative computation of robust control invariant sets has been extensively studied for linear and piecewise affine systems , where intensive polyhedral operations are required to carry out set iterations.
In this work, we focus on a special class of systems that are monotone, or order preserving, and provide an alternative computational approach to the safety control problem. cooperative systems are common in models of biological, socio-economical and transportation networks. Monotonicity, in general, is a mathematical property that indicates a type of order preserving law. Monotone autonomous systems are thoroughly studied in . In , the authors introduced cooperative control systems and provided results on steady state responses and stability.
We consider discrete time uncertain control systems that are monotone with respect to positive orthant in the state and adversarial inputs space. In contrast to , we do not assume monotonicity with respect to controls. We do not even require the control space to be partially ordered. On the other hand, we assume a more restrictive form of the safety region in the problem formulation. Our consideration of such systems and specifications is motivated by the dynamics of urban traffic networks , which are described in more detail later in the paper. The key result of this work is to show that computing robust control invariant sets maps to computing finite sequences of control actions, which we call s-sequences. We show that repeated executions of s-sequences are safe control policies that do not require state feedback. We also show that, under some mild assumptions, the existence of s-sequences is almost necessary. To the best of our knowledge, these fundamental insights were not established before.
Safety control of monotone systems has also been considered in  and . However, in these papers, monotonicity with respect to the controls was also assumed. Therefore, the results of this paper are more general in this respect. Set-invariance theories are also closely related to stability analysis. In ,  and , the authors studied the stability of monotone and mixed monotone deterministic systems with no control inputs. Extending these results to cooperative systems with partially ordered adversarial inputs is relatively straightforward, but it is not so obvious for systems with control inputs, specifically for discontinuous control admissible sets.
This work is also related to finite state abstraction based control of (mixed) monotone systems . This approach enables control synthesis from rich temporal logic  specifications, of which safety is a special yet important class. However, discretization of the state space is computationally expensive and its complexity grows exponentially with respect to the size of the system. Furthermore, with particular focus on safety specifications of the form assumed in this paper, our results are stronger in the following ways. First, if our approach does not find a solution to the safety control problem, we are almost certain that a solution by any approach does not exist. This result is rarely achieved in finite state abstraction based control, unless a bisimulation quotient is constructed (see, e.g, ). Second, we find policies that do not require feedback, hence implementing the control loop does not require sensing. Third, our method is computationally more efficient.
This paper is organized as follows. We provide the necessary notation in Sec. II and formulate the problem in Sec. III. In Sec. IV, we show how to compute robust control invariant sets and s-sequences. In Sec. V, we characterize the long term response of the system to repeated s-sequences. In Sec. VI, we explain the underlying assumptions and formalize the notion of almost necessity for the existence of s-sequences. Finally, we provide two case studies in Sec. VII.
We denote the positive orthant of an -dimensional space by . For two vectors , we use the following notations:
for all . We denote the -dimensional vector of all ones by .
Given a vector , the set is defined as:
 The set is a lower-set if we have .
A graphical illustration of a lower-set is depicted in Figure 1. Note that lower-sets can be non-convex.
The set of lower-sets is closed under union and intersection, i.e. if the sets and are lower-sets, then and are also lower-sets.
Iii Problem Statement and Approach
Iii-a Motivating Application: Urban Traffic Networks
An urban traffic network is usually modeled as a directed graph, where its edges and vertices represent traffic links and junctions, respectively. An example of an urban traffic network is shown in Figure 2. We adopt the discrete time fluid-like vehicular flow model from , which is briefly explained in Sec. VII-B. The control input is the set of red/green light decisions at the junctions and the adversarial inputs are the numbers of exogenous vehicles arriving in each link in one time step. An upper bound for the adversarial input of each link is assumed to be known. From a game theoretical view, the aim of the adversary is to congest the network, while the winning condition for the player is to keep the links free of congestion.
Monotonicity in traffic networks indicates that given a fixed sequence of control actions, an increase in the vehicular occupancy of some link leads to subsequent higher or at least equal level of occupancy in the whole network at later times. However, traffic networks are not fully cooperative. It is shown in  that under a first in first out (FIFO) rule, monotonicity does not hold at diverging junctions. For instance, consider the flow in links in Figure 2. If the number of vehicles on link is near its capacity, then it limits the vehicular flow from link . On the other hand, under FIFO policy, the flow of the vehicles from link to is also impeded. Consequently, an increase in the occupancy of link may actually decrease the occupancy of link . The authors in  studied this phenomenon and showed that traffic networks are mixed monotone, which is a weaker property than monotonicity.
We desire that links do not impede the vehicular flow from their upstream links, i.e. the situation described above never happens. In other words, we desire the traffic network to behave as a cooperative system. The set of states that correspond to cooperative dynamics is called cooperative region, which is straightforward to show that is a lower-set in the state space, i.e. it always favors less amount of vehicles. Therefore, it is practically meaningful to design a control strategy that keeps the traffic dynamics cooperative, which literally means free of congestion. From safety control perspective, the safe set is defined as the cooperative region (or a subset of the cooperative region, as the whole cooperative region might require a large number of equations to characterize). In addition, since the model in  is a hybrid system, restriction to this type of safe sets discards a substantial amount of modes that are capturing the non-cooperative behavior. As a result, the equations governing the evolution in the safe set (cooperative region) are much simpler than the dynamics of the system in the whole state space. This issue is discussed further in the case study at the end of the paper.
Iii-B Problem Formulation
We consider discrete time systems in the form of
where is the state, is the adversarial input and is the control input from an admissible set . We assume that the set is a rectangle in the form of:
which is a reasonable assumption for many networked systems where the components of the adversarial inputs are stochastically independent. Note that any set can be over-approximated by a . We do not make any restrictive assumptions on . For instance, is an index set in an urban traffic network.
System (3) is cooperative if for all :
We assume that system (3) is cooperative. Apart from this property, we do not further restrict the function . In particular, we are interested in hybrid systems. For example, the urban traffic model in  is a piece-wise affine hybrid system. See Sec. VII or  for further details.
In this paper, monotonicity is defined with respect to the state and adversarial inputs, which is different from the definitions in ,  and . In the mentioned works 111In  only deterministic control systems are considered. , for all :
Such systems are also cooperative with respect to the control inputs. We have relaxed this condition in this paper. We do not even assume that the set is partially ordered.
We wish to restrict the evolution of the state of the system to a user-defined set, which is referred to as safe set in the rest of the paper. We assume that safe sets are lower-sets. This is a restrictive assumption that is specifically motivated by the nature of the urban traffic networks and is also closely related to the stabilization of cooperative systems in the first orthant. The problems formulated in  and  consider a more general form of safe sets that are not necessarily lower-sets. In this paper, we consider the following problem:
Given a cooperative system (3) and a lower-set safe-set , find a set of initial conditions and a control strategy such that the evolution of the system, for any sequence of admissible adversarial inputs, is confined to for all times.
The solution to the problem above involves computation of a set and a control policy , such that the evolution of the system is restricted to . The set is a robust control invariant set (RCIS), which is formally defined in Sec. IV. We may also find the maximal robust control invariant set (MRCIS), which corresponds to the complete solution to Problem 1. However, finding MRCIS is not always computationally practical. Instead, we focus on a more tractable solution with some possible conservativeness. The main drawback of conservativeness is that if we can not find a RCIS, we can not claim that the MRCIS is non-existent (empty). We investigate the limitations of our approach in Sec. VI. Informally, we show that if our approach is not able to find a RCIS (a solution to Problem 1), it is very likely that MRCIS is empty (there does not exist a solution to Problem 1).
Iv Robust Controlled Invariant Set
In this section, we explain how to find a RCIS inside the safe set . We begin with the definition of RCIS. Next, we focus on MRCIS and explain its geometrical features and computational limitations. Then the key method of this paper is presented.
Given system (3), the set is RCIS if and only if:
The following statements are well known results (see, e.g., ) that are stated without proof.
If , are RCISs, then is also a RCIS.
If there exist a RCIS , then there exist a unique MRCIS such that .
Implementing the MRCIS fixed point algorithm for a hybrid system is computationally intensive and is limited to very small systems subject to convex sets (see, e.g.,  for discussion) . Specifically, computing the robust predecessor involves set projection that is computationally challenging for complex systems. Moreover, finite termination is not guaranteed and early termination does not result in a RCIS (a solution to Problem 1). Instead, we exploit monotonicity to introduce a new approach. The following lemma is the key idea of the paper.
If there exist and a control sequence such that
satisfies the following conditions:
then the set
is a RCIS inside .
We show that for any point in , there exist a control such that for all adversarial inputs, the successor is in . For all . Now we apply . Monotonicity implies . Therefore, . But we know that for all , where follows from condition (2). Therefore, .
An s-sequence is a finite length sequence of controls, denoted by:
where there exist such that
where is the length of the sequence and .
The conditions in the definition above can be formulated as the set of the following constraints:
The theorem below immediately follows from Lemma 1.
If , is a feasible solution to the set of constraints (10), then is an s-sequence and the set
is a RCIS inside .
We now explain how to use the theorem above and find an . If is fixed, finding a solution for (10) is a feasibility problem. One way to approach this problem is formulating (10) as an SMT (satisfiability modulo theories) problem. There exist powerful SMT solvers that are able to handle nonlinearities in the constraints . An alternative approach is formulating (10) as the constraints of an optimization problem, where the cost function aims to maximize a notion of size for the set . For instance, the following optimization problem:
The dynamics of a large class of systems can be written as mixed integer constraints. In particular, piecewise affine hybrid systems and safe sets that are unions of polyhedra (not necessarily convex) can be encoded using mixed integer linear constraints (see, e.g., ). Therefore, the optimization problem above can be written as a mixed integer linear programming (MILP) problem, which is solved using efficient state of the art solvers. If (3) is a linear system and is a polyhedron, then (12) is solved in polynomial time. Otherwise, the time required for solving (12) grows polynomially with respect to the size of system (3) and exponentially with respect to and the number of integer constraints (e.g., the number of modes of the hybrid system).
If the set of constraints (10) is infeasible, one has to change to search for feasibility. Algorithmically, we start from and implement until (10) becomes feasible and a solution to Problem 1 is obtained. Large values of makes finding a feasible solution for (10) impractical. In Sec. VI, we establish a relation for the necessity of the existence of s-sequences.
As mentioned earlier, for any feasible solution, we may use (11) to find a RCIS. If multiple feasible solutions are available, we may find the union of all the RCISs provided by (11) to find a larger RCIS. Practically, RCIS are useful as terminal constraints of model predictive controllers (see ). Therefore, larger RCISs might be desirable. We do not yet have a proof that by taking the union of all RCISs, in the limit , we are able to get arbitrarily close to the MRCIS.
V Controlled Limit Cycles and Attractive Sets
In the last section, we provided a solution to Problem 1: is the set of initial conditions and the control strategy is based on s-sequences. In this section, we characterize the infinite time system response to the repetitions of an s-sequence and show its relation to controlled limit cycles and attractive sets.
Let be the s-sequence that corresponds to . Then the trajectory of the following system:
converges to a limit cycle, i.e. exists.
It follows from the definition of s-sequences that . Monotonicity implies:
By continuing the argument above we draw the conclusion that:
Therefore, each vector component of the following sequence is non-increasing:
and it is already known that is lower bounded (by the origin). As a result, it follows from the cooperative convergence theorem  that the limit exists. We denote:
As a result, and the trajectory of (13) converges to .
We introduce the following repetitive sequence:
The sequence above is basically the control strategy. Its applicability solely requires the initial condition to be in (it is straightforward to see from the proof of Lemma 1 that is reachable from any point in ). In other words, our solution to the control strategy in Problem 1 is unexpectedly a simple policy that does not require state feedback.
(sketch) Let and , represent the trajectories of and , respectively. Monotonicity indicates that:
As , the right hand side approaches . Therefore, all the left hand side values also finally reach and remain there forever.
Vi Necessity of existence of s-sequences
In the last sections, we showed that the existence of s-sequences is sufficient for providing a solution to Problem 1. In this section we provide a fundamental result on the necessity conditions for the existence of s-sequences. We show that, under some assumptions, the existence of s-sequences is almost necessary.
The safe set is bounded.
(Strict monotonicity with respect to the adversarial inputs) There exist such that for all and such that:
where is a n-dimensional vector of all ones and , the following relation holds:
We now use the assumptions above to provide the key idea of this section.
Consider a uniform grid over the set with cube cells of length . The number of cells is proportional to , so we let , where depends on the shape of . Now consider a safe trajectory for system such that the trajectory does not meet the conditions in Lemma 1. By the virtue of the pigeonhole principle, after points obtained from the trajectory, there exist a cell that contains at least two points. In other words, without loss of generality, by redefining as the earlier point in the cell, there exist such that
If the same control sequence, , is applied to the system , , it follows from Assumption 2 that
where and are independent constants.
The theorem above addresses the concern of searching for very long s-sequences. Starting from and ending at some that is beyond our computational resources, without having an s-sequence found, we know that the existence of a solution to Problem 1 is highly unlikely. Informally, such a policy, if exists, is fragile, in the sense that, a slight increase in the adversarial inputs makes the policy invalid.
We conclude this section by mentioning that the results of this section are still theoretical and preliminary. We did not explain how to determine for a cooperative system. Furthermore, the approach based on the number of cells in a uniform grid may lead to very wide bounds in Theorem 3 that seem conservative for practical use.
Vii Case studies
In this section, we provide two case studies. The first case study is an academic example in two dimensions hence it is convenient to graphically illustrate the results. The second case study is of practical interest, where we apply our methods to the urban traffic network shown in Fig. 2.
Vii-a Case Study 1: Two-mode planar hybrid system
Consider (3) to be the following system in :
where , , , and
The system above represents a two-mode hybrid (switched) system with additive disturbances where the control input set is . Note that if is fixed, trajectories grow unbounded. We wish to find a control policy that restricts the evolution of the system to the safe set
which is a triangular lower-set. We encode the system above as the set of the following mixed-integer constraints:
where is a sufficiently large number ( in our implementation). We setup the optimization problem (12) as a MILP.
Using the Gurobi MILP solver , we find that the smallest that renders the MILP feasible is . The solution is found almost instantly on a personal computer. The following s-sequence is obtained:
which corresponds to , . We find the RCIS using (11). As explained in Sec. V, by applying the control sequence to , we arrive at the limit cycle , where . The attractive set is found using (19). We also simulate a trajectory of system . The values of are drawn from a uniform distribution over . The results are illustrated in Fig. 4.
Vii-B Case study 2: Urban traffic network
First, we explain the details of the model in . Let and represent the set of links and junctions, respectively. Link is characterized by its tail junction and head junction , where indicates that link is an entry link to the network. We say that link is a downstream link for if . Similarly, link is an upstream link for . For simplicity, we consider networks in which all links are either in north-south () or east-west () directions. We denote the direction of link by . The traffic light at junction is denoted by . The control input is a dimensional tuple representing all the traffic lights in the network. The state is , where and is the number of vehicles on link . The number of vehicles that flow out of link in one time step, denoted by , is:
where is the maximum outflow of vehicles from in one time step and is the supply available from downstream link to . The FIFO-based model for supply is , where is the capacity ratio of dedicated to , is the ratio of flow turning from to and is the vehicular capacity of link . As mentioned in Sec. III, monotonicity does not hold when supply limits the flow at diverging junctions. Therefore, by restricting the state to the following rectangular safe set:
The discrete time evolution of is given by:
where is the adversarial input corresponding to link . It is straightforward to check that , , and . Therefore, the evolution of each state component is cooperative with respect to the state and adversarial inputs. Finally, in a compact form, the evolution can be written in the form (3). We wish to find a control policy for the urban traffic network shown in Fig. 2 such that the state is always in . The network parameters are given in Table I.
|, , ,|
We formulate (12) as a MILP. The smallest for which an s-sequence is found is . The time required to solve the MILP using Gurobi is 79 seconds on a 3GHz Core i7 MacBook Pro. In comparison to finite state-based safety game implemented in, a problem of this size (12 links, 6 junctions) is intractable, unless a very coarse partitioning of the state space is considered.
Table II shows the traffic light at each junction for each control input in . We also find that:
We obtain a RCIS and an attractive set that lie in . As explained in Sec. VI, we can simulate the system to obtain the limit cycle, which is illustrated in Fig. 5. A trajectory of the system starting from with chosen from a uniform distribution over is also shown in Fig. 6. Note that all the components of the trajectory in Fig. 6 are upper bounded by their corresponding values in the trajectory in Fig. 5.
-  F. Blanchini, “Set invariance in controlâa survey,” Automatica, vol. 35, no. 11, pp. 1747–1767, 1999.
-  E. C. Kerrigan, “Robust Constraint Satisfaction: Invariant Sets and Predictive Control,” Ph.D. dissertation, University of Cambridge, 2000.
-  S. V. Raković, P. Grieder, M. Kvasnica, D. Q. Mayne, and M. Morari, “Computation of invariant sets for piecewise affine discrete time systems subject to bounded disturbances,” in Decision and Control, 2004. CDC. 43rd IEEE Conference on, vol. 2. IEEE, 2004, pp. 1418–1423.
-  H. Smith, Monotone dynamical systems: an introduction to the theory of competitive and cooperative systems. American Mathematical Soc., 2008, no. 41.
-  D. Angeli and E. D. Sontag, “Monotone control systems,” IEEE Transactions on Automatic Control, vol. 48, no. 10, pp. 1684–1698, 2003.
-  S. Coogan, E. A. Gol, M. Arcak, and C. Belta, “Controlling a network of signalized intersections from temporal logical specifications,” in American Control Conference (ACC), 2015. IEEE, 2015, pp. 3919–3924.
-  R. Ghaemi and D. Del Vecchio, “Safety control of piece-wise continuous order preserving systems,” in Proceedings of the IEEE Conference on Decision and Control. IEEE, 2011, pp. 545–551.
-  P.-J. Meyer, A. Girard, and E. Witrant, “Safety control with performance guarantees of cooperative systems using compositional abstractions,” in 5th IFAC Conference on Analysis and Design of Hybrid Systems (ADHS),, Atlanta, GA, 2015.
-  S. Coogan and M. Arcak, “Dynamical properties of a compartmental model for traffic networks,” in 2014 American Control Conference, 2014, pp. 2511–2516.
-  E. Lovisari, G. Como, and K. Savla, “Stability of monotone dynamical flow networks,” in Decision and Control (CDC), 2014 IEEE 53rd Annual Conference on. IEEE, 2014, pp. 2384–2389.
-  S. Coogan and M. Arcak, “Efficient finite abstraction of mixed monotone systems,” in Proceedings of the 18th International Conference …. ACM, 2015, pp. 58–67. [Online]. Available: http://dl.acm.org/citation.cfm?id=2728607
-  C. Baier, J.-P. Katoen, and Others, Principles of model checking. MIT press Cambridge, 2008, vol. 26202649.
-  P. Tabuada, Verification and Control of Hybrid Systems . Springer Science & Business Media, 2008.
-  E. S. Kim, M. Arcak, and S. A. Seshia, “Directed Specifications and Assumption Mining for Monotone Dynamical Systems,” in 19th ACM International Conference on Hybrid Systems: Computation and Control (HSCC), Vienna, Austria, 2016.
-  S. Coogan, M. Arcak, and A. a. Kurzhanskiy, “On the Mixed Monotonicity of FIFO Traffic Flow Models,” arXiv preprint arXiv:1511.05081, 2015. [Online]. Available: http://arxiv.org/abs/1511.05081
-  S. Gao, S. Kong, and E. M. Clarke, “dReal : An SMT Solver for Nonlinear Theories over the Reals,” in Automated Deduction–CADE-24. Springer, 2013, no. 1041377, pp. 208–214.
-  A. Bemporad and M. Morari, “Control of systems integrating logic, dynamics, and constraints,” Automatica, vol. 35, no. 3, pp. 407–427, 1999.
-  J. Yeh, Real analysis: theory of measure and integration. World Scientific, 2006.
-  G. O. Inc., “Gurobi Optimizer reference manual,” p. 572, 2014.
-  S. Sadraddini and C. Belta, “A Provably Correct MPC Approach to Safety Control of Urban Traffic Networks,” arXiv preprint arXiv:1602.01028, 2016.