# Fast Reachable Set Approximations via State Decoupling Disturbances

###### Abstract

With the recent surge of interest in using robotics and automation for civil purposes, providing safety and performance guarantees has become extremely important. In the past, differential games have been successfully used for the analysis of safety-critical systems. In particular, the Hamilton-Jacobi (HJ) formulation of differential games provides a flexible way to compute the reachable set, which can characterize the set of states which lead to either desirable or undesirable configurations, depending on the application. While HJ reachability is applicable to many small practical systems, the curse of dimensionality prevents the direct application of HJ reachability to many larger systems. To address computation complexity issues, various efficient computation methods in the literature have been developed for approximating or exactly computing the solution to HJ partial differential equations, but only when the system dynamics are of specific forms. In this paper, we propose a flexible method to trade off optimality with computation complexity in HJ reachability analysis. To achieve this, we propose to simplify system dynamics by treating state variables as disturbances. We prove that the resulting approximation is conservative in the desired direction, and demonstrate our method using a four-dimensional plane model.

## I Introduction

Optimal control problems and differential games are important and powerful theoretical tools for analyzing a wide variety of systems, particularly in safety-critical scenarios. They have been extensively studied in the past several decades [1, 2, 3, 4, 5, 6, 7], and have been successfully applied to practical problems such as pairwise collision avoidance [5], aircraft in-flight refueling [8], vehicle platooning [9], and many others [10, 11]. With the recent growing interest in using safety-critical autonomous systems such as autonomous cars and unmanned aerial vehicles for civil purposes [12, 13, 14, 15, 16, 17], the importance and necessity of having flexible tools that can provide safety guarantees have substantially increased.

Intuitively, in an optimal control problem, one seeks to find the cheapest way a system described by an ordinary differential equation (ODE) model can perform a certain task. In a differential game, a system is controlled by two adversarial agents competing to respectively minimize and maximize a joint cost function. Hamilton-Jacobi (HJ) reachability is a common and effective way to analyze both optimal control problems and differential games because of the guarantees that it provides and its flexibility with respect to the system dynamics.

In a reachability problem, one is given some system dynamics described by an ODE, and a target set which describes the set of final conditions under consideration. Depending on the application, the target set can represent either a set of desired or undesired states. The goal in reachability analysis is to compute the backward reachable set (BRS). When the target set is a set of desired states, the BRS represents the set of states from which the system can be guaranteed to be driven to the target set, despite the worst case disturbance. In contrast, when the target set is a set of undesired states, the BRS represents the set of states from which the system may be driven into the target set under some disturbance, despite its best control efforts to remain outside. Because of the theoretical guarantees that reachability analysis provides, it is ideal for analyzing the newest problems involving autonomous systems.

Despite these advantages, there are serious drawbacks to using HJ Reachability on large systems. In order to compute the BRS, an HJ partial differential equation (PDE) must be solved on a grid representing a discretization of the state space. This means that the complexity of computing a BRS grows exponentially with the number of system states, making the standard HJ reachability formulation intractable for systems higher than approximately five dimensions. To address this difficulty, a number of approximation techniques have been developed, such as those involving projections, approximate dynamic programming, and occupation measure theory [18, 19, 20].

In this paper, we propose a general method to remove coupling in systems by treating coupling variables as disturbances. This uncoupling of dynamics transforms the system into a form that is suitable for analysis using methods such as [21] and [22], which exploit system structure. This method can also be combined with previous work such as [18, 19, 20]. to reduce computation complexity even further. We show that our approach results in BRSs that are conservative in the desired direction, and demonstrate the performance of our method when combined with the decoupled formulation in [22].

## Ii Problem Formulation

Consider a differential game between two players described by the time-invariant system with state evolving according to the ODE

(1) | ||||

where is the control of Player 1, and is the control of Player 2. Often, and are regarded as the control and disturbance, respectively, of a system described by (1). We assume is uniformly continuous, bounded, and Lipschitz continuous in for fixed , and the control functions are drawn from the set of measurable functions^{1}^{1}1A function between two measurable spaces and is said to be measurable if the preimage of a measurable set in is a measurable set in , that is: , with -algebras on ,.. As in [1, 2, 5], we allow Player 2 to only use nonanticipative strategies , defined by

(2) | ||||

Suppose that the state can be written as such that the control and disturbance can be written as , and such a decomposition of the control leads to the following form of system dynamics:

(3) | ||||

where , and are components of the system dynamics that involve , respectively. Note that this assumption on and is a mild one, and is satisfied by any system in which each of the control components and disturbance components have independent control sets and disturbance sets , respectively; note that we can also write , and . This decomposition is very common in real world systems, where control input bounds such as maximum acceleration and maximum turn rate are independent of each other.

For the system in the form of (3), we would like to compute the BRS of time horizon , denoted . Intuitively, is the set of states from which there exists a control strategy to drive the system into a target set within a duration of despite worst-case disturbances. Formally, the BRS is defined as^{2}^{2}2Similar definitions of BRSs and their relationships can be found in, for example, [23]

(4) | ||||

Standard HJ formulations exist for computing the BRS in the full dimensionality [3, 5, 6, 7]. In addition, special HJ formulations can be used to substantially reduce computation complexity for systems with special properties such as having terminal integrators or having fully decoupled dynamics [21, 22]. The goal of this paper will be to demonstrate how to take advantage of previous work on BRS computation for systems of particular forms, even when the actual system dynamics do not exactly satisfy necessary assumptions. For concreteness, in this paper we will focus on removing coupling to put systems into a fully decoupled form that satisfies the assumptions in [22].

Our proposed approach computes an approximation of the BRS in dimension instead of in dimension , dramatically reducing computation complexity. This is done by removing coupling in the dynamics by treating certain variables as disturbances. The computed approximation is conservative in the desired direction, meaning any state in the approximate BRS is also in the true BRS.

## Iii Background

Given the system in (1), the BRS defined in (4) can be computed using the following terminal value HJ PDE:

(5) | ||||

In (5), the target set is represented as the sub-zero level set of the function : . The BRS is represented as the viscosity solution of (5) : [5, 24]. The definition given in (4) assumes that the control tries to grow the BRS as much as possible, and the disturbance tries to do the opposite. Similar definitions of the BRS, for example one in which the control tries to inhibit its growth, can be computed by adjusting the minimum and maximum. For simplicity, we will assume that the BRS is defined by , and thus determined by solving given in (5).

In the case that the system is fully decoupled in the form of (6), [22] provides a method to exactly compute . The computation is done in each of the decoupled components, substantially reducing computation complexity as long as the system is decoupled and as long as the target can be written as the intersection .

(6) | ||||

The reason for the dimension reduction is intuitive, and can be easily verified by checking that the following statements are equivalent:

## Iv Coupling as Disturbance

The decoupled formulation summarized in Section III enables the exact computation of dimensional BRSs in the space of the dimensional decoupled components. However, this substantial computation benefit can only be gained if the system dynamics satisfy (6).

In the case where the dynamics are in the form of (3), the decoupled formulation cannot be directly used. However, if one treats as a disturbance in the function , and as a disturbance in the function , the system would become decoupled. Mathematically, (3) becomes the following:

(7) |

where represents the full-dimensional domain over which computation is done. By treating the coupled variables as a disturbance, we have uncoupled the original system dynamics (3), and produced approximate dynamics (7) that are decoupled, allowing us to do the computation in the space of each decoupled component.

Compared to the original system dynamics given in (3), the uncoupled dynamics given in (7) experiences a larger disturbance, since the dependence of the function and the dependence on the function are treated as disturbances. With the definition of BRS in (4), the approximate BRS computed using the dynamics (7) is an under-approximation of the true BRS. We formalize this in the proposition below.

###### Proposition 1

Let be the BRSs of the subsystem (7) from the target sets , and let be the BRS of the system (3) from the target set . Then^{3}^{3}3Strictly speaking, would need to be “back projected” into the higher dimensional space before their intersections can be taken, but we will use the abuse of notation for convenience., .

It suffices to show that given any state such that are in the BRSs for the system in (7), respectively, then is in the BRS for the system in (3).

For convenience, we will use to denote , with having the analogous meaning. Applying the definition of BRS in (4) to the subsystems in (7), at the state we have

The above two conditions together imply

(8) | |||

In particular, since , the above is true also when , so

(9) | |||

By treating the coupled states as disturbance, the computation complexity reduces from for the full formulation to for the decoupled approximate system (7).

## V Disturbance Splitting

By treating coupling variables and as disturbances in and , respectively, we introduce conservatism in the BRS computation. This conservatism is always in the desired direction. In situations where and are large, the degree of conservatism can be reduced by splitting the disturbance and into multiple sections, as long as the target set does not depend on the state variables being split. For example, can be split as follows:

(11) | |||

This disturbance splitting results in the following family of approximate system dynamics

(12) |

from which a BRS can be computed in . Since , the uncoupling disturbance is reduced whenever the disturbance is split. In addition, the uncoupling disturbance can also be split into , for a total of total “pieces” of uncoupling disturbances. However, a smaller disturbance bound also restricts the allowable trajectories of each approximate system, so overall it is difficult to determine a priori the optimal way to split the uncoupling disturbances. The trade-off between the size of disturbance bound and degree of restriction placed on trajectories can be seen in Fig. 2.

### V-a Examples of Decoupling System Dynamics

Our proposed method applies to any system of the form (3), as we will demonstrate with the example in Section VI. Systems with light coupling between groups of state variables are particularly suitable for the application or our proposed method. Below are other example systems for which treating the coupling variables in or in as disturbances would lead to decoupling.

Linear systems with large Jordan blocks, for example,

(13) |

If is treated as a disturbance in the equation for , we would obtain the decoupled components and .

Lateral Quadrotor Dynamics near Hover [25]:

(14) |

where is the longitudinal position, is the longitudinal velocity, is the pitch angle, is the pitch rate, and is the desired pitch angle. These dynamics are valid for small pitch angles. The system would become decoupled into the and components if is treated as a disturbance in . In fact, the full ten-dimensional (10D) quadrotor dynamics given in [25] can be decomposed into five decoupled components of 2D systems, allowing an approximation of the 10D BRS to be computed in 2D space.

It is worth noting that after decoupling the 4D system in (14) into two 2D systems, each decoupled component is in the form . This is exactly the form of the dynamics in [21], allowing the 4D BRS to be exactly computed in 1D! In general, removing coupling may bring the system dynamics into a form suitable for analysis using methods that require specific assumptions on the dynamics, potentially greatly reducing computation complexity.

## Vi Numerical Results

We demonstrate our proposed method using a 4D model of an aircraft flying at constant altitude, given by

(15) |

where represent the plane’s position in the and directions, represents the plane’s heading, and represents the plane’s longitudinal velocity. The plane has a limited turn rate and a limited longitudinal acceleration as its control variables. For our example, the computation bounds are

Using the decoupled approximation technique, we create the following decoupled approximation of the system with and as the decoupled components:

(16) |

We define the target set as a square of length 4 m centered at , described by . This can be interpreted as a positional goal centered at the origin that can be achieved for all angles and velocities within the computation grid bounds. From the target set, we define such that . To analyze our newly decoupled system we must likewise decouple the target set by letting be

(17) | ||||

These target sets have corresponding implicit surface functions , which then form the 4D target set represented by .

We set as the target set in our reachability problem and computed the BRS from using both the direct 4D computation as well as the proposed decoupled approximation method.

### Vi-a Backward Reachable Sets

The BRS describes in this case the set of initial conditions from which the system is guaranteed to reach the target set within a given time period despite worst possible disturbances. To analyze the BRS we vary the degree of conservatism using the disturbance splitting method described in section V. After applying splitting, we arrive at the following piece-wise system:

(18) |

We analyzed the decoupled approximation formulation with and ranging from to . To visually depict the computed 4D BRS, we plot 3D slices of the BRS in Fig. 1. In these slices the green sets are the BRS computed using the full formulation, and the gray sets are the decoupled approximations. With the definition of BRS given in (4), all the decoupled approximations are constructed to be under-approximations.

The top row of plots shows the 3D projections through all values of . The bottom row of plots shows the 3D projections through all values of . Moving from left to right, each column of plots shows the decoupled approximations with an increasing number of split sections and .

### Vi-B Reconstruction Performance

To compare the degree of conservatism of the decoupled approximations, we determined the total 4D volume of the BRS computed using both methods. We then took the ratio of the decoupled approximation volume to the full formulation BRS volume. Since under-approximations are computed, a higher volume ratio indicates a lower degree of conservatism.

Fig. 2 shows this volume ratio as a function of and . For example, the purple curve represents the volume ratio for across various values of , and on the other extreme, the yellow curve represents the the volume ratio for across various values of . The highest number of sections computed was with .

Initially the decoupled approximations become less conservative as and increase. This is because splitting the disturbance range has the effect of mitigating the strength of the disturbances. However, splitting the disturbance range also restricts the allowable trajectories of the system and can eventually introduce more conservatism. For example, if the velocity disturbance range is , the trajectories must stay within this velocity range for the duration of . If this range is split, the set of disturbances has a smaller range, but likewise the trajectories for each subsection must remain within the smaller split velocity range for the time period. Therefore, there is an optimal point past which splitting does not help decrease conservatism.

In this system the least conservative approximation was for . The volume ratio for this approximation was 0.217, meaning that the decoupled approximation had a volume that is 21.7% of the volume of the BRS computed using the full formulation. The 3D projections of the set computed by can be seen in Fig. 3.

### Vi-C Computation Time Performance

In Fig. 4 we compare the computation time of the two methods as a function of the number of grid points in each dimension. Computations were done on a desktop computer with a Core i7-5820K CPU and 128 GB of random-access memory (RAM). The full formulation (yellow curve) quickly becomes intractable as grid points are added; 100 grid points in each dimension requires 12.7 hours and 97 GB of RAM.

The decoupled approximation is orders of magnitude faster than the full formulation, and therefore can be done with many more grid points in each dimension. The decoupled approximation used was for sections, as this provided the most accurate BRS as determined in VI-B. The runtimes would be even faster using sections. We plot both the runtimes for reachability computation with 4D-reconstruction (red curve) as well as the runtimes for reachability computation alone (blue curve). Compared to the full formulation, at 100 grid points in each dimension the decoupled approximation takes 50 seconds to run and 36 seconds to reconstruct, with 625 MB of RAM to run and 6.75 GB of RAM to reconstruct. At 200 grid points the decoupled approximation takes 3.37 minutes to run and 44.1 minutes to reconstruct, with 1 GB of RAM to run and 120 GB of RAM to reconstruct.

In general we recommend computing the value function in only a region near a state of interest, bypassing the time and RAM required to reconstruct the function over the entire grid. Without full reconstruction of the value function, we are able to obtain results faster and for higher numbers of grid points before running out of memory, improving the accuracy of the computation.

In Fig. 5 we compare the computation time of the 2D computations in the decoupled approximation as a function and . Each line of the graph represents the computation time for a fixed number of across various values of . As the number of sections increases, the computation time required increases approximately linearly, as expected.

## Vii Conclusions

Hamilton-Jacobi reachability analysis can provide safety and performance guarantees for many practical systems, but the curse of dimensionality limits its application to systems with less than approximately five state variables. By treating state variables as disturbances, key state dependencies can be eliminated, reducing the system dynamics to a simpler form and allowing reachable sets to be calculated conservatively using available efficient methods in the literature.

## References

- [1] P. Varaiya, “On the existence of solutions to a differential game,” SIAM Journal on Control, vol. 5, no. 1, pp. 153–162, 1967.
- [2] L. C. Evans and P. E. Souganidis, “Differential games and representation formulas for solutions of Hamilton-Jacobi-Isaacs equations,” Indiana Univ. Math. J., vol. 33, no. 5, pp. 773–797, 1984.
- [3] E. Barron, “Differential Games with Maximum Cost,” Nonlinear analysis: Theory, methods & applications, pp. 971–989, 1990.
- [4] C. J. Tomlin, J. Lygeros, and S. S. Sastry, “A game theoretic approach to controller design for hybrid systems,” Proceedings of the IEEE, vol. 88, no. 7, pp. 949 –970, July 2000.
- [5] I. Mitchell, A. Bayen, and C. Tomlin, “A time-dependent Hamilton-Jacobi formulation of reachable sets for continuous dynamic games,” IEEE Transactions on Automatic Control, vol. 50, no. 7, pp. 947–957, July 2005.
- [6] O. Bokanowski, N. Forcadel, and H. Zidani, “Reachability and minimal times for state constrained nonlinear problems without any controllability assumption,” SIAM Journal on Control and Optimization, pp. 1–24, 2010.
- [7] J. F. Fisac, M. Chen, C. J. Tomlin, and S. S. Shankar, “Reach-avoid problems with time-varying dynamics, targets and constraints,” in 18th International Conference on Hybrid Systems: Computation and Controls, 2015.
- [8] J. Ding, J. Sprinkle, S. S. Sastry, and C. J. Tomlin, “Reachability calculations for automated aerial refueling,” in IEEE Conference on Decision and Control, Cancun, Mexico, 2008.
- [9] M. Chen, Q. Hu, C. Mackin, J. Fisac, and C. J. Tomlin, “Safe platooning of unmanned aerial vehicles via reachability,” in IEEE Conference on Decision and Control, 2015.
- [10] A. M. Bayen, I. M. Mitchell, M. Oishi, and C. J. Tomlin, “Aircraft autolander safety analysis through optimal control-based reach set computation,” Journal of Guidance, Control, and Dynamics, vol. 30, no. 1, 2007.
- [11] H. Huang, J. Ding, W. Zhang, and C. Tomlin, “A differential game approach to planning in adversarial scenarios: A case study on capture-the-flag,” in Robotics and Automation (ICRA), 2011 IEEE International Conference on, 2011, pp. 1451–1456.
- [12] Jointed Planning and Development Office (JPDO), “Unmanned aircraft systems (UAS) comprehensive plan – a report on the nation’s UAS path forward,” Federal Aviation Administration, Tech. Rep., 2013.
- [13] Amazon.com, Inc. (2016) Amazon prime air. [Online]. Available: http://www.amazon.com/b?node=8037720011
- [14] BBC Technology. (2016) Google plans drone delivery service for 2017. [Online]. Available: http://www.bbc.com/news/technology-34704868
- [15] AUVSI News. (2016) Uas aid in south carolina tornado investigation. [Online]. Available: http://www.auvsi.org/blogs/auvsi-news/2016/01/29/tornado
- [16] National Aeronautics and Space Administration. (2016) Challenge is on to design sky for all. [Online]. Available: http://www.nasa.gov/feature/challenge-is-on-to-design-sky-for-all
- [17] P. Kopardekar, J. Rios, T. Prevot, M. Johnson, J. Jung, and J. E. R. III, “Uas traffic management (utm) concept of operations to safely enable low altitude flight operations,” in AIAA Aviation Technology, Integration, and Operations Conference, 2016.
- [18] I. M. Mitchell and C. J. Tomlin, “Overapproximating reachable sets by hamilton-jacobi projections,” Journal of Scientific Computing, vol. 19, no. 1-3, pp. 323–346, 2003.
- [19] J. S. McGrew, J. P. How, L. Bush, B. Williams, and N. Roy, “Air combat strategy using approximate dynamic programming,” AIAA Guidance, Navigation, and Control Conference, Aug 2008.
- [20] J. B. Lasserre, D. Henrion, C. Prieur, and E. Trélat, “Nonlinear optimal control via occupation measures and lmi-relaxations,” SIAM Journal Control and Optimization, vol. 47, no. 4, pp. 1643–1666, June 2008.
- [21] I. M. Mitchell, “Scalable calculation of reach sets and tubes for nonlinear systems with terminal integrators: A mixed implicit explicit formulation,” in Proceedings of the 14th International Conference on Hybrid Systems: Computation and Control, 2011, pp. 103–112.
- [22] M. Chen and C. J. Tomlin, “Exact and efficient hamilton-jacobi reachability for decoupled systems,” in 54th IEEE Conference on Decision and Control, December 2015.
- [23] I. M. Mitchell, “Comparing forward and backward reachability as tools for safety analysis,” in Proceedings of the 10th International Conference on Hybrid Systems: Computation and Control, 2007.
- [24] M. G. Crandall and P.-L. Lions, “Viscosity solutions of Hamilton-Jacobi equations,” Transactions of the American Mathematical Society, vol. 277, no. 1, pp. 1–42, 1983.
- [25] P. Bouffard, “On-board model predictive control of a quadrotor helicopter: Design, implementation, and experiments,” Master’s thesis, University of California, Berkeley, 2012.