Convergence of the shooting algorithm for singular optimal control problems
In this article we propose a shooting algorithm for optimal control problems governed by systems that are affine in one part of the control variable. Finitely many equality constraints on the initial and final state are considered. We recall a second order sufficient condition for weak optimality, and show that it guarantees the local quadratic convergence of the algorithm. We show an example and solve it numerically.
ptimal control, singular control, second order optimality condition, weak optimality, shooting algorithm, Gauss-Newton method
We investigate optimal control problems governed by ordinary differential equations that are affine in one part of the control variable. This class of system includes both the totally affine and the nonlinear cases. This study is motivated by many models that are found in practice. Among them we can cite the followings: the Goddard’s problem analyzed in Martinon et al. [4, 5, 16], other models concerning the motion of a rocket in Lawden , Bell and Jacobson , Goh , Oberle , and an optimal production process in Cho et al. .
We can find shooting-like methods applied to the numerical solution of partially affine problems in, for instance, Oberle [18, 20] and Oberle-Taubert , where the authors use a generalization of the algorithm that Maurer  suggested for totally affine systems. These works present interesting implementations of a shooting-like algorithm, but they do not link the convergence of the method with sufficient conditions of optimality as it is done in this article.
In this paper we propose a shooting algorithm which can be also used to solve problems with bounds on the controls. We give a theoretical support to this method, by showing that a second order sufficient condition for optimality proved in Aronna  ensures the local quadratic convergence of the algorithm.
The article is organized as follows. In Section 2 we give the statement of the problem, the main definitions and assumptions, and a first order optimality condition. The shooting algorithm is described in Section 3. In Section 4 we recall a second order sufficient condition for weak optimality. We state the main result of the article in Section 5. In Section 6 we work out an example and solve it numerically.
NOTATIONS. Let denote the value of function at time if is a function that depends only on and the th component of evaluated at Partial derivatives of a function of are referred as or for the derivative in time, and or for the differentiations with respect to space variables. The same convention is extended to higher order derivatives. By we mean the Lebesgue space with domain equal to the interval and with values in The notation refers to the Sobolev spaces.
2 Statement of the Problem
We study the optimal control problem (P) given by
Here for for and we put, in sake of simplicity of notation, which is not a variable. The nonlinear control belongs to while denotes the space of affine controls and refers to the state space. When needed, we write for a point in Assume throughout the article that data functions and have Lipschitz-continuous second derivatives. A trajectory is an element that satisfies the state equation (2). If in addition, the constraints in (3) hold, we say that is a feasible trajectory of problem (P).
Set the space of Lipschitz-continuous functions with values in the dimensional space of row-vectors with real components Consider an element and define the pre-Hamiltonian function
the initial-final Lagrangian function
and the Lagrangian function
Throughout the article we study a nominal feasible trajectory that we assume to be smooth. We present now an hypothesis for the endpoint constraints. Consider the mapping
where is the solution of (2) associated with
The derivative of at is onto.
The latter hypothesis is usually known as qualification of the endpoint equality constraints.
It is said that the feasible trajectory is a weak minimum of problem (P) if there exists such that is a minimum in the set of feasible trajectories satisfying
The following first order necessary condition holds for See the book by Pontryagin et al.  for a proof.
Let be a weak solution satisfying Assumption 1, then there exists a unique such that is solution of the costate equation
with transversality conditions
and the stationarity condition
3 The shooting algorithm
The purpose of this section is to present an appropriate numerical scheme to solve the problem (P). More precisely, we investigate the formulation and the convergence of an algorithm that approximates an optimal solution provided an initial estimate exists.
3.1 Optimality system
In what follows we use the first order optimality conditions (9) to provide a set of equations from which we can determine We obtain an optimality system in the form of a two-point boundary value problem (TPBVP).
Throughout the rest of the article we assume, in sake of simplicity, that whenever some argument of or their derivatives is omitted, they are evaluated at and
We shall recall that for the case where all the control variables appear nonlinearly (), the classical technique is using the stationarity equation
In case (11) holds, due to the Implicit Function Theorem, we can write with being a smooth function. Hence, replacing the occurrences of by in the state and costate equations yields a two-point boundary value problem.
On the other hand, when the system is affine in all the control variables (), we cannot eliminate the control from the equation and, therefore, a different technique is employed (see e.g. [16, 2, 24]). The idea is to consider an index and to take to be the lowest order derivative of in which appears with a coefficient that is not identically zero. Goh [11, 10], Kelley et al.  and Robbins  proved that is even. This implies that the control does not appear the first time we derive with respect to time, i.e. depends only on and and consequently, it is differentiable in time. Thus the expression
is well-defined. The control can be retrieved from (12) provided that, for instance, the strengthened generalized Legendre-Clebsch condition
holds (see Goh [10, 12, 13]). In this case, we can write with being differentiable. By replacing by in the state-costate equations, we get an optimality system in the form of a boundary value problem.
In the problem studied here, where and we aim to use both equations (10) and (12) to retrieve the control as a function of the state and the multiplier We next describe a procedure to achieve this elimination that was proposed in Goh [12, 13]. First let as recall a necessary condition proved in Goh  and in [1, Lemma 3.10 and Corollary 5.2]. Define which is referred as the Lie bracket in the variable of and
Lemma 3.1 (Necessary conditions for weak optimality)
If is a smooth weak minimum for (P) satisfying Assumption 1, then
Let us show that can be differentiated twice with respect to the time variable, as it was done in the totally affine case. Observe that (10) may be used to write as a function of In fact, in view of Lemma 3.1, the coefficient of in is zero. Consequently,
Take now an index and observe that
where Lemma 3.1 is used in the last equality. Therefore, We can then differentiate one more time, replace the occurrence of by and obtain (12) as it was desired. See that (12) together with the boundary conditions
guarantee the second identity in the stationarity condition (9).
Observe now that the derivative with respect to of the mapping is given by
On the other hand, if (11) and (13) are verified, is definite positive along and, consequently, it is nonsingular. In this case we may write and from (10) and (12). Thus (OS) can be regarded as a TPBVP whenever the following hypothesis is verified.
Summing up we get the following result.
Proposition 3.1 (Elimination of the control)
When the linear and nonlinear controls are uncoupled, this elimination of the controls is much simpler. An example is shown in Oberle  where a nonlinear control variable can be eliminated by the stationarity of the pre-Hamiltonian, and the remaining problem has two uncoupled controls, one linear and one nonlinear. Another example is the one presented in Section 6.
3.2 The algorithm
The aim of this section is to present a numerical scheme to solve system (OS). In view of Proposition 3.1 we can define the following mapping.
Note that solving (OS) consists of finding such that
Since the number of equations in (21) is greater than the number of unknowns, the Gauss-Newton method is a suitable approach to solve it. The shooting algorithm we propose here consists of solving the equation (21) by the Gauss-Newton method.
3.3 The Gauss-Newton Method
This algorithm solves the equivalent least squares problem
At each iteration given the approximate value it looks for that gives the minimum of the linear approximation of problem
Afterwards it updates
In order to solve the linear approximation of problem (22) at each iteration we look for in the kernel of the derivative of the objective function, i.e. satisfying
Hence, to compute direction the matrix must be nonsingular. Thus, Gauss-Newton method will be applicable provided that is invertible, where It follows easily that is nonsingular if and only if is one-to-one.
Furthermore, since the right hand-side of system (21) is zero, it can be proved that the Gauss-Newton algorithm converges locally quadratically if the function has Lipschitz continuous derivative. The latter holds true here given the regularity hypotheses on the data functions. This convergence result is stated in the proposition below. See e.g. Fletcher  for a proof.
If is one-to-one then the shooting algorithm is locally quadratically convergent.
4 Second order sufficient condition
In this section we present a sufficient condition for optimality proved in , and we state in Section 5 afterwards that this condition guarantees the local quadratic convergence of the shooting algorithm proposed above.
Given consider the linearized state equation
where refers to the partial derivative of with respect to i.e. and equivalent notations hold for the other involved derivatives. Take an element and define the second variation of the Lagrangian function
It can be proved that can be written as
Note that this mapping does not contain a quadratic term on since Hence, one cannot state a sufficient condition in terms of the uniform positivity of on the set of critical directions, as it is done in the totally nonlinear case. Therefore, we use a change of variables introduced by Goh in  and transform into a quadratic mapping that may result uniformly positive in an associated transformed set of critical directions.
Consider hence the linear differential system in (23) and the change of variables
This change of variables can be done in any linear system of differential equations, and it is often called Goh’s transformation. Observe that defined in that way satisfies the linear equation
4.1 Critical cones
We define now the sets of critical directions associated with Even if we are working with control variables in and hence the control perturbations are naturally taken in the second order analysis involves quadratic mappings and it is useful to extend them continuously to Given satisfying (23)-(24), consider the linearization of the endpoint constraints and cost function,
Define the critical cone in by
Since we aim to state an optimality condition in terms of the variables after Goh’s transformation, we transform the equations defining Let be a critical direction. Define by transformation (25) and set Then the transformed of (27)-(28) is
Consequently, the transformed critical cone is given by
4.2 Second variation
Next we state that if is a weak minimum, then the transformation of yields the quadratic mapping
If is a smooth weak minimum, then
for all and given by (25).
4.3 The sufficient condition
We state now a second order sufficient condition for strict weak optimality.
Define the order by
for It can also be considered as a function of by setting
with being the primitive of defined in (25).
[growth] We say that satisfies growth condition in the weak sense if there exist such that
for every feasible trajectory with
Theorem 4.2 (Sufficient condition for weak optimality)
Let be a smooth feasible trajectory such that Assumption 1 is satisfied. Then the following assertions hold.
Assume that there exists such that
Then is a weak minimum satisfying growth in the weak sense.
Conversely, if is a weak solution satisfying growth in the weak sense then (36) holds for some
5 Main result: Convergence of the shooting algorithm
The main result of this article is the theorem below that gives a condition guaranteeing the quadratic convergence of the shooting method near an optimal local solution.
The complete proof of this theorem can be found in . The idea of the proof is to show that (36) yields the injectivity of and then use Proposition 3.2. In order to prove that (36) implies that is one-to-one, the following elements are employed: the linearization of (OS) which gives an expression of the derivative the Goh’s transformed of this linearized system and an associated linear-quadratic optimal control problem in the variables involving (26) and (33).
Remark 5.2 (Bang-singular solutions)
Finally we claim that the formulation of the shooting algorithm above and the proof of its local convergence (Theorem 5.1) can be done also for problems where the controls are subject to bounds of the type
More precisely, it holds for solutions for which each control component is a concatenation of bang and singular arcs, i.e. arcs saturating the corresponding inequality in (37), and arcs in the interior of the constraint. This extension follows from a transformation of the problem to one without bounds, and it is detailed in [2, Section 8] for the totally-affine case.
6 An example
Consider the following optimal control problem treated in Dmitruk and Shishov :
Here, Assumption 1 holds since no final constraints are considered. The pre-Hamiltonian function associated with (38) is, omitting arguments,
We can then write the optimality system (OS) related to (38). The state and costate equations are
where we do not include since it is constantly equal to 1. The boundary conditions are
Observe that the last line in (41) can be removed since it is implied by the first equation in the second line. Here the shooting function is given by
In  it was checked that the second order sufficient condition (36) held for the control The solution associated with this control has In view of Theorem 4.2, we know that the shooting algorithm converges quadratically for appropriate initial values of
We investigated optimal control problems with systems that are affine in some components of the control variable and that have finitely many equality endpoint constraints. For a Mayer problem of this kind of system we proposed a numerical indirect method for approximating a weak solution. For qualified solutions, we proved that the local convergence of the method is guaranteed by a second order sufficient condition for optimality proved before by the author.
We presented an example, in which we showed how to eliminate the control by using the optimality conditions, proposed a shooting formulation and solved it numerically. The tests converged, as it was expected in view of the theoretical result.
Part of this work was done under the supervision of Frédéric Bonnans during my Ph.D. study. I acknowledge him for his great guidance.
I also thank Xavier Dupuis for his careful reading, and the three anonymous reviewers for their useful remarks.
This work is supported by the European Union under the 7th Framework Programme FP7-PEOPLE-2010-ITN Grant agreement number 264735-SADCO.
- M.S. Aronna. Singular solutions in optimal control: second order conditions and a shooting algorithm. INRIA Research Rapport Nr. 7764 or arXiv:1210.7425, 2011.
- M.S. Aronna, J.F. Bonnans, and Martinon P. A shooting algorithm for problems with singular arcs. J. Optim. Theory Appl., Published as ‘Online First’, 2013.
- D.J. Bell and D.H. Jacobson. Singular Optimal Control Problems. Academic Press, 1975.
- F. Bonnans, J. Laurent-Varin, P. Martinon, and E. Trélat. Numerical study of optimal trajectories with singular arcs for an Ariane 5 launcher. J. Guidance, Control, and Dynamics, 32(1):51–55, 2009.
- A.E. Bryson, Jr. and Y.C. Ho. Applied optimal control. Hemisphere Publishing Corp. Washington, D. C., 1975. Optimization, estimation, and control, Revised printing.
- D.I. Cho, P.L. Abad, and M. Parlar. Optimal production and maintenance decisions when a system experience age-dependent deterioration. Optimal Control Appl. Methods, 14(3):153–167, 1993.
- A. V. Dmitruk and K. K. Shishov. Analysis of a Quadratic Functional with a Partly Singular Legendre Condition. Moscow University Computational Mathematics and Cybernetics, 34(1):16–25, 2010.
- R. Fletcher. Practical methods of optimization. Vol. 1. John Wiley & Sons Ltd., Chichester, 1980. Unconstrained optimization, A Wiley-Interscience Publication.
- B.S. Goh. Necessary conditions for singular extremals involving multiple control variables. J. SIAM Control, 4:716–731, 1966.
- B.S. Goh. Necessary Conditions for the Singular Extremals in the Calculus of Variations. University of Canterbury, 1966.
- B.S. Goh. The second variation for the singular Bolza problem. J. SIAM Control, 4(2):309–325, 1966.
- B.S. Goh. Compact forms of the generalized Legendre-Clebsch conditions and the computation of singular control trajectories. In Proceedings of the American Control Conference, volume 5, pages 3410–3413, 1995.
- B.S. Goh. Optimal singular rocket and aircraft trajectories. In Control and Decision Conference, 2008. CCDC 2008, pages 1531 –1536, 2008.
- H.J. Kelley, R.E. Kopp, and H.G. Moyer. Singular extremals. In Topics in Optimization, pages 63–101. Academic Press, New York, 1967.
- D. F. Lawden. Optimal trajectories for space navigation. Butterworths, London, 1963.
- H. Maurer. Numerical solution of singular control problems using multiple shooting techniques. J. Optim. Theory Appl., 18(2):235–257, 1976.
- H. Maurer and W. Gillessen. Application of multiple shooting to the numerical solution of optimal control problems with bounded state variables. Computing, 15(2):105–126, 1975.
- H.J. Oberle. Numerische Behandlung singulärer Steuerungen mit der Mehrzielmethode am Beispiel der Klimatisierung von Sonnenhäusern. PhD thesis. Technische Universität München, 1977.
- H.J. Oberle. On the numerical computation of minimum-fuel, Earth-Mars transfer. J. Optimization Theory Appl., 22(3):447–453, 1977.
- H.J. Oberle. Numerical computation of singular control functions in trajectory optimization problems. J. Guidance Control Dynam., 13(1):153–159, 1990.
- H.J. Oberle and K. Taubert. Existence and multiple solutions of the minimum-fuel orbit transfer problem. J. Optim. Theory Appl., 95(2):243–262, 1997.
- L. Pontryagin, V. Boltyanski, R. Gamkrelidze, and E. Michtchenko. The Mathematical Theory of Optimal Processes. Wiley Interscience, New York, 1962.
- H.M. Robbins. A generalized Legendre-Clebsch condition for the singular case of optimal control. IBM J. of Research and Development, 11:361–372, 1967.
- E. Trélat. Optimal Control and Applications to Aerospace: Some Results and Challenges. J. Optim. Theory Appl., 154(3):713–758, 2012.