Convergence rate for a Gauss collocation method applied to constrained optimal control ††thanks: July 10, 2016. Revised December 16, 2017. The authors gratefully acknowledge support by the Office of Naval Research under grant N00014-15-1-2048, by the National Science Foundation under grant DMS-1522629, and by the U.S. Air Force Research Laboratory under contract FA8651-08-D-0108/0054.
A local convergence rate is established for a Gauss orthogonal collocation method applied to optimal control problems with control constraints. If the Hamiltonian possesses a strong convexity property, then the theory yields convergence for problems whose optimal state and costate possess two square integrable derivatives. The convergence theory is based on a stability result for the sup-norm change in the solution of a variational inequality relative to a 2-norm perturbation, and on a Sobolev space bound for the error in interpolation at the Gauss quadrature points and the additional point . The tightness of the convergence theory is examined using a numerical example.
Key words. Gauss collocation method, convergence rate, optimal control, orthogonal collocation
AMS subject classifications. 49M25, 49M37, 65K05, 90C30
In earlier work [24, 25, 26], we analyze the convergence rate for orthogonal collocation methods applied to unconstrained control problems. In this analysis, it is assumed that the problem solution is smooth, in which case the theory implies that the discrete approximations converge to the solution of the continuous problem at potentially an exponential rate. But when control constraints are present, the solution often possesses limited regularity. The convergence theory developed in the earlier work for unconstrained problems required that the optimal state had at least four derivatives, while for constrained problems, the optimal state may have only two derivatives, at best [4, 7, 20, 28]. The earlier convergence theory was based on a stability analysis for a linearization of the unconstrained control problem; the theory showed that the sup-norm change in the solution was bounded relative to the sup-norm perturbation in the linear system. Here we introduce a convex control constraint, in which case the linearized problem is a variational inequality, or equivalently a differential inclusion, not a linear system. We obtain a bound for the sup-norm change in the solution relative to a 2-norm perturbation in the variational inequality. By using the 2-norm for the perturbation rather than the sup-norm, we are able to avoid both Lebesgue constants and the Markov bound  for the sup-norm of the derivative of a polynomial relative to the sup-norm of the original polynomial. Using best approximation results in Sobolev spaces [3, 13], we obtain convergence when the optimal state and costate have only two square integrable derivatives, which implies that the theory is applicable to a class of control constrained problems for which the optimal control is Lipschitz continuous.
The specific collocation scheme analyzed in this paper, presented in [2, 18], is based on collocation at the Gauss quadrature points, or equivalently, at the roots of a Legendre polynomial. Other sets of collocation points that have been studied in the literature include the Lobatto quadrature points [11, 14, 19], the Chebyshev quadrature points [12, 15], the Radau quadrature points [16, 17, 33, 36], and extrema of Jacobi polynomials . Kang [31, 32] obtains a convergence rate for the Lobatto scheme applied to control systems in feedback linearizable normal form by inserting bounds in the discrete problem for the states, the controls, and certain Legendre polynomial expansion coefficients. In our approach, the discretized problem is obtained by simply collocating at the Gauss quadrature points.
Our approximation to the control problem uses a global polynomial defined on the problem domain. Earlier work, including [6, 8, 9, 10, 22, 30, 37], utilizes a piecewise polynomial approximation, in which case convergence is achieved by letting the mesh spacing approach zero, while keeping the polynomial degree fixed. For an orthogonal collocation scheme based on global polynomials, convergence is achieved by letting the degree of the polynomials tend to infinity. Our results show that even when control constraints are present, and a solution possesses limited regularity, convergence can still be achieved with global polynomials.
We consider control problems of the form
where , the control constraint set is closed and convex with nonempty interior, the state , denotes the derivative of with respect to , is the initial condition which we assume is given, , , denotes the space of times continuously differentiable functions mapping to . It is assumed that and are at least continuous.
Let denote the space of polynomials of degree at most , and let denote the -fold Cartesian product . We analyze the discretization of (LABEL:P) given by
The polynomials used to approximate the state should satisfy the dynamics exactly at the collocation points , . The parameter represents an approximation to the control at time . The dimension of is , while there are equations in (LABEL:D) corresponding to the collocated dynamics at points and the initial condition. We collocate at the Gauss quadrature points, which are symmetric about and which satisfy
The analysis also makes use of the two noncollocated points
For , we use the sup-norm given by
where is the Euclidean norm. Given , the ball with center and radius is denoted
The following regularity assumption is assumed to hold throughout the paper.
Smoothness. The problem (LABEL:P) has a local minimizer in . There exists an open set and such that
Moreover, the first two derivatives of and are Lipschitz continuous on the closure of and on respectively.
Let denote the solution of the linear costate equation
where is the Hamiltonian defined by and denotes gradient. From the first-order optimality conditions (Pontryagin’s minimum principle), it follows that
where is the normal cone. For any ,
while if .
Since the collocation problem (LABEL:D) is finite dimensional, the first-order optimality conditions, or Karush-Kuhn-Tucker conditions, hold when a constraint qualification  is satisfied. We show in Lemma LABEL:equiv that the first-order optimality conditions are equivalent to the existence of such that
The following assumptions are utilized in the convergence analysis.
For some , the smallest eigenvalue of the Hessian matrices and are greater than , uniformly for .
For some , the Jacobian of the dynamics satisfies
for all where is the matrix sup-norm (largest absolute row sum), and the Jacobian is an by matrix whose -th row is .
The condition (A2) ensures (see Lemma LABEL:feasible) that in the discrete linearized problem, it is possible to solve for the discrete state in terms of the discrete control. As shown in , this property holds in an -collocation framework when the domain is partitioned into mesh intervals with large enough that
for all .
The coercivity assumption (A1) is not only a sufficient condition for the local optimality of a feasible point of (LABEL:P), but it yields the stability of the discrete linearized problem (see Lemma LABEL:inf-bounds). One would hope that (A1) could be weakened to only require coercivity relative to a subspace associated with the linearized dynamics similar to what is done in . To formulate this weakened condition, we introduce the following 6 matrices:
With this notation and with denoting the inner product, the weaker version of (A1) is that
whenever satisfies with and for some and satisfying and for almost every . For the Euler integration scheme, we show in [6, Lem. 11] that this weaker condition implies an analogous coercivity property for the discrete problem. The extension of this result from the Euler scheme to orthogonal collocation schemes remains an open problem.
Let be the by matrix defined by
The matrix is a differentiation matrix in the sense that , , whenever is the polynomial that satisfies for . The submatrix , consisting of the trailing columns of , has the following properties which are utilized in the analysis:
is invertible and .
If is the diagonal matrix containing the Gauss quadrature weights , , on the diagonal, then the rows of the matrix have Euclidean norm bounded by .
The invertibility of is proved in [18, Prop. 1]. The bound for the inverse appearing in (P1) is established in Appendix 1. (P2) has been checked numerically for up to 300 in . Some intuition concerning the general validity of (P2) is as follows: It is observed numerically that the last row of the matrix has the largest Euclidean norm among all the rows. Based on the formula for given in [18, Sect. 4.1.2], the -th element in the last row approaches as tends to infinity. Hence, the -th element in the last row of approaches as tends to infinity. Since the quadrature weights sum to 2, the Euclidean norm of the last row of should be close to . Despite the strong numerical evidence for (P2), a proof of (P2) for general is still missing.
The properties (P1) and (P2) are stated separately since they are used in different ways in the analysis. However, (P2) implies (P1) by the Schwarz inequality. That is, if is a row from , then we have
since the quadrature weights sum to 2 and when (P2) holds, the Euclidean norm of a row from is at most .
If is a solution of (LABEL:D) associated with the discrete controls , , and if satisfies (LABEL:dcostate)–(LABEL:dcontrolmin), then we define
The following convergence result relative to the vector -norm (largest absolute element) is established. Here denotes the Sobolev space of functions with square integrable derivatives through order and norm denoted .
Suppose is a local minimizer for the continuous problem with for some . If both (A1)–(A2) and (P1)–(P2) hold, then for sufficiently large, the discrete problem has a local minimizer and , and an associated multiplier satisfying –; moreover, there exists a constant independent of and such that
This result was established in  for unconstrained control problem, but with the exponent 3/2 replaced by 3 and with . Hence, the analysis is extended to control constrained problems and the exponent of in the convergence estimate is improved by 1.5. Since typical control constrained problems have regularity at most when (A1) holds, there is no guarantee of convergence with the previous estimate.
The paper is organized as follows. In Section LABEL:abstract the discrete optimization problem (LABEL:D) is reformulated as a differential inclusion obtained from the first-order optimality conditions, and a general approach to convergence analysis is presented. We also establish the connection between the Karush-Kuhn-Tucker conditions and the polynomial conditions (LABEL:dcostate)–(LABEL:dcontrolmin). In Section LABEL:sect_interp we use results from  to bound the derivative of the interpolation error in . Section LABEL:residual estimates how closely the solution to the continuous problem satisfies the first-order optimality conditions for the discrete problem, while Section LABEL:inverse establishes the invertibility of the linearized dynamics for the discrete problem. Section LABEL:Lip proves a Lipschitz property for the linearized optimality conditions, which yields a proof of Theorem LABEL:maintheorem. A numerical example given in Section LABEL:numerical indicates the potential for further improvements to the convergence rate exponent. Section LABEL:appendix2 contains a result of Yvon Maday concerning the error in best approximation relative to an norm with a singular weight function.
Notation. We let denote the space of polynomials of degree at most , while is the subspace consisting of polynomials in that vanish at and . The Gauss collocation points , , are the roots of the Legendre polynomial of degree . The associated Gauss quadrature weights , , are given by
For any , we have [38, Thm. 3.6.24]
Derivatives with respect to are denoted with either a dot above the function as in , which is common in the optimal control literature, or with an accent as in , which is common in the numerical analysis literature. The meaning of the norm is based on context. If , then denotes the maximum of over , where is the Euclidean norm. For a vector , is the maximum of over . If , then is the largest absolute row sum (the matrix norm induced by the vector sup-norm). We often partition a vector into subvectors , . Similarly, if , then . The dimension of the identity matrix is often clear from context; when necessary, the dimension of is specified by a subscript. For example, is the by identity matrix. The gradient is denoted , while denotes the Hessian; subscripts indicate the differentiation variables. Throughout the paper, is a generic constant which is independent of the polynomial degree and the smoothness , and which may have different values in different equations. The vector has all entries equal to one, while the vector has all entries equal to zero; again, their dimension should be clear from context. If is the differentiation matrix introduced in (LABEL:Ddef), then is the -th column of and is the submatrix formed by columns through . We let denote the Kronecker product. If and , then is the by matrix composed of blocks; the block is . We let denote the usual space of functions square integrable on , while is the Sobolev space consisting of functions with square integrable derivatives through order . The norm in is denoted . The seminorm in corresponding to the norm of the derivative is denoted . The subspace of corresponding to functions that vanish at and is denoted . We let denote the -fold Cartesian product .
2 Abstract Setting
In the introduction, we formulated the discrete optimization problem (LABEL:D) and the necessary conditions (LABEL:dcostate)–(LABEL:dcontrolmin) in polynomial spaces. However, to prove Theorem LABEL:maintheorem, we reformulate the first-order optimality conditions in Cartesian space. Given a feasible point and for the discrete problem (LABEL:D), define , , and , . As noted earlier, is a differentiation matrix in the sense that
Since , it follows from the exactness result (LABEL:exact) for Gaussian quadrature that when satisfies the dynamics of (LABEL:D), we have
Hence, the discrete problem (LABEL:D) can be reformulated as the nonlinear programming problem
To prove Theorem LABEL:maintheorem, we analyze the existence and stability of solutions to the first-order optimality conditions associated with the nonlinear programming problem.
We introduce multipliers , corresponding to each of the constraints in the nonlinear program. The first-order optimality conditions correspond to stationary points of the Lagrangian
The stationarity conditions for the Lagrangian appear below.
Since there are no state constraints, the conditions (LABEL:NC0)–(LABEL:NC2) are obtained by setting to zero the derivative of the Lagrangian with respect to the indicated variables. The condition (LABEL:NC3) corresponds to stationarity of the Lagrangian respect to the control. The relation between multipliers satisfying (LABEL:NC0)–(LABEL:NC3) and satisfying (LABEL:dcostate)–(LABEL:dcontrolmin) is as follows.
The multipliers satisfy – if and only if the polynomial satisfying the interpolation conditions and , , is a solution of – and .
Proof. We start with multipliers satisfying (LABEL:NC0)–(LABEL:NC3) and show that satisfying the interpolation conditions and , , is a solution of – with . The converse follows by reversing all the steps in the derivation. Define for , , and . Hence, we have for . In (LABEL:NC3) we divide by and substitute . In (LABEL:NC1) we divide by , and substitute and
With these modifications, (LABEL:NC1)–(LABEL:NC3) become
. In [18, Thm. 1] it is shown that if is a polynomial that satisfies the conditions for , then
This identity coupled with (LABEL:Dadjoint)–(LABEL:Dcontrolmin) imply that – hold.
Now let us consider the final term in (LABEL:NC0). Since the polynomial that is identically equal to has derivative and since is a differentiation matrix, we have , which implies that , where is the -th column of . Hence, the final term in (LABEL:NC0) can be written
Again, if is the interpolating polynomial that satisfies for , then by (LABEL:dif), (LABEL:mu0expand), and the exactness of Gaussian quadrature for polynomials in , we have
Since , we deduce from (LABEL:NC0) and (LABEL:Di0) that .
In the proof of Proposition LABEL:equiv, and . We combine (LABEL:NC0), (LABEL:Dadjoint), and (LABEL:mu0expand) to obtain
Based on Proposition LABEL:equiv, the optimality conditions (LABEL:NC0)–(LABEL:NC3) are equivalent to (LABEL:dcostate)–(LABEL:dcontrolmin), which are equivalent to (LABEL:Dadjoint)–(LABEL:Dcontrolmin) and (LABEL:mu0). This latter formulation, which we refer to as the transformed adjoint system in our earlier work , is most convenient for the subsequent analysis. This leads us to write the first-order optimality conditions for (LABEL:D) as an inclusion where
The 7 components of are defined as