On the Optimal Control of Impulsive Hybrid Systems On Riemannian ManifoldsThis work was supported by NSERC and AFOSR.

# On the Optimal Control of Impulsive Hybrid Systems On Riemannian Manifolds††thanks: This work was supported by NSERC and AFOSR.

Farzin Taringoo    Peter E. Caines Department of Electrical and Computer Engineering and Centre for Intelligent Machines, McGill University, Montreal, Canada, {taringoo, peterc@cim.mcgill.ca}.
###### Abstract

This paper provides a geometrical derivation of the Hybrid Minimum Principle (HMP) for autonomous impulsive hybrid systems on Riemannian manifolds, i.e. systems where the manifold valued component of the hybrid state trajectory may have a jump discontinuity when the discrete component changes value. The analysis is expressed in terms of extremal trajectories on the cotangent bundle of the manifold state space. In the case of autonomous hybrid systems, switching manifolds are defined as smooth embedded submanifolds of the state manifold and the jump function is defined as a smooth map on the switching manifold. The HMP results are obtained in the case of time invariant switching manifolds and state jumps on Riemannian manifolds.

H

ybrid Minimum Principle, Riemannian Manifolds.

{AMS}

34A38, 49N25, 34K34, 49K30, 93B27

## 1 Introduction

The problem of hybrid systems optimal control (HSOC) in Euclidean spaces has been studied in many papers, see e.g. [6, 8, 11, 12, 14, 15, 17, 18, 24, 27, 30, 32, 39, 41]. In particular, [4, 17, 30, 32] present an extension of the Minimum Principle to hybrid systems and [30] gives an iterative algorithm which is based upon the Hybrid Minimum Principle (HMP) necessary conditions for both autonomous and controlled switching systems. In general the previously cited papers consider HSOC problems with a priori given sequences of discrete transitions. In addition, [17] includes the case of switching costs.

We note that historically optimal control theory has mainly used the term Maximum Principle since optimal controls were derived via the maximization of a Hamiltonian function, see e.g. [25]. However, since we work with problems in the Bolza form we formulate the theory in terms of the minimization of a suitably defined Hamiltonian function and consequently shall consistently use the term Minimum Principle.

A geometric version of Pontryagin’s Minimum Principle for a general class of state manifolds is given in [1, 5, 32]. In this paper, we employ the control needle variation method of [5], [13] and [37] to analyze state variation propagation through switching manifolds and hence we obtain a Hybrid Minimum Principle for autonomous hybrid systems (i.e. systems without controlled distinct state switchings) on Riemannian manifolds. It is shown that under appropriate hypotheses on the differentiability of the hybrid value function, the discontinuity of the adjoint variable at the optimal switching state and switching time is proportional to a differential form of the hybrid value function defined on the cotangent bundle of the state manifold. In the case of open control sets and Euclidean state spaces this result for impulsive hybrid systems appeared in [26] without using the language of differential geometry. We note that the analysis in this paper extends to the case of multiple autonomous switchings which has been treated in [30] for hybrid systems defined on Euclidean spaces.

The continuity of the Hamiltonian function in the case of time invariant switching manifolds is derived in [30] for open control value sets by employing the methods of the calculus of variations. In this paper, for compact control value sets, we obtain the continuity result for the Hamiltonian function at the optimal switching time by use of the needle variation method. In particular we note that here the needle variation method is generalized to a class of autonomous hybrid systems associated with time varying embedded switching manifolds when the Hamiltonian function is discontinuous at optimal switching times. It is shown that the discontinuity is related to a differential form of an augmented hybrid value function.

In this paper, in Section 2 we give a general definition of hybrid systems on differentiable manifolds and then in Section 3 present a geometric version of the Pontryagin Minimum Principle for optimal control systems. In Section 4 we obtain the Hybrid Minimum Principle for impulsive hybrid systems using the method of needle variations. Complete proofs of the results of Section 3 are given in the Appendices A-C. Furthermore the analysis of those cases where the hybrid value functions are differentiable, and the switching manifolds and impulsive jumps are time varying, are given in the referenced link [35].

## 2 Hybrid Systems

In the following definition the standard hybrid systems framework (see e.g. [8, 30]) is generalized to the case where the continuous state space is a smooth manifold, where henceforth in this paper smooth means . {definition} A hybrid system with autonomous discrete transitions is a five-tuple

 H:={\textslH=\textslQ×M,\textslU,\textslF,S,J} (1)

where:
is a finite set of discrete (valued) states (components) and is a smooth dimensional Riemannian continuous (valued) state (component) manifold with associated metric .
is called the hybrid state space of .
is a set of admissible input control values, where is a compact set in . The set of admissible input control functions is , the set of all bounded measurable functions on some interval , taking values in .
is an indexed collection of smooth, i.e. , vector fields , where is a controlled vector field assigned to each discrete state; hence each is continuous on and continuously differentiable on for all .
is a collection of embedded time independent pairwise disjoint switching manifolds except in the case where is identified with such that for any ordered pair is an open smooth, oriented codimension 1 submanifold of , possibly with boundary . By abuse of notation, we describe the manifolds locally by .
shall denote the family of the state jump functions on the manifold . For an autonomous switching event from to , the corresponding jump function is given by a smooth map : if the state trajectory jumps to , . The non-jump special case is given by .
We use the term impulsive hybrid systems for those hybrid systems where the continuous part of the state trajectory may have discontinuous transitions (i.e. jump) at controlled or autonomous discrete state switching times.
We assume:
A1: The initial state is such that for all . A (hybrid) input function is defined on a half open interval , where further . A (hybrid) state trajectory with initial state and (hybrid) input function is a triple consisting of a finite strictly increasing sequence of times (boundary and switching times) , an associated sequence of discrete states , and a sequence of absolutely continuous functions satisfying the continuous and discrete dynamics given by the following definition. {definition} The continuous dynamics of a hybrid system with initial condition , input control function and hybrid state trajectory are specified piecewise in time via the mappings

 (xqi,u):[ti,ti+1)→M×U,i=0,...,L,0

where is an integral curve of satisfying

 ˙xqi(t)=fqi(xqi(t),u(t)),a.e.t∈[ti,ti+1),

where is given recursively by

 xqi+1(ti+1)=limt↑t−i+1ζqi,qi+1(xqi(t)),h0=(q0,x0),t

The discrete autonomous switching dynamics are defined as follows:
For all , whenever an admissible hybrid system trajectory governed by the controlled vector field meets any given switching manifold transversally, i.e. , there is an autonomous switching to the controlled vector field , equivalently, discrete state transition . Conversely, any autonomous discrete state transition corresponds to a transversal intersection.

A system trajectory is not continued after a non-transversal intersection with a switching manifold. Given the definitions and assumptions above, standard arguments give the existence and uniqueness of a hybrid state trajectory , with initial state and input function , up to defined to be the least of an explosion time or an instant of non-transversal intersection with a switching manifold.

A2:  (Controllability) For any , all pairs of states are mutually accessible in any given time period , via the controlled vector field for some .

A3:   , is a family of loss functions such that , and is a terminal cost function such that .

Henceforth, Hypotheses A1-A3 will be in force unless otherwise stated. Let be the number of switchings and then we define the hybrid cost function as

 J(t0,tf,h0;L,u):=L∑i=0∫ti+1tilqi(xqi(s),u(s))ds+h(xqL(tf)), tL+1=tf

where we observe the conditions above yield . {definition} For a hybrid system , given the data , the Bolza Hybrid Optimal Control Problem (BHOCP) is defined as the infimization of the hybrid cost function over the hybrid input functions , i.e.

 Jo(t0,tf,h0;L)=infu∈IJ(t0,tf,h0;L,u).
{definition}

A Mayer Hybrid Optimal Control Problem (MHOCP) is defined as the special case of the BHOCP where the cost function given in (2) is evaluated only on the terminal state of the system, i.e. .

In general, different control inputs result in different sequences of discrete states of different cardinality. However, in this paper, we shall restrict the infimization to be over the class of control functions, generically denoted , which generates an a priori given sequence of discrete transition events.

We adopt the following standard notation and terminology, see [9]. The time dependent flow associated to a differentiable time independent vector field is a map satisfying (where for economy of notation ):

 Φfuqi:[ti,ti+1)×[ti,ti+1)×M→M,(t,s,x)→Φ(t,s)fuqi(x):=Φfuqi((t,s),x)∈M,

where

 Φ(t,s)fuqi:M→M,Φ(s,s)fuqi(x)=x, (5)
 ddtΦ(t,s)fuqi(x)|t=fqi(Φ(t,s)fuqi(x(s))),t,s∈[ti,ti+1). (6)

We associate to via the push-forward of .

 TΦ(t,s)fuqi:TxM→TΦ(t,s)fuqi(x)M. (7)

Following [9], the corresponding tangent lift of is the time dependent vector field on

 fT,uqi(vx):=ddt|t=sTΦ(t,s)fuqi(vx),vx∈TxM, (8)

which is given locally as

 fT,uqi(x,vx)=[fu,iqi(x)∂∂xi+(∂fu,iqi∂xjvj)∂∂vi]ni,j=1, (9)

and is evaluated on , see [9]. The following lemma gives the relation between the push-forward of and the tangent lift introduced in (9). For simplicity and uniformity of notation, we use instead of . The following lemma is taken from [5] and its results are essential to obtain the Minimum Principle along the optimal trajectory for standard optimal control problems. In this paper we use the same results to obtain the HMP statement for hybrid systems. {lemma}[[5]] Consider as a time dependent vector field on and as its corresponding flow. The flow of , denoted by , satisfies:

 Ψ(t,s,(x,v))=(Φ(t,s)fqi(x),TΦ(t,s)fqi(v))∈TM,(x,v)∈TM.

## 3 The Pontryagin Minimum Principle for standard optimal control problems

In this section we focus on the Pontryagin Minimum Principle (PMP) for standard (non-hybrid) optimal control problems defined on a Riemannian manifold . A standard optimal control problem (OCP) can be obtained from a BHOCP, see (2), by fixing the discrete states to , and hence to the value 0. The resulting optimal control problem in Bolza form becomes that of the infimization of the cost (2) with respect to state dynamics which by suppressing notation of may be written

### 3.1 The Relationship between Bolza and Mayer Problems

In Section 2 both the BHOCP and the MHOCP were introduced; since the results in this paper are only stated for the Mayer problem we now briefly explain the relationship between them.

In general (see [5]), a Bolza problem can be converted to a Mayer problem with state variable by adjoining an auxiliary state to the state , one then defines the dynamics to be given by

 ˙^x(t)=[˙x(t)˙xn+1(t)]=[f(x(t),u(t))l(x(t),u(t))], (10)

where and are respectively the dynamics and the running cost of the Bolza problem. Then the equivalent Mayer problem is obtained by the infimization of the penalty function defined as follows:

 ^h(^x(tf))≡^h(x(tf),xn+1(tf)):=xn+1(tf)+h(x(tf))=J(t0,tf,x0,u), (11)

where is the terminal cost function of the Bolza problem. Note that after such a transformation from a Bolza problem the state space of the resulting Mayer problem is , where is the state manifold of the Bolza problem.

### 3.2 Elementary Control and Tangent Perturbations

We now present some results from [1], [5] and [21]. It is essential to note that henceforth in this paper we treat the general Mayer problem with state space manifold denoted by . In the special case where Mayer OCP is derived from a Bolza problem takes the product form given in the previous section.

Consider the nominal control input and define the associated perturbed control as

 uπ(t1,u1)(t,ϵ)={u1t1−ϵ≤t≤t1,u(t)elsewhere, (12)

where . For brevity in notation shall be written .

Associated to we have the corresponding state trajectory on . It may be shown under suitable hypotheses, uniformly for , see [17] and [21]. Following (5), the flow resulting from the perturbed control is defined as:

 Φ(t,s),xπ,f(.):[0,τ]→M,x∈M,t,s∈[t0,tf],τ∈\mathdsR+,Φ(t,s),xπ,f(ϵ)∈M,

where is the flow corresponding to the perturbed control , i.e. . The following lemma gives the formula of the variation of at the limit from the right . We recall that the point is called a Lebesgue point of if, ([1]):

 lims1↓t11|s1−t1|∫s1t1|u(τ)−u(t1)|dτ=0.

For any , may be modified on a set of measure zero so that all points are Lebesgue points (see [28], page 158, and [29]) in which case, necessarily, the value of any cost function is unchanged. {lemma}[[5]] For a Lebesgue time , the curve is differentiable from the right at and the corresponding tangent vector is given by

 ddϵΦ(t1,s),xπ,f|ϵ=0=f(x(t1),u1)−f(x(t1),u(t1))∈Tx(t1)M. (13)

The tangent vector is called the elementary perturbation vector associated to the perturbed control at . The displacement of the tangent vectors at is given by the push-forward of the vector field , see sections below.

### 3.3 Adjoint Processes and the Hamiltonian

In this section we present the definitions of the adjoint process and the Hamiltonian function which appear in the statement of the Minimum Principle. In the case , by the smoothness of we may define the following system of differential equations:

 ˙λT(t)=−λT(t)∂f∂x(x(t),u(t)),t∈[t0,tf],x(t0)∈\mathdsRn. (14)

The matrix solution of where gives the transformation between tangent vectors on the state trajectory from time to (see [21]), in other words, considering as a tangent vector at the push-forward of under is

 v2=TΦ(t2,t1)f(v1)=φ(t2−t1)v1,v1∈Tx(t1)\mathdsRn≃\mathdsRn,t1,t2∈[t0,tf].

Evidently the vector is the solution of the following differential equation:

 ˙v(t)=∂f∂x(x(t),u(t))v(t),v(0)=v0,v(t)∈Tx(t)\mathdsRn≃\mathdsRn. (15)

A key feature of the solution of (14) is that along , remains constant since

 ddt(λT(t)v(t))=˙λT(t)v(t)+λT(t)˙v(t)=−λT(t)∂f∂xv(t)+λT(t)∂f∂xv(t)=0.

For a general Riemannian manifold , the role of the adjoint process is played by a trajectory in the cotangent bundle of , i.e. . As in the definition of the tangent lift, we define the cotangent lift which corresponds to the variation of a differential form (see [40]):

 fT∗,u(αx):=ddt|t=−sT∗Φ(−t,s)fu(αx),αx∈T∗xM, (17)

where . As in (9), in the local coordinates, , of , we have

 fT∗,u(x,p)=[fu,i(x)∂∂xi−(∂fu,i∂xjpj)∂∂pi]ni,j=1, (18)

where is the pull back of applied to differential forms . The minus sign in front of in (17) is due to the fact that pull backs act in the opposite sense to push forwards, therefore the variation of a covector at depends upon which notationally corresponds to , see [40]. The following lemma gives the connection between the cotangent lift defined in (17) and its corresponding flow on . Let (, the pull back of , whose existence is guaranteed since is a diffeomorphism, see [40].

{lemma}

[[5]] Consider as a time dependent vector field on , then the flow , satisfies

 Γ(t,s,(x,p))=(Φ(t,s)f(x),(T∗Φ(t,s)f)−1(p)),(x,p)∈T∗M, (19)

and is the corresponding integral flow of .

We now generalize (14) and (15) to differentiable manifolds. Along a given trajectory , the variation with respect to time, , is an element of . The vector field defined in (17) is thus the mapping , which generalizes (14) to a mapping from to . The generalization of (3.3) to is given in the following proposition. {proposition}[[5]] Let be a time dependent vector field giving rise to the associated pair ; then along an integral curve of on

 ⟨Γ,Ψ⟩:I→\mathdsR,

is a constant map, where is an integral curve of in and is an integral curve of in .

The integral curves and are the generalizations of and in (15) and (3.3) in to the case of a differentiable manifold . The corresponding variation of the elementary tangent perturbation in Lemma 3.2 is given in the following proposition. {proposition}[[5]] Let be the integral curve of with the initial condition , then

 ddϵΦ(t,t1),xπ,f|ϵ=0=Ψ(t),t∈[t1,tf].

By the result above and Lemma 2 we have

 ddϵΦ(t,t1),xπ,f|ϵ=0=TΦ(t,t1)f([f(x(t1),u1)−f(x(t1),u(t1))])∈Tx(t1)M.

### 3.4 Hamiltonian Functions and Vector Fields

Here we recall the notions of Hamiltonian vector fields (see e.g. [3]), which were employed in [1] to obtain a Minimum Principle for optimal control problems in a geometrical framework.

For an optimal (non-hybrid) control problem defined on the state manifold , with controlled vector field , the Hamiltonian function for the Mayer problem is defined as:

 H:T∗M×U→\mathdsR, (20)
 H(p,x,u)=⟨p,f(x,u)⟩,p∈T∗xM,f(x,u)∈TxM. (21)

In general, the Hamiltonian is a smooth function with an associated Hamiltonian vector field defined by (see [1])

 ωλ(.,→H)=dH,λ∈T∗M,

where is the symplectic form (see e.g. [16], [23]) defined on (see [1, 20]) and is the space of smooth vector fields defined on . The Hamiltonian vector field satisfies , (see [1]) where is the contraction mapping (see [19, 23]) along the vector field . In the local coordinates of , we have:

 dH=n∑i=1∂H∂pidpi+∂H∂xidxi,→H=n∑i=1∂H∂pi∂∂xi−∂H∂xi∂∂pi. (22)

So the Hamiltonain system is locally written as:

 ˙x(t)=∂H∂pi,˙p(t)=−∂H∂xi, whereλ(t)=(x(t),p(t))∈T∗M,x(t0)=x0,λ(tf)=dh(x(tf))∈T∗x(tf)M,

where

 dh=n∑i=1∂h∂xidxi∈Ω1(M). (23)

### 3.5 Pontryagin Minimum Principle

For standard (non-hybrid) optimal control problems defined on a Riemannian manifold we have the following result known as Pontryganin Minimum Principle. {theorem}[[21]] Consider an OCP satisfying hypotheses A1-A3 () defined on a Riemannian manifold . Then corresponding to an optimal control and optimal state trajectory pair, there exists a nontrivial adjoint trajectory defined along the optimal state trajectory, such that:

 H(xo(t),po(t),uo(t))≤H(xo(t),po(t),v),∀v∈U,t∈[t0,tf],

and the corresponding optimal adjoint trajectory satisfies:

 ˙λo(t)=→H(λo(t)),t∈[t0,tf].

The Minimum Principle gives necessary conditions for optimality; conditions under which the Minimum Principle is sufficient for optimality are given in [7] and [10].

## 4 The Hybrid Minimum Principle for Autonomous Impulsive Hybrid Systems

Here we consider a simple impulsive autonomous hybrid system consisting of one switching manifold. Consider a hybrid system with a single switching from the discrete state to the discrete state at the unique switching time on the optimal trajectory associated with the dynamics:

 ˙xq0(t)=fq0(x(t),u(t)),a.e.t∈[t0,ts),
 ˙xq1(t)=fq1(x(t),u(t)),a.e.t∈[ts,tf],

where and

 fqi(.,u(.)):M×[ti,ti+1)→TM,i=0,1,

together with a smooth state jump with the following action:

 xo(ts)=ζ(xo(t−s))=limt→t−sζ(x(t)),xo(t−s)∈S⊂M.

We shall assume the switching manifold is an embedded dimensional submanifold which consists of a single switching manifold (see Section 2). Following [30], the control needle variation analysis is performed in two distinct cases. In the first case, the variation is applied after the optimal switching time, therefore there is no state variation propagation along the state trajectory before the switching manifold, while in the second case, the control needle variation is applied before the optimal switching time. In this case there exists a state variation propagation along the state trajectory which passes through the switching manifold, see [30] (see Figure 1).

Recalling assumption A2 in the Bolza problem and assuming the existence of optimal controls for each pair of given switching state and switching time, let us define a function for a hybrid system with one autonomous switching, i.e. , as follows:

 v(x,t)=infu∈UJ(t0,tf,h0,u), (24)

where

 x=Φ(t−,t0)fq0(x0)∈S⊂M.

### 4.1 Non-Interior Optimal Switching States

In this subsection, we show that the optimal switching state for an MHOCP derived from a BHOCP (see (10)) cannot be an interior point of the attainable switching set for an MHOCP which is defined as

Note that the state manifold of a Mayer problem derived from a Bolza problem is where is the state manifold of the Bolza problem. In this paper, for simplicity and uniformity of notation, the state manifold and the switching manifold of a Mayer problem shall also be denoted by and respectively. {lemma} Consider an MHOCP derived from a BHOCP as in (10), (11) with a single switching from the discrete state to the discrete state at the unique switching time on the optimal trajectory and an dimensional switching manifold defined in an dimensional manifold , where is the switching manifold of the BHOCP. Then an optimal switching state at the optimal switching time cannot be an interior point of in the induced topology of from . {proof} If has empty interior in the topology induced on from the result is immediate. Assume is an interior point of , i.e. there exists an open neighbourhood of . Let us denote a coordinate system around by where corresponds to the running cost of the Bolza problem, see (10). Since the switching manifold of the MHOCP is defined by , we may choose a neighbourhood of in the induced topology of with the last coordinate free to vary in an open set in . Hence fixing , there exists such that

 yi=xoi(t−s),i=1,...,n,yn+1

which is accessible by subject to a new control , where is not necessarily equal to . Set the control ; then results in an identical state trajectory on for the Bolza problem (since the variables do not change). However, the final hybrid cost corresponding to the new switching state is

 J(t0,tf,(x0,q0);1,(u,uo))=yn+1+∫tftsl1(xo(t),uo(t))dt+h(xo(tf))

where , contradicting the optimality of ; we conclude lies on the boundary of .

However the lemma above implies that the hybrid value function defined by (24) cannot be differentiated in all directions at the optimal switching state for MHOCPs derived from BHOCPs. Hence the main HMP Theorem 4.3 for MHOCPs below applies in potential to all MHOCPs derived from BHOCPs. The general HMP statement given below employs a differential form corresponding to the normal vector to the switching manifold at the optimal switching state . Now in the special case where the value function can be differentiated in all directions at , it may be shown that for some scalar , see [35], Lemma A.1; this fact has significant implications for the theory of HMP as is shown in [33, 34, 38].

### 4.2 Preliminary Lemmas

In order to use the methods introduced in [1, 5, 21], we establish Lemma 4.2 using the perturbed control and the associated state variation at the final state . Denote by the switching time corresponding to . Note that, in general, does not necessarily intersect the switching manifold at