Relaxation Methods for Mixed-Integer Optimal Control of PDEs

# Relaxation Methods for Mixed-Integer Optimal Control of Partial Differential Equations

Falk M. Hante and Sebastian Sager
November 22, 2012
###### Abstract.

We consider integer-restricted optimal control of systems governed by abstract semilinear evolution equations. This includes the problem of optimal control design for certain distributed parameter systems endowed with multiple actuators, where the task is to minimize costs associated with the dynamics of the system by choosing, for each instant in time, one of the actuators together with ordinary controls. We consider relaxation techniques that are already used successfully for mixed-integer optimal control of ordinary differential equations. Our analysis yields sufficient conditions such that the optimal value and the optimal state of the relaxed problem can be approximated with arbitrary precision by a control satisfying the integer restrictions. The results are obtained by semigroup theory methods. The approach is constructive and gives rise to a numerical method. We supplement the analysis with numerical experiments.

F.M. Hante is with Department of Mathematics, University of Erlangen-Nuremberg, Cauerstraße 11, 91058 Erlangen, Germany. Phone: +49 (0)9131 85-67128, hante@math.fau.de.
S. Sager is with Institute of Mathematical Optimization, Otto-von-Guericke University, Universitätsplatz 2, 39106 Magdeburg, Germany. Phone: +49 (0)391 6718745, sager@ovgu.de

## 1. Introduction and Problem Formulation

The factoring of decision processes interacting with continuous evolution plays an important role in model-based optimization for many applications. For example, when optimally controlling chemical processes there are often both continuous decisions such as inlet and outlet flows, as well as discrete decisions, such as the operation of on–off valves and pumps that may redirect flows within the reactor, [21, 14]. Such mixed-integer optimal control problems are therefore studied in different communities with different approaches. Most of these approaches address problems that are governed by systems of ordinary differential equations in Euclidean spaces, see [22] for a survey on this topic.

Total discretization of the underlying system obviously leads to typically large mixed-integer nonlinear programs. Hence, relaxation techniques have become an integral part of efficient mixed-integer optimal control algorithms, either in the context of branch-and-bound type methods or, more directly, by means of nonlinear optimal control methods combined with suitable rounding strategies. An important result is that the solution of the relaxed problem can be approximated with arbitrary precision by a solution fulfilling the integer requirements [23].

In this paper, we extend such relaxation techniques to problems that are governed by certain systems of partial differential equations. Motivating applications are for example to switch between reductive and oxidative conditions in order to maximize the performance in a monolithic catalyst [27], port switching in chromatographic separation processes [5, 14], or to optimize switching control within photochemical reactions [26]. Our problem setting also includes the switching control design in the sense that systems are equipped with multiple actuators and the optimizer has to choose one of these together with ordinary controls for each instant in time.

Concerning systems involving partial differential equations, such switching control design has already been studied using several techniques: In [18, Chapter 8] optimal switching controls are constructed for systems governed by abstract semilinear evolution equations by combining ideas from dynamic programming and approximations of the value function using viscosity solutions of the Hamilton-Jacobi-Bellman equations. Switching boundary control for linear transport equations using switching time sensitivities has been studied in [9]. Exemplary for the heat equation and based on variational methods, the controllability in case of switching among several actuators has been considered in [29] and null-controllability for the one-dimensional wave equation with switching boundary control has been considered in [7]. Based on linear quadratic regulator optimal control techniques and enumeration of the integer values for a fixed time discretization, optimal switching control of abstract linear systems has been considered in [13].

Our approach is complementary to the above, as we break the computationally very expensive combinatorial complexity of the problem by relaxation. This comes at the downside of providing only a suboptimal solution and possibly at the price of fast switching but, as we will see, with arbitrary small integer-optimality gap, depending on discretization, and extensions to limit the number of switching.

We will be concerned with the following problem of mixed-integer optimal control: Minimize a cost functional

 J=ϕ(z(tf))+∫tf0ψ(z(t),u(t))dt (1)

over trajectories and control functions and subject to the constraints that is a mild solution of the operator differential equation

 {˙z(t)=Az(t)+f(t,z(t),u(t),v(t)),t∈(0,tf]z(0)=z0∈X (2)

and that the control functions satisfy

 u(t)∈Uad⊂U, v(t)∈Vad⊂V,t∈[0,tf] (3)

where , and are Banach spaces, , and are normed linear spaces, is the infinitesimal generator of a strongly continuous semigroup on , is a fixed real number, , and are given functions, is some subset of and is a finite subset of . This setting is for the most part classical, except for the assumptions on .

We will refer to the above infinite-dimensional dynamic optimization problem as mixed-integer optimal control problem, short (MIOCP), and to the control function as a mixed-integer control. This accounts for the fact that we do not impose restrictions on the set while we can always identify the finite set of the feasible control values for with a finite number of integers

 Vad={v1,…,vN}≃{1,…,N}. (4)

Moreover, the operator differential equation (2) is an abstract representation of certain initial-boundary value problems governed by linear and semilinear partial differential equations, see, e. g., [20].

The existence of an optimal solution of the problem (MIOCP) depends, inter alia, on the spaces , and where we seek , and , respectively. Common choices are, for , the spaces of square integrable (), piecewise -times differentiable () or (piecewise) -times weakly differentiable () functions and, for , the spaces of essentially bounded () or piecewise constant (PC) functions . We defer these considerations by assuming later that there exists an optimal solution of a related (to a certain extent convexified and relaxed) optimal control problem and present sufficient conditions guaranteeing that the solution of the relaxed problem can be approximated with arbitrary precision by a solution satisfying the integer restrictions. We do only assume that , and to ensure that certain quantities in problem (MIOCP) are well-defined.

This relaxation method becomes most easily evident from writing the problem (MIOCP) using a differential inclusion, that is, minimize (1) subject to the constraints that is a solution of

 {˙z(t)∈Az(t)+{f(t,z(t),u(t),vi):vi∈Vad},t∈(0,tf]z(0)=z0. (5)

and satisfies

 u(t)∈Uad,  t∈[0,tf].

It is well known that, under certain technical assumptions, the solution set of (5) is dense in the solution set of the convexified differential inclusion

 {˙z(t)∈Az(t)+¯¯¯¯¯co{f(t,z(t),u(t),vi):vi∈Vad},t∈(0,tf]z(0)=z0, (6)

where denotes the closure of the convex hull. This is proved in [6] for the case when is a separable Banach space and in [2] for non-separable Banach spaces. While these results rely on powerful selection theorems, our main contribution is a constructive proof based on discretization, giving rise to a numerical method at the prize of additional regularity assumptions.

We will see that the advantage of such a relaxation method is that the convexified problem, using a particular representation of (6), falls into the class of optimal control problems with partial differential equations without integer-restrictions. The already known theory, in particular concerning existence, uniqueness and regularity of optimal solutions as well as numerical considerations such as sensitivities, error analysis for finite element approximations, etc., can thus be carried over to the mixed-integer problem under consideration here. We also discuss the possibility to include state-constraints for example enforcing time-periodicity constraints

 z(tf)=z(0), (7)

as occurring in chromatographic separation processes [5, 14]. The disadvantage of this approach is that we target at a solution that is only suboptimal (though with arbitrary precision) and that switching costs, a standard regularization of mixed-integer problems to prevent chattering solutions, or additional combinatorial constraints can lead to larger optimality/feasibility gaps. Nevertheless, we will show how a-priori bounds for such a gap can be obtained when constraints on the number of switches are incorporated.

The framework we use for the analysis here will be semigroup theory. Recall that for given and given control functions , the mild solution of the state equation (2) is given by a function satisfying the variation of constants formula

 z(t)=T(t)z0+∫t0T(t−s)f(s,z(s),u(s),v(s))ds,0≤t≤tf (8)

in the Lebesgue-Bochner sense. This abstract setting covers in particular the usual setup for weak solutions of linear parabolic partial differential equations with distributed control on reflexive Banach spaces where arises from a time-invariant variational problem, see [1, Section 1.3].

We include in our analysis explicitly the possibility to approximate the state equation (2) and say that is an -accurate solution of (2) if is the mild solution and for all . Accordingly, we define for a given the set of -admissible solutions for (MIOCP) as

 Ξε:={(u,v,z)∈U[0,tf]×V[0,tf]×X[0,tf]:u,v satisfy (???) (9) and z=z(u,v) is an ε-accurate solution of (???)}.

Further, we denote throughout the paper by the space of -valued functions defined on the interval and being piecewise once-weakly differentiable with a piecewise defined weak derivative that is square-integrable in the Lebesgue-Bochner sense. Consistently, we denote by the space of -valued functions defined on the interval being piecewise Hölder-continuous with a Hölder-constant . In both constructions, piecewise means that there exists a finite partition of the interval

 0=τ0<τ1<τ2<…<τKtf<τKtf+1=tf (10)

so that the function has the respective regularity on all intervals , . We denote by the norm on and by the operator norm induced by . Further, we denote by and the norm of and , respectively. For simplicity of notation, we also define for all , denoting the identity on .

The paper is organized as follows. In Section 2, we present details of the relaxation method and the main results concerning estimates of the approximation error. In Section 3, we discuss extensions of the method to incorporate certain combinatorial constraints. In Section 4, we discuss applications for linear and semilinear equations and present numerical results for the heat equation with spatial scheduling of different actuators and a semilinear reaction-diffusion system with an on-off type control. In Section 5, we conclude with some additional remarks and point out open problems.

## 2. Relaxation Method

Consider the following problem involving a particular representation of the convexified differential inclusion (6)

 minimize[ω,α,y]∈U[0,tf]×~V[0,tf]×X[0,tf] J=ϕ(y(tf))+∫tf0ψ(y(t),ω(t))dts.\,t. (11a) ˙y(t)=Ay(t)+N∑i=1αi(t)f(t,y(t),ω(t),vi),t∈(0,tf] (11b) y(0)=y0:=z0 (11c) N∑i=1ω(t)∈Uad, t∈[0,tf] (11d) α(t)=(α1(t),…,αN(t))∈[0,1]N, t∈[0,tf] (11e) N∑i=1αi(t)=1, t∈[0,tf], (11f)

where we write for , respectively, to emphasize the relaxation of the original problem (MIOCP). Here, .

Observe that the control functions take values on the full interval , but that any optimal controls of (11) yields an optimal mixed-integer control

 [u∗,v∗]:=[ω∗,N∑i=1α∗ivi] (12)

of problem (MIOCP) if for almost every . However, it is not very difficult to construct examples where for on some interval of positive measure. It is only in some special cases where optimality of implies that takes only values on the boundary of its feasible set. For examples where this property, known as the bang-bang principle, can be verified in the context of partial differential equations, see [28, Section 3.2.4] and the references therein. Moreover, the relaxed problem (11) can in most applications only be solved approximately.

Therefore, consider the following hypothesis.

• Problem (11) has an optimal solution in .

Under this assumption, we will accept -accurate optimal solutions of the relaxed problem (11) and propose in Algorithm 1 an iterative procedure to obtain from these solutions a mixed-integer control taking values in . We then show in Theorem 3 under certain technical assumptions that the optimal value and the optimal state of problem (11) can be approximated by the proposed procedure with arbitrary precision. To make our terminology precise, we include the following definition.

###### Definition 2.

Given some , we say that is an -accurate optimal solution of (11) if is an -accurate solution of (11b)–(11c) with and , the constraints (11d)–(11f) hold with and and

 J(ωε,αε,yε)≤infω,α,y s.t. (???)--(???)J(ω,α,y)+ε. (13)

So any -accurate optimal solution of the relaxed problem (11) is admissible with respect to the control constraints on and , -close admissible with respect to the state variable and the corresponding value of the cost function is -close to the optimal value of problem (11).

Now, consider Algorithm 1 on page 1 to obtain a mixed-integer control for problem (MIOCP). The main result is the following.

###### Theorem 3.

Assuming (H), let denote an optimal solution of the relaxed problem (11) and assume that the following assumptions hold true.

• The functions , , and satisfy the Lipschitz-estimates

 |ϕ(y1)−ϕ(y2)| ≤η∥y1−y2∥X, |ψ(y1,ω1)−ψ(y2,ω2)| ≤ξ(∥y1−y2∥X+∥ω1−ω2∥U), ∥∥f(t,y1,ω1,vi)−f(t,y2,ω2,vi)∥∥X ≤L(∥y1−y2∥X+∥ω1−ω2∥U),

for all , and with positive constants , and .

• For all and the function

 s↦T(t−s)f(s,y∗(s),ω∗(s),vi)

is in and there exists a positive constant such that

 ∥∥∥dds(T(t−s)f(s,y∗(s),ω∗(s),vi))∥∥∥X≤Cia.\,e. in 0

Let .

• For all , there exists a positive constant such that

 supt∈[0,tf]∥f(t,y∗(t),ω∗(t),vi)∥X≤Mi.

Let .

• The solution is stable in the sense that there exists a positive constant such that

 ∥ω∗(t)−ω(t)∥U+∥α∗(t)−α(t)∥RN≤ (14) CJ|J(ω∗,α∗,y(ω∗,α∗))−J(ω,α,y(ω,α))| a.\,e. in (0,tf)

for in some neighborhood of in .

Moreover, assume that and that the sequence in Algorithm 1 is such that with

 Δtk=maxi=1,…,nk{tki−tki−1}. (15)

Define the constants , , and , where and the constants , , , , and are given by hypothesis (H)(H). Then, defined by Algorithm 1 is in for all and satisfies the estimates

 ∥y∗(t)−zk(t)∥X≤C1εk+C2(N−1)Δtk,t∈[0,tf], (16)

and

 |J(ω∗,α∗,y∗)−J(uk,vk,zk)|≤C3εk+C4(N−1)Δtk. (17)

In particular, Algorithm 1 terminates in a finite number of steps with an -feasible mixed-integer solution of Problem (MIOCP) satisfying the estimate

 |J(ω∗,α∗,y∗)−J(u∗,v∗,z∗)|≤ε, (18)

where is chosen arbitrarily in step :.

Before we prove Theorem 3, we first prove a result saying that the deviation of two mild solutions in equipped with the uniform norm can be estimated in terms of the absolute value of the integrated difference of two linearly entering control functions. This estimate is non-standard and generalizes the result in [23] to a Banach space setting, noting that the absolute value of the integrated difference is not a norm and in particular not comparable to the -norm as a natural choice. This estimate, together with an approximation result for this integrated difference is the key ingredient in order to prove all a-priori estimates needed for the proof of Theorem 3.

###### Lemma 4.

Let and . Suppose that is a feasible solution of the relaxed problem (11) and assume that the hypotheses (H)(H) of Theorem 3 hold true. Let be such that

 maxi=1,…,Nsupt∈[0,tf]∣∣∣∫t0α∗i(τ)−βi(τ)dτ∣∣∣≤ε (19)

and let be the mild solution of (A) in Algorithm 1 with , , and . Then

 ∥y∗(t)−z(t)∥X≤((M+Ct)e¯MLt)ε,t∈[0,tf]. (20)
###### Proof.

Fix and set, for the sake of brevity, and . Recalling (10) and using hypothesis (H), let be the set of partition points of the functions as objects in , , so that and . From the definition of the mild solutions for (11) and (A), we have

 δ(t)=∥∥ ∥∥N∑i=1∫t0T(t−s)fi(s,y∗(s))α∗i(s)−T(t−s)fi(s,z(s))βi(s)ds∥∥ ∥∥X.

Adding under the integral, applying the triangular inequality and rearranging terms this yields

 δ(t) ≤N∑i=1∥∥∥∫t0T(t−s)[fi(s,y∗(s))−fi(s,z(s))]βi(s)ds∥∥∥X +N∑i=1∥∥ ∥∥K∑k=0∫τk+1τkT(t−s)fi(s,y∗(s))[α∗i(s)−βi(s)]ds∥∥ ∥∥X.

Now using integration by parts in the second part, we obtain

 δ(t) ≤N∑i=1∥∥∥∫t0T(t−s)[fi(s,y∗(s))−fi(s,z(s))]βi(s)ds∥∥∥X +N∑i=1∥∥∥K∑k=0(T(t−τk+1)fi(τk+1,y∗(τk+1))∫τk+10α∗i(s)−βi(s)ds −T(t−τk)fi(τk,y∗(τk))∫τk0α∗i(s)−βi(s)ds −∫τk+1τkdds(T(t−s)fi(s,y∗(s)))∫s0α∗i(ϑ)−βi(ϑ)dϑds)∥∥∥X.

Then by rearranging terms, noting that the appearing telescopic sum evaluates as

 K∑k=0(T(t−τk+1)fi(τk+1,y∗(τk+1))∫τk+10α∗i(s)−βi(s)ds −T(t−τk)fi(τk,y∗(τk))∫τk0α∗i(s)−βi(s)ds) =fi(t,y∗(t))∫t0α∗i(s)−βi(s)ds

because of , , and , and by applying the triangular inequality this estimate simplifies to

 δ(t) ≤N∑i=1∫t0∥T(t−s)∥L(X)∥fi(s,y∗(s))−fi(s,z(s))∥X|βi(s)|ds +N∑i=1∥∥fi(t,y∗(t))∥∥X∣∣∣∫t0α∗i(s)−βi(s)∣∣∣ds +N∑i=1∫t0∥∥∥dds(T(t−s)fi(s,y∗(s)))∥∥∥X∣∣∣∫s0α∗i(ϑ)−βi(ϑ)dϑ∣∣∣ds.

Then, by definition of , and the constant , the definition of the constants , and in hypotheses (H)–(H), the assumption (19) and the fact that , this yields

 δ(t) ≤¯ML∫t0δ(s)ds+Mε+Ctε.

Finally, using the Gronwall lemma and rearranging terms, we obtain the desired estimate

 δ(t)≤((M+Ct)e¯MLt)ε.

∎∎

Next, we recall from [23] the following result on integral approximations.

###### Lemma 5.

Let be a measurable function satisfying for all . Define a piecewise constant function by

 βi(t)=pi,j,t∈[tj,tj+1), i=1,…,N, j=0,…,n−1 (21)

where for all , , is defined by (A) in Algorithm 1 with . Then it holds for

1. for all ,

2. for all .

###### Proof.

See Theorem 5 of [23]. ∎

With the above two results we are now in the position to prove Theorem 3.

###### Theorem 3.

Let the assumptions of Theorem 3 hold true. First we show that the sequence obtained by Algorithm 1 is -feasible for the problem (MIOCP). We have for all by construction of in step :, , , by construction in step : so , , because for all and as seen from (A) and (A). By construction in step :, is an -accurate solution of (A). Thus, recalling (9), for all .

Next, we show (16). To this end, let , and denote the mild solutions of (11b), (11c) with the respective controls. The stability assumption (H) and the continuity assumption (H) implies that

 ∥α∗(t)−αk(t)∥RN+∥ω∗(t)−ωk(t)∥U (22) ≤CJ|J(ω∗,α∗,y(ω∗,α∗))−J(ωk,αk,y(ωk,αk))| ≤CJ(|J(ω∗,α∗,y(ω∗,α∗))−J(ωk,αk,yk)|+ |J(ωk,αk,yk)−J(ωk,αk,y(ωk,αk))|) ≤CJ(εk+|ϕ(yk(tf))−ϕ(y(tf;ωk,αk))|+ ∫tf0|ψ(yk(t),ω