Optimal Control Problems in Transport Dynamics

# Optimal Control Problems in Transport Dynamics

## Abstract

In the present paper we deal with an optimal control problem related to a model in population dynamics; more precisely, the goal is to modify the behavior of a given density of individuals via another population of agents interacting with the first. The cost functional to be minimized to determine the dynamics of the second population takes into account the desired target or configuration to be reached as well as the quantity of control agents. Several applications may fall into this framework, as for instance driving a mass of pedestrian in (or out of) a certain location; influencing the stock market by acting on a small quantity of key investors; controlling a swarm of unmanned aerial vehicles by means of few piloted drones.

Keywords: Transport dynamics; optimal control problems; Wasserstein distance; functionals on measures.

AMS Subject Classification: 49J20, 49J45, 60K30, 35B37.

## 1 Introduction

In recent years several models of transport dynamics have been studied; if represents the density of a given population at time in a space location , the evolution of , whenever the total mass of the population is conserved, is described by means of the continuity equation

 ∂ρ∂t(t,x)=−divx(v(t,x)ρ(t,x)),

where is the velocity of the population motion. The vector field may depend on in a rather general way; here we are interested in the cases where

 v(t,x)=(K∗ρ)(t,x)+f(t,x),

being an external velocity field, a self-interaction kernel, and the convolution operator

 (K∗ρ)(t,x)=∫ΩK(t,x−y)dρ(t,y).

Our ambient space is a domain of , which we take bounded and regular enough; the case of unbounded domains can be treated in a similar way with some technical modifications. Models of the kind above have been widely considered in the literature; we refer for instance to [3, 7, 15, 25, 30] and to the references therein.

In the present paper we deal with an optimal control problem related to the dynamics above; more precisely, the goal is to modify the behavior of the density of the population by influencing the behavior of another population of agents interacting with , that we denote by . This means that the function above is of the form

 f(t,x)=(H∗ν)(t,x)for every (t,x)∈[0,T]×Ω,

for a given cross-interaction kernel . The resulting state equation governing our optimal control problem is

 ∂ρ∂t(t,x)=−divx(((K∗ρ)(t,x)+(H∗ν)(t,x))ρ(t,x)), (1)

with initial condition

 ρ(0,x)=ρ0(x)on Ω,

and boundary conditions

 v(t,x)⋅n(x)=0on (0,T]×∂Ω,

where

 v(t,x)=(K∗ρ)(t,x)+(H∗ν)(t,x),

with suitable convolution kernels. Notice that, by setting , equation (1) has the form of a continuity equation, where is an external velocity field.

The dynamics of is determined by the minimization of a given functional taking into account the desired behavior of as well as the cost of the control agents (whose mass is allowed to vary). It is introduced in detail in Section 4 by using the general theory of functionals defined on the space of measures, developed in [9, 10, 11]. Under rather mild assumptions on we establish the existence of solutions for the optimal control problem with cost functional subject to the PDE constraint (1).

Notice that the formulation of our control problem differs significantly, for instance, from that of mean-field games, introduced in [22], as rather than embedding decentralized control rules inside the dynamics of we introduce an external control mass that interacts with the original population with the goal to modify its behavior.

The reason to study such infinite dimensional optimal control problems instead of their discrete counterparts lies in the so-called curse of dimensionality, term introduced by Richard Bellman in [4] to describe the difficulty in solving optimization problems where the dimension of the state variable (which depends on the number of agents, in this case) is large: the goal is to compute a nearly optimal control strategy that does not depend anymore on the number of agents.

Several applications may fall into our framework; for instance

• driving a mass of pedestrian to (or out of) a certain location using a small number of stewards;

• trying to stabilize the stock market in order to avoid systemic failures, by acting on few key investors with a relatively limited amount of resources;

• computing the minimal amount of manually-controlled units such that a swarm of drones performs a given task (as, for instance, wind harvesting or the recognition of a given area).

In the present paper we do not perform numerical simulations; we want to stress that this issue presents several difficulties, mainly related to the nonlocal behavior of the governing state equations and to the nonconvexity of the cost functional. Some numerical simulations of problems of similar type have been performed in [1, 2].

After introducing the model in Section 2 and the class of admissible controls in Section 3, we state in Section 4 the optimal control problem rigorously and we study its well-posedness; some variants are also considered. Section 5 is devoted to a list of functionals falling into our framework, and Section 6 to the analysis of a natural control problem arising in pedestrian dynamics.

## 2 Preliminaries

### 2.1 The Wasserstein space of probability measures

Let ; we denote by the set of finite positive measures on , and by the set of positive measures with total mass less than or equal to . It is well-known that the class admits a metric topologically equivalent to the weak* convergence.

The space is the subset of whose elements are the probability measures on , i.e., for which . The space is the subset of whose elements have finite -th moment, i.e.,

 ∫Ω|x|pdμ(x)<+∞.

Clearly when is bounded. Finally, we denote by the subset of which consists of all probability measures with compact support.

For any and any Borel function , we denote by the push-forward of through , defined by

 f#μ(B):=μ(f−1(B))for every Borel set B of Rd2.

In particular, if one considers the projection operators and defined on the product space , for every we call first (resp., second) marginal of the probability measure (resp., ). Given and , we denote by the subset of all probability measures in with first marginal and second marginal .

On the set we consider the Wasserstein or Monge-Kantorovich-Rubinstein distance,

 Wp(μ,ν)=inf{∫R2d|x−y|pdρ(x,y) : ρ∈Γ(μ,ν)}1/p. (2)

If we have the equivalent expression for the Wasserstein distance:

 W1(μ,ν)=sup{∫Rdφ(x)d(μ−ν)(x) : φ∈\textupLip(Rd), \textupLipRd(φ)≤1},

where stands for the Lipschitz constant of on . We denote by the set of optimal plans for which the minimum is attained, i.e.,

 ρ∈Γo(μ,ν)⟺ρ∈Γ(μ,ν) and ∫R2d|x−y|pdρ(x,y)=Wp(μ,ν)p.

It is well-known that is non-empty for every , hence the infimum in (2) is actually a minimum. For more details, see e.g. [3, 30].

### 2.2 The model

Let be a finite-time horizon and let be a bounded open regular set, admitting the possibility of not being convex, i.e., may have internal “obstacles” and “walls”.

The dynamics of a conserved quantity under the effect of an external vector field is described by means of the continuity equation, given by

 ∂ρ∂t(t,x)=−divx(v(t,x)ρ(t,x)). (3)

A detailed analysis of (3) in the case can be found in [3]. To model the interaction of with the possible obstacles in , we prescribe reflecting boundary conditions of the form

 v(t,x)⋅n(x)=0on [0,T]×∂Ω,

where is the outer normal to the boundary of .

The evolution of the measure-valued curve is then given by

 ⎧⎪ ⎪ ⎪⎨⎪ ⎪ ⎪⎩∂ρ∂t(t,x)=−divx(v(t,x)ρ(t,x))in (0,T]×Ω,ρ(0,x)=ρ0(x)on Ω,v(t,x)⋅n(x)=0on (0,T]×∂Ω, (4)

where is an initial probability distribution with support contained in the interior of .

###### Remark 2.1.

Notice that, thanks to the boundary conditions and , then for all .

We now proceed to clarify our notion of solution for (4).

###### Definition 2.2.

Given and , we say that is a solution of (4) if

• is continuous with respect to the Wasserstein distance ;

• satisfies and for every it holds

 ∫T0∫Ω(∂ϕ∂t(t,x)+v(t,x)⋅∇ϕ(t,x))dρ(t,x)dt=0.

Notice that no continuity assumptions are made on the velocity field , the definition of solution above is given in the weak distributional sense. Our main interest lies in the case that has a specific dependency on , namely

 v(t,x):=(K∗ρ)(t,x)+f(t,x),for all (t,x)∈[0,T]×Rd. (5)

In the expression above, the function is an external velocity field and denotes the convolution operator

 (K∗ρ)(t,x):=∫RdK(t,x−y)dρ(t,y).

Here is a self-interaction kernel which models the self-interaction of . Several instances of such interaction kernels can be found in biology, chemistry and social sciences, see for instance [15, 16, 21, 23, 27, 29].

## 3 The class of admissible velocity fields

We now turn our attention to the solutions of system (4); we show that, under mild conditions on the functions and appearing in (5), they exist and are unique. The following results generalize those in [8], and are reported to keep track of the explicit dependencies of the constants.

We start by introducing the class of -admissible functions.

###### Definition 3.1.

Fix and . The class is the set of all functions satisfying:

1. is a Carathéodory function;

2. for all ;

3. for all .

The following result, which can be found in [20], shows that is compact with respect to a topology interacting with the convergence of measures.

###### Theorem 3.2.

Let and . For any there exists a subsequence and such that

 limk→∞∫T0⟨ϕ(t),gnk(t,⋅)−g(t,⋅)⟩dt=0, (6)

for all such that for all , for some . Here the symbol denotes the duality pairing between and its dual .

Moreover, given a compact set , if is a sequence of functions from to converging to in the Wasserstein distance, i.e.,

 limn→∞W1(μn(t),μ(t))=0for all t∈[0,T],

then for all and for all it holds

 limn→∞∫t0⟨φ,gn(s,⋅)μn(s)⟩ds=∫t0⟨φ,g(s,⋅)μ(s)⟩ds. (7)

 ∫T0⟨ψ(g(t,⋅)),μ(t)⟩dt≤liminfn→∞∫T0⟨ψ(gn(t,⋅)),μn(t)⟩dt, (8)

holds for any nonnegative convex globally Lipschitz function .

###### Proof.

It is straightforward to show that is contained within the class of Carathéodory functions satisfying

1. for almost every ;

2. for almost every ;

3. for almost every .

The result follows by Corollary 2.7, Theorem 2.10 and Theorem 2.12 of [20]. ∎

For the sake of brevity, from now on we set . The following result, whose proof is reported in the Appendix, shows that whenever and belong to the class , a solution of system (4) exists, is unique and is uniformly continuous in time: remarkably, the modulus of continuity depends only on , and .

###### Theorem 3.3.

Fix , and . If , then there exists a unique solution of system (4). Furthermore, there exist depending only on , and such that

• for every ;

• is uniformly continuous with modulus of continuity

###### Remark 3.4.

In what follows, for the sake of simplicity, we assume that the function is in , so that Theorem 3.3 applies and turns out to be Lipschitz continuous with a constant (see equation (18) in the Appendix). The more general case would provide in the sense of [3], and all the results below follow along the same lines. This assumption helps us to keep the notation compact without any loss of generality.

## 4 The variational problem

We now pass to study how to control the behavior of by means of another mass of individuals – representing, for instance, the officers and stewards of a building to be evacuated – whose evolution is obtained by the minimization of a suitable given cost functional . Formally, this means coupling the dynamics of with through an interaction kernel as follows

 ⎧⎨⎩∂ρ∂t(t)=−divx(((K∗ρ)(t)+(H∗ν)(t))ρ(t)) for t∈(0,T],ρ(0)=ρ0. (9)

Notice that system (9) is again of the form of system (4) with velocity field

 v(t,x):=(K∗ρ)(t,x)+(H∗ν)(t,x),

hence of the same nature of (5). This implies, by Theorem 3.3 and Remark 3.4, that the solutions of (9) are Lipschitz curves with a Lipschitz constant and values in , whose support is uniformly bounded in time inside , i.e., they belong to the class

 \textupLipL([0,T];P1(Ω))={ρ∈C([0,T];P1(Ω)):W1(ρ(t),ρ(s))≤L|t−s| ∀t,s∈[0,T]}.

We now make the assumption that, similarly to , also the control mass has a characteristic limit speed (or acceleration) . We thus prescribe to belong to the class

 \textupLipL′([0,T];MM(Ω))={ν∈C([0,T]; MM(Ω)): d(ν(t),ν(s))≤L′|t−s| ∀t,s∈[0,T]},

where is a metric on equivalent to the weak* topology. The modeling reason for considering as the space where the curve takes its values is that we want to allow the mass of to change over the time, up to a maximal mass (in the interpretation of as the probability distribution of stewards, we would like to change their number as the needs come).

The dynamics of is given by the minimization of a cost functional , encoding a certain goal that and have to reach, subject to system (9), which prescribes the evolution of . We assume that the cost functional

 J:\textupLipL′([0,T];MM(Ω))×\textupLipL([0,T];P1(Ω))→R∪{+∞},

that we optimize in our control problem, satisfies the following assumptions:

(J1)

is bounded from below;

(J2)

is lower semicontinuous with respect to the pointwise (in time) weak* convergence of measures, i.e., for any such that weakly* for every , it holds

 J(ν,ρ)≤liminfn→∞J(νn,ρn).

Some examples of interesting cost functionals satisfying (J1) and (J2) are listed in Section 5. We can now state the optimal control problem we study henceforth.

###### Problem 1.

Given and , solve

 min{J(ν,ρ):(ν,ρ)∈\textupLipL′([0,T];MM(Ω))×\textupLipL([0,T];P1(Ω))}

subject to the state equation (9).

It is straightforward to see that Problem 1 can then be rewritten as

 min{J(ν,ρ)+χA(ν,ρ):(ν,ρ)∈\textupLipL′([0,T];MM(Ω))×\textupLipL([0,T];P1(Ω))} (10)

where is the characteristic function (with value on and elsewhere) of the set

###### Lemma 4.1.

The set is closed under the topology of pointwise weak* convergence of measures. Therefore, also satisfies the assumption (J2).

###### Proof.

Take such that weakly* for every . By Definition 2.2, to prove that we have to show that and for every

 ∫T0∫Ω(∂ϕ∂t(t,x)+((K∗ρ)(t,x)+(H∗ν)(t,x))⋅∇ϕ(t,x))dρ(t,x)dt=0.

The fact that simply follows from the assumption that for every and the uniqueness of the weak* limit. Since we have that for every

 ∫T0∫Ω(∂ϕ∂t(t,x)+((K∗ρn)(t,x)+(H∗νn)(t,x))⋅∇ϕ(t,x))dρn(t,x)dt=0.

Hence, by the weak* convergence, the regularity of the test functions and the dominated convergence theorem, we obtain

 limn→∞∫T0∫Ω∂ϕ∂t(t,x)dρn(t,x)dt=∫T0∫Ω∂ϕ∂t(t,x)dρ(t,x)dt.

For the same reasons, and the continuity of , we have

 limn→∞∇ϕ(t,x)⋅∫ΩH(t,x−y)dνn(t,y)=∇ϕ(t,x)⋅∫ΩH(t,x−y)dν(t,y)

for every , while its admissibility and the uniform compact support of the measures gives us the upper bound

 ∣∣∣∇ϕ(t,x)⋅∫ΩH(t,x−y)dνn(t,y)∣∣∣≤Mℓ(t)(1+δ(Ω))sup(t,x)∈[0,T]×Ω|∇ϕ(t,x)|, (11)

where we have set

 δ(Ω)=sup{|x| : x∈Ω}. (12)

Notice that the bound (11) belongs to (notice that this also holds if simply belongs to ), and that the same holds true with in place of , in place of and in place of . By the dominated convergence theorem and the compact support of the measures, we obtain finally

which concludes the proof. ∎

The compactness of the set , where is endowed with the metric, was already discussed in the proof of Theorem 3.3. The following result shows that also the set , where is equipped with the metric of the weak* convergence, is compact.

###### Lemma 4.2.

Consider equipped with the metric of weak* convergence. Then, the set is compact with respect to the uniform convergence.

###### Proof.

Without loss of generality, assume . Notice that for a positive measure its total variation coincides with itself; hence, the set coincides with the closed unit ball , which is compact in the weak* topology from the Banach-Alaoglu Theorem. Therefore, consider a sequence . Similarly to the proof of Theorem 3.3, we have that

• is equicontinuous and is contained in a closed subset of the set , because of the uniform bound on the Lipschitz constant;

• for every , the sequence is relatively compact in equipped with the weak* topology, since this metric space is compact.

Hence, an application of the Ascoli-Arzelà Theorem for functions with values in a metric space concludes the proof. ∎

###### Proof.

We prove the statement by means of the direct methods in the Calculus of Variations. Rewrite Problem 1 in the form (10) and notice that from the hypothesis (J1) the functional is bounded from below. We can thus consider a minimizing sequence , which by Lemma 4.2 admits a subsequence uniformly (and thus pointwise) converging to some . By Lemma 4.1 and hypothesis (J2), the functional is lower semicontinuous with respect to the pointwise weak* convergence, and this concludes the proof. ∎

For use below, in particular in Section 6, we mention the following remark.

###### Remark 4.4.

In several applications, the cost functional as well as the PDE constraint (9) may depend on some extra term , where is a function space with topology . Whenever it is possible to rewrite the problem as

 min{F(ν,ρ,f):(ν,ρ,f)∈\textupLipL′([0,T];MM(Ω))×\textupLipL([0,T];P1(Ω))×X}

for a certain cost functional , then Theorem 4.3 is still valid provided that

• is compact with respect to the topology ;

• for every , the functional is lower semicontinuous with respect to the topology . Note that in this formulation the state equation is included in the functional as done in (10).

## 5 The cost functional J

In this section we show some examples of cost functionals appearing in Problem 1. Clearly, any linear combination of the following terms is still a valid functional for which Theorem 4.3 applies. We start with a preliminary result on lower semicontinuous functionals defined on (see for instance [12] or Lemma 1.6 of [28]).

###### Proposition 5.1.

Let be a metric space and let be a lower semicontinuous function bounded from below. Then the functional defined by

 J(μ)=∫Xf(x)dμ(x)for every μ∈MM(X)

is lower semicontinuous with respect to the weak* convergence of measures.

In several applications of optimal control in opinion dynamics and crowd motion (see [8, 19]), the functional to be minimized consists of a Lagrangian term of the form

 J1(ν,ρ)=∫T0L(ν(t),ρ(t))dt.

The Lagrangian may prescribe, for instance, a certain mutual interaction between the measures and , or one can use to model the distance to the basin of attraction of the target configurations of the measure , as in [14]. In this case, in order for to be lower semicontinuous with respect to the pointwise weak* convergence, the lower semicontinuity of with respect to the weak* convergence suffices. Some interesting particular cases of the functional above are listed below.

1. Our decision to let the mass of vary comes from the choice to allow the optimization of the quantity of control agents, in accordance with the goal to achieve. We can model the cost of employing a quantity of agents at time by considering the Lagrangian

 L(ν(t),ρ(t))=∫Ωf(t,x)dν(t,x).

Here is a lower semicontinuous function, for instance

 f(t,x)=c(t)|x−x0|p,

where , is a nonnegative integrable function, and represents a sort of manpower storage room.

The addition of the term with the above choice of to the general cost functional can be used to penalize the mass of .

2. A common example is the one where we require the dynamics of the measure to satisfy a specific feature, like the collapse of one of its moments or marginals. An example is given by alignment models like the Cucker-Smale one (see [14]), where one is interested in a population of individuals function of a spatial variable and a consensus or velocity variable . The goal of a control strategy, in this case, is to force the alignment of the group, which in term of the state variables means that all the velocities ’s tend to coincide. This is ensured by minimizing at every instant and for any individual with velocity the square distance between and the mean :

 ∫R2d∣∣v−¯¯¯v(t)∣∣2dρ(t,x,v).

In more general terms, given a projection (with , then the Lagrangian may be of the form

 L(ν(t),ρ(t))=∫Ω∣∣∣x−∫Ωydπ#ρ(t,y)∣∣∣2dπ#ρ(t,x).

Indeed, denoting by the center of mass of the projection of at time , the minimization of the above Lagrangian leads to the convergence of to the measure For a similar problem in the context of the Hegselmann-Krause model for opinion formation, see [31].

3. Another relevant particular case of the functional is given by

 J2(ν,ρ)=∫T0∫Cdρ(t,x)dt (13)

where is a given subset of . By Proposition 5.1, the functional above is pointwise weakly* lower semicontinuous as soon as is an open set. This also happens when is closed (which is the most common case in optimal evacuation problems), with Lebesgue negligible, and is in . Minimizing this functional corresponds to the evacuation of from the set .

4. In the cases where a desired final configuration of is given, we may use one of the following functionals

 J3(ν,ρ)=∫T0W1(ρ(t),¯¯¯ρ)dtorJ4(ν,ρ)=W1(ρ(T),¯¯¯ρ)

to force to adhere to . As already noticed, the distance is continuous with respect to the weak* convergence whenever the measures have uniformly compact support.

There is a slight difference between the two functionals above. The first one prescribes a somewhat greedy approach for the optimization procedure, by asking that the distance cannot be too large on average (in time). The second one, instead, allows a greater freedom in the behavior of which is only prescribed to have a final distribution as close as possible to . However, one should always be aware that a greater liberty may translate into a more difficult numerical implementation (since the number of degrees of freedom may grow out of control), which is an ingredient that should play a relevant role in the design of any control problem.

We remark here the connection with the Benamou-Brenier formulation of transport problems [6] where the initial and the final configurations and are both prescribed and the kinetic energy of the system has to be minimized.

5. The adoption of the space is made for the sake of generality. However, it is often the case that the dynamics of possesses some extra structure which let belong to a narrower subset of . For instance, when represents a conserved quantity in time, we already noticed that its evolution can be described by means of a continuity equation like

 ∂ν∂t(t,x)=−divx((v(t,x)+u(t,x))ν(t,x)),

where only the component of the external velocity field is optimized (specifically, could be the drift depending on the interaction with the other agents, like in (5), hence stands for the optimal strategy subject to the underlying dynamics). We come back to this case in Section 6. Several families of PDEs (Fokker-Planck, Vlasov, etc) determine subsets which are closed under the pointwise weak* convergence of measures: in all those cases, the functional

 J5(ν,ρ)=χB(ν,ρ)

is lower semicontinuous with respect to this topology, and can be used in the context of Problem 1.

6. Problem 1 does not prescribe any constraint on the dynamics of the control agents , except for its maximal speed. One way to impose extra conditions on the curve can be by means of the restriction to a closed subset of , like the set of solutions of a particular PDE, as argued above. This is a very powerful tool from the modeling point of view, but it may create extra difficulties if we want to identify the minimizers of , unless optimality conditions are available, as for instance in [8]. Another way to give more structure to the dynamics of is to embed the desired features of it inside the functional , like

 J6(ν,ρ)=∫T0∫Ω2Q(x,y)dν(t,x)dν(t,y)dt,

where is a lower semicontinuous function. The functional is clearly lower semicontinuous with respect to pointwise weak* convergence by Proposition 5.1, and forces to self-interact via the kernel . For instance, if we want to avoid high concentrations of , we could opt for kernels like

 Q(x,y)=−|x−y|p,orQ(x,y)=|x−y|−p,

while if, on the contrary, we want to remain as concentrated as possible, we may consider

 Q(x,y)=|x−y|p.
7. Another interesting class of functionals to model the cost of is given by

 ∫T0[∫Ωh(t,νa(t,x))dx+∑x∈Ωk(ν#(t,x))]dt,

where and are respectively the absolutely continuous and atomic parts of (hence, the sum over all reduces to the atoms of ), while the function (resp. ) is nonnegative and convex (resp. concave), with the properties and such that the slopes of at infinity and of at are . This class of functionals over the measures has been studied in [9], where it is proved their weak* lower semicontinuity. To this class belongs for instance the Mumford-Shah functional (see [24]), that can be obtained by taking

 h(s)=s2,k(s)=1R∖{0}(s)={0if% s=01otherwise.

Another interesting choice of the functions and is

 h(s)=χ{0}(s)={0if s=0+∞otherwise,k(s)=1R∖{0}(s),

which gives

 J7(ν,ρ)=∫T0H0(ν(t))dt,

being the counting measure. This functional has a drastic effect: it forces the measure to be discrete at every instant, i.e.,

 ν(t)=N∑i=1δxi(t),for some xi(t)∈Ω,

and minimizes the number of atoms it is supported on. However, the points cannot vary “too wildly” on in time since : indeed, belonging to this class implies that, for any , the atoms of and of must satisfy

 d(N∑i=1δxi(t),N′∑j=1δyj(s))≤L′|t−s|.

## 6 A control problem in pedestrian dynamics

In this section, we consider a rather natural control problem arising in pedestrian dynamics, that can be reformulated as Problem 1 with a specific choice of the cost functional .

###### Problem 2.

Given , , and , solve