Sparse Control of Multiagent Systems

Sparse Control of Multiagent Systems

Abstract

In recent years, numerous studies have focused on the mathematical modeling of social dynamics, with self-organization, i.e., the autonomous pattern formation, as the main driving concept. Usually, first or second order models are employed to reproduce, at least qualitatively, certain global patterns (such as bird flocking, milling schools of fish or queue formations in pedestrian flows, just to mention a few). It is, however, common experience that self-organization does not always spontaneously occur in a society. In this review chapter we aim to describe the limitations of decentralized controls in restoring certain desired configurations and to address the question of whether it is possible to externally and parsimoniously influence the dynamics to reach a given outcome. More specifically, we address the issue of finding the sparsest control strategy for finite agent-based models in order to lead the dynamics optimally towards a desired pattern.

1 Introduction

The autonomous formation of patterns in multiagent dynamical systems is a fascinating phenomenon which has spawned an enormous wealth of interdisciplinary studies: from social and economic networks battiston (); currarini2009economic (), passing through cell aggregation and motility camazineselforganization (); kese70 (); KocWhi98 (); be07 (), all the way to coordinated animal motion MR2507454 (); ChuDorMarBerCha07 (); cristiani2010modeling (); couzinlaneformation (); couzin2005N (); CS (); Niw94 (); PE99 (); ParVisGru02 (); Rom96 (); TonTu95 (); YEECBKMS09 () and crowd dynamics albi2015invisible (); cristiani2011multiscale (); CucSmaZho04 (); MR2438215 (). Beyond biology and sociology, the principles of self-organization in multiagent systems are employed in engineering and information science to produce cheap, resilient, and efficient squadrons of autonomous machines to perform predefined tasks arvin2014development () and to render swarms of animals reynolds1987flocks () and hair/fur textures in CGI animations pixarhair (). The scientific literature on the subject is vast and ever-growing: the interested reader may be addressed to bak2013nature (); CCH13 (); cafotove10 (); viza12 () and references therein for further insights on the topic.

A common feature of all those studies is that self-organization is the result of the superimposition of binary interactions between agents amplified by an accelerating feedback loop. This reinforcement process is necessary to give momentum to the multitude of feeble local interactions and to eventually let a global pattern appear. Typically, the strength of such interaction forces is a function of the “social distance” between agents: for instance, birds align with their closest neighbors parisi08 () and people agree easier with those who already conform to their beliefs krause02 (). Some of the forces of the system may be of cohesive type, i.e., they tend to reduce the distance between agents: whenever cohesive forces have a comparable strength at short and long range, we call these systems heterophilious; if, instead, there is a long-range bias we speak of homophilious societies motsch2014heterophilious (). Heterophilious systems have a natural tendency to keep the trajectories of the agents inside a compact region, and therefore to exhibit stable asymptotic profiles, modeling the autonomous emergence of global patterns. On the other hand, self-organization in homophilious societies can be accomplished only conditionally to sufficiently high levels of initial coherence that allow the cohesive forces to keep the dynamics compact birdsofafeather (). Being such systems ubiquitous in real life (e.g., see kirman2007marginal ()), it is legitimate to ask whether – in case of lost cohesion – additional forces acting on the agents of the system may restore stability and achieve pattern formation.

A first solution to facilitate self-organization is to consider decentralized control strategies: these consist in assuming that each agent, besides being subjected to forces induced in a feedback manner by the rest of the population, follows an individual strategy to coordinate with the other agents. However, as it was clarified in bongini2015conditional (), even if we allow agents to self-steer towards consensus according to additional decentralized feedback rules computed with local information, their action results in general in a minor modification of the initial homophilious model, with no improvement in terms of promoting unconditional pattern formation. Hence, blindly insisting and believing on decentralized control is certainly fascinating, but rather wishful, as it does not secure self-organization.

Such additional forces may eventually be the result of an offline optimization among perfectly informed players: in this case we fall into the realm of Game Theory nash1950equilibrium (); von2007theory (). Games without an external regulator model situations where it is assumed that an automatic tendency to reach “correct” equilibria exists, like the stock market. However, also in this case such an optimistic view of the dynamics is often frustrated by evidences of the convergence to suboptimal configurations hardin1968tragedy (), whence the need of an external figure controlling the evolution of the system.

For all these reasons, in the seminal papers caponigro2013sparsefake (); caponigro2015sparse () external controls with limited strength were considered to promote self-organization in multiagent systems. Notice that, in such situations, efficient control strategies should target only few individuals of the population, instead of squandering resources on the entire group at once: taking advantage of the mutual dependencies between the agents, they should trigger a ripple effect that would spread their influence to the whole system, thus indirectly controlling the rest of the agents. The property of control strategies to target only a small fraction of the total population is known in the mathematical literature as sparsity tao (); donoho (); fora10 (). The fundamental issue is the selection of the few agents to control: an effective criterion is to choose them as to maximize the decay rate of some Lyapunov functional associated to the stability of the desired pattern cohen1983absolute ().

As a paradigmatic case study, let us consider alignment models CS (); krause02 (): these are dissipative systems where imitation is the dominant feedback mechanism and in which the emerging pattern is a state where agents are fully aligned, also called consensus. For several of such models it has been proved that consensus emergence can be guaranteed regardless of the initial conditions of the system only if the alignment forces are sufficiently strong at far distance, see HaHaKim (); haskovec2014note (); in case they are not, it is easy to provide counterexamples to the emergence of a consensus. If we were to use the criterion above to select a control strategy to steer the system to consensus, it would lead to a sparse control targeting at each instant only the agent farthest away from the mean consensus parameter. Surprisingly enough, for such systems not only this strategy works for every initial condition, but the control of the instantaneous leaders of the dynamics is more convenient than controlling simultaneously all agents. Therefore if, on the one side, the homophilious character of a society plays against its compactness, on the other side, it may plays at its advantage if we allow for sparse interventions to restore consensus.

The above results have more far-reaching potential as they can be extended to non-dissipative systems as well, like the Cucker-Dong model of attraction and repulsion cucker14 (). In this model, agents autonomously organize themselves in a cohesive and collision-avoiding configuration provided that the total energy is below a certain level. The sparse control strategy is able to raise this level considerably and it is optimal in maximizing the convergence of the energy functional towards it. However in this case, due to the singular non-conservative forces in play, it may be seen that sparse controllability is in general conditional to the choice of the initial conditions, as opposed to the unconditional controllability of alignment models.

The essential scope of this review chapter is to describe in more detail the aforementioned mechanisms relating sparse controllability and pattern formation. We do so by condensing the results of the papers bofo13 (); bongini2014sparse (); bongini2015conditional (); caponigro2015sparse (), addressing the limitations of decentralized control strategies, the sparse controllability of alignment models and the one of attraction repulsion models.

2 Self-organization in dynamical communication networks

We start from the analysis of general properties of alignment models. Instances of these models are ubiquitous in nature since several species are able to interpret and instinctively reproduce certain manoeuvres that they perceive (e.g., fleeing from a danger, searching for food, performing defense tactics, etc.), see animalbehavior (). Such systems may be seen as networks of agents with oriented information flow under possible link failure or creation, and can be effectively represented by means of directed graphs with edges possibly switching in time.

A directed graph on a set of nodes is any subset of . Each pair is called an edge from to , and a directed path from to in is a sequence of edges . The graph is said to be strongly connected if for any pair of distinct nodes there is a directed path from to and a directed path from to .

When studying under which conditions networks of agents are able to self-organize, it is usually not enough to know if two nodes are connected: the strength of the interaction between them also matters. Hence, given a system of agents, for each pair of agents we denote by the weight of the link connecting with : clearly, if , is not connected to at time . The value can be seen as the relative intensity of the information exchange flowing from agent to agent at time . We shall assume for the moment that each weight function is piecewise continuous.

The weights naturally induce a directed graph structure on the set of agents: we define, for any and , the graph as

The adjacency matrix is the set of pairs for which the communication channel from to is active at time .

As a prototypical example of a multiagent system and to quantitatively illustrate the concept of self-organization, we introduce alignment models: if we denote by the states of the agents of our systems, then the instantaneous evolution of the state of agent at time is given by

(1)

The meaning of the above system of differential equations is the following: at each instant , the state of agent tends to the state of agent with a speed that depends on the strength of the information exchange . Since (1) is a system of ODEs with possibly discontinuous coefficients, we need for it a proper notion of solution.

Definition 1

Let denote a countable family of open intervals such that all the functions are continuous on every and . Given , we say that the curve is a solution of (1) with initial datum if

  1. ;

  2. for every and , satisfies (1) on .

The notion of self-organization that we are considering for system (1) is that of consensus or flocking, which is the situation where the state variables of the agents asymptotically coincide.

Definition 2 (Consensus for system (1))

Let denote a solution of (1) with initial datum . We say that converges to consensus if there exists a such that, for every , it holds

The value is called the consensus state.

In the definition above, stands for the Euclidean norm on . The subscript shall often be omitted whenever clear from context.

Roughly speaking, a system of agents satisfying (1) converges to consensus regardless of the initial condition provided that the underlying communication graph is “sufficiently connected”. With this we mean that each node must possess, over some dense collection of time intervals, a strong enough communication path to every other node in the network. This intuitive idea is made precise in the following result, whose proof can be found in haskovec2014note (). A similar answer for discrete-time systems was also provided in moreau2005stability ()

Theorem 2.1

Let be a solution of (1) with initial datum . Suppose that there exists an and a strongly connected directed graph on the set of agents on which the system spends an infinite amount of time, i.e.,

Then converges to consensus with consensus state belonging to the convex hull of .

The above result is closely related, for instance, to (motsch2014heterophilious, , Theorem 2.3), which requires a stronger connectivity of the network of agents (the quantity in (motsch2014heterophilious, , Equation (2.5))) but also gives an explicit rate for the convergence towards (see (motsch2014heterophilious, , Equation (2.6b))).

Theorem 2.1 also says that, without further hypotheses on the interaction weights , the value of is rather an emergent property of the global dynamics of system (1) than a mere function of the initial datum . Nonetheless, it is relatively simple to identify assumptions on for which the latter is true. For example, from a trivial computation follows

Hence, if for every the weight matrix has the property that for every , then the average

(2)

is an invariant of the dynamics. This implies that holds, i.e., the consensus state is only a function of the initial datum .

3 Consensus emergence in alignment models

In this section we shall see that the assumptions of Theorem 2.1 can actually be very restrictive and seldom met when dealing with specific instances of alignment models.

3.1 Some classic examples of alignment models

A general principle in opinion formation is the conformity bias, i.e., agents weight more opinions that already conform to their beliefs. This can, actually, be extended to coordination in general, since intuitively it is easier to coordinate with “near” agents than “far away” ones. Formally, this is equivalent to asking that the weights are a nonincreasing function of the distance between the states of the agents, i.e.,

(3)

where is a nonincreasing interaction kernel. Notice that (3) trivially implies the invariance of the mean (given by (2)), and that , if it exists.

Several classic opinion formation models combine conformity bias with alignment. In the DW model, see weisbuch (), two random agents and update their opinions and to , provided they originally satisfy , where is fixed a priori. Instead, in the popular bounded confidence model of Hegselmann and Krause krause02 (), opinions evolves according to the dynamics (1) where the function has the form

for some fixed confidence radius . The dynamics is thus given by the system of ODEs

(4)

where we have set

(5)

and stands for its cardinality. It is straightforward to design an instance of this model not fulfilling the hypothesis of Theorem 2.1. Indeed, consider a group of agents in dimension with initial conditions and . Since , it follows that for all and for all .

Second-order models are necessary whenever we want to describe the dynamics of physical agents, like flocks of birds, herds of quadrupeds, schools of fish, and colonies of bacteria, where individuals are considered aligned whenever they move in the same direction, regardless of their position. Since in such cases it is necessary to perceive the velocities of the others in order to align, to describe the motion of the agents we need the pair position-velocity , but this time only the velocity variable is the consensus parameter.

One of the first of such models, named Vicsek’s model in honor of one of its fathers, was introduced in vicsek1995novel (). Very much in the spirit of (4), it postulates that the evolution of the spatial coordinate and of the orientation in the plane of the -th agent follows the law of motion given by

(6)

where denotes the constant modulus of .In this model, the orientation of the consensus parameter is adjusted with respect to the other agents according to a weighted average of the differences . The influence of the -th agent on the dynamics of the -th one is a function of the (physical or social) distance between the two agents: if this distance is less than , the agents interact by appearing in the computation of the respective future orientation.

Figure 1: On the left: a typical evolution of the Hegselmann-Krause model. On the right: mill patterns in the Vicsek model. (Kind courtesy of G. Albi)

In CS (), the authors proposed a possible extension of system (6) to dimensions as follows

The substitution of the function with a strictly positive kernel let us drop the highly irregular and nonsymmetric normalizing factor in favor of a simple , and leads to the system

(7)

Notice that the equation governing the evolution of has the same form as (1), and since now the weights are symmetric (i.e., for all ) then is a conserved quantity.

An example of a system of the form (7) is the influential model of Cucker and Smale, introduced in CS (), in which the function is

(8)

where , , and are constants accounting for the social properties of the group. Systems like (7) are usually referred to as Cucker-Smale systems due to the influence of their work, as can be witnessed by the wealth of literature focusing on their model, see for instance ahn2010stochastic (); carrillo2010asymptotic (); dalmao2011cucker (); ha2009simple (); perea2009extension (); shen2007cucker ().

3.2 Pattern formation for the Cucker-Smale model

We now focus on consensus emergence for system (7). In the following, we shall consider a kernel which is decreasing, strictly positive, bounded and Lipschitz continuous.

As already noticed, in second-order models alignment means that all agents move with the same velocity, but not necessarily are in the same position. Therefore, Definition 2 of consensus applies here on the variables only.

Definition 3 (Consensus for system (7))

We say that a solution

of system (7) tends to consensus if the consensus parameter vectors tend to the mean , i.e.,

Figure 2: Consensus behavior of a Cucker-Smale system. On the left: agents align with the mean velocity. On the right: agents fail to reach consensus.

The following result is an easy corollary of Theorem 2.1.

Corollary 1

Let be a solution of system (7), where the interaction kernel is decreasing and strictly positive. Suppose that there exists for which it holds

Then converges to consensus.

Proof

Since is decreasing and strictly positive, from the initial assumptions follows

for every for which holds for every . Therefore, the condition for every implies which yields

The statement then follows from Theorem 2.1 for the choice .

Unfortunately, the result above has the serious flaw that it cannot be invoked directly to infer convergence to consensus, since establishing a uniform bound in time for the distances of the agents is very difficult, even for smooth kernels like (8). Intuitively, consider the case where the interaction strength is too weak and the agents too dispersed in space to let the velocities align. In this case, nothing prevents the distances to grow indefinitely, violating the hypothesis of Corollary 1. Hence, in order to obtain more satisfactory consensus results, we need to follow approaches that take into account the extra information at our disposal, which are the strength of the interaction and the initial configuration of the system.

Originally, this problem was studied in CS (); cusm07 () borrowing several tools from Spectral Graph Theory, see as a reference chung1997spectral (). Indeed, system (7) can be rewritten in the following compact form

(9)

where is the Laplacian3 of the matrix , which is a function of . Being the Laplacian of a positive definite, symmetric matrix, encodes plenty of information regarding the adjacency matrix of the system, see mohar1991laplacian (). In particular, the second smallest eigenvalue of , called the Fiedler’s number of is deeply linked with consensus emergence: provided that a sufficiently strong bound from below of is available, the system converges to consensus.

To establish under which conditions we have convergence to consensus, we shall follow a different approach. The advantage of it is that it can be employed also to study the issue of the controllability of several multiagent systems (see Section 5).

3.3 The consensus region

A natural strategy to improve Corollary 1 would be to look for quantities which are invariant with respect to , since it is conserved in systems like (7).

Definition 4

The symmetric bilinear form is defined, for any , as

where denotes the usual scalar product on .

Remark 1

It is trivial to prove that

(10)

where stands for the average of the elements of the vector given by (2). From this representation of follows easily that the two spaces

are perpendicular with respect to the scalar product , i.e., . This means that every can be written uniquely as , where and . A closer inspection reveals that it holds and for every . Notice that, since , for any vector it holds

(11)

Since for every we have , it holds

This means that distinguishes two vectors modulo their projection on . Moreover, from (10) immediately follows that restricted to coincides, up to a factor , with the usual scalar product on .

Remark 2 (Consensus manifold)

Notice that whenever the initial datum belongs to the set , the right-hand size of in (7) is 0, hence the equality is satisfied for all and the system is already in consensus. For this reason, the set is called the consensus manifold.

The bilinear form can be used to characterize consensus emergence for solutions of system (7) by setting

The functionals and provide a description of consensus by measuring the spread, both in positions and velocities, of the trajectories of the solution , as the following trivial result shows.

Proposition 1

The following statements are equivalent:

  1. for every ;

  2. for every ;

  3. .

The following Lemma shows that is a Lyapunov functional for system (7).

Lemma 1 ((caponigro2015sparse, , Lemma 1))

Let be a solution of system (7). Then for every it holds

(12)

Therefore, is decreasing.

By means of the quantities and we can provide a sufficient condition for consensus emergence for solutions of system (7).

Theorem 3.1 ((HaHaKim, , Theorem 3.1))

Let and set and . If the following inequality is satisfied

(13)

then the solution of (7) with initial datum tends to consensus.

The inequality (13) defines a region in the space of initial conditions for which the balance between , and the kernel is such that the system tends to consensus autonomously.

Definition 5 (Consensus region)

We call consensus region the set of points satisfying (13).

The size of the consensus region gives an estimate of how large the basin of attraction of the consensus manifold is. If the rate of communication function is integrable, i.e., far distant agents are only weakly influencing the dynamics, then such a region is essentially bounded, and actually not all initial conditions will realize self-organization, as the following example shows.

Example 1 ((Cs, , Proposition 5))

Consider agents in dimension subject to system (7) with interaction kernel given by (8) with , , and . If we denote by and the trajectories of the two agents, it is easy to show that the evolution of the relative main state and of the relative consensus state is given for every by

(14)

with initial condition and (without loss of generality, we may assume that ). An explicit solution of the above system can be easily derived by means of direct integration:

Condition (13) in this case reads Hence, suppose (13) is violated, i.e., . This means for some , which implies

for every . Therefore, the solution of system (14) with initial datum satisfying does not converge to consensus, since otherwise we would have for .

Remark 3

Notice that, if diverges for every , then the consensus region coincides with the entire space . In other words, in this case the interaction force between the agents is so strong that the system will reach consensus no matter what the initial conditions are.

As the following example shows, there may be initial configurations from which the system can reach consensus automatically even if condition (13) is not satisfied.

Example 2

Consider an instance of the Cucker-Smale system (7) without control in dimension with agents, where the interaction function is of the form

for some given and positive continuous function satisfying

The constant is to be properly chosen later on. Assume that the initial state and consensus parameters of the two agents are and respectively, for some .

Due to the nature of the situation, is fairly easy to check if condition (13) of Theorem 3.1 is satisfied or not. Indeed we have and , and, by the particular form of , after a change of variables the computation below follows

Therefore at time we are not in the consensus region given by (13), since

We now show that there exists a time such that

(15)

i.e., the system enters the consensus region autonomously at time .

To do so, we first compute a lower bound for the integral. Notice that, since we are considering a Cucker-Smale system with mean consensus parameter , the speeds and are decreasing by Lemma 1. Therefore, we can estimate from above the time until holds by (since the agents are moving on the real line in opposite directions). Hence for every , which yields the following lower bound

valid for any .

We now compute an upper bound for the functional for . Notice that

hence by (12) we have

which, by integration, implies that for every .

We now plug together the two bounds. In order for (15) to hold at some time , simply choose

For this choice of , it follows

From Theorem 3.1 we can then conclude that any solution of the above system tends autonomously to consensus.

4 The effect of perturbations on consensus emergence

An immediate way to enhance the alignment capabilities of systems like (7) consists in adding a feedback term penalizing the distance of each agent’s velocity from the average one, i.e.,

(16)

where is a prescribed constant, modeling the strength of the additional alignment term.

This approach to the enforcement of consensus is a particular instance of what in the literature is known as decentralized control strategy, which has been thoroughly studied especially for its application in the self-organization of unmanned aerial vehicles (UAVs) fax2004information (), congestion control in communication networks paganini2001scalable (), and distributed sensor newtorks cortes2005coordination (). We also refer to tanner2007flocking () for the stability analysis of a decentralized coordination method for dynamical systems with switching underlying communication network.

As system (16) can be rewritten as (7) with the interaction kernel replacing , by Theorem 3.1 and Remark 3 each solution of (16) tends to consensus.

However, the apparently innocent fix of adding the extra term above has actually a huge impact on the interpretation of the model: as pointed out in caponigro2015sparse (), this approach requires that each agent must possess at every instant a perfect information of the whole system, since it has to correctly compute the mean velocity of the group in order to compute its trajectory. This condition is seldom met in real-life situations, where it is usually only possible to ask that each agent computes an approximated mean velocity vector , instead of the true . These considerations lead us to the model

(17)

In studying under which conditions the solutions of system (17) tend to consensus, it is often desirable to express the approximated feedback as a combination of a term consisting on a true information feedback, i.e., a feedback based on the real average , and a perturbation term, which models the deviation of from . To this end, we rewrite system (17) in the following form:

(18)

where and are two nonnegative, piecewise continuous functions, and is the deviation acting on the estimate of by agent (which can, of course, depend on ). Therefore, solutions in this context have to be understood in terms of weak solutions in the Carathéodory sense, see filipov ().

Remark 4

In what follows, we will not be interested in the well-posedness of system (18), but rather in finding assumptions on the functions , , , and for which we can guarantee its asymptotic convergence to consensus.

System (18) provides the advantage of encompassing all the previously introduced models, as can be readily seen:

  • if and , or , then we recover system (7),

  • the choices , (or equivalently ) yield system (16),

  • if and we obtain system (17).

The introduction of the perturbation term in system (18) may deeply modify the nature of the original model: for instance, an immediate consequence is that the mean velocity of the system is, in general, no longer a conserved quantity.

Proposition 2

For system (18), with perturbations given by the vector-valued function , for every it holds

Remark 5

As we have already pointed out, it is possible to recover system (7) by setting , whereas we can recover system (16) for the choice . Note that in both cases we have for every , therefore the mean velocity is conserved both in systems (7) and (16).

We also highlight the fact that is not conserved even in the case that for every and for every we have , where , i.e., the case in which all agents make the same mistake in evaluating the mean velocity.

4.1 General results for consensus stabilization under perturbations

The following is a generalization of Lemma 1 to systems like (18).

Lemma 2 ((bongini2015conditional, , Lemma 3.1))

Let be a solution of system (18). For every it holds

(19)
Proof

Differentiating for every we have

Hence, inserting the expression for , using the fact that is nonincreasing, and invoking Proposition 2, we get (19).

Since we are interested in the case where plays an active role in the dynamics, in what follows we assume for all . As a direct consequence of Lemma 2 we get that, by controlling the magnitude of the deviations , we can establish the unconditional convergence to consensus.

Theorem 4.1

Let be a solution of system (18), and suppose that there exists a such that for every ,

(20)

for some function , where

(21)

Then tends to consensus.

Proof

Under the assumption (20), for every the upper bound in (19) can be simplified to

Integrating between and (where ) we get and as the factor is negative while is nonnegative, approaches exponentially fast.

We then immediately get the following

Corollary 2

If there exists such that for every and for every , then any solution of system (18) tends to consensus.

Proof

Noting that implies , by (11) we have Hence, we can apply Theorem 4.1 with for every to obtain the result.

Remark 6

A trivial implication of Corollary 2 is that any solution of system (16) tends to consensus (this was already a consequence of Theorem 3.1), but has moreover a rather nontrivial implication: also any solution of systems subjected to deviated uniform control, i.e., systems like (18) where for every and for every , tends to consensus, because it holds

for every and for every , therefore Corollary 2 applies. This means that systems of this kind converge to consensus even if the agents have an incorrect knowledge of the mean velocity, provided they all make the same mistake.

Another consequence of the previous results is the following corollary, which provides an upper bound for tolerable perturbations under which consensus emergence can be unconditionally guaranteed.

Corollary 3

For every