Multi-Objective Linear Quadratic Team Optimization This work was supported by the Swedish Research Council.

# Multi-Objective Linear Quadratic Team Optimization ††thanks: This work was supported by the Swedish Research Council.

Ather Gattami A. Gattami is with the Electrical Engineering School, KTH-Royal Institute of Technology, SE-100 44 Stockholm, Sweden. gattami@kth.se
###### Abstract

In this paper, we consider linear quadratic team problems with an arbitrary number of quadratic constraints in both stochastic and deterministic settings. The team consists of players with different measurements about the state of nature. The objective of the team is to minimize a quadratic cost subject to additional finite number of quadratic constraints. We will first consider the Gaussian case, where the state of nature is assumed to have a Gaussian distribution, and show that the linear decisions are optimal and can be found by solving a semidefinite program We then consider the problem of minimizing a quadratic objective for the worst case scenario, subject to an arbitrary number of deterministic quadratic constraints. We show that linear decisions can be found by solving a semidefinite program.

Key words. Team Decision Theory, Game Theory, Convex Optimization.

AMS subject classifications. 99J04, 49K04

## 1 Introduction

We consider the problem of distributed decision making with information constraints under linear quadratic settings. For instance, information constraints appear naturally when making decisions over networks. These problems can be formulated as team problems. The team problem is an optimization problem with several decision makers possessing different information aiming to optimize a common objective. Early results in  considered static team theory in stochastic settings and a more general framework was introduced by Radner , where existence and uniqueness of solutions where shown. Connections to dynamic team problems for control purposes where introduced in . In , the team problem with two team members was solved. The solution cannot be easily extended to more than two players since it uses the fact that the two members have common information; a property that doesn’t necessarily hold for more than two players. Also, a nonlinear team problem with two team members was considered in , where one of the team members is assumed to have full information whereas the other member has only access to partial information about the state of the world. Related team problems with exponential cost criterion were considered in . Optimizing team problems with respect to affine decisions in a minimax quadratic cost was shown to be equivalent to stochastic team problems with exponential cost, see . The connection is not clear when the optimization is carried out over nonlinear decision functions. The deterministic version (minimizing the worst case scenario) of the linear quadratic team decision problem was solved in .

In this paper, we will consider both Gaussian and deterministic settings(worst case scenario) for team decision problems under additional quadratic constraints. It’s well-known that additional constraints, although convex, could give rise to complex optimization problems if the optimized variables are functions (as opposed to real numbers). For instance linear functions, that is functions of the form where is a real matrix, are no longer optimal. We will illustrate this fact by the following example:

###### Example 1

For , we want to minimize the objective function

 |u|2

subject to

 |x−u|2≤γ

Some Hilbert space theory shows that the optimal is given by

 u=μ(x)=(|x|−√γ)x/|x|  if  |x|2>γ,

and

 u=μ(x)=0  otherwise.

Obviously, the optimal is a nonlinear function of .

Increasing the dimension of , and adding constraints on the structure of , for instance and , certainly makes the constrained optimization more complicated. The example above shows that, in spite of having a convex optimization carried out over a Hilbert space, the optimal decision function is nonlinear. However, we show in the upcoming sections that multi-objective problems behave nicely when considering the expected values of the objectives in the Gaussian case, in the sense that linear decisions are optimal. For the deterministic counterpart which is not an optimization problem over a Hilbert space, we show how to find the linear optimal decisions by semidefinite programming. However, the optimality of the linear decisions remains and open question.

## 2 Notation

The following table gives a list of the notation we are going to usee throughout the text:

Sn Sn+ The set of n×n symmetric matrices. The set of n×n symmetric positive semidefinite matrices. The set of n×n symmetric positive definite matrices. The set of measurable functions. The set of functions μ:Rp→Rm with μ(y)=(μT1(y1),μT2(y2),...,μTN(yN))T, μi:Rpi→Rmi, ∑imi=m, ∑ipi=p. The element of A in position (i,j). A⪰B ⟺ A−B∈Sn+. A≻B ⟺ A−B∈Sn++. The Kronecker binary operation between two matrices A and B, A⊗B. Tr[A] is the trace of the matrix A. The set of Gaussian variables with mean m and covariance X.

## 3 Linear Quadratic Gaussian Team Theory

In this section we will review some classical results in stochastic team theory with new simpler proofs for the linear quadratic case, that first appeared in  and .

In the static team decision problem, one would like to solve

 minμ E[xu]T[QxxQxuQuxQuu][xu] \hb@xt@.01(3.1) subject to yi=Cix+vi ui=μi(yi) for i=1,...,N.

Here, and are independent Gaussian variables taking values in and , respectively, with and . Also, and will be stochastic variables taking values in , , respectively, and . We assume that

 [QxxQxuQuxQuu]∈Sm+n, \hb@xt@.01(3.2)

and , .

If full state information about is available to each decision maker , the minimizing can be found easily by completion of squares. It is given by , where is the solution to

 QuuL=−Qux.

Then, the cost function in (LABEL:static) can be rewritten as

 J(x,u) =E{xT(Qxx−LTQuuL)x}+E{(u−Lx)TQuu(u−Lx)}. \hb@xt@.01(3.3)

Minimizing the cost function , is equivalent to minimizing

 E{(u−Lx)TQuu(u−Lx)},

since nothing can be done about (the cost when has full information).

The next theorem is due to Radner , but we give a different formulation and proof that is simpler, which relies on the structure of the linear quadratic Gaussian setting:

###### Theorem 1

Let and be Gaussian variables with zero mean, taking values in and , respectively, with . Also, let be a stochastic variable taking values in , , , , , for . Then, the optimal decision to the optimization problem

 minμ E{(u−Lx)TQuu(u−Lx)} \hb@xt@.01(3.4) subject to yi=Cix+vi ui=μi(yi) for i=1,...,N.

is unique and linear in .

Proof. Let be the linear space of functions such that if is a linear transformation of , that is for some real matrix . Since , is a linear space under the inner product

 ⟨g,h⟩=E{gTQuuh},

and norm

 ||g||2=E{gTQuug}.

The optimization problem in (LABEL:opt1) where we search for the linear optimal decision can be written as

 minu∈Z ||u−Lx||2 \hb@xt@.01(3.5)

Finding the best linear optimal decision to the above problem is equivalent to finding the shortest distance from the subspace to the element , where the minimizing is the projection of on , and hence unique. Also, since is the projection, we have

 0=⟨u∗−Lx,μ⟩=E{(u∗−Lx)TQuuu},

for all . In particular, for , we have

 E{(u∗−Lx)TQuufi}=E{[(u∗−Lx)TQuu]izi}=0.

The Gaussian assumption implies that is independent of , for all linear transformations . This gives in turn that is independent of . Hence, for any decision , linear or nonlinear, we have that

 E(u∗−Lx)TQuuμ(y)=∑iE{[(u∗−Lx)TQuu]iμi(yi)}=0,

and

 E(μ(y)− Lx)TQuu(μ(y)−Lx) =E(u∗−Lx+μ(y)−u∗)TQuu(u∗−Lx+μ(y)−u∗) =E(u∗−Lx)TQuu(u∗−Lx)+E(μ(y)−u∗)TQuu(μ(y)−u∗) +2E(u∗−Lx)TQuu(μ(y)−u∗) =E(u∗−Lx)TQuu(u∗−Lx)+E(μ(y)−u∗)TQuu(μ(y)−u∗) ≥E(u∗−Lx)TQuu(u∗−Lx)

with equality if and only if . This concludes the proof.

###### Proposition 1

Let and be independent Gaussian variables taking values in and , respectively with , . Also, let be a stochastic variable taking values in , , , , , and . Set . Then, the optimal solution to the optimization problem

 minKi E(u−Lx)TQuu(u−Lx) \hb@xt@.01(3.6) subject to ui=Kiyi for i=1,...,N.

is the solution of the linear system of equations

 N∑j=1[Quu]ijKj(CjVxxCTi+[Vvv]ji)=−[Qux]iVxxCTi,for i=1,...,N. \hb@xt@.01(3.7)

Proof. Let and The problem of finding the optimal linear feedback law can be written as

 minKi Tr[E{Quu(u−Lx)(u−Lx)T}] \hb@xt@.01(3.8) subject to u=K(Cx+v)

Now

 f(K) =TrE{Quu(u−Lx)(u−Lx)T} \hb@xt@.01(3.9) =TrE{Quu(KCx+Kv−Lx)(KCx+Kv−Lx)T} =Tr[E{QuuK(CxxTCT+vvT)KT−2QuuLxxTCTKT +QuuLxxTLT+2Quu(KC−L)xvTKT}] =Tr[QuuK(CVxxCT+Vvv)KT−2QuuLVxxCTKT+QuuLVxxLT)] =Tr[N∑i,j=1[Quu]ijKj(CjVxxCTi+[Vvv]ji)KTi−2N∑i,j=1[Quu]ijLjVxxCTiKTi] +Tr[QuuLVxxLT].

A minimizing is obtained by solving :

 0 =∇Kif(K) \hb@xt@.01(3.10) =2N∑j=1[Quu]ijKj(CjVxxCTi+[Vvv]ji)−2N∑j=1[Quu]ijLjVxxCTi.

Since , we get that

 ∑Nj=1[Quu]ijLjVxxCTi=−[Qux]iVxxCTi,

and the equality in (LABEL:almost) is equivalent to

 ∑Nj=1[Quu]ijKj(CjVxxCTi+[Vvv]ji)=−[Qux]iVxxCTi,

and the proof is complete.

In general, separation does not hold for the static team problem when constraints on the information available for every decision maker are imposed. That is, the optimal decision is not given by , where is the optimal estimated value of by decision maker . We show it by considering the following example.

###### Example 2

Consider the team problem

 minimize E[xu]T[QxxQxuQuxQuu][xu] subject to yi=Cix+vi ui=μi(yi) for i=1,...,N

The data we will consider is:

 N=2,C1=C2=1,x∼N(0,1),v1∼N(0,1),v2∼N(0,1)

The best decision with full information is given by

 u =−Q−1uuQuxx=⎡⎣23−13−1323⎤⎦x=⎡⎣1313⎤⎦x.

The optimal estimate of of decision maker 1 is

 ^x1=E{x|y1}=12y1,

and of decision maker 2

 ^x2=E{x|y2}=12y2.

Hence, the decision where each decision maker combines the best deterministic decision with her best estimate of is given by

 ui =13^xi=16yi,

for . This policy gives a cost equal to . However, solving the team problem yields , and hence the optimal team decision is given by

 ui=15yi.

The cost obtained from the team problem is . Clearly, separation does not hold in team decision problems.

## 4 Team Decision Problems with Power Constraints

Consider the modified version of the optimization problem (LABEL:static):

 minμ E[xu]T[QSSTR][xu] \hb@xt@.01(4.1) subject to yi=Cix ui=μi(yi) γi≥E∥μi(yi)∥2 for i=1,...,N.

The difference from Radner’s original formulation is that we have added power constraints to the decision functions, .

In optimization (minimization) problems, you define the value to be infinite if there doesn’t exist any feasible decision variable that satisfy the constraints. Therefore, usually, one assumes that there is a feasible point, and hence the value must be finite. Existence conditions are hard to derive usually in spite of problems might be convex. So in practice, you run the algorithm and either you get a finite number, or it goes indefinitely. Conditions where you a decide whether you have a feasible problem or not are of great interest of course. It’s a nontrivial problem that is outside the scope of this paper.

In the sequel, we will prove a more general theorem, where we consider power constraints on a set of quadratic forms in both the state and the decision function .

###### Theorem 2

Let be a Gaussian variable with zero mean and given covariance matrix , taking values in . Also, let , , , and , for . Assume that the optimization problem

 minμ∈C E[xμ(x)]T[Q0S0ST0R0][xμ(x)] \hb@xt@.01(4.2) subject to E[xμ(x)]T[QjSjSTjRj][xμ(x)]≤γj j=1,...,M

is feasible. Then, linear decisions given by , with , are optimal.

Proof. Consider the expression

 E[xμ(x)]T[Q0S0ST0R0][xμ(x)]+M∑j=1λj(E[xμ(x)][QjSjSTjRj][xμ(x)]−γj).

Take the expectation of a quadratic form with index to be larger than . Then, makes the value of the expression above infinite. On the other hand, if the expectation of a quadratic form with index is smaller than , then the maximizer is optimal for .

Now let be the optimal value of the optimization problem (LABEL:main), and consider the objective function

 [xu]T[Q0S0ST0R0][xu]=xT(Q0−S0R−10ST0)x+(u−R−10ST0x)TR0(u−R−10ST0x).

We have that , since it’s the Schur complement of in the positive semi-definite matrix Since , a necessary condition for the objective function to be zero is that , and so must be linear (In order for to have the structure given by , must be in , to satisfy the information constraints).

Now assume that . We have

 p⋆ =minμ∈Cmaxλi∈R+E[xμ(x)]T[Q0S0ST0R0][xμ(x)]+M∑j=1λj(E[xμ(x)][QjSjSTjRj][xμ(x)]−γj) \hb@xt@.01(4.3)

Now introduce and the matrix

 [QSSTR]=M∑j=0λj[QjSjSTjRj],

and consider the minimax problem

 p0=minμ∈Cmaxλj≥0∑Mj=0λj=1E[xμ(x)]T[QSSTR][xμ(x)]−M∑j=1λjγj. \hb@xt@.01(4.4)

Note that a maximizing must be positive, since implies that , while gives . We can always recover the optimal solutions of (LABEL:pstar) from that of (LABEL:p0) by dividing all variables by , that is , , and . Now we have the obvious inequality ()

 p0

For any fixed values of , we have , so Theorem LABEL:radner1 gives the equality

 minμ∈CE[xμ(x)][QSSTR][xμ(x)]=minK∈KE[xKx][QSSTR][xKx],

where the minimizing is unique. Thus,

 p0

The objective function is radially unbounded in since . Hence, it can be restricted to a compact subset of . Thus,

 p0 =minK∈Kmaxλj≥0∑Mj=0λj=1E[xKx]T(M∑j=0λj[QjSjSTjRj])[xKx]−M∑j=1λjγj =p0,

where the equality is obtained by applying Proposition LABEL:minmaxtheorem in the Appendix, the second inequality follows from the fact that the set of linear decisions , , is a subset of , and the second equality follows from the definition of . Hence, linear decisions are optimal, and the proof is complete.

Remark: Although Theorem 2 is stated and proved for and , it extends easily to the case for any matrix , which often is the case in applications.

## 5 Computation of The Optimal Team Decisions

The optimization problem that we would like to solve when assuming linear decisions is

 minγ0,K∈K γ0 \hb@xt@.01(5.1) subject to E[xKCx]T[QjSjSTjRj][xKCx]≤γj,j=0,...,M, x∼N(0,X2).

Note that we can write the constraints as

 E[xKCx]T[QjSjSTjRj][xKCx] =E{Tr[IKC]T[QjSjSTjRj][IKC]xxT} \hb@xt@.01(5.2) =TrX[IKC]T[QjSjSTjRj][IKC]X,

where we used that . Hence, we obtain a set of convex quadratic inequalities (convex since for all )

 TrX[IKC]T[QjSjSTjRj][IKC]X≤γj.

There are many existing computational methods to solve convex quadratic optimization problems (see ).

Alternatively, we can formulate the optimization problem as a set of linear matrix inequalities as follows. For simplicity, we will assume that for all (The case is analogue with some technical conditions).

###### Theorem 3

The team optimization problem (LABEL:linopt) is equivalent to the semi-definite program

 minγ0,K∈K γ0 \hb@xt@.01(5.3) subject to TrPj≤γj 0⪯[Pj−XQjX−XSjKCX−XCTKTSTjXXCTKTRjRjKCXRj] j=0,...,M.

Proof. Introduce the matrices , and write the given constraints as

 γj≥TrPj
 Pj−X[IKC]T[QjSjSTjRj][IKC]X⪰0. \hb@xt@.01(5.4)

Now we have that

 0 ⪯X[IKC]T[QjSjSTjRj][IKC]X \hb@xt@.01(5.5) =Pj−XQjX−XSjKCX−XCTKTSTjX−XCTKTRjKCX.

Since , the quadratic inequality above can be transformed to a linear matrix inequality using the Schur complement (), which is given by

 [Pj−XQjX−XSjKCX−XCTKTSTjXXCTKTRjRjKCXRj]⪰0.

Hence, our optimization problem to be solved is given by

 minK∈K γ0 \hb@xt@.01(5.6) subject to TrPj≤γj 0⪯[Pj−XQjX−XSjKCX−XCTKTSTjXXCTKTRjRjKCXRj] j=0,...,M,

which proves our theorem.

## 6 Minimax Team Theory

We considered the problem of static stochastic team decision in the previous sections. This section treats an analogous version for the deterministic (or worst case) problem. Although the problem formulation is very similar, the ideas of the solution are considerably different, and in a sense more difficult.

The deterministic problem considered is a quadratic game between a team of players and nature. Each player has limited information that could be different from the other players in the team. This game is formulated as a minimax problem, where the team is the minimizer and nature is the maximizer.

### 6.1 Deterministic Team Problems

Consider the following team decision problem

 infμsupx≠0 J(x,u)||x||2 \hb@xt@.01(6.1) subject to yi=Cix ui=μi(yi) for i=1,...,N

where , , .
is a quadratic cost given by

 J(x,u)=[xu]T[QxxQxuQuxQuu][xu],

where

 [QxxQxuQuxQuu]∈Sm+n.

We will be interested in the case . The players ,…, make up a team, which plays against nature represented by the vector , using , that is

 μ(Cx)=⎡⎢ ⎢⎣μ1(C1x)⋮μN(CNx)⎤⎥ ⎥⎦.
###### Theorem 4

If the value of the game (LABEL:minimax) is equal to , then there is a linear decision , with , achieving that value.

Proof. For a proof, consult .

### 6.2 Relation with The Stochastic Minimax Team Decision Problem

Now consider the stochastic minimax team decision problem

 minKmaxE∥x∥2=1E{xT[IKC]T[QxxQxuQuxQuu][IKC]x}.

Taking the expectation of the cost in the stochastic problem above yields the equivalent problem

 minKmaxTrX=1Tr[IKC]T[QxxQxuQuxQuu][IKC]X

where is a positive semi-definite matrix, and is the covariance matrix of , i. e. . Hence, we see that the stochastic minimax team problem is equivalent to the deterministic minimax team problem, where nature maximizes with respect to all covariance matrices of the stochastic variable with variance .

## 7 Deterministic Team Problems with Quadratic Constraints

Consider the team problem (LABEL:minimax). An equivalent condition for the existence of a decision function that achieves the value of the game is that

 [xμ⋆(Cx)]T[QSSTR][xμ⋆(Cx)]≤γ⋆∥x∥2

for all , which is equivalent to

 [xμ⋆(Cx)]T[Q−γ⋆ISSTR][xμ⋆(Cx)]≤0

for all . This is an example of a power constraint. We could also have a set of power constraints that have to be mutually satisfied. For instance, in addition to the minimization of the worst case quadratic cost, we could have constraints on the induced norms of the decision functions

 ∥μi(Cix)∥2∥x∥2≤γi%forallx≠0,i=1,...,M,

or equivalently given by the quadratic inequalities

 ∥μi(Cix)∥2−γi∥x∥2≤0for % all x,i=1,...,M.

Also, the team members could share a common power source, and the power is proportional to the squared norm of the decisions :

 M∑i=1∥μi(Cix)∥2−c∥x∥2≤0% for all x,

for some positive real number .

It’s not clear whether linear decisions are optimal, since the example give at the introduction indicates that, in deterministic settings, nonlinear decision are optimal. However, the next result shows how to obtain the linear optimal decisions by solving a semidefinite program.

###### Theorem 5

Let , for . Let for , and for