Push-sum on random graphs

# Push-sum on random graphs

Pouya Rezaienia, Bahman Gharesifard,
Tamás Linder, and Behrouz Touri
The first three authors are with the Department of Mathematics and Statistics at Queen’s University, Kingston, ON, Canada. The last author is with the Department of Electrical and Computer Engineering at the University of California, San Diego.
###### Abstract

In this paper, we study the problem of achieving average consensus over a random time-varying sequence of directed graphs by extending the class of so-called push-sum algorithms to such random scenarios. Provided that an ergodicity notion, which we term the directed infinite flow property, holds and the auxiliary states of agents are uniformly bounded away from zero infinitely often, we prove the almost sure convergence of the evolutions of this class of algorithms to the average of initial states. Moreover, for a random sequence of graphs generated using a time-varying -irreducible probability matrix, we establish convergence rates for the proposed push-sum algorithm.

## 1 Introduction

Many distributed algorithms, executed with limited information over a network of agents, rely on estimating the average value of the initial state of the individual agents. These include the distributed optimization protocols [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], distributed regret minimization algorithms in machine learning [12], and dynamics for fusion of information in sensor networks [13]. There is a large body of work devoted to the average consensus problem, starting with the pioneering work [14], where the so-called push-sum algorithm is first introduced. The key differentiating factor of the push-sum algorithm from consensus dynamics is that it takes advantage of a paralleled scalar-valued agreement dynamics, initiated uniformly across the agents, that tracks the imbalances of the network and adjusts for them when estimating the consensus value.

In addition to the earlier work [14], several recent papers have studied the problem of average consensus, see for example [15], where other classes of algorithms based on weight adaptation are considered, ensuring convergence to the average on fixed directed graphs. The study of convergence properties of push-sum algorithms on time-varying deterministic sequences of directed graphs, to best of our knowledge, was initiated in [16] and extended in [11], where push-sum protocols are intricately utilized to prove the convergence of a class of distributed optimization protocols on a sequence of time-varying directed graphs. The key assumption in [11] is the -connectedness of the sequence, which means that in any window of size the union of the underlying directed graphs over time is strongly connected. As we demonstrate, a by product of our work in deterministic settings is the generalization of the sequences on which the convergence of the push-sum algorithms is valid to the ones which satisfy the infinite flow property; in this sense, this extension mimics the properties required for the convergence of consensus dynamics, along the lines of [17].

This paper is concerned with the problem of average consensus for scenarios where communication between nodes is time-varying and possibly random. The convergence properties of consensus dynamics on random sequences of directed graphs are by this time well-established, see for example [17, 18, 19]. Average consensus on random graphs has also been studied in [16], under the assumption that the corresponding random sequence of stochastic matrices is stationary and ergodic with positive diagonals and irreducible expectation. One of our main objectives in this work is to extend these result to more general sequences of random stochastic matrices, in particular, beyond stationary. More importantly, to best of our knowledge, we establish for the first time convergence rates for the push-sum algorithms on random sequences of directed graphs.

The remainder of this paper is organized as follows. Section 2 contains mathematical preliminaries. In Section 3, we give a formal description of our consensus problem. In Section 4, we describe the push-sum algorithm. Section 5 studies the ergodicity of row-stochastic matrices, and Section 6 contains our main convergence results. In Section 7, we derive convergence rates for the push-sum algorithm for a class of random column-stochastic matrices. Finally, we gather our conclusions and ideas for future directions in Section 8.

## 2 Mathematical Preliminaries

We start with introducing some notational conventions. Let and denote the set of real and integer numbers, respectively, and let and denote the set of non-negative real numbers and integers, respectively. For a set , we write if is a proper subset of , and we call the empty set and trivial subsets of . The complement of is denoted by . Let denote the cardinality of a finite set . We view all vectors in as column vectors, where . We denote by , and , the standard Euclidean norm, the -norm, and the infinity norm on , respectively. The th unit vector in , whose th component is and all other components are , is denoted by . We will also use the short-hand notation and . A vector is stochastic if its elements are nonnegative real numbers that sum to . We use to denote the set of non-negative real-valued matrices. A matrix is row-stochastic (column-stochastic) if each of its rows (columns) sums to 1. For a given and any nontrivial , we let . The notation and will refer to the transpose of the matrix and the vector , respectively. A positive matrix is a real matrix all of whose elements are positive. Finally, denotes the th row of matrix and denotes the th column of .

### 2.1 Graph theory

A (weighted) directed graph consists of a node set , an edge set , and a weighted adjacency matrix , with if and only if , in which case we say that is connected to . Similarly, given a matrix , one can associate to a directed graph , where if and only if , and hence is the corresponding adjacency matrix for . The in-neighbors and the out-neighbors of are the set of nodes and , respectively. The out-degree of is . A path is a sequence of nodes connected by edges. A directed graph is strongly connected if there is a path between any pair of nodes. A directed graph is complete if every pair of distinct vertices is connected by an edge. If the directed graph is strongly connected, we say that is irreducible.

### 2.2 Sequences of random stochastic matrices

Let be the set of column-stochastic matrices that have positive diagonal entries, and let denote the Borel -algebra on . Given a probability space , a measurable function is called a random column-stochastic matrix, and a sequence of such measurable functions on is called a random column-stochastic matrix sequence; throughout, we assume that . Note that for any , one can associate a sequence of directed graphs to , where if and only if . This in turn defines a sequence of random directed graphs on , which we denote by .

## 3 Problem Statement

Consider a network of nodes , where node has an initial state (or opinion) ; the assumption that this initial state is a scalar is without loss of generality, and our treatment can easily be extended to the vector case. The objective of each node is to achieve average consensus; that is to compute the average with the constraint that only limited exchange of information between nodes is permitted. The communication layer between nodes at each time is specified by a sequence of random directed graphs , where . Specifically, at each time , node updates its value based on the values of its in-neighbors , where . One standing assumption throughout this paper is that each node knows its out-degree at every time ; this assumption is indeed necessary, as shown in [20]. Our main objective is to show that the class of so-called push-sum algorithms can be used to achieve average consensus at every node, under the assumption that the communication network is random. This key point distinguishes our work from the existing results in the literature [14][11][15]. Another key objective that we pursue in this paper is to obtain rates of convergence for such algorithms. We start our treatment with reviewing the push-sum algorithm.

## 4 Random Push-Sum

Consider a network of nodes , where node has an initial state (or opinion) . The push-sum algorithm, proposed originally in [14], is defined as follows. Each node maintains and updates, at each time , two state variables and . The first state variable is initialized to and the second one is initialized to , for all . At time , node sends and to its out-neighbors in the random directed graph , which we assume to contain self-loops at each node for all . At time , node updates its state variables according to

 xi(t+1) =∑j∈Nini(t)xj(t)d\textupoutj(t), (1) yi(t+1) =∑j∈Nini(t)yj(t)d\textupoutj(t). (2)

It is useful to define another auxiliary variable ; as we will show later, is the estimate by node of the average . One can rewrite this algorithm in a vector form; let the column-stochastic matrix to be a function of with entries

 Wij(t)=⎧⎨⎩1d\textupoutj(t)if j∈N\textupini(t),0otherwise. (3)

Using these weighted adjacency matrices, for every , we can rewrite the dynamics (1) as

 x(t+1) =W(t)x(t), (4) y(t+1) =W(t)y(t), (5)

where

 x(t) =(x1(t),…,xn(t))′, y(t) =(y1(t),…,yn(t))′.

## 5 Ergodicity

In this section, we establish some important auxiliary results regarding the convergence of products of matrices which satisfy the so-called directed infinite flow property (c.f. Definition 3). We study the products of a class of matrices in a deterministic setting, which we then use to study the push-sum algorithm in the next section. We start by some definitions.

###### Definition 1 (Ergodicity [21], [17]).

Let be a sequence of row-stochastic matrices, and for , let denote the product

 A(t:s)=A(t)A(t−1)⋯A(s), (6)

where . The sequence is said to be weakly ergodic, if for all and any , . The sequence is said to be strongly ergodic if for any , where is a stochastic vector.

It can be shown that weak ergodicity and strong ergodicity are equivalent [21, Theorem 1]. We will simply call such a sequence of row-stochastic matrices ergodic.

We first establish a sufficient condition for ergodicity of a sequence of row-stochastic matrices, Proposition 2, which we subsequently use in our convergence result for the push-sum algorithm. For this reason, we consider the following dynamical system:

 x(t+1)=A(t)x(t),for all t≥0. (7)

Let us start by two key definitions.

###### Definition 2 (Strong Aperiodicity [17]).

We say that a sequence of matrices is strongly aperiodic if there exists such that , for all and .

Motivated by the infinite flow property [17, Definition 3.2.], we provide the following definition.

###### Definition 3 (Directed Infinite Flow Property).

We say that a sequence of matrices has the directed infinite flow property if for any non-trivial , .

Consider now a sequence of matrices that is strongly aperiodic and has the directed infinite flow property. Let , and for any , define

 kq=argmint′>kq−1⎛⎝minS⊂[n]t′−1∑t=kq−1AS¯S(t)>0⎞⎠. (8)

Note that is the minimal time instance after , such that there is nonzero information flow between any non-trivial subset of and its complement; consequently, the directed graph associated with the product is strongly connected.

###### Proposition 1.

If a sequence of matrices has the directed infinite flow property, is finite for all .

###### Proof.

Suppose that is not finite for some . Then, using (8), there exists a non-trivial subset such that . This implies that , which contradicts the assumption that has the directed infinite flow property. ∎

To establish convergence results for the products of row-stochastic matrices satisfying Proposition 3, we argue that in each time window where the underlying directed graph becomes strongly connected for times, i.e., after time steps for some , significant mixing will occur. To formalize this statement, let and

 ℓq=kqn−k(q−1)n, (9)

for . For , we also define

 Qt,s={q:s≤k(q−1)n,kqn≤t}.

We are now ready to state our first result.

###### Proposition 2.

Consider the dynamics (7), where the sequence of row-stochastic matrices is such that satisfies (3). Suppose, additionally, that is strongly aperiodic and has the directed infinite flow property. Then,

1. there is a vector such that, for all and ,

 ∣∣∣[A(t:s)]ij−ϕj(s)∣∣∣≤Λt,s,

where and ;

2. if, for the sequence associated with , we have

 ∞∑q=11nℓq=∞, (10)

then the sequence is ergodic.

###### Proof.

We start by proving the first statement. By definition of , we know that for all , is irreducible. Since each is strongly aperiodic, by Lemma A.1, the matrix

 A(kn(q+1)−1:knq)=A(kn(q+1)−1:kn(q+1)−1)×⋯×A(knq+2−1:knq+1)×A(knq+1−1:knq),

which is the product of irreducible matrices, is positive for all . Hence, by Lemma A.2 (ii), for all , we have

 [A(kn(q+1)−1:knq)]ij≥1nkn(q+1)−knq=1nlq+1.

Now, since and for all , , using [22, Lemma 3], we obtain

 maxi∈[n][A(t:s)]ij−mini∈[n][A(t:s)]ij≤Λt,s. (11)

Note that if we let for all , we have

 ∣∣∣[A(t:s)]ij−ϕj(s)∣∣∣≤maxi∈[n][A(t:s)]ij−mini∈[n][A(t:s)]ij. (12)

Using (11) and (12), we conclude that

 ∣∣∣[A(t:s)]ij−ϕj(s)∣∣∣≤Λt,s,

for all .

We next prove part (ii); since for all , we have that , where we have used the fact that for all . This implies

 ∞∑q=1ln(λq) ≤−∞∑q=11nℓq. (13)

On the other hand, we have

 limt→∞Λt,0=limt→∞∏q∈Qt,0λq=limt→∞exp⎛⎜⎝∑q∈Qt,0ln(λq)⎞⎟⎠.

The definition of the sets implies that we can write the right hand side as , which gives

 limt→∞Λt,0=exp(∞∑q=1ln(λq))=0,

where the last equality follows from (13) and the assumption . Using the fact that , we have that , for any . Hence, by Proposition 2, part (i), we conclude that is weakly (and thus strongly) ergodic. ∎

Following similar steps as in Proposition 2 we obtain the following result for sequences of column-stochastic matrices of the form (3).

###### Proposition 3.

Consider the dynamics (7) and assume that sequence of matrices is strongly aperiodic and has the directed infinite flow property, where the are weighted adjacency matrices in the form of (3). Then,

1. there is a vector such that, for all and ,

 ∣∣∣[A(t:s)]ij−ϕi(t)∣∣∣≤Λt,s,

where and ;

2. for the sequence associated with , if

 ∞∑q=11nℓq=∞,

then for all , .

It is worth pointing out that in Proposition 2, since the are row-stochastic, approaches a vector with identical entries. However, in Proposition 3 the does not necessarily approach a fixed vector.

## 6 Convergence of Push-Sum

With all the pieces in place, we are now ready to study the behavior of the push-sum algorithm in a random setting.

###### Theorem 1.

Consider the push-sum algorithm (4) and suppose that the sequence of random column-stochastic matrices has the directed infinite flow property, almost surely. Then, we have

 |zi(t+1)−¯x|≤2∥x(0)∥1yi(t+1)Λt,0,

where and .

###### Proof.

Define

 D(t:s)≜W(t:s)−ϕ(t)1′n,

where is a (random) vector from part (i) of Proposition 3. In addition, under the push-sum algorithm we have that

 x(t+1)=W(t:0)x(0), y(t+1)=W(t:0)y(0),

for all . Hence, for every and all , we have

 zi(t+1)−¯x =xi(t+1)yi(t+1)−1′nx(0)n =[W(t:0)x(0)]i[W(t:0)y(0)]i−1′nx(0)n =[D(t:0)x(0)]i+ϕi(t)1′nx(0)[D(t:0)y(0)]i+ϕi(t)1′ny(0)−1′nx(0)n.

Using the fact that and by bringing the fractions to a common denominator, we have

 zi(t+1)−¯x= [D(t:0)x(0)]i+ϕi(t)1′nx(0)[D(t:0)1n]i+nϕi(t)−1′nx(0)n = n[D(t:0)x(0)]i+nϕi(t)1′nx(0)n([D(t:0)1n]i+nϕi(t)) −[D(t:0)1n]i1′nx(0)+nϕi(t)1′nx(0)n([D(t:0)1n]i+nϕi(t)) = n[D(t:0)x(0)]i+[D(t:0)1n]i1′nx(0)n([D(t:0)1n]i+nϕi(t)).

Note that the denominator in the last equation is equal to . Hence, for all and we have

 |zi(t+1)−¯x|≤ ∥x(0)∥1yi(t+1)(maxj|[D(t:0)]ij|) +|1′nx(0)|nyi(t+1)(maxj|[D(t:0)]ij|)n = |1′nx(0)|+∥x(0)∥1yi(t+1)(maxj∣∣[D(t:0)]ij∣∣),

where the inequality follows from the triangle inequality. Since , we have that

 |zi(t+1)−¯x|≤2∥x(0)∥1yi(t+1)(maxj∣∣[D(t:0)]ij∣∣).

Using the upper bound in part (i) of Proposition 3, we obtain

 |zi(t+1)−¯x|≤2∥x(0)∥1yi(t+1)Λt,0. (14)

###### Proposition 4.

Consider the push-sum algorithm (4) and suppose that the sequence of random column-stochastic matrices has the directed infinite flow property, almost surely. Moreover, suppose that the sequence associated with satisfies (10), almost surely. If there exists , such that for any , there is such that for all , then

 limt→∞ |zi(t+1)−¯x|=0,almost surely.
###### Remark 1.

In the next section we exhibit a class of random matrix sequences that satisfy the conditions of Proposition 4 and thus admit average consensus almost surely.

###### Proof.

Proof of this proposition is similar to the proof of Theorem 4.1 in [16], where the sequence is assumed to be stationary; however, since we do not assume stationarity, we provide a proof. By Proposition 3 part (ii), for any there is a time such that for all and ,

 n∑j=1|[W(t:0)]ij−1nn∑k=1[W(t:0)]ik|<δε.

By assumption, there exists such that , which implies that , where is defined as in Lemma A.3. Since by Lemma A.3, is non-increasing, for all , meaning that converges to zero as and hence, , almost surely. ∎

## 7 B-Irreducible Sequences

In this section we characterize a class of random column-stochastic matrices that admits average consensus and we provide a rate of convergence of the push-sum algorithm for this class. To achieve this, we restrict the class of random matrices that we consider; as we will point out later, this restricted class still includes many interesting sequences of random matrices.

In the following discussion, we assume that the push-sum dynamics is generated by a column-stochastic matrix sequence where

 Wij(t)=Rij(t)∑ni=1Rij(t), (15)

for all , where is with probability , and is with probability such that are independent random variables. In other words, there is a random communication link between node and at time with probability . Note that is a sequence of independent random column-stochastic matrices.

Furthermore, for the probability matrix sequence , we assume that the following holds.

###### Assumption 1.

is a sequence of matrices with . Additionally, we assume that , for all . Also, for some constant , we assume that for all and all such that . Finally, we assume that the sequence is -irreducible, i.e. for some integer ,

 (t+1)B−1∑t′=tBP(t)

is irreducible for all .

We next state the main result of this section.

###### Theorem 2.

Consider the push-sum algorithm (4) and let be a sequence of random column-stochastic matrices defined by (15), where satisfies Assumption 1. Let . Then, for any , where

 E[ln(|zi(t+1)−¯x|)]≤c0−c1t

where

 c0= ln(2∥x(0)∥1)+ln(n)(nBp+B)+ln(15), c1= −p2nBln(1−1n4nBp).

The proof relies on the following results.

###### Lemma 1.

Let be a sequence of random column-stochastic matrices defined by (15), where satisfies Assumption 1. Let and be the sequences defined, respectively, in (8) and (9) along each sample path. Then

1. the sequence has the directed infinite flow property almost surely, and

2. for the sequence , we have

 ∞∑q=01nℓq=∞,% almost surely.
###### Proof.

We start by proving (i). For any , let us define the sequence of events

 At={(t+1)B−1∑t′=tBW(t′) is irreducible}. (16)

Note that for all , the events are independent and that implies , for any non-trivial . Since , for all , we have

 P(At)≥ϵ2(n−1).

This follows from [23, Corollary 5.3.6] and the fact that is -irreducible and hence, there is at least a subset of size of the edges that form a strongly connected graph and for some .

Since the events are independent, hence, by the second Borel-Contelli lemma [24, Theorem 2.3.6], infinitely often, almost surely. Moreover, since every positive entry of is bounded below by , for any non-trivial , , almost surely, implying that has the directed infinite flow property, almost surely. This also implies that and are finite for all , almost surely. This completes the proof of (i).

To prove (ii), let us define, for all ,

 Ct=(t+1)n−1⋂t′=tnAt′, (17)

where is defined in (16). Since the are independent, for all . This implies that . Again, since the are independent, by the Borel-Contelli lemma, occurs infinitely often, almost surely. This implies that infinitely often, almost surely. Hence, , almost surely. ∎

###### Lemma 2.

In the push-sum algorithm (4) let be a sequence of random column-stochastic matrices corresponding to the sequence satisfying Assumption 1. Then for all there exists such that for all , .

###### Proof.

Consider the event defined in (17). At any time occurs, by Lemma A.1, the product is positive; moreover, by Lemma A.2, for all . Since is column-stochastic, we have . By Lemma 1, occurs infinitely often, almost surely; therefore, for all there exists such that for all , . ∎

The preceding two lemmas and Proposition 4 imply the following.

###### Corollary 1.

Let be a sequence of random column-stochastic matrices corresponding to the sequence satisfying Assumption 1. Then admits average consensus, almost surely.

###### Lemma 3.

Let be a sequence of random column-stochastic matrices corresponding to the sequence satisfying Assumption 1. Let be the sequence defined in (9) along each sample path. For all , we have

 E[Λt,0]≤exp(−β2t(tB−2))+2(1−1n4nBp)pt2nB,

where , , and .

###### Proof.

Let be the indicator of the event , i.e.,

 XB(t)={1if ∑(t+1)B−1t′=tBW(t′) is irreducible,0otherwise.

By the preceding argument, we have . Note that the are independent. We let for all , and define

 qt≜max{q:kq≤t}.

By definition of and , we have that

 qt≥HB(⌊tB⌋−1). (18)

Now, we have that

 E[Λt,0]= E[Λt,0 ∣∣∣ qt≤pt2B]P(qt≤pt2B) +E[Λt,0 ∣∣∣ qt>pt2B]P(qt>pt2B).

Since all terms on the right-hand side are less than or equal to , we have

 E[Λt,0]≤P(qt≤pt2B)+E[Λt,0 ∣∣∣ qt>pt2B].

Using (18), we have

 E[Λt,0]≤ P(HB(⌊tB⌋−1)≤pt2B) +E[Λt,0 ∣∣∣ qt>pt2B].

Let us consider the second term on the right-hand side. When , we have . Using Lemma A.5 to maximize the second term on the right-hand side over the choices of , we obtain

 E[Λt,0 ∣∣∣ qt>pt2B]≤ (19) ≤ (20)

To further simplify the above inequality, we show that . To show this, we note that for all , we have and hence, . Now, assume that . We have . Therefore,

 t⌊pt2nB⌋≤2nBp(ξ+1ξ)≤4nBp,

where the last inequality follows from the fact that .

Using this inequality in (19), we get

 E[Λt,0 ∣∣∣ qt>pt2B]≤ (21) ≤ 2(1−1n4nBp)pt2nB. (22)

On the other hand, since for all , we have

 P(H(⌊tB⌋−1)≤pt2B) =P⎛⎝⌊t/B⌋−1∑t′=0X