Block-Structured Supermarket ModelsThe main results of this paper will be published in ”Discrete Event Dynamic Systems” 2014. On the other hand, the three appendices are the online supplementary material for this paper published in ”Discrete Event Dynamic Systems” 2014

# Block-Structured Supermarket Models1

## Abstract

Supermarket models are a class of parallel queueing networks with an adaptive control scheme that play a key role in the study of resource management of, such as, computer networks, manufacturing systems and transportation networks. When the arrival processes are non-Poisson and the service times are non-exponential, analysis of such a supermarket model is always limited, interesting, and challenging.

This paper describes a supermarket model with non-Poisson inputs: Markovian Arrival Processes (MAPs) and with non-exponential service times: Phase-type (PH) distributions, and provides a generalized matrix-analytic method which is first combined with the operator semigroup and the mean-field limit. When discussing such a more general supermarket model, this paper makes some new results and advances as follows: (1) Providing a detailed probability analysis for setting up an infinite-dimensional system of differential vector equations satisfied by the expected fraction vector, where the invariance of environment factors is given as an important result. (2) Introducing the phase-type structure to the operator semigroup and to the mean-field limit, and a Lipschitz condition can be obtained by means of a unified matrix-differential algorithm. (3) The matrix-analytic method is used to compute the fixed point which leads to performance computation of this system. Finally, we use some numerical examples to illustrate how the performance measures of this supermarket model depend on the non-Poisson inputs and on the non-exponential service times. Thus the results of this paper give new highlight on understanding influence of non-Poisson inputs and of non-exponential service times on performance measures of more general supermarket models.

Keywords: Randomized load balancing; Supermarket model; Matrix-analytic method; Operator semigroup; Mean-field limit; Markovian arrival processes (MAP); Phase-type (PH) distribution; Invariance of environment factors; Doubly exponential tail; -factorization.

## 1 Introduction

Supermarket models are a class of parallel queueing networks with an adaptive control scheme that play a key role in the study of resource management of, such as computer networks (e.g., see the dynamic randomized load balancing), manufacturing systems and transportation networks. Since a simple supermarket model was discussed by Mitzenmacher [23], Vvedenskaya et al [32] and Turner [30] through queueing theory as well as Markov processes, subsequent papers have been published on this theme, among which, see, Vvedenskaya and Suhov [33], Jacquet and Vvedenskaya [8], Jacquet et al [9], Mitzenmacher [24], Graham [5, 6, 7], Mitzenmacher et al [25], Vvedenskaya and Suhov [34], Luczak and Norris [20], Luczak and McDiarmid [18, 19], Bramson et al [1, 2, 3], Li et al [17], Li [13] and Li et al [15]. For the fast Jackson networks (or the supermarket networks), readers may refer to Martin and Suhov [22], Martin [21] and Suhov and Vvedenskaya [29].

The available results of the supermarket models with non-exponential service times are still few in the literature. Important examples include an approximate method of integral equations by Vvedenskaya and Suhov [33], the Erlang service times by Mitzenmacher [24] and Mitzenmacher et al [25], the PH service times by Li et al [17] and Li and Lui [16], and the ansatz-based modularized program for the general service times by Bramson et al [1, 2, 3].

Little work has been done on the analysis of the supermarket models with non-Poisson inputs, which are more difficult and challenging due to the higher complexity of that arrival processes are superposed. Li and Lui [16] and Li [12] used the superposition of MAP inputs to study the infinite-dimensional Markov processes of supermarket modeling type. Comparing with the results given in Li and Lui [16] and Li [12], this paper provides more necessary phase-level probability analysis in setting up the infinite-dimensional system of differential vector equations, which leads some new results and methodologies in the study of block-structured supermarket models. Note that the PH distributions constitute a versatile class of distributions that can approximate arbitrarily closely any probability distribution defined on the nonnegative real line, and the MAPs are a broad class of renewal or non-renewal point processes that can approximate arbitrarily closely any stochastic counting process (e.g., see Neuts [27, 28] and Li [11] for more details), thus the results of this paper are a key advance of those given in Mitzenmacher [23] and Vvedenskaya et al [32] under the Poisson and exponential setting.

The main contributions of this paper are threefold. The first one is to use the MAP inputs and the PH service times to describe a more general supermarket model with non-Poisson inputs and with non-exponential service times. Based on the phase structure, we define the random fraction vector and construct an infinite-dimensional Markov process, which expresses the state of this supermarket model by means of an infinite-dimensional Markov process. Furthermore, we set up an infinite-dimensional system of differential vector equations satisfied by the expected fraction vector through a detailed probability analysis. To that end, we obtain an important result: The invariance of environment factors, which is a key for being able to simplify the differential equations in a vector form. Based on the differential vector equations, we can provide a generalized matrix-analytic method to investigate more general supermarket models with non-Poisson inputs and with non-exponential service times. The second contribution of this paper is to provide phase-structured expression for the operator semigroup with respect to the MAP inputs and to the PH service times, and use the operator semigroup to provide the mean-field limit for the sequence of Markov processes who asymptotically approaches a single trajectory identified by the unique and global solution to the infinite-dimensional system of limiting differential vector equations. To prove the existence and uniqueness of solution through the Picard approximation, we provide a unified computational method for establishing a Lipschitz condition, which is crucial in all the rigor proofs involved. The third contribution of this paper is to provide an effective matrix-analytic method both for computing the fixed point and for analyzing performance measures of this supermarket model. Furthermore, we use some numerical examples to indicate how the performance measures of this supermarket model depend on the non-Poisson MAP inputs and on the non-exponential PH service times. Therefore, the results of this paper gives new highlight on understanding performance analysis and nonlinear Markov processes for more general supermarket models with non-Poisson inputs and non-exponential service times.

The remainder of this paper is organized as follows. In Section 2, we first introduce a new MAP whose transition rates are controlled by the number of servers in the system. Then we describe a more general supermarket model of identical servers with MAP inputs and PH service times. In Section 3, we define a random fraction vector and construct an infinite-dimensional Markov process, which expresses the state of this supermarket model. In Section 4, we set up an infinite-dimensional system of differential vector equations satisfied by the expected fraction vector through a detailed probability analysis, and establish an important result: The invariance of environment factors. In Section 5, we show that the mean-field limit for the sequence of Markov processes who asymptotically approaches a single trajectory identified by the unique and global solution to the infinite-dimensional system of limiting differential vector equations. To prove the existence and uniqueness of the solution, we provide a unified matrix-differential algorithm for establishing the Lipschitz condition. In Section 6, we first discuss the stability of this supermarket model in terms of a coupling method. Then we provide a generalized matrix-analytic method for computing the fixed point whose doubly exponential solution and phase-structured tail are obtained. Finally, we discuss some useful limits of the fraction vector as and . In Section 7, we provide two performance measures of this supermarket model, and use some numerical examples to indicate how the performance measures of this system depend on the non-Poisson MAP inputs and on the non-exponential PH service times. Some concluding remarks are given in Section 8. Finally, Appendices A and C are respectively designed for the proofs of Theorems 1 and 3, and Appendix B contains the proof of Theorem 2, where the mean-field limit of the sequence of Markov processes in this supermarket model is given a detailed analysis through the operator semigroup.

## 2 Supermarket Model Description

In this section, we first introduce a new MAP whose transition rates are controlled by the number of servers in the system. Then we describe a more general supermarket model of identical servers with MAP inputs and PH service times.

### 2.1 A new Markovian arrival process

Based on Chapter 5 in Neuts [28], the MAP is a bivariate Markov process with state space , where is a counting process of arrivals and is a Markov environment process. When , if the random environment shall go to state in the next time, then the counting process is a Poisson process with arrival rate for . The matrix with elements satisfies . The matrix with elements has negative diagonal elements and nonnegative off-diagonal elements, and the matrix is invertible, where is a state transition rate of the Markov chain from state to state for . The matrix is the infinitesimal generator of an irreducible Markov chain. We assume that , where is a column vector of ones with a suitable size. Hence, we have

 ci,i=−⎡⎣mA∑j=1di,j+mA∑j≠ici,j⎤⎦.

Let

 C=⎛⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜⎝−mA∑j≠1c1,jc1,2⋯c1,mAc2,1−mA∑j≠2c2,j⋯c2,mA⋮⋮⋱⋮cmA,1cmA,2⋯−mA∑j≠mAcmA,j⎞⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟⎠,
 C(N)=C−Ndiag(De),
 D(N)=ND,

where

 diag(De)=diag(mA∑j=1d1,j,mA∑j=1d2,j,…,mA∑j=1dmA,j).

Then

 Q(N)=C(N)+D(N)=[C−Ndiag(De)]+ND

is obviously the infinitesimal generator of an irreducible Markov chain with states. Thus is the irreducible matrix descriptor of a new MAP of order . Note that the new MAP is non-Poisson and may also be non-renewal, and its arrival rate at each environment state is controlled by the number of servers in the system.

Note that

 Q(N)e=[C−Ndiag(De)]e+NDe=0,

the Markov chain with states is irreducible and positive recurrent. Let be the stationary probability vector of the Markov chain . Then depends on the number , and the stationary arrival rate of the MAP is given by .

### 2.2 Model description

Based on the new MAP, we describe a more general supermarket model of identical servers with MAP inputs and PH service times as follows:

Non-Poisson inputs: Customers arrive at this system as the MAP of irreducible matrix descriptor of size , whose stationary arrival rate is given by .

Non-exponential service times: The service times of each server are i.i.d. and are of phase type with an irreducible representation of order , where the row vector is a probability vector whose th entry is the probability that a service begins in phase for ; is a matrix of size  whose entry is denoted by with for , and for . Let , where “” denotes the transpose of matrix (or vector) . When a PH service time is in phase , the transition rate from phase to phase is , the service completion rate is , and the output rate from phase is . At the same time, the mean of the PH service time is given by .

Arrival and service disciplines: Each arriving customer chooses servers independently and uniformly at random from the identical servers, and waits for its service at the server which currently contains the fewest number of customers. If there is a tie, servers with the fewest number of customers will be chosen randomly. All customers in any server will be served in the FCFS manner. Figure 1 gives a physical interpretation for this supermarket model.

###### Remark 1

The block-structured supermarket models can have many practical applications to, such as, computer networks and manufacturing system, where it is a key to introduce the PH service times and the MAP inputs to such a practical model, because the PH distributions contain many useful distributions such as exponential, hyper-exponential and Erlang distributions; while the MAPs include, for example, Poisson process, PH-renewal processes, and Markovian modulated Poisson processes (MMPPs). Note that the probability distributions and stochastic point processes have extensively been used in most practical stochastic modeling. On the other hand, in many practical applications, the block-structured supermarket model is an important queueing model to analyze the relation between the system performance and the job routing rule, and it can also help to design reasonable architecture to improve the performance and to balance the load.

## 3 An Infinite-Dimensional Markov Process

In this section, we first define the random fraction vector of this supermarket model. Then we use the the random fraction vector to construct an infinite-dimensional Markov process, which describes the state of this supermarket model.

For this supermarket model, let be the number of servers with at least customers (note that the serving customer is also taken into account), and with the MAP be in phase and the PH service time be in phase at time . Clearly, and for , and . Let

 U(N)0;i(t)=n(N)0;i(t)N, \ 1≤i≤mA,

and for

 Missing or unrecognized delimiter for \left

Then is the fraction of servers with at least customers, and with the MAP be in phase and the PH service time be in phase at time . Using the lexicographic order we write

 U(N)0(t)=(U(N)0;1(t),U(N)0;2(t),…,U(N)0;mA(t))

for

 U(N)k(t)= (U(N)k;1,1(t),U(N)k;1,2(t),…,U(N)k;1,mB(t);…; U(N)k;mA,1(t),U(N)k;mA,2(t),…,U(N)k;mA,mB(t)),

and

 U(N)(t)=(U(N)0(t),U(N)1(t),U(N)2(t),…). (1)

Let and . We write if for some ; if for every .

For a fixed quaternary array with and , it is easy to see from the stochastic order that for . This gives

 U(N)1(t)≥U(N)2(t)≥U(N)3(t)⋯≥0 (2)

and

 1=U(N)0(t)e≥U(N)1(t)e≥U(N)2(t)e≥U(N)3(t)e≥⋯≥0. (3)

Note that the state of this supermarket model is described as the random fraction vector for , and is a stochastic vector process for each . Since the arrival process to this supermarket model is the MAP and the service times in each server are of phase type, is an infinite-dimensional Markov process whose state space is given by

 ˜ΩN= {(h(N)0,h(N)1,h(N)2…):h(N)0 is a probability % vector of size mA, h(N)1≥h(N)2≥h(N)3≥⋯≥0,h(N)k is a row vector of size mAmB for k≥1, 1=h(N)0e≥h(N)1e≥h(N)2e≥⋯≥0, and \ Nh(N)k \ is a row % vector of nonnegative integers for k≥0}, (4)

We write

 u(N)0;i(t)=E[U(N)0;i(t)]

and for

 u(N)k;i,j(t)=E[U(N)k;i,j(t)].

Using the lexicographic order we write

 u(N)0(t)=(u(N)0;1(t),u(N)0;2(t),…,u(N)0;mA(t))

and for

 u(N)k(t)= (u(N)k;1,1(t),u(N)k;1,2(t),…,u(N)k;1,mB(t);…; u(N)k;mA,1(t),u(N)k;mA,2(t),…,u(N)k;mA,mB(t)),
 u(N)(t)=(u(N)0(t),u(N)1(t),u(N)2(t),…).

It is easy to see from Equations (2) and (3) that

 u(N)1(t)≥u(N)2(t)≥u(N)3(t)⋯≥0 (5)

and

 1=u(N)0(t)e≥u(N)1(t)e≥u(N)2(t)e≥⋯≥0. (6)

In the remainder of this section, for convenience of readers, it is necessary to explain the structure of this long paper which is outlined as follows. Part one: The limit of the sequence of Markov processes. It is seen from (1) and (4) that we need to deal with the limit of the sequence of infinite-dimensional Markov processes. This is organized in Appendix B by means of the convergence theorems of operator semigroups, e.g., see Ethier and Kurtz [4] for more details. Part two: The existence and uniqueness of the solution. As seen from Theorem 2 and (27), we need to study the two means and , or and . To that end, Section 4 sets up the system of differential vector equations satisfied by , while Section 5 provides a unified matrix-differential algorithm for establishing the Lipschitz condition, which is a key in proving the existence and uniqueness of the solution to the limiting system of differential vector equations satisfied by  through the Picard approximation. Part three: Computation of the fixed point and performance analysis. Section 6 discusses the stability of this supermarket model in terms of a coupling method, and provide an effective matrix-analytic method for computing the fixed point. Section 7 analyzes the performance of this supermarket model by means of some numerical examples.

## 4 The System of Differential Vector Equations

In this section, we set up an infinite-dimensional system of differential vector equations satisfied by the expected fraction vector through a detailed probability analysis. Specifically, we obtain an important result: The invariance of environment factors, which is a key to rewriting the differential equations as a simple vector form.

To derive the system of differential vector equations, we first discuss an example with the number of customers through the following three steps:

Step one: Analysis of the Arrival Processes

In this supermarket model of identical servers, we need to determine the change in the expected number of servers with at least customers over a small time period . When the MAP environment process jumps form state to state for and the PH service environment process sojourns at state for , one arrival occurs in a small time period . In this case, the rate that any arriving customer selects servers with at least customers at random and joins the shortest one with customers, is given by

 Missing or unrecognized delimiter for \right ×L(N)k;l(uk−1(t),uk(t))Ndt, (7)

where

 Missing or unrecognized delimiter for \left +d−1∑m=1Cmd{mB∑j=1[uk−1;l,j(N)(t)−u(N)k;l,j(t)]}m−1∑r1+r2+⋯+rmA=d−m∑mAi≠lri≥10≤rj≤d−m,1≤j≤mA(d−mr1,r2,…,rmA) ×mA∏i=1{mB∑j=1[u(N)k;i,j(t)]}ri+d∑m=2Cmdm−1∑m1=1m1mCm1m{mB∑j=1[u(N)k−1;l,j(t)−u(N)k;l,j(t)]}m1−1
 Missing or unrecognized delimiter for \right ×∑r1+r2+⋯+rmA=d−m0≤rj≤d−m,1≤j≤mA(d−mr1,r2,…,rmA)mA∏i=1{mB∑j=1[u(N)k;i,j(t)]}ri. (8)

Note that is the rate that any arriving customer joins one server with the shortest queue length , where the MAP goes to phase from phase , and the PH service time is in phase .

Now, we provide a detailed interpretation for how to derive (8) through a set decomposition of all possible events given in Figure 2, where each of the selected servers has at least customers, the MAP arrival environment is in phase or , and the PH service environment is in phase . Hence, the probability that any arriving customer selects servers with at least customers at random and joins a server with the shortest queue length and with the MAP phase or is determined by means of Figure 2 through the following three parts:

Part I: The probability that any arriving customer joins a server with the shortest queue length and with the MAP phase , and the queue lengths of the other selected servers are not shorter than , is given by

 d∑m=1Cmd{mB∑j=1[u(N)k−1;l,j(t)−u(N)k;l,j(t)]}m−1{mB∑j=1[u(N)k;l,j(t)]}d−m,

where is a binomial coefficient, and

 {mB∑j=1[u(N)k−1;l,j(t)−u(N)k;l,j(t)]}m−1

is the probability that any arriving customer who can only choose one server makes independent selections during the servers with the queue length and with the MAP phase at time ; while is the probability that there are servers whose queue lengths are not shorter than and with the MAP phase .

Part II: The probability that any arriving customer joins a server with the shortest queue length and with the MAP phase ; and the queue lengths of the other selected servers are not shorter than , and there exist at least one server with no less than customers and with the MAP phase , is given by

 d−1∑m=1Cmd{mB∑j=1[u(N)k−1;l,j(t)−u(N)k;l,j(t)]}m−1 ×∑r1+r2+⋯+rmA=d−m∑mAi≠lri≥10≤rj≤d−m,1≤j≤mA(d−mr1,r2,…,rmA)mA∏i=1{mB∑j=1[u(N)k;i,j(t)]}ri,

where when , is a multinomial coefficient.

Part III: If there are selected servers with the shortest queue length where there are servers with the MAP phase and servers with the MAP phases , then the probability that any arriving customer joins a server with the shortest queue length and with the MAP phase is equal to . In this case, the probability that any arriving customer joins a server with the shortest queue length and with the MAP phase , the queue lengths of the other selected servers are not shorter than , is given by

 d∑m=2Cmdm−1∑m1=1m1mCm1m{mB∑j=1[u(N)k−1;l,j(t)−u(N)k;l,j(t)]}m1−1 Missing or unrecognized delimiter for \right ×∑r1+r2+⋯+rmA=d−m0≤rj≤d−m,1≤j≤mA(d−mr1,r2,…,rmA)mA∏i=1{mB∑j=1[u(N)k;i,j(t)]}ri.

Using the above three parts, (7) and (8) can be obtained immediately.

For any two matrices and , their Kronecker product is defined as , and their Kronecker sum is given by .

The following theorem gives an important result, called the invariance of environment factors, which will play an important role in setting up the infinite-dimensional system of differential vector equations. This enables us to apply the matrix-analytic method to the study of more general supermarket models with non-Poisson inputs and non-exponential service times.

###### Theorem 1
 L(N)1;l(u(N)0(t)⊗α,u(N)1(t))= d∑m=1Cmd[mA∑l=1mB∑j=1(u(N)0;l(t)αj−u(N)1;l,j(t))]m−1 ×[mA∑l=1mB∑j=1u(N)1;l,j(t)]d−m (9)

and for

 L(N)k;l(u(N)k−1(t),u(N)k(t))= d∑m=1Cmd[mA∑l=1mB∑j=1(u(N)k−1;l,j(t)−u(N)k;l,j(t))]m−1 ×[mA∑l=1mB∑j=1u(N)k;l,j(t)]d−m. (10)

Thus  and for  are independent of the MAP phase . In this case, we have

 L(N)1;l(u(N)0(t)⊗α,u(N)1(t))def=L(N)1(u(N)0(t)⊗α,u(N)1(t)) (11)

and for

 L(N)k;l(u(N)k−1(t),u(N)k(t))def=L(N)k(u(N)k−1(t),u(N)k(t)). (12)

Proof: See Appendix A.

It is seen from the invariance of environment factors in Theorem 1 that Equation (7) is rewritten as, in a vector form,

 {u(N)k−1(t)(D⊗I)−u(N)k(t)[diag(De)⊗I]} ×L(N)k(u(N)k−1(t),u(N)k(t))Ndt. (13)

Note that and are scale for .

Step two: Analysis of the Environment State Transitions in the MAP

When there are at least customers in the server, the rate that the MAP environment process jumps from state to state with rate , and no arrival of the MAP occurs during a small time period , is given by

 [mA∑l=1u(N)k;l,j(t)cl,i+uk,i,j(N)(t)(di,1,di,2,…,di,mA)e]Ndt.

This gives, in a vector form,

 u(N)k(t)([C+diag(De)]⊗I)Ndt. (14)

Step three: Analysis of the Service Processes

To analyze the PH service process, we need to consider the following two cases:

Case one: One service completion occurs with rate during a small time period . In this case, when there are at least customers in the server, the rate that a customer is completed its service with entering PH phase and the MAP is in phase  is given by

 [u(N)k+1;i,1(t)t01αj+u(N)k+1;i,2(t)t02αj+⋯+u(N)k+1;i,mB(t)t0mBαj]N%dt.

Case two: No service completion occurs during a small time period , but the MAP is in phase and the PH service environment process goes to phase . Thus, when there are at least customers in the server, the rate of this case is given by

 [u(N)k;i,1(t)t1,j+u(N)k;i,2(t)t2,j+u(N)k;i,3(t)t3,j+⋯+u(N)k;i,mB(t)tmB,j]Ndt.

Thus, for the PH service process, we obtain that in a vector form,

 [u(N)k(t)(I⊗T)+u(N)k+1(t)(I⊗T0α)]Ndt (15)

Let

 n(N)k(t)= (n(N)k;1,1(t),n(N)k;1,2(t),…,n(N)k;1,mB(t);…; n(N)k;mA,1(t),n(N)k;mA,2(t),…,n(N)k;mA,mB(t)).

Then it follows from Equation (13) to (15) that

 dE[n(N)k(t)]= {{u(N)k−1(t)(D⊗I)−u(N)k(t)[diag(De)⊗I]}L(N)k(u(N)k−1(t),u(N)k(t)) +u(N)k(t){[C+% diag(De)]⊕T}+u(N)k+1(t)(I⊗T0α)}Ndt.

Since and , we obtain

 du(N)k(t)dt= {u(N)k−1(t)(D⊗I)−u(N)k(t)(t)[diag(De)⊗I]}L(N)k(u(N)k−1(t),u(N)k(t)) +u(N)k(t){[C+diag(De)]⊕T}+u(N)k+1(t)(I⊗T0α). (16)

Using a similar analysis to Equation (16), we obtain an infinite-dimensional system of differential vector equations satisfied by the expected fraction vector as follows:

 du(N)1(t)dt= {[u(N)0(t)⊗α](D⊗I)−u(N)1(t)[%diag(De)⊗I]}L(N)1(u(N)0(t)⊗α,u(N)1(t)) +u(N)1(t){[C+diag(De)]⊕T}+u(N)2(t)(I⊗T0α), (17)

and for

 du(N)k(t)dt= {u(N)k−1(t)(D⊗I)−u(N)k(t)[diag(De)⊗I]}L(N)k(u(N)k−1(t),u(N)k(t)) +u(N)k(t){[C+diag(De)]⊕T}+u(N)k+1(t)(I⊗T0α), (18)

with the boundary condition

 du(N)0(t)dt=u(N)0(t)(C+D), (19)
 u(N)0(t)e=1; (20)

and with the initial condition

 u(N)k(0)=gk, k≥1, (21)

where

 g1≥g2≥g3≥⋯≥0

and

 1=g0e≥g1e≥g2e≥⋯≥0.
###### Remark 2

It is necessary to explain some probability setting for the invariance of environment factors. It follows from Theorem 1 that

 Missing or unrecognized delimiter for \right

and for

 Missing or unrecognized delimiter for \left

Note that the two expressions will be useful in our later study, for example, establishing the Lipschitz condition, and computing the fixed point. Specifically, for we have

 L(N)1(u(N)0(t)⊗α,u(N)1(t))=1

and for

 L(N)k(u(N)k−1(t),u(N)k(t))=1.

For we have

 L(N)1(u(N)0(t)⊗α,u(N)1(t))=u(N)0(t)e+u(N)1(t)e>1

and for

 L(N)k(u(N)k−1(t),u(N)k(t))=u(N)k−1(t)e+u(N)k(t)e.

This shows that is not a probability vector.

## 5 The Lipschitz Condition

In this section, we show that the mean-field limit of the sequence of Markov processes asymptotically approaches a single trajectory identified by the unique and global solution to the infinite-dimensional system of limiting differential vector equations. To that end, we provide a unified matrix-differential algorithm for establishing the Lipschitz condition, which is a key in proving the existence and uniqueness of the solution by means of the Picard approximation according to the basic results of the Banach space.

Let be the operator semigroup of the Markov process . If , where , then for and