# Distributed Game Theoretic Optimization and Management of Multichannel ALOHA Networks

## Abstract

The problem of distributed rate maximization in multi-channel ALOHA networks is considered. First, we study the problem of constrained distributed rate maximization, where user rates are subject to total transmission probability constraints. We propose a best-response algorithm, where each user updates its strategy to increase its rate according to the channel state information and the current channel utilization. We prove the convergence of the algorithm to a Nash equilibrium in both homogeneous and heterogeneous networks using the theory of potential games. The performance of the best-response dynamic is analyzed and compared to a simple transmission scheme, where users transmit over the channel with the highest collision-free utility. Then, we consider the case where users are not restricted by transmission probability constraints. Distributed rate maximization under uncertainty is considered to achieve both efficiency and fairness among users. We propose a distributed scheme where users adjust their transmission probability to maximize their rates according to the current network state, while maintaining the desired load on the channels. We show that our approach plays an important role in achieving the Nash bargaining solution among users. Sequential and parallel algorithms are proposed to achieve the target solution in a distributed manner. The efficiencies of the algorithms are demonstrated through both theoretical and simulation results.

Collision channels, multi-channel ALOHA, best-response dynamics, Nash equilibrium, Nash bargaining solution, potential games.

## 1Introduction

Random access schemes have been widely used for data transmission of a large number of users sharing a common channel. In multi-channel systems, the users transmit over orthogonal channels (i.e., sub-bands) using Orthogonal Frequency Division Multiple Access (OFDMA). Each channel can be a cluster of multiple carriers. A common way to increase user rates in multi-channel systems is to exploit the channel diversity using local channel state information (CSI). Recently, multi-channel systems have been studied extensively in wireless communication [1].

In this paper we examine Medium Access Control (MAC) schemes used to enable a large number of users to co-exist in a typically low number of shared channels. We investigate multi-channel ALOHA networks, where users access a channel according to a specific transmission probability. Transmission is successful if only a single user transmits over a shared channel in a given time-slot. However, if two or more users transmit simultaneously over the same channel, a collision occurs. ALOHA-based protocols are widely used in wireless communication primarily because of their ease of implementation and their random nature. Simple transmitters can randomly access a channel without a carrier sensing operation. Past and recent works on single and multi-channel ALOHA networks can be found in [7] and references therein. In [7], stability of multi-channel networks in which a single channel is chosen randomly (from a uniform distribution) for transmission among multiple channels is studied. In [8], a multi-channel ALOHA model, in which a single channel is used for transmissions of new packets and other channels for retransmissions, was analyzed. A Price of Anarchy (PoA) of Nash equilibria in multi-channel ALOHA networks is studied in [9]. Queuing delay analysis for a single-channel ALOHA is provided in [10]. Analysis of a generalized ALOHA protocol under adversarial environments is given in [11].

In wireless communication networks, distributed algorithms are generally preferred over centralized solutions. In this paper we mainly focus on distributed algorithms in multi-channel ALOHA networks. We examine distributed algorithms with dynamic systems where users make autonomous decisions based on local information. Such techniques have been presented in the literature. A related work on distributed optimization in cognitive radio networks can be found in [12]. The problem of distributed learning in cognitive radio networks using multi-armed bandit technique with distributed multiple players was investigated in [15], where the number of channels is greater than the number of users and users implement carrier sensing operation before transmission. However, in this paper we adopt the ALOHA protocol for transmissions and the number of users is typically greater than the number of channels. The problem of multi-radio multi-channel allocation was investigated in [16]. In [17], a distributed learning algorithm was proposed that converges in some special cases. In the multi-radio multi-channel allocation model, the utility of each channel decreases with the number of radios transmitting over it. This is generally done by a TDMA protocol, for instance, among users who transmit over the same channel. As a result, users are encouraged to spread resources over channels. In this paper, however, the achievable rate of a user on a channel increases with the transmission probability (based on the ALOHA-network model) which results in strategies that allocate more resources on better channels. In [19], the multi-channel ALOHA protocol in cognitive radio networks was analyzed, where the focus is on a hierarchical model of primary and secondary users in the network. The secondary users choose randomly one of the idle channels for transmission. In this paper, however, we focus on the open sharing model among users (e.g., ISM band), in which users exploit local information to choose better channels for transmissions. In [20], the opportunistic multi-channel ALOHA scheme was analyzed for i.i.d Rayleigh fading channels. In this scheme, a user transmits over channels with instantaneous gains greater than some threshold. In this paper, however, long-term rates are assumed (i.e., mean-rates) and the interference caused by other users is also taken into consideration when designing effective algorithms for the spectrum access problem.

There is a significant amount of work in wireless networking that make use of game theory. Related works on networking games can be found in [22]. Random access games were studied in [25]. Game theoretic techniques were used in [25] to analyze single-channel ALOHA networks. In [25], distributed optimization algorithms of single-channel ALOHA networks using game theoretic tools are studied, where the utility of each user increases with the transmission probability. Here, we consider a similar model. Specifically, in [26] energy-efficient Nash equilibria under user-rate demands have been established. However, the analysis of the energy-efficient equilibria does not hold under the multi-channel setting. Here, we extend this model to a multi-channel setting and study a distributed optimization of the user rates under constraints on the transmission probabilities. Another related work considered a non-cooperative power control game in multichannel networks with energy-efficiency perspectives [27], where the goal is to maximize the number of reliable bits transmitted per joule of energy consumed in a distributed fashion. In this paper, however, we focus on efficiency and fairness with respect to the achievable rates across users.

Cooperative game theory has been widely used to study channel sharing problems in wireless communication networks. In a non-cooperative game, players individually attempt to maximize their own utility regardless of the utility achieved by other players. On the other hand, in a cooperative game, players bargain with each other. If an agreement is reached, they act according to the agreement. If they disagree, they do not cooperate [32]. An efficient solution for cooperative games is the Nash Bargaining Solution (NBS) [33]. In recent years, the NBS has been analyzed for the frequency flat *interference* channel in the SISO [34], MISO [36] and MIMO cases [38], as well as for a frequency selective interference channel [39]. In this paper, however, we apply cooperative game theoretic techniques to analyze the efficiency of our approach for the channel sharing problem over *collision* channels in multi-channel ALOHA networks.

In our previous work [4] we mainly focused on networks containing homogeneous users, where all users have the same transmission probability constraint. However, in this paper we focus on more general heterogenous networks, where each user in the network is allowed to transit with a different probability. Handling such cases creates additional challenges when designing effective protocols for the system. First, fairness should be considered when defining the target solutions for all users. Second, further refinements of the user dynamics are required to stabilize the system.

First, we consider the case where heterogenous users exploit their own CSI and the channel utilization to increase their utility, where each user in the network has an individual transmission probability constraint. We present the best-response algorithm that solves the distributed rate maximization. A best-response approach is a common method in non-cooperative games to achieve a Nash Equilibrium Point (NEP) [44]. The idea of best-response dynamics is that every user produces its best response in terms of the current state of all other users. Here, users need to decide which channels to access to improve their utility. The proposed best-response dynamics in this paper enable users to make autonomous decisions using their local CSI and by monitoring the load on the channels. We show that users’ dynamic behavior obeys a global potential function [47], which implies the convergence of the dynamics.

Next, we study a simpler transmission scheme where users transmit over the channel with the highest collision-free utility (i.e., the utility that the user receives conditioned on the event that the channel is free), which is an approximate solution to the best-response dynamics as increases. The performance of the best-response dynamic are analyzed as compared to this simple transmission scheme for a finite , which serves as a benchmark of the performance that could be obtained by exploiting the channel utilization. We also propose a centralized log-concave optimization problem to determine the transmission probabilities of heterogeneous users under this setting.

Finally, we consider the case where users are not restricted by a transmission probability constraint. Users are required to implement a distributed rate maximization under uncertainty since the transmission probabilities of the other users are unknown. In this case, fairness must be taken into consideration when formulating the target solution for all users. We examine the problem from a cooperative game theoretic perspective. We suggest a distributed learning scheme, where users adjust their transmission probability based on local information only to achieve the desired load on the channels to maximize their rates. We show that our approach plays an important role in achieving NBS among users. We propose sequential and parallel algorithms to reach the target solution in a distributed manner. The efficiencies of the algorithms are demonstrated through both theoretical and simulation results. Specifically, we show that the global NBS of the network can be achieved by both the sequential and parallel algorithms under mild conditions on user utilities.

The rest of this paper is organized as follows. In Section 2 we present the network model for the multi-channel ALOHA system. In Section 3 we focus on distributed dynamics for the distributed rate maximization problem under given transmission probability constraints. In Section 4 we focus on simpler solutions to rate maximization using CSI alone. In Sections Section 5 and Section 6 we discuss cooperative game considerations and distributed algorithms for the rate maximization problem under uncertainty of user transmission probabilities. In Section 7 we provide simulation results to demonstrate the algorithms performance.

## 2Network Model

We consider a wireless network containing users who transmit over orthogonal channels, where . The users transmit over the shared channels using the slotted ALOHA protocol. In each time slot each user is allowed to access a single channel according to a specific transmission probability. Transmission is successful if only a single user transmits over a shared channel in a given time-slot. However, if two or more users transmit simultaneously over the same channel, a collision occurs. We assume that users are backlogged, i.e., all users always have packets to transmit. The achievable rate of user at channel given that the channel is free, referred to as collision-free utility, is denoted by and is proportional to the bandwidth of channel . For convenience, we define as a virtual zero-rate channel. Transmitting over channel refers to no-transmission. Throughout the paper, it is assumed that the collision-free utilities are fixed during the running-time of the algorithm (i.e, represents the mean-rate or long-term rate where the channel statistics change slowly). It is assumed that every user knows its own collision-free utility, while collision-free utilities of other users are unknown. The collision-free rate matrix of all users in all channels is given by:

Let be the probability that user transmits over channel . Let be the set of all transmission probability vectors of user in all channels. A transmission probability vector of user is given by:

Since we are mainly interested in high-loaded systems, where the number of users is greater (or even much greater) than the number of channels, it is desirable to limit the congestion level over the channels. Thus, we consider only single-channel strategies, where every user selects a single channel for transmission:

for some , and for all . We define as the set of all transmission probability matrices of all users in all channels. The probability matrix is given by:

where .

We define as the set of all probability matrices of all users in all channels, except user . The probability matrix is given by:

When user perfectly monitors the channel utilization^{1}

which is the success probability of user on channel . Roughly speaking, can be viewed as the load that user observes on channel . Increasing decreases the rate that user can achieve over channel .

We further define

which is the probability that channel is not used by the users.

The expected rate of user in the channel is given by:

Hence, the expected rate of user is given by:

## 3The Distributed Rate Maximization Problem

In this section we extend the results reported in [4] for the special case of a homogenous network to the general case of a heterogenous network, where every user may have a different probability constraint. Throughout this section we consider a non-cooperative setting in the sense that every user maximizes its own rate under a constraint on the allowed transmission probability. Thus, the constraints on the attempt probabilities are used to prioritize users in the network^{2}

We are interested in solving the distributed rate maximization problem, where each user tries to maximize its own expected rate subject to a total transmission probability constraint:

Since we are mainly interested in high-loaded systems, throughout the paper we restrict users to select at most a single channel for transmission (to reduce the collision level). Thus, . Note that when user solves (Equation 10) given the current system state, the resulting strategy is given by:

where^{3}

Note that in practical systems, is generally estimated from a pilot signal. On the other hand, complete information on matrix is not required. Knowing the channel utilization to obtain is sufficient to make a decision.

The probability matrix is called the multi-strategy matrix and contains all the users’ strategies, whereas is the multi-strategy matrix containing all users’ strategies except the strategy of user .

In the following, we define the non-cooperative multi-channel ALOHA game^{4}

When users cannot increase their rates by unilaterally changing their strategy, an equilibrium is obtained.

### 3.1Best-Response Dynamics

Here, we propose a best-response dynamics to solve the distributed rate maximization problem. We initialize the algorithm by a simple solution where every user picks the channel with the highest collision-free utility . In the learning process step, each user monitors the channel utilization to obtain for all . Then the user updates its strategy by selecting the channel with the maximal achievable rate based on the estimated load.

In the best-response dynamics users can change their selected channels according to the dynamic load. In this section we show that the dynamics converge. In the following we use the theory of potential games to show that any sequential updating dynamics across users of the proposed best-response algorithm converges in finite time, starting from any point. In potential games, users’ encouragement to change their strategy obeys a global potential function. Any local maximum of the potential function is a NEP of the game. In Theorem ?, we show that is an ordinal potential game, where the utility of a player increases by unilaterally changing its strategy, if and only if the potential function increases. For the following definition are given in Definition ? and is a payoff function for the users.

To prove the theorem we modify the distributed rate maximization problem (Equation 10). Since every user selects a single channel for transmission (and ), (Equation 10) is equivalent to the following optimization problem:

Note that the constraint implies (and also implies , ). As a result, for every , we can multiply the objective by a constant without affecting the solution’s argument. Hence, using the monotonicity of the logarithm, (Equation 10) is equivalent to the following optimization problem:

We further define:

where is determined by the chosen channel and .

Next, assume that user selects channel according to strategy and changes its strategy by selecting channel according to strategy . In what follows refer to with respect to strategy , for . The difference in the payoff function is given by:

We apply the ordinal potential function that was introduced in [48] to show that the difference in the proposed function ( ?) is given by:

where .

Hence, ( ?) follows. Furthermore, is upper bounded by .

Due to the monotonicity of the logarithm increasing increases the actual rate . As a result, ( ?) is a bounded ordinal potential function of which completes the proof.

## 4Competitive Approach Under the Totally Greedy (TG) Access Algorithm

In this section we focus on the simple transmission scheme where users access the channel with the highest collision-free utility, without considering the channel utilization. The users have constraints on the transmission probability, as in the previous section. We refer to this scheme as the Totally Greedy (TG) access scheme. The disadvantage of this scheme is that users do not exploit the channel load information to increase their rate. For instance, consider the case of two channels , . Assume that an interferer exists on channel ; thus all users observe . Using the TG scheme, all users transmit over channel even if the load on this channel is significantly higher than the load on channel . This scheme may lead to inefficient exploitation of the spectrum band. On the other hand, it is simple to implement and only a single iteration is required. Furthermore, under some mild conditions on the utility matrix it provides a good solution as the number of users increases (as will be discussed in subsequent sections). Thus, it can serve as a benchmark of the performance that could be obtained by exploiting the channel utilization when implementing the best-response dynamics. In Section 4.1 we examine the system performance in terms of user sum rate, when users exploit the channel utilization to improve their rates in a distributed fashion as compared to the TG scheme.

Let be the actual utility matrix, which is obtained by removing the first column (i.e., the all-zero vector) from , defined in (Equation 1). For purposes of analysis in this section we assume some weak conditions on the utility matrix :

- A(1)
The rows in the matrix are statistically independent.

- A(2)
The columns in the matrix are identically distributed.

Due to path loss attenuation, the rows in the matrix (which refer to users) are assumed to be independent but not-necessarily identically distributed. Due to the frequency selective fading effect, the columns for each row in the matrix (which refer to channels) are assumed to be identically distributed but not-necessarily independent. It was shown in [5] that when assumptions hold, the TG scheme provides an approximate solution to the best-response dynamics discussed in the previous section as increases. The intuition for this result is that for a large number of users, the number of users that select channel approaches . Hence, the load approaches a constant value and selecting the channel with the highest collision-free utility is more dominant. Furthermore, setting maximizes the network throughput since the expected number of users that select channel is .

### 4.1Totally Greedy Vs. Best Response

Here, we examine the loss of the simple TG scheme as compared to the best-response dynamics for a finite number of users in the case where every user experiences equal rates for all channels, i.e., for all . We consider the case where all users set to maximize the network throughput in terms of sum rate [5]. In this case the TG scheme randomly picks a channel.

Let be the number of users that select channel and assume that . Then, the best-response dynamics converge when for all . The achievable rate of user is given by:

Hence, the sum rate achieved by the best-response dynamics is given by:

Next, we compute the expected user sum rate achieved by the TG scheme. Assume that user transmits over channel . Note that channel is selected by all other users with a probability and then every user that picks channel actually transmits over it with a probability . Therefore, the expected rate of user on channel is: . Since every channel is selected with an equal probability , the expected rate of user achieved by the TG scheme is given by:

Hence, the expected sum rate achieved by the TG scheme is given by:

Note that the sum rate achieved by both schemes approaches as increases.

The gain of the best response algorithm over the TG scheme is defined as the ratio between the sum rate achieved by the best-response dynamics and the sum rate achieved by the TG scheme. The gain is given by:

It can be shown that and that as increases. The intuition for this result is that as increases, the number of users that select channel approaches . Hence, the load approaches a constant value and the TG selection is more dominant. To illustrate the result, we depict in Fig. ?. It can be seen that the best-response algorithm outperforms the TG scheme by roughly when and by when .

### 4.2Determining for Heterogenous Networks

In this section we discuss the choice of , . Assume that . A natural criterion for rate maximization in communication networks is to maximize the rate of a specific user (say user ) subject to the target rate constraints of all other users [49]. Note that as long as the demands for users are inside the rate region (i.e., feasible demands), maximizing the rate of user brings the system to operate on the boundary of the rate region, which is a desired operating point. We assume that hold. Let . Since we assume identically distributed channels, the probability that is for all and for all , and the probability. Hence, the expected rate of user is given by:

We consider the problem of maximizing the rate of a specific user such that all other user rates satisfy the target rate demands, for all . Let . Since is a constant independent of , we need to solve the following optimization problem:

We optimize over to maximize user ’s expected rate, such that target rate demands for all other users are satisfied.

The optimization problem (Equation 14) is log-concave. Complexity does not depend on the number of channels . Note that reducing increases all the other user rates . Hence, the optimal solution lies on the boundary of the rate constraints.

## 5Cooperative Game Theoretic Learning

In previous sections we examined the dynamics of multi-channel ALOHA networks, when users try to maximize their rates under given transmission probability constraints. In this section we consider a different problem in multi-channel ALOHA networks, where the transmission probability constraints are not given. As a result, a self control on the transmission probability is mandatory to avoid high load on the channels and consequently a significant loss in data rate.

Unlike the homogenous users scenario, here we do not consider the sum rate as a performance measure of the network due to fairness considerations. Note that the optimal solution for the the sum rate maximization is when a single user with the highest collision-free utility on every channel transmits with probability , while all the other users do not transmit. This operating point is clearly very bad from a fairness perspective. Therefore, in this section the sum log rate is considered to be a performance measure of the network, which is a common measure to evaluate the tradeoff between efficiency and fairness among users [50]. We show that our approach plays an important role in achieving NBS among users [33].

First, in Section 5.1 we motivate our approach by analyzing the performance among users that transmit over the same channel. Roughly speaking, we show that is essential to achieve both efficiency and fairness among users that transmit over channel . Based on this observation, we formulate the distributed rate maximization for a multi-channel network in Section 5.2. In Section 6.4 we analyze the performance for the entire network. We show that when assumptions hold, our approach achieves the target solution among all the users in the multi-channel network (and not just for each channel separately).

### 5.1Rationale

Let be the set of users that transmit over channel and its cardinality, respectively. In this section, we show that is essential to achieving both efficiency and fairness among users in .

#### Fairness in Channel Sharing

Applying the equal share transmission scheme is reasonable from a fairness perspective, where users that transmit over the same channel are required to equally share the expected number of successful time slots. Thus, in Proposition ? we consider the case where users that transmit over channel are restricted to using the equal share transmission scheme. It is shown that is a necessary condition to maximize the user rates under this setting as the number of users increases.

Proposition ? follows from standard results on a single-channel ALOHA network [52].

Setting for all yields:

#### The Efficiency and Fairness Tradeoff

Next, to further strengthen the rationale, we examine the case when the transmission probability may be different for every user and users may transmit with a probability close to . Note that the sum rate is maximized by setting for , and for all , which obviously does not maintain fairness. On the other hand, Theorem ? shows that the equal share transmission scheme still maximizes the user sum log rate over channel (i.e., the tradeoff between efficiency and fairness among users that share channel is good).

The achievable rate of user is given by:

Taking log on both sides yields:

Let be the sum log rate on channel . Hence, for we obtain:

and for we have:

.

By the monotonicity of the logarithm, it is clear that for , maximizing yields for all . Next, we focus on the case where . Note that is a strictly concave function of . Therefore, it has a unique global maximum. Differentiating with respect to , and equating to zero yields a unique solution for all .

As a result, we obtain the following corollary, as was done in (Equation 15).

#### Bargaining Over the Collision Channel

Here, we provide an interpretation of our approach from a cooperative game theory perspective. In a non-cooperative game, players (i.e., users) individually attempt to maximize their own utility regardless of the utility achieved by other players. On the other hand, in a cooperative game, players bargain with each other. If an agreement is reached, they act according to the agreement. If they disagree, they do not cooperate. For more details on cooperative game theory and applications to network games, the reader is referred to [33].

Let be the set of players. The underlying structure for Nash bargaining in an players scenario is a set of outcomes of the bargaining process (which in our model represents the set of achievable rates that the users can get by cooperating) and a designated disagreement outcome (where in our model represents the minimal rate that user would expect to achieve. Otherwise, it will not cooperate). Cooperative game theories prove that there exists a unique and efficient solution under intuitive axioms of fairness, symmetry and scaling-invariant and this solution is given by [32]:

dubbed the Nash Bargaining Solution (NBS) among players in .

Next, we show that maximizing the sum log rate (i.e., applying the equal share transmission scheme) over channel is also an NBS among users in .

Note that by non-cooperating all the users in will increase their transmission probabilities to to increase their rates. Thus, every user in (say ) expects to obtain by non-cooperating. Thus, substituting for all in (Equation 17) yields the sum log rate maximization. The rest of the proof follows from the proof of Theorem ?.

In Section 6.4 we show that when assumptions holds, our approach achieves the global NBS of the network.

### 5.2The Optimization Problem

In this section we formulate the distributed rate maximization for a multi-channel network aimed to achieve both efficiency and fairness on every channel. In subsequent sections we examine two schemes used to solve the proposed optimization problem in a distributed fashion. Moreover, in Section 6.4 we show that when hold, not just the sum log rate on every channel is maximized, but also the global sum log rate of the network is maximized as increases (which is also the global NBS of the network as shown in Theorem ?).

Based on the observation that for a large number of users should approach , the goal in this section is to cause the system to operate with the desired load on each channel in a distributed fashion. Let be the estimate of at user by monitoring the channel utilizations. Hence, each user is required to maximize its rate, but maintain a desired load on the channels (which is affected by ):

We refer to this formulation as the *adaptive rate maximization* problem, since the transmission probabilities are adapted to the channel loads.

Note that solving this problem may lead to undesirable solutions depending on the dynamic updating of the transmission probabilities across users (note that is a necessary but not a sufficient condition to maximize the sum log rate). For instance, assume that user monitors and wants to force its transmission probability to satisfy the constraint: . In this case, the update of yields

As a result, if user detects channel as a free channel, i.e., , it maximizes its probability to get which satisfies the constraint. Then, in the next iteration, any other user that accesses this channel will detect and will force its probability to zero to satisfy the constraint (as a result, ). Hence, in the next section we propose two schemes to obtain the target solutions for all users.

## 6Distributed Algorithms for the Adaptive Rate Maximization Problem

In this section we propose parallel and sequential mechanisms to solve (Equation 18) efficiently. The proposed mechanisms are executed from time to time until convergence. It should be noted that the proposed algorithms apply for all and perform well as can be seen via simulation results. Performance analysis, however, will be presented under the asymptotic regime (i.e., as approaches infinity) and an accurate estimate of .

### 6.1Sequential Updating

In the sequential updating mechanism, users adjust their transmission probability until they get the desired channel load. Let . The users’ goal is to reduce sequentially until convergence.

In the initialization step, all users select the channels with the highest collision-free utility and set their transmission probability to .

Next, in the learning step, each user occasionally monitors the channel utilization of all channels. After the user has estimated it does the following. First, it computes the highest transmission probability allowed on each channel based on the estimated load:

This operation will encourage users to move to channels with low loads.

Next, the user computes the potential achievable rates on all the channels:

If there is a channel with a higher potential rate than its current channel, the user switches to this channel; i.e., it updates as follows:

Next, the user reduces to obtain the desired load. If , user increases its transmission probability to increase the load on the channel: . Otherwise, it reduces its transmission probability to reduce the load on the channel: .

Note that as approaches for all , the potential transmission probability that user computes for all other channels approaches zero to maintain the desired load. Hence, users are encouraged to remain in their channels as the load approaches the desired load.

To stabilize the algorithm, we allow user to switch to channel from only if it gains at least percents of its current rate: . Users may update dynamically to speed up convergence (i.e., by increasing ) or to increase their data rate (i.e., by reducing ) from time to time^{5}

- Initialize: |

- for users do: |

- estimate for all |

- |

- |

- |

- |

- end for |

- repeat: |

- for users do: |

- estimate for all |

- compute |

for all |

- compute potential rates: |

- if do: |

- end if |

- compute |

- if do: |

- else, do: |

- end if |

- |

- |

- end for |

- until for all |

### 6.2Parallel Updating

The parallel algorithm is based on the observation that for a large number of users (and when A(1), A(2) hold) the maximal network throughput in multi-channel ALOHA networks approaches , where users transmit with probability [20]. The parallel algorithm is described as follows. In the initialization step, all users set their transmission probability to . In the learning step, all users monitor the channel utilization for all and compute . Hence, all users can estimate the number of users by:

Then all users set their transmission probability:

and implement the best-response dynamics, discussed in Section 3.1, with a given transmission probability . Theorem ? shows that under , for all as . The parallel updating mechanism is given in Table ?.

- Initialize: |

- for users do: |

- estimate for all |

- |

- |

- |

- |

- end for |

- for users do: |

- estimate for all |

- compute |

for all |

- compute |

- end for |

- for users do: |

- |

- |

- |

- end for |

- perform the best-response dynamics |

with given until convergence |

### 6.3Convergence of the Sequential and Parallel Updating Algorithms

When applying the sequential and parallel updating algorithms, users can change their selected channels according to the dynamic load. In this section we show that the dynamics converge in finite time, starting from any point.

The following theorem establishes the convergence of the sequential updating algorithm. For purposes of analysis, we assume that users do not reduce their transmission probability to zero (thus, users with a high transmission probability should reduce their rates). Therefore, we assume that the transmission probability of every user is lower bounded by for some .

Assume that users play a multi-strategy matrix . Assume that user has computed the potential rates and wants to update its strategy. User will switch to a different channel only if

holds.

Note that for all . Thus,

and

.

Let

Then,

.

As a result, user will not switch strategy in the next iterations for any multi-strategy of the other users once (which occurs in finite time by increasing from time to time). Once for all occurs, the entire system is in equilibrium.

It should be noted that practically, simulation results show fast convergence of the sequential updating algorithm for very small values of .

The following theorem establishes the convergence of the parallel updating algorithm.

After the initialization step, all users set their transmission probability to . Then, all the users implement the best-response dynamics discussed in Section 3.1 with a given transmission probability . As a result, convergence is guaranteed in finite time, starting from any point by Corollary .

### 6.4Achieving the Global NBS via Best Response

In this section we examine the performance of the algorithms in the asymptotic regime (i.e., as , where is fixed). For purposes of analysis, we assume that and can be arbitrarily small when applying the sequential updating algorithm. Theorem ? shows that under assumption , both the sequential and parallel updating algorithms maximize the global sum log rate of the network as increases. Theorem ? shows that the global NBS of the network is achieved in this case.

We prove the theorem in two steps. First, we establish the upper bound on the sum log rate that can be achieved by any algorithm. Then, we show that the proposed algorithms achieve the bound in the asymptotic regime.

We use the same notation as in the proof of Theorem ?. Substituting in (Equation 16) yields:

where .

Let be the sum log rate of the network. Hence^{6}

where is a constant independent of and is a function of .

It can be verified that the second derivative of with respect to is strictly negative in its domain. Therefore, by the strict concavity of , for any partition of , , , such that , we have: , where equality holds iff for all . Therefore, maximizing the upper bound with respect to , yields a solution for all . Substituting in (Equation 21) yields:

Next, to show that the parallel algorithm achieves this bound (Equation 22), it suffices to show the following: the users transmit with probability for all