Deploy-As-You-Go Wireless Relay Placement: An Optimal Sequential Decision Approach using the Multi-Relay Channel ModelThis work was supported by the Department of Science and Technology (DST), India, through the J.C. Bose Fellowship, by an Indo-Brazil cooperative project on “WIreless Networks and techniques with applications to SOcial Needs (WINSON)," and by a project funded by the Department of Electronics and Information Technology, India, and NSF, USA, titled Wireless Sensor Networks for Protecting Wildlife and Humans in Forests.This paper is an extension of [1], and is also available in [2]. Arpan Chattopadhyay and Anurag Kumar are with the Electrical Communication Engineering (ECE) Department, Indian Institute of Science (IISc), Bangalore-560012, India (e-mail: arpanc.ju@gmail.com, anurag@ece.iisc.ernet.in). Abhishek Sinha is with the Laboratory for Information and Decision Systems (LIDS), Massachusetts Institute of Technology, Cambridge, MA 02139 (e-mail: sinhaa@mit.edu). Marceau Coupechoux is with Telecom ParisTech and CNRS LTCI, Dept. Informatique et Réseaux, 23, avenue d’Italie, 75013 Paris, France (e-mail: marceau.coupechoux@telecom-paristech.fr). This work was done during the period when he was a Visiting Scientist in the ECE Deparment, IISc.

# Deploy-As-You-Go Wireless Relay Placement: An Optimal Sequential Decision Approach using the Multi-Relay Channel ModelThis work was supported by the Department of Science and Technology (DST), India, through the J.C. Bose Fellowship, by an Indo-Brazil cooperative project on “WIreless Networks and techniques with applications to SOcial Needs (WINSON)," and by a project funded by the Department of Electronics and Information Technology, India, and NSF, USA, titled “Wireless Sensor Networks for Protecting Wildlife and Humans in Forests.”This paper is an extension of [1], and is also available in [2]. Arpan Chattopadhyay and Anurag Kumar are with the Electrical Communication Engineering (ECE) Department, Indian Institute of Science (IISc), Bangalore-560012, India (e-mail: arpanc.ju@gmail.com, anurag@ece.iisc.ernet.in). Abhishek Sinha is with the Laboratory for Information and Decision Systems (LIDS), Massachusetts Institute of Technology, Cambridge, MA 02139 (e-mail: sinhaa@mit.edu). Marceau Coupechoux is with Telecom ParisTech and CNRS LTCI, Dept. Informatique et Réseaux, 23, avenue d’Italie, 75013 Paris, France (e-mail: marceau.coupechoux@telecom-paristech.fr). This work was done during the period when he was a Visiting Scientist in the ECE Deparment, IISc.

## 1Introduction

Wireless interconnection of devices such as smart phones, or wireless sensors, to the wireline communication infrastructure is an important requirement. These are battery operated, resource constrained devices. Hence, due to the physical placement of these devices, or due to channel conditions, a direct one-hop link to the infrastructure “base-station” might not be feasible. In such situations, other nodes could serve as relays in order to realize a multi-hop path between the source device and the infrastructure. In the wireless sensor network context, the relays could be other wireless sensors or battery operated radio routers deployed specifically as relays. The relays are also resource constrained and a cost might be involved in placing them. Hence, there arises the problem of optimal relay placement. Such a problem involves the joint optimization of node placement and operation of the resulting network, where by “operation” we mean activities such as transmission scheduling, power allocation, and channel coding.

Our work in this paper is motivated by recent interest in problems of impromptu (as-you-go) deployment of wireless relay networks in various situations; for example, “first responders” in emergency situations, or quick deployment (and redeployment) of sensor networks in large terrains, such as forests (see [3], [4], [5], [6], [7]). In this paper, we are concerned with the situation in which a deployment agent walks from the source node to the sink node, along the line joining these two nodes, and places wireless relays (in an “as-you-go” manner) so as to create a source-to-sink multi-relay channel network with high data rate; see Figure 1. We first consider the scenario where the length of the line in Figure 1 is known; the results of this case are used to formulate the as-you-go deployment in the case where is a priori unknown, but has exponential distribution with known mean .

In order to capture the fundamental trade-offs involved in such problems, we consider an information theoretic model. For a placement of the relay nodes and allocation of transmission powers to these relays, we model the “quality” of communication between the source and the sink by the information theoretic achievable rate of the multi-relay channel (see [8], [9] and [10] for the single and multi-relay channel models). The relays are equipped with full-duplex radios1, and carry out decode-and-forward relaying. We consider scalar, memoryless, time-invariant, additive white Gaussian noise (AWGN) channels. We assume synchronous operation across all transmitters and receivers, and consider the exponential path-loss model for radio wave propagation.

### 1.1Related Work

A formulation of the relay placement problem requires a model of the wireless network at the physical (PHY) and medium access control (MAC) layers. Most researchers have adopted the link scheduling and interference model, i.e., a scheduling algorithm determines radio resource allocation (channel and power) and interference is treated as noise (see [15]); treating interference as noise leads to the model that simultaneous transmissions “collide” at receiving nodes, and transmission scheduling aims to avoid collisions.

However, node placement for throughput maximization with this model is intractable because the optimal throughput is obtained by first solving for the optimum schedule assuming fixed node locations, followed by an optimization over those locations. Hence, with such a model, there appears to be little work on the problem of jointly optimizing the relay node placement and the transmission schedule. Reference [16] is one such work where the authors considered placing a set of nodes in an existing network such that a certain network utility is optimized subject to a set of linear constraints on link rates, under the link scheduling and interference model. They posed the problem as one of geometric programming assuming exponential path-loss, and proposed a distributed solution. The authors of [17] consider relay placement for utility maximization, assuming there are several source nodes, sink nodes and a few candidate locations for placing relays; they ignore interference because of highly directional antennas used in GHz mmWave networks, which may not always be valid. Relay placement for capacity enhancement has been studied in [18], but there interference is mitigated by scheduling transmissions over multiple channels.

On the other hand, an information theoretic model for a wireless network often provides a closed-form expression for the channel capacity, or at least an achievable rate region. These results are asymptotic, and make idealized assumptions such as full-duplex radios, perfect interference cancellation, etc., but provide algebraic expressions that can be used to formulate tractable optimization problems which can provide useful insights. In the context of optimal relay placement, some researchers have already exploited this approach. Thakur et al., in [19], report on the problem of placing a single relay node to maximize the capacity of a broadcast relay channel in a wideband regime. Lin et al., in [20], numerically solve the problem of a single relay node placement, under power-law path loss and individual power constraints at the source and the relay; however, our work is primarily focused on multi-relay placement, under the exponential path-loss model and a sum power constraint among the nodes. The linear deterministic channel model ([21]) is used by Appuswamy et al. in [22] to study the problem of placing two or more relay nodes along a line so as to maximize the end-to-end data rate. Our present paper is in a similar spirit; however, we use the achievable rate formulas for the -relay channel (with decode and forward relays) to study the problem of placing relays on a line having length , under a sum power constraint over the nodes.

The most important difference of our paper with the literature reported above is that we address the problem of sequential placement of relay nodes along a line of an unknown random length. This paper extends our previous work in [1], which presents the analysis for the case of given and ; the study under given and is a precursor to the formulation of as-you-go deployment problem, since it motivates an additive cost structure that is essential for the formulation of the sequential deployment problem as a Markov decision process (MDP).

The deploy-as-you-go problem has been addressed by previous researchers. For example, Howard et al., in [4], provide heuristic algorithms for incremental deployment of sensors in order to cover a deployment area. Souryal et al., in [3], propose heuristic deployment algorithms for the problem of impromptu wireless network deployment, with an experimental study of indoor RF link quality variation. The authors of [5] propose a collaborative deployment method for multiple deployment agents, so that the contiguous coverage area of relays is maximized subject to a total number of relays constraint. However, until the work in [23] and [6], there appears to have been no effort to rigorously formulate as-you-go deployment problem in order to derive optimal deployment algorithms. The authors of [23] and [6] used MDP based formulations to address the problem of placing relay nodes sequentially along a line and along a random lattice path, respectively. The formulations in [23] and [6] are based on the so-called “lone packet traffic model” under which, at any time instant, there can be no more than one packet traversing the network, thereby eliminating contention between wireless links. This work was later extended in [7] to the scenario where the traffic is still lone packet, but a measurement-based approach is employed to account for the spatial variation of link qualities due to shadowing.

In this paper, we consider as-you-go deployment along a line, but move away from the lone-packet traffic assumption by employing information theoretic achievable rate formulas (for full-duplex radios and decode-and-forward relaying). We assume exponential path-loss model (see [24] and Section 6.1). To the best of our knowledge, there is no prior work that considers as-you-go deployment under this physical layer model.

### 1.2Our Contribution

• Optimal Offline Deployment: Given the location of full-duplex relays to connect a source and a sink separated by a given distance , and under the exponential path-loss model and a sum power constraint among the nodes, the optimal power split among the nodes and the achievable rate are expressed (Theorem ?) in terms of the channel gains. We find expression for optimal relay location in the single relay placement problem (Theorem ?). For the relay placement problem, numerical study shows that, the relay nodes are clustered near the source at low attenuation and are placed uniformly at high attenuation. Theorem ? shows that, by placing large number of relays uniformly, we can achieve a rate arbitrarily close to the AWGN capacity. Only this part of our current paper was published in the conference version [1].

• Optimal As-You-Go Deployment: In Section 4, we consider the problem of placing relay nodes in a deploy-as-you-go manner, so as to connect a source and a sink separated by an unknown distance, modeled as an exponentially distributed random variable . Specifically, the problem is to start from a source, and walk along a line, placing relay nodes as we go, until the line ends, at which point the sink is placed. With a sum power constraint, the aim is to maximize a capacity limiting term derived from the deployment problem for known , while constraining the expected number of relays. We “relax” the expected number of relays constraint via a Lagrange multiplier, and formulate the problem as a total cost MDP with uncountable state space and non-compact action sets. We prove the existence of an optimal policy and convergence of value iteration (Theorem ?); these results for uncountable state space and non-compact action space are not evident from standard literature. We study properties of the value function analytically. This is the first time that the as-you-go deployment problem is formulated to maximize the end-to-end data rate under the full-duplex multi-relay channel model.

• Numerical Results on As-You-Go Deployment: In Section 5, we study the policy structure numerically. We also demonstrate numerically that the proposed as-you-go algorithm achieves an end-to-end data rate sufficiently close to the maximum possible achievable data rate for offline placement. This is particularly important since there is no other benchmark in the literature, with which we can make a fair comparison of our policy.

• The material in Section 4 and Section 5 were absent in the conference version [1].

### 1.3Organization of the Paper

In Section 2, we describe our system model and notation. In Section 3, we address the problem of relay placement on a line of known length. Section 4 deals with the problem of as-you-go deployment along a line of unknown random length. Numerical work on as-you-go deployment has been presented in Section 5. Some discussions are provided in Section 6. Conclusions are drawn in Section 7.

## 2System Model and Notation

### 2.1The Multi-Relay Channel

The multi-relay channel was studied in [10] and [9] and is an extension of the single relay model presented in [8]. We consider a network deployed on a line with a source node and a sink node at the end of the line, and full-duplex relay nodes as shown in Figure Figure 1. The relay nodes are numbered as . The source and sink are indexed by and , respectively. The distance of the -th node from the source is denoted by . Thus, . As in [10] and [9], we consider the scalar, time-invariant, memoryless, AWGN setting.

We use the model that a symbol transmitted by node is received at node after multiplication by the (positive, real valued) channel gain (an assumption often made in the literature, see e.g., [10] and [25]). The power gain from Node to Node is denoted by . We define and . The Gaussian additive noise at any receiver is independent and identically distributed from symbol to symbol and has variance .

### 2.2An Inner Bound to the Capacity

For the multi-relay channel, we denote the symbol transmitted by the -th node at time ( is discrete) by for . is the additive white Gaussian noise at node and time , and is assumed to be independent and identically distributed across and . Thus, at symbol time , node receives:

An inner bound to the capacity of this network, under any path-loss model, is given by (see [10]):

where , and node transmits to node at power (expressed in mW).

In Appendix Section 8, we provide a descriptive overview of the coding and decoding scheme proposed in [10]. A sequence of messages are sent from the source to the sink; each message is encoded in a block of symbols and transmitted by using the relay nodes. The scheme involves coherent transmission by the source and relay nodes (this requires symbol-level synchronization among the nodes), and successive interference cancellation at the relay nodes and the sink. A node receives information about a message in two ways (i) by the message being directed to it cooperatively by all the previous nodes, and (ii) by overhearing previous transmissions of the message to the previous nodes. Thus node receives codes corresponding to a message times before it attempts to decode the message (a discussion on the practical feasibility of full-duplex decode-and-forward relaying scheme is provided in Section 6.3). Note that, in (Equation 2), for any , denotes a possible rate that can be achieved by node from the transmissions from nodes . The smallest of these terms becomes the bottleneck, see (Equation 2).

For the single relay channel, . Thus, by (Equation 2), an achievable rate is given by (see also [8]):

Here, the first term in the of (Equation 3) is the achievable rate at node (i.e., the relay node) due to the transmission from the source. The second term in the corresponds to the possible achievable rate at the sink node due to direct coherent transmission from the source and the relay and due to the overheard transmission from the source to the relay. The higher the channel attenuation, the less will be the contribution of farther nodes, “overheard” transmissions become less relevant, and coherent transmission reduces to a simple transmission from the previous relay. The system is then closer to simple store-and-forward relaying.

The authors of [10] have shown that any rate strictly less than is achievable through the coding and decoding scheme. This achievable rate formula can also be obtained from the capacity formula of a physically degraded multi-relay channel (see [9]), since the capacity of the degraded relay channel is a lower bound to the actual channel capacity. In this paper, we will seek to optimize in (Equation 2) over power allocations to the nodes and the node locations, keeping in mind that is a lower bound to the actual capacity. We denote the value of optimized over power allocation and relay locations by .

### 2.3Path-Loss Model

We model the power gain via the exponential path-loss model: the power gain at a distance is where . This is a simple model used for tractability (see [16], [22]) and [26] for prior work assuming exponential path-loss). However, for propagation scenarios involving randomly placed scatterers (as would be the case in a dense urban environment, or a forest, for example) analytical and experimental support has been provided for the exponential path-loss model (a discussion has been provided in Section 6.1). We also discuss in Section 6.4 how the insights obtained from the results for exponential path-loss can be used for power-law path-loss (power gain at a distance is , ). Deployment with other path-loss models is left in this paper as a possible future work.

Under exponential path-loss, the channel gains and power gains in the line network become multiplicative, e.g., and for .

We discuss in Section 6.2 how shadowing and fading can be taken care of in our model, by providing a fade- margin in the power at each transmitter.

### 2.4Motivation for the Sum Power Constraint

In this paper we consider the sum power constraint (in mW) over the source and the relays. This constraint has the following motivation. Let the fixed power expended in a relay (for reception and driving the electronic circuits) be denoted by (expressed in mW), and the initial battery energy in each node be denoted by (in mJ unit). The information theoretic approach utilized in this paper requires that the nodes in the network are always on. Hence, the lifetime of node , is , the lifetime of the source is , and that of the sink is . The rate of battery replacement at node is . Hence, the rate at which we have to replace the batteries in the network is . The depletion rate is inevitable at any node, and it does not affect the achievable data rate. Hence, in order to reduce the battery replacement rate, we must reduce the sum transmit power in the entire network.

## 3Placement on a Line of Known Length

As a precursor to addressing the deploy-as-you-go problem over a line of unknown length, in this section we solve the problem of power constrained deployment of a given number of relays on a line of known length. We will often refer to this problem as offline deployment problem. The results of this section provide (i) first insights into the relay placements we obtain using the multi-relay channel model, (ii) a starting point for the formulation of as-you-go deployment problem, and (iii) a benchmark with which we can compare the performance of our as-you-go deployment algorithm.

### 3.1Optimal Power Allocation

In this section, we consider the optimal placement of relay nodes on a line of given length, , so as to to maximize (see (Equation 2)), subject to a total power constraint on the source and relay nodes given by . We will first maximize in (Equation 2) over for any given placement of nodes (i.e., given ). This will provide an expression of achievable rate in terms of channel gains, which has to be maximized over . Let for (expressed in mW). Hence, the sum power constraint becomes .

The basic idea is to choose the power levels (i.e., ) in (Equation 2) so that all the terms in the in (Equation 2) become equal. We provide explicit expressions for and the achievable rate (optimized over power allocation) in terms of the power gains. See Appendix Section 9 for the detailed proof. A result on the equality of certain terms under optimal power allocation has also been proved in [9] for the coding scheme used in [9]. But it was proved in the context of a degraded Gaussian multi-relay channel, and the proof depends on an inductive argument, whereas our proof utilizes LP (linear programming) duality.

Recalling the exponential path-loss parameter , and the source-sink distance , let us define , which can be treated as a measure of attenuation in the line.

Let us now comment on the results of Theorem ?:

• In order to maximize , we need to place the relay nodes such that is minimized. This quantity is viewed as the net attenuation the power faces.

• When no relay is placed, the attenuation is . The ratio of attenuation with no relay and attenuation with relays is called the “relaying gain” .

Rate is increasing with the number of relays, and is bounded above by . Hence, . Also, note that does not depend on .

• By Theorem ?, we have for .

• Note that we have derived Theorem ? using the fact that is nonincreasing in . If there exists some such that , i.e, if -th and -st nodes are placed at the same position, then , i.e., the nodes do not direct any power specifically to relay . However, relay can decode the symbols received at relay , and those transmitted by relay . Then relay can transmit coherently with the nodes to improve effective received power in the nodes .

### 3.2Optimal Placement of a Single Relay Node

In the following theorem, we derive the optimal power allocation, node location and data rate when a single relay is placed.

See Appendix Section 10.

Discussion: It is easy to check that obtained in Theorem ? is strictly greater than the AWGN capacity for all . This happens because the source and relay transmit coherently to the sink. becomes equal to the AWGN capacity only at . At , we do not use the relay since the sink can decode any message that the relay is able to decode.

The variation of and with has been shown in Figure 2. We observe that (from Figure 2 and Theorem ?) , and . For large values of , source and relay cooperation provides negligible benefit since source to sink attenuation is very high. So it is optimal to place the relay at a distance . The relay works as a repeater which forwards data received from the source to the sink. For small , the gain obtained from coherent transmission is dominant, and, in order to receive sufficient information (required for coherent transmission) from the source, the relay is placed near the source.

### 3.3Optimal Placement for a Multi-Relay Channel

As we discussed earlier, we need to place relay nodes such that is minimized. Here . We have the constraint . Now, writing , and defining , we arrive at the following problem:

The objective function is convex in each of the variables . The objective function is sum of linear fractionals, and the constraints are linear.

Remark: From optimization problem (Equation 5) we observe that optimum depend only on . Since , the normalized optimal distance of relays from the source depend only on and .

See Appendix Section 11.

See Appendix Section 12.

A numerical study of multi-relay placement: We discretize the interval and run a search program to find normalized optimal relay locations for different values of and . The results are summarized in Figure ?.

We observe that at low attenuation (small ), relay nodes are clustered near the source node and are often at the source node, whereas at high attenuation (large ) they are almost uniformly placed along the line. For large , the effect of long distance between any two adjacent nodes dominates the gain obtained by coherent relaying. Hence, it is beneficial to minimize the maximum distance between any two adjacent nodes and thus multihopping is a better strategy in this case. For small , the gain obtained by coherent transmission is dominant. In order to allow this, relays should be able to receive sufficient information from their previous nodes. Thus, they tend to be clustered near the source.

In Figure 3 we plot the relaying gain in dB vs. the number of relays , for various values of . As proved in Theorem ?, we see that increases with for fixed . On the other hand, increases with for fixed , as proved in Theorem ?.

### 3.4Uniformly Placed Relays, Large N

When the relays are uniformly placed, the behaviour of (called in the next theorem) for large number of relays is captured by the following:

See Appendix Section 13.

Remark: From Theorem ?, it is clear that we can achieve a rate arbitrarily close to (i.e., the effect of path-loss can completely be mitigated) by placing a large enough number of relay nodes. In this context, we would like to mention that the variation of broadcast capacity as a function of the number of nodes (located randomly inside a unit square) was studied in [27]; but the broadcast capacity in their paper increases with since they assume per-node power constraint.

## 4As-You-Go Deployment of Relays on a Line of Unknown Length

Having developed the problem of placing a given number of relays over a line of fixed, given length, we now turn to the deploy-as-you-go problem. An agent walks along a line, starting from the source and heading towards the sink which is at an unknown distance from the source location, deploying relays as he goes, so as to achieve a multi-relay network when he encounters the sink location (and places the sink there). We model the distance from the source to sink as an exponentially distributed random variable with mean .2 The deployment objective is to achieve a high data rate from the source to the sink, subject to a total power constraint and a constraint on the expected number of relays placed (note that, the number of relay nodes, , is a random variable here, due to the randomness in ). Using the rate expression from Theorem ?, we formulate the problem as a total cost MDP.

Such a deployment problem could be motivated by a situation where it is required to place a sensor (say, a video camera) to monitor an event or an object from a safe distance (e.g., the battlefront in urban combat, or a suspicious object that needs to be detonated, or a group of animals in a forest). In such a situation, the deployment agent, after placing the sensor, walk away from the scene of the event, along a forest trail, or a road, or a building corridor, placing relays as he walks, until a suitable safe sink location is found, in such a way that the number of relays is kept small while the end-to-end data rate is maximized.

### 4.1Formulation as an MDP

We now formulate the as-you-go deployment problem as an MDP.

#### Deployment Policies

In the as-you-go placement problem, the person carries a number of nodes and places them as he walks, under the control of a placement policy. A deployment policy is a sequence of mappings from the state space to the action space; at the -th decision instant (i.e., after placing the -st relay), provides the distance at which the next relay should be placed (provided that the line does not end before that point), given the system state which is a function of the locations of previously placed nodes. Thus, the decisions are made based on the locations of the relays placed earlier. The first decision instant is the start of the line, and the subsequent decision instants are the placement points of the relays. Let denote the set (possibly uncountable) of all deployment policies. Let denote the expectation under policy .

#### The Unconstrained Problem

We recall from ( ?) that for a fixed length of the line and a fixed , has to be minimized in order to maximize . is basically a scaling factor which captures the effect of attenuation and relaying on the maximum possible SNR .

Let be the cost of placing a relay. We are interested in solving the following problem:

The “cost” function inside the outer parentheses in (Equation 6) has two terms, one is the denominator of in (Equation 4), and the other is a linear multiple of the number of relays. Thus, the cost function captures the tradeoff between the cost of placing relays (quantified as per relay), and the need to achieve high end-to-end data rate by making the denominator of small. Note that, due to the randomness in the length of the line, the and are all random variables.3

We will see in Theorem ? that an optimal policy always exists for this problem.

#### The Constrained Problem

Solving the problem in (Equation 6) also helps in solving the following constrained problem:

where is a constraint on the expected number of relays. 4 The following standard result (see [28]) gives the optimal :

The motivation behind formulation (Equation 7) is as follows. Suppose that one seeks to solve the following problem:

Since is convex in , we can argue by Jensen’s inequality that by solving (Equation 7) we actually find a relay placement policy that maximizes a lower bound to the expected achievable data rate obtained from (Equation 8). But formulation (Equation 7) (and hence formulation (Equation 6), by Lemma ?) allows us to write the objective function as a summation of hop-costs; this motivates us to formulate the as-you-go deployment problem as an MDP, resulting in a substantial reduction in policy computation. However, in Section 5, we will show numerically that solving (Equation 7) is a reasonable approach to deal with the computational complexity of (Equation 8); we will see that formulation (Equation 7) allows us to achieve a reasonable performance.

We now formulate the above “as-you-go” relay placement problem (Equation 6) as a total cost Markov decision process.

#### State Space, Action Space and Cost Structure

Let us define , . Also, recall that . Thus, we can rewrite (Equation 6) as:

When the person starts walking from the source along the line, the state of the system is set to . At this instant the placement policy provides the location at which the first relay should be placed. The person walks towards the prescribed placement point. If the sink placement location is encountered before reaching this point, the sink is placed; if not, then the first relay is placed at the placement point. In general, the state after placing the -th relay is denoted by (a function of the location of the nodes up to the -th instant), for . At state , the action is the distance where the next relay has to be placed (action means that no further relay will be placed). If the line ends before this distance, the sink node has to be placed at the end. The randomness is coming from the random residual length of the line. Let denote the residual length at the -th instant.

With this notation, the state of the system evolves as:

Here denotes the end of the line, i.e., the termination state.

The single stage cost (for problem (Equation 9)) for state , action and residual length , is:

Also, for all .

From (Equation 10), it is clear that the next state depends on the current state , the current action and the residual length of the line. Since the length of the line is exponentially distributed, from any placement point, the residual line length is exponentially distributed, and independent of the history of the process. The cost incurred at the -th decision instant is given by (Equation 11), which depends on , and .

Hence, our formulation in (Equation 9) is an MDP with state space and action space where .

Remark: An optimal policy (if it exists) for the problem (Equation 6) will be used to place relay nodes along a line whose length is a sample from an exponential distribution with mean . After the deployment is over, the power will be shared optimally among the source and the deployed relay nodes (according to Theorem ?).

### 4.2Optimal Value Function

Suppose for some . Then, the optimal value function (cost-to-go) at state is defined by:

If we decide to place the next relay at a distance and follow the optimal policy thereafter, the expected cost-to-go at a state becomes:

The first term in (Equation 12) corresponds to the case in which the line ends at a distance less than and we are forced to place the sink node. The second term corresponds to the case where the residual length of the line is greater than and a relay is placed at a distance .

Note that our MDP has an uncountable state space and a non-compact action space . Several technical issues arise in this kind of problems, such as the existence of optimal or -optimal policies, measurability of the policies, etc. We, therefore, invoke the results provided by Schäl [29], which deal with such issues. Our problem is one of minimizing total, undiscounted, non-negative costs over an infinite horizon. Equivalently, in the context of [29], we have a problem of total reward maximization where the rewards are the negative of the costs. Thus, our problem specifically fits into the negative dynamic programming setting of [29] (i.e., the case where single-stage rewards are non-positive).

Now, the state is absorbing. Also, no action is taken at this state and the cost at this state is . Hence, we can think of this state as state in order to make our state space a Borel subset of the real line.

Thus, satisfies the following Bellman equation for each :

where the second term inside is the cost of not placing any relay (i.e., ).

We analyze the MDP for and .

#### Case I (β>ρ)

We observe that the cost of not placing any relay (i.e., ) at state is given by:

where (using the fact that ). Since not placing a relay (i.e., ) is a possible action for every , it follows that .

The cost in (Equation 12), upon simplification, can be written as:

Since for all , the expression in (Equation 14) is strictly less that for large enough . Hence, according to (Equation 13), it is not optimal to not place any relay and the Bellman equation (Equation 13) can be rewritten as:

#### Case II (β≤ρ)

Here the cost in (Equation 12) is if we do not place a relay (i.e., if ). Let us consider a policy where we place the next relay at a fixed distance from the current relay, irrespective of the current state. If the residual length of the line is at any state , we will place less than additional relays, and for each relay a cost less than is incurred (since ). At the last step when we place the sink, a cost less than is incurred. Thus, the value function of this policy is upper bounded by:

Hence, . Thus, by the same argument as in the case , the minimizer in the Bellman equation lies in , i.e., the optimal placement distance lies in . Hence, (Equation 13) can be rewritten as:

### 4.3Upper Bound on the Optimal Value Function

We know that . Now, let us consider the Bellman equation (Equation 15). It is easy to see that is strictly negative for sufficiently large . Hence, the R.H.S of (Equation 15) is strictly less than .

Follows from Proposition ?.

Follows from (Equation 16), since the analysis is valid even for .

### 4.4Convergence of the Value Iteration

The value iteration for our MDP is given by:

Here is the -th iterate of the value iteration. Let us start with for all . We set for all . is the optimal value function for a problem with the same single-stage cost and the same transition structure, but with the horizon length being (instead of infinite horizon as in our original problem) and terminal cost. Here, by horizon length , we mean that there are number of relays available for deployment.

Let be the set of minimizers of (Equation 18) at the -th iteration at state , if the infimum is achieved at some . Let be an accumulation point of some sequence where each . Let be the set of minimizers in (Equation 17). In Appendix Section 14, we show that for each , and are nonempty.

The proof is given in Appendix ?. It uses some results from [29], which have been discussed first. Next, we provide a general theorem (Theorem ?) on the convergence of value iteration, which has been used to prove Theorem ?.

Remark: Since the action space is noncompact, it is not obvious from standard results whether the optimal policy exists. However, we are able to show that in our problem, for each state , the optimal action will lie in a compact set of the from , where is continuous in , and could possibly go to as . The results of [29] allow us to work with the scenario where for each state , it is sufficient to focus only on a compact action space .

### 4.5Properties of the Value Function Jξ(s)

See Appendix ? for the proofs of these propositions.

### 4.6A Useful Normalization

Note that, is exponentially distributed with mean . Defining and , , we can rewrite (Equation 6) as follows:

Note that, plays the same role as played in the known case (see Section 3.2). Since is the mean length of the line, can be considered as a measure of attenuation in the network. We can think of the new problem (Equation 19) in the same way as (Equation 6), but with the length of the line being exponentially distributed with mean () and the path-loss exponent being changed to . The relay locations are also normalized (). One can solve the new problem (Equation 19) and obtain the optimal policy. Then the solution to (Equation 6) can be obtained by multiplying each control distance (from the optimal policy of (Equation 19)) with the constant . Hence, it suffices to work with .

## 5A Numerical Study of As-You-Go Deployment

Let us recall that the state of the system after placing the -th relay is given by . The action is the normalized distance of the next relay to be placed from the current location. The single stage cost function for our total cost minimization problem is given by (Equation 11).

In our numerical work, we discretized the state space into steps as , and discretized the action space into steps of size , i.e., the action space becomes .

### 5.1Structure of the Optimal Policy

We performed numerical experiments to study the structure of the optimal policy obtained through value iteration for and some values of . The value iteration in these experiments converged and we obtained a stationary optimal policy, though Theorem ? does not guarantee the uniqueness of the stationary optimal policy.

Figure 4 shows that the normalized optimal placement distance is decreasing with the state . This can be understood as follows. The state (at a placement point) is small only if a sufficiently large number of relays have already been placed.5 Hence, if several relays have already been placed and is sufficiently large compared to (i.e., is small), the -st relay will be able to receive sufficient amount of power from the previous nodes, and hence does not need to be placed close to the -th relay. A value of close to indicates that there is a large gap between relay and relay , the power received at the next relay from the previous relays is small and hence the next relay must be placed closer to the previous one.

On the other hand, is increasing with (see Figure 5). Recall that is the price of placing a relay. This figure confirms the intuition that if the relay price is high, then the relays should be placed less frequently.

Figure 6 shows that is decreasing with , for fixed values of and . This happens because increased attenuation will require frequent placement of the relays.

### 5.2Relay Placement Patterns

The policy that we use corresponds to a line having exponentially distributed length with mean , but it is applied to the scenario where the actual realization of the (normalised) length (see Section 4.6) of the line is .

Tables Table 1, Table 3, and Table 5 illustrate some examples of as-you-go placement of relay nodes along a line of normalised length , using various values of and . Tables Table 2, Table 4, and Table 6 illustrate the corresponding evolution of state as the relays are placed in the examples in Tables Table 1, Table 3 and Table 5. If the line actually ends at some point before (normalised) distance , the process would end there with the corresponding placement of relays (as can be obtained from Tables Table 1, Table 3, and Table 5) before the sink being placed at the end-point. Thus, for example, reading from Table 3 for and , if the actual normalised length of the line is , then one relay will be placed at (the source itself), followed by relays at normalised distances from the source, and finally the sink is placed at a normalised distance , the end of the line.

We observe that as increases, more relays need to be placed since the optimal control decreases with for each (see Figure 6). On the other hand, the number of relays decreases with increasing (the relay cost); this is in confirmation of the observations from Figure 5.

Note that, initially one or more relays are placed at or near the source if is or small. But, after some relays have been placed, the relays are placed equally spaced apart. We see that this happens because, after a few relays have been placed, the state, , does not change, hence, resulting in the relays being subsequently placed equally spaced apart. This phenomenon is evident in Table 2, Table 4, Table 6, and Figure 7. The state will remain unchanged after a relay placement if , since we have discretized the state space. After some relays are placed, the state becomes equal to a fixed point of the function . Note that the deployment starts from , but for any value of (even with smaller than ), we numerically observe the same phenomenon. Hence, is an absorbing state.

### 5.3Numerical Examples for Practical Deployment

In order to provide a more concrete illustration we adopt a path loss parameter from [24]. Figure of [24] shows that the attenuation in the received signal power in a dense urban environment is roughly dB when we move from m distance to m distance away from the transmitter. This yields a value of to be per meter for the exponential path-loss (see the discussion in Section 6.1 on the motivation for choosing the exponential path-loss model in the light of the results from [24]). Then, m corresponds to , and m corresponds to . For , normalised relay locations and state evolution are available in Tables Table 3-Table 6, and, for , normalised relay locations and state evolution are available in Tables Table 3-Table 4. Note that, under per meter and , one unit normalised distance in the tables correspond to m distance in the dense urban environment (due to the normalization as in Section 4.6).

For the sake of illustration, let us consider the sample deployment for , (Table Table 5). In this case, the first relay will be placed at a distance m from the source, the second relay will be placed at a distance m from the source, etc. Also, if we choose such that few relays will be placed on a typical line whose length is several hundreds of meters, then the relays will be placed almost uniformly on the line. But, for small , more relays will be placed and some of them will be clustered near the source (see the deployment for and in Table 3).

### 5.4Comparison with Optimal Offline Deployment

Since there is no prior work in the literature with which we can make a fair comparison of our as-you-go deployment policy for the full-duplex wireless multi-relay network, we compare the performance of our policy with optimal offline deployment. Thus, the numerical experiments reported in Table 7 are a result of asking the following question: how does the cost of as-you-go deployment over a line of exponentially distributed length compare with the cost of placing the same number of relays optimally over the line, once the length of the line has been revealed?

For several combinations of and , we generated random numbers independently from an exponential distribution with parameter . Each of these numbers was considered as a possible realization of the length of the line. Then we computed the placement of the relay nodes for each realization by optimal sequential placement policy, which gave us , a quantity that we use to evaluate the quality of the relay placement. The significance of can be recalled from ( ?) where we found that the rate can be achieved if total power is available to distribute among the source and the relays; i.e., can be interpreted as the net effective attenuation after power has been allocated optimally over the nodes. Also, for each realization, we computed for optimal relay placement, assuming that the length of the line is known before deployment and that the number of relays available is the same as the number of relays used by the corresponding sequential placement policy. For a given combination of and , for the -th realization of the length of the line, let us denote the two values by and . Then the percentage difference for the -th realization is:

The average percentage difference in Table 7 is the quantity . The maximum percentage difference is the quantity .

Discussion of Table 7:

1. For small enough , some relays will be placed at the source itself. For example, for and , we will place two relays at the source (Table Table 1). After placing the first relay, the next state will become , and . The state after placing the second relay becomes , for which (see the placement in Table 1). Now, the line having an exponentially distributed length with mean will end before distance with high probability, and the probability of placing the third relay will be very small. As a result, the mean number of relays will be . In case only relays are placed by the sequential deployment policy and we seek to place relays optimally for the same length of the line (with the length known), the optimal locations for both relays are close to the source location if the length of the line is small (i.e., if the attenuation is small, recall the definition of from Section 3.2). If the line is long (which has a very small probability), the optimal placement will be significantly different from the sequential placement. Altogether, the difference (from (Equation 20)) will be small.

2. For and , is so large that with high probability the line will end in a distance less than and no relay will be placed.

3. From ( ?) we know that for a given placement of relays on a line of given length , the optimal power allocation yields an achievable rate . At the end of as-you-go deployment the power is allocated optimally among the nodes deployed, and a rate can be achieved. If the same number of relays are optimally placed over the same line, with the same total power, then the inner bound is given by . We seek to compare these two rates numerically.

The maximum fractional difference in Table 7 is less than , and substantially smaller than in most cases. Since, in (Equation 20), is always greater than , we have for all (i.e., for all realizations of in the simulation). Now, by the monotonicity of :

Since is a concave function, for any , we have . Using this inequality, we can upper bound the difference in achievable rate from the previous equation by:

This calculation implies that, for the large number of cases reported in Table 7, by using the approximation in (Equation 7) and by using the corresponding optimal policy for as-you-go deployment, we lose at most bits per channel use, compared to the case when the realization of the exponentially distributed source to sink distance is known apriori and when we use the same number of relays as used in the as-you-go deployment case. Note that, the statement of this claim holds with high probability since the maximum difference is taken over sample deployments. Hence, it is reasonable to solve (Equation 7) instead of (Equation 8) which is intractable.

## 6Discussion

### 6.1Exponential Path-Loss Model

Exponential path-loss model has been used before in the context of relay placement (see [16], [22]) and in the context of cellular networks (see [26]). Analytical and experimental support for the exponential path-loss model have been provided by Franceschetti et al. ([24]). Franceschetti et al. used a random scattering model (applicable to an urban environment, or a forest environment) to show that the path-loss in such an environment is the product of an exponential function and a power function of the distance (see [24]). Figure of their paper, which is obtained from measurements made in an urban environment, shows that path-loss (in dB) varies linearly with distance beyond a distance of meters, which implies exponential path-loss for longer distance. These distances are practical for urban scenarios where the network is deployed over several hundreds of meters or several kilometers.

Exponential path-loss was also proposed by Marano and Franceschetti for urban environment, and validated by theory and experiment (see [30]).6

Shadowing (which is typically viewed as being static once a link is deployed) and time varying fading, can be incorporated in our setting by providing a fade-margin in the power at each transmitter. Thus, when expressed in dBm, the actual transmit power for any transmitter-receiver pair is the fade-margin plus the power used in the information theoretic capacity formulas; this fade-margin does not depend on the distance between the transmitter-receiver pair. Note that, this approach, though very conservative in nature, can remove the complexity in analysis arising out of fading in the network. Also note that, if the actual power gain between two nodes distance apart is with , then can be absorbed in the fade margin.

### 6.3Full-Duplex Decode-and-Forward Relaying

Full-duplex radios might become a reality soon; see [11], [12], [13], [14] for recent efforts to realize them. Decode-and-forward relaying requires symbol-level synchronous operation across all nodes in the network. The requirement of globally coherent transmission and reception seems to be restrictive at the moment, but this problem will be solved with the advent of better clocks (with less drift) and efficient clock synchronization algorithms. Any research on impromptu deployment assuming imperfect synchronization, or half-duplex communication, or no interference cancellation, can use this paper as a benchmark for performance analysis.

### 6.4Insights on Power-Law Path-Loss

In [1], we studied the problem of single-relay placement under a per-node power constraint at the source and the relay, for both exponential and power-law path-loss models. The variation of optimal relay location, as the amount of attenuation in the network varies, follow slightly different (but mostly similar) trends (see Figures and of [1]) because of the fact that power-law model allows unbounded power gain (unlike the exponential model) when the distance tends to (). The findings are even more similar when we bound the power gain from above by some constant value in case of the power-law model (power gain is for some ); see the similarity between Figures and in [1]. The results on the fixed node power case provide the insight that when the power gain is or ; under the sum power constraint, the variation of the relay locations as a function of attenuation will follow a pattern similar to that in case of exponential path-loss.

## 7Conclusion

Motivated by the problem of as-you-go deployment of wireless relay networks, we first studied the problem of placing relay nodes along a line, in order to connect a sink at the end of the line to a source at the start of the line, so as to maximize the end-to-end achievable data rate. For the multi-relay channel with exponential path-loss and sum power constraint, we derived an expression for the achievable rate in terms of the power gains among all possible node pairs, and formulated an optimization problem in order to maximize the end-to-end data rate. Numerical work for the fixed source-sink distance suggests that at low attenuation the relays are mostly clustered close to the source in order to be able to cooperate among themselves, whereas at high attenuation they are uniformly placed and work as repeaters. Next, the deploy-as-you-go sequential placement problem was addressed; a sequential relay placement problem along a line having unknown random length was formulated as an MDP, the value function was characterized analytically, and the policy structure was investigated numerically. We found numerically that at the initial stage of the deployment process the inter-relay distances are smaller, and, as deployment progresses, the inter-relay distances increase gradually, and finally the relays start being placed at regular intervals.

Our results are based on information theoretic achievable rate results. In order to utilize currently commercially available wireless devices, we have also been exploring non-information theoretic, packet forwarding models for optimal relay placement, with the aim of obtaining placement algorithms that can be easily reduced to practice (see [7] for reference). The study of as-you-go deployment under the information theoretic model and under the packet forwarding model provides two complementary approaches for two different conditions in the physical layer and the MAC layer, and provides a more comprehensive development of the problem.

obtained his B.E. in Electronics and Telecommunication Engineering from Jadavpur University, Kolkata, India in the year 2008, and M.E. and Ph.D in Telecommunication Engineering from Indian Institute of Science, Bangalore, India in the year 2010 and 2015, respectively. He is currently working in INRIA, Paris as a postdoctoral researcher. His research interests include networks and machine learning.

Abhishek Sinha

is currently a graduate student in the Laboratory for Information and Decision Systems (LIDS), at Massachusetts Institute of Technology, Cambridge, MA. Prior to joining MIT, he completed his Master’s studies in Telecommunication Engineering at the Indian Institute of Science, Bangalore, in the year 2012. His areas of interests include stochastic processes, information theory and network control.

Marceau Coupechoux

is an Associate Professor at Telecom ParisTech since 2005. He obtained his master from Telecom ParisTech in 1999 and from University of Stuttgart, Germany in 2000, and his Ph.D. from Institut Eurecom, Sophia-Antipolis, France, in 2004. From 2000 to 2005, he was with Alcatel-Lucent (Bell Labs former Research & Innovation and then in the Network Design department). In the Computer and Network Science department of Telecom ParisTech, he is working on cellular networks, wireless networks, ad hoc networks, cognitive networks, focusing mainly on layer 2 protocols, scheduling and resource management. From August 2011 to August 2012 he was a visiting scientist at IISc Bangalore.

Anurag Kumar

obtained his B.Tech. degree from the Indian Institute of Technology at Kanpur, and the PhD degree from Cornell University, both in Electrical Engineering. He was then with Bell Laboratories, Holmdel, N.J., for over 6 years. Since 1988 he has been on the faculty of the Indian Institute of Science (IISc), Bangalore, in the Department of Electrical Communication Engineering. He is currently also the Director of the Institute. From 1988 to 2003 he was the Coordinator at IISc of the Education and Research Network Project (ERNET), India’s first wide-area packet switching network. His area of research is communication networking, specifically, modeling, analysis, control and optimisation problems arising in communication networks and distributed systems. Recently his research has focused primarily on wireless networking. He is a Fellow of the IEEE, of the Indian National Science Academy (INSA), of the Indian Academy of Science (IASc), of the Indian National Academy of Engineering (INAE), and of The World Academy of Sciences (TWAS). He is a recepient of the Indian Institute of Science Alumni Award for Engineering Research for 2008.

## 8A Brief Description of the Coding Scheme of

Transmissions take place via block codes of symbols each. The transmission blocks at the source and the relays are synchronized. The coding and decoding scheme is such that a message generated at the source at the beginning of block is decoded by the sink at the end of block , i.e., block durations after the message was generated (with probability tending to 1, as . Thus, at the end of blocks, , the sink is able to decode messages. It follows, by taking