Optimal Capacity Relay Node Placement in a Multihop Wireless Network on a Line^{†}^{†}thanks: This paper is a substantial extension of the workshop paper [1]. The work was supported by the Department of Science and Technology (DST), India, through the J.C. Bose Fellowship and a project funded by the Department of Electronics and Information Technology, India, and the National Science Foundation, USA, titled “Wireless Sensor Networks for Protecting Wildlife and Humans in Forests."^{†}^{†}thanks: This work was done during the period when M. Coupechoux was a Visiting Scientist in the ECE Deparment, IISc, Bangalore.
Abstract
We use information theoretic achievable rate formulas for the multirelay channel to study the problem of optimal placement of relay nodes along the straight line joining a source node and a sink node. The achievable rate formulas that we use are for fullduplex radios at the relays and decodeandforward relaying. For the single relay case, and individual power constraints at the source node and the relay node, we provide explicit formulas for the optimal relay location and the optimal power allocation to the sourcerelay channel, for the exponential and the powerlaw pathloss channel models. For the multiple relay case, we consider exponential pathloss and a total power constraint over the source and the relays, and derive an optimization problem, the solution of which provides the optimal relay locations. Numerical results suggest that at low attenuation the relays are mostly clustered close to the source in order to be able to cooperate among themselves, whereas at high attenuation they are uniformly placed and work as repeaters.
The structure of the optimal power allocation for a given placement of the nodes, then motivates us to formulate the problem of impromptu (“asyougo") placement of relays along a line of exponentially distributed length, with exponential pathloss, so as to minimize a cost function that is additive over hops. The hop cost trades off a capacity limiting term, motivated from the optimal power allocation solution, against the cost of adding a relay node. We formulate the problem as a total cost Markov decision process, for which we prove results for the value function, and provide insights into the placement policy via numerical exploration.
I Introduction
Wireless interconnection of mobile user devices (such as smart phones or mobile computers) or wireless sensors to the wireline communication infrastructure is an important requirement. These are battery operated, resource constrained devices. Hence, due to the physical placement of these devices, or due to the channel conditions, a direct onehop link to the infrastructure “basestation” might not be feasible. In such situations, other nodes could serve as relays in order to realize a multihop path between the source device and the infrastructure. In the cellular context, these relays might themselves be other users’ devices. In the wireless sensor network context, the relays could be other wireless sensors or battery operated radio routers deployed specifically as relays. In either case, the relays are also resource constrained and a cost might be involved in engaging or placing them. Hence, there arises the problem of optimal relay placement. Such an optimal relay placement problem involves the joint optimization of node placement and of the operation of the resulting network, where by “operation" we mean activities such as transmission scheduling, power allocation, and channel coding.
In this paper we consider the problem of maximizing the data rate between a source node and a sink by means of optimally placing relay nodes on the line segment joining the source and the sink; see Figure 1. More specifically, we consider the two scenarios where the length of the line in Figure 1 is known, and where is an unknown random variable whose distribution is known, the latter being motivated by recent interest in problems of “asyougo" deployment of wireless networks in emergency situations (see Section IA, Related Work). In order to understand the fundamental tradeoffs involved in such problems, we consider an information theoretic model. For a placement of the relay nodes along the line and allocation of transmission powers to these relays, we model the “quality" of communication between the source and the sink by the information theoretic achievable rate of the relay channel^{1}^{1}1The term “relay channel" will include the term “multirelay channel.". The relays are equipped with fullduplex radios^{2}^{2}2See [2], [3] for recent efforts to realize practical fullduplex radios., and carry out decodeandforward relaying. We consider scalar, memoryless, timeinvariant, additive white Gaussian noise (AWGN) channels. Since we work in the information theoretic framework, we also assume synchronous operation across all transmitters and receivers. A pathloss model is important for our study, and we consider both powerlaw and exponential pathloss models.
Ia Related Work
A formulation of the problem of relay placement requires a model of the wireless network at the physical (PHY) and medium access control (MAC) layers. Most researchers have adopted the link scheduling and interference model, i.e., a scheduling algorithm determines radio resource allocation (channel and power) and interference is treated as noise (see [4]). But node placement for throughput maximization with this model is intractable because the optimal throughput is obtained by first solving for the optimum schedule assuming fixed node locations, followed by an optimization over those locations. Hence, with such a model, there appears to be little work on the problem of jointly optimizing the relay node placement and the transmission schedule. In [5], the authors considered placing a set of nodes in an existing network such that certain network utility (e.g., total transmit power) is optimized subject to a set of linear constraints on link rates. They posed the problem as one of geometric programming assuming exponential pathloss, and showed that it can be solved in a distributed fashion. To the best of our knowledge, there appears to be no other work which considers joint optimization of link scheduling and node placement using the link scheduling model.
On the other hand, an information theoretic model for a wireless network often provides a closedform expression for the channel capacity, or at least an achievable rate region. These results are asymptotic, and make idealized assumptions such as fullduplex radios, perfect interference cancellation, etc., but provide algebraic expressions that can be used to formulate tractable optimization problems. The results from these formulations can provide useful insights. In the context of optimal relay placement, some researchers have already exploited this approach. For example, Thakur et al. in [6] report on the problem of placing a single relay node to maximize the capacity of a broadcast relay channel in a wideband regime. The linear deterministic channel model ([7]) is used in [8] to study the problem of placing two or more relay nodes along a line so as to maximize the endtoend data rate. Our present paper is in a similar spirit; however, we use the achievable rate formulas for the relay channel (with decode and forward relays) to study the problem of placing relays on a line under individual node power constraints as well as with sum power constraints over the source and the relays.
Another major difference of our paper with [6] and [8] is that we also address the problem of sequential placement of the relay nodes along a line having unknown random length. Such a problem is motivated by the problem faced by “firstresponders" to emergencies (large building fires, buildings with terrorists and hostages), in which these personnel might need to deploy wireless sensor networks “astheygo." Howard et al., in [9], provide heuristic algorithms for incremental deployment of sensors (such as surveillance cameras) with the objective of covering the deployment area. Souryal et al., in [10], address the problem of impromptu deployment of static wireless networks with an extensive study of indoor RF link quality variation. More recently, Sinha et al. ([11]) have provided a Markov Decision Process based formulation of a problem to establish a multihop network between a sink and an unknown source location by placing relay nodes along a random lattice path. The formulation in [11] is based on the socalled “lone packet traffic model" under which, at any time instant, there can be no more than one packet traversing the network, thereby eliminating contention between wireless links. In our present paper, we consider the problem of asyougo relay node placement so as to maximize a certain information theoretic capacity limiting term which is derived from the results on the fixed length line segment.
IB Our Contribution

In Section III, we consider the problem of placing a single relay with individual power constraints at the source and the relay. In this context, we provide explicit formulas for the optimal relay location and the optimal source power split (between providing new information to the relay and cooperating with the relay to assist the sink), for the exponential pathloss model (Theorem 2) and for the power law pathloss model (Theorem 3). We find that at low attenuation it is better to place the relay near the source, whereas at very high attenuation the relay should be placed at halfdistance between the source and the sink.

In Section IV, we focus on the relay placement problem with exponential pathloss model and a sum power constraint among the source and the relays. For given relay locations, the optimal power split among the nodes and the achievable rate are given in Theorem 4 in terms of the channel gains. We explicitly solve the single relay placement problem in this context (Theorem 5). A numerical study shows that, the relay nodes are clustered near the source at low attenuation and are placed uniformly between the source and the sink at high attenuation. We have also studied the asymptotic behaviour of the achievable rate when relay nodes are placed uniformly on a line of fixed length, and show that for a total power constraint among the the source and the relays, , where is the AWGN capacity formula and is the power of the additive white Gaussian noise at each node.

In Section V, we consider the problem of placing relay nodes along a line of random length. Specifically, the problem is to start from a source, and walk along a line, placing relay nodes as we go, until the line ends, at which point the sink is placed. We are given that the distance between the source and the sink is exponentially distributed. With a sum power constraint over the source and the deployed relays, the aim is to maximize a certain capacity limiting term that is derived from the problem over a fixed length line segment, while constraining the expected number of relays that are used. Since the objective can be expressed as a sum of certain terms over the interrelay links, we “relax" the expected number of relays constraint via a Lagrange multiplier, and thus formulate the problem as a total cost Markov Decision Process (MDP), with an uncountable state space and noncompact action sets. We establish the existence of an optimal policy, and convergence of value iteration, and also provide some properties of the value function. A numerical study supports certain intuitive structural properties of the optimal policy.
The rest of the paper is organized as follows. In Section II, we describe our system model and notation. In Section III, node placement with pernode power constraint is discussed. Node placement for total power constraint is discussed in Section IV. Sequential placement of relay nodes for sum power constraint is discussed in Section V. Conclusions are drawn in Section VI. The proofs of all the theorems are given in the appendices.
Ii The Multirelay Channel: Notation and Review
Iia Network and Propagation Models
The multirelay channel was studied in [12] and [13] and is an extension of the single relay model presented in [14]. We consider a network deployed on a line with a source node, a sink node at the end of the line, and fullduplex relay nodes as shown in Figure 1. The relay nodes are numbered as . The source and sink are indexed by and , respectively. The distance of the th node from the source is denoted by . Thus, . As in [12] and [13], we consider the scalar, timeinvariant, memoryless, additive white Gaussian noise setting. We do not consider propagation delays, hence, we use the model that a symbol transmitted by node is received at node after multiplication by the (positive, real valued) channel gain ; such a model has been used by previous researchers, see [12], [15]. The Gaussian additive noise at any receiver is independent and identically distributed from symbol to symbol and has variance . The power gain from Node to Node is denoted by . We model the power gain via two alternative pathloss models: exponential pathloss and powerlaw pathloss. The power gain at a distance is for the exponential pathloss model and for the powerlaw pathloss model, where , . These path loss models have their roots in the physics of wireless channels ([16]). For the exponential pathloss model, we will denote ; can be treated as a measure of attenuation in the line network. Under the exponential pathloss model, the channel gains and power gains in the line network become multiplicative, e.g., and for . In this case, we define and . The powerlaw pathloss expression fails to characterize nearfield transmission, since it goes to as . One alternative is the “modified powerlaw pathloss” model where the pathloss is with a reference distance. In this paper we consider both powerlaw and modified powerlaw path loss models, apart from the exponential pathloss model.
IiB An Inner Bound to the Capacity of the MultiRelay Channel
For the multirelay channel, we denote the symbol transmitted by the th node at time ( is discrete) by for . is the additive white Gaussian noise^{3}^{3}3We consider real symbols in this paper. But similar analysis will carry through if the symbols are complexvalued, so long as the channel gains, i.e., s are real positive numbers. at node and time , and is assumed to be independent and identically distributed across and . Thus, at symbol time , node receives:
(1) 
In [12], the authors showed that an inner bound to the capacity of this network is given by (defining ): ^{4}^{4}4 in this paper will mean the natural logarithm unless the base is specified.
(2) 
where denotes the power at which node transmits to node .
In order to provide insight into the expression in (2) and the relay placement results in this paper, we provide a descriptive overview of the coding and decoding scheme described in [12]. Transmissions take place via block codes of symbols each. The transmission blocks at the source and the relays are synchronized. The coding and decoding scheme is such that a message generated at the source at the beginning of block is decoded by the sink at the end of block , i.e., block durations after the message was generated (with probability tending to 1, as . Thus, at the end of blocks, , the sink is able to decode messages. It follows, by taking , that, if the code rate is bits per symbol, then an information rate of bits per symbol can be achieved from the source to the sink.
As mentioned earlier, we index the source by , the relays by , and the sink by . There are independent Gaussian random codebooks, each containing codes, each code being of length ; these codebooks are available to all nodes. At the beginning of block , the source generates a new message , and, at this stage, we assume that each node has a reliable estimate of all the messages . In block , the source uses a new codebook to encode . In addition, relay and all of its previous transmitters (indexed ), use another codebook to encode (or their estimate of it). Thus, if the relays have a perfect estimate of at the beginning of block , they will transmit the same codeword for . Therefore, in block , the source and relays coherently transmit the codeword for . In this manner, in block , transmitter generates codewords, corresponding to , which are transmitted with powers . In block , node receives a superposition of transmissions from all other nodes. Assuming that node knows all the powers, and all the channel gains, and recalling that it has a reliable estimate of all the messages , it can subtract the interference from transmitters . At the end of block , after subtracting the signals it knows, node is left with the received signals from nodes (received in blocks ), which all carry an encoding of the message . These signals are then jointly used to decode using joint typicality decoding. The codebooks are cycled through in a manner so that in any block all nodes encoding a message (or their estimate of it) use the same codebook, but different (thus, independent) codebooks are used for different messages. Under this encoding and decoding scheme, relatively simple arguments lead to the conclusion that any rate strictly less than displayed in (2) is achievable.
From the above description we see that a node receives information about a message in two ways (i) by the message being directed to it cooperatively by all the previous nodes, and (ii) by overhearing previous transmissions of the message to the previous nodes. Thus node receives codes corresponding to a message times before it attempts to decode the message. Note that, in (2), for any , denotes a possible rate that can be achieved by node from the transmissions from nodes . The smallest of these terms become the bottleneck, which has been reflected in (2).
(3)  
Here, the first term in the of (3) is the achievable rate at node (i.e., the relay node) due to the transmission from the source. The second term in the corresponds to the possible achievable rate at the sink node due to direct coherent transmission to itself from the source and the relay and due to the overheard transmission from the source to the relay.
The higher the channel attenuation, the less will be the contribution of farther nodes, “overheard" transmissions become less relevant, and coherent transmission reduces to a simple transmission from the previous relay. The system is then closer to simple storeandforward relaying.
The authors of [12] have shown that any rate strictly less than is achievable through the abovementioned coding and decoding scheme which involves coherent multistage relaying and interference subtraction. This achievable rate formula can also be obtained from the capacity formula of a physically degraded multirelay channel (see [13] for the channel model), since the capacity of the degraded relay channel is a lower bound to the actual channel capacity. In this paper, we will seek to optimize in (2) over power allocations to the nodes and the node locations, keeping in mind that is a lower bound to the actual capacity. We denote the value of optimized over power allocation and relay locations by .
The following result justifies our aim to seek optimal relay placement on the line joining the source and the sink (rather than anywhere else on the plane).
Theorem 1
For given source and sink locations, the relays should always be placed on the line segment joining the source and the sink in order to maximize the endtoend data rate between the source and the sink.
See Appendix A.
(4) 
(5) 
Iii Single Relay Node Placement: Node Power Constraints
In this section, we aim at placing a single relay node between the source and the sink in order to maximize the achievable rate given by (3). Let the distance between the source and the relay be , i.e., . Let . We assume that the source and the relay use the same transmit power . Hence, , and . Thus, for a given placement of the relay node, we obtain:
(6)  
Maximizing over and , and exploiting the monotonicity of yields the following problem:
(7) 
Note that and in the above equation depend on , but does not depend on .
Iiia Exponential Path Loss Model
IiiA1 Optimum Relay Location
Here , and . Let , and the optimum relay location be . Let be the normalized optimal relay location. Let be the optimum value of when relay is optimally placed.
Theorem 2
See Appendix B.
IiiA2 Numerical Work
In Figure 2, we plot and provided by Theorem 2, versus . Recalling the discussion of the coding/decoding scheme in Section II, we note that the relay provides two benefits to the source: (i) being nearer to the source than the sink, the power required to transmit a message to the relay is less than that to the sink, and (ii) having received the message, the relay coherently assists the source in resolving ambiguity at the sink. These two benefits correspond to the two terms in (3). Hence, Theorem 2 and Figure 2 provide the following insights:

At very low attenuation (), (since ). Since and , we will always have for all and for all . The minimum of the two terms in (7) is maximized by and .
Note that the maximum possible achievable rate from the source to the sink for any value of is , since the source has a transmit power . In the low attenuation regime, we can dedicate the entire source power to the relay and place the relay at the source, so that the relay can receive at a rate from the source. The attenuation is so small that even at this position of the relay, the sink can receive data at a rate by receiving data from the relay and overhearing the transmission from the source to the relay; since , there is no coherent transmission involved in this situation.

For , we again have . decreases from to as increases. In this case since attenuation is higher, the source needs to direct some power to the sink for transmitting coherently with the relay to the sink to balance the source to relay data rate. is the critical value of at which source to relay channel ceases to be the bottleneck. Hence, for , we still place the relay at the source but the source reserves some of its power to transmit to the sink coherently with the relay’s transmission to the sink. As attenuation increases, the source has to direct more power to the sink, and so decreases with .

For , . Since attenuation is high, the sink overhears less the source to relay transmissions and the relay to sink transmissions become more important. Hence, it is no longer optimal to keep the relay close to the source. Thus, increases with . Since the link between the source and the sink becomes less and less useful as increases, we dedicate less and less power for the direct transmission from the source to the sink. Hence, increases with . As , and . We observe that the ratio of powers received by the sink from the source and the relay is less than , since . This ratio tends to zero as . Thus, at high attenuation, the source transmits at full power to the relay and the relay acts just as a repeater.

We observe that . The two data rates in (6) can be equal only if . Otherwise we will readily have , which means that the two rates will not be equal.
IiiB Power Law Path Loss Model
IiiB1 Optimum Relay Location
For the powerlaw pathloss model, , and . Let be the normalized optimum relay location which maximizes . Then the following theorem states how to compute .
Theorem 3

For the single relay channel and powerlaw pathloss model (with ), there is a unique optimal placement point for the relay on the line joining the source and the sink, maximizing . The normalized distance of the point from source node is , where is precisely the unique real root of the Equation in the interval .

For the “modified powerlaw pathloss” model with , the normalized optimum relay location is .
See Appendix B.
IiiB2 Numerical Work
The variation of and as a function of for the powerlaw path loss model are shown in Figure 3. As increases, both and increase. For large , they are close to and respectively, which means that the relay works just as a repeater. At low attenuation the behaviour is different from exponential pathloss because in that case the two rates in (6) can be equalized since channels gains are unbounded. For small , relay is placed close to the source, is high and hence a small suffices to equalize the two rates. Thus, we are in a situation similar to the case for exponential pathloss model. Hence, and increase with .
On the other hand, the variation of and with under the modified powerlaw pathloss model are shown in Figure 4, with . Note that, for a fixed and , remains constant at if , but achieves its maximum over this set of relay locations at . Hence, for any , the achievable rate is maximized at over the interval . For small values of , and hence . Beyond the point where is the solution of , the behaviour is similar to the powerlaw pathloss model. However, for those values of which result in , the value of decreases with . This happens because in this region, if increases, decreases at a slower rate compared to (since ) and decreases at a slower rate compared to , and, hence, more source power should be dedicated for direct transmission to the sink as increases. The variation of is not similar to that in Figure 2 because here, unlike the exponential pathloss model, remains constant over the region and also because .
Iv Multiple Relay Placement : Sum Power Constraint
In this section, we consider the optimal placement of relay nodes to maximize (see (2)), subject to a total power constraint on the source and relay nodes given by . We consider only the exponential pathloss model. We will first maximize in (2) over for any given placement of nodes (i.e., given ). This will provide an expression of achievable rate in terms of channel gains, which has to be maximized over . Let for . Hence, the sum power constraint becomes .
Theorem 4

For fixed location of relay nodes, the optimal power allocation that maximizes the achievable rate for the sum power constraint is given by:
(8) where
(9) 
The achievable rate optimized over the power allocation for a given placement of nodes is given by:
(10)
The basic idea of the proof is to choose the power levels (i.e., ) in (2) so that all the terms in the in (2) become equal. This has been proved in [13] via an inductive argument based on the coding scheme used in [13], for a degraded Gaussian multirelay channel. However, we provide explicit expressions for and the achievable rate (optimized over power allocation) in terms of the power gains, and the proof depends on LP duality theory.
See Appendix C for the detailed proof.
Remarks and Discussion:

We find that in order to maximize , we need to place the relay nodes such that is minimized. This quantity can be seen as the net attenuation the power faces.

When no relay is placed, the effective attenuation is . The ratio of the attenuation with no relay and the attenuation with relays is called the “relaying gain” , and is defined as follows:
(11) The relaying gain captures the effect of placing the relay nodes on the SNR . Since, by placing relays we can only improve the rate, and we cannot increase the rate beyond , can take values only between and .

From (8), it follows that any node will transmit at a higher power to node , compared to any node preceding node .

Note that we have derived Theorem 4 using the fact that is nonincreasing in . If there exists some such that , i.e, if th and st nodes are placed at the same position, then , i.e., the nodes do not direct any power specifically to relay . However, relay can decode the symbols received at relay , and those transmitted by relay . Then relay can transmit coherently with the nodes to improve effective received power in the nodes .
Iva Optimal Placement of a Single Relay Node
Theorem 5
For the single relay node placement problem with sum power constraint and exponential pathloss model, the normalized optimum relay location , power allocation and optimized achievable rate are given as follows :

For , , , and .

For , , , , and
See Appendix C.
Remarks and Discussion:

It is easy to check that obtained in Theorem 5 is strictly greater than the AWGN capacity for all . This happens because the source and relay transmit coherently to the sink. becomes equal to the AWGN capacity only at . At , we do not use the relay since the sink can decode any message that the relay is able to decode.

The variation of and with has been shown in Figure 5. We observe that (from Figure 5 and Theorem 5) , and . For large values of , source and relay cooperation provides negligible benefit since source to sink attenuation is very high. So it is optimal to place the relay at a distance . The relay works as a repeater which forwards data received from the source to the sink.
IvB Optimal Relay Placement for a MultiRelay Channel
As we discussed earlier, we need to place relay nodes such that is minimized. Here . We have the constraint . Now, writing , and defining , we arrive at the following optimization problem:
(12)  
One special feature of the objective function is that it is convex in each of the variables . The objective function is sum of linear fractionals, and the constraints are linear.
Remark: From optimization problem (12) we observe that optimum depend only on . Since , we find that normalized optimal distance of relays from the source depend only on and .
Theorem 6
For fixed , and , the optimized achievable rate for a sum power constraint strictly increases with the number of relay nodes.
See Appendix C.
Let us denote the relaying gain in (11) by .
Theorem 7
For any fixed number of relays , is increasing in .
See Appendix C.
IvC Numerical Work
We discretize the interval and run a search program to find normalized optimal relay locations for different values of and . The results are summarized in Figure 6. We observe that at low attenuation (small ), relay nodes are clustered near the source node and are often at the source node, whereas at high attenuation (large ) they are almost uniformly placed along the line. For large , the effect of long distance between any two adjacent nodes dominates the gain obtained by coherent relaying. Hence, it is beneficial to minimize the maximum distance between any two adjacent nodes and thus multihopping is a better strategy in this case. On the other hand, if attenuation is low, the gain obtained by coherent transmission is dominant. In order to allow this, relays should be able to receive sufficient information from their previous nodes. Thus, they tend to be clustered near the source.^{5}^{5}5At low attenuation, one or more than one relay nodes are placed very close to the source. We believe that some of them will indeed be placed at the source, but it is not showing up in Figure 6 because we discretized the interval to run a search program to find optimum relay locations.
IvD Uniformly Placed Relays, Large
If the relays are uniformly placed, the behaviour of (called in the next theorem) for large number of relays is captured by the following Theorem.
Theorem 8
For the exponential pathloss model with total power constraint, if relay nodes are placed uniformly between the source and sink, resulting in achievable rate, then and .
See Appendix C.
Remark: From Theorem 8, it is clear that we can ensure at least data rate (roughly) by placing a large enough number of relay nodes.
V Optimal Sequential Placement of Relay Nodes on a Line of Random Length
In this section, we consider the sequential placement of relay nodes along a straight line having unknown random length, subject to a total power constraint on the source node and the relay nodes. For analytical tractability, we model the length of the line as an exponential random variable with mean .^{6}^{6}6One justification for the use of the exponential distribution, given the prior knowledge of the mean length , is that it is the maximum entropy continuous probability density function with the given mean. Thus, by using the exponential distribution, we are leaving the length of the line as uncertain as we can, given the prior knowledge of its mean. We are motivated by the scenario where, starting from the data source, a person is walking along the line, and places relays at appropriate places in order to maximize the endtoend achievable data rate (to a sink at the end of the line) subject to a total power constraint and a constraint on the mean number of relays placed. We formulate the problem as a total cost Markov decision process (MDP).
Va Formulation as an MDP
We recall from (10) that for a fixed length of the line and a fixed , has to be minimized in order to maximize . is basically a scaling factor which captures the effect of attenuation and relaying on the maximum possible SNR .
In the “asyougo" placement problem, the person carries a number of nodes and places them as he walks, under the control of a placement policy. A deployment policy is a sequence of mappings , where at the th decision instant provides the distance at which the next relay should be placed (provided that the line does not end before that point). The decisions are made based on the locations of the relays placed earlier. The first decision instant is the start of the line, and the subsequent decision instants are the placement points of the relays. Let denote the set of all deployment policies. Let denote the expectation under policy . Let be the cost of placing a relay. We are interested in solving the following problem:
(13) 
Note that, due to the randomness of the length of the line, the are also random variables.
We now formulate the above “asyougo" relay placement problem as a total cost Markov decision process. Let us define , . Also, recall that . Thus, we can rewrite (13) as follows:
(14) 
When the person starts walking from the source along the line, the state of the system is set to . At this instant the placement policy provides the location at which the first relay should be placed. The person walks towards the prescribed placement point. If the line ends before reaching this point, the sink is placed; if not, then the first relay is placed at the placement point. In general, the state after placing the th relay is denoted by , for . At state , the action is the distance where the next relay has to be placed. If the line ends before this distance, the sink node has to be placed at the end. The randomness is coming from the random residual length of the line. Let denote the residual length of the line at the th instant.
With this notation, the state of the system evolves as follows:
(15) 
Here denotes the end of the line, i.e., the termination state. The single stage cost incurred when the state is , the action is and the residual length of the line is , is given by:
(16) 
Also, for all .
Now, from (15), it is clear that the next state depends on the current state , the current action and the residual length of the line. Since the line is exponentially distributed the residual length need not be retained in the state description; from any placement point, the residual line length is exponentially distributed, and independent of the history of the process. Also, the cost incurred at the th decision instant is given by (16), which depends on , and . Hence, our problem (13) is an MDP problem with state space and action space where . Action means that no further relay will be placed.
Solving the problem in (13) also helps in solving the following constrained problem:
(17)  
s.t., 
where is the constraint on the average number of relays. The following standard result tells us how to choose the optimal :
Lemma 1
The constraint on the mean number of relays can be justified if we consider the relay deployment problem for multiple sourcesink pairs over several different lines of mean length , given a large pool of relays, and we are only interested in keeping small the total number of relays over all these deployments.
Remark: The optimal policy for the problem (13) will be used to place relay nodes along a line whose length is a sample from an exponential distribution with mean . After the deployment is over, the power will be shared optimally among the source and the deployed relay nodes (according to the formulas in Theorem 4), in order to implement the coding scheme (as described in Section IIB) in the network.
VB Analysis of the MDP
Suppose for some . Then, the optimal value function (costtogo) at state is defined by:
If we decide to place the next relay at a distance and follow the optimal policy thereafter, the expected costtogo at a state becomes:
(18)  
The first term in (18) corresponds to the case in which the line ends at a distance less than and we are forced to place the sink node. The second term corresponds to the case where the residual length of the line is greater than and a relay is placed at a distance .
Note that our MDP has an uncountable state space and a noncompact action space . Several technical issues arise in this kind of problems, such as the existence of optimal or optimal policies, measurability of the policies, etc. We, therefore, invoke the results provided by Schäl [17], which deal with such issues. Our problem is one of minimizing total, undiscounted, nonnegative costs over an infinite horizon. Equivalently, in the context of [17], we have a problem of total reward maximization where the rewards are the negative of the costs. Thus, our problem specifically fits into the negative dynamic programming setting of [17] (i.e., the case where singlestage rewards are nonpositive).
Now, we observe that the state is absorbing. Also no action is taken at this state and the cost at this state is . Hence, we can think of this state as state in order to make our state space a Borel subset of the real line. We present the following theorem from [17] in our present context.
Theorem 9
[[17], Equation (3.6)] The optimal value function satisfies the Bellman equation.
Thus, satisfies the following Bellman equation for each :
(19)  
where the second term inside is the cost of not placing any relay (i.e., ).
Recall that the line has a length having exponential distribution with mean and that the pathloss exponent is for the exponential pathloss model. We analyze the MDP problem for the two different cases: and .
VB1 Case I ()
We observe that the cost of not placing any relay (i.e., ) at state is given by:
where (using the fact that ). Since not placing a relay (i.e., ) is a possible action for every , it follows that .
The cost in (18), upon simplification, can be written as:
(20) 
Since for all , the expression in (20) is strictly less that for large enough . Hence, according to (19), it is not optimal to not place any relay.
Thus, in this case, the Bellman equation (19) can be rewritten as: