Self-Organization Scheme for Balanced Routing in Large-Scale Multi-Hop Networks

Self-Organization Scheme for Balanced Routing in Large-Scale Multi-Hop Networks

Mihai-Alin Badiu, David Saad, and Justin P. Coon,  M.-A Badiu is with the Department of Electronic Systems, Aalborg University, Aalborg 9220, Denmark (e-mail: and the Department of Engineering Science, University of Oxford, Parks Road, Oxford OX1 3PJ, UK (email: Saad is with the School of Engineering and Applied Science, Aston University, Birmingham B4 7ET, United Kingdom (e-mail: P. Coon is with the Department of Engineering Science, University of Oxford, Parks Road, Oxford OX1 3PJ, UK (email: work was supported by the Independent Research Fund Denmark under Grant ID DFF–5054-00212, the Leverhulme Trust Grant No. RPG-2013-48 (D.S.) and EPSRC grant number EP/N002350/1 (“Spatially Embedded Networks”). Most of the research was carried out during an extended visit to Aston University, and M.-A Badiu would like to thank the Mathematics group for the hospitality.

We propose a self-organization scheme for cost-effective and load-balanced routing in multi-hop networks. To avoid overloading nodes that provide favourable routing conditions, we assign each node with a cost function that penalizes high loads. Thus, finding routes to sink nodes is formulated as an optimization problem in which the global objective function strikes a balance between route costs and node loads. We apply belief propagation (its min-sum version) to solve the network optimization problem and obtain a distributed algorithm whereby the nodes collectively discover globally optimal routes by performing low-complexity computations and exchanging messages with their neighbours. We prove that the proposed method converges to the global optimum after a finite number of local exchanges of messages. Finally, we demonstrate numerically our framework’s efficacy in balancing the node loads and study the trade-off between load reduction and total cost minimization.

Load balancing, routing, Internet of things, distributed optimization, belief propagation, self-organization.

I Introduction

Large-scale wireless networks employing multi-hop transmissions are an integral component of the Internet of Things [1]. For example, such networks can consist of a massive number of sensors that collect data from the environment and send it to central controllers. Since in multi-hop networks each wireless node can relay other nodes’ messages, it is highly relevant to direct the information flows from the source nodes to the destinations efficiently in terms of, e.g., energy consumption or reliability. Sending the flows along the minimim-cost paths towards the destinations potentially leads to overloading those nodes that provide favourable routes, which can cause quick battery depletion or decrease the resilience of the network against node failures [2, 3, 4, 5]. Therefore, information should be routed through the network so as to minimize costs while trying to balance the node-loads. Moreover, given their scale, such networks must be designed to be self-organizing and adaptive.

There is a large body of work studying energy efficient routing protocols (see, e.g., the survey [6]). A typical objective is to maximize the network lifetime by maximizing the minimum lifetime over all nodes, where the lifetime of a node is defined as the ratio between its residual energy and its energy expenditure [2, 3, 4]. However, the network lifetime objective does not account for the total routing cost (total energy in this case) and thus can be inefficient in this respect, similar to minimum-cost routing being suboptimal for node balancing. It is therefore relevant to investigate objectives that favour solutions that are somewhere “in-between” these two extremes.

In this work, we propose an algorithmic strategy for distributed multi-hop networking whereby the nodes coordinate and organize themselves so as to route the information to the destinations in an efficient and balanced way. To this end, we model balanced routing as the minimization of a network objective function, which includes the overall cost of the routes (given by generic link costs) and an additional term that penalizes the node-loads. The objective function provides a tunable trade-off between total cost efficiency and fairness of the distribution of the node loads. The possible routes from source nodes to destinations are coupled in the objective function, which creates a competition for the shortest (i.e., least cost) routes to the sinks. To solve the optimization problem, we use the min-sum version of the belief propagation (BP) method [7]. In this way, we obtain a distributed algorithm which finds globally optimal routes in a decentralized manner with low-complexity local computations and message exchanges between neighbouring nodes. We also show that the proposed method converges to the global optimum in a finite number of iterations.

Ii Network Model and Problem Formulation

We assume a data collection scenario in which a set of nodes generate and/or relay information that has to be delivered to any subset of the destination nodes (e.g., gateways, access points) in . The nodes in are simple devices with constrained resources (energy, memory, processing capabilities, etc.) and can participate in routing each other’s packets towards the destination nodes. Packets generated by a source node in can travel to a destination in over different routes; moreover, they can be delivered to different destination nodes.

We model the wireless network as a directed graph , with and being the set of edges (links). An edge indicates that node can transmit to node directly. For each , denotes the set of all edges incident to , while and stand for the sets of its outgoing and respectively incoming edges. Node generates information at a rate of units (we assume a certain unit rate ), where ; if , the node is just a relay node. The capacity of edge is units, , such that the amount of flow units carried by satisfies . The assumption that the rates and capacities are integer multiples of is not restrictive, because any set of rational numbers can be expressed in this way by finding an appropriate unit . Moreover, if any of the rates and capacities have irrational values, it is necessary to convert them to rational numbers to represent them on a computer. We associate each link with the weight representing the cost of transferring a unit over edge . For example, the cost can be the transmit power required to ensure a certain data rate, the expected transmission count (ETX), or hop-count (when ). We further assume that the network is in the unsaturated traffic regime and packets are transferred between neighbours according to a medium access scheme, which we do not concern ourselves with here.

The routing solution space consists of those configurations which satisfy the flow conservation constraints


and the capacity constraints , for all . The two constraints ensure that all generated flows are delivered to the destinations such that edge flows do not exceed the respective capacities. We assume that the solution space is non-empty. The total cost of a configuration is . Furthermore, we define the load of node to be the amount of flow it has to forward.

In general, there are many feasible configurations, each implying different sets of routes, path lengths, total costs, distribution of node loads, etc. A common objective is to minimize the total cost, which, as one can notice, turns data collection into a (linear) minimum cost network flow problem [8]. However, such an approach may yield solutions wherein some nodes that provide low-cost forwarding edges experience high loads. We are therefore interested in balancing the node loads in a cost-effective manner.

Iii Proposed Objective for Load Balancing

We seek a trade-off between minimization of the total cost and minimization of the loads of individual nodes. To this end, for each we introduce the strictly-increasing convex function to penalize the load of the th node. The functions can vary over the nodes to reflect their different load-tolerances depending on residual energies, capabilities etc. Now, we formulate the optimization problem

subject to

where is a parameter that balances cost-efficiency and load minimization. When , we recover the linear minimum cost flow problem [8], which gives the most cost-efficient flow configuration; however, this setting usually does not provide well-balanced loads and therefore we focus on .

In the following, we assume that the functions are piecewise-linear convex (PLC) with integral breakpoints, which is very convenient for obtaining a simple message-passing algorithm with provable convergence to the correct solution, as we show next in Prop. 1 and Prop. 2. An example of such function is one that takes the value , with , at each breakpoint and varies linearly between consecutive breakpoints; the higher the value of , the stronger the load is penalized. Such a choice provides a simple way to select the efficiency-fairness trade-off by tuning the parameter .

Iv BP Algorithm for Balanced Routing

BP is a generic message-passing algorithm for solving large-scale inference and optimization problems in graphical models. It has a distributed nature whereby the nodes of the graph perform simple local computations and exchange messages with their neighbours. While BP provides correct solutions when the underlying graph is a tree, its correctness and convergence cannot be generally guaranteed for graphs with cycles, with few exceptions [7, 9]. Nonetheless, for graphs with cycles, the BP heuristic often performs very well. In network problems, the min-sum algorithm is applied to find the shortest path between two nodes [10] or minimize path lengths and link congestion [11]. For the min-cost network flow problem with linear or PLC costs on edges, BP was shown in [9] to converge to the correct solution (if the solution is unique). Compared to [9], our objective (2) (with ) additionally includes node costs given by the PLC functions ; therefore, the application of BP gives the novel algorithm described next.111Alternatively, by using the node splitting technique [8, p. 41], one can transform (2) into a min-cost network flow problem with PLC costs on edges, which can be solved using BP [9, Th. 6.1]. However, BP on the transformed graph is different from Algorithm 1 that we obtain here, see footnote 2.

For each node , we define a function that reflects the flow conservation constraint at node , i.e., it maps each vector of edge flows to

Furthermore, we define

which additionally includes the load penalty for node . On the contrary, destination nodes do not have any constraints and “accept” any flows on their incoming edges, so we set , for any and . Next, we capture the cost and capacity constraint of edge by introducing the function given by

We can now reformulate (2) as the equivalent problem


where includes those components of with indices in .

We apply the min-sum version of BP to solve (3). Given that each edge variable node has exactly two neighbour function nodes from the set , we simplify the standard message updates by defining the messages (4) in Algorithm 1. At iteration , for each node and incident edge , where either or , the algorithm computes the message , which becomes an input to neighbour at the next iteration. Since is the zero function for all , the messages computed by destination nodes do not change with and thus are not updated.

0:  The graph , edge costs and capacities , data rates , parameters and
0:  Estimates of the optimal edge flows of (2)
1:  Initialize , for all , ,
2:  for  to  do
3:     For each and , update
for all , where .
4:  end for
5:  For each , compute the belief function
and determine its minimizer
6:  return  
Algorithm 1 Distributed algorithm for balanced routing.

Algorithm 1 has the following interpretation. Every node is seeking to determine the flow on each of its incident edges while satisfying its local flow conservation constraint and minimizing its load. The message can be viewed as a local cost that node attributes to allocating units to edge ; thus, the message is a function of the flow. For any , the message update (4) includes: (i) the cost of sending flow over edge and (ii) the minimum cost of allocating flows to the rest of the edges that are incident to such that flow conservation is ensured. The latter cost is the result of a local optimization, which looks for the feasible configuration of the flows on the incident edges that minimizes an objective function that includes the cost of the load of node and the local costs (messages) estimated by the neighbouring nodes.222If using BP on the augmented graph obtained by node-splitting (see footnote 1), then effectively depends on messages over incoming (if ) or outgoing (if ) edges that were computed at , i.e., it uses outdated information, which slows down convergence, as Fig. (c)c shows. The message updates have low-complexity, as we show next.

Proposition 1 (Complexity)

For each , and , the message is a piecewise-linear convex (PLC) function with breakpoints in . The complexity of its update (4) is linear in the total capacity of the input and output edges of node and logarithmic in and .


The proof is by induction on . At , Algorithm 1 initializes the messages to trivial PLC functions. Suppose at iteration all messages are PLC functions with integral breakpoints. We provide the proof for with , as the case is very similar. Let , , be

and define , , given by

Now, we define the function

The minimization in the r.h.s. is a so-called interpolation of PLC functions whose complexity is logarithmic in the number of functions and linear in the total number of their linear pieces [9]. Since has breakpoints in and is also PLC with integral breakpoints, it follows that the function is itself PLC with integral breakpoints and at most pieces, where ; moreover, can be computed in operations. Now, we write (4)

Given that and the messages at are PLC with integral breakpoints, the interpolation in the second line gives again a PLC function; its computation takes operations, where . The addition of , which is linear in , makes PLC with integral breakpoints. \qed







Fig. 1: Simulation results for , and various fractions of source nodes: (a) Improvement of the total cost and maximum load relative to minimum-cost routing; (b) Jain’s fairness index for the node loads; (c) empirical cdf of the minimum number of iterations required for Algorithm 1 to converge when .

We establish that Algorithm 1 outputs the optimal solution after a finite number of iterations.

Proposition 2 (Convergence)

Suppose (2) has a unique optimal solution .333When the costs are generic (e.g., random), it is highly likely that (2) has a unique solution. Otherwise, it is possible to add small noise to the costs such that the modified problem has a unique solution which very closely approximates the solution of the original problem [9]. Then, there exists a finite integer such that the output of Algorithm 1 satisfies , for any .


Although our objective function (2) is different than that of the min-cost network flow problem with linear (or PLC) edge costs, we can use the same proof strategy as in [9, Th. 4.1, Th. 6.1]. The difference is that we need to define an appropriate residual graph [8]. Denote by the residual graph of with respect to the flow . has the same vertices , while we define its edges and their costs as follows: for any , if , then is also an edge in with capacity and cost , where is the load of node ; if , then additionally includes the directed edge with capacity and cost . At the unique optimal solution , all the directed cycles of the residual graph must have positive costs (according to the negative cycle optimality criterion [8]). The proof relies on this property and follows the same steps as that of [9, Th. 4.1]; therefore we omit the details. \qed

V Numerical Results

We consider nodes independently and uniformly distributed inside the unit square and sink node at the center of the square. Any two nodes and that are spaced by less than are connected by the directed edges and . We discard the network realizations that are not connected. For each realization, we randomly select sources out of the nodes; the sources generate information at unit rate, while the remaining nodes act as relays. The cost associated with each link is the expected transmission count (ETX), which is drawn uniformly at random from the interval .

For the proposed balanced routing scheme (Algorithm 1 with and power ), we evaluate the total cost, the maximum of the node loads , the Jain’s index, , as a measure of fairness in the distribution of the loads and the empirical distribution of the minimum number of iterations required for Algorithm 1 to converge. We compare the results obtained using our algorithm against minimum-cost routing which is instantiated by setting in Algorithm 1. The results in Fig. 1 are obtained by averaging from independent trials. In Fig. (a)a, we observe that balancing with reduces the maximum load by compared to minimum-cost routing across all fractions of source nodes, while the total cost increases by ; increasing to two brings larger reduction of the maximum load, of about , and a higher relative total cost of about . Fig. (b)b shows that the balanced routing scheme provides significantly fairer load-distributions. As illustrated in Fig. (c)c, the number of iterations required to find a balanced solution is higher than for min-cost routing and increases with . We also evaluated BP on the graph transformed by node splitting (see footnotes 1 and 2) and, while it outputs the same solutions, it requires a higher number of iterations than our method, as shown in Fig. (c)c.

Vi Conclusion

We formulated balanced routing in large-scale networks (such as Internet of Things) as optimization of an objective function that provides a tunable trade-off between total cost efficiency and fairness of the distribution of the node loads. In the proposed decentralized scheme, the nodes collectively find the globally optimal routing solution through low-complexity local computations and exchanges of messages with neighbours. The scheme provides significantly fairer solutions than minimum-cost routing at the expense of slightly increased total cost and higher number of required iterations.

There are several interesting directions to explore further, such as adapting the framework to specific models of energy consumption, including in the design the notions of reliability, trust among nodes and security, but also extending the framework to take into account the scheduling of the transmissions.


  • [1] A. Al-Fuqaha, M. Guizani, M. Mohammadi, M. Aledhari, and M. Ayyash, “Internet of things: A survey on enabling technologies, protocols, and applications,” IEEE Communications Surveys Tutorials, vol. 17, no. 4, pp. 2347–2376, Fourthquarter 2015.
  • [2] C. K. Toh, “Maximum battery life routing to support ubiquitous mobile computing in wireless ad hoc networks,” IEEE Communications Magazine, vol. 39, no. 6, pp. 138–147, Jun 2001.
  • [3] J.-H. Chang and L. Tassiulas, “Maximum lifetime routing in wireless sensor networks,” IEEE/ACM Transactions on Networking, vol. 12, no. 4, pp. 609–619, Aug 2004.
  • [4] R. Madan and S. Lall, “Distributed algorithms for maximum lifetime routing in wireless sensor networks,” IEEE Transactions on Wireless Communications, vol. 5, no. 8, pp. 2185–2193, Aug 2006.
  • [5] J. Vazifehdan, R. V. Prasad, and I. Niemegeers, “Energy-efficient reliable routing considering residual energy in wireless ad hoc networks,” IEEE Trans. on Mobile Computing, vol. 13, no. 2, pp. 434–447, Feb 2014.
  • [6] N. A. Pantazis, S. A. Nikolidakis, and D. D. Vergados, “Energy-efficient routing protocols in wireless sensor networks: A survey,” IEEE Commun. Surveys Tutorials, vol. 15, no. 2, pp. 551–591, Second 2013.
  • [7] M. Mezard and A. Montanari, Information, Physics, and Computation.   New York, NY, USA: Oxford University Press, 2009.
  • [8] R. K. Ahuja, T. L. Magnanti, and J. B. Orlin, Network Flows: Theory, Algorithms, and Applications.   Upper Saddle River, NJ, USA: Prentice-Hall, Inc., 1993.
  • [9] D. Gamarnik, D. Shah, and Y. Wei, “Belief propagation for min-cost network flow: Convergence and correctness,” Operations Research, vol. 60, no. 2, pp. 410–428, 2012.
  • [10] N. Ruozzi and S. Tatikonda, “s-t paths using the min-sum algorithm,” in 2008 46th Annual Allerton Conference on Communication, Control, and Computing, Sep 2008, pp. 918–921.
  • [11] C. H. Yeung and D. Saad, “Competition for shortest paths on sparse graphs,” Phys. Rev. Lett., vol. 108, p. 208701, May 2012.
Comments 0
Request Comment
You are adding the first comment!
How to quickly get a good reply:
  • Give credit where it’s due by listing out the positive aspects of a paper before getting into which changes should be made.
  • Be specific in your critique, and provide supporting evidence with appropriate references to substantiate general statements.
  • Your comment should inspire ideas to flow and help the author improves the paper.

The better we are at sharing our knowledge with each other, the faster we move forward.
The feedback must be of minimum 40 characters and the title a minimum of 5 characters
Add comment
Loading ...
This is a comment super asjknd jkasnjk adsnkj
The feedback must be of minumum 40 characters
The feedback must be of minumum 40 characters

You are asking your first question!
How to quickly get a good answer:
  • Keep your question short and to the point
  • Check for grammar or spelling errors.
  • Phrase it like a question
Test description