Message Passing for Optimization and Control of Power Grid: Model of Distribution System with Redundancy
Abstract
We use a power grid model with generators and consumption units to optimize the grid and its control. Each consumer demand is drawn from a predefined finitesizesupport distribution, thus simulating the instantaneous load fluctuations. Each generator has a maximum power capability. A generator is not overloaded if the sum of the loads of consumers connected to a generator does not exceed its maximum production. In the standard grid each consumer is connected only to its designated generator, while we consider a more general organization of the grid allowing each consumer to select one generator depending on the load from a predefined consumerdependent and sufficiently small set of generators which can all serve the load. The model grid is interconnected in a graph with loops, drawn from an ensemble of random bipartite graphs, while each allowed configuration of loaded links represent a set of graph covering trees. Losses, the reactive character of the grid and the transmissionlevel connections between generators (and many other details relevant to realistic power grid) are ignored in this proofofprinciples study. We focus on the asymptotic limit, and , and we show that the interconnects allow significant expansion of the parameter domains for which the probability of a generator overload is asymptotically zero. Our construction explores the formal relation between the problem of grid optimization and the modern theory of sparse graphical models. We also design heuristic algorithms that achieve the asymptotically optimal selection of loaded links. We conclude discussing the ability of this approach to include other effects, such as a more realistic modeling of the power grid and related optimization and control algorithms.
I Introduction
The existing power grid is complex and far from being optimized. The anticipated installation of smallscale distributed generators and storage devices, as well as the addition of many ancillary backup lines and control devices at both transmission and distribution levels, imply that the main future challenges will require intelligent planning, optimization and control of this ever growing grid. This optimized and efficient grid of the future, that incorporates new hardware and concepts such as renewable generation and distributed storage, has been labeled ”smart grid” 08D (2008); 09S (). Accounting for many important details of the power distribution and transmission is not feasible without much simplification. Simplified models must identify the significant effects, extracting and analyzing and later probing each of them separately and in combinations. (See, e.g. related review articles Massoud Amin and Wollenberg (2005); Ilic (2007); Amin (2008).)
We adopt this ”discovery though simplification” approach and focus in this paper on improving the functioning and control of the electric grid on the power distribution level. Specifically, our prime focus is on preventing overloads of the power generation units caused by fluctuations in the demand by efficiently utilizing ancillary lines. This approach is justified in the context of a cityscale transmission system, in which many significant loads and generators are in geometric proximity of each other and the cost of building new ancillary lines is not prohibitive. We assume that shedding excess load is not an option and we consider the possibility of redistributing the load via a system of interconnects over a larger grid than what is used under normal (no overload) conditions. Fundamentally, we ask the following question: can an intelligent arrangement of ancillary lines among a system of generators, each serving customers, possibly reduce the probability of a generator failure from a finite number to zero in the limit of and ? In fact, an assumption that a drastic improvement is possible stems from basic informationtheoretic intuition: any finite error probability can be reduced to zero via properly introduced redundancy Shannon (1948). We show that adding interconnects to the power system is related to adding redundancy to the information system, in the sense that achieving the asymptotic overloadfree distribution is indeed a possibility for an idealized grid.
The model of power line considered in the paper mimics how the ancillary power lines are operated on the distribution level power grid. A cityscale power grid has an intervene loopy structure, however these loops are typically used for ancillary (backup and/or maintenance) purposes and operators aim at avoiding running current over loops. We adopt this strategy and assume that the system contain (or may be built with) many switches and that it is operated in the manner that for each given configuration of the preinstalled onoff switches the currents flow over trees, i.e. subgraphs of the full loopy distribution graph without loops. We also assume that the lines are sufficiently short (kilometers, not hundreds or thousand of kilometers), and thus thermal losses are not important and reactive parts of line impedances can be ignored. These assumptions correspond to the socalled DC approximation Wood and Wollenberg (1996). Combination of the loopfree structure of power flows (for any given configuration of switches) with DC approximation allows to model electricity delivery as an abstract commodity flow Cormen et al. (1990). For completeness of our description, let us also note that some other effects of realistic power flows are ignored in our first publication on this emerging subject. Thus, we do not consider back flows (consumers turned into distributed generators) and for that matter we do not discuss at all the entire scale up (transmission level) structure of the grid, inhomogeneities and spatiotemporal correlations in loads and generation, and all economy, pricing and regulation effects Baldick and Hogan (2001). In essence, our main task here consists in establishing the existence of fundamental limits, and bounds on the idealized asymptotic failurefree regime, and followed by developing efficient and simple algorithm controlling switching in the grid.
To solve our model we use the cavity method and its computational realization via population dynamics, introduced in the statistical physics of disordered systems Mézard and Parisi (1985, 2001) and recently adapted to the analysis of Shannon (phase) transitions in constraint satisfaction Mezard et al. (2002) and errorcorrection Richardson and Urbanke (2008); Mezard and Montanari (2009). This method explores the famous fact that the BethePeierls Bethe (1935); Peierls (1936) or Belief Propagation Gallager (1963); Pearl (1988) (BP) scheme exactly solves probabilistic models on graphs without loops (a tree). The method allows to evaluate the ensemble average over configurations of allowed discrete switchings on the grid balancing load with generation. We study how the number of allowed solutions scale with the size of the grid, and identify a transition (in the space of model parameters) from a satisfiable (SAT) domain, in which the number of solutions is exponentially large (in the system size) and where BP algorithm finds a valid solution easily, to an unsatisfiable (UNSAT) domain in which the number of valid solutions is with high probability zero (thus load shedding would be required).
Assuming that the two ways communications between consumers and generators exists we design a stochastic local search algorithm, coined WalkGrid, that is able to find optimal configuration of switches almost anywhere in the SAT phase. We also utilize aforementioned BP analysis and develop a BPbased message passing scheme for efficient search of SAT configuration of switches. Generally, WalkGrid outperforms the BPbased algorithm almost anywhere in the SAT phase. Note however that one useful feature of BP (not readily available in WalkGrid) consists in its ability to count number of available SAT solutions, and thus have a direct algorithmic test of the distance to failure, i.e. distance to the SATUNSAT transition.
The material in the manuscript is organized as follows. We introduce our model of the distribution grid in Section II. The Belief Propagation method is detailed in Section III. We describe population dynamics algorithms and present SATUNSAT transition results in IV. Control algorithms are explained in Section V. Section VI summarizes our approach and proposes extensions of the study for future explorations relevant to intelligent optimization and control of the power grid.
Ii Grid model
Consider sources/generators each connected to distinct consumers, so that the total number of consumers is . Greek/Latin indices will be reserved for generators/consumers. For simplicity we assume that each generator has a maximum production rate of unity, i.e. , though inhomogeneities in the production can be easily incorporated into our approach. The configuration of loads, , is drawn from an assumed known distribution with support on the interval . Our enabling example is the flatbox ensemble with a fraction of customers dropped off from the network, i.e. drawing no electricity at all,
(1) 
The mean consumption then is , and is the width of the part of the distribution, correspondent to nonzero demand. We require that the consumption of each individual customer is nonnegative
(2) 
and we assume that under normal conditions, i.e. when demand is not excessive, there is always enough power produced in the network to supply electricity to all consumers, i.e.
(3) 
In our ”standard” model of the grid each consumer is assigned to strictly one generator, while each generator feeds exactly consumers. This corresponds to a graph decomposed into simple trees, where is the number of generators. To guarantee in this standard case that a generator is always SAT, i.e. it is capable of supplying electricity to all of its customers, one requires
(4) 
This later condition will be challenged below, in the sense that we will show that by adding ancillary lines one does not need to impose (4), but instead have a weaker condition, thus extending the domain of the asymptotically failurefree generation.
To achieve this improvement we consider an interconnected grid built from the standard/separated one by adding ancillary lines between consumers and generators. For our quantitative analysis we choose a random graph drawn in the following way. First, consumers are connected to every producer. Second, we choose consumers from each producer and connect them to a second producer in such a way that every producer is connected to consumers. Of these connections go to singly connected consumers and to doubly connected consumers. So that a total of consumers are connected to two generators and are connected to one generator. See Fig. b for illustration. The choice of the two parameter graph ensemble is used for simplicity. Our analysis method is valid for any random network as long as it is bipartite and locally treelike, meaning that the length of the shortest loop going through a random node is .
We say that given configuration of loads is SAT (satisfiable) if there exists a matching , where is the bipartite graph accounting for all generators, consumers and lines, such that the following set of conditions are simultaneously satisfied
(5)  
(6) 
where , resp. , stand for all the nodes to which , resp. , is connected. If the reverse is true, i.e. there exists no valid with all Eqs. (5,6) satisfied, we say that is UNSAT (unsatisfiable). First, we aim to solve the decision problem: is is SAT or UNSAT? Furthermore, if is SAT, we would like to find at least one valid solution, . Both problems can be stated for a given graph or, alternatively, can be considered ”in average” for ensemble of graphs. We describe the ”average” solution of the former problem in Sections IV and the algorithmic solution of the latter in Section V. Both Sections will be preceded by a preparatory discussion of the BP approach in Section III.
Related models have been studied in the context of resource allocation. One example is the problem of online advertising known as AdWords, in particular its uniform offline version Lehmann et al. (2001); Andelman and Mansour (2004); Azar et al. (2008). The main difference between our model and the model for budgetconstrained advertising problem is that condition (6) is replaced by the maximization of the total revenue . This relaxation translates into allowing one to shed loads, that is not an option in our setting. Discussion of the BPbased approaches to the AdWords problem and their methodology in Altarelli et al. (2009) is very much related to ours.
Of other models developed in Computer Sciences, the one most similar to ours is the socalled offline weighted ballsintobins games (sometimes called balanced loading) Talwar and Wieder (2007); Berenbrink et al. (2008). The offline version of the ballsintobins games is known to be polynomial in the nonweighted case (i.e. for a special discrete choice of the distribution (1)), as in that case it can be mapped into a maxflow (mincut) problem Sanders et al. (2000). On the other hand the general (weighted) version on a fully connected graph (every customer being possibly connected to every generator) is equivalent to the NPcomplete bin packing problem (number partitioning in the case of two generators). We are not aware of any computational complexity results for the case of bounded number of consumers per generator. Therefore, and given that the maxflow mapping of Sanders et al. (2000) does not generalize to the most general version of Eq. (1), we conjecture that computationally the general problem described by Eq. (1) is NPhard.
Iii Belief Propagation
In the asymptotic limit of an infinite system, for which while and are , the interconnected grid is locally treelike. Therefore, the BethePeierls (Belief Propagation) BP approach for evaluating the generalization of (4) is expected to be asymptotically exact (in a sense to be clarified later in this text). This Section describes details of the BP approach
We introduce the following set of marginal probabilities:

is the probability that generator is satisfied given that the edge , connecting with his consumerneighbor on , is in the active state, i.e. .

is the probability that generator is satisfied given that the edge , where , is inactive, i.e. .

is the probability that is satisfied (i.e. it is connected to exactly one generator) given that the edge , where , is active, i.e. .

is the probability that is satisfied (i.e. it is connected to exactly one generator) given that the edge , where , is inactive, i.e. .
BP relates these marginal probabilities to each other assuming that the relations are graph local, i.e. as if the graph would contain no loops. The resulting Belief Propagation (BP) equations are
(7)  
(8)  
(9)  
(10) 
where is a standard notation for the set of generator nodes linked to consumer , however excluding generator , and similarly stands for the set of consumer nodes linked to generator excluding consumer . In Eqs. (710), and are normalizations ensuring that, and . The is the step function enforcing the generation constraints. It is unity if the argument is positive and zero otherwise. The probability for the link/edge to be active, stated in terms of the related and , is
(11) 
where . Note, for the sake of completeness, that the BP equations can be derived in the spirit of Yedidia et al. (2001) as conditions for a fixedpoint (minimum) of a functional of marginal probabilities (beliefs), the socalled Bethe free energy functional.
The Bethe entropy, defined as the logarithm of the number of possible SAT configurations (and also equal to the Bethe free energy evaluated at the solution of the previously mentioned BP equations), is
(12)  
(13)  
(14)  
(15) 
The entropy is extensive, , and selfaveraging, i.e. the distribution of is concentrated around its mean value with dispersion being at .
Iv Average Bethe Entropy via Population Dynamics
Fixed point of the BP Eqs. (710) and the corresponding entropy (15) can be obtained by solving Eqs. (710) iteratively for a given instance of the problem. Repeating these simulations many times for different instances of the graph and load ensembles one can also calculate the average Bethe entropy. However, the average behavior, corresponding to the limits of the infinite graph, can be obtained more efficiently via the population dynamics technique. This technique offers a computationally efficient sampling from the distribution of the marginal probabilities on infinite random graphs and subsequent evaluation of the average Bethe entropy. Below we give a very brief exposition of the population dynamics approach applied to our model. The interested reader is referred to Mézard and Parisi (2001); Zdeborová (2009) for further details.
In population dynamics, we create a pool of components, each characterized by the vector with drawn from the original distribution of demands (1) and the initial selected arbitrarily. Messages leaving the generator and entering the consumer connected to a single generator will always be kept fixed to , , while other messages will be updated iteratively, so that at any new sweep a new pool is derived from the old one. A sweep consists of the following step repeated times (each time one of the components is updated). A step consists of choosing a random number , random numbers , and random elements from the pool representing the incoming messages , and computing the message based on Eqs. (9,10). Then we replace a random element of the pool by the vector , in accordance with Eqs. (78), thus guaranteeing that and . We repeat this sweep procedure many times until convergence is achieved. To compute the average Bethe entropy we apply similar procedure of sampling from the resulting pool and thus averaging all the terms in Eq. (15).
Implementing the population dynamics method we observed three possible outcomes

SAT phase: The Bethe entropy is positive suggesting that the number of SAT configuration (valid redistribution of the demand over the generators) is exponential in the system size
^{2} . 
UNSAT phase, type 1: The Bethe entropy is negative, suggesting that there is almost surely no valid redistribution of demand over the generators.
Fig. 2 shows results of the population dynamics simulations, with lines connecting the marks indicating the boundaries of the respective SAT and UNSAT phases. These boundaries are established by fixing and traversing different values of starting from SAT phase moving towards UNSAT phase and catching the value where the UNSAT conditions are first time observed. We observe a significant improvement of the SAT/UNSAT threshold from what Eq. (4) suggests.
Recall that in the case the SAT/UNSAT threshold is described by (4). Similar upper bound can be derived for cases . Then for any given , one can find (with high probability) a place on a very large network where all consumers connected to two generators (which are neighbors trough one of the consumers) have demands near to the maximum value . Then at least of these loads have to be connected to one generator and hence
(16) 
Moreover, to observe an improvement in the condition (4) at one needs to require existence of such configuration of loads that a set of consumers connected to one producer and drawing the minimal amount of electricity does not overload the generator, i.e.
(17) 
To conclude, this study of the average Bethe entropy shows that the network with added ancillary lines is able to withstand larger fluctuations in the demand, , than the naive network, in which every consumer has a predesigned provider independently of the current demand. This effect is amplified with increasing .
At this point it is also appropriate to recall that the SATUNSAT transition is actually an abrupt transition only in the sense of the asymptotic limit. Thus for large finite the generator failure probability is small but finite at any point of the SAT domain.
V Control Algorithms
In this Section we discuss the problem of calculating a valid loadtogenerators assignment for a given graph and given configuration of loads. We designed two heuristic methods to identify SAT configuration of switches. Our first algorithm, coined WalkGrid, is an adaptation of the WalkSAT Selman et al. (1994); Ardelius and Aurell (2006), which is a stochastic local search heuristic solver for the Ksatisfiability problem. The WalkGrid algorithm is very fast and shows flawless performance in discovering a valid configuration almost anywhere in the SAT region.
Our second algorithm corresponds to solving BP equations. The BP scheme, designed in the spirit of Mezard et al. (2002), is used to find most biased/stressed link and then proceeds with decimation towards a valid configuration Mezard et al. (2002). Our implementation of the BPdecimation is so far slower and a bit less efficient than performance shown by the WalkGrid algorithm. To this point, let us note that there exists a more efficient way of using BP to find valid configurations  the reinforcement strategy Chavas et al. (2005); Braunstein and Zecchina (2006), which is fully distributed, linear in the number of consumers and typically outperforms decimation. The reinforcement strategy has been implemented in Altarelli et al. (2009) for a related online advertising problem. However, we had a difficulty to find an implementation of the reinforcement which would work efficiently in our problem.
v.1 WalkGrid
WalkGrid is a stochastic local search algorithm inspired and closely related to its KSAT ancestor called WalkSAT Selman et al. (1994); Ardelius and Aurell (2006). It can also be viewed as a MonteCarlolike algorithm which in order to gain speed violates the detailed balance condition (not needed here as we are not interested in sampling, but are rather focused on a local search). Our implementation works as follows {codebox} \Procname \liAssign each value of 0 or 1 randomly (but such that ); \li\RepeatPick a random power generator which shows an overload, and denote the value of the overload, ; \liChoose a random consumer connected to the generator , i.e. ; \liPick an arbitrary other generator which is not overloaded and consider switching connection from to . \li\If(in the result of this switch is relieved from being overloaded \liand either remains under the allowed load or it is overloaded but by the amount less than ) \liAccept the move, i.e. disconnect from and connect it to thus setting , . \li\ElseWith probability connect consumer to instead of ; \li\UntilSolution found or number of iterations exceeds .
The WalkGrid algorithm depends on two parameters: the maximum number of iterations , and the temperaturelike (greediness) parameter . The parameter needs to be optimized, just as in the original WalkSAT solver.
Fig. 3 shows performance of the WalkGrid algorithm. The running time of this algorithm scales close to linear with the system size, it is thus relatively easy to resolve fast network with many thousands of nodes. Whereas for , , the separated architecture requires , the WalkGrid algorithm is able to find valid configurations up to , while our theoretical analysis suggests that valid configurations should exist up to .
v.2 BeliefPropagation decimation
In the BPbased decimation algorithm one updates Eqs. (710) iteratively, thus passing messages from generators to consumers and back. After fixed number of steps the most biased consumer is chosen and the more probable value for its consumption is assigned, the graph is reduced and the procedure is repeated. Note that updating Eqs. (78) takes steps per message thus making the algorithm exponential in . However, building new connections is expensive and one should realistically assume that actual is any case not very large. As far as the scaling in goes, the algorithm is quadratic in the number of consumers.
\li\RepeatUpdate BP messages on every edge according to (710) times. \liCompute the marginals (11). \liChoose the most biased edge and assign is the more probable values; \liSimplify the formula, cutting off the assigned edge from the graph; \li\UntilSolution or contradiction is found;
The algorithm performance is illustrated in Fig. 5 where the percentage of success in the BPbased decimation is shown. These data average over random instances from the , ensemble with variance in consumption and varying mean consumption. For such a set of parameters the separated network would be able to achieve , whereas with two consumers per producer connected to two producers the mean consumption increases to .
Vi Summary and Path Forward
This manuscript reports a first study of the power distribution networks with transition from the SAT regime, in which shedding of loads is avoidable, to the UNSAT regime, where shedding is the only available option for balancing the demand. We have shown that a significant enlargement of the SATdomain is possible employing ancillary connections between power consumers and generators. Even though our model represents a gross oversimplification over the actual power grid, it offers a significant step forward in providing a framework and guidance for analysis of more involved and realistic problems. The general approach we pursued in this study is based on recent developments in the field of graphical models that merges statistical physics, computer science, optimization theory and information theory Richardson and Urbanke (2008); Mezard and Montanari (2009). The BP approach is asymptotically exact on infinite sparse graphs and as such is useful for the asymptotic (capacity/phasetransition style) analysis. The BP scheme also provides heuristic tools for graphical models on finite sparse graphs that can be used for algorithmic optimization and control of the power grid. We tested this BP approach, and also developed in parallel another an apparently more efficient alternative to BP called WalkGrid. This algorithm finds valid configuration of switching practically anywhere inside the SAT phase. Note, that the two algorithms, BP and WalkGrid, are truly complementary and there utility for practical problems in power networks are yet to be explored.
It is important to emphasize that many generalizations of our model are very straightforward and can be used directly within the framework presented here. This includes implementing different probability distributions for demands (as long as the distribution support is bounded), and nonuniformity for both generator and consumer levels, i.e. varying production caps for generators and introducing distinct distributions of demands for different consumers. Also the network itself can be easily extended in many ways from the simplified case, parameterized by and , that we discussed in the manuscript. Our equations are straightforwardly valid for every bipartite locally treelike random network. Another case, which allows very natural and straightforward generalization for all the statements made in this manuscript, corresponds to breaking the equivalence between different edges in the graph and thus assigning nonuniform weights to them. These weights may e.g. represent cost of construction, geographical length, proxy for losses, cost of exploitation, etc.
There are many other more realistic extensions of our model associated with the description, optimization and control of power grids which can benefit from utilizing a graphical model approach of the kind discussed in this paper, even though actual implementation may prove to be more involved. We conclude listing some of these more interesting but difficult problems that we plan to address in the future, based on the general method sketched in this manuscript:

Generators in a real grid are interconnected on higher (still power distribution, but also power generation) levels. In combination with item [A], this represents a major challenge for extending our approach. However, we still believe that posing the joint optimization and control problem in terms of a complex graphical model and then addressing capacity/transition as well as algorithmic issues with the host of BPrelated techniques is a feasible and exciting path forward.

The joint optimization setting, mentioned in item [B], may also include various additional factors associated with economic policies (e.g. prices, incentives, etc) Baldick and Hogan (2001); Walawalkar et al. (2008), government regulations and load/generation forecasting. These complications can be accounted for in the form of extra soft or hard constraints. This united framework should also take advantage of the progress made in developing optimal power flow solutions Huneault and Galiana (1991); Pandya and Joshi (2008).

Our consideration in this paper was purely static, thus ignoring important transients. Constructing dynamical (discrete and continuous time) models that also accounts for all the aforementioned problems, is an important future task. It is also important to notice that the graphical model approach can in fact be extended to the dynamic framework, see e.g. Hutter et al. (2004), thus suggesting yet another intriguing future opportunity.

An essential part of statistical studies of the power grid focuses on estimating probabilities and mitigating very costly and dangerous large scale outages Dobson et al. (2007); Hines and Talukdar (2008). The graphical model framework this manuscript describes allows an extension which can analyze rare events of special interest, such as dangerous but rare configurations of the channel noise and degrading performance of LowDensityParityCheck codes Chernyak et al. (2004); Chertkov and Stepanov (2008). We plan to extend this approach to the analysis of outages and their cascading through the grid.
Vii Acknowledgments
We thank Florent Krzakala, David Gamarnik and Scott Backhaus for very stimulating and critical comments, and all members of LANL study group on ”Optimization and Control Theory for Smart Grids” for many discussions. The work at LANL was carried out under the auspices of the National Nuclear Security Administration of the U.S. Department of Energy at Los Alamos National Laboratory under Contract No. DEAC5206NA25396.
Footnotes
 This assumption is not crucial and can be easily removed, allowing consumers not only to consume but also to generate electricity. The inclusion of this capability potentially has a far reaching consequences because it allows distributed generation. We postpone our discussion of this interesting possibility to future publications.
 This is the case unless the so called replica symmetry breaking takes place in our model. We have done a local stability check of the BP solution and have not seen any indication for a break down of the replica symmetry. Note also that this stability is also equivalent to convergence of the underlying BP algorithm for an individual realization of the graph and the loads.
References
 The smart grid: An introduction (2008), eprint http://www.oe.energy.gov/1165.htm.
 Smart grid at wikipedia, eprint http://en.wikipedia.org/wiki/Smart_grid.
 S. Massoud Amin and B. Wollenberg, Power and Energy Magazine, IEEE 3, 34 (2005).
 M. Ilic, Proceedings of the IEEE 95, 1060 (2007).
 S. Amin, Power and Energy Magazine, IEEE 6, 48 (2008).
 C. Shannon, The Bell System Technical Journal 27, 379423, 623656 (1948).
 A. J. Wood and B. F. Wollenberg, Power Generation Operation And Control (New York, 1996), 2nd ed.
 T. H. Cormen, C. E. Leiserson, and R. L. Rivest, Introduction to algorithms (MIT Press ; New York : McGrawHill, 1990).
 R. Baldick and W. Hogan, in University of California Energy Institute (2001).
 M. Mézard and G. Parisi, J. Physique 46, L771 (1985).
 M. Mézard and G. Parisi, Eur. Phys. J. B 20, 217 (2001).
 M. Mezard, G. Parisi, and R. Zecchina, Science 297, 812 (2002), eprint http://www.sciencemag.org/cgi/reprint/297/5582/812.pdf.
 T. Richardson and R. Urbanke, Modern Coding Theory (Cambridge University Press, 2008).
 M. Mezard and A. Montanari, Information, Physics, and Computation (Oxford University Press, 2009).
 H. Bethe, Proceedings of Royal Society of London A 150, 552 (1935).
 H. Peierls, Proceedings of Cambridge Philosophical Society 32, 477 (1936).
 R. Gallager, Low density parity check codes (MIT PressCambridhe, MA, 1963).
 J. Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference (Morgan Kaufmann Publishers, Inc., 1988).
 B. Lehmann, D. Lehmann, and N. Nisan, in In ACM conference on electronic commerce (2001), pp. 18–28.
 N. Andelman and Y. Mansour, in In 9th Scandinavian Workshop on Algorithm Theory (SWAT (2004), pp. 26–38.
 Y. Azar, B. Birnbaum, A. Karlin, C. Mathieu, and C. Nguyen, in Lecture Notes in Computer Science 5125 (Springer, 2008), p. 186.
 F. Altarelli, A. Braunstein, J. RealpeGomez, and R. Zecchina, J. Stat. Mech. p. P07002 (2009).
 K. Talwar and U. Wieder, in STOC ’07: Proceedings of the thirtyninth annual ACM symposium on Theory of computing (ACM, New York, NY, USA, 2007), pp. 256–265, ISBN 9781595936318.
 P. Berenbrink, T. Friedetzky, Z. Hu, and R. Martin, Theor. Comput. Sci. 409, 511 (2008), ISSN 03043975.
 P. Sanders, S. Egner, and J. Korst, in SODA ’00: Proceedings of the eleventh annual ACMSIAM symposium on Discrete algorithms (Society for Industrial and Applied Mathematics, Philadelphia, PA, USA, 2000), pp. 849–858, ISBN 0898714532.
 J. Yedidia, W. Freeman, and Y. Weiss, Generalized belief propagation (Cambridge, MA, MIT Press, 2001), vol. 13, pp. 689–695.
 L. Zdeborová, Acta Physica Slovaca 59, 169 (2009).
 B. Selman, H. A. Kautz, and B. Cohen, in Proc. 12th AAAI (AAAI Press, Menlo Park, CA, USA, 1994), pp. 337–343.
 J. Ardelius and E. Aurell, Phys. Rev. E 74, 037702 (2006).
 J. Chavas, C. Furtlehner, M. Mézard, and R. Zecchina, J. Stat. Mech. p. P11016 (2005).
 A. Braunstein and R. Zecchina, Physical Review Letters 96, 030201 (2006).
 B. Bollobas, Modern Graph Theory (Springer, 1998).
 R. Walawalkar, S. Blumsack, J. Apt, and S. Fernands, Energy Policy 36, 3692 (2008), ISSN 03014215.
 M. Huneault and F. Galiana, Power Systems, IEEE Transactions on 6, 762 (1991).
 K. Pandya and S. Joshi, Journal of Theoretical and Applied Technology pp. 450–457 (2008).
 F. Hutter, B. Ng, and R. Dearden, Incremental thin junction trees for dynamic bayesian networks (2004), eprint http://www.eecs.harvard.edu/b̃mng/papers/tud_tr04itjt.pdf.
 I. Dobson, B. A. Carreras, V. E. Lynch, and D. E. Newman, Chaos 17, 026103 (2007).
 P. Hines and S. Talukdar, in Proc. of the International Conference on Infrastructure Systems, Rotterdam (2008), eprint http://www.cems.uvm.edu/p̃hines/pdfs/2008/Hines_2008_nginfras.pdf.
 V. Chernyak, M. Chertkov, M. G. Stepanov, and B. Vasic, Phys. Rev. Lett. 93, 198702 (2004).
 M. Chertkov and M. Stepanov, Information Theory, IEEE Transactions on 54, 1514 (2008).