Exact Topology and Parameter Estimation in Distribution Grids with Minimal Observability

Exact Topology and Parameter Estimation in Distribution Grids with Minimal Observability


Limited presence of nodal and line meters in distribution grids hinders their optimal operation and participation in real-time markets. In particular lack of real-time information on the grid topology and infrequently calibrated line parameters (impedances) adversely affect the accuracy of any operational power flow control. This paper suggests a novel algorithm for learning the topology of distribution grid and estimating impedances of the operational lines with minimal observational requirements - it provably reconstructs topology and impedances using voltage and injection measured only at the terminal (end-user) nodes of the distribution grid. All other (intermediate) nodes in the network may be unobserved/hidden. Furthermore no additional input (e.g., number of grid nodes, historical information on injections at hidden nodes) is needed for the learning to succeed. Performance of the algorithm is illustrated in numerical experiments on the IEEE and custom power distribution models.


Distribution networks, Missing data, Power flows, Sample complexity, Topology learning, Impedance estimation \thankstoThis work is supported by the US DOE/OE’s Grid Modernization Laboratory Consortium (GMLC) program.

1 Introduction

Distribution grids enable supply of power to end-users/ loads. With the advent of smart grids, new resources like controllable loads, small-scale/household renewable generators (e.g. solar panels) and storage units (e.g. batteries, electric vehicles) have been introduced in the distribution grid. This paradigm shifting change is turning traditionally passive and largely static distribution grids, that acted solely as power sinks, into dynamic, reconfigurable and active resources for novel controls. Emerging smart grid control technologies, relying on these dynamic and reconfigurable features, include participation in demand response, use of batteries in frequency regulation and inter-household energy settlements/transactions. These new technologies also require much more accurate estimation of the grid characteristics. The radial topology of current operational lines, and their impedances are the most significant of these frequently changing characteristics of the smart distribution grids. However, reliable and real-time estimation of the distribution grid topology and line impedances is impeded by limited observability. Even though Phasor Measurement Unit (PMU) technology has become available recently, it still largely limited to power transmission systems [1]. It is not obvious if a wide-spread (full coverage) use of the PMU technology will ever be economically justified at the distribution level. Moreover, access to underground distribution grid in urban areas (e.g. New York City) is technically challenging. These constraints/limitations justify importance of developing techniques capable of estimating operational topology and line impedances in the situation of infrequent calibration and sparse access.

In spite of the access limitations PMUs, micro-PMUs [2], FNETs [3] have being placed into many distribution grids. In addition, and most importantly for the setting discussed in the manuscript, smart end-user measurement units, such as ones associated with smart house-hold devices and EVs, have been installed. These modern end-user devices have the ability to record and communicate nodal voltages and injections. Inspired by this novel capabilities we analyze in this manuscript the joint problem of topology and impedance estimation using measurements collected from smart meters located only at the end/terminal nodes of the smart distribution grids.

1.1 Prior Work

Topology estimation in the power grid is an active area of research. Researchers have proposed different methods depending on availability and type of measurements. [4] uses cycle basis and maximum likelihood tests to reconstruct topology from line measurements. For measurements of nodal voltages collected at all nodes, [5, 6] uses the signs of inverse covariance of complex voltages to identify the operational lines. In a similar regime, [7, 8] presents greedy schemes based on trends in second moments of voltage magnitudes to identify grid topology. [9, 10] utilizes statistical independence tests, in the setting of a graphical model associated with nodal voltages, to reconstruct topology. A number of data driven and model-free schemes, using signature and regression based methods, were developed to reconstruct topology and line parameters [11, 12, 13]. In the work [14], most closely related to this manuscript, an iterative scheme to estimate topology and line impedances from voltage measurements is proposed.

A common feature of the aforementioned papers is that they all rely on availability of nodal measurements (voltage and/or injection) at all nodes of the grid. Reconstruction in the case with limited voltage observability as discussed in [7, 8, 15], require full information about all line impedances in the network. However, these assumptions can be easily broken due to non-ubiquitous presence of nodal meters and lack of historical and state-estimation information.

In this manuscript, we overcome these impediments and suggest an efficient algorithm which estimates jointly operational topology and impedances of the operationally radial distribution grids in the setting when voltage and injection/consumption measurements at all but end-user nodes are missing/hidden.

1.2 Technical Contribution

We develop a provable method/algorithm for topology estimation in the radial distribution grids from samples of nodal voltage and injection measurements collected from smart meters at the end-user locations. The remaining nodes (including all intermediate nodes) in the grid may be unobserved, i.e., their states are unmeasured and further even the knowledge of their existence and number may be unknown. This represents a realistic scenario where majority of measurements are limited to residential devices installed at the end-user locations. Our reconstruction is model-based. We consider a linearized power flow model [16, 17, 18, 7]. Under this model, we develop an algorithm which can provably learn the impedance distance, i.e. sum of line impedances along the unique operational path connecting any two observed nodes in the radial grid. Once the impedance distance is reconstructed, we utilize the recursive grouping algorithm [19] to learn topology and impedances including impedances between missing nodes. Our algorithm is characterized by computational complexity and sample complexity. We demonstrate performance of the algorithm on some useful toy examples and then present results of experiments based on ac power flows on realistic IEEE test cases. To the best of our knowledge, this is the first work which provides guaranteed topology and impedance reconstruction in the distribution grid where only terminal nodes are observed.

The rest of the manuscript is organized as follows. Section 2 introduces nomenclature and power flow relations in the distribution grid. Our main algorithm is described in Section 3. Analysis of the computational/sample complexity of the algorithm is also presented in Section 3. Numerical experiments on IEEE test cases are presented in Section 4. Finally, Section 5 is reserved for conclusions and discussion of future work.

2 Distribution Grid and Power Flow Model

Figure 1: Illustration of a radial distribution grid. The large red node denotes the substation bus. Leaf nodes that represent households/end-users are observed and colored solid blue, while unobserved and possibly unknown intermediate nodes are denoted by dotted blue circles. The unknown operational lines without impedance information are colored grey.

Radial Structure: The distribution grid is defined over graph , where the set of buses/nodes is denoted by and the set of undirected operational lines/edges is denoted by . The operational grid is assumed to have ‘radial’ operational structure, that is it forms the forest consisting of disjoint trees with roots corresponding to substations. Fig 1 shows a disjoint tree part of a radial grid where a red node represents substation, blue nodes represent leaf nodes and dotted nodes represent internal nodes. Notationally, we use alphabets to represent buses/nodes and stands for a line/edge between nodes and . We denote as a root node (reference bus). We denote as a unique path from node to node from the same (operational) tree.

Power Flow Models: Given a radial distribution grid on a tree structured graph , the grid satisfies the following Kirchhoff’s law of power flow which express the complex power injection at a node via node-voltages and line-impedances as follows:


In this equation, denote impedance of , voltage magnitude, voltage phase, active and reactive power at respectively. The substation/root/reference nodes, maintained at unit/nominal voltage, are assumed known/fixed. Since (1) is non-convex, we simplify the model by making a realistic assumption that the second order terms in (1) is negligible. Under this assumption, we introduce the linearized lossless power flow equation describing the linear coupled power flow (LC-PF) model [14, 7]:


where , and are resistance, reactance of line respectively, i.e., . By considering only deviations from the respective steady state reference values, are modeled as random variables with zero mean (counted from the known reference values).

Linear Coupled Power Flow Models: Note that under assumption that voltage deviations from nominal and power injections deviations from the base case are both small the LC-PF model (2) is equivalent to the LinDistFlow equations introduced in [20, 21, 22]. The LC-PF model (2) can also be stated in the following matrix form [7]


where are respectively vectors of voltage magnitude, voltage phase, active and reactive power at the non-substation buses of the grid. represents the reduced weight Laplacian matrices for where are used edge-weights respectively.1 We remind that are normalized to have zero mean.

3 Topology and Impedance Learning Algorithm

In this section, we introduce our main algorithm for learning topology and impedances. We assume that time-stamped observations of voltage magnitudes, voltage phase, active and reactive injections at the end-nodes are available to the observer. Our algorithm is built on the notion of additive ‘distance’ defined as a distance over the graph which thus satisfies the weighted metric property, . We first estimate the distance, and then utilize the recursive grouping algorithm [19] to learn operational topology of the grid. Before introducing our algorithm, let us make the following assumption about the missing intermediate nodes.

Assumption 1.

All missing intermediate nodes have a degree at least 3.

We note that Assumption 1 is necessary to recover the true topology of the grid. See Assumption 2 in [14] for the details. In addition, we assume that the complex power injections at different nodes are uncorrelated

Assumption 2.


As considered in prior studies [5, 7], the uncorrelated assumption 2 is well-justified over sufficiently short time intervals while considering deviations of injections at end-users. Further, for intermediate nodes that involved in separation of power into downstream lines and without any major nodal usage, leakage or device losses cause the net power injection, and hence may be considered as independent from the rest. Note also that the Assumption 2 does not specify the class of distributions that can model individual node’s power injection. It is applicable when nodal injections are negative (loads), positive (due to local generation) or are a mixture of both. In a future work, we will relax this assumption and discuss learning in the presence of correlated end-user injection profiles that are only uncorrelated to injections at intermediate nodes.

Now, we refer the following key property of the inverse of a reduced weight Laplacian matrix which is necessary to define the grid-based distance metric [7]


See Section 4 in [7] for details. In (4), is the unique path from to , is the resistance of and is the root (reference bus) of the grid. One can also derive a similar formulation for line reactances and .

Under Assumption 2 and using (3) in the case of observed nodes , one derives the following identity


where are quantities that can be computed from measurements at observed nodes and . Notice that ability to estimate the expectations in (5) implies that one can also estimate the value of and for any observed unless . To avoid such pathological situation, we make the following assumption.

Assumption 3.

There exists a constant such that for all node ,

Using the estimated , we can now estimate the resistance distance (effective resistance) between observed nodes as


Note that effective resistance is an additive distance metric between nodes and in the grid. Similarly, one can also estimate the additive reactance distance between observed nodes . Once we estimate for all pairs of observed nodes, we can utilize the recursive grouping algorithm (RG) [19] which directly leads us, under Assumption 1, to consistent topology and impedance estimation of the power grid.


0.15   {subfigure}0.15   {subfigure}0.15   {subfigure}0.15   {subfigure}0.15   {subfigure}0.15

Figure 2:
Figure 3:
Figure 4:
Figure 5:
Figure 6:
Figure 7:
Figure 8: (a) an original topology where skyblue nodes are missing and green nodes are observed ( in Algorithm 1) (b) a partition of generated by sibling groups and its parent in the first iteration of RG where each set in is marked by the dashed line (c) updated after the first iteration of RG and a partition of in the second iteration of RG (d) updated after the second iteration of RG and a partition of in the third iteration of RG (e) a result after the third iteration of RG (f) a recovered topology

3.1 Recursive Grouping Algorithm with Exact Distance

The recursive grouping algorithm (RG) is an algorithm which recovers the true radial topology given any additive distance of observed nodes on the tree where it requires to observe every leaf nodes. Now, we first make the ideal setting assumption that exact values of are known for every pair of observed nodes. We now need to introduce the following lemma [19].

Lemma 1.

For , the following relation holds:

  • for all if and only if is a leaf node and is its parent.

  • for all if and only if are leaf nodes with common parent, i.e., they belong to the same group of siblings.

Using Lemma 1 (i), one can figure out the parent-child relationship for a set of observed nodes . In addition, Lemma 1 (ii) enables us to find the groups/sets of siblings of .

Now we are ready to describe how the RG works. The RG steps are illustrated in Figure 8. The input of RG is a set of observed nodes and the additive distance for all . For example, in Figure 8, green nodes represent . First, RG finds groups of siblings and parents of a node using Lemma 1, also as illustrated in Figure 8. After recovering the parent-child, sibling relationships, it adds edges to all identified parent-child pairs. For siblings without observed parent, RG adds a new node for a potential parent and adds edges to the newly added parents and its children. The procedure is illustrated in Figure 8. Once nodes/edges update is done, RG updates the distance between the newly added parents. For siblings and their newly added parent , the distance is calculated by


for any . Also, for any , RG also computes by


Finally, RG updates the set with (newly added) parents and nodes which have no (established) relations which requires to update the parent-child, sibling relationships at the next iteration of RG. For example, green nodes in Figure 8 is an updated . After updating , RG starts over the whole procedure anew unless , which implies that we can add an edge to remaining vertices or only a single vertex left. Figure 8-8 illustrates advanced iterations of the RG (following the first one). Formal description of RG is given in the Algorithm 1.

1:Input: ,
2:Output: ,
4:while  do
5:      .
6:      Compute for all .
7:      Find a coarsest partition of such that any two nodes in are either leaves and sibling, or a parent and a leaf child.2
8:      for  do
9:            if  then
10:                 .
11:            else if A parent exists then
14:            else
15:                 Add a parent of as follows
19:            end if
20:      end for
21:      Update for using (7), (8).
22:      .
23:end while
24:if  then
26:end if
Algorithm 1 Recursive Grouping Algorithm (

Overall, we propose the following two stage algorithm for topology learning of grids with missing modes:

  • For all pair of observed nodes , calculate and using (5), (6) and second order moments.

  • Recover missing nodes and lines using the recursive grouping algorithm.

for learning the topology and impedance of the grid. The formal statement of the algorithm is presented in Algorithm 2.

1:Input: ,
2:Output: ,
3:for  do
5:end for
6:for  do
9:end for
12:for  do
13:      ,
14:end for
Algorithm 2 Topology/Impedance Learning Algorithm with Missing Nodes

3.2 Recursive Grouping Algorithm with Samples

In the practical scenario, we can only observe the approximated value of calculated from samples rather than the exact value. Given finite number of samples the variance of the distance is nonzero. To account for the variance, we allow some tolerance for finding parent-child and sibling relationships. In addition, we test the relationship of only using nodes which are close enough to both and , i.e., nodes in where satisfies

for some constant . Let us now present rules which guide the relationships of nodes using samples.

  • Set as a parent of if for all .

  • Set as siblings if

One can observe that the newly introduced rules are equivalent to the RG rules with exact except for a tolerance . Update of the distance is done in a similar manner. For and its newly added parent , one sets

where denotes the children set of . Likewise, update of the distance for ,


0.3   {subfigure}0.3   {subfigure}0.3
{subfigure}0.3   {subfigure}0.3   {subfigure}0.3

Figure 9: 10 vertices
Figure 10: 50 vertices
Figure 11: 100 vertices
Figure 12: Experimental results for synthetic grids with 10, 50 and 100 vertices averaged (each case) over 100 random radial grids. Upper graphs show average accuracy of our algorithm while lower graphs represent the impedance error observed within the correctly recovered topologies. ”eps” in the graphs denotes .

3.3 Sample and Computational Complexity

We observe that our algorithm terminates in steps where is the depth of the grid and follows from the calculation of . In addition, our algorithm requires (under some mild assumptions) only samples to correctly recover the operational topology and line impedances. The following theorem makes a formal statement on the sample complexity of RG.

Theorem 1.

Suppose that a radial graph/grid has a constant depth. Under Assumptions 1-3 and assuming the LC-PF model, if line impedances are bounded from below by nonzero value, nodal power injections are sub-Gaussian with constantly bounded sub-Gaussian parameter and the number of samples is greater than for some constant , then Algorithm 2 recovers the true topology and impedances with probability .

Proof of the theorem is omitted as it is analogous to the proof of Theorem 11 in [19]. Note that Algorithm 2 has an extra term in sample complexity compare to Theorem 11 in [19]. This is because are expressed by summations of independent complex power injections in (3). In Assumption 2, we assume the independence of complex power injections in different nodes and achieve (5). However, the variance should be considered in (5) as

which increases the sample complexity.

4 Experiments

In this section, we present experimental results of our algorithm on custom and IEEE models.

Custom Examples: In each of our simulation runs we construct the random radial grid and complex power injections. For the topology of the grid, we generate a random tree with maximum degree 5. The line resistance and reactance are independently sampled from the distribution .3 For complex power injections, we sample these from the independent normal distribution, i.e., . Using the generated complex power injections, we compute nodal voltage magnitudes and phases using LC-PF equation (3). The input of the algorithm is the complex power injections, voltage magnitude and phases of the leaf/end-user nodes.

Under this setting, we run experiments changing number of vertices from 10 to 100, the number of samples from 1000 to 10000, and also changing tolerance . To quantify performance of our algorithm, we record accuracy of the correct recovery and errors in the recovered topology and impedances. For each number of vertices and number of samples, we generate 100 random radial grids to measure the performance. Figure 12 shows the correct recovery ratio and the average error in estimating line impedances. The average error is defined for the grid with correct recovery according to . One observes that our algorithm recover line impedances with small error even in the demanding case of 1000 samples. We also observe that larger results in a higher accuracy for the small number of samples but it becomes less accurate for the large number of samples (compare to ). However, if the threshold is too small (), the algorithm performance decreases for all samples sizes. Note that similar results (thus not shown) are derived when changing variance of the complex power injectiona/consumptions.

Figure 13: Illustration of an IEEE radial distribution grid with 33 leaf/end-user nodes and 22 internal nodes.

IEEE models: For the more realistic experiments we use a IEEE model with 56 nodes [5] where the topology was modified (to be radial) and the internal nodes all to be of degree . The modified grid is illustrated in Figure 13. We generate the complex power injections from the independent normal distribution as in the case of the custom models. From the complex power injections, we obtain the corresponding voltage magnitude and phase by using ac power flow equations in MATPOWER [23]. The input of the algorithm is the complex power injections, voltage magnitude and phases of the leaf/end-user nodes.

Under this setting, we measure performance of our algorithm by varying the input number of samples, the variance of the complex power injection and the threshold value, . In particular, as a way to quatify errors, we count the number of edge difference between the recovered topology and the true topology. We also compare the performance of our algorithm with MATPOWER samples and LC-PF samples generated with the same complex power injections. Figure 14 and Figure 15 show our IEEE model experimental results. In Figure 14, one observes that the algorithm works similarly for both MATPOWER samples and the LC-PF samples. In Figure 15, similarly to what we saw in the custom model experiments, the algorithm performance decreases as the threshold increases.

Figure 14: Error of the algorithm observed as the variance of the complex power injection changes. ”var” denotes the variance of the complex power injections and MP, LC implies that the samples are from MATPOWER and LC-PF respectively.
Figure 15: Error of our algorithm by varying the threshold .

5 Conclusion

Topology learning of the distribution grids in real time from sparse data is critically important for a number of operational/control applications. In this manuscript, we propose a novel algorithm which recovers topology and line impedances by only using measurements at the end-user nodes. In this approach we utilize LC-PF model to approximate the resistance distance (also called effective resistance) between any two observed nodes and apply the recursive grouping algorithm to recover the topology. Computational complexity of the algorithm is and the algorithm guarantees to output the correct topology with only samples (under some mild technical conditions regarding statistics of loads/generation). Furthermore, our experimental results, derived for custom (randomized) and IEEE models, shows that the algorithm performs remarkably well. In the future, we plan to extend our algorithm to the case of correlated injections/consumptions and also attempt to generalize to the case of sparse but loopy operations grids/graphs.


  1. denotes subgraph of induced by .
  2. is a coarsest partition if for any and for any , there exists such that . The coarsest partition in Algorithm 1 represents a collection of sets of siblings and their parent.
  3. is a uniform distribution on an interval .


  1. R. Hoffman, “Practical state estimation for electric distribution networks,” in IEEE PES Power Systems Conference and Exposition.   IEEE, 2006, pp. 510–517.
  2. A. von Meier, D. Culler, A. McEachern, and R. Arghandeh, “Micro-synchrophasors for distribution systems,” Innovative Smart Grid Technologies Conference (ISGT), 2014 IEEE PES, pp. 1–5, 2014.
  3. Z. Zhong, C. Xu, B. J. Billian, L. Zhang, S.-J. S. Tsai, R. W. Conners, V. A. Centeno, A. G. Phadke, and Y. Liu, “Power system frequency monitoring network (fnet) implementation,” Power Systems, IEEE Transactions on, vol. 20, no. 4, pp. 1914–1921, 2005.
  4. R. Sevlian and R. Rajagopal, “Feeder topology identification,” arXiv preprint arXiv:1503.07224, 2015.
  5. S. Bolognani, N. Bof, D. Michelotti, R. Muraro, and L. Schenato, “Identification of power distribution network topology via voltage correlation analysis,” in Decision and Control (CDC), 2013 IEEE 52nd Annual Conference on.   IEEE, 2013, pp. 1659–1664.
  6. D. Deka, M.Chertkov, S. Talukdar, and M. V. Salapaka, “Topology estimation in bulk power grids: Theoretical guarantees and limits,” in accepted in the Bulk Power Systems Dynamics and Control Symposium-IREP, 2017.
  7. D. Deka, M. Chertkov, and S. Backhaus, “Structure learning in power distribution networks,” IEEE Transactions on Control of Network Systems, 2017.
  8. D. Deka, S. Backhaus, and M. Chertkov, “Learning topology of the power distribution grid with and without missing data,” in Control Conference (ECC), 2016 European.   IEEE, 2016, pp. 313–320.
  9. ——, “Estimating distribution grid topologies: A graphical learning based approach,” in Power Systems Computation Conference (PSCC), 2016.   IEEE, 2016, pp. 1–7.
  10. Y. Liao, Y. Weng, G. Liu, and R. Rajagopal, “Urban distribution grid topology estimation via group lasso,” arXiv preprint arXiv:1611.01845, 2016.
  11. G. Cavraro, R. Arghandeh, A. von Meier, and K. Poolla, “Data-driven approach for distribution network topology detection,” arXiv preprint arXiv:1504.00724, 2015.
  12. V. Arya, T. Jayram, S. Pal, and S. Kalyanaraman, “Inferring connectivity model from meter measurements in distribution networks,” in Proceedings of the fourth international conference on Future energy systems.   ACM, 2013, pp. 173–182.
  13. J. Peppanen, J. Grimaldo, M. J. Reno, S. Grijalva, and R. G. Harley, “Increasing distribution system model accuracy with extensive deployment of smart meters,” in PES General Meeting— Conference & Exposition, 2014 IEEE.   IEEE, 2014, pp. 1–5.
  14. D. Deka, S. Backhaus, and M. Chertkov, “Learning topology of distribution grids using only terminal node measurements,” in Smart Grid Communications (SmartGridComm), 2016 IEEE International Conference on.   IEEE, 2016, pp. 205–211.
  15. ——, “Learning topology of distribution grids using only terminal node measurements,” in IEEE Smartgridcomm, 2016.
  16. M. Baran and F. Wu, “Optimal sizing of capacitors placed on a radial distribution system,” Power Delivery, IEEE Transactions on, vol. 4, no. 1, pp. 735–743, Jan 1989.
  17. ——, “Optimal capacitor placement on radial distribution systems,” Power Delivery, IEEE Transactions on, vol. 4, no. 1, pp. 725–734, Jan 1989.
  18. S. Bolognani and S. Zampieri, “On the existence and linear approximation of the power flow solution in power distribution networks,” Power Systems, IEEE Transactions on, vol. 31, no. 1, pp. 163–172, 2016.
  19. M. J. Choi, V. Y. Tan, A. Anandkumar, and A. S. Willsky, “Learning latent tree graphical models,” The Journal of Machine Learning Research, vol. 12, pp. 1771–1812, 2011.
  20. M. E. Baran and F. F. Wu, “Optimal sizing of capacitors placed on a radial distribution system,” IEEE Transactions on power Delivery, vol. 4, no. 1, pp. 735–743, 1989.
  21. ——, “Optimal capacitor placement on radial distribution systems,” IEEE Transactions on power Delivery, vol. 4, no. 1, pp. 725–734, 1989.
  22. ——, “Network reconfiguration in distribution systems for loss reduction and load balancing,” IEEE Transactions on Power delivery, vol. 4, no. 2, pp. 1401–1407, 1989.
  23. R. D. Zimmerman, C. E. Murillo-Sánchez, and R. J. Thomas, “Matpower: Steady-state operations, planning, and analysis tools for power systems research and education,” IEEE Transactions on power systems, vol. 26, no. 1, pp. 12–19, 2011.
Comments 0
Request Comment
You are adding the first comment!
How to quickly get a good reply:
  • Give credit where it’s due by listing out the positive aspects of a paper before getting into which changes should be made.
  • Be specific in your critique, and provide supporting evidence with appropriate references to substantiate general statements.
  • Your comment should inspire ideas to flow and help the author improves the paper.

The better we are at sharing our knowledge with each other, the faster we move forward.
The feedback must be of minimum 40 characters and the title a minimum of 5 characters
Add comment
Loading ...
This is a comment super asjknd jkasnjk adsnkj
The feedback must be of minumum 40 characters
The feedback must be of minumum 40 characters

You are asking your first question!
How to quickly get a good answer:
  • Keep your question short and to the point
  • Check for grammar or spelling errors.
  • Phrase it like a question
Test description