Tackling information asymmetry in networks: a new entropy-based ranking index

# Tackling information asymmetry in networks: a new entropy-based ranking index

## Abstract

Information is a valuable asset for agents in socio-economic systems, a significant part of the information being entailed into the very network of connections between agents. The different interlinkages patterns that agents establish may, in fact, lead to asymmetries in the knowledge of the network structure; since this entails a different ability of quantifying relevant systemic properties (e.g. the risk of financial contagion in a network of liabilities), agents capable of providing a better estimate of (otherwise) unaccessible network properties, ultimately have a competitive advantage. In this paper, we address for the first time the issue of quantifying the information asymmetry arising from the network topology. To this aim, we define a novel index - InfoRank - intended to measure the quality of the information possessed by each node, computing the Shannon entropy of the ensemble conditioned on the node-specific information. Further, we test the performance of our novel ranking procedure in terms of the reconstruction accuracy of the (unaccessible) network structure and show that it outperforms other popular centrality measures in identifying the “most informative” nodes. Finally, we discuss the socio-economic implications of network information asymmetry.

## Introduction

Recognizing the most relevant nodes in a networked system represents a topic of growing interest. This translates into identifying nodes with key features, be they structural or functional. Depending on the system under study, in fact, possessing certain features may translate into accessing a competitive advantage or prominent position in the system. The problem has been tackled by defining a plethora of indices, aiming at quantifying the importance of a node in a given system: the so-called centrality measures [1].

The latter are intended to capture the role played by each node within the network by optimizing an opportunely-defined objective function: examples are provided by the degree-centrality (defined by the number of neighbors of each vertex) [2], the closeness-centrality (defined by the average distance of the reachable nodes from any, given, node) [5], the PageRank-centrality (defined by the number of “authoritative” nodes pointing at the vertex under consideration) [6], etc.

All these centrality measures look at the topological role that nodes have in a network, disregarding the ability of a node to obtain information about the rest of the system. On the contrary, our methodology focuses on the difference in the information contained in the different pattern of interlinkages of each node. We refer to this difference as to network information asymmetry and we will show how it allows nodes to obtain a significantly-better estimation of the (otherwise unaccessible) network properties.

Our novel index measures the reduction of uncertainty over the remaining interlinkages that a node’s own ego-network allows: the node whose accessible information provides the largest uncertainty reduction will be identified as being the “most informative” one. More quantitatively, for each node the uncertainty reduction is computed by comparing the Shannon entropy benchmark value - measurable by all nodes - with the one obtained by also conditioning on the information ego-network on top of it.

Several attempts to define entropy-based indices have been made [8]; however, the measures that have been proposed so far are based on specific definitions of Shannon entropy, an evidence that severely affects their applicability. As we will show in what follows, InfoRank can be understood as a generalization of these measures, applicable to any maximum-entropy ensemble and to any subset of nodes.

The paper is organized as follows. In Section 1, we introduce the general methodology for computing InfoRank in any ensemble of networks, given any set of commonly shared information. Then, we apply this general methodology to the case of the configuration model. In Section 2, we measure InfoRank on a number of real-world networks and verify its correlation with the reconstruction accuracy achieved by each node. Finally, in Section 3 we discuss the role of network information asymmetry in social, economic and financial systems.

## 2Methods

### InfoRank: theoretical foundations

This first subsection is devoted to illustrate the theoretical foundations of our proposed methodology, rooted into information theory. Let us focus on the simplest case of a single node: in order to calculate InfoRank each node (hereafter indexed by ) can be imagined to solve two different problems. The first one reads

with indicating the constrained Shannon entropy and encoding the normalization condition. By solving the constrained-optimization problem above, node finds that

(where ). On the other hand, the second problem node has to solve reads

with being nothing else than the functional in Equation 1 further constrained by imposing the node specific pattern of interconnections (i.e. where runs over all the other nodes of the network). Upon solving the second constrained-optimization problem, the expression

(where ) is found. Both functionals achieve a minimum in their stationary point (consistently, since we are trying to minimize each node - residual - uncertainty). This can be easily proven, upon noticing that the Hessian matrix of both and is the covariance matrix of the constraints and, as such, positive-semidefinite.

In order to find the stationary point of , node must solve the equations , i.e.

for each of the values. In order to numerically evaluate the parameters , let us focus on a specific value, e.g. ; we can then divide into two subsums: one where and one where . Thus, condition Equation 4 can be rewritten as

where the sum runs over the network configurations having (and briefly indicated with the symbol ). Analogously, where

and

the first sum runs over the networks having and the second sum runs over the networks having . Solving the likelihood equation in the case brings to ; solving the likelihood equation in the case brings to . Thus, if one has since the term is suppressed by the coefficient which converges to zero; otherwise, if then since the term is suppressed by the coefficient which converges to zero.

The estimation of the other parameters proceeds in an analogous way, by applying the same line of reasoning to the “surviving” partition functions. In other words, specifying the single patterns of connections means reducing the number of configurations over which the estimation of the constraints is carried out: thus, runs over a smaller number of configurations than .

Let us now evaluate the expressions and for the same value of the parameters (say ): since the number of addenda in is larger than the number of addenda in , it also holds true that and the inequivalence is true as well. Let us now choose a particular value of the parameters, i.e. the point of minimum of : . Thus,

where the second inequality follows from the very definition of minimum. Our ranking procedure builds upon the evidence that, by imposing more information on top of the common one, each node further reduces its uncertainty about the unknown network structure: the one reducing the residual uncertainty to the largest extent is identified as the “most informative” one. The same reasoning and calculations apply for the case of subsets of nodes, only quantifying the InfoRank of all possible subsets of nodes in a network of size would require computing Shannon entropies.

The next subsections will be devoted to illustrate the steps defining our approach, by focusing, for the sake of simplicity, on (the simplest case of) binary, undirected networks.

### Quantifying the benchmark information

Let us suppose the benchmark information, which is accessible to all nodes in the network , to be represented by the degree sequence. The benchmark model is, thus, represented by the usual Configuration Model (CM) [12], defined by the following system of equations:

The informativeness of the degree sequence in explaining the network structure can be quantified by calculating the value of the Shannon entropy defined by the chosen constraints:

with indicating the contribution of node to the benchmark entropy (the subscript stresses the benchmark-like value of this functional, encoding a kind of information which is accessible to all nodes). Intuitively, the closer the value to zero, the larger the explanatory power of the degree sequence with respect to the network structure.

### Quantifying the node-specific information

Let us now focus on a specific node, e.g. . Constraining the information encoded into its specific pattern of connections implies letting node impose

i.e. treating as deterministic the links constituting its ego-network. As an example, let us suppose that node is linked only with nodes 2 and 3 out of the constituting our ideal network, i.e. that and . This implies that the system of equations that node has to solve becomes

where we have explicitly specified the value of the coefficients quantifying the probability of node to establish a connection with every other node (notice that we have omitted the equation controlling for the value of the -th degree, since trivially satisfied). The system above can be rearranged by moving at the left hand side the known entries of the adjacency matrix:

i.e. the zeros and the ones characterizing the (missing) links with node . More explicitly:

where the superscript stresses that the numerical value of the probability coefficients is induced by the specification of node patterns of connections and, in general, . Notice, in fact, that the problem of quantifying the informativeness of the ego-network of each node can be restated by imagining that the node itself is removed from the network, in such a way that a reduced adjacency matrix remains naturally defined, inducing, in turn, a reduced system of equations:

### Calculating the node-specific InfoRank

Once has been removed from the network, the entropy of the “surviving” topological structure can be computed by employing the novel probability coefficients defined by the system of equations in Equation 8, i.e.

Since removing different nodes will, in general, impact on the benchmark entropy differently, a ranking is naturally induced by the amount of “uncertainty reduction” caused by the removal of each node. Since our aim is identifying the node(s) possessing the largest amount of information, in order to define a novel ranking index, let us divide by and take the complement to 1:

as apparent from the definition, the larger the entropy reduction, the higher the rank of the node causing it. In what follows, the index will be referred to as to the InfoRank index.

### Approximating the node-specific InfoRank

Although formally similar, the quantities and , respectively defined in Equation 6 and Equation 9, are conceptually very different and must not be confused. In fact, while just represents the contribution of node to the benchmark entropy , the second index quantifies the residual uncertainty after the removal of node , accounting, in particular, for the effect that removing such a node has on the remaining vertices. Whenever such an effect can be ignored (i.e. when diminishing the nodes degree by one unit doesn’t affect much the magnitude of the surviving probability coefficients), can be indeed approximated by , further implying that .

Remarkably, the term can be simplified as well. For what concerns our analysis, two cases are worth to be mentioned. The first one concerns sparse networks: when assuming the probability coefficients to satisfy the requirement , then , implying that . The second approximation is valid whenever the probability coefficients controlling for the connections of node are well represented by their average value, i.e. ; in this case, one obtains that .

## 3Results

### Ranking nodes in synthetic networks

In order to better illustrate the meaning of the InfoRank index, let us consider two extreme cases, i.e. the removal of either an isolated or a fully-connected node. It is intuitive that, in both cases, the knowledge of the connections of the considered nodes adds no information or, equivalently, that removing these nodes doesn’t lead to any uncertainty reduction. This is readily seen by comparing the systems Equation 5 and Equation 8. In presence of a hub, in fact, the system of equations can be rewritten as (with denoting the hub). Since solving the latter system with respect to is equivalent to solve the former system, removing a hub doesn’t change the information content of the network configuration; analogously, when considering an isolated node. On the other hand, the value characterizes a node whose removal induces a configuration which is perfectly deterministic (i.e. composed by isolated nodes, cliques or both). Naturally, in the very special case of a star graph, the central node is both the hub and the vertex with largest InfoRank value.

A relationship between InfoRank and the node degree, nonetheless, exists. In order to understand it, let us start by considering the quantity , which corresponds to the Shannon entropy of the possible connections of node : the node bringing the largest contribution to , then, is the one maximizing the aforementioned sum, i.e. the one encoding nats into its connections (this unit of measure, also known as “natural bit”, is a consequence of having chosen the base of the logarithm to be the natural one - in base 2, the overall contribution would have been of bits). Each of the addenda can, thus, be imagined to contribute with an average coefficient , further implying that a ranking based on the naïve contribution of each node to would privilege nodes with neighbors.

InfoRank, instead, accounts also for the effect that constraining the pattern of connections of a given node has on the connections of the neighboring ones. Let us consider the synthetic network shown in Figure 1. Upon computing the vector , one finds that the largest contribution to comes from the hub, consistently with the previous discussion (notice, in fact, that ). Let us know imagine, instead, to remove the center of the star: this would cause, in turn, an entire portion of the network to become deterministically determinable (7 nodes would become isolated, in fact). As a consequence, while the contribution would be enriched by an additional amount of (with ), removing the hub would just disconnect two more nodes (by retaining only the main contribution). InfoRank correctly assigns the highest score to the center of the star, pointing it out as the vertex establishing the most informative set of interconnections. Our index, in other words, encodes higher-order corrections to the naïve contribution , by including the “effects” of the additional constraints on the neighboring vertices.

### Ranking nodes in real-world networks

Let us now employ InfoRank to analyse real-world configurations. The core of our analysis will consist in a thorough comparison of a number of alternative ranking indices (in what follows, binary, directed networks will be considered, since one of the chosen indicators becomes trivial in the undirected case): in order to consistently compare the ranking scores output by the selected algorithms, the former ones are normalized in order to let them range within the same interval. More specifically, if we let represent the rank of node according to the chosen algorithm , the applied transformation reads and ensures that nodes with minimum rank are assigned a value (in blue, according to the color scale adopted throughout the paper); viceversa, nodes with maximum rank are assigned a value (in red, according to the color scale adopted throughout the paper).

The first alternative index is represented by the degree-centrality, identifying the nodes characterized by the largest degree as the most important (i.e. central) ones. A first limitation of such an index lies in the nature of the connectivity concept, which lacks an obvious generalization to, e.g. the directed case we are considering in the present paper. In what follows we will adopt the following definition

which ranks the nodes according to the number of their out-neighbors. As evident from the first panel of Figure 2, Figure 3 and Figure 4, the (out-)degree-centrality trivially identifies the hubs as the most central nodes.

The second indicator we have considered is the so-called closeness-centrality [5], defined as

i.e. as the reciprocal of the average topological distance of a vertex from the other, connected ones ( is the number of nodes that can be reached from - following the links direction - and is the topological distance separating from any reachable node ). Intuitively, any two nodes are said to be “close” when their distance is “small”, i.e. few links must be walked to reach one from the other. Naturally, the nodes with are the ones with zero out-degree, while a node with exactly connections will be also the most central one. Generally speaking, however, nodes with small degree do not necessarily have a small closeness-centrality value: an example is provided by the second panel of Figure 3, where nodes behaving like “local hubs” (e.g. at the center of star-like subgraphs) are, in fact, characterized by a large independently from their degree. On the other hand, nodes with a large degree do not necessarily have a large closeness-centrality value: in fact, the first panel of Figure 4 shows that although a large number of nodes can be reached from the hub, many lie at a large distance from it.

Interestingly, as the second panels of Figure 2 and Figure 4 show, the nodes minimizing the (average) topological distance from them are the ones connected to a strongly connected component (SCC - either belonging to it or not). In the case of C. Elegans neural network, its large reciprocity () further levels out the differences between the values of such vertices.

The third indicator considered in the present analysis is the PageRank-centrality [6]. It is computed by solving to iterative equation

which can be imagined to describe a Markov chain: if a walker moves from to with probability ; if such a probability becomes (in a sense, the walker “jumps” from to ). The introduction of the addendum accounting for jumps guarantees the convergence of the formula above to the stationary distribution of this dynamical process (which exists and is unique - its Markov chain, in fact, becomes strongly connected and aperiodic by construction) which also provides the searched ranking scores. In what follows we have set .

By oversimplifying a bit, PageRank scores higher nodes that are pointed either 1) by a large number of nodes which, in turn, have low out-degree (thus becoming “authoritative” nodes) or 2) by authoritative nodes themselves [6]. The evidence that nodes with a large PageRank value do not necessarily coincide with the nodes having a large in-degree is provided by the Little Rock food web: in this particular case, a couple of species predated by a limited number of predators can be, indeed, observed; the former, however, constitute the only preys of the latter. In all the other cases the correlation coefficient between the vectors and is quite large: 0.70 for the US airports network (in 1997), 0.82 for the C. Elegans neural network, for both the World Trade Web and the e-MID interbank network (notice that upon lowering the two vectors become less correlated, since the random contribution to the dynamics becomes the prevalent one). Such a correlation has been also noticed elsewhere [16].

Let us now consider our novel InfoRank index. As a first observation, the ranking induced by it shows a little overlap with the one provided by the other indices, thus confirming its degree of novelty. The intuitive idea according to which the nodes with largest InfoRank are the ones disconnecting the largest number of subgraphs is confirmed upon looking at the fourth panel of Figure 2 and the third panel of Figure 3: when considering either the C. Elegans neural network or the US airport networks, in fact, vertices acting as “junctions” between a group of leaves and the remaining part of the network are often assigned an InfoRank value that is larger than the one assigned to the “most internal” nodes. Naturally, when directed networks are considered, reducing uncertainty does not necessarily imply isolating nodes: it is often enough to exactly determine either their out- or in-degree to gain a notable amount of information.

### Exploring the InfoRank degree-dependence

Let us now consider the World Trade Web (WTW) [17]. The main reason we include it in our analysis is its link density: being much denser than the other networks considered so far, it also allows us to better understand the relationship between InfoRank and the degree sequence(s).

Since the WTW topological structure can be deduced with great accuracy from the knowledge of its degree-sequence(s) [18], we may also expect the latter to be correlated with the ranking indices considered for the present analysis. This is indeed the case. As the fourth panel of Figure 5 shows, both the closeness-centrality and the PageRank are highly correlated with the total degree (i.e. ). The monotonic, increasing, relatioship between total degree and closeness-centrality can be straightforwardly explained by noticing that all countries have established a direct connection with the nodes that can be reached by them via some other (indirect) path. This is not true, for example, for the Little Rock food web shown in Figure 4: in that case, the node with largest out-degree is connected to only some of the nodes constituting a strongly connected sub-component; as a consequence, the overall distance from the set of reachable nodes increases.

The monotonic, increasing, relationship between total degree and PageRank, instead, rests upon a double (empirical) evidence: countries with a large out-degree are 1) also characterized by a large in-degree and are usually 2) “pointed” by countries with a small out-degree.

InfoRank, on the other hand, shows an overall bell-shaped trend with a maximum in correspondence of the values . This means that the nodes providing the largest contribution to the entropy reduction overlap with the ones maximizing . However, as evident upon inspecting Figure 8, this doesn’t mean that the removal of a given node has a small impact on the other vertices; evident deviations from the trend are, in fact, clearly visible: InfoRank adjusts the estimation provided by exclusively accounting for the nodes degrees, although its functional dependence on them is, overall, similar to the one characterizing .

### Exploring the relationship between InfoRank and the reconstruction accuracy

As we have seen, InfoRank individuates the node(s) reducing the network residual uncertainty to the largest extent. We may, thus, suspect InfoRank to also “select” the nodes able to provide the best reconstruction of the network itself. In order to verify our conjecture, we have explicitly tested the agreement between the reconstruction achieved by each node and the observed network structure. In order to do so, we have computed an index often employed to test the (global) goodness of a reconstruction algorithm: the accuracy, defined as where is the expected number of true positives, i.e. , is the expected number of true negatives, i.e. and is the total number of vertices [7]. We have then summarized our findings by calculating the correlation between the vector and the vector .

The results are reported in Table 1: while the correlation between InfoRank and accuracy is almost 1, when comparing the goodness of the reconstruction achieved by nodes ranked via alternative indices a worse agreement is found. In particular, e-MID and the WTW show a negative correlation value: this is due to the bell-shaped trend recovered, e.g. when scattering the accuracy value versus any of the chosen ranking indicators, consistently with the results illustrated in Figure 6.

### Exploring the relationship between InfoRank and systemic risk estimation

In this subsection, we try to give a quantitative illustration of how reconstruction accuracy can lead to better estimation of relevant properties of a financial system, focusing on a real network of transaction in an interbank money market. Interbank money markets are essential for financial institutions for the provision of liquidity. In such markets, information asymmetry [19] on the network of connections can translate in a better estimation of the expected payments, widely recognized as a measure of systemic risk in networks of interbank liabilities [20]. Here, we focus on data from the e-MID (electronic market of interbank deposit) platform, that served a significant percentage () of the unsecured money market in the Euro Area before the 2007-2008 crisis [22]. We apply the clearing mechanism originally proposed in [20], in the generalization discussed in [21], and compute the payment vector, whose components represent the amount a financial institution is able to repay to its creditors. When the payment of a bank is less then its corresponding obligation, that bank is considered insolvent. Hence, computing the payment vector corresponds to identify insolvencies and estimate systemic risk in a financial network. A detailed discussion on such measures of systemic risk is found in [23]. Insolvency of bank occurs when its equity, the difference between assets and liabilities, becomes negative. The external cash flow is given by the external assets , affected by fire sales in case of insolvency, and external liabilities . Both are sampled from a Gaussian distribution, with parameters and , respectively. For our analysis on the role of InfoRank, first, we compute the payment vector - that entails the information on systemic risk losses - starting from the real e-MID network. Secondly, for each node, we compute the payment vector using a sample of networks from its specific ensemble, then evaluate the normalized squared error of each these sampled payment vectors with respect to the real payment vector. Finally, we calculate the mean over the set of sampled payment vectors and obtain the mean squared error that each node is subjected to in computing this systemic risk measure. In Figure 7, we recognize that a larger InfoRank yields a smaller error in the estimation of systemic risk.

## 4Discussion

We introduced a new index of node relevance in networks based on an information-theoretic approach. Differently from other indices, InfoRank can be generalized in several, highly non-trivial, ways. First, its perfectly general derivation allows it to be employed to analyze directed, as well as weighted, networks with any set of commonly shared information. Secondly, it can be extended to quantify the informativeness of whole subsets of nodes: this is usually a major limitation for the other centrality indicators, tailored to provide single nodes estimates. Thirdly, it does not depend on any arbitrary parameter (as PageRank, for example), but only on a maximum entropy principle. We verified the financial consequences of the information-theoretic approach we followed in this paper, by applying our algorithm to the estimation of contagion processes in the e-MID network of interbank loans. To the best of our knowledge, this work is the first to provide a quantitative measure of network information asymmetry leading to competitive advantage for some agents upon others in socio-economic and financial systems.

Finally, the ability of identifying highly informed nodes - characterized by high InfoRank values - may also provide strategies to optimally sample networks, when gathering information on individual nodes is costly, e.g. when surveying a financial system for regulatory purposes.

## Acknowledgements

PB and TS acknowledge support from: FET Project DOLFINS nr. 640772 and FET IP Project MULTIPLEX nr. 317532.

### References

1. Newman M.E.J. Networks: An Introduction. Oxford University Press (2010).
2. Bloch F., Jackson M.O. & Tebaldi P. Centrality Measures in Networks, arXiv:1608.05845 (2017).
3. Borgatti S.P. Centrality and network flow, Social Networks 27, 55-71 (2005).
4. Benzi M. & Klymko C. A matrix analysis of different centrality measures, SIAM Journal on Matrix Analysis and Applications 36, 686–706, doi:10.1137/130950550 (2013).
5. Sabidussi G. The centrality index of a graph, Psychometrika 31, 581–603 (1966).
6. Langville A.N. & Meyer C. Google’s PageRank and Beyond. Princeton University Press (2006).
7. Squartini T., Cimini G., Gabrielli A. & Garlaschelli D. Network reconstruction via density sampling, Applied Network Science 2 (3), doi:10.1007/s41109-017-0021-8 (2017).
8. Zhang Q., Meizhu L., Yuxian D. & Yong D. Local structure entropy of complex networks, arXiv:1412.3910v1 (2014).
9. Bianconi G., Pin P. & Marsili M., Assessing the relevance of node features for network structure, PNAS 106 (28), 11433–11438, doi: 10.1073/pnas.0811511106 (2009).
10. Bianconi, G. The entropy of randomized network ensembles. Europhysics Letters 81 (2), 28005 (2007).
11. Borgatti S.P. Identifying sets of key players in a social network, Computational and Mathematical Organization Theory 12, 21–34, doi:10.1007/s10588-006-7084-x (2006).
12. Park J. & Newman M.E.J. The statistical mechanics of networks, Physical Review E 70, 066117, doi:10.1103/PhysRevE.70.066117 (2004).
13. Oshio K., Iwasaki Y., Morita S., Osana Y., Gomi S., Akiyama E., Omata K., Oka K. & Kawamura K., Tech. Rep. of CCeP, Keio Future 3, (Keio University, 2003).
14. Colizza V., Pastor–Satorras R. & Vespignani A., Reaction–diffusion processes and metapopulation models in heterogeneous networks Nature Physics 3, 276-282 (2007).
15. Martinez N.D. Artifacts or Attributes? Effects of Resolution on the Little Rock Lake Food Web, Ecological Monographs 61 (4), 367-392 (1991).
16. Fortunato S., Boguna M., Flammini A. & Menczer F. Approximating PageRank from In-Degree in Lecture Notes in Computer Science 4936. Springer, Berlin (2008).
17. Gleditsch, K.S. Expanded trade and GDP data, Journal of Conflicts Resolution 46, 712–724 (2002).
18. Squartini T., Fagiolo G., Garlaschelli D. Randomizing world trade. I. A binary network analysis, Physical Review E 84, 046117, doi:10.1103/PhysRevE.84.046117 (2011).
19. Wittenberg-Moerman R., The role of information asymmetry and financial reporting quality in debt trading: Evidence from the secondary loan market,Journal of Accounting and Economics 46 (2), 240-260 (2008).
20. Eisenberg L. & Noe T.H. Systemic risk in financial systems, Management Science 47 (2), 236-249 (2001).
21. Rogers L.C.G. & Veraart L.A.M. Failure and rescue in an interbank network, Management Science 59 (4), 882-898, (2013).
22. Barucca P. & and Lillo F. The organization of the interbank network and how ECB unconventional measures affected the e-MID overnight market, arXiv:1511.08068 (2015).
23. Glasserman P. & Young P.H. Contagion in financial networks, Journal of Economic Literature 54 (3), 779-831 (2016).
24. Barucca P., Bardoscia M., Caccioli F., D’Errico M., Visentin G., Battiston S. & Caldarelli, G. Network valuation in financial systems, arXiv:1606.05164 (2016).
10003