A universal data based method for reconstructing complex networks with binary-state dynamics

A universal data based method for reconstructing complex networks with binary-state dynamics

Jingwen Li School of Systems Science, Beijing Normal University, Beijing, 100875, China    Zhesi Shen School of Systems Science, Beijing Normal University, Beijing, 100875, China    Wen-Xu Wang wenxuwang@bnu.edu.cn School of Systems Science, Beijing Normal University, Beijing, 100875, China Business School, University of Shanghai for Science and Technology, Shanghai 200093, China    Celso Grebogi Institute for Complex Systems and Mathematical Biology, King’s College, University of Aberdeen, Aberdeen AB24 3UE, UK    Ying-Cheng Lai Institute for Complex Systems and Mathematical Biology, King’s College, University of Aberdeen, Aberdeen AB24 3UE, UK School of Electrical, Computer and Energy Engineering, Arizona State University, Tempe, Arizona 85287, USA Department of Physics, Arizona State University, Tempe, Arizona 85287, USA
Abstract

To understand, predict, and control complex networked systems, a prerequisite is to reconstruct the network structure from observable data. Despite recent progress in network reconstruction, binary-state dynamics that are ubiquitous in nature, technology and society still present an outstanding challenge in this field. Here we offer a framework for reconstructing complex networks with binary-state dynamics by developing a universal data-based linearization approach that is applicable to systems with linear, nonlinear, discontinuous, or stochastic dynamics governed by monotonous functions. The linearization procedure enables us to convert the network reconstruction into a sparse signal reconstruction problem that can be resolved through convex optimization. We demonstrate generally high reconstruction accuracy for a number of complex networks associated with distinct binary-state dynamics from using binary data contaminated by noise and missing data. Our framework is completely data driven, efficient and robust, and does not require any a priori knowledge about the detailed dynamical process on the network. The framework represents a general paradigm for reconstructing, understanding, and exploiting complex networked systems with binary-state dynamics.

I Introduction

Complex networked systems consisting of units with binary-state dynamics are common in nature, technology, and society barrat2008 (). In such a system, each unit can be in one of the two possible states, e.g., being active or inactive in neuronal and gene regulatory networks kumar2010 (), cooperation or defection in networks hosting evolutionary game dynamics game (), being susceptible or infected in epidemic spreading on social and technological networks pastor2015 (), two competing opinions in social communities shao2009 (), etc. The interactions among the units are complex and a state change can be triggered either deterministically (e.g., depending on the states of their neighbors) or randomly. Indeed, deterministic and stochastic state changes can account for a variety of emergent phenomena, such as the outbreak of epidemic spreading granell2013 (), cooperation among selfish individuals santos2005 (), oscillations in biological systems koseska2013 (), power blackout buldyrev2010 (), financial crisis galbiati2013 (), and phase transitions in natural systems balcan2011 (). A variety of models have been introduced to gain insights into binary-state dynamics on complex networks newman2010 (), such as the voter models for competition of two opinions voter (), stochastic propagation models for epidemic spreading sis (), models of rumor diffusion and adoption of new technologies castellano2009 (), cascading failure models bashan2013 (), Ising spin models for ferromagnetic phase transition ising (), and evolutionary games for cooperation and altruism santos2008 (). A general theoretical approach to dealing with networks hosting binary state dynamics was developed recently gleeson2013 () based on pair approximation and master equations, providing a good understanding of the effect of the network structure on the emergent phenomena.

In this paper, we address the inverse problem of binary-state dynamics on complex networks, i.e., the problem of reconstructing the network structure and binary dynamics from data. Deciphering the network structure from data has always been a fundamental problem in complexity science, as the structure can determine the type of collective dynamics on the network boccaletti2006 (). More generally, for a complex networked system, reductionism is not effective and it is necessary to reconstruct and study the system as a whole barabasi2011 (). The importance of network reconstruction has been increasingly recognized and effective methodologies have been developed GdiBLC:2003 (); friedman2004 (); Timme:2007 (); CMN:2008 (); guo2008 (); RWLL:2010 (); HKKN:2011 (); WLGY:2011 (); barzel2013 (); feizi2013 (); CCPGP:2013 (); SWFDL:2014 (); han2015 (). Of particular relevance to our work is spreading dynamics on complex networks, where the available data are binary: a node is either infected or healthy. In such cases, a recent work SWFDL:2014 () demonstrated that the propagation network structure can be reconstructed and the sources of spreading can be detected by exploiting compressive sensing CRT:2006a (); CRT:2006b (); Donoho:2006 (); Baraniuk:2007 (); CW:2008 (); Romberg:2008 (). However, for binary state network dynamics, a general reconstruction framework was lacking (prior to the present work). The problem of reconstructing complex networks with binary-state dynamics is extremely challenging, for the following reasons. (i) The switching probability of a node depends on the states of its neighbors according to a variety of functions for different systems, which can be linear, nonlinear, piecewise, or stochastic. If the function that governs the switching probability is unknown, a tremendous difficulty would arise in obtaining a solution of the reconstruction problem. (ii) Structural information is often hidden in the binary states of the nodes in an unknown manner and the dimension of the solution space can be extremely high, rendering impractical (computationally prohibitive) brute-force enumeration of all possible network configurations. (iii) The presence of measurement noise, missing data, and stochastic effects in the switching probability make the reconstruction task even more challenging, calling for the development of effective methods that are robust against internal and external random effects.

To meet the challenges, we develop a general and robust framework for reconstructing complex networks based solely on the binary states of the nodes without any knowledge about the switching functions. Our idea is centered around developing a general method to linearize the switching functions from binary data. The data-based linearization method is applicable to linear, nonlinear, piecewise, or stochastic switching functions. The method allows us to convert the network reconstruction problem into a sparse signal reconstruction problem for local structures associated with each node. Exploiting the natural sparsity of complex networks, we employ the lasso lasso (), an L constrained fitting method for statistics and data mining, to identify the neighbors of each node in the network from sparse binary data contaminated by noise. We establish the underlying mechanism that justifies the linearization procedure by conducting tests using a number of linear, nonlinear and piecewise binary-state dynamics on a large number of model and real complex networks. We find universally high reconstruction accuracy even for small data amount with noise. Because of its high accuracy, efficiency and robustness against noise and missing data, our framework is promising as a general solution to the inverse problem of network reconstruction from binary-state time series, which is key to articulating effective strategies to control complex networks with binary state dynamics using, e.g., the recently developed network controllability frameworks liu2011 (); nepusz2012 (); yan2012 (); yuan2013 (); RR:2014 (); Wuchty:2014 (). The data-based linearization method is also useful for dealing with general nonlinear systems with a wide range of applications.

Ii Binary-state dynamics

We consider a large number of representative binary state processes on complex networks, which model a plethora of physical, social and biological phenomena gleeson2013 (). In such a dynamical process, the state of a node can be (inactive) or (active). In general, the process can be characterized by two switching functions, and , which determine the probabilities for a node to change its state from to and vice versa, respectively. The variables in these functions, and , are the degree of the node and the number of active neighbors of the node, respectively. The switching functions can be linear, nonlinear, piecewise, bounded and stochastic for characterizing and generating all kinds of binary-sate dynamical processes occurring on complex networks. Despite the difference among the switching functions, the feature that a node’s switching probability depends on its degree and its number of active neighbors is generic. Table 1 lists the switching functions of different models, and the brief descriptions of each model can be found in Appendix.

Model
Voter voter ()
Kirman kirman ()
Ising Glauber ising (); glauber ()
SIS sis ()
Game game ()
Language language ()
Threshold threshold ()
Majority vote majority ()
Table 1: Switching functions for various binary state dynamical processes on complex networks. The function is the probability that a node switches its state from to while represents the probability of the reverse process, where is the degree of the node, is the number of neighbors of this node in the active state . The models and the other parameters are described in Methods. The parameter values used in the simulations are listed in Supplementary Table S1.

Iii Reconstruction method

Our goal is to articulate a general framework to reconstruct the network structure from binary states of nodes without knowing a priori the specific switching functions. A key step is to develop a universal procedure to obtain the linearization of the switching functions from binary data. We demonstrate that this can be accomplished by taking advantage of certain common features of the binary state dynamics.

Figure 1: Schematic illustration of data based linearization from a merging process. (a) The original binary-state time series, where the dark blue squares denote the state and the white squares denote the state. The variable consists of for all . Only strings with (highlighted by the green frames) contain useful information for reconstruction. We identify the time steps with and use . (b) Method of choosing bases. We first construct a network where the vertices denote strings of with (green squares) and the edges are weighted by the normalized Hamming distance between the strings. We then eliminate the edges with weight smaller than the threshold . Setting another threshold , we obtain the top vertices with large degrees (yellow squares) and remove the other vertices together with their edges. Finally, we pick out the vertices with smaller degree (red squares) according to the number of base strings needed for reconstruction. (c) Selection of subordinate strings from a base. We take as a base and calculate between and other strings so as to sort out the time steps satisfying in this set. (d) Establishing average node states. We calculate the average value to represent the state of the data set subject to the base, and the average value to linearize the switching probability [Eqs. (4)]. The average values are shown in blue. Similarly, we obtain a sequence of and the associated average values for reconstructing network structure by employing the lasso to solve (see Methods for details).

iii.1 Data based linearization of switching functions

To proceed, we note that the number of active neighbors at time can be expressed as

(1)

where if nodes and are connected and otherwise, and denotes the state of node at time step . In general, the switching probability for node to change its state from to at time step can be written as

(2)

where is a monotonous function characterizing different dynamical models, e.g., those listed in Table 1. In Eq. (2), all the matrix elements () that are to be inferred from data characterize the network structure. In general this is a difficult problem, because in Eq. (2), only nodal state is measurable, whereas neither of the quantities and nor the form of is known. In fact, not knowing the function is the main difficulty in reconstructing the adjacency matrix . To overcome this difficulty, we propose a merging process to linearize , i.e.,

(3)

where and are constants associated with node . Insofar as the linearization is realized, we can solve . The idea of linearization is first proposed and used in Ref. SWFDL:2014 (), but the mathematical form of is assumed to be known in that case. It is worth noting that the linearization approach is highly nontrivial and is fundamentally different from that in the standard canonical nonlinear analysis because, in our case, the mathematical form of is not available, which can be a nonlinear, discrete and piecewise function. The fully data based linearization procedure is the main contribution of this paper.

iii.2 Procedure of dealing with binary-state data

We present the procedure of dealing with binary-state data. The merging based linearization process enables the probability to be estimated according to the law of large numbers, from which the solution of can be obtained. In particular, as shown in Fig. 1(a), for an arbitrary node , we first identify all the time steps with as information about the switching probability is contained only in the flipping behavior from state 0. Then we propose a method to choose the optimal base strings that are neither too special nor too similar to each other(see Fig. 1(b)). A base string at is a state vector based on which a set of similar strings are identified and averaged to estimate the swiching probability. Specifically, we first construct a network whose vertices represent strings composed of at different time steps for and the edges are weighted by the normalized pairwise Hamming distances among the strings. We then eliminate edges whose weight is smaller than a threshold, say . Setting another threshold , we can extract a subnetwork where only the top vertices of large degree are preserved, while other vertices and their edges are removed. Finally, we pick out vertices with smallest degrees ensuring that the selected base strings are sufficiently different, where is the number of equations in Eq. (16) For each chosen base string, we set a threshold in the normalized Hamming distance between strings to select a set of subordinate strings that belong to each base string, as shown in Fig. 1(c). A subordinate string is a string whose normalized Hamming distance to the base string is less than the selected threshold . Using the average of to represent the state of node and the average of to estimate the switching probability of node according to the law of large numbers, we obtain , where denotes the time of the base string (see Fig. 1(d)).

The whole process leads to the linearization of with the following data-based relationship

(4)

where is the average over all time of the subordinate strings within . The constant parameter is incorporated into the linear coefficient and the intercept . It is not necessary to estimate the quantities , and in Eq. (4) separately - it is only necessary to infer value of the product . In particular, if and are not connected, we have , but a nonzero value of means that there is a link between the two nodes. As we will show, the value of can be obtained but this quantity plays little role in the reconstruction.

Figure 2 shows some representative examples to validate the linearization procedure. Four types of dynamics, including two with continuous and nonlinear switching functions and two with discontinuous and piecewise functions, are tested. We see that the switching functions for different parameter values are linearized, enabling the network structure in the linearized system (4) to be reconstructed by distinguishing between zero and nonzero values of the reconstructed product . As compared to the original function , the range of in the linearized function typically shrinks considerably as a result of the merging process, as shown in Figs. 2(a) and 2(b). For the discrete piecewise functions in Figs. 2(c) and 2(d), approximately linear functions arise for different parameter values. This is particularly striking, because even given a switching function, it is still difficult to linearize a piecewise function. We have achieve a data-based linearization of nonlinear and piecewise functions without any knowledge a priori.

Figure 2: Data based linearization procedure for nonlinear and piecewise binary-state dynamics. Linearization of the switching probability function for (a) Ising model, (b) evolutionary game model, (c) threshold model, and (d) majority model. The grey lines represent Eq. (2) with the function from the different models, where is the node’s degree and is the number of active neighbors. Data points are the result of the linearization procedure from time series, which corresponds to Eq. (4). For the linearized function, is obtained from and the value of the function is obtained via . For the data points, each color represents a set of subordinate strings whose base string has active neighbors. The colors demonstrate that bases with different values are needed to produce a linear function with a sufficient range of for reconstruction, which justifies the base selection based on the normalized Hamming distance in Fig. 1. For both nonlinear and piecewise switching functions, a linearized function in the form of Eq. (4) can be generated based entirely on data, which is the key to reconstruction. The data points are obtained from an ER random network of nodes and average degree .

iii.3 Theoretical validation of data-based linearization

We provide an analysis for the completely data-based linearization that gives rise to the general relationship (Eq. (4)) from general binary-state dynamics characterized by the switching probability (Eq. (2)),

For nodes with only one neighbor, the linear relationship (4) can be rigorously proved. In this scenario, the number of active neighbors is either or . Let denote the proportion of strings with single active neighbors in the set of base , and denote the proportion of strings with null active neighbors as . Let the switching probability of null active neighbors and single active neighbors be and . Then we have

(5)

and

(6)

Inserting Eq. (6) into Eq. (5), we have

(7)

which is a linear form that is subject to Eq. (4), because both and are constants and they are determined by the specific binary-state dynamics.

Figures 3(a, b) shows two representative examples of reconstructing the local structure of a node with one neighbor for the evolutionary game model and the threshold model. We see explicitly linear relationship for both models. With respect to different number of active neighbors in the original bases, two sets of groups are classified.

Figure 3: Theoretical analysis of the data-based linearization. (a, b)Linearization of switching function for nodes with a single neighbor for the game model (a) and the threshold model (b). The grey solid curves are the original switching functions, data points are the results of data-based linearization (Eq, (4)), and the dashed lines are theoretical predictions from Eq. (7). The color of data points represents two sets of subordinate strings whose base string has no active neighbors () or has a single active neighbors ().For both nonlinear and piecewise switching functions, the theoretical predictions are in exact agreement with data-based linearization, because for the linearization is rigorous without any approximation. (c, d) The distribution of active neighbors in subordinate strings subject to each base string and binomial distributions for reconstructing node with for the game model (c) and for the majority model (d), respectively. Each color of curves represents a set of subordinate strings whose base string has active neighbors. The distribution can be well described by binomial distributions under different success probability in each trial, as exemplified by black curves. There is a good agreement between the distribution of active neighbors in subordinate strings and binomial distributions. (e, f) The original switching function and the linearized function with theoretical prediction based on binomial distribution for the game model (e) and the majority model (f), respectively. The color of data points represents different sets of subordinate strings whose base string has different number of active neighbors (the same meaning as in (c), (d)). The grey curves are the original switching function in the binary-state dynamics. The black dashed lines are the theoretical prediction of the linear relationship through Eq. (15) based on binomial distribution and Taylor linear approximation. The theoretical predictions are in good agreement with numerical results.

For nodes with more than one neighbor, the linear relationship can be justified and predicted based on binomial distribution and Taylor linear approximation. For an arbitrary node, say, node with neighbors, we will substantiate the linear relationship between and resulting from the data-based linearization, where

(8)

and

(9)

where represents the proportion of strings with active neighbors among all strings that belong to the set of base . The key to validating the linear relationship lies in the distribution that obeys.

Regarding the effect of the merging process as shown in Fig. 1, we hypothesize that follows binomial distributions with different success probability . We denote the proportion of state in data to be . If the strings are randomly chosen for each set of a base, exactly obeys binomial distribution with success probability . However, due to the process of selecting strings that are similar to each set of a base, the distribution will be biased toward the number of active neighbors in the base. Despite the original complex influence of the base and string selections based on Hamming distance, their effects can be simply regarded as selecting a group of strings with similar proportion of state since we actually do not know which the node’s neighbors are. This process leads to the success probability that depends on the base string. Figs. 3(c, d) shows the comparison between the actual distribution of obtained from numerical simulations and the binomial distributions with different success probability in each trial in the game and majority model, where the success probability in each trial approximately range from to because in the data. We see that can be well approximated by binomial distributions with different parameter values, which indeed validates our binomial distribution hypothesis.

Based on the binomial distribution hypothesis, we have

(10)

Inserting Eq. (10) into Eq. (8) yields

(11)

The fact that fluctuates around allows us to apply the Taylor series expansion around to Eq. (11), leading to

(12)

Omitting the high-order term , we have

On the other hand, substitute Eq. (10) into Eq. (9) yields

(14)

Combining Eq. (LABEL:eq:Taylor_y) and Eq. (14), we have

(15)

Note that all variables in the first term on the right hand side of Eq. (15) are only determined by the binary-state dynamics and the node degree of . Hence, the first term corresponding to is a constant with respect to node state . In analogy, all variables in the coefficient of the second term are determined by the binary-state dynamics and the node degree of as well, indicating the coefficient is a constant corresponding to in Eq. (4). Taken together, we theoretically justified that Eq. (15) is approximately a linear equation in the form of Eq. (4).

Figures 3(e, f) shows the relationship between and (namely ) of each set of bases and the linear relationship calculated by using Eq. (15) for the game model and the majority model with nonlinear and piecewise switching dynamics. We see that the theoretical predictions are in good agreement with the results from the merging process for linearization, which strongly validates the data-based linearization for general binary-state dynamics.

It is noteworthy that the key to the success of the data-based linearization lies in selecting similar strings subject to a base and the average over each set of bases. The selection of similar strings accounts for the binomial distribution of active neighbors in a set, and different bases induces different success probability in each trial. Then the average of the binomial distributions leads to the relatively small range of compared to the original range in the switching function, allowing us to use Taylor linear approximation. Moreover, high-order terms in the Taylor series expansion contribute little to the binomial distribution, which justifies the low-order approximation. Based on the linear relationship, the reconstruction of local structure can be realized by employing the lasso without requiring the linear coefficients and intercept. In other words, the data-based linearization is general valid for arbitrary binary-state dynamics without any knowledge of the switching function.

iii.4 Reconstruction of local structure based on the lasso

The linear relationaship, Eq. (4) allows us to ascertain the neighbors of any node from different values of the base time, e.g., , and their subordinate times. In particular, with respect to , Eq. (4) can be expressed in the matrix form as Eq. (16) , where the vector is to be solved for obtaining the neighbors of , and the vector and the matrix can be constructed entirely from binary time series without requiring any other information.

(16)

The natural sparsity of complex networks ensures that, on average, the number of neighbors for a node is much smaller than the network size , implying that is typically sparse with most of its elements being zero and the number of nonzero elements is in fact the node degree with . We can then exploit the sparsity to reconstruct by employing the lasso lasso (), a convex optimization method for sparse signal reconstruction. The lasso incorporating an L1-norm and an error control term is efficient and robust, enabling a reliable reconstruction of the local network structure as represented by from a small amount of data. In particular, the problem is to optimize

(17)

where is the norm of assuring the sparsity of the solution, and the least squares term guarantees the robustness of the solution against noise in data. In Eq. (17), is a nonnegative regularization parameter that affects the reconstruction performance in terms of the sparsity of the network, which can be determined by a cross-validation method lasso_python (). An advantage of using the lasso is that , i.e., the number of bases needed, can be much less than the length of . For each base of each node, the strings included can be collected and calculated from only one set of data sample in the time series, ensuring the sparse data requirement.

After the vector has been reconstructed, the direct neighbors of node are simply those associated with nonzero elements in . In the same manner, we can uncover the neighborhoods of all other nodes, so that the full structure of the network can be obtained by matching the neighbors of all nodes.

Iv Reconstruction performance

iv.1 Measurement indices

To quantify the performance of our reconstruction method, we introduce two standard measurement indices, the area under the receiver operating characteristic curve (AUROC) and the area under the precision-recall curve (AUPR). True positive rate (TPR), false positive rate (FPR), Precision and Recall that are used to calculate AUROC and AUPR are defined as follows:

(18)

where is the cutoff in the edge list, is the number of true positives in the top predictions in the edge list, and is the number of positives in the gold standard.

(19)

where is the number of false positive in the top predictions in the edge list, and is the number of negatives in the gold standard.

(20)
(21)

where , which is called sensitivity, is equivalent to . By varying from 0 to , two sequences of points ) and are measured respectively, and the receiver operating characteristic curve and the precision-recall curve are obtained, as shown in Fig. 3(d) and (f). The area under the two curves, denoted as AUROC and AUPR repectively, repensent the reconstruction performance: AUROC(AUPR) ranges from AUC=0.5(AUPR=) for random guessing to AUROC=1(AUPR=1) for perfect reconstructibility.

Because the links of each node are actually identified separately, the AUROC and AUPR are calculated for each node, and we use the mean index values over all the nodes to characterize the reconstruction performance for the whole network.

iv.2 Reconstruction performance affected by network structure and amount of data

We test our method by implementing different dynamical processes on Erdös-Rényi random (ER) erd6s1960 (), scale-free (SF) barabasi1999 (), small-world (SW) watts1998 (), and empirical networks. For network reconstruction, knowledge about the switching dynamics and network details is not necessary - only the states of the nodes at different time steps need to be recorded. See Sec. 1 in Supplementary Materials for computational details.

Figure 4: Reconstruction performance. (a) Reconstructed values of the neighboring vector for all nodes in SW and SF networks with the voter model, where , , and the length of time series used is . The red dashed line represents the threshold for determining whether a reconstructed value is regarded as representing an actual link (a value larger than the threshold) or a null link (a value smaller than the threshold). The correctly reconstructed links (true positive), falsely reconstructed links (false positive), and missing links (false negative) are represented by the dark blue, red and light blue points, respectively, while the yellow points indicate the true negative links. (b, c) Visualization of the reconstructed SW and SF networks, respectively. The color legends of the reconstructed links are the same as those in a. There are more missing links (false negative) in the SF network than in the ER network. (d, e) ROC curves of reconstructed values for SW and SF networks for different values of . (f, g) PR curves of the reconstructed values for SW and SF networks for different values of .
Figure 5: Reconstruction performance with respect to the number of base strings. LassoLasso(a,b,c) AUROC and (d,e,f) AUPR as functions of the normalized number of base strings for the voter, game and majority model on ER, SF and SW networks. The network size and . The length of time series is . Other parameter values of binary-state dynamics are shown in Supplementary Table S1.

Figure 4 illustrates the reconstruction performance, where Fig. 4(a) shows the element values in the reconstructed neighboring vector of all nodes for SW and SF networks with the voter model. We note that the values of corresponding to actual links are markedly and distinctly greater than those of null connections. Setting a cut-off value in the gap between the two groups of points in Fig. 4(a), we can separate the actual links from the null connections, enabling a reconstruction of the whole SW network. For the SF network, it is difficult to fully reconstruct the neighbors of the hub nodes, for the following two reasons: (i) in general the linearization procedure works better for small node degree, as shown in Fig. 2; (ii) the lasso based reconstruction requires smaller data amount and offers better accuracy for sparser vector associated with small degree nodes. However, for an SF network, a vast majority of the nodes in an SF network are not hubs, which can be precisely reconstructed. The reconstructed SW and SF networks are shown in Figs. 4(b) and 4(c), respectively.

AUROC/AUPR Voter Kirman Ising SIS Game Language Threshold Majority
ER / / / / / / / /
SF / / / / / / / /
SW / / / / / / / /
Dolphins / / / / / / / /
Football / / / / / / / /
Karate / / / / / / / /
Leader / / / / / / / /
Polbooks / / / / / / / /
Prison / / / / / / / /
Santa Fe / / / / / / / /
Table 2: AUROC and AUPR measures for various dynamics on a variety of model and empirical networks. The parameter values in the dynamical models are listed in Supplementary Table S1. The size and mean degree of ER, SF and SW networks are and , and the length of time series used is . The length of time series used for empirical networks is .

To assess how the number of base strings affects the reconstruction accuracy, we define to be the number of divided by the network size to quantify the relative amount of the base strings. As shown in Figs. 4(d-g), the receiver operating characteristic (ROC) and the precision-recall (PR) curves show better performance as is increased for both SW and SF networks, implying that high accuracy can be achieved for reasonably large values of . Fig. 5 shows the AUROC and AUPR measures as a function of for different dynamical models on ER, SW and SF networks. Due to the advantage of the lasso for sparse vectors, nearly perfect reconstruction is achieved after exceeds a relatively small critical value, e.g., .

It is also important to assess how the length of the binary time series affects the reconstruction accuracy and efficiency. We have calculated the AUROC and AUPR measures as a function of the normalized time-series length for various dynamical processes on ER, SF and SW networks (see (see Supplementary Sec. 2). In general, high reconstruction accuracy can be achieved for relatively short time series. We systematically test our method on a variety of model and real networks in combination with eight binary-state dynamics (Table 2) and find high values of AUROC and AUPR for all cases.

We explore the effects of network properties such as the average degree and the size on reconstruction performance. As shown in Fig. 6. The reconstruction accuracy decreases as increases. The main reason for this result is that the low-order approximation in the data-based linearization is better for smaller node degree. Moreover, with the increase of , the vector to be reconstructed will become denser. Note that it usually requires larger amounts of data to reconstruct a denser signal by using the lasso according to the compressive sensing theory. Thus, in general a network with larger will be more difficult to be reconstructed. Fig. 7 shows the minimum normalized length of time series to acquire at least AUROC and AUPR simultaneously as a function of network size . We see that decreases as increases, which is because of network sparsity as well. In general, for the same average node degree , a network with larger size will be sparser, leading to a sparser vector . According to the compressive sensing theory, less data are required for reconstructing a sparser , accounting for the decrease of with the increase of . These results indicate that our reconstruction method is scalable and of practical importance for dealing with large real networked systems.

Figure 6: Reconstruction performance affected by average node degree. (a,b,c) AUROC and (d,e,f) AUPR as functions of the average node degree for the voter, game and majority model on ER, SF and SW networks. The network size and normalized length of time series . Other parameter values of binary-state dynamics are shown in Supplementary Table S1.
Figure 7: Reconstruction performance affected by network size. The minimum normalized length to acquire at least AUROC and AUPR simultaneously as a function of network size for the voter, Ising and majority model on (a) ER, (b) SF and (c) SW networks. The mean degree of networks is . Other parameter values of binary-state dynamics are shown in Supplementary Table S1.
AUROC/AUPR Voter Game Majority Voter Game Majority
ER / / / / / /
SF / / / / / /
SW / / / / / /
Table 3: Robustness of reconstruction against noise and missing data. AUROC and AUPR measures for voter, game, and majority models on ER, SF and SW networks for measurement noise and the fraction of inaccessible nodes . The network size is and the mean degree is . The length of the time series used is . Details of the parameter values in the dynamical models are listed in Supplementary Table S1.

iv.3 Robustness of reconstruction against noise and missing data

In real applications, time series are often contaminated by noise and the data from certain nodes may be lost or inaccessible. To address these practical issues, we test the robustness of our method. Specifically, we instill noise into the time series by randomly flipping a fraction of binary states and assume a fraction of nodes are inaccessible. The results are shown in Table 3, where voter, game, and majority models are used as examples of linear, nonlinear and piecewise dynamics, respectively. Strikingly, we obtain high values of AUROC and AUPR even in presence of measurement noise or inaccessible nodes, providing strong evidence for the robustness of our framework against noise and missing data. More detailed characterization associated with the results in Table 3, i.e., AUROC and AUPR as functions of and , are provided in Supplementary Sec. 3.

V Discussion

Reconstructing the topological structure and dynamics of complex systems from data is a central issue in both network science and engineering community CCPGP:2013 (); GdiBLC:2003 (); Timme:2007 (); BL:2007 (); CMN:2008 (); RWLL:2010 (); LP:2011 (); HKKN:2011 (). A framework WYLKG:2011 (); WYLKH:2011 (); WLGY:2011 () of network reconstruction is based on compressive sensing CRT:2006a (); CRT:2006b (); Donoho:2006 (); Baraniuk:2007 (); CW:2008 (); Romberg:2008 (), a sparse signal recovery method developed in applied mathematics and engineering signal processing. A recent work SWFDL:2014 () also demonstrated that compressive sensing can be exploited for network reconstruction in situations where the available time series are polarized (binary), e.g., virus spreading and information diffusion in social and computer networks. While the structure of the virus propagation network and the spreading sources can be obtained, the method is unable to predict the network dynamical systems that generate the binary data.

The contribution of this paper is a general framework to solve the challenging problem of reconstructing complex networks hosting binary-state dynamics, based only on time series without any knowledge of the network structure and the switching functions that generate the binary data. The key to our success is the formulation of a universal data-based linearization method, which is powerful for reconstructing the neighborhood of nodes for any type of nodal dynamics: linear, nonlinear, discontinuous, or stochastic. The natural sparsity of real complex networks allows us to address the local reconstruction as a sparse signal reconstruction problem that can be solved by employing the lasso, a convex optimization method, from small amounts of binary data. The optimization is robust against measurement noise and missing data. Once the neighborhoods of all nodes have been reconstructed, the whole network can be mapped out by assembling all the local structures and making adjustments to ensure consistency. We have validated our framework using a variety of binary-state dynamical models on a number of model and real complex networks. High reconstruction accuracy has been obtained for all cases, even for relatively small amounts of binary data contaminated by noise and when partial data are lost. These results suggest the practical applicability of our framework.

While our framework potentially offers a general, completely data driven approach to reconstructing binary dynamical processes on complex networks, there are still challenges. For example, our framework can deal with various types of switching functions underlying the binary-state dynamics, but in its present form the framework is not applicable to non-monotonous functions or non-Markovian type of dynamics. Especially, when the switching functions are not monotonous, the data-based linearization would fail due to the violation of the one-to-one correspondence between the switching probability and the number of active neighbors. For non-Markovian dynamics, the merging procedure inherent in our method would fail. To predict the interaction strength among nodes presents another challenge, especially where noise is present and there is missing data. The results reported in this paper suggest strongly that our present framework can serve as a starting point to meet the challenges, eventually leading to a complete and universally applicable solution to the inverse problem of binary network structure and dynamics.

References

  • (1) Barrat, A., Barthelemy, M. & Vespignani, A. Dynamical Processes on Complex Networks (Cambridge University Press, 2008).
  • (2) Kumar, A., Rotter, S. & Aertsen, A. Spiking activity propagation in neuronal networks: reconciling different perspectives on neural coding. Nature Rev. Neuro. 11, 615–627 (2010).
  • (3) Szabó, G. & Fath, G. Evolutionary games on graphs. Phys. Rep. 446, 97–216 (2007).
  • (4) Pastor-Satorras, R., Castellano, C., Van Mieghem, P. & Vespignani, A. Epidemic processes in complex networks. Rev. Mod. Phys. 87, 925–979 (2015).
  • (5) Shao, J., Havlin, S. & Stanley, H. E. Dynamic opinion model and invasion percolation. Phys. Rev. Lett. 103, 018701 (2009).
  • (6) Granell, C., Gómez, S. & Arenas, A. Dynamical interplay between awareness and epidemic spreading in multiplex networks. Phys. Rev. Lett. 111, 128701 (2013).
  • (7) Santos, F. C. & Pacheco, J. M. Scale-free networks provide a unifying framework for the emergence of cooperation. Phys. Rev. Lett. 95, 098104 (2005).
  • (8) Koseska, A., Volkov, E. & Kurths, J. Oscillation quenching mechanisms: Amplitude vs. oscillation death. Phys. Rep. 531, 173–199 (2013).
  • (9) Buldyrev, S. V., Parshani, R., Paul, G., Stanley, H. E. & Havlin, S. Catastrophic cascade of failures in interdependent networks. Nature 464, 1025–1028 (2010).
  • (10) Galbiati, M., Delpini, D. & Battiston, S. The power to control. Nature Phys. 9, 126–128 (2013).
  • (11) Balcan, D. & Vespignani, A. Phase transitions in contagion processes mediated by recurrent mobility patterns. Nature Phys. 7, 581–586 (2011).
  • (12) Newman, M. Networks: An Introduction (Oxford University Press, 2010).
  • (13) Sood, V. & Redner, S. Voter model on heterogeneous graphs. Phys. Rev. Lett. 94, 178701 (2005).
  • (14) Pastor-Satorras, R. & Vespignani, A. Epidemic spreading in scale-free networks. Phys. Rev. Lett. 86, 3200 (2001).
  • (15) Castellano, C., Fortunato, S. & Loreto, V. Statistical physics of social dynamics. Rev. Mod. Phys. 81, 591 (2009).
  • (16) Bashan, A., Berezin, Y., Buldyrev, S. V. & Havlin, S. The extreme vulnerability of interdependent spatially embedded networks. Nature Phys. 9, 667–672 (2013).
  • (17) Krapivsky, P. L., Redner, S. & Ben-Naim, E. A Kinetic View of Statistical Physics (Cambridge University Press, 2010).
  • (18) Santos, F. C., Santos, M. D. & Pacheco, J. M. Social diversity promotes the emergence of cooperation in public goods games. Nature 454, 213–216 (2008).
  • (19) Gleeson, J. P. Binary-state dynamics on complex networks: pair approximation and beyond. Phys. Rev. X 3, 021004 (2013).
  • (20) Boccaletti, S., Latora, V., Moreno, Y., Chavez, M. & Hwang, D.-U. Complex networks: Structure and dynamics. Phys. Rep. 424, 175–308 (2006).
  • (21) Barabási, A.-L. The network takeover. Nature Phys. 8, 14 (2011).
  • (22) Gardner, T. S., di Bernardo, D., Lorenz, D. & Collins, J. J. Inferring genetic networks and identifying compound mode of action via expression profiling. Science 301, 102–105 (2003).
  • (23) Friedman, N. Inferring cellular networks using probabilistic graphical models. Science 303, 799–805 (2004).
  • (24) Timme, M. Revealing network connectivity from response dynamics. Phys. Rev. Lett. 98, 224101 (2007).
  • (25) Clauset, A., Moore, C. & Newman, M. E. J. Hierarchical structure and the prediction of missing links in networks. Nature 453, 98–101 (2008).
  • (26) Guo, S., Wu, J., Ding, M. & Feng, J. Uncovering interactions in the frequency domain. PLoS Comput. Biol. 4, e1000087 (2008).
  • (27) Ren, J., Wang, W.-X., Li, B. & Lai, Y.-C. Noise bridges dynamical correlation and topology in coupled oscillator networks. Phys. Rev. Lett. 104, 058701 (2010).
  • (28) Hempel, S., Koseska, A., Kurths, J. & Nikoloski, Z. Inner composition alignment for inferring directed networks from short time series. Phys. Rev. Lett. 107, 054101 (2011).
  • (29) Wang, W.-X., Lai, Y.-C., Grebogi, C. & Ye, J.-P. Network reconstruction based on evolutionary-game data via compressive sensing. Phys. Rev. X 1, 021021 (2011).
  • (30) Barzel, B. & Barabási, A.-L. Network link prediction by global silencing of indirect correlations. Nature Biotechnol. 31, 720–725 (2013).
  • (31) Feizi, S., Marbach, D., Médard, M. & Kellis, M. Network deconvolution as a general method to distinguish direct dependencies in networks. Nature Biotechnol. 31, 726–733 (2013).
  • (32) Caldarelli, G., Chessa, A., Pammolli, F., Gabrielli, A. & Puliga, M. Reconstructing a credit network. Nature Phys. 9, 125–126 (2013).
  • (33) Shen, Z.-S., Wang, W.-X., Fan, Y., Di, Z.-R. & Lai, Y.-C. Reconstructing propagation networks with natural diversity and identifying hidden source. Nature Commun. 5, 4323 (2014).
  • (34) Han, X., Shen, Z., Wang, W.-X. & Di, Z. Robust reconstruction of complex networks from sparse data. Phys. Rev. Lett. 114, 028701 (2015).
  • (35) Candès, E. J., Romberg, J. K. & Tao, T. Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Info. Theo. 52, 489–509 (2006).
  • (36) Candes, E. J., Romberg, J. K. & Tao, T. Stable signal recovery from incomplete and inaccurate measurements. Commun. Pure Appl. Math. 59, 1207–1223 (2006).
  • (37) Donoho, D. L. Compressed sensing. IEEE Trans. Info. Theo. 52, 1289–1306 (2006).
  • (38) Baraniuk, R. G. Compressive sensing. IEEE Sig. Proc. Mag. 24, 118–121 (2007).
  • (39) Candès, E. J. & Wakin, M. B. An introduction to compressive sampling. IEEE Sig. Proc. Mag. 25, 21–30 (2008).
  • (40) Romberg, J. Imaging via compressive sampling. IEEE Sig. Proc. Mag. 25, 14–20 (2008).
  • (41) Hastie, T., Tibshirani, R., Friedman, J. & Franklin, J. The elements of statistical learning: data mining, inference and prediction. Math. Intell. 27, 83–85 (2005).
  • (42) Liu, Y.-Y., Slotine, J.-J. & Barabási, A.-L. Controllability of complex networks. Nature 473, 167–173 (2011).
  • (43) Nepusz, T. & Vicsek, T. Controlling edge dynamics in complex networks. Nature Phys. 8, 568–573 (2012).
  • (44) Yan, G., Ren, J., Lai, Y.-C., Lai, C.-H. & Li, B. Controlling complex networks: How much energy is needed? Phys. Rev. Lett. 108, 218703 (2012).
  • (45) Yuan, Z., Zhao, C., Di, Z., Wang, W.-X. & Lai, Y.-C. Exact controllability of complex networks. Nature Commun. 4 (2013).
  • (46) Ruths, J. & Ruths, D. Control profiles of complex networks. Science 343, 1373–1376 (2014).
  • (47) Wuchty, S. Controllability in protein interaction networks. Proc. Natl. Acad. Sci. (USA) 111, 7156–7160 (2014).
  • (48) Erdös, P. & Rényi, A. On the evolution of random graphs. Publ. Math. Inst. Hungar. Acad. Sci 5, 17–61 (1960).
  • (49) Barabási, A.-L. & Albert, R. Emergence of scaling in random networks. Science 286, 509–512 (1999).
  • (50) Watts, D. J. & Strogatz, S. H. Collective dynamics of ¡®small-world¡¯networks. Nature 393, 440–442 (1998).
  • (51) Davis, J. & Goadrich, M. The relationship between precision-recall and roc curves. In Proceedings of the 23rd international conference on Machine learning, 233–240 (ACM, 2006).
  • (52) Bongard, J. & Lipson, H. Automated reverse engineering of nonlinear dynamical systems. Proc. Natl. Acad. Sci. (USA) 104, 9943–9948 (2007).
  • (53) Levnajić, Z. & Pikovsky, A. Network reconstruction from random phase resetting. Phys. Rev. Lett. 107, 034101 (2011).
  • (54) Wang, W.-X., Yang, R., Lai, Y.-C., Kovanis, V. & Grebogi, C. Predicting catastrophes in nonlinear dynamical systems by compressive sensing. Phys. Rev. Lett. 106, 154101 (2011).
  • (55) Wang, W.-X., Yang, R., Lai, Y.-C., Kovanis, V. & Harrison, M. A. F. Time-series–based prediction of complex oscillator networks via compressive sensing. EPL 94, 48006 (2011).
  • (56) de Oliveira, M. J. Isotropic majority-vote model on a square lattice. J. Stat. Phys. 66, 273–281 (1992).
  • (57) Kirman, A. Ants, rationality, and recruitment. Quart. J. Econo. 137–156 (1993).
  • (58) Abrams, D. M. & Strogatz, S. H. Linguistics: Modelling the dynamics of language death. Nature 424, 900–900 (2003).
  • (59) Granovetter, M. Threshold models of collective behavior. Ame. J. Socio. 1420–1443 (1978).
  • (60) Glauber, R. J. Time-dependent statistics of the ising model. J. Math. Phys. 4, 294–307 (1963).
  • (61) Pedregosa, F. et al. Scikit-learn: Machine learning in python. J. Mach. Learn. Res. 12, 2825–2830 (2011).

Appendix A Description of used Binary-state Dynamics

The voter model voter () assumes that a node randomly chooses and then adopts one of its neighbors’ state at each time step. The total number of neighbors is its degree , of which are active, i.e., they are in state . The probabilities that the node will become active and inactive are and , respectively. In the majority-voter model majority (), a node tends to align with the majority state of its neighbors, and the probability of misalignment is .

In the Kirman’s ant colony model kirman (), a node switches from state to with the probability (with being the number of active neighbors) and the rate of transition from to is , where the parameters and quantify the individual action that is independent of the states of the neighbors and characterizes the action of copying from neighbors’ state.

The Ising model ising () is a classic paradigm to study ferromagnetism at the microscopic level of spins. In the model, a node can assume either one of the two states: spin-up or spin-down. Switching in the state occurs with the probability determined by minimizing the energy (Hamiltonian) of the system. In our study, we chose the transition rates according to the Glauber dynamics glauber (), as shown in Table 1, where the parameter quantifies the combining effect of temperature and the ferromagnetic-interaction parameter.

The SIS model sis () describes the epidemic process of disease spreading with infection and recovery. Each susceptible individual contracts the disease from each of its infected neighbors at the rate , so at each time step a susceptible node with infected neighbors has the probability of remaining susceptible. The infection rate is then . The recovery rate of an infected node is at each time step.

The game model game () originates from the evolutionary game theory. In a network, each node is a player, and the two states means that the player can take on two different strategies. A player plays with each of his/her neighbors using one chosen strategy at each time step. The profit of a rational player , when playing with a neighbor , is characterized by the payoff matrix where and are parameters. Different games can be generated by adjusting and . The payoff of a player is the sum of profit from playing game with all its neighbors. A player switches the strategy with a probability that depends on the payoff it may gain in the next round under the current circumstance by switching its strategy, as illustrated in Table 1, where the parameter qualifies the willingness for an individual to change its strategy according to those of its neighbors, and is associated with the effect of the expected payoff.

For the language model language (), the two states denote two different language choices of a person. Transition from the primary language to the secondary occurs with the probability that is proportional to the fraction of speakers in the neighbors with the power , multiplied by the parameter (or ) according to the respective language.

The threshold model threshold () is a deterministic model, where for each node a certain threshold is set which can be, for example, a function of the node’s degree. At each time step, a node becomes active if the number of its active neighbors exceeds the threshold , and no recovery transformation is permitted.

Acknowledgements

W.-X.W. was supported by NSFC under Grant No. 11105011, CNNSF under Grant No. 61074116 and the Fundamental Research Funds for the Central Universities. Y.-C.L. was supported by ARO under Grant W911NF-14-1-0504.

Author contributions

W.-X.W. designed research; J.L. and Z.S. performed research; all analyzed data; J.L., W.-X.W. and Y.-C.L. wrote the paper; all edited the paper.

Additional information

Competing financial interests: The authors declare no competing financial interests.

Comments 0
Request Comment
You are adding the first comment!
How to quickly get a good reply:
  • Give credit where it’s due by listing out the positive aspects of a paper before getting into which changes should be made.
  • Be specific in your critique, and provide supporting evidence with appropriate references to substantiate general statements.
  • Your comment should inspire ideas to flow and help the author improves the paper.

The better we are at sharing our knowledge with each other, the faster we move forward.
""
The feedback must be of minimum 40 characters and the title a minimum of 5 characters
   
Add comment
Cancel
Loading ...
67455
This is a comment super asjknd jkasnjk adsnkj
Upvote
Downvote
""
The feedback must be of minumum 40 characters
The feedback must be of minumum 40 characters
Submit
Cancel

You are asking your first question!
How to quickly get a good answer:
  • Keep your question short and to the point
  • Check for grammar or spelling errors.
  • Phrase it like a question
Test
Test description