Network Cartography: Seeing the Forest and the Trees

Network Cartography: Seeing the Forest and the Trees

Abstract

Real-world networks are often complex and large with millions of nodes, posing a great challenge for analysts to quickly see the big picture for more productive subsequent analysis. We aim at facilitating exploration of node-attributed networks by creating representations with conciseness, expressiveness, interpretability, and multi-resolution views. We develop such a representation as a map — among the first to explore principled network cartography for general networks. In parallel with common maps, ours has landmarks, which aggregate nodes homogeneous in their traits and interactions with nodes elsewhere, and roads, which represent the interactions between the landmarks. We capture such homogeneity by the similar roles the nodes played. Next, to concretely model the landmarks, we propose a probabilistic generative model of networks with roles as latent factors. Furthermore, to enable interactive zooming, we formulate novel model-based constrained optimization. Then, we design efficient linear-time algorithms for the optimizations. Experiments using real-world and synthetic networks show that our method produces more expressive maps than existing methods, with up to 10 times improvement in network reconstruction quality. We also show that our method extracts landmarks with more homogeneous nodes, with up to 90% improvement in the average attribute/link entropy among the nodes over each landmark. Sense-making of a real-world network using a map computed by our method qualitatively verify the effectiveness of our method.

Network Cartography: Seeing the Forest and the Trees


Jia Wang, Kevin Chen-Chuan Chang, Hari Sundaram
Department of Computer Science
University of Illinois at Urbana-Champaign
Champaign, IL 61820

{jiawang4, kcchang, hs1}@illinois.edu


\@float

copyrightbox[b]

\end@float

Over the last decades, the size of datasets for real-world networks has expanded by orders of magnitude— While this data deluge has blessed our building more robust analytics, it has challenged our ability to explore these large networks with intuition. Often, a network (e.g., Facebook) captures real-world information of entities as nodes on the network, with attributes (e.g., gender, age) describing the properties of and edges indicating the interactions (e.g., friend-link between users) between entities. Such attributed, unweighted networks proliferate in many domains, e.g., social (e.g., Facebook, Twitter) and biology (e.g., protein-protein interaction) networks. We thus address the problem of exploring large, real-world networks by creating compact representations that reveal the essential interactions in the data, at desirable resolutions to see from the “trees” to the “forest” on the network.

We believe a compact representation for effectively exploring a network should satisfy the following requirements. Conciseness: The representation should preserve essential attribute and link patterns, while minimizing uninteresting detail, of the network. Expressiveness: It should be able to encapsulate a diversity of characteristic patterns that may exist in the network. Multi-resoultuons: It can represent the network at various controllable levels of details. Locatability: It should capture not only the big picture but also every node on the network– i.e., we can locate each individual node on the representation.

Network cartography.  The metaphor of a map satisfies our requirements. Maps, by definition of cartography [?], are “graphic representations that facilitate a spatial understanding of things, concepts, conditions, processes, or events in the human world.” In particular, for navigating the physical world, we are familiar with common geographic maps with “landmarks” and their spatial connections of “roads.” Such geographic maps are necessarily concise and expressive of all kinds of terrains, and allows zooming to various resolutions and locating every point of interest. Thus, for our objective, to compactly represent a large network, a network map should comprise landmarks, representing a group of individual nodes, and roads, abstracting the interactions between these landmarks— We aim to study this cartography of networks.

While the needs are pressing, unfortunately, there has been little prior work for network maps. Rosvall et al. [?] and Guimera et al. [?] made the early attempts to create maps to summarize complex networks. However, they either assume a specific domain (metabolic networks) or restrict to only link topologies. To our knowledge, our study is the first to create maps for general networks of both attribute and link information. We informally state our problem as follows.

Problem: For an attributed, unweighted network , given the number of landmarks , create a map for as a graph with nodes to represent .

Challenge: Discovering landmarks. What is a landmark on a network? While the important landmarks in the familiar geographical map have emerged due to centuries of cartographic practice, the semantics of the landmarks in a map designed to compactly represent a network are unclear.

Insight: From homogeneity to roles.  We observe that landmarks on a map signify homogeneity of points of interests— E.g., on a geographic map, a landmark (such as “Navy Pier”) is an aggregate representation of points (shops, restaurants, piers) sharing similar properties (at very close spatial coordinates) and similar interconnections with other landmarks. Thus, in a network, a landmark should aggregate nodes that are similar, in terms of their attributes and interactions. Such homogeneous nodes, in sociology, define roles [?] in an interconnected society, as individuals that share characteristic patterns of behaviors. We thus propose to discover roles in a network as landmarks on its map.

Figure \thefigure: Example network with node attributes (in red) representing the people and their interaction in an academic department. The middle figure colors each node by its role(s) resulted from role discovery. The bottom set of figures show a sequence of three maps with increasing resolution constructed from the discoverd roles, the relationships between the finer roles and those of the previous map are shown in dashed lines; each role is also labeled by the attributes it will likely have.

To illustrate, Figure Network Cartography: Seeing the Forest and the Trees shows an example network in part (a) and the roles (as discovered via our algorithm) of the network in part (b). In , each node is a person in an academic department, with attributes such as a:“age over 30” and t:“teach” for node . While , a toy network of 15 nodes, already appears “complex,” its nodes share common patterns of behaviors. To reveal, we annotate the network with the implicit roles in part (b) by coloring the nodes– nodes with a color share a certain behavior pattern. Now, we can observe that people in orange-role “teach and interact with green-role .” And, the latter group comprises young people. The red-role “does not teach, has many publications, are aged over 30, and interacts with blue role ”. A plausible theory is that is undergrad and grad students; then, and are perhaps researcher and teacher. Further, with roles as landmarks, in part (c), we can construct maps of different resolutions, e.g., with three roles and five roles.

Challenge: Formulating roles for landmarks. The challenge of formulating roles for landmarks lies in properly capturing the desired homogeneity. While there exist diverse formulations [?, ?, ?], they fall short for landmark homogeneity, as Section Network Cartography: Seeing the Forest and the Trees will further discuss. First, they are too restricted for real-world networks, as none of them captures overlapping roles and inter-role interactions. Second, they address only links but not attributes. We thus need to develop formulation for coherently integrating the dual-mode behaviors in potentially overlapping roles.

Insight: Probabilistic dual-mode regular equivalence.  First, we conceptually capture link homogeneity by regular equivalence (Re) [?], defined as ”two nodes are regularly equivalent if they are equally related to equivalent others”. This definition, however, is too restrictive for real-world networks. Our model for role extraction probabilistically captures the essence of Re to handle noisy real-world data, and incorporates overlapping roles and non-uniform inter-role interactions to flexibly express the variations in complex networks. Second, we inject attributes as the second mode of the role behavior, in addition to links. Thus, a role represents both link and attribute patterns, enforcing homogeneity in both modes.

Challenge: Multi-resolution continuum. As maps should support multi-resolutions, maintaining an intuitive continuum between resolutions is challenging. On a geographic map, when we zoom into a region (e.g., Chicago), the changes are limited to within the region (we may now see Millennium Park), and the interconnections to the rest of the world should be stable (the whole region is still located in the state of Illinois). Such continuum is intuitive for network maps too. On map (Figure Network Cartography: Seeing the Forest and the Treesc), when we zoom into landmark faculty, it gets refined into two finer landmarks in : teacher and researcher, while the other landmarks are not changed; later, when we zoom into student, resulting in undergrad and grad in , the continuum maintains. While desired, realizing such continuum is challenging– Since our landmarks are determined interdependently, changing one landmark will necessarily change others.

Insight: Multi-scale constraints. To maintain continuum in multi-resolution views, as we use roles for landmarks, we must enforce the semantics of continuum in role discovery. As we model role discovery as a likelihood maximization problem, we enforce the continuum as an additional constraint in the objective function, which thus enables a principled realization of multi-resolution stability.

Our solution. We summarize our framework of constructing a map for a given network as follows, as Figure Network Cartography: Seeing the Forest and the Trees shows.

  • Role discovery. Compute as the parameters of a model that describes how the observed network is generated probabilistically from the latent roles (Figure Network Cartography: Seeing the Forest and the Treesb).

  • Map construction. Use to compute and construct the map as a node-labeled weighted graph: each node of represent a role, the weight of the edge between two roles represent how likely they will interact, and each node is labeled with how likely that role will have each attribute (Figure Network Cartography: Seeing the Forest and the Treesc).

Specifically, we propose a new probabilistic, generative model for networks to extract roles as landmarks of the map (Section Network Cartography: Seeing the Forest and the Trees). The model explicitly parameterizes each node-role affiliation pair and each role-role interaction pair. The same role affiliation parameters will be the explanators in the logistic regression models for the node attributes. We design an objective function to optimize the likelihood of the observations under the model; our objective function formulation also encourages solution sparsity. Then, we process the model parameters into more intuitive quantities to construct the map, and achieve a multi-scale view by constraining the new solution to maintain the relationships and landmarks from the coarser scale, while determining the new landmarks at the finer scale and their relationship with the landmarks at the coarser scale (Section Network Cartography: Seeing the Forest and the Trees). We use an efficient block-coordinate gradient ascent algorithm to perform the optimization in linear time w.r.t the network size (Section Network Cartography: Seeing the Forest and the Trees).

In experiments, compared with baseline methods, ours shows significant improvement in model expressiveness measured by network reconstruction quality— up to 10 times for synthetic networks and 27% for real-world networks. In terms of homogeneity, our method outperforms existing methods by up to 90% in terms the attribute/link entropy among the nodes over each landmark. In a case study using real-world networks, a set of multi-resolution maps created by our method actually help us to easily see the salient structures of the network from complex observed interactions.

Contributions. We summarize our contributions as follows.

  • Network cartography through role discovery. We are the first to propose the use of roles to serve as map landmarks for attributed networks; in this, we are inspired by earlier work on regular equivalence [?]. The landmarks along with their attribute distribution and the resulting interrelationships between landmarks result in meaningful maps.

  • A probabilistic model for the map. We propose an expressive, probabilistic generative model of the network with roles as the latent factors. The expressive nature of the model allows us to model a wide variety of attributed networks. During the optimization process, we enforce sparsity constraints to achieve concise maps.

  • Simultaneous multiple resolution. We are able to create maps that simultaneously show different parts of the map at different resolutions.

  • Scalable cartography. Our algorithm runs in linear time (with respect to network size). Furthermore, it is easy to implement using standard optimization toolboxes.

  • Extensive experiments. We not only synthesize networks with diverse structures to verify the effectiveness of our method in a wide range of networks, but also use real-world networks to show the practicality of our method, evaluated using a variety of measures. Case studies further evaluate our method’s usability in action.

Network cartography.  Cartography [?] has a long history and was intensively used in visualizing geographic data, but their use in exploring large general networks has received attention only recently. Rosvall et al [?] creates map of sciences by revealing community structures through efficiently encoding of random walk paths. Guimera et al  [?] used universal roles detected from optimizing network modularity to create maps for metabolic networks.

Attributed networks.  We model node-attributed networks, for which alternative models with different goals have been proposed; e.g, Bagc [?] uses a Bayesian approach, Cesna [?] models overlapping communities, and Mag [?] models clusters as generated from attributes. Methods for graph clustering using both structure/attribute information have also been proposed [?, ?]. Issues of these methods will be discussed later in this section.

Role theory.  We create maps with landmarks as roles. The theory of roles have been studied in sociology [?] where roles are defined as “a set of connected behaviors, rights, obligations, beliefs, and norms as conceptualized by people in a social situation” [?], and various formulations have been proposed [?, ?, ?, ?]. For the alternative formulation of universal roles, which does not fit in our picture, see survey [?].

Intergroup interactions.  Our model accounts for intergroup interactions in contrast with only intra-group found in communities. Some other methods also accounts for this aspect but with limitations for creating maps. Mixed-Membership Stochastic Blockmodeling (Mmb) [?] models each observed edge of each node as generated from one of its roles, but it does not model the intensity of role affiliation, as also pointed out in [?, ?]. (Bagc [?] suffers from similar issues) as Mmb. In fact, Mmb cannot model hub and peripheral nodes, which in our model can be represented by nodes with strong affiliations to other roles. Mmb does not model node attributes as key to information-enriched maps either. In terms of goals, Mmb focus more on generalizability of model while we focus on summarization quality in computing maps. Generalized Blockmodeling [?] does not capture varying interaction intensities between blocks (roles). Dedicom [?] uses matrix factorization for real-valued data, thus cannot model binary links; other Nmf-based methods [?, ?] suffers from similar issues. The model by Han et al [?] uses separate parameters to explain node attributes and links.

Overlapping groups.  Models for overlapping groups of nodes has less been studied. Dedicom [?] and Cesna [?] are two examples. Note that mixed memberships [?, ?] do not sufficiently capture the concept of overlapping groups, because their models only model the probability, rather than the intensity, of the affiliation of nodes to the groups.

Multi-resolution views.  Local multiple-resolution views implies a hierarchical structure of the data (zooming corresponds to branching in a hierarchy). Methods for hierarchical clustering [?] cannot be directly applied to our case in which groups are overlapping. For graph summarization at different levels of granularity, Tian et al [?] proposed -Snap, but this operation does not support local views of part of the map.

In this section, we develop our probabilistic generative model of networks, with roles as latent factors, to extract the landmarks of a map.

Probabilistic regular equivalence.  We first identify Re (regular equivalence) as the most appropriate semantics of link homogeneity for maps, compared to RolX [?] and structural equivalence [?] (Se) as representatives of alternative formulations. RolX defines roles as universal in terms of structural functions, i.e, whether it is a bridge connecting otherwise separated groups, or a hub connecting to many nodes, etc.; it misses the interaction patterns of specific networks, thus falls short in expressness. Se considers nodes equivalent (with same role) only if they connects to the same other nodes — too restrictive since there can be too many nodes for Se to find a small number of useful roles. On the other hand, Re is free from both issues by considering nodes equivalent if they connects to the same roles rather than the same nodes.

We make significant extension of Re with a probabilistic generative model, regarding roles as the latent factors that drives the noisy observed networks. For extracting latent factors [?], although matrix factorizations (such as Nmf [?]) are also widely studied and used, they have not yet been well studied for effectively handling binary data matrices. Since binary data carry special semantics, thus methods for factorizing general real-valued data cannot be applied. From an efficiency perspective, Nmf for a general data matrix (adjacency matrix in our case) needs time [?], which is prohibitive for large networks; the Nmf-based algorithm Asalsan [?] specialized for graph processing has linear complexity but can only be used with weighted links. Probabilistic generative models, on the other hand, are a much more expressive class of tool which model various types of observations, including binary observations, e.g, by Bernoulli models. As shown later, an efficient algorithm can also be derived from a well designed probabilistic model.

Setting.  In this paper, we consider the extraction of landmarks given an undirected and unweighted graph of nodes and links, where represents the node set and the edge set (directed graphs can be handled with slight modification of our method as described later; handling weighted graphs will be discussed till the end of the paper). Each node is labeled by binary attributes , and we use to denote the set of observed node-attribute pairs .

Regarding the notation, we will use boldface capital letters like for matrices, calligraphic capital letters like for sets, boldface lowercase letters like for vectors, and lowercase letters for scalars. The roles will be indexed by and the attributes by .

We model the landmarks in a map as roles. In the example network in Figure Network Cartography: Seeing the Forest and the Trees, we observe group “often interact with ()” and “have more than 15 publications” but “do not teach courses”. A plausible theory would be that are research professors and graduates, and ’s behaviors can be explained by the pure role of “researcher”. We also observe “aged over 30” and “frequent interactions with group ” for group ; prior knowledge tells us are lecturers and are undergrads, and ’s behaviors are due to the pure role of “teacher”. Other roles include “admin”, “undergrad”, and “graduate”, as extracted in map in Figure Network Cartography: Seeing the Forest and the Trees.

Plural and non-uniform role affiliation.  To characterize the roles, we observe that each node can participate in multiple roles at different intensities. First, role affiliation is plural. In , group (known as professors in prior) do both teaching and research and interact with both undergrad and graduate students; these behaviors are explained by their simultaneous roles “teacher” (abbrv. tea) and “researcher” (abbrv. res). Second, the affiliation is non-uniform. Note that professor might not conduct role “teacher” so much as lecturers as indicated by less interactions with undergrads.

To capture the plurality and non-uniformity of role affiliation, we represent the affiliation of each node to a set of roles by vector , where quantifies how strongly is affiliated to the role ; we also require for easier interpretation of the results. For instance, in , the lecturer will have dominantly large compared to other components of due to a pure role of “teacher”, while the professor who does research but less teaching will have . Stacking form the node-role membership matrix .

Non-uniform inter-role interaction.  We observe that inter-role interactions are also non-uniform in intensities. In , although (married female aged over 30, possibly administrative staff) interact with both (professors) and (graduates), administrative staff usually interact more with professors, because they often assist the professors in scheduling events.

To reflect the non-uniformity of inter-role interactions, we quantify the tendency for two nodes of roles and to interact by . We then define the inter-role interaction matrix as . For instance, in , the strong interaction between professors and graduates (abbrv. gra) will be captured by and possibly also as graduates usually more actively turn to professors.

Based on the above notions, we model our belief for two nodes interact as follows. We know that if nodes are affiliated to roles and respectively and roles and tends to interact, nodes and also tends to interact. In addition, our belief for being formed will add up as we observe more such evidences from other pairs of roles . For simplicity, we assume that our belief grows linearly with the role affiliation and inter-role interaction intensities. Thus we formulate this belief, the predictor of link , as

Since our observations are binary, we need a link function that transform into probability values. Observing that the marginal potential of forming links normally decrease as more evidences are present, we model the probability of forming link with the predictor as

(1)

While another common choice for transforming predictors to have range would be the sigmoid function , it has domain but is restricted to be nonnegative, hence inappropriate for our case. We also support our use of by efficiency considerations as we will discuss in Section Network Cartography: Seeing the Forest and the Trees.

Links and attributes as covariates.  Finally, we observe that the links and attributes of each node are covariates with its roles as the underlying factor. In , the attribute “age > 30” and interactions of “being approached by graduate students” of the professor can both be explained by a “researcher” role. Similarly, the attribute “female” and the interactions of “being contacted by professors” can be explained by a “admin” role.

To capture the role-based covariation of attributes and links, we will use an attribute predictor as the linear combination of the role affiliation vector and some parameter to model our belief of node to have attribute (for simplicity, we assume that the attributes of a node are conditionally independent given the roles of that node). For example, in , the role “researcher” will indicate a high weight so that it would contributes positively and strongly to observing many publications for professors. Then, we transform the predictor to probability using a link function. For the above purpose, a standard logistic regression would suffice, by which we specify the attribute predictor as and the probability of observing as

(2)

Note the intercept term is ignored in the predictor to simplify our presentation. We denote .

Now, our problem of discovery roles as landmarks of the map for a given network is defined as follows.

Definition 1 (Role discovery)

Given an undirected unweighted graph with node attributes and the number of roles , we will find a set of roles represented in the following parameters: the matrix for inter-role interactions, a vector for each attribute for predicting the attributes of any node given the roles of , and a vector for each node for representing the role affiliations of .

Model extensibility.  Although binary attributes are discussed here, the model can be easily adapted to categorical attributes (e.g, occupation) using a multinomial logistic model in place of the binary model described earlier. The model also directly applies to directed graphs which will result in an asymmetric matrix . To support weighted graphs, we can modify the model by replacing in (1) with an identity function, while the algorithm needs to be designed differently (e.g, the Asalsan algorithm [?] with some adaption), for which we will not discuss in more details here.

We extract the roles from network by optimizing the parameters of our model of in the following aspects. First, we optimize the parameters such that they accurately express the observations in the original network using a set of roles. Since the number of landmarks is usually small for concise maps, it implies that each landmark must be homogeneous for the observations to be represented compactly using the roles. In particular, we capture the accuracy of by how well our model can be used to reconstruct . Second, we optimize for sparsity of the parameters for conciseness of representation. That is, we wish be represented using as few inter-role interactions as possible and each node can be described by their most important roles, such that users can focus on the most salient patterns of .

To encode the above requirements, we propose an objective function with respect to parameters . captures model accuracy by the log-likelihood of observing under a model with parameters , while the sparsity requirement is enforced by the -norm regularizations of and . Formally,

(3)

where and are the log-likelihoods of links and attributes given by

(4)
(5)

and weight is used to impose emphasis on links or attributes, and to control the degree of representation sparsity.

In summary, the optimal parameters , which maximizes subject to the constraints that and , will represent our extracted roles. That is,

(6)

The detailed algorithm for optimizing will be introduced later in Section Network Cartography: Seeing the Forest and the Trees.

Thus far we have encoded the information needed to create a map for the given network in the model parameters . In this section, we will develop methods to render as an intuitively interpretable map. Further, we propose methods to allow for local multiple-resolution views of any part of the map.

The computed model parameters in the stage of role discovery needs to be transformed to interpretable quantities to construct a map. For instance, while a larger apparently indicates the stronger affiliation of node to role , does not have an intuitive physical meaning, such as probability, length, or frequency which are natural for human interpretation. The same issue also holds for parameters and . This makes it non-intuitive, if not deceptive, to make sense of a map constructed directly from (e.g, using as the weighted adjacency matrix for the metagraph).

In order to construct maps as metagraphs with interpretable weights and labels, we introduce a virtual node for each role , which is an imaginary node affiliated purely to role , i.e., for some , where is the unit vector along the th dimension, and controls the affiliation strength of to role . Intuitively, the virtual node can be viewed as an idealized representative for role so that its behaviors as expected from our model naturally characterize role . We might choose the value of parameter as follows. First, we may set to some constant independent of the actual affiliations of the nodes to role , thus in effect characterize role itself by . Second, we may set , where , as the average affiliation of nodes with role ; such created virtual node will instead represent the role as actually practiced by the real nodes in data.

Using the virtual node as a representative for role , we characterize the behaviors of role intuitively by the probability of observing each attribute on , given by

Similarly, to characterize the inte-role interaction between each pair of roles and , we use the probability of interaction between an virtual node with purely role and a virtual node with purely role , which we give by

For later reference, we denote and . Such transformed will be finally visualized as the output map.

In summary, we formally define the steps for creating a map as follows.

Definition 2 (Map construction)

Given an unweighted network with binary node attributes, a map for with landmarks as a metagraph is constructed by

  1. Compute given ;

  2. Use to compute for each and for each ;

  3. Construct graph : each node in represents a landmark, each edge is weighted by , and each node is labeled with for each attribute .

With an overview of the network by a coarser map, we allow users to select part of the map for higher-resolution views in the same global context, as in map in Figure Network Cartography: Seeing the Forest and the Trees where we selected the landmark “student” to zoom in, and revealed the finer-granular landmarks “undergrad” and “graduate” in , while other landmarks remains unchanged, and the finer-granular connections between the new landmarks and the existing landmarks are revealed too, such as that “graduate” interacts more with professors than “undergrad”.

As a first attempt to changing the resolution locally, one might extract a subgraph for the selected landmark and compute a map with that subgraph as input, but this method loses the connections between the newly created landmarks with those existing outside the subgraph. One might also attempt the alternative way of computing from scratch a new map with more landmarks, but it violates our locality requirement and destructs the existing non-selected landmarks. Although hierarchical clustering also produces finer-granular views of data, it does hard partitioning of the data points while our groups of nodes are overlapping; it also considers only intra-cluster connections but not the connections between the clusters.

Figure \thefigure: Obtaning from to initialize

We achieve local multiple-resolution views for some role in a given map constructed with by constructing a new map with parameters , and constraining our optimization of such that the non-selected roles (those other than ) in remain little changed. We measure the change between maps and by the distance between their respective parameters and with respect to the non-split roles. Formally, we define the distance between models and using sum of squared errors as

where (resp. ) transforms by removing its components associated with role (resp. ), for which we illustrate the details in Figure Network Cartography: Seeing the Forest and the Trees. Using the operator , we actually measure the divergence between the maps only with respect to roles other than the selected .

In summary, we formally define the problem of computing local multi-resolution views on a map as follows.

Definition 3 (Local multi-resolution views)

Given a previously computed map constructed from with roles, and a specified landmark to view at finer resolution, we compute a map with model of roles by splitting role into the sub-roles and , such that satisfies our requirements for appropriate landmarks by maximizing , meanwhile keeps the roles other than relatively unchanged by minimizing

Note that while we demonstrate the case of two sub-roles, the algorithm can be easily generalized to split any role into multiple sub-roles.

In order to find that both maximizes for quality landmarks and minimizes for locality of the finer view, we optimize under a revised objective function

(7)

where is the hyperperameter controlling the tradeoff between the landmark quality and the locality of the resolution change. For the revised optimization objective, we will design an efficient algorithm in Section Network Cartography: Seeing the Forest and the Trees.

In this section, we develop efficient algorithms to optimize the objective functions formulated in Section Network Cartography: Seeing the Forest and the Trees for role discovery and Section Network Cartography: Seeing the Forest and the Trees for computing local multi-resolution views in selected landmarks.

Algorithm for landmark extraction.  Given the optimization objective function formulated in (3), we derive the gradients of with respect to variables as follows. For easier derivation, we split the matrix variables and into the vector variables and . Then

(8)
(9)
(10)

Note that the optimization problem (6) is nonconvex. The problem of optimizing the variable with all other variable fixed, however, is convex. This is also true for variables and . Thus we resort to the block-coordinate gradient ascent algorithm to optimize , where we cycle through the variables and optimize one of them at a time until convergence to a local minima. To optimize each variable, we employ the gradient ascent algorithm with the learning rate set proportional to the normalized gradient. After each gradient ascent update of any variable subject to the non-negativity constraint, we need an additional step to ensure the updated still satisfies the constraint. To this end, we project the negative entries of to after each iteration, denoted by . The steps are summarized in Algorithm 1.

Input : Network , number of landmarks
Output : Landmarks , ,
// initialization
1 Initialize entries of with small random values;
2 Initialize with for some matrix with small random entries;
3 repeat
       // evaluate gradients with precomputed
4       ;
5       Repeat ;
       // enforcing symmetry of by projection
6       if  is undirected then
7             ;
8            
9      for each node  do
10             Repeat ;
11             Update incrementally due to new ;
12            
13      for each attribute  do
14             Repeat ;
15            
16until  not improved;
Algorithm 1 NetLand

On Initialization.  At the beginning of the optimization, we need to initialize each variable properly. Initializing the variables to small uniformly distributed random values around turns out to work well in practice. An initialization with large random values, however, will typically lead to useless results with the algorithm stopping at an early stage. In addition, special attention must be paid to the when the given network is undirected. In this case, needs to be initialized as some symmetric matrix. It can be easily shown that the symmetry property will then continue to hold throughout the optimization. In practice, setting to for some random matrix will do the work, while failures in confirming to symmetry will result in unexpected nonsense results in most cases.

Linear-time computation.  The cost of Algorithm 1 concentrates on the evaluation of the log-likelihoods (4), (5) for evaluating the objective function and the gradients (8)–(10). To analyze the time for objective function evaluation, it amounts to analyzing the evaluation of for each link individually and the sum of for all absent links .

Simply evaluating for each pair of vertices is prohibitive. While for each observed link , the term takes time for evaluating , there are separated pairs of nodes, for which computing once for each such pair is impossible.

Fortunately, the sum of for missing links can be improved by first rewriting it into

We observe that the first term on the right side can be simplified into where . The value of can be precomputed in time and cached before optimizing in each outer iteration. The total time for evaluating is then . The costly last term in (8) can be similarly computed.

With the precomputation technique, the time for a full (outer) iteration of the algorithm scales linearly w.r.t the size of . In details, each gradient ascent step for takes time, each takes time where is the degree of node , and each takes time. Summing up, the overall time complexity of Algorithm 1 is ; the memory needed is proportional to the size of . The performance in practice can be further boosted using optimized matrix operation libraries.

Note that the precomputation technique is valid for our choice of as the link function but not for the logistic function . With , term has to be evaluated once for each such link. In fact, with , term in (8) will be replaced by which cannot be decomposed as the product of two cheap sums as in case of mean function. This results in prohibitive complexity.

Algorithm for zooming.  We create locally refined view with respect to the selected landmark in a map with parameters by optimizing as formulated in (7). Let the sub-roles to be created be and . We derive the gradients of as follows.

where is the extended which will be used to initialize . In particular, we create from by removing its parameters for the role and appending new (randomly initialized) parameters for the sub-roles and ; see Figure Network Cartography: Seeing the Forest and the Trees for an illustration of this process. The operator sets the parameters in corresponding to the sub-roles to 0. Intuitively, keeps the newly created sub-roles from being penalized by its distance from , . The weight controls the degree of the closeness enforcement.

We optimize using Algorithm 1 with the adapted gradients and also a special initialization of for faster convergence. In particular, we initialize to which inherits the roles other than , so that the algorithm’s overhead will be limited to the optimization of the sub-roles. The algorithm for zooming is summarized in Algorithm 2.

Input : Network , landmarks , selected landmark
Output : Landmarks with refined
1 ;
2 Append parameters for to and initialize them with small random numbers;
3 Initialize by ;
Compute using Algorithm 1 with adapted gradients
Algorithm 2 Zoom

We will both quantitatively verify if the maps computed by our method have the desired properties of maps we identified in Section Network Cartography: Seeing the Forest and the Trees. We will also demonstrate the usability of our method by creating multi-resolution maps for real-world networks.

Datasets.  The following real-world and synthetic datasets will be used in our tests. We chose the real networks so that they exhibit interesting strong intergroup interactions instead of only intra-group interactions found in communities.

  • Hospital 111http://www.sociopatterns.org/datasets. This dataset [?] contains the directed network of contacts among 75 people in a hospital ward in France during five days. Each person is labeled with one of {patient, admin, nurse, doctor}. An arc is created for each pair of people if they interacted at least 5 times during the period.

  • Enron 222http://cis.jhu.edu/~parky/Enron. This dataset represents the directed network of email communications between 184 users. Each user is labeled by one or more the 32 topics (manually extracted in [?]) they ever used. We create an arc for each (ordered) pair of users if they communicated at least 15 emails.

  • Syn-x. We synthesize as parameters of our proposed model with landmarks, which in turn generates random undirected networks, such that the network corresponds to one of {bip,star,comm,rand} representing bipartite, star (wheel), community, and random structures of (constructed from ). Thus, a variety of common graph structures will be captured in the experiments. We then synthesize networks according our probabilistic generative model with parameter .

Methods.  We will evaluate and compare the following methods.

  • NetLand . Our model of networks for creating maps. By default, we set the weight for attribute likelihoods as and sparsity regularization weights to 0 when compared with other method and 0.2 otherwise.

  • Cesna [?]. This the state-of-the-art model for large node-attributed networks. It aims at overlapping groups, models attributes as generated from group memberships, as well allows flexible community affiliation. By default, which has the same role as ours. In Cesna, represents the number of communities, analogous to our landmarks.

  • BinDedi. The original Dedicom [?, ?] model uses a similar form to model links, which allows arbitrary membership affiliation and pairwise intergroup interactions, but handles only weighted graphs and does not handle attributes. We non-trivially adapt it for binary links by applying the link function to and . The modified Dedicom is equivalent to our model with .

Convergence.  We evaluate the convergence performance of Algorithm 1 as indicator for efficiency. Figure Network Cartography: Seeing the Forest and the Trees shows how the value of the objective function changes with the number of full iterations. We observe that the algorithm improves significantly at the first few iterations and converges quickly. Thus our algorithm is scalable given also the linear time complexity of each iteration.

Figure \thefigure: Convergence of NetLand (Algorithm 1).

Expressiveness.  We measure a map’s expressiveness by quality of network reconstruction using the map , in terms of the log-likelihood of observing the network given the model underlying our map. In Figure (a), we show the log-likelihoods of the real-world networks at different computed by NetLand and Cesna (method BinDedi is not considered here due to its inability to handle node attributes; it corresponds to NetLand with and the effect of will be evaluated later). We observe that our method outperforms Cesna by up to on hospital and on enron in reconstruction quality. For both methods, the log-likelihoods improve with larger , as expected due to more parameters.

To evaluate the reconstruction quality of NetLand on networks with diverse structural characteristics, we further use four synthetic networks with structures often found in real-world networks, as shown in Figure (b). For syn-star, we observe that NetLand drastically outperforms Censa by orders of magnitude. Obvious advantage of our method is also observed for syn-bip and syn-rand. In these networks, non-community patterns dominate or mix with other inter-group interactions — structures which Cesna cannot model. On the other hand, for syn-comm, our improvement is marginal as increases since both methods are able to handle community structures (corresponding to an with only diagonal nonzero elements).

We also evaluate the impact of weight which controls the tradeoff between link and attribute reconstruction qualities. The log-likelihoods at different for methods NetLand and Cesna are shown in Figure Network Cartography: Seeing the Forest and the Trees (Censa also has as the weight for attribute log-likelihoods). We observe that log-likelihoods peaks at for NetLand on both networks, suggesting as a possible starting point to set generally. On both datasets, the performance drops quickly as approaches 1 with an overemphasis on attributes. On the other hand, a small has less negative impact on performance. This is possibly due to the more observations () for links than for attributes ( observations) so that the decrease in attribute likelihoods less affects the overall. A higher weight for attributes would impact more on the overall likelihood and make the attributes better expressed.

(a) Real-world networks
(b) Synthetic networks
Figure \thefigure: Expressiveness: log-likelihood of network at varying using Cesna and NetLand on real-world and synthetic datasets.
Figure \thefigure: Impact of on log-likelihoods

Homogeneity.  The key property our method must satisfy is the homogeneity of links and attributes among the nodes in each landmark. We evaluate the homogeneity by the entropy of the link and attribute distributions in each landmark.

Since each node can belong to multiple landmarks, we approximately measure the entropy of the overlapping groups by first grouping the nodes by their main roles . Let be the distribution of attribute in group , where and , and be the link distribution of node with respect to the roles, where . The mean link distribution (with respect to the roles) of group is thus .

Given the grouping of nodes, we measure the homogeneity of attributes in by summing up the entropy of each attribute over the nodes in , and measure the homogeneity of links in by the summed deviation of each node’s link distribution among the roles from the mean of summed entropy over each attribute and links of map . Finally, we give the homogeneity measures of map as follows.

where is (binary) entropy function and is the KL-divergence function (relative entropy). measures how much each node’s link distribution diverges from the mean distribution of its group.

We show the results for homogeneity of both links and attributes in Figure Network Cartography: Seeing the Forest and the Trees. For our method, as increases, it shows an overall trend of increasing homogeneity (decreasing entropy) on both attributes and links, as expected due to unmixed roles. On the other hand, a smaller generally results in higher homogeneity for attributes and slightly lower homogeneity for links, as more weight is posed on attributes in optimization.

Next, we compare our method to BinDedi in Figure (a), We observe that NetLand consistently outperforms BinDedi which only considers only the topology aspect of networks. For link homogeneity, although our methods model the additional attribute observations, the resulted link homogeneity is still quite comparable to that of BinDedi which focuses entirely on links.

We also compare our method to Cesna which handles attributes but does not model intergroup interactions, with results given in Figure (b). Our method outperforms Cesna in most cases by a significant margin (up to for enron at ). We explain this advantage by the greater expressiveness of our model which accounts for intergroup interactions, beyond Censa’s state-of-the-art modeling of node-attributed networks. Moreover, we observe improvement also in attribute homogeneity over Cesna in most cases (up to for hospital at ), as roles turns out a better concept than communities in explaining attributes.

(a) NetLand v.s BinDedi
(b) NetLand v.s CESNA
Figure \thefigure: Homogeneity: average attribute entropy of landmarks and link entropy of nodes.
Figure \thefigure: Cartography of the directed hospital network (enlarge for details): a continuous sequence of maps for with increasing levels of resolution. The rows from top to bottom are: of inter-landmark interaction probabilities, the map (landmarks selected for zooming are circled in bold blue and created sub-roles in bold black), input network (colored by main role), and of probability of each attribute for each landmark. Landmarks in and are labeled by hypothesized names.
Figure \thefigure: Locations of nodes: coordinates of nodes ( for node ) with respect to map in Figure Network Cartography: Seeing the Forest and the Trees, in the space defined by each pair of landmarks. In each subplot, the set of points represent the set of all nodes.

In Figure Network Cartography: Seeing the Forest and the Trees, we show a sequence of maps by Algorithm 2 computed by Algorithm 2 with for the hospital network. While we allow users to select any landmark to split, for this example, we split the largest role (in term of how many nodes have it as the main role) in each map. We used for and , as defined in Section Network Cartography: Seeing the Forest and the Trees. For reference, we index the roles as .

Multi-resolution maps.  First of all, we can intuitively see the localized incremental changes between consecutive maps from the coloring of the non-split landmarks and the heatmaps of , and . The interactions between non-split landmarks are not disturbed by the zooming, too.

To start, we see in map that nurses has the largest population in the hospital (see corresponding ). In the subsequent map , role is characterized by mainly involving nurses and by patients. With help of indicating strong interaction from to , we see that the major interactions in this hospital are those from nurses to patients (possibly explained by that nurses actively perform regular checks for patients).

Increasing reveals finer-resolution structures. In the map , role are administrative staff who actively approach which involves mainly doctors and some patients; possibly corresponds to “administrative staff members assigned with specific doctors or patients to assist”. mainly involves patients and mainly nurses. involves mainly administrative staff and some other nurses who are passively approached by patients and doctors; they are possibly “on-duty staff”.

To study in map in more details, we computed map by splitting the of into the sub-roles and . Role still corresponds to nurses but now those inactive ones, captures active nurses, and now also some administrative staff absent in . This is because while we enforce stability of affiliations of nodes to the non-selected roles, our formulation still allows a small number of nodes to change their role compositions (controlled by in (7)); particularly, some administrative staff become affiliated to the new in , resulting in the added adm attribute of . We interpret as “active hospital staff (in approaching patients)”.

Locatability.  Recall that each node can be located using affiliation vector . In Figure Network Cartography: Seeing the Forest and the Trees, we show a pairwise plot of ’s columns for map in Figure Network Cartography: Seeing the Forest and the Trees. Each subplot shows the overall distribution of nodes in the corresponding landmarks. For instance, we observe in subplot-(5,1) that no nodes simultaneously takes roles (personal admins) and (on-calls), probably because a personal assistant does not often perform general on-call duties.

The task of extracting a big picture from complex real-world networks is challenging. In this paper, we presented effective cartographic methods to help with this task. We created intuitively understandable visual maps. To extract landmarks of the map, we recognized roles as the appropriate concept to model landmarks, and proposed a novel expressive model for role discovery. Furthermore, we allow for local multiple-resolution views for knowledge acquisition from networks at different granularities. We also propose efficient algorithms for computing the maps. In addition to an intuitive representation, experiments show our method outperforms state-of-the-art network models in representation accuracy by up to 27% for real networks and up to 10 times on synthetic networks. Case study also shows our created maps actually help users to quickly capture the big picture of a network.

  • [1] E. M. Airoldi, D. M. Blei, S. E. Fienberg, and E. P. Xing. Mixed membership stochastic blockmodels. Advances in Neural Information Processing Systems, pages 33–40, 2009.
  • [2] B. W. Bader, R. Harshman, and T. G. Kolda. Temporal analysis of semantic graphs using asalsan. In Data Mining, 2007. ICDM 2007. Seventh IEEE International Conference on, pages 33–42. IEEE, 2007.
  • [3] M. W. Berry, M. Browne, and B. Signer. topic annotated enron email data set. Philadelphia: Linguistic Data Consortium, 2001.
  • [4] B. J. Biddle. Role theory: Expectations, identities, and behaviors. Academic Press, 2013.
  • [5] S. P. Borgatti and M. G. Everett. The class of all regular equivalences: Algebraic structure and computation. Social Networks, 11(1):65–88, 1989.
  • [6] R. S. Burt. Positions in networks. Social forces, 55(1):93–122, 1976.
  • [7] P. Doreian, V. Batagelj, and A. Ferligoj. Generalized blockmodeling. 2005.
  • [8] M. G. Everett and S. P. Borgatti. Regular equivalence: General theory. Journal of mathematical sociology, 19(1):29–52, 1994.
  • [9] J. Friedman, T. Hastie, and R. Tibshirani. The elements of statistical learning, volume 1. Springer series in statistics Springer, Berlin, 2001.
  • [10] A. Giddens, M. Duneier, R. P. Appelbaum, and D. Carr. Introduction to sociology. WW Norton New York, 2000.
  • [11] R. Guimera and L. A. N. Amaral. Functional cartography of complex metabolic networks. Nature, 433(7028):895–900, 2005.
  • [12] Y. Han and J. Tang. Probabilistic community and role model for social networks. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 407–416. ACM, 2015.
  • [13] J. B. Harley, D. Woodward, et al. Cartography in prehistoric, ancient, and medieval Europe and the Mediterranean, volume 1. University of Chicago Press Chicago, 1987.
  • [14] R. A. Harshman, P. E. Green, Y. Wind, and M. E. Lundy. A model for the analysis of asymmetric data in marketing research. Marketing Science, 1(2):205–242, 1982.
  • [15] K. Henderson, B. Gallagher, T. Eliassi-Rad, H. Tong, S. Basu, L. Akoglu, D. Koutra, C. Faloutsos, and L. Li. Rolx: structural role extraction & mining in large graphs. In Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 1231–1239. ACM, 2012.
  • [16] M. Kim and J. Leskovec. Latent multi-group membership graph model. arXiv preprint arXiv:1205.4546, 2012.
  • [17] C.-b. Lin. Projected gradient methods for nonnegative matrix factorization. Neural computation, 19(10):2756–2779, 2007.
  • [18] M. J. Mandel. Local roles and social networks. American sociological review, pages 376–386, 1983.
  • [19] M. Nickel, V. Tresp, and H.-P. Kriegel. A three-way model for collective learning on multi-relational data. In Proceedings of the 28th international conference on machine learning (ICML-11), pages 809–816, 2011.
  • [20] R. Rossi, N. K. Ahmed, et al. Role discovery in networks. Knowledge and Data Engineering, IEEE Transactions on, 27(4):1112–1131, 2015.
  • [21] M. Rosvall and C. T. Bergstrom. Maps of random walks on complex networks reveal community structure. Proceedings of the National Academy of Sciences, 105(4):1118–1123, 2008.
  • [22] A. Silva, W. Meira Jr, and M. J. Zaki. Mining attribute-structure correlated patterns in large attributed graphs. Proceedings of the VLDB Endowment, 5(5):466–477, 2012.
  • [23] Y. Tian, R. A. Hankins, and J. M. Patel. Efficient aggregation for graph summarization. In Proceedings of the 2008 ACM SIGMOD international conference on Management of data, pages 567–580. ACM, 2008.
  • [24] P. Vanhems, A. Barrat, C. Cattuto, J.-F. Pinton, N. Khanafer, C. Régis, B.-a. Kim, B. Comte, and N. Voirin. Estimating potential infection transmission routes in hospital wards using wearable proximity sensors. PloS one, 8(9):e73970, 2013.
  • [25] F. Wang, T. Li, X. Wang, S. Zhu, and C. Ding. Community discovery using nonnegative matrix factorization. Data Mining and Knowledge Discovery, 22(3):493–521, 2011.
  • [26] Z. Xu, Y. Ke, Y. Wang, H. Cheng, and J. Cheng. A model-based approach to attributed graph clustering. In Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data, pages 505–516. ACM, 2012.
  • [27] J. Yang and J. Leskovec. Overlapping community detection at scale: a nonnegative matrix factorization approach. In Proceedings of the sixth ACM international conference on Web search and data mining, pages 587–596. ACM, 2013.
  • [28] J. Yang, J. McAuley, and J. Leskovec. Community detection in networks with node attributes. In Data Mining (ICDM), 2013 IEEE 13th international conference on, pages 1151–1156. IEEE, 2013.
  • [29] Y. Zhou, H. Cheng, and J. X. Yu. Graph clustering based on structural/attribute similarities. Proceedings of the VLDB Endowment, 2(1):718–729, 2009.
Comments 0
Request Comment
You are adding the first comment!
How to quickly get a good reply:
  • Give credit where it’s due by listing out the positive aspects of a paper before getting into which changes should be made.
  • Be specific in your critique, and provide supporting evidence with appropriate references to substantiate general statements.
  • Your comment should inspire ideas to flow and help the author improves the paper.

The better we are at sharing our knowledge with each other, the faster we move forward.
""
The feedback must be of minimum 40 characters and the title a minimum of 5 characters
   
Add comment
Cancel
Loading ...
10622
This is a comment super asjknd jkasnjk adsnkj
Upvote
Downvote
""
The feedback must be of minumum 40 characters
The feedback must be of minumum 40 characters
Submit
Cancel

You are asking your first question!
How to quickly get a good answer:
  • Keep your question short and to the point
  • Check for grammar or spelling errors.
  • Phrase it like a question
Test
Test description