The Link Prediction Problem
in Bipartite Networks
We define and study the link prediction problem in bipartite networks, specializing general link prediction algorithms to the bipartite case. In a graph, a link prediction function of two vertices denotes the similarity or proximity of the vertices. Common link prediction functions for general graphs are defined using paths of length two between two nodes. Since in a bipartite graph adjacency vertices can only be connected by paths of odd lengths, these functions do not apply to bipartite graphs. Instead, a certain class of graph kernels (spectral transformation kernels) can be generalized to bipartite graphs when the positive-semidefinite kernel constraint is relaxed. This generalization is realized by the odd component of the underlying spectral transformation. This construction leads to several new link prediction pseudokernels such as the matrix hyperbolic sine, which we examine for rating graphs, authorship graphs, folksonomies, document–feature networks and other types of bipartite networks.
In networks where edges appear over time, the problem of predicting such edges is called link prediction [1, 2]. Common approaches to link prediction can be described as local when only the immediate neighborhood of vertices is considered and latent when a latent model of the network is used. An example for local link prediction methods is the triangle closing model, and these models are conceptually very simple. Latent link prediction methods are instead derived using algebraic graph theory: The network’s adjacency matrix is decomposed and a transformation is applied to the network’s spectrum. This approach is predicted by several graph growth models and results in graph kernels, positive-semidefinite functions of the adjacency matrix .
Many networks contain edges between two types of entities, for instance item rating graphs, authorship graphs and document–feature networks. These graphs are called bipartite , and while they are a special case of general graphs, link prediction methods cannot be generalized to them. As we show in Section 2, this is the case for all link prediction functions based on the triangle closing model, as well as all positive-semidefinite graph kernels. Instead, we will see that their odd components can be used, in Section 3. For each positive-semidefinite graph kernel, we derive the corresponding odd pseudokernel. One example is the exponential graph kernel . Its odd component is , the hyperbolic sine. We also introduce the bipartite von Neumann pseudokernel, and study the bipartite versions of polynomials with only odd powers. We show experimentally (in Section 4) how these odd pseudokernels perform on the task of link prediction in bipartite networks in comparison to their positive counterparts, and give an overview of their relative performances . We also sketch their usage for detecting near-bipartite graphs.
2 Bipartite Link Prediction
The link prediction problem is usually defined on unipartite graphs, where common link prediction algorithms make several assumptions :
Triangle closing: New edges tend to form triangles.
Clustering: Nodes tend to form well-connected clusters in the graph.
In bipartite graphs these assumptions are not true, since triangles and larger cliques cannot appear. Other assumptions have therefore to be used. While a unipartite link prediction algorithm technically applies to bipartite graphs, it will not perform well. Methods based on common neighbors of two vertices will for instance not be able to predict anything in bipartite graphs, since two vertices that would be connected (from different clusters) do not have any common neighbors.
Several important classes of networks are bipartite: authorship networks, interaction networks, usage logs, ontologies and many more. Many unipartite networks (such as coauthorship networks) can be reinterpreted as bipartite networks when edges or cliques are modeled as vertices. In these cases, special bipartite link prediction algorithms are necessary. The following two sections will review local and algebraic link prediction methods for bipartite graphs. Examples of specific networks of these types will be given in Section 4.
Given an undirected graph with vertex set and edge set , its adjacency matrix is defined as when and otherwise. For a bipartite graph , the adjacency matrix can be written as , where is the biadjacency matrix of .
2.1 Local Link Prediction Functions
Some link prediction functions only depend on the immediate neighborhood of two nodes; we will call these functions local link prediction functions .
Let and be two nodes in the graph for which a link prediction score is to be computed. Local link prediction functions depend on the common neighbors of and . In the bipartite link prediction problem, and are in different clusters, and thus have no common neighbors. The following link prediction functions are therefore not applicable to bipartite graphs: The number of common neighbors , the measure of Adamic and Adar  and the Jaccard coefficient . These methods are all based on the triangle closing model, which is not valid for bipartite graphs.
2.1.1 Preferential Attachment
If is the number of neighbors of node , the preferential attachment models gives a prediction between and of . The factor normalizes the sum of predictions for a vertex to its degree.
3 Algebraic Link Prediction Functions
Link prediction algorithms that not only take into account the immediate neighborhood of two nodes but the complete graph can be formulated using algebraic graph theory, whereby a decomposition of the graph’s adjacency matrix is computed . By considering transformations of a graph’s adjacency matrix, link prediction methods can be defined and learned. Algebraic link prediction methods are motivated by their scalability and their learnability. They are scalable because they rely on a model that is built once and which makes computation of recommendations fast. These models correspond to decomposed matrices and can usually be updated using iterative algorithms . In contrast, local link prediction algorithms are memory-based, meaning they access the adjacency data directly during link prediction. Algebraic link prediction methods are learnable because their parameters can be learned in a unified way .
In this section, we describe how algebraic link prediction methods apply to bipartite networks. Let be a (not necessarily bipartite) graph. Algebraic link prediction algorithms are based on the eigenvalue decomposition of its adjacency matrix :
To predict links, a spectral transformation is usually applied:
where applies a real function to each eigenvalue . then contains link prediction scores that, for each node, give a ranking of all other nodes, which is then used for link prediction. If is positive, is a graph kernel, otherwise, we will call a pseudokernel.
Several spectral transformations can be written as polynomials of the adjacency matrix in the following way. The matrix power gives, for each vertex pair , the number of paths of length between and . Therefore, a polynomial of gives, for a pair , the sum of all paths between and , weighted by the polynomial coefficients. This fact can be exploited to find link prediction functions that fulfill the two following requirements:
The link prediction score should be higher when two nodes are connected by many paths.
The link prediction score should be higher when paths are short.
These requirements suggest the use of polynomials with decreasing coefficients.
3.1 Odd Pseudokernels
In bipartite networks, only paths of odd length are significant, since an edge can only appear between two vertices if they are already connected by paths of odd lengths. Therefore, only odd powers are relevant, and we can restrict the spectral transformation to odd polynomials, i.e. polynomials with odd powers.
The resulting spectral transformation is then an odd function and except in the trivial and undesired case of a constant zero function, will be negative at some point. Therefore, all spectral transformations described below are only pseudokernels and not kernels.
3.1.1 The Hyperbolic Sine
where coefficients are decreasing with path length. Keeping only the odd component, we arrive at the matrix hyperbolic sine .
Figure 2 shows the hyperbolic sine applied to the (positive) spectrum of the bipartite Slovak Wikipedia user–article edit network.
3.1.2 The Odd von Neumann Pseudokernel
The von Neumann kernel for unipartite graphs is given by the following expression .
We call its odd component the odd von Neumann pseudokernel:
The hyperbolic sine and von Neumann pseudokernels are compared in Figure 3, based on the path weights they produce.
3.1.3 Rank Reduction
Similarly, rank reduction of the matrix can be described as a pseudokernel. Let be the eigenvalue with -th largest absolute value, then rank reduction is defined by
This function is odd, but does not have an (odd) Taylor series expansion.
3.2 Computing Latent Graph Models
Bipartite graphs have adjacency matrices of the form
where is the biadjacency matrix of the graph. This form can be exploited to reduce the eigenvalue decomposition of to the equivalent singular value decomposition .
with , and each singular value corresponds to the eigenvalue pair .
3.3 Learning Pseudokernels
The hyperbolic sine and the von Neumann pseudokernel are parametrized by , and rank reduction has the parameter , or equivalently . These parameters can be learned by reducing the spectral transformation problem to a one-dimensional curve fitting problem, as described in . In the bipartite case, we can apply the curve fitting method to only the graph’s singular value, since odd spectral transformations fit the negative eigenvalue in a similar way they fit the positive eigenvalues. This kernel learning method is shown in Figure 4.
As experiments, we show the performance of bipartite link prediction functions on several large datasets, and present a simple method for detecting bipartite or near-bipartite datasets.
4.1 Performance on Large Bipartite Networks
We evaluate all bipartite link prediction functions on the following bipartite network datasets. BibSonomy is a folksonomy of scientific publications . BookCrossing is a bipartite user–book interaction network . CiteULike is a network of tagged scientific papers . DBpedia is the semantic network of relations extracted from Wikipedia, of which we study the five largest bipartite relations . Epinions is the rating network from the product review site Epinions.com . Jester is a user–joke network . MovieLens is a user–movie rating dataset, and a folksonomy of tags attached to these movies . Netflix is the large user–item rating network associated with the Netflix Prize . The Wikipedia edit graphs are the bipartite user–article graphs of edits on various language Wikipedias. The Wikipedia categories are represented by the bipartite article–category network . All datasets are bipartite and unweighted. In rating datasets, we only consider the presence of a rating, not the rating itself. Table 1 gives the number of nodes and edges in each dataset.
In the experiments, we withhold 30% of each network’s edges as the test set to predict. For datasets in which edges are labeled by timestamps, the test set consists of the newest edges. The remaining training set is used to compute link prediction scores using the preferential attachment model and the pseudokernel learning methods described in the previous sections. For the pseudokernel learning methods, the training set is again split into 70% / 30% subsets for training. Link prediction accuracy is measured by the mean average precision (MAP), averaged over all users present in the test set . The evaluation results are summarized in Table 1.
|DBpedia football club||41,846||131,084||0.685||0.678||0.674||0.505||0.159||0.680|
4.2 Detecting Near-bipartite Networks
Some networks are not bipartite, but nearly so. An example would be a network of “fan” relationships between persons where there are clear “hubs” and “authorities”, i.e. popular persons and persons being fan of many people. While these networks are not strictly bipartite, they are mostly bipartite in a sense that has to be made precise. Measures for the level of bipartivity exist in several forms [4, 27], and spectral transformations offer another method. Using the link prediction method described in Section 3.3, nearly bipartite graphs can be recognized by the odd shape of the learned curve fitting function.
Figure 5 shows the method applied to two unipartite networks: the Advogato trust network  and the hyperlink network in the English Wikipedia . The curves indicate that the Advogato trust network is not bipartite, while the Wikipedia link network is nearly so.
While technically the link prediction problem in bipartite graphs is a subproblem of the general link prediction problem, the special structure of bipartite graphs makes common link prediction algorithms ineffective. In particular, all methods based on the triangle closing model cannot work in the bipartite case. Out of the simple local link prediction methods, only the preferential attachment model can be used in bipartite networks.
Algebraic link prediction methods can be used instead, by restricting spectral transformations to odd functions, leading to the matrix hyperbolic sine as a link prediction function, and an odd variant of the von Neumann kernel. As in the unipartite case, no single link prediction method is best for all datasets.
-  Liben-Nowell, D., Kleinberg, J.: The link prediction problem for social networks. In: Proc. Int. Conf. on Information and Knowledge Management. (2003) 556–559
-  Taskar, B., Wong, M.F., Abbeel, P., Koller, D.: Link prediction in relational data. In: Advances in Neural Information Processing Systems. (2003)
-  Gärtner, T., Horváth, T., Le, Q.V., Smola, A., Wrobel, S.: Kernel Methods for Graphs. In: Mining Graph Data. John Wiley & Sons (2006)
-  Holme, P., Liljeros, F., Edling, C.R., Kim, B.J.: On network bipartivity. Phys. Rev. E 68 (2003) 6653–6673
-  Leskovec, J., Backstrom, L., Kumar, R., Tomkins, A.: Microscopic evolution of social networks. In: Proc. Int. Conf. on Knowledge Discovery and Data Mining. (2008) 462–470
-  Adamic, L., Adar, E.: Friends and neighbors on the web. Social Networks 25 (2001) 211–230
-  Barabási, A.L., Albert, R.: Emergence of scaling in random networks. Science 286(5439) (1999) 509–512
-  Zhang, D., Mao, R.: Classifying networked entities with modularity kernels. In: Proc. Conf. on Information and Knowledge Management. (2008) 113–122
-  Newman, M.E.J.: Finding community structure in networks using the eigenvectors of matrices. Phys. Rev. E 74 (2006)
-  Chung, F.: Spectral Graph Theory. American Mathematical Society (1997)
-  Rendle, S., Schmidt-Thieme, L.: Online-updating regularized kernel matrix factorization models for large-scale recommender systems. In: Proc. Int. Conf. on Recommender Systems. (2008) 251–258
-  Kunegis, J., Lommatzsch, A.: Learning spectral graph transformations for link prediction. In: Proc. Int. Conf. on Machine Learning. (2009) 561–568
-  Ito, T., Shimbo, M., Kudo, T., Matsumoto, Y.: Application of kernels to link analysis. In: Proc. Int. Conf. on Knowledge Discovery in Data Mining. (2005) 586–592
-  Wu, Y., Chang, E.Y.: Distance-function design and fusion for sequence data. In: Proc. Int. Conf. on Information and Knowledge Management. (2004) 324–333
-  Kandola, J., Shawe-Taylor, J., Cristianini, N.: Learning semantic similarity. In: Advances in Neural Information Processing Systems. (2002) 657–664
-  Cardoso, J.R., Leite, F.S.: Computing the inverse matrix hyperbolic sine. In: Proc. Int. Conf. on Numerical Analysis and its Applications. (2001) 160–169
-  Hotho, A., Jäschke, R., Schmitz, C., Stumme, G.: BibSonomy: A social bookmark and publication sharing system. In: Proc. Workshop on Conceptual Structure Tool Interoperability. (2006) 87–102
-  Ziegler, C.N., McNee, S.M., Konstan, J.A., Lausen, G.: Improving recommendation lists through topic diversification. In: Proc. Int. World Wide Web Conf. (2005) 22–32
-  Emamy, K., Cameron, R.: CiteULike: A researcher’s social bookmarking service. Ariadne (51) (2007)
-  Bizer, C., Cyganiak, R., Auer, S., Kobilarov, G.: DBpedia.org–querying Wikipedia like a database. In: Proc. Int. World Wide Web Conf. (2007)
-  Massa, P., Avesani, P.: Controversial users demand local trust metrics: an experimental study on epinions.com community. In: Proc. American Association for Artificial Intelligence Conf. (2005) 121–126
-  Goldberg, K., Roeder, T., Gupta, D., Perkins, C.: Eigentaste: A constant time collaborative filtering algorithm. Information Retrieval 4(2) (2001) 133–151
-  GroupLens Research: MovieLens data sets. http://www.grouplens.org/node/73 (October 2006)
-  Bennett, J., Lanning, S.: The Netflix prize. In: Proc. KDD Cup. (2007) 3–6
-  Wikimedia Foundation: Wikimedia downloads. http://download.wikimedia.org/ (January 2010)
-  Manning, C.D., Raghavan, P., Schütze, H.: Introduction to Information Retrieval. Cambridge University Press (2008)
-  Estrada, E., Rodríguez-Velázquez, J.A.: Spectral measures of bipartivity in complex networks. Phys. Rev. E 72 (2005)
-  Stewart, D.: Social status in an open-source community. American Sociological Review 70(5) (2005) 823–842