Laplacian embedding with tuned localization and spectral bandwidth using graph Slepians
Guiding Network Analysis using Graph Slepians:
An Illustration for the C. Elegans Connectome
Spectral approaches of network analysis heavily rely upon the eigendecomposition of the graph Laplacian. For instance, in graph signal processing, the Laplacian eigendecomposition is used to define the graph Fourier transform and then transpose signal processing operations to graphs by implementing them in the spectral domain. Here, we build on recent work that generalized Slepian functions to the graph setting. In particular, graph Slepians are band-limited graph signals with maximal energy concentration in a given subgraph. We show how this approach can be used to guide network analysis; i.e., we propose a visualization that reveals network organization of a subgraph, but while striking a balance with global network structure. These developments are illustrated for the structural connectome of the C. Elegans.
Guiding Network Analysis using Graph Slepians:
An Illustration for the C. Elegans Connectome
Dimitri Van De Ville, Robin Demesmaeker, Maria Giulia Preti
École Polytechnique Fédérale de Lausanne (EPFL), Switzerland
University of Geneva, Switzerland
Keywords: Graph signal processing, Graph Laplacian, Slepian functions, Graph cut, Laplacian embedding
In many applications, data can be advantageously modeled and analyzed using graphs, in particular, to reveal emergent, global network properties. In addition, increased availability of large and rich datasets, combined with better computational resources, have been driving the growing interest in network science and graph signal processing, respectively by different communities. One fundamental concept in network analysis is graph partitioning and embedding; e.g., how nodes can be grouped in subgraphs or how they can be represented in a low-dimensional space based on the graph topology . Spectral graph theory relates these problems to the graph Laplacian and its eigendecomposition . For instance, the convex relaxation of the graph cut can be solved by thresholding the Laplacian eigenvector with the smallest non-zero eigenvalue—known as the Fiedler vector . Moreover, the Laplacian eigenvectors also provide a meaningful embedding by mapping nodes onto a line (or higher-dimensional space) that minimizes distance between connected nodes . For graph signal processing, the eigenvectors of the graph Laplacian can be used to define a graph Fourier transform (GFT) and then to define processing steps in the spectral domain ; e.g., the spectral graph wavelet transform  is such a notable example.
Recently, we have introduced the graph equivalent of Slepian functions  that were proposed fifty years ago on regular domains to find a trade-off between temporal and spectral energy concentration [8, 9]. The design of graph Slepians is based on an optimization criterion that expresses energy concentration of a band-limited graph signal in a predefined subgraph. In our previous work , we proposed to generalize this criterion such that it reflects a modified Laplacian embedding distance, which basically gives a meaning to eigenvalues as “localized frequencies”, instead of as energy concentrations.
Here, we apply the graph Slepian framework to guide network analysis. After a brief introduction on the theory and design of graph Slepians, we show how the information revealed by graph Slepians can be summarized to provide insightful visualizations. We then focus on the structural connectome of the C. Elegans organism as an example to guide network analysis.
2 Graph Slepians in a Nutshell
2.1 Graph Fourier Transform
Let us consider an undirected weighted graph with nodes. Its connectivity is characterized by an symmetric adjacency matrix , where the elements indicate the edge weights between nodes . We assume the graph is connected and the edge weights are positive. Next to the graph and its topology, we can also consider graph signals that define a mapping from the nodes to a length- vector associating a value with every node; e.g., the graph topology models the presence of communication links between nodes, while the graph signal represents measures taken at the nodes.
The graph Laplacian is defined as , where is the diagonal degree matrix with elements . We further consider the normalized graph Laplacian that factors out differences in degree and thus is only reflecting relative connectivity.
Spectral methods for graphs are based on the insight that the eigendecomposition of the graph Laplacian gives access to a GFT. The eigenvectors of minimize
and can be ordered according to increasing eigenvalues . The eigenvectors play the role of basis vectors of the graph spectrum, and the associated eigenvalues the one of frequencies . By stacking the vectors as the columns of , which is an unitary matrix, we obtain the GFT that links a graph signal and its spectral coefficient vector as
2.2 Graph Slepians
To generalize Slepians to graphs, we introduce selectivity and bandwidth for the graph signals [10, 7]. First, selectivity can be specified by a subset that contains the nodes in which we want the energy concentration to be optimal. We introduce the diagonal selection matrix with elements that indicate the presence of index in ; so the number of selected indices is . Second, for the notion of “bandwidth”, we propose to restrict the spectrum to the eigenvectors with the smallest eigenvalues . We then consider the truncated graph spectrum matrix of size ; i.e., it only contains the first columns of . Consequently, any band-limited graph signal can be represented by
Finding the linear combination of eigenvectors within the bandlimit and with maximal energy in reverts to optimizing the Rayleigh quotient
where is the concentration matrix. Since is real and symmetric, is real as well and we revert to the regular transpose . The Slepian vectors are orthonormal over the entire graph as well as orthogonal over the subset ; i.e., we have as well as , where is the Kronecker delta.
By convention, the eigenvalues of the Slepian eigendecomposition are sorted according to decreasing energy concentration in . The eigenvalue spectrum presents a sudden transition between well-localized and poorly-localized eigenvectors.
2.3 Graph Slepians Revisited: The Link with Laplacian Embedding
The GFT plays a central role in Laplacian embedding and graph clustering. Specifically, in Laplacian embedding, the aim is to find a mapping from the nodes onto a line such that strongly connected nodes stay as close as possible, which can be expressed as :
where and . The solution is the eigenvector of the Laplacian with the smallest non-zero eigenvalue; i.e., the Fiedler vector . The optimization (3) is also related to the classical graph cut problem that consists of partitioning the graph into clusters of nodes such that the cut size is minimal. In this setting, indicate the labels of the nodes, which can be relaxed to take any real values.
We now use the Slepian design to generalize the criterion (3) as to find a bandlimited solution that is optimized for a selected set of nodes. To reveal this link, we first rewrite the quadratic form (3) as
Therefore, we propose to obtain a generalized Slepian criterion by introducing in this quadratic form bandwidth and node selection in the spectral and graph domains, respectively:
where is the band-limited diagonal matrix. Consequently, we propose to solve the eigendecomposition of the modified concentration matrix . As before, the Slepian vectors are double orthogonal as we have and . We store the graph Slepians as columns of the matrix of size . The concentration value of each graph Slepian can still be recovered using the quadratic form , or, vice versa, the modified embedded distance can be found for the previous design as .
This demonstrates that Laplacian embedding can be generalized as a Slepian problem with the additional weighting of the Laplacian eigenvalues. It is interesting to note that the eigenvalues of represent the modified embedded distance, or, equivalently, a “frequency” that is localized in the subgraph , whereas the eigenvalues of represent the energy concentration in the subgraph.
3 An Illustration for the Worm Connectome
The C. Elegans is an intensely studied organism in biology. In particular, the wiring diagram of its 302 neurons has been carefully mapped during a long and effortful study . Here, we use the adjacency matrix that considered 279 somatic neurons and combined connectivity from chemical synapses and gap junctions . We retrieved the type of each neuron (including polymodal status) from the WormAtlas***http://www.wormatlas.org/.
3.1 Graph Laplacian Embedding
In their modeling work, Varshney et al  studied network properties of the worm connectome using different approaches, including Laplacian embedding. In particular, the topological view generated by mapping on the first two eigenvectors with smallest non-zero eigenvalues already reveals an interesting organization, see Fig. 1A. The horizontal dimension () reveals two lobes of neurons mostly according to ventral and neck neurons, respectively. The vertical dimension () show mostly a sensory-inter-motor neuron gradient (from bottom to top), which is very similar to previous results .
3.2 Graph Slepian Analysis
To provide a 2-D visualization, we want to maximize the variance captured by the Slepians in the selected subgraph and avoid the selection of the “most important” Slepian vectors. Therefore, we construct the restricted Gram matrix that contains the inner products of the graph Slepians computed on the subgraph ; i.e., we have . We then diagonalize this matrix using the SVD as and project onto this basis to obtain coordinates as , from which we keep the two dimensions with largest eigenvalues in . The columns of can be considered as a data-driven spectral window that allows maximizing the variance in the visualization for nodes that are part of .
We specifically pick the subgraph as the sensory neurons, or polymodal neurons with a sensory component. A total of 83 neurons are selected, which represents of all neurons. For low bandwidth up to , the Slepian view on the network is basically identical to the Laplacian embedding. Then, for higher bandwidth such as , the energy concentration in the subset of sensory neurons is able to increase and a different structure is revealed, as shown in Fig. 1B. First, many of the motorneurons have moved towards the center of the representation, which means that many of them are not so well connected to sensory neurons. Second, the two lobes of ventral/neck neurons are replaced by two lobes of mainly left/right neurons, including some remaining inter- and motorneurons. This structure is preserved as the bandwidth is further increased, although some additional organization within is uncovered, such as the branch of pharynx neurons for , see Fig. 1C.
For these results, we used the graph Slepian design based on the modified embedded distance, however, the graph Slepian design based on the energy concentration leads to almost identical results. The reason is that the summarizing method described before relies on the subspace spanned by the Slepians and not on their eigenvalues, and these spaces are very similar. However, for any spectral graph operations (e.g., filtering  or wavelet construction ), the Slepian design based on the modified embedded distance is advantageously as the eigenvalues can be used as frequencies; for an exemple see .
One well known phenomenon for classical Slepian functions is the phase transition between concentrated and non-concentrated functions, which is given by the Shannon number or time-bandwidth product. For the graph Slepians, the graph-bandwidth product corresponds to . However, as opposed to the conventional case, this is not necessarily equal to the sum of concentration values due to the irregularity of the domain. In Fig. 2A, we plot the graph-bandwidth product and the sum of concentration values for the two Slepian designs when selecting the sensory neurons. We notice that for the concentration design, the sum is almost identical to as the bandwidth varies. For the embedded-distance design, we notice a gap between both measures, although the sum of concentration values still increases approximately linearly. In Fig. 2B, we show a similar behavior when selecting the motor neurons (140 in total).
We have shown how graph Slepians can be designed and used to guide network analysis. The proposed method is flexible as it allows to easily recompute the Slepian vectors for different choices of the subgraph and bandwidth, once the (truncated) eigendecomposition of the graph Laplacian is available. For large(r) graphs, one can rely on efficient large-scale solvers  implemented in widely available software libraries such as ARPACK. Future extensions of this approach could include the relationship with graph uncertainty principles [15, 10, 16], statistical resampling for graphs , or discovery of hierarchical graph structure [18, 19] by gradual refinement of the subgraph. The design could also be extended for directed graphs using recent extensions of spectral decompositions for directed graphs [20, 21].
The Matlab implementation of the graph Slepians is publicly available†††https://codeocean.com/2017/05/10/graph-slepians.
This work was supported in part by the Swiss National Science Foundation, and in part by the Center for Biomedical Imaging (CIBM) of the Geneva - Lausanne Universities and the EPFL, the foundations Leenaards and Louis-Jeantet.
-  U. Luxburg, “A tutorial on spectral clustering,” Statist. Comput. 17(4), pp. 395–416, 2007.
-  F. R. K. Chung, Spectral Graph Theory, Amer. Math. Soc., Providence, 1997.
-  M. Fiedler, “Laplacian of graphs and algebraic connectivity,” Combinatorics and Graph Theory 25, pp. 57–70, 1989.
-  M. Belkin and P. Niyogi, “Laplacian eigenmaps for dimensionality reduction and data representation,” Neural Computation 15(6), pp. 1373–1396, 2003.
-  D. I. Shuman, S. K. Narang, P. Frossard, A. Ortega, and P. Vandergheynst, “The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains,” IEEE Signal Processing Magazine 30(3), pp. 83–98, 2013.
-  D. K. Hammond, P. Vandergheynst, and R. Gribonval, “Wavelets on graphs via spectral graph theory,” Appl. and Comp. Harm. Anal. 30(2), pp. 129–150, 2011.
-  D. Van De Ville, R. Demesmaeker, and M. G. Preti, “When Slepian meets Fiedler: Putting a focus on the graph spectrum,” IEEE Signal Processing Letters 24, pp. 1001–1004, 2017.
-  D. Slepian and H. O. Pollak, “Prolate spheroidal wave functions, fourier analysis and uncertainty,” Bell System Technical Journal 40(1), pp. 43–63, 1961.
-  D. Slepian, “Prolate spheroidal wave-functions, fourier-analysis, and uncertainty. Discrete case,” Bell System Technical Journal 57(5), pp. 1371–1430, 1978.
-  M. Tsitsvero, S. Barbarossa, and P. Di Lorenzo, “Signals on graphs: Uncertainty principle and sampling,” IEEE Transactions on Signal Processing 64(18), pp. 4845–4860, 2016.
-  J. G. White, E. Southgate, J. N. Thomson, and S. Brenner, “The structure of the nervous system of the nematode Caenorhabditis elegans,” Phil Trans R Soc Lond B 314, pp. 1–340, 1986.
-  B. L. Chen, D. H. Hall, and D. B. Chklovskii, “Wiring optimization can relate neuronal structure and function,” Proceedings of the National Academy of Sciences of the United States of America 103, pp. 4723–4728, Mar. 2006.
-  L. R. Varshney, B. L. Chen, E. Paniagua, D. H. Hall, and D. B. Chklovskii, “Structural properties of the Caenorhabditis elegans neuronal network,” PLoS computational biology 7(2), p. e1001066, 2011.
-  R. B. Lehoucq and D. C. Sorensen, “Deflation techniques for an implicitly restarted Arnoldi iteration,” SIAM Journal on Matrix Analysis and Applications 17(4), pp. 789–821, 1996.
-  A. Agaskar and Y. M. Lu, “A spectral graph uncertainty principle,” IEEE Transactions on Information Theory 59(7), pp. 4338–4356, 2013.
-  O. Teke and P. P. Vaidyanathan, “Uncertainty principles and sparse eigenvectors of graphs,” IEEE Transactions on Signal Processing , in press.
-  E. Pirondini, A. Vybornova, M. Coscia, and D. Van De Ville, “Spectral method for generating surrogate graph signals,” IEEE Signal Processing Letters 23, pp. 1275–1278, September 2016.
-  A. Arenas, A. Fernández, and S. Gómez, “Analysis of the structure of complex networks at different resolution levels,” New Journal of Physics 10, p. 053039, May 2008.
-  J. Irion and N. Saito, “Hierarchical graph Laplacian eigen transforms,” JSIAM Letters 6, pp. 21–24, 2014.
-  A. Sandryhaila and J. M. F. Moura, “Discrete signal processing on graphs,” IEEE Transactions on Signal Processing 61, pp. 1644–1656, Mar. 2013.
-  H. N. Mhaskar, “A unified framework for harmonic analysis of functions on directed graphs and changing data,” Applied and Computational Harmonic Analysis , in press.