Subgraph Frequencies: Mapping the Empirical and Extremal Geography of Large Graph Collections
Abstract
A growing set of online applications are generating data that can be viewed as very large collections of small, dense social graphs — these range from sets of social groups, events, or collaboration projects to the vast collection of graph neighborhoods in large social networks. A natural question is how to usefully define a domainindependent ‘coordinate system’ for such a collection of graphs, so that the set of possible structures can be compactly represented and understood within a common space. In this work, we draw on the theory of graph homomorphisms to formulate and analyze such a representation, based on computing the frequencies of small induced subgraphs within each graph. We find that the space of subgraph frequencies is governed both by its combinatorial properties — based on extremal results that constrain all graphs — as well as by its empirical properties — manifested in the way that real social graphs appear to lie near a simple onedimensional curve through this space.
We develop flexible frameworks for studying each of these aspects. For capturing empirical properties, we characterize a simple stochastic generative model, a singleparameter extension of ErdősRényi random graphs, whose stationary distribution over subgraphs closely tracks the onedimensional concentration of the real social graph families. For the extremal properties, we develop a tractable linear program for bounding the feasible space of subgraph frequencies by harnessing a toolkit of known extremal graph theory. Together, these two complementary frameworks shed light on a fundamental question pertaining to social graphs: what properties of social graphs are ‘social’ properties and what properties are ‘graph’ properties?
We conclude with a brief demonstration of how the coordinate system we examine can also be used to perform classification tasks, distinguishing between structures arising from different types of social graphs.
3
Categories and Subject Descriptors: H.2.8 [Database Management]: Database applications—Data mining
Keywords: Social Networks, Triadic Closure, Induced Subgraphs, Subgraph Census, Graph Homomorphisms.
1 Introduction
The standard approach to modeling a large online social network is to treat it as a single graph with an enormous number of nodes and a sparse pattern of connections. Increasingly, however, many of the key problems encountered in managing an online social network involve working with large collections of small, dense graphs contained within the network.
On Facebook, for example, the set of people belonging to a group or attending an event determines such a graph, and considering the set of all groups or all events leads to a very large number of such graphs. On any social network, the network neighborhood of each individual — consisting of his or her friends and the links among them — is also generally a small dense graph with a rich structure, on a few hundred nodes or fewer [29]. If we consider the neighborhood of each user as defining a distinct graph, we again obtain an enormous collection of graphs. Indeed, this view of a large underlying social network in terms of its overlapping node neighborhoods suggests a potentially valuable perspective on the analysis of the network: rather than thinking of Facebook, for example, as a single billionnode network, with a global structure that quickly becomes incomprehensible, we argue that it can be useful to think of it as the superposition of a billion small dense graphs — the network neighborhoods, one centered at each user, and each accessible to a closer and more tractable investigation.
Nor is this view limited to a site such as Facebook; one can find collections of small dense graphs in the interactions within a set of discussion forums [9], within a set of collaborative online projects [31], and in a range of other settings.
Our focus in the present work is on a fundamental global question about these types of graph collections: given a large set of small dense graphs, can we study this set by defining a meaningful ‘coordinate system’ on it, so that the graphs it contains can be represented and understood within a common space? With such a coordinate system providing a generalpurpose framework for analysis, additional questions become possible. For example, when considering collections of a billion or more social graphs, it may seem as though almost any graph is possible; is that the case, or are there underlying properties guiding the observed structures? And how do these properties relate to more fundamental combinatorial constraints deriving from the extremal limits that govern all graphs? As a further example, we can ask how different graph collections compare to one another; do network neighborhoods differ in some systematic way, for instance, from social graphs induced by other contexts, such as the graphs implicit in social groups, organized events, or other arrangements?
The Present Work
In this paper we develop and analyze such a representation, drawing on the theory of graph homomorphisms. Roughly speaking, the coordinate system we examine begins by describing a graph by the frequencies with which all possible small subgraphs occur within it. More precisely, we choose a small number (e.g. or ); then, for each graph in a collection, we create a vector with a coordinate for each distinct node subgraph , specifying the fraction of tuples of nodes in that induce a copy of (in other words, the frequency of as an induced subgraph of ). For , this description corresponds to what is sometimes referred to as the triad census [6, 7, 8, 32]. The literature on frequent subgraph mining [11, 13, 33], and motif counting [18] is also is closely related, but focuses on connected subgraphs.
With each graph in the collection mapped to such a vector, we can ask how the full collection of graphs fills out this space of subgraph frequencies. This turns out to be a subtle issue, because the arrangement of the graphs in this space is governed by two distinct sets of effects: extremal combinatorial constraints showing that certain combinations of subgraph frequencies are genuinely impossible; and empirical properties, which reveal that the bulk of the graphs tend to lie close to a simple onedimensional curve through the space. We formulate results on both these types of properties, in the former case building on an expanding body of combinatorial theory [4, 17] for bounding the frequencies at which different types of subgraphs can occur in a larger ambient graph.
The fact that the space of subgraph frequencies is constrained in these multiple ways also allows us to concretely address the following type of question: When we see that human social networks do not exhibit a certain type of structure, is that because such a structure is mathematically impossible, or simply because human beings do not create it when they form social connections? In other words, what is a property of graphs and what is a property of people? Although this question is implicit in many studies of social networks, it is hard to separate the two effects without a formal framework such as we have here.
Indeed, our framework offers a direct contribution to one of the most wellknown observations about social graphs: the tendency of social relationships to close triangles, and the relative infrequency of what is sometimes called the ‘forbidden triad’: three people with two social relationships between them, but one absent relationship [22]. There are many sociological theories for why one would expect this subgraph to be underrepresented in empirical social networks [10]. Our framework shows that the frequency of this ‘forbidden triad’ has a nontrivial upper bound in not just social graphs, but in all graphs. Harnessing our framework more generally, we are in fact able to show that any node subgraph that is not a complete or empty subgraph has a frequency that is bounded away from one. Thus, there is an extent to which almost all subgraphs are mathematically ‘forbidden’ from occurring beyond a certain frequency.
We aim to separate these mathematical limits of graphs from the complementary empirical properties of real social graphs. The fact that real graph collections have a roughly onedimensional structure in our coordinate system leads directly to our first main question: is it possible to succinctly characterize the underlying backbone for this onedimensional structure, and can we use such a characterization to usefully describe graphs within our coordinate system in terms of their deviation from this backbone?
The subgraph frequencies of the standard ErdősRényi random graph [3] produce a onedimensional curve (parametrized by ) that weakly approximates the layout of the real graphs in the space, but the curve arising from this random graph model systematically deviates from the real graphs in that the random graph contains fewer triangles and more trianglefree subgraphs. This observation is consistent with the sociological principle of triadic closure — that triangles tend to form in social networks. As a means of closing this deviation from , we develop a tractable stochastic model of graph generation with a single additional parameter, determining the relative rates of arbitrary edge formation and triangleclosing edge formation. The model exhibits rich behaviors, and for appropriately chosen settings of its single parameter, it produce remarkably close agreement with the subgraph frequencies observed in real data for the suite of all possible 3node and 4node subgraphs.
Finally, we use this representation to study how different collections of graphs may differ from one another. This arises as a question of basic interest in the analysis of large social media platforms, where users continuously manage multiple audiences [2] — ranging from their set of friends, to the members of a groups they’ve joined, to the attendees of events and beyond. Do these audiences differ from each other at a structural level, and if so what are the distinguishing characteristics? Using Facebook data, we identify structural differences between the graphs induced on network neighborhoods, groups, and events. The underlying basis for these differences suggests corresponding distinctions in each user’s reaction to these different audiences with whom they interact.
2 Data description
Throughout our presentation, we analyze several collections of graphs collected from Facebook’s social network. The collections we study are all induced graphs from the Facebook friendship graph, which records friendship connections as undirected edges between users, and thus all our induced graphs are also undirected. The framework we characterize in this work would naturally extend to provide insights about directed graphs, an extension we do not discuss. We do not include edges formed by Facebook ‘subscriptions’ in our study, nor do we include Facebook ‘pages’ or connections from users to such pages. All Facebook social graph data was analyzed in an anonymous, aggregated form.
For this work, we extracted three different collections of graphs, around which we organize our discussion:

Neighborhoods: Graphs induced by the friends of a single Facebook user ego and the friendship connections among these individuals (excluding the ego).

Groups: Graphs induced by the members of a ‘Facebook group’, a Facebook feature for organizing focused conversations between a small or moderatesized set of users.

Events: Graphs induced by the confirmed attendees of ‘Facebook events’, a Facebook feature for coordinating invitations to calendar events. Users can response ‘Yes’, ‘No’, and ‘Maybe’ to such invitations, and we consider only users who respond ‘Yes’.
The neighborhood and groups collections were assembled in October 2012 based on monthly active user egos and current groups, while the events data was collected from all events during 2010 and 2011. For event graphs, only friendship edges formed prior to the date of the event were used. Subgraph frequencies for fournode subgraphs were computed by sampling 11,000 induced subgraphs uniformly with replacement, providing sufficiently precise frequencies without enumeration. The graph collections were targeted at a variety of different graph sizes, as will be discussed in the text.
3 Subgraph space
In this section, we study the space of subgraph frequencies that form the basis of our coordinate system, and the onedimensional concentration of empirical graphs within this coordinate system. We derive a model capable of accurately identifying the backbone of this empirical concentration using only the basic principle of triadic closure, showing how the subgraph frequencies of empirical social graphs are seemingly restricted to the vicinity of a simple onedimensional structure.
Formally, the subgraph frequency of a node graph in an node graph (where ) is the probability that a random node subset of induces a copy of . It is clear that for any integer , the subgraph frequencies of all the node graphs sum to one, constraining the vector of frequencies to an appropriately dimensioned simplex. In the case of , this vector is simply the relative frequency of induced threenode subgraphs restricted to the 4simplex; there are just four such subgraphs, with zero, one, two, and three edges respectively. When considering the frequency of larger subgraphs, the dimension of the simplex grows very quickly, and already for , the space of fournode subgraph frequencies lives in an 11simplex.
Empirical distribution
In Figure 1, the threenode subgraph frequencies of 50node, 100node, and 200node graph collections are shown, with each subplot showing a balanced mixture of 17,000 neighborhood, group and event graphs – the three collections discussed in Section 2, totaling 51,000 graphs at each size. Because these frequency vectors are constrained to the 4simplex, their distribution can be visualized in with three of the frequencies as axes.
Notice that these graph collections, induced from disparate contexts, all occupy a sharply concentrated subregion of the unit simplex. The points in the space have been represented simply as an unordered scatterplot, and two striking phenomena already stand out: first, the particular concentrated structure within the simplex that the points follow; and second, the fact that we can already discern a nonuniform distribution of the three contexts (neighborhoods, groups and events) within the space — that is, the different contexts can already be seen to have different structural loci. Notice also that as the sizes of the graphs increases – from 50 to 100 to 200 – the distribution appears to sharpen around the onedimensional backbone. The vast number of graphs that we are able to consider by studying Facebook data is here illuminating a structure that is simply not discernible in previous examinations of subgraph frequencies [8], since no analysis has previously considered a collection near this scale.
The imagery of Figure 1 directly motivates our work, by visually framing the essence of our investigation: what facets of this curious structure derive from our graphs being social graphs, and what facets are simply universal properties of all graphs? We will find, in particular, that parts of the space of subgraph frequencies are in fact inaccessible to graphs for purely combinatorial reasons — it is mathematically impossible for one of the points in the scatterplot to occupy these parts of the space. But there are other parts of the space that are mathematically possible; it is simply that no real social graphs appear to be located within them. Intuitively, then, we are looking at a population density within an ambient space (the Facebook graphs within the space of subgraph frequencies), and we would like to understand both the geography of the inhabited terrain (what are the properties of the areas where the population has in fact settled?) and also the properties of the boundaries of the space as a whole (where, in principle, would it be possible for the population to settle?).
Also in Figure 1, we plot the curve for the frequencies for 3 node subgraphs in as a function of . The curves are given simply by the probability of obtaining the desired number of edges in a three node graph, . This curve closely tracks the empirical density through the space, with a single notable discrepancy: the real world graphs systemically contain more triangles when compared to at the same edge density. We emphasize that it is not a priori clear why would at all be a good model of subgraph frequencies in modestlysized dense social graphs such as the neighborhoods, groups, and events that we have here; we believe the fact that it tracks the data with any fidelity at all is an interesting issue for future work. Beyond , in the following subsection, we present a stochastic model of edge formation and deletion on graphs specifically designed to close the remaining discrepancy. As such, our model provides a means of accurately characterizing the backbone of subgraph frequencies for social graphs.
Stochastic model of edge formation
The classic ErdősRényi model of random graphs, , produces a distribution over node undirected graphs defined by a simple parameter , the probability of each edge independently appearing in the graph. We now introduce and analyze a related random graph model, the Edge Formation Random Walk, defined as a random walk over the space of all unlabeled node graphs. In its simplest form, this model is closely related to , and will we show via detailed balance that the distribution defined by on node graphs is precisely the stationary distribution of this simplest version of the random walk on the space of node graphs. We first describe this basic version of the model; we then add a component to the model that captures a triadic closure process, which produces a close fit to the properties we observe in real graphs.
Let be the space of all unlabeled node graphs, and let be the following continuous time Markov chain on the state space . The transition rates between the graphs in are defined by random additions and deletions of edges, with all edges having a uniform formation rate and a uniform deletion rate . Thus the single parameter , the effective formation rate of edges, completely characterizes the process. Notice that this process is clearly irreducible, since it is possible to transition between any two graphs via edge additions and deletions.
Since is irreducible, it possesses a unique stationary distribution. The stationary distribution of an irreducible continuous time Markov chain can be found as the unique stable fixed point of the linear dynamical system that describes the diffusion of probability mass during a random walk on node graphs, where is the generator matrix with transition rates and , all depending only on . The stationary distribution then satisfies .
The following proposition shows the clear relationship between the stationary distribution of this simplest random walk and the frequencies of .
Proposition 3.1
The probabilities assigned to (unlabeled) graphs by satisfy the detailed balance condition for the Edge Formation Random Walk with edge formation rate , and thus characterizes the stationary distribution.
We first describe an equivalent Markov chain based on labeled graphs: there is a state for each labeled node graph; the transition rate from a labelled graph to a labelled graph is if can be obtained from by adding an edge; and if can be obtained from by removing an edge. All other transition rates are zero. We call this new chain the labeled chain, and the original chain the unlabeled chain.
Now, suppose there is a transition from unlabeled graph to unlabeled graph in the unlabeled chain, with transition probability . This means that there are ways to add an edge to a labeled copy of to produce a graph isomorphic to . Now, let be any graph in the labeled chain that is isomorphic to . In the labeled chain, there are transitions out of leading to a graph isomorphic to , and each of these has probability . Thus, with probability , a transition out of leads to a graph isomorphic to . A strictly analogous argument can be made for edge deletions, rather than edge additions.
This argument shows that the following describes a Markov chain equivalent to the original unlabeled chain: we draw a sequence of labeled graphs from the labeled chain, and we output the isomorphism classes of these labeled graphs. Hence, to compute the stationary distribution of the original unlabeled chain, which is what we seek, we can compute the stationary distribution of the labeled chain and then sum stationary probabilities in the labeled chain over the isomorphism classes of labeled graphs.
It thus suffices to verify the detailed balance condition for the distribution on the labeled chain that assigns probability to each labeled graph . Since every transition of the labeled walk occurs between two labeled graphs and , with , the only nontrivial detailed balance equations are of the form:
Since the probability assigned to the labeled graph by is simply , detailed balance is clearly satisfied.
Incorporating triadic closure
The above modeling framework provides a simple analog of that notably exposes itself to subtle adjustments. By simply adjusting the transition rates between select graphs, this framework makes it possible to model random graphs where certain types of edge formations or deletions have irregular probabilities of occurring, simply via small perturbations away from the classic model. Using this principle, we now characterize a random graph model that differs from by a single parameter, , the rate at which 3node paths in the graph tend to form triangles. We call this model the Edge Formation Random Walk with Triadic Closure.
Again let be the space of all unlabeled node graphs, and let be a continuous time Markov chain on the state space . As with the ordinary Edge Formation Random Walk, let edges have a uniform formation rate and a uniform deletion rate , but now also add a triadic closure formation rate for every 3node path that a transition would close. The process is still clearly irreducible, and the stationary distribution obeys the stationary conditions , where the generator matrix now also depends on . We can express the stationary distribution directly in the parameters as . For the model reduces to the ordinary Edge Formation Random Walk.
The state transitions of this random graph model are easy to construct for and , and transitions for the case of are shown in Figure 2. Proposition 3.1 above tells us that for , the stationary distribution of a random walk on this state space is given by the graph frequencies of . As we increase away from zero, we should therefore expect to see a stationary distribution that departs from precisely by observing more graphs with triangles and less graphs with open triangles.
The framework of our Edge Formation Random Walk makes it possible to model triadic closure precisely; in this sense the model forms an interesting contrast with other models of triangleclosing in graphs that are very challenging to analyze (e.g. [5, 12, 15, 21, 27]). We will now show how the addition of this single parameter makes it possible to describe the subgraph frequencies of empirical social graphs with remarkable accuracy.
Fitting subgraph frequencies
The stationary distribution of an Edge Formation Random Walk model describes the frequency of different graphs, while the coordinate system we are developing focuses on the frequency of node subgraphs within node graphs. For these two questions are in fact the same, since the distribution of random induced node subgraphs of is simply . When we introduce , however, our model departs from this symmetry, and the stationary probabilities in a random walk on node graphs is no longer precisely the frequencies of induced node subgraphs in a single node graph.
But if we view this as a model for the frequency of small graphs as objects in themselves, rather than as subgraphs of a larger ambient graph, the model provides a highly tractable parameterization that we can use to approximate the structure of subgraph frequencies observed in our families of larger graphs. In doing so, we aim to fit as a function of , where is the rate parameter that produces edge density for the specific value of . For this relationship is simply , but for the relation is not so tidy, and in practice it is easier to fit numerically rather than evaluate the expression.
When considering a collection of graph frequencies we can fit by minimizing residuals with respect to the model. Given a collection of graphs, let be the vectors of node subgraph frequencies for each graph and be the edge densities. We can then fit as:
In Figure 3 we plot the threenode subgraph frequencies as a function of edge density , for a collection of 300,000 50node subgraphs, again a balanced mixture of neighborhoods, groups, and events. In this figure we also plot (in yellow) the curve resulting from fitting our random walk model with triadic closure, , which is thus parameterized as a function of edge density . For this mixture of collections and , the optimal fit is . Notice how the yellow line deviates from the black curve to better represent the backbone of natural graph frequencies. From the figure it is clear that almost all graphs have more triangles than a sample from of corresponding edge density. When describing extremal bounds in Section 4, we will discuss how is in fact by no means the extremal lower bound.
As suggested by Figure 2, examining the subgraph frequencies for fournode subgraphs is fully tractable. In Figure 4, we fit to the mean subgraph frequencies of our three different collections of graphs separately. Note that the mean of the subgraph frequencies over a set of graphs is not necessarily itself a subgraph frequency corresponding to a graph, but we fit these mean 11vectors as a demonstration of the model’s ability to fit an ‘average’ graph. The subgraph frequency of at the edge density corresponding to the data is shown as a black dashed line in each plot — with poor agreement — and gray dashed lines illustrate an incremental transition in , starting from zero (when it corresponds to ) and ending at .
The striking agreement between the fit model and the mean of each collection is achieved at the corresponding edge density by fitting only . For neighborhood graphs, this agreement deviates measurably on only a single subgraph frequency, the fournode star. The yaxis is plotted on a logarithmic scale, which makes it rather remarkable how precisely the model describes the scarcity of the fournode cycle. The scarcity of squares has been previously observed in email neighborhoods on Facebook [28], and our model provides the first intuitive explanation of this scarcity.
The model’s ability to characterize the backbone of the empirical graph frequencies suggests that the subgraph frequencies of individual graphs can be usefully studied as deviations from this backbone. In fact, we can interpret the fitting procedure for as a variance minimization procedure. Recall that the mean of a set of points in is the point that minimizes the sum of squared residuals. In this way, the procedure is in fact fitting the ‘mean curve’ of the model distribution to the empirical subgraph frequencies.
Finally, our model can be used to provide a measure of the triadic closure strength differentially between graph collections, investigating the difference in for the subgraph frequencies of different graph collections. In Figure 4, the three different graph types resulted in notably different ratios of — the ratio of the triadic closure formation rate to the basic process rate — with a significantly higher value for this ratio in neighborhoods. We can interpret this as saying that open triads in neighborhoods are more prone to triadic closure than open triads in groups or events.
4 Extremal bounds
As discussed at the beginning of the previous section, we face two problems in analyzing the subgraph frequencies of real graphs: to characterize the distribution of values we observe in practice, and to understand the combinatorial structure of the overall space in which these empirical subgraph frequencies lie. Having developed stochastic models to address the former question, we now consider the latter question.
Specifically, in this section we characterize extremal bounds on the set of possible subgraph frequencies. Using machinery from the theory of graph homomorphisms, we identify fundamental bounds on the space of subgraph frequencies that are not properties of social graphs, but rather, are universal properties of all graphs. By identifying these bounds, we make apparent large tracts of the feasible region that are theoretically inhabitable but not populated by any of the empirical social graphs we examine.
We first review a body of techniques based in extremal graph theory and the theory of graph homomorphisms [17]. We use these techniques to formulate a set of inequalities on subgraph frequencies; these inequalities are all linear for a fixed edge density, an observation that allows us to cleanly construct a linear program to maximize and minimize each subgraph frequency within the combined constraints. In this manner, we show how it is possible to map outer bounds on the geography of all these structural constraints. We conclude by offering two basic propositions that transcend all edge densities, thus identifying fundamental limits on subgraph frequencies of all sizes.
4.1 Background on subgraph frequency and homomorphism density
In this subsection, we review some background arising from the theory of graph homomorphisms. We will use this homomorphism machinery to develop inequalities governing subgraph frequencies. These inequalities allow us to describe the outlines of the space underlying Figure 1(a) — the first step in understanding which aspects of the distribution of subgraph frequencies in the simplex are the result of empirical properties of human social networks, and which are the consequences of purely combinatorial constraints.
Linear constraints on subgraph frequency
Let denote the subgraph frequency of in , as defined in the last section: the probability that a random node subset of induces a copy of . Note that since is a probability over outcomes, it is subject to the law of total probability. The law of total probability for subgraph frequencies takes the following form.
Proposition 4.1
For any graph and any integer , where , the subgraph density of in , satisfies the equality
Let be a random vertex induced subgraph of . Now, the set of outcomes form a partition of the sample space, each with probability . Furthermore, conditional upon an vertex induced subgraph being isomorphic to , is the probability that a random vertex induced subgraph of is isomorphic to .
This proposition characterizes an important property of subgraph frequencies: the vector of subgraph frequencies on nodes exists in a linear subspace of the vector of subgraph frequencies on nodes. Furthermore, this means that any constraint on the frequency of a subgraph will also constrain the frequency of any subgraph for which or .
Graph homomorphisms
A number of fundamental inequalities on the occurrence of subgraphs are most naturally formulated in terms of graph homomorphisms, a notion that is connected to but distinct from the notion of induced subgraphs. In order to describe this machinery, we first review some basic definitions [4]. if and are labelled graphs, a map is a homomorphism if each edge of maps to an edge of . We now write for the probability that a random map from into is a homomorphism, and we refer to as a homomorphism density of and .
There are three key differences between the homomorphism density and the subgraph frequency defined earlier in this section. First, is based on mappings of into that can be manytoone — multiple nodes of can map to the same node of — while is based on onetoone mappings. Second, is based on mappings of into that must map edges to edges, but impose no condition on pairs of nodes in that do not form edges: in other words, a homomorphism is allowed to map a pair of unlinked nodes in to an edge of . This is not the case for , which is based on maps that require nonedges of to be mapped to nonedges of . Third, is a frequency among mappings from labeled graphs to labelled graphs , while is a frequency among mappings from unlabeled to unlabeled .
From these three differences, it is not difficult to write down a basic relationship governing the functions and [4]. To do this, it is useful to define the intermediate notion , which is the probability that a random onetoone map from to is a homomorphism. Since only an fraction of all maps from to are not onetoone, we have
(1) 
Next, by definition, a onetoone map of into is a homomorphism if and only if the image , when viewed as an induced subgraph of , contains all of ’s edges and possibly others. Correcting also for the conversion from labelled to unlabeled graphs, we have
(2) 
where is the number of automorphisms of and is the number of ways that a labelled graph can be extended (by adding edges) to form a labelled graph isomorphic to .
Homomorphism inequalities
There are a number of nontrivial results bounding the graph homomorphism density, which we now review. By translating these to the language of subgraph frequencies, we can begin to develop bounds on the simplexes in Figure 1.
For complete graphs, the KruskalKatona Theorem produces upper bounds on homomorphism density in terms of the edge density while the MoonMoser Theorem provides lower bounds, also in terms of the edge density.
Proposition 4.2 (KruskalKatona [17])
For a complete graph on nodes and graph with edge density ,
The MoonMoser bound is well known to not be sharp, and Razborov has recently given an impressive sharp lower bound for the homomorphism density of the triangle [24] using sophisticated machinery [23]. We limit our discussion to the simpler MoonMoser lower bound which takes the form of a concise polynomial and provides bounds for arbitrary , not just the triangle ().
Finally, we employ a powerful inequality that is known to lower bound the homomorphism density of any graph that is either a forest, an even cycle, or a complete bipartite graph. Stated as such, it is the solved special cases of the open Sidorenko Conjecture, which posits that the result could be extended to all bipartite graphs . We will use the following proposition in particular when is a tree, and will refer to this part of the result as the Sidorenko tree bound.
Proposition 4.4 (Sidorenko [17, 26])
For a graph that is a forest, even cycle, or complete bipartite graph, with edge set E(F), and with edge density ,
Using Equations (1) and (2), we can translate statements about homomorphisms into asymptotic statements about the combined frequency of particular sets of subgraphs. We can also translate statements about frequencies of subgraphs to frequencies of their complements using the following basic fact.
Lemma 4.5
If for graphs , coefficients , and a function ,
then
Note that . Thus if
then
where .
4.2 An LP for subgraph frequency bounds
In the previous section, we reviewed linear constraints between the frequencies of subgraphs of different sizes, and upper and lower bounds on graph homomorphism densities with applications to subgraph frequencies. We will now use these constraints to assemble a linear program capable to mapping out bounds on the extremal geography of the subgraph space we are considering. To do this, we will maximize and minimize the frequency of each individual subgraph frequency, subject to the constraints we have just catalogued.
We will focus our analysis on the cases , the triad frequencies, and , the quad frequencies. Let denote the subgraph frequencies of the four possible 3vertex undirected graphs, ordered by increasing edge count.
Program 4.6
The frequency of a 3node subgraph in any graph with edge density is bounded asymptotically (in ) by subject to and:
(3)  
(4)  
(5)  
(6)  
(7) 
Here the equalities in (3) derive from the linear constraints, the constraints in (4) derive from KruskalKatona, the constraints (56) derive from MoonMoser, and the constraints in (7) derive from the Sidorenko tree bound. More generally, we obtain the following general linear program that can be used to find nontrivial bounds for any subgraph frequency:
Program 4.7
The frequency of a node subgraph in any graph with edge density is bounded asymptotically (in ) by , subject to , appropriately assembled.
From Program 1 given above it is possible to derive a simple upper bound on the frequency of the 3nodepath (sometimes described in the social networks literature as the “forbidden triad”, as mentioned earlier).
Proposition 4.8
The subgraph frequency of the 3nodepath obeys .
Let again denote the subgraph frequencies of the four possible 3vertex undirected graphs, ordered by increasing edge count, where is the frequency of the 3nodepath. By the linear constraints,
while by MoonMoser, . Combining these two constraints we have:
The polynomial in is maximized at , giving an upper bound of . This bound on the “forbidden triad” is immediately apparent from Figure 5 as well, which shows the bounds constructed via linear programs for all 3node and 4node subgraph frequencies. In fact, the subgraph frequency of the ‘forbidden” 3nodepath in the balanced complete bipartite graph , which has edge density , is exactly , demonstrating that this bound is asymptotically tight. (In fact, we can perform a more careful analysis showing that it is exactly tight for even .)
Figure 5 illustrates these bounds for and . Notice that our empirical distributions of subgraph frequencies fall well within these bounds, leaving large tracts of the bounded area uninhabited by any observed dense social graph. While the bounds do not fully characterize the feasible region of subgraph frequencies, the fact that the bound is asymptotically tight at for the complete bipartite graph is important — practically no empirical social graphs come close to the boundary, despite this evidence that it is feasibly approachable. We emphasize that an exact characterization of the feasible space would necessitate machinery at least as sophisticated as that used by Razborov.
In the next subsection we develop two more general observations about the subgraph frequencies of arbitrary graphs, the latter of which illustrates that, with the exception of clique subgraphs and empty subgraphs, it is always possible to be free from a subgraph. This shows that the lower regions of the nonclique nonempty frequency bounds in Figure 5 are always inhabitable, despite the fact that social graphs do not empirically populate these regions.
4.3 Bounding frequencies of arbitrary subgraphs
The upper bound for the frequency of the 3nodepath given in Proposition 4.8 amounted to simply combining appropriate upper bounds for different regions of possible edge densities . In this section, we provide two general bounds pertaining to the subgraph frequency of an arbitrary subgraph . First, we show that any subgraph that is not a clique and is not empty must have a subgraph density bounded strictly away from one. Second, we show that for every subgraph that is not a clique and not empty, it is always possible to construct a family of graphs with any specified asymptotic edge density that contains no induced copies of .
With regard to Figures 5, the first of the results in this subsection uses the Sidorenko tree bound to show that in fact no subgraph other than the clique or the empty graph, not even for large values of , has a feasible region that can reach a frequency of . The second statement demonstrates that it is always possible to be free of any subgraph that is not a clique or an empty graph, even if this does not occur in the real social graphs we observe.
Proposition 4.9
For every , there exist constants and such that the following holds. If is a node subgraph that is not a clique and not empty, and is any graph on nodes, then .
Let denote the node star — in other words the tree consisting of a single node linked to leaves. By Equation (1), if has nodes, then for an absolute constant . We now state our condition on and in the statement of the proposition: we choose small enough and large enough so that
(8) 
For a node graph , let denote the property that for all graphs on at least nodes, we have . Our goal is to show that holds for all node that are neither the clique nor the empty graph. We observe that since , the property holds if and only if holds.
The basic idea of the proof is to consider any node graph that is neither complete nor empty, and to argue that the star lacks a onetoone homomorphism into at least one of or — suppose it is . The Sidorenko tree bound says that must have a nontrivial number of onetoone homomorphisms into ; but the images of these homomorphisms must be places where is not found as an induced subgraph, and this puts an upper bound on the frequency of .
We now describe this argument in more detail; we start by considering any specific node graph that is neither a clique nor an empty graph. We first claim that there cannot be a onetoone homomorphism from into both of and . For if there is a onetoone homomorphism from into , then must contain a node of degree ; this node would then be isolated in , and hence there would be no onetoone homomorphism from into . Now, since it is enough to prove that just one of or holds, we choose one of or for which there is no onetoone homomorphism from . Renaming if necessary, let us assume it is .
Suppose by way of contradiction that . Let denote the edge density of — that is, . The edge density of can be written, using Proposition 4.1, as
By a set of , we mean a set of nodes in . We color the sets of according to the following rule. Let be a set of : we color blue if is isomorphic to , and we color red if there is a onetoone homomorphism from to . We leave the set uncolored if it is neither blue nor red under these rules. We observe that no set can be colored both blue and red, for if it is blue, then is isomorphic to , and hence there is no onetoone homomorphism from into . Also, note that is equivalent to saying that at least a fraction of all sets are blue.
Finally, what fraction of sets are red? By the Sidorenko tree bound, we have
where the last inequality follows from the fact that is not the empty graph, and hence . Since , our condition on from (8) implies that
Now, let denote the number of onetoone homomorphisms of into ; by definition,
and hence
Now, at most different onetoone homomorphisms can map to the same set of , and hence more than many sets of are red. It follows that the fraction of sets that are red is ; but this contradicts our assumption that at least a fraction of sets are blue, since no set can be both blue and red.
Proposition 4.10
Assume is not a clique and not empty. Then for each edge density there exists a sequence of asymptotic edge density for which does not appear as an induced subgraph in any . Equivalently, .
We call a nearclique if it has at most one connected component of size greater than one, and this component is a clique. For any , it is possible to construct an infinite sequence of nearcliques with asymptotic density , by simply taking the nontrivial component of each to be a clique of the appropriate size.
Now, fix any , and let be any graph that is neither a clique nor an empty graph. If is not a nearclique, then the required sequence is the sequence of nearcliques , since all the induced subgraphs of a nearclique are themselves nearcliques.
On the other hand, if is a nearclique, then since is neither a clique nor an empty graph, the complement of is not a nearclique. It follows that the required sequence is the sequence of complements of the nearcliques .
Note that it is possible to take an free graph with asymptotic density and append nodes with local edge density and random (ErdősRényi) connections to obtain a graph with any intermediate subgraph frequency between zero and that of . The same blending arguement can be applied to any graph with a subgraph frequency above to again find graphs with intermediate subgraph frequencies. In this way we see that large tracts of the subgraph frequency simplex are fully feasible for arbitrary graphs, yet by Figure 5 are clearly not inhabited by any real world social graph.
5 Classification of audiences
The previous two sections characterize empirical and extremal properties of the space of subgraph frequencies, providing two complementary frameworks for understanding the structure of social graphs. In this section, we conclude our work with a demonstration of how subgraph frequencies can also provide a useful tool for distinguishing between different categories of graphs. The Edge Formation Random Walk model introduced in Section 3 figures notably, providing a meaningful baseline for constructing classification features, contributing to the best overall classification accuracy we are able to produce.
Thus, concretely our classification task is to take a social graph and determine whether it is a node neighborhood, the set of people in a group, or the set of people at an event. This is a specific version of a broader characterization problem that arises generally in social media — namely how social audiences differ in terms of social graph structure [1]. Each of the three graph types we discuss — neighborhoods, groups, and events — define an audience with which a user may choose to converse. The defining feature of such audience decisions has typically been their size — as users choose to share something online, do they want to share it publicly, with their friends, or with a select subgroup of their friends? Products such as Facebook groups exist in part to address this audience problem, enabling the creation of small conversation circles. Our classification task is essentially asking: do audiences differ in meaningful structural ways other than just size?
In Figure 1 and subsequently in Figure 5, we saw how the three types of graphs that we study — neighborhoods, groups, and events — are noticeably clustered around different structural foci in the space of subgraph frequencies. Figure 5 focused on graphs consisting of exactly 50nodes, where it is visibly apparent that both neighborhoods and events tend to have a lower edge density than groups of that size. Neighborhood edge density — equivalent to the local clustering coefficient — is known to generally decrease with graph size [20, 29], but it is not clear that all three of the graph types we consider here should decrease at the same rate.
In Figure 6, we see that in fact the three graph types do not decrease uniformly, with the average edge density of neighborhoods decreasing more slowly than groups or events. Thus, small groups are denser than neighborhoods while large groups are sparser, with the transition occurring at around 400 nodes. Similarly, small event graphs are denser than neighborhoods while large events are much sparser, with the transition occurring already at around 75 nodes.
The two crossing points in Figure 6 suggest a curious challenge: are their structural features of audience graphs that distinguish them from each other even when they exhibit the same edge density? Here we use the language of subgraph frequencies to formulate a classification task for classifying audience graphs based on subgraph frequencies. We compare our classification accuracy to the accuracy achieved when also considering a generous vector of much more sophisticated graph features. We approach this classification task using a simple logistic regression model. While more advanced machine learning models capable of learning richer relationships would likely produce better classification accuracies, our goal here is to establish that this vocabulary of features based on subgraph frequencies can produce nontrivial classification results even in conjunction with simple techniques such as logistic regression. Evaluating our features in other contexts such as graph matching [14, 16, 30], where frequencies of connected subgraphs have been used previously [25], would be interesting future work.
When considering neighborhood graphs, recall that we are not including the ego of the neighborhoods as part of the graph, while for groups and events the administrators as members of their graphs. As such, neighborhoods without their ego deviate systematicallly from analogous audience graphs created as groups or as events. In Figure 6 we also show the average edge density of neighborhoods with their ego, adding one node and edges, noting that the difference is small for larger graphs.
Classification features
Subgraph frequencies has been the motivating coordinate system for the present work, and will serve as our main feature set. Employing the Edge Formation Random Walk model from Section 3, we additionally describe each graph by its residuals with respect to a backbone — described by the parameter — fit to the complete unclassified training set.
Features based on subgraph frequencies are local features, computable by examining only a few local nodes of the graph at a time. Note that the subgraph frequencies of arbitrarily large graphs can be accurately approximated by sampling a small number of induced graphs. Comparatively, it is relevant to ask: can these simple local features do as well as more sophisticated global graph features? Perhaps the number of connected components, the size of the largest component, or other global features provide highly informative features for graph classification.
To answer this question, we compare our classification accuracy using subgraph frequencies with the accuracy we are able to achieve using a set of global graph features. We consider:

Size of the largest components, for .

Size of the core, for .

Number of components in the core, for .

Degeneracy, the largest for which the core is nonempty.

Size of the brace [28], for .

Number of components in the brace, for .
These features combine linearly to produce a rich set of graph properties. For example, the number of components in the core minus the number of components in the core yields the number of singletons in the graph.
Classification results
The results of the classification model are shown in Table 1, reported in terms of classification accuracy — the fraction of correct classifications on the test data – measured using fivefold crossvalidation on a balanced set of 10,000 instances. The classification tasks were chosen to be thwart classification based solely on edge density, which indeed performs poorly. Using only 4node subgraph frequencies and residuals, an accuracy of is achieved in both tasks.
In comparison, classification based on a set of global graph features performed worse, achieving just and accuracy for the two tasks. Meanwhile, combining global and subgraph frequency features performed best of all, with a classification accuracy of . In each case we also report the accuracy with and without residuals as features. Incorporating residuals with respect to either a or Edge Formation Random Walk baseline consistently improved classification, and examining residuals with respect to either baseline clearly provides a useful orientation of the subgraph coordinate system for empirical graphs.
Model Features  N vs. E,  N vs. G, 

Edges  0.487  0.482 
Triads  0.719  0.647 
Triads +  0.737  0.673 
Triads +  0.736  0.668 
Quads  0.751  0.755 
Quads +  0.765  0.769 
Quads +  0.765  0.769 
Global + Edges  0.694  0.763 
Global + Triads  0.785  0.766 
Global + Triads +  0.784  0.766 
Global + Triads +  0.789  0.767 
Global + Quads  0.797  0.812 
Global + Quads +  0.807  0.815 
Global + Quads +  0.809  0.820 
6 Conclusion
The modern study of social graphs has primarily focused on the examination of the sparse largescale structure of human relationships. This global perspective has led to fruitful theoretical frameworks for the study of many networked domains, notably the world wide web, computer networks, and biological ecosystems [20]. However, in this work we argue that the locally dense structure of social graphs admit an additional framework for analyzing the structure of social graphs.
In this work, we examine the structure of social graphs through the coordinate system of subgraph frequencies, developing two complementary frameworks that allow us to identify both ‘social’ structure and ‘graph’ structure. The framework developed in Section 3 enables us to characterize the apparent social forces guiding graph formation, while the framework developed in Section 4 characterizes fundamental limits of all graphs, delivered through combinatorial constraints. Our coordinate system and frameworks are not only useful for developing intuition, but we also demonstrate how they can be used to accurately classify graph types using only these simple descriptions in terms of subgraph frequency.
Distribution note
Implementations of the Edge Formation Random Walk equilibrium solver and the subgraph frequency bounds optimization program are available from the first author’s webpage.
Acknowledgments
We thank Peter Grassberger for helpful comments. This work was supported in part by NSF grants IIS0910664 and IIS1016099.
References
 [1] L. Adamic, J. Zhang, E. Bakshy, and M. S. Ackerman. Knowledge sharing and Yahoo Answers: everyone knows something. In WWW, pages 665–674. ACM, 2008.
 [2] L. Backstrom, E. Bakshy, J. Kleinberg, T. Lento, and I. Rosenn. Center of attention: how Facebook users allocate attention across friends. In ICWSM, 2011.
 [3] B. Bollobás. Random Graphs. Cambridge University Press, second edition, 2001.
 [4] C. Borgs, J. T. Chayes, L. Lovász, V. Sos, B. Szegedy, and K. Vesztergombi. Counting graph homomorphisms. In Topics in Discrete Mathematics, eds. M. Klazar, J. Kratochvil, M. Loebl, J. Matousek, R. Thomas, and P. Valtr, pages 315–371. Springer, 2006.
 [5] S. Chatterjee and P. Diaconis. Estimating and understanding exponential random graph models. arXiv preprint, arXiv:1102.2650v3, 2011.
 [6] J. Davis and S. Leinhardt. The structure of positive interpersonal relations in small groups. In Sociological Theories in Progress. Vol. 2, eds. J. Berger, M. Zelditch, and B. Anderson. HoughtonMifflin, 1971.
 [7] K. Faust. Very local structure in social networks. Sociological Methodology, 37(1):209–256, 2007.
 [8] K. Faust. A puzzle concerning triads in social networks: Graph constraints and the triad census. Social Networks, 32(3):221–233, 2010.
 [9] D. Fisher, M. A. Smith, and H. T. Welser. You are who you talk to: Detecting roles in usenet newsgroups. In HICSS, 2006.
 [10] M. Granovetter. The strength of weak ties. American Journal of Sociology, 78:1360–1380, 1973.
 [11] A. Inokuchi, T. Washio, and H. Motoda. An aprioribased algorithm for mining frequent substructures from graph data. In PKDD ’00, pages 13–23, 2000.
 [12] E. M. Jin, M. Girvan, and M. E. J. Newman. The structure of growing social networks. Phys. Rev. E, 64:046132, 2001.
 [13] M. Kuramochi and G. Karypis. An efficient algorithm for discovering frequent subgraphs. IEEE Trans. on Knowledge and Data Engineering, 16(9), 2004.
 [14] A. Khan, N. Li, X. Yan, Z. Guan, S. Chakraborty, and S. Tao. Neighborhood based fast graph search in large networks. In SIGMOD, pages 901–912, 2011.
 [15] J. Leskovec, J. Kleinberg, and C. Faloutsos. Graphs over time: densification laws, shrinking diameters and possible explanations. In KDD, pages 177–187. ACM, 2005.
 [16] G. Li, M. Semerci, B. Yener, and M. Zaki. Effective graph classification based on topological and label attributes. Statistical Analysis and Data Mining, 4(5):265–283, 2012.
 [17] L. Lovász. Very large graphs. In D. Jerison, B. Mazur, T. Mrowka, W. Schmid, R. Stanley, and S. T. Yau, editors, Current Developments in Mathematics, pages 67–128. International Press, 2009.
 [18] R. Milo, S. ShenOrr, S. Itzkovitz, N. Kashtan, D. Chklovskii, and U. Alon. Network motifs: simple building blocks of complex networks. Science, 298:824–827, 2002.
 [19] J. W. Moon and L. Moser. On a problem of Turan. Magyar Tud. Akad. Mat. Kutat Int. Kzl, 7:283–286, 1962.
 [20] M. E. J. Newman. Networks: An Introduction. Oxford University Press, 2010.
 [21] J. Park and M. E. J. Newman. Solution for the properties of a clustered network. Phys Rev E, 72(2):26136, 2005.
 [22] A. Rapoport. Spread of information through a population with sociostructural bias I: Assumption of transitivity. Bulletin of Mathematical Biophysics, 15(4):523–533, December 1953.
 [23] A. Razborov. Flag algebras. Journal of Symbolic Logic, 72:1239–1282, 2007.
 [24] A. Razborov. On the minimal density of triangles in graphs. Combinatorics, Probability and Computing, 17:603–618, 2008.
 [25] N. Shervashidze, S. Vishwanathan, T. Petri, K. Mehlhorn, and K. Borgwardt. Efficient graphlet kernels for large graph comparison. In AISTATS, 2009.
 [26] A. Sidorenko. A correlation inequality for bipartite graphs. Graphs and Combinatorics, 9:201–204, 1993.
 [27] D. Strauss. On a general class of models for interaction. SIAM Review, 28(4):513–527, 1986.
 [28] J. Ugander, L. Backstrom, C. Marlow, and J. Kleinberg. Structural diversity in social contagion. PNAS, 109(16):5962–5966, 2012.
 [29] J. Ugander, B. Karrer, L. Backstrom, and C. Marlow. The anatomy of the Facebook social graph. Technical Report cs.SI/1111.4503, arxiv, November 2011.
 [30] S. Vishwanathan, N. Schraudolph, R. Kondor, and K. Borgwardt. Graph kernels. Journal of Machine Learning Research, 99:1201–1242, 2010.
 [31] J. Voss. Measuring Wikipedia. In ICISSI, 2005.
 [32] S. Wasserman and K. Faust. Social Network Analysis: Methods and Applications. Cambridge Univ. Press, 1994.
 [33] X. Yan and J. Han. gpsan: Graphbased substructure pattern mining. In ICDM ’02, pages 721–724, 2002.