A Hierarchy of Lower Bounds for Sublinear Additive Spanners††thanks: Supported by NSF grants CCF-1217338, CNS-1318294, CCF-1417238, CCF-1514339, CCF-1514383, CCF-1637546, and BSF Grant 2012338. Email: firstname.lastname@example.org, email@example.com, firstname.lastname@example.org. A preliminary version of this paper will appear in the conference proceedings of SODA 2017.
Spanners, emulators, and approximate distance oracles can be viewed as lossy compression schemes that represent an unweighted graph metric in small space, say bits. There is an inherent tradeoff between the sparsity parameter and the stretch function of the compression scheme, but the qualitative nature of this tradeoff has remained a persistent open problem.
It has been known for some time that when there are schemes with constant additive stretch (distance is stretched to at most ), and recent results of Abboud and Bodwin show that when there are no such schemes. Thus, to get practically efficient graph compression with we must pay super-constant additive stretch, but exactly how much do we have to pay?
In this paper we show that the lower bound of Abboud and Bodwin is just the first step in a hierarchy of lower bounds that characterize the asymptotic behavior of the optimal stretch function for sparsity parameter . Specifically, for any integer , any compression scheme with size has a sublinear additive stretch function :
This lower bound matches Thorup and Zwick’s (2006) construction of sublinear additive emulators. It also shows that Elkin and Peleg’s -spanners have an essentially optimal tradeoff between and , and that the sublinear additive spanners of Pettie (2009) and Chechik (2013) are not too far from optimal. To complement these lower bounds we present a new construction of -spanners with size , where . This size bound improves on the spanners of Elkin and Peleg (2004), Thorup and Zwick (2006), and Pettie (2009). According to our lower bounds neither the size nor stretch function can be substantially improved.
Our lower bound technique exhibits several interesting degrees of freedom in the framework of Abboud and Bodwin. By carefully exploiting these freedoms, we are able to obtain lower bounds for several related combinatorial objects. We get lower bounds on the size of -hopsets, matching Elkin and Neiman’s construction (2016), and lower bounds on shortcutting sets for digraphs that preserve the transitive closure. Our lower bound simplifies Hesse’s (2003) refutation of Thorup’s conjecture (1992), which stated that adding a linear number of shortcuts suffices to reduce the diameter to polylogarithmic. Finally, we show matching upper and lower bounds for graph compression schemes that work for graph metrics with girth at least . One consequence is that Baswana et al.’s (2010) additive -spanners with size cannot be improved in the exponent.
Spanners , emulators [27, 57], and approximate distance oracles  can be viewed as kinds of compression schemes that approximately encode the distance metric of a (dense) undirected input graph in small space, where the notion of approximation is captured by a non-decreasing stretch function .
An -emulator (also called a Steiner spanner ) is a weighted graph such that for each , . In other words, one is allowed to add Steiner points () and long-range (weighted) edges such that distances are non-contracting.
- (Unconstrained) Distance Oracles.
For our purposes, an -approximate distance oracle using space is a bit string in such that given , an estimate can be computed by examining only the bit string. Note: the term “oracle” was used in  to indicate that is computed in constant time [44, 3, 21]. Later work considered distance oracles with non-constant query time [48, 5, 4, 31]. In this paper we make no restrictions on the query time at all. Thus, for our purposes distance oracles generalize spanners, emulators, and related objects.
In this paper we establish essentially optimal tradeoffs between the size of the compressed graph representation and the asymptotic behavior of its stretch function . In order to put our results in context we must recount the developments of the last 30 years that investigated multiplicative, additive, , and sublinear additive stretch functions.
1.1 Multiplicative Stretch
Historically, the first notion of stretch studied in the literature was purely multiplicative stretch. Althöfer et al.  quickly settled the problem by showing that any graph contains an -spanner with at most edges, and that the claim is false for . Here is the maximum number of edges in a graph with vertices and girth . The upper bound of  follows directly from the observation that a natural greedy construction never closes a cycle with length at most ; the lower bound follows from the fact that no strict subgraph of a graph with girth is an -spanner.111Removing any edge stretches the distance between its endpoints from 1 to at least . Moreover, since every graph contains a bipartite subgraph with at least half the edges, for every . Thus, there are -spanners with size . It has been conjectured [32, 17, 15] that the trivial upper bound is sharp up to the leading constant, but this Girth Conjecture has only been proved for (trivial), and [18, 33, 49, 60, 58, 12, 39]. See [40, 41, 61] for lower bounds on .
1.2 Additive Stretch
The Girth Conjecture implies that a spanner with size must stretch some pair of adjacent vertices at original distance to distance . If “stretch” is defined a priori to be multiplicative, then such -spanners are optimal. However, there is no reason to believe that is an optimal stretch function for size . The girth argument could also be interpreted as lower bounding additive stretch or -stretch. In general, the Girth Conjecture only implies that -spanners with size have .
Aingworth, Chekuri, Indyk, and Motwani  gave a construction of an additive -spanner with size , which is optimal in the sense that neither the additive stretch nor exponent can be unilaterally improved.222Moreover, later results of Bollobás et al.  show that for spanner size , the stretch function is optimal for . See [30, 57, 9, 38] for constructions of additive 2-spanners with size . This result raised the tantalizing possibility that there exist arbitrarily sparse additive spanners. Dor, Halperin, and Zwick  observed that additive -emulators exist with size , i.e., the emulator introduces weighted edges connecting distant vertex pairs. Baswana, Kavitha, Mehlhorn, and Pettie  constructed additive 6-spanners with size and Chechik  constructed additive-4 spanners with size . See [62, 38, 30, 57, 27, 9] for other constructions of additive 2- and 6-spanners.
The “” exponent proved to be very resilient, for both emulators and spanners with additive stretch. This led to a line of work establishing additive spanners below the threshold with stretch polynomial in [16, 9, 47, 20, 13]. The additive spanners of Bodwin and Williams  with stretch function have size that is the minimum of and .
1.3 Sublinear Additive Stretch
Elkin and Peleg  showed that the “4/3 barrier” could also be broken by tolerating multiplicative stretch. In particular, for any integer and real , there are -spanners with size , where . The construction algorithm and size-bound both depend on . Thorup and Zwick  gave a surprisingly simple construction of an -size emulator with -type stretch.
Thorup and Zwick’s emulator has the special property that its stretch holds for every simultaneously, i.e., it can be selected as a function of . Judiciously choosing leads to an emulator with a sublinear additive stretch function .333The Thorup-Zwick emulator can easily be converted to a -spanner by replacing weighted edges with paths up to length . A careful analysis shows the size of the resulting spanner can be made (see Section 3) which would slightly improve on . Elkin [personal communication, 2013] has stated that with minor changes, the Elkin-Peleg  spanners can also be expressed as -spanners with size . We state these bounds in Figure 1 rather than those of  in order to facilitate easier comparisons with subsequent constructions [57, 20, 47], and the new constructions of Section 3. Thorup and Zwick also showed that this same stretch function also applies to their earlier  construction of multiplicative -spanners with size . Pettie  gave a construction of sublinear additive spanners whose size-stretch tradeoff is closer to the Thorup-Zwick emulators. For stretch function the size is , which is always for any fixed . At their sparsest, Thorup and Zwick’s emulators  and Pettie’s spanners  have size and stretch . Pettie  gave an even sparser -spanner with size .
1.4 Lower Bounds
Woodruff proved that any -size spanner with stretch function must have . As a corollary, additive -spanners must have size , independent of the status of the Girth Conjecture. Bollobás, Coppersmith, and Elkin  showed that if the stretch is such that for , then -size is necessary and sufficient for spanners and emulators.
In a recent surprise, Abboud and Bodwin  proved that no additive -spanners, emulators, nor distance oracles exist with and exponent less than . More precisely, any construction of these three objects with additive stretch has size and any construction with size has additive stretch for some . This result explained why all prior additive spanner constructions had a strange transition at [27, 57, 9, 20, 14, 38, 62], but it did not suggest what the optimal stretch function should be for sparsity when .
|Elkin & Peleg Span.|
|& Zwick Span.|
1.5 New Results
Distance Oracle Lower Bounds.
Our main result is a hierarchy of lower bounds for spanners, emulators, and distance oracles, which shows that tradeoffs offered by Thorup and Zwick’s  sublinear additive emulators  and Elkin and Peleg’s -spanners cannot be substantially improved. Building on Abboud and Bodwin’s  lower bounds for additive spanners, we prove that for every integer and , there is a graph on vertices and edges such that any spanner with size , , stretches vertices at distance to at least for a constant . More generally, we exhibit graph families that cannot be compressed into distance oracles on bits such that distances can be recovered below this error threshold. The consequences of this construction are that the existing sublinear additive emulators , sublinear additive spanners [47, 20], and -spanners [30, 57, 47] are, to varying degrees, close to optimal. Specifically,
The -emulator  with size cannot be improved by more than a constant factor in the stretch , or by a in the exponent .
When , the existing constructions of -spanners [30, 57, 47] with size cannot be substantially improved in either the additive term or the exponent . This follows from the fact that any spanner with stretch of type , for every functions as a -spanner for distances . However, there is no reason to believe that the size of such -spanners must depend on , as it does in the current constructions.
There is an interesting new hierarchy of phase transitions in the interplay between our lower bounds previous upper bounds . Let be a sufficiently large constant and be a sufficiently small constant. If one wants a graph compression scheme with stretch , then one needs only bits of space to store an emulator . However, if we want a slightly improved stretch , then, by our lower bound, the space requirement leaps to . In general, the optimal space for stretch function takes a polynomial jump as we shift from some sufficiently large constant to a sufficiently small constant .
An important take-away message from our work is that the sublinear additive stretch functions of type used by Thorup and Zwick  are exactly of the “right” form. For example, such plausible-looking stretch functions as and could only exist in the narrow bands not covered by our lower bounds: between space and and between space and .
Spanner Upper Bounds.
To complement our lower bounds we provide new upper bounds on the sparsity of spanners with stretch of type , which holds for every . Our new spanners have size , where . This construction improves on the bounds that can be derived from [57, 30, 47] in the dependence on .444No bounds of this type are stated explicitly in  or . In order to get a bound of this type—with the exponent and some dependence on — one must only adjust the sampling probabilities of ; however, adapting  requires slightly more significant changes [Elkin, personal communication, 2013]. For example, one consequence of this result is an -size spanner that functions as a -spanner for all . This size bound is an improvement on Chechik’s -spanner, as long as .
Hopset Lower Bounds.
Hopsets are fundamental objects that are morally similar to emulators. They were explicitly defined by Cohen  but used implicitly in many earlier works [59, 37, 22, 51]. Let be an arbitrary undirected weighted graph and be a set of edges called the hopset. In the united graph , the weight of an edge is the length of the shortest path in between and . Define the -limited distance in , denoted , to be the length of the shortest path from to that uses at most edges in .555Note that whereas is metric, does not necessarily satisfy the triangle inequality for finite . We call a -hopset, where , if, for any , we have
There is clearly some three-way tradeoff between and . Elkin and Neiman  recently showed that any graph has a -hopset with size , where .666It is likely that Elkin and Neiman’s tradeoff could be more precisely stated as follows: for any positive integer and , there is an size -hopset with .
In this work, we show that any construction of -hopsets with worst-case size , where is an integer and , must have . For example, hopsets with must have size and those with must have size . This essentially matches the Elkin-Neiman tradeoff, up to a constant in that depends on .
Lower Bounds on Shortcutting Digraphs.
In 1992, Thorup  conjectured that the diameter of any directed graph could be drastically reduced with a small number of shortcuts. In particular, there exists another directed graph with and the same transitive closure relation as (), such that if , then there is a -length path from to in . Thorup’s conjecture was confirmed for trees [53, 55, 19] and planar graphs , but finally refuted by Hesse  for general graphs. In this paper we give a simpler 1-page proof of Hesse’s refutation by modifying our spanner lower bound construction.
Spanners for High-Girth Graphs.
Our lower bounds apply to the class of all undirected graph metrics. Baswana, Kavitha, Mehlhorn, and Pettie  gave sparser spanners for a restricted class of graph metrics. Specifically, graphs with girth at least contain additive -spanners with size . We adapt our lower bound construction to prove that the exponent is optimal, assuming the Girth Conjecture, and more generally we give lower bounds on compression schemes for the class of graphs with girth at least . Any scheme that uses bits must have stretch , for any . We also give new constructions of emulators and spanners for girth- graphs that shows that the exponent is the best possible.
1.6 Related Work
Much of the recent work on spanners has focused on preserving or approximating distances between specified pairs of vertices. See [25, 2, 1] for lower bounds on pairwise spanners and [25, 47, 26, 36, 35, 2, 43, 50] for upper bounds. Pairwise spanners have proven to be useful tools for constructing (sublinear) additive spanners; see [47, 20, 13].
The space/stretch tradeoffs offered by the best distance oracles [21, 44, 45, 3, 5, 4, 31] are strictly worse than those of the best spanners and emulators, even though distance oracles are entirely unconstrained in how they encode the graph metric. This is primarily due to the requirement that distance oracles respond to queries quickly. There are both unconditional  and conditional [24, 44, 45] lower bounds suggesting that distance oracles with reasonable query time cannot match the best spanners or emulators.
In Section 2 we generalize Abboud and Bodwin’s construction  to give a spectrum of lower bounds against graph compression schemes with sublinear additive stretch and -stretch. In Section 3 we combine ideas from Thorup and Zwick’s emulators  and Pettie’s spanners  to attain a new bound on sparse -spanners. In Section 4 we prove tight bounds on -hopsets. In Section 5 we generalize the construction of Section 2 to give stretch-sparseness lower bounds on the class of graphs with girth at least . Matching upper bounds for graphs of gith are given in Section 5.1. In Section 6 we give a simpler refutation of Thorup’s shortcutting conjecture. In Section 7 we highlight some remaining open problems.
2 The Lower Bound Construction
The graphs in this section are parameterized by an integer , which determines the length of the hardest shortest paths to approximate. Each graph has a layered structure, consisting of a layer of input ports, some number of interior layers, and a layer of output ports. In any given graph construction, is the number of input/output ports. The construction of , , and is essentially the same as the graphs constructed by Abboud and Bodwin .
2.1 The First Base Graph
Let be an -layer graph with the following properties:
has vertices per layer, and all edges connect vertices in adjacent layers.
Each edge is assigned a . For any vertex , the edges connecting to the previous layer have distinct labels and the edges connecting to the subsequent layer have distinct labels.
Let be a set of pairs of input/output ports. Each has the property that there exists a unique shortest path . Moreover, , any two of these paths are edge disjoint, and the edge set is precisely the union of these paths over all pairs in .
These properties imply that the number of vertices and edges in is and .
Refer to [7, 1] for constructions of satisfying these requirements, or to  for a construction without the layered structure. For the sake of completeness we give a short sketch of how is constructed using average-free sets [7, 1]. Let be an -average-free set, i.e., one for which the equation
|has no solutions, except the trivial . Let denote the th vertex in . The edge set consists of|
|with . The pair set consists of|
The -average free property of ensures that is the unique shortest path between its endpoints.
2.2 The Second Base Graph
Roughly speaking, is obtained by taking a certain product of two copies of .777Here we let be short for . Ignoring issues of integrality only introduces factors in all the bounds. Let and be the vertex sets of copies and , each with respective pair-sets and . is a layered graph with vertex set where when is even and when is odd. Vertices in are identified with vertex pairs from . When is even, an edge exists between layers and iff . Similarly, when is odd, an edge exists between layers and iff . An edge in inherits the label of the corresponding edge in , so the label set for is . The pair-set for is defined to be
Observe that any length- path from layer to corresponds to picking edges alternately from two paths, one from to in and one from to in . Lemma 2.1 summarizes the relevant properties of and .
Let be a non-decreasing function of such that , , and . The graph has the following properties.
It has vertices and edges.
The vertices of each pair in are connected by a unique shortest path in , whose edge labels alternate between two labels in .
By definition, .
Part 1. Each layer of contains vertices; there is no harm in adding dummy vertices to round it up to . There are at least edges in each of and , and each edge of is duplicated times in the construction of . Parts 2,3. Follows directly from the construction of , and that has unique shortest paths in . ∎
2.3 A Recursive Construction
In this section we construct a hierarchy of hard graphs and corresponding pair-sets such that each pair in has a unique shortest path in . We will show that, for any and sufficiently small constant , any spanner of with stretch function must include at least edges. Each is a layered graph with input ports, output ports, and some number of interior layers. In other words, the first layer (“input ports”) and last layer (“output ports”) have size each while the interior layers may have different sizes, and each node pair in is composed of one input port and one output port. Let denote the graph with the same topology as but with layers reversed; that is, the roles of input and output ports are swapped.
The Base Case.
The base case graph is a complete bipartite graph on vertices and its corresponding pair-set has size .
The Inductive Case.
Let us first give a very informal overview of the construction, then discuss how we plan to prove its correctness. The goal is to produce a new graph that contains within it many copies of . The shortest path for each joins an input port to an output port , in , and meanders through many copies of . When goes through a copy of it enters and exits it at a particular input/output port pair, say . We hope that (a success); if this holds for all the copies of intersected by then any aggressive sparsification of these copies will introduce a significant additive error in each copy. Unfortunately, while is large, it is not that large. Only a tiny -fraction of the set of input/output port pairs of appear in . Thus, if walks into and out of each through random ports, it is likely to miss the pairs in (a failure).
The problem with this approach is not the random assignment of input/output ports but the independence across
copies of . We solve this problem by correlating the success or failure events associated with .
That is, we ensure that either enters/leaves every copy of along a pair in ,
or it enters/leaves no copy of using a pair in .
Thus, many of the potential pairs are useless and may be discarded, but some of the pairs
must accumulate lots of error at each copy of that touches.
We now give this argument in more formality. When we construct from and as follows. Let the label-set of be and . Let be the standard and reversed copies of and be a port assignment permutation selected uniformly at random.
Recall that consists of layers . Layers and become the input and output ports of and are left as-is. For each vertex in an interior layer , we replace with a graph , which is a copy of if is odd and if is even. For each former edge in with , we replace it with a path of length connecting the th output port of (or leave it at if ) and the th input port of (or leave it at if .) The resulting graph is ; see Figure 2 for a diagram. It remains to define the new pair-set .
Let be one of the pairs in , and suppose the edges on the unique shortest path from to alternate between labels ‘’ and ‘’. The corresponding path in passes through some , where are copies of and are copies of .
By construction, enters at the input port and leaves at the output port, if is odd, or the reverse if is even. Up to reversal, the input/output terminals through each are identical, for all . The pair-set consists of all (whose unique shortest path in is labeled with, say, ) for which .
The expected size of is . Assuming is a nondecreasing function for all , the expected size of is on the order of .
By definition of and the construction of , there are candidate pairs in , each of which, say , is associated with two alternating labels, say and . Since a uniformly random input/output pair is in with probability