Arithmetic expanders and deviation bounds for sums of random tensors

# Arithmetic expanders and deviation bounds for sums of random tensors

## Abstract

We prove hypergraph variants of the celebrated Alon–Roichman theorem on spectral expansion of sparse random Cayley graphs. One of these variants implies that for every prime and any , there exists a set of directions of size such that for every set of density , the fraction of lines in with direction in is within of the fraction of all lines in . Our proof uses new deviation bounds for sums of independent random multi-linear forms taking values in a generalization of the Birkhoff polytope. The proof of our deviation bound is based on Dudley’s integral inequality and a probabilistic construction of -nets. Using the polynomial method we prove that a Cayley hypergraph with edges generated by a set as above requires for (our notion of) spectral expansion for hypergraphs.

j.briet@cwi.nl J. B. was supported by a VENI grant from the Netherlands Organisation for Scientific Research (NWO)

rao@cims.nyu.edu This material is based upon work supported by the National Science Foundation Graduate Research Fellowship Program under Grant No. DGE-1342536.

## 1Introduction

In the following all graphs are undirected and may have loops and parallel edges. For an -vertex graph and denote by the number of edges connecting and . If is -regular then its normalized adjacency matrix is given by . Let be the eigenvalues of arranged in decreasing order and denote .

### 1.1Spectral expanders.

Spectral expanders are infinite families of graphs of size increasing with such that the spectral gap is at least some that is independent of . A single graph is said to be an expander if it is tacitly understood to belong to such a family. Spectral expansion, the property of having large spectral gap, occurs in random graphs have with high probability. Seminal work on quasirandomness of Thomason [35], and Chung, Graham, and Wilson [10] showed that for dense graphs, this property is equivalent to a number of other likely features of random graphs. One of these is expansion, a measure of connectedness showing that no large set of vertices can be disconnected from its complement by cutting only a few edges. Another is discrepancy, which refers the property that the edge density of any sufficiently large induced subgraph is close to the overall edge density.

A long line of research extending the results of [10] to dense hypergraphs was initiated by Chung and Graham [8], culminating in recent work of Lenz and Mubayi [19] (which we refer to for a more detailed account). Partially motivated by an application in Theoretical Computer Science concerning special types of error-correcting codes (locally decodable codes) [3], we study the extent to which some known results on sparse expanders generalize to hypergraphs. Along the way we establish a new deviation inequality for sums of independent random multi-linear forms (Theorem ?) that we hope will find applications elsewhere.

### 1.2Cayley graphs and the Alon–Roichman Theorem.

Most known examples of sparse expanders are Cayley graphs, which are defined as follows. For a finite group and an element , the Cayley graph is the 2-regular graph with vertex set and edge set , where in case , all edges are doubled. For a multiset1 , the Cayley graph is the -regular graph formed by the union of the graphs .

The group over which Cayley graphs are defined strongly influences the minimal degree required for spectral expansion. The famous examples of constant-degree expanders of Margulis [?] and Lubotzky, Phillips, and Sarnak [21] are Cayley graphs which, crucially, are defined over non-Abelian groups. It is easy to see that a Cayley graph over the Abelian group , for example, requires degree at least to be an expander [2].

Similarly, because expanders must be connected, it follows that spectral expansion requires degree in any Cayley graph over any -element Abelian group [15]. A celebrated result of Alon and Roichman [2], however, shows that Abelian groups are extreme in this sense.

Our main results are hypergraph versions of Proposition ? and Theorem ?.

### 1.3Hypergraphs

A -uniform hypergraph with vertex set has as edge set a family of unordered -element multisets with possible parallel edges. For let denote the number of edges equal to . The adjacency form of is the -linear form defined by . The degree of a vertex is defined by and is -regular if every vertex has degree exactly , in which case its normalized adjacency form is . Of particular importance here are hypergraphs whose edge set is given by a multiset of the form , , where are permutations on . In this case we set

where runs over all permutations of , giving a -regular hypergraph.

### 1.4Hypergraph spectral expansion.

To define spectral expansion for hypergraphs we build on the following characterisation of . Recall that the Schatten- norm (or spectral norm) of a matrix is given by . If is symmetric, then this norm is precisely the maximum absolute value of the eigenvalues of . Since for an -vertex graph , the eigenvector associated with the first eigenvalue is the normalized all-ones vector , we have , where is the all-ones matrix. Our definition of spectral expansion for hypergraphs is based on the following norm on multilinear forms. For a -linear form on and define

The notion of spectral expansion we shall use is relative to a fixed regular -uniform hypergraph . In particular, for a regular -uniform hypergraph , we define

For graphs, this parameter coincides with if is the complete graph with all loops.

### 1.5Cayley hypergraphs.

A Cayley hypergraph over a finite group is a disjoint union of particular permutation hypergraphs as mentioned in Section 1.3. Let be an integer vector such that no element of has order for every . This ensures that for every , the maps are permutations. For , we define to be the hypergraph as in Section 1.3 based on the permutations . For a multiset , we let be the -regular hypergraph given by the union of for .
To connect the above definitions, consider a Cayley hypergraph . For a subset , let be a sub-hypergraph of and let . Then, for every set of density , we have . Dividing by shows that the fraction of edges that induces in is within of the fraction of edges it induces in .

### 1.6Translation invariant equations

To motivate the above definitions we focus on a special class of Cayley hypergraphs that arises from systems of translation invariant equations. Such a system can be given in terms of a matrix and a vector such that . For an Abelian group without elements of order for every , we then consider the set of solutions in to the linear equations defined by ,

There is a large body of literature on the problem of bounding the maximum size of a set such that contains only trivial solutions. Well-studied examples involving a single equation (where ) include Sidon sets [26], where , and sets without 3-term arithmetic progressions (APs), where (sometimes referred to as cap sets) [27]. Sets avoiding a general -variate translation invariant equation were studied in [28]. Probably the most-studied examples involving more than one equation are -term APs [31], where

Translation invariance refers to the fact that for every in and every , the tuple belongs to as well. As such, is a union of cosets of the subgroup . If is a set of representatives of these cosets, then the edge set of the hypergraph is furnished precisely by the (unordered) tuples in , which leads to the following definition.

The preceding discussion shows that an arithmetic expander has the property that for every set of density , the fraction of solutions in among the cosets represented by is within of the fraction of all solutions in . For APs, this means the following. The matrix as in satisfies for , from which it follows that consists of cosets represented by APs through zero, , which correspond to the possible steps that an AP can take. In this case, an arithmetic expander is thus characterized by a small set of steps such that the fraction of APs in any set taking steps from gives an accurate estimate of the fraction of all APs in . The AP matrix also satisfies for , from which it follows that consists of the cosets with representatives given by the points through which -term APs travel, . In this case, an arithmetic expander thus estimates the fraction of all APs by the fraction of APs travelling through a small fixed set of points.

## 2Our results

### 2.1Spectral expansion of Cayley hypergraphs.

Our first result is an extension of Proposition ? concerning arithmetic expanders for -APs where is a prime.

Our second result is a version of Theorem ?, showing for instance that in the AP case, for as in , there exist -arithmetic expanders of size for both options of , where depends on and only.

### 2.2A deviation bound for sums of random tensors

Our proof of Theorem ? follows similar lines as a slick proof of Theorem ? due to Landau and Russel [22]. Their proof is based on a matrix-valued deviation inequality called the matrix-Chernoff bound. One can also use the following matrix version of the Hoeffding bound, which follows from a non-commutative Khintchine inequality of Tomczak-Jaegermann [37] (see Appendix A) and which is more in line with the tools we shall use below.

The proof of Theorem ? is similarly based on a new deviation bound for multi-linear forms that belong to a generalization of the Birkhoff polytope (of doubly-stochastic matrices). To define this polytope, we first consider the following generalization of a doubly-stochastic matrix. Let be the standard basis vectors and let denote the all-ones vector. A -linear form on is plane sub-stochastic if is nonnegative on the standard basis vectors and if for every , we have

Let be the polytope of -linear forms on such that the form defined by for , is plane sub-stochastic. Observe that the set is the set of matrices such that is doubly sub-stochastic.2 Our deviation bound then is as follows.

For example, for , we have , and . The proof of Theorem ? is now nearly identical to the proof of the Alon–Roichman theorem shown above.

### Open problems

Our results leave open the problem of determining the minimal degree required for spectral expansion of random Cayley hypergraphs. Remark ? could be interpreted as suggesting the intriguing possibility that, in stark contrast with the Alon–Roichman Theorem, this degree must be quasi-polynomial in the size of the group. Another problem is to determine the optimal form of Theorem ?. Finally, it is open if the straightforward generalization of the Expander Mixing Lemma given in Proposition ? below admits a converse for Cayley hypergraphs. A converse was shown to hold for Cayley graphs by Kohayakawa, Rödl, and Schacht [16] and Conlon and Zhao [12].

### Acknowledgements

J. B. would like to thank Jozef Skokan for pointing him to [19] and Zeev Dvir and Sivakanth Gopi for helpful discussions.

## 3Proof of Theorem

In this section we prove Theorem ?. To rephrase this result, consider for a set the Cayley hypergraph

Then, by Definition ?, Theorem ? says that for every of size , the hypergraphs and satisfy . The first ingredient of the proof is the following straightforward generalization of the Expander Mixing Lemma [1], which follows directly from the above definitions.

To prove the theorem it thus suffices to show that for every of size , there exist such that on the one hand, , while on the other hand, , which is precisely what we shall do with sets satisfying . We achieve this by constructing a combinatorial rectangle that contains many lines, but no lines with direction in , by which we mean the following. Define the line through in direction , denoted , to be the sequence . Say that contains if for every . Denote by the number of lines contained in that have direction . The following proposition shows why considering lines through rectangles suffices.

Theorem ? will thus follow from the following result.

The proof of Theorem ? uses the polynomial method. For the remainder of this section let . For an -variate polynomial denote .

The following basic and standard result (see for example [34]) shows that for any small set , we can always find a low-degree homogeneous polynomial such that .

We also use the following standard result bounding the zero-set of a polynomial in terms of its degree; the specific form quoted below is from [11].

Finally, we use the Chevalley–Warning Theorem to lower bound the number of common zeros of a system of polynomials [20].

We include a quick proof we learned from Dion Gijswijt, which is based on Lemma ?.

With this, we are set up to prove Theorem ?.

## 4Proof of Theorem

In this section we prove Theorem ?. Throughout this section, let be independent uniformly distributed -valued random variables and let .

### 4.1Reduction to Bernoulli processes

The main new ingredient needed for the proof of Theorem ? is a bound showing that for fixed , the expected norm of the Rademacher sum is at most a constant times . From this, we derive the result using standard techniques based on combining a symmetrization trick, the Kahane–Khintchine inequality and an exponential Markov inequality. The details follow below. Recall that a real-valued random variable is centered if it has expectation zero.

The following standard symmetrization lemma allows us to bound the moments of the random variable whose tail we aim to bound in in terms of the moments of the norm of a Rademacher sum of fixed plane sub-stochastic forms.

Next, the Kahane–Khintchine inequality reduces the problem of bounding the numbers from Lemma ? to bounding only (see for example [23]).

Lemma ? and Theorem ? thus show that the moments on the left-hand side of can be bounded in terms of the average on the right-hand side of . The following upper bound and a standard exponential Markov argument will now allow us to prove Theorem ?.

The remainder of this section is devoted to the proof of Theorem ?.

The first step towards proving Theorem ? is to break the problem up into more manageable pieces using the following lemma. For every define the set

The above lemma thus reduces the problem of bounding the expectations of Theorem ? to bounding each of the expectations appearing in the right-hand side of . The following lemma provides the bounds we need.

### 4.3Dudley’s integral inequality

To prove Lemma ? we use Dudley’s integral inequality (see for example [32]), which bounds the expected supremum of a stochastic process endowed with a metric space structure in terms of covering numbers. For a metric space and , an -net is a subset such that for every there exists an with distance and the covering number is the smallest integer such that admits an -net of size . The diameter of a metric space is given by .

The following set is relevant to our setting:

For each consider the (centered) random variable , so that the left-hand side of equals . Moreover, for every and , we have

where the second line follows from Hoeffding’s inequality [4]. We shall therefore consider the metric space . For our setting, the relevant form of Dudley’s inequality is then as follows.

To apply the above result we need a bound on the diameter of and its covering numbers.

### 4.4Bounds on the covering numbers

To bound the covering numbers of we use a technique akin to Maurey’s empirical method for bounding , the covering numbers of the unit ball of under (see for instance [9]). In particular, for every -tuple as in , we use the probabilistic method to show that there exists another -tuple of vectors such that each is a “sparse” version of . By this we mean that it has few nonzero entries, each of which has relatively small magnitude, and such that is close to in Euclidean distance. This implies that there exists a net composed of all points such that is sparse and that the covering numbers can be bounded by the number of -tuples of sparse vectors. The sparse vectors themselves are obtained by taking the empirical average of independent samples from a signed and scaled standard basis vector whose average equals . Quantitatively, we get the following lemma.

## AProof of Theorem

For completeness, we derive Theorem ? (the matrix Hoeffding bound) from the following special case of a result of Tomczak-Jaegermann [37].

### Footnotes

1. We use curly brackets to delimit multisets: unordered lists that may contain repeated elements.
2. Recall that the Birkhoff–von Neumann Theorem states that the Birkhoff polytope is the convex hull of the set of permutation matrices. In [17] it is shown that for , the natural analogue of this fails for the set of forms in that attain equalities in and are nonnegative on standard basis vectors.

### References

1. Explicit construction of linear sized tolerant networks.
N. Alon and F. R. Chung. Annals of Discrete Mathematics
2. Random Cayley graphs and expanders.
N. Alon and Y. Roichman. Random Structures & Algorithms
3. Outlaw distributions and locally decodable codes.
J. Briët, Z. Dvir, and S. Gopi. In Proceedings of the 8th Annual Innovations in Theoretical Computer Science conference (ITCS 2017). 2017.
4. Concentration inequalities

S. Boucheron, G. Lugosi, and P. Massart. .
5. Translation invariant equations and the method of Sanders.
T. F. Bloom. Bulletin of the London Mathematical Society
6. Locally decodable codes and the failure of cotype for projective tensor products.
J. Briët, A. Naor, and O. Regev. Electronic Research Announcements in Mathematical Sciences (ERA-MS)
7. On embeddings of from locally decodable codes.
J. Briët. Electronic Colloquium on Computational Complexity (ECCC)
8. Quasi-random hypergraphs.
F. R. K. Chung and R. L. Graham. Random Structures & Algorithms
9. Interactions between compressed sensing random matrices and high dimensional geometry

D. Chafaï, O. Guédon, G. Lecué, and A. Pajor. , volume 37 of Panoramas et Synthèses [Panoramas and Syntheses].
10. Quasi-random graphs.
F. R. K. Chung, R. L. Graham, and R. M. Wilson. Combinatorica
11. Two structural results for low degree polynomials and applications.
G. Cohen and A. Tal. arXiv preprint arXiv:1404.0654
12. Quasirandom Cayley graphs.
D. Conlon and Y. Zhao. preprint
13. On large subsets of with no three-term arithmetic progression.
J. S. Ellenberg and D. Gijswijt. arXiv preprint arXiv:1605.09223
14. Montréal notes on quadratic Fourier analysis.
B. Green. In Additive combinatorics, volume 43 of CRM Proc. Lecture Notes, pages 69–102. Amer. Math. Soc., Providence, RI, 2007.
15. Expander graphs and their applications.
S. Hoory, N. Linial, and A. Wigderson. Bull. Amer. Math. Soc.
16. Discrepancy and eigenvalues of Cayley graphs.
Y. Kohayakawa, V. Rödl, and M. Schacht. preprint
17. On the vertices of the -dimensional Birkhoff polytope.
N. Linial and Z. Luria. Discrete Comput. Geom.
18. Eigenvalues and linear quasirandom hypergraphs.
J. Lenz and D. Mubayi. In Forum of Mathematics, Sigma, volume 3. Cambridge Univ Press, 2015.
19. The poset of hypergraph quasirandomness.
J. Lenz and D. Mubayi. Random Structures Algorithms
20. Finite fields.
R. Lidl and H. Niederreiter. In G.-C. Rota, editor, Finite Fields, volume 20 of Encyclopedia of Mathematics and its Applications. Addison-Wesley, Reading, Massachusetts, 1983.
21. Ramanujan graphs.
A. Lubotzky, R. Phillips, and P. Sarnak. Combinatorica
22. Random Cayley graphs are expanders: a simplified proof of the Alon-Roichman theorem.
Z. Landau and A. Russell. Electron. J. Combin.
23. Probability in Banach Spaces

M. Ledoux and M. Talagrand. .
24. Explicit constructions of expanders.
G. A. Margulis. Problemy Peredači Informacii
25. Explicit group-theoretical constructions of combinatorial schemes and their application to the design of expanders and concentrators.
G. A. Margulis. Problems of Information Transmission
26. A complete annotated bibliography of work related to Sidon sequences.
K. O’Bryant. Electron. J. Combin
27. Sets of integers that do not contain long arithmetic progressions.
K. O’Bryant. Electron. J. Combin.
28. Solving a linear equation in a set of integers. I.
I. Z. Ruzsa. Acta Arith.
29. On Roth’s theorem on progressions.
T. Sanders. Ann. of Math. (2)
30. Roth’s theorem in many variables.
T. Schoen and I. D. Shkredov. Israel Journal of Mathematics
31. Integer sets containing no arithmetic progressions.
E. Szemerédi. Acta Math. Hungar.
32. Upper and lower bounds for stochastic processes

M. Talagrand. , volume 60 of Ergebnisse der Mathematik und ihrer Grenzgebiete. 3. Folge. A Series of Modern Surveys in Mathematics [Results in Mathematics and Related Areas. 3rd Series. A Series of Modern Surveys in Mathematics].
33. The dichotomy between structure and randomness, arithmetic progressions, and the primes.
T. Tao. In International Congress of Mathematicians. Vol. I, pages 581–608. Eur. Math. Soc., Zürich, 2007.
34. Algebraic combinatorial geometry: the polynomial method in arithmetic combinatorics, incidence combinatorics, and number theory.
T. Tao. EMS Surv. Math. Sci.
35. Pseudorandom graphs.
A. Thomason. In Random graphs ’85 (Poznań, 1985), volume 144 of North-Holland Math. Stud., pages 307–331. North-Holland, Amsterdam, 1987.
36. Random graphs, strongly regular graphs and pseudorandom graphs.
A. Thomason. In Surveys in combinatorics 1987 (New Cross, 1987), volume 123 of London Math. Soc. Lecture Note Ser., pages 173–195. Cambridge Univ. Press, Cambridge, 1987.
37. The moduli of smoothness and convexity and the Rademacher averages of trace classes ().
N. Tomczak-Jaegermann. Studia Math.
13721