Limitations of sum of products of Read-Once Polynomials
We study limitations of polynomials computed by depth two circuits built over read-once polynomials (ROPs) and depth three syntactically multi-linear formulas. We prove an exponential lower bound for the size of the arithmetic circuits built over syntactically multi-linear arithmetic circuits computing a product of variable disjoint linear forms on variables. We extend the result to the case of arithmetic circuits built over ROPs of unbounded depth, where the number of variables with gates as a parent in an proper sub formula is bounded by . We show that the same lower bound holds for the permanent polynomial. Finally we obtain an exponential lower bound for the sum of ROPs computing a polynomial in defined by Raz and Yehudayoff .
Our results demonstrate a class of formulas of unbounded depth with exponential size lower bound against the permanent and can be seen as an exponential improvement over the multilinear formula size lower bounds given by Raz  for a sub-class of multi-linear and non-multi-linear formulas. Our proof techniques are built on the one developed by Raz  and later extended by Kumar et. al.  and are based on non-trivial analysis of ROPs under random partitions. Further, our results exhibit strengths and limitations of the lower bound techniques introduced by Raz .
More than three decades ago, Valiant  developed the theory of Algebraic Complexity classes based on arithmetic circuits as the model of algebraic computation. Valiant considered the permanent polynomial defined over an matrix of variables:
where is the set of all permutations on symbols. Valiant  showed that the polynomial family is complete for the complexity class . Further, Valiant  conjectured that does not have polynomial size arithmetic circuits. Since then, obtaining super polynomial size lower bounds for arithmetic circuits computing has been a pivotal problem in Algebraic Complexity Theory. However, for general classes of arithmetic circuits, the best known lower bound is quadratic in the number of variables .
Naturally, the focus has been on proving lower bound for against restricted classes of circuits. Grigoriev and Karpinski  proved an exponential size lower bound for depth three circuits of constant size over finite fields. Agrawal and Vinay  (See also [20, 9]) showed that proving exponential lower bounds against depth four arithmetic circuits is enough to resolve Valiant’s conjecture, and hence explaining the lack of progress in extending the results in  to higher depth circuits. This was strengthened to depth three circuits over infinite fields by Gupta et. al. .
Recently, Gupta et. al.  obtained a size lower bound for homogeneous depth four circuits computing where the bottom fan-in is bounded by . The techniques introduced in [5, 6] have been generalized and applied to prove lower bounds against various classes of constant depth arithmetic circuits, regular arithmetic formulas and homogeneous arithmetic formulas. (See e.g., [7, 11, 8].) Exhibiting polynomials that have exponential lower bound against concrete classes of arithmetic circuits is an important research direction.
In 2004, Raz  showed that any multilinear formula computing requires size , which was one of the first super polynomial lower bounds against formulas of unbounded depth. Further, in , Raz extended this to separate multilinear formulas from multilinear circuits. Raz’s work lead to several lower bound results, most significant being an exponential separation of constant depth multilinear circuits . More recently, Kumar et. al  extended the techniques developed in  to prove lower bounds against non-multilinear circuits and formulas.
Motivation and our Model : Depth three circuits are in fact circuits built over linear forms. A linear form can be seen as the simplest form of read-once formulas (): formulas where a variable appears at most once as a leaf label. Polynomials computed by s are called read-once polynomials or s. There are two natural generalizations of the model: 1) Replace linear forms by sparse polynomials, this leads to the well studied circuits; and 2) Replace linear forms with more general read-once formulae, this leads to the class of circuits over read-once formulas or for short.
In this paper, we consider the second extension, i.e., . Restricted forms of were already considered in the literature. For example, Shpilka and Volkovich  obtained identity testing algorithms for the sum of s. Further  gives identity tests for when the top fan-in is restricted to two.
Apart from being a natural generalization of circuits, the class can be seen as building non-multi-linear polynomials using the simplest possible multi-linear polynomials viz. s.
Our Results : We study the limitations of the model for some restricted class of circuits. Firstly, we prove,
Let be -variate syntactic multi-linear formulas with bottom -fan-in at most where , and top -fan-in at most for and . Also assume that . There is a product of variable disjoint linear forms such that, if then .
Our arguments do not directly generalize to the case of unbounded depth s with small bottom fan-in. Nevertheless, we obtain a generalization of Theorem 1, allowing s of unbounded depth with a more stringent restriction than bottom -fan-in. Let be an and for a gate in , let be the number of variables in the sub-formula rooted at whose parents are labelled as . Then is the maximum value of , where the maximum is taken over all gates in excluding the top layer of gates. Note that, in the case of s, is equivalent to the bottom fan-in. For an , is the smallest value of among all s computing . We prove,
Let be s with for , and , where . There is a product of linear forms such that, if then .
As far as we know, this is the first exponential lower bound for a sub-class of non-multi-linear formulas of unbounded depth. It can be noted that our result above does not depend on the depth of the s. Further, note that even though a product of linear forms is a simple linear projection of , Theorem 2 does not imply a lower bound for due to restrictions on , since linear projections might change the bottom fan-in of the resulting s. However, we prove,
Let be s with for , and , for and . Then, if then for some .
Finally, we show that the polynomial defined by Raz-Yehudayoff  cannot be written as sum of sub-exponentially many s:
There is a polynomial such that for any s , if , then we have .
Related Results : Shpilka and Volkovich  proved a linear lower bound for a special class of s to sum-represent the polynomial and used it crucially in their identity testing algorithm. Theorem 4 is an exponential lower bound against the same model as in , however against a polynomial in . It should be noted that the results in Raz  combined with  immediately implies a lower bound of for the sum of s. Our results are an exponential improvement of bound given by .
Kayal  showed that at least many polynomials of degree are required to represent the polynomial as sum of powers. Our model is significantly different from the one in  since it includes high degree monomials, though the powers are restricted to be sub-linear, whereas Kayal’s argument works against arbitrary powers.
Our Techniques Our techniques are broadly based on the partial derivative matrix technique introduced by Raz  and later extended by Kumar et. al . It can be noted that the lower bounds obtained in  are super polynomial and not exponential. Though Raz-Yehudayoff  proved exponential lower bounds, their argument works only against bounded depth multilinear circuits. Further, the arguments in [14, 18] do not work for the case of non-multilinear circuits, and fail even in the case of products of two multilinear formulas. This is because rank of the partial derivative matrix, a complexity measure used by [14, 18] (see Section 2 for a definition) is defined only for multi-linear polynomials. Even though this issue can be overcome by a generalization introduced by Kumar et. al , the limitation lies in the fact that the upper bound of for an or variate polynomial, obtained in  or  on the measure for the underlying arithmetic formula model is insufficient to handle products of two s.
Our approach to prove Theorems 2 and 3 lie in obtaining an exponentially stronger upper bounds (see Lemma 17 ) on the rank of the partial derivative matrix of an on variables where . Our proof is a technically involved analysis of the structure of s under random partitions of the variables. Even though the restriction on might look un-natural, in Lemma 18, we show that a simple product of variable disjoint linear forms in -variables, with achieve exponential rank with probability . Thus our results highlight the strength and limitations of the techniques developed in [18, 10] to the case of non-multi-linear formulas.
Finally proof of Theorem 4 is based on an observation pointed out to the authors by an anonymous reviewer. We have included it here since the details have been worked out completely by the authors.
Due to space limitations, all the missing proofs can be found in Sections 5,6 and 7.
Let be an arbitrary field and be a set of variables. An arithmetic circuit over is a directed acyclic graph with vertices of in-degree 0 or 2 and exactly one vertex of out-degree 0 called the output gate. The vertices of in-degree 0 are called input gates and are labeled by elements from . The vertices of in-degree more than 0 are labeled by either or . Thus every gate of the circuit naturally computes a polynomial. The polynomial computed by is the polynomial computed by the output gate of the circuit.
An arithmetic formula is a an arithmetic circuit where every gate has out-degree bounded by , i.e., the underlying undirected graph is a tree.
The size of an arithmetic circuit is the number of gates in . For any gate depth of is the length of the longest path from an input gate to gate in . Depth of is defined as the depth of its output gate.
An arithmetic read-once formula ( for short) is an arithmetic formula over where every input variable occurs as a label of at most once . The polynomial computed by an is called a read-once-polynomial or .
Let be a multilinear polynomial. The partial derivative matrix of denoted by  is a matrix defined as follows: The rows of are labeled by all possible multilinear monomials in and the columns of be labeled by all possible multilinear monomials in . For any two multilinear monomials and , the entry is the coefficient of in .
(Sub-Additivity.) Let where and are multilinear polynomials in . Then, Moreover, if then
(Sub-Multiplicativity.) Let , where and are multilinear polynomials in , and . Then,
Kumar et. al.  generalized the notion of partial derivative matrix to include polynomials that are not multilinear. Let and . Let be a polynomial. The polynomial coefficient matrix of denoted by is a matrix defined as follows. For multilinear monomials and in variables and respectively, the entry if and only if can be uniquely expressed as where such that and does not have any monomial that is divisible by and contains only variables present in and .
 For a multilinear polynomial , we have .
Observe that the matrix has polynomial entries. Therefore is defined only under a substitution function that substitutes every variable in to a field element.
For any substitution function , let us denote by the matrix obtained by substituting every variable in at each entry of to the field element given by . Define, Having defined polynomial coefficient matrix and we now look at properties of with respect to .
(Sub-additivity.) Let . Then,
(Sub-multiplicativity.) Let and such that and . Then for any polynomials , we have: .
A partition of is a function such that is an injection when restricted to , i.e., , if and then . Let be an and be a partition function. Define to be the formula obtained by replacing every variable that appears as a leaf in by . Then the polynomial computed by is . Observe that .
An arithmetic formula is said to be a constant-minimal formula if no gate in has both its children to be constants. Observe that for any arithmetic formula , if there exists a gate in such that then we can replace in by the constant , where . Thus we assume without loss of generality that is constant-minimal.
We need some observations on formulas that compute natural numbers. Recall that an arithmetic formula is said to be monotone if does not contain any negative constants.
Let be a monotone arithmetic formula were the leaves are labeled numbers in . Then for any gate in , the value of denoted by is defined as : If is a leaf then where is the label of and then , where . Finally, is the value of the output gate of . Let be a monotone arithmetic formula with leaves labelled by either or . A node a is called a if is a leaf and or with and . The following is a simple upper bound on the value computed by a formula.
Let be a binary monotone arithmetic formula with leaves. If every leaf in takes a value at most , then .
The proof is by induction on the size of the formula. Base Case :
If has a single gate then .
If has a single gate then .
Induction Step : Let be the output gate of with children and . Let the number of leaves in the sub formula rooted at and be and .
If is a gate. Then, . By induction hypothesis, .
If is a gate. Then, . By induction hypothesis, .
Any formula with a large value should have a large number of s.
Let be a binary monotone arithmetic formula with leaves labeled by either or . If then there exists at least nodes that are s.
Let be a binary monotone arithmetic formula with leaves labeled by either or . First mark every node such that is a and remove sub-formula rooted at except . Consider any leaf that remains unmarked and along the path from to root there is no node that is marked. Then . Consider the unique path from to root in . Let the first node in the path such that . Let and be the children of . Without loss of generality let be an ancestor of . Then observe that there is atleast one marked node(say ) in the sub-formula rooted at . Set . If is a gate set else set . Let be the leaves of the resulting formula at the end of this process. For every , we have . Therefore by Lemma 5, . Since , we have . Therefore as required. ∎
Finally, we will use the following variants of Chernoff-Hoeffding bounds.
(Chernoff-Hoeffding bound) Let be independent random variables. Let and . Then for any ,
3 Hardness of representation for Sum of s
Let . Define as a distribution on the functions as follows : For ,
Observe that is not necessarily true. Let be a binary arithmetic formula computing a polynomial on the variables . Note that any gate with at least one variable as a child can be classified as:
type- gates : sum gates both of whose children are variables,
type- gates : product gates both of whose children are variables,
type- gates : sum gates exactly one child of which is a variable and the other an internal gate; and
type- gates: product gates exactly one child of which is a variable and the other an internal gate
Given any , let there be type- gates, type-, type- and type- gates in . Note that . Let . Let there be gates of type- that achieve -1 under and let gates of type- that achieve -2 under . Then, .
Let be an computing an and . Then,
Observe that for any type- gate , , and hence type- gates do not contribute to the rank.
The proof is by induction on the structure of . Base case is when is of depth 1. Let be the root gate of computing the polynomial . Then
is an type- gate with children : . For any , we have . Then . Therefore either or . In either case, .
is a type- gate with children : . For any we have . Then . Therefore .
For the induction step, we have the following cases based on the structure of .
is a type- gate with children , i.e., . For any , we have by sub-additivity . Let be the number of type- gates in the sub-formula rooted at that achieve rank-1 and rank-2 under respectively. Let be the number of type- and type- gates in the sub-formula rooted at . We now have , and By Induction hypothesis . First suppose the case when , then, . Now suppose and hence (since and are integers). If , then . Finally, if , for all of the remaining possibilities, we have .
be an internal gate. For , let be the number of type- gates that achieve rank-1 and rank-2 under respectively and be the number of type- and type- gates in the sub-formula rooted at . Then where . In either case, , and from Induction hypothesis Since we have
Let be a and . Let be the number type- gates that achieve - under . Then,
Let be a type- gate in . Then for some . Then . Therefore, . Applying Theorem 5 (2) and (3) with , we get the required bounds for . ∎
Let be a computing and variables, and . Then with probability at least , .
Consider the following two cases:
Case 1 : . Then either or .
Firstly, suppose , then by Lemma 7, we have . Since , we have . By Lemma 8, . Therefore, .
Now suppose . Since , we have . Therefore by Lemma 7,
Case 2 : . Observe that . Since any type gate achieves rank 1 under any , by a simple inductive argument we have for any . Therefore ∎
The following polynomial was introduced by Raz and Yehudayoff .
Let be an integer. Let and . For any two integers , we define an interval . Let be the length of the interval . Let and . For every such that is even we define a polynomial as when and if then, where , are distinct variables, and the summation is over such that the interval is of even length. Let
In the following, we view as polynomial in with coefficients from the rational function field .
Let Let , , and be sets of variables. Suppose such that . Then for the polynomial as in Definition 1 we have,
Proof builds on Lemma 4.3 in  as a base case and is by induction on .
Base case: Either or . For , the statement follows by Lemma 4.3 in . When , then .
Induction step: Without loss of generality, assume that . There are three possibilities:
- Case 1 :
Let and or vice versa. In this case
- Case 2 :
and . Then
For the penultimate inequality above, note that is defined on and and hence by Induction Hypothesis, .
- Case 3
and . Then there is an such that and , where . Then by the definition of , over , since by Lemma 4.3 in , and by Induction Hypothesis.
Proof is a simple application of Chernoff’s bound (Theorem 5) with . ∎