We give a nontrivial algorithm for the satisfiability problem for -wire threshold circuits of depth two which is better than exhaustive search by a factor where . We believe that this is the first nontrivial satisfiability algorithm for -wire threshold circuits of depth two. The independently interesting problem of the feasibility of sparse - integer linear programs is a special case. To our knowledge, our algorithm is the first to achieve constant savings even for the special case of Integer Linear Programming. The key idea is to reduce the satisfiability problem to the Vector Domination Problem, the problem of checking whether there are two vectors in a given collection of vectors such that one dominates the other component-wise.
We also provide a satisfiability algorithm with constant savings for depth two circuits with symmetric gates where the total weighted fan-in is at most .
One of our motivations is proving strong lower bounds for circuits, exploiting the connection (established by Williams) between satisfiability algorithms and lower bounds. Our second motivation is to explore the connection between the expressive power of the circuits and the complexity of the corresponding circuit satisfiability problem.
Satisfiability testing is both a canonical -complete problem [6, 13] and one of the most successful general approaches to solving real-world constraint satisfaction problems. In particular, optimized heuristics are used to address a variety of combinatorial search problems successfully in practice, such as circuit and protocol design verification. The exact complexity of the satisfiability problem is also central to complexity theory, as demonstrated by Williams , who has showed that any improvement (by even a superpolynomial factor compared to exhaustive search) for the satisfiability problem for general circuits implies circuit lower bounds. Furthermore he has successfully used the connection to prove superpolynomial size bounds for circuits using a novel nontrivial satisfiability algorithm for circuits, solving a long standing open problem .
This raises the questions: For which circuit models do nontrivial satisfiability algorithms exist? How does the amount of improvement over exhaustive search relate to the expressive power of the model (and hence to lower bounds)? Can satisfiability heuristics for stronger models than be useful for real-world instances?
Both the connection to circuit lower bounds and to heuristic search algorithms point to threshold circuits as the model to study next. Bounded depth polynomial size threshold circuits are the next natural circuit class stronger than . is a powerful bounded depth computational model. It has been shown that basic operations like addition, multiplication, division, and sorting can be performed by bounded depth polynomial size threshold circuits. [5, 2]. In contrast, unbounded fan-in bounded depth polynomial size circuits over the standard basis (even when supplemented with mod gates for prime ) cannot compute the majority function . However, our understanding of the limitations of bounded depth threshold circuits is extremely weak. Exponential lower bounds for such circuits are only known for the special case of depth two and bounded weight . For larger depth circuits, barely superlinear lower bounds are known on the number of wires .
On the other hand, satisfiability for depth two threshold circuits contains as special cases some well known problems of both theoretical and practical significance. is one such special case, since both conjunctions and disjunction are a special case of threshold gates. max--sat, the optimization form of satisfiability, is another special case, since the top threshold gate can count the number of satisfied clauses for an assignment. Even for max--sat, no algorithms with a constant factor savings over exhaustive search are known (although such an algorithm is provided for max--sat in ). Another special case is Integer Linear Programming (ILP), a problem that is very useful in expressing optimization problems both in theory and practice. Testing the feasibility for a - ILP is equivalent to testing the satisfiability of a circuit with two levels, the bottom consisting of threshold gates and the top level being a conjunction. So both theoretical and real-world motivation points us to trying to understand the satisfiability problem for depth two threshold circuits.
Santhanam  gives an algorithm with constant savings for linear size formulas of AND and OR gates with fan-in two. However, this does not directly give an algorithm for depth two threshold circuits, as converting a linear size threshold circuit into a formula over AND and OR gates gives quadratic size.
In all of these related problems, a key distinction is between the cases of linear size and superlinear size circuits. In particular, an algorithm with constant savings for depth two threshold circuits of superlinear size would refute the Strong Exponential Time Hypothesis () , since for all can be reduced (via Sparsification Lemma ) to superlinear size depth two threshold circuits . ( says that for every , there is a such that cannot be solved in time .) However, for and , algorithms with constant savings are known when the formula is linear size [15, 7, 1]. So, short of refuting , the best we could hope for is to extend such an improvement to the linear size depth two threshold circuit satisfiability problem.
In this paper, we give the first improved algorithm, which obtains a constant savings in the exponent over exhaustive search for the satisfiability of -wire, depth two threshold circuits for every constant . As a consequence, we also get a similar result for linear-size ILP. Under , this is qualitatively the best we could hope for, but we expect that further work will improve our results quantitatively. For example, our savings is exponentially small in , whereas in, e.g., the satisfiability algorithm of  for constant depth and-or circuits, it is polylogarithmic in . We consider this just a first step towards a real understanding of the satisfiability problem for threshold circuits, and hope that future work will get improvements both in depth and in savings.
While we do not obtain any new circuit lower bounds, there is some chance that this line of work could eventually yield such bounds. For example, if there is an algorithm for any constant depth threshold circuit with super-inverse-polynomial savings in , then by applying .
Our main sub-routine is an algorithm for the Vector Domination Problem: given vectors in , is there a pair of vectors so that the first is larger than the second in every coordinate? We show that, when for a constant , this problem can be solved in subquadratic time. In contrast, Williams  shows that solving even the Boolean special case of vector domination with a subquadratic algorithm when would refute . We think the Vector Domination Problem may be of independent interest, and might be used to reason about the likely complexities of other geometric problems within polynomial time.
Let be a set of variables with . An assignment on is a function that assigns every variable a Boolean value. A restriction is an assignment on a set . For an assignment and a variable , denotes the value of under the assignment .
A threshold gate on variables is defined by weights for and a threshold . The output of the gate is , if and otherwise. The fan-in of the threshold gate is the number of nonzero weights. We call a variable an input to a gate if the corresponding weight is nonzero. We also extend the definition of a threshold gate to -ary symmetric gates whose inputs and outputs are -ary.
For a collection of threshold gates, the number of wires is the sum of their fan-ins. A depth two threshold circuit consists of a collection of threshold gates (called the bottom-level gates) on the same variables and a threshold gate (called the top-level gate) on the outputs of the bottom-level gates plus the variables. The output of the circuit is the output of the top-level gate. We call a variable with nonzero weight at the top-level gate a direct wire. For a -ary depth two threshold circuit, the gates are -ary gates and the top-level gate only outputs Boolean values. The number of wires of a depth two threshold circuit is the number of wires of the bottom-level gates. We call a threshold circuit sparse if the the number of wires is linear in the number of variables.
A satisfiability algorithm for depth two threshold circuits is an algorithm that takes as input a depth two threshold circuit and outputs an assignment such that the circuit evaluates to 1 under the assignment.
A linear function on a variable set is a function , where are called the coefficients. The size of a linear function is the number of nonzero coefficients. A linear inequality is an inequality of the form .
An algorithm for the Integer Programming Feasibility Problem takes as input a collection of linear inequalities on variables and outputs an assignment such that all inequalities are satisfied. We call an inequality of the form a capacity constraint. In a 0-1 Integer Programming Feasibility Problem each variables is constrained to be 0 or 1.
We use to denote the asymptotic growth of a function ignoring polynomial factors. Informally, we say an algorithm is nontrivial, if its time is significantly better than exhaustive search. If is a satisfiability algorithm for circuits with variables with run time , we call the savings of the algorithm over exhaustive search.
For a vector , we use for coordinate .
All logarithms are base unless noted otherwise.
3 Results and Techniques
The main contribution of the paper is a nontrivial satisfiability algorithm for sparse threshold circuits of depth 2. More precisely, we prove the following:
There is a satisfiability algorithm for depth two threshold circuits on variables with wires that runs in time where
While the proof in Section 5 assumes a Boolean inputs for simplicity, the proof easily extends to threshold circuits with -ary inputs, yielding the following corollary.
There is a satisfiability algorithm for depth two threshold circuits on -ary variables with wires that runs in time where
In the following, we provide a high level description of our algorithm. Intuitively, there are two extreme cases for the bottom layer of a linear size threshold circuits of depth two.
The first extreme case is when we have a linear number of gates each with bounded fan-in . This case is almost equivalent to max--sat and can be handled in a way similar to [4, 1]. Consider the family of -sets of variables given by the support of each bottom-level gate. A probabilistic argument shows that, for some constant , there exists a subset of about variables so that at most one element from each of the -sets in the family is outside of . Then for any assignment to the variables in , each bottom-level gate becomes either constant or a single literal, and the top-level gate becomes a threshold function of the remaining inputs. To check if a threshold function is satisfiable, we set each variable according to the sign of its weight.
The second extreme case is when we have a relatively small number of bottom-level gates, say, at most , but some of them might have a large fan-in. In this case, we could first reduce the problem to - ILP by guessing the truth value of all bottom-level gates and the top gate, and then verifying the consistency of our guesses. Each of our guesses are threshold functions of the variables, so testing consistency of our guesses is equivalent to testing whether the feasible region of about linear inequalities has a Boolean solution.
We then reduce such an ILP to the Vector Domination problem. To do this, we partition the variables arbitrarily into two equal size sets. For each assignment to the first set, we compute a vector where the ’th component corresponds to the weighted sum contributed by the first set of variables to the ’th threshold gate. For the second set of variables, we do the same, but subtract the contribution from the threshold for the gate. It is easy to see that the vectors corresponding to a satisfying assignment are a dominating pair. Since there are vectors in our set, and each vector is of dimension , to get constant savings, we need a Vector Domination algorithm that is subquadratic when the dimension is much less than the logarithm of the number of vectors. The last step is to give such an algorithm, using a simple but delicate divide-and-conquer strategy.
Finally, to put these pieces together, we need to reduce the arbitrary case to a “convex combination” of the two extreme cases mentioned above. To do this, we use the Fan-In Separation Lemma which asserts that there must be a relatively small value of so that there are relatively few gates of fan-in bigger than but less than , for some constant . We show that, as in the first extreme case, for a random subset of variables, the gates with fan-in less or equal to almost entirely simplify to constants or literals after setting the variables in . Our selection of ensures that the number of gates of fan-in greater than is small relative to the number of remaining variables. So we can apply the method outlined for the second extreme case. The Fan-In Separation Lemma is where our savings becomes exponentially small. Unfortunately, this lemma is essentially tight, so a new method of handling this step would be needed to make the savings polynomially small.
Since the Integer Programming Feasibility problem with capacity constraints can be expressed as a depth two threshold circuit with an AND gate as the top-level gate, the results translate directly to the feasibility version of sparse integer programs with capacity constraints. We get
Let be a collection of linear inequalities in variables with total size at most . There is an algorithm that finds an integer solution to the linear inequalities with capacity constraints for all in time for
The following two sections contain the details of the proof. Section 4 introduces the Vector Domination problem and, for small dimension, gives an algorithm faster than the trivial quadratic time. The feasibility of a - ILP with a small number of inequalities is then reduced to the Vector Domination problem, yielding an algorithm for such - ILP with constant savings. A reduction from depth two threshold circuits to - ILP concludes that section. In Section 5, we show how to reduce the -wire depth two threshold circuits satisfiability problem to the special case with a small number of bottom-level gates relative to the number of variables. The remaining sections discuss generalizations of our result.
4 Vector Domination Problem
In this section we introduce the Vector Domination problem and give an algorithm faster than the trivial for small dimension.
Given two sets of -dimensional real vectors and , the Vector Domination Problem is the problem of finding two vectors and such that for all .
Let and with . There is an algorithm for the Vector Domination problem that runs in time
The claim is trivial for or . In the latter case we can sort the set and then decide if such a pair exists in linear time.
Otherwise, let be the median of the first coordinates of . We partition the set into three sets , and , where contains all vectors such that , contains all vectors such that and contains all vectors such that . We further partition into set , and in the same way. A vector can only dominate a vector in one of three cases:
For the first two cases we have and as we split at the median. For the third case, we know , hence we can recurse on vectors of dimension . Since finding the median takes time we get for the running time of vectors of dimension
To solve this recurrence relation, we want to count the number of nodes in the recurrence tree with and . There are possible paths from the root node to such a node, as in every step we either decrease or , and there are possible combinations to do so, and if we decrease there are two possible children. Since computing the median of numbers takes time the total time is upper bounded by
We can reduce - ILP with few inequalities to the Vector Domination Problem.
Consider a - Integer Linear Program on variables and inequalities for some . Then we can find a solution in time
Note that this algorithm is faster than for .
Separate the variable set into two sets and of equal size. We assign every assignment to the variables in and a -dimensional vector where every dimension corresponds to an inequality. Let be an assignment to and let be the -th inequality for all . Let be the vector with and let be the set of such vectors. For an assignment to , let be the vector with and let be the set of all such vectors .
An assignment to all variables corresponds to an assignment to and an assignment to , and hence to a pair and . The pair satisfies all inequalities if and only if dominates . Since and the dimension is , we can solve the domination problem in time
We now reduce the satisfiability of a depth two threshold circuit with bottom-level gates and any number of direct wires to the union of ILP problems.
Consider a depth two threshold circuit on variables and bottom-level gates for some . We allow an arbitrary number of direct wires to the top-level gate. Then there is a satisfiability algorithm that runs in time
Note that this algorithm is faster than for .
For every subset of bottom-level gates, we solve the satisfiability problem under the condition that only the bottom-level gates of are satisfied. For an assignment to satisfy both the circuit and the condition that only gates in are satisfied, it must satisfy the following system of inequalities:
For gates in with weights and threshold , we have .
For gates not in we require , which is equivalent to .
Let be the weights of the direct wires and let be the threshold of the top-level gate. Further let be the sum of the weights of the gates in . Then .
Note that this system contains inequalities, and the additional dimension adds only a polynomial factor to the time.
Since we need to solve a system of inequalities for every possible subset of bottom-level gates to be satisfied, we have an additional factor of , which gives the running time as claimed. ∎
Williams  introduced the reductions used in Corollaries 4.2 and 4.3. He considered a special case of the Vector Domination problem (called the Cooperative Subset Query problem) where the entries in the vectors are 0 and 1 instead of arbitrary real numbers . Applying the reduction from Corollary 4.2 to , he concludes that an algorithm for solving the Cooperative Subset Query problem with that runs in time for some and a time-constructible gives a algorithm in time where is the number of clauses. Our algorithm only works for , so it would be interesting to see how far this can be pushed.
5 Fan-In Separation
In this section we reduce the satisfiability of a depth two threshold circuit with wires to depth two threshold circuits with at most bottom-level gates by considering all possible assignments to a random subset of variables. The goal of the restriction is to eliminate all but a small fraction of gates. will consist of all but a fraction of the variables where is chosen such that there are only a small number of gates of fan-in larger than relative to the number of remaining variables. Fan-In Separation Lemma shows how to find such a .
Lemma 5.1 (Fan-In Separation Lemma).
Let be a family of sets such that . Further let and be parameters. There is an such that
Assume otherwise for the sake of contradiction. For , let be the sum of where . By assumption we have for all . Hence , which is a contradiction. ∎
Consider a depth two threshold circuit with variables and wires. Let and let be a random set of variables such that each variable is in with some probability independently. There exists a such that the expected number of bottom-level gates that depend on at least two variables not in is at most .
Let and and let be the smallest value such that there are at most wires as inputs to gates with fan-in between and . Further let .
Using the Fan-In Separation Lemma we get . We distinguish three types of bottom-level gates: Small gates, with fan-in at most , medium gates with fan-in between and , and large gates with fan-in at least . For each type of gates, we argue that the expected number of gates that depend on at least two variables not in is bounded by .
For medium gates, the total number of wires is bounded by and each gate contains at least wires. Hence the number of medium gates is bounded by .
Large gates contain at least wires, hence the number of large gates is bounded by .
For small gates, we argue as follows. Let be the number of small gates and let be their fan-ins. Let denote the event that gate depends on at least two variables not in and let be the number of such events. We have and therefore
There is a satisfiability algorithm for depth two threshold circuits with wires that runs in time for .
Let and as well as other parameters be as above. For every assignment to , we have a depth two threshold circuit with variables and bottom-level gates in expectation. Since , we can decide the satisfiability of such a circuit using Corollary 4.3 with constant savings. Let be the savings with our parameters.
Let be the time for carrying out the entire procedure. Since we are interested in the expected savings we consider the logarithm of the time and get
and the lemma follows from . ∎
Since is bounded by above by , we can repeat the process a constant number of times until we find a restriction such that the savings is at least its expectation. This gives us our main result Theorem 3.1.
6 Generalization to Formulas
In this section we discuss an extension of our main result to linear size, constant depth threshold formulas. A formula is a circuit such that the output of every gate is an input to at most one other gate. A formula can be viewed as a tree where the internal nodes correspond to gates and the leaves to bottom variables. Note that a circuit of depth two is always a formula. The proof is a direct generalization of our main proof.
There is a satisfiability algorithm for depth threshold formulas with wires that runs in time where
The algorithm chooses a random restriction such that at most gates depend on more than one variable after restriction, where as before. As in the original proof, we take into account that there is only a single top-level gate, which does not simplify after restriction. The main difference to our main proof is the notion of the fan-in. Instead of considering the number of inputs to a gate, consider the size of a gate. The size of a gate is the size of the subtree rooted at that gate. It is also an upper bound to the number of variables the gate depends on.
For all , the sum of sizes of all gates at depth is at most , since the circuit is a tree with at most wires. Hence the sum of sizes of all gates (minus the top-level gate) is at most .
Using the Fan-In Separation Lemma we can select a set of size where such that the number of gates that depend on at least two variables not in is at most . We can then write each remaining gate as a linear inequality, as each input is either a variable, a negated variable or a constant, which allows us to apply Corollary 4.2. ∎
7 Generalization to Symmetric Gates
In this section we discuss a second extension, to symmetric gates. A gate is symmetric if the output depends only on the weighted sums of the inputs. In particular, threshold gates are symmetric. The proof of our main result does not directly generalize to symmetric gates, but we give a different algorithm to decide the satisfiability of depth two circuits consisting of symmetric gates that follows similar ideas as our main proof. For this algorithm we do however require that the weights are integer and small. Specifically, we define the weighted fan-in of a gate as the sum of the absolute weights and the weighted number of wires as the sum of the fan-ins of all the gates. The result applies to circuits with a weighted fan-in of .
The main difference between the two algorithms is the problem we reduce it to after applying a restriction. In our main result, we reduce the satisfiability of the simplified circuit to a (small) system of linear inequalities. Here, we reduce to a system of linear equations. We first give an algorithm for linear equations.
There is an algorithm to find a Boolean solution to a system of linear equations on variables in time .
We first reduce the problem to subset sum. Let be the weight of in the -th equation and let be right-hand side of the -th equation. Further let be the largest such value. We define and . Then there is a solution to the system of linear equations if and only if there is a subset of the that sums to .
To solve the subset sum problem, partition the set of into two sets of equal size and list all possible subset sums each. We can then sort the lists in time and determine if there is a pair of numbers that sums to . ∎
We reduce the satisfiability problem of depth two threshold circuits with small integer weights to a system of linear equations to get the following result.
There is a satisfiability algorithm for depth circuits with symmetric gates and weighted number of wires that runs in time where
As before, we pick a random restriction with some parameter , such that most gates depend on at most one variable.
Given an assignment, we distinguish between the Boolean output of a gate and the value. The value is defined as the weighted sum of the inputs. Note that the value uniquely defines the output of a symmetric gate. Unlike our main proof, we guess the value of the remaining gates, including the top-level gate. Given a value for every gate, we can write a system of linear equation. We then solve the system of linear equations on Boolean variables in time using Lemma 7.1.
We need the overhead for guessing the values to be smaller than the savings achieved with solving the system of linear equations. For this, it is crucial that both the number of remaining gates and the number of values they can obtain is small. Here we use the requirement that the weights are small. We defer the details of the calculation on how many systems of linear equations we need to solve until section 7.1
One possible approach would be to select using a fan-in separation technique. However, we only achieved savings that are doubly exponentially small in using this approach. To get better savings, it is useful to model the interplay between the parameter and the circuit as an explicit zero-sum game, where the first player’s (the algorithm designer) pure strategies are the values of and the second player’s (the circuit designer) pure strategies are the circuits where all the gates have the same fan-in. The payoff is the difference between the saving of solving the subset sum problem and the overhead of guessing the values of the gates.
The mixed strategies of the circuit designer are circuits of symmetric gates with a weighted number of wires of at most , where each such circuit is viewed as a distribution of the total number of wires among gates of different weighted fan-in. The mixed strategies of the algorithm designer are distributions on the values of . We then apply the Min-Max theorem to lower bound the expected value of the game by exhibiting a distribution (with finite support) on the values of . We search through the values in the support of the distribution to find a that produces the expected value. This novel game-theoretic analysis yields an overall savings which is only single exponentially small in . Section 7.2 contains the details of the Min-Max approach.
7.1 The Algorithm
We develop the algorithm of Lemma 7.2 in three stages. In this section, we consider a parameter and present a satisfiability algorithm for depth two circuits with symmetric gates and weighted number of wires of . We further assume that all the bottom-level gates have the same weighted fan-in . The algorithm achieves savings and for certain combinations of and the savings might be negative. In the second stage we extend the algorithm to circuits with varying fan-in and show that the savings of the algorithm is a convex combination of . In the last stage, in Section 7.2 we show how to select a such that the savings is at least for any distribution on .
As we are mainly interested in the savings, we look at the logarithm of the time complexity and bound its expectation.
Let be a parameter and be a depth two circuit with symmetric gates, variables , a weighted number of wires of , and weighted fan-in for all bottom-level gates. There is an algorithm that decides the satisfiability for such with time complexity such that for
We select a random subset such that a variable is in with probability independently. We note that . For each of the assignments to , we solve the satisfiability problem of the simplified circuit. Bottom-level gates where all inputs are in are removed and the top-level gate is adjusted appropriately. Gates that only depend on one input are replaced by a direct wire to the top-level gate with an appropriate weight and adjustment to the top-level gate. For all gates with at least two remaining inputs, we guess the value of the gate and express the gate as a linear equation. Similarly, we guess the value of the top-level gate to get another linear equation. We then solve the resulting system of linear equations on variables in time using Lemma 7.1.
The critical part of the analysis is bounding the overhead from guessing the values of the gates. We first bound the number of distinct values a gate can take. The top-level gate can only take polynomially many different values. Consider a bottom-level gate with fan-in after applying an assignment to the variables in . We bounds the number of distinct values that the gate can take in two different ways. The number of possible inputs, and hence the number of possible values is bounded by . On the other hand, since the value is an integer between and , the number of possible values for the gates is also upper bounded by . Hence, we use as an upper bound for the number of values of a bottom-level gate with fan-in .
Since we have a control on the number of distinct values taken by a gate by assumption, our overhead crucially depends on the number of exceptional gates, gates that depend on more than one variable after applying an assignment to the variables in . Intuition says that the number of exceptional gates should be small. If the fan-in of a gate is small, then we expect that it will simplified to depend on at most one variable after assigning values to the variables in . On the other hand, there cannot be too many gates of large fan-in. While the intuition is simple, it is tricky to make it work for us in the general context. At this stage, our focus is on estimating the savings for the probability parameter and weighted fan-in .
Let be a random variable denoting the number of possible values the remaining gates can obtain. Our estimation of and involves two cases. Let . We first consider the case . Let be the set of variables that appear in exceptional gates. Our goal is to upper bound .
Consider a bottom-level gate. Let be the random variable denoting the number of its inputs not in . Let be the number of variables the gate depends on, and let be the random variable denoting the number of its inputs not in . The distribution of is , hence we have . Let the random variable denote the number of variables that the gate can contribute to . Since is the set of variables appearing in exceptional gates, we have for and otherwise. Hence
by Bernoulli’s inequality. Hence, for any variable which is an input to the gate, the probability belongs to is at most . Since the total number of wires is bounded by , we have .
For the logarithm of the time complexity this yields
where the logarithmic summand stems from guessing the value of the top-level gate. We have .
We now consider the case . Suppose the -th gate has inputs that are not in . The expected value of is . There are at most possible values for the gate. Since all the bottom-level gates have the same weighted fan-in , the number of bottom-level gates is at most and . We bound the expected logarithm of the number of possible values of all gates by
where we use the concavity of the logarithm function in the penultimate step and the fact in the last step.
For the logarithm of the time complexity we get,
with savings . ∎
We now extend the algorithm to circuits with varying fan-in and show that the logarithm of the time complexities is lower bounded by a convex combination of the savings . We model the -wire circuits of varying weighted fan-in by a distribution on wires. For each weighted fan-in , the wire distribution specifies the number of wires of bottom-level gates of weighted fan-in . We denote the savings of our algorithm on circuits with wire distribution by .
Let be a parameter and be a depth two circuit with symmetric gates, variables and a weighted number of wires of , where the wires are distributed according to . There is a satisfiability algorithm for such with time complexity such that for
The algorithm is the same as above. The logarithm of the overhead of guessing the values for all bottom-level gates with fan-in is if and otherwise. Solving the system of linear equations and using linearity of expectation then yields the savings as claimed. ∎
7.2 The Algorithm as a Zero-Sum Game
The time complexity of the algorithm in Section 7.1 depends crucially on choosing a suitable parameter . Instead of trying to directly determine a good parameter by analyzing the wire distribution of the circuit, we apply a trick from game theory.
A zero-sum game with two players and is a game where both players pick a strategy and the outcome is determined by a function of the two strategies. Player tries to maximize the outcome, while player tries to minimize it. The Min-Max Theorem states that it does not matter which player moves first, as long as we allow mixed strategies for the players.
We model the task of choosing the parameter as the following zero-sum game: Player , the algorithm designer, picks some probability , and player , the circuit designer, picks a value . The outcome is , the savings of the algorithm. The algorithm designer tries to maximize the savings, and the circuit designer tries to minimize it. The wire distribution of a circuit is a mixed strategy for the circuit designer. A mixed strategy for the algorithm designer would be a distribution on the probabilities.
A direct approach for designing the algorithm would be to select the parameter depending on the circuit so that we obtain large savings. Specifically, given the wire distribution of the circuit , the algorithm designer picks a and and the outcome is a convex combination of the values . Using the Min-Max Theorem we turn this game around: The algorithm designer picks a mixed strategy and the circuit designer responds with a pure strategy , a circuit where all bottom-level gates have weighted fan-in . The following lemma shows that there is a good strategy for the algorithm designer.
There is a distribution on parameters such that for all ,
Let be the following distribution on : For with suitable constants, and , we set with probability , where is the normalization factor. We know that . The expectation of is .
Let be any pure strategy of the circuit designer and . The expected outcome of the game for these strategies is
To lower bound the expected outcome, we use a case analysis on the savings similar to the one in Section 7.1. Let as defined in the previous section. Let be the largest value such that for , we have and for we have .
Using the savings from Lemma 7.3, we have for and otherwise. The expected savings is then
Let . By the definition of we have . Hence
Hence for we get
We now conclude that for every there is a for , such that . Using that for every mixed strategy for , the savings is a convex combination of the savings for pure strategies, we conclude the same for any strategy on .
This gives us the final algorithm: Given a circuit with wire distribution , evaluate with for each as above and use the optimal for the random restriction.
The savings is tight in the sense that there is a mixed strategy on such that the expected savings is at most .
There is a wire distribution such that for any
Let be the strategy of the algorithm designer and let be the distribution such that for , and for any other . By lemma 7.4 we have
We argue that for large and , the savings is negative. Assume . There is some such that for , . Using that for any and , the savings is upper bounded by we get
For large , the expectation is therefore negative. On the other hand, if , then . ∎
In this paper, we present the first nontrivial algorithm for deciding the satisfiability of -wire threshold circuits of depth . The same result also applies to the special of case of - Integer Linear Programming with sparse constraints. The algorithm improves over exhaustive search by a factor where .
Several straightforward open questions remain. Can we further improve the savings? The savings in our algorithm is exponentially small in , while the best known savings for -size circuits is only polylogarithmically small in . Can we decrease this gap? If not, can we explain it in terms of the expressive power of the circuits?
Our algorithm handles only linear size threshold circuits of depth two. Can we obtain nontrivial satisfiability algorithms for slightly more relaxed models? For example, it would be very interesting to extend the result to larger depth circuits. It would also be nice to generalize the algorithm to deal with depth two threshold circuits with linearly many gates.
It would also be interesting to relax the restriction on the number of wires. Unfortunately, as discussed earlier, it is not be possible to obtain a constant savings algorithm for depth two threshold circuits of superlinearly many wires under .
It would be extremely interesting to find a subquadratic algorithm for the Vector Domination Problem for dimension , which would imply the refutation of .
Our algorithm is a “Split and List” algorithm , split the variable set into subsets and list all assignments to the subsets. As such, it inherently takes exponential space. Can we reduce the space requirement to polynomial space?
Acknowledgments: We thank Dominik Scheder for the fruitful discussions on the Vector Domination Problem. We also thank Shachar Lovett and Ryan Williams for pointing us to a simpler algorithm for systems of linear equations.
-  P. Austrin, S. Benabbas, A. Chattopadhyay, and T. Pitassi. Personal Communication, 2012.
-  Ravi B. Boppana and Michael Sipser. Handbook of theoretical computer science (vol. a). chapter The complexity of finite functions, pages 757–804. MIT Press, Cambridge, MA, USA, 1990.
-  Chris Calabro. The exponential complexity of satisfiability problems. PhD thesis, University of California, San Diego, 2009.
-  Chris Calabro, Russell Impagliazzo, and Ramamohan Paturi. The complexity of satisfiability of small depth circuits. In Parameterized and Exact Computation: 4th International Workshop, IWPEC 2009, Copenhagen, Denmark, September 10-11, 2009, Revised Selected Papers, pages 75–85, Berlin, Heidelberg, 2009. Springer-Verlag.
-  Ashok K Chandra, Larry Stockmeyer, and Uzi Vishkin. Constant depth reducibility. SIAM Journal on Computing, 13(2):423–439, 1984.
-  S.A. Cook. The complexity of theorem-proving procedures. In Proceedings of the Third Annual ACM Symposium on Theory of Computing, pages 151–158, 1971.
-  Evgeny Dantsin and Alexander Wolpert. Max-sat for formulas with constant clause density can be solved faster than in time. In Armin Biere and CarlaP. Gomes, editors, Theory and Applications of Satisfiability Testing - SAT 2006, volume 4121 of Lecture Notes in Computer Science, pages 266–276. Springer Berlin Heidelberg, 2006.
-  Andras Hajnal, Wolfgang Maass, Pavel Pudlak, Mario Szegedy, and Gyorgy Turan. Threshold circuits of bounded depth. In Proceedings of the 28th Annual Symposium on Foundations of Computer Science, SFCS ’87, pages 99–110, Washington, DC, USA, 1987. IEEE Computer Society.
-  R. Impagliazzo and R. Paturi. The complexity of . Journal of Computer and Systems Sciences, 62(2):367–375, March 2001. Preliminary version in 14th Annual IEEE Conference on Computational Complexity, pages 237–240, 1999.
-  R. Impagliazzo, R. Paturi, and F. Zane. Which problems have strongly exponential complexity? Journal of Computer and System Sciences, 63:512–530, 1998. Preliminary version in 39th Annual IEEE Symposium on Foundations of Computer Science, pp 653-662, 1998.
-  Russell Impagliazzo, Williams Matthews, and Ramamohan Paturi. A Satisfiability Algorithm for . In Proceedings of the 23rd Annual ACM-SIAM Symposium on Discrete Algorithms, 2012.
-  Russell Impagliazzo, Ramamohan Paturi, and Michael E. Saks. Size–depth tradeoffs for threshold circuits. SIAM J. Comput., 26(3):693–707, 1997. preliminary version published in STOC 1993.
-  L. Levin. Universal sorting problems. Problems of Information Transmission, 9:265–266, 1973.
-  Rahul Santhanam. Fighting perebor: New and improved algorithms for formula and qbf satisfiability. In Proceedings of the 2010 IEEE 51st Annual Symposium on Foundations of Computer Science, FOCS ’10, pages 183–192, Washington, DC, USA, 2010. IEEE Computer Society.
-  R. Schuler. An algorithm for the satisfiability problem of formulas in conjunctive normal form. Journal of Algorithms, 54(1):40–44, 2005.
-  Ryan Williams. A new algorithm for optimal 2-constraint satisfaction and its implications. Theoretical Computer Science, 348:357–365, 2005.
-  Ryan Williams. Improving exhaustive search implies superpolynomial lower bounds. In Proceedings of the 42nd ACM symposium on Theory of computing, STOC ’10, pages 231–240, New York, NY, USA, 2010. ACM.
-  Ryan Williams. Non-Uniform ACC Circuit Lower Bounds. In Proceedings of the Twenty-Sixth Annual IEEE Conference on Computational Complexity, 2011.