Quantifier Elimination over Finite Fields Using Gröbner BasesThis research was sponsored by National Science Foundation under contracts no. CNS0926181, no. CCF0541245, and no. CNS0931985, the SRC under contract no. 2005TJ1366, General Motors under contract no. GMCMUCRLNV301, Air Force (Vanderbilt University) under contract no. 18727S3, the GSRC under contract no. 1041377 (Princeton University), the Office of Naval Research under award no. N000141010188, and DARPA under contract FA8650-10-C-7077.

Quantifier Elimination over Finite Fields Using Gröbner Bases1

Abstract

We give an algebraic quantifier elimination algorithm for the first-order theory over any given finite field using Gröbner basis methods. The algorithm relies on the strong Nullstellensatz and properties of elimination ideals over finite fields. We analyze the theoretical complexity of the algorithm and show its application in the formal analysis of a biological controller model.

1 Introduction

We consider the problem of quantifier elimination of first-order logic formulas in the theory of arithmetic in any given finite field . Namely, given a quantified formula in the language, where is a vector of quantified variables and a vector of free variables, we describe a procedure that outputs a quantifier-free formula , such that and are equivalent in .

Clearly, admits quantifier elimination. A naive algorithm is to enumerate the exponentially many assignments to the free variables , and for each assignment , evaluate the truth value of the closed formula (with a decision procedure). Then the quantifier-free formula equivalent to is , where . This naive algorithm always requires exponential time and space, and cannot be used in practice. Note that a quantifier elimination procedure is more general and complex than a decision procedure: Quantifier elimination yields an equivalent quantifier-free formula while a decision procedure outputs a yes/no answer. For instance, fully quantified formulas over finite fields can be “bit-blasted” and encoded as Quantified Boolean Formulas (QBF), whose truth value can, in principle, be determined by QBF decision procedures. However, for formulas with free variables, the use of decision procedures can only serve as an intermediate step in the naive algorithm mentioned above, and does not avoid the exponential enumeration of values for the free variables. We believe there has been no investigation into quantifier elimination procedures that can be practically used for this theory.

Such procedures are needed, for instance, in the formal verification of cipher programs involving finite field arithmetic [16, 8] and polynomial dynamical systems over finite fields that arise in systems biology [11, 12, 4]. Take the S2VD virus competition model [11] as an example, which we study in detail in Section 6: The dynamics of the system is given by a set of polynomial equations over the field . We can encode image computation and invariant analysis problems as quantified formulas, which are solvable using quantifier elimination. As is mentioned in [11], there exists no verification method suitable for such systems over general finite fields so far.

In this paper we give an algebraic quantifier elimination algorithm for . The algorithm relies on strong Nullstellensatz and Gröbner basis methods. We analyze its theoretical complexity, and show its practical application.

In Section 3, we exploit the strong Nullstellensatz over finite fields and properties of elimination ideals, to show that Gröbner basis computation gives a way of eliminating quantifiers in formulas of the form , where the s are atomic formulas and is a quantifier block. We then show, in Section 4, that the DNF-expansion of formulas can be avoided by using standard ideal operations to “flatten” the formulas. Any quantifier-free formula can be transformed into conjunctions of atomic formulas at the cost of introducing existentially quantified variables. This transformation is linear in the size of the formula, and can be seen as a generalization of the Tseitin transformation. Combining the techniques, we obtain a complete quantifier elimination algorithm.

In Section 5, we analyze the complexity of our algorithm, which depends on the complexity of Gröbner basis computation over finite fields. For ideals in that contain for each , Buchberger’s Algorithm computes Gröbner bases within exponential time and space [13]. Using this result, the worst-case time/space complexity of our algorithm is bounded by when contains no more than two alternating blocks of quantifiers, and for more alternations. Recently a polynomial-space algorithm for Gröbner basis computation over finite fields has been proposed in [17], but it remains theoretical so far. If the new algorithm can be practically used, the worst-case complexity of quantifier elimination is for arbitrary alternations.

Note that this seemingly high worst-case complexity, as is common for Gröbner basis methods, does not prevent the algorithm from being useful on practical problems. This is crucially different from the naive algorithm, which always requires exponential cost, not just in worst cases. In Section 6, we show how the algorithm is successfully applied in the analysis of a controller design in the S2VD virus competition model [11], which is a polynomial dynamical system over finite fields. The authors developed control strategies to ensure a safety property in the model, and used simulations to conclude that the controller is effective. However, using the quantifier elimination algorithm, we found bugs that show inconsistency between specifications of the system and its formal model. This shows how our algorithm can provide a practical way of extending formal verification techniques to models over finite fields.

Throughout the paper, omitted proofs are provided in the Appendix.

2 Preliminaries

2.1 Ideals, Varieties, Nullstellensatz, and Gröbner Bases

Let be any field and the polynomial ring over with indeterminates . An ideal generated by is , Let be an arbitrary point, and be a polynomial. We say that vanishes on if .

Definition 1

For any subset of , the affine variety of over is

Definition 2

For any subset of , the vanishing ideal of is defined as

Definition 3

Let be any ideal in , the radical of is defined as

When , we say is a radical ideal. The celebrated Hilbert Nullstellensatz established the correspondence between radical ideals and varieties:

Theorem 2.1 (Strong Nullstellensatz [14])

For an arbitrary field , let be an ideal in . We have where is the algebraic closure of and

The method of Gröbner bases was introduced by Buchberger [6] for the algorithmic solution of various fundamental problems in commutative algebra. For an ideal in a polynomial ring, Gröbner basis computation transforms to a canonical representation that has many useful properties. Detailed treatment of the theory can be found in [3].

Definition 4

Let be the set of monomials in . A monomial ordering on is a well-ordering on T satisfying
(1) For any ,
(2) For all , then .

We order the monomials appearing in any single polynomial with respect to . We write to denote the leading monomial in (the maximal monomial under ), and to denote the leading term of ( multiplied by its coefficient). We write where is a set of polynomials.

Let be an ideal in . Fix any monomial order on . The ideal of leading monomials of , , is the ideal generated by the leading monomials of all polynomials in . Now we are ready to define:

Definition 5 (Gröbner Basis [3])

A Gröbner basis for is a set satisfying

2.2 The First-order Theory over a Finite Field

Let be an arbitrary finite field of size , where is a prime power. We fix the structure to be and the signature (“” is a logical predicate). For quantified formulas, we write to emphasize that the is a vector of quantified variables and is a vector of free variables.

The standard first-order theory for each consists of the usual axioms for fields [15] plus , which fixes the size of the domain. We write this theory as . In , we consider all the atomic formulas as polynomial equations . The realization of a formula is the set of assignments to its free variables that makes the formula true over . Formally:

Definition 6 (Realization)

Let be a formula with free variables . The realization of , written as , is inductively defined as:

  • (in particular, )

Proposition 1 (Fermat’s Little Theorem)

Let be a finite field. For any , we have . Conversely, .

Definition 7 (Quantifier Elimination)

admits quantifier elimination if for any formula , where the variables are quantified and the variables free, there exists a quantifier-free formula such that .

2.3 Nullstellensatz in Finite Fields

The strong Nullstellensatz admits a special form over finite fields. This was proved for prime fields in [10] and used in [4, 5]. Here we give a short proof that the special form holds over arbitrary finite fields, as a corollary of Theorem 2.1.

Lemma 1

For any ideal , is radical.

Theorem 2.2 (Strong Nullstellensatz in Finite Fields)

For an arbitrary finite field , let be an ideal, then

Proof

Apply Theorem 2.1 to and use Lemma 1. We have . But since , it follows that

Thus we obtain

3 Quantifier Elimination Using Gröbner Bases

In this section, we show that the key step in quantifier elimination can be realized by Gröbner basis computation. Namely, for any formula of the form , we can compute a quantifier-free formula such that . We use the following notational conventions:

  • is the number of quantified variables and the number of free variables. We write and , and call them field polynomials (following [10]).

  • We use to denote the assignment for the variables, and for the variables. is a complete assignment for all the variables in .

  • When we write or a formula , we assume that all the variables do occur in or . We assume that the variables always rank higher than the variables in the lexicographic order.

3.1 Existential Quantification and Elimination Ideals

First, we show that eliminating the variables is equivalent to projecting the variety from to .

Lemma 2

For , we have .

Definition 8 (Projection)

The -th projection mapping is defined as:

where . For any set , we write

Lemma 3

.

Next, we show that the projection of the variety from to , is exactly the variety .

Definition 9 (Elimination Ideal [7])

Let be an ideal. The l-th elimination ideal , for , is the ideal of defined by

The following lemma shows that adding field polynomials does not change the realization. For , we have:

Lemma 4

Now we can prove the key equivalence between projection operations and elimination ideals. This requires the use of Nullstellensatz for finite fields.

Theorem 3.1

Let be an ideal which contains the field polynomials for all the variables in . We have

Proof

We show inclusion in both directions.

  • For any , there exists such that . That is, satisfies all polynomials in ; in particular, satisfies all polynomials in that only contain the variables ( is not assigned to variables). Thus,

  • Let be a point in such that . Consider the polynomial

    vanishes on all the points in , except , since is excluded in the product for all . In particular, vanishes on all the points in , because for each , must be different from , and (since there are no variables). Therefore, is contained in the vanishing ideal of , i.e., .

    Now, Theorem 2.2 shows . Since already contains the field polynomials, we know , and consequently Since , we must have . But on the other hand, . Hence . But since , we know .∎

3.2 Quantifier Elimination using Elimination Ideals

Theorem 3.1 shows that to obtain the projection of a variety over , we only need to take the variety of the corresponding elimination ideal. In fact, this can be easily done using the Gröbner basis of the original ideal:

Proposition 2 (cf. [7])

Let be an ideal and let be the Gröbner basis of with respect to the lexicographic order . Then for every , is a Gröbner basis of the -th elimination ideal . That is,

Now, putting all the lemmas together, we arrive at the following theorem:

Theorem 3.2

Let be be a formula in , with . Let be the Gröbner basis of . Suppose then we have

Proof

We write for convenience. First, by Lemma 4, adding the polynomials and does not change the realization:

Next, by Lemma 3, the quantification on corresponds to projecting a variety:

Using Theorem 3.1, we know that the projection of a variety is equivalent to the variety of the corresponding elimination ideal, i.e., . Now, using the property of Gröbner bases in Proposition 2, we know the elimination ideal is generated by :

Finally, by Lemma 2, an ideal is equivalent to the conjunction of atomic formulas given by the generators of the ideal:

Connecting all the equations above, we have shown Note that (they do not contain variables).∎

4 Formula Flattening with Ideal Operations

If negations on atomic formulas can be eliminated (to be shown in Lemma 5), Theorem 3.2 already gives a direct quantifier elimination algorithm. That is, we can always use duality to make the innermost quantifier block an existential one, and expand the quantifier-free part to DNF. Then the existential block can be distributed over the disjuncts and Theorem 3.2 is applied. However, this direct algorithm always requires exponential blow-up in expanding formulas into DNF.

We show that the DNF-expansion can be avoided: Any quantifier-free formula can be transformed into an equivalent formula of the form , where are new variables and s are polynomials. The key is that Boolean conjunctions and disjunctions can both be turned into additions of ideals; in the latter case new variables need be introduced. This transformation can be done in linear time and space, and is a generalization of the Tseitin transformation from to general finite fields.

We use the usual definition of ideal addition and multiplication. Let and be ideals, and be a polynomial. Then and .

Lemma 5 (Elimination of Negations)

Suppose is a quantifier free formula in in NNF and contains negative atomic formulas. Then there is a formula , where contains new variables but no negative atoms, such that .

Lemma 6 (Elimination of Disjunctions)

Suppose and are two formulas in variables , and and are ideals in satisfying and . Then, using as a new variable, we have

Theorem 4.1

For any quantifier-free formula given in NNF, there exists a formula of the form such that . Furthermore, can be generated in time , and also .

Proof

Since is in NNF, all the negations occur in front of atomic formulas. We first use Lemma 5 to eliminate the negations. Suppose there are negative atomic formulas in , we obtain . Now does not contain negations.

We then prove that there exists an ideal for satisfying , where are the introduced variables (which rank higher than the existing variables in the variable ordering, so that the projection truncates assignments on the variables).

  • If is an atomic formula , then ;

  • If is of the form , then ;

  • If is of the form , then , where is new.

Note that the new variables are only introduced in the disjunction case, and therefore the number of variables equals the number of disjunctions. Following Lemma 2 and 6, the transformation preserves the realization of the formula in each case. Hence, we have . Writing , we know Notice that the number of rewriting steps is bounded by the number of logical symbols appearing in . Hence the transformation is done in time linear in the size of the formula. The number of new variables is equal to the number of negations and disjunctions. ∎

5 Algorithm Description and Complexity Analysis

We now describe the full algorithm using the following notations:

  • The input formula is given by . Each represents a quantifier block, where is either or . and are different quantifiers. We write . is a quantifier-free formula in and given in NNF, where are free variables.

  • We assume the innermost quantifier is existential, . (Otherwise we apply quantifier elimination on the negation of the formula.)

5.1 Algorithm Description

1:  Input: where is the number of quantifier alternations, is an existential block (), and is in negation normal form.
2:  Output: A quantifier-free equivalent formula of
3:  Procedure QE()
4:  while  do
5:      Eliminate_Negations()
6:      Formula_Flattening()
7:     
8:      = Gröbner_Basis()
9:     if  then
10:        
11:        break
12:     end if
13:      where
14:     
15:     for  to  do
16:        QE
17:     end for
18:     
19:     
20:  end while
21:  return  
Algorithm 1 Quantifier Elimination for

Section 3 shows how to eliminate existential quantifiers over conjunctions of positive atomic formulas. Section 4 shows how formulas can be put into conjunctions of positive atoms with new quantified variables. It follows that we can always eliminate the innermost existential quantifiers, and iterate the process by flipping the universal quantifiers into existential ones. We first emphasize some special features of the algorithm:

  • In each elimination step, a full quantifier block is eliminated. This is desirable in practical problems, which usually contain many variables but few alternating quantifier blocks. For instance, many verification problems are expressible using two blocks of quantifiers (-formulas).

  • The quantifier elimination step essentially transforms an ideal to another ideal. This corresponds to transforming conjunctions of atomic formulas to conjunctions of new atomic formulas. Therefore, the quantifier elimination steps do not introduce new nesting of Boolean operators.

  • The algorithm always directly outputs CNF formulas.

A formal description of the full algorithm is given in Algorithm 1. The main steps in the algorithm are explained below. Each loop of the algorithm contains three main steps. In Step 1, is flattened; in Step 2, the innermost existential quantifier block is eliminated; in Step 3, the next (universal) quantifier block is eliminated and the process loops back to Step 1. The algorithm terminates either after Step 2 or Step 3, when there are no remaining quantifiers to be eliminated.

Step 1: (Line 5-7)

First, since is in NNF, we use Theorem 4.1 to eliminate the negations and disjunctions in to get , where are the variables introduced for eliminating negations (Lemma 5), and are the variables introduced for eliminating disjunctions (Lemma 6).

Step 2: (Line 8-12)

Since , using Theorem 4.1, we can eliminate the variables simultaneously by computing

Now we have

If there are no more quantifiers, the output is , which is in CNF.

Step 3: (Line 13-18)

Since , we distribute the block over the conjuncts:

Now we do elimination recursively on for each , which can be done using only Step 1 and Step 2. We obtain:

(1)

and the formula becomes (note that the extra negation is distributed)

(2)

If there are no more quantifiers left, the output formula is , which is in CNF. Otherwise, , and we return to Step 1.

Theorem 5.1 (Correctness)

Let be a formula where and is in NNF. Algorithm 1 computes a quantifier-free formula , such that and is in CNF.

5.2 Complexity Analysis

The worst-case complexity of Gröbner basis computation on ideals in that contain for each variable is known to be single exponential in the number of variables in time and space. This follows from the complexity result for Gröbner basis computation of zero-dimensional radical ideals [13] (a direct proof can be found in [9]).

Proposition 3

Let be an ideal. The time and space complexity of Buchberger’s Algorithm is bounded by , assuming that the length of input () is dominated by .

Now we are ready to estimate the complexity of our algorithm.

Theorem 5.2 (Complexity)

Let be the input formula with quantifier blocks. When , the time/space complexity of Algorithm 1 is bounded by . Otherwise, it is bounded by .

Proof

The complexity is dominated by Gröbner basis computation, whose complexity is determined by the number of variables occurring in the ideal. When , the main loop is executed once, and the number of newly introduced variables is bounded by the original length of the input formula. Therefore, Gröbner basis computations can be done in single exponential time/space. When , the number of newly introduced variables is bounded by the length of the formula obtained from the previous run of the main loop, which can itself be exponential in the number of the remaining variables. In that case, Gröbner basis computation can take double exponential time/space.

Case :

In Step 1, the number of the introduced and variables equals to the number of negations and disjunctions that appear in the . Hence the total number of variables is bounded by the length of