Counting Value Sets: Algorithm and Complexity

# Counting Value Sets: Algorithm and Complexity

Qi Cheng School of Computer Science, The University of Oklahoma, Norman, OK 73019, USA. Email: qcheng@cs.ou.edu. Partially supported by NSF.    Joshua E. Hill Department of Mathematics, University of California, Irvine, CA 92697, USA. Email: hillje@math.uci.edu. Partially supported by NSF.    Daqing Wan Department of Mathematics, University of California, Irvine, CA 92697, USA. Email: dwan@math.uci.edu. Partially supported by NSF.
###### Abstract

Let be a prime. Given a polynomial in of degree over the finite field , one can view it as a map from to , and examine the image of this map, also known as the value set. In this paper, we present the first non-trivial algorithm and the first complexity result on computing the cardinality of this value set. We show an elementary connection between this cardinality and the number of points on a family of varieties in affine space. We then apply Lauder and Wan’s -adic point-counting algorithm to count these points, resulting in a non-trivial algorithm for calculating the cardinality of the value set. The running time of our algorithm is . In particular, this is a polynomial time algorithm for fixed if is reasonably small. We also show that the problem is #P-hard when the polynomial is given in a sparse representation, , and is allowed to vary, or when the polynomial is given as a straight-line program, and is allowed to vary. Additionally, we prove that it is NP-hard to decide whether a polynomial represented by a straight-line program has a root in a prime-order finite field, thus resolving an open problem proposed by Kaltofen and Koiran in [4, 5].

## 1 Introduction

In a finite field with ( prime) elements, , take a polynomial, with degree . Denote the image set of this polynomial as

 Vf={f(α)∣α∈Fq}

and denote the cardinality of this set as .

There are a few trivial bounds that can be immediately established. There are only elements in the field, so . Additionally, any polynomial of degree can have at most roots, thus for all , is satisfied at most times. This is true for every element in , so , whence

 ⌈qd⌉≤#(Vf)≤q

(where is the ceiling function).

Both of these bounds can be achieved: if , then is called a permutation polynomial and if , then is said to have a “minimal value set”.

The problem of establishing has been studied in various forms for at least the last 115 years, but exact formulations for are known only for polynomials in very specific forms. Results that apply to general polynomials are asymptotic in nature, or provide estimates whose errors have reasonable bounds only on average .

The fundamental problem of counting the value set cardinality can be thought of as a much more general version of the problem of determining if a particular polynomial is a permutation polynomial. Shparlinski  provided a baby-step giant-step type test that determines if a given polynomial is a permutation polynomial by extending  to an algorithm that runs in . This is still fully exponential in . Ma and von zur Gathen  provide a ZPP (zero-error probabilistic polynomial time) algorithm for testing if a given polynomial is a permutation polynomial. According to , the first deterministic polynomial time algorithm for testing permutation polynomials is obtained by Lenstra using the classification of exceptional polynomials which in turn depends on the classification of finite simple groups. Subsequently, an elementary approach based on the Gao-Kaltofen-Lauder factorization algorithm is given by Kayal .

For the more general problem of exactly computing , essentially nothing is known about this problem’s complexity and no non-trivial algorithms are known. For instance, no baby-step giant-step type algorithm is known in computing . No probabilistic polynomial time algorithm is known. Finding a non-trivial algorithm and proving a non-trivial complexity result for the value counting were raised as open problems in , where a probabilistic approximation algorithm is given. In this paper, we provide the first non-trivial algorithm and the first non-trivial complexity result for the exact counting of the value set problem.

### 1.1 Our results

Perhaps the most obvious method to calculate is to evaluate the polynomial at each point in and count how many distinct images result. This algorithm has a time and space complexity . One can also approach this problem by operating on points in the co-domain. One has for some if and only if has a zero in ; this algorithm again has a time complexity , but the space complexity is improved considerably to .

In this paper we present several results on determining the cardinality of value sets. On the algorithmic side, we show an elementary connection between this cardinality and the number of points on a family of varieties in affine space. We then apply the Lauder-Wan -adic point-counting algorithm, resulting in a non-trivial algorithm for calculating the image set cardinality in the case that is sufficiently small (i.e., for some positive constant ). Precisely, we have

###### Theorem 6.

There exists an explicit deterministic algorithm and an explicit polynomial such that for any of degree , where ( prime), the algorithm computes the cardinality of the image set, , in a number of bit operations bounded by .

The running time of this algorithm is polynomial in both and , but is exponential in . In particular, this is a polynomial time algorithm for fixed if the characteristic is small ( can be large).

On the complexity side, we have several hardness results on the value set problem. With a field of characteristic , we have

###### Theorem 3.

The problem of counting the value set of a sparse polynomial over a finite field of characteristic is #P-hard.

The idea of our proof of this theorem is to reduce the problem of counting satisfying assignments for a 3SAT formula to the problem of value set counting.

Over a prime-order finite field, we have

###### Theorem 5.

Over a prime-order finite field , the problem of counting the value set is #P-hard under RP-reduction, if the polynomial is given as a straight-line program.

Additionally, we prove that it is NP-hard to decide whether a polynomial in represented by a straight-line program has a root in a prime-order finite field, thus resolving an open problem proposed in [4, 5]. We accomplish the complexity results over prime-order finite fields by reducing the prime-order finite field subset sum problem (PFFSSP) to these problems.

In the PFFSSP, given a prime , an integer and a set of integers , we want to decide the solvability of the equation

 a1x1+a2x2+⋯+atxt≡b(modp)

with for . The main idea comes from the observation that if , there is a sparse polynomial such that as runs over , the vector

 (α(x),α(x+1),⋯,α(x+t−1))

runs over all the elements in . In fact, a lightly modified version of the quadratic character suffices. So the PFFSSP can be reduced to deciding whether the shift sparse polynomial has a solution in .

## 2 Background

### 2.1 The subset sum problem

To prove the complexity results, we use the subset sum problem (SSP) extensively. The SSP is a well-known problem in computer science. In one instance of the SSP, given an integer and a set of positive integers ,

1. (Decision version) the goal is to decide whether there exists a subset such that the sum of all the integers in equals ,

2. (Search version) the goal is to find a subset such that the sum of all the integers in equals ,

3. (Counting version) the goal is to count the number of subsets such that the sum of all the integers in equals .

The decision version of the SSP is a classical NP-complete problem. The counting version of the SSP is #P-complete, which can be easily derived from proofs of the NP-completeness of the decision version, e.g. [2, Theorem 34.15].

One can view the SSP as a problem of solving the linear equation

 a1x1+a2x2+⋯+atxt=b

with for . The prime-order finite field subset sum problem is a similar problem where in addition to and , one is given a prime , and the goal is to decide the solvability of the equation

 a1x1+a2x2+⋯+atxt≡b(modp)

with for .

###### Proposition 1.

The prime-order finite field subset sum problem is NP-hard under RP-reduction.

###### Proof.

To reduce the subset sum problem to the prime-order finite field subset sum problem, one finds a prime , which can be done in randomized polynomial time. ∎

###### Remark 1.

To make the reduction deterministic, one needs to de-randomize the problem of finding a large prime, which appears to be hard .

### 2.2 Polynomial representations

There are different ways to represent a polynomial over a field . The dense representation lists all the coefficients of a polynomial, including the zero coefficients. The sparse representation lists only the nonzero coefficients, along with the degrees of the corresponding terms. If most of the coefficients of a polynomial are zero, then the sparse representation is much shorter than the dense representation. A sparse shift representation of a polynomial in is a list of triples which represents the polynomial

 ∑1≤i≤nai(x+bi)ei.

More generally, a straight-line program for a univariate polynomial in or is a sequence of assignments, starting from and . After that, the -th assignment has the form

 xi=xj⊙xk

where and is one of the three operations . We first let be an element in such that . A straight-line program for a univariate polynomial in can be defined similarly, except that the sequence starts from and . One can verify that a straight-line program computes a univariate polynomial, and that sparse polynomials and sparse shift polynomials have short straight-line programs. A polynomial produced by a short straight-line program may have very high degree, and most of its coefficients may be nonzero, so it may be costly to write it in either a dense form or a sparse form.

## 3 Hardness of solving straight-line polynomials

It is known that deciding whether there is a root in a finite field extension for a sparse polynomial is NP-hard . In a related work, it was shown that deciding whether there is a -adic rational root for a sparse polynomial is NP-hard . However, the complexity of deciding the solvability of a straight-line polynomial in within a prime-order finite field was not known. This open problem was proposed in  and . We resolve this problem within this section, and this same idea will be used later on to prove the hardness result of the value set counting problem.

Let be an odd prime. Let be the quadratic character modulo , namely equals , depending on whether is a quadratic residue, a quadratic non-residue, or is congruent to modulo . For , . Consider the list

 χ(1),χ(2),⋯,χ(p−1). (1)

It is a sequence in . The following bound is a standard consequence of the celebrated Weil bound for character sums, see  for a detailed proof.

###### Proposition 2.

Let be a sequence in . Then the number of such that

 χ(x)=b1,χ(x+1)=b2,⋯,χ(x+t−1)=bt

is in the range .

The proposition implies that if , then every possible sequence in occurs as a consecutive sub-sequence in expression (1). In many situations it is more convenient to use binary sequences, which suggests instead using the polynomial , but this results in a small problem at . We instead use the sparse polynomial

 α(x)=(x(p−1)/2+xp−1)/2.

takes value in if and iff .

###### Corollary 1.

If , then for any binary sequence , there exists a such that

 α(x)=b1,α(x+1)=b2,⋯,α(x+t−1)=bt.

In other words, if , the map

 x↦(α(x),α(x+1),⋯,α(x+t−1))

is an onto map from to ; this map thus sends an algebraic object to a combinatorial object.

Given a straight-line polynomial and a prime , how hard is it to decide whether the polynomial has a solution in ? We now prove that this problem is NP-hard.

###### Theorem 1.

Given a sparse shift polynomial , and a large prime , it is NP-hard to decide whether has a root in .

###### Proof.

We reduce the (decision version of the) subset sum problem to this problem. Given and , one finds a prime and constructs a sparse shift polynomial

 β(x)=t−1∑i=0aiα(x+i)−b. (2)

If the polynomial has a solution modulo , then the answer to the subset sum problem is “yes”, since for any , .

In the other direction, if the answer to the subset sum problem is “yes”, then according to Corollary 1, the polynomial has a solution in . Note that the reduction can be computed in randomized polynomial time. ∎

## 4 Complexity of the value set counting problem

In this section, we prove several results about the complexity of the value set counting problem.

### 4.1 Finite field extensions

We will use a problem about circuits to prove that counting the value set of a sparse polynomial in a binary field is #P-hard. A Boolean circuit is in if every output bit of the circuit depends only on at most input bits. We can view a circuit with input bits and output bits as a map from to and call the image of the map the value set of the circuit. The following proposition is implied in . We will sketch the proof for completeness.

###### Proposition 3.

Given a 3SAT formula with variables and clauses, one can construct in polynomial time an circuit with input bits and outputs bits, such that if there are satisfying assignments for the 3SAT formula, then the cardinality of the value set of the circuit is . In particular, if the 3SAT formula can not be satisfied, then the circuit computes a permutation from to .

###### Proof.

Denote the variables of the 3SAT formula by , and the clauses of the 3SAT formula by . Build a circuit with input bits and output bits as follows. The input bits will be denoted by and output bits will be denoted by . Set for . And set

 wi=(Ci∧(yi⊕y(i+1(modm))))∨(¬Ci∧yi)

for . In other words, if is evaluated to be TRUE, then output as , and otherwise output as . Note that depends only on variables from , thus we obtain an circuit. After fixing an assignment to ’s, ’s are also fixed, and the transformation from to is linear over . One can verify that the linear transformation has rank if the assignment satisfies all the clauses, and it has rank (namely it is full rank) if some of the clauses are not satisfied. So the cardinality of the value set of the circuit is

 M2m−1+(2n−M)2m=2n+m−2m−1M.

If we replace the Boolean gates in the circuit by algebraic gates over , we obtain an algebraic circuit that computes a polynomial map from to itself, where each polynomial depends only on variables and has degree equal to or less than . There is an -basis for , say which induces a bijection from to given by

 (x1,x2,⋯,xn+m)↦x=n+m∑i=1xiωi

which has an inverse that can be represented by sparse polynomials in . Using this fact, we can replace the input bits of the algebraic circuit by sparse polynomials, and collect the output bits together using the base to form a single element in . We thus obtain a sparse univariate polynomial in from the circuit such that their value sets have the same cardinality. We thus have the following theorem:

###### Theorem 2.

Given a 3SAT formula with variables and clauses, one can construct in polynomial time a sparse polynomial in such that the value set of has cardinality , where is the number of satisfying assignments of the 3SAT formula.

Since counting the number of satisfying assignments for a 3SAT formula is #P-complete, we have our main theorem:

###### Theorem 3.

The problem of counting the value set of a sparse polynomial over a finite field of characteristic is #P-hard.

### 4.2 Prime-order finite fields

The construction in Theorem 2 relies on building field extensions. The technique cannot be adopted easily to the prime-order finite field case. We will prove that counting the value set of a straight-line polynomial over prime-order finite field is #P-hard. We reduce the counting version of subset sum problem to the value set counting problem.

###### Theorem 4.

Given access to an oracle that solves the value set counting problem for straight-line polynomials over prime-order finite fields, there is a randomized polynomial-time algorithm solving the counting version of the SSP.

###### Proof.

Given an instance of the counting subset sum problem, and , if , we answer ; if , then we answer . Otherwise we find a prime and ask the oracle to count the value set of the shift sparse polynomial

 f(x):=(1−β(x)p−1)(t−1∑i=0α(x+i)2i)

over the prime-order field . We output the answer , which is easily seen to be exactly the number of subsets of which sum to . ∎

Since the counting version of the SSP is #P-complete, this theorem yields

###### Theorem 5.

Over a prime-order finite field , the problem of counting the value set is #P-hard under RP-reduction, if the polynomial is given as a straight-line program.

## 5 The Image Set and Point Counting

###### Proposition 4.

If is a polynomial of degree , then the cardinality of its image set is

 #(Vf)=d∑i=1(−1)i−1Niσi(1,12,…,1d) (3)

where and denotes the th elementary symmetric function on elements.

###### Proof.

For any , define

 ~Nk,y={(x1,…,xk)∈Fkq∣f(x1)=⋯=f(xk)=y}

and denote the corresponding cardinality of these sets as

 Nk,y=#(~Nk,y)

and finally, note that

 Nk=∑y∈VfNk,y (4)

Let us refer to the right hand side of (3) as ; plugging (4) into this expression and rearranging, we get

 η=∑y∈Vfd∑i=1(−1)i−1Ni,yσi(1,12,…,1d).

Let us call the inner sum , that is:

 ωy=d∑i=1(−1)i−1Ni,yσi(1,12,…,1d).

If we can show that for all we have , then we clearly have .

Let be fixed. Let . It is clear that and for . Substituting this in, our expression mercifully becomes somewhat nicer:

 ωy =1−d∑i=0(−1)ikiσi(1,12,…,1d) =1−d∑i=0(−1)iσi(k1,k12,…,k1d) (5) =1−[(1−k1)(1−k12)⋯(1−k1d)] (6) =1.

From step (5) to step (6), we are using the identity

 n∏j=1(λ−Xj)=n∑j=0(−1)jλn−jσj(X1,…,Xn).

Note that the bracketed term of (6) is , as must be an integer such that , so one term in the product will be .

Thus, we have , as desired. ∎

Proposition 4 gives us a way to express in terms of the numbers of rational points on a sequence of curves over . If we had a way of getting for , then it would be easy to calculate .

The spaces aren’t of any nice form (in particular, we cannot assume they are non-singular projective, abelian varieties, etc.), so we proceed by using the -adic point counting method described in , which works for any variety over a field of small characteristic (i.e., for some positive constant ).

###### Theorem 6.

There exists an explicit deterministic algorithm and an explicit polynomial such that for any of degree , where ( prime), the algorithm computes the cardinality of the image set, , in a number of bit operations bounded by .

###### Proof.

Recall that with

 ~Nk ={(x1,…,xk)∈Fkq∣f(x1)=⋯=f(xk)} =⎧⎪ ⎪ ⎪ ⎪ ⎪⎨⎪ ⎪ ⎪ ⎪ ⎪⎩(x1,…,xk)∈Fkq∣∣ ∣ ∣ ∣ ∣∣f(x1)−f(x2)=0f(x1)−f(x3)=0⋮f(x1)−f(xk)=0⎫⎪ ⎪ ⎪ ⎪ ⎪⎬⎪ ⎪ ⎪ ⎪ ⎪⎭.

For reasons soon to become clear, we need to represent this as the solution set of a single polynomial. Let us introduce additional variables to , and denote and . Now examine the auxiliary function

 Fk(x,z)=z1(f(x1)−f(x2))+⋯+zk−1(f(x1)−f(xk)). (7)

Clearly, if , then is the zero function. If , then the solutions of specify a -dimensional -linear subspace of . Thus, if we denote the cardinality of the solution set to as , then we see that

 #(Fk) =qk−1Nk+qk−2(qk−Nk) =Nkqk−2(q−1)+q2k−2.

Solving for , we find that

 Nk=#(Fk)−q2k−2qk−2(q−1). (8)

Thus we have an easy way to determine what is depending on the number of points on this hypersurface defined by the single polynomial equation .

The main theorem in  yields an algorithm for toric point counting in for small characteristic (i.e., for some positive constant ) that works for general varieties. In [8, §6.4], this theorem is adapted to be a generic point counting algorithm.

Adapting this result to our problem, we see that has a total degree of , is in variables, and that we only care about the case where . Thus, the runtime for this algorithm is bit operations. In order to calculate using equation (3), we calculate for , scaled by an elementary symmetric polynomial. All of the necessary elementary symmetric polynomials can be evaluated using Newton’s identity (see ) in less than multiplications. As such, the entire calculation has a runtime of bit operations. For consistency with , we can then note that as , we can write . Thus, there is a polynomial, , in one variable such that the runtime of this algorithm is bounded by bit operations. In the dense polynomial model, the polynomial has input size , so this algorithm does not have polynomial runtime with respect to the input length. This algorithm has runtime that is exponential in the degree of the polynomial, , and polynomial in and . ∎

## 6 Open Problems

Though value sets of polynomials appear to be closely related to zero sets, they are not as well-studied. There are many interesting open problems about value sets. The most important one is to find a counting algorithm with running time , that is, a deterministic polynomial time algorithm in the dense model. It is not clear if this is always possible. Our result affirmatively solves this problem for fixed if characteristic is reasonably small. We conjecture that the same result is true for fixed and all characteristic .

For the complexity side, can one prove that the counting problem for sparse polynomials in prime-order finite fields is hard? Can one prove that the counting problem for dense input model is hard for general degree ?

#### Acknowledgment:

We thank Dr. Tsuyoshi Ito for pointing out the reference  to us.

## References

•  Martin Avendano, Ashraf Ibrahim, J. Maurice Rojas, and Korben Rusek. Randomized np-completeness for p-adic rational roots of sparse polynomials in one variable. In ISSAC, pages 331–338, 2010.
•  T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein. Introduction to algorithms. MIT electrical engineering and computer science series. MIT Press, 2001.
•  B. Durand. Inversion of 2d cellular automata: some complexity results. Theoretical Computer Science, 134(2):387–401, 1994.
•  Erich Kaltofen. Polynomial factorization: a success story. In The 2003 international symposium on Symbolic and algebraic computation (presentation), ISSAC ’03, 2003.
•  Erich Kaltofen and Pascal Koiran. On the complexity of factoring bivariate supersparse (lacunary) polynomials. In Proceedings of the 2005 international symposium on Symbolic and algebraic computation, ISSAC ’05, pages 208–215, New York, NY, USA, 2005. ACM.
•  Neeraj Kayal. Solvability of a system of bivariate polynomial equations over a finite field (extended abstract). In Automata, languages and programming, volume 3580 of Lecture Notes in Comput. Sci., pages 551–562. Springer, Berlin, 2005.
•  Aviad Kipnis and Adi Shamir. Cryptanalysis of the hfe public key cryptosystem by relinearization. In Proceedings of the 19th Annual International Cryptology Conference on Advances in Cryptology, CRYPTO ’99, pages 19–30, London, UK, 1999. Springer-Verlag.
•  Alan G. B. Lauder and Daqing Wan. Counting points on varieties over finite fields of small characteristic. In J.P. Buhler and P. Stevenhagen, editors, Algorithmic Number Theory, pages 579 – 612. Cambridge University Press, 2008.
•  Keju Ma and Joachim von zur Gathen. The computational complexity of recognizing permutation functions. Computational Complexity, 5(1):76–97, 1995.
•  Keju Ma and Joachim von zur Gathen. Tests for permutation functions. Finite Fields and their Applications, 1(1):31–56, 1995.
•  D. G. Mead. Newton’s identities. The American Mathematical Monthly, 99(8):pp. 749–751, 1992.
•  René Peralta. On the distribution of quadratic residues and nonresidues modulo a prime number. Mathematics of Computation, 58(197):433–440, 1992.
•  I. E. Shparlinski. A deterministic test for permutation polynomials. Computational Complexity, 2(2):129–132, 1992.
•  Terence Tao, Ernie Croot III, and Harald Helfgott. Deterministic methods to find primes. Mathematics of Computation, 2011. To appear.
•  Joachim von zur Gathen. Tests for permutation polynomials. SIAM Journal on Computing, 20(3):591–602, 1991.
You are adding the first comment!
How to quickly get a good reply:
• Give credit where it’s due by listing out the positive aspects of a paper before getting into which changes should be made.
• Be specific in your critique, and provide supporting evidence with appropriate references to substantiate general statements.
• Your comment should inspire ideas to flow and help the author improves the paper.

The better we are at sharing our knowledge with each other, the faster we move forward.
The feedback must be of minimum 40 characters and the title a minimum of 5 characters   