Invariant Generation with Hypergeometric Sequences

Automated Generation of Non-Linear Loop Invariants Utilizing Hypergeometric Sequences

Andreas Humenberger, Maximilian Jaroschek, Laura Kovács Technische Universität WienInstitut für Informationssysteme 184Favoritenstraße 9–11ViennaA–1040Austria

Analyzing and reasoning about safety properties of software systems becomes an especially challenging task for programs with complex flow and, in particular, with loops or recursion. For such programs one needs additional information, for example in the form of loop invariants, expressing properties to hold at intermediate program points. In this paper we study program loops with non-trivial arithmetic, implementing addition and multiplication among numeric program variables. We present a new approach for automatically generating all polynomial invariants of a class of such programs. Our approach turns programs into linear ordinary recurrence equations and computes closed form solutions of these equations. The computed closed forms express the most precise inductive property, and hence invariant. We apply Gröbner basis computation to compute a basis of the polynomial invariant ideal, yielding thus a finite representation of all polynomial invariants. Our work significantly extends the class of so-called P-solvable loops by handling multiplication with the loop counter variable. We implemented our method in the Mathematica package Aligator and showcase the practical use of our approach.

program analysis, loop invariants, recurrence relations, hypergeometric sequences
copyright: rightsretainedconference: The 42nd International Symposium on Symbolic and Algebraic Computation; July 2017; Kaiserslautern, Rheinland-Pfalz, Germanyjournalyear: 2017ccs: Theory of computation Invariantsccs: Theory of computation Automated reasoningccs: Theory of computation Program verificationccs: Mathematics of computing Discrete mathematics\mmaDefineMathReplacement

[≤]¡=≤ \mmaDefineMathReplacement[≥]¿=≥ \mmaDefineMathReplacement[≠]!=≠ \mmaDefineMathReplacement[→]-¿→[2] \mmaDefineMathReplacement[⧴]:¿:→[2] \mmaDefineMathReplacement∉∉ \mmaDefineMathReplacement∞∞ \mmaSet morefv=gobble=2, linklocaluri=mma/symbol/definition:#1, morecellgraphics=yoffset=1.9ex

1. Introduction

1.1. Overview

Analysis and verification of software systems requires non-trivial automation. Automatic generation of program properties describing safety and/or liveness is a key step to such automation, in particular in the presence of program loops (or recursion). For programs with loops one needs additional information, in the form of loop invariants or conditions on ranking functions.

In this paper we focus on loop invariant generation for programs with assignments implementing numeric computations over scalar variables. Our programming model extends the class of so-called P-solvable loops. Our work is based on and extends results of (Rodríguez-Carbonell and Kapur, 2007; Kovács, 2007), in particular it relies on the fact that the set of polynomial invariants of P-solvable loops form a polynomial ideal and we employ reasoning about C-finite and hypergeometric sequences to determine algebraic dependencies. We show how to compute the ideal of polynomial invariants of extended P-solvable loops as follows: we model programs as a system of recurrence equations and compute closed form sequence solutions of these recurrences. If these sequences are of a certain type, which includes, among others, polynomials, rational functions, exponential and factorial sequences, then we compute a set of generators of the polynomial invariant ideal via Gröbner bases. We implemented our approach in the Mathematica package Aligator (Kovács, 2008) that is able to compute polynomial loop invariants for programs that, to the best of our knowledge, no other approach is able to handle.

This paper is organized as follows. In Section 2, we state basic definitions and facts about the algebra of linear ordinary recurrence operators as well as C-finite and hypergeometric sequences. We also give a precise definition of the programming model we take into consideration, particularly the notion of imperative loops with assignment statements only. This is followed by a description of the class of P-solvable loops and its reach and limitations in Section 3. In Section 4 we present our main contribution, an extension of P-solvable loops by reasoning about hypergeometric sequences and we derive the necessary theoretical and algorithmical results to offer fully automated polynomial invariant generation therein. We conclude the paper with a presentation of our implementation in the Mathematica package Aligator in Section 5 and a summary of possible future research directions in Section 6.

1.2. Related Work

Many classical data flow analysis problems, such as constant propagation and finding definite equalities among program variables, can be seen as problems about polynomial identities expressing loop invariants. In (Müller-Olm and Seidl, 2004; Sankaranarayanan et al., 2004) a method built upon linear and polynomial algebra is developed for computing polynomial equalities of a bounded degree. A related approach was also proposed by (Rodriguez-Carbonell and Kapur, 2007) using abstract interpretation. Abstract interpretation is also used in (Farzan and Kincaid, 2015; de Oliveira et al., 2016) for computing polynomial invariants of programs whose assignments can be described by C-finite recurrences. In our work we do not rely on abstract interpretation but use algebraic reasoning about holonomic sequences. For program loops with assignments only, our technique can handle programs with more complex arithmetic than the previously mentioned methods. Our work is currently restricted though to single-path loops.

Without an a priori fixed polynomial degree, in (Rodríguez-Carbonell and Kapur, 2007) the polynomial invariant ideal is approximated by a fixed point procedure based on polynomial algebra and abstract interpretation. In (Kovács, 2007), the author defines the notion of P-solvable loops which strictly generalizes the programming model of (Rodríguez-Carbonell and Kapur, 2007). Given a P-solvable loop with assignments and nested conditionals, the results in (Kovács, 2007) yield an automatic approach for computing all polynomial loop invariants. Our work extends (Kovács, 2007; Rodríguez-Carbonell and Kapur, 2007) in new ways: it handles a richer class of P-solvable loops where multiplication with the loop counter is allowed. Our technique relies on manipulating hypergeometric sequences and relaxes the algebraic restrictions of (Kovács, 2007; Rodríguez-Carbonell and Kapur, 2007) on program operations. To the best of our knowledge, no other method is able to derive polynomial invariants for extended P-solvable loops. Unlike (Kovács, 2007; Rodríguez-Carbonell and Kapur, 2007), we however only treat loops with assignments; that is, invariants for extended P-solvable loops with conditionals are not yet treated by our approach.

2. Preliminaries

In this section we give a brief overview of the algebra of linear ordinary recurrence operators as well as C-finite and hypergeometric sequences that we use further on. We also describe our programming model in detail.

2.1. Recurrence Operators and Holonomic Sequences

Let be a computable field of characteristic zero.

The algebra of linear ordinary recurrence operators in one variable will serve as the algebraic foundation to deal with recurrence equations. For details on general Ore algebras, see (Bronstein and Petkovšek, 1996; Ore, 1933).

Definition 2.1 ().

Let be the set of univariate polynomials in the variable over the set of rational functions in and let be the forward shift operator in , i.e.  for . We define the Ore polynomial ring of ordinary recurrence operators with component-wise addition and the unique distributive and associative extension of the multiplication rule

to arbitrary polynomials in . To clearly distinguish this ring from the commutative polynomial ring over , we denote it by . The order of an operator is its degree in .

Without loss of generality, we assume that the leading coefficient of any operator is equal to 1. Otherwise, we can divide by the leading coefficient of from the left. is a right Euclidean domain, i.e. we have the notion of the greatest common right divisor and the least common left multiple of operators and we are able to determine both algorithmically. Consequently, is a principal left ideal domain and every left ideal is generated by the greatest common right divisor of a given set of generators.

Consider the ring of all sequences in with component-wise addition and the Hadamard product (i.e. component-wise product) as multiplication. We follow (Petkovšek et al., 1996) in identifying sequences as equal if they only differ in finitely many terms. This will prove beneficial in two ways. Firstly, it allows us to define the action of operators on sequences in a natural way. Secondly, disregarding finitely many starting values makes it possible to identify unnecessary loop variables, whose values are eventually equal to the values of another variable, and therefore can be computed outside of any while loop. Let be the equivalence relation on defined by

We then set to be the quotient ring . Subsequently, it will not be necessary to distinguish between and , where is the canonical homomorphism. The field  can be embedded in via the map . The action of an operator in on an element in is defined by the map

where the evaluation is well defined for all for some , and we set . If , then we say that is an annihilator of ( annihilates ) and is a solution of . A sequence that is annihilated by a non-zero operator in is called holonomic sequence. For a given sequence , the set of all its annihilators forms a left ideal in . We call it the annihilator ideal of and denote it by .

Example 2.2 ().

Let be a polynomial in . The polynomial sequence is annihilated by the operator

is a generator of the annihilator ideal of . Set . Then is again a polynomial sequence with . It follows that is another annihilator of in and its coefficients are independent of . Since generates , there exists an operator with .

In our work, we focus on two different special kinds of holonomic sequences:

Definition 2.3 ().

Let . Then

  • is called C-finite if it is annihilated by an operator in with only constant coefficients. )

  • is called hypergeometric if it is annihilated by an order 1 operator in .

Example 2.4 ().

We give some examples of commonly encountered sequences.

  • As was shown in Example 2.2, polynomial sequences are both, C-finite and hypergeometric.

  • Rational function sequences , , are hypergeometric but not C-finite.

  • The factorial sequence is hypergeometric but not C-finite.

  • The Fibonacci sequence with

    is C-finite but not hypergeometric.

  • The sequence of harmonic numbers with

    is neither hypergeometric nor C-finite.

In a sufficiently large algebraic field extension , every C-finite sequence can be uniquely written (up to reordering) in the form

for some and for with for . For any and , is defined as . Then every hypergeometric sequence can be uniquely written (up to reordering) in the form

for some , , , and for , and the difference is not an integer for . From these closed forms it is immediate that finite sums and products of C-finite sequences are again C-finite and finite products of hypergeometric sequences are again hypergeometric. Sums of hypergeometric sequences are not necessarily hypergeometric, see Lemma 4.3. Subsequently, we will assume that is large enough so that all occurring C-finite and hypergeometric sequences have a closed form representation in .

For more details on C-finite and hypergeometric sequences, as well as proofs for the facts given in this section, see (Kauers and Paule, 2011).

For functions with that are algebraically independent over , we distinguish between the polynomial ring , where are used as variables, and the ring of all sequences of the form with . This distinction is important, as e.g. the function is algebraically independent over , but the sequence is not, and thus is isomorphic to , but is not.

Remark 0 ().

In the context of this paper, since the operators in question emerge from program loops, we can safely assume that the rational function coefficients of any operator do not have poles in . Otherwise, a division by zero error would occur for some program input.

2.2. Programming Model

We consider a simple programming model of single-path loops with rational function assignments. That is, nested loops and/or loops with conditionals are not yet handled in our work. Our programming model is thus given by the following loop pattern, written in a C-like syntax:

while  do
end while

where are (scalar) variables with values from , the are rational functions over in variables and is a a Boolean formula (loop condition) over . In our approach however we ignore loop conditions and treat program loops as non-deterministic programs. In (Müller-Olm and Seidl, 2004), it is shown that the set of all affine equality invariants is not computable if the programming model includes affine equality tests/conditions. With this consideration, our programming model from (1) becomes:

while true do
end while

Due to particular importance in our reasoning, we suppose that there is always a variable denoting the loop iteration counter. The initial value of will always be and will be incremented by at the end of each iteration.

Each program variable gives rise to a sequence . For a program variable , we allow ourselves to abuse the notation and also use the identifier as a variable in polynomial rings as well as an identifier for the sequence .

A polynomial loop invariant is a non-zero polynomial over in variables such that for all . As observed in (Rodríguez-Carbonell and Kapur, 2007; Kovács, 2007), the set of all polynomial invariants forms a polynomial ideal in , called the polynomial invariant ideal and is denoted by . For a subset , we define

In general, polynomial loop invariants depend on the initial values of program variables. To simplify the presentation, we fix to be

for a computable field of characteristic zero that allows us to represent all occurring C-finite and hypergeometric sequences in closed form, and sufficiently many variables that represent the initial values of the program variables .

3. Polynomial Invariants for P-Solvable Loops

We now turn our attention to the class of P-solvable loops introduced in (Kovács, 2007) that allows for computing all polynomial loop invariants..

Definition 3.1 ().

An imperative loop with assignment statements only is called P-solvable if the sequence of each recursively changed program variable is C-finite and the ideal of all polynomial invariants over is not the zero ideal.

Example 3.2 ().

In (Kovács, 2007), it is shown that the Euclidean algorithm is P-solvable. Given the program:

while  do
end while

The ideal of polynomial loop invariants is shown to be

With and , this gives .

While P-solvable loops cover a wide class of program loops, there are several significant cases which do not fall into this class. Notably, multiplication with the loop counter will generally result in loops that are not P-solvable.

Example 3.3 ().

Consider the following loop with relevant loop variables . The variables are temporary variables used to access previous values of . Along with the loop counter , we will not take them into consideration for the loop invariants in this example.

while true do
end while

The program then satisfies the following system of recurrences:

This loop is not P-solvable as, for example, the variable is updated by a sequence that is not C-finite (due to the multiplication between the program variables and ). To the best of our knowledge, none of the existing invariant generation techniques is able to to compute polynomial invariants for this loop. In the next section, we extend the class of P-solvable loops, covering also programs as the one above, and introduce an automated approach to derive all polynomial invariants of such loops.

4. Extension of P-Solvable Loops

4.1. Definition of Extended P-Solvable Loops

Consider the sequences with values in given by


where , the are polynomials in , not identically zero for finitely many , and the and are elements of for , with and for .

In particular, this class of sequences comprises C-finite sequences as well as hypergeometric sequences and Hadamard products of C-finite and hypergeometric sequences, which could not be handled in automated invariant generation before. We give an extension of Definition 3.1 based on this class of sequences

Definition 4.1 ().

An imperative loop with assignment statements only is called extended P-solvable if the sequence of each recursively changed program variable is of the form (3).

Note that in Definition 4.1, we drop the requirement of Definition 3.1 that the ideal of algebraic relations is not the zero ideal. This change is just for convenience.

While it is obvious that the inclusion of hypergeometric terms in extended P-solvable loops allows assignments of the form , where is a rational function in , it also allows assignments that turn into higher order recurrences, as illustrated in Example 4.2. It also allows for assignments of the form , with , as long as the closed form of is a rational function in .

4.2. Detecting Extended P-Solvable Loops

In order to employ the ideas we develop in Section 4.3 for finding algebraic relations in extended P-solvable loops, we have to be able to detect sequences of the form (3). This means, given a recurrence operator of order and starting values , compute, if possible, , and as in (3) such that is a solution of with for . We can write as a sum of hypergeometric sequences:

with , , and . Note that we use instead of since the exponential sequence for each summand can be a product of several . We can assume without loss of generality that the are linearly independent over . In fact, if , we can set and get . Let be the least common left multiple of the first order operators that annihilate respectively in the Ore algebra and let be a generator of . We show that and are equal. (Note that we required all operators to have leading coefficient .)

By right division with remainder, we can write as

with and some . We then get

Since the are linearly independent, we have , and so, are right factors of . This proves the claim.

Since every annihilator of is a multiple of and therefore also an annihilator of , we can use Petkovšek’s algorithm (Petkovšek, 1992) to determine , and as in (3). More precisely, given an operator of order and starting values , we compute as in (3) such that (if possible), by computing all hypergeometric solutions of . This gives and , linearly dependent on parameters . Next, we solve the linear system in terms of . Any solution then gives rise to a sequence with the desired properties.

Example 4.2 ().

For the recurrence for in Example 3.3, we compute two hypergeometric solutions using Petkovšek’s algorithm:

Thus, we get

with the relations and stemming from the starting values of . Since are given by first order recurrences, their closed forms can be easily computed:

It follows that the program loop given in Example 3.3 is extended P-solvable.

4.3. The Ideal of Algebraic Relations

We now turn to the problem of, given sequences as in (3), how to compute a basis for the ideal of all algebraic relations among the . We proceed by identifying the terms that are algebraically independent over . For this, we use basic properties of sums and products of hypergeometric terms. First, we state a necessary condition for a finite sum of hypergeometric terms to be again hypergeometric.

Lemma 4.3 ().

Let be hypergeometric sequences. If the sum is hypergeometric, then there exist integers , , and a rational function such that .


We prove the claim by induction on . For the case , there is nothing to show. Now suppose the claim holds for some . There is a rational function such that

Let be such that . We then get


We first treat the case in which for all , is not zero. Then, bringing in (4) to the other side yields

The sequence is hypergeometric, and by the induction hypothesis it follows that there are and a rational function with . Dividing by proves the claim. For the case that there is an with , the left hand side of (4) is a sum of fewer than  hypergeometric terms and the right hand side is hypergeometric. The induction hypothesis then again yields suitable and . ∎

Example 4.4 ().

The sums and are hypergeometric, whereas is not.

The next lemma gives a characterization of when the quotient of two hypergeometric sequences is a rational function sequence. Together with Lemma 4.3, this then will yield the algebraic independence of certain hypergeometric sequences in Lemma 4.6.

Lemma 4.5 ().

Let be such that for all with , we have . Then for , , and , there is a rational function such that

if and only if and .


If and , then we can set . For the other direction, we have

A hypergeometric term is a rational function if and only if its shift quotient can be written in the form

with . Therefore, for any root in the numerator of there is a root in integer distance in the denominator of , which, by the condition on the , is not possible if or

Lemma 4.6 ().

Let and . The sequences are algebraically independent over if and only if there are no , such that .


If there are , with , then we get the algebraic relation

Conversely, let be a nonzero polynomial over in  variables. We can write as a sum of the form

Assume that Then, by Lemma 4.3, there have to be terms , and a rational function with

By Lemma 4.5, this can only be the case if there are in integer distance, which contradicts the condition on the . ∎

Example 4.7 ().

Let be hypergeometric sequences given by and

The closed forms then are

From Lemma 4.6 it follows that are algebraically independent over , but are not.

Lemma 4.6 allows us to represent the sequences arising in extended P-solvable loops as rational function sequences over the field as follows: Let be of the form (3) and let be a subset of such that there are no , , with and for each there exists an such that . Let be such that

for all and . Then there exist with

Substituting variables for , for , for and for then gives

where is a rational function over in variables. We now can compute the ideal of all algebraic dependencies among the program variables of a P-solvable loop as the ideal of algebraic relations among rational functions.

Proposition 4.8 ().

Let be sequences of the form (3) and consider the corresponding rational functions in as above. For each , write with coprime polynomials over . Denote by the ideal of algebraic relations among in . Then the ideal of algebraic relations among the sequences in is given by


The proposition follows immediately from the fact that the ideal of algebraic dependencies among a set of rational functions

in the polynomial ring is given by

and that by Lemma 4.6 there are no algebraic relations over the field among the terms with as above for . ∎

Example 4.9 ().

We compute the ideal of algebraic relations among given in Example 3.3. First, we compute the ideal of algebraic relations among and with corresponding variables . We get

Now we can compute the ideal of algebraic relations among by adding the relations