Polynomial Template Generation using Sum-of-Squares Programming

Polynomial Template Generation using Sum-of-Squares Programming

Assalé Adjé and Victor Magron Onera, the French Aerospace Lab, France.
Université de Toulouse, F-31400 Toulouse, France.
assale.adje@onera.fr Circuits and Systems Group, Department of Electrical and Electronic Engineering, Imperial College London, South Kensington Campus, London SW7 2AZ, UK.

Template abstract domains allow to express more interesting properties than classical abstract domains. However, template generation is a challenging problem when one uses template abstract domains for program analysis. In this paper, we relate template generation with the program properties that we want to prove. We focus on one-loop programs with nested conditional branches. We formally define the notion of well-representative template basis with respect to such programs and a given property. The definition relies on the fact that template abstract domains produce inductive invariants. We show that these invariants can be obtained by solving certain systems of functional inequalities. Then, such systems can be strengthened using a hierarchy of sum-of-squares (SOS) problems when we consider programs written in polynomial arithmetic. Each step of the SOS hierarchy can possibly provide a solution which in turn yields an invariant together with a certificate that the desired property holds. The interest of this approach is illustrated on nontrivial program examples in polynomial arithmetic.

static analysis, abstract interpretation, template abstract domains, sum-of-squares programming, piecewise discrete-time polynomial systems
11footnotetext: The author is supported by the RTRA /STAE Project BRIEFCASE and the ANR ASTRID VORACE Project.22footnotetext: The author is supported by EPSRC (EP/I020457/1) Challenging Engineering Grant.

1 Introduction

The concept of templates was introduced in a linear setting. They answered to the computational issue of the polyhedra domain, that is, the number of faces and the number of vertices both explode when performing the code analysis. Recently, generalizations of linear templates appeared, such as quadratic Lyapunov functions as nonlinear templates. Nevertheless, no precise characterization of the templates to use have been developed for program analysis purpose. Indeed, depending on the property to show, prefixing a template basis without any rules can lead to unuseful information on the programs. For instance, suppose that we want to show that the values taken by the variables of the program are bounded. Then, it is natural to use intervals or norm functions as templates. Unfortunately, these functions are not sufficient to show the desired property. In the context of linear systems in optimal control, it is well known that Lyapunov functions provide useful templates to bound the variable values. This result can be extended to polynomial systems using polynomial Lyapunov functions. The crucial notion behind is that these polynomial functions allow to define sublevel sets which are invariant by the dynamics -in our case, the dynamics being the loop body. In static analysis, Lyapunov functions provide inductive invariants, which are precisely the results of computation while using template abstract domains.

Related works.

Template domains were introduced by Sankaranarayanan et al. [SSM05], see also [SCSM06]. The latter authors only considered a finite set of linear templates and did not provide an automatic method to generate templates. Linear template domains were generalized to nonlinear quadratic cases by Adjé et al. in [AGG11, AGG10], where the authors used in practice quadratic Lyapunov templates for affine arithmetic programs. These templates are again not automatically generated. Roux et al. [RJGF12] provide an automatic method to compute floating-point certified Lyapunov functions of perturbed affine loop body updates. They use Lyapunov functions with squares of coordinate functions as quadratic template bases in case of single loop programs written in affine arithmetic. The extension proposed in [AGMW13, AGMW14] relies on combining polynomial templates with sum-of-squares (SOS) techniques to certify nonlinear inequalities.

Proving polynomial inequalities is already NP-hard and boils down to show that the infimum of a given polynomial is positive. However, one can obtain lower bounds of the infimum by solving a hierarchy of Moment-SOS relaxations, introduced by Lasserre in [Las01]. Recent advances in SOS optimization allowed to extensively apply these relaxations to various fields, including parametric polynomial optimization, optimal control, combinatorial optimization, etc. (see e.g. [Par03, Lau09] for more details). In the context of hybrid systems, certified inductive invariants can be computed by using SOS approximations of parametric polynomial optimization problems [LWYZ14]. In [PJ04], the authors develop an SOS-based methodology to certify that the trajectories of hybrid systems avoid an unsafe region. Recently, Ahmadi et al. [AJ13] investigate necessary or sufficient conditions for SOS-convex Lyapunov functions to stabilize switched systems, either in the linear case or when the switched system is the convex hull of a finite number of nonlinear criteria.

In a static analysis context, polynomial invariants appear in [BRCZ05], where invariants are given by polynomial inequalities (of bounded degree) but the method relies on a reduction to linear inequalities (the polyhedra domain). Template polyhedra domains allow to analyze reachability for polynomial systems: in [STDG12], the authors propose a method that computes linear templates to improve the accuracy of reachable set approximations, whereas the procedure in [DT12] relies on Bernstein polynomials and linear programming, with linear templates being fixed in advance. Bernstein polynomials also appear in  [RG13] as template polynomials but there are not generated automatically. In [SG09], the authors use SMT-based techniques to automatically generate templates which are defined as formulas built with arbitrary logical structures and predicate conjunctions. Other reductions to systems of polynomial equalities (by contrast with polynomial inequalities, as we consider here) were studied in [MOS04, RCK07] and more recently in [CJJK14].

Contribution and methodology.

In this paper, we generate polynomial templates by combining the approach of SOS approximations extensively used in control theory with template abstract domains originally introduced in static analysis. We focus on analyzing programs composed of a single loop with polynomial conditional branches in the loop body and polynomial assignments. For such programs, our method consists in computing certificates which yield sufficient conditions that a given property holds. We introduce the notion of well-representative templates with respect to this property. Computing inductive invariant and polynomial templates boils down to solving a system of functional inequalities. For computational purpose, we strengthen this system as follows:

  1. We impose that the functions involved in each inequality of the system belong to a convex cone included in the set of nonnegative functions. This allows in turn to define the stronger notion of well-representative templates.

  2. Instantiating to the cone of SOS polynomials leads to consider a hierarchy of SOS programs, parametrized by the degrees of the polynomial templates. While solving the hierarchy, we extract polynomial template bases and feasible invariant bounds together with (SOS-based) certificates that the desired property holds.

The potential of the method is demonstrated on several “toy” nonlinear programs, defined with medium-size polynomial conditionals/assignments, involving at most 4 variables and of degree up to 3. Numerical experiments illustrate the hardness of program analysis in this context, as simple nonlinear examples can already yield unexpected behaviors.

Organization of the paper.

The paper is organized as follows. In Section 2, we present the programs that we want to analyze and their representation as constrained piecewise discrete-time dynamical system. Next, we recall the collecting semantics that we use and finally remind some required background about abstract semantics for generalized template domains. Section 3 contains the main contribution of the paper, namely the definition of well representative templates and how to generate such templates in practice using SOS programming. Section 4 provides practical computation examples for program analysis.

2 Static analysis context and abstract template domains

In this section, we describe the programs which are considered in this paper. Next, we explain how to analyze them through their representation as discrete-time dynamical systems. Then, we give details about the special properties which can be inferred on such programs. Finally, we recall mandatory results for abstract template domains that are used in the sequel of the paper.

2.1 Program syntax and constrained piecewise discrete-time dynamical system representations

In this paper, we are interested in analyzing computer science programs. We focus on programs composed of a single loop with a possibly complicated switch-case type loop body. This loop is supposed to be written as a nested sequence of if statements. Moreover we suppose that the analyzed programs are written in Static Single Assignment (SSA) form, that is each variable is initialized at most once. We denote by the vector of the program variables. Finally, we consider assignments of variables using only parallel assignments . Tests are either weak inequalities or strict inequalities . We assume that assignments are functions from to and test functions are functions from to . In the program syntax, the notation will be either or . The form of the analyzed program is described in Figure 1.

x  ;
while ((x)0 and  and (x)0){
        x = (x);
           x = (x);
Figure 1: One-loop programs with nested conditional branches

As depicted in Figure 1, an update of the -th condition branch is executed if and only if the conjunction of tests holds. The variable is updated by if the current value of belongs to . Consequently, we interpret programs as constrained piecewise discrete-time dynamical systems (CPDS for short). The term piecewise means that there exists a partition of such that for all , the dynamics of the system is represented by the following relation, for :


We assume that the initial condition belongs to some compact set . For the program, is the set where the variables are supposed to be initialized in. Since the test entry for the loop condition can be nontrivial, we add the term constrained and denotes the set representing the conjunctions of tests for the loop condition. The iterates of the CPDS are constrained to live in : if for some step , then the CPDS is stopped at this iterate with the terminal value . We define a partition as a family of nonempty sets such that:


From Equation (2), for all there exists a unique such that . A set can contain both strict and weak inequalities and characterizes the set of the conjunctions of tests functions . Let stands for the vector of tests functions associated to the set . Moreover, for , we denote by (resp. ) the part of corresponding to strict (resp. weak) inequalities. Finally, we obtain the representation of the set given by Equation (3):


We insist on the notation: (resp. ) means that for all coordinates , (resp. ).

We suppose that the sets and also admits the representation given by Equation (3) and we denote by the vector of tests functions and by the vector of tests functions . We also decompose and as strict and weak inequality parts denoted respectively by , , and . To sum up, we give a formal definition of CPDS.

Definition 1 (Cpds)

A constrained piecewise discrete-time dynamical system (CPDS) is the quadruple with:

  • is the compact of the possible initial conditions;

  • is the set of the constraints which must be respected by the state variable;

  • is a partition as defined in Equation (2);

  • is the family of the functions from to , w.r.t. the partition satisfying Equation (1).

From now on, we associate a CPDS representation to each program of the form described at Figure 1. Since a program admits several CPDS representations, we choose one of them, but this arbitrary choice does not change the results provided in this paper. In the sequel, we will often refer to the running example described in Example 1.

Example 1 (Running example)

The program below involves four variables and contains an infinite loop with a conditional branch in the loop body. The update of each branch is polynomial. The parameters (resp. ) are given parameters. During the analysis, we only keep the variables and since and are just memories.

 = ;
 = ;
while (-1 <= 0){
  if (^2 + ^2 <= 1){
       = ;
       = ;
       =  * ^2 +  * ^3;
       =  * ^3 +  * ^2;
       = ;
       = ;
       =  * ^3 +  * ^2;
       =  * ^2 +  * ^2;

Its constrained piecewise discrete-time dynamical system representation corresponds to the quadruple , where the set of initial conditions is:

the set in which the variable lies is:

the partition verifying Equation (2) is:

and the functions relative to the partition are:

2.2 Program invariants

The main goal of the paper is to decide automatically if a given property holds for the analyzed program. We are interested in numerical properties and more precisely in properties on the values taken by the -uplet of the variables of the program. Hence, in our point-of-view, a property is just the membership of some set . In particular, we study properties which are valid after an arbitrary number of loop iterates. Such properties are called loop invariants of the program. Formally, we use the CPDS representation of a given program and we say that is a loop invariant of this program if:

where is defined at Equation (1) as the state variable at step of the CPDS representation of the program.

Now, let us consider a program of the form described in Figure 1 and let us denote by the CPDS representation of this program. The set of reachable values is the set of all possible values taken by the state variable along the running of . We define as follows:


To prove that a set is a loop invariant of the program is equivalent to prove that . We can rewrite by introducing auxiliary variables , :


Let us denote by the set of subsets of and introduce the map defined by:


We equip with the partial order of inclusion and by the standard component-wise partial order. The infimum is understood in this sense i.e. as the greatest lower bound with respect to this order. The smallest fixed point problem is:

It is well-known from Tarski’s theorem that the solution of this problem exists, is unique and in this case, it corresponds to where are defined in Equation (5). Tarski’s theorem also states that is the smallest solution of the following Problem:

We warn the reader that the construction of is completely determined by the data of the CPDS . But for the sake of conciseness, we do not make it explicit on the notations. Note also that the map corresponds to a standard transfer function (or collecting semantics functional) applied to the CPDS representation of a program.

Example 2 (Transfer function of the running example)

Since , the transfer function associated to the CPDS of Example 1 is given by:

To prove that a subset is a loop invariant, it suffices to show that satisfies . Nevertheless, is still not computable and we use abstract interpretation [CC77] to provide safe over-approximations of . Next, we use generalized abstract template domains as abstract domains and we construct a safe over-approximation of using a Galois connection. In this paper, we consider invariants defined from properties which are encoded with sublevel sets of given functions. A loop invariant is supposed to be the union of sublevel sets of a given function from to .

Definition 2 (Sublevel property)

Given a function from to , we define the sublevel property as follows:

Example 3 (Sublevel property examples)
  1. Let be a norm on , then is the property “the values taken by the variables are bounded”.

  2. Let , then is the property “the values taken by the variable are bounded from above”.

  3. We can ensure that the set of possible values taken by the program variables avoids an unsafe region with a fixed level sublevel property. For example, if the property to show consists in proving that the square norm of the variable is still greater than 1, we can set and restrict the sublevel sets to those for which .

A sublevel property is called sublevel invariant when this property is a loop invariant. We describe how to construct template bases, so that we can prove that a sublevel property is a sublevel invariant.

2.3 Abstract template domains

The concept of generalized templates was introduced in [AGG10, AGG11]. Let stands for the set of functions from to .

Definition 3 (Generalized templates)

A generalized template is a function from to over the vector of variables .

Templates can be viewed as implicit functional relations on variables to prove certain properties on the analyzed program. We denote by the set of templates. First, we suppose that is given by some oracle and say that forms a template basis. Here, we recall the required background about generalized templates (see [AGG10, AGG11] for more details).

2.3.1 Basic notions

We replace the classical concrete semantics by meaning of sublevel sets i.e. we have a functional representation of numerical invariants through the functions of . An invariant is determined as the intersection of sublevel sets. The problem is thus reduced to find optimal level sets on each template . Let stands for the set of functions from to .

Definition 4 (-sublevel sets)

For , we associate the -sublevel set given by:

In convex analysis, a closed convex set can be represented by its support function i.e. the supremum of linear forms on the set (e.g.  [Roc96, § 13]). Here, we use the generalization by Moreau [Mor70] (see also [Rub00, Sin97]) which consists in replacing the linear forms by the functions .

Definition 5 (-support functions)

To , we associate the abstract support function denoted by and defined by:

Let and be two ordered sets equipped respectively by the order and . Let be a map from to and be a map from to . We say that the pair defines a Galois connection between and if and only if and are monotonic and the equivalence holds for all and all .

We equip with the partial order of real-valued functions i.e. . The set is equipped with the inclusion order.

Proposition 1

The pair of maps and defines a Galois connection between and the set of subsets of .

In the terminology of abstract interpretation, is the abstraction function, and is the concretisation function. The Galois connection result provides the correctness of the semantics. We also remind the following property:


2.3.2 The lattices of -convex sets and -convex functions

Now, we are interested in closed elements (in term of Galois connection), called -convex elements.

Definition 6 (-convexity)

Let , we say that is a -convex function if . A set is a -convex set if . We respectively denote by and the set of -convex functions of and the set of -convex sets of .

The family of functions is ordered by the partial order of real-valued functions. The family of sets is ordered by the inclusion order. Galois connection allows to construct lattice operations on -convex elements.

Definition 7 (The meet and join)

Let and be in . We denote by and the functions defined respectively by, and . We equip with the join operator and the meet operator . Similarly, we equip with the join operator and the meet operator .

The next theorem follows readily from the fact that the pair of and defines a Galois connection (see e.g. [DP02, § 7.27]).

Theorem 2.1

The complete lattices and are isomorphic.

2.3.3 Abstract semantics

Since the pair of maps and is a Galois connection (Proposition 1), we can construct abstract semantics functional from this pair and the map defined at Equation (6). We obtain a map from to itself defined for and by:

Since is conditioned by the data of the CPDS , it is also the case for . As a corollary of Theorem 2.1, the best abstraction of in the lattice is the smallest fixed point of Equation (8).


The infimum is understood in the sense of the order of the component-wise order of the complete lattice . Using Tarski’s theorem, the solution of Equation (8) exists and is unique and is usually called the abstract semantics. This latter solution is optimal but any feasible solution could provide an answer to decide whether a sublevel property is an invariant of the program.

Definition 8 (Feasible invariant bound)

The function is a feasible invariant bound w.r.t. to the CPDS iff it exists such that:


In the sequel, we denote by the set of feasible invariant bounds.

From the definition of feasible invariant bound, we state the following proposition.

Proposition 2

Let us consider a CPDS . The following statements are true:

  1. Let be a solution of Problem (8), then is the smallest feasible invariant bound w.r.t. ;

  2. For all , .

For a given program represented by the CPDS , we recall that an invariant is to said be an inductive invariant of this program if for all , the implication holds for the state variable . Next, for a given function , we give a simple condition in term of inductive invariants (up to test functions) for to be a feasible invariant bound.

Proposition 3 (Loop head invariants in template domains)

Let us consider the CPDS and . Suppose that:


Then .


From the definition of the operator and Proposition 1, Conjunction (9) holds with for all . ∎

We recalled that abstract template domains produce invariants, i.e. -sublevel sets of feasible invariant bounds. It is not surprising since abstract template domains are abstract domains. The main issue is that is supposed to be given. The question is which templates basis can produce a nontrivial (strictly smaller that ) feasible invariant bound? This question can be refined when we want to show that some sublevel property is an invariant: which templates basis can ensure that the sublevel property is an invariant of the program? We propose an answer by considering Equation (10) as a system of equations, where unknowns are the template basis and . Given a sublevel , we also impose that and satisfy . This latter constraint leads to the computation of a level for which is an invariant of the program.

3 Proving program properties using sum-of-squares

Here, we describe how to certify that a sublevel property is a loop invariant using sum-of-squares (SOS) approximations. In Section 3.1, we provide a formal definition of the set of template bases that we shall use to the latter certification. Then we describe how to construct template bases so that we can prove sublevel properties (Section 3.2). In the end, we explain how to compute such bases in practice, by solving a hierarchy of SOS programs (Section 3.3).

3.1 The general setting

Definition 9 (Well-representative template basis w.r.t. a CPDS and a sublevel property)

Let be a sublevel property and be a CPDS. The template basis is well-representative w.r.t. and iff there exists such that .

In the sequel, we fix a CPDS and a sublevel property .

Well-representative template bases explicit the sets of implicit functional relations on the program variables, needed to prove that a sublevel property is an invariant. Next, we define a cone structure to strengthen the notion of well-representative bases.

Definition 10 (Convex cones containing the scalars in )

A non-empty subset of is a convex cone containing the scalars iff:

  1. for all , for all , ;

  2. for all , ;

  3. for all , ;

In the sequel, we write instead of , for each . For a convex cone containing the scalars , stands for the set of vectors of elements of and stands for the set of tableaux of elements of . For , we denote the “row m” of by and the “column j” of by . Thus refers to the element of the tableau .

We derive a stronger notion of well-representative template bases, namely well-representative template bases This notion is more restrictive, as a well-representative template basis deals with a system of inequalities instead of conjunctions of implications.

Definition 11 ( well-representative template basis)

A finite template basis is a well-representative template basis w.r.t. and iff there exist , , and for all , there exist , , such that:

  1. Initial condition satisfiability: ,

  2. “Local” branch satisfiability: , :

  3. Property satisfiability:

For the sake of presentation, let us define for all , for all :

Example 4 ( well-representative template basis)

Consider Example 1. We are interested in proving the boundedness of the values taken by the variables of the program. For , let consider . Recall that , and . Let and be a singleton template basis. Then is well-representative w.r.t. the CPDS and iff there exists , , , and such that:

Note that generating inductive invariants is well known to yield undesirable nonlinear optimization problems (e.g. bilinearity, as in [CSS03]). Here nonlinearity is avoided by fixing the parameters and to 1, so that the two last inequalities of Definition 11 become linear in the variables , , and the parameters .

The next lemma states that well-representative templates bases are well-representative template bases. This result is an application of S-Lemma with “nonnegative functions multipliers”.

Lemma 1 (Functional S-Lemma)

Let and . If there exists such that




Assuming that the inequality (13) holds for some , we obtain . The positivity of yields the desired result. ∎

Theorem 3.1 ( well-representative is well-representative )

Assume that a finite template basis is well-representative w.r.t. and . Then is well-representative w.r.t. and .


is well-representative. Then there exists , and and for all , , , such that, for all , for all , , ( and defined at Equation (11)) and . We set, for all , . From Proposition 1, for all is equivalent to and for all and for all imply respectively, by Lemma 1 for all , . Taking , we have from Equation (7), and . By Proposition 3, . Finally implies that by Lemma 1. ∎

This proof exhibits a feasible invariant bound which is given by the variable of the system of inequalities in Definition 11.

3.2 Simple construction of well-representative template bases

In this subsection, we discuss how to simply construct well-representative template bases.

Proposition 4 (With one well-representative template)

Let be a well-representative template basis w.r.t. and and be a finite subset of for all , , for all , . Then is a well-representative template basis w.r.t. and .


Suppose that is well-representative w.r.t. . By definition, there exists , and and for all , , , , such that the functions for all , , belong to ( and defined at Equation (11)) and . Let us take such that . It follows that and thus: . Now let , since then there exists such that