Solving QBF by AbstractionThis work was partially supported by the German Research Foundation (DFG) as part of the Transregional Collaborative Research Center “Automatic Verification and Analysis of Complex Systems” (SFB/TR 14 AVACS).

Solving QBF by Abstractionthanks: This work was partially supported by the German Research Foundation (DFG) as part of the Transregional Collaborative Research Center “Automatic Verification and Analysis of Complex Systems” (SFB/TR 14 AVACS).

Leander Tentrup Reactive Systems Group
Saarland University
Abstract

Many synthesis and verification approaches rely on solving techniques for quantified Boolean formulas (QBF). Consequently, solution witnesses, i.e., Boolean functions, become more important as they represent implementations or counterexamples. We present a new recursive counterexample guided abstraction refinement algorithm for solving and certifying QBFs that exploits structural reasoning on the formula level. The algorithm decomposes the given QBF into one propositional formula for every scope—a maximal block of consecutive quantifiers of the same type—that abstracts from assignments of outer and inner quantifiers. Further, we define a format for proof traces and present an efficient method to extract certificates from traces that allows building witnessing functions and certifying the solver result. We report on experimental evaluation of this algorithm in a prototype solver QuAbS (Quantified Abstraction Solver) using synthesis benchmark sets and show the effectiveness of the certification approach.

1 Introduction

Efficient solving techniques for Boolean theories are an integral part of modern verification and synthesis methods. The ever growing complexity of verification and synthesis problems led to propositional problems of enormous size. To see further advances in these areas, we believe that it is necessary to move to more compact representations of these requirements. Quantified Boolean formulas (QBFs) have repeatedly been considered as a candidate theory to compactly encode Boolean problems [3, 42, 29, 19, 13, 32, 38, 10, 9]. Recent advances in QBF solvers give rise to hope that QBF may help to increase the scalability of verification and synthesis approaches.

Most current QBF solving algorithms are limited to formulas in prenex conjunctive normal form (PCNF). Starting with formulas in CNF enables efficient optimizations and preprocessing methods, like blocked clause elimination [8], that are more difficult to implement on more general structures. However, there is experimental evidence that CNF solvers can be ineffective even when the satisfiability of the formula is trivial [1, 41]. Further, problems are rarely naturally expressed in CNF and the CNF transformation increases the formula’s size linearly, disrupts its structure, and artificially introduces new variables. We argue that negation normal form (NNF) is a good compromise between structure and complexity. Arbitrary quantified Boolean formulas can be transformed to NNF with maximal increase of formula size of factor two while preserving most of the structure and without artificially adding variables. At the same time, NNF inherits important properties from CNF, especially that negations appear only in front of variables, enabling pure literal analysis and other optimizations that we describe further in Sec. 3.2.

Certifying the results is particularly important for QBF, as the pure yes/no answer is of little use. Like the propositional SAT problem [7, 12], QBF can be used to encode objects of interest, like error paths [13, 29] and implementations [9, 19, 20]. While the yes/no answer of a SAT or QBF solver then provides the information about the existence of this object, we often want to construct a concrete instance for further use. For the propositional SAT problem the object can typically be extracted as the assignment of the variables, but for QBF the object potentially consists of Skolem functions for the existential quantifiers or Herbrand functions for the universal quantifiers. Certification has been explored in the setting of CNF solving [2, 34, 24, 37] but in practice it involves performance penalties due to non-applicable solving optimizations and limited preprocessing. We believe that certification must be treated as a first-class citizen of QBF solving and show that it is possible to have competitive performance and solution extraction at the same time.

We present a new counterexample guided abstraction refinement (CEGAR) algorithm for solving quantified Boolean formulas that exploits the propositional formula’s structure. The algorithm decomposes the given QBF into one propositional formula for every scope—a maximal block of consecutive quantifiers of the same type—that abstracts from assignments of outer and inner scopes. This propositional abstraction contains special interface literals that are used to communicate abstraction entries between the scopes. In every scope, we use a SAT solver as an oracle to generate new abstraction entries and to provide us with witnesses for unsatisfiable queries. On the algorithmic side, we introduce a new element to QBF refinement algorithms by maintaining, for every scope, a dual abstraction and make twofold use of it: It provides a method for optimizing abstraction entries and it is used to translate counterexamples from inner scopes to outer scopes.

To sum up, this paper makes the following contributions:

  • We provide a general counterexample guided abstraction refinement (CEGAR) algorithm for solving QBFs in negation normal form.

  • We provide requirements that an abstraction must fulfill to be compatible with this algorithm and provide an instance of an abstraction that is experimentally evaluated.

  • We show an exponential gap between the abstraction algorithm and algorithms based on assignment and expansion.

  • We describe an efficient certification approach, i.e., proof traces and function extraction. It is evaluated on synthesis benchmarks where implementations can be directly obtained from certificates.

2 Quantified Boolean Formulas

A quantified Boolean formula (QBF) is a propositional formula over a finite set of variables with domain extended with quantification. The syntax is given by the grammar

where . For readability, we lift the quantification over variables to the quantification over sets of variables and denote a maximal consecutive block of quantifiers of the same type by and by , accordingly, where .

Given a subset of variables , an assignment of is a function that maps each variable to either true () or false (). When the domain of is not clear from context, we write . A partial assignment may additional set variables to an undefined value . We say that is compatible with , for short, if they have the same domains () and for all where . For two assignments and with disjoint domains and we define the combination as We define the complement to be for all . The set of assignments of is denoted by .

A quantifier for binds the variable in the scope . Variables that are not bound by a quantifier are called free. The set of free variables of formula is defined as . The semantics of the satisfaction relation is given as

QBF satisfiability is the problem to determine, for a given QBF , the existence of an assignment for the free variables of , such that the relation holds.

The dependency set of an existentially (universally) quantified variable , denoted by , is the set of universally (existentially) quantified variables  such that is in the scope of . A universal variable additionally depends on the free variables. A Boolean function maps assignments of to true or false. An assignment over variables can be identified by the conjunctive formula . Similarly, Boolean functions can be identified by propositional formulas over the variables in their domain. Let be the propositional formula where all quantifiers are removed and occurrences of variables are replaced by their corresponding functions. A witness for a satisfiable QBF is a Skolem function for every existential variable , such that for every assignment of the universal variables is satisfiable. For unsatisfiable QBFs, the witnesses are defined dually and called Herbrand functions.

A closed QBF is a formula without free variables. Closed QBFs are either true or false. A formula is in prenex form, if the formula consists of a quantifier prefix followed by a propositional formula. Every QBF can be transformed into a closed QBF and into prenex form while maintaining satisfiability.

A literal is either a variable , or its negation . Given a set of literals , the disjunctive combination is called a clause and the conjunctive combination is called a cube. Given a literal , the polarity of , for short, is if is positive and otherwise. The variable corresponding to is defined as where if and otherwise.

A QBF is in negation normal form (NNF) if negation is only applied to variables. Every QBF can be transformed into NNF by at most doubling the size of the formula and without introducing new variables. For formulas in NNF, we treat literals as atoms.

3 Solving Quantified Boolean Formulas

For this section, we assume an arbitrary quantified formula with quantifier prefix and propositional body in NNF.

We use a generic solving function for propositional formula under assumptions , that returns whether is satisfiable and either a satisfying assignment for variables or a partial assignment such that is unsatisfiable. Further, we define to be if and otherwise ( analogously).

3.1 Assignment Based Solving

Algorithm 1 shows a recursive solving algorithm that communicates variable assignments through the quantifier levels of the QBF.

Every quantifier has a propositional formulas that is initialized with the body of the quantified formula ( for universal quantifiers). In every iteration of the loop, an assignment is generated in line 3. In the innermost quantifier, we optimize the assignment by generating a partial assignment of that is sufficient to falsify  [33]. Consequently, this partial assignment is sufficient to satisfy . If is quantified, we recursively solve under the assignment . In the positive case (), we return the obtained witness while in the negative case, we refine a clause that excludes the counterexample assignment before reentering the loop. If there is no satisfying assignment in line 3, the algorithm returns a partial assignment that witnesses the unsatisfiability of the query.

A simple inductive argument shows that, given a QBF, the algorithm terminates with the correct result.

1:procedure assignment-qbf()
2:     while  do
3:         
4:         if  then
5:              return          
6:         
7:         if  is propositional then
8:               ,
9:              return          
10:         
11:         if  then
12:              return          
13:               
Algorithm 1 Assignment Based Algorithm

3.2 Abstraction Based Solving

Refinements based on variable assignments are in many cases not optimal. This insight led to algorithms that generalizes from assignment based refinement to clause based refinement [28, 37] for QBFs given in conjunctive normal form. The idea in these works is that multiple assignments may lead to the satisfaction of the same clauses, hence, instead of communicating assignments, the information whether a clause is satisfied or not is communicated between quantifier levels. While in the assignment based algorithm, a refinement (line 13) states that at least one assignment of must be different in the next iteration, the refinement in the clause based algorithm states that from a set of clauses/cubes one element must be satisfied (depending on the quantifier).

In the following, we propose another generalization to negation normal form formulas, i.e., we base the communication on the satisfaction of individual subformulas. To facilitate working with arbitrary Boolean formulas, we start with introducing additional notation. Let be the set of Boolean formulas and let () be the set of (direct) subformulas of (note that but ). For a propositional formula , returns the Boolean connector if is not a literal. For example, given , , , and .

To communicate the satisfaction of subformulas, we introduce two special types of literals which we call interface literals. The valuation of those literals are communicated between the quantifier levels. For a given quantifier and a subformula , the literal represents the assignments made by the outer quantifiers while the literal represents the assignments from the current quantifier including assumptions on the satisfaction of subformulas by inner quantifiers. Thus, is true if is satisfied by the outer quantifiers and a valuation that sets to true indicates that is satisfied by the quantifier . Before going in more detail on the abstraction, we introduce the basic algorithm first.

Recursive CEGAR Algorithm.

Algorithm 2 shows the abstraction based algorithm abstraction-qbf. On a high level, the algorithm has the same structure as assignment-qbf as it does recursion on the quantifier blocks. Unlike to previous approaches that utilize SAT solvers [26, 28, 37, 40], we keep for every scope a dual abstraction that is used for optimization of abstraction entries and translating interface literals.

Given a quantifier , the sets and contain the and literals corresponding to this quantifier. When translating a literal to a literal, we use the same index, e.g., in line 9, the literals are translated to literals of the inner scope. We initialize with the abstraction described below, which is a propositional formula over variables in , and literals. The innermost quantifier does not contain literals. The dual abstraction is defined as the abstraction for .

In every iteration of the while loop, a literal assignment is generated according to the outer assignment (line 3). Before the recursive call, is translated to a assignment for the inner quantifier . If the recursive call is (w.r.t. the current quantifier), the abstraction is refined by excluding the counterexample , i.e., in the following iterations one of the such that must be set to true. Due to the negation during translation, those were set to false in . In case the recursive call is , we update the dual abstraction accordingly and use it to optimize and translate the witness from a assignment to a assignment. If the query in line 3 fails, together with an assignment witnessing the unsatisfiability is returned.

1:procedure abstraction-qbf()
2:     while  do
3:         
4:         if  then
5:              return
6:         else if  is propositional then
7:              
8:              return          
9:         Let and define s.t. for all
10:         
11:         if  then
12:              
13:              
14:              return          
15:               
Algorithm 2 Abstraction Based Algorithm

To ensure correctness, we need requirements on the abstraction being used. Given a quantifier , we say that a subformula is good if it is not yet falsified (), respectively satisfied (). For every quantifier and literal it must hold that if is set to true then is good for quantifier , which gives us proper refinement semantics (line 15). The same property holds for literals . Further, for a quantifier alternation , it must hold that the set of outer literals matches the set of inner literals to enable the translation in line 9. Combining these properties gives us that a good subformula of quantifier is a bad subformula of quantifier and vice versa. Termination then follows from progress due to refinements (line 15) and correctness can be showed by induction over the quantifier blocks.

Abstraction.

Before giving the formal definition, we give an intuition for the abstraction and its interference with the algorithm.

Example 1

Consider the example and its negation . The abstraction for the universal quantifier is and the abstraction for the existential quantifier is . The only satisfying assignment for is and , i.e., the universal quantifier can satisfy in , while assuming that will be satisfied by the inner level (as is set to false). This assignment is translated into assignment of the literals in by negation ( and ). At the existential quantifier, the abstraction is solved under the assumption , resulting in a satisfiable query. The assumption is needed to satisfy (assumptions and are enough to make the dual abstraction unsatisfiable, line 7), thus, the partial assignment with is returned (line 8). The following refinement forces that must be set to true in the next iteration, i.e., . This depletes all possible assignments of the universal quantifier, thus, proving that the instance is true.

We now give a formal definition of the abstraction. Given a propositional formula in NNF and a scope , we build the following propositional formula in conjunctive normal form representing the abstraction for this scope, where encodes that must hold and defines a CNF formula that encodes the truth of subformula with respect to the valuations of the current, inner and outer scopes represented by and literals, respectively:

An undefined result form means that the subformula is ignored in the encoding . The abstraction of a scope is defined as the existential abstraction for . The dual abstraction is defined as the abstraction for . Note that not every literal that is used in the abstraction may be exposed as an interface literal. For this abstraction we define

These sets can be further reduced as described in the next section.

Optimizations.

There are a number of optimizations that can be applied to the basic abstraction algorithm. First, the SAT solver used in line 3 does not know about the semantics of the and literals, hence, the returned assignment may not be optimal. Depending on the solver used, there is limited influence on the solving, like setting default polarities, control decision heuristics, etc. A SAT solver independent solution to this problem is a pass after the sat call that adjusts the literal assignment according to the chosen variable assignment. This is possible since in NNF (as in CNF), the optimization objective for the existential and universal quantifiers is to maximize and minimize the number of satisfied subformulas, respectively. More sophisticated optimization could be achieved by taking the formula structure into account or even modify the variable assignment  to optimize the literal assignment.

The abstraction can be further improved to reduce the number of interface literals. By computing which subformula is influenced by which scopes, one can combine a whole formula subtree into a single literal if all subformulas in the subtree are only influenced by the current or outer scopes. We depict such a situation in Fig. 1.

*

Figure 1: Visualization of the scope influence optimization: In this picture, the subtree  is only influenced by outer scope , so we collapse the whole subtree into a single  literal (indicated by grey dots) in the inner scopes. For the abstraction at scope , we correspondingly introduce a just single  literal for the output of the subtree.

Preprocessing.

Due to the richer structure of formulas compared to CNF, it is more difficult to implement preprocessing techniques and we consider only basic ones. We normalize the formula by combining consecutive subformulas with the same operator, removing tautologies and forced literals. As in NNF negation appears only in front of variables, detection of pure literals [15] is still efficient. Since usually circuits are used to represent formulas, sophisticated circuit minimization techniques would likely benefit solving although we do not make use of them.

4 Certification

Certification is an essential complement of QBF solving. Certification amounts to extracting witnessing functions from a QBF, either Skolem functions for true QBFs or functions for false QBFs. Not only does it allow to verify the solver result, but also can the resulting functions be used in the context of the application. More recently, QBF proofs were used to establish differentiations between different kind of PCNF solving methods [27, 4]. We use some of these results to separate our algorithm from other QBF solving algorithms in Sec. 5. The main result of this section is a proof format for our abstraction algorithm and an efficient algorithm to transform proof traces into Boolean functions.

Proof Format.

To extract a witness from a run of abstraction-qbf, we need to remember situations and reactions, represented by assignments to and , that were satisfiable for the respective quantifier. Hence, the proof consists of a sequence of pairs and these pairs can be obtained from the algorithm by the result of the query to the dual abstraction  in lines 7 and 13. As an immediate consequence, the number of pairs in the proof trace is linear in the number of iterations of the algorithm. Further, the proof format is independent of the abstraction that is used for the algorithm.

As the function extraction depends on the abstraction, we define a function which maps a literal and a scope to a Boolean formula over the variables of the outer scopes (with respect to ). Intuitively, describes these assignments that enable in the abstraction of scope .

Example 2

Consider again our earlier example formula . It holds that and .

Function Extraction.

Prior to the function extraction, we filter out those pairs from the proof that correspond to the variables that are dependencies and do not describe a function, i.e., universal variables for true QBFs and existential variables for false QBFs. The remaining proof consists of pairs where is a set of variables that were assigned positively and is a formula that represents the situation were the response is correct. Let be the pairs corresponding to scope and let be some variable, the function is defined as

(1)

This construction is similar to previous extraction algorithms, including [37, 4]. The definition of allows that may depend on variables in outer scopes corresponding to functions instead of dependencies. By replacing those variables with their extracted functions, one can make sure that depends only on . The size of , measured in terms of distinct subformulas, is linear in the number of pairs and, hence, linear in the size of the proof.

Example 3

The proof trace for example is and the Skolem function is .

Theorem 4.1

Given a QBF and proof trace . The run time of the function extraction algorithm is in . The size of the resulting functions is linear in the size of .

Certification.

A certificate is a representation of all functions that witnesses the result of the QBF. A certificate is correct, if two conditions are satisfied: it is (1) functionally correct and (2) well-formed. Functionally correctness can be checked by a propositional SAT query to (respectively for false QBFs) where every occurrence of a function variable is replaced by the function . The unsatisfiability of this query witnesses functionally correctness. The well-formedness criterion concerns the representation of the certificate, usually as a circuit, and requires that the representation of a function depends only on its dependencies. One can further differentiate syntactical and semantical well-formedness. A certificate is syntactical ill-formed if a non-dependency is reachable from the output of a function. A certificate is semantical ill-formed if a valuation change of a set of non-dependencies changes the valuation of a function. Our function extraction guarantees syntactical well-formedness and therefore any further circuit simplification guarantees at least semantical well-formedness.

5 Comparison to other Solving Algorithms

In this section we compare abstraction-qbf to well-established solving methods. For comparing the runtime of abstraction-qbf and assignment-qbf, we use the number of iterations of the while loops as a measurement.

First, we show that there is an exponential gap between abstraction-qbf and assignment-qbf. We use the family of QBFs that was used to show separation between -resolution and expansion based solving [5]. is defined as with for every and . is false for every bound and the unique Herbrand function is defined as .

Lemma 1

Every run of assignment-qbf on needs exponentially many iterations in .

Proof

We show that every refinement at the existential quantifier excludes exactly one function entry of . As there are function entries, assignment-qbf needs an exponential number of iterations.

Let be an arbitrary assignment. Assume that is true under this assignment (the other case is analogous). At the universal quantifier, the query is satisfied under assumption by choosing . As flipping the value of already one would make false, one need the complete assignment to falsify (line 8 of Algorithm 1). Hence, the assignment is returned by the universal level and the refinement eliminates exactly one function entry. ∎

Lemma 2

Every run of abstraction-qbf on needs constant number of iterations.

Proof

The abstraction has only two relevant literals, namely and and the existential quantifier has to set or to false (it may be the case that both are set to false). The universal quantifier needs just one literal set to true to satisfy , hence, either or is refined (line 15 of Algorithm 2). The abstraction becomes unsatisfiable after the next iteration, which is analogous to the first one. ∎

We use a simulation argument on the proof traces to show that abstraction-qbf is at least as good as assignment-qbf.

Lemma 3

Given an arbitrary QBF . Every proof trace of assignment-qbf on can be polynomially simulated by a proof trace of abstraction-qbf.

Proof (Sketch)

A proof trace for assignment-qbf can be defined by a sequence of pairs where for a given scope , is a set (cube) of variables from outer scopes that indicates is a satisfying assignment under this condition. Given such a trace , we translate it into a proof trace of abstraction-qbf by computing for each pair which interface literals are satisfied by , resulting in the pair . By construction, the resulting proof trace is correct and .

Theorem 5.1
  1. assignment-qbf does not simulate abstraction-qbf.

  2. abstraction-qbf simulates assignment-qbf.

We compare abstraction-qbf to expansion based algorithms in the following. To do so, we use a family of formulas

(2)

that was used to show that ExpRes does not simulate -resolution [27]. The refutation of (2) needs to expand each variable in both polarities at all depths [27], hence, there is no polynomial refutation in expansion based solvers like RAReQS [26].

Lemma 4

There exists a run of abstraction-qbf on formula (2) that has a linear number of iterations with respect to .

Proof (Sketch)

We consider the NNF translation (3) of (2). W.l.o.g. we can consider the quantifier prefix as blocks of quantifiers as the subformula contains only variables from block for every , i.e., the only subformula shared between quantifiers is .

(3)

For every quantifier , either cube or must be falsified. Hence, by using , the inner quantifier can falsify both cubes and , forcing and to be true. Further, and unlike to the expansion based setting, the situations where and are the same with respect to our abstraction (both cubes are falsified), hence, these situations fall together.

Using this, we can construct a run that proceeds in this way until the innermost quantifier . The corresponding query is as clause is not satisfied yet, i.e., requires , and as well as are forced to be true. The failed assumptions of this query correspond to the literals of , , and . Due to the optimization in the dual abstraction of quantifier , this reduces to either , or , if or , respectively. The refinement on forces to satisfy the previous falsified cube, i.e., the value of is swapped and the algorithm proceeds as above, but this time returns the other refinement , or , . This establishes that must be satisfied at this level, thus, eliminating one block .

6 Experimental Results

We implemented the algorithms described in this paper and their optimizations in a solver called QuAbS111Available at https://www.react.uni-saarland.de/tools/quabs/ (Quantified Abstraction Solver) that takes QBFs in the standard format QCIR[36]. As the underlying SAT solver, we use PicoSAT [6] and MiniSat [14]. We compare QuAbS against the two publicly available QBF solvers that support the QCIR format, namely RAReQS [26] and GhostQ [30]. For our experiments, we used a machine with a quad-core Intel Xeon processor and of memory. The timeout and memout were set to minutes and , respectively.

Benchmarks.

We evaluate the solver on three novel benchmark sets that are designed to take advantage of the structural problem definition. The syntcomp benchmark set was created from the safety benchmarks of SYNTCOMP 2014 [25]. In this terminology, and are the state and next state bits, is the environment input, is the system output, is the transition relation, and denotes the winning region. Starting from all safe states , we iteratively solve the query and adapting for false queries [10], i.e., we exclude states represented by form where the environment can force the system into a bad state. If the query becomes satisfiable, the benchmark is realizable, if excludes the initial state , the benchmark is unrealizable. The bounded-synthesis benchmark set was generated from the synthesis benchmarks bundled with Acacia+ [11] using a QBF encoding of the bounded synthesis approach [18]. The petri-games benchmark set was generated using a variant of the bounded synthesis approach for Petri games [16] on the benchmark set introduced by Finkbeiner et al. [17]. All those benchmarks have in common that it is possible to directly build implementations from satisfiable queries. The qbfgallery2014 and eval2012 are originally CNF benchmark sets that were translated to QCIR using a conversion tool provided by the QBFGallery organizers.

Family total QuAbS assignment RAReQS GhostQ
PicoSAT MiniSat PicoSAT MiniSat plain CEGAR
syntcomp 160 154 156 135 138 109 156 147
bounded-synt. 330 281 276 261 237 322 278 319
petri-games 244 213 206 161 137 174 125 161
structural 734 648 638 557 512 605 559 627
qbfgallery2014 699 290 241 202 122 241 316 333
eval2012 337 177 166 98 77 101 161 187
translated 1036 467 407 300 199 342 477 520
total 1770 1115 1045 857 711 947 1036 1147
Table 1: Number of instances of the benchmark sets solved within 10 minutes.

Evaluation.

Table 1 shows the result of the evaluation broken down by benchmark families. Overall, QuAbS is very competitive. It clearly outperforms the assignment based approach, confirming the theoretical results established in the previous section. Compared to the established solver, the results are mixed. There are benchmarks where QuAbS outperforms both RAReQS and GhostQ like the Petri games family as well as benchmarks where RAReQS (bounded synthesis) and GhostQ (translated benchmarks) have an advantage. In the internal comparison, the variant using PicoSAT has a small lead over the variant using MiniSat, however, both variants solve some benchmarks that the other does not. Combining both leads to and solved structural and translated benchmarks, respectively.

Certification.

We implemented the certification approach described in Sec. 4, but instead of generating proof traces, we build the certificates (repesented as AIGER files) within the solving loop. This allows us to build candidate Skolem and Herbrand functions in parallel during solving and minimizes the overhead due to certification. In the verification step, we build a SAT query that encodes the functional correctness property and solve this query with MiniSat. The size of a certificate is given in the number of AND gates in the AIGER file.

Family total variant #solved #verified avg. size   solving verification
syntcomp 160 PicoSAT 154 142 39,900 19.7% 80.3%
MiniSat 156 138 15,264 8.9% 91.1%
bounded-synthesis 330 PicoSAT 281 278 22,209 83.1% 16.9%
MiniSat 276 270 54,865 45.3% 54.7%
petri-games 244 PicoSAT 213 204 129,542 42.7% 57.3%
MiniSat 206 183 172,738 30.6% 69.4%
qbfgallery2014 699 PicoSAT 285 262   1,200,879 79.2% 20.3%
MiniSat 238 208 2,742,076 64.0% 36.0%
eval2012 337 PicoSAT 177 161 523,123 48.7% 51.3%
MiniSat 166 147 477,519 54.0% 46.0%
Table 2: Number of solved instances with certification enabled, the number of verified instances, average certificate sizes, and a breakdown of the relative times needed to solve and verify certificates by benchmark families.

The overall effect of the certification approach on the running times is negligible (less than increase for both variants) which confirms our goal that the combination of solving and certification can be implemented efficiently. Table 2 shows the results of the certification run. Compared to Table 1 only in the gbfgallery2014 benchmark set there were some instances that could not be solved within timeout when certification was enabled. In most benchmarks, the time spent verifying the result exceeded the solving times. Consequently, not all of the solved instance could also be verified within timeout. We did not expect the verification to be a bottleneck, for future results we want to try more sophisticated SAT solvers reduce the time needed for verification.

The spike in the average size in the translated benchmarks is partially due to the (incomplete) translation, i.e., not every auxiliary CNF variable was detected and influences the certificate size negatively. To further reduce the size of certificates, we plan to employ available circuit minimization techniques.

QuAbS PicoSAT certificate size

QuAbS MiniSat certificate size

Figure 2: The size of certificates computed by QuAbS depending on SAT solver on the structural benchmarks.

Figure 2 compares the size produced by both solver variants on the commonly solved instances of the structural benchmarks. The observation correlates with Table 1 and 2 in that it shows a small advantage of the PicoSAT version, especially in seemingly more difficult instances, that are instances with large certificates.

7 Related Work

There are other solving techniques that use structural information, but they are conceptually very different, including DPLL like [15, 22, 30] and expansion [31, 35, 26]. SAT solvers have been used to solve quantifier alternations [26, 28, 40, 37], also in the context of synthesis [10]. We extend work on QBF solving techniques that communicate the satisfaction of clauses through a recursive refinement algorithm [28, 37] that were limited to conjunctive normal form. Further, the maintenance of a dual abstraction for optimization is new in this context and the certification approach is different than [37]. Certification has been considered in the context of CNF solving techniques [2, 34, 24, 37] but we are not aware of another work considering certification in the more general setting. The duality of circuit based QBF solving has been used to enhance search based CNF solvers [21, 23] but our use of a dual abstraction during solving is inspired by recent SMT solving enhancements [33].

8 Conclusion

We presented a new QBF solving algorithm that exploits the structure in the propositional formula. Further, we defined a certification format suitable for this algorithm and described an efficient algorithm to extract solution witnesses from true, respectively false, QBFs. We have implemented the solving and certification techniques in a tool called QuAbS. Several evaluations on different synthesis benchmarks and translated standard benchmark sets showed competitive performance. In subsequent work, we have generalized the algorithm to exploit independent quantifiers in non-prenex formulas [39] for parallel solving.

References

  • [1] Ansótegui, C., Gomes, C.P., Selman, B.: The achilles’ heel of QBF. In: Proceedings of AAAI. pp. 275–281. AAAI Press / The MIT Press (2005)
  • [2] Balabanov, V., Jiang, J.R.: Unified QBF certification and its applications. Formal Methods in System Design 41(1), 45–65 (2012)
  • [3] Benedetti, M., Mangassarian, H.: QBF-based formal verification: Experience and perspectives. JSAT 5(1-4), 133–191 (2008)
  • [4] Beyersdorff, O., Bonacina, I., Chew, L.: Lower bounds: from circuits to QBF proof systems. Electronic Colloquium on Computational Complexity (ECCC) 22, 133 (2015)
  • [5] Beyersdorff, O., Chew, L., Janota, M.: Proof complexity of resolution-based QBF calculi. Electronic Colloquium on Computational Complexity (ECCC) 21, 120 (2014)
  • [6] Biere, A.: PicoSAT essentials. JSAT 4(2-4), 75–97 (2008)
  • [7] Biere, A., Cimatti, A., Clarke, E.M., Zhu, Y.: Symbolic model checking without BDDs. In: Proceedings of TACAS. LNCS, vol. 1579, pp. 193–207. Springer (1999)
  • [8] Biere, A., Lonsing, F., Seidl, M.: Blocked clause elimination for QBF. In: Automated Deduction - CADE-23 - 23rd International Conference on Automated Deduction, Wroclaw, Poland, July 31 - August 5, 2011. Proceedings. LNCS, vol. 6803, pp. 101–115. Springer (2011)
  • [9] Bloem, R., Egly, U., Klampfl, P., Könighofer, R., Lonsing, F.: SAT-based methods for circuit synthesis. In: Proceedings of FMCAD. pp. 31–34. IEEE (2014)
  • [10] Bloem, R., Könighofer, R., Seidl, M.: SAT-based synthesis methods for safety specs. In: Proceedings of VMCAI. LNCS, vol. 8318, pp. 1–20. Springer (2014)
  • [11] Bohy, A., Bruyère, V., Filiot, E., Jin, N., Raskin, J.: Acacia+, a tool for LTL synthesis. In: Proceedings of CAV. LNCS, vol. 7358, pp. 652–657. Springer (2012)
  • [12] Bradley, A.R.: SAT-based model checking without unrolling. In: Proceedings of VMCAI. LNCS, vol. 6538, pp. 70–87. Springer (2011)
  • [13] Dershowitz, N., Hanna, Z., Katz, J.: Bounded model checking with QBF. In: Proceedings of SAT. LNCS, vol. 3569, pp. 408–414. Springer (2005)
  • [14] Eén, N., Sörensson, N.: An extensible SAT-solver. In: Proceedings of SAT. LNCS, vol. 2919, pp. 502–518. Springer (2003)
  • [15] Egly, U., Seidl, M., Woltran, S.: A solver for QBFs in negation normal form. Constraints 14(1), 38–79 (2009)
  • [16] Finkbeiner, B.: Bounded synthesis for Petri games. In: Proceedings of Correct System Design. LNCS, vol. 9360, pp. 223–237. Springer (2015)
  • [17] Finkbeiner, B., Gieseking, M., Olderog, E.: Adam: Causality-based synthesis of distributed systems. In: Proceedings of CAV. LNCS, vol. 9206, pp. 433–439. Springer (2015)
  • [18] Finkbeiner, B., Schewe, S.: Bounded synthesis. STTT 15(5-6), 519–539 (2013)
  • [19] Finkbeiner, B., Tentrup, L.: Detecting unrealizable specifications of distributed systems. In: Proceedings of TACAS. LNCS, vol. 8413, pp. 78–92. Springer (2014)
  • [20] Finkbeiner, B., Tentrup, L.: Fast DQBF refutation. In: Proceedings of SAT. LNCS, vol. 8561, pp. 243–251. Springer (2014)
  • [21] Goultiaeva, A., Bacchus, F.: Exploiting QBF duality on a circuit representation. In: Proceedings of AAAI. AAAI Press (2010)
  • [22] Goultiaeva, A., Iverson, V., Bacchus, F.: Beyond CNF: A circuit-based QBF solver. In: Proceedings of SAT. LNCS, vol. 5584, pp. 412–426. Springer (2009)
  • [23] Goultiaeva, A., Seidl, M., Biere, A.: Bridging the gap between dual propagation and CNF-based QBF solving. In: Proceedings of DATE. pp. 811–814. EDA Consortium San Jose, CA, USA / ACM DL (2013)
  • [24] Heule, M., Seidl, M., Biere, A.: Efficient extraction of Skolem functions from QRAT proofs. In: Proceedings of FMCAD. pp. 107–114. IEEE (2014)
  • [25] Jacobs, S., Bloem, R., Brenguier, R., Ehlers, R., Hell, T., Könighofer, R., Pérez, G.A., Raskin, J., Ryzhyk, L., Sankur, O., Seidl, M., Tentrup, L., Walker, A.: The first reactive synthesis competition (SYNTCOMP 2014). CoRR abs/1506.08726 (2015)
  • [26] Janota, M., Klieber, W., Marques-Silva, J., Clarke, E.M.: Solving QBF with counterexample guided refinement. In: Proceedings of SAT. LNCS, vol. 7317, pp. 114–128. Springer (2012)
  • [27] Janota, M., Marques-Silva, J.: Expansion-based QBF solving versus Q-resolution. Theor. Comput. Sci. 577, 25–42 (2015)
  • [28] Janota, M., Marques-Silva, J.: Solving QBF by clause selection. In: Proceedings of IJCAI. pp. 325–331. AAAI Press (2015)
  • [29] Jussila, T., Biere, A.: Compressing BMC encodings with QBF. Electr. Notes Theor. Comput. Sci. 174(3), 45–56 (2007)
  • [30] Klieber, W., Sapra, S., Gao, S., Clarke, E.M.: A non-prenex, non-clausal QBF solver with game-state learning. In: Proceedings of SAT. LNCS, vol. 6175, pp. 128–142. Springer (2010)
  • [31] Lonsing, F., Biere, A.: Nenofex: Expanding NNF for QBF solving. In: Proceedings of SAT. LNCS, vol. 4996, pp. 196–210. Springer (2008)
  • [32] Miller, C., Scholl, C., Becker, B.: Proving QBF-hardness in bounded model checking for incomplete designs. In: Proceedings of MTV. pp. 23–28. IEEE Computer Society (2013)
  • [33] Niemetz, A., Preiner, M., Biere, A.: Turbo-charging lemmas on demand with don’t care reasoning. In: Proceedings of FMCAD. pp. 179–186. IEEE (2014)
  • [34] Niemetz, A., Preiner, M., Lonsing, F., Seidl, M., Biere, A.: Resolution-based certificate extraction for QBF - (tool presentation). In: Proceedings of SAT. LNCS, vol. 7317, pp. 430–435. Springer (2012)
  • [35] Pigorsch, F., Scholl, C.: Exploiting structure in an AIG based QBF solver. In: Proceedings of DATE. pp. 1596–1601. IEEE (2009)
  • [36] QBF Gallery 2014: QCIR-G14: A non-prenex non-CNF format for quantified Boolean formulas http://qbf.satisfiability.org/gallery/qcir-gallery14.pdf
  • [37] Rabe, M.N., Tentrup, L.: CAQE: A certifying QBF solver. In: Proceedings of FMCAD. pp. 136–143. IEEE (2015)
  • [38] Rabe, M.N., Wintersteiger, C.M., Kugler, H., Yordanov, B., Hamadi, Y.: Symbolic approximation of the bounded reachability probability in large markov chains. In: Proceedings of QEST. LNCS, vol. 8657, pp. 388–403. Springer (2014)
  • [39] Tentrup, L.: Non-prenex QBF solving using abstraction, to appear at SAT 2016
  • [40] Tu, K., Hsu, T., Jiang, J.R.: QELL: QBF reasoning with extended clause learning and levelized SAT solving. In: Proceedings of SAT. LNCS, vol. 9340, pp. 343–359. Springer (2015)
  • [41] Zhang, L.: Solving QBF by combining conjunctive and disjunctive normal forms. In: Proceedings of AAAI. pp. 143–150. AAAI Press (2006)
  • [42] Zhang, W.: QBF encoding of temporal properties and QBF-based verification. In: Proceedings of IJCAR. LNCS, vol. 8562, pp. 224–239. Springer (2014)
Comments 0
Request Comment
You are adding the first comment!
How to quickly get a good reply:
  • Give credit where it’s due by listing out the positive aspects of a paper before getting into which changes should be made.
  • Be specific in your critique, and provide supporting evidence with appropriate references to substantiate general statements.
  • Your comment should inspire ideas to flow and help the author improves the paper.

The better we are at sharing our knowledge with each other, the faster we move forward.
""
The feedback must be of minimum 40 characters and the title a minimum of 5 characters
   
Add comment
Cancel
Loading ...
29609
This is a comment super asjknd jkasnjk adsnkj
Upvote
Downvote
""
The feedback must be of minumum 40 characters
The feedback must be of minumum 40 characters
Submit
Cancel

You are asking your first question!
How to quickly get a good answer:
  • Keep your question short and to the point
  • Check for grammar or spelling errors.
  • Phrase it like a question
Test
Test description