SAT-based Explicit \mathsf{LTL}_{f} Satisfiability CheckingGeguang Pu and Kristin Y. Rozier are corresponding authors.

SAT-based Explicit Satisfiability Checkingthanks: Geguang Pu and Kristin Y. Rozier are corresponding authors.

Jianwen Li, Kristin Y. Rozier
Iowa State University
Ames, IA, USA
{jianwen,kyrozier}@iastate.edu &Geguang Pu, Yueling Zhang
East China Normal University
Shanghai, China
{ggpu,ylzhang}@sei.ecnu.edu.cn &Moshe Y. Vardi
Rice University
Houston, TX, USA
vardi@cs.rice.edu
Abstract

We present here a SAT-based framework for (Linear Temporal Logic on Finite Traces) satisfiability checking. We use propositional SAT-solving techniques to construct a transition system for the input formula; satisfiability checking is then reduced to a path-search problem over this transition system. Furthermore, we introduce CDLSC (Conflict-Driven Satisfiability Checking), a novel algorithm that leverages information produced by propositional SAT solvers from both satisfiability and unsatisfiability results. Experimental evaluations show that CDLSC outperforms all other existing approaches for satisfiability checking, by demonstrating an approximate four-fold speed-up compared to the second-best solver.

SAT-based Explicit Satisfiability Checkingthanks: Geguang Pu and Kristin Y. Rozier are corresponding authors.


Jianwen Li, Kristin Y. Rozier Iowa State University Ames, IA, USA {jianwen,kyrozier}@iastate.edu                        Geguang Pu, Yueling Zhang East China Normal University Shanghai, China {ggpu,ylzhang}@sei.ecnu.edu.cn                        Moshe Y. Vardi Rice University Houston, TX, USA vardi@cs.rice.edu

Copyright © 2019, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.

Introduction

Linear Temporal Logic over Finite Traces, or , is a formal language gaining popularity in the AI community for formalizing and validating system behaviors. While standard Linear Temporal Logic (LTL) is interpreted on infinite traces (?), is interpreted over finite traces (?). While LTL is typically used in formal-verification settings, where we are interested in nonterminating computations, cf. (?), is more attractive in AI scenarios focusing on finite behaviors, such as planning (?????), plan constraints (??), and user preferences (???). Due to the wide spectrum of applications of in the AI community (?), it is worthwhile to study and develop an efficient framework for solving -reasoning problems. Just as propositional satisfiability checking is one of the most fundamental propositional reasoning tasks, satisfiability checking is a fundamental task for reasoning.

Given an formula, the satisfiability problem asks whether there is a finite trace that satisfies the formula. A “classical” solution to this problem is to reduce it to the LTL satisfiability problem (?). The advantage of this approach is that the LTL satisfiability problem has been studied for at least a decade, and many mature tools are available, cf. (??). Thus, satisfiability checking can benefit from progress in LTL satisfiability checking. There is, however, an inherent drawback that an extra cost has to be paid when checking LTL formulas, as the tool searches for a “lasso” (a lasso consists of a finite path plus a cycle, representing an infinite trace), whereas models of formulas are just finite traces. Based on this motivation, (?) presented a tableau-style algorithm for satisfiability checking. They showed that the dedicated tool, Aalta-finite, which conducts an explicit-state search for a satisfying trace, outperforms extant tools for satisfiability checking.

The conclusion of a dedicated solver being superior to satisfiability checking from (?), seems to be out of date by now because of the recent dramatic improvement in propositional SAT solving, cf. (?). On one hand, SAT-based techniques have led to a significant improvement on LTL satisfiability checking, outperforming the tableau-based techniques of Aalta-finite (?). (Also, the SAT-based tool ltl2sat for satisfiability checking outperforms Aalta-finite on particular benchmarks (?).) On the other hand, SAT-based techniques are now dominant in symbolic model checking (??). Our preliminary evaluation indicates that satisfiability checking via SAT-based model checking (??) or via SAT-based LTL satisfiability checking (?) both outperform the tableau-based tool Aalta-finite. Thus, the question raised initially in (?) needs to be re-opened with respect to satisfiability checking: is it best to reduce to SAT-based model checking or develop a dedicated SAT-based tool?

Inspired by (?), we present an explicit-state SAT-based framework for satisfiability. We construct the transition system by utilizing SAT solvers to compute the states explicitly. Furthermore, by making use of both satisfiability and unsatisfiability information from SAT solvers, we propose a conflict-driven algorithm, CDLSC, for efficient satisfiability checking. We show that by specializing the transition-system approach of (?) to and its finite-trace semantics, we get a framework that is significantly simpler and yields a much more efficient algorithm CDLSC than the one in (?).

We conduct a comprehensive comparison among different approaches. Our experimental results show that the performance of CDLSC dominates all other existing -satisfiability-checking algorithms. On average, CDLSC achieves an approximate four-fold speed-up, compared to the second-best solution (IC3 (?)+K-LIVE (?)) tested in our experiments. Our results re-affirm the conclusion of (?) that the best approach to satisfiability solving is via a dedicated tool, based on explicit-state techniques.

LTL over Finite Traces

Given a set of atomic propositions, an formula has the form:

;

where is true, is the negation operator, is the and operator, is the strong Next operator and is the Until operator. We also have the duals (false) for , for , (weak Next) for and for . A literal is an atom or its negation (). Moreover, we use the notation (Globally) and (Eventually) to represent and . Notably, is the standard next operator, while is weak next; requires the existence of a successor state, while does not. Thus is always true in the last state of a finite trace, since no successor exists there. This distinction is specific to .

formulas are interpreted over finite traces (?). Given an atom set , we define be the family of sets of atoms. Let be a finite nonempty trace, with . we use to denote the length of . Moreover, for , we denote as the i-th position of , and to represent , which is the suffix of from position . We define the satisfaction relation as follows:

  • ; and , if and ;

  • , if ;

  • , if and ;

  • if and ;

  • , if there exists such that and for every it holds that ;

Definition 1 ( Satisfiability Problem).

Given an formula over the alphabet , we say is satisfiable iff there is a finite nonempty trace such that .

Notations. We use to denote the set of subformulas of . Let be a set of formulas, we denote to be the formula . The two formulas are semantically equivalent, denoted as , iff for every finite trace , iff . Obviously, we have , and .

We say an formula is in Tail Normal Form (TNF) if is in Negated Normal Form (NNF) and -free. It is trivial to know that every formula has an equivalent NNF. Assume is in NNF, is defined as , where is a new atom to identify the last state of satisfying traces (Motivated from (?)), and is an formula defined recursively as follows: (1) if is or a literal; (2) ; (3) ; (4) ; (5) ; (6) ; (7) .

Theorem 1.

is satisfiable iff is satisfiable.

In the rest of the paper, unless clearly specified, the input formula is in TNF.

Approach Overview

There is a Non-deterministic Finite Automaton (NFA) that accepts exactly the same language as an formula (?). Instead of constructing the NFA for , we generate the corresponding transition system (Definition 5), by leveraging SAT solvers. The transition system can be considered as an intermediate structure of the NFA, in which every state consists of a set of subformulas of .

The classic approach to generate the NFA from an formula, i.e. Tableau Construction (?), creates the set of all one-transition next states of the current state. However, the number of these states is extremely large. To mitigate the overload, we leverage SAT solvers to compute the next states of the current state iteratively. Although both approaches share the same worst case (computing all states in the state space), our new approach is better for on-the-fly checking, as it computes new states only if the satisfiability of the formula cannot be determined based on existing states.

We show the SAT-based approach via an example. Consider the formula . The initial state of the transition system is . To compute the next states of , we translate to its equivalent neXt Normal Form (XNF), e.g. , see Definition 4. If we replace in with a new propositions , the new formula, denoted , is a pure Boolean formula. As a result, a SAT solver can compute an assignment for the formula . Assume the assignment is , then we can induce that is true, which indicates is a one-transition next state of , i.e. has a self-loop with the label . To compute another next state of , we add the constraint to the input of the SAT solver. Repeat the above process and we can construct all states in the transition system.

Checking the satisfiability of is then reduced to finding a final state (Definition 6) in the corresponding transition system. Since is in TNF, a final state meets the constraint that (recall is a set of subformulas of ) is satisfiable. For the above example, the initial state is actually a final state, as is satisfiable. Because all states computed by the SAT solver in the transition system are reachable from the initial state, we can prove that is satisfiable iff there is a final state in the system (Theorem 4).

We present a conflict-driven algorithm, i.e. CDLSC, to accelerate the satisfiability checking. CDLSC maintains a conflict sequence , in which each element, denoted as (), is a set of states in the transition system that cannot reach a final state in steps. Starting from the initial state, CDLSC iteratively checks whether a final state can be reached, and makes use of the conflict sequence to accelerate the search. Consider the formula . In the first iteration, CDLSC checks whether the initial state is a final state, i.e. whether is satisfiable. The answer is negative, so cannot reach a final state in 0 steps and can be added into . However, we can do better by leveraging the Unsatisfiable Core (UC) returned from the SAT solver. Assume that we get the UC . That indicates every state containing , i.e. , is not a final state. As a result, we can add instead of into , making the algorithm much more efficient.

Now in the second iteration, CDLSC first tries to compute a one-transition next state of that is not included in . (Otherwise the new state cannot reach a final state in 0 step.) This can be encoded as a Boolean formula where represent and respectively. Assume the new state is generated from the assignment of the SAT solver. Then CDLSC checks whether can reach a final state in 0 step, i.e. is satisfiable. The answer is negative and we can add the UC to as well. Now to compute a next state of that is not included in , the encoded Boolean formula becomes where , represent and respectively. Assume the new state is generated from the assignment of the SAT solver. Since is satisfiable, is a final state and we conclude that the formula is satisfiable. In principle, there are a total of states in the transition system of , but CDLSC succeeds to find the answer by computing only 3 of them (including the initial state).

CDLSC also leverages the conflict sequence to accelerate checking unsatisfiable formulas. Similar to Bounded Model Checking (BMC) (?), CDLSC searches the model iteratively. However, BMC invokes only 1 SAT call for each iteration, while CDLSC invokes multiple SAT calls. CDLSC is more like an IC3-style algorithm, but achieves a much simpler implementation by using UC instead of the Minimal Inductive Core (MIC) like IC3 (?).

SAT-based Explicit-State Checking

Given an formula , we construct the transition system (??) by SAT solvers and then check the satisfiability of the formula over its corresponding transition system.

Transition System

First, we show how one can consider formulas as propositional ones. This requires considering temporal subformulas as propositional atoms.

Definition 2 (Propositional Atoms).

For an formula , we define the set of propositional atoms of , i.e. , as follows: (1) if is an atom, Next, Until or Release formula; (2) if ; (3) if or .

Consider . We have . Intuitively, the propositional atoms are obtained by treating all temporal subformulas of as atomic propositions. Thus, an formula can be viewed as a propositional formula over .

Definition 3.

For an formula , let be considered as a propositional formula over . A propositional assignment of , is in and satisfies .

Consider the formula . From Definition 3, is where , are two Boolean variables representing the truth values of and . Moreover, the set is a propositional assignment of . In the rest of the paper, we do not introduce the intermediate variables and directly say is a propositional assignment of . The following theorem shows the relationship between the propositional assignment of and the satisfaction of .

Theorem 2.

For an formula and a finite trace , implies there exists a propositional assignment of such that ; On the other hand, where is a propositional assignment of , also implies .

We now introduce the neXt Normal Form (XNF) of formulas, which is useful for the construction of the transition system.

Definition 4 (neXt Normal Form).

An formula is in neXt Normal Form (XNF) if there are no Until or Release subformulas of in .

For example, is not in XNF, while is. Every formula has a linear-time conversion to an equivalent formula in XNF, which we denoted as .

Theorem 3.

For an formula , there is a corresponding formula in XNF such that . Furthermore, the cost of the conversion is linear.

Observe that when is in XNF, there can be only Next (no Until or Release) temporal formulas in the propositional assignment of . For , the set is a propositional assignment of . Based on semantics, we can induce from that if a finite trace satisfying and , is true. This motivates us to construct the transition system for , in which is a next state of and is the transition label between these two states.

Let be an formula and be a propositional assignment of , we denote and . Now we define the transition system for an formula.

Definition 5.

Given an formula and its literal set , let . We define the transition system for , where is the set of states, is the initial state, and

  • is the transition relation, where () holds iff there is a propositional assignment of such that and .

A run of on a finite trace is a finite sequence such that is the initial state and holds for all .

We define the notation for a run , to represent the length of , i.e. number of states in . We say state is reachable from state in steps (resp. in up to steps), if there is a run on some finite trace leading from to and (resp. ). In particular, we say is a one-transition next state of if is reachable from in 1 steps. Since a state is a subset of , which essentially is a formula with the form of , we mix the usage of the state and formula in the rest of the paper. That is, a state can be a formula of , and a formula can be a set of states, i.e. iff .

Lemma 1.

Let be the transition system of . Every state is reachable from the initial state .

Definition 6 (Final State).

Let be a state of a transition system . Then is a final state of iff the Boolean formula is satisfiable.

By introducing the concept of final state, we are able to check the satisfiability of the formula over .

Theorem 4.

Let be an formula. Then is satisfiable iff there is a final state in .

An intuitive solution from Theorem 4 to check the satisfiability of is to construct states of until (1) either a final state is found by Definition 6, meaning is satisfiable; or (2) all states in are generated but no final state can be found, meaning is unsatisfiable. This approach is simple and easy to implement, however, it does not perform well according to our preliminary experiments.

Conflict-Driven Satisfiability Checking

In this section, we present a conflict-driven algorithm for satisfiability checking. The new algorithm is inspired by (?), where information of both satisfiability and unsatisfiability results of SAT solvers are used. The motivation is as follows: In Definition 6, if the Boolean formula is unsatisfiable, the SAT solver is able to provide a UC (Unsatisfiable Core) such that and is still unsatisfiable. It means that represents a set of states that are not final states. By adding a new constraint , the SAT solver can provide a model (if exists) that avoids re-generation of those states in , which accelerates the search of final states. More generally, we define the conflict sequence, which is used to maintain all information of UCs acquired during the checking process.

Definition 7 (Conflict Sequence).

Given an formula , a conflict sequence for the transition system is a finite sequence of set of states such that:

  1. The initial state is in for ;

  2. Every state in is not a final state;

  3. For every state (), all the one-transition next states of are included in .

We call each is a frame, and is the frame level.

In the definition, represents the length of and denotes the -th element of . Consider the transition system shown in Figure 1, in which is the initial state and is the final state. Based on Definition 7, the sequence is a conflict sequence. Notably, the conflict sequence for a transition system may not be unique. For the above example, the sequence is also a conflict sequence for the system. This suggests that the construction of a conflict sequence is algorithm-specific. Moreover, it is not hard to induce that every non-empty prefix of a conflict sequence is also a conflict sequence. For example, a prefix of above, i.e. , is a conflict sequence. As a result, a conflict sequence can be constructed iteratively, i.e. the elements can be generated (and updated) in order. Our new algorithm is motivated by these two observations.

start

Figure 1: An example transition system for the conflict sequence.

An inherent property of conflict sequences is described in the following lemma.

Lemma 2.

Let be an formula with a conflict sequence for the transition system , then represents a set of states that cannot reach a final state in up to steps.

Proof.

We first prove is a set of states that cannot reach a final state in step. Basically from Definition 7, is a set of states that are not final states. Inductively, assume is a set of states that cannot reach a final state in steps. From Item 3 of Definition 7, every state satisfies all its one-transition next states are in , thus every state cannot reach a final state in steps. Now since is a set of states that cannot reach a final state in steps, is a set of states that cannot reach a final state in up to steps. ∎

We are able to utilize the conflict sequence to accelerate the satisfiability checking of formulas, using the theoretical foundations provided by Theorem 5 and 6 below.

Theorem 5.

The formula is satisfiable iff there is a run of such that (1) is a final state; and (2) () is not in for every conflict sequence of with .

Proof.

() Since is a final state, is satisfiable according to Theorem 4. () Since is satisfiable, there is a finite trace such that the corresponding run of on ends with a final state (according to Theorem 4). Let be where is the final state. It holds that () is a state that can reach a final state in steps. Moreover for every of with , ( is meaningless when ) represents a set of states that cannot reach a final state in steps (From the proof of Lemma 2). As a result, it is true that is not in if . ∎

Theorem 5 suggests that to check whether a state can reach a final state in steps (), finding a one-transition next state of that is not in is necessary; as imples cannot reach a final state in steps (From the proof of Lemma 2). If all one-transition next states of are in , cannot reach a final state in steps.

Theorem 6.

The formula is unsatisfiable iff there is a conflict sequence and such that .

Proof.

() is true implies that is true. Also from Lemma 2 we know is a set of states that cannot reach a final state in up to i steps. Since is true for each , is in . Moreover, is true implies all reachable states from are included in . We have known all states in are not final states, so is unsatisfiable.

() If is unsatisfiable, every state in is not a final state. Let be the set of states of . According to Lemma 2, contains the set of states that are not final in up to steps. Now we let satisfy that contains all states that are not final in up to steps, so includes all reachable states from , as is unsatisfiable. However, because , there must be an such that , which indicates that is true. ∎

Algorithm Design. The algorithm, named CDLSC (Conflict-Driven Satisfiability Checking), constructs the transition system on-the-fly. The initial state is fixed to be where is the input formula. From Definition 6, whether a state is final is reducible to the satisfiability checking of the Boolean formula . If is a final state, there is no need to maintain the conflict sequence in CDLSC, and the algorithm can return SAT immediately; Otherwise, the conflict sequence is maintained as follows.

  • In CDLSC, every element of is a set of set of subformulas of the input formula . Formally, each () can be represented by the formula where is a set of subformulas of . We mix-use the notation for the corresponding formula as well. Every state satisfying is included in .

  • is created iteratively. In each iteration , is initialized as the empty set.

  • To compute elements in , we consider an existing state (e.g. ). If the Boolean formula is unsatisfiable, is not a final state and can be added into from Item 2 of Definition 7. Moreover, CDLSC leverages the Unsatisfiable Core (UC) technique from the SAT community to add a set of states, all of which are not final and include , to . This set of states, denoted as , is also represented by a set of formulas and satisfies . The detail to obtain is discussed below.

  • To compute elements in (), we consider the Boolean formula , where represents the formula . The above Boolean formula is used to check whether there is a one-transition next state of that is not in . If the formula is unsatisfiable, all the one-transition next states of are in , thus can be added into according to Item 3 of Definition 7. Similarly, we also utilize the UC technique to obtain a subset of , such that represents a set of states that can be added into .

As shown above, every Boolean formula sent to a SAT solver has the form of ( where is a state and is either or . Since every state consists of a set of formulas, the Boolean formula can be rewritten as . Moreover, we introduce a new Boolean variable for each , and re-encode the formula to be . is satisfiable iff is satisfiable, and is an assignment of iff is an assignment of . Sending instead of to the SAT solver that supports assumptions (e.g. Minisat (?)) enables the SAT solver to return the UC, which is a set of , when is unsatisfiable. For example, assume and is sent to the SAT solver with being the assumptions. If the SAT solver returns unsatisfiable and the UC , the set , which represents every state including , is the one to be added into the corresponding . We use the notation for the above procedure.

The pseudo-code of CDLSC is shown in Algorithm 1. Line 1-2 considers the situation when the input formula is a final state itself. Otherwise, the first frame is initialized to (Line 3), and the current frame level is set to 0 (Line 4). After that, the loop body (Line 5-11) keeps updating the elements of iteratively, until either the procedure returns true, which means to find a model of , or the procedure returns true, which is the implementation of Theorem 6. The loop continues to create a new frame in if neither of the procedures succeeds to return true. To describe conveniently, we say every run of the while loop body in Algorithm 1 is an iteration.

0:  An formula .
0:  SAT or UNSAT.
1:  if  is satisfiable then
2:     return  SAT;
3:  Set ;
4:  Set ;
5:  while true do
6:     if  returns true then
7:        return  SAT;
8:     if  returns true then
9:        return  UNSAT;
10:     ;
11:     Set ;
Algorithm 1 Implementation of CDLSC

The procedure is responsible for updating . Taking an formula and the frame level currently working on, returns true iff a model of can be found, with the length of . As shown in Algorithm 2, is implemented in a recursive way. Each time it checks whether a next state of the input , which belongs to a lower level (than the input ) frame can be found (Line 2). If the result is positive and such a new state is constructed, first checks whether is a final state when is 0 (in which case returns true). If is not a final state, a UC is extracted from the SAT solver and added to (Line 5-11). If is not 0, recursively checks whether a model of can be found with the length of (Line 12-13). If the result is negative and such a state cannot be constructed, a UC is extracted from the SAT solver and added into (Line 14-15).

0:  : The formula is working on;: The frame level is working on.
0:  true or false.
1:  Let ;
2:  while  is satisfiable do
3:     Let be the model of ;
4:     Let , i.e. be the next state of extracted from ;
5:     if  then
6:        if  is satisfiable then
7:           return  true;
8:        else
9:           Let ;
10:           Add into ;
11:           Continue;
12:     if  is true then
13:        return  true;
14:  Let ;
15:  Add into ;
16:  return  false;
Algorithm 2 Implementation of

Notably, Item 1 of Definition 7, i.e. , is guaranteed for each , as the original input formula of is always (Line 6 in Algorithm 1) and there is some (Line 15 in Algorithm 2) including that is added into , if no model can be found in the current iteration.

The procedure in Algorithm 1 implements Theorem 6 in a straightforward way: It reduces the checking of whether being true on some frame level , to the satisfiability checking of the Boolean formula . Finally, we state Theorem 7 below to provide the theoretical guarantee that CDLSC always terminates correctly.

Lemma 3.

After each iteration of CDLSC with no model found, the sequence is a conflict sequence of for the transition system .

Theorem 7.

The CDLSC algorithm terminates with a correct result.

CDLSC is shown how to accelerate the checking of satisfiable formulas in the previous section. For unsatisfiable instances, consider . CDLSC first checks that is unsatisfiable, where the SAT solver returns as the UC. So is added into . Then CDLSC checks that is still unsatisfiable, in which is still the UC. So is added into as well. Since and according to Theorem 6, CDLSC terminates with the unsatisfiable result. In this case, CDLSC only visits one state for the whole checking process. For a more general instance like , where is a large formula, checking by CDLSC enables to achieve a significantly improvement compared to the checking by traditional tableau approach.

Summarily, CDLSC is a conflict-driven on-the-fly satisfiability checking algorithm, which successfully leads to either an earlier finding of a satisfying model, or the faster termination with the unsatisfiable result.

Experimental Evaluation

Figure 2: Result for Satisfiability Checking on LTL-as- Benchmarks. The X axis represents the number of benchmarks, and the Y axis is the accumulated checking time (s).
Type Number Result IC3+K-LIVE Aalta-finite Aalta-infinite ltl2sat CDLSC
Alternate Response 100 sat 134 1 48 123 3
Alternate Precedence 100 sat 154 3 70 380 4
Chain Precedence 100 sat 127 2 45 83 2
Chain Response 100 sat 79 1 41 49 2
Precedence 100 sat 132 2 14 124 1
Responded Existence 100 sat 130 1 14 327 1
Response 100 sat 155 1 41 53 2
Practical Conjunction 1000 varies 1669 19564 4443 20477 115
Table 1: Results for Satisfiability Checking on -specific Benchmarks.

Benchmarks We first consider the LTL-as- benchmark, which is evaluated by previous works on satisfiability checking (??). This benchmark consists of 7442 instances that are originally LTL formulas but are treated as formulas, as both logics share the same syntax. Previous works (??) have shown that the benchmark is useful to test the scalability of solvers.

Secondly, we consider the 7 -specific patterns that are introduced in recent researches on , e.g. (??), and we create 100 instances for each pattern. As shown in Table 1, it is trivial to check the satisfiability of these patterns by most tested solvers, as either they have small sizes or dedicated heuristics for , which are encoded in both Aalta-finite and CDLSC, enable to solve them quickly. Inspired from the observation in (?) that an LTL specification in practice is often the conjunction of a set of small and frequently-used patterns, we randomly choose a subset of the instances of the 7 patterns to imitate a real specification in practice. We generate 1000 such instances as the practical conjunction pattern shown in the last row of Table 1. Unlike the random benchmarks in SAT community, which are often considered not interesting, we argue that the new practical conjunction pattern is a representative for real specifications in industry.

Experimental Setup We implement CDLSC in C++, and use Minisat 2.2.0 (?) as the SAT engine111https://github.com/lijwen2748/aaltaf. We compare it with two extant satisfiability solvers: Aalta-finite (?) and ltl2sat (?). We also compared with the state-of-art LTL solver Aalta-infinite (?), using the -to-LTL satisfiability-preserving reduction described in (?). As LTL satisfiability checking is reducible to model checking, as described in (?), we also compared with this reduction, using nuXmv with the IC3+K-LIVE back-end (?), as an satisfiability checker.

We ran the experiments on a RedHat 6.0 cluster with 2304 processor cores in 192 nodes (12 processor cores per node), running at 2.83 GHz with 48GB of RAM per node. Each tool executed on a dedicated node with a timeout of 60 seconds, measuring execution time with Unix time. Excluding timeouts, all solvers found correct verdicts for all formulas. All artifacts are available in the supplemental material.

Results Figure 2 shows the results for satisfiability checking on LTL-as- benchmarks. CDLSC outperforms all other approaches. On average, CDLSC performs about 4 times faster than the second-best approach IC3+K-LIVE (1705 seconds vs. 6075 seconds). CDLSC checks the formula directly, while IC3+K-LIVE must take the input of the LTL formula translated from the formula. As a result, IC3-KLIVE may take extra cost, e.g. finding a satisfying lasso for the model, to the satisfiability checking. Meanwhile, CDLSC can benefit from the heuristics dedicated for that are proposed in (?). Finally, the performance of ltl2sat is highly tied to its performance of unsatisfiability checking as most of the timeout cases for ltl2sat are unsatifiable. For Aalta-finite, its performance is restricted by the heavy cost of Tableau Construction.

Table 1 shows the results for -specific experiments. Columns 1-3 show the types of formulas under test, the number of test instances for each formula type, and the results by formula type. Columns 4-8 show the checking times by formula types in seconds. The dedicated solvers perform extremely fast on the seven scalable pattern formulas (Column 5 and 8), because their heuristics work well on these patterns. For the difficult conjunctive benchmarks, CDLSC still outperforms all other solvers.

Discussion and Concluding Remarks

Bounded Model Checking (BMC) (?) is also a popular SAT-based technique, which is however, not necessary to compare. There are two ways to apply BMC to satisfiability checking. The first one is to check the satisfiability of the LTL formula from the input formula. (?) has shown that this approach cannot perform better than IC3+K-LIVE, and the fact of CDLSC outperforming IC3+K-LIVE induces CDLSC also outperforms BMC. The second approach is to check the satisfiability of the formula directly, by unrolling iteratively. In the worst case, BMC can terminate (with UNSAT) once the iteration reaches the upper bound. This is exactly what is implemented in ltl2sat (?).

In this paper, we introduce a new SAT-based framework, based on which we present a conflict-driven algorithm CDLSC, for satisfiability checking. Our experiments demonstrate that CDLSC outperforms Aalta-infinite and IC3+K-LIVE, which are designed for LTL satisfiability checking, showing the advantage of a dedicated algorithm for . Notably, CDLSC maintains a conflict sequence, which is similar to the state-of-art model checking technique IC3 (?). CDLSC does not require the conflict sequence to be monotone, and simply use the UC from SAT solvers to update the sequence. Meanwhile, IC3 requires the sequence to be strictly monotone, and has to compute its dedicated MIC (Minimal Inductive Core) to update the sequence. We conclude that CDLSC outperforms other existing approaches for satisfiability checking.

Acknowledgement. We thank anonymous reviewers for the helpful comments. This work is supported in part by NASA ECF NNX16AR57G, NSF CAREER Award CNS-1552934, NSF grants CCF-1319459 and IIS-1527668, NSF Expeditions in Computing project “ExCAPE: Expeditions in Computer Augmented Program Engineering”, NSFC projects No. 6157297, No. 61632005 and No. 61532019, and China HGJ project No. 2017ZX01038102-002.

References

  • [Bacchus and Kabanza 1998] F. Bacchus and F. Kabanza. Planning for temporally extended goals. Ann. of Mathematics and Artificial Intelligence, 22:5–27, 1998.
  • [Bacchus and Kabanza 2000] F. Bacchus and F. Kabanza. Using temporal logic to express search control knowledge for planning. Artificial Intelligence, 116(1–2):123–191, 2000.
  • [Bienvenu et al. ] M. Bienvenu, C. Fritz, and S. McIlraith. In KR, Lake District, UK.
  • [Bienvenu et al. 2011] M. Bienvenu, C. Fritz, and S. A. McIlraith. Specifying and computing preferred plans. Artificial Intelligence, 175(7¨C8):1308 – 1345, 2011.
  • [Biere et al. 1999] A. Biere, A. Cimatti, E.M. Clarke, and Y. Zhu. Symbolic model checking without BDDs. In Proc. 5th Int. Conf. on Tools and Algorithms for the Construction and Analysis of Systems, volume 1579 of Lecture Notes in Computer Science. Springer, 1999.
  • [Bradley 2011] A. Bradley. SAT-based model checking without unrolling. In Ranjit Jhala and David Schmidt, editors, Verification, Model Checking, and Abstract Interpretation, volume 6538 of LNCS, pages 70–87. Springer, 2011.
  • [Calvanese et al. 2002] D. Calvanese, G. De Giacomo, and M.Y. Vardi. Reasoning about actions and planning in LTL action theories. In Principles of Knowledge Representation and Reasoning, pages 593–602. Morgan Kaufmann, 2002.
  • [Camacho et al. 2017] A. Camacho, J.A. Baier, C. Muise, and A.S. McIlraith. Bridging the gap between LTL synthesis and automated planning. Technical report, U. Toronto, 2017.
  • [Cavada et al. 2014] R. Cavada, A. Cimatti, M. Dorigatti, A. Griggio, A. Mariotti, A. Micheli, S. Mover, M. Roveri, and S. Tonetta. The NuXMV symbolic model checker. In CAV, pages 334–342, 2014.
  • [Claessen and Sörensson 2012] K. Claessen and N. Sörensson. A liveness checking algorithm that counts. In FMCAD, pages 52–59. IEEE, 2012.
  • [De Giacomo and Vardi 1999] G. De Giacomo and M.Y. Vardi. Automata-theoretic approach to planning for temporally extended goals. In Proc. European Conf. on Planning, Lecture Notes in AI 1809, pages 226–238. Springer, 1999.
  • [De Giacomo and Vardi 2013] G. De Giacomo and M. Vardi. Linear temporal logic and linear dynamic logic on finite traces. In IJCAI, pages 2000–2007. AAAI Press, 2013.
  • [De Giacomo et al. 2014] G. De Giacomo, R. De Masellis, and M. Montali. Reasoning on LTL on finite traces: Insensitivity to infiniteness. In AAAI, pages 1027–1033, 2014.
  • [Di Ciccio et al. 2016] C. Di Ciccio, F. Maggi, and J. Mendling. Efficient discovery of target-branched declare constraints. Inf. Syst., 56(C):258–283, March 2016.
  • [Eén and Sörensson 2003] N. Eén and N. Sörensson. An extensible SAT-solver. In SAT, pages 502–518, 2003.
  • [Een et al. 2011] N. Een, A. Mishchenko, and R. Brayton. Efficient implementation of property directed reachability. In FMCAD, pages 125–134, 2011.
  • [Fionda and Greco 2016] V. Fionda and G. Greco. The complexity of LTL on finite traces: Hard and easy fragments. In AAAI, pages 971–977. AAAI Press, 2016.
  • [Gabaldon 2004] A. Gabaldon. Precondition control and the progression algorithm. In KR, pages 634–643. AAAI Press, 2004.
  • [Gerth et al. 1995] R. Gerth, D. Peled, M.Y. Vardi, and P. Wolper. Simple on-the-fly automatic verification of linear temporal logic. In P. Dembiski and M. Sredniawa, editors, Protocol Specification, Testing, and Verification, pages 3–18. Chapman & Hall, 1995.
  • [Li et al. 2013] J. Li, L. Zhang, G. Pu, M. Vardi, and J. He. LTL satisfibility checking revisited. In TIME, pages 91–98, 2013.
  • [Li et al. 2014] J. Li, L. Zhang, G. Pu, M. Y. Vardi, and J. He. LTL satisfibility checking. In ECAI, pages 91–98, 2014.
  • [Li et al. 2015] J. Li, S. Zhu, G. Pu, and M.Y. Vardi. SAT-based explicit LTL reasoning. In HVC, pages 209–224. Springer, 2015.
  • [Malik and Zhang 2009] S. Malik and L. Zhang. Boolean satisfiability from theoretical hardness to practical success. Commun. ACM, 52(8):76–82, 2009.
  • [Patrizi et al. 2011] F. Patrizi, N. Lipoveztky, G. De Giacomo, and H. Geffner. Computing infinite plans for LTL goals using a classical planner. In IJCAI, pages 2003–2008. AAAI Press, 2011.
  • [Pnueli 1977] A. Pnueli. The temporal logic of programs. In IEEE FOCS, pages 46–57, 1977.
  • [Rozier and Vardi 2007] K.Y. Rozier and M.Y. Vardi. LTL satisfiability checking. In SPIN, volume 4595 of LNCS, pages 149–167. Springer, 2007.
  • [Rozier and Vardi 2010] K.Y. Rozier and M.Y. Vardi. LTL satisfiability checking. STTT, 12(2):123–137, 2010.
  • [Sohrabi et al. 2011] S. Sohrabi, J. A. Baier, and S. A. McIlraith. Preferred explanations: Theory and generation via planning. In AAAI, pages 261–267, August 2011.
  • [Vardi 2007] M.Y. Vardi. Automata-theoretic model checking revisited. In VMCAI, LNCS 4349, pages 137–150. Springer, 2007.
  • [Vizel et al. 2015] Y. Vizel, G. Weissenbacher, and S. Malik. Boolean satisfiability solvers and their applications in model checking. Proceedings of the IEEE, 103(11):2021–2035, 2015.

Appendix A Missing Proofs

Proof of Theorem 1

We first introduce the following lemmas that are useful for the proof.

Lemma 4.

If is satisfiable, there is a non-empty finite trace such that for , and .

Proof.

Since is satisfiable, there is a non-empty finite trace such that . Recall that has the form of , so implies and there is such that and for every . We define , and first prove that implies . Let , and we prove by induction over the type of that .

  1. If , then and of course ;

  2. If is a literal, then and implies . Therefore, ;

  3. If , then , and implies and . By hypothesis assumption, implies and implies . So is true. If , the proof is similar;

  4. If , then , and implies that and . Let . By hypothesis assumption, implies is true. Moreover, because , from its definition. As a result, is true;

  5. If , then , and implies that