Weighted Model Counting in FO{}^{2} with Cardinality Constraints and Counting Quantifiers:A Closed Form Formula

# Weighted Model Counting in FO2 with Cardinality Constraints and Counting Quantifiers: A Closed Form Formula

## Abstract

Weighted First Order Model Counting (WFOMC) computes the weighted sum of the models of a first order theory on a domain of a given finite size. WFOMC has emerged as a fundamental tool for probabilistic inference. Algorithms for WFOMC that run in polynomial time w.r.t. the domain size are called lifted inference algorithms. Such algorithms have been developed for multiple extensions of FO (the fragment of First Order Logic with two variables) for the special case of symmetric weight functions. In this paper, instead of developing a specific algorithm, we derive a closed form formula for WFOMC in FO. The three key advantages of our proposal are: (i) it deals with existential quantifiers without introducing negative weights; (ii) it easily extends to FO with cardinality constraints and counting quantifiers (aka C); finally, (iii) it supports WFOMC for a class of weight functions strictly larger than symmetric weight functions, which can model count distributions, without introducing complex or negative weights.

## Introduction

Statistical Relational Learning (SRL) attempts to reason about probabilistic distributions over properties of relational domains [7, 15]. Most SRL frameworks use formulas in a logical language to provide a compact representation of the domain structure. Probabilistic knowledge on relational domain can be specified by assigning a weight to every interpretation of the logical language. One of the advantage of this approach is that probabilistic inference can be cast as Weighted Model Counting [3].

First Order Logic (FOL) allows specifying structural knowledge with formulas that contain individual variables that range over all the individuals of the domain. Probabilistic inference on domains described in FOL requires the grounding (aka instantiation) of all the individual variables with all the occurrences of the domain elements. This grounding leads to an exponential blow up of the complexity of the model description and hence the probabilistic inference.

Lifted inference [14, 2] aims at resolving this problem by carrying out probabilistic inference without the need for grounding. In recent years, Weighted First Order Model Counting has emerged as a useful formulation for probabilistic inference in SRL. Formally, WFOMC refers to the task of calculating the weighted sum of the models of a formula over a domain of a finite size.

where is the cardinality of the domain and is a weight function that associates a real number to each interpretation . FOL formulas which admit an algorithm that computes in a polynomial time w.r.t. are referred to as domain-liftable [18].

In the past decade, multiple extensions of FO (the fragment of First Order Logic with two variables) have been proven to be domain-liftable [19, 9, 11, 13]. These results are formulated over a special class of weight functions known as symmetric weight functions [1] and utilise lifted inference rules which are able to exploit the symmetry of FOL formulas in a rule based manner.

In this paper, we derive a closed form formula for WFOMC in FO, the fragment of first order logic consisting of the formulas that contain at most two individual variables. The formula can be computed in polynomial time with respect to the size of the domain, and therefore it constitutes a lifted inference algorithm. We see three main advantages of the proposed approach:

1. The proposed formula solves the weighted model counting problem without introducing negative weights. [6] proposes a method for WFOMC of formulas containing existential quantifiers by introducing negative weights. Though it has been shown that WFOMC is well defined also with negative weights, the connections between negative weights and probability turns out to be less clean, as it leads to negative and larger than one probability values.

2. The formula is easily extended to FO with cardinality constraints and counting quantifiers, without loosing domain-liftability. A cardinality constraint on an interpretation, is a constraint on the number of elements for which a certain predicate holds; counting quantifiers admit quantification of the variables with operators like , they allow expressions of the form expressing the fact that there exist at least elements that satisfy .

3. The formula computes WFOMC for a class of weight functions larger than symmetric weight functions. This extended class of weight functions allow to models the recently introduced count distributions [12].

For the sake of presentation, most of the paper focuses on First Order Model Counting (FOMC), i.e., the weight function is constantly equal to 1. We then show how weighted model counting can be obtained by multiplying each term of the resulting formula with the corresponding weight. This allows us to simplify the treatment of the counting part and renders an easier notation for the reader. The paper is therefore structured as follows: The next section describes the related work in the literature on WFOMC. We then introduce the main result, i.e., a closed form formula for First Order Model Counting for universally quantified formulas in FO, and it’s extension with cardinality restriction. In the successive section, we show how this formula can be used to compute FOMC also in presence of existential quantifiers and counting quantifiers. The last part of the paper elaborates how these formulas extend to WFOMC. In particular we show how the formula for FOMC can be adapted to WFOMC for symmetric weight functions and a larger class of weight function that allow us to define counting distributions [12].

## Related work

Weighted first order model counting (WFOMC) was initially defined in [18]. The paper provides an algorithm for WFOMC over universally quantified theories based on the knowledge compilation technique, which transforms a FOL theory to a first order deterministic decomposable normal form (FO d-DNNF)1. A successive paper [19] has formalized the notion of domain lifted theory, i.e. a first order theory for which WFOMC can be computed in polynomial time in the size of the domain. The same paper shows that a theory composed of a set of universally quantified clauses containing at most two variables is domain liftable. This is done by combining knowledge compilation with a technique called domain recursion. In domain recursion WFOMC of theory on a domain of elements is rewritten in terms of WFOMC of the same theory in a domain with elements, by partially grounding the theory with a single element of the domain. A successive paper [6] extends this procedure to theories in full (i.e, where existential quantification is allowed) by applying skolemization to remove existentially quantified variables. The major drawback of these technique is that it introduces negative weights, and therefore it makes it more complex to use it for probabilistic inference which requires non-negative weights. [11] extends the domain-liftability results for FO with a functionality axiom, and for sentences in uniform one-dimensional fragment U. This is a recently introduced [10] extension of FO that admits more than two variables. [11] proposed a closed form formula for WFOMC in with functionality constraints. [13] recently proposes a uniform treatment of WFOMC of FO with cardinality constraints on predicate interpretation and counting quantifiers. This work builds on the previous ideas of domain recursion and does not provide closed form counting for any of the newly lifted classes. Finally, [8] re-investigates the problem of skolemisation arguing that negative weights can be prohibitive and that the skolemisation procedure is computationally expensive. The paper gives examples of theories for which skolemisation can be bypassed using domain recursion. With respect to the state of the art approaches to WFOMC, we propose an approach that provides a closed form for WFOMC with cardinality constraints from which the PTIME complexity is immediately evident. Importantly, this doesn’t require the introduction of negative weights. Furthermore, w.r.t. the closed form proposed in [11], we believe that our proposal has a more direct and intuitive interpretation. Finally, [12] introduces Complex Markov Logic Networks, which use complex-valued weights and allow for full expressivity over a class of distributions called count distributions. We show in the last section of the paper that our formalisation is complete w.r.t. these class of distributions.

## FOMC for Universal Formulas

Let be a first-order function free language with equality. A pure universal formula in is a formula of the form

 ∀x1…∀xm.Φ(x1,…,xm) (1)

where is a set of distinct variables occurring in , and is a quantifier free formula that does not contain any constant symbol. We use the compact notation for , where . Notice that we distinguish between the -tuple of variables and the set of variables denoted by . We denote an arbitrary -tuple of constants or variables by , is the result of uniform substitution of with in . If is the set of constants or variables of and a pure universal formula then denotes the formula:

 Φ(Σ)=⋀σ∈ΣmΦ(σ) (2)
###### Lemma 1.

For any arbitrary quantifier free formula , the following equivalence holds:

 ∀xΦ(x)↔∀xΦ(X) (3)
###### Proof.

For any , we have that is valid. Which implies that is also valid. Since and commute, we have that . The viceversa is obvious since is one of the conjuncts in . ∎

###### Example 1.

Let , then is the following formula

 (A(x)∧R(x,x)∧x≠x→A(x)) ∧ (A(x)∧R(x,y)∧x≠y→A(y)) ∧ (A(y)∧R(y,x)∧y≠x→A(x)) ∧ (A(y)∧R(y,y)∧y≠y→A(y)) (4)

Notice that in we can assume that two distinct variables and are grounded to different domain elements. Indeed, the cases in which and are grounded to the same domain element is taken into account by the conjunct in which is replaced by . See for instance the first and the last conjunct of (4).

A first order truth assignment of a quantifier free formula is a function that assigns to each atom of either or ( means false and true) and assigns 1 to and 0 to if and are two distinct individual variables. A first order truth assignment allows associating a truth value to a first order quantifier free formula. The truth value of under the truth assignment , denoted by , is obtained by applying the classical propositional logic of the connectives. Notice that is not a FOL interpretation as it assigns truth values to atoms that contain free variables, and not to their groundings.

###### Example 2.

An example of first order truth assignment for the formula (4) of Example 1 is the following:

 A(x)R(x,x)A(y)R(y,y)R(x,y)R(y,x)τ\pagecolorgreen!100\pagecolorgreen!101\pagecolorred!101\pagecolorred!101\pagecolorgreen!100\pagecolorred!101\omit\span\omitτx\omit\span\omitτy\omit\span\omitτxy

We omit the truth assignments of equality atoms, since it is fixed. We have that .

As highlighted in the previous example, any assignment can be split in three partial assignments , , and that assign only the atoms containing , and both and , respectively.

###### Example 3.

Consider the assignment of example 2 and the one obtained by the permutation that exchanges and

 A(x)R(x,x)A(y)R(y,y)R(x,y)R(y,x)τ\pagecolorgreen!100\pagecolorgreen!101\pagecolorred!101\pagecolorred!101\pagecolorgreen!100\pagecolorred!101τπ\pagecolorred!101\pagecolorred!101\pagecolorgreen!100\pagecolorgreen!101\pagecolorred!101\pagecolorgreen!100

It is easy to see that . This is not a coincidence, it is actually a property that derives from the shape of . This is stated in the following property.

###### Proposition 1.

For every quantifier free formula , every permutation of and every first order truth assignment for , ; where , for every atom of .

###### Proof.

If then for some . This implies that , which implies that . The proof of the opposite direction follows form the fact that . ∎

From now on, we concentrate on the special case where . For any assignment of , let and be the partial assignments that assign only the atoms containing and respectively. Notice that if is an atom of , so is and vice-versa. This implies that and assign two sets of atoms that are isomorphic under the exchange of with . Let be the number of atoms contained in each of these two sets and let be an enumeration of the predicate symbols of these atoms. In other words, we have that assigns truth value to and that assigns to .2 This implies that and can be represented by two integers and respectively between and , such that if and only if , and if and only if , where refers to the number (0 or 1) of the binary encoding of the integer . For every , we define as the number of models of which are extensions of the partial assignments and . Hence, can be written as follows

 nij=\sc mc(Φ(X)∧u−1⋀k=0(¬1−bin(i)kPk(x)∧¬1−bin(j)kPk(y))

where is the empty string and is . Notice that Proposition 1 guarantees that .

###### Example 4 (Example 1 cont’d).

The set of atoms containing only or only in the formula (4) are and respectively. In this case . The partial assignments and corresponding to the assignment of Example 2 are: and . is the number of first order assignments satisfying (4) and agreeing with and . In this case . The other cases are as follows:

 n00n01n02n03n11n12n13n22n23n334444222224

For any set of constants and any -tuple such that , let be any partition of such that . We define as follows:

 Φ(Ck)=Φ(C)∧2u−1⋀i=0⋀c∈Ciu−1⋀j=0(¬)1−bin(i)jPj(c) (5)
###### Example 5.

Examples of , on are and .

 Φ(C{{a},∅,{b,c},∅})=Φ(C) ∧¬A(a)∧¬R(a,a) ∧A(b)∧¬R(b,b) ∧A(c)∧¬R(c,c)

Note there are such partitions, and all the for such partitions will have the same model count. These observations have been formalised in lemma 2

###### Proof.

Let and , be two partitions with the same . Notice that can be obtained by applying some permutation on from . From Proposition 1 we have that

 \sc mc(Ck)=\sc mc(C′k)

Furthermore notice that if is different from then and cannot be simultaneously satisfied. This implies that

 \sc mc(Φ(C))=∑k∑Ck\sc mc(Ck)

Since there are partitions of , of the form , then

###### Lemma 3.

For any partition

 \sc mc(Φ(Ck))=∏c≠dc,d∈Cnicid

Where for all , are the indices such that and .

###### Proof.

can be rewritten in

 ⋀{c,d}⊆Cc≠dΦic,id({c,d})

is obtained by replacing each atom with if and otherwise and each atom with if and otherwise. Notice that all the atoms of contain both and . Furthermore notice that if then and do not contain common atoms. Finally we have that . Hence

 \sc mc⎛⎜ ⎜⎝⋀c,d∈Cc≠dΦic,id({c,d})⎞⎟ ⎟⎠=∏c≠dc,d∈Cnicid

###### Theorem 1.

For any pure universal formula in

 \sc fomc(∀x.Φ(x),n)=∑∑k=n(nk)∏0≤i≤j≤2u−1nk(i,j)ij (6)
 k(i,j)={ki(ki−1)2if i=jkikjotherwise (7)
###### Proof.

Notice that for a set of constants with . Therefore, by Lemma 2, to prove the theorem it is enough to show that for all , . By the Lemma 3 we have that . Then:

 ∏c≠dnicid =∏i∏c≠dc,d∈Cinii⋅∏i

###### Example 6 (Example 1 cont’d).

Consider a domain of 3 elements (i.e., n=3). Each term of the summation (6) is of the form

 (3k0,k1,k2,k3)∏0≤i≤j<2u−1nk(i,j)ij

which is the number of models with elements for which and are both false; elements for which is false and true, elements for which is true and is false and elements for which and are both true. For instance

 (32,0,0,1)n100n203=(32,0,0,1)41⋅22=3⋅16=48

is the number of models in which 2 elements are such that and is false and element where and are both true.

As a final remark for this section, notice that the computational cost of computing is constant with respect to the domain cardinality. We assume the cost of multiplication to be constant. Hence, the computational complexity of computing (6) depends on the domain only through the multinomial coefficients and the multiplications involved in . The computational cost of computing is polynomial in and the total number of are , which has as an upper-bound[5]. Also, the term has multiplication operations. Hence, we can conclude that the (6) is computable in polynomial time with respect to the domain cardinality.

## FOMC for Cardinality Constraints

Cardinality constraints are arithmetic constraints on the number of true interpretations of a set of predicates in a given FOL formula. In Example 6, we showed how different values of can represent different unary predicate cardinalities. Let’s formalize the correspondence between the multinomial factor and the cardinality of the unary predicates of the models that satisfy . For every with and for every unary predicate , we define

 k(Pj)=∑0≤i≤2u−1bin(i)j⋅ki

The following lemma states that is the number of such that .

###### Lemma 4.

For every -tuple of integers with , and every unary predicate , if then

###### Proof.

The lemma follows immediately from the definition of given in equation (5). ∎

Let be any arithmetic constraint on the integer variables representing the cardinality of unary predicates. We say that , if is satisfied when each is substituted for the integer .

###### Corollary 1 (of Theorem 1).

For every cardinality restriction on unary predicates,

 \sc fomc(∀xΦ(x)∧ρ,n) =∑k⊨ρ(nk)∏0≤i≤j≤2u−1nk(i,j)ij (8)
###### Example 7.

To count the models of (4) with the additional constraint that is balanced i.e., , we have to consider only the terms where is such . Equivalently in equation (8) we should consider only the such that . (Notice that is the number of elements that satisfy and and is the number of elements that satisfy and ).

To count models that satisfy cardinality restriction on binary predicates, we need to extend the result of Theorem 1. Similarly to what we have done for unary atoms, let be an enumeration of the atoms of that contain both variables and . Notice that the order of variables is not respected; for instance in Example 1, we have two predicates and . Every partial assignment to these predicates can be represented with an integer , with , with the usual convention that, if , then . Now for every and every , , where , and . We start by observing that

 nij=2b−1∑v=0nijv (9)
###### Example 8.

For instance introduced in Example 4 expands to where corresponds to the following assignments:

 A(x)R(x,x)\tabularcell@hboxA(y)\tabularcell@hboxR(y,y)R(x,y)R(y,x)\tabularcell@hboxv\tabularcell@hboxn13v\multirowsetup0\multirowsetup1\tabularcell@hbox\multirowsetup1\tabularcell@hbox\multirowsetup100\tabularcell@hbox0\tabularcell@hboxn130=1\tabularcell@hbox\tabularcell@hbox01\tabularcell@hbox1\tabularcell@hboxn131=0\tabularcell@hbox\tabularcell@hbox10\tabularcell@hbox2\tabularcell@hboxn132=1\tabularcell@hbox\tabularcell@hbox11\tabularcell@hbox3\tabularcell@hboxn133=0\omit\span\omitτx=1\omit\span\omitτy=3\omit\span\omitτxy=v

Notice that in the case of formulas in FO, is either or . By replacing in equation (6) with with its expansion (9) we obtain that is equal to

 ∑∑k=n(nk)∏0≤i≤j≤2u−1⎛⎝∑0≤v≤2b−1nijv⎞⎠k(i,j)

which can be rewritten as

 ∑k,h(nk)∏0≤i≤j≤2u−1(k(i,j)hij)∏0≤v≤2b−1nhijvijv (10)

where, for every , is a vector of integers that sum up to . Similarly to what we have done for unary predicates, we define for every binary predicate as follows:

 hij(R) =2b−1∑v=0(bin(v)l+bin(v)r)⋅hijv (11)

where and are the indexes such that corresponds to and to .

Let be any arithmetic constraint on the integer variables representing the cardinality of arbitrarily selected unary and binary predicates. We say that , if is satisfied when each is substituted for the integer if is unary and with if is binary.

###### Example 9.

A graphical representation of the pair for the formula (4) is provided in the following picture:

This configuration represent the models in which a set of constants are partitioned in four sets , each containing elements (hence ). Furthermore, for each pair and the relation is partitioned in 4 sub relations where each contains pairs (hence ). For instance if the pair it means that we are considering assignments that satisfy .

###### Corollary 2 (of Theorem 1).

For every cardinality restriction is equal to

 ∑k,h⊨ρ(nk)∏0≤i≤j≤2u−1(k(i,j)hij)∏0≤v≤2b−1nhijvijv (12)
###### Example 10.

Consider formula (4) with the additional conjunct and . The constraint implies that we have to consider such that . constraint translates to only considering monomials with .

## FOMC for the Existential Quantifier

Any arbitrary formula in FO can be reduced to an equisatisfiable reduction called Scott’s Normal Form(SNF) [17], see equation (19). In [11] prove that SNF also preserves WFOMC of the FO formulas. In this section, we extend our result for FOMC in universally quantified formulas to the whole FO fragment by providing an FOMC formula for SNF. We first show our method for the following special case of SNF:

 ∀x∀y.Φ(x,y)∧∀x∃y.Ψ(x,y) (13)

where and are formulae without quantifiers. First of all notice that:

 \sc fomc((???),n) =\sc fomc(∀xy.Φ(x,y),n) (14) −\sc fomc(∀xy.Φ(x,y)∧∃x∀y¬Ψ(x,y),n)

The first term of (14) can be computed by Theorem 1; for the second term we need to prove an auxiliary lemma, which uses the following notation:

 em =\sc fomc(∀xy.Φ(x,y)∧∃=mx∀y¬Ψ(x,y),n) pm =\sc fomc(∀xy.Φ(x,y)∧P(x)→¬Ψ(x,y)∧|P|=m,n)

where is a new unary predicate. In the following lemma we show that can be expressed as a function of ’s.

###### Lemma 5.
 em=n∑k=m(−1)k−m(km)pk

By induction on

#### m=n

The lemma holds since is equivalent to when the domain cardinality is .

#### m+1⟹m

(15) (16) (17) The equality of (16) and (17) can be obtained by expanding the summation and showing that all the terms of every internal summation cancel but one. We omit this expansion since it is routinary. ∎

###### Example 11.

An expansion of the statement of Lemma 5 with and is

Since is the first order model counting of a pure universal formula with cardinality restriction, it can be computed by the formula of Corollary 1. Lemma 5 tells us how to compute also starting from the ’s. Finally notice that, the second term of equation  (14) can be computed by summing from . This is possible since the set of models counted in are disjoint from the set of models counted in , where . This allows us to state the following theorem:

###### Theorem 2.

Let be the formula and let be the number of models of which are extensions of the partial assignments and , then

 \sc fomc((???),n)=∑∑k=n(nk)(−1)k(P)∏0≤i≤j≤2u−1nk(i,j)ij (18)
###### Proof.
 \sc fomc((???),n) =p0−n∑m=1em =p0−n∑m=1n∑k=m(−1)k−m(km)pk by Lemma 5

The expansion of the right terms is as follows:

 −(11)p1+(21)p2−(31)p3+…+(−1)n−1(n