Typed Answer Set Programming and Inverse Lambda Algorithms
Abstract
Our broader goal is to automatically translate English sentences into formulas in appropriate knowledge representation languages as a step towards understanding and thus answering questions with respect to English text. Our focus in this paper is on the language of Answer Set Programming (ASP). Our approach to translate sentences to ASP rules is inspired by Montague’s use of lambda calculus formulas as meaning of words and phrases. With ASP as the target language the meaning of words and phrases are ASPlambda formulas. In an earlier work we illustrated our approach by manually developing a dictionary of words and their ASPlambda formulas. However such an approach is not scalable. In this paper our focus is on two algorithms that allow one to construct ASPlambda formulas in an inverse manner. In particular the two algorithms take as input two lambdacalculus expressions G and H and compute a lambdacalculus expression F such that F with input as G, denoted by F@G, is equal to H; and similarly G@F = H. We present correctness and complexity results about these algorithms. To do that we develop the notion of typed ASPlambda calculus theories and their orders and use it in developing the completeness results.
Typed Answer Set Programming and Inverse Lambda Algorithms]Typed Answer Set Programming Lambda Calculus Theories and
Correctness of Inverse Lambda Algorithms with respect to them
Chitta Baral et al.] Chitta Baral, Juraj Dzifcak, Marcos A. Gonzalez and Aaron Gottesman
School of Computing, Informatics, and Decision Systems Engineering
Arizona State University, Tempe, AZ
1 Introduction
The broader goal of our proposed research is to translate English sentences to appropriate knowledge representation (KR) and reasoning languages. This will help in understanding text and answering questions with respect to it. Such an ability is important in developing various systems that need “understanding of natural language”. This includes systems that can acquire knowledge from text, systems that can interact in English with robots and other systems, intelligent training and tutoring systems, and systems that can process existing scientific literature in particular domains and formulate hypothesis.
Our approach is inspired by Montague’s work [Montague:Book] where the meaning of words and phrases are expressed as calculus expressions and the meaning of a sentence is built from the meaning of its words by making appropriate applications of the corresponding calculus expressions. This approach has also been used elsewhere, such as in [Bos:Book, Collins:2005, Baral:2008, Me:2009, stefania2010, Baral:Festschrift]; but the question that we address here is how do we obtain the calculus like meaning of words? They get complex quickly and hand crafting them is not scalable.
In this paper we use ASP as our target KR language and address the issue of automatically obtaining meaning of words as ASPcalculus formulas instead of the underlying logic of first order logic in traditional use of calculus. Thus, the meanings of words are expressed as formulas of ASPcalculus and using them sentences are translated to ASP rules. In [Baral:2008] the ASPcalculus formulas that represent words were handcrafted and it was remarked that the human engineering needed to generate the ASPcalculus expressions need to be substituted by an automatic process.
Our main idea in automating this process is through a a notion of inverse application of ASPcalculus formulas and use them in constructing the ASPcalculus expressions of words. We discuss two algorithms from [marcosthesis10] that compute a ASPcalculus expression
We illustrate the basic idea of inverse application of ASPcalculus formulas and how they can be used in constructing the ASPcalculus expressions of words through the following example.
Most  birds  fly 

Most  birds  fly 

???  .  . 
???  .  
It is assumed in Table 1 that the meaning of “Most birds fly” and the ASPcalculus formulas for “fly” and “birds” is known. We would like to determine the appropriate semantic representation for the word “most”. To do so we must first compute the semantic representation of “Most birds”. This can be done using the meaning of the sentence “Most birds fly” and the word “fly”. However, we first must know whether the meaning of “Most birds” is to be used as input to the meaning of “fly” or vice versa. To obtain this directionality information we make use of combinatory categorial grammars (CCG) [Steedman:book].
A CCG parse of a sentence assigns categories to the words of the sentence. There are several basic categories, with representing a sentence and representing a noun phrase. More complex categories are formed from these basic categories by using “” and “/” which specify directionality. For example, a nontransitive verb, like “fly” above, would have category meaning that if a noun phrase, , precedes the verb then a sentence is formed. Similarly, a category for a simple adjective, would be , meaning that if a noun phrase, , comes after the adjective then a would result.
Note that the category of “most” given here is not that of a simple adjective, . If “most” had this category then the result of applying “birds” to “most” would result in category which would then be applied to the right of “fly”. However, it is not possible to form the meaning of the sentence by substituting into the given meaning of “fly”. Therefore, an alternative CCG parse of the sentence must be used that swaps the application of “fly” to be on the right side. This is done by raising the category of “Most birds” to , which in turns means that the category of “most” must be .
The top part of Table 1 gives a CCG parse of the sentence “Most birds fly”. The meaning of the phrase “Most birds”, which has a category , must have the meaning of “fly” applied from the right since it has the category . Therefore, to get the meaning of the sentence, , we let be the meaning of “fly”. Then we have to find an such that . From inspection . will satisfy this equation.
Now, having the expressions for “Most birds” and “birds”, we can calculate the meaning of the desired word “most”. Since “most” has category , we have to apply the meaning of “birds” to the right of it to obtain the meaning of “Most birds”. From inspection taking the meaning of “most” to be .. produces the desired result.
As this example demonstrates, given the meaning of most words in a sentence and a CCG parse for the sentence, we can find a new semantic representation for words and phrases whose meanings are unknown. The question is how exactly do we determine the new representation? In this paper we discuss the Inverse Algorithms [marcosthesis10] to solve this task, which is known as the Inverse problem.
To help in showing the correctness and applicability of our and algorithms we extrapolate the notion of typed first order theories to define the notion of a typed ASP theory. We then define the notion of orders of such theories. Using these notions we present the soundness, completeness and complexity results of the Inverse Algorithms. For example, the completeness result is with respect to typed ASPcalculus formulas up to the second order
As mentioned earlier, these algorithms are key to developing systems that can translate English sentences to KR languages. However, such systems need to address additional issues such as dealing with possible multiple meaning of words, and developing appropriate ontologies that maximize the accuracy of the translation. These aspects are separately discussed in [Me:2009]. A simpler version of the Inverse Algorithms discussed in this paper is used in developing a system that learns to translate combinatorial puzzles to ASP rules and solve those puzzles [baral2012kr]
In summary the main contributions of this paper are:

We formulate the notion of typed ASPcalculus theories and define the notion of orders of such theories.

We illustrate the use of Inverse Algorithms with respect to typed ASPcalculus formulas.

We present soundness, completeness and complexity results for these algorithms with respect to typed ASPcalculus theories.
The rest of this paper is organized as follows. In the next section, we present some background material and pointers on typed lambda calculus and ASP. In Section 3 we introduce typed Answer Set Programming lambda calculus. In Section 4 we present the Inverse Algorithms. We then illustrate our algorithms with respect to several examples and give a use of our algorithms in sections 5 and 6, respectively. In Section 7 we present the soundness, completeness and complexity results. Finally, we conclude and briefly mention the companion natural language semantics work that uses our algorithms.
2 Background
2.1 Typed Lambda Calculus
Since Montague’s groundbreaking work [Montague:Book], calculus has been accepted and used as a tool by many in natural language semantics. Montague was the first to introduce the use of calculus to represent the meaning of words and application as a mechanism to construct the meaning of phrases and sentences. However, to properly understand the notion of “meaning” (or semantics), it is useful to consider models of calculus expressions. When referring to a model, one is looking for a semantic tool that can give it two elements: the entities that are part of the domain, and for every element in the signature, the semantic value associated with it. By creating this model with the corresponding denotations for types, expressions of the system will have a defined type and a semantic value associated with it.
Both untyped and typed calculus can be characterized using models, but typed calculus has had the most impact on natural language semantics, which also became familiar to linguistics after the mentioned works by Montague. In this paper we will follow the Simply Typed Lambda Calculus of Church [Church:1940] to have ASP as the core logic. This is the most commonly used approach in linguistics where only one type constructor is used to build types, “”, and each term has a single type associated with it [Barendregt:Book].
Because of space constraints, we do not present the typed lambda calculus definitions that we use to define typed ASP calculus. The books [Hindley:Book] and [HindleyType:Book] are good reference points for typed lambda calculus.
2.2 Answer Set Programming
Answer Set Programming is the language of logic programming with answer set semantics [GelfondLifschitz]. This language is one of the most suitable declarative languages [Baral:Book] for knowledge representation, reasoning and declarative problem solving; all important aspects that are needed to develop natural language understanding systems. It has a large body of support structure, including efficient implementations and theoretical building block studies. It allows the representation, in an intuitive way, of various kinds of knowledge that cannot be adequately expressed in firstorder logic. These include, for instance, default statements ( ) and normative statements ( ). We now present some basic definitions related to Answer Set Programming syntax and semantics [Baral:Book].
Definition 1 (ASP rule)
An ASP rule is of the form:
………
where are literals and k 0, m k and n m.
The literals to the left of the “” belong to the Head of the rule, and the literals to the right of the “” belong to the Body of the rule. An ASP program is a set of ASP rules.
Definition 2 (Satisfiability)
An ASP rule of the form:
………
of an ASP program is said to be satisfied by a set of ground literals I of if:
{…} I and {…} I = implies that {…} I .
Definition 3
An Answer Set of an ASP Program without the “not” operator, is a consistent set of ground literals such that satisfies all rules of and no subset of satisfies all rules of .
Definition 4 (Answer Set)
A consistent set of ground literals is an Answer Set of an ASP Program if is an answer set of the reduct , where is obtained from by

Deleting all rules from that contain some in their body for some .

Removing all occurrences of from the remaining rules.
3 Typed Answer Set Programming Lambda Calculus
We start by presenting the signature for the language Typed Answer Set Programming Lambda Calculus (Typed ASP Lambda Calculus). It consists of the following:

the lambda operator, also called abstractor, ;

the lambda application ;

the parenthesis symbols and ;

for every type , an infinite set of variables for each natural number ;

for every type , a (possibly empty) set of constants of type ;

the connectives , , , , “,” and “.”; and

predicate and function symbols.
Variables and constants in the signature for Typed ASP Lambda Calculus will be referred to as terms.
Next, we introduce the set of types that will be used with Typed ASP Lambda Calculus, in conjunction with the definition of the semantics of types assigned to the different expressions of the language. We will follow the principles presented in [MathLing:Book], where represents the set of possible objects (denotations) that describe the meanings of expressions of type a.
Definition 5 (Types)
The set of types is defined recursively as follows:

are types, called base types, and

if and are types, then is a type.
Intuitively, refers to terms, which is either a variable or a constant in ASP, or a function symbol with terms as input; refers to atoms of ASP, which are predicate symbols with terms as input
Definition 6 (Type Semantics)
Given an ASP Program , the semantics of is defined using:

= the set of terms and functions in the language of ;

= the set of atoms in the language of ;

= the set of literals;

= the set of genliterals;

= the set of “”connected literals that belong to heads of ASP rules;

= the set of “”connected genliterals that belong to bodies of ASP rules;

, the set of satisfiability values for an ASP program; and

the set of functions from to .
Expressions of type denote satisfiability values of ASP programs. An ASP program can be true under certain Herbrand interpretations, and false under others. denotes functions whose input is in and output values are in . For example, the type corresponds to functions from terms to satisfiability values.
This section continues by introducing the definition for ASP typed term, followed by the definition of ASP calculus formula:
Definition 7 (ASP Typed Term)
The elements which belong to the set of ASP typed terms of type are inductively defined as follows:

For each type , every term of type belongs to .

For any types and

if and , then @

if is a variable of type and has free occurrences of the variable , then u. and the free occurrences of are now bound to the abstractor u.
^{5}


If is a function symbol with arity , and , then
. 
If is a predicate symbol with arity , and , then
. 
If and is not a term
^{6} , then and (. 
If and is not a term, then and ( .

If and is not a term, then .

If , then .

If and is not a term, then .

If , then .

If and , then (, (, and (.

If , then .
Definition 8 (Typed ASP Calculus Formula)
A typed ASP calculus formula is an ASP typed term where every variable is bound to an abstractor and every abstractor binds to a variable.
A typed ASP calculus formula is in normal form if it does not contain any redex occurrences. An example of a redex is a typed term of the form (v.v)@. The typed term or v.v, do not have any redex occurrences [HindleyType:Book].
The binding of every variable to an abstractor and every abstractor to a variable in definition 8 correspond to closed and Iterms in the classic theory of lambda calculus, respectively. These conditions ensure that one obtains Answer Set Programming programs when the typed ASP calculus formulas are in normal form. By the way in which the ASP typed terms have been defined and the two properties that we are enforcing, when there are no lambda abstractors left in an ASP typed term we obtain an expression that belongs to the Answer Set Programming language presented above. Some examples of typed ASP calculus formulas are the following:

... with type where has type , has type , and has type .

... with type where has type and has type .

. with type where has type .

... with type where has type , and have type .
Let the fourth formula of the examples be . In , has type because when an ASP typed formula is applied , it will be placed in the variable and will receive as argument the expression .. This expression has type and therefore the input of the formula applied to needs to have as input and as output to lead to an ASP formula. Thus, has type . has type meaning that one expects a term to be placed inside the literal .
The following are not typed ASP calculus formulas:

.., where has type and have type .

.., where has type , has type , and has type .
The first expression is not a calculus formula since is of type which means has type from above. Then must have type from . However, there is no rule that allows us to combine , an element of type , with , an element of type , with the connective .
The second expression violates the rules of a typed ASP calculus formulas since there is no rule saying that the connective can be applied to terms of type , which in this example is the type of .
This section concludes with two more definitions.
Definition 9 (Occurrence)
The relation P occurs in Q is defined by induction on as follows:

an ASP typed term occurs in .

if occurs in or in , then occurs in .

if occurs in , then occurs in .

if occurs in or occurs in , then occurs in , and .

if occurs in , then occurs in and .

if occurs in any term , then occurs in …. Where is a function symbol.

if occurs in any term , then occurs in …. Where is a predicate symbol of an atom.
Definition 10 (subterm)
A subterm of a typed ASP calculus formula is any term that occurs in .
3.1 Type Order
We have introduced the types that will be assigned to typed ASP lambda calculus terms and formulas. Next, we present the notion of order, which is associated with types and establishes a hierarchical structure that separates typed calculus formulas to several classes. Order will be an important concept when we state the completeness proof for the Inverse Algorithms since we will show that they are complete for typed calculus formulas up to order two. Each typed term has a type, and each type will be assigned an order.
Definition 11 (Type Order)
The order of a type is defined as:

Base types have order 0.

For function types, ) = .
The definition from [Stirling:Match] gives order one to base types. In our case, we consider order zero for base types since this is the common approach in linguistics. Next, some examples of typed ASP lambda calculus formulas with different orders are presented:

Order zero:  type .

Order one: ..  type .

Order two: ..  type .

Order three: ..  type .
With these simple examples, one can see the intuition behind the order of typed ASP lambda calculus formulas. Formulas of order zero correspond to expressions with base types. Formulas of order one correspond to expressions which start with a series of lambda abstractors followed by an ASP program with variables bound to the initial lambda abstractors.
Formulas of order two extend the expressions allowed in order one by including applications. Formulas of order zero can be applied to variables inside the formula. Formulas of order three extend those present in order two by allowing lambda abstractors inside the expression after the initial lambda abstractors. In this case, formulas of order one can be applied to variables, this is why now, we can find lambda abstractors at the beginning and in the middle of the formulas. These claims can be easily proved by contradiction using the given definitions.
4 The Inverse Lambda Operators
This section presents the formal definition of the two components of the Inverse Algorithms, and , from [marcosthesis10]. The objective of and is that, given typed calculus formulas and , the formula is computed such that and , respectively. We now define the different symbols used in the algorithm and their meaning:

Let , and represent typed calculus formulas, ,,…, represent typed terms; , and ,…, represent variables.

Typed terms that are subterms of a typed term are denoted as .
We also consider the following two statements:

A list of abstractors of the form … can be empty if the corresponding variables … are not present in the formula they belong to.

If the formulas being processed within the algorithm do not satisfy any of the conditions then the algorithm returns .
Definition 12 (Operator :)
Consider two lists (of same length) of typed ASP calculus formulas … and …, and a typed ASP calculus formula . The result of the operation …… is defined as:

find the first occurrence of formulas … in .

replace each by the corresponding .

find the next occurrence of formulas … in and go to 2. Otherwise, stop.
We now give the two inverse algorithms.
Definition 13 ()
The algorithm , is defined as:
Given and :

If is .

then = .


If is a subterm of

then = .


If is not ., … … … … are subterms of , and , is ………
^{7} with 1 and , 1 .
then = .…… … where each maps to a different in .


If is … and … is a subterm of ,
is .…… with , .
then = .…………

Definition 14 ()
The algorithm , is defined as:
Given and :

If is .

then


If is a subterm of and is .

then =


If is not ., …… … … are subterms of and is .……… … with 1 and , 1 .

then = ….…….


If is …. and … is a subterm of ,
is .….……… with , .
then = ……

Please note that the final cases for both operators involve formulas of third order.
5 Inverse Lambda Algorithm Examples with Typed ASP Lambda Calculus
This section presents several examples demonstrating how the Inverse Algorithms can be applied to find in various settings given ASPcalculus formulas and . A use case example follows in the next section.
5.1 Example 1
Let and be typed ASP calculus formulas where = . and = .. needs to be calculated such that = @ . Here case 1 of will be applicable. Then = . @ and in this case = . @ . Then, @ = . @ @ . = (. @ = = .
5.2 Example 2
Let and be typed ASP calculus formulas where = . . and = . needs to be calculated such that = @ . Here case 2 of is applicable. Thus, we get = . : = . : = .. .
5.3 Example 3
Let and be typed ASP calculus formulas with = . and = .. . now needs to be calculated such that = @ . Therefore, case 3 of will be applicable.
is not . so the first condition is satisfied. From , one has the following formulas that are subterms: = with subsubterms = and = (from ); = with subsubterms = and = (from ). Therefore the second condition of case 2 is satisfied.
The third condition is satisfied since, : = .. : , for = 1,2. For example, for , = .., : , = .. .
Therefore one can now calculate that = . : @ @ : @ @ = . : @ @ : @ @ = .. @ @ , @ @ .
5.4 Example 4
Let and be typed ASP calculus formulas with = , , . and = . @ @ @ @ . needs to be calculated such that = @ . Case 3 of will be applied.
is clearly not . @ . has the following subterms: , = , with subsubterms = and = ; , , = , with = and = . Then, =., : @ @ , , : @ @ = ., : @ @ , : @ @ . = . @ @ @ @ . Therefore, satisfies the second condition of case 3.
Thus, we calculate =.., : , = .., : , = .., .
5.5 Example 5
Let and be typed ASP calculus formulas where = . , . and = .. @ ., . needs to be calculated such that = @ . Therefore, case 4 of will be applied.
= . with = . , , ) = with = 2 and = . Relabeling the variables of to better match the conditions of the case by substituting for and for , we see = .. @ . : . Therefore, satisfies the second condition of case 4.
Thus, we calculate = . : @ = . @ .
6 Use Case Examples
In this section we present a use case of our inverse lambda algorithms to show how meaning of words are computed when one knows meaning of the sentences and meaning of some of the words. We consider the following sentences from [Baral:2008].

Most birds fly.

Penguins are birds.

Penguins do not fly.
We will consider an initial lexicon that has the semantics for simple nouns and verbs. Combinatory Categorial Grammar (CCG) [Clark:CCG] is used to construct the meaning of a sentence from the meaning of its constituent words and phrases. After parsing the first two sentences using CCG and adding the semantics from the initial lexicon, we obtain the output of a simplified CCG parsing with two categories “S”(sentence) and “NP”(noun phrase), as shown in Table 2.
Most  birds  fly 

Penguins  are  birds 

Most  birds  fly 

???  .  . 
???  .  
Penguins  are  birds 
.  ???  . 
.  ???  
In Table 2, one can see that the semantic representations for the words “most” and “are” are missing. We already discussed how we can obtain the semantic representation of “most” before. Now we will illustrate how we can compute it using the presented Inverse Algorithms. Starting with the first sentence, one can take the meaning of the sentence and the meaning of the word “fly” to calculate the representation of “Most birds”.
“Most birds” has category and the category of “fly” is being applied from its right. Therefore, if we take as the meaning of the sentence and as the meaning of “fly”, we can use to obtain the expression for “Most birds”. In this case, option three of the algorithm is satisfied and = ..
Now, we have the expression for “Most birds” and “birds”. Since the word “birds” is being applied to the right of “Most birds”, we need to again call to obtain the representation for “Most”. Option three of the algorithm is again satisfied and one obtains as ... This is the Typed ASP calculus representation for the word “most”.
The process to obtain the word “are” from the second sentence is very similar. First one calls with being the meaning of the sentence and being “Penguins” to obtain the meaning of “are birds”. Option three of the algorithm is satisfied and = .. Next, one calls with “are birds” and “birds” to obtain the desired meaning of “are”. Option three of the algorithm is satisfied again and = ..). This expression corresponds to the Typed ASP calculus formula for the word “are”. Next, the last sentence is presented in Table 3.
Penguins  do not  fly 

Penguins  do not  fly 
.  ???  . 
???  .  
In this case, the semantics of the phrase “do not” is missing. This phrase was not part of the initial lexicon. For this sentence, we call first, obtaining . as the meaning of “Penguins do not”, and afterwards to obtain the representation for “do not”, which is ..).
7 Correctness and Complexity of the Inverse Algorithms
{theorem}[Soundness of ] Given two typed calculus formulas and in normal form, if returns a nonnull value , then = .
[Soundness ] Given two typed calculus formulas and in normal form, if returns a nonnull value , then = .
[Completeness of ] For any two typed calculus formulas and in normal form, where H is of order two or less, and G is of order one or less, if there exists a set of typed calculus formulas of order two or less in normal form, such that , , then will give an where .
[Completeness of ] For any two typed calculus formulas and of order two or less in normal form, if there exists a set of typed calculus formulas of order one or less in normal form, such that , , then will give an , where .
[ complexity] The Algorithm runs in exponential time in the number of variables in and polynomial time in the size of the formulas and .
[ complexity] The Algorithm runs in exponential time in the number of variables in and polynomial time in the size of the formulas and .
Due to lack of space, we will only comment on how the soundness and completeness proofs are structured. The complete proofs are given in the online appendix of the paper. The soundness proof shows how in each of the four cases of , the typed ASP calculus formula is obtained by applying to . The application @ is computed using the expressions from the algorithm for and , generating the expression for given in the algorithm. The proof of Theorem 1 is given in the online appendix of the paper, pp. 2–3. The same reasoning is followed for . The complete proof of Theorem 2 is given in the online appendix of the paper, pp. 3–4.
The completeness proof is divided to six cases, which correspond to the six possible valid combinations of orders that , and may have, such that the order of the terms will be less than 2. These are shown in Table 4. For each case, it is proven by contradiction that and return a formula if one such exists. It is done by assuming that they return a value and reaching a contradiction at the end of the proof. In the process, each of the four conditions of the algorithms are analyzed, where it is shown that at least one of the conditions of the algorithm has be satisfied for each of the six cases. The complete proof of Theorem 3 is given in the online appendix of the paper, pp. 4–7 and Theorem 4 is given in pp. 7–11.
Finally, the proof for the complexity results, Theorem 5 and Theorem 6, are given in the online appendix of the paper, pp. 11–12.
H  F  G  ASP type examples for formula F  
0  1  0  
1 
1  0  
2 
2  0  
0 
2  1  
1 
2  1  
2 
2  1 
8 Conclusion
In this paper we developed the language of typed answer set programming lambda calculus and defined associated notions such as ASP typed term, ASP calculus formulas, and type orders. We used these notions to formulate soundness and completeness of Inverse Algorithms with respect to typed answer set programming lambda calculus. These algorithms are important in that they allow automatic construction of ASPlambda representations of new words using information already available about known sentences and words. They have been used in a system that is able to learns to translate combinatorial logic puzzle descriptions to ASP rules [baral2012kr] that obtain solutions to the puzzles; however that (short) paper does not go into the details of the algorithm, as we do here.
Acknowledgement
We acknowledge support by NSF (grant number 0950440), IARPA and ONRMURI for this work.
References
Footnotes
 This algorithm also works for typed firstorder logic lambda calculus. We show that in [Baral:Festschrift]. But its applicability to ASP is not discussed there as that requires additional machinery, which we present in this paper.
 The Inverse problem can be shown to be a special cases of the “higherorder matching” [Dowek:Match] and “Interpolation problem” [Stirling:Match] . Specifically, the problem corresponds to an Interpolation problem and problem corresponds to the Higherorder matching problem. The higher order matching problem is known to be undecidable in the general case [Loader]. Higher order matching can be further considered as a special case of higher order unification which has been explored in [huet73, huet75] and recently used in [kwiat10]. None of these works consider ASP.
 Blackburn and Bos say in page 101 of their book [Bos:Book]: “Now, arguably natural language semantics never requires types much above order three or so–nonetheless the ability to take a logical perspective on higherorder types really is useful.” Note that their definition of order three corresponds to our definition of order 2.
 An atom is said to be ground if none of the terms in the atom contain a variable
 Refer to the definition of occurrence presented at the end of this section.
 This is to guarantee that each term only corresponds to its unique type.
 When the formula is being generated, the indexes of the abstractors