An Expressive Probabilistic Temporal Logic
This paper argues that a combined treatment of probabilities, time and actions is essential for an appropriate logical account of the notion of probability; and, based on this intuition, describes an expressive probabilistic temporal logic for reasoning about actions with uncertain outcomes. The logic is modal and higher-order: modalities annotated by actions are used to express possibility and necessity of propositions in the next states resulting from the actions, and a higher-order function is needed to express the probability operator. The proposed logic is shown to be an adequate extension of classical mathematical probability theory, and its expressiveness is illustrated through the formalization of the Monty Hall problem.
In order to reason about probabilistic knowledge, we must reason about time and actions as well. When we say, for example, that “the probability of ‘heads’ after a coin toss is 50% and that of ‘tails’ is 50%”, we implicitly assume that there is an action (in this example, tossing a coin) which can bring the world to different states in the next moment in time. The uncertainty lies in the state transition: the world may end up in a state where the coin shows heads or in a state where it shows tails.
Despite the evident dependence of our informal notion of probability on the notions of action and time, the formal mathematical languages that we use to talk about probabilities rarely support mentioning action and time explicitly. Kolmogorov’s probability theory, for example, merely defines probability as the measure function in a measure space with total measure 1 . The task of modeling time-dependent actions and their possible outcomes in terms of events in a probabilistic space remains informal. While this informality is not problematic in the simplest situations (e.g. when we are interested in the possible outcomes of a single action, or when multiple actions are independent of each other), slightly more complex situations may already lead to confusion and difficulty. A famous example is the Monty Hall problem .
Another inconvenience of dealing with probabilities just in terms of a measure space is that its set-theoretic language (where events are represented as subsets of the sample space) is rather limited. There are obvious parallels between, for instance, set intersection and conjunction or set union and disjunction, which allow us to represent propositional probabilistic knowledge easily (e.g. the event of a randomly picked coin showing heads and the same coin being made of silver can be represented as the intersection of the event of showing heads with the event of being made of silver). However, it is not clear how this analogy could be extended to more expressive logics with quantifiers.
The main contribution of this paper, addressing the above mentioned issues, is the development of the syntax (in Section 2) and the semantics (in Section 3) of an expressive probabilistic temporal logic (PTL) for reasoning about actions with uncertain outcomes. PTL is an adequate extension of classical probability theory (as demonstrated in Section 4), and its greater expressiveness allows us to reason explicitly about event independence (as discussed in Section 5.1) and to avoid typical ambiguities of natural language discourse about probabilities (as shown in Section 5.2). This capacity of PTL to avoid ambiguities related to outcomes and events is one of its main conceptual novelties in comparison to related work (cf. Section 8). PTL’s convenience and expressive power are illustrated through the formalization of the Monty Hall problem (in Section 7).
The aim of PTL’s language is to be sufficiently expressive to capture typical probabilistic statements, conveniently similar to natural language, and yet more precise than natural language in cases when the latter is ambiguous. Intuitively, probability is an inherently higher-order function, since it takes a proposition (representing an event) as an argument. Therefore, if a probabilistic logical language is to include a probability operator in the syntactic level, it is only natural that it should be a higher-order language. Furthermore, because thinking probabilistically involves numerical computation and reasoning about states and actions, it is convenient to have a typed language, with distinct basic types for numbers, states and actions. The types used here are mostly the well-known simple types, but a list type constructor is included as well, in order to allow the representation of temporal sequences of actions and propositions.
Types are freely generated from the set of basic types , the right-associative function type constructor and the list type constructor list. is the type for states, is the type for booleans, is the type for objects and is the type for real numbers. The set of all types is denoted . The type of (local) propositions is defined to be an abbreviation for and the type of actions is defined to be an abbreviation for .
The definition of ensures that the truth of a proposition depends on states. The definition of follows the intuition that an action can be seen as a function that maps a state to a list of possible next states.
As shown in Definition 2.3, PTL contains, besides the usual logical symbols, also symbols for arithmetical functions and relations, the hybrid logic symbols for explicitly referring to states, list constructors and functions, and a probability operator. As modal operators ( and ) implicitly bind states, they have a more fundamental role, which reminds that of the binder. Therefore, they are treated separately in Definition 2.4.
For every type , is a countably infinite set of uninterpreted symbols of type . The set of arithmetic function symbols is the set . The set of arithmetic relation symbols is . The set of propositional logical symbols is . The set of hybrid logical symbols is . The set of quantifiers is . The symbol has type , has type , has type and the length operator has type . The probability operator has type . The set of all symbols is defined as .
Expressions are constructed as in the lambda calculus, using the symbols from , application, abstraction and modalities.
Expressions are constructed according to the following rules:
if , then is an expression of type .
if is an expression of type and is an expression of type , then is an expression of type .
if and is an expression of type , then is an expression of type .
if is an expression of type , is an expression of type and is an expression of type , then are expressions of type .
Formulas are expressions of type . Actions are expressions of type . The set of expressions of type is denoted . .
Types are omitted when they can be inferred from the context. The usual parenthesis conventions are followed. Numerals are occasionally written in decimal notation. Infix notation is employed as usual for logical connectives, arithmetical functions and relations and the list constructor . Binding notation is used for quantifiers. Additionally, the following notation conventions and abbreviations are used:
Probabilities appear in the logical language in two ways: firstly, as annotations on the diamond modal operator, in order to indicate how probable the corresponding state transition is; and secondly, through the higher-order probability function , which takes a list of actions and a proposition as arguments and returns the probability that the proposition will hold after the execution of the listed actions.
The following are some simple examples of probabilistic statements and their corresponding formalizations in PTL:
Tossing a coin has a transition with probability to a state where the coin shows heads:
The probability of a coin showing heads after it is tossed is :
The probability of a coin showing heads twice after it is tossed twice is less than : , where and .
After a coin is tossed it is necessarily either heads or tails:
After a coin is tossed it is possibly tails:
For each type , we need a domain of elements on which expressions of type are interpreted. For numerical expressions, we assume the domain to be a real closed field. For booleans, we assume the set with the usual two truth values. For function types, we require all functions to be present in the type’s domain. This effectively results in a standard higher-order semantics. For Henkin semantics, it would suffice to drop this last condition.
A domain for a type is a non-empty set such that is the set of all functions from to (for every and ), , and is the set of all lists of elements from .
As in the most common modal logics , we use frames as the foundation for the modal aspects of the semantics. A frame is essentially a set of states and a relation for the transitions between states. What is different here is that transitions are labeled by actions and by probabilities, and the transition relation and actions must be mutually consistent.
A probabilistic labeled frame is a triple such that is a non-empty set of states, satisfying the condition that if then for every , and is a probability function satisfying the condition that for all and for all such that there exists with ,
The relation in definition 3.2 may be cyclic. This is convenient, for instance, when specifying Markov chains.
A model extends a frame with an interpretation function that assigns denotations to expressions. The denotation of an expression may generally vary with the state. In such cases, we say that the interpretation is flexible; otherwise, it is rigid . In the examples considered in this paper, boolean expressions and probabilistic expressions are always flexibly interpreted, whereas other expressions are always rigidly interpreted.
A model is a tuple where is a probabilistic labelled frame, is a domain, and is an interpretation function that maps states and expressions of any type to elements in . It is assumed that any interpretation maps arithmetic symbols, list constructors and functions, and logical constants to their usual fixed denotations. Therefore (as usual, non-exhaustively):
iff for every
iff for some
is the function taking an element and returning .
where and for any distinct from .
Furthermore, and most importantly, the interpretations of expressions formed with modal and probabilistic operators are defined as follows:
for every such that
for some such that
In the probabilistic logic PTL, validity and satisfaction of a formula by a model are standard non-probabilistic notions, as defined below. The logic handles probabilities explicitly in its language; not at the semantic level.
A formula is satisfied in a model in a state , denoted iff . A formula is globally satisfied in a model , denoted iff for all . A formula is valid, denoted iff for every model . A set of formulas entails a formula , denoted , iff for every model such that .
This section shows how the usual mathematical presentation of probability theory, as recalled in Definition 4.4, can be considered a special case of the probabilistic logic presented here. This is done by showing (in Theorem 4.5) how to translate probability spaces into models and the usual set-theoretic language for probabilistic events into PTL’s language.
Set expressions over a set are expressions freely generated from singleton subsets of and operators for complementation (), union () and intersection ().
As usual, by abuse of notation, set expressions and the sets they denote are not explicitly distinguished.
If then the following are examples of set expressions: , , (denoting the set ), (denoting the set ), (denoting the empty set), …
A probability space is a triple where is the sample space (whose elements are outcomes), is a -algebra on (i.e. a collection of subsets of (events) closed under complementation, countable union and countable intersection) and is a probability function satisfying Kolmogorov’s axioms:
, for all
For any countable collection of mutually disjoint events
For every probability space , there is a model and a language translation function from set expressions over to formulas such that iff , for some and some .
Proof. Let be . For each , let be a distinct atomic proposition. Let be any interpretation such that iff . Let be . Let the probabilistic transition function be defined such that . Since , all are mutually disjoint and , the condition (from Definition 3.2) that
holds. Finally let be the model . The translation function is defined recursively:
By Definition 3.4 again and the definition of , the summation above can be simplified, resulting in the following equation:
Furthermore, unfolding the definition of , the equation above reduces to:
Therefore, it suffices to prove that the equation above holds iff , or equivalently, that:
By Kolmogorov’s third axiom, . Hence, letting be the following set:
A sufficient condition for the equation above to hold is that . This is proven below by induction on the structure of :
Base case (): then and, by definition of , iff .
(): then and hence:
By induction hypothesis, . Therefore, .
(): then and hence:
By induction hypothesis, and . Therefore, .
(): this case is analogous to the case above.
Informally, the idea of the proof of Theorem 4.5 is to translate a probability space into a model with a distinguished initial state and a future state for each possible outcome in the space. Any set expression (specifying an event) has a corresponding logical formula. The correspondence is as expected: union corresponds to disjunction, intersection to conjunction and complementation to negation. The translation is adequate in the sense that the probability of an event in the space is equal to the probability of the corresponding formula in the model.
A corollary of Theorem 4.5 is that the probabilistic logic PTL is more expressive than classical probability theory, in two distinct informal senses. The first one is syntactical: whereas the usual language of classical probability theory (which relies on set expressions) can naturally express formulas containing propositional connectives such as negation, conjunction and disjunction (through the inverse of the translation function defined in the proof of the theorem), there are formulas in PTL’s language (e.g. formulas containing quantifiers or nested probability operators) which have no (natural) counterpart in the language of classical probability theory. The second one is semantical: the proof of Theorem 4.5 shows that probability spaces correspond to models with a very simple frame; it would be inconvenient to express models with more complex frames in terms of probability spaces, because the frame structure would have to be flattened.
Shortcomings and limitations of probability spaces for knowledge representation become apparent in situations where a sequence of independent actions is performed over time. Suppose that a fair coin is tossed twice. Representing this as a probability space requires a sample space with four outcomes . Saying that, for instance, (where and ) requires the assumption of independence for the tosses. Two events and are often defined to be independent if and only if . But this definition is epistemologically unsatisfactory. How do we actually come to know that and are independent? According to this definition, we must know in advance. But that is precisely what, in practice, we do not know and would like to compute (based on our knowledge of and )! We can easily get trapped in circular reasoning, trying to justify, for instance, our claim that by saying that and are independent and then trying to justify that they are independent by saying that . Of course, we tend to escape from such cases of fallacious circular reasoning by simply assuming that the events are independent. However, the assumption is tacit. Classical probability theory provides no way to represent knowledge of the independence and any reason that we might have for justifying the assumption of independence of the events remains at an informal level, external to the representation.
In the PTL, on the other hand, the possibility to represent independence comes naturally and for free. For instance, when the axiom is assumed, it follows from the semantics of the logic that it holds in any state of the model. And since the axiom states the equal probabilities for heads and tails in a way that does not depend on anything except the action of the toss itself, it is clear that tossing a coin at a state has no effect on tossing the coin at another state . Therefore, the two tosses must be independent, and consequently it follows that:
Also dependence can be easily represented. For example, consider a magical coin that behaves as a fair coin in an initial state, but when tossed in any other state always gives the opposite result of the previous toss. This may be represented by the following axioms:
(no state is a predecessor of )
The inadequacy of classical probability theory’s usual definition of independence can be further illustrated in a situation where we have to randomly get an object from a bag with four objects: a black sphere, a white sphere, a black cube and a white cube. For simplicity, we assume tacitly that we put the object back in the bag after the action. This can be represented by the following axioms: A1: ; A2: ; A3: ; A4: ; A5: ; and A6: .
It then follows, by the semantics, that:
Nevertheless, we should not be willing to conclude from this result, as classical probability theory does, that the event of getting a spherical object and the event of getting a black object are independent. It is merely coincidental that . If the bag had an additional black tetrahedral, for instance, the two sides of this equation would not be equal anymore. In the formalization above, it is evident that both events are correlated, because they consist of outcomes from a single action.
A simple formal theory of (in)dependence of actions could provide the following definition for independence of an action from an action :
It follows from the semantics that entails the following shortcut theorem:
The notion of independence defined in is non-circular. We may, from the logical specification of a system in the PTL’s language, explicitly reason about the actions of the system, conclude that some of them are mutually independent and use the general theorem above as a shortcut for computing probabilities of sequences of actions. This is arguably more satisfactory than the teleological definition of independence from classical probability theory, which depends on the very shortcut theorem that we would have liked to derive.
It is not an aim of this paper to discuss or other theories of independence in detail. is just a (very simple) example showing that PTL is expressive enough to allow explicit reasoning about concepts that are very relevant in a probabilistic context.
Informal statements about probabilities are sometimes imprecise and ambiguous. Their intended meanings are not always clear. If a person tried to describe to a person the random effects of an action , her description might include a sentence such as: “the probability of after is ”. The most straightforward and literal logical meaning for this sentence would be . However, it is often the case that the meaning intended by is actually . must guess, from the context of the conversation and the common knowledge, which of the two alternatives is actually meant.
A formula such as provides fine-grained information about one particular state transition that is made possible by the action, whereas provides coarse-grained aggregated information about transitions to all states where holds. The aggregated information is incomplete, because it doesn’t say how many such states there are and it doesn’t specify the transition probability to each of these states.
The power to disambiguate is an interesting qualitative criterium to estimate the usefulness of a formal language. The formal probabilistic logical language proposed here is expressive enough to precisely disambiguate between and , which are subtly but importantly different in meaning, even though they are often expressed indistinguishably in natural language. It is important to note that neither nor is valid. Understanding the difference between and is crucial for a correct use of PTL. Furthermore, the difference in the meanings of and is essential to a semantics for probabilities that is compatible with our intuition about probabilities. Therefore, any sufficiently rich probabilistic logic should strive to distinguish between these important notions. PTL does so explicitly and syntactically.
In natural language dialogues, tends to cope with the ambiguity by subconsciously attempting to presuppose that fully specifies a single outcome of , in which case means . If this presupposition is incompatible with pre-existing knowledge or even with knowledge acquired later during the dialogue, the presupposition is canceled and the meaning falls back to . Fully understanding the dynamics of presuppositions is an open linguistic challenge, and probabilities bring yet another dimension of complexity to this difficult problem.
Consider the following statement:
“the probability of picking number (for ) is ”
Upon hearing this sentence, we tend to presuppose that there are six outcomes (i.e. ). However, if we are later told that:
“the number is picked by throwing a 12-faced fair dice where each (for ) occurs in two distinct faces.”
we are forced to cancel our presupposition and revise our logical interpretation of the previous sentence.
6 Implementation and Automation
A preliminary implementation of PTL in Coq is available in https://github.com/Paradoxika/ProbLogic. It follows the embedding methodology used in [1, 2], which is based on a higher-order and typed version of the standard translation of modal logics into predicate logic, with three important differences. Firstly, whereas in the standard translation the accessibility relation is a primitive constant, in the embedding of PTL it is derived from the primitive notion of action. Secondly, the higher-order modal logics used in  were rigid, while PTL includes a flexible probability function (which is simulated by a flexible predicate in the implementation). Finally, in contrast to the logics from , PTL requires numerical reasoning. It is this last point that makes the embedding of PTL significantly harder than previous embeddings and justifies its preliminary status. The current implementation still does not provide convenient modal tactics (as those described in ) and numerical reasoning is done with Coq’s standard QArith library for rationals (instead of real-closed fields). Decidability (of the satisfiability, validity and entailment problems) is indeed, of course, hopeless for the proposed higher-order logic. But even for logics with undecidability issues, automated theorem provers are occasionaly sufficiently efficient for practical applications . It is also important to note that, even if arithmetical expressions (of type ) are restricted to be ground (i.e. by forbidding quantifiers of type ), PTL thus restricted would still be expressive enough to formalize all the examples shown in this paper. In this restricted logic, the only automation of arithmetic needed is simplification/computation of arithmetic expressions and reduction of ground simplified arithmetic propositions to or . With the recent progress in SMT-solving and automated theorem proving modulo arithmetic (even with quantifiers), it is reasonable to hope that automated provers will soon be able to cope with PTL problems. In the meanwhile, the current implementation in Coq has already proven to be sufficient for a fully interactive formalization of the Monty Hall problem, as described in the next section.
7 The Monty Hall Problem
PTL is used here in the formalization of vos Savant’s famous Monty Hall problem , whose description is reproduced below:
Suppose you’re on a game show, and you’re given the choice of three doors: Behind one door is a car; behind the others, goats. You pick a door, say No. 1, and the host, who knows what’s behind the doors, opens another door, say No. 3, which has a goat. He then says to you, ‘Do you want to pick door No. 2?’ Is it to your advantage to switch your choice?
This probabilistic puzzle is seemingly paradoxical, because people very often make mistakes when they reason informally about the problem, as they tend to wrongly compute the probabilities. Therefore, despite its apparent simplicity, this problem is an interesting benchmark for evaluating formal probabilistic logics. A good probabilistic logic should allow a sufficiently natural and unambiguous formal representation of the problem and should entail correct probability values. From the player’s point of view, the Monty Hall problem can be formalized in PTL by the following axioms:
Axiom 1: “you’re given the choice of three doors”:
Axiom 2: “behind one door is a car”:
Axiom 3: “behind the others, goats”:
Axiom 4: “you pick a door, say No. 1, and the host, who knows what’s behind the doors, opens another door, say No. 3, which has a goat.”:
A more literal reading of Axiom 4 would be that “there exists a state (the current state), reachable from the initial state by the sequence of actions in which the host hides the car (), the player picks the first door (), and the host opens a door (), where the third door is open and has a goat.”. It is fair to say that the axioms shown above capture the intended meanings of their corresponding informal natural language sentences. As desired, the axioms are reasonably similar to the corresponding sentences, although there are interesting differences worth discussing, particularly in relation to Axiom 4. Firstly, it illustrates the need for the hybrid logic operators and in in situations where it is important to declare local conditions, which hold only in a single given state. Secondly, it shows the convenience of having a versatile approach to actions. The () action, for instance, takes the picked door as an argument whereas the () action takes no argument. This allows us to express that, from the point of view of the player, the action of picking a door is an action of the player and he can choose which door to pick, while opening a door is an action performed by the host, with uncertain outcomes to the player. The opening of the third door is represented as a random event of the action, through the proposition . These subtle differences between Axiom 4 and its corresponding sentence in the informal description of the problem are evidence that, as expected from a formal language, PTL offers a higher degree of precision than what we are used to in natural language.
There are many assumptions that are not explicitly mentioned in the description of the problem. But they must be formalized as well. We list below only some of them. Other axioms (e.g. stating what remains unchanged when actions are excuted) can be see in the Coq formalization discussed in Section 6.
Axiom 5: Each door has equal probability of having the car after the () action:
Axiom 6: The () action marks the picked door:
Axiom 7: The host opens a door containing a goat with uniform probability among the doors that are neither picked nor contain a car:
Axiom 8: When the player does the switch () action, the newly picked door is different from the previously picked door and from the open door:
Axiom 9: When the player does the no switch () action, the newly picked door is the same as the previously picked door:
Axiom 10: A state is a victorious state if and only if the car is behind the picked door:
The next step is the formalization of (the intended meaning of) the question (“Do you want to pick door No. 2? Is it to your advantage to switch your choice?”) as a conjecture. However, this is significantly less straightforward than the formalization of the axioms. A naive and literal reading of the question could result in the following tentative conjecture:
But the formula above is only satisfied in models where the probability of victory by switching is greater than the probability of victory by not switching in all states, whereas the question is interested in a few states only, namely those reachable by a given sequence of actions (i.e. hiding, picking, opening and re-picking). Taking this into account, an apparently plausible alternative formalization could be:
But this is trivially false in any model that satisfies the axioms above, because the action has a successor state (where the car was hidden behind the first door) such that:
Yet another possible attempt would be to formalize the conjecture as:
where is the current state when the question is asked and is a formula specifying whether is a posible current state (i.e. consistent with the player’s observations). However, for a similar reason, this formula is also false in any model that satisfies the axioms: there is a possible current state , where and . In fact, it is easy to see that, in any possible current state , and are always either and , because the action of switching has always only one possible outcome.
As evidenced by the failed conjectures above, there is a structural gap between the natural language question and the correct formalization of its intended meaning, and therein lies a potential reason (though probably not the only one) why people tend to have difficulties to reason about the Monty Hall problem. As it is posed, the question induces the player to think in terms of probabilistic outcomes of the action of switching or not switching in the current state. In contrast, the correct thinking requires the player to hypothetically backtrack to the initial state and formulate the conjecture as follows:
In any model satisfying the axioms (including the omitted axioms), and . Therefore, the conjecture is a theorem\@footnotemark\@footnotetextAn interactive proof of this theorem using the embedding of PTL in Coq is freely available in the online repository of the implementation discussed in Section 6. For the sake of simplicity, this formalization of the Monty Hall problem does not concern itself with specifying in which states each action is allowed or disallowed. But this could also be done..
8 Related Work
Many probabilistic logics are surveyed in . Among those logics, most depart from classical logic by adopting a probabilistic notion of validity and entailment. PTL, on the other hand, remains strictly classical in this respect. The probabilistic modal logics described in Sections 4.1 and 4.2 of  are probably the most similar to PTL. However, they are propositional, lack the probabilistic diamond operator, and are atemporal.
Probabilistic logics that incorporate time include PCTL [8, 4], which extends CTL by replacing the existential and universal path quantifiers by a probabilistic operator. PCTL is an excellent logic for model checking Markov chains. However, its lack of a probabilistic diamond operator makes it susceptible to the issues discussed in Section 5.2, thereby limiting its use beyond model checking. They also lack an explicit handling of actions, which is necessary for a convenient formalization of the Monty Hall problem and other examples discussed here. On the other hand, PCTL’s temporal modalities (which include, for instance, the until operator) are more sophisticated than PTL’s temporal modalities (which can only make statements about the next moment in time). PTL’s parsimony is intentional: it includes only the minimal set of temporal modalities needed to capture the desired notion of probability. Nevertheless, in practical applications where other temporal modalities are needed, they could be easily added to PTL as well.
9 Conclusion and Future Work
The large number of available probabilistic logics indicates that conciliating logic and probability is a non-trivial task. The expressive probabilistic temporal logic PTL described here provides a novel alternative approach, based on the simple intuition that the notion of probability can only be fully grasped in combination with the notions of action and time. The complex interaction of time, action and probability naturally leads to a modal and higher-order logic. PTL is adequate with respect to classical probability theory, of which it can be considered an extension (as shown in Section 4, where a correspondence between events and formulas has been established in detail). PTL’s convenient expressive power allowed a natural formalization of the famous Monty Hall problem. One of the main insights in the development of PTL came with the discovery of the need for both a higher-order probability function and a probabilistic diamond operator, as discussed in Section 5.2. Besides the higher order, the satisfaction of this need is a distinguishing feature of PTL.
In the near future, the implementation of PTL in Coq needs to be made more user-friendly, through the implementation of tactics that automate and hide technical details for users. On the philosophical side, it would be interesting to extend PTL with past temporal modalities, since we often need to reason about actions that have happened in the past but whose outcomes we have not yet observed, and to define conditional probabilities, in order to explore the question about the relationship between probabilities of conditionals (e.g. ) and conditional probabilities (e.g. )  from PTL’s perspective.
Benzmüller, C. and B. W. Paleo, Automating gödel’s
ontological proof of god’s existence with higher-order automated theorem
provers, in: ECAI 2014 - 21st European Conference on Artificial
Intelligence, 18-22 August 2014, Prague, Czech Republic - Including
Prestigious Applications of Intelligent Systems (PAIS 2014), 2014, pp.
Benzmüller, C. and B. W. Paleo, Interacting with modal logics in
the coq proof assistant, in: Computer Science - Theory and
Applications - 10th International Computer Science Symposium in Russia, CSR
2015, Listvyanka, Russia, July 13-17, 2015, Proceedings, 2015, pp. 398–411.
-  Blackburn, P., M. de Rijke and Y. Venema, “Modal Logic,” Cambridge University Press, New York, NY, USA, 2001.
Brázdil, T., V. Forejt, J. Kretínský and A. Kucera,
The satisfiability problem for probabilistic CTL, in:
Proceedings of the Twenty-Third Annual IEEE Symposium on Logic in
Computer Science, LICS 2008, 24-27 June 2008, Pittsburgh, PA, USA, 2008,
-  Demey, L., B. Kooi and J. Sack, Logic and probability, in: E. N. Zalta, editor, The Stanford Encyclopedia of Philosophy, 2014, fall 2014 edition .
-  Fitting, M., “Types, Tableaus, and Gödel’s God,” Kluwer Academic Publishers, 2002.
Hajek, A., “The Conditional Construal of Conditional Probability,”
Ph.D. thesis, Princeton (1993).
-  Hansson, H. and B. Jonsson, A logic for reasoning about time and reliability, Formal Aspects of Computing 6 (1994), pp. 512–535.
-  Kolmogorov, A., “Grundbegriffe der Wahrscheinlichkeitsrechnung,” 1933.
-  vos Savant, M., Ask marilyn, Parade Magazine 16 (1990).