The New Frontier
At this point in time, two major areas of physics, statistical mechanics and quantum mechanics, rest on the foundations of probability and entropy. The last century saw several significant fundamental advances in our understanding of the process of inference, which make it clear that these are inferential theories. That is, rather than being a description of the behavior of the universe, these theories describe how observers can make optimal predictions about the universe. In such a picture, information plays a critical role. What is more is that little clues, such as the fact that black holes have entropy, continue to suggest that information is fundamental to physics in general.
In the last decade, our fundamental understanding of probability theory has led to a Bayesian revolution. In addition, we have come to recognize that the foundations go far deeper and that Cox’s approach of generalizing a Boolean algebra to a probability calculus is the first specific example of the more fundamental idea of assigning valuations to partially-ordered sets. By considering this as a natural way to introduce quantification to the more fundamental notion of ordering, one obtains an entirely new way of deriving physical laws. I will introduce this new way of thinking by demonstrating how one can quantify partially-ordered sets and, in the process, derive physical laws. The implication is that physical law does not reflect the order in the universe, instead it is derived from the order imposed by our description of the universe. Information physics, which is based on understanding the ways in which we both quantify and process information about the world around us, is a fundamentally new approach to science.
address=University at Albany (SUNY), NY, USA
The New Frontier
Keywords: entropy, information physics, lattice, measure, order, poset, probability, quantification, quantum mechanics, relativity, valuation.
PACS: 03.65.Ta, 03.30.+p, 02.50.Cw, 89.70.Cf, 89.70.-a
“Measure what is measurable, and make measurable what is not so.”
Galileo Galilei (1564-1642)
In the last century, there were three individuals whose ideas revolutionized the way we view information and probability. The first of these individuals was Claude Shannon who, while in graduate school, realized that Boolean algebra could be used to simplify telephone networks. This insight paved the way for digital computers, which clearly have revolutionized all aspects of human society. However, it also led to a more subtle revolution based on Shannon’s quantification of information transmitted by a communication channel. Shannon’s information took the curious form of entropy Shannon and Weaver (1949), which at the time was believed to be a physical property of a thermodynamic system.
Around the same time, a physicist, Richard Threlkeld Cox, published a paper where he obtained probability theory as a unique quantification of degrees of plausibility deriving from a generalization of Boolean algebra Cox (1946). To this day, Cox’s results are not fully appreciated by the scientific community. His approach forms a foundation for probability theory that stands alongside of the measure-theoretic foundation provided by Kolmogorov. While Kolmogorov’s approach is founded in traditional mathematical rigor, Cox’s approach relies on a purpose-driven generalization, which is perhaps more satisfying to physicists, but less so to mathematicians. However, the motivation behind the specific generalization that Cox proposes gives meaning to the concept of probability, which is something that Kolmogorov’s approach lacks. As Bayesians, we often view probabilities as degrees of plausibility, or degrees of belief, and many of us have come to find Cox’s views quite natural.
Edwin T. Jaynes discovered Shannon’s paper in the Princeton library, and as he says, he disappeared for about a week Jaynes (1956). Upon re-emerging, he declared to anyone who would listen that this was the greatest piece of work since the discovery of the Dirac equation. Jaynes writes,
It’s almost impossible to describe the psychological effect of seeing our old familiar expression for entropy derived in a completely new way, and then applied with great success to problems of engineering which apparently have no relation to thermodynamics. But all of the inequalities, which are usually associated with the second law of thermodynamics, turn out to be statements of the greatest practical usefulness in engineering problems. It seemed to me that there must be something pretty important that we could learn from this situation. (Jaynes, 1956, p. 3)
Many of the early attempts to employ information theory in physics were based on making analogies between the communication theory and statistical mechanics. Jaynes realized that the connection was not in the form of a simple analogy, but was something far more subtle. He writes
the essential content of both statistical mechanics and communication theory, of course, does not lie in the equations; it lies in the ideas that lead to those equations. (Jaynes, 1956, p. 4)
Jaynes continues by writing
the job as I saw it was not to try to invent any fancy new mathematics. That would presumably come later if we were successful. The job was to find the viewpoint from which we could see that the reasoning behind communication theory and statistical mechanics was really the same. (Jaynes, 1956, p. 5)
This critical insight will be relevant again when we look at extending these ideas to quantum mechanics and beyond.
Jaynes was also aware of Cox’s work in 1956 when he gave his lectures on Probability Theory in Science and Engineering. Jaynes appreciated Cox’s approach as it made clear that probability quantified a state of belief about a physical system rather than the state of the physical system itself. He recognized that the latter viewpoint, led to potential misconceptions when probability theory was applied in physics. While he was clearly convinced of the interpretation of probability as a degree of plausibility, he, like many of us, was not satisfied with Cox’s derivation of the product rule. Jaynes writes
I might say that I am not entirely satisfied with the argument that we went through to get this; not because I think its wrong, but because I think it is too long. The final result we get is so simple that there must be a simpler way of deriving it; but I haven’t found it. (Jaynes, 1956, p. 35)
A year after his lectures on the topic, Jaynes published his paper revealing the ideas behind both communication theory and statistical mechanics, which results in the principle of maximum entropy (Jaynes, 1956, pp. 110—151), Jaynes (1957). Since the entropy quantifies the degree of uncertainty in a probability distribution, assigning a probability that maximizes the entropy subject to a set of constraints amounts to using the information provided by the known constraints, while being careful not to inadvertently assume too much. Jaynes’ maximum entropy principle provided the justification that Gibbs so carefully avoided in his works on statistical mechanics to ensure acceptance.
With the benefit of the insights provided by these three individuals, we have come to view probability, entropy and information in a new light. Probability and entropy describe states of knowledge about systems—not the systems themselves. What is more, we now realize that information acts a constraint on our beliefs. Free from the previous confusion surrounding probability, entropy and information, and the misconceptions that ensue, we can take these new ideas and re-examine the laws of physics. Several of us from this community have been doing just that. In addition to a more clear understanding of statistical mechanics we have seen the principle of maximum entropy used to derive properties of systems ranging from the physics of foam Rivier et al. (1999) to the physics of planetary atmospheres Lorenz et al. (2001). More profound perhaps is Ariel Caticha’s investigation of entropic dynamics Caticha (2002) where he is working to utilize maximum entropy to derive the dynamical behavior of systems ranging from Newtonian mechanics Caticha and Cafaro (2007) to quantum mechanics Caticha (2010).
Inspired by Cox, I have been working to understand how to derive calculi from algebras in general by selecting consistent quantification schemes for partially-ordered sets and lattices. At one level, this more fundamental understanding has resulted in a much simpler derivation of the product rule that might have been more to Jaynes’ liking. However, at a deeper level, we now understand how constraints imposed by ordering relations can result in the derivation of physical laws. This recently has been demonstrated with a novel derivation of the complex arithmetic in Feynman’s path integral approach to quantum mechanics Goyal et al. (2010a, b) as well as a derivation of special relativity from a partial order on a set of events Knuth and Bahreyni (2010). Each of these examples is related to information in a different way. In some examples the connection to information is direct as we consider a partial order on states of knowledge themselves. However, we have also employed these ideas by considering the partial order that arises from the way that events can be informed about one another or the partial order that arises from composing sequences of measurements aimed at gaining information.
In this tutorial, which is still very much a work in progress, I will introduce this new way of thinking by explaining how one can derive physical laws by quantifying partially-ordered sets. The implication is that physical law does not reflect the order in the universe, instead it is derived from the order imposed by our description of the universe. This occurs both through the acts of quantification of information (which I will discuss here) and processing of information, which is related to the use of entropy and probability. We have now demonstrated these ideas by deriving a surprising amount of old physics. New physics now awaits as we enter this new frontier of Information Physics.
Order Theory, Posets, Lattices and Algebras
While group theory has become an essential tool for theoretical physics, order theory remains entirely overlooked. At the most fundamental level, group theory is concerned with equivalence relations among partitioned sets, whereas order theory is concerned with ordering relations among ordered sets. In this sense these two theories stand side-by-side and both can place extremely strong constraints on physical theories. I will use these theories in concert with one another. First, I will rely on ordering relations to obtain algebraic operations that have specific symmetry properties. I will then use these symmetries to place strong constraints on any quantified description. The resulting constraints correspond to the physical laws.
I begin by introducing the concept of a binary ordering relation and a partially-ordered set. Two elements of a set are ordered by comparing them according to a binary ordering relation, generically denoted and read ‘is included by’. The simplest example is the ordering of the integers according to the usual meaning of the symbol ‘is less than or equal to’. This results in a totally ordered structure called a chain (Fig. 1A). To illustrate the hierarchy, we simply draw element above element if and connect them with a line if there does not exist an element in the set such that .
In some cases, elements of the set are incomparable to one another, as in the popular example of comparing apples and oranges. A set of incomparable elements is called antichain. I illustrate this in Figure 1B with a set of card suits where the elements are placed side-by-side to indicate that no element includes any other.
More interesting examples involve both inclusion and incomparability, which is why we refer to these structures in general as partially ordered sets, or posets for short. Figure 1C illustrates the poset that results from partitioning three objects. One could consider all three objects together , or each separately . These objects can also be partitioned in three ways: , or . Any two partitions from this set can be compared according to a relation that decides whether one partition includes another. For example, the partition includes the partition since it can be obtained by simply sub-dividing into three separate cells. However, the partitions and are incomparable since, for example, there is no way to sub-divide the partition to obtain the partition .
Given a set of elements in a poset, their upper bound is the set of elements that contain each of the elements of the set. For example, the upper bound of the partition in Fig. 1C is the set . Given a pair of elements and , the least element of their upper bound is called the join, which is denoted . The lower bound of a set of elements is defined dually by considering all the elements included by each of the elements of the set. Given a pair of elements and , the greatest element of their lower bound is called the meet, which is denoted . A lattice is a partially ordered set where each pair of elements has a unique meet and a unique join (Fig. 2). Graphically, the join can be found by starting at both elements and following the lines upward until they first intersect. The meet is found similarly by moving downward. There often exist elements that are not formed from the join of any pair of elements. These elements are called join-irreducible elements. Meet-irreducible elements are defined similarly. For example, the partitions , or cannot be formed by joining any other pair of partitions and therefore are join-irreducible. In this case, these elements are also meet-irreducible.
We can choose to view the join and meet as algebraic operations that take any two lattice elements to a unique third lattice element. From this perspective, the lattice is an algebra. This results in both a structural and operational perspective which are related by a set of equations called consistency relations
In short, a lattice is an algebra. Where an algebra considers a set of elements along with a set of operations that takes one or more elements to another element, the lattice considers a set of elements along with a binary ordering relation that sets up a hierarchy among the elements. The algebraic perspective is operational, whereas the lattice perspective is structural. Both the operational and structural relationships among elements are useful.
Given a specific lattice, we find that the consistency relations result in a specific algebraic identity. For example, the integers ordered by the usual ‘less than or equal to’ leads to
whereas the positive integers ordered by ‘divides’ leads to
Sets ordered by the usual ‘is a subset of’ leads to
Such examples highlight the generality of the order-theoretic approach.
There are many ways to quantify a poset. Here I will describe some of the ways that we have been exploring Knuth (2004, 2009); Knuth and Bahreyni (2010): valuations, bi-valuations, and projections. However, I will leave a more general discussion of the pair formalism of quantum mechanics and the origin of the complex sum and product rules as described in Goyal et al. (2010b) to a future work. It is important to keep in mind that the quantification techniques I will cover does not comprise an exhaustive list, as we are only beginning to explore the possibilities.
We begin by considering the quantification of lattices. We will see that this is equivalent to extending an algebra to a calculus by defining functions that take lattice elements to real numbers. Such functions enable one to quantify the relationships between the lattice elements. This leads to probability theory on the lattice of logical statements and information theory on the partition sublattice of questions Knuth (2009).
Valuations and Bi-valuations
A valuation is a function that takes a single lattice element to a real number in a way that respects the partial order, so that iff . This means that the lattice structure imposes constraints on the valuation assignments, which can be expressed as a set of constraint equations.
The valuation assigned to element can be defined with respect to a second lattice element called the context. The result is a function called a bi-valuation , which takes two lattice elements and to a real number. Here a solidus is used as an argument separator so that one reads as the degree to which includes .
In the following sections, I consider three operations than can be performed on lattices, each of which obeys associativity. The symmetries exhibited by associativity impose strong constraints on quantification, namely additivity. This, in turn, constrains valuation and bi-valuation assignments. The first two operations, the lattice join and the lattice product, are associated with the lattice structure and thus impose the same constraints on both the valuation and bi-valuation assignments; whereas the last symmetry, associativity of context, is specific to bi-valuations.
The Lattice Join
I now show that associativity of the lattice join forces valuations to be additive. I begin by considering a very special case depicted in Fig. 3 (left) of two elements and with join and a null meet (not shown). The value assigned to the join , written , must be a function of the values assigned to both and , and , since if there did not exist any functional relationship, then the valuation could not possibly reflect the underlying lattice structure. This functional relationship can be written in terms of an unknown binary operator
Now consider another case where we have three elements , , and , such that their meets are again disjoint. The least upper bound of these three elements can be written in at least two different ways: and . Consequently, the value assigned to this join can also be written in two different ways
This functional equation for the operator has a general solution given by Aczel Aczél (1966)
where is an arbitrary invertible function. We take advantage of this freedom to choose a valuation that simplifies this constraint
By letting , equation (8) implies that .
We now seek a solution for the general case. Consider the lattice in Figure 3 (center) and note that the elements and have a null meet, as do the elements and . Applying (8) to these two cases, we get
Simple substitution results in the general constraint equation known as the sum rule
In general for bi-valuations we have
for any context . Note that the sum rule is not focused solely on joins since it is symmetric with respect to interchange of joins and meets. That is, this result simultaneously respects associativity of the lattice join and the lattice meet.
We have derived that associativity constrains us to additive valuations—there is no other option. The cartoon at the right of Fig. 3 illustrates the symmetry of the sum rule. The sum of the valuations of the elements at the top and bottom of the diamond equals the sum of the valuations of the elements on the right and left sides
The Lattice Product
One can combine two lattices via the lattice product where elements themselves are combined in as in a Cartesian product. That is, the product of a lattice with a lattice will result in a lattice with elements of the form , where and . The lattice product is associative, so that for three lattices , , and , we have
with elements of the form .
The valuation assigned to an element clearly must be a function of the valuations assigned to and in their respective original lattices. Again, associativity will require that they are combined in an additive fashion
where is an arbitrary function.
In some cases, such as in probability theory, we expect associativity of the lattice product to hold simultaneously with associativity of the lattice join within a given lattice. Given the linearity of the constraint imposed by associativity of lattice join (13), the only remaining freedom is that of rescaling. This means that any further constraints must have a multiplicative form. The result is that the valuation assigned to an element formed by a lattice product is given by
which is a product rule applicable to combining lattices.
The Chain Rule
We now focus on bi-valuations and explore changes in context. Changes in context are again associative, which again results in an additive constraint.
We begin with the special case of a chain and consider four ordered elements . The relationship can be divided into two relations, and . By considering to be the context, this sub-division implies that the context can be considered in parts. Thus the bi-valuation we assign to with respect to context , , must be related to both the bi-valuation assigned to with respect to context , , and the bi-valuation assigned to with respect to context , . That is, there exists a binary operator that relates the bi-valuations assigned to the two steps to the bi-valuation assigned to the one step
Extending this to three steps (Fig. 4A) and considering the bi-valuation relating and , via intermediate contexts and , we obtain another associative relationship
Using the associativity theorem again results in a constraint equation for non-negative bi-valuations involving changes in context Skilling (67–79). We call this the chain rule
This result can be extended by considering the following lemma. The sum rule applied to the diamond in Fig. 4B defined by , , , and with context gives
Since and , we have , reducing the sum rule to
This relationship, illustrated by the equivalence of the arrows in Fig. 4B, will used several times in the derivation that follows.
We now consider the more general lattice in Fig. 4C and focus on the chain along the lower left side. Using the chain rule, we decompose the bi-valuation with context into two parts by introducing the intermediate context
We apply the lemma to the diamond defined by , , , (Fig. 4C, center) to obtain
Similarly, the diamond defined by , , , and (Fig. 4C, right) results in
The Valuation Calculus
We have derived that associativity of the lattice join results in the sum rule
which is a central axiom of measure theory. Associativity of the lattice product imposes an additional constraint, which results in a product rule
Extending the concept of valuation to that of a context-dependent bi-valuation, we obtain a sum rule
a product rule for combining spaces
and a product rule for context change
The valuation calculus differs from traditional measure theory in two important ways. First, additivity is not postulated, but rather is derived from associativity. Second, the valuation calculus generalizes measure theory by introducing the concept of context, which is quantified using bi-valuations and manipulated using the product rule. These rules are constraint equations ensuring that the assigned valuations respect the order-theoretic properties of the lattice.
The previous sections describe the consistent quantification of lattices, which is made possible by the fact that lattices possess extra structure that allows one to define a unique join and meet of each pair of elements thus making it an algebra. It is precisely this extra structure that constrains any proposed quantification scheme via the sum and product rules. However, such constraints do not apply to posets in general since they lack this extra structure possessed by lattices.
Consistent quantification of a poset can proceed by artificially imposing additional lattice-like structure. One way to do this is to select a distinguished a set of elements in the poset that form a lattice, and attempt to relate the remaining elements in the poset to the elements of this distinguished set. We have recently demonstrated this quantification technique by selecting one or more chains as the distinguished set (or sets) and projecting poset elements onto the chains Knuth and Bahreyni (2010). In general, it may not be possible to quantify all poset elements in this way, but here we show that one can certainly quantify a subset of the elements. Surprisingly, this proposed quantification scheme results in the Minkowski metric and Lorentz transformations Knuth and Bahreyni (2010).
First we consider quantification using a single chain. We select a chain to be used for quantification and label its elements with . In a finite poset, such a chain is described by . In an infinite poset where the chain is countably infinite the label can be any integer and the chain is described by . If the chain is uncountably infinite, a real number index can be used.
An element can be projected onto a chain if there exists an element such that . If this is the case, then the projection of onto the chain is given by the least element on the chain such that . If one considers the sub-poset consisting only of the element and the elements comprising the chain , then in this sub-poset covers , (Fig. 5A). If the projection exists, we say that is quantifiable with respect to , and assign to the element the numeric label assigned to the element . Note that, in general, not all elements of a poset are quantifiable with respect to a given chain. Any chain potentially divides the poset into two classes: elements quantifiable with respect to the chain and elements not quantifiable with respect to the chain (Fig. 5B). Thus, one can only be assured to quantify some subset of the poset.
One can project to N different chains and use the corresponding numeric labels to coordinatize the poset elements that are quantifiable with respect to each of the selected chains with numbers taken as a Cartesian product (Fig. 5C).
The interval between two poset elements can be quantified using two chains. These chains must be synchronized so that successive events in one chain project to successive events in the other chain (Fig. 6A). Figure 6B illustrates the quantification of an interval given by where and . This pair-wise quantification can be decomposed into the sum of a symmetric and an antisymmetric pair Knuth and Bahreyni (2010) given by
The two integer labels can be used to obtain a single scalar. This is done by taking the lattice product of the two chains, which, as we saw earlier, results in a valuation found by taking the product of the two original valuations, so that
we can rewrite the pair as
and the scalar as
This is the Minkowski metric, familiar from special relativity, and here it arises from a simple method for quantifying a poset Knuth and Bahreyni (2010). This is not a coincidence. Our recent paper demonstrates that the scalar interval is invariant when computed with respect to any synchronized pair of chains. In addition, the parameters and are shown to transform according to the Lorentz transformations of time and space.
It should be noted that such a consistent decomposition of an interval is not always possible given more than two synchronized chains Knuth and Bahreyni (2010), and that this is related to the multi-dimensionality of space.
It is not possible in this tutorial to cover the applications derived using this methodology in requisite detail. For this reason, I will simply outline the basic applications and point to appropriate references. Since these quantification techniques are applicable to a wide array of posets and lattices, we can expect that they will be relevant to numerous applications. At this point, we have five examples where we have derived a theory from first principles based on quantifying posets and lattices.
The most general of these applications, measure theory, has been discussed here as the derivation of the valuation calculus and the related bi-valuations. The valuation calculus both encompasses and extends traditional measure theory. Additivity of measures, which is an axiom of measure theory is derived here as a consequence of associativity. Furthermore, the valuation calculus generalizes measure theory by introducing the concept of context. A valuation with respect to a context is quantified using bi-valuations and manipulated using the product rule. Earlier works discussing these results can be found here Knuth (2004, 2009).
The second example, which was the original inspiration for this work is the derivation of probability theory Knuth (2004, 2007, 2008, 2009). By founding probability theory as a quantification of implication among logical statements, we obtain a theory that encompasses and generalizes both the Cox and Kolmogorov formulations. By introducing probability as a bi-valuation defined on a lattice of statements we can quantify the degree to which one statement implies another. Rather than deriving probability theory from a set of desiderata derived from Cox’s particular notion of plausibility, the properties of the lattice of statements form the basis of the theory. Furthermore, the meaning of the derived measure is inherited from the ordering relation, which in this case is implication. The fact that these lattices are derived from sets means that this work encompasses Kolmogorov’s formulation of probability theory as a measure on sets. However, mathematically this theory improves on Kolmogorov’s foundation by not only deriving, rather than assuming, additivity of the measure, but also by introducing the concept of context and endowing the measure with meaning.
The third example involves the derivation of information theory as a valuation on the partition subspace of questions. The space of questions is generated from the space of statements by virtue of Birkhoff’s Representation Theorem Knuth (2003). The result is the free distributive lattice of questions, which by virtue of its being a lattice imposes a sum rule and a product rule. By postulating that the relevance of a question is a function of the probabilities that answer it, we couple the probability measure on the statement space with the relevance measure on the question space. Due to a conflict of constraints, to be discussed in more detail in a future work, one can show that an objective non-trivial measure can be defined only on the subspace of questions that are isomorphic to partitions. The result is that the most basic relevance measures are quantified by the Shannon entropy of the set of assertions that potentially answer the question. The sum rule, when relating partitions, results in a relationship between mutual information and joint entropy
The result is not only a novel derivation of information theory, but a natural extension of the theory to include the relevance of a question quantified with respect to a given context Knuth (2003, 2006, 2008).
Deriving mathematical theories is one thing, but deriving physical theories is an another thing altogether. The first such example is a derivation of the complex sum and product rules of the Feynman formulation of quantum mechanics Goyal et al. (2010a, b). This was achieved by considering a pair-wise valuation on the space of sequences of measurements. The logic of the process of measuring served to generate the algebra, which implicitly defines a poset of measurement sequences. By combining measurements in two ways: parallel and serial, which correspond to the lattice join and the lattice product, and mapping the pair-wise valuation to a scalar-valued probability, we obtain the complex sum and product rule along with the Born rule, which maps our pair-wise valuation to a scalar-valued probability Goyal et al. (2010a, b).
The most recent application has been a derivation of special relativity as a quantification of a poset of causally related events Knuth and Bahreyni (2010). As discussed above, this is achieved by distinguishing two chains of elements (events) as observers and projecting events onto the observer chains. The result is that intervals are quantified by a pair of numbers and that this pair maps to a unique scalar, which gives rise to the Minkowski metric. What is strange is that in this picture space and time emerge as nothing more than a convenient decomposition, which along with other results, strongly suggests that they are not fundamental.
In his derivation of probability theory Cox provided the first example of generalizing an algebra to a calculus Cox (1946). That such an activity is generally possible or even useful is not obvious until one begins to notice the great many similarities between a variety of mathematical theories and physical laws, such as the various incarnations of the sum rule or the fact that quantum mechanics looks like a complex version of probability theory. As Jaynes recognized, it is not a matter of simple analogy, but rather something far more subtle. The theories are similar because the ideas that lead to the theories are similar. These ideas are based on the quantification of order.
In this tutorial, I have shown how a variety of rules involving quantification arise as constraint equations to ensure that any quantification does not violate the underlying order. What is more striking is that this entire procedure is based on the quantification of order underlying our descriptions of physical reality—not necessarily physical reality itself. The consequence is that the physical laws we obtain are constraints on quantification imposed by our descriptions. This is where we arrive at Information Physics.
At the heart of this new methodology lies the valuation calculus which is applicable to any lattice. Associativity of the lattice join (or meet) gives rise to the sum rule. Associativity of the lattice product results in a product rule, which dictates how valuations are to be combined when taking lattice products. Associativity of changes of context result in a product rule for bi-valuations that dictates how valuations should be manipulated when changing context. The techniques based on projections are based on distinguishing a sub-lattice that can be used to employ valuations to quantify a poset in general.
Most exciting is the range of theories that have been successfully derived using this foundation: measure theory, probability theory, information theory, quantum mechanics, and special relativity. These results provide strong support for the claim that Information Physics, which relies on information about our descriptions of reality to derive physical laws, is a potentially useful general approach. With these positive examples as guideposts, we now aim to use these techniques to quantify new problems and derive new physical laws.
I would like to thank Janos Aczél, Newshaw Bahreyni, Ariel Caticha, Julian Center, Seth Chaiken, Keith Earle, Adom Giffin, Philip Goyal, Steve Gull, Jeffrey Jewell, Vassilis Kaburlasos, Nabin Malakar, Carlos Rodríguez, and John Skilling for inspiring discussions, invaluable remarks and comments, and much encouragement. This work was supported in part by the College of Arts and Sciences and the College of Computing and Information of the University at Albany (SUNY).
- Shannon and Weaver (1949) C. F. Shannon, and W. Weaver, The Mathematical Theory of Communication, Univ. of Illinois Press, Chicago, 1949.
- Cox (1946) R. T. Cox, Am. J. Physics 14, 1–13 (1946).
- Jaynes (1956) E. T. Jaynes, Probability Theory in Science and Engineering, No. 4 in Colloquium Lectures in Pure and Applied Science, Socony-Mobil Oil Co., 1956.
- Jaynes (1957) E. T. Jaynes, Physical Review 106, 620–630 (1957).
- Rivier et al. (1999) N. Rivier, B. Dubertret, T. Aste, and H. Ohlenbusch, “Universality, prior information and maximum entropy in foams.,” in Maximum Entropy and Bayesian Methods, Munich 1998, edited by W. von der Linden, V. Dose, R. Fischer, and R. Preuss, Kluwer, Dordrecht, 1999, pp. 57–64.
- Lorenz et al. (2001) R. D. Lorenz, J. I. Lunine, P. G. Withers, and C. P. McKay, Geophys. Res. Lett. 28, 415–418 (2001).
- Caticha (2002) A. Caticha, “Entropic dynamics.,” in Bayesian Inference and Maximum Entropy Methods in Science and Engineering, Baltimore MD, USA, 2002, edited by R. L. Fry, AIP Conference Proceedings 617, American Institute of Physics, New York, 2002, p. 302.
- Caticha and Cafaro (2007) A. Caticha, and C. Cafaro, “From information geometry to Newtonian dynamics.,” in Bayesian Inference and Maximum Entropy Methods in Science and Engineering, Saratoga Springs, NY, USA, 2007, edited by K. H. Knuth, A. Caticha, J. L. Center, A. Giffin, and C. C. Rodríguez, AIP Conference Proceedings 954, American Institute of Physics, New York, 2007, pp. 165–175.
- Caticha (2010) A. Caticha, Entropic dynamics, time and quantum theory. (2010), arXiv:1005.2357v1[quant-ph].
- Goyal et al. (2010a) P. Goyal, K. H. Knuth, and J. Skilling, “The origin of complex quantum amplitudes,” in Bayesian Inference and Maximum Entropy Methods in Science and Engineering, Oxford MS, USA, 2009, edited by P. Goggans, AIP Conference Proceedings 1193, American Institute of Physics, New York, 2010a, pp. 89–96.
- Goyal et al. (2010b) P. Goyal, K. H. Knuth, and J. Skilling, Phys. Rev. A 81, 022109 (2010b), arXiv:0907.0909 [quant-ph].
- Knuth and Bahreyni (2010) K. H. Knuth, and N. Bahreyni, A derivation of special relativity from causal sets. (2010), arXiv:1005.4172v1[math-ph].
- Knuth (2004) K. H. Knuth, “Deriving laws from ordering relations.,” in Bayesian Inference and Maximum Entropy Methods in Science and Engineering, Jackson Hole WY, USA, August 2003, edited by G. J. Erickson, and Y. Zhai, AIP Conference Proceedings 707, American Institute of Physics, New York, 2004, pp. 204–235, arXiv:physics/0403031v1 [physics.data-an].
- Knuth (2009) K. H. Knuth, “Measuring on lattices,” in Bayesian Inference and Maximum Entropy Methods in Science and Engineering, Oxford, MS, USA, 2009, edited by P. Goggans, and C.-Y. Chan, AIP Conference Proceedings 1193, American Institute of Physics, New York, 2009, pp. 132–144, arXiv:0909.3684v1 [math.GM].
- Aczél (1966) J. Aczél, Lectures on Functional Equations and Their Applications, Academic Press, New York, 1966.
- Skilling (67–79) J. Skilling, “The canvas of rationality,” in Bayesian Inference and Maximum Entropy Methods in Science and Engineering, São Paulo, Brazil, 2008, edited by M. S. Lauretto, C. A. B. Pereira, and S. J. M., AIP Conference Proceedings, American Institute of Physics, New York, 67–79.
- Knuth (2007) K. H. Knuth, “Lattice theory, measures, and probability,” in Bayesian Inference and Maximum Entropy Methods in Science and Engineering, Saratoga Springs NY, USA, July 2007, edited by K. H. Knuth, A. Caticha, J. L. Center, A. Giffin, and C. C. Rodríguez, AIP Conference Proceedings 954, American Institute of Physics, New York, 2007, pp. 23–36.
- Knuth (2008) K. H. Knuth, “The origin of probability and entropy,” in Bayesian Inference and Maximum Entropy Methods in Science and Engineering, São Paulo, Brazil, 2008, edited by M. S. Lauretto, C. A. B. Pereira, and S. J. M., AIP Conference Proceedings 1073, American Institute of Physics, New York, 2008, pp. 35–48.
- Knuth (2003) K. H. Knuth, “What is a question?,” in Bayesian Inference and Maximum Entropy Methods in Science and Engineering, Moscow ID, USA, 2002, edited by C. Williams, AIP Conference Proceedings 659, American Institute of Physics, New York, 2003, pp. 227–242.
- Knuth (2006) K. H. Knuth, “Valuations on lattices and their application to information theory,” in Proceedings of the 2006 IEEE World Congress on Computational Intelligence (IEEE WCCI 2006), Vancouver, BC, Canada, July 2006., 2006.