Blending Entropy: A Term for Addressing Information Density in Mediated Reality
The virtuality continuum describes the degrees of positive virtuality under the umbrella term mixed reality. Besides adding virtual information within a mixed environment, diminished reality aims at reducing real world information. Mann defined the term mediated reality (MR), which also considered diminished reality, but without the possibility to describe different degrees of fusion between a mixed and a diminished reality. That is why this work defines the new term blending entropy that captures the relations between a mixed and a diminished reality. The blending entropy is based on the information density of the mediated reality and the actual area the user has to comprehend, which is named perceptual frustum. We describe the blending entropy’s two-dimensional dependencies and detail important points in the blending entropy’s space.
1 Definition of the Blending Entropy
In information theory, entropy is the average information within a message. In a MR, entropy can be viewed as the total of any information perceived by the user. The blending entropy is characterized by the entropy density and the perceptual frustum as illustrated in Figure 1.
1.1 Entropy Density
The information density or entropy density of an environment depends on the number and kind of physical objects and other visible information such as text, speech and haptic. The blending of the realities as defined in the virtuality continuum  fuses these entropy densities. Mann  called it MR if this mixture can lead to a diminished reality. There is no clear concept yet that defines the stages between diminished and mixed reality. That is why the new term blending entropy is introduced.
The lower limit of entropy density is a reality with an entropy density of zero. The user that interacts in such a void reality would receive no information or knowledge. This void reality could be an environment that just consists of emptiness. Decreasing the entropy density in relation to the equilibrated reality leads to the diminished reality (DR), which aims at removing real-world objects unnoticeably, i.e, reducing the entropy density of the real world.
The mixed reality comprises a larger entropy density than the equilibrated reality, since its goal is usually to add valuable information to the real- or virtual environment. The upper limit of the entropy density raises the uncertainty to a maximum, i.e., the environment only comprises independent and dense information. Such an environment can be accomplished by combining the real with (multiple) virtual realities into a multiversed reality. This definition of a multiversed reality motivates the importance of the term blending entropy. A multiversed reality would overwhelm the user’s mind, i.e., the user would not be able to interact and reasonably process the information regardless of the size of the perceptual frustum. This is also called information overload and describes the overmuch processing of information . This issue can be addressed via the blending entropy. For instance, minimizing the blending entropy circumvents an information overload, which can be achieved by reducing the entropy density or the perceptual frustum.
1.2 Perceptual Frustum
The perceptual frustum is similar to the field of regard, but the perceptual frustum is dynamic. It depends on the situation and the spatial distribution of the necessary information. If the user has to be aware of information distributed in a vast environment for a certain task or purpose, the perceptual frustum is large. When compressing the required information into a small space, the perceptual frustum is tiny.
The largest perceptual frustum is dubbed “god-like”. It expresses a (theoretical) perceptual frustum that requires the user to concurrently access the total information of the world. In the work of Stafford et al. , a “god-like” user tagged navigation clues on a tabletop for a user that received them in the real world via augmented reality (AR). This work is close to but still different from a “god-like” perceptual frustum, since the “god-like” user just focused on a single person and it was not necessary to conceive the whole reality.
A perceptual frustum smaller than the “god-like” one is an omnidirectional perceptual frustum. The difference to “god-like” is that only the immediate environment of the virtual avatar or the person has to be comprehended and it is not necessary to consider multiple places of the world. Reducing the size of the perceptual frustum even further results in a tunnel-like perceptual frustum, which presents the necessary information right in front of the user. The ParaFrustum proposed by Sukan et al.  is a practicable example of a tunnel-like perceptual frustum. The ParaFrustum did not add further expert knowledge but it guided the user’s view to the important parts of the current manual step.
The smallest (theoretical) perceptual frustum is a point-like spot. A scenario of a user that has to press a lone button depending on the augmentation would comply to a point-like perceptual frustum. A study by Marner et al.  is close to such a perceptual frustum. Here, the users pressed buttons sequentially according to spatial AR cues.
1.3 Blending Entropy
Finally, the blending entropy is the integration of the entropy density along the perceptual frustum. It holds the information of the joint realities that have to be perceived by the user. An increased or decreased blending entropy states that more or less information has to be processed by the user, respectively. The dotted line in Figure 1 illustrates the direction of raising blending entropy. In consequence, the largest blending entropy exists for a multiversed reality with a “god-like” perceptual frustum.
1.4 Equilibrated Reality
It is important to note that the origin of the blending entropy is defined by the entropies of the extrema of the virtuality continuum, i.e., the real or virtual environments. A blending entropy of zero is called equilibrated reality. Here, the entropy density and the perceptual frustum of either the purely real or the purely virtual environments is identical to the density and the frustum of the equilibrated reality. Therefore, the equilibrated reality shares the same perceptual frustum and information density than the unmixed environments but it still consists of a modulated environment. Thus, the equilibrated reality can simply be described as a modulated reality that shares the identical information complexity as the purely virtual or real environments.
One of the goals of this work is to link mixed and diminished reality within a single representation. Mixed reality typically describes techniques and applications that advance the reality. The extension of the realities occurs for both extrema of the virtuality continuum. On the one hand, physical objects enhance the virtual reality, which is known as augmented virtuality . On the other hand, virtual objects are integrated into the real world, i.e., AR. In every case, additional knowledge or material complements the environment, which introduces a higher entropy density according to the new terms of this work.
Then reducing the entropy density connects the DR with the MR. An example for a DR application is the work of Korkalo et al. . They concealed AR markers via DR from the real environment, i.e., the goal was to lower the entropy density. Diminishing contents in the environment, while keeping the same perceptual frustum realizes a lower blending entropy. A term that describes all degrees of blending entropy already exists and is named mediated reality . The mediated reality consists of positive (MR) and negative (DR) entropy densities. The shortcoming of the original definition of the mediated reality is that there are no levels between mixed and diminished reality that describes the underlying relation. The definition of the entropy density allows such a classification.
The size of the blending entropy depends on the user as well as on the task. For example, the real environment of an industrial worker may already comprise a lot of information such as a console with different gauges, buttons and switches, i.e., the entropy density of the non-modulated environment is rather large. The worker, however, is accustomed to this environment and the specific tasks therein. This knowledge of the worker about the scene may produce a tunnel view, since the worker exactly knows where to look at, which minimizes complexity and creates an acceptable entropy for the worker. Consequently, the goal of a mediated reality can be the increase of the blending entropy and certain expert knowledge can be presented to the worker.
On the other hand, an untrained user that has not worked in such an environment before may be near an information overload. Because of the definition of the blending entropy, also a variety of options are available for this user. The mediated reality application could aim at reducing the blending entropy or, at least, produce a balanced blending entropy, i.e., stay at the equilibrated state. This can be achieved via an AR-guided reduction of the perceptual frustum , which narrows the user’s perception (tunnel vision). Moreover, the entropy density can be minimized via DR, i.e., the application identifies the actual relevant parts and diminishes the unimportant parts. Diminishing parts that are close to the border of the perceptual frustum may also decrease the perceptual frustum, which would generate an even smaller blending entropy.
Most MR applications aim at modifying the blending entropy. For instance, some MR applications guide the user (lowering perceptual frustum) or present special information (higher entropy density), which can now be expressed via the blending entropy. Furthermore, the introduction of entropy density allowed to define a relationship between diminished and mixed reality.
- Korkalo et al.  O. Korkalo, M. Aittala, and S. Siltanen. Light-weight marker hiding for augmented reality. In Proceedings of the International Symposium on Mixed and Augmented Reality, pages 247–248. IEEE, 2010.
- Mann  S. Mann. Mediated reality. Techreport 260, M.I.T. Media Lab Perceptual Computing Section, 1994.
- Marner et al.  M. R. Marner, A. Irlitti, and B. H. Thomas. Improving procedural task performance with augmented reality annotations. In Proceedings of the International Symposium on Mixed and Augmented Reality, pages 39–48. IEEE, 2013.
- Milgram et al.  P. Milgram, H. Takemura, A. Utsumi, and F. Kishino. Augmented reality: A class of displays on the reality-virtuality continuum. In Proceedings of the Conference on Photonics for Industrial Applications, pages 282–292. SPIE, 1995.
- Speier et al.  C. Speier, J. S. Valacich, and I. Vessey. The influence of task interruption on individual decision making: An information overload perspective. Wiley Online Library Decision Sciences, 30(2):337–360, 1999.
- Stafford et al.  A. Stafford, W. Piekarski, and B. H. Thomas. Implementation of god-like interaction techniques for supporting collaboration between outdoor ar and indoor tabletop users. In Proceedings of the International Symposium on Mixed and Augmented Reality, pages 165–172. IEEE, 2006. doi: 10.1109/ISMAR.2006.297809.
- Sukan et al.  M. Sukan, C. Elvezio, O. Oda, S. Feiner, and B. Tversky. ParaFrustum: Visualization techniques for guiding a user to a constrained set of viewing positions and orientations. In Proceedings of the Symposium on User Interface Software and Technology, pages 331–340. ACM, 2014. ISBN 978-1-4503-3069-5.