Air Mounted Eyepiece:
Design Methods for Aerial Optical Functions of
Near-Eye and See-Through Display using
Transmissive Mirror Device
We propose a novel method to implement an optical see-through head mounted display which renders real aerial images with a wide viewing angle, called an Air Mounted Eyepiece (AME). To achieve the AMD design, we employ an off-the-shelf head mounted display and Transmissive Mirror Device (TMD) which is usually used in aerial real imaging systems. In the proposed method, we replicate the function of the head mounted display (HMD) itself, which is used in the air by using the TMD and presenting a real image of eyepiece in front of the eye. Moreover, it can realize a wide viewing angle 3D display by placing a virtual lens in front of the eye without wearing an HMD. In addition to enhancing the experience of mixed reality and augmented reality, our proposed method can be used as a 3D imaging method for use in other applications such as in automobiles and desktop work. We aim to contribute to the field of human-computer interaction and the research on eyepiece interfaces by discussing the advantages and the limitations of this near-eye optical system.
In contemporary times, the realization of immersive experiences in Mixed Reality and Augmented Reality are desirable. An optical see-through type information presentation method with a wide viewing angle is required for this purpose. In recent years, several research studies have proposed employing optical elements such as transmitted liquid crystal, half mirrors, holographic optical element and waveguide to achieve these requirements. However, there are several problems when using these techniques in an optical see-through system (Figure 2). When a single lens VR goggle is combined with a transmissive liquid crystal or half mirror, the scene transmitted through the lens and the scenery are distorted owing to the fact that there is a single lens which blocks the lights from the scenery. In addition, even though the lens is presented in front of the eye using a half mirror or holographic optical element or waveguide, a wide viewing angle cannot be achieved because the distance from the eye to the lens becomes longer. If a convex lens used in optical party, it restricts magnify rate.
In this paper, we introduce a novel method to present “virtual lenses” in front of the eyes using a Transmissive Mirror Device (TMD) with a Micro Dihedral Corner Reflector Array (MDCRA). A TMD with an MDCRA has been proposed   and it has been used for various aerial interactions . We propose a method to create the functionality of a head-mounted display in the air by using a TMD to present a real image of eyepiece through single lens VR goggles. We call this method an Air Mounted Eyepiece (AME). By looking into the virtual lens in the air, the user obtains the same experience as the head mounted display without wearing goggles. It is also considered useful to provide a see-through and non-wearable immersive experience with a wide viewing angle from the safety standpoint for vehicle mounting and medical applications. This paper introduces the design methods of AME instead of convex mirror or waveguide or half mirror for building HMD and near-eye displays.
2 Design Method
Aerial Image Formation of the single lens
We introduce a method to generate the light field of a single lens as an aerial imaging formulation just close to or inside the eyeball. By using this method, a transmission type aerial image with a wider viewing angle than an ordinary transparent HMD can be obtained as the lens position can be brought closer to the eyeball. When using an HMD as the light source, the HMD is arranged at the position where the single lens of the HMD is in focus with respect to the TMD eye position, such that they build an optical system that includes the TMD. This allows the aerial imaging of the single lens to be placed in front of the eyeball. By looking into the aerial image presented in front of the eyeball, the same effect as with an HMD can be obtained. Digital transformation is necessary because the image output through the aerial imaging of the single lens is inverted vertically and horizontally.
In Figure.2 we note the parameters in design process of the AME and conventional methods frequently used in see-through HMD design. Half-mirror based HMD is raised in Figure. 3(a). A view angle of this setup is
where is size of screen, a is the distance between half-mirror and eyeball, d is distance between half-mirror and screen. Additional lens are sometimes inserted between screen and half-mirror however it doesnât change the maximum view angle . To solve this problem, convex mirror or prisms are used instead of half-mirror. We show in Figure. 3(b). In this case the view angle is
where a is the magnification ratio of convex mirror. Note that if a is higher it can not be used as see-through type HMD, because it distorts the see-through view. Then we introduce our methods by using TMD, in Figure. 3(c), a view angle is
where is the size of lens, is the size of TMD, is distance between TMD and screen, is distance between TMD and eyeball. This method can easily be applied to self-designing for HMD in research-prototype, because the is shown on the spec sheet of HMD in many cases. So that prototypers just put HMD close to the HMD on the screen side, it work as see through HMD.
Optical ray tracing in a computer simulation was performed using Zemax OpticStudio (Figure 4). The TMD structure with a micro dihedral corner reflector array was reproduced with CAD. The blue line represents an optical ray of light from the scenery. The red, green, and yellow lines represent the optical ray emitted from the image source. The half mirror optical see-though display provides a narrow viewing angle. In our proposed method, the Air Mounted Eyepiece creates the function of the head mounted display in the air, thus the virtual lens is placed while in the air. The relationship between the viewing angle and the change in the mirror pitch is shown in the following Figure 5. The higher resolution images are obtained when the mirror pitch is small.
See  for more details on the behavior of TMD.
We conducted an experiment to evaluate AME (Figure 6 and Figure 7). The camera (Sony Î±7R II) was placed at the focal distance. We moved the camera back and forth, and captured aerial images near the focal length. Focal length of a camera is , ISO sensitivity was . LCD and lens (SAL100M28) that are parts of Oculus DK2 were employed. Pitch size of TMD was .
We described the design method above with regards to the concept behind the optical element of the TMD before and during the aerial imaging of the eye. We implemented the application based on that concept. Figure 8 shows the application of the see-through wide view angle HMD that consists of an off-the-shelf HMD. Figure 9 shows the application of a non-wearable HMD that enables viewing without wearing goggles.
An orthogonal corner mirror cube (a non-commercial product) was used as the TMD. The pitch of the mirrors in the mirror cube was 0.5 mm and 0.3 mm, and the mirror ratio was 1:3. An aluminum vapor deposition mirror was used. The laminated thin plate mirror was cut such that it was attached at an angle of with respect to the long side of the rectangle. The following mountings use different mirrors having these pitch values.
Aerial Image Formation of the single lens
A Google cardboard and Oculus DK 2 were used as the HMD in the prototype. Based on the relationship for the focus in front of the eye, the maximum viewing angle depends on the viewing angle of each device. The positional relationships corresponding to the variables described in each viewing angle and resolution are summarized in the Table 1. The image obtained is as shown in the Figure 1 (d). A polarizing filter was used to prevent the narrowing of the visual field due to the secondary reflected light. The sense of resolution changes depending on the pitch of the mirror. The presumed changes in resolution are summarized in the Figure 5.
As an optical element that functions by forming an image in the air, it is only possible to change the light emitting position and the input position of a light field which possesses the capability of changing input / output relationship with respect to the light field. That is, it is difficult to absorb or diffuse light. In addition, as the resolution is increased, the mirror functions as a diffraction grating. With respect to the finer structure, it is difficult to form imaging and projection systems. Moreover, because the optical path length is extended, caution is required when the device is used for applications such as the ultrashort pulse laser used in Lasik surgery.
It is known that the corner cube and Nanoink printing TMDs are available on the market. In this research, a TMD that is easy to obtain and process is prepared and it is easy to perform simulation calculations for this device. The same method can be applied to Nanoink print TMDs as well. In fact, there is a high possibility that the resolution of the Nanoink print TMD will be improved.
7 Conclusion and Futurework
In this research, we propose a new HMD design that functions as an aerial virtual optical system in front of the eye by using Transmissive Mirror Devices. This is the first challenge: to use a TMD, which is usually used in aerial real imaging systems, for a near-eye display. We call this method an Air Mounted Eyepiece. AMD enabled the extension of the conventional HMD to a see-through HMD with a wide viewing angle. In future research, we will consider gaze tracking. Because the eyeball image is transferable to the HMD side, gaze tracking is possible without disturbing the user’s line of sight in wide-view retinal observation. Furthermore, by physically moving the position of the head mount display in accordance with head tracking, an immersive weightless environment can be realized. This development will be useful and safe for vehicle mounting and medical applications.
|( per eye)|
|( per eye)|
8 Related Work
Near-eye See-though Display
A near-eye see-though display for Augmented Reality has recently been proposed for various public uses. In the commercial field, Microsoft Hololens111https://www.microsoft.com/en-us/hololens (last accessed October 10, 2017) and Meta222https://www.metavision.com/ (last accessed October 10, 2017) are available for end users.
Additionally, many optical see-though displays were proposed in previous research. Displays using half mirrors  or free-form prisms  were proposed. However, the display with a half mirror distorts the scene transmitted through the lens or only allows for a narrow viewing angle, such that distortion correction has been proposed . In another method, Waveguide, Retinal scanning, and Holograpy  were proposed. However, complex optical configurations or high computer performance were required. Using transmissive liquid crystals enabled optical see-through and simple HMD configurations  employing a stacked transmissive LCD reproduced the light field . By modulating the point light source, a wide field of view was achieved. However, there are several problems that block the light from the real scene.
In our proposed method, we employ a TMD as a new optical element for a near-eye see-through display. By using a TMD, the real image of a single lens of VR googles is placed in front of the eye, the function of the head mounted display itself is created in the air. In contrast to previous methods, the scenery viewed through a TMD is not distorted and a wide viewing angle is obtained.
-  Satoshi Maekawa, Kouichi Nitta, and Osamu Matoba. Transmissive optical imaging device with micromirror array, 2006.
-  Daisuke Miyazaki, Noboru Hirano, Yuki Maeda, Siori Yamamoto, Takaaki Mukai, and Satoshi Maekawa. Floating volumetric image formation using a dihedral corner reflector array device. Appl. Opt., 52(1):A281–A289, Jan 2013.
-  Yasutoshi Makino, Yoshikazu Furuyama, Seki Inoue, and Hiroyuki Shinoda. Haptoclone (haptic-optical clone) for mutual tele-environment by real-time 3d image transfer with midair force feedback. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, CHI ’16, pages 1980–1990, New York, NY, USA, 2016. ACM.
-  Ronald Azuma and Gary Bishop. Improving static and dynamic registration in an optical see-through hmd. In Proceedings of the 21st Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’94, pages 197–204, New York, NY, USA, 1994. ACM.
-  Hong Hua and Bahram Javidi. A 3d integral imaging optical see-through head-mounted display. Opt. Express, 22(11):13484–13491, Jun 2014.
-  Yuta Itoh and Gudrun Klinker. Simultaneous Direct and Augmented View Distortion Calibration of Optical See-Through Head-Mounted Displays. In 14th IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2015, FUkuoka, Japan, Sep. 29 - Oct. 3, 2015, pages 43–48, 2015.
-  Andrew Maimone, Andreas Georgiou, and Joel Kollin. Holographic near-eye displays for virtual and augmented reality. ACM Transactions on Graphics, 36:85:1–85:16, July 2017.
-  Andrew Maimone and Henry Fuchs. Computational augmented reality eyeglasses. In ISMAR, pages 29–38. IEEE Computer Society, 2013.
-  Andrew Maimone, Douglas Lanman, Kishore Rathinavel, Kurtis Keller, David Luebke, and Henry Fuchs. Pinlight displays: Wide field of view augmented reality eyeglasses using defocused point light sources. ACM Trans. Graph., 33(4):89:1–89:11, July 2014.