Recyglide : A Wearable Multi-modal Stimuli Haptic Display aims to Improve User VR Immersion

Recyglide : A Wearable Multi-modal Stimuli Haptic Display aims to Improve User VR Immersion

Juan Heredia Skolkovo Institute of Science and Technology Juan.Heredia@skoltech.ru Jonathan Tirado Skolkovo Institute of Science and Technology Jonathan.Tirado@skoltech.ru Vladislav Panov Skolkovo Institute of Science and Technology Vladislav.Panov@skoltech.ru Miguel Altamirano Cabrera Skolkovo Institute of Science and Technology Miguel.Altamirano@skoltech.ru Kamal Youcef-Toumi Massachusetts Institute of Technology youcef@mit.edu  and  Dzmitry Tsetserukou Skolkovo Institute of Science and Technology D.Tsetserukou@skoltech.ru

RecyGlide : A Forearm-worn Multi-modal Haptic Display aimed to Improve User VR Immersion

Juan Heredia Skolkovo Institute of Science and Technology Juan.Heredia@skoltech.ru Jonathan Tirado Skolkovo Institute of Science and Technology Jonathan.Tirado@skoltech.ru Vladislav Panov Skolkovo Institute of Science and Technology Vladislav.Panov@skoltech.ru Miguel Altamirano Cabrera Skolkovo Institute of Science and Technology Miguel.Altamirano@skoltech.ru Kamal Youcef-Toumi Massachusetts Institute of Technology youcef@mit.edu  and  Dzmitry Tsetserukou Skolkovo Institute of Science and Technology D.Tsetserukou@skoltech.ru
Abstract.

Haptic devices have been employed to immerse users in VR environments. In particular, hand and finger haptic devices have been deeply developed. However, this type of devices occlude the hand detection by some tracking systems, or in other tracking systems, it is uncomfortable for the users to wear two hand devices (haptic and tracking device). We introduce RecyGlide, which is a novel wearable forearm multimodal display at the forearm. The RecyGlide is composed of inverted five-bar linkages and vibration motors. The device provides multimodal tactile feedback such as slippage, a force vector, pressure, and vibration. We tested the discrimination ability of monomodal and multimodal stimuli patterns in the forearm, and confirmed that the multimodal stimuli patterns are more recognizable. This haptic device was used in VR applications, and we proved that it enhances VR experience and makes it more interactive.

Multi-modal haptics, vibrotactile feedback, VR aplications
journalyear: 2019copyright: rightsretainedconference: VRST ’19; November 12-15, 2019; Sidney, Australiabooktitle: VRST ’19, November 12-15, 2019, Sidney, Australiabooktitle: VRST ’19, November 12-15, 2019, Sidney, Australiaccs: Human-centered computing Haptic devices
Figure 1. a) Novel wearable haptic display RecyGlide. b) The user with RecyGlade and Tracking System HTC c) VR Appications
\Description

Enjoying the baseball game from the third-base seats. Ichiro Suzuki preparing to bat.

1. Introduction

Recently, several VR applications have been proposed in many fields: medicine, design, marketing, etc. , and to improve user immersion new methods or instruments are need. Haptics provides a solution, and enhances user experience using stimuli (Han et al., 2018). Most of the haptics devices are located in the palm or fingers. However, in some cases the haptic device position is a problem because VR Hand Tracking Systems need a free hand (Leapmotion) or a hand instrument (HTC) to recognize the position. Therefore, we propose a novel multi-modal haptic display located in the forearm.

Various haptic devices have been developed in the forearm with mono-modal stimulus (Dobbelstein et al., 2018; Moriyama et al., 2018). However, they have a persistent problem: users have more difficulty perceiving a stimulus in the forearm. The forearm is a not advantageous zone since it does not have as many nerves as the palm or fingertips have. Consequently, our device produces multi stimuli for improving user perceptions. Furthermore, multimodal stimuli experiments have been employed on the user’s hands, where the results show improvements in patterns recognition (Cabrera and Tsetserukou, 2019).

RecyGlide is a novel forearm haptic display to provide multimodal stimuli. RecyGlide consists of one inverted five-bar linkage 2-DoF system installed parallel to the radius, which produce a sliding force along the user’s forearm. The second stimulus is vibration; two vibration motors are placed in the device’s edges (see Fig. 2 (b)). The schematic and 3D representation are shown in Fig 2.

The user study aims to identify the advantages of multimodal stimuli use in comparison with monomodal sensations. The experiment consists of multimodal and monomodal pattern recognition by users. In our hypothesis, two tactile channels could improve the perception of patterns.

2. Device Development

provides the sensation of vibration, contact at one point, and sliding at the user’s forearm. The location of the haptic contact point is determined using kinematics model of inverted five-bar linkages, inspired by technology (Tsetserukou et al., 2014). The second stimulus is generated by two vibration motors located on the extreme sides of the device. The two types of stimuli allow creating different patterns at the forearm. Also, it could be used to interact with the VR environment, e.g., the sensation of submerging in liquids, the felling of an animal movement in the forearm, or information delivery about the relative location.

(a) Isometric view of the 3D model.
(b) Front view of the 3D model.
Figure 2. 3D CAD model of wearable tactile display.

The device has 2-DOF defined by the action of two servomotors. The slippage stimulus is produced by the movement of the motors in the same direction. Conversely, the movement of the motors in the opposite direction generates the force stimulus. The derivation of the previous stimuli creates other stimuli like temperature or pressure.

location is convenient for hand tracking systems. Commercial tracking systems have different principles to achieve their objective. However, the use of a device in hand decreases the performance. In the case of a visual track, if the haptic device is located in hand, the typical shape of the hand changes producing poor tracking performance. For systems with trackers like HTC, it is uncomfortable for the user to wear two devices in hand. The device has been designed to adapt ergonomically to the user’s forearm, allowing free movement of the hand when working in the virtual reality environment.

The technical characteristics are listed in the Table 1. The table shows the type of servomotors,and also the weight, the material of the device.

Motors Hitec HS-40
Weight
Material PLA and TPU 95A
Max. normal force at
contact point
Table 1. Technical specification of RecyGlide.

RecyGlide is electronically composed by an Arduino MKR 1000, servomotors and vibration motors. This Arduino model helps for IoT applications because of its wifi module. The Arduino is in charge of signal generation to the motors and communication wifi TCP/IP to the computer. The device maintains constant wifi communication with the computer throws a python script. A virtual socket transmits the data from python to the VR application in Unity.

In the experiment, a TCP/ IP server in Arduino was designed. The server provides direct access to the device, avoiding the use of the computer. This utility allows generating apps for cellphones too.

3. User Study

The objective of the following experiment is to analyze the user’s perception and recognition of patterns when mononodal and multimodal stimuli are rendered on the forearm, and to determine if the multimodal stimuli increase the user’s perception of the contact point position. The first user experience is a contact stimulus over the cutaneous area by the sliding action of the contact point generated by the inverted five-bar linkages device. The second user experience implements stimuli by the combination of vibrating motors and the sliding of the contact point generated by the inverted five-bar linkages device. The results of the experiment will help to understand if the multimodal stimuli improves the perception the position generated by RecyGlide device.

3.1. Experimental Design

Figure 3. (a)Displacement patterns: (A) Small Distance (SD): progress of 25% , (B) Medium Distance (MD)progress of 50% , (C) Large Distance (LG): progress of 75% , (D) Small Distance with vibration (SDV): progress of 25%, (E) Medium Distance with vibration (MDV): progress of 50%, (F) Large Distance with vibration (LDV): progress of 75%.

For the execution of this experiment, a pattern bank of 6 different combinations between displacement and vibratory signals was designed and is shown in the Fig. 5. In the first group of patterns, A (Small Distance (SD): progress of 25%), B (Medium Distance (MD): progress of 50%) and C (Large Distance (LG): progress of 75%), where monomodal stimuli are delivered; the second group, D (Small Distance with vibration (SDV): progress of 25%), E (Medium Distance with vibration (MDV): progress of 50%) and F (Large Distance with vibration (LDV): progress of 75%), includes multimodal stimuli.The vibration is delivered progressively according to the position of the contact point: while the position of the contact point is nearest to the edge where the vibration motor is located, the vibration frequency is higher; when the contact point is located at the middle distance, the vibration is the same in both vibration motors. The higher frequency used in the vibration motors is equal to . The sliding speed of the contact point over the skin is constant and has a value of .

3.2. Experimental Setup

The user was asked to sit in front of a desk, and to wear the device on the right forearm. The device was connected to one Arduino MKR1000. From the python console, the six patterns were sent to the micro-controller throw TCP/IP communication. To reduce the external stimuli, the users wear headphones with white noise. A physical barrier interrupted the vision of the users to their right arm. Before each section of the experiment, a training session was conducted, where all the patterns were delivered to the users five times. Six participants volunteering complete the experiments, two women and four men, with an average age of 27 years old. Each pattern was delivered on their forearm five times in a random order.

3.3. Results

To summarize the data obtained in the experiment, a confusion matrix is tabulated in the Table 2. The diagonal term of the confusion matrix indicates the percentage of correct responses of subjects.

Category Answers(predicted class)
SD MD LD SDV MDV LDV

Patterns

Small Distance (SD) 77 20 3 0 0 0
Medium Distance (MD) 0 80 20 0 0 0
Large Distance (LD) 3 3 93 0 0 0
Small Distance Vibration (SDV) 0 0 0 97 0 3
Medium Distance Vibration (MDV) 0 0 0 3 93 3
Large Distance Vibration (LDV) 0 0 0 0 0 100
Table 2. Confusion Matrix for Patterns Recognition.

The results of the experiment revealed that the mean percent correct scores for each subject averaged over all six patterns ranged from 80 to 96.7 percent, with an overall group mean of 90.6 percent of correct answers. Table 1 shows that the distinctive patterns LDV and SDV have higher percentages of recognition 100 and 97, respectively. On the other hand, patterns MD and SD have lower recognition rates of 80 and 77 percent, respectively. For most participants, it was difficult to recognize pattern SD, which usually was confused with pattern MD. Therefore, it is prove the requirement of more distinctive tactile stimuli (vibration) to improve the recognition rate.

The ANOVA results showed a statistically significant difference in the recognition of different patterns (F(5, 30) = 3.2432, p = 0.018532 0.05). The paired t-tests showed statistically significant differences between the SD and SDV (p=0.0410.05), and statistical difference between MD and MDV (p=0.0250.05). This results confirm our hypothesis, that the multimodal stimuli improves the perception of the humans in the forearm for short distances. However, the results of paired t-tests between the long distances with and without vibration do not reveal significant differences, thus for long distances perceptions, the multimodal stimuli are not required.

4. Applications

For the demonstration of RecyGlide, some applications were developed using the game engine Unity 3D. The SubmergingHand application clearly illustrate how the device improves the user immersion in a virtual reality environment. The device provides a sensation of the liquid level by the position of the contact point in the forearm. The viscosity of the liquid is represented by the normal force applied in the contact point. When the liquid is viscous, the applied force is high and the vibration motors work in a higher frequency. The application is shown in the Fig. 4.

Figure 4. Submerging Hand : moving the contact point the user recognizes the immersion in the liquid.

The application ”Boundaries Recognition” is an example of how our device helps to perceive the VR environment. Habitually VR users go forward the object boundaries and don’t respect the physics limits of static objects in the scene, such as walls and tables. RecyGlide informs the boundary collision of the hand tracker by activating the vibration motors and moving the contact point to one of the two sides, which is call collision side.

Figure 5. Boundaries Recognition: The display informs when the user has crashed whit an environment boundery

These two applications are basic examples, that can be applied in more complex ones. Additionally, the patterns can represent the current state of some variable or environment characteristic; for example, they can communicate the tracker battery status, the selected environment or the distance to an objective, etc.

4.1. Conclusions

We proposed a new haptic device in the forearm and made some experiments with it. Due to the results of the experiment, we demonstrated that multimodal stimuli patrons are easily recognizable. Therefore we consider that the device is suitable for communicating some VR messages through the use of patterns.

Though the forearm is not an advantageous area, the device has excellent performance and improves VR realism and the user immersion. The feeling of submersion in liquids can be used in numerous applications such as summing simulators, medical operations, games, etc. Boundaries collision detection is useful for all kind of VR applications because it is a constant problem in VR environments.

References

  • M. A. Cabrera and D. Tsetserukou (2019) LinkGlide: a wearable haptic display with inverted five-bar linkages for delivering multi-contact and multi-modal tactile stimuli. In Haptic Interaction, H. Kajimoto, D. Lee, S. Kim, M. Konyo, and K. Kyung (Eds.), Singapore, pp. 149–154. External Links: ISBN 978-981-13-3194-7 Cited by: §1.
  • D. Dobbelstein, E. Stemasov, D. Besserer, I. Stenske, and E. Rukzio (2018) Movelet: a self-actuated movable bracelet for positional haptic feedback on the user’s forearm. In Proceedings of the 2018 ACM International Symposium on Wearable Computers, ISWC ’18, New York, NY, USA, pp. 33–39. External Links: ISBN 978-1-4503-5967-2, Link, Document Cited by: §1.
  • P. Han, Y. Chen, K. Lee, H. Wang, C. Hsieh, J. Hsiao, C. Chou, and Y. Hung (2018) Haptic around: multiple tactile sensations for immersive environment and interaction in virtual reality. In Proceedings of the 24th ACM Symposium on Virtual Reality Software and Technology, VRST ’18, New York, NY, USA, pp. 35:1–35:10. External Links: ISBN 978-1-4503-6086-9, Link, Document Cited by: §1.
  • T. Moriyama, T. Nakamura, and H. Kajimoto (2018) Development of a wearable haptic device that presents the haptic sensation corresponding to three fingers on the forearm. In Proceedings of the Symposium on Spatial User Interaction, SUI ’18, New York, NY, USA, pp. 158–162. External Links: ISBN 978-1-4503-5708-1, Link, Document Cited by: §1.
  • D. Tsetserukou, S. Hosokawa, and K. Terashima (2014) LinkTouch: a wearable haptic device with five-bar linkage mechanism for presentation of two-dof force feedback at the fingerpad. In 2014 IEEE Haptics Symposium (HAPTICS), Vol. , pp. 307–312. External Links: Document, ISSN 2324-7355 Cited by: §2.
Comments 0
Request Comment
You are adding the first comment!
How to quickly get a good reply:
  • Give credit where it’s due by listing out the positive aspects of a paper before getting into which changes should be made.
  • Be specific in your critique, and provide supporting evidence with appropriate references to substantiate general statements.
  • Your comment should inspire ideas to flow and help the author improves the paper.

The better we are at sharing our knowledge with each other, the faster we move forward.
""
The feedback must be of minimum 40 characters and the title a minimum of 5 characters
   
Add comment
Cancel
Loading ...
398402
This is a comment super asjknd jkasnjk adsnkj
Upvote
Downvote
""
The feedback must be of minumum 40 characters
The feedback must be of minumum 40 characters
Submit
Cancel

You are asking your first question!
How to quickly get a good answer:
  • Keep your question short and to the point
  • Check for grammar or spelling errors.
  • Phrase it like a question
Test
Test description