ReHabgame: A non-immersive virtual reality rehabilitation system with applications in neuroscience
This paper proposes the use of a non-immersive virtual reality rehabilitation system ”ReHabgame” developed using Microsoft Kinect and the Thalmic Labs Myo gesture control armband. The ReHabgame was developed based on two third-person video games that provide a feasible possibility of assessing postural control and functional reach tests. It accurately quantifies specific postural control mechanisms including timed standing balance, functional reach tests using real-time anatomical landmark orientation, joint velocity, and acceleration while end trajectories were calculated using an inverse kinematics algorithm. The game was designed to help patients with neurological impairment to be subjected to physiotherapy activity and practice postures of daily activities. The subjective experience of the ReHabgame was studied through the development of an Engagement Questionnaire (EQ) for qualitative, quantitative and Rasch model.
The Monte-Carlo Tree Search (MCTS) and Random object (ROG) generator algorithms were used to adapt the physical and gameplay intensity in the ReHabgame based on the Motor Assessment Scale (MAS) and Hierarchical Scoring System (HSS). Rasch analysis was conducted to assess the psychometric characteristics of the ReHabgame and to identify if these are any misfitting items in the game.
Rasch rating scale model (RSM) was used to assess the engagement of players in the ReHabgame and evaluate the effectiveness and attractiveness of the game. The results showed that the scales assessing the rehabilitation process met Rasch expectations of reliability, and unidimensionality. Infit and outfit mean squares values are in the range of for all considered 16 items. The Root Mean Square Residual (RMSR) and the person separation reliability were acceptable. The item/person map showed that the persons and items were clustered symmetrically.
Rehabilitation is a complicated and active process by which those affected by injury or disease achieve a full recovery or, if a full recovery is not possible, realise their optimal physical, mental and social potential that allows them to regain much of their independence and quality of life Ortiz-Catalan et al. (2014). It is agreed that intensive, repetitive, and goal-oriented rehabilitation process improves motor function and cortical reorganisation in stroke patients with both acute and long-term (chronic) impairments. However, this recovery process is typically slow and labour-intensive, usually involving extensive interaction between one or more therapists and one patient Kan et al. (2011). Motor disability does not only cause limitations in functional motor control, muscle strength, and range of motion (ROM) but can limit the ability to perform daily tasks, that could cause isolation, and reduce participation in community activities Nowak (2008) and Piron et al. (2009b). Participating in repetitive exercises can help these people to overcome the limitations they experience, but lack of action and isolation stops them to perform recommended practices thus, become weaker, and cause obesity-related chronic health conditions and so on. Presumably, every person participating in rehabilitation hopes to be happily situated, productively occupied, and efficiently supported throughout the process McColl et al. (2001). A lack of motivation is another impediment that stops them to participate in physiotherapy sessions regularly.
Studies in computational neuroscience have shown that virtual reality (VR) rehabilitation techniques could create an environment that recovers health conditions as well as community integration Turolla et al. (2013) and Piron et al. (2009b). A study by Turolla et al. (2013) among post-stroke patients was conducted to investigate the effects of the two treatment methods: (I) combined VR and conventional therapy and (II) standard therapy alone. Their findings showed that the first process (combined VR and traditional treatment) was more efficient than the other. The past decade has witnessed the creation of a vast number of successful VR based rehabilitation, educational and commercial video games. They use new technology to simulate realistic features of the real world, Mohanty and Cantu (2011), Weiss et al. (2013), Mirelman et al. (2009), O’Neil et al. (2013), Ustinova et al. (2014), Levin et al. (2015), Saposnik et al. (2010), Dobkin (2004), Tsekleves et al. (2016), Smith et al. (1999), Page et al. (2004) and Holden and Dyar (2002). Piron et al. (2001) conducted the application of VR in psychology as an assessment tool considered as a highly sophisticated form of adaptive testing. Holden (2005) reviewed the applications of VR in the field of motor rehabilitation. They reported that the people with disabilities appear capable of motor learning within VR environments and the movements learned in VR are transferable to real-world equivalent. Mirelman et al. (2009) investigated the effects of training of the lower extremity using a robot coupled with virtual reality (VR). They also showed that movements learned in a VR environment are transferable to real-world similar motor tasks. Villiger et al. (2011) have used interesting VR scenarios to reshape cortical networks in patients which resulted in significant improvement in controlling the body, a decrease in neuropathic pain, and an increase in muscle strength.
The ”ReHabgame” was developed based on the neuromuscular rehabilitation system that assists players to combat the physical impairment and improve the upper limb performance. It is a flexible, non-immersive, low-cost VR system that does not use any head mounted devices but simulates the VR features integrating off-the-shelf devices such as the Kinect V2 (Microsoft in 2014)111https://en.wikipedia.org/wiki/Kinect and Myo armband (Thalmic Labs in 2016). This system encourages motor impaired participants to interact with virtual objects which are spawned in the scene. This system provides an easy and engaging rehabilitation process compared to using other haptic devices. It could be used by patients for long-term recovery which leads to better rehabilitation outcome. It was developed in Unity game engine that integrates the motion/gesture capture, image processing, skeleton tracking, IR emitter, inertial measurement units and EMG sensors capabilities into the games. The postural control/balance, gestures and range of motion (ROM) of the upper body was tracked through this marker less system. It facilitates the calibration of joints orientation, position of the leading joints and muscle activities throughout the gameplay. The ReHabgame was built as a single-player (the clinician’s avatar instructs the patient’s avatar to copy the exercises) and two-player game (two patients play at the same time and compete against one another). All the data were collected and saved in the background (hard drive) through the devices and were used for qualitative and quantitative analysis.
In this study, we define VR as a computer-based, interactive, multi-sensory environment that occurs in real-time. It provides enhanced feedback about movement characteristics, and advanced motor task learning and execution. Our study shows that the use of interactive VR system improves upper limb rehabilitation and habilitation which concurs with findings in literature including Piron et al. (2001, 2005); Holden (2005); Piron et al. (2009a); Fluet et al. (2010). Fig 1 illustrates the architecture of the system with its components. Two algorithms were adapted to acquire the gameplay to playersâ ability using MCTS algorithm and Random Object Generator (ROG). Players interact with the virtual objects in the virtual park through three scenarios (case studies) such as; (I) ”Reach-Grasp-Release fruits game”, (II) ”Reach-Press-Hold buttons game” and (III) ”Reach-Press buttons game” which are analysed together as the ReHabgame. An Engagement Questionnaire (EQ) was developed and administered to a group of 20 subjects to assess subjectively the effectiveness and efficiency of the ReHabgame using item response theory (IRT). The results of the Rasch analysis of the outcomes of this experiment are presented in this paper. To our knowledge, this is the first study that proposes an instrument to measure the subjective experience and employs Rasch analysis to evaluate the engagement in a motor rehabilitation game. Three features of game engagement were considered: flow, presence and absorption Brockmyer et al. (2009); Ryan et al. (2006). The Rasch model analysis of the gameplay shows that there is an appropriate balance between skill and challenge (flow). Participants feel present in the game and perceive a high-level of control (presence), which in turn offers a raised degree of engagement, thus, distracting them from the pain associated with the impairment (absorption).
2.1 ReHabgame principles, design and operation
The player reaches and grasps virtual fruits generated in the 3D world and releases them above the virtual basket Esfahlani et al. (2017); Esfahlani and Thompson (2016). The system is designed for one or two players to play at the same time independently as illustrated in Fig 2. It can be set up to rehabilitate the upper limbs (right hand, the left hand or both hands at the same time) and postural balance. The system combines the abduction, adduction, flexion, extension, internal/external rotation and horizontal abduction/adduction. The two-player game may be arranged as a clinician (first avatar/ instructor) shows the movements of the patient (second avatar) in that the patient follows the actions of the clinician and accordingly the patient’s ROM is recorded by the system. These data are used by the artificial intelligence (AI) control system to generate virtual objects within patient’s comfort zone initially and then gradually push them further out of the comfort zone. The algorithm is designed based on inverse kinematic in that the joints (such as head, neck, shoulder, elbow, wrist, etc.) location are monitored through the Kinect for further analysis. The ReHabgame was developed as follows:
”Reach-Grasp-Release”: The players gather the fruits spawned in the scene and release them above the virtual baskets to gain points. As soon as the fruit is collected it must be released above the basket to hit the bottom of the basket to collect points. It was developed using the gravity component and raycast hit function in Unity. Fig 1(a) shows the female avatar who has collected the orange with the left hand and aims to release it above the basket. The downward red circles indicate that it shouldn’t be released unless the downward circles changed to green colour. Thus, the player should adjust her arm to receive positive visual feedback (the colour changes from red circles to green). The male character’s shoulder joint follows a horizontal abduction (extension), the elbow joint is in a flexion mode and the hand is extended. The male avatar’s head and neck are tilted towards right shoulder that is detected as a problem in the postural control system. The streamed video and joint locations collected via the Kinect are used for further analysis. Fig 1(b) illustrates the female avatar who aims to reach the kiwi while the right shoulder’s joint is abducted. The male character in this figure has picked the peach and wants to release it above the basket but the visual feedback indicating the red downward circles which means it is not safe to release it. In Fig 1(c) the female avatar collected the strawberry but cannot release unless the circles indicating green. The male character is idle in this figure. Fig 1(d) shows the female avatar has collected the strawberry and wants to release it since the visual feedback specifies the green circles meaning that she can release the fruit to gain points. The male’s avatar hand is extended and the elbow join is flexed.
”Reach-Press-Hold”: In the reach-and-press button game, the player reaches out to a button to press it; the latter task can be configured to require a press-and-hold action or a press-for-specific-time action from the player in that the steadiness and the vibration of the hand are recorded based on fps.
Fig 3 illustrates the player & avatar in button Reach-Press-Hold game. Buttons are generated in a matrix format defined in the system based on patient’s range of movements (ROM). Fig 2(a) shows the button generated around the player’s head height in that the arm is abducted (away from the midline of the body). The player should raise the hand above the shoulder’s height to reach the button which requires higher ROM. Fig 2(b) illustrates the button generated in an area around the player’s heap height with arm abduction. Fig 2(c) shows the arm’s flexion (horizontal adduction) to reach the button generated at the shoulder’s height. The player reached the button and pressed it for one second with a visual feedback via the green dot around the white circle. Fig 2(d) illustrates the red part of the button is pressed for three seconds with three green visual feedbacks appeared around the button. If the player’s press and hold is not steady enough, the timer restarts itself.
Figure 2: Screen shots of a male and a female players with the representative avatars in the âReach-Grasp-Releaseâ fruits game. (a) shows the female avatar collected the peach and aiming to release it above the basket, and the male avatar collected a kiwi, (b) displays the female avatar aiming to collect the kiwi and the male avatar has collected the peach, (c) the female has collected the fruit and the male avatar is idle and (d) demonstrations the female avatar released the strawberry above the basket with green dots provided the positive feedback. Figure 3: Screen shots of a player in the âReach-Press-Holdâ button game. (a) the avatar’s right hand is engaged with the game which aiming towards the button to press its red part, (b) illustrates the avatar pressed the red button for three sec with three green dots appeared next to it which illustrates the visual feedback, (c) displays the avatar reached and pressed another button and holds it for one sec and (d) demonstrates the avatar with three seconds holding with its visual feedback.
2.2 Virtual object generator algorithms
Monte Carlo tree search algorithm (MCTS) has revolutionised the game’s performance where partial information requires decision making under uncertainties Chaslot (2010). MCTS is used as a method of finding optimal choices in a given domain by taking random samples in the decision space and building a search tree accordingly. The primary mechanism of generating adaptive patient-specific tasks in the ReHabgame resides in the object generator algorithm, which employs the MCTS procedure and Random Object Generator (ROG). To explore the effects of physical and gameplay intensity the MCTS object generation algorithm was used versus ROG.
MCTS was based on Monte Carlo sampling with a large number of random simulated games. MCTS makes the computation converge to the right value much more quickly than simple Monte Carlo itself Ciancarini and Favini (2010). This probabilistic algorithm uses lightweight random simulations of a problem space to grow a search tree that represents the unique states of the problem domain, with child nodes representing the outcome of specific actions Browne et al. (2012). In the ReHabgame the algorithm was broken down into four stages as highlighted below:
Selection: The algorithm starts at the root node, builds a tree of possible future game states. Each node in the tree represents a state and each link represents an action taken in that state and leads to a new state. While the state is built in the tree, the next action is chosen according to the statistics stored, in a way that balances between exploitation and exploration. The task is either selecting an action that leads to the best results so far called exploitation or a less likely action still needs to explore (known as exploration due to the uncertainty of the evaluation).
Expansion: More state (child nodes) needs to be added as new nodes to expand the tree. The tree is grown by one node for each simulated game according to the available actions. The first action is chosen by an expansion strategy and subsequently simulated. This results in a new game state, for which a new node is created.
Simulation: After expansion, a rollout is done from the new node, which means that a simulation is run from the new node applying random actions until a pre-defined stop criterion is met or the game ends. The satisfactory weighting of action selection probabilities has a significant effect on the level of play. If all legal actions are selected with equal probability, then the strategy played is often weak, and the level of the Monte-Carlo program is sub-optimal Chaslot (2010).
Backpropagation (backup): The score difference resulting from the rollout is backed up to the root node, which means that the reward is saved to the visited nodes then a new iteration starts. Backpropagation updates node statistics that are used for future decisions and ultimately inform the final decision made by the system for the root node.
An effective variant and selection strategy is the modified version of MCTS called upper confidence bound for trees (UCT) controller that construct statistical confidence intervals, de Waard (2016); Frydenberg et al. (2015). UCB is efficient and guaranteed to be within the best possible bound to address the exploration-exploitation in MCTS Browne et al. (2012) as illustrated in Eqn. 1. A child node is selected to maximise UCT, where is the number of times the current node (parent) has been visited, is the number of times child has been visited and is a constant that shifts priority from exploration to exploitation.
By increasing , the priority is shifted to exploration which means; states that have been visited less will be visited with a higher priority than states that have been visited more. A decrease changes priority to exploitation means that; states which have a great value are visited more to maximise reward. If more than one child node has the same maximal value, the tie is usually broken randomly. The estimated value for every node that has been visited in the iteration is updated with the reward. The estimated value of a node is the average of all rewards backed up to that node.
3.1 Motor Assessment Scale
The Motor Assessment Scale (MAS) Sabari et al. (2005, 2014) is the standardised assessment tool in the clinical evaluation and outcome studies of stroke rehabilitation. It consists of 8 items representing different areas of motor behaviour: supine-to-side lying onto entire
side; supine to sitting over a side of the bed; balanced sitting; sitting to standing; walking; upper-arm function; hand movements; and advanced hand activities. The four items of MAS was used in this study for the design of VR system. The postural balance (increments of displacement of the personâs centre of mass and upper body joints) were monitored by the Kinect, upper-arm function; hand movements; and advanced hand activities data were assessed via both the Kinect and Myo armband devices.
A hierarchical scoring system (HSS) was used in ROG in that when the player achieves the criterion for a score of more difficult level the easier steps do not need to be tested. The study considers that the force of gravity and higher degrees of freedom of joints require greater control thus considered as advanced steps in the game level design Sabari et al. (2005)
Upper-Arm Function: Shoulder’s depression, elevation, protraction, and retraction while placing elbow in a predefined position. Add elbow’s Flexion and extension at different angles. Take hand to an area around a head position, and behind it. Fig 4a shows the elbowsâ fixed position that was selected by the virtual purple button, and Fig 4b illustrates the handsâ orientation that was picked by the player. The fruit cannot be collected unless the purple button was pressed by the elbow (which was fixed to this location) and changed to green colour as an indicator of a positive feedback.
Hand Movements: Radial deviation of the wrist, lift the hand, reach forward (extend arms fully/ partially) in a pickup shape. Show palm to the screen while shoulder and elbow are in different positions (i.e. shoulder protracted, elbow extended, wrist neutral or extended and palm in contact with virtual objects).
Advanced Hand Activities: Grasp and releasing an object (I) the hand transport for carrying the hand toward the purpose considering shoulder, elbow and wrist movements and (II) the grasp component for shaping hand (fingers and thumb) for collection Jeannerod (1984) and Nowak (2008). Hand gestures to pick up, and release open/close fingers and the thumb.
The scoring system was designed based on these components. If the attempt was successful the score is one: , if partially completed is half: , if was unsuccessful is zero: , as defined in Eqn. 2:
is the score of the play where the setting could be graded for implementing the time as another factor to measure the count. It was determined by confronting the player’s time with a linear interpolation between a best possible time (max score) and a maximum period allowed (0 counts).
The choice of the MCTS/ROG algorithms and other parameters including the number of iterations left/right-handed gameplay, type of the game and single/double player game can be defined through the main menu. Fig 5a and Fig 5b, illustrates the Game Engine and the main menu of the ReHabgame, respectively. The maximum and minimum range of orientation and movement are predefined (i.e. in the MCTS the shoulder’s yaw is in the range . The pitch is , the roll of the shoulder is between and the elbow is within . Each range is given a density based on the number of samples selected from the menu. This limited number of options allows for a non-infinite search-space. Fig 5c, Fig 5d shows the virtual fruit that is spawned in the 3D-world based on the confidence of the search-tree in the pre-established range of the upper limb orientation. The black and red dots, next to the avatar in Fig 5, show the effects of the parameters picked for shoulder and elbow freedom of movement. samples were selected for shoulder’s yaw range (Sh yaw: first row of the black and red dots), samples for its pitch (Sh pitch: second row of the black and red dots), samples for its roll (Sh roll: third row of the black and red dots) and for the elbow (Elbow: fourth row of the black and red dots). The red dots in each picture corresponds to the orientation sample picked by the exploration tree to spawn the fruit. In Fig 5c the red dot in the ”sh yaw” row represents the desired rotation of needed to be explored. Likewise, the ”sh pitch”, ”sh roll” and ”elbow” are determined based on their range, samples and confidence.
3.2 Inverse Kinematics (IK)
Kinematics refers to the mathematical description of motion without considering the underlying physical forces. The kinematics of the human body is specifically concerned with formulating and solving for the translation, rotation, position, and velocity of each body segment in real-world motions. The movement of a kinematic chain, whether it is a robot or an animated character, is modelled by the kinematic equations of the chain Papaleo et al. (2012); Tolani and Badler (1996); Lura (2012). The IK determines the typical parameters that provide a desired position of the end-effector to achieve the task and is known as action planning Lura (2012). IK techniques provide direct control over the placement of an end effector object at the end of a kinematic chain of joints, solving for the joint rotations which place the object at the desired location. IK is adapted for calculating the end effector for the virtual object’s location. The algorithm that was used to calculate the joint angles required to collect the objects is defined in Eqn. 3:
Where are orientation around each axis and are each segment’s length.
4.1 Design of the EQ
Deep engagement in game-playing has the potential to be an important factor to predict the impact of using video games for rehabilitation purposes. Three key concepts are used in the literature to describe subjective experience relevant to engagement in a game: flow, presence and absorption Brockmyer et al. (2009); Ryan et al. (2006).
Flow describes the enjoyment feelings due to a balance between skill and challenge in the process of performing an exercise Csikszentmihalyi and Csikszentmihalyi (1992); Sinclair et al. (2009). The dual flow model measures a combination of attractiveness in terms of compelling gameplay and effectiveness in terms of physical outcomes Sinclair et al. (2009). Defining a specific goal and an immediate performance feedback increase the likelihood of flow, and being in a flow state seems to enhance learning. Flow states also include a feeling of being in control, being one doing the activity, and experiencing time distortions.
Presence is another intrinsic motivation in gaming contexts which is the sense of being within the game world by controlling the game, a need for challenge and feeling of effectence (the feeling of efficacy White (1959)). Concepts of presence are widely discussed by game designers, who attempt to make the experience of virtual worlds feel real and authentic Ryan et al. (2006).
Absorption defined as the total engagement in the present experience Brockmyer et al. (2009). In contrast to presence and in common with the flow, being in a state of psychological absorption induces an altered state of consciousness Ryan and Deci (2000). In this altered state, there is a separation of thoughts, feelings, experiences and effect are less accessible to consciousness Glicksohn and Avnon (1997); Sinclair et al. (2010). While this feature is perceived as having a negative psychological effect, we are looking at its potential to detach and distract the players from feeling the discomfort associated with their impairment and, thus, avoid frustration and giving up. An Engagement Questionnaire (EQ) was developed to evaluate user involvement in the ReHabgame using a self-report measure. The questionnaire has two parts, in part 1, a set of 16 questions require the user to rate on a Likert scale their experience in terms of flow (questions 3, 6, 8, 9, 16), presence (questions 5, 12, 13, 14, 15) and absorption (questions 1, 2, 4, 7, 10, 11). In the second part of the questionnaire, open-ended questions were asked, so that the players could explain their experience and provide feedback. The 5-point numeric rating scale is defined in that; 4 is ”always agree”, three is ”sometimes agree”, two is ”neutral”, one is ”sometimes disagree”, and 0 is ”always disagree”. The questions were listed in order from less engaged to more engaged ones. The answers provided for part 1 of the questionnaire were used to perform a psychometric analysis to determine the validity of the measurements. The part 2 of the questionnaire will be analysed and used to develop a beta version of the game in the future work.
Part one of the Engagement Questionnaire (EQ):
I feel spaced out (Absorption)
I feel intimidated (Absorption)
I get wound up (Flow)
The game kept me on my toes but did not overwhelm me (Absorption)
My actions seem to happen automatically (Presence)
I play without thinking about how to play (Flow)
I get into the game (Absorption)
I am so involved in what I am doing and cannot see myself separate from what I am doing (Flow)
If someone talks to me, I donât hear them (Flow)
I lose track of time (Absorption)
I feel different (Absorption)
When moving through the game world, I feel as if I am there (presence)
I experience feelings as deeply in the game as I have in real life (Presence)
When playing the game, I feel as if I am an important participant (Presence)
I felt competent and effective (Presence)
Playing makes me feel calm (Flow)
EQ-part two, answer the questions below in your terms:
What do you think about the appropriateness of the length of time the rehabilitation took?
During the rehabilitation session, did you focus on the remaining time?
How exhausted do you feel?
How difficult did you feel today’s session was?
How interesting did you find the game?
How quick did the time pass during the session?
How difficult was it to focus on the game?
How difficult was it to play the game?
Do you ever become so involved in the game as if you are inside the game rather than moving your limbs?
Please, tell us how we can improve the therapy sessions or the ReHabgame;
The pilot testing was conducted with post-stroke individuals (8 women, 12 men; mean age 54; range 4163 years) who were recruited voluntarily. All participants were discharged from a hospital, years ago. The ethical approval was obtained before starting the study, from the University Computing and Technology Departmentâs Research Ethics committee.
First, we described the study procedure and pattern of engagement and invited participants to read and sign an informed consent form. The study was continued for eight consecutive weeks in that player’s joint velocity, acceleration, a ROM, head and spine tilt and hand gestures were collected and saved on the hard disk. The EQ was designed and administered post-play to determine the player’s engagement in the game.
The upper limbs (arm/hand) are designed to perform very skilled movements which allow us to do day to day activities. According to the therapist, to allow us to perform all of the intricate activities, the brain is wired to the upper limb for speed and accuracy Weinstein et al. (2018). This process makes it vulnerable to an injured person, and consequently, rehabilitation can be slow Ramos-Murguialday et al. (2013). The rehabilitation system has adapted an algorithm to allow the patients to use their arm and hand in ways that they would otherwise be unable to do so. The games in this study are the computer-generated simulation of 3D images and environments that can be interacted with in a seemingly real and physical way by a player. It does not use any head-mounted equipment such as Oculus/Google to avoid any motion sickness or discomfort for players (patients). The player’s action partially determines what happens in the environment. The games were developed with direct collaboration and consultancy with physiotherapists and literature Pedraza-Hueso et al. (2015); Hung et al. (2016); Fluet et al. (2010); Ortiz-Catalan et al. (2014); Kim et al. (2005). The consultancy was performed throughout the development process as well as before starting to build the game. The information was gathered by asking the therapists what is important to be considered when we design our game and what elements must be implemented? Why donât people attend their physiotherapy sessions? Why do they find it boring? etc. We use the virtual rehabilitation system to promote rehabilitation and maximise the outcomes of the therapy. It aims to keep the muscles and joints of the hand and arm as healthy as possible, and this remains a key focus throughout the rehabilitation. This physiotherapy seeks to achieve as much function as possible and work towards an individualâs own goals.
Three/four minutes before starting the sessions the Myo armband was fitted to the participant’s arm to warm-up and synchronise with the game and computer. Patients then were asked to stand within 1.52.0 m from the screen facing the Kinect as illustrated in Fig 6. When the player stands in the Kinect’s field-of-view, the avatar figures are automatically generated. The participants were informed that the task is to interact with virtual objects generated on the screen. The virtual object’s trajectories were stored on the hard disk which enables the estimation of joint centres using IK. The postural control or balance was monitored while interacting with the virtual objects through the avatar and player acts as a controller. The reach tests were conducted with identical instructions, with the subject advised that the movement of the player won’t help in reaching object (the avatar’s spine is locked in a global coordinate system). The players must keep their body and shoulders straight, show palm to the screen and close fingers for collection. Both games continued for one hour under the supervision of a medical scientist and a psychologist to assess the usability of the games for the target group.
5 Results and Data Analysis
The ROM and physical parameters, such as joint positions, orientations and relative angles were captured by the Kinect. The Kinect was used due to its possibility to enable an efficient and continuing rehabilitation programme in the home environment. It makes treatment more enjoyable thus increasing motivation and therefore adherence. The system has been developed so that the physical presence of a therapist is not required during the session and there is no need for the player to wear any marker or wires to be connected to sensors. The inertial measurement unit (IMU) and muscle activity were collected via the Myo with the blue tooth connection. The data were used by the algorithm and also saved for off-line analysis. The player receives real-time feedback on performance as the MCTS simulation incorporates data efficiently from the player’s previous performances for a subsequent generation in the game. The motion data from the Kinect and Myo undergo a process of smoothing to achieve higher accuracy and efficiency. A smoothing algorithm was used to remove any noise from the signals collected by the sensors. The angular measurements and accurate linear motions were obtained by the IMU (gyroscope, accelerometer and magnetometer) sensors to distinguish precise linear movement from the accelerometer readings.
The EQ and ROM data were collected from each of the study participants. The participants played the ReHabgame with small breaks between each one to prevent fatigue. All feedback, suggestions and observations were collected to improve the game and tested again by the same persons and using the same EQ. The data gathered from the various stages of the therapy sessions were used for qualitative analysis.
Rasch analysis was implemented using Ministep (Winsteps) software222http://www.winsteps.com/ministep.htm. In 1969, Georg Rasch333https://www.rasch.org/ proposed a statistical model that complied with the fundamental assumptions made in measurements in neuroscience and has been used to aid in the construction and validation of health status questionnaire in neurology Rasch (1993); Tesio (2003); Duncan et al. (2003).
|Results of the Rasch rating scale model (RSM)|
|Item||Item difficulty||Item Fit (mean squares)||RMSR (root-mean-square residual)|
|No.||(Logit)||Infit Outfit||(observation - expectation)|
|Item Mean (SD): 45.8 (14.1), and Person Mean (SD): 36.6 (7.3)|
In EQ, 16 multiple choice questions were developed with 5-point numeric rating scale answers from (always disagree) to (always agree). Due to the itemsâ equal rating scale structure, the Rash Rating Scale Model (RSM) Andrich (1978) was used to determine the participantsâ engagement in the ReHabgame 444https://www.rasch.org/rmt/rmt94n.htm. Unidimensionality, reliability, targeting and response categories were examined.
To assess the unidimensionality (i.e. fitness for the Rasch model), a principal component analysis of Rasch residuals was used in which the observed response is compared with the predicted response of the model. Under good fit conditions for rating surveys, infit and outfit mean squares must be within the range of to Millis et al. (2014). The internal consistency of person and item performance were studied using separation reliability estimates and separation ratios. Separation reliability for person refers to the consistency of a persons responses across items. The items separation reliability refers to the consistency of item performances across persons. Separation ratios to provide evidence of internal consistency. Targeting was done using an examination of the item/person map in which the distribution and spread of items and persons along an ordinary logit were compared. Targeting is measured as a tool for participantsâ engagement and is defined as the extent to which items are of appropriate difficulty for the sample or item endurability to a personâs level of engagement. A logit is a Log-odds unit or a logarithmic transformation of the ratio of the probabilities of a correct and incorrect response scale of the latent construct or the probabilities of adjacent categories on a rating scale. Cronbachsâ alpha (Rasch Person Reliability) was used to measure the internal consistency between items and its strength Duncan et al. (2003). Item Response Theory (IRT) model or the probability of an individualsâ response to an item and model item characteristic curve (ICC) or the relationship between the probability of success to an item were assessed.
The Rasch analysis and RSM are outlined in Table 1. It shows the item numbers which correspond to the question numbers, the itemsâ difficulty (Logit), infit and outfit mean squares and RMSR for each item. Infit and outfit mean squares values are in the range of (0.681.52) for all 16 items, which satisfy the unidimensionality condition. The initial fit analysis statistics for the individual items show that the easiest items were ”(11): I feel different (Absorption)”, and ”(16): Playing makes me feel calm (Flow)” with the logits equal to . The most difficult item was item number ”(1): I feel spaced out” with the highest logit value . RMSR has its highest value of for item ”(6)” and the lowest value of for item ”(11)”. The Ministeps item/person map is shown in Fig 7. Item/person map would have a good targeting when the items and persons are clustered symmetrically all around the vertical axis. The item measures ranged from to logit and personsâ logit was in the range of to . Therefore, the Item/person map has a good targeting of items. The left side of the Fig 7 represented by illustrates the items and the right side are the participants (-) with good distribution to characterise the participantsâ engagement.
The Rasch expectation is that individuals with higher engagement in a game would have a higher likelihood of agreeing with more engaged items in the categories (higher response category). Fig 8 sketched out the category and conditional probability curves relative to item difficulty. The peaks of the curves for the categories (”always disagree = 0” to ”always agree = 4”) are always in ascending order along the latent variable. The crossover point for the category is within category and , which means this category could be collapsed into category and . The person/item threshold distribution map is outlined in Fig 8(b) with the x-axes displaying location or difficulty of item thresholds and the y-axes display the frequencies of item thresholds and participants. It can be seen that the offset of person to item suggests targeted scale.
Some of the important findings taken from the second part of the EQ are listed below. The majority of participants enjoyed the rehabilitation sessions and the ReHabgame activities. Overall, the participants commented very positively and reported that the activities covered a good ROM for the upper body. They suggested that audio feedback could be useful to create a more exciting environment in the game.
Fig 10a, Fig 10b illustrate the muscle activity during the gameplay collected by one of the EMG sensors of Myo from two participants while playing the same game. Fig 10c, Fig 10d shows the elbow rotation data generated in the hard disk and collected from the Kinect.
Some of the feedback & suggestions received from players are listed below:
”I like to play longer, and I feel it helps me to score better”
”The game becomes intense occasionally but still motivating, and I like to improve my score each time”
”The ReHabgame makes my muscles to be used [and] without realising they are getting stronger”
”The positive impact of the game was noticed by third parties (family members and my physiotherapist)”
”I found it very interesting. I feel the positive effects of the exercise on my whole body”
”My whole body was engaged with the game, and I feel I have more control over my posture”
”The difficulty level of the game seemed to match my ability, and I score better each session”
”The more I play I find that my finger movements are improving”
”I like that the gameplay was challenging but not exhaustive”
”I needed to concentrate a lot, and I couldn’t hear what people were saying around me”
”The game helps and improves the upper limb reaction and reduces its rigidity.”
”I found the visual feedback very instructive, but adding sound to the game would make it more stimulating”
”I would like to be able to compare my performance and score with previous sessions on graphs”
”I prefer VR games like the ReHabgame since head-mounted displays in other VR systems cause motion sickness”
6.1 Summary of main findings
To our knowledge, our study is the first to develop the rehabilitation game with the capabilities build in the ReHabgame. A Rasch model was developed to evaluate the participant engagement in the game as well as the system effective and attractiveness. The developed system enables a therapist to present a variety of controlled stimuli to measure and monitor the progress of the participants. The finding showed that the therapy provides an adequate physical and mental engagement through sufficient visual feedback throughout the sessions. It gives the opportunity for patients to perform additional exercises independently. It offers the possibility for a clinician to recommend an appropriate and individually aligned exercise. The attractiveness of the ReHabgame was measured by the standard flow, presence and absorption (psychological model balancing the playerâs perceived skill with the perceived challenge). The effectiveness was assessed to evaluate the physical balance between fitness (the bodyâs ability in tolerating exercise), and intensity (the difficulty of the training on the body). The Rasch analysis in this study showed that a unidimensional construct was created and fit statistics were obtained for the measured properties. The ReHabgame successfully calibrated the ROM of the players, muscle activities, linear and angular motions. The required movement or exercise level was adopted through the MCTS and the ROG based on HSS and MAS. In that skill level and game difficulty were gradually increased to challenge the player.
6.2 Strengths and limitations
Our study had some limitations in that the sample of participants were not fully representative of the diverse population of stroke patients (41 63) years old which was not wide enough to include younger or older people. The percentage of female participants to male was . Additionally, the ethics approval process to have access to the NHS was cumbersome and time-consuming therefore we decided to limit our study to the volunteers of communities.
The feasibility and effectiveness of the ReHabgame were studied as stimuli for improving the motor performance in a post-stroke participant. It facilitates autonomous physical rehabilitation at patient’s state as prescribed by the therapist. This system provided an environment that encourages patients to build greater strength and endurance on tasks that increase independence and enrich daily activity. The system monitors the trajectory of the hand during movements about the shoulder, arm, and elbow. The data taken from the devices are saved and accessed for further analysis as well as to generate the best game suitable for a patient with a certain disability through the intelligent computing system. The algorithm generates objects randomly and monitors the execution of the exercise, and the system provides direct feedback in real-time accordingly.
The therapy provided an adequate physical and mental engagement that offered sufficient visual feedback throughout treatment sessions. The MCTS and ROG algorithms were used to control the intensity of the physical activity that happened gradually based on the player’s performance. The data were obtained at devices local sampling frequency followed by performing linear interpolations to create a constant sampling interval while maintaining time and frequency domain signal integrity to achieve an identical sampling frequency to the kinematic data.
The subjective experience was conducted through the EQ to gather further information from the subjects about their experience of game playing. Some positive elements of the ReHabgame that was claimed by the player were the beautiful scenery and appealing feedbacks they received which had a direct relation to the engagement of a player. It aimed to distract the player’s attention towards the exciting aspects of the game to avoid fatigue. The training engages users to allow for the constant intensive practice required for motor plasticity. It monitors and progressively corrects the abnormalities of the kinematic movement, coordination and assess the transmitted electrical signals as the muscles strain. The subjective experience was performed through the EQ to gather further information from the subjects about their experience in playing the games. Future work includes experiments involving individuals with motor impairment, with a specific focus on gait and balance. It also aims to incorporate the possibility of implementing social interactions through online gameplay at patient’s home or could be arranged in social communities by patients carer or charity organisation. That increases the motivation by gathering more patients to cooperatively (or competitively) interact and play the game. It could enable a long-term recovery and could lead to better rehabilitation outcome Hung et al. (2016); Pedraza-Hueso et al. (2015). The open-ended questionnaire also will be used for developing the beta version of the ReHabgame in the future work.
The authors acknowledge Anglia Ruskin University for funding, organising and hosting the pilot study and testing interventions.
- Andrich (1978) Andrich, D., 1978. A rating formulation for ordered response categories. Psychometrika 43 (4), 561–573.
- Brockmyer et al. (2009) Brockmyer, J. H., Fox, C. M., Curtiss, K. A., McBroom, E., Burkhart, K. M., Pidruzny, J. N., 2009. The development of the game engagement questionnaire: A measure of engagement in video game-playing. Journal of Experimental Social Psychology 45 (4), 624–634, doi:https://doi.org/10.1016/j.jesp.2009.02.016.
- Browne et al. (2012) Browne, C. B., Powley, E., Whitehouse, D., Lucas, S. M., Cowling, P. I., Rohlfshagen, P., Tavener, S., Perez, D., Samothrakis, S., Colton, S., 2012. A survey of monte carlo tree search methods. IEEE Transactions on Computational Intelligence and AI in Games 4 (1), 1–43, doi:https://doi.org/10.1109/TCIAIG.2012.2186810.
- Chaslot (2010) Chaslot, G., 2010. Monte-carlo tree search. url:https://project.dke.maastrichtuniversity.nl/games/files/phd/Chaslot_thesis.pdf.
- Ciancarini and Favini (2010) Ciancarini, P., Favini, G. P., 2010. Monte carlo tree search in kriegspiel. Artificial Intelligence 174 (11), 670–684, doi:https://doi.org/10.1016/j.artint.2010.04.017.
- Csikszentmihalyi and Csikszentmihalyi (1992) Csikszentmihalyi, M., Csikszentmihalyi, I. S., 1992. Optimal experience: Psychological studies of flow in consciousness. Cambridge university press.
- de Waard (2016) de Waard, M., 2016. Monte carlo tree search with options for general video game playing. Ph.D. thesis, Universiteit van Amsterdam, doi:https://doi.org/10.1109/CIG.2016.7860383.
- Dobkin (2004) Dobkin, B. H., 2004. Strategies for stroke rehabilitation. The Lancet Neurology 3 (9), 528–536, doi:http://dx.doi.org/10.1016/S1474-4422(04)00851-8.
- Duncan et al. (2003) Duncan, P. W., Bode, R. K., Lai, S. M., Perera, S., in Neuroprotection Americas Investigators, G. A., et al., 2003. Rasch analysis of a new stroke-specific outcome scale: the stroke impact scale. Archives of physical medicine and rehabilitation 84 (7), 950–963.
- Esfahlani et al. (2017) Esfahlani, S. S., Cirstea, S., Sanaei, A., Wilson, G., 2017. An adaptive self-organizing fuzzy logic controller in a serious game for motor impairment rehabilitation. In: Industrial Electronics (ISIE), 2017 IEEE 26th International Symposium on. IEEE, pp. 1311–1318.
- Esfahlani and Thompson (2016) Esfahlani, S. S., Thompson, T., 2016. Intelligent physiotherapy through procedural content generation. In: Twelfth Artificial Intelligence and Interactive Digital Entertainment Conference. url:https://www.aaai.org/ocs/index.php/AIIDE/AIIDE16/paper/view/14055.
- Fluet et al. (2010) Fluet, G. G., Qiu, Q., Kelly, D., Parikh, H. D., Ramirez, D., Saleh, S., Adamovich, S. V., 2010. Interfacing a haptic robotic system with complex virtual environments to treat impaired upper extremity motor function in children with cerebral palsy. Developmental neurorehabilitation 13 (5), 335–345.
- Frydenberg et al. (2015) Frydenberg, F., Andersen, K. R., Risi, S., Togelius, J., 2015. Investigating mcts modifications in general video game playing. In: 2015 IEEE Conference on Computational Intelligence and Games (CIG). IEEE, pp. 107–113, doi:https://doi.org/10.1109/CIG.2015.7317937.
- Glicksohn and Avnon (1997) Glicksohn, J., Avnon, M., 1997. Explorations in virtual reality: Absorption, cognition and altered state of consciousness. Imagination, Cognition and Personality 17 (2), 141–151, doi:https://doi.org/10.2190/FTUU-GLC5-GBT8-9RUW.
- Holden (2005) Holden, M. K., 2005. Virtual environments for motor rehabilitation: review. Cyberpsychology & behavior 8 (3), 187–211, doi:https://doi.org/10.1089/cpb.2005.8.187.
- Holden and Dyar (2002) Holden, M. K., Dyar, T., 2002. Virtual environment training: a new tool for neurorehabilitation. Journal of Neurologic Physical Therapy 26 (2), 62–71, url:web.mit.edu/bcs/bizzilab/publications/holden2002a.pdf.
- Hung et al. (2016) Hung, Y.-X., Huang, P.-C., Chen, K.-T., Chu, W.-C., 2016. What do stroke patients look for in game-based rehabilitation: a survey study. Medicine 95 (11).
- Jeannerod (1984) Jeannerod, M., 1984. The timing of natural prehension movements. Journal of motor behavior 16 (3), 235–254, doi:http://dx.doi.org/10.1080/00222895.1984.10735319.
- Kan et al. (2011) Kan, P., Huq, R., Hoey, J., Goetschalckx, R., Mihailidis, A., 2011. The development of an adaptive upper-limb stroke rehabilitation robotic system. Journal of neuroengineering and rehabilitation 8 (1), 33, doi:https://doi.org/10.1186/1743-0003-8-33.
- Kim et al. (2005) Kim, G. J., et al., 2005. A swot analysis of the field of virtual reality rehabilitation and therapy. Presence: Teleoperators and Virtual Environments 14 (2), 119–146, doi:https://doi.org/10.1162/1054746053967094.
- Levin et al. (2015) Levin, M. F., Weiss, P. L., Keshner, E. A., 2015. Emergence of virtual reality as a tool for upper limb rehabilitation: incorporation of motor control and motor learning principles. Physical therapy 95 (3), 415–425, doi:https://doi.org/10.2522/ptj.20130579.
- Lura (2012) Lura, D. J., 2012. The creation of a robotics based human upper body model for predictive simulation of prostheses performance. Ph.D. thesis, University of South Florida, url:http://scholarcommons.usf.edu/etd/4133.
- McColl et al. (2001) McColl, M. A., Davies, D., Carlson, P., Johnston, J., Minnes, P., 2001. The community integration measure: development and preliminary validation. Archives of physical medicine and rehabilitation 82 (4), 429–434.
- Millis et al. (2014) Millis, S. R., Meachen, S.-J., Griffen, J. A., Hanks, R. A., Rapport, L. J., 2014. Rasch analysis of the community integration measure in persons with traumatic brain injury. Archives of physical medicine and rehabilitation 95 (4), 734–740.
- Mirelman et al. (2009) Mirelman, A., Bonato, P., Deutsch, J. E., 2009. Effects of training with a robot-virtual reality system compared with a robot alone on the gait of individuals after stroke. Stroke 40 (1), 169–174, doi:https://doi.org/10.1161/STROKEAHA.108.516328.
- Mohanty and Cantu (2011) Mohanty, S. D., Cantu, S., 2011. Teaching introductory undergraduate physics using commercial video games. Physics Education 46 (5), 570, doi:https://doi.org/10.1088/0031-9120/46/5/009.
- Nowak (2008) Nowak, D. A., 2008. The impact of stroke on the performance of grasping: usefulness of kinetic and kinematic motion analysis. Neuroscience & Biobehavioral Reviews 32 (8), 1439–1450, PMID: 18582943.
- O’Neil et al. (2013) O’Neil, R. L., Skeel, R. L., Ustinova, K. I., 2013. Cognitive ability predicts motor learning on a virtual reality game in patients with tbi. NeuroRehabilitation 33 (4), 667–680, PubMed:24018367.
- Ortiz-Catalan et al. (2014) Ortiz-Catalan, M., Nijenhuis, S., Ambrosch, K., Bovend, Eerdt, T., Koenig, S., Lange, B., 2014. Virtual reality. In: Emerging therapies in neurorehabilitation. Springer, pp. 249–265, url:http://ict.usc.edu/pubs/Virtual%20Reality.pdf.
- Page et al. (2004) Page, S. J., Sisto, S., Levine, P., McGrath, R. E., 2004. Efficacy of modified constraint-induced movement therapy in chronic stroke: a single-blinded randomized controlled trial. Archives of physical medicine and rehabilitation 85 (1), 14–18, PubMed:14970962.
- Papaleo et al. (2012) Papaleo, E., Zollo, L., Sterzi, S., Guglielmelli, E., 2012. An inverse kinematics algorithm for upper-limb joint reconstruction during robot-aided motor therapy. In: 2012 4th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob). IEEE, pp. 1983–1988, doi:http://dx.doi.org/10.1109/BioRob.2012.6290861.
- Pedraza-Hueso et al. (2015) Pedraza-Hueso, M., Martín-Calzón, S., Díaz-Pernas, F. J., Martínez-Zarzuela, M., 2015. Rehabilitation using kinect-based games and virtual reality. Procedia Computer Science 75, 161–168, doi:https://doi.org/10.1016/j.procs.2015.12.233.
- Piron et al. (2001) Piron, L., CENNIS, F., TONINS, P., Dam, M., 2001. Virtual reality as an assessment tool for arm motor deficits after brain lesions. Medicine Meets Virtual Reality 2001: Outer Space, Inner Space, Virtual Space 81, 386, PMID:11317774.
- Piron et al. (2005) Piron, L., Tonin, P., Piccione, F., Iaia, V., Trivello, E., Dam, M., 2005. Virtual environment training therapy for arm motor rehabilitation. Presence: Teleoperators and Virtual Environments 14 (6), 732–740, doi:https://doi.org/10.1162/105474605775196580.
- Piron et al. (2009a) Piron, L., Turolla, A., Agostini, M., Zucconi, C., Cortese, F., Zampolini, M., Zannini, M., Dam, M., Ventura, L., Battauz, M., et al., 2009a. Exercises for paretic upper limb after stroke: a combined virtual-reality and telemedicine approach. Journal of Rehabilitation Medicine 41 (12), 1016–1020.
- Piron et al. (2009b) Piron, L., Turolla, A., Agostini, M., Zucconi, C., Tonin, P., Piccione, F., Dam, M., 2009b. Assessment and treatment of the upper limb by means of virtual reality in post-stroke patients. Stud Health Technol Inform 145, 55–62, PMID:19592786.
- Ramos-Murguialday et al. (2013) Ramos-Murguialday, A., Broetz, D., Rea, M., Läer, L., Yilmaz, Ö., Brasil, F. L., Liberati, G., Curado, M. R., Garcia-Cossio, E., Vyziotis, A., et al., 2013. Brain–machine interface in chronic stroke rehabilitation: a controlled study. Annals of neurology 74 (1), 100–108.
- Rasch (1993) Rasch, G., 1993. Probabilistic models for some intelligence and attainment tests. ERIC, iSBN: 0-941938-05-0.
- Ryan and Deci (2000) Ryan, R. M., Deci, E. L., 2000. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American psychologist 55 (1), 68, doi:https://doi.org/10.1037110003-066X.55.1.68.
- Ryan et al. (2006) Ryan, R. M., Rigby, C. S., Przybylski, A., 2006. The motivational pull of video games: A self-determination theory approach. Motivation and emotion 30 (4), 344–360, doi:https://doi.org/10.1007/s11031-006-9051-8.
- Sabari et al. (2005) Sabari, J. S., Lim, A. L., Velozo, C. A., Lehman, L., Kieran, O., Lai, J.-S., 2005. Assessing arm and hand function after stroke: a validity test of the hierarchical scoring system used in the motor assessment scale for stroke. Archives of physical medicine and rehabilitation 86 (8), 1609–1615.
- Sabari et al. (2014) Sabari, J. S., Woodbury, M., Velozo, C. A., 2014. Rasch analysis of a new hierarchical scoring system for evaluating hand function on the motor assessment scale for stroke. Stroke research and treatment 2014.
- Saposnik et al. (2010) Saposnik, G., Teasell, R., Mamdani, M., Hall, J., McIlroy, W., Cheung, D., Thorpe, K. E., Cohen, L. G., Bayley, M., Group, S. O. R. C. S. W., et al., 2010. Effectiveness of virtual reality using wii gaming technology in stroke rehabilitation a pilot randomized clinical trial and proof of principle. Stroke 41 (7), 1477–1484, PMID:20508185.
- Sinclair et al. (2009) Sinclair, J., Hingston, P., Masek, M., 2009. Exergame development using the dual flow model. In: Proceedings of the Sixth Australasian Conference on Interactive Entertainment. ACM, p. 11, doi:https://doi.org/10.1145/1746050.1746061.
- Sinclair et al. (2010) Sinclair, J., Hingston, P., Masek, M., Nosaka, K., 2010. Testing an exergame for effectiveness and attractiveness. In: Games Innovations Conference (ICE-GIC), 2010 International IEEE Consumer Electronics Society’s. IEEE, pp. 1–8, doi:https://doi.org/10.1109/ICEGIC.2010.5716909.
- Smith et al. (1999) Smith, G. V., Silver, K. H., Goldberg, A. P., Macko, R. F., 1999. Task-oriented exercise improves hamstring strength and spastic reflexes in chronic stroke patients. Stroke 30 (10), 2112–2118, PMID:10512915.
- Tesio (2003) Tesio, L., 2003. Measuring behaviours and perceptions: Rasch analysis as a tool for rehabilitation research. Journal of rehabilitation medicine 35 (3), 105–115.
- Tolani and Badler (1996) Tolani, D., Badler, N. I., 1996. Real-time inverse kinematics of the human arm. Presence: Teleoperators & Virtual Environments 5 (4), 393–401, url:https://www.ncbi.nlm.nih.gov/pubmed/11539377.
- Tsekleves et al. (2016) Tsekleves, E., Paraskevopoulos, I. T., Warland, A., Kilbride, C., 2016. Development and preliminary evaluation of a novel low cost vr-based upper limb stroke rehabilitation platform using wii technology. Disability and Rehabilitation: Assistive Technology 11 (5), 413–422, PMID:25391221.
- Turolla et al. (2013) Turolla, A., Dam, M., Ventura, L., Tonin, P., Agostini, M., Zucconi, C., Kiper, P., Cagnin, A., Piron, L., 2013. Virtual reality for the rehabilitation of the upper limb motor function after stroke: a prospective controlled trial. Journal of neuroengineering and rehabilitation 10 (1), 1, PMID:23914733.
- Ustinova et al. (2014) Ustinova, K., Perkins, J., Leonard, W., Hausbeck, C., 2014. Virtual reality game-based therapy for treatment of postural and co-ordination abnormalities secondary to tbi: A pilot study. Brain injury 28 (4), 486–495, doi:https://doi.org/10.3109/02699052.2014.888593.
- Villiger et al. (2011) Villiger, M., Hepp-Reymond, M.-C., Pyk, P., Kiper, D., Eng, K., Spillman, J., Meilick, B., Estévez, N., Kollias, S. S., Curt, A., et al., 2011. Virtual reality rehabilitation system for neuropathic pain and motor dysfunction in spinal cord injury patients. In: Virtual Rehabilitation (ICVR), 2011 International Conference on. IEEE, pp. 1–4.
- Weinstein et al. (2018) Weinstein, M., Green, D., Rudisch, J., Zielinski, I. M., Benthem-Muñiz, M., Jongsma, M. L., McClelland, V., Steenbergen, B., Shiran, S., Bashat, D. B., et al., 2018. Understanding the relationship between brain and upper limb function in children with unilateral motor impairments: A multimodal approach. European Journal of Paediatric Neurology 22 (1), 143–154.
- Weiss et al. (2013) Weiss, P. L., Sveistrup, H., Rand, D., Kizony, R., 2013. Video capture virtual reality: a decade of rehabilitation assessment and intervention. Physical Therapy Reviewsdoi:http://dx.doi.org/10.1179/108331909X12488667117339.
- White (1959) White, R. W., 1959. Motivation reconsidered: The concept of competence. Psychological review 66 (5), 297, doi:http://dx.doi.org/10.1037/h0040934.