Task-space Synergies for Reaching using Upper-limb Prostheses

Task-space Synergies for Reaching using Upper-limb Prostheses

Abstract

Synergistic prostheses enable the coordinated movement of the human-prosthetic arm, as required by activities of daily living. This is achieved by coupling the motion of the prosthesis to the human command, such as residual limb movement in motion-based interfaces. Previous studies demonstrated that developing human-prosthetic synergies in joint-space must consider individual motor behaviour and the intended task to be performed, requiring personalisation and task calibration. In this work, an alternative synergy-based strategy, utilising a synergistic relationship expressed in task-space, is proposed. This task-space synergy has the potential to replace the need for personalisation and task calibration with a model-based approach requiring knowledge of the individual user’s arm kinematics, the anticipated hand motion during the task and voluntary information from the prosthetic user. The proposed method is compared with surface electromyography-based and joint-space synergy-based prosthetic interfaces in a study of motor behaviour and task performance on able-bodied subjects using a VR-based transhumeral prosthesis. Experimental results showed that for a set of forward reaching tasks the proposed task-space synergy achieves comparable performance to joint-space synergies and superior to conventional surface electromyography. Case study results with an amputee subject motivate the further development of the proposed task-space synergy method.

I Introduction

Synergy-based interfaces in prosthetics enable the coordinated motion of multiple degrees of freedom in the human-prosthetic arm as needed by many activities of daily living (ADLs) [1]. Specifically, kinematic synergies relate the movement of the prosthetic device to the movement of the residual limb [2], e.g. regulating the movement of an elbow prosthesis as a function of the movement of the shoulder (residual limb) in a transhumeral amputation. However, the kinematic synergies investigated in the literature to date have been found to be dependent on individual motor behaviour and the task, making them difficult to generalise and implement practically [3, 4, 5, 6].

Kinematic synergies are typically realised by a joint-space relationship, such that movements of the joints in the residual limb correspond to movements of the joints in the prosthesis [2, 4, 6, 7, 8, 1, 9]. So far, two approaches to realise joint-space kinematic synergies have been reported in the transhumeral prostheses literature: postural [2, 7] and differential synergies [4, 6, 8, 1, 9]. Postural synergies define the human-prosthesis relationship in terms of joint displacements; i.e. for a given task and combination of joint displacements in the residual limb, there will be a resultant prosthesis joint displacement. Differential synergies define the human-prosthesis relationship in terms of joint velocities. As such, at any given position, the movement of the prosthesis is a function of only the movement of the residual limb in both cases of kinematic synergies. Therefore, this relationship is dependent on the initial limb position and the task used to calibrate the synergy. Both types of synergies require extensive calibration, and have been found to be dependent on individual motor behaviour and the task to be performed [2, 3, 4, 5, 7]. While recent results have demonstrated that differential kinematic synergies can be personalised to their user during task execution [6], a library of task-specific and individually personalised synergies would be needed to provide the arbitrary range of movements required by ADLs.

The dependency of synergies on human individuality and the task performed is an unavoidable challenge to their application in fields such as prosthetics. Current methods do not utilise task information, such as the location of the target, as it is challenging to obtain it in a prosthetic setting. However, using this information may be beneficial in improving the robustness and performance of synergistic interfaces, and avert the need of generating a library of task specific synergies. While obtaining this information may be a challenge with current methods, this may be overcome with future technologies or interface modalities that assist with determining reaching intention and direction [10].

It is therefore necessary to develop and evaluate alternative strategies capable of explicitly using task information in the design of synergistic prosthetic interfaces. This serves to overcome task dependency and human motor behaviour variations while maintaining voluntary user control and involvement. In this work, an alternative synergy-based strategy, utilising a synergistic relationship expressed in task-space, is proposed. This method has the potential to replace the need for personalisation and task calibration with a model-based approach requiring knowledge of the individual user’s arm kinematics, the anticipated hand motion during the task and voluntary information from the prosthetic user. Using classic robotics tools, the proposed task-space synergy can be generated using the manipulator’s Jacobian [11, 12], under the assumption that knowledge of the initial conditions and the desired end-effector (hand) motion is available. Here, a simplified case of such formulation under constrained motion is proposed. Location of the target is explicitly specified by the human user prior to the reaching motion, thus overcoming the assumption of the knowledge of the target location.

This paper presents the concept of task-space kinematic synergies in the context of forward reaching tasks with an elbow prosthesis. The performance of the resulting coordinated motion is compared against that produced in able-bodied motion, with a conventional sEMG interface, and a joint-space synergy [9]. Experiments were carried out in a Virtual Reality Environment to allow the emulation of prosthesis use in able-bodied subjects. The outcomes show that, for a set of forward reaching tasks, the proposed task-space synergy achieved better performance than conventional sEMG and comparable performance to a joint-space synergy interface [9] in terms of the closeness to able-bodied motor behaviour and task performance.The results achieved from a case study with an amputee subject motivate the further development of the proposed task-space synergy method.

Ii A Task-space Kinematic Synergy

The objective of a task-space kinematic synergy is to determine the motion of the prosthesis given the motion of the residual limb and the desired hand path. However, knowing the exact desired hand path before execution of the task is a challenge as information on the desired target is only known to the prosthesis user. While there exist methods to obtain this information through other interfaces (e.g. gaze-tracking [10]); here, a path-constraint approach is taken to allow the prosthetic user to convey the information of the target prior to commencing the reaching motion. This is done by the user aligning an imaginary straight line between the shoulder point, the target, and end effector (Figure 0(a)) prior to a forward reaching motion. This determines the path of the hand reaching motion, described by the vector (Figure 0(b)). The path can be thought of as the radial direction of a polar coordinate centered at the shoulder point S. Once determined, the reaching motion, which is now constrained to move only along , can be produced through coupling the elbow prosthesis flexion-extension to the user’s shoulder flexion-extension. This stage of the user aligning the hand along the target and the shoulder in a straight line will be referred to in this paper as the “aiming” stage, used to provide the information of the target in the proposed Task Space synergy approach.

(a) Target reaching example. The green dotted line shows the shortest path to target. The orange dotted line the forward straight reaching path. The reference frame has its origin at the shoulder joint and is attached to the shoulder.
(b) Simplified geometry for a reaching task by re-positioning the hand. The hand slides across the forward reaching path, defined by the vector () given from the shoulder to the hand frame origins.
(c) Definition of the direction of motion () and hand () frames that determine the forward reaching motion. The direction frame is defined by the triangle created by the shoulder (S), elbow (E), and hand (H) joints and it has its origin at the shoulder joint.
Fig. 1: Formulation of the forward reaching task and reference motion for the task-space synergy.

The protocol for the user to determine the constrained path is presented as follows. Let be the hand velocity in task-space. and are the residual limb joint angular displacement and velocity respectively (Figure 0(c)). In the transhumeral amputation case under consideration, and . and represent prosthesis joint angular displacement and velocity respectively (Figure 0(c)), for the elbow prosthesis case and . The reference frame has its origin at the center of the shoulder joint and is attached to the shoulder with its -axis pointing forward and the -axis upwards w.r.t. the upper body. The coordinate frame has its origin at the shoulder joint and is attached to the direction of reach. The shoulder angle is given by the angle between the upper-arm (SE) and the direction of reach (SH). The frame has its origin at the centre of the hand (end-effector) with aligned to . (See Figure 1).

Initially, the length of the upper arm () and lower arm () are measured and determined. Once the aiming process is completed, the following is carried out:

  1. A coordinate frame is defined with its -axis () aligned with the direction of reaching .

  2. Determine the frame transformation () in order to express shoulder and elbow joint angular displacements with reference to frame .

  3. Obtain the Jacobian matrix that relates the incremental movement of the hand (end-effector) to the incremental joint displacement of the shoulder and elbow, taking into account the reaching path the hand motion is constrained to ().

  4. Solve for as a function of , , and to determine the elbow motion reference that satisfies the desired task-space synergy.

These four steps are elaborated in the subsections below. The following assumption is used:

Assumption 1.

The angular position and velocities of the residual limb are measurable.

Remark 1.

Intertial Measurement Unit (IMU)-based measurements of residual limb motion have been demonstrated for synergistic elbow prostheses in [9, 13].

Ii-a Direction of Motion Reference Frame

The direction of motion frame determines the direction of the forward reaching task as exemplified in Figure 1, and is defined w.r.t. the reference frame as follows:

The -axis is defined by the unit vector given in the direction from the shoulder joint center to the initial hand position , as . The -axis is defined by the unit vector perpendicular to the plane given by the S-E-H triangle in the direction of , as . The -axis follows the right hand rule.

With this the direction of motion reference frame is defined and motion of the arm and hand can be expressed with it as reference. This concludes step A.

Ii-B Frame Transformation

Assuming that measured data is obtained in frame then it is necessary to transform these angles to the task frame . This assumption arises from the common use of relative measurements from two IMUs to produce residual limb pose measurements [9, 13]. This transformation can be performed through the Euler angles (, , ) in a Y-X-Z sequence. The transformation angles can be obtained as follows:

The first rotation () is along the -axis. can be obtained from , and is defined as , where and are the and components of . The second rotation () is along the -axis. can be obtained from , and is defined as . The last rotation () is along the -axis. can be obtained from , and is defined as .

The transformation is therefore given by:

(1)

where is an -axis standard rotation of the given angle . Thus it is possible to transform arm motion data from the reference frame to the task frame . This concludes step B.

Ii-C Arm Jacobian

By performing the adequate transformations, as previously presented, the forward reaching task can be simplified to a 2D task on the plane of the frame. This simplifies the kinematics required for the synergy to those of a 2-DOF planar arm. The differential kinematics of the simplified arm are well known and given by [12]:

(2)

where , , , , , and . and are the lengths of the upper and lower arm respectively. This concludes step C.

Ii-D Elbow Motion Reference

As previously discussed, through the simplification of the task only forward reaching motion is required along the direction of motion plane, i.e. only motion along the -axis. This means that it is desired to satisfy the constraint . With this constraint in consideration, it is possible to solve for the desired elbow velocity from Equation (2), such that

(3)

It is important to note that a singularity exists when , ; and needs to be properly handled. This concludes step D.

Iii Experimental Methodology

An experiment was designed to analyse the performance of the proposed task-space synergy method when used in a basic prosthetic elbow. The resultant behaviour with the proposed task-space synergy-based prosthetic interface was compared with able-bodied behaviour, a traditional sEMG-based prosthetic interface, and a basic joint-space synergy-based prosthetic interface. The experiment was performed in a Head Mounted Display-based (HMD) Virtual Reality Environment (VRE) on able-bodied subjects (Figure 1(d)). The use of a VRE allowed the gathering of data from able-bodied subjects for both able-bodied and prosthetic cases by virtually removing their lower arm and fitting them with a virtual prosthesis. The case study with an amputee subject used the same VRE-based methodology. A video description of the VR platform used for the experiment can be accessed from https://youtu.be/vvq4tjShdB4.

(a) Side view of the task. The object start position and target locations were normalised to the subject’s height and arm length.
(b) Subject point of view of the task. Only one target location was presented per repetition of the task. Subjects were required to place the virtual hand on the orange circle before the start of the task.
(c) Amputee subject with motion tracking sensors. Sensors were placed on: C7 vertebrae, shoulder acromion, and upper arm.
(d) Subject with motion tracking sensors, elbow brace, and VR headset. Sensors were placed on: C7 vertebrae, shoulder acromion, upper arm, forearm, and hand. An elbow immobiliser was used for the virtual prosthetic cases. In the EMG case, two sEMG sensors were placed on the subject’s forearm.
(e) Able-bodied VR avatar. Motion trackers determine the motion of the forearm and hand avatars, and the white markers. The elbow immobiliser was inactive to allow the subject to perform the task naturally.
(f) Prosthetic VR avatar. The upper arm motion tracker was used to create a “residual limb” avatar to which the virtual prosthesis was attached. The rest of the trackers were presented as white markers. The elbow immobiliser was engaged, and forearm and hand tracking was disabled.
Fig. 2: Experimental platform and task set-up. (a) presents the side view of the task and (b) the subject point of view in VR. (c) shows the motion tracking sensor placement, and (d) and (e) the VR avatars for the able-bodied and prosthetic cases.

Iii-a Experiment Task and Set-up

The task required subjects to pick and place an object in 3D space in the forward direction from a standing position without stepping. The execution of the task is considered as one iteration from the moment the object was grasped until the object was placed. One object start position and four target positions were used and were placed at a location relative to each subject’s height () and arm length (). The target location details are presented in Table I, these locations are relative to the subject’s initial standing position. Figure 1(a) shows the start and target locations in the VRE from a side view, while Figure 1(b) shows them from a subject point of view.

Target Comment
Start 0 Reachable from a neutral, relaxed position with the upper-arm down and elbow bent .
Close 0.12 Reachable with upper-body motion only.
Mid -0.12 Reachable with either upper-body motion or arm extension only.
Far 0 Reachable with both upper-body motion and arm extension only.
High 0 Reachable with both upper-body motion and arm extension only.
TABLE I: Object start and target positions.

Four different experiment cases were used, one able-bodied and three transhumeral prosthetic.

  1. AB (Able-bodied): This case was used to determine the benchmark motor behaviour and task performance for each subject.

  2. EP (sEMG Proportional): This is a standard dual-site differential surface EMG proportional prosthetic interface used to command the prosthetic elbow’s joint velocity [14]. This case was used as the traditional prosthetic interface for comparison.

  3. TS (Task-space Synergy): The proposed task-space synergy method as described in equation (3).

  4. JS (Joint-space Synergy): A Joint-space synergy method (JS) given by a shoulder-elbow flexion synergy was used as the synergistic prosthetic interface case for comparison.

The JS was given by ; where is the prosthetic elbow velocity, the shoulder extension velocity, and the synergy parameter. This joint-space synergy, based on [9], was chosen because it is the closest to the proposed task-space synergy as it does not require training or calibration by allowing the user to compensate for the synergy by choosing when in the reach to enable it.

Subjects were fitted with an elbow orthosis with two motion tracking sensors attached to it, one for the lower-arm and one for the upper-arm. Two additional motion tracking sensors were placed on the C7 vertebrae (trunk motion), the shoulder acromion (shoulder motion), and the hand (see Figure 1(d)). The elbow orthosis allowed to immobilise the subject’s elbow in the prosthetic cases to better represent muscle conditions in amputees, as done in [9]. The amputee subject was fitted as able-bodied on his right arm for the AB case, while for the prosthetic cases he had the upper-arm sensor strapped to his left upper arm (see Figure 1(c)).

In the AB case, motion of the upper-body, the whole arm and hand was directly mapped to the VRE through a virtual avatar as shown in Figure 1(e). In the prosthetic cases, transhumeral amputation was emulated in able-bodied subjects in the VRE by using the data from the upper-arm motion tracker to determine the motion of the virtual residual limb. A virtual prosthetic elbow, forearm, and hand were attached to the virtual residual limb as seen in Figure 1(f). Only the virtual prosthetic elbow was actuated, while the forearm and hand were fixed.

Muscle activation was gathered using sEMG electrodes placed on the forearm as the elbow orthosis interfered with the biceps/triceps electrode placement typically used in transhumeral amputees. While the forearm electrode position does not reflect the physiological placement of sEMG sensors for transhumeral prostheses, it can provide comparable functional capabilities in able-bodied subjects as observed in [9].

The following prosthetic interface simplifications were done to reduce the cognitive load on the subjects and to focus the experiment on the motor behaviour aspect of prosthesis use. Firstly, the functionality of the prosthetic elbow was enabled with a button press instead of relying on complex sEMG switching commands, which are a major issue with active prostheses to date [15]. Secondly, grasping and releasing of the object were done automatically when the hand stopped moving and was touching the object (grasp) or within the target area (release), avoiding the need to switch between elbow and hand control.

Iii-B Hardware Set-up

The experiment was performed on an HTC Vive Pro HMD with the application developed in Unity3D. The experimental platform runs on an Intel Core i7-8700K processor at 3.7GHz, with 32GB RAM, and an Gigabyte GeForce GTX 1080Ti video card with 11GB GDDR5. HTC Vive Trackers were used for motion tracking, except for hand motion tracking where an HTC Vive Controller was used (HTC Vive system tracking capabilities can be found in [16]). The C7 and shoulder acromion Vive Trackers were attached directly to the subject’s body using medical tape. Data gathering, and VR update were performed at 90Hz. Myoware sensors were used for sEMG data gathering with Ag-AgCl electrodes. The VR platform used for the experiment can be downloaded from https://github.com/Rigaro/VRProEP.

Iii-C Experimental Protocol

The experiment was performed on 15 able-bodied subjects, six female and nine male. The age range was and the median age . The amputee case study was performed on a single male subject aged . The subject has a congenital limb difference in his left forearm as seen in Figure 1(c). He is not a prosthesis user and does not use his left arm in daily tasks relevant to the task tested in this experiment. The experiment was performed in two sessions lasting two hours each, on two different days not more than a week apart. Each session tested two cases. The first case tested for all subjects was AB which was used as the benchmark for individual motor behaviour. The order of the following cases was randomly selected from the three cases representing prosthetic-use (EP, TS or JS) to minimise the bias introduced by the effects of learning on the data gathered. The session procedure is presented in Table II.

Step Time (minutes) VR/No-VR
Introduction 5 No-VR
Sensor placement 15 No-VR
Case 1 50 VR
Break 5 No-VR
Case 2 50 VR
TABLE II: Experiment session protocol.

Each block of case testing consisted of 120 iterations of the task separated into three sets of 40 iterations, the procedure is presented in Table III. During the Training block of the experiment, subjects were instructed outside the VRE on how to use the prosthetic interface and were allowed to familiarise themselves with it. This was done by presenting a third person view of the virtual residual limb and prosthetic device on a screen. The procedure was approved by the University of Melbourne Human Research Ethics Committee, project number 1750711.2. Informed consent was received from all subjects in the study.

Step Task iterations per target Time (minutes) VR/No-VR
Training - 5 No-VR
Practice 5 - VR
Set 1 10 - VR
Break - 1 VR
Set 2 10 - VR
Break - 1 VR
Set 3 10 - VR
TABLE III: Case testing block protocol.

Iii-D Data Gathering and Analysis

The data gathered from motion tracking sensors and the VRE was used to analyse the performance of subjects in terms of the task, quality of motion, and the resulting motor behaviour. The metrics used for data analysis are presented next.

Two metrics of task performance were used: completion time and reach accuracy. The mean task completion time () over iterations of the task is used to determine the effects of prosthetic interfaces on the time required to complete the task. As this metric is commonly used in the literature, it is used to compare the performance of the synergistic interfaces in this paper to other studies in the literature. The mean reaching accuracy () over iterations of the task is used to verify that the task is being performed adequately.

Two metrics of quality of motion were used: path smoothness and variability. Continuous, well coordinated movements are characteristic of trained and healthy motor behaviour. Hand and joint path smoothness are well documented and accepted measures of the quality of upper-limb motion [17, 18, 19]. Here Spectral Arc Length (SAL), represented by (), is used as the metric for smoothness [20] due to its characteristics which include sensitivity to changes in motor characteristics, robustness to measurement noise, and dimensionless monotonic response. Low path variability, both of the hand and joints, is characteristic of trained and healthy motor behaviour [18, 21]. Here the standard deviation of the hand and joint path (), is used as the metric for path variability.

Two metrics of motor behaviour were used: hand/joint path difference and upper-body displacement. The first metric of motor behaviour is path difference (). This metric evaluates the difference between the mean able-bodied hand/joint path and the mean prosthetic hand/joint path. It is used to determine the closeness of the resultant prosthetic motor behaviour to the individual’s natural able-bodied motor behaviour. Let the mean hand/joint path for the able-bodied case () be given by and the mean path for the prosthetic case () given by , where is the sampling time. The path difference () is defined as the sum of the absolute differences between the mean paths across the time domain as follows.

(4)

The second metric of motor behaviour is upper-body displacement (). This metric evaluates the amount of upper-body motion used for the task. When a prosthetic device is not used effectively or is not adequate for the task this metric represents compensation motion. Some level of upper-body motion is also part of natural able-bodied motion during reaching tasks.

Iv Results & Discussion

Results for the experiment with able-bodied subjects and the case study with an amputee subject are presented next. Given that target locations were personalised to each subject, hand paths for all cases and targets were normalised to the subject’s arm length. This was done so that all analysed paths were on the same plane, such that the starting position coordinates were and the target position coordinates were [22]. The data-set of the results (raw and processed) can be downloaded from https://github.com/Rigaro/TaskSpaceSynergyData2020.

Iv-a Experiment with Able-bodied Subjects

The results for the hand path difference over iterations of the task are presented in Figure 3, where the population mean of the hand path difference for each iteration of the task is presented for each target. From this it can be seen how subjects, over iterations, try to reproduce their able-bodied hand path with the prosthesis. In general, it can be observed that there was a gradual reduction of the difference over the first 15 iterations until steady-state is reached. The reduction of path difference over iterations suggests that individuals try to reproduce AB motion thus leading to an improvement on their performance. The clear steady-state difference from the AB path shows that despite their best effort, the introduction of limb loss and the artificial limb inherently changes the dynamics of reaching. This means that the natural/dynamically optimal motion when using a prosthesis is different from one’s AB motion. For the Far target, which requires more joints to be coordinated to be reached (trunk, shoulder and arm), it is observed that TS and JS performed 15% and 25% better than EP, respectively. This shows the advantage of kinematic synergies over traditional sEMG, which only allows for joint-by-joint control of the prosthesis as opposed to coordinated motion. Task performance and motor behaviour results are presented next.

Fig. 3: Population mean experimental results for hand path difference from AB () over iterations of the task for each target.

The improvement in task performance can be observed in the task completion time, shown in Figure 4, which is reduced over iterations of the task. While the AB completion time remains mostly unchanged, the prosthetic cases see a significant improvement in completion time, which is correlated to the hand path difference. Synergistic interfaces on average had a 20% shorter task completion time than sEMG for the Far target, which requires more joints to be coordinated (upper-body and upper-arm), with both TS and JS performing similarly. This suggests that TS is as effective as JS in reducing the time required to perform reaching tasks. While in [9] it was reported that JS had a significant advantage over EP in terms of completion time, from the results there it was not possible to conclude whether it was because of the motion-based prosthetic joint-switching mechanism proposed therein or the synergy (JS) itself. The results presented herein suggest that the significant difference between EP and JS reported in [9] may be due to to the switching mechanism, as a major difference is only observed for the reaching task that requires more joints to be coordinated. On the other hand, accuracy showed similar results across all cases which indicates that subjects were capable of performing the task with similar accuracy for all cases and targets.

Fig. 4: Population mean experimental results for task completion time () over iterations of the task for each target.

Given that the results for the Far target show the most significant difference between cases, the hand and joint path analysis will focus on that target. The mean and standard deviation hand paths and joint angles of the last 10 iterations (steady-state) of the Far target for a representative subject are presented in Figure 5. Figure 6 presents the population mean hand path and joint path variation, hand path and joint-space smoothness, and trunk and shoulder displacement results.

(a) Mean and standard deviation hand paths.
(b) Mean and standard deviation joint paths.
Fig. 5: Mean and standard deviation hand paths and joint angles of the last 10 iterations (steady-state) of the Far target for a representative subject
(a) Population hand path variability () mean and standard deviation.
(b) Population joint path variability () mean and standard deviation.
(c) Population hand path smoothness () mean and standard deviation. The less negative the smoother the path.
(d) Population joint path smoothness () mean and standard deviation. The less negative the smoother the path.
(e) Population C7 displacement () mean and standard deviation.
(f) Population shoulder acromion displacement () mean and standard deviation.
Fig. 6: Population mean experimental results for the Quality of Motion and Motor Behaviour metrics for the last 10 iterations per target.

Reaching Motion Variability and Smoothness

The hand path in Figure 4(a) is normalised between zero and one, representing the start of the movement and the target location, respectively. From the TS results, the initial aiming movement can be identified from the low variability part of the reach, as this is only performed with the residual limb (shoulder) and thus is less variable. This is followed by the straight hand forward movement generated by the synergy, which has higher variability. The aiming phase can also be seen in the joint movement, as shown in Figure 4(b), where the mean and standard deviation of shoulder and elbow angles are presented. The TS movement starts with a shoulder only aiming motion, followed by a smooth curve. This is further corroborated by the variability and smoothness results, presented in Figure 6. Synergistic interfaces show lower hand path variability over EP for the Far target, with TS showing a 16% reduction in variability over EP, and JS a 31% reduction over EP. TS shows a 20% reduction in joint path variability over EP while JS a 31% reduction over EP. In terms of hand path smoothness the JS interface shows a 10% improvement in smoothness for the Far target with respect to EP, while TS shows no improvement. This may be attributed to the need with TS to pre-position the hand before executing the movement, which affects the resultant smoothness metric. On the other hand, joint motion was 18%, 24%, and 15% smoother than EP for the Mid, Far, and High targets respectively. On the other hand, the TS case only shows a 17% improvement in joint smoothness for the Far target with respect to EP.

The joint path of subjects with the EP interface, in Figure 4(b), shows a mostly horizontal line representing the initial shoulder only movement, followed by a combined elbow and shoulder movement where subjects use EMG to extend the elbow while using the shoulder to do the fine adjustments of hand position. Lastly, JS shows a mostly smooth curve as it was identified that subjects learned the timing to activate the synergistic movement to minimise the error at the end of the motion. Interestingly, this resulted in more natural and joint paths, further demonstrating the capabilities of human motor learning and adaptation. In the end it appears that, regardless of the type of synergy, humans will adapt to the best of their capabilities to use the synergy effectively.

Joint Range and Trunk Movement

Another important feature that arises from Figure 4(b) is the range of joint angles used. It can be observed that out of all cases, TS utilises a wider range of arm joint angles suggesting more arm utilisation. This is corroborated with the trunk and shoulder displacement metrics, shown in Figures 5(e) and 5(f), which show that TS has the lowest trunk and shoulder forward displacements. Trunk displacement for the TS case shows a 60% and 45% reduction over EP and JS, respectively, for the Mid target; while a 60% reduction over EP and JS for the Far target. These results agree with those reported in [3], which present a fully calibrated synergy, where a reduction in trunk compensation motion was reported with a JS method over EP, suggesting that TS may further reduce compensatory movement at the expense of a less natural movement (point and reach). However, this is due to the constraint imposed on the reaching path. If the location of the reaching target is known in the formulation of the task-space synergy, then it may be possible to generate a human-prosthesis motion that closely resembles able-bodied behaviour, while retaining the user’s control over the movement. Moreover, as the task-space synergy is based on the arm kinematics, no individual synergy training or personalisation may be required, and could be generalised to any point-to-point reaching task. Therefore, this scenario will be explored in future studies.

Iv-B Case Study with Amputee Subject

Results for the case study are presented in Figures 7, 8, 9, and 10. The results for the subject’s hand path difference over iterations of the task are presented in Figure 7. From this, it can be seen that the subject did not show a significant change in reaching strategy over iterations of the task, regardless of the interface modality. This may be due to reaching motion being new to the subject as he had never done this type of coordinated motion with his left arm.

The results for task completion time over iterations of the task are presented in Figure 8. Similarly to hand path difference, there is no apparent improvement in time over iterations of the task. Further analysis of the results did not reveal any trends. This may be a result of the short time allowed for the experiment and the novelty of the experience for the subject. As he had never used a prosthetic device nor performed the type of tasks in the experiment with his left arm, the duration of the experiment may not be sufficient to see significant improvement in his performance.

Similarly to the experimental results, the results for hand and joint paths are presented only for the Far target. Hand path results are presented in 8(a) while joint path results are presented in 8(b). For this subject, the TS case is the one that observes the lowest variability hand path variability (Figure 8(a)), which is supported by the results in Figure 9(a), except for the High target. However, in terms of the closeness to the AB hand path, all three prosthetic cases perform similarly. On the other hand, the experimental results suggested that the JS modality achieved the lowest variability.

Regarding the joint path, the case study results in Figure 8(b) are closer to the experimental results in Figure 4(b) in terms of their shape. While the amputee subject showed smaller joint-range utilisation, similar features highlighting the interface use strategy arise. For instance, the EP strategy shows the same straight line shapes characteristic of sequential motion, while both TS and JS show curves. In the TS case, it can be seen that the amputee subject also followed the aim-and-extend strategy as seen by the horizontal line segment with lower variability. In general, both TS and JS outperform EP in terms of joint path variability as shown in Figure 9(b).

Lastly, results for hand and joint path smoothness (Figures 9(c) and 9(d)), and trunk and shoulder displacement (Figures 9(e) and 9(f)) show no significant difference between the three prosthetic cases.

These results highlight human-to-human variation, where preference to different interface modalities may arise. Moreover, the type of amputation, or in this case limb difference, may affect the subject’s preference towards an interface modality. Anecdotally, the subject stated a preference towards the TS modality as he found it easier to use because of the point-and-extend strategy. This motivates further studies with subjects with different amputations and limb differences that focus on the human aspect of the human-prosthetic interface.

Fig. 7: Case study experimental results for hand path difference from AB () over iterations of the task for each target.
Fig. 8: Case study experimental results for task completion time () over iterations of the task for each target.
(a) Mean and standard deviation hand paths.
(b) Mean and standard deviation joint paths.
Fig. 9: Mean and standard deviation hand paths and joint angles of the last 10 iterations (steady-state) of the Far target for the amputee subject
(a) Amputee subject hand path variability () mean and standard deviation.
(b) Amputee subject joint path variability () mean and standard deviation.
(c) Amputee subject hand path smoothness () mean and standard deviation. The less negative the smoother the path.
(d) Amputee subject joint path smoothness () mean and standard deviation. The less negative the smoother the path.
(e) Amputee subject C7 displacement () mean and standard deviation.
(f) Amputee subject shoulder acromion displacement () mean and standard deviation.
Fig. 10: Case study experimental results for the Quality of Motion and Motor Behaviour metrics for the last 10 iterations per target.

V Conclusion

In this paper a realisation of a task-space synergy for motion-based human-prosthetic interfaces was presented which allows the prosthesis to be controlled in task-space through the anticipated path of the prosthetic hand. Such type of synergistic interface has the potential to be generalisable to any reaching task, is inherently personalised, and may prove useful in complex tasks that require the coordination of multiple degrees of freedom. An experimental study with able-bodied subjects and a case study with an amputee subject showed that the proposed task-space synergy achieves comparable performance to joint-space synergistic interfaces in terms of the closeness to able-bodied motor behaviour and task performance. While superior performance over personalised synergies or calibration were inconclusive, it provides the advantage of not relying on time-consuming calibration processes or human motor learning. Future studies will investigate the application of task-space synergies to higher complexity tasks and multi-degree-of-freedom prosthetic devices, and the case when full information on the reaching target is available.

References

  1. M. Merad, A. Roby-Brami, and N. Jarrasse, “Towards the Implementation of Natural Prosthetic Elbow Motion Using Upper Limb Joint Coordination,” in IEEE RAS EMBS Int. Conf. Biomed. Robot. Biomechatronics, pp. 821–826, 2016.
  2. R. R. Kaliki, R. Davoodi, and G. E. Loeb, “Evaluation of a noninvasive command scheme for upper-limb prostheses in a virtual reality reach and grasp task,” IEEE Trans. Biomed. Eng., vol. 60, no. 3, pp. 792–802, 2013.
  3. M. Merad, É. de Montalivet, A. Touillet, N. Martinet, A. Roby-Brami, and N. Jarrassé, “Can we achieve intuitive prosthetic elbow control based on healthy upper limb motor strategies?,” Front. Neurorobot., vol. 12, 2018.
  4. M. Legrand, M. Merad, E. de Montalivet, A. Roby-Brami, and N. Jarrassé, “Movement-based control for upper-limb prosthetics: Is the regression technique the key to a robust and accurate control?,” Front. Neurorobot., vol. 12, 2018.
  5. R. Garcia-Rosas, D. Oetomo, C. Manzie, Y. Tan, and P. Choong, “On the relationship between human motor control performance and kinematic synergies in upper limb prosthetics,” in Eng. Med. Biol. Conf., pp. 3194 – 3197, 2018.
  6. R. Garcia-Rosas, Y. Tan, D. Oetomo, C. Manzie, and P. Choong, “Personalized Online Adaptation of Kinematic Synergies for Human-Prosthesis Interfaces,” IEEE Trans. Cybern., pp. 1–15, 2019.
  7. A. Akhtar, N. Aghasadeghi, L. Hargrove, and T. Bretl, “Estimation of distal arm joint angles from EMG and shoulder orientation for transhumeral prostheses,” J. Electromyogr. Kinesiol., vol. 35, pp. 86–94, 2017.
  8. D. Blana, T. Kyriacou, J. M. Lambrecht, and E. K. Chadwick, “Feasibility of using combined EMG and kinematic signals for prosthesis control: A simulation study using a virtual reality environment,” J. Electromyogr. Kinesiol., vol. 29, pp. 21–27, 2016.
  9. N. A. Alshammary, D. A. Bennett, and M. Goldfarb, “Synergistic elbow control for a myoelectric transhumeral prosthesis,” IEEE Trans. Neural Syst. Rehabil. Eng., vol. 26, no. 2, pp. 468–476, 2018.
  10. S. Li, X. Zhang, and J. D. Webb, “3-D-Gaze-Based Robotic Grasping Through Mimicking Human Visuomotor Function for People with Motion Impairments,” IEEE Trans. Biomed. Eng., vol. 64, no. 12, pp. 2824–2835, 2017.
  11. D. Whitney, “Resolved Motion Rate Control of Manipulators and Human Prostheses,” IEEE Trans. Man Mach. Syst., vol. 10, pp. 47–53, jun 1969.
  12. R. M. Murray, Z. Li, and S. S. Sastry, A Mathematical Introduction to Robotic Manipulation, vol. 29. CRC Press, 1994.
  13. M. Merad, E. de Montalivet, A. Roby-Brami, and N. Jarrasse, “Intuitive prosthetic control using upper limb inter-joint coordinations and IMU-based shoulder angles measurement: A pilot study,” in Int. Conf. Intell. Robot. Syst., pp. 5677–5682, 2016.
  14. A. Fougner, O. Stavdahl, P. J. Kyberd, Y. G. Losier, and P. A. Parker, “Control of upper limb prostheses: Terminology and proportional myoelectric control - A review,” IEEE Trans. Neural Syst. Rehabil. Eng., vol. 20, no. 5, pp. 663–677, 2012.
  15. E. A. Biddiss and T. T. Chau, “Upper limb prosthesis use and abandonment: A survey of the last 25 years,” Prosthet. Orthot. Int., vol. 31, no. 3, pp. 236–257, 2007.
  16. D. C. Niehorster, L. Li, and M. Lappe, “The accuracy and precision of position and orientation tracking in the HTC vive virtual reality system for scientific research,” Iperception., vol. 8, no. 3, pp. 1–23, 2017.
  17. D. A. Kistemaker, J. D. Wong, and P. L. Gribble, “The cost of moving optimally: kinematic path selection,” J. Neurophysiol., vol. 112, no. 8, pp. 1815–1824, 2014.
  18. S. Balasubramanian, R. Colombo, I. Sterpi, V. Sanguineti, and E. Burdet, “Robotic assessment of upper limb motor function after stroke,” 2012.
  19. A. De Los Reyes-Guzmán, I. Dimbwadyo-Terrer, F. Trincado-Alonso, F. Monasterio-Huelin, D. Torricelli, and A. Gil-Agudo, “Quantitative assessment based on kinematic measures of functional impairments during upper extremity movements: A review,” Clin. Biomech., vol. 29, no. 7, pp. 719–727, 2014.
  20. S. Balasubramanian, A. Melendez-Calderon, and E. Burdet, “A robust and sensitive metric for quantifying movement smoothness,” IEEE Trans. Biomed. Eng., vol. 59, no. 8, pp. 2126–2136, 2012.
  21. W. G. Darling and J. D. Cooke, “Changes in the variability of movement trajectories with practice,” J. Mot. Behav., vol. 19, no. 3, pp. 291–309, 1987.
  22. N. Yang, M. Zhang, C. Huang, and D. Jin, “Synergic analysis of upper limb target reaching movements,” J. Biomech., vol. 35, pp. 739–746, 2002.
Comments 0
Request Comment
You are adding the first comment!
How to quickly get a good reply:
  • Give credit where it’s due by listing out the positive aspects of a paper before getting into which changes should be made.
  • Be specific in your critique, and provide supporting evidence with appropriate references to substantiate general statements.
  • Your comment should inspire ideas to flow and help the author improves the paper.

The better we are at sharing our knowledge with each other, the faster we move forward.
""
The feedback must be of minimum 40 characters and the title a minimum of 5 characters
   
Add comment
Cancel
Loading ...
408794
This is a comment super asjknd jkasnjk adsnkj
Upvote
Downvote
""
The feedback must be of minumum 40 characters
The feedback must be of minumum 40 characters
Submit
Cancel

You are asking your first question!
How to quickly get a good answer:
  • Keep your question short and to the point
  • Check for grammar or spelling errors.
  • Phrase it like a question
Test
Test description