The Potential of Neuroimaging-Guided Sensorimotor Rehabilitation

James Sulzer¹ and Roger Gassert², IEEE Senior Member
¹ Department of Mechanical Engineering, University of Texas at Austin
² Department of Health Sciences and Technology, ETH Zurich

Stroke, caused by a cerebrovascular lesion, is one of the most debilitating diseases in the world. While physical and occupational therapy play an important role in the rehabilitation process, we are still unable to determine effective treatment strategies for the reduction of stroke-related impairments. It appears that reducing impairments after stroke may be mostly spontaneous and that therapy primarily supports compensation [1]. Despite the source of the injury in the brain, treatment strategies are only at the limb level. The focus on the limbs while brain reorganization goes unmonitored could controversially result in compensatory neuroplasticity that limits recovery, for example, increased reliance on the healthy hemisphere during hand movement [2]. In this way, conventional therapy is somewhat akin to banging a TV set with poor reception in the hope that it will improve – we know the problem is inside the box, but we lack the tools and knowledge to intervene. As we learn more about how neural changes underlie recovery, we can begin to monitor activity at the source of the problem in addition to the limbs, followed by more effective interventions in the form of neuroimaging-guided neurotherapy.

For decades, real-time neuroimaging and neural recording has enabled the use of brain-machine (BMI) and brain-computer interfaces (BCI) in neuroprostheses. Some clinical examples include controlling robotic prostheses with signals recorded from intracortical electrodes [3], or using those signals to drive functional electrical stimulation of related muscles [4]. Non-invasive imaging such as electroencephalograpy (EEG) and functional near infrared spectroscopy (fNIRS) have been used in numerous applications for control and communication, including web browsers, spellers, and games (for review see [5]). While BCIs for assistive applications have soared in popularity, their therapeutic applications have garnered increasing attention on their own.

A potential advance in neurotherapy comes from using functional magnetic resonance imaging (fMRI) as a therapeutic brain-computer interface, a procedure known as fMRI neurofeedback. Neurofeedback using fMRI typically presents visual feedback of activity in a prespecified brain region or set of regions to the participant to help him or her gain self-control of the activity in that circuit. Although the mechanism of neuroplasticity underlying neurofeedback is not well-known, it is hypothesized that repeated self-activation of neural circuits will result in long-term potentiation (or depression) associated with neuroplasticity. While considered a byproduct of BCIs, this targeted neuroplasticity is the goal of neurofeedback. EEG-based neurofeedback began in the late 1950’s, whereas fMRI more recently took the stage, offering the advantage of more physiologically specific feedback (for review, see [6]). For example, by isolating activity from the primary motor cortex in healthy participants, we observed a link between the ability to self-regulate motor cortical activity and improvements in fine motor control [7].

The potential for fMRI neurofeedback as a therapeutic tool is enormous because it allows people to target neuroplasticity internally across the whole brain on the level of millimeters without surgery or drugs. For instance, one seminal study showed that fine patterns of neural activity in early visual cortex corresponding to a grating orientation could be self-regulated, and this control resulted in improvements in sensitivity to the specific grating [8]. The application of fMRI neurofeedback towards stroke rehabilitation remains almost entirely unexplored. While cortical neurofeedback can be achieved in different imaging modalities, techniques for rehabilitation unique to fMRI involve deep brain imaging such as neurofeedback of thalamo-cortical connectivity [9] or the dopaminergic reward system [10].

The success of neurofeedback will in part be limited by our model of neurorecovery following injury. Recent work is beginning to clarify these mechanisms, for instance, how activity shifts in different brain regions [2] or the connectivity between them [11]. We have just begun to discover the array of mechanisms of functional recovery we can infer using fMRI, all of which are potential neuromodulation targets with neurofeedback. However, our ability to understand neural recovery is not only affected by the measurement system, but also the interventions.

Robotics is an excellent tool for well-controlled and reproducible experimental interactions with and manipulations of the sensorimotor system, both in terms of movement assessment and providing somatosensory and motor stimuli. However, the MRI environment greatly restricts participant movement and use of ferrous materials, and is highly sensitive to electromagnetic interference, making the use of robotic systems challenging. We have made pioneering contributions to the field of neuroscience robotics, proposing a number of approaches for dynamic human-robot interaction within the MRI environment during neuroimaging (some examples shown in Fig. 1) and their application in neuroscience. These rely on hydrostatic and cable transmissions, as well as ultrasonic and shielded electromagnetic actuation, coupled with flexible structures that allow measurement of interaction forces via optical fiber based reflected light intensity measurement [12]. With the advent of this new technology, we can now obtain a more detailed characterization of neural activation in response to active and passive movements as well as haptic stimuli than ever before.

Fig. 1 fMRI-compatible finger interfaces.

Fig. 1 fMRI-compatible finger interfaces. Upper panel: finger interface used in [13] with hydrostatic transmission (electromagnetic actuator located outside the magnet room) and fiber optic force sensor (left). The device allows linear displacement of the index finger in opposition to the thumb, and can be flexibly placed at the entrance of the scanner bore (right). Lower panel: high-performance haptic interface with adjustable cable transmission driven by shielded DC actuators located at end of scanner bed [14] which will be used in follow-up studies.

To illustrate this concept, our recent study used a robotic finger interface actuated via hydrostatic transmission to learn how sensorimotor brain regions respond to passive forefinger motion [13]. We developed a quantitative stimulus-response relationship of the degree of forefinger movement amplitude and velocity to sensorimotor activation, while controlling for interaction force. In two independent experiments with 41 healthy participants, we found that sensorimotor activity increased linearly with movement velocity, but, surprisingly, decreased non-linearly with amplitude (Fig. 2). With the ability to now characterize normal human responses to sensorimotor stimuli, we can begin to develop a template of healthy activity to compare with those who have suffered neurological injury for a detailed diagnosis. Being able to observe the problem is the first step towards addressing it.

Fig. 2 BOLD signal in sensorimotor regions

Fig. 2 BOLD signal in sensorimotor regions reveals differential encoding of passive forefinger movement velocity and amplitude. The index finger was moved against the thumb using the MRI-compatible robot presented in Fig. 1. While sensorimotor activity increased linearly with movement velocity (upper panel), it decreased non-linearly with amplitude (lower panel). S1/M1: primary somatosensory/motor cortex; SMA: supplementary motor area; S2: secondary somatosensory cortex; Amax: maximal aperture in index finger/thumb opposition. Figure adapted from [13].

The future of rehabilitation after stroke may lie in the fusion of real-time neuroimaging and robotics for well-controlled, targeted neurotherapy. Rehabilitation robots provide a standardized training environment with continuously adapting physical support tailored to the individual patient. This allows an increase of both therapy intensity and dose while reducing the burden on therapists, with comparable efficacy to conventional therapy [15]. This concept of a therapeutic Brain-Robot Interface (BRI) has already been explored with different neuroimaging modalities. For example, magnetoencephalography (MEG) neurofeedback research pioneered the use of the sensorimotor rhythms to control the opening and closing of a hand orthosis for stroke rehabilitation [16]. This paradigm developed into a randomized, controlled EEG-based study showing modest improvements in motor function after stroke when applied in addition to conventional training [17].

For the purpose of sensorimotor rehabilitation, robotic stimuli could be adapted using online feedback of brain activity to ensure that the brain activates in the best possible way for recovery. This approach inverts the conventional BRI for targeted neurotherapy, in the form of a Robot-Brain Interface (RBI), building on recent approaches in non-sensorimotor, non-robotic adaptive neurofeedback [18, 19]. Our study examining the neural correlates of forefinger kinematics [13] is an initial step in this direction, as it presents a model that could be used to drive activation of sensorimotor networks to a desired activation level through the adaptation of movement kinematics. However, there are fundamental challenges to face before this future can be realized. We need a quantitative understanding of the brain-behavior relationship and the neuroplastic effects of the RBI. Future research will need to further characterize how these relationships change with recovery following stroke. Finally, while fMRI is currently the best neuroimaging modality we have, its potential for clinical translation is likely limited. The solution here will come from technological advances and novel approaches, such as functional ultrasound neuroimaging [20], which may enable the use of neurally-guided neurorehabilitation in the clinic and one day even at home.


  1. Huang, V.S. and Krakauer, J.W. Robotic neurorehabilitation: a computational motor learning perspective. Journal of NeuroEngineering and Rehabilitation. 2009;6:5. DOI: 1186/1743-0003-6-5
  2. Ward, N.S. and Cohen, L.G. Mechanisms underlying recovery of motor function after stroke. Archives of neurology. 2004;61(12):1844-8. DOI: 1001/archneur.61.12.1844
  3. Collinger, J.L. et al. High-performance neuroprosthetic control by an individual with tetraplegia. The Lancet. 2013;381(9866):557-564. DOI: 1016/S0140-6736(12)61816-9
  4. Ajiboye, A.B. et al. Restoration of reaching and grasping movements through brain-controlled muscle stimulation in a person with tetraplegia: a proof-of-concept demonstration. The Lancet. 2017;389(10081):1821-1830. DOI: 1016/S0140-6736(17)30601-3
  5. Chaudhary, U. et al. Brain–computer interfaces for communication and rehabilitation. Nature Reviews Neurology. 2016;12(9):513. DOI:
  6. Sulzer, J. et al. Real-time fMRI neurofeedback: progress and challenges. Neuroimage. 2013;76:386-99. DOI: 1016/j.neuroimage.2013.03.033
  7. Blefari, M.L. et al. Improvement in precision grip force control with self-modulation of primary motor cortex during motor imagery. Frontiers in behavioral neuroscience. 2015;9:18. DOI: 3389/fnbeh.2015.00018
  8. Shibata, K. et al. Perceptual learning incepted by decoded fMRI neurofeedback without stimulus presentation. Science. 2011;334(6061):1413-5. DOI: 1126/science.1212003
  9. Liew, S.L. et al. Improving motor corticothalamic communication after stroke using real-time fMRI connectivity-based neurofeedback. Neurorehabilitation and neural repair. 2016;30(7):671-675. DOI: 1177/1545968315619699
  10. Sulzer, J. et al. Neurofeedback-mediated self-regulation of the dopaminergic midbrain. Neuroimage. 2013;83:817-25. DOI: 1016/j.neuroimage.2013.05.115
  11. Grefkes, C. and Fink, G.R. Connectivity-based approaches in stroke and recovery of function. The Lancet Neurology. 2014;13(2):206-216. DOI: 1016/S1474-4422(13)70264-3
  12. Neuroscience Robotics. Rehabilitation Engineering Laboratory, ETH Zurich:
  13. Dueñas, J. et al. BOLD signal in sensorimotor regions reveals differential encoding of passive forefinger velocity and displacement amplitude. NeuroImage. 2018;173:332-40. DOI: 1016/j.neuroimage.2018.02.052
  14. Vigaru, B. et al. Design and evaluation of a cable-driven fMRI-compatible haptic interface to investigate precision grip control. IEEE Transactions on Haptics. 2016;9(1):20-32. DOI: 1109/TOH.2015.2485201
  15. Gassert, R. and Dietz. V. Rehabilitation Robots for the Treatment of Sensorimotor Deficits: A Neurophysiological Perspective. Journal of NeuroEngineering and Rehabilitation. 2018;15:46. DOI: 1186/s12984-018-0383-x
  16. Buch, E. et al. Think to move: a neuromagnetic brain-computer interface (BCI) system for chronic stroke. Stroke. 2008;39(3):910-917. DOI: 1161/STROKEAHA.107.505313
  17. Ramos-Murguialday, A. et al. Brain–machine interface in chronic stroke rehabilitation: a controlled study. Annals of neurology. 2013;74(1):100-108. DOI: 1002/ana.23879
  18. DeBettencourt, M.T. et al. Closed-loop training of attention with real-time brain imaging. Nature Neuroscience. 2015;18(3). DOI: 1038/nn.3940
  19. Lorenz, R. et al. The Automatic Neuroscientist: A framework for optimizing experimental design with closed-loop real-time fMRI. NeuroImage. 2016;129:320-34. DOI: 1016/j.neuroimage.2016.01.032
  20. Macé, E. et al. Functional ultrasound imaging of the brain. Nature methods. 2011;8(8):662. DOI: 1038/nmeth.1641

About the Authors:

James Sulzer, PhDJames Sulzer, PhD, is an Assistant Professor of Mechanical Engineering at the University of Texas at Austin. His research focuses on novel techniques, including neurofeedback, to understand and intervene in the recovery process following neurological insult such as stroke and spinal cord injury.


Roger GassertRoger Gassert, PhD, is an Associate Professor of Rehabilitation Engineering at the Department of Health Sciences and Technology at ETH Zurich. His research focuses on robotics technology, wearable sensors and non-invasive neuroimaging for the exploration, assessment and restoration of sensorimotor function. He is a senior member of the IEEE.