Cortico-Striatal Circuits are a Key Component of Learning in brain-machine interface tasks

RESEARCH
Ryan Neely, Aaron Koralek, Vivek Athalye, Rui Costa, Jose Carmena

Establishing a functional link between the human nervous system and computer systems could enable a broad range of applications, from medical treatments to consumer-focused products. Brain-machine interface (BMI) technologies have shown early promise in restoring communication and movement capabilities to paralyzed individuals, and there remains a strong research as well as commercial interest in developing these technologies further. Many BMI systems work by measuring neural signals, and “decoding” these signals to produce activity in an artificial effector- for example, a computer cursor or robotic appendage. Successful decoding of these signals involves numerous hardware and software challenges. However, another important factor in designing successful BMI systems is understanding how brain circuits respond when their activity is used to control a new device or effector. Even BMI systems designed to replicate natural movements cannot currently monitor the full complement of neurons involved or replicate the feedback provided by a natural limb. As a result, some adaptation on the part of the users’ neural circuitry must occur when learning to control a BMI system. Indeed, work on this subject has shown that neural circuits undergo plastic and state-dependent changes when learning to control a BMI [1,2,3] Identifying general rules or mechanisms by which cortical neurons adapt their outputs to generate new or altered patterns could guide the design of future BMI systems, and also provide a unique window into how learning occurs in cortical circuits.

Although a large body of work in BMI research as focused on interfacing with motor systems in the brain, it remains possible to capture neural activity from most any brain region and utilize these signals as BMI control signal to accomplish a variety of tasks. Previous work has demonstrated that learning to control the activity of primary motor cortex (M1) neurons requires neuroplasticity in the synaptic connection between M1 neurons and neurons in the dorsal striatum [4]. The striatum is known to play an important role in motor learning and control: these circuits are disrupted in disorders such as Parkinson’s Disease that are characterized by abnormal movement patterns. However, much of the cortex, including numerous sensory and cognitive circuits, also project to the dorsal striatum [5]. We wondered if this connection was an important component of operant learning across a diverse range of cortical systems, regardless of function.

To test this question, we designed a BMI system that virtually re-routed activity from neurons in the primary visual cortex (V1) of rats and mice into a computer that translated this activity into an auditory tone. Typically, a major source of “bottom-up” input to V1 neurons comes from the visual system which is driven by light entering the eyes. However, other “top-down” systems can also influence the activity of these neurons- for example to amplify signals corresponding to salient features in the environment  [6]. We tested whether rats could operantly learn to control the activity of V1 neurons in order to generate a particular tone, which was rewarded with the delivery of a sweetened water drink (Figure 1). The transform used to generate tones from neural activity was arbitrary, and therefore animals were required to learn to generate the rewarded patterns in order to increase the occurrence of reward.

Figure 1: BMI training with V1 neurons.

Figure 1: BMI training with V1 neurons. a: Rats and mice were implanted with microwire electrode arrays that recorded spikes from the primary visual cortex. Neural activity was fed into a computer that used an arbitrary transform to generate a tone pitch that changed according to neural spikes in real time. Animals received a sweet liquid reward when they produced a particular target tone. b: Plot shows mean performance of 9 animals over 10 training days. Animals quickly learned to perform above change level (dashed lines). Solid lines show mean and S.E.M.; dashed lines show data from individual animals.

We observed that animals were able learn to generate the rewarded patterns, and surprisingly, that performance was similar in both a light and total dark behavioral chamber. This observation suggested that the driving signals behind the learned neural activity were internally generated- i.e. not dependent on external light. Furthermore, as animals became increasingly proficient at the task, activation of neurons in the dorso-medial striatum increased around the time of target tone generation. We also observed that spike-field coherence (a measure of functional connectivity) increased specifically between the BMI control neurons in V1 and the local field activity in the dorsomedial striatum. These data suggested that the cortex-dorsal striatum connection was involved in learning to control a BMI system using neurons in the visual cortex, which echoed similar studies showing the role of cortico-striatal circuits in motor learning [7,8,9].

Because animals were able to learn to modulate activity in the primary visual cortex, and this activity was associated with dorsal striatal activation, we further explored the role of the cortico-striatal circuit in learning a V1-driven brain-machine interface. Because mice are unable to see light in the far red range, we infected mice with a red-light triggered microbial opsin (Jaws) that allowed us to shut down neurons in the dorsal medial striatum (the area that receives input from the V1 control neurons) using an optical fiber implanted in the brains of the mice [10]. Using this method, we observed that mice were unable to learn to perform the BMI task when the neurons in the dorsomedial striatum were inactivated. Interestingly, mice who had already learned to perform the task were able to continue unimpaired when the striatum was inactivated after the learning had taken place.

Figure 2: Dorsal striatum activity is required to learn a neuroprosthetic task with visual cortex neurons.

Figure 2: Dorsal striatum activity is required to learn a neuroprosthetic task with visual cortex neurons. a: Spike-field coherence between spikes from V1 BMI neurons and dorsal striatum field potentials, time-locked to successful target hits, increased over the course of training. b: Inhibiting dorsomedial striatum neurons with the microbial opsin Jaws combined with red light (LED 50, red bar) prevented mice from learning the task, while GFP-infected control animals were unaffected. However, Jaws-expressing animals were able to learn over subsequent training days if the red light was inactive and the dorsal striatum was able to function normally.

These results suggest that the connection between the cortex and striatum is a key circuit by which cortical neurons learn to change their output in order to perform a novel neuroprosthetic task. This connection may be part of a larger reciprocal circuit that involves additional brain areas, and is able to re-train cortical outputs based on behavioral outcomes. The striatum is an input nucleus for a larger structure called the basal ganglia, which is crucial for processing reward [11]. This structure seems well-suited to guide the tuning of cortical activity in order to generate activity that best suits the animal in a given situation. This system may help to drive cortical plasticity as an animal adapts to new environmental conditions, and it may also be a key pathway involved in learning to control a brain-machine interface. As BMI systems expand and begin to tap into larger numbers of neurons and more diverse cortical circuits, understanding how the brain adapts to these increasingly complex systems can help researchers identify strategies to improve control and performance.

Acknowledgements

The results of this study were published in the March 2018 edition of Neuron:

Neely, R. M., Koralek, A. C., Athalye, V. R., Costa, R. M., & Carmena, J. M. (2018). Volitional Modulation of Primary Visual Cortex Activity Requires the Basal Ganglia. Neuron, 97(6), 1356-1368. https://doi.org/10.1016/j.neuron.2018.01.051

References

  1. Arduin, P.-J., Frégnac, Y., Shulz, D. E., & Ego-Stengel, V. (2013). “Master” neurons induced by operant conditioning in rat motor cortex during a brain-machine interface task. The Journal of Neuroscience : The Official Journal of the Society for Neuroscience, 33(19), 8308–8320. https://doi.org/10.1523/JNEUROSCI.2744-12.2013
  2. Ganguly, K., & Carmena, J. M. J. (2009). Emergence of a stable cortical map for neuroprosthetic control. PLoS Biology, 7(7), e1000153. https://doi.org/10.1371/journal.pbio.1000153
  3. Sadtler, P. T., Quick, K. M., Golub, M. D., Chase, S. M., Ryu, S. I., Tyler-Kabara, E. C., … Batista, A. P. (2014). Neural constraints on learning. Nature, 512(7515), 423–426. https://doi.org/10.1038/nature13665
  4. Koralek, A. C. A., Jin, X., II, J. L., Costa, R. R. M., Carmena, J. M. J., Long, J. D., … Carmena, J. M. J. (2012). Corticostriatal plasticity is necessary for learning intentional neuroprosthetic skills. Nature, 483(7389), 331–335. https://doi.org/10.1038/nature10845
  5. Hintiryan, H., Foster, N. N., Bowman, I., Bay, M., Song, M. Y., Gou, L., … Dong, H.-W. (2016). The mouse cortico-striatal projectome. Nature Neuroscience, 19(8), 1100. https://doi.org/10.1038/nn.4332
  6. Zhang, S., Xu, M., Kamigaki, T., Hoang Do, J. P., Chang, W.-C., Jenvay, S., … Dan, Y. (2014). Long-range and local circuits for top-down modulation of visual cortex processing. Science (New York, N.Y.), 345(6197), 660–665. https://doi.org/10.1126/science.1254126
  7. Hikosaka, O., Nakahara, H., Rand, M. K., Sakai, K., Lu, X., Nakamura, K., … Doya, K. (1999). Parallel neural networks for learning sequential procedures. Trends in Neurosciences, 22(10), 464–471. Retrieved from http://www.sciencedirect.com/science/article/pii/S0166223699014393
  8. Koralek, A. C. A., Costa, R. R. M., & Carmena, J. M. J. (2013). Temporally precise cell-specific coherence develops in corticostriatal networks during learning. Neuron, 79(5), 865–872. https://doi.org/10.1016/j.neuron.2013.06.047
  9. Yin, H. H., Ostlund, S. B., Knowlton, B. J., & Balleine, B. W. (2005). The role of the dorsomedial striatum in instrumental conditioning. European Journal of Neuroscience, 22(2), 513–523. https://doi.org/10.1111/j.1460-9568.2005.04218.x
  10. Chuong, A. S., Miri, M. L., Busskamp, V., Matthews, G. A. C., Acker, L. C., Sørensen, A. T., … Boyden, E. S. (2014). Noninvasive optical inhibition with a red-shifted microbial rhodopsin. Nature Neuroscience, 17(8), 1123–1129. https://doi.org/10.1038/nn.3752
  11. Redgrave, P., Rodriguez, M., Smith, Y., Rodriguez-Oroz, M. C., Lehericy, S., Bergman, H., … Obeso, J. A. (2010). Goal-directed and habitual control in the basal ganglia: implications for Parkinson’s disease. Nature Reviews. Neuroscience, 11(11), 760–772. https://doi.org/10.1038/nrn2915

About the Authors:

Ryan Neely, Ph.D., Helen Wills Neuroscience Institute, University of California, Berkeley

Aaron Koralek, Ph.D, Champalimaud Neuroscience Programme, Champalimaud Centre for the Unknown. http://costa-lab.org/

Vivek Athalye, Department of Electrical Engineering and Computer Sciences, University of California, Berkeley

Rui Costa, DVM, Ph.D., Department of Neuroscience, Zuckerman Mind Brain Behavior Institute, Columbia University. https://www.actingbrain.com/

Jose Carmena, Ph.D., Helen Wills Neuroscience Institute and Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, and Joint Graduate Group in Bioengineering UC Berkeley/UC San Francisco. Co-Chair, IEEE Brain Initiative. http://carmenalab.org/