RESEARCH
April 2019
Violante I.R., Leech R., Lorenz R.
Brain stimulation is an increasingly popular field, capturing the creativity of DIY brain hackers and neurotechnology enthusiasts. Among scientists and clinicians, non-invasive brain stimulation (NIBS) is seen as a tool to causally investigate brain-behaviour relations and a promising treatment for a variety of neurological and psychiatric disorders [1]. Here we introduce our novel approach, Neuroadaptive Bayesian Optimization, designed to tackle limitations associated with conventional applications of NIBS, and define individualised stimulation protocols to inform clinical applications with optimal efficacy.
The idea of using brain stimulation is not new. Reports on the use of electrical stimulation date back almost 2000 years, and they include the use of the torpedo fish’s electric discharge in the treatment of headaches and epilepsy [2]. However, the last decades have seen an exponential growth in the development and application of NIBS technologies [1]. This boom can be partly attributed to the fact that NIBS approaches do not require surgery, are well tolerated, have a good side effect profile and are easy to scale up due to the low costs involved [3]. Therefore, if effective, they have vast clinical potential, such as improving the cognitive impairments (e.g., remembering and learning new things or concentrating on a task) that result from brain injury or psychiatric conditions – a field with few currently available therapies.
For some time, the application of NIBS to improve cognitive performance has been controversial, particularly regarding the replicability of the effects reported across studies [4]. This issue is partly associated with the traditional application of NIBS, which involves defining the stimulation parameters (e.g. frequency, phase between scalp electrodes, intensity and location) ad hoc and testing them on a cohort of participants. Indeed, this approach exhibits two main limitations: 1) the effects of stimulation on brain function are hard to verify without multimodal NIBS-recording techniques; 2) the stimulation parameters may vary across subjects due to inter-individual variations in the neurobiology of the brain and pathology.
To tackle these issues, we have recently developed a framework that uses adaptive closed-loop neurostimulation to determine the optimal stimulation parameters for the individual: Neuroadaptive Bayesian Optimization (NBO). Our approach uses Bayesian optimization and real-time functional neuroimaging (summarized in Figure 1). The experiment starts with selecting a random combination of stimulation parameters (for example, frequency and phase) from the large stimulation parameter space (1). Those parameters are then applied to the subject in the magnetic resonance (MR) scanner (2). Whole-brain functional images are acquired and analysed in real-time (3). Information about the current brain state is extracted and compared to the pre-defined target brain state (4). This result is then fed into the machine-learning algorithm (5). Based on this, the algorithm chooses a new combination of stimulation parameters that optimises for the target brain state. This loop continues until the optimum stimulation parameters have been identified.
Figure 1. Neuroadaptive Bayesian Optimization (NBO)
Thus, NBO addresses the two main limitations of NIBS (verifiability and inter-individual variability). The first limitation is probably the easiest to tackle, and there is a growing literature demonstrating the potential of combining NIBS with functional neuroimaging approaches such as fMRI or electroencephalography (EEG) to provide information on brain function. Although, there are still challenges to be overcome regarding artefact removal when combining NIBS with electrophysiological techniques such as EEG/MEG, researchers can use fMRI to map the effects of NIBS on large-scale networks across the brain [5], as proposed in our approach.
The second limitation concerns the suboptimal nature of applying NIBS through a ‘one-size-fits-all’ approach. Indeed, variability has been identified as one of the major obstacles for the broad therapeutic use of NIBS [6]. Unfortunately, sources of inter-individual differences are vast, including neuroanatomy (e.g. skull thickness, grey matter orientation, white matter structure), neurochemistry (e.g. concentration of neurotransmitters, hormone levels), age, genetic traits, disease heterogeneity, as well as brain activity related to the task the participant is performing [7]. This means that, even when employing rigorous methodological standards, many of these variables cannot be feasibly determined beforehand. Even if we did have access to them, we do not currently have models that can integrate all this information to propose the best stimulation parameters for the individual. This is why closed-loop neuroadaptive stimulation approaches can be powerful, as they can identify the ideal parameters in loco, surpassing the need to determine how different stimulation parameters relate to inter-individual variability beforehand [7].
The machine learning algorithm that we employ, Bayesian optimization constitutes an active sampling technique. Active sampling is particularly useful for scenarios when we are interested in exploring a large space of possible experimental conditions but where the acquisition of appropriate data comes at a cost, either in terms of time, financial costs, data quality, subject comfort or the amount of stimulation that can safely be delivered in one session. Active sampling approaches are already making progress in the field of invasive brain stimulation using implanted devices [8].
Bayesian optimization is characterized by automatically choosing samples, from which it progressively learns in real-time. It can be understood as an iterative two-stage procedure that repeats in a closed-loop. The first stage is the data modelling stage, in which a probabilistic surrogate model is used to approximately estimate the objective functions (i.e., the relationship between stimulation parameters and the subject’s brain activation in response to these parameters). Commonly, non-parametric surrogate models such as Gaussian processes (GP) are employed due to their versatility and flexibility [9]. At any given iteration, GP regression is used to update the algorithm’s estimate of the objective function across the entire parameter space by using all available observations obtained up to that point. The second stage is the guided search stage, in which an acquisition function is used to propose a point in the parameter space from where to sample next (i.e., the stimulation parameters the subject will receive in the next iteration). This new observation will then be used to update the algorithm’s probabilistic surrogate model. The role of the acquisition function is to guide exploration of the parameter space for achieving its learning goal (i.e., finding the stimulation parameters that maximize a particular brain state – the target brain state) by evaluating the utility of each candidate point [10]. As such, the acquisition function must balance a trade-off between exploring the parameter space and exploiting the current set of parameters for which measurements have already been collected; this allows for an efficient and reliable search over an exhaustive parameter space. There are different acquisition functions available and the specific choice of acquisition function determines how rapidly the algorithm transitions from exploration to exploitation [11].
Those interested in combining NBO with brain stimulation should however reflect upon some of the challenges they might encounter. Two of the most prominent challenges are inherently associated with the complexities of studying a non-stationary system like the human brain. Firstly, stimulation effects are brain-state dependent [5], which means that the choice of the task that participants perform during the optimisation is critical. In our opinion, the task should recruit the brain networks one wishes to target, be engaging so fatigue is minimised, and avoid learning effects that would result in targeting different processes across the experiment. Secondly, there is still a lot to learn about the relationship between the duration of brain stimulation and its after-effects, such as brain plasticity. Although plasticity is desirable in clinical applications, it should be minimised until the optimal stimulation parameters have been identified.
Despite these limitations, we have recently performed several studies that show that NBO provides a powerful strategy to efficiently explore many more experimental conditions than currently possible with standard methodology: 1) NBO identified tasks that reliably isolate the activity of frontoparietal networks, from a large space of possible behavioural tasks [12]; 2) our first pilot study demonstrated the technical feasibility of combining NBO and NIBS with real-time fMRI [13]; 3) NBO can be combined with transcranial alternating current stimulation (tACS), a form of NIBS, to identify frequencies and phases that elicit phosphene perception (illusory flash like visual percepts) in a manner that would be intractable with conventional approaches (83 vs 5 minutes to identify the optimum frequency and phase parameters that elicit maximum phosphene perception for each participant) [14].
By harnessing the knowledge from these studies, we are confident NBO will become a valid framework capable of overcoming the limitations of conventional NIBS studies and providing personalised interventions to inform clinical applications with optimal efficacy.
References
- Polania, R., et al. Studying and modifying brain function with non-invasive brain stimulation. Nat Neurosci. 2018;21(2):174-187. DOI: 10.1038/s41593-017-0054-4
- Priori, A. Brain polarization in humans: a reappraisal of an old tool for prolonged non-invasive modulation of brain excitability. Clin Neurophysiol. 2003;114(4):589-595. DOI:
- Brunoni, A.R., et al. Clinical research with transcranial direct current stimulation (tDCS): challenges and future directions. Brain Stimul. 2012;5(3):175-195. DOI: 10.1016/j.brs.2011.03.002
- Horvath, J.C., et al. Quantitative Review Finds No Evidence of Cognitive Effects in Healthy Populations From Single-session Transcranial Direct Current Stimulation (tDCS). Brain Stimul. 2015;8(3):535-550. DOI: 10.1016/j.brs.2015.01.400
- Violante, I.R., et al. Externally induced frontoparietal synchronization modulates network dynamics and enhances working memory performance. Elife. 2017;6(DOI: 10.7554/eLife.22001
- Ziemann, U., et al. Inter-subject and Inter-session Variability of Plasticity Induction by Non-invasive Brain Stimulation: Boon or Bane? Brain Stimul. 2015;8(3):662-663. DOI: 10.1016/j.brs.2015.01.409
- Li, L.M., et al. The contribution of interindividual factors to variability of response in transcranial direct current stimulation studies. Front Cell Neurosci. 2015;9(181. DOI: 10.3389/fncel.2015.00181
- Rosin, B., et al. Closed-loop deep brain stimulation is superior in ameliorating parkinsonism. Neuron. 2011;72(2):370-384. DOI: 10.1016/j.neuron.2011.08.023
- Rasmussen, C.E., et al. (2006). Gaussian Processes for Machine Learning, (MIT Press).
- Shahriari, B., et al. Taking the Human Out of the Loop: A Review of Bayesian Optimization. Proceedings of the IEEE. 2016;104(1):148-175. DOI: 10.1109/JPROC.2015.2494218
- Lorenz, R., et al. Neuroadaptive Bayesian Optimization and Hypothesis Testing. Trends Cogn Sci. 2017;21(3):155-167. DOI: 10.1016/j.tics.2017.01.006
- Lorenz, R., et al. Dissociating frontoparietal brain networks with neuroadaptive Bayesian optimization. Nat Commun. 2018;9(1):1227. DOI: 10.1038/s41467-018-03657-3
- Lorenz, R., et al. (2016). Towards tailoring non-invasive brain stimulation using real-time fMRI and Bayesian optimization. In arXiv e-prints.
- Lorenz, R., et al. Assessing tACS-induced phosphene perception using closed-loop Bayesian optimization. bioRxiv. 2017. DOI: 10.1101/150086
About the Authors
Ines R. Violante is a lecturer in Psychological Neuroscience at the University of Surrey. She received a BSc in Biochemistry (2007) and PhD in Biomedical Sciences (2012) from the University of Coimbra, Portugal. She was a Sir Henry Wellcome Postdoctoral Fellow in Imperial College London and University College London (2014-2017). Her research focuses on understanding the role of brain networks on cognitive functions and the use of brain stimulation as a neuromodulator. She is developing the framework to apply Neuroadaptive Bayesian Optimization for non-invasive brain stimulation.
Robert Leech is Professor of neuroimaging analytics at King’s College London. He is a neuroscientist and computer scientist. His research is inherently multi-disciplinary touching on neuroimaging, cognitive psychology and computational modelling; he has collaborated with clinicians working in neurology, psychiatry, hepatology, endocrinology and infectious diseases.
Romy Lorenz is a cognitive neuroscientist with a multidisciplinary background in psychology and biomedical engineering. Currently, she is a Sir Henry Wellcome Postdoctoral Fellow at the University of Cambridge and the Max Planck Institute for Human Cognitive and Brain Science. Her research interest lies in developing brain-computer interfaces (BCIs) and artificial intelligence (AI) as a new experimental paradigm in cognitive neuroscience; allowing her to address research questions that have been historically challenging to tackle with conventional methods. She received her PhD from Imperial College London in 2017, for which she developed Neuroadaptive Bayesian Optimization (or “The AI Neuroscientist”).
