Anna W. Roe, Director & Professor
Interdisciplinary Institute of Neuroscience and Technology
School of Medicine
268 Kaixuan Rd
Hangzhou, Zhejiang Province, China
Brain Neuromorphics Needs An Architectural Framework.
Scientists and engineers have long drawn inspiration from the biological world to understand how architecture gives rise to function. To learn how to fly, study the architecture of bird and insect wings . To build a master swimmer, study the architecture of fish and amphibian neuromuscular oscillators . In the same vein, to understand intelligence, study the architecture of human and nonhuman primate brains. This last endeavor (Neuromorphics or Neuromorphic Computing), has generated ‘smart machines’ that can mimic perception and motor behavior, and have been modelled on the currency of brain function, neuronal spike firing [3,4]. Such approaches have driven the development of new computing architectures that overcome von Neumann bottlenecks, GPUs that accelerate via mass parallelism, in-memory processors, and implementation of attractor networks and finite state machines . Year-by-year, we see accelerations in benchmark performance, expansion of hardware and software technology, and computational deep neural network sophistication . However, despite these breathtaking advances, many of the basic functions of intelligent systems–rapid and efficient memory access, behaviorally targeted resource allocation, on-the-fly response to ever-changing contexts, and energy efficient computation–remain fundamentally out of reach.
Here, I highlight a potentially game-changing avenue of investigation. While neuronal spikes are the messengers of information, understanding the brain’s architecture (within which these neuronal messages flow) establishes key constraints that reduce, simplify, and systematize, thereby providing a formal framework for possible solutions. The focus shifts to studying brain architecture constrained networks of submillimeter nodes. Following a brief summary of the unique submillimeter organization of human and nonhuman primate brains, I describe an new technology that enables study of brain circuits at submillimeter scale and discuss the potential impact of this work on neuromorphic intelligence. To my knowledge, no current neuromorphic intelligence effort encapsulates such a concept and approach.
Networks in the Primate Brain Are Based in Submillimeter Functional Architecture.
There are unique aspects of primate brains that make them distinct from brains of mice or birds. A very fundamental aspect is its submillimeter scale architecture. As established by Nobel Laureates Hubel and Wiesel and primate developmental scientist Pasko Rakic, the primate cerebral cortex contains genetically derived arrays of fundamental submillimeter nodes (‘columns’) [7,8]. These arrays contain systematically mapped parameters that can be viewed as variables in which the nodes represent values of the parameter space . Activation of a specific nodal network is in essence activating a specific behavior. (For example, ‘Picking up a red apple and moving it towards my mouth makes me smile’ might activate one value in each of the following parameter maps: In visual cortex, the hue ‘red’ in color space, the ‘round’ curvature in shape space, ‘moving to the right’ in direction of motion space, and ‘towards me’ from near to far values in depth space; in motor cortex, digit flexor node in flexor to extensor values in hand grasp space; in limbic cortex, motor smile node in face gesture space.) A behavior would be defined by the set of nodes, one node selected from each parameter map; the total collection of such nodes across the brain (e.g. 50-100 values) would comprise a network that defines the behavior. While the topographic organization of nodal maps (sensory, motor, cognitive, and limbic maps) have been well described in many cortical and subcortical areas, little is understood about the connections between these nodal information arrays.
A Novel Approach for Establishing a Google Map of ‘Mesoscale’ Networks in Primate Brains.
The importance of understanding connections in the brain has been underscored by the huge investment in the US, Europe, and Asia in the mapping of brain ‘connectomes’ (map of all the connections in the brain). While these efforts provided great advances in our understanding of brain structure and function in the human population, the studies were based on data samples (MRI voxels) which were 2-3mm in size, thus failing to capture the submillimeter nature of information representation. To address this need, over the past few years, our group developed a technology (termed ‘infrared neural stimulation in ultrahigh field magnetic resonance imaging’, or ‘INS-fMRI’, Science Adv. 2019, chosen as one of the top 10 medical technologies in China) . This is a brain network mapping method which uses pulsed near infrared (1875nm) stimulation of single brain nodes to activate functionally connected sites, the locations of which are mapped at high spatial resolution in ultrahigh field fMRI. The result of a single node stimulation is a brainwide nodal network (1 stimulation site : 1 brain network). The advantages of this method are that, for the first time, submillimeter nodal networks can be mapped with high precision (stimulates single node) at brainwide scale. Because it is conducted in vivo, many different sites can be stimulated within a single brain, producing architectural understanding of nodal networks. Furthermore, it is fast (network seen immediately on MRI console) and requires no time-consuming reconstruction (unlike anatomical tract tracing studies). The speed and systematic aspects of this methodology will provide the first primate ‘mesoscale’ functional connectome. This connectome database will be useful for multiple fields of investigation.
Findings and Implications.
This ongoing ‘mesoscale connectome’ project has already revealed previously unknown fundamental rules of primate brain network architecture.
(1) Brainwide mesoscale networks: INS-fMRI has revealed that single nodal networks in the primate brain link nodes across multiple brain areas, both cortical and subcortical; this is the first time networks encompassing sensory, motor, cognitive, and limbic areas of the brain have been revealed at submillimeter scale. [10,11]
(2) Behavioral networks are topographically constrained: Hebbian ‘use it or lose it’ rules have previously predicted the presence of organized topographic maps in the cortex [12,13]. An extension of these rules to brainwide circuits predicts that organized brain network topography will also result. Parallel to how statistics of natural scenes influence the establishment of cortical maps, human behaviors also have structure, with one behavior is commonly followed by another related behavior; this results in a systematic, Hebbian topographic mapping of behavioral networks. Such an idea is supported by our finding that networks of nodes are topographically arranged (i.e. a shift in the stimulated node leads to a shift in the connected nodes), thus extending organization of ‘brain maps’ to ‘brain network maps’. This result constitutes an important constraint regarding baseline connection architecture. At the neuronal level, it means the statistics of change in neuronal firing is statistically related to neighboring architecturally constrained behavioral networks.
(3) Behavioral networks are highly sparse: By counting the total number of nodes activated from stimulation of a single node, we find the number of nodes in each network is remarkably sparse. This raises implications for network conciseness, efficiency, and speed as well as is relevant for understanding brain neuroenergetics.
(4) A mathematics of brain connections: These findings shift our concept of brain connectivity from ‘area-to-area’ to a highly organized ‘2Darray-to-2Darray’ architecture . The mathematical challenge will be representing ‘2Darray-to-2Darray’ relationships amongst N cortical areas. Interestingly, we also find that nodal networks in the brain appear to be canonical in nature, as illustrated by repeated circuit motifs across different featural modalities (e.g. in visual and somatosensory cortex, in different visual feature modalities) . This suggests that there may be ‘elements’ (microcircuit motifs) and ‘axioms’ (rules governing the relationships between these elements). Mathematical expression of such relationships will lead to a systematic conceptualization of an intelligent architecture.
(5) Network dynamics: This is perhaps the most intriguing and the most challenging aspect of understanding brain function. We are in the process of examining the simultaneous effects of single node stimulation on modulation of network behavior and modulation of sensory behavior in behaving monkeys.
In sum, the future of neuromorphics can be aided by a mesoscale architectural view of the primate brain, a machine known to generate intelligent behavior. The understanding of this architecture has been advanced by modern mapping methods and will prove useful for the neuroscience community as a guide for other functional, anatomical, and behavioral studies. Other impacts may also come in the fields of brain-machine interface design by guiding targeted and multi-site interfaces, and guidance of circuit-based advances in precision and personalized medicine. For the field of neuromorphics, impact may come from the constraints offered by systematic description of behavioral network architecture.
- Thielicke W, Stamhuis EJ (2018) The effects of wingtwist in slow-speed flapping flight of birds: trading brute force against efficiency. Bioinspir Biomim. 13(5):056015. doi: 10.1088/1748-3190/aad5a3.
- Di Santo V, …, Lauder GV (2021) Convergence of undulatory swimming kinematics across a diversity of fishes. Proc Natl Acad Sci U S A. 118(49):e2113206118.
- Yang S, …, Loparo K (2022) BiCoSS: Toward Large-Scale Cognition Brain With Multigranular Neuromorphic Architecture. IEEE Trans Neural Netw Learn Syst. 33(7):2801-2815.
- Bartolozzi C, …, Donati E (2022) Embodied neuromorphic intelligence. Nat Commun. 13(1):1024.
- Wang F, …, Zhou P (2021) Multi-functional computing in-memory SRAM cells based on two-surface-channel MoS 2 iScience. 2021 Sep 16;24(10):103138.
- LeCun Y, …, Hinton G (2015)Deeplearning. 521(7553):436-44.
- Hubel DH, Wiesel TN (1977) Ferrier lecture. Functional architecture of macaque monkey visual cortex. Proc R Soc Lond B Biol Sci.198(1130):1-59.
- Rakic P,… Dominguez MM (2009) Decision by division: making cortical maps. Trends Neurosci 32(5):291-301.
- Hu J*, Song XM, Wang Q, Roe AW** (2020) Curvature domains in V4 of Macaque Monkey. Elife. 9:e57261. doi: 10.7554/eLife.57261.
- Xu AG, … Roe AW (2019) Focal infrared neural stimulation with high-field functional MRI: a rapid way to map mesoscale brain connections. Science Advances, 5(4):eaau7046;
- Shi S,… Roe AW (2021) Infrared neural stimulation with 7T fMRI: a rapid in vivo method for mapping cortical connections of primate amygdala. Neuroimage, 231, 117818.
- Hebb, D. O. (1949). The Organisation of Behaviour. New York, NY: John Wiley & Sons.
- Swindale NV (1998) Corticalorganization: modules, polymaps and mosaics. Curr Biol. 8(8):R270-3. doi: 10.1016/s0960-9822(98)70170-8.
- Roe AW (2019) Columnar connectome: towards a mathematics of brain function. Network Neuroscience 3(3):779-791.
- Hu JM, Roe AW (2022) Functionally specific and sparse domain-based micro-networks in monkey V1 and V2. Curr Biol, 32(13):2797-2809.e3.