Please consider helping support the Arts & Humanities (at individual, university, city, state & federal levels) so they (SHTEAM) may grow together with STEM. "Brain in Violet" by Dr. Greg Dunn.
We conduct neuroscience, neuroengineering and translational research to better understand how the brain controls movement, and to design medical systems to assist people with paralysis. These medical systems are referred to as brain-machine interfaces (BMIs), brain-computer interfaces (BCIs) and intra-cortical neural prostheses. We conduct this research as part of our Neural Prosthetic Systems Lab (NPSL), which focuses on more basic systems and computational neuroscience and neuroengineering, and as part of our Neural Prosthetics Translational Lab (NPTL), which focuses on more translational systems and computational neuroscience and neuroengineering.
Our neuroscience research investigates the neural basis of movement preparation and generation using a combination of electro- / opto-physiological (e.g., chronic electrode-array recordings and optogenetic stimulation), behavioral, computational and theoretical techniques (e.g., dynamical systems, dimensionality reduction, single-trial neural analyses). For example, how do neurons in premotor (PMd) and primary motor (M1) cortex plan and guide reaching arm movements?
Our neuroengineering research investigates the design of high-performance and robust intra-cortical neural prostheses. These systems translate neural activity from the brain into control signals for prosthetic devices, which can assist people with paralysis by restoring lost motor functions. This work includes statistical signal processing, machine learning, and real-time system modeling and implementation. For example, how can we design motor prostheses with performance rivaling the natural arm, or communication prostheses rivaling the throughput of spoken language?
Our translational research, including an FDA pilot clinical trial (BrainGate2), are conducted as part of the the Neural Prosthetics Translational Laboratory (NPTL)
. For example, how do pre-clinical laboratory designs actually work with people with paralysis in real-world settings?
Figure 1. Various neural measurement and perturbation techniques, mathematical analysis methods and real-time closed-loop brain-computer interface (BCI) systems approaches advanced in order to better understand how populations of cortical neurons coordinate and cooperate to prepare, generate and control movement.
(a) NeuroPixel electrode array (with 960 electrodes, 384 simultaneously recordable) probe and multiplexing electronics (HHMI & IMEC; e.g., Dutta et al. IEDM 2019).
(b) Voltage vs. time traces from 184 (of 384) electodes from a NeuroPixel probe in an awake-behaving rhesus monkey, showing how neurons' action potentials are seen on multiple electrodes which facilitates high-quality spike sorting (e.g., Trautmann et al. Neuron 2019).
(c) Optogenetics in rhesus monkeys for increasing or decreasing activity in specific neurons (e.g., Diester et al. Nature Neuroscience 2011).
(d) 2-photon (GCaMP6f) optical recordings from reaching rhesus monkeys, with reach directional tuning (2D color wheel shows reach direction) of example neurons overlaid in color (Trautmann*, O'Shea*, Sun*, et al. bioRxiv 2019).
(e) These methods (and Utah arrays and U/V/S linear probes) allow information to be read out and written into the rhesus monkey brain.
(f) In both monkeys and in people, the number of action potentials in a short time period, across all measured neurons, constitutes a neural state and can be plotted in a neuron firing-rate space or, more generally, in a lower-dimensional space such as PCA, GPFA or dPCA (Churchland et al. J Neurosci 2006; Yu et al. NIPS 2006; Yu et al. J Neurophysiology 2006).
(g) This neural state evolves in time and forms a neural trajectory (Churchland*, Yu*, et al. Nature Neuroscience 2010).
(h) The neural population dynamics, formed by the underlying neural network circuitry, can be expressed as a neural flow field. The neural flow field express the system dynamics which can be modeled as a linear dynamical system (LDS) or non-linear dynamical system such as a recurrent neural network (RNN) (e.g., Shenoy, Sahani & Churchland Annual Review of Neuroscience 2013; Sussillo et al. Nature Neuroscience 2015; Vyas et al. Annual Review of Neuroscience 2020 in press). Neural states remain in a muscle-null space while preparing a reach, and then enter muscle-potent space to generate the movement (e.g., Kaufman et al. Nature Neuroscience 2014). These movement dynamics have a substantial time-limited rotatory component (Churchland*, Cunningham*, et al. Nature 2012).
(i) The initial conditions influence the neural trajectory (e.g., Afshar et al. Neuron 2011).
(j) A neural state starts at some initial-condition location, and then moves as dictated by the neural flow field (assuming no inputs).
(k) If the initial-condition location is perturbed, by changing the task or by electrical or optical stimulation, a different (perturbed) neural trajectory results (e.g., Ames et al. Neuron 2014).
(l) If the input to the dynamical system is changed, then the dynamics are changed, and cause the same initial-condition location to follow different neural trajectories (e.g., Mante*, Sussillo*, et al. Nature 2013).
(m) A Utah electrode array (Blackrock Microsystems Inc.), used in rhesus monkeys and in human clinical trials.
(n) Top to bottom: action potential (spike) rasters from one delayed-reach trial, spike clustering in PCA (or similar) space and sorting, and one example voltage vs. time trace.
(o) Two electrode array locations marked on a scaled-down 3D printed brain based on imaging from participant T5, with signals related to many/most body parts (Willett*, Deo*, et al. Cell 2020) and to speech production (Stavisky et al. eLife 2019).
(p) Illustration of a BCI that measures neural activity, translates this neural activity into movement control signals using a variety of statistical-signal processing decoding algorithms (e.g., Santhanam et al. Nature 2006; Gilja*, Nuyujukian*, et al. Nature Neuroscience 2012; Sussillo et al. Journal of Neural Engineering 2012; Nuyujukian et al. Journal of Neural Engineering 2014; Nuyujukian et al. IEEE Transactions on Biomedical Engineering 2015; Gilja*, Pandarinath* et al. Nature Medicine 2015; Kao et al. Nature Communications 2015; Sussillo*, Stavisky*, Kao*, et al. Nature Communications 2016; Nuyujukian et al. IEEE Proceedings 2017; Kao*, Nuyujukian*, et al. IEEE Transactions on Biomedical Engineering 2017; Pandarinath*, Nuyujukian*, et al. eLife 2017), and uses these control signals to guide robotic/prosthetic arms and hands, electrically stimulate paralyzed musculature, and guide computer cursors and click signals on general-purpose computer interfaces (e.g., Nuyujukian et al. PLoS 2018).