Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Plug-and-play control of a brain–computer interface through neural map stabilization

Nature Biotechnology volume 39, pages 326–335 (2021)Cite this article

Subjects

Abstract

Brain–computer interfaces (BCIs) enable control of assistive devices in individuals with severe motor impairments. A limitation of BCIs that has hindered real-world adoption is poor long-term reliability and lengthy daily recalibration times. To develop methods that allow stable performance without recalibration, we used a 128-channel chronic electrocorticography (ECoG) implant in a paralyzed individual, which allowed stable monitoring of signals. We show that long-term closed-loop decoder adaptation, in which decoder weights are carried across sessions over multiple days, results in consolidation of a neural map and ‘plug-and-play’ control. In contrast, daily reinitialization led to degradation of performance with variable relearning. Consolidation also allowed the addition of control features over days, that is, long-term stacking of dimensions. Our results offer an approach for reliable, stable BCI control by leveraging the stability of ECoG interfaces and neural plasticity.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: ltCLDA drives performance improvements.
Fig. 2: Emergence of a refined decoder map for cursor control.
Fig. 3: Neural representations emerge and stabilize over multiple timescales.
Fig. 4: Long-term PnP and decoder reset.
Fig. 5: Stacking of two neural clickers.

Similar content being viewed by others

Data availability

The data that support the findings of this study are available on reasonable request from the corresponding author. The data are not publicly available because they contain information that might compromise the privacy of the research participant.

Code availability

The custom MATLAB code used for analyses is available upon request from the corresponding author.

References

  1. Schwartz, A. B. Cortical neural prosthetics. Annu. Rev. Neurosci. 27, 487–507 (2004).

    CAS  PubMed  Google Scholar 

  2. Taylor, D. M., Tillery, S. I. & Schwartz, A. B. Direct cortical control of 3D neuroprosthetic devices. Science 296, 1829–1832 (2002).

    CAS  PubMed  Google Scholar 

  3. Gilja, V. et al. A high-performance neural prosthesis enabled by control algorithm design. Nat. Neurosci. 15, 1752–1757 (2012).

    CAS  PubMed  PubMed Central  Google Scholar 

  4. Hochberg, L. R. et al. Neuronal ensemble control of prosthetic devices by a human with tetraplegia. Nature 442, 164–171 (2006).

    CAS  PubMed  Google Scholar 

  5. Carmena, J. M. et al. Learning to control a brain–machine interface for reaching and grasping by primates. PLoS Biol. 1, E42 (2003).

    PubMed  PubMed Central  Google Scholar 

  6. Degenhart, A. D. et al. Remapping cortical modulation for electrocorticographic brain–computer interfaces: a somatotopy-based approach in participants with upper-limb paralysis. J. Neural Eng. 15, 026021 (2018).

    PubMed  PubMed Central  Google Scholar 

  7. Pandarinath, C. et al. High performance communication by people with paralysis using an intracortical brain–computer interface. eLife 6, e18554 https://doi.org/10.7554/eLife.18554 (2017).

  8. Wang, W. et al. An electrocorticographic brain interface in an individual with tetraplegia. PLoS ONE 8, e55344 (2013).

    CAS  PubMed  PubMed Central  Google Scholar 

  9. Aflalo, T. et al. Neurophysiology. Decoding motor imagery from the posterior parietal cortex of a tetraplegic human. Science 348, 906–910 (2015).

    CAS  PubMed  PubMed Central  Google Scholar 

  10. Wolpaw, J. R. & McFarland, D. J. Control of a two-dimensional movement signal by a noninvasive brain–computer interface in humans. Proc. Natl Acad. Sci. USA 101, 17849–17854 (2004).

    CAS  PubMed  PubMed Central  Google Scholar 

  11. Sellers, E. W., Vaughan, T. M. & Wolpaw, J. R. A brain–computer interface for long-term independent home use. Amyotroph. Lateral Scler. 11, 449–455 (2010).

    PubMed  Google Scholar 

  12. Ajiboye, A. B. et al. Restoration of reaching and grasping movements through brain-controlled muscle stimulation in a person with tetraplegia: a proof-of-concept demonstration. Lancet 389, 1821–1830 (2017).

    PubMed  PubMed Central  Google Scholar 

  13. Benabid, A. L. et al. An exoskeleton controlled by an epidural wireless brain–machine interface in a tetraplegic patient: a proof-of-concept demonstration. Lancet Neurol. 18, https://doi.org/10.1016/S1474-4422(19)30321-7 (2019).

  14. Collinger, J. L. et al. High-performance neuroprosthetic control by an individual with tetraplegia. Lancet 381, 557–564 (2013).

    PubMed  PubMed Central  Google Scholar 

  15. Ganguly, K. & Carmena, J. M. Emergence of a stable cortical map for neuroprosthetic control. PLoS Biol. 7, e1000153 (2009).

    PubMed  PubMed Central  Google Scholar 

  16. Gulati, T., Ramanathan, D. S., Wong, C. C. & Ganguly, K. Reactivation of emergent task-related ensembles during slow-wave sleep after neuroprosthetic learning. Nat Neurosci. 17, 1107–1113 https://doi.org/10.1038/nn.3759 (2014).

  17. Flint, R. D., Scheid, M. R., Wright, Z. A., Solla, S. A. & Slutzky, M. W. Long-term stability of motor cortical activity: implications for brain–machine interfaces and optimal feedback control. J. Neurosci. 36, 3623–3632 (2016).

    CAS  PubMed  PubMed Central  Google Scholar 

  18. Dayan, E. & Cohen, L. G. Neuroplasticity subserving motor skill learning. Neuron 72, 443–454 (2011).

    CAS  PubMed  PubMed Central  Google Scholar 

  19. Wodlinger, B. et al. Ten-dimensional anthropomorphic arm control in a human brain–machine interface: difficulties, solutions and limitations. J. Neural Eng. 12, 016011 (2015).

    CAS  PubMed  Google Scholar 

  20. Wolpaw, J. R. et al. Independent home use of a brain–computer interface by people with amyotrophic lateral sclerosis. Neurology 91, e258–e267 (2018).

    PubMed  PubMed Central  Google Scholar 

  21. Ganguly, K., Dimitrov, D. F., Wallis, J. D. & Carmena, J. M. Reversible large-scale modification of cortical networks during neuroprosthetic control. Nat. Neurosci. 14, 662–667 (2011).

    CAS  PubMed  PubMed Central  Google Scholar 

  22. Farina, D. et al. The extraction of neural information from the surface EMG for the control of upper-limb prostheses: emerging avenues and challenges. IEEE Trans. Neural Syst. Rehabil. Eng. 22, 797–809 (2014).

    PubMed  Google Scholar 

  23. Liu, J., Sheng, X., Zhang, D., He, J. & Zhu, X. Reduced daily recalibration of myoelectric prosthesis classifiers based on domain adaptation. IEEE J. Biomed. Health Inform. 20, 166–176 (2016).

    PubMed  Google Scholar 

  24. Cordella, F. et al. Literature review on needs of upper limb prosthesis users. Front Neurosci. 10, 209 (2016).

    PubMed  PubMed Central  Google Scholar 

  25. Phillips, B. & Zhao, H. Predictors of assistive technology abandonment. Assist. Technol. 5, 36–45 (1993).

    CAS  PubMed  Google Scholar 

  26. Green, A. M. & Kalaska, J. F. Learning to move machines with the mind. Trends Neurosci. 34, 61–75 (2011).

    CAS  PubMed  Google Scholar 

  27. Chao, Z. C., Nagasaka, Y. & Fujii, N. Long-term asynchronous decoding of arm motion using electrocorticographic signals in monkeys. Front Neuroeng. 3, 3 (2010).

    PubMed  PubMed Central  Google Scholar 

  28. Degenhart, A. D. et al. Histological evaluation of a chronically-implanted electrocorticographic electrode grid in a non-human primate. J. Neural Eng. 13, 046019 (2016).

    PubMed  PubMed Central  Google Scholar 

  29. Leuthardt, E. C., Schalk, G., Wolpaw, J. R., Ojemann, J. G. & Moran, D. W. A brain–computer interface using electrocorticographic signals in humans. J. Neural Eng. 1, 63–71 (2004).

    PubMed  Google Scholar 

  30. Rouse, A. G., Williams, J. J., Wheeler, J. J. & Moran, D. W. Spatial coadaptation of cortical control columns in a micro-ECoG brain–computer interface. J. Neural Eng. 13, 056018 (2016).

    CAS  PubMed  Google Scholar 

  31. Chang, E. F. Towards large-scale, human-based, mesoscopic neurotechnologies. Neuron 86, 68–78 (2015).

    CAS  PubMed  PubMed Central  Google Scholar 

  32. Vansteensel, M. J. et al. Fully implanted brain–computer interface in a locked-in patient with ALS. N. Engl. J. Med. 375, 2060–2066 (2016).

    PubMed  PubMed Central  Google Scholar 

  33. Yanagisawa, T. et al. Electrocorticographic control of a prosthetic arm in paralyzed patients. Ann. Neurol. 71, 353–361 (2012).

    PubMed  Google Scholar 

  34. Orsborn, A. L. et al. Closed-loop decoder adaptation shapes neural plasticity for skillful neuroprosthetic control. Neuron 82, 1380–1393 (2014).

    CAS  PubMed  Google Scholar 

  35. Dangi, S., Orsborn, A. L., Moorman, H. G. & Carmena, J. M. Design and analysis of closed-loop decoder adaptation algorithms for brain–machine interfaces. Neural Comput. 25, 1693–1731 (2013).

    PubMed  Google Scholar 

  36. Shenoy, K. V. & Carmena, J. M. Combining decoder design and neural adaptation in brain–machine interfaces. Neuron 84, 665–680 (2014).

    CAS  PubMed  Google Scholar 

  37. Gulati, T., Guo, L., Ramanathan, D. S., Bodepudi, A. & Ganguly, K. Neural reactivations during sleep determine network credit assignment. Nat. Neurosci. 20, 1277–1284 (2017).

    CAS  PubMed  PubMed Central  Google Scholar 

  38. Georgopoulos, A. P., Schwartz, A. B. & Kettner, R. E. Neuronal population coding of movement direction. Science 233, 1416–1419 (1986).

    CAS  PubMed  Google Scholar 

  39. Ajemian, R., D’Ausilio, A., Moorman, H. & Bizzi, E. Why professional athletes need a prolonged period of warm-up and other peculiarities of human motor learning. J. Mot. Behav. 42, 381–388 (2010).

    PubMed  Google Scholar 

  40. Ajemian, R., D’Ausilio, A., Moorman, H. & Bizzi, E. A theory for how sensorimotor skills are learned and retained in noisy and nonstationary neural circuits. Proc. Natl Acad. Sci. USA 110, E5078–E5087 (2013).

    CAS  PubMed  PubMed Central  Google Scholar 

  41. Chestek, C. A. et al. Single-neuron stability during repeated reaching in macaque premotor cortex. J. Neurosci. 27, 10742–10750 (2007).

    CAS  PubMed  PubMed Central  Google Scholar 

  42. Stevenson, I. H. et al. Statistical assessment of the stability of neural movement representations. J. Neurophysiol. 106, 764–774 (2011).

    PubMed  PubMed Central  Google Scholar 

  43. Gallego, J. A., Perich, M. G., Chowdhury, R. H., Solla, S. A. & Miller, L. E. Long-term stability of cortical population dynamics underlying consistent behavior. Nat. Neurosci. 23, 260–270 (2020).

    CAS  PubMed  PubMed Central  Google Scholar 

  44. Murthy, V. N. & Fetz, E. E. Coherent 25- to 35-Hz oscillations in the sensorimotor cortex of awake behaving monkeys. Proc. Natl Acad. Sci. USA 89, 5670–5674 (1992).

    CAS  PubMed  PubMed Central  Google Scholar 

  45. Crone, N. E., Miglioretti, D. L., Gordon, B. & Lesser, R. P. Functional mapping of human sensorimotor cortex with electrocorticographic spectral analysis. II. Event-related synchronization in the gamma band. Brain 121(Pt 12), 2301–2315 (1998).

    PubMed  Google Scholar 

  46. Hall, T. M., de Carvalho, F. & Jackson, A. A common structure underlies low-frequency cortical dynamics in movement, sleep and sedation. Neuron 83, 1185–1199 (2014).

    CAS  PubMed  PubMed Central  Google Scholar 

  47. Ramanathan, D. S. et al. Low-frequency cortical activity is a neuromodulatory target that tracks recovery after stroke. Nat. Med. 24, 1257–1267 (2018).

    CAS  PubMed  PubMed Central  Google Scholar 

  48. Buzsaki, G., Anastassiou, C. A. & Koch, C. The origin of extracellular fields and currents—EEG, ECoG, LFP and spikes. Nat. Rev. Neurosci. 13, 407–420 (2012).

    CAS  PubMed  PubMed Central  Google Scholar 

  49. Lemke, S. M., Ramanathan, D. S., Guo, L., Won, S. J. & Ganguly, K. Emergent modular neural control drives coordinated motor actions. Nat. Neurosci. 22, 1122–1131 (2019).

    CAS  PubMed  PubMed Central  Google Scholar 

  50. Downey, J. E. et al. Blending of brain–machine interface and vision-guided autonomous robotics improves neuroprosthetic arm performance during grasping. J. Neuroeng. Rehabil. 13, 28–28 (2016).

    PubMed  PubMed Central  Google Scholar 

  51. Chestek, C. A. et al. Long-term stability of neural prosthetic control signals from silicon cortical arrays in rhesus macaque motor cortex. J. Neural Eng. 8, 045005 (2011).

    PubMed  PubMed Central  Google Scholar 

  52. Jarosiewicz, B. et al. Virtual typing by people with tetraplegia using a self-calibrating intracortical brain–computer interface. Sci. Transl. Med. 7, 313ra179 (2015).

    PubMed  PubMed Central  Google Scholar 

  53. Hochberg, L. R. et al. Reach and grasp by people with tetraplegia using a neurally controlled robotic arm. Nature 485, 372–375 (2012).

    CAS  PubMed  PubMed Central  Google Scholar 

  54. Moses, D. A., Leonard, M. K., Makin, J. G. & Chang, E. F. Real-time decoding of question-and-answer speech dialogue using human cortical activity. Nat. Commun. 10, 3096 (2019).

    PubMed  PubMed Central  Google Scholar 

  55. Brainard, D. H. The psychophysics toolbox. Spat. Vis. 10, 433–436 (1997).

    CAS  PubMed  Google Scholar 

  56. Kao, J. C., Nuyujukian, P., Ryu, S. I. & Shenoy, K. V. A high-performance neural prosthesis incorporating discrete state selection with hidden Markov models. IEEE Trans. Biomed. Eng. 64, 935–945 (2017).

    PubMed  Google Scholar 

  57. Fan, R., Chang, K., Hsieh, C., Wang, X. & Lin, C. Liblinear: a library for large linear classification. J. Mach. Learn. Res. 9, 1871–1875 (2008).

    Google Scholar 

  58. Heliot, R., Ganguly, K., Jimenez, J. & Carmena, J. M. Learning in closed-loop brain–machine interfaces: modeling and experimental validation. IEEE Trans. Syst. Man Cybern. B Cybern. 40, 1387–1397 (2010).

    PubMed  Google Scholar 

Download references

Acknowledgements

This work was funded by the National Institutes of Health (NIH) through the NIH Director’s New Innovator Award Program (grant no. 1 DP2 HD087955]. The development of the signal processing and decoding approach used in this study was supported by the Doris Duke Charitable Foundation (grant no. 2013101). The authors especially thank our participant ‘Bravo-1’ for his unwavering commitment to this project. We also thank C.C. Rodriguez for assisting with data collection and D.A. Moses, J.R. Liu and M. Dougherty for assistance with the clinical trial.

Author information

Author notes

  1. These authors contributed equally: Daniel B. Silversmith, Reza Abiri, Nicholas F. Hardy, Nikhilesh Natraj.

Authors and Affiliations

  1. Department of Neurology, University of California, San Francisco, San Francisco, CA, USA

    Daniel B. Silversmith, Reza Abiri, Nicholas F. Hardy, Nikhilesh Natraj, Adelyn Tu-Chan & Karunesh Ganguly

  2. Neurology and Rehabilitation Service, San Francisco Veterans Affairs Medical Center, San Francisco, CA, USA

    Daniel B. Silversmith, Reza Abiri, Nicholas F. Hardy, Nikhilesh Natraj, Adelyn Tu-Chan & Karunesh Ganguly

  3. Department of Neurosurgery, University of California, San Francisco, San Francisco, CA, USA

    Edward F. Chang

Authors
  1. Daniel B. Silversmith
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Reza Abiri
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nicholas F. Hardy
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nikhilesh Natraj
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Adelyn Tu-Chan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Edward F. Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Karunesh Ganguly
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

D.S. established the decoder and real-time setup for cursor control. D.S., R.A., N.F.H. and N.N. collected data. R.A. and D.S. performed data analysis. N.F.H. specifically conducted the analysis of ECoG power. N.N. established the decoder framework for clickers, designed and implemented the ltCLDA-BCI learning simulation and wrote supplementary note. A.T.C. conducted patient recruitment and patient care. E.F.C. performed the surgical implantation and participated in patient care. K.G. supervised all aspects of this study. D.S., R.A. and K.G. wrote the manuscript. All authors read and revised the manuscript.

Corresponding author

Correspondence to Karunesh Ganguly.

Ethics declarations

Competing interests

E.F.C., K.G., N.N. and A.T.C. receive some salary support from Facebook Reality Labs for a separate project on speech decoding. The funders had no role in study design, data collection, data analysis and the contents of this paper.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Figs. 1–12, Supplementary Table 1, Supplementary Video Legends 1 and 2 and Supplementary Note

Reporting Summary

Supplementary Video 1

Example of BCI cursor control during center-out task. This video shows performance during a fixed block after ltCLDA. The 8 targets are shown.

Supplementary Video 2

Example of point-and-click task using ECoG-based BCI. Neural control of the cursor during reach to an instructed target in a 4x4 grid, clicking on that target was a correct selection. The instructed target was illuminated green to indicate that it was the instructed target. After each trial, the next instructed target was randomly selected from the grid and immediately illuminated. Overall, for this example of block, the bitrate was calculated as 0.92.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Silversmith, D.B., Abiri, R., Hardy, N.F. et al. Plug-and-play control of a brain–computer interface through neural map stabilization. Nat Biotechnol 39, 326–335 (2021). https://doi.org/10.1038/s41587-020-0662-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41587-020-0662-5

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing