Customize your JAMA Network experience by selecting one or more topics from the list below.
Writing in Nature, researchers at the University of California, San Francisco (UCSF) described a system that synthesizes intelligible speech from brain activity recorded during speaking or mimed speaking. Although their study involved people without speech impairments, the technology could be a stepping stone toward restoring communication for people with stroke, amyotrophic lateral sclerosis, or other neurological disorders that impede speech, the researchers said.
Investigators recorded neural activity patterns within brain regions controlling lip, tongue, larynx, and jaw movements during speech in 5 volunteers at the UCSF Epilepsy Center using a high-density intracranial electrode array placed in preparation for neurosurgical treatment. An artificial neural network translated the patterns into computer simulations of the participants’ vocal tract movements, which were then decoded to produce synthesized speech.
A separate group of volunteers transcribed the synthesized sentences by choosing from a pool of 25 or 50 words that included both target and random words. The median word error rates from transcribed sentences was 31% with the smaller set of words and 53% with the larger set. The system was also able to synthesize sentences from mimed speech, although with poorer performance, demonstrating “that it is possible to decode important spectral features of speech that were never audibly uttered,” the authors wrote.
The proof-of-concept work is part of the broader field of brain-computer interfaces being designed to restore communication and movement in people with paralysis. In the future, “providing a closed-loop brain-computer interface to a paralyzed participant may allow them to learn to neurally control a virtual vocal tract,” said Gopala Anumanchipalli, PhD, a speech scientist at UCSF and co–lead author of the study.
Abbasi J. Synthesizing Speech From Brain Activity. JAMA. 2019;321(22):2155. doi:10.1001/jama.2019.7064
Create a personal account or sign in to: