From counting the number of people in a room to reading this sentence in your head, we are constantly harnessing our ability for inner speech. But what happens when scientists make that internal monologue external? In a paper published today in Cell, researchers describe using brain implants and new data analysis techniques to isolate the neural signals associated with the inner speech of four people with movement disorders that limit their speech, creating a brain-computer interface (BCI) that can vocalize these inner thoughts. The team also explored ways to prevent the system from blurting out internal thoughts a person might want to keep to themselves.
The paper is “impactful” in elucidating how inner speech is produced, says computer scientist Vikash Gilja, who studies speech decoding and is chief scientific officer of the BCI company Paradromics. “This is a paper my whole team is going to pay attention to.”
Over the past decade, engineers have created increasingly sophisticated computer systems that generate words from the brain activity of people paralyzed by strokes or neurological conditions such as amyotrophic lateral sclerosis (ALS). A few months ago, researchers at the University of California, Davis unveiled a device that could decode speech with startling accuracy and speed from electrodes implanted in the motor cortex of a man unable to speak intelligibly because of ALS.
To power this machine, the man had to try as hard as possible to form words with his mouth—to induce signals of “attempted speech” that scientists could pick up from the electrodes. In similar experiments performed at Stanford University, electrical engineer Erin Kunz heard from participants that trying to vocalize their thoughts was tiring and uncomfortable.
People find generating inner speech less stressful, but researchers had previously decoded only a small repertoire of possible words from those brain signals. Kunz and her team sought to expand on that work. First, the researchers analyzed recordings from electrodes that had been implanted in the brains of four people with movement disorders, in a part of the motor cortex known to be involved in attempted speech. The researchers looked for an overlap in the patterns of brain activity involved in two situations: when participants tried to read words aloud and when they simply thought the words to themselves. The team found that neurons in this region were active during both attempted and inner speech, although inner speech produced lower amplitude signals.
Focusing on the areas associated with inner speech, Kunz and her colleagues developed an algorithm that could translate the signals into audio. They trained their model by asking participants to “internally say” words from a 125,000-word vocabulary bank while recording neural activity. Then, the participants were told to imagine saying sentences using those words. The model could decode the sentences based on inner speech in real time in real time, with an error rate from 26% to 54%—the highest accuracy for decoding inner speech reported to date.