Sensational: continuous speech decoded from brain scans

Sensational: continuous speech decoded from brain scans

2. May 2023 0 By Horst Buchwald

Sensational: continuous speech decoded from brain scans

San Francisco, 5/2/2023

Researchers at the University of Texas at Austin have developed a non-intrusive approach that uses AI and brain scans to turn a person’s thoughts into sentences.

The “semantic decoder” can transcribe the essence of what a person has heard, seen or imagined and translate it into a stream of text, according to their research. Although the technology is still in its early stages, it has the potential to serve as the basis for non-invasive brain-computer interfaces that could help people with disabilities, such as paralysis, communicate intelligibly.

For the first time, scientists have been able to decode a person’s “continuous speech” from their brain recordings without the use of surgical implants.

The study involved three human participants who listened to narrative podcasts for 16 hours while inside an fMRI machine. The researchers mapped how the words were translated into responses in language-processing parts of the brain.

A large language model was used to match patterns in participants’ brain activity to heard phrases and words. Later, the participants heard a new story or imagined telling a story, and the decoder generated a corresponding text from their brain activity.

Although it’s not a word-for-word transcription, the decoder “was able to capture the essence of what the user heard,” said lead author Jerry Tang.

The study, published in Nature Neuroscience, emphasizes that “brain-computer interfaces should respect intellectual privacy.”

Hits: 2