Using implants and artificial intelligence (AI) to convert brain impulses into speech is an intriguing and quickly expanding field of research and technology. This procedure combines neuroengineering, neurology, and advanced AI algorithms.
Radboud University and UMC Utrecht researchers have succeeded in converting brain impulses into audible speech. They were able to predict the words people wanted to utter with an accuracy of 92 to 100% by decoding signals from the brain using a mix of implants and AI. This month, their findings were published in the Journal of Neural Engineering.
The research indicates a promising development in the field of Brain-Computer Interfaces, according to lead author Julia Berezutskaya, researcher at Radboud University’s Donders Institute for Brain, Cognition and Behaviour and UMC Utrecht. Berezutskaya and colleagues at the UMC Utrecht and Radboud University used brain implants in patients with epilepsy to infer what people were saying.
Our goal is to predict entire sentences and paragraphs of what people are attempting to say based just on brain activity. More studies, more powerful implants, greater datasets, and advanced AI models will be required to get there.
Julia Berezutskaya
Bringing back voices
‘Ultimately, we intend to make this technology available to patients who are paralyzed and unable to communicate in a locked-in state,’ adds Berezutskaya. ‘These folks lose the ability to move their muscles and, as a result, speak. We can analyze brain activity and give them a voice again by constructing a brain-computer interface.’
The researchers invited non-paralyzed persons with temporary brain implants to utter a number of words out loud while their brain activity was measured for the experiment in their new study. ‘We were then able to create direct mapping between brain activity on the one hand and speech on the other,’ says Berezutskaya. We also used sophisticated artificial intelligence algorithms to directly translate brain activity into audible speech. That is, we could not only estimate what individuals were saying, but we could also instantly convert those words into intelligible, understandable sounds. Furthermore, the reconstructed speech sounded exactly like the real speaker in tone of voice and manner of speaking.’
Researchers all across the world are trying to figure out how to distinguish words and sentences in brain patterns. The researchers were able to reconstruct understandable speech from tiny samples, demonstrating that their models can uncover the complicated mapping between brain activity and speech even with insufficient data.
They also conducted listening tests with participants to see how distinguishable the synthesized words were. The positive results of those tests indicate that the technology not only properly identifies words, but also conveys those words audibly and understandably, just like a human voice.
Limitations
‘For the time being, there are some limitations,’ adds Berezutskaya. ‘In these experiments, we asked individuals to say twelve words aloud, and those were the terms we looked for. guessing individual words is generally easier than guessing complete phrases. Large language models utilized in AI research may be useful in the future.
Our goal is to predict entire sentences and paragraphs of what people are attempting to say based just on brain activity. More studies, more powerful implants, greater datasets, and advanced AI models will be required to get there. All of these processes will take time, but they appear to be moving in the right way.’
