New Delhi: A team of researchers at the University of Texas at Austin has created a novel artificial intelligence (AI) system which can translate the brain activity of a person who is listening to a story or visualizing telling a story, into a continuous flow of text.
The system incorporates a transformer model, akin to the ones employed by Google’s Bard and Open AI’s ChatGPT, to achieve this feat.
This breakthrough has the potential to revolutionize how we communicate and interact with machines.
A recent study published in the journal Nature Neuroscience suggests that the new AI system, known as the semantic decoder, could be a potential solution for individuals who are mentally aware but are unable to speak due to debilitating conditions like strokes.
Unlike other language decoding methods, this non-invasive system doesn’t require surgical implants for its functioning.
Moreover, the system allows the participants to communicate freely without being constrained by a pre-defined set of words.
The semantic decoder system involves the use of functional MRI scanning to measure brain activity.
To train the decoder, the participants spend several hours listening to podcasts while being scanned.
Once the training is complete, the system can generate text from a person’s brain activity when they listen to a new story or imagine telling a story, provided they consent to having their thoughts decoded.
This process eliminates the need for surgical implants, and the individual can freely communicate without any restriction on the words they use.
Alex Huth, an assistant professor of neuroscience and computer science at UT Austin, expressed his enthusiasm about the new non-invasive system, stating that it represents a significant step forward compared to previous methods, which typically only decoded single words or short sentences.
According to Alex Huth, the new system is capable of decoding complex ideas and generating a continuous stream of language over an extended period.
The resulting text is not an exact transcription of the spoken or imagined words. Instead, it attempts to capture the essence of what is being expressed, albeit imperfectly.
After training the decoder to monitor an individual’s brain activity, the machine is able to produce text that closely matches, and at times precisely matches, the intended meaning of the original words about half of the time.
During the experiments, the researchers provided an example where a participant heard the speaker say, “I don’t have my driver’s license yet,” and the decoded thought was “She has not even started to learn to drive yet.”
Additionally, the team also discussed the issue of potential misuse of the technology.
They emphasized that the decoding process was only successful with willing and cooperative participants who were trained to use the decoder.
This means that the technology can only be used with the individual’s consent and cannot be used without their knowledge.
The researchers found that the decoder was unable to generate intelligible results for individuals who had not undergone training with the system.
Additionally, if the trained participants resisted by thinking other thoughts, the results were also unusable.
Jerry Tang, a doctoral student in computer science and a member of the research team, emphasized the team’s commitment to addressing concerns regarding the potential misuse of the technology.
He stated that they are dedicated to ensuring that individuals use these types of technologies only when they desire to do so, and when it is beneficial for them.
Aside from having participants listen or think about stories, the researchers also instructed them to watch four brief silent videos while inside the scanner.
Using the individual’s brain activity, the semantic decoder was able to accurately describe specific events from the videos.
However, the current system is not suitable for practical use outside of the laboratory due to its reliance on fMRI machines, which require a significant amount of time.
Nonetheless, the researchers believe that their findings could be applied to other more portable brain-imaging systems such as functional near-infrared spectroscopy (fNIRS).
