Computer word prediction models such as those used on smartphones could teach us a lot about how our brains process language.
- Like smartphone text prediction technology, our brains are able to predict which words will be spoken next in a sentence.
- Our human language system uses the same activity patterns as artificial intelligence to, for example, fill in a blank in a sentence.
What do your cell phone’s text prediction app and your brain have in common? The way they process language! This is demonstrated by a new study conducted by neuroscientists of the Massachusetts Institute of Technology (MIT). To be published in PNASit reveals that the underlying function of these models of artificial intelligence bears strong similarities to the function of language processing centers in the human brain.
A computer model close to the human language system
In recent years, artificial intelligence models of language have become very good at certain tasks. They particularly excel in predicting the next word in a text string; this technology helps search engines and texting apps predict the next word you’ll type. The new generation is even able to learn something about the underlying meaning of language: they can not only predict the next word, but also perform tasks that seem to require some degree of genuine understanding, such as answering questions, summarizing documents and completing stories.
According to the MIT researchers, the underlying function of these patterns resembles the function of language processing centers in the human brain. This would prove, according to them, that the human brain is able to also use the prediction of the next word to manage language processing.
“The better the model performs in predicting the next word, the closer it gets to the human brain, explains Nancy Kanwisher, lead author of the study. It’s amazing that the models fit so well, and it very indirectly suggests that perhaps what the human language system is doing is predicting what’s going to happen next.”
Network nodes comparable to those observed in the human brain
The new, high-performance next-word prediction models belong to a class of models called deep neural networks. These networks contain computer “nodes” that form connections of varying strength, and layers that pass information between them in prescribed ways.
For this study, the researchers compared language processing centers in the human brain with language processing patterns. For this, they analyzed 43 different language models, several of which are optimized for next-word prediction. These include a model called GPT-3 (Generative Pre-trained Transformer 3), which, given a prompt, can generate text similar to what a human would produce. Other models have been designed to perform different linguistic tasks, such as filling in a blank in a sentence.
When a string of words was presented to each model, the researchers measured the activity of the nodes that make up the network. They then compared these patterns to human brain activity, measured in subjects performing three language tasks: listening to stories, reading sentences one at a time, and reading sentences in which only one word is revealed at a time.
They found that the most successful next-word prediction models exhibited patterns of activity very similar to those seen in the human brain.
The researchers now plan to build variants of these language processing models to see how small changes in their architecture affect their performance and their ability to adapt to human neural data.
.
