Foundation model predicts brain activity from speech across 324 participants
Foundation models are now crossing into neuroscience. RABBiT demonstrates that compact encoder architectures trained on brain imaging data can generalize across participants and stimuli with minimal fine-tuning, achieving zero-shot fMRI prediction from speech. This represents a shift in how AI systems model human cognition: rather than treating brain activity as a downstream prediction task, the work treats it as a learnable domain where shared linguistic structure emerges across individuals. The implication is significant for both neuroscience and AI interpretability. If brain responses follow learnable patterns that transfer across people, it suggests language models and human brains may encode meaning through comparable structural principles, offering a new empirical lens for understanding what foundation models actually learn.
Modelwire context
ExplainerThe key technical claim worth scrutinizing is zero-shot generalization across participants, meaning the model predicts fMRI responses for people it has never seen during training. That cross-subject transfer is the hard part of brain encoding research, and most prior work quietly sidesteps it by fine-tuning on held-out subjects before reporting numbers.
This sits at the intersection of two threads Modelwire has been tracking. The non-invasive brain-to-text work from Meta's FAIR team (covered July 1, via The Decoder) approached the brain-computer interface problem from the decoding side, reconstructing language from neural signals. RABBiT runs the problem in the opposite direction, predicting what brain activity a given speech stimulus should produce. Together they sketch a two-way bridge between language and neural representation. The broader question of whether foundation models encode meaning in ways that resemble human cognition connects directly to the LLM survey covered the same week, which examined which cognitive phenomena are genuine architectural artifacts versus surface mimicry.
If RABBiT's zero-shot accuracy holds on held-out language datasets beyond the stimuli used in training, specifically naturalistic speech corpora not drawn from the same recording sessions, that would substantiate the cross-individual generalization claim. If performance drops sharply outside training-adjacent stimuli, the transfer story is narrower than advertised.
Coverage we drew on
This analysis is generated by Modelwire’s editorial layer from our archive and the summary above. It is not a substitute for the original reporting. How we write it.
MentionsRABBiT · fMRI · foundation models · speech-to-brain encoding
Modelwire Editorial
This synthesis and analysis was prepared by the Modelwire editorial team. We use advanced language models to read, ground, and connect the day’s most significant AI developments, providing original strategic context that helps practitioners and leaders stay ahead of the frontier.
Modelwire summarizes, we don’t republish. arXiv cs.CL originally reported this story as “RABBiT: Rapidly adaptive BOLD foundation model via brain-tuning for accurate zero-shot and few-shot prediction of speech-elicited responses in the brain”. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.