Reading minds to answer questions about what people see

A new method called Brain-IT-VQA decodes language directly from fMRI brain activity and uses a language model to answer questions about images a person has seen. The team also created NSD-VQA, a dataset with 20 carefully controlled questions per image, allowing them to measure which visual and semantic information the brain actually encodes. The work bridges neuroscience and AI: it outperforms prior fMRI decoding approaches while simultaneously mapping how different brain regions process different aspects of vision.