Using AI-selected natural images and AI-generated synthetic images as neuroscientific tools, scientists at Weill Cornell Medicine, Cornell Tech, and Cornell’s Ithaca campus probe the visual processing areas of the brain. Their study aimed to investigate the structure of vision through the use of data-driven methods, with the potential to eliminate biases that may occur from examining responses to a smaller group of chosen images.
Scientists think that their approach is promising to study the neuroscience of vision. Through this, scientists investigated the selectivity and inter-individual differences in human brain responses to images via functional MRI.
Scientists used an AI model of the human visual system to select or create images that study participants were asked to look at. It was anticipated that the images would stimulate several visual processing regions. Functional magnetic resonance imaging (fMRI) captured the volunteers’ brain activity. Scientists found that the images considerably more activated the target regions than the control images.
Scientists also found that one visual processing region may activate strongly in response to an image of a face, whereas another may respond to a landscape.
To achieve this goal, scientists must primarily rely on non-invasive techniques because direct recording of brain activity using implanted electrodes is risky and challenging. fMRI is the preferred non-invasive technique. It can read the minute alterations in three dimensions throughout the brain by an fMRI scanner at a resolution of a few cubic millimeters.
In the study, scientists trained an artificial neural network (ANN), a form of AI system, to mimic the visual processing system of the human brain using an existing dataset that included tens of thousands of natural images and associated fMRI responses from human individuals. Afterward, they applied this model to forecast which photos from the dataset should maximally stimulate several specific brain regions related to vision.
To achieve the same goal, they also combined the model with an AI-based picture generator to produce synthetic images.
Six volunteers were enrolled in the study. Scientists recorded their fMRI responses to the images. They mainly focused on the responses in several visual processing areas.
The findings demonstrated that, for synthetic and natural images, the predicted maximal activator images strongly activated the targeted brain regions on average across individuals, compared to a set of images that were generated or selected to be only average activators. This demonstrates the general validity of the ANN-based model and implies that synthetic images could also be helpful as probes for evaluating and refining models of this kind.
In another study, scientists built distinct ANN-based visual system models for each of the six participants using the image and fMRI-response data from the first session. They then selected or generated predicted maximal-activator images for every person using these customized models.
Compared to the responses to images based on the group model, the fMRI responses to these images showed that, at least for the synthetic images, the targeted visual region was more activated, a face-processing region termed FFA1.
Scientists noted, “This result suggests that AI and fMRI can be useful for individualized visual-system modeling, for example, to study differences in visual system organization across populations.”
Scientists are now conducting more experiments using a more advanced version of the image generator called Stable Diffusion. It could help study other senses, such as hearing.
Dr. Amy Kuceyeski, a professor of mathematics in radiology and of mathematics in neuroscience at the Feil Family Brain and Mind Research Institute at Weill Cornell Medicine- also hopes to study the therapeutic potential of this approach.
“In principle, we could alter the connectivity between two parts of the brain using specifically designed stimuli, for example, to weaken a connection that causes excess anxiety.”
Journal Reference:
- Gu, Z., Jamison, K., Sabuncu, M.R. et al. Human brain responses are modulated when exposed to optimized natural images or synthetically generated images. Commun Biol 6, 1076 (2023). DOI: 10.1038/s42003-023-05440-7