Mind-to-Image: Projecting Visual Mental Imagination of the Brain from fMRI
arxiv(2024)
摘要
The reconstruction of images observed by subjects from fMRI data collected
during visual stimuli has made significant strides in the past decade, thanks
to the availability of extensive fMRI datasets and advancements in generative
models for image generation. However, the application of visual reconstruction
has remained limited. Reconstructing visual imagination presents a greater
challenge, with potentially revolutionary applications ranging from aiding
individuals with disabilities to verifying witness accounts in court. The
primary hurdles in this field are the absence of data collection protocols for
visual imagery and the lack of datasets on the subject. Traditionally,
fMRI-to-image relies on data collected from subjects exposed to visual stimuli,
which poses issues for generating visual imagery based on the difference of
brain activity between visual stimulation and visual imagery. For the first
time, we have compiled a substantial dataset (around 6h of scans) on visual
imagery along with a proposed data collection protocol. We then train a
modified version of an fMRI-to-image model and demonstrate the feasibility of
reconstructing images from two modes of imagination: from memory and from pure
imagination. This marks an important step towards creating a technology that
allow direct reconstruction of visual imagery.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要