Feedback processing shapes the categorical organization of the ventral stream

The human ventral stream shows a categorical organization, with distinct regions responding selectively to faces, houses, tools, etc. Recent neuroimaging and computational studies have shown that this organization partly reflects the feedforward processing of category-specific visual features. However, other work has provided evidence for a similar categorical organization in the absence of visual input, suggesting that it may also be shaped by top-down feedback processing. Here, to reveal such feedback processing, we focus on the selective response to large objects (buildings) in the scene-selective parahippocampal place area (PPA). Specifically, we tested whether the selective response to buildings in the PPA: 1) can be observed when controlling for visual features typical of buildings (e.g., rectilinearity), 2) is delayed relative to the PPA response to scenes, and 3) reflects top-down activation of scene representations. In an fMRI study with high temporal resolution (TR=140 ms), participants (N=30) viewed images of isolated buildings, visually matched boxes, scenes, and chairs. Results showed a selective PPA response to buildings (vs boxes), despite their closely matched visual features. Interestingly, analyses of BOLD peak latency showed that building-selective PPA responses peaked about 200 ms later (4.87 s) than scene-selective PPA responses (4.66 s), consistent with the hypothesized delayed responses reflecting top-down feedback. This delayed PPA response to buildings was corroborated by an EEG study (N=32): multivariate decoding analyses across posterior electrodes revealed that building-selective response patterns emerged relatively late (350 ms after stimulus onset), about 200 ms later than scene-selective response patterns. Finally, building-selective response patterns at 350 ms after stimulus onset generalized to scene-selective response patterns at 200 ms after stimulus onset. Taken together, these results provide information about the nature of large-object selectivity in the PPA and, more generally, indicate that (at least some) category-selective responses in visual cortex can be decoupled from visual feature processing.