Intuitive physics guides visual tracking and working memory: The dynamics of neural processing in expectation violation

People have strong expectations about the basic behavior of everyday objects (e.g., that they shouldn’t suddenly disappear), which guide our perception, actions, and predictions. Core physical knowledge is so basic it is present in very early infancy and in non-human animals, as demonstrated in numerous behavioral experiments relying on surprise in the face of physically-impossible events. Yet, the online dynamics of processing physically-impossible events, leading to surprise and following it, remain hidden. To uncover them, we leveraged neural markers of online processing, by embedding short videos of physically-possible or physically-impossible scenes in a visual working memory task. In two EEG experiments, we measured the contralateral delay activity (CDA) as the scene unfolded, to examine whether violations of physical-expectations (a) disrupt the ability to smoothly track objects in the scene, and (b) later modify the scene’s interpretation. Our stimuli were modeled after those used in behavioral studies with adults and infants, and more recently AI benchmarks. In Experiment 1, objects crossed a stage, moving behind an occluder which briefly hid them. The animations either proceeded as expected, or included violations of object-permanence, in which an object disappeared or appeared. We found that the violations disrupted object tracking, causing working memory to reset before it could represent the new and correct number of objects in the scene. In Experiment 2, the stimuli were minimally changed to include a hole behind the occluder, meaning that appearance and disappearance could be explained away. Despite perceptual input during the critical period being identical to Experiment 1, the changes in high-level expectations translated to uninterrupted object tracking in Experiment 2, and even allowed participants to continue representing the object that disappeared ‘down the hole’. Our results suggest that intuitive physics expectations shape online representations, and are critical for the ability to track objects in dynamic scenes.