State-trace analysis — Misrepresented and misunderstood: Reply to Ashby (2019)

Stephens, Matzke, and Hayes (SMH; 2019) used state-trace analysis to re-analyze databases of studies of reasoning and category learning. They found that many behavioral dissociations that had been viewed as support for distinct cognitive processes (or systems) were consistent with the operation of only one latent psychological variable. Ashby (2019) discussed several concerns about the application and interpretation of state-trace analysis in relation to the COVIS dual-systems model of category learning. The current reply addresses these concerns, showing that Ashby’s arguments reflect a misunderstanding of some aspects of state-trace analysis and a misrepresentation of claims made by SMH. We do not claim that state-trace analysis is the final arbiter of competing theories about cognitive systems, and do not assert that it directly assesses model parsimony. Nevertheless, we argue that state-trace analysis is an important advance over existing methods for evaluating claims about unobservable psychological processes based on ordinal data patterns, and is a useful tool as part of theory testing.