How fast can I get to an answer? Sample size, power, and observing behavior

Behavior analysts primarily use “steady-state responding” when deciding when to stop collecting data and to analyze the resulting patterns in their data. These decisions often occur at three levels of data collection: within sessions, within conditions, and—for researchers—within studies. But, exactly when can behavior analysts feel comfortable stopping their data collection? That is, how do they know they have collected enough data to answer their question but have not wasted time and resources by collecting too much data. The short-and-sweet answer is to stop collecting data when you have sufficiently measured the behavior-environment relations of interest. What this looks like analytically, however, is a much more nuanced decision than it reads. In this chapter, we review common answers to the question “When can I stop collecting data?” Throughout, we review answers based on statistics as well as common methods behavior analysts have used in the past. We also review different ways to estimate how many observations are needed before beginning a research study (i.e., power analyses) and how this family of calculations can be used at a variety of time points. By the end of the chapter, readers should know how to quantitatively justify that what they see in their data accurately reflects the behavior-environment relations under study.