Statistical power of spatial earthquake forecast tests

The Collaboratory for the Study of Earthquake Predictability (CSEP) is an international effort to evaluate earthquake forecast models prospectively. In CSEP, one way to express earthquake forecasts is through a grid-based format: the expected number of earthquake occurrences within 0.1 degrees x 0.1 degrees spatial cells. The spatial distribution of seismicity is thereby evaluated using the Spatial test (S-test). The high-resolution grid combined with sparse and inhomogeneous earthquake distributions leads to a huge number of cells causing disparity in the number of cells, and the number of earthquakes to evaluate the forecasts, thereby affecting the statistical power of the S-test. In order to explore this issue, we conducted a global earthquake forecast experiment, in which we computed the power of the S-test to reject a spatially non-informative uniform forecast model. The S-test loses its power to reject the non-informative model when the spatial resolution is so high that every earthquake of the observed catalog tends to get a separate cell. Upon analysing the statistical power of the S-test, we found, as expected, that the statistical power of the S-test depends upon the number of earthquakes available for testing, e.g. with the conventional high-resolution grid for the global region, we would need more than 32 000 earthquakes in the observed catalog for powerful testing, which would require approximately 300 yr to record M >= 5.95. The other factor affecting the power is more interesting and new; it is related to the spatial grid representation of the forecast model. Aggregating forecasts on multi-resolution grids can significantly increase the statistical power of the S-test. Using the recently introduced Quadtree to generate data-based multi-resolution grids, we show that the S-test reaches its maximum power in this case already for as few as eight earthquakes in the test period. Thus, we recommend for future CSEP experiments the use of Quadtree-based multi-resolution grids, where available data determine the resolution.