Comparison of relative change with effect size metrics in Alzheimer's disease clinical trials

Background Per cent slowing of decline is frequently used as a metric of outcome in Alzheimer's disease (AD) clinical trials, but it may be misleading. Our objective was to determine whether per cent slowing of decline or Cohen's d is the more valid and informative measure of efficacy.Methods Outcome measures of interest were per cent slowing of decline; Cohen's d effect size and number-needed-to-treat (NNT). Data from a graphic were used to model the inter-relationships among Cohen's d, placebo decline in raw score units and per cent slowing of decline with active treatment. NNTs were computed based on different magnitudes of d. Last, we tabulated recent AD anti-amyloid clinical trials that reported per cent slowing and for which we computed their respective d's and NNTs.Results We demonstrated that d and per cent slowing were potentially independent. While per cent slowing of decline was dependent on placebo decline and did not include variance in its computation, d was dependent on both group mean difference and pooled SD. We next showed that d was a critical determinant of NNT, such that NNT was uniformly smaller when d was larger. In recent AD associated trials including those focused on anti-amyloid biologics, d's were below 0.23 and thus considered small, while per cent slowing was in the 22-29% range and NNTs ranged from 14 to 18.Conclusions Standardised effect size is a more meaningful outcome than per cent slowing of decline because it determines group overlap, which can directly influence NNT computations, and yield information on the likelihood of minimum clinically important differences. In AD, greater use of effect sizes, NNTs, rather than relative per cent slowing, will improve the ability to interpret clinical trial results and evaluate the clinical meaningfulness of statistically significant results.