On Selecting a Multiple Comparison Procedure for Analysis of a Clinical Trial: Fallback, Fixed Sequence, and Related Procedures

In a clinical trial with multiple statistical hypotheses of interest, control of the Type I error rate must be considered. There are many ways to address control of the Type I error rate and often the various options differ only by subtle nuances. In this article we describe several similar methods for a general case of hypotheses without logical restrictions, including the fallback procedure and the fixed sequence procedure, which order testing a priori, and procedures with a data-driven ordering of hypotheses, which order testing according to the observed data. Comparisons on power, including the probability of rejecting at least one or of rejecting all hypotheses, are provided. In general, if the treatment effects are known or accurately estimated, an optimal testing procedure can be chosen. However, if the treatment effects are unknown and especially if there is doubt about whether one or more null hypotheses are true, an optimal testing procedure may not be available and compromise procedures which retain some level of power for many plausible scenarios are preferable. Examples are discussed to compare some of the proposed procedures.