The problem of ROC analysis without truth: the EM algorithm and the information matrix

Henkelman, Kay, and Bronskill (HKB) showed that although the problem of ROC analysis without truth is underconstrained and thus not uniquely solvable in one dimension (one diagnostic test), it is in principle solvable in two or more dimensions. However, they gave no analysis of the resulting uncertainties. The present work provides a maximum-likelihood solution using the EM (expectation-maximization) algorithm for the two-dimensional case. We also provide an analysis of uncertainties in terms of Monte Carlo simulations as well as estimates based on Fisher Information Matrices for the complete- and the missing-data problem. We find that the number of patients required for a given precision of estimate for the truth-unknown problem is a very large multiple of that required for the corresponding truth-known case.