A strategy for detecting an anomalous batch of kappa serum free light chains measurements using nonparametric analysis of patient deltas

Abstract Patient-based quality control (PBQC) involves monitoring for shifts in patient results to detect analytical problems. Established methods perform well, but only for assays with high volumes and approximately normal result distributions. Recent publications have described monitoring the average of patient deltas (AOD) as a way generalize PBQC methods more broadly. Here we propose a nonparametric analysis of patient deltas (NPAOD). To illustrate the concept we present a case of an analytical issue in the kappa serum free light chain assay (kappa sFLC) caused by a bad reagent pack and how it could have been detected using NPAOD. We were alerted to a possible issue by the ordering clinicians who noticed a series of unexpectedly high results. Significantly lower values were observed when the samples were repeated with a new reagent pack (median [IQR]: 5.3 [3.12-13.52] versus 2.52 [0.95-4.25], p < 0.5 e-7 by paired Wilcoxon test). To evaluate the use of PBQC in detecting this error, we analyzed one year of retrospective data comprising 7657 results from 3061 patients. The distribution of kappa sFLC values was positively skewed (mean = 14.5 g/dL, median = 2.2 g/dL, skew = 26.6). Both volumes and distributions varied significantly by day of the week following the scheduling of subspecialty clinics. These properties make the conventional approach of simple moving averages poorly suited for monitoring kappa sFLC at our institution. We computed delta values as the difference between the current result and the most recent previous result and produced delta values for 4596/7657 (60%) of the samples. The distribution of deltas was zero centered and symmetric (mean = 0.03 g/dL, median = 0.01 g/dL). Using the historical distribution as a reference, we found that the average of deltas was not significantly different for the results of the bad reagent pack (p = 0.78 by Student’s T test). In contrast, a nonparametric approach yielded a significant difference whether by comparison of medians using a Wilcoxon test (p = 0.0004) or comparison of empirical distributions using a Kolmogorov-Smirnov test (p = 1e-7). The latter approach may offer several benefits. First, the Komogorov-Smirnov test has greater statistical power because it considers both the position and the shape of the distribution. Second, an algorithm based on comparison of empirical distributions may have fewer hyperparameters to tune. We continue this work with an in-silico characterization of detection and false alarm rates at different magnitudes of analytical bias. In conclusion, this case study provides initial evidence that a nonparametric analysis of patient deltas may help to extend PBQC to additional assays that have unfavorable distributions and volumes for the simple moving average algorithm.