Evaluating the utility of brightfield image data for mechanism of action prediction

Fluorescence staining techniques, such as Cell Painting, together with fluorescence microscopy have proven invaluable for visualizing and quantifying the effects that drugs and other perturbations have on cultured cells. However, fluorescence microscopy is expensive, time-consuming, labor-intensive, and the stains applied can be cytotoxic, interfering with the activity under study. The simplest form of microscopy, brightfield microscopy, lacks these downsides, but the images produced have low contrast and the cellular compartments are difficult to discern. Nevertheless, by harnessing deep learning, these brightfield images may still be sufficient for various predictive purposes. In this study, we compared the predictive performance of models trained on fluorescence images to those trained on brightfield images for predicting the mechanism of action (MoA) of different drugs. We also extracted CellProfiler features from the fluorescence images and used them to benchmark the performance. Overall, we found comparable and largely correlated predictive performance for the two imaging modalities. This is promising for future studies of MoAs in time-lapse experiments for which using fluorescence images is problematic. Explorations based on explainable AI techniques also provided valuable insights regarding compounds that were better predicted by one modality over the other. Author summaryIn our work we provide a thorough comparison of deep learning models trained on fluorescence images versus those trained on brightfield images for the purpose of mechanism of action prediction. Fluorescence microscopy, which uses fluorescent dyes to stain specific cellular compartments, is an invaluable method in image cytometry but it is time-consuming, expensive, labour intensive, and the dyes used can be toxic to the cells. Brightfield microscopy, the simplest form of light microscopy, lacks these downsides but the images produced do not have the clear contrast of the cellular compartments. We show that with brightfield images it is possible to obtain very similar predictive performance, and in some cases superior performance, due to the brightfield images containing additional information not available in the fluorescence images. We validate these claims using explainable AI techniques. This competitive performance of the models based on brightfield is very promising for live cell time-lapse experiments for which using fluorescence microscopy is problematic especially due to the toxicity of some of the dyes.