Multi-Study Learning for Real-time Neurochemical Sensing in Humans using the ``Study Strap Ensemble"

Real-time neurochemical sensing during awake behavior in humans allows for direct investigation of the neural signals that drive human decision-making. Current techniques to perform this kind of research require measurements from an electrode placed in the brain during awake neurosurgery and rely on prediction models to estimate neurotransmitter concentrations from the electrical measurements recorded by the electrode. These models are trained by combining multiple datasets (“studies”) that are collected in vitro under heterogeneous conditions in order to promote accuracy of the models when applied to data collected in the brain. A prevailing challenge is deciding how to combine studies or ensemble models trained on different studies to enhance model generalizability. We propose the “study strap ensemble”, which leverages advantages of two common approaches to multi-study learning problems: pooling studies and fitting one model versus averaging predictions from multiple models each fit to individual studies. The study strap ensemble fits models to bootstrapped datasets, or “pseudo-studies.” These are generated by resampling from multiple studies with a hierarchical resampling scheme that generalizes the randomized cluster bootstrap. The study strap is controlled by a tuning parameter that determines the proportion of observations to draw from each study. When the parameter is set to its lowest value, each pseudostudy is resampled from only a single study. When it is high, the study strap ignores the multi-study structure and generates pseudo-studies by merging the datasets and drawing observations like a standard bootstrap. We empirically show the optimal tuning value often lies in between. We prove that special cases of the study strap draw the merged dataset and the set of original studies as pseudo-studies. We further extend the study strap approach by introducing an ensemble weighting scheme that utilizes information in the distribution of the covariates of the test dataset. Our methods produce marked improvements in simulations and in our motivating application. All methods are available in the studyStrap CRAN package.