Forecasting the spatial spread of an Ebola epidemic in real-time: comparing predictions of mathematical models and experts

Ebola virus disease outbreaks can often be controlled, but require rapid response efforts frequently with profound operational complexities. Mathematical models can be used to support response planning, but it is unclear if models improve the prior understanding of experts. We performed repeated surveys of Ebola response experts during an outbreak. From each expert we elicited the probability of cases exceeding four thresholds between two and 20 cases in a set of small geographical areas in the following calendar month. We compared the predictive performance of these forecasts to those of two mathematical models with different spatial interaction components. An ensemble combining the forecasts of all experts performed similarly to the two models. Experts showed stronger bias than models forecasting two-case threshold exceedance. Experts and models both performed better when predicting exceedance of higher thresholds. The models also tended to be better at risk-ranking areas than experts. Our results support the use of models in outbreak contexts, offering a convenient and scalable route to a quantified situational awareness, which can provide confidence in or to call into question existing advice of experts. There could be value in combining expert opinion and modelled forecasts to support the response to future outbreaks. ### Competing Interest Statement The authors have declared no competing interest. ### Funding Statement This study was partly funded by the Department of Health and Social Care using UK Aid funding and is managed by the National Institute for Health and Care Research (VEEPED: PR-OD-1017-20002; AR and WJE). This study was partly funded by the Wellcome Trust (210758/Z/18/Z : JDM and SF). The views expressed in this publication are those of the authors and not necessarily those of the funders. ### Author Declarations I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained. Yes The details of the IRB/oversight body that provided approval or exemption for the research described are given below: LSHTM ethics approval was obtained for this study (reference: 17633). Signed informed consent was taken from experts willing to participate and their verbal consent was requested again at the beginning of each elicitation. I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals. Yes I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance). Yes I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable. Yes All data and code used to process the expert interview responses can be found here: https://github.com/epiforecasts/Ebola-Expert-Interviews. The forecasts were performed using the EpiCastR package https://github.com/epiforecasts/EpiCastR. The code used for the analysis and scoring of the forecasts can be found here: https://github.com/epiforecasts/Ebola-Expert-Ellicitation