Beyond streamflow predictions: A continental scale hydrologic model intercomparison experiment

Conducting highly standardized model intercomparison studies of hydrologic models across large scales is beneficial in various aspects such as improving model accuracy and robustness, informing decision making, addressing uncertainties, enhancing educational and outreach opportunities and facilitating model benchmarking among others. However, looking beyond streamflow for hydrologic models is required to ensure that models simulate the right results for the right reasons. Continental scale analyses provide further insights into which systematic limitations a model has. In this study, seven models (GR4J-CemaNeige, HMETS, Blended-v1, Blended-v2, HBV-EC, HYPR, SAC-SMA) have been setup for more than 2500 watersheds across Canada and the US using the RAVEN modeling framework. The models are setup using a standardized set of meteorologic and geophysical datasets to inform the model regarding forcings, soil, landcover, and terrain. All models are calibrated with respect to daily streamflow (2001-2015) and are subsequently validated on an independent time period (1986-2000). Calibration was performed using 10 independent trials of the Dynamically Dimensioned Search algorithm each using a budget of 2000 model evaluations and Kling-Gupta Efficiency (KGE) as the objective function. Additional variables such as actual evapotranspiration (AET), surface soil moisture (SSM), and snow water equivalent (SWE) for the calibrated model setups were recorded and compared against independent gridded reference datasets (AET and SSM from GLEAM, SWE from ERA5-Land).  The results (surprisingly) show that all tested models perform equally well for streamflow prediction (range of median KGE values across all sites during calibration period is [0.83, 0.87] and validation period is [0.46, 0.54]).  Differences between models are most apparent for the auxiliary variables analyzed, i.e. AET, SSM, and SWE. The most interesting differences between the models lie in their abilities to predict AET, with median KGE being the highest for SAC-SMA (0.71), followed by GR4J-CemaNeige (0.65), while the lowest values were observed for HMETS (0.37) and HBC-EC (0.17). Indeed SAC-SMA showed highest performances across 51% of locations while the second-best model is GR4J-CemaNeige with best performance at 13% of locations.  The SSM, evaluated using the Pearson correlation (r) coefficient, was predicted relatively well by all models (r ranging between 0.62 and 0.72); however, while most models had poorer predictions in the Rocky mountains and at higher latitudes, the SAC-SMA was definitely a better predictor of the temporal dynamics in SSM in these regions. While the median performance for SWE prediction was relatively low across all models (median KGE between 0.23 and 0.40), poorer predictions mostly occurred in regions with low annual SWE, and predictions improved with increasing annual snow amounts.  The study reveals novel insights regarding the consistent ability of a suite of models to predict streamflow, while clear ranking of models was apparent based on their ability to simulate spatially distributed variables like AET. Such differences likely arise due to model equifinality highlighting the value of model evaluation against multiple spatially distributed and lumped metrics, generating the correct streamflow for the right reasons.