The Data Synergy Effects of Time-Series Deep Learning Models in Hydrology

When fitting statistical models to variables in geoscientific disciplines such as hydrology, it is a customary practice to stratify a large domain into multiple regions (or regimes) and study each region separately. Traditional wisdom suggests that models built for each region separately will have higher performance because of homogeneity within each region. However, each stratified model has access to fewer and less diverse data points. Here, through two hydrologic examples (soil moisture and streamflow), we show that conventional wisdom may no longer hold in the era of big data and deep learning (DL). We systematically examined an effect we call data synergy, where the results of the DL models improved when data were pooled together from characteristically different regions. The performance of the DL models benefited from modest diversity in the training data compared to a homogeneous training set, even with similar data quantity. Moreover, allowing heterogeneous training data makes eligible much larger training datasets, which is an inherent advantage of DL. A large, diverse data set is advantageous in terms of representing extreme events and future scenarios, which has strong implications for climate change impact assessment. The results here suggest the research community should place greater emphasis on data sharing. Plain Language Summary Traditionally with statistical methods used in hydrology, we split the domain into relatively homogeneous regimes, for each of which we can create a simple model, that is, a local model. However, in the era of big data machine learning, we show that this is often the opposite of what should be done. With deep learning models, we should compile a large and heterogeneous data set and compare the local model to a model trained with all the data (global model). Including heterogeneous training samples may improve the results compared to the local model. We call this the data synergy effect, and it results from two main factors. First, deep learning models are complex enough to accommodate different training instances, inherently permitting larger training datasets with more extreme events and changing trends. Second, with a heterogeneous training data set, deep learning models may be able to learn both the underlying similarities and factors contributing to differences between regions.