Runoff predictions in new-gauged basins using two transformer-based models

In hydrology, runoff predictions are challenging when the data is lacking (e.g., predictions in un-gauged basins (PUB) and predictions with limited data (PLD)). Here, PLD refers to the case that the data is not enough for training or fine-tuning a data-driven model well (e.g., a new-gauged basin). We also name PLD as PNB (predictions in new-gauged basins). The difference between PNB and PUB is that the new-gauged basins can provide some data (e.g., runoff observations) while the un-gauged basins cannot. The long short-term memory (LSTM)-based models have shown good performance in runoff predictions due to their advanced structures. However, those structures have low level of flexibility and two nonadjacent positions cannot communicate directly. For high level of flexibility and better performance, we propose a simple Transformer-based rainfall-runoff model named RRS-Former, and want to show its power and also the power of the previously proposed RR-Former in PNB compared with LSTM-based models. The main part of RRS-Former is attention modules in which two arbitrary positions can be connected directly. Four hydrological units including 241 basins in the Catchment Attributes and Meteorology for Large-Sample Studies (CAMELS) dataset are used to compare the performance of our Transformer-based models and that of LSTM-based models. Besides using the k-fold validation to test performance for PNB, we propose a new way named unit-fold validation, in which we train the models by using the basins in three hydrological units and then test the performance using basins in the rest one hydrological unit. The results based on both k-fold and unit-fold show that our RRS-Former and RR-Former have better performance and are more reliable compared with the LSTM-based models.