Classifying raw irregular time series (CRIT) for large area land cover mapping by adapting transformer model

For Landsat land cover classification, the time series observations are typically irregular in the number of observations in a period (e.g., a year) and acquisition dates due to cloud cover variations over large areas and acquisition plan variations over long periods. Compositing or temporal percentile calculation are usually used to transform the irregular time series to regular temporal variables so that the machine and deep learning classifiers can be applied. Recognizing that the composite and percentile calculations have information loss, this study presents a method directly Classifying the Raw Irregular Time series (CRIT) ('raw' means irregular good-quality surface reflectance time series without any composite or temporal percentile derivation) by adapting Transformer. CRIT uses the acquisition day of year as classification input to align time series and also takes the Landsat satellite platform (Landsat 5, 7 and 8) as input to address the inter-sensor reflectance differences. The CRIT was demonstrated by classifying Landsat analysis ready data (ARD) surface reflectance time series acquired across one year for three years (1985, 2006 and 2018) over the Conterminous United States (CONUS) with both spatial and temporal variations in Landsat availability. 20,047 training and 4949 evaluation 30-m pixel were used where each pixel was annotated as one of seven land cover classes for each year. The CRIT was compared with classifying 16-day composite time series and temporal percentiles and compared with a 1D convolution neural network (CNN) method. Results showed that the CRIT trained with three years of samples had 1.4-1.5% higher overall accuracies with less computation time than classifying 16-day composites and 2.3-2.4% higher than classifying temporal percentiles. The CRIT advantages over 16-day composites were pronounced for developed (0.05 F1-score) and cropland (0.02 F1-score) classes and for mixed or boundary pixels. This was reasonable as the 16-day composites had only on average 7.02, 16.49 and 15.78 good quality observations for the three years, respectively, in contrast to 7.89, 27.72, and 26.60 for the raw irregular time series. The CNN was not as good as CRIT in classifying the raw irregular time series as CNN simply filling temporal positions with no observations as zeros while the CRIT used a masking mechanism to rule out their contribution. The CRIT can also take the pixel coordinates and DEM variables as input which further increased the overall accuracies by 1.1-2.6% and achieved 84.33%, 87.54% and 87.01% overall accuracies for the 1985, 2006 and 2018 classifications, respectively. The CRIT land cover maps were shown consistent with the USGS Land Change Monitoring, Assessment, and Projection (LCMAP) maps. The developed codes, training data and maps were made publicly available.