STeInFormer: Spatial-Temporal Interaction Transformer Architecture for Remote Sensing Change Detection

Abstract: Convolutional neural networks and attention mechanisms have greatly benefited remote sensing change detection (RSCD) because of their outstanding discriminative ability. Existent RSCD methods often follow a paradigm of using a non-interactive Siamese neural network for multi-temporal feature extraction and change detection heads for feature fusion and change representation. However, this paradigm lacks the contemplation of the characteristics of RSCD in temporal and spatial dimensions, and causes the drawback on spatial-temporal interaction that hinders high-quality feature extraction. To address this problem, we present STeInFormer, a spatial-temporal interaction Transformer architecture for multi-temporal feature extraction, which is the first general backbone network specifically designed for RSCD. In addition, we propose a parameter-free multi-frequency token mixer to integrate frequency-domain features that provide spectral information for RSCD. Experimental results on three datasets validate the effectiveness of the proposed method, which can outperform the state-of-the-art methods and achieve the most satisfactory efficiency-accuracy trade-off. Code is available at https://github.com/xwmaxwma/rschange.

• The paper proposes STeInFormer, a Transformer architecture combining cross-temporal/spatial interactions and frequency-domain mixing for enhanced remote sensing change detection.
• STeInFormer achieved superior F1-scores compared to state-of-the-art methods across three datasets while demonstrating efficiency with reduced parameters and FLOPs.
• STeInFormer offers a promising approach for future remote sensing change detection tasks by effectively integrating spatial, temporal, and frequency-domain information.

An Overview of STeInFormer: Spatial-Temporal Interaction Transformer for Remote Sensing Change Detection

This paper introduces "STeInFormer," a specialized Transformer architecture developed to address the unique challenges inherent in remote sensing change detection (RSCD). The proposed method distinguishes itself from traditional RSCD workflows by integrating spatial-temporal interactions directly within its architectural design, taking advantage of both convolutional neural networks (CNNs) and attention mechanisms for enhanced feature extraction.

Key Components of STeInFormer

1. Cross-Temporal Interaction: The STeInFormer incorporates cross-temporal interactor (CTI) modules, leveraging gating mechanisms to prioritize relevant changes while suppressing non-interest changes. This focuses on enhancing the dynamic features between multi-temporal data, significantly aiding in the discrimination of subtle changes.
2. Cross-Spatial Interaction: It uses cross-spatial interactors (CSI), which are inspired by the U-Net architecture to fuse high-level semantic information with low-level spatial details effectively. This arrangement allows for maintaining spatial precision that is particularly important in high-resolution remote sensing imagery processing.
3. Multi-Frequency Token Mixer: The paper innovatively includes a parameter-free multi-frequency token mixer to integrate frequency-domain information using pre-designed discrete cosine transform (DCT) base functions. This mixer is characterized by its linear complexity, making it resource-efficient while contributing to robust feature representation.

Experimental Validation

Empirical results demonstrated the superiority of STeInFormer compared to existing state-of-the-art methods over three datasets: WHU-CD, LEVIR-CD, and CLCD. It achieved higher F1-scores, showcasing better performance without incurring a substantial computational cost. The architecture not only proved effective in capturing complex changes but also efficient, characterized by a notably reduced number of parameters and FLOPs.

Implications and Future Directions

The proposed STeInFormer exhibits promise for improving RSCD tasks, potentially serving as a backbone network for future change detection applications. This shift to integrating frequency domain features could represent a key advancement in how RSCD tasks approach feature extraction, allowing for nuanced understanding and analysis of earth observation datasets. Moving forward, investigating the potential of frequency-domain approaches and further optimizing spatial-temporal interactions could chart new directions for AI-based remote sensing applications.

Conclusion

Overall, the STeInFormer represents a well-crafted novel approach exploiting both spatial and temporal dimensions in remote sensing imagery, driving performance improvements while maintaining efficient computations. As research in AI and RSCD progresses, methodologies such as this could provide foundational insights and tools for advancing automated change detection capabilities.