STAMICS: Splat, Track And Map with Integrated Consistency and Semantics for Dense RGB-D SLAM

Abstract: Simultaneous Localization and Mapping (SLAM) is a critical task in robotics, enabling systems to autonomously navigate and understand complex environments. Current SLAM approaches predominantly rely on geometric cues for mapping and localization, but they often fail to ensure semantic consistency, particularly in dynamic or densely populated scenes. To address this limitation, we introduce STAMICS, a novel method that integrates semantic information with 3D Gaussian representations to enhance both localization and mapping accuracy. STAMICS consists of three key components: a 3D Gaussian-based scene representation for high-fidelity reconstruction, a graph-based clustering technique that enforces temporal semantic consistency, and an open-vocabulary system that allows for the classification of unseen objects. Extensive experiments show that STAMICS significantly improves camera pose estimation and map quality, outperforming state-of-the-art methods while reducing reconstruction errors. Code will be public available.

• The paper introduces a novel approach that integrates semantic data with Gaussian splatting to enhance mapping and reduce reconstruction errors.
• It employs a graph-based clustering technique to ensure temporal semantic consistency, effectively mitigating semantic drift in dynamic scenes.
• The framework demonstrates improved localization and pose estimation accuracy across multiple benchmark datasets compared to state-of-the-art SLAM methods.

STAMICS: Integrating Consistency and Semantics in Dense RGB-D SLAM

Introduction

"STAMICS: Splat, Track And Map with Integrated Consistency and Semantics for Dense RGB-D SLAM" (2503.21425) introduces a novel approach to Simultaneous Localization and Mapping (SLAM) by integrating semantic information with 3D Gaussian representations for enhanced localization and mapping accuracy. The authors aim to address the limitations of traditional SLAM systems, which typically rely on geometric data but struggle to maintain semantic consistency in dynamic or densely populated environments.

Methodology

Semantic Integration with Gaussian Splatting:

STAMICS introduces a framework that integrates semantic data directly into the SLAM process using Gaussian Splatting (Figure 1). This involves enhancing 3D Gaussian scene representation with semantic data to effectively manage both geometric and semantic consistency. The scene representation is improved through a Semantic-Enhanced Gaussian Splatting technique, ensuring alignment of geometry and semantics by encoding semantics into Gaussian parameters.

Figure 1: Overview of STAMICS showing the integration of semantic data into geometric reconstruction, managed by various loss functions including semantic consistency loss.

Temporal Semantic Consistency:

A significant innovation is the graph-based clustering approach to enforce temporal semantic consistency (Figure 2). This ensures stable and coherent labeling of objects over time, combating the challenge of semantic drift wherein the same objects are inconsistently labeled in different frames.

Figure 2: Illustration of graph clustering showing how nodes with high semantic consistency scores are grouped, creating a new graph with improved consistency.

Open Vocabulary System:

STAMICS also incorporates an open-vocabulary system to dynamically classify unseen objects, enhancing adaptability to unfamiliar environments. This expansion enables the system to maintain semantic coherence even when encountering novel objects, thereby improving both flexibility and robustness.

Results

STAMICS excels in reducing reconstruction errors and improving pose estimation accuracy as compared to state-of-the-art SLAM methods (Figure 3). Quantitative analyses highlight substantial improvements in trajectory estimation accuracy with lower Average Trajectory Error (ATE) across diverse benchmark datasets including TUM-RGBD, Replica, and Scannet.

Figure 3: Comparison of reconstruction results with existing methods demonstrating reduced errors and enhanced detail.

In particular, STAMICS exhibited outstanding performances in maintaining low ATE in high-complexity sequences, demonstrating its robustness across both indoor and dynamic environments.

Implications and Future Directions

The integration of semantic information into SLAM enhances both theoretical and practical applications in fields such as robotics and autonomous navigation. By maintaining semantic and geometric coherence, STAMICS enables more accurate and reliable mapping in complex environments. Future research could extend this approach to cover broader environmental variations and explore further scalability of the system in real-time applications.

Conclusion

STAMICS represents a significant advancement in SLAM technology by effectively integrating semantics with 3D scene representation to tackle challenges of semantic drift and object consistency. This approach not only improves localization and mapping accuracy but also positions STAMICS as a robust framework adaptable to environments with varying semantic complexities.