Polar Hierarchical Mamba: Towards Streaming LiDAR Object Detection with Point Clouds as Egocentric Sequences

Abstract: Accurate and efficient object detection is essential for autonomous vehicles, where real-time perception requires low latency and high throughput. LiDAR sensors provide robust depth information, but conventional methods process full 360{\deg} scans in a single pass, introducing significant delay. Streaming approaches address this by sequentially processing partial scans in the native polar coordinate system, yet they rely on translation-invariant convolutions that are misaligned with polar geometry -- resulting in degraded performance or requiring complex distortion mitigation. Recent Mamba-based state space models (SSMs) have shown promise for LiDAR perception, but only in the full-scan setting, relying on geometric serialization and positional embeddings that are memory-intensive and ill-suited to streaming. We propose Polar Hierarchical Mamba (PHiM), a novel SSM architecture designed for polar-coordinate streaming LiDAR. PHiM uses local bidirectional Mamba blocks for intra-sector spatial encoding and a global forward Mamba for inter-sector temporal modeling, replacing convolutions and positional encodings with distortion-aware, dimensionally-decomposed operations. PHiM sets a new state-of-the-art among streaming detectors on the Waymo Open Dataset, outperforming the previous best by 10\% and matching full-scan baselines at twice the throughput. Code will be available at https://github.com/meilongzhang/Polar-Hierarchical-Mamba .

• The paper introduces a novel state-space model, Polar Hierarchical Mamba, which utilizes local bidirectional and global forward Mamba blocks to handle polar LiDAR data effectively.
• It achieves a 10% performance boost over existing streaming detectors and doubles throughput compared to full-scan methods on the Waymo Open Dataset.
• It reduces computational and memory demands by nearly 50% through dimensionally-decomposed convolutions, optimizing real-time detection for autonomous systems.

An Analytical Summary of "Polar Hierarchical Mamba: Towards Streaming LiDAR Object Detection with Point Clouds as Egocentric Sequences"

The paper "Polar Hierarchical Mamba: Towards Streaming LiDAR Object Detection with Point Clouds as Egocentric Sequences" introduces a new state space model (SSM)-based architecture named Polar Hierarchical Mamba (PHiM), specifically designed for streaming LiDAR perception. This study aims to address the challenge of real-time object detection in autonomous vehicles, where conventional methods often face limitations due to high latency when processing full 360-degree LiDAR scans.

Key Contributions:

1. Streamlined SSM Architecture: The Polar Hierarchical Mamba (PHiM) represents a novel approach using local bidirectional Mamba blocks for spatial encoding within sectors and a global forward Mamba for temporal modeling between sectors. This design replaces traditional translation-invariant convolutions, which are not well-suited for polar geometry, with more tailored, distortion-aware operations.
2. Improved Performance Metrics: PHiM demonstrates enhanced performance in comparison to existing streaming detectors on the challenging Waymo Open Dataset, producing a reported 10% improvement over previous methods. It achieves comparable results to full-scan baselines while allowing for double the throughput, demonstrating its efficiency and robustness.
3. Efficient Use of Resources: The architecture reduces both memory usage and computational cost by avoiding complex geometric serialization and positional embeddings, which are typically memory-intensive. PHiM employs dimensionally-decomposed convolutions to enhance spatial locality awareness, reducing model parameters by nearly 50% compared to full-scan Mamba architectures.

Detailed Insights:

The increasing demand for efficient real-time perception modules in autonomous driving has motivated this research. The traditional single-pass processing approach used in LiDAR systems introduces a notable delay, which is untenable in dynamic driving scenarios. Streaming methods address this by leveraging the sequential nature of LiDAR's rotation to process data as it arrives. However, a direct adaptation of euclidean convolution methods tends to degrade performance in polar coordinate systems due to inherent spatial distortions.

The PHiM architecture elegantly circumvents these challenges by utilizing Mamba-based spatiotemporal modeling with a hierarchy that allows capturing detailed intra-sector and inter-sector relationships efficiently. The strategic avoidance of distortion-heavy convolutions across the polar dimensions ensures that computational resources are optimized without the necessity for bias-heavy heuristics or positional encodings.

Implications and Speculations on Future Developments:

The approach highlighted in this paper not only advances the current methodologies in LiDAR processing but also opens new avenues for deploying complex models in edge applications where computational resources and latency are constrained. By leveraging the native streaming capacity of LiDAR sensors, PHiM underscores the potential for SSMs in transforming real-time perception systems.

The theoretical implications of this research extend to future exploration of long-horizon operation scenarios across heterogeneous environments, potentially influencing multimodal fusion strategies in autonomous systems. Future improvements might focus on integrating this architecture in a multimodal framework that can simultaneously process different data types, such as camera feeds, to build a more holistic perception system.

Conclusion:

Polar Hierarchical Mamba signifies a significant evolutionary step towards efficient and practical LiDAR object detection models for autonomous systems. Its pronounced gains in efficiency and performance, achieved by circumventing conventional processing paradigms, indicate a robust direction for future research and application in real-time perception technologies. Further exploration of SSM integration may yield state-of-the-art advancements applicable to a broad spectrum of sensor modalities beyond LiDAR.