4D mmWave Radar Dataset Overview

• 4D mmWave radar datasets are comprehensive collections capturing range, azimuth, elevation, and Doppler velocity via FMCW sensors, enabling robust detection and tracking.
• They facilitate advanced research in automotive, robotics, and SLAM applications by integrating synchronized radar, camera, and LiDAR data for multi-modal perception.
• Annotated with object-level ground truth and processed through FFTs, beamforming, and clustering, these datasets benchmark performance under adverse conditions.

A 4D mmWave radar dataset comprises measurement data from automotive or robotic frequency-modulated continuous-wave (FMCW) radar sensors providing per-point range, azimuth, elevation, and Doppler velocity, captured in traffic, indoor, marine, or other operational scenarios. These resources serve as the foundational benchmarks for 3D object detection, multi-object tracking, SLAM, panoptic perception, and sensor fusion in conditions where cameras and LiDARs may be impaired by occlusion, weather, or limited illumination. Public 4D mmWave radar datasets typically package synchronized radar, camera, and optionally LiDAR data, annotated with object-level ground truth (e.g., 3D bounding boxes, tracks, or occupancy voxels), and span diverse environments including urban streets, highways, industrial zones, indoor spaces, and maritime domains.

1. Principles of 4D mmWave Radar Data Acquisition

4D mmWave radar systems sample the reflected RF spectrum via an FMCW chirp protocol and MIMO antenna array, enabling high-resolution estimation of:

• Range ($r$): Determined via fast-time (range) FFT across ADC samples.
• Azimuth ($\theta$) and Elevation ($\phi$): Recovered by digital beamforming (angle FFT or Capon/MUSIC) across virtual channels.
• Doppler velocity ($v_D$): Extracted from slow-time (chirp) FFT, yielding relative radial velocities per point.

Typical raw outputs are either tensor-format (ADC cubes: $R[n_{r}, n_{d}, n_{a}, n_{e}]$), or post-processed, sparse point clouds with Cartesian $[x,y,z]$, $v_D$, and reflectivity attributes. Processing includes range/Doppler/angle FFTs, multichannel calibration, CFAR detection for clutter suppression, and clustering for point grouping. The radar hardware is characterized by parameters including carrier frequency (76–81 GHz), bandwidth (1–4 GHz), virtual channel count (16–128+), range and angular resolutions ($\sim$3–10 cm, 1–5°), and frame rates (5–20 Hz) (Wu et al., 23 Sep 2025, Guan et al., 2024, Peng et al., 31 Mar 2025).

2. Public Datasets: Scope, Modalities, and Specifications

Object Detection and Tracking

Key public 4D mmWave radar datasets include:

Dataset	Sensor Model	Az/El Res.	Env.	Labels	Frames
View-of-Delft (VoD)	ZF FRGen21	1–1.5°	Urban, campus	3D boxes, tracks (Car, Ped, Cyc)	8,682
TJ4DRadSet	Oculii Eagle	2°	Urban, highway	3D boxes, tracks (Car, Ped, Cyc, Truck)	7,757
K-Radar	TI 4D @ 77 GHz	1°	Adverse WX	3D boxes, tracks	35,000
aiMotive	(unspecified)	–	Mixed WX	3D boxes, tracks	26,500
V2X-Radar	Oculii/Arbe Phoenix	$\sim$1.5°	Urban/co-op.	3D boxes (Car, Ped, Cyc, Bus, Truck)	20,000

VoD provides five-scan accumulated point clouds, enabling denser frames for LiDAR-comparable performance, while TJ4DRadSet offers sparser single-sweep data containing a wide elevation FOV and challenging illumination. Additional datasets target long-range, multi-agent, large-scale, or adverse weather coverage (e.g., L-RadSet, Dual Radar, Bosch Street) (Wu et al., 23 Sep 2025, Han et al., 2023, Peng et al., 31 Mar 2025, Yang et al., 2024).

SLAM and Mapping

SLAM-oriented datasets such as MSC-RAD4R (urban/suburban, 90k frames), NTU4DRadLM, and DIDLM provide synchronized radar, LiDAR, GPS/IMU, and sometimes IR/depth modalities. They focus on large-scale localization, trajectory estimation, and mapping in adverse weather, low light, and challenging surfaces (Gong et al., 2024, Wu et al., 2024).

Special Contexts

WaterScenes (Oculii Eagle, 54k frames), RadarRGBD (TI AWR2243), and mmFall (TI AWR1843, indoor gait/fall detection) extend 4D mmWave datasets to maritime navigation, indoor RGB-D fusion, and health monitoring, respectively (Guan et al., 2023, Song et al., 21 May 2025, Jin et al., 2020).

3. Data Structure, Annotation, and Calibration

Typical 4D radar point cloud data is organized as variable-length $N \times k$ arrays per frame (e.g., $N$ points $\times$ 7–9 features: $[x,y,z,v_D,$ RCS/SNR, angle, time, \dots]$$), sometimes as higher-dimensional spectra. Annotation formats include:</p> <ul> <li><strong>3D bounding boxes:</strong>$$[x_c, y_c, z_b, l, w, h, \theta]$

Track IDs: for temporal association and MOT

Per-point/voxel semantic labels: WaterScenes, RadarRGBD

Explicit cross-calibration: Rigid-body transforms between radar, camera, LiDAR, and vehicle/world frames; timestamp alignment via hardware trigger, GNSS/IMU synchronization

Ground truth is obtained either by co-registered LiDAR with manual annotation or by camera-guided 3D projection and review. Dataset splits (train/val/test) vary—some datasets (VoD, TJ4DRadSet) provide official partitions, others require protocol agreement (Wu et al., 23 Sep 2025, Guan et al., 2024, Yang et al., 2024).

4. Preprocessing and Mathematical Formulations

Common pipelines include:

• Ego-motion compensation: For multi-scan accumulation, radar points are compensated using synchronized vehicle pose estimates to suppress ghosting (VoD: five-scan accumulation) (Wu et al., 23 Sep 2025).
• Hard spatial filtering: Applied to all datasets prior to voxelization or grid construction.
• Instance-guided densification: Radar points projected onto camera or segmentation masks, with Gaussian/uniform sampling and nearest-neighbor depth assignment to augment sparse returns, producing virtual points for feature learning (Wu et al., 23 Sep 2025).
• Voxelization and attention encoding: Voxel grids (resolution $\Delta x, \Delta y, \Delta z$) hold limited point counts per cell, with triple-attention mechanisms learning point-, channel-, and voxel-wise weights for robust representation (Eqs. 8–14 in (Wu et al., 23 Sep 2025)).
• Kernel density estimation (KDE): Used in SMURF to construct multi-dimensional density features per point, denoising and compensating for scatter (Liu et al., 2023).
• Clustering and filtering: DBSCAN employed for static/dynamic separation (e.g., EFEAR-4D Doppler-circle fitting for ego-velocity) (Wu et al., 2024).
• Projection and coordinate transforms: Rigid and affine mappings standardize radar, LiDAR, and image geometry for fusion and annotation (Yang et al., 2024, Wu et al., 23 Sep 2025).

5. Dataset Utility and Benchmark Performance

Fusion and radar-centric algorithms evaluate datasets using tasks such as 3D detection, tracking, panoptic segmentation, SLAM, and referring expression comprehension. Performance is reported as 3D mAP/BEV mAP at various IoU thresholds per class and area (e.g., entire annotated area [EAA], driving corridor [DCA]):

• MLF-4DRCNet (VoD): EAA mAP 60.28%, DCA mAP 82.57%; DCA mAP exceeds LiDAR PointPillars baseline (Wu et al., 23 Sep 2025).
• SMURF: Multi-representation fusion delivers state-of-the-art BEV/3D mAP on radar-only inputs (Liu et al., 2023).
• RadarNeXt: Demonstrates radar-only 3D mAP of 50.48 (VoD five-scan) and high inference throughput (>67 FPS on RTX A4000) (Jia et al., 4 Jan 2025).
• Talk2Radar: Establishes a benchmark for 3D referring expression comprehension on 4D radar with language grounding (Guan et al., 2024).
• WaterScenes: Multi-task segmentation (PointNet++) achieves mIoU ≈ 60.7% for point-cloud semantic segmentation (Guan et al., 2023).
• DIDLM, MSC-RAD4R (SLAM): Baselines are set for 3D mapping error (ATE ≈ 0.15 m/100 m, RMSE ≈ 0.20 m depending on sequence/weather) (Gong et al., 2024).

Reported results demonstrate 4D mmWave radar resilience, with only modest drops in adverse scenarios where camera/LiDAR degrade by 30–50% (Peng et al., 31 Mar 2025).

6. Expansion, Simulation, and Limitations

Efforts to overcome hardware and scenario limitations include:

• Dataset simulation: Neural pipelines (DIS-Net, RSS-Net) synthesize realistic 4D radar from LiDAR/camera, generating point clouds and signal strengths statistically matched to real data (Dis-Net KL divergence $D_{KL}=0.154$ on VoD) (Song et al., 11 Mar 2025).
• Scaling and Pretraining: GRT collects 1M samples (29 h) of raw I/Q cube data as a foundation for large radar pretraining; ablations show amplitude-only, AoA, CFAR, and Doppler-omitting representations incur substantial accuracy loss (Huang et al., 15 Sep 2025).
• Coverage gaps: Many datasets omit full publication of RF parameters (bandwidth, FMCW chirp, angular resolution), limiting absolute performance comparison and low-level algorithmic development (Han et al., 2023).
• Sparsity and clutter: Standalone radar performance is constrained by point sparsity, measurement noise, and multi-path/ghosting; modern pipelines use multi-modal densification, KDE filtering, and attention-based encoding as mitigation (Wu et al., 23 Sep 2025, Liu et al., 2023).
• Resolution constraints: Even state-of-the-art single-chip radars deliver 8–12 azimuthal bins and coarse elevation, stressing the need for robust downstream architectures or fusion (Huang et al., 15 Sep 2025).

7. Access, Licensing, and Future Directions

The majority of published 4D mmWave radar datasets are available under research-consent, CC-BY, or academic licenses, with data and tooling (calibration, preprocessing, loader scripts) released on GitHub or institutional pages (Guan et al., 2024, Song et al., 21 May 2025, Yang et al., 2024). Ongoing development trends include:

• Broader environment coverage: New datasets target extreme weather, long-range, multi-agent (cooperative V2X), marine, and low-light scenarios (Peng et al., 31 Mar 2025).
• Standardization: Increased call for full hardware parameter disclosure (Tx/Rx count, chirp parameters, bandwidth) to support reproducibility and raw tensor/ADC access (Han et al., 2023).
• Raw data utility: Foundational models leveraging raw complex I/Q cubes outperform traditional processed representations, with scaling laws indicating $10^8$–$10^9$ frames may be needed to fully saturate such models (Huang et al., 15 Sep 2025).
• Annotation diversity: Moving from bounding boxes and tracks to per-point/voxel semantic labels and bespoke tasks (e.g., 3D REC, panoptic segmentation) (Guan et al., 2023, Guan et al., 2024).

Overall, 4D mmWave radar datasets have become indispensable in pushing the limits of autonomous perception, robust mapping, and intelligent multisensor fusion—particularly in challenging, safety-critical, and previously inaccessible domains (Wu et al., 23 Sep 2025, Han et al., 2023, Peng et al., 31 Mar 2025).