Deformation-Aware Observation Modeling for Radar-Based Human Sensing via 3D Scan-Depth Sequence Fusion

Abstract: Non-contact radar-based human sensing is often interpreted using simplified motion assumptions. However, respiration induces non-rigid surface deformation of the human body that impacts electromagnetic wave scattering and can degrade the robustness of measurements. To address this, we propose a surface-deformation-aware observation model for radar-based human sensing that fuses static high-resolution three-dimensional scanner measurements with temporal depth camera data to represent time-varying human surface geometry. Non-rigid registration using the coherent point drift algorithm is employed to align a static template with dynamic depth frames. Frame-wise electromagnetic scattering is subsequently computed using the physical optics approximation, allowing the reconstruction of intermediate-frequency radar signals that emulate radar observations. Validation against experimental radar data demonstrated that the proposed model exhibited greater robustness than a depth-sequence-only model under low-signal-quality conditions involving complex surface dynamics and multiple reflective sites. For two participants, the proposed model achieved higher Pearson correlation coefficients of 0.943 and 0.887 between model-derived and experimentally measured displacement waveforms, compared with 0.868 and 0.796 for the depth-sequence-only model. Furthermore, in a favorable case characterized by a single relatively-stationary reflective site, the proposed method achieved a correlation coefficient of 0.789 between model-derived and experimentally measured in-phase-quadrature magnitude variations. These results suggest that our sensor-fusion-based deformation-aware observation modeling can realistically reproduce radar observations and provide physically grounded insights into the interpretation of radar measurement variations.