Cooperative Robotics Reinforced by Collective Perception for Traffic Moderation

Abstract: Collisions at non-line-of-sight (NLOS) intersections remain a major safety concern because drivers have limited visibility of approaching traffic. V2X based warnings can reduce these risks, yet many vehicles are not equipped with V2X and drivers may ignore in vehicle alerts. Collective perception (CP) can compensate for low V2X penetration by extending the awareness of connected vehicles, but it cannot influence unconnected vehicles. To fill this gap, our work introduces a complementary concept that adds a cooperative humanoid robot as an active traffic moderator capable of physically stopping a vehicle that attempts to merge into an unseen traffic stream. The system operates on two parallel perception pathways. A dual camera infrastructure unit detects the position, speed and motion of approaching vehicles and transmits this information to the robot as a collective perception message (CPM). The robot also receives cooperative awareness messages (CAM) from connected vehicles through its onboard V2X unit and can act as a relay for decentralized environmental notification messages (DENM) when safety events originate elsewhere along the road. A fusion module combines these streams to maintain a robust real time view of the main road. A Zone of Danger (ZoD) is defined and used to predict whether an approaching vehicle creates a collision risk for a merging road user. When such a risk is detected, the robot issues a human-like STOP gesture and blocks the merging path until the hazard disappears. The full system was deployed at the Future Mobility Park (FMP) in Rotterdam. Experiments show that the combined vision and V2X perception allows the robot to detect approaching vehicles early, predict hazards reliably and prevent unsafe merges in real world NLOS conditions.

• The paper introduces a novel robotic traffic moderator that fuses infrastructure vision, V2X, and onboard sensing to prevent unsafe merges.
• It deploys a dual-camera system with YOLOv12 and V2X data fusion to forecast hazards with a CPM latency of 1.3 seconds, ensuring timely intervention.
• The research demonstrates that physical robot actuation effectively influences driver behavior and extends safety to unequipped vehicles.

Cooperative Robotics Reinforced by Collective Perception for Traffic Moderation

Motivation and Problem Statement

The persistent risk of collisions at non-line-of-sight (NLOS) intersections stems from limited driver visibility and insufficient prioritization signaling in unsignalized merging scenarios. Even modern advanced driver-assistance systems (ADAS) and connected vehicles fail to guarantee situation awareness when visibility is obscured, and penetration of V2X-enabled vehicles remains insufficient to rely on cooperative perception exclusively. Existing methods that use infrastructure-vision sensing or V2X communication only partially address the problem, either by supporting only equipped vehicles or serving an advisory role with no direct physical intervention capabilities.

This paper proposes a unified framework combining infrastructure-based vision sensing, V2X communication, and a physically embodied humanoid robot, ARI, that actively moderates traffic merging—a novel approach to bridging gaps between equipped and unequipped participants.

System Architecture and Technical Components

The system comprises two parallel perception channels. The dual-camera infrastructure unit executes real-time detection of oncoming traffic, projecting bounding boxes onto the road reference line and generating CPMs. These CPMs encode position, velocity, and class attributes for each tracked object and are transmitted to ARI via V2X (IEEE 802.11p DSRC).

ARI is a mobile humanoid robot equipped with a front-mounted RGB-D camera (Intel RealSense D435), 2D LiDAR for SLAM-based localization, a differentiated-drive base, multi-DOF arms for gesture actuation, and a dedicated V2X OBU supporting ETSI-compliant CAM and CPM protocols. Onboard vision inference employs a lightweight YOLOv12 model, supporting real-time detection and robust classification of vehicles and cyclists, even in high-variance and occlusion-heavy environments.

Figure 1: ARI robot showing perception modules, V2X hardware, and actuators for STOP gesture execution.

The fusion module on the robot receives CPMs from the infrastructure, CAMs from vehicles, and DENMs from remote safety units. It builds a consolidated, real-time environmental model, projecting all object states onto an annotated road reference line. Priority-based data fusion trusts V2X information foremost and supplements with vision-based detections after redundancy checks. A scenario-based hazard prediction module formalizes the Zone of Danger (ZoD), approximating the conflict region and forecasting entry/exit timing for all detected objects.

Figure 2: System overview for NLOS road merging with roadside vision sensing, V2X communication, and ARI robotic moderator.

Perception, Fusion, and Traffic Moderation Logic

The infrastructure perception system comprises two Intel D455 cameras, with image processing performed by a Jetson AGX Orin. YOLOv12 enables efficient vehicle detection, with bounding box projections mapped onto the calibrated road reference. Polynomial regression models estimate distance from camera-calibrated positions, and tracked object velocities are calculated using consecutive timestamped estimates.

Figure 3: Output of dual-camera infrastructure system; vehicle tracking with projected bounding boxes on road reference line.

V2X-based data exchange utilizes ETSI-standardized message protocols. The combined fusion module on ARI maintains an updated environmental topology, favoring low-latency V2X packets and supplementing perception gaps via camera-based detections.

The core moderation logic uses a region-based conflict predictor instead of unreliable ToC metrics in non-aligned observation geometries. ZoD is parameterized to account for braking distance at velocities up to $80\,\mathrm{km/h}$, typical driver reaction time, and ARI’s actuation latency. When an oncoming vehicle is predicted to enter the ZoD within the temporal threshold ($\tau_{\mathrm{th}} = 5\,\mathrm{s}$) and a merging vehicle is detected, ARI initiates STOP gesture actuation: moving to the center of the lane and signaling for merging vehicles to halt.

Figure 4: Experimental deployment at FMP; site map, camera placement, and ARI executing STOP gesture at the merging gate.

Experimental Evaluation

Deployment at the Future Mobility Park (FMP) involved a four-hour real-world test with mixed traffic. The system recorded 798 vehicles, with the infrastructure cameras achieving an average first detection distance of $110.1\,\mathrm{m}$, ensuring enough lead time for hazard intervention.

A V2X-equipped VW ID.4 was tracked in repeated runs for direct evaluation. ARI initiated STOP gesture at ZoD entry and maintained lane blockage until the vehicle exited the conflict zone, demonstrating precise and minimal intervention time. Communication-level KPIs, including ARI’s CAM inter-generation gap (IGG), VW’s CAM inter-packet gap (IPG), and CPM latency, remained within prescribed bounds. Notably, CPM latency was measured at $1.3\,\mathrm{s}$, supporting timely perception fusion.

In addition, the DENM repeater capability was validated. ARI received DENMs from a safety vehicle outside the range of the VW ID.4 and immediately rebroadcasted them, enabling vehicles otherwise unreachable to the RSU to be notified of hazards, thus expanding the spatial range of event-driven warnings.

Strong Results and Claims

• Robotic moderation with fused perception consistently prevented unsafe merges in all observed hazardous scenarios. The system reliably detected and forecasted conflicts in real-world NLOS conditions.
• Infrastructure vision and V2X fusion resulted in robust detection of approaching vehicles at $>100\,\mathrm{m}$, with minimal communication delays and precise hazard prediction.
• Physical intervention by the robot directly influenced driver behavior, unlike conventional V2X warnings, thus incorporating unequipped vehicles into the cooperative ecosystem.
• DENM relaying by the robot extended the effective range of hazard notifications, mitigating NLOS communication blind spots.

Practical and Theoretical Implications

The framework demonstrates that cooperative robotics integrated with collective perception greatly expands the operational envelope for traffic moderation at NLOS intersections. Robotic moderation offers a scalable, adaptive solution that physically actuates roadside interventions, moving beyond advisory warnings or limited V2X-based collaborative control.

System limitations include restricted camera-based detection range and robot actuation speed. The authors propose future enhancements via radar integration for extended coverage, refined response thresholds adapting to real-time traffic and environmental parameters, and predictive models for multi-agent interactions.

For practical deployment, safety and robustness concerns arise due to potential robot exposure to vandalism, accidental collisions, and system malfunctions during active intervention. Future system designs should incorporate elevated signaling, collision-proof enclosures, and comprehensive fail-safe protocols.

Theoretically, robotic moderation in traffic environments introduces new approaches to human-robot interaction and autonomous multi-agent negotiation at intersections, promoting further exploration in multi-modal perception fusion, adaptive control, and collaborative robotics for traffic safety.

Conclusion

The study validates a cooperative robotic traffic moderation system based on infrastructure vision and V2X data fusion, deployed at a real-world NLOS merging site. Robust perception, reliable hazard forecasting, and precise physical intervention were exhibited, effectively mitigating collision risks in mixed-equipment traffic. The system’s modularity and mobility position it as a viable solution for diverse intersection layouts, with future work needed to address response time, detection range, and operational safety for broader adoption.