Safety-Oriented Spatial Reasoning

• Safety-Oriented Spatial Reasoning is a framework that enforces spatial safety by modeling occlusions, hazards, and uncertainties using set-based reachability and safety metrics.
• It employs explicit sensor coverage modeling and worst-case occupancy prediction in autonomous driving to guarantee collision avoidance under ambiguous conditions.
• SOSR extends to language and multimodal systems by verifying spatial instructions and UI layouts, ensuring robust safety even amidst adversarial and uncertain inputs.

Safety-Oriented Spatial Reasoning (SOSR) encompasses a class of computational reasoning and verification strategies aimed at ensuring that automated agents—whether physical robots, vehicles, or digital decision engines—operate in a manner that provably respects safety constraints grounded in spatial configuration, occlusion, perception limits, and the consequences of discrete actions. Originating in safety-critical domains like autonomous driving and robotic control, SOSR methods generalize beyond classical spatial reasoning by explicitly codifying the avoidance of both known and potentially unknown hazards, often under conditions of partial observability, ambiguous instructions, or adversarial scene construction. The approach unites set-based verification, safety-oriented metrics, formal evaluation over language and multimodal models, and structural UI/environment understanding to maximally reduce risk of catastrophic failure.

1. Set-Based SOSR for Autonomous Driving

Set-based SOSR methods, as exemplified by (Orzechowski et al., 2018), formalize spatial safety as the absence of trajectory-occupancy intersections under the worst-case assumptions about occlusions and hidden obstacles. The workflow begins with explicit modeling of the ego vehicle’s sensing field, typically as a polygonal region computed from sensor coverage (lidar, radar, camera) or precomputed visibility maps. The field’s boundary is segmented into “critical sensing-field edges,” retained only if they represent loci from which an unobserved obstacle could influence the ego’s trajectory—i.e., on valid lanelets with topological/train-of-right-of-way alignment.

For each critical edge, the method constructs over-approximated state intervals for hypothetical obstacles, modeling potential $(x,y)$ pose, orientation $\psi$, and speed $v$ as independent bounded intervals: $$x(0)\in[\underline{x},\overline{x}],\ y(0)\in[\underline{y},\overline{y}],\ \psi(0)\in[\psi_{\min},\psi_{\max}],\ v(0)\in[v_{\min},v_{\max}]$$ or $$\mathbf{s}(0)\in[\underline{\mathbf{s}},\overline{\mathbf{s}}]$$ in Cartesian space. These intervals are then propagated forward by set-based reachability methods.

Two reachability models underpin occupancy prediction:

• M₁ (Acceleration-based occupancy): Uses Kamm’s circle to over-approximate future positions from all admissible initial states and velocities, constructing unions and convex hulls as conservative enclosures.
• M₂ (Lane-following occupancy): Extends M₁ along the lane network by estimating maximal longitudinal advancement under piecewise acceleration limits.

A candidate ego-trajectory $T_e$ is partitioned into an intended action segment $T_\text{int}$ and a fail-safe segment $T_\text{fail}$ that guarantees entry into a “safe state.” The core safety guarantee is expressed as: $T_e(\tau_k)\cap\mathcal{O}_i(\tau_k)=\emptyset$ for every obstacle $i$ and time-slice $\tau_k$. These intersections are efficiently tested using polygonal operations.

The recursive safety property, proven inductively, establishes that so long as $T_\text{fail}$ is always valid and pre-verified, the system can never be forced into a state where no collision-free action exists. This principle, coupled with exclusive use of over-approximating operations, yields provable soundness: no possible real-world collision configuration could escape detection by the verification process (Orzechowski et al., 2018).

Empirical evaluation in urban intersection scenarios demonstrates that this approach preempts failures due to occlusion and guarantees a valid escape maneuver even under adversarially delayed perception.

2. Safety-Oriented Metrics and Spatial Constraints in Perception

Safety-Oriented Spatial Reasoning also encompasses metric and constraint-driven approaches to perception, as in the Uncompromising Spatial Constraints (USC) framework (Liao et al., 2022). While traditional object detectors are evaluated by accuracy and mean average precision, these metrics do not directly penalize unsafe under-coverage—where the predicted bounding box fails to fully cover a true object, potentially leading to unanticipated collisions.

USC metricization establishes dual constraints:

• Perspective-View (PV) enclosure: The ground-truth object projection must be fully contained in the detector’s prediction, $$\mathbf{G}^{\mathcal{PV}}\subset\mathbf{P}^{\mathcal{PV}}$$.
• Bird’s Eye View (BEV) under-estimation: The closest points and AV-facing edges of ground-truth and predicted boxes must not expose any risky region closer to the AV than predicted, enforcing $\|v_P^c\|\leq\|v_G^c\|$ and preventing edge crossing.

Two scalar metrics quantify this safety margin:

• IoGT (Intersection-over-Ground-Truth in PV): Fraction of the true box covered (no penalty for safe over-coverage), $[0,1]$.
• ADR (Average Distance Ratio in BEV): Penalizes closeness of AV-facing real corners not shielded by the predicted box.

Their product, $\text{USC}=\text{IoGT}\times\text{ADR}$, correlates more strongly with actual collision rates ($r\approx0.93$ in closed-loop AV tests) than classical metrics. USC can be directly integrated into fine-tuning via loss augmentation, ensuring that learning-based perception systems not only achieve high detection accuracy but also minimize spatial underestimation risks (Liao et al., 2022).

3. Language-Based and Multimodal SOSR in Safety-Critical Decision-Making

Recent advances evaluate SOSR in LLM and multimodal model settings, where spatial safety cannot rely on geometric map data or precise sensor inputs, but must instead be inferred from unstructured or multimodal representations (Han et al., 9 Jan 2026, Liu et al., 26 Sep 2025).

In (Han et al., 9 Jan 2026), SOSR is operationalized as a set of natural-language decision tasks where safety hinges on spatial understanding:

• Input: Scenario $S$ in natural language; Output: decision $a$ from finite $\mathcal{A}$
• Predicate: $\mathrm{safe}(S,a)$ determines if $a$ preserves safety constraints
• Metric: Success Rate (SR), i.e., fraction of safe decisions across $N$ trials

Subtasks probe direction inference under sequential instructions (with or without distractors), and evacuation decisions in hazardous settings, requiring models to avoid secondary objectives or hallucinated options. Failure to meet the safety predicate in even 1% of decisions is shown to constitute unacceptable risk for real-world robotic deployment.

In multimodal domains, as with WebRSSBench (Liu et al., 26 Sep 2025), SOSR tasks involve spatial relation inference between UI elements and detecting safety-critical buttons (leading to irreversible or high-risk actions). Accuracy in relationships and recall in safety-critical detection are reported as:

Model	Relational Accuracy (Hard)	Safety Recall
GPT-5	41.3%	71.1%
Gemini 2.5-Pro	44.2%	91.1%
Qwen2.5-VL-7B	6.8%	91.1%

Systematic weaknesses include over-reliance on visual salience, fragility to small layout perturbations, and both conservative and reckless errors in safety detection (Liu et al., 26 Sep 2025).

4. Evaluation Protocols and Performance Analysis

SOSR performance is quantified with application-specific metrics aligned to outcome safety. In set-based trajectory verification, the absence of candidate–occupancy intersections provides a binary certificate of safety for each plan and time horizon (Orzechowski et al., 2018). Perception modules integrate the USC metric, reporting aggregate mAUSC and the hybrid USC-NDS to reflect both coverage and standard detection performance (Liao et al., 2022).

In language-based and GUI-oriented benchmarks:

• For LLMs, success rate and risk are computed as: $$\mathrm{SR} = \frac{1}{N}\sum_{i=1}^N I_i, \quad \text{where } I_i=1 \iff \mathrm{safe}(S_i,a_i)=1.$$
• For UI tasks, accuracy in spatial relations and recall in safety detection are central: $$\mathrm{Acc}_{\rm pos} =\frac1{N_{\rm pos}} \sum_{i=1}^{N_{\rm pos}} \mathbf{1}\{\hat r_i=r_i^*\};\quad \mathrm{Recall}_{\rm safe} =\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}$$

Empirical studies confirm that traditional accuracy and mAP can be misleading for safety: models achieving 99% overall accuracy may still incur unacceptable absolute rates of catastrophic errors in SOSR tasks (Han et al., 9 Jan 2026).

5. Failure Modes, Limitations, and Countermeasures

Documented failure patterns in SOSR include:

• Unsafe decision prioritization: LLMs sometimes recommend actions leading toward hazards or hallucinated entities, driven by spurious alignment to secondary goals (e.g., retrieving data rather than evacuation in fire scenarios) (Han et al., 9 Jan 2026).
• Distractor susceptibility: Inclusion of irrelevant sentences in language-based direction-following can degrade accuracy, even in high-capacity models.
• Nondeterminism: Outputs for identical SOSR prompts can be inconsistent, undermining reliability.
• Compositional weaknesses: Multimodal models exhibit sharp accuracy drops under minor layout changes or visually similar decoys (Liu et al., 26 Sep 2025).
• Conservative or reckless safety judgment: Some models default to “no risk” (failing to flag dangers), others to false positives, with substantial model-to-model variance.

Suggested mitigations include explicit downstream safety-verification layers (enforcing $\mathrm{safe}(S,a)$ checks), refusal or abstention policies, richer spatial grounding (e.g., mapping language into intermediate representations), adversarial/counterfactual fine-tuning, and step-wise spatial reasoning scaffolds (Han et al., 9 Jan 2026, Liu et al., 26 Sep 2025).

6. Implications and Research Directions

Safety-Oriented Spatial Reasoning establishes that, in safety-critical systems, guarantees must extend to the worst-case feasible hazards, not merely averaged risk or mean accuracy. The set-based framework in autonomous navigation demonstrates the viability of provable soundness and recursive safety without resorting to probabilistic surrogates (Orzechowski et al., 2018). However, challenges in integrating SOSR into deep learning and LLM-driven systems include the inability of prevailing models to (a) consistently respect safety constraints in language-only prompts, and (b) maintain robust spatial grounding under adversarial scenarios (Han et al., 9 Jan 2026, Liu et al., 26 Sep 2025).

Proposed research avenues emphasize the integration of symbolic safety checks into learning architectures, dataset expansion to encompass counterfactual and adversarial hazards, enforcement of model abstention in high-risk ambiguous cases, and hybrid techniques blending formal set-based verification with data-driven perception and language understanding.

Advancements in SOSR are required for closing the gap between raw task performance and real-world dependability in autonomous vehicles, robotics, web agents, and any decision-making context in which spatially-grounded safety is non-negotiable.