DriveVer: Lightweight Trajectory Evaluator as Test-Time Verifier for Autonomous Driving

Abstract: End-to-end autonomous driving models often encounter performance bottlenecks, as training-time scaling leads to high computational costs and diminishing marginal returns. Existing planners typically adopt a one-shot generation paradigm, lacking secondary validation and active correction mechanisms to detect and revise suboptimal or unsafe trajectories during inference. To address this issue, we propose DriveVer, a lightweight, plug-and-play Test-Time Verifier that leverages the test-time scaling paradigm to enable autonomous driving systems to validate and refine trajectories without costly and heavy training. We construct a dedicated trajectory dataset based on the NAVSIM benchmark through condition-driven clustering and balanced sampling according to ego-vehicle states and navigation commands. Employing a dual-head architecture, DriveVer efficiently fuses candidate trajectories with multi-view visual representations and ego-vehicle kinematic features to simultaneously predict a safety confidence score and an absolute geometric refinement vector. Extensive experiments on the NAVSIM benchmark show that DriveVer significantly improves the performance of base planning models. Notably, as an extremely compact model with only 34M parameters, DriveVer introduces minimal computational overhead, achieving competitive results while maintaining real-time inference efficiency.

• The paper introduces a test-time trajectory verification and refinement module that evaluates and corrects base planner outputs to boost safety.
• It employs a dual-head architecture with confidence scoring and Transformer-based geometric refinement for timely and accurate trajectory adjustments.
• Empirical results on NAVSIM benchmarks demonstrate consistent improvements in safety metrics and sub-100ms inference suitable for real-time deployment.

DriveVer: Lightweight Test-Time Trajectory Verification and Refinement for Autonomous Driving

Introduction and Motivation

The paradigm of end-to-end (E2E) autonomous driving has advanced significantly due to powerful deep learning architectures, yet these models regularly encounter two fundamental challenges: prohibitive training-time scaling costs and the absence of robust test-time evaluation or correction mechanisms. Mainstream E2E planners perform one-shot trajectory generation and directly execute the predicted path, lacking the capacity for on-the-fly safety validation or corrective interventions when confronted with out-of-distribution or adversarial scenarios. This limitation severely constrains both generalization and safety.

DriveVer addresses this deficiency by introducing a test-time trajectory verification and refinement framework. Operable as a plug-and-play module, DriveVer is designed to post-process the output of any E2E base planner—assessing the safety of the initial prediction and applying geometric corrections as warranted. This approach leverages the emerging test-time scaling paradigm, where additional inference-time compute is utilized—not for massive retraining, but for iterative verification and correction, all in real-time.

The DriveVer pipeline is conceptually distinguished from conventional one-shot planners by its insertion of a test-time validation and correction loop (Figure 1).

Figure 1: A comparison between the traditional one-shot trajectory generation and the DriveVer post-hoc refinement paradigm.

DriveVer Architecture

The DriveVer model is architected around two distinct but interdependent functional branches: confidence scoring and geometric refinement. Observational inputs include multi-view visual data, high-level navigation intents, ego kinematic states, and the initial candidate trajectory. Visual features are efficiently encoded via a register-token-based backbone (adapted from DrivoR), providing a compact, information-rich intermediate representation conducive to real-time inference.

The dual-head output branches share cross-modal encoded features but serve orthogonal tasks. The confidence branch yields a scalar safety/confidence score determining the necessity of intervention, while the refinement branch predicts a geometric direction vector indicating how the trajectory should be corrected (Figure 2).

Figure 2: The dual-head DriveVer architecture for joint trajectory evaluation (confidence scoring) and iterative geometric refinement.

Model Internals and Supervision Strategies

Both branches employ Transformer Decoder architectures, allowing rich cross-attention between visual and state/trajectory tokens for scene-adaptive reasoning (Figure 3). Unlike classical selection-based evaluation modules, DriveVer directly outputs a refinement vector in the geometric space of the trajectory, providing a continuous, intent-aware correction mechanism rather than mere candidate selection or rule-based smoothing.

Figure 3: Structure of the Transformer Decoder stack underpinning both the confidence and refinement branches.

For optimization, DriveVer innovates in its supervision signals:

• The confidence branch uses a binary classification target derived from whether a trajectory surpasses an expert-derived quality threshold. This enables DriveVer to learn when test-time intervention is unnecessary, minimizing detrimental over-corrections.
• The refinement branch is trained by minimizing misalignment in the cosine similarity between predicted and ground-truth direction vectors in trajectory space, rather than strict coordinate residuals. This favours robust, global optimization of the path towards expert demonstration, as evidenced by ablation studies.

Dedicated Dataset Construction

A central challenge for test-time verifiers is the absence of finely annotated negative trajectories in existing autonomous driving datasets. DriveVer circumvents this by generating a balanced candidate library across kinematic manifolds and navigational commands, clustered and sampled from the NAVSIM benchmark. Trajectories are partitioned into high- and low-quality subsets via the Predictive Driver Model Score (PDMS), providing dense, unbiased supervision across edge cases, unsafe maneuvers, and diverse driving contexts.

Evaluation and Empirical Findings

Quantitative Performance

DriveVer is empirically validated by integration with several SOTA E2E planners (e.g., DiffusionDrive, DrivoR, AdaThinkDrive, ELF-VLA) on the NAVSIM v1/v2 suites. Analysis focuses on closed-loop, safety-centric metrics: PDMS and the extended EPDMS.

• Integration of DriveVer yields consistent, measurable improvements in PDMS and EPDMS for all base planners. The most pronounced gains are observed for planners with larger residual room for error (e.g., DiffusionDrive), while SOTA systems such as DrivoR still benefit from marginal yet persistent improvements, highlighting the general applicability and reliability of the approach.
• The lightweight design (34M parameters) ensures sub-100ms per-sample inference on commodity GPUs, fully compatible with real-time deployment constraints.

Qualitative Insights

DriveVer's corrections exhibit strong scene-specific semantic awareness, for example:

• Steering away from parked vehicles or encroaching on drivable area boundaries.
• Increasing lateral clearance from lead vehicles to prevent collision.
• Realigning to ground-truth trajectories when the base planner produces significant deviations.

  Figure 4: Overlay of baseline, DriveVer-refined, and expert trajectories in various challenging urban driving cases.

Ablation Studies

• The selective gating of the refinement logic via the confidence branch is essential; indiscriminate refining of already-optimal base planner outputs degrades performance.
• Directional cosine loss for the refinement branch outperforms $L_1$ residual losses, improving generalization and reducing overfitting, especially for small refinements required by high-performing base planners.

Practical and Theoretical Implications

On the practical side, DriveVer tightens the safety upper-bounds of E2E autonomous planning stacks at negligible computational costs, avoiding the diminishing returns and environmental footprint associated with brute-force scaling. This enables a modular upgrade path for refinement and safety enhancing without retraining massive models.

Theoretically, DriveVer demonstrates the effectiveness of lightweight test-time scaling and end-to-end gradient-based geometric refinement for closed-loop autonomous systems. It bridges the gap between corpus-scale offline training and adaptive, real-time self-correction, pointing towards hybrid architectures with dynamic test-time reasoning—an important direction for future safe AI systems.

Potential extensions include:

• Integration with uncertainty-aware or risk-sensitive base planners.
• Incorporation into chain-of-thought or VLA reasoning frameworks for transparent, step-wise driving rationales.
• Progressive test-time iterative refinement beyond single-pass correction, subject to bounded latency budgets.

Conclusion

DriveVer introduces a computationally efficient, interpretable, and generalizable mechanism for test-time trajectory evaluation and refinement in autonomous driving. By decoupling safety-critical verification from end-to-end planner scaling and leveraging a dual-head cross-modal architecture, DriveVer achieves consistent performance gains across diverse planning paradigms. Its construction of a balanced trajectory database, robust training objectives, and modular deployment profile position it as a reference for future research in adaptive, self-verifying AI planning systems.

For further experimental details and full model specification, see "DriveVer: Lightweight Trajectory Evaluator as Test-Time Verifier for Autonomous Driving" (2607.00399).