CLEAR: Cognition and Latent Evaluation for Adaptive Routing in End-to-End Autonomous Driving

Abstract: End-to-end autonomous driving models often struggle to balance multi-modal maneuver generation with real-time inference constraints. While diffusion models successfully capture diverse driving behaviors, their iterative denoising process incurs unacceptable latency for safety-critical deployment. To address this, we propose CLEAR (Cognition and Latent Evaluation for Adaptive Routing), a framework that combines ultra-fast generative planning with deep semantic reasoning. CLEAR employs Drive-JEPA as the visual encoder and replaces the multi-step denoising chain with a single-step conditional drift in a VAE latent space, introducing a conditioning coefficient to balance diversity and expert precision. Meanwhile, we fully fine-tune Qwen~3.5~0.8B on driving QA pairs to extract scene-aware hidden states. These states guide both an Adaptive Scheduler, which selects the conditioning coefficient $α$ and sample count $N$ from a discrete set of predefined schemes, and a cross-attention scorer that selects the optimal trajectory from candidates. On the NAVSIM v1 benchmark, CLEAR achieves a state-of-the-art PDMS of 93.7. Our results demonstrate that high-fidelity, multi-modal planning can be executed efficiently without dense geometric annotations or iterative sampling.

• The paper introduces a novel single-step drift mechanism that replaces iterative diffusion for rapid, kinematically consistent trajectory generation.
• It leverages cognition-driven adaptive scheduling and LLM-based cross-attention scoring to dynamically balance trajectory diversity and precision.
• Empirical evaluations on NAVSIM benchmarks show state-of-the-art safety with 93.7 PDMS and efficient real-time performance at up to 99 FPS.

CLEAR: Cognition and Latent Evaluation for Adaptive Routing in End-to-End Autonomous Driving

Motivation and Problem Statement

End-to-end autonomous driving frameworks have shifted towards learning unified policies that map sensory and intent inputs directly to feasible trajectories. The primary challenge in such systems is reconciling the need for multi-modal trajectory reasoning—critical in ambiguous or densely interactive urban environments—with the stringent real-time constraints required for safety-critical applications. Existing planners based on diffusion models are able to capture behavioral diversity through iterative denoising but at the cost of inference latency that exceeds practical margins. Moreover, leveraging large vision-LLMs for semantic understanding introduces additional delays if not architecturally constrained. The CLEAR framework addresses these latencies by decoupling geometric and semantic planning via a fast latent-space generative mechanism tightly coupled with cognitive scene understanding, using drive-oriented JEPA visual encoders and a task-specialized LLM.

Figure 1: CLEAR framework: frozen Drive-JEPA and fine-tuned Qwen~3.5~0.8B extract visual/semantic features; the Adaptive Scheduler predicts $(\alpha, N)$; the CLEAR Decoder generates $N$ trajectory candidates via conditional drift; the Cross-Attention Scorer selects the optimal candidate.

Architecture and Methodological Contributions

The core of CLEAR consists of three synergistic design elements:

Single-Step Conditional Drift Generation

CLEAR replaces the iterative denoising sequence typical of diffusion-based trajectory samplers with a single-step drift in a structured VAE latent manifold. The latent space is organized using an auxiliary classifier into driving primitives, with a pre-fitted PCA projection enabling efficient and kinematically consistent decoding of physical trajectories. A key parameter, the conditioning coefficient $\alpha$, modulates the drift endpoint between geometric attractors and expert demonstrations:

• $\alpha \to 0$ maximizes trajectory diversity (multi-modal coverage).
• $\alpha \to 1$ yields high-precision, expert-matching (deterministic) behavior.

Multi-candidate repulsion further ensures multi-modal support by preventing mode collapse—a common pitfall in single-step generative frameworks.

Cognition-Driven Adaptive Scheduling and Scoring

Rather than using the LLM as a trajectory generator, CLEAR refines and leverages the scene-level hidden representations of the compact Qwen~3.5~0.8B, which is fully fine-tuned on closed-domain driving QA. These serve two roles:

1. The Adaptive Scheduler selects from a finite, supervised set of $(\alpha, N)$ schemes, dynamically trading off sample count and diversity based on contextual complexity.
2. The Cross-Attention Scorer uses the same LLM representations to evaluate trajectory candidates through learned, context-sensitive ranking rather than fixed geometric heuristics.

Intent-Driven Efficient Decoder

Physical candidate trajectory generation is achieved with an MLP-Mixer core, with per-scene adaptive layer normalization controlled by the $\alpha$ forecast. Dense vision streams are first collapsed with learnable scene queries, assuring information compactness and intent conditioning.

Figure 2: Training progression: high drift ($\alpha=0.9$) yields mode collapse to ground-truth; low drift ($\alpha=0.1$) maintains broad geometric modality; intermediate $\alpha$ interpolates between the two.

Empirical Evaluation and Analysis

Benchmarks

CLEAR is evaluated on the NAVSIM v1 and v2 closed-loop driving benchmarks, focusing on PDM Score (PDMS) and Extended PDMS (EPDMS), which aggregate collision safety, comfort, progress, compliance, and anticipation metrics without dependency on dense geometric labels. Key findings:

• State-of-the-art closed-loop safety and quality: CLEAR achieves 93.7 PDMS on v1, surpassing DriveSuprim and Drive-JEPA, with notable improvements in safety-critical sub-metrics such as time-to-collision (TTC: 97.2).
• Strong generalization: On v2, CLEAR attains the highest EPDMS (88.6) among ViT/L-scale methods, and leads on NC, DAC, EP, and EC, indicating robust physical and semantic scene understanding.
• Efficiency: Sample generation operates at up to 99 FPS, with single-pass decoding introducing negligible latency, critical for deployable safety systems.

Training Dynamics and Drift Analysis

Latent distribution analysis throughout training demonstrates that the decoupled drift mechanism precisely realizes the intended coverage/precision trade-off controlled by $N$0. Early training yields scattered modality with little topological alignment; late training (500 epochs) cleanly separates exploratory and expert modes, with intermediate $N$1 interpolating multi-modality and precision as designed.

Ablation reveals that the addition of the LLM-driven scorer and adaptive scheduler yields additive gains: the scorer alone improves PDMS from 93.1 to 93.3, while full adaptivity further boosts PDMS to 93.7—the highest fidelity and safety.

Theoretical and Practical Implications

This work delivers two significant insights:

• Efficacy of minimal, modular cognitive-geometry coupling: Full geometric and semantic reasoning for real-world driving does not require either massive multimodal LLMs or dense geometric annotation pipelines. Compact, frozen feature extractors with a lightweight, adaptively tuned latent-space generator suffice for state-of-the-art closed-loop planning.
• Drift-based generative matching as latency-optimal: The paradigm of replacing iterative stochastic search (diffusion) with single-step drift (complemented by context-adaptive scoring) stands as an efficient alternative for real-time, high-dimensional control tasks.

Practical implications include reduced infrastructure for dataset construction, significantly relaxed latency constraints for edge deployment, and modular adaptability to scene semantics, enabling rapid adjustment across diverse driving domains.

Limitations and Future Directions

Despite its strengths, CLEAR's discrete set of $N$2 scheduling schemes may not capture the true optimum for novel or out-of-distribution scenes; continuous or differentiable scheduling policies offer potential for further gains. Additionally, modular training—pre-training VAE, PCA bases, LLM, and then downstream scorers—introduces pipeline complexity. End-to-end differentiable training and direct geometric-LLM joint optimization are promising avenues for further efficiency and performance improvements.

Conclusion

CLEAR introduces a unified framework for efficient, multi-modal trajectory planning in end-to-end autonomous driving by fusing fast latent-space generative models with cognitive semantic scene evaluation. It sets a new state-of-the-art for closed-loop planning with minimal reliance on explicit geometric labels or large language/backbone models and demonstrates that inference speed and multi-modality need not be mutually exclusive in practice.