Scale and architecture generalization of trajectory geometry

Ascertain whether the same step‑indexed reasoning‑trajectory geometric organization found in Llama 3.1 8B variants persists at larger model scales and across architecturally distinct model families.

Background

The study analyzes Base, Instruct, and reasoning‑distilled variants of Llama 3.1 8B and reports consistent step‑structured geometry across these training regimes. However, the analysis is confined to this model family and scale.

The authors explicitly state that they have not verified whether this geometric organization carries over to larger models or to different architectures, leaving cross‑scale and cross‑architecture generalization unresolved.

References

Although the consistency of trajectory-level phenomena across three substantially different post-training objectives suggests that these structures are not paradigm-specific, we have not verified whether the same geometric organization holds at larger scales or across architecturally distinct model families.

— LLM Reasoning as Trajectories: Step-Specific Representation Geometry and Correctness Signals  (2604.05655 - Sun et al., 7 Apr 2026) in Limitations, paragraph 2