Conditions under which unguided rollouts reach guidance-like computations

Ascertain whether, and under what conditions, an unguided rollout from a base large language model will sample traces that perform computations similar to those induced by an oracle-provided guidance prefix, particularly when the guidance segment required for success is long.

Background

The transfer from guided to unguided problems in POPE hinges on the model encountering or approximating states similar to those reached under guidance. The authors note that, especially when long guidance is needed, it may be difficult for the base model to sample comparable computations without conditioning.

Understanding when such states are reachable without conditioning would clarify the limits of POPE’s transfer and inform the design of guidance strategies and curricula.