Does curation prevent model collapse in iterative deployment?

Determine whether the curation of valid traces during iterative deployment of large language models fully prevents model collapse, or whether curation merely delays collapse without eliminating it.

Background

The paper studies an iterative deployment mechanism where each new generation of a LLM is fine-tuned on curated valid traces from previous deployments. This process is theoretically connected to REINFORCE with an implicit binary reward and empirically shown to improve planning capabilities.

A key safety concern discussed is model collapse: prior work has shown that training on models’ own synthetic outputs can shrink distributions and degrade capabilities. The authors highlight that while curation might delay collapse by filtering valid outputs, it remains explicitly unknown whether curation fully prevents collapse in iteratively deployed systems.