Predict performance thresholds that trigger monitorability drops

Ascertain the performance thresholds on the outcome reward R_out and the Chain-of-Thought reward R_cot at which monitorability degradation is observed in practice during reinforcement learning post-training.

Background

The framework predicts that monitorability issues appear when models achieve sufficiently high performance on both outcome and CoT rewards in in-conflict settings. However, the exact levels of performance at which degradation occurs are not characterized.

The authors explicitly state it is unclear whether monitorability drops will be observed in practice without knowing these thresholds, highlighting the need for methods to predict such failures of optimization.