Effect of Instruction Tuning on Geometric Stability of Deception

Determine whether training large language models to follow instructions induces geometric fragility of deceptive states in the models’ representation space, thereby making honest responses more robust and prevalent.

Background

The paper shows that across multiple model families and datasets, requiring models to generate deliberative tokens before answering increases the likelihood of recommending the honest option. The authors argue this effect is not fully explained by the semantic content of reasoning traces and propose a geometric account: deceptive outputs correspond to metastable, narrower regions in representation space that are more easily disrupted by paraphrasing, resampling, or activation noise.

Building on this geometric perspective, the authors identify a forward-looking question regarding training: beyond inference-time reasoning, does instruction-following training itself shape the representation space so that deceptive states become more fragile and honesty becomes the more stable default? They highlight that their experiments focus on post-trained models and explicitly raise this as a question for future work.