Ideal Linear Probes: Criteria and Identification

Characterize the properties that define an ideal linear probe β_W for a binary concept W in softmax-based representation spaces where the concept probability is modeled as P(W=1 | λ) = σ(β_W^T λ + b_W), and develop principled procedures to identify or estimate such probes from data.

Background

The paper studies how semantic concepts are encoded in representation spaces that define softmax distributions and focuses on steering such representations using linear probes. A linear probe β_W for a binary concept W is assumed to satisfy a logistic relationship P(W=1 | λ) = σ(β_W^T λ + b_W), aligning with standard logistic regression.

The theoretical results on dual steering’s robustness rely on the availability of a suitable probe; however, the authors explicitly note that identifying what constitutes an ideal probe and how to find it is not currently understood. This uncertainty is significant because the effectiveness and reliability of steering depend critically on probe quality, as further discussed in their probing assumption and empirical sections.