Capacity for Orthogonal Raters Before Diminishing Returns

Determine how many orthogonal capability-specific raters the SkillRater framework can support before diminishing returns in downstream performance, in order to inform how finely to decompose capabilities for multimodal training.

Background

SkillRater decomposes data filtering into multiple capability-aligned raters and demonstrates that three learned raters (visual understanding, OCR, and STEM reasoning) produce near-orthogonal signals with measurable gains over monolithic scoring. The authors currently instantiate three capability dimensions and observe strong complementarity.

However, the optimal number of independent raters that yields benefits without over-fragmenting the data signal is not established. Determining this capacity is important for guiding practical decomposition granularity and for understanding limits of orthogonalization in capability-targeted data selection.