Relative Classification Accuracy Metrics

• Relative Classification Accuracy (RCA) is a family of metrics that quantify semantic alignment and classifier performance by normalizing intrinsic task difficulty.
• The framework employs calibration steps to compare outputs across groups and domains, aiding in fairness assessments and identity consistency in generative models.
• RCA is applied in fine-grained conditional generation, domain adaptation for segmentation, and multiclass extrapolation, providing actionable insights into model behavior.

Relative Classification Accuracy (RCA) is a family of calibrated metrics developed for measuring classification performance in contexts where traditional accuracy or distributional similarity metrics are insufficient, such as fine-grained generative modeling, domain adaptation for segmentation, multiclass extrapolation, and fairness-driven subpopulation classification. The RCA framework aims to quantify either the semantic alignment of generated or predicted outputs with intended labels, the comparative performance of classifiers across groups or domains, or the extrapolative behavior of classifiers as the class set expands. These metrics introduce normalization or calibration steps to disentangle inherent task difficulty from model-related performance, offering domain-invariant and comparable standards across tasks.

1. Formal Definitions and Mathematical Formulations

RCA is instantiated in several distinct but conceptually aligned forms across the literature:

a) Identity Consistency in Conditional Generation:

In fine-grained image generation (e.g., K-pop face synthesis), RCA quantifies the capacity of a class-conditional generative model to preserve intended semantic identity under the constraints of image fidelity and class ambiguity (Lin et al., 22 Jan 2026). Given an "oracle" classifier $C$ trained on the real dataset:

• $\mathrm{Acc}_{\mathrm{gen}}$: top-1 accuracy of $C$ on generated images.
• $\mathrm{Acc}_{\mathrm{oracle}}$: top-1 accuracy of $C$ on held-out real images.

The RCA is defined as:

$$\mathrm{RCA} = \frac{\mathrm{Acc}_{\mathrm{gen}}}{\mathrm{Acc}_{\mathrm{oracle}}}$$

with $\mathrm{RCA} \in [0,1]$ by construction. 1 indicates perfect generative semantic consistency matching real-data label recoverability.

b) Subpopulation Fairness and Group-Level Calibration:

RCA formalizes the alignment of classification rates across subpopulations or groups, often in fairness settings (Amit et al., 22 May 2025). For group $g$ with reference classification rate $R^*_g$ (e.g., under the Bayes-optimal classifier) and observed rate $R_g(h)$ for classifier $h$:

$\mathrm{RCA}_g(h) = \frac{R_g(h) - R^*_g}{R^*_g}$

A classifier is said to be classification-accurate on group $g$ if $|\mathrm{RCA}_g(h)| \leq \epsilon$.

c) Reverse Accuracy for Segmentation Quality Prediction:

"Reverse Classification Accuracy" is introduced in domain adaptation for medical image segmentation (Valindria et al., 2018). For test image $x$:

• Predict segmentation $\hat S(x)$ using a model trained on source domain $S$.
• Train a "reverse classifier" $C_x$ using $(x, \hat S(x))$ as the only labeled example.
• Apply $C_x$ to $M$ reference images $(x_{r_i}, y_{r_i})$, $i=1\dots M$.
• Compute Dice Similarity Coefficient for each:

$$DSC(\hat y_i, y_{r_i}) = \frac{2|\hat y_i \cap y_{r_i}|}{|\hat y_i| + |y_{r_i}|}$$

• Define:

$$\mathrm{RCA}(x) = \max_{1 \leq i \leq M} DSC(\hat y_i, y_{r_i})$$

d) Predictive Extrapolation in Multiclass Classification:

RCA, in this context, relates to predicting classification accuracy as the number of unseen classes grows (Slavutsky et al., 2020). Let $C_x = P_{Y'}(S_{y^*}(x) \geq S_{Y'}(x))$ denote the probability that a data point's correct class score beats a random incorrect class. For $k$ classes:

$E(k) = E_X \left[ C_x^{k-1} \right]$

RCA is functionally tied to the power moment of the reversed ROC (rROC) curve.

2. Calibration Procedures and Algorithmic Workflow

RCA for Identity Consistency (Lin et al., 22 Jan 2026):

1. Train an oracle classifier $C$ (e.g., ResNet-34) on real images.
2. Compute $\mathrm{Acc}_{\mathrm{oracle}}$ on a held-out set.
3. Generate a large, balanced sample of images per class with the generative model.
4. Assign intended class labels to each generated image.
5. Compute $\mathrm{Acc}_{\mathrm{gen}}$ as the oracle's accuracy on these synthetic samples.
6. Calculate RCA as the ratio, $$\mathrm{Acc}_{\mathrm{gen}}/\mathrm{Acc}_{\mathrm{oracle}}$$.

RCA for Subpopulation Fairness (Amit et al., 22 May 2025):

• For each group $g$, estimate $R^*_g$ from an optimal or reference classifier and $R_g(h)$ from the model under study.
• Compute groupwise RCA.
• For overall guarantees, require small maximum $|\mathrm{RCA}_g(h)|$ across all $g$.

Reverse RCA for Domain Adaptation (Valindria et al., 2018):

• For each test image $x$, train $C_x$ with $(x, \hat S(x))$.
• Segment all $M$ reference images using $C_x$.
• Compute and store $DSC$ with ground-truth per reference.
• Assign RCA to $x$ as the maximal $DSC$ across reference cases.

Multiclass Accuracy Prediction via rROC (Slavutsky et al., 2020):

• For each test point $x$, collect the scores $S_{y^*}(x)$ (true class) and $S_{y'_i}(x)$ (wrong classes).
• Estimate empirical CDF $C_x$.
• Fit a neural network to map observed test statistics to $C_x$, calibrate so that the implied empirical accuracies match observed ones across $2\leq k \leq k_1$.
• Extrapolate $E(k)$ for larger $k$ using the power moment formula.

3. Comparison with Standard Metrics

Metric	What It Measures	Key Limitations
FID / IS	Distributional similarity, feature diversity	Blind to semantic alignment; not class-aware
Raw Accuracy	Model success rate (e.g., on generated data)	Conflates task difficulty with model capacity
RCA	Task-normalized, semantic class preservation	Relies on oracle or reference baseline; not sensitive to visual fidelity
rROC/Power-Moment	Score separation, extrapolative accuracy	Requires marginality; assumes no retraining

RCA (in all its forms) specifically addresses the inability of distributional and even classification-based metrics to separate intrinsic class ambiguity from representational or generative performance. In generative settings, for example, $\mathrm{FID}=8.93$ and $\mathrm{IS}=1.06$ may suggest high visual quality and poor entropy respectively, yet $\mathrm{RCA}=0.27$ directly reveals inadequate semantic preservation (Lin et al., 22 Jan 2026).

4. Empirical Insights, Failure Modes, and Trade-offs

Empirical Findings

• In fine-grained face generation (KoIn10 dataset), an RCA of $0.27$ was reported despite excellent FID, indicating only 27% of identity information was maintained in generated samples (Lin et al., 22 Jan 2026).
• Confusion matrices reveal strong recall for some classes (e.g., RCA$_1 \approx 0.55$), but near-random performance or semantic collapse for visually ambiguous classes (RCA$_2 \approx 0.02$).

Diagnosed Failure Modes

• Resolution Bottleneck: Low resolutions impede the encoding of subtle, identity-specific features.
• Intra-gender Ambiguity: Models collapse to gender-level representations, masking fine-grained label distinctions.
• Mode Dominance: Partial mode collapse, where the generator favors prevalent or visually distinct identities.

Theoretical Trade-offs

• In fairness-focused classification, a core impossibility result (conditional on cryptographic assumptions) states that no polynomial-time algorithm can simultaneously guarantee both near-optimal Bayes loss and arbitrarily small RCA deviation across groups in worst case (Amit et al., 22 May 2025). This necessitates domain-specific prioritization between utility and fairness-driven accuracy.

5. Applications and Interpretation Across Domains

Fine-Grained Conditional Generation

RCA is fundamental for validating semantic controllability in generative models, enabling direct comparison of conditional label preservation across architectures and resolutions. It underpins empirical investigations of semantic mode collapse, which are invisible under metrics like FID/IS (Lin et al., 22 Jan 2026).

Fair Classification

RCA serves as a calibrator for groupwise fairness, facilitating audits of rate-preserving classification and aligning model selection processes with subpopulation equity constraints (Amit et al., 22 May 2025).

Domain Adaptation in Segmentation

Reverse RCA enables cost-effective, active learning by predicting per-sample segmentation quality and guiding targeted annotation for domain adaptation, achieving comparable performance to full supervision with a fraction of manual effort (Valindria et al., 2018).

Multiclass Extrapolation

RCA, via power-moment or rROC methods, allows extrapolation of classifier accuracy as the number of target classes increases, supporting robust prediction in real-world deployment where new unseen classes are encountered (Slavutsky et al., 2020).

6. Practical Guidelines for RCA Computation

• Use a high-quality, domain-specific oracle or reference classifier, and ensure tight correspondence in input preprocessing and resolution between the oracle and the evaluated data (Lin et al., 22 Jan 2026).
• For reverse RCA, assemble a representative reference set capturing domain variability (Valindria et al., 2018).
• In multiclass extrapolation, ensure the marginality assumption holds (scores for each class are independent of which other classes are present) for validity (Slavutsky et al., 2020).
• Pair RCA with diversity and fidelity metrics (e.g., FID, LPIPS) for a complete assessment of generative or predictive pipelines.

7. Limitations and Future Directions

RCA's strengths—domain invariance, semantic calibration—come with limitations. Its output is only as reliable as the oracle or reference benchmark it employs. For low-accuracy or poor-calibrated oracles, RCA loses interpretability (Lin et al., 22 Jan 2026). It fails to measure visual quality or intra-class diversity directly and is sensitive to resolution and label granularity constraints. Extensions proposed include integration with metric-learning losses (ArcFace), deployment in combination with super-resolution methods, and application across other fine-grained domains such as species identification or product categorization. In multiclass extrapolation, extending RCA to non-marginal classifiers and adaptive sampling scenarios remains open (Slavutsky et al., 2020).

---

References:

• "Relative Classification Accuracy: A Calibrated Metric for Identity Consistency in Fine-Grained K-pop Face Generation" (Lin et al., 22 Jan 2026)
• "Accuracy vs. Accuracy: Computational Tradeoffs Between Classification Rates and Utility" (Amit et al., 22 May 2025)
• "Domain Adaptation for MRI Organ Segmentation using Reverse Classification Accuracy" (Valindria et al., 2018)
• "Predicting Classification Accuracy When Adding New Unobserved Classes" (Slavutsky et al., 2020)