E-ICL+FCP: Efficient Full Conformal Prediction

• E-ICL+FCP is an efficient framework for full conformal prediction that combines enhanced in-context learning with a permutation-invariant Transformer to simulate retraining without extra overhead.
• It employs a CP-aware meta-training loss with smooth approximations for quantile and indicator functions, optimizing predictive set tightness while maintaining nominal coverage.
• Empirical evaluations on synthetic and real-world tasks demonstrate that E-ICL+FCP achieves smaller prediction sets and optimal efficiency–coverage trade-offs compared to conventional methods.

E-ICL+FCP is an efficient framework for full @@@@1@@@@ (FCP) based on enhanced in-context learning (ICL) with a permutation-invariant Transformer and a conformal prediction-aware training objective. The principal advance is the simulation of the retrained-models requirement intrinsic to classical FCP using a single ICL model. This design preserves coverage guarantees associated with FCP, eliminates retraining overhead, and provides smaller typical prediction sets compared to conventional split CP (SCP) and prior ICL-FCP approaches. E-ICL+FCP achieves optimal efficiency–coverage trade-offs and is validated on both synthetic and real-world classification tasks (Deng et al., 1 Sep 2025).

1. Permutation-Invariant Transformer Architecture

E-ICL+FCP employs a Transformer encoder explicitly constructed to be permutation-invariant with respect to the augmented calibration set. For each candidate label $y \in \mathcal Y$, an augmented dataset $\mathcal D^y = \mathcal D \cup \{(x_{n+1}, y)\}$ is formed, where $\mathcal D$ contains $n$ calibration points and $(x_{n+1}, y)$ is the test point paired with label $y$. Each data point $(x_i, y_i)$ is encoded as context token $\mathbf c_i = h_1(x_i, y_i)$ and query token $\mathbf q_i = h_2(x_i, y_i)$. The token sequence $$[\mathbf c_1, ..., \mathbf c_{n+1}, \mathbf q_1, ..., \mathbf q_{n+1}]$$ is processed through $E$ identical encoder layers:

$$H^{(e)} = H^{(e-1)} + \mathrm{MHA}(\mathrm{LN}(H^{(e-1)}), M) + \mathrm{FFN}(\mathrm{LN}(\cdot))$$

where $M$ is the attention mask comprising zero blocks for permitted attention and $-\infty$ for non-permitted (masked) attention. Specifically, context tokens attend freely among themselves, and each query attends to all context tokens and itself, but never to other queries. This architectural property ensures invariance to all permutations of $\mathcal D^y$. The resulting output $G_\theta(\mathcal D^y, x)$ thus simulates the retraining of base models for FCP and guarantees the correctness of the coverage properties.

2. CP-Aware Meta-Training Loss

Standard ICL objectives (cross-entropy over tasks) are insufficient to optimize FCP set efficiency. E-ICL+FCP introduces a meta-training loss designed to produce small predictive sets while maintaining nominal coverage. For a meta-training task $t$ with support $\{(x_i^t, y_i^t)\}_{i=1}^{n}$, query $(x_{n+1}^t, y_{n+1}^t)$, and model $G_\theta$:

• The non-conformity score: $$s_i^{t,y} = -\log G_\theta(\mathcal D^{t,y}, x_i^t)[y]$$.
• The predictive set:

$$C_\theta^t = \{y \in \mathcal Y : s_{n+1}^{t,y} \le Q_{1-\alpha}(\{s_i^{t,y}\}_{i=1}^{n+1})\}$$

where $Q_{1-\alpha}$ is the $(1-\alpha)$ quantile.

To enable differentiability, E-ICL+FCP replaces hard quantile and indicator functions with smooth approximations. The soft quantile $\hat Q_{1-\alpha}$ employs a pinball loss and a temperature parameter $c_q$, while the soft indicator is a sigmoid $$\sigma(r, \tau) = [1 + \exp(-(r - \tau)/\kappa)]^{-1}$$ with smoothing $\kappa$. The inefficiency surrogate $\mathrm{L_{ineff}}^t(\theta)$ approximates $|C_\theta^t|$ via summed soft indicators and the classification surrogate $\mathrm{L_{class}}^t(\theta)$ enforces coverage of the true label. The aggregate meta-training loss over tasks is:

$$L(\theta) = \sum_{t=1}^T \mathrm{L_{ineff}}^t(\theta) + \lambda\,\mathrm{L_{class}}^t(\theta)$$

where $\lambda$ controls the coverage–efficiency trade-off. All components are amenable to gradient-based optimization.

3. E-ICL+FCP Inference Algorithm and Computational Complexity

Inference uses a fixed ICL model $G_\theta$, requiring no further retraining. For a test context $\mathcal D$ and query $x_{n+1}$:

1. For each $y \in \mathcal Y$, construct $\mathcal D^y$.
2. Run $G_\theta$ on tokens encoding $\mathcal D^y$.
3. For every $i = 1, ..., n+1$, compute non-conformity scores $s_i^y$.
4. Calculate empirical quantile $Q^y$ for each $y$.
5. Include $y$ in the conformal set $C$ if $s_{n+1}^y \le Q^y$.

This process avoids retraining the base model per candidate label and test point. E-ICL+FCP requires $|\mathcal Y|$ model forward passes per query, each with $2n+2$ tokens, and $O(|\mathcal Y| n)$ cost per test instance. Relative to classical FCP (which requires $|\mathcal Y| (n+1)$ retrainings) and SCP ($O(n)$ per query), E-ICL+FCP preserves optimal scaling and allows parallelization across the candidate label space.

Method	# Model Training	# Forward Passes	Big-O per Test
Split CP (JL/MAML)	$0$ or $1$	$n+1$ or $m+1$	$O(n)$
Full CP (retrain)	$|\mathcal Y|$	$|\mathcal Y|(n+1)$	$O(|\mathcal Y| n)$
E-ICL+FCP	$0$	$|\mathcal Y|$	$O(|\mathcal Y| n)$

This algorithmic design provides a marked reduction in retraining requirements, yielding substantial practical computational savings.

4. Distribution-Free Coverage Guarantee

The framework preserves the classical distribution-free coverage guarantee under data exchangeability. Theorem 1 states: If $\{(x_i, y_i)\}_{i=1}^{n+1}$ are exchangeable and $G_\theta$ is permutation-invariant, the E-ICL+FCP predictive set

$$C(x_{n+1}) = \{y : s_{n+1}^y \le Q_{1-\alpha}(\{s_i^y\}_{i=1}^{n+1})\}$$

satisfies:

$\Pr\{y_{n+1} \in C(x_{n+1})\} \ge 1 - \alpha$

This result follows from the uniformity of the rank of $s_{n+1}^y$ among $\{s_i^y\}_{i=1}^{n+1}$ guaranteed by permutation invariance, replicating the coverage argument of classical FCP formulations (Vovk et al., Barber et al.). This suggests that the absence of actual retraining does not affect coverage, provided the architecture and loss adhere strictly to permutation invariance and the CP-aware prescription.

5. Empirical Evaluation and Efficiency–Coverage Trade-off

E-ICL+FCP demonstrates empirical superiority in both synthetic and real-world scenarios. On the QPSK symbol demodulation task ($y \in \{\pm 1 \pm j\}$, $n=19$, $\alpha = 0.1$), E-ICL+FCP attains coverage $0.903$ and average set size $1.216$, a relative reduction of $6.99\%$ in set size over standard ICL-FCP ($0.901$ coverage, $1.309$ set size). On CIFAR-FS binary few-shot classification ($n=19$ support, $\alpha = 0.1$), it achieves coverage $0.900$ and average set size $1.091$ ($–5.73\%$ improvement in set size over ICL-FCP). Other split CP and meta-learning baselines exhibit inferior efficiency–coverage profiles.

Method	Coverage	Avg. Set Size	Gain Over ICL-FCP
ICL+FCP	0.901	1.309	Baseline
E-ICL+FCP	0.903	1.216	–6.99% (QPSK)
ICL+FCP	0.899	1.157	Baseline
E-ICL+FCP	0.900	1.091	–5.73% (CIFAR-FS)

A plausible implication is that the CP-aware objective enables predictive sets nearly as tight as possible under the coverage constraint and the in-context learning paradigm. Table 1 (Deng et al., 1 Sep 2025) confirms that E-ICL+FCP requires zero retraining and only $|\mathcal Y|$ forward passes, realizing a superior practical trade-off.

6. Related Methodologies and Theoretical Context

Traditional FCP requires retraining the model for each candidate label to produce prediction sets with marginal coverage $1-\alpha$, incurring computational cost $O(|\mathcal Y| n)$. SCP attempts to mitigate complexity by splitting the data, but suffers coverage–efficiency trade-offs due to reduced calibration information. Prior meta-learning approaches, including JL+SCP and MAML+SCP, as well as standard ICL+FCP, do not tailor training to the conformal objective and yield wider predictive sets. E-ICL+FCP improves upon these by directly optimizing for set tightness and coverage with conformal-aware smoothing and permutation invariance, without compromising the distribution-free guarantee. This approach leverages deep Transformer architectures for exchangeability without explicit retraining.

7. Practical Significance and Implications

The elimination of retraining cycles in E-ICL+FCP constitutes a major step in making full conformal inference tractable within large-scale and few-shot settings. The combination of Transformer-based permutation invariance and CP-specific loss structure enables distribution-free, computationally scalable uncertainty quantification. A plausible implication is that E-ICL+FCP could be foundational in applications requiring both data-efficient and reliably calibrated predictive sets, including trustworthy AI, medical diagnosis, and robust automated decision systems. The approach is validated extensively and offers immediate practical improvements in both computational load and predictive set precision (Deng et al., 1 Sep 2025).