Feature-space Smoothing (FS)

• Feature-space Smoothing (FS) is a method that perturbs learned feature representations to enforce local regularity, stability, and robustness in models.
• It utilizes stochastic, geometric, and multiplicative techniques to enhance certified performance in tasks like segmentation, adversarial defense, and interpretability.
• Experimental results demonstrate significant gains in robustness and efficiency, making FS valuable for complex vision and multimodal language applications.

Feature-space Smoothing (FS) encompasses a class of methodologies and theoretical frameworks that enforce or exploit local regularity, perturbation resilience, or geometric stability by operating directly on learned representations within a model, rather than on original input data. FS methods have been proposed and empirically validated for certified robustness, segmentation stability, adversarial defense, interpretability, and performance in diverse vision and multimodal language systems. Typical implementations instantiate stochastic, geometric, or multiplicative smoothing at various feature layers, and are often accompanied by formal robustness or stability guarantees that tie feature-space perturbation budgets to input-space threat models.

1. Core Definitions and Variants

Feature-space Smoothing in its canonical form injects perturbations, noise, or structured modifications into an intermediate feature representation produced by a backbone or encoder network, and then aggregates downstream responses, predictions, or attributions. Formally, for a network $f_e:\mathbb{R}^d\rightarrow\mathbb{R}^D$ mapping input $x$ to a feature vector, the smoothed encoder is

$$\hat f_e(x) = \mathbb{E}_{\varepsilon\sim \mathcal{N}(0,I_d)}[f_e(x+\varepsilon)]$$

or its Monte Carlo approximation. Aggregation in the downstream pipeline may occur by mean, median, voting, or more sophisticated schemes depending on the task. FS can also comprise deterministic geometric transformations, e.g., neighborhood-based Chebyshev smoothing ("GeloVec"), or multiplicative masking with Bernoulli noise ("MuS").

Empirical instantiations include:

• Gaussian-feature stochastic smoothing for certified regression/classification (Shumitskaya et al., 7 Aug 2025, Addepalli et al., 2023, Xia et al., 22 Jan 2026)
• Geometric/metric-based manifold smoothing for semantic segmentation (Kriuk et al., 2 May 2025)
• Multiplicative feature masking for stable explanations (Xue et al., 2023)

2. Methodological Frameworks

Stochastic Feature-space Smoothing

In adversarial robustness and IQA, Gaussian noise is injected into a low-dimensional feature vector $f_{\mathrm{norm}}$ (post-backbone, post-normalization). Multiple samples are scored, and their aggregation yields a "smoothed" output with robustness properties.

Key methodological steps (Shumitskaya et al., 7 Aug 2025, Xia et al., 22 Jan 2026):

1. Extract deterministic features: $f_{\mathrm{norm}} = FTN(b(x))$.
2. Sample $e_i \sim \mathcal{N}(0,\sigma_f^2 I_d)$, form $f_{\mathrm{noised}}^i = f_{\mathrm{norm}} + e_i$.
3. Compute predictions or quality scores $S(f_{\mathrm{noised}}^i)$; aggregate (e.g., median).
4. Derive certified input-space radii via Jacobian spectral norm or feature-cosine bounds.

Geometric/Manifold-based Feature Smoothing

In segmentation, GeloVec formalizes feature-space smoothing as enforcing coherence along a learned Riemannian manifold: local neighborhoods are regularized via adaptive Chebyshev distances, and multispatial orthogonal projections increase discriminativity while preserving locality. Smoothing weights adaptively gate smoothing based on proximity to boundaries (Kriuk et al., 2 May 2025).

Multiplicative Smoothing

MuS achieves smoothing via random feature masking: $f(x) = \mathbb{E}_{s\sim D}[h(x \odot s)]$, where $s$ is a binary mask, $x$0 the base classifier, and $x$1 defines Bernoulli marginals (Xue et al., 2023). The operator preserves the equivalence $x$2 and supports certified stability via Lipschitz continuity under masking.

3. Theoretical Guarantees

Certified Input-space Robustness

In randomized feature-space smoothing for IQA, the relationship between feature-noise standard deviation $x$3 and the maximal safe input-space perturbation $x$4 is established via Jacobian spectral norm: $x$5 where $x$6 is the Jacobian of the composite feature map. Theorems from regression certification (e.g., Chiang et al.) provide bounds on prediction range across all admissible perturbations (Shumitskaya et al., 7 Aug 2025).

Certified Feature-cosine Similarity

For multimodal LLMs, under $x$7-bounded adversarial perturbation $x$8, FS yields: $x$9 where $$\hat f_e(x) = \mathbb{E}_{\varepsilon\sim \mathcal{N}(0,I_d)}[f_e(x+\varepsilon)]$$0 is the Gaussian CDF, and $$\hat f_e(x) = \mathbb{E}_{\varepsilon\sim \mathcal{N}(0,I_d)}[f_e(x+\varepsilon)]$$1 is the “Gaussian robustness score” (Xia et al., 22 Jan 2026). This gives explicit FCSB certificates linking the underlying feature distribution's noise resilience to certified robustness radii.

Stability and Attribution Guarantees

Multiplicative smoothing certifies incremental and decremental stability radii for feature attributions based on model confidence margins and smoothing rate $$\hat f_e(x) = \mathbb{E}_{\varepsilon\sim \mathcal{N}(0,I_d)}[f_e(x+\varepsilon)]$$2 (Xue et al., 2023). The Lipschitz property of the smoothed classifier under masking is leveraged to state precise bounds on the allowable mask Hamming perturbations before prediction flips.

Geometric Stability on Manifolds

In GeloVec, smoothing weights gate local attention based on adaptive Chebyshev distances, which are interpreted as proxies for local geodesic distances. While not furnishing explicit Riemannian stability theorems, the framework implicitly relies on bi-Lipschitz regularity and local linearity (Kriuk et al., 2 May 2025).

4. Algorithmic Procedures and Computational Complexity

FS-IQA (Certified Feature-space Smoothing for Robust IQA)

FS-IQA requires only a single backbone and FTN pass per image, followed by multiple lightweight scorer passes for noise-perturbed features (Shumitskaya et al., 7 Aug 2025). This reduces inference time by 99.5% (no certification) and 20.6% (with certification) compared to input-space smoothing. Key components:

• FTN dimensionality reduction ($$\hat f_e(x) = \mathbb{E}_{\varepsilon\sim \mathcal{N}(0,I_d)}[f_e(x+\varepsilon)]$$3)
• Gaussian feature noise and scorer aggregation
• Jacobian-based certification or abstention

GeloVec (Geometric Feature Space Smoothing)

GeloVec’s block comprises:

1. Multispatial orthogonal basis transformation (1×1 convolutions, normalization).
2. Geometric adaptive sampling using maximum adaptive Chebyshev distance for local neighborhood.
3. Edge gating to selectively apply or suppress smoothing.
4. Geometry-modulated attention; attention weights penalized by local distances.
5. Residual connections and multi-scale variants.

Parallelizable by design; ablation analysis credits each component with incremental mIoU gains (Kriuk et al., 2 May 2025).

FLSS (Feature-Level Stochastic Smoothing) & MuS

FLSS implements VAE-like stochastic smoothing in the penultimate feature layer, with downstream MLP voting or averaging over 100 Monte Carlo draws; only a single MLP pass per sample is needed. MuS evaluation is quantized, typically requiring 32–64 forward passes, batched for GPU efficiency (Xue et al., 2023).

5. Empirical Results and Practical Outcomes

Performance Gains

Method	Certified Gain	Key Benchmarks	Noted Speedup/Overhead
FS-IQA (Shumitskaya et al., 7 Aug 2025)	30.9% higher SRCC/PLCC	KonIQ-10k, Kadid-10k, 6 IQA models	99.5% faster (~33ms img, no cert); 7.5s cert
GeloVec (Kriuk et al., 2 May 2025)	+2.1–2.7 mIoU (segm.)	Caltech Birds-200, LSDSC, FSSD	ResNet-34 backbones, highly parallel
FLSS (Addepalli et al., 2023)	+3.15–3.89% robust acc.	CIFAR-10/100, adversarial accuracy	2% wall-clock test increase, 100× MLP
FS-PSM (Xia et al., 22 Jan 2026)	ASR $$\hat f_e(x) = \mathbb{E}_{\varepsilon\sim \mathcal{N}(0,I_d)}[f_e(x+\varepsilon)]$$4 90%$$\hat f_e(x) = \mathbb{E}_{\varepsilon\sim \mathcal{N}(0,I_d)}[f_e(x+\varepsilon)]$$51%	LLaVA-1.5-7B, OpenFlamingo-9B, CLIP	4–8 MC samples at inference
MuS (Xue et al., 2023)	Stability radii (20–40%)	ViT, ResNet-50, RoBERTa, ImageNet, LIME/SHAP	<5% acc drop at $$\hat f_e(x) = \mathbb{E}_{\varepsilon\sim \mathcal{N}(0,I_d)}[f_e(x+\varepsilon)]$$6

Downstream Robustness

• IQA: FS-IQA improves SRCC/PLCC with both full-reference and no-reference models, matching subjective quality more closely under attack.
• Segmentation: GeloVec outperforms SegFormer, preserves boundaries, and improves precision, recall, and F1 on challenging benchmarks.
• Multimodal LLMs: FS-PSM reduces white-box ASR from $$\hat f_e(x) = \mathbb{E}_{\varepsilon\sim \mathcal{N}(0,I_d)}[f_e(x+\varepsilon)]$$790% to $$\hat f_e(x) = \mathbb{E}_{\varepsilon\sim \mathcal{N}(0,I_d)}[f_e(x+\varepsilon)]$$81%, with substantial feature-cosine preservation; complements existing adversarial training (Xia et al., 22 Jan 2026).

6. Limitations, Practicalities, and Extensions

FS methods evaluated so far are primarily focused on $$\hat f_e(x) = \mathbb{E}_{\varepsilon\sim \mathcal{N}(0,I_d)}[f_e(x+\varepsilon)]$$9-bounded Gaussian noise models. Certified robustness does not always directly transfer from feature-cosine similarity or feature domain stability to strict output-level correctness in structured or sequence models; a gap remains between feature and output guarantees in complex multimodal tasks (Xia et al., 22 Jan 2026). Monte Carlo estimation introduces computational variance; though overhead is low for low-dimensional scorer heads, for high-throughput or real-time pipelines, practitioners may select "prediction-only" modes and conduct certification offline.

In segmentation, GeloVec’s implicit Riemannian interpretation has not been formalized with explicit curvature or geodesic regularity theorems, but provides an empirically effective geometric prior (Kriuk et al., 2 May 2025). For interpretability, MuS achieves robust attribution stability while introducing minor accuracy reductions, with best results for mask rates $f_{\mathrm{norm}}$0 (Xue et al., 2023).

Finally, plug-and-play components (e.g., FS-PSM) enable retrofitting pre-trained models for certified robustness without retraining full-core encoders, although tuning of auxiliary module losses and noise levels remains an open engineering problem (Xia et al., 22 Jan 2026).

7. Connections to Related Areas and Outlook

Feature-space Smoothing connects and contrasts with input-level randomized smoothing (Cohen et al.), variational bottlenecks, geometric deep learning, and certified interpretability by leveraging the low intrinsic dimension and semantic locality of learned features. FS methods address prohibitive compute in high-dimensional input smoothing, enable certified out-of-sample generalization, stabilize attribution schemes, and provide a template for domain-adapted, architecture-agnostic certified defenses.

Research directions include extending certification to $f_{\mathrm{norm}}$1 or semantic threat models, formalizing the geometry of feature manifolds in smoothing schemes, and integrating FS into dynamically adaptive architectures for real-time robust deployment (Shumitskaya et al., 7 Aug 2025, Kriuk et al., 2 May 2025, Xia et al., 22 Jan 2026, Addepalli et al., 2023, Xue et al., 2023).