Subspace Injection (OSI) Techniques

• Subspace Injection (OSI) is a method of injecting vectors, signals, or noise into defined subspaces to optimize coding, privacy, and model alignment tasks.
• In randomized linear algebra, OSI employs isotropy and injectivity properties to preserve subspace structures, ensuring reliable low-rank approximations and effective signal recovery.
• Applied in differential privacy and LLM alignment, sequential subspace noise injection and activation steering enable refined privacy controls and precise model behavior adjustments.

Subspace Injection (OSI) encompasses a family of concepts unified by the principle of injecting vectors, signals, noise, or information into specific subspaces of high-dimensional spaces or parameter manifolds. This operation appears fundamentally in the study of subspace codes for network coding, certified unlearning in differential privacy, activation steering in LLMs, and randomized linear algebra via dimension reduction. OSI techniques exploit structural decompositions and subspace geometry to gain efficiency, modularity, and robustness, with each field applying specialized forms of injection, tailored metrics, and algorithmic frameworks.

1. Injection Metrics in Subspace Codes

The injection metric is central to the theory of subspace codes, especially under adversarial models in random linear network coding. Formally, given subspaces $U, V \subseteq \mathbb{F}_q^n$, the injection distance is

$$d_I(U,V) = \max\{\dim U, \dim V\} - \dim(U \cap V).$$

This metric quantifies the number of “injected” dimensions needed from the larger subspace to cover the smaller. Equivalently, $d_I(U,V) = \dim(U+V) - \min\{\dim U, \dim V\}$ (0904.0813, 0811.4163, Ghatak, 2013). For constant-dimension codes, $d_I(U,V)$ reduces to half the classical subspace distance $d_S(U,V)$, but for variable dimension it penalizes misalignments due to rank mismatch only once: $$d_I(U,V) = \tfrac{1}{2}d_S(U,V) + \tfrac{1}{2}|\dim U - \dim V|.$$ The injection metric thus refines the geometry of projective space and better models adversarial insertions/deletions than $d_S$.

2. OSI in Randomized Linear Algebra: Oblivious Subspace Injection

In randomized linear algebra, the Oblivious Subspace Injection (OSI) property describes a random linear map $S \in F^{d \times k}$ with two essential features:

• Isotropy: For any $x \in F^d$, $\mathbb{E}[\|S^*x\|_2^2] = \|x\|_2^2$.
• Injectivity: For any $$d_I(U,V) = \max\{\dim U, \dim V\} - \dim(U \cap V).$$0-dimensional subspace $$d_I(U,V) = \max\{\dim U, \dim V\} - \dim(U \cap V).$$1, with high probability, $$d_I(U,V) = \max\{\dim U, \dim V\} - \dim(U \cap V).$$2 for all $$d_I(U,V) = \max\{\dim U, \dim V\} - \dim(U \cap V).$$3.

Here, $$d_I(U,V) = \max\{\dim U, \dim V\} - \dim(U \cap V).$$4 or $$d_I(U,V) = \max\{\dim U, \dim V\} - \dim(U \cap V).$$5, and the injectivity parameter $$d_I(U,V) = \max\{\dim U, \dim V\} - \dim(U \cap V).$$6 quantifies the preservation of length in low-dimensional subspaces (Camaño et al., 28 Aug 2025).

The OSI property is strictly weaker than the Oblivious Subspace Embedding (OSE) property, omitting a uniform upper bound on dilation, yet suffices for algorithmic correctness in randomized SVD, low-rank approximation, least-squares, and Nyström methods. The analysis modularizes into (i) correctness under OSI and (ii) constructive verification for structured random matrices (e.g., Gaussian, SparseStack, randomized trigonometric transforms, Khatri–Rao).

3. Sequential Subspace Noise Injection for Certified Unlearning

In differential privacy-driven certified unlearning, sequential subspace noise injection implements “block-wise” noisy fine-tuning, distributing the noise budget over orthogonal subspaces of the model parameter space rather than all-at-once isotropic injection. Given a $$d_I(U,V) = \max\{\dim U, \dim V\} - \dim(U \cap V).$$7-dimensional parameter vector $$d_I(U,V) = \max\{\dim U, \dim V\} - \dim(U \cap V).$$8, the decomposition $$d_I(U,V) = \max\{\dim U, \dim V\} - \dim(U \cap V).$$9, with $d_I(U,V) = \dim(U+V) - \min\{\dim U, \dim V\}$0 forming orthonormal bases for $d_I(U,V) = \dim(U+V) - \min\{\dim U, \dim V\}$1 disjoint subspaces, enables fine-tuning and noise injection one block $d_I(U,V) = \dim(U+V) - \min\{\dim U, \dim V\}$2 at a time (Dolgova et al., 8 Jan 2026).

This approach has several advantages:

• Reduced Per-step Distortion: Perturbations $d_I(U,V) = \dim(U+V) - \min\{\dim U, \dim V\}$3 applied in subspaces of dimension $d_I(U,V) = \dim(U+V) - \min\{\dim U, \dim V\}$4 incur smaller $d_I(U,V) = \dim(U+V) - \min\{\dim U, \dim V\}$5-norm fluctuations per step, attenuating accuracy collapse compared to standard NFT.
• Privacy Composition: Each block update is analyzed via Rénnyi Differential Privacy (RDP), with privacy loss composed additively across blocks, maintaining the global $d_I(U,V) = \dim(U+V) - \min\{\dim U, \dim V\}$6 certificate.
• Empirical Superiority: On MNIST and CIFAR-10, block-wise OSI preserves model utility near the retraining baseline while maintaining robust unlearning guarantees, in contrast to isotropic NFT which triggers catastrophic accuracy degradation.

4. Construction of Codes Under Injection Metrics

Modern code constructions for the injection metric exploit algebraic and combinatorial subspace structure. The Etzion–Silberstein framework utilizes profile vectors, Ferrers diagrams, and liftings of Ferrers-diagram rank-metric codes to generate non-constant-dimension codes with improved size and rate over comparable subspace-metric designs (0904.0813, Ghatak, 2013). Specifically:

• Every subspace is uniquely represented by its reduced row-echelon form (RREF), encoded as a profile vector $d_I(U,V) = \dim(U+V) - \min\{\dim U, \dim V\}$7.
• The profile defines a Ferrers diagram, with the support determining the positions of leading ones, and a “star” submatrix $d_I(U,V) = \dim(U+V) - \min\{\dim U, \dim V\}$8 where free entries are optimized.
• Rank-metric codes of fixed minimum rank distance are built within these structured submatrices, and codewords are lifted back to the projective space.

The selection of code parameters and Schubert cells (partitioned via Plücker coordinates) is optimized to ensure minimum injection distance $d_I(U,V) = \dim(U+V) - \min\{\dim U, \dim V\}$9 both within and between subcodes. These non-constant-dimension codes demonstrably achieve larger cardinalities and better asymptotic packing/covering rates than constant-dimension or subspace-metric-based codes (0811.4163, Ghatak, 2013).

5. Subspace Injection in Activation Steering and Model Alignment

In the context of LLM alignment, subspace injection refers to the procedure of steering network activations along direction(s) defined within a learned property-aligned subspace. Frameworks such as PIXEL (Position-wise Injection with eXact Estimated Levels) identify low-dimensional subspaces (e.g., via dual-view contrastive PCA), select steering vectors, and compute closed-form minimal interventions per position (Yu et al., 11 Oct 2025). The process consists of:

• Subspace learning from attribute-contrasted data,
• Per-position injection with closed-form amplitude to reach prescribed alignment,
• Orthogonal residual calibration for sample-level adaptation.

These separately-calibrated subspace injections ensure the desired margin alignment at minimal activation change, with representation-level guarantees and generalization bounds.

6. Asymptotic Rates, Bounds, and Geometric Insights

The injection metric induces rich geometric and asymptotic phenomena in projective space. Ball volumes around subspaces under $d_I(U,V)$0 are “essentially flat”, scaling as $d_I(U,V)$1, implying “sphere-packing” and “Gilbert-Varshamov” bounds that are largely dimension-independent for packing and covering (0811.4163). Asymptotic rates for optimal code sizes satisfy

$d_I(U,V)$2

for normalized minimum injection distance $d_I(U,V)$3, and covering rates are quadratic: $d_I(U,V)$4 with $d_I(U,V)$5 the normalized covering radius. Notably, constant-dimension codes occupying the “half-dimension” stratum ($d_I(U,V)$6) achieve packing rates within constant factors of the global optimum.

7. Implementation Practices and Extensions

In randomized algorithms, calibration of the OSI property to structured random matrices (e.g., SparseStack, SparseRTT, Khatri–Rao) yields near-optimal low-rank approximations, least-squares, and trace estimation at lower computational cost, without requiring uniform operator norm bounds on the sketching map (Camaño et al., 28 Aug 2025). In privacy-aware learning, subspace-blocking strategies for certified unlearning are parameterized by the number and sizes of orthogonal subspaces, trading computational cost per iteration for reduction in accuracy loss (Dolgova et al., 8 Jan 2026).

Potential research extensions include tighter upper bounds (e.g., Johnson-like for $d_I(U,V)$7 codes), explicit asymptotic analysis of mixed-dimension injection codes, optimally covering projective spaces under $d_I(U,V)$8, advances in asymmetric binary code construction for code design, and balancing decoding complexity against injection-metric performance (0904.0813).

---

References:

(0904.0813): Projective Space Codes for the Injection Metric (0811.4163): Packing and Covering Properties of Subspace Codes for Error Control in Random Linear Network Coding (Ghatak, 2013): Subspace Codes for Random Networks Based on Plücker Coordinates and Schubert Cells (Camaño et al., 28 Aug 2025): Faster Linear Algebra Algorithms with Structured Random Matrices (Dolgova et al., 8 Jan 2026): Sequential Subspace Noise Injection Prevents Accuracy Collapse in Certified Unlearning (Yu et al., 11 Oct 2025): PIXEL: Adaptive Steering Via Position-wise Injection with eXact Estimated Levels under Subspace Calibration