Freq-Domain KV Cache Compression

• Frequency-domain KV cache compression is a family of techniques that applies DCT/DFT to key and value matrices, exploiting spectral energy concentration for efficient memory and computational savings.
• The methodology integrates spectral transforms, low-pass filtering, and outlier identification to achieve up to 80% memory reduction and significant decoding speedups with minimal accuracy degradation.
• Practical implementations like FlashCache, FAEDKV, and EliteKV demonstrate that these techniques can be seamlessly integrated into transformer architectures for large-context and multimodal applications.

Frequency-domain KV cache compression encompasses a family of techniques that exploit the spectral (frequency) structure of the key and value matrices stored in transformer-based models to reduce memory and computational requirements at inference time. Unlike score-based or geometric compression, frequency-domain approaches analyze and modify the distribution of KV activations using transforms such as the Discrete Cosine Transform (DCT) or Discrete Fourier Transform (DFT), revealing intrinsic energy concentration patterns and enabling informed pruning or projection. This class of methods achieves significant cache reduction and acceleration, with minimal loss in downstream accuracy, and underpins recent algorithms such as FlashCache, FAEDKV, and EliteKV (Yang et al., 20 Nov 2025, Li et al., 26 Jul 2025, Zhou et al., 3 Mar 2025). These approaches render KV cache management more efficient and in some cases agnostic to token position, making them particularly attractive for large-context and multimodal transformers.

1. Spectral Properties of KV Matrices in Transformers

Empirical studies on multimodal and language transformers reveal that the frequency-domain energy of the per-layer key and value matrices is predominantly concentrated in low-frequency bands. Specifically, it is observed that over 90% of the total energy in both keys and values lies in the low-frequency region, as measured after applying a 1D DCT or DFT along the cache sequence axis (Yang et al., 20 Nov 2025). This regularity suggests that much of the information encoded by the KV cache is smooth in token position and can be well-represented by a small subset of spectral components.

In the context of RoPE-based attention, each attention head implicitly encodes a distribution over base rotation frequencies, further motivating frequency selection as a means of compression (Zhou et al., 3 Mar 2025).

2. Frequency-Domain Transformation and Principal Component Extraction

To exploit these spectral properties, the KV caches $K^\ell, V^\ell \in \mathbb{R}^{N\times D}$ (where $N$ is context length, $D$ the hidden size, and $\ell$ the layer) are transformed via DCT (Yang et al., 20 Nov 2025) or DFT (Li et al., 26 Jul 2025) along the token axis. Formally, for DCT:

$$C_k^\ell[m] = \alpha[m] \sum_{i=0}^{N-1} k_i^\ell \cos[\pi m(i+1/2)/N]$$

with energy spectrum $P_k^\ell[m] = |C_k^\ell[m]|^2$ and $\alpha[m] = \sqrt{1/N}$ for $m=0$, $\sqrt{2/N}$ otherwise (Yang et al., 20 Nov 2025). For DFT/IWDFT, analogous formulas apply.

Low-pass filtering is then performed by zeroing all coefficients above a cutoff frequency $\omega = \gamma N$ (with $0<\gamma<1$). The inverse transform (IDCT or IDFT) of this truncated spectrum yields the "base" KV sequence, representing the principal, smooth component.

In EliteKV, the intrinsic frequency preference of each RoPE attention head is identified, and only the top-$r$ "elite" frequency components are retained for rotation, restoring linearity to other dimensions (Zhou et al., 3 Mar 2025).

3. Outlier KV Definition and Recognition

High-frequency KV pairs—tokens whose keys and/or values substantially deviate from the base—are disproportionately critical for model inference. The deviation is quantified as:

$$\operatorname{Dev}[i] = \|k_i^\ell - k_{\rm base}^\ell[i]\|_2^2 + \|v_i^\ell - v_{\rm base}^\ell[i]\|_2^2$$

Pairs with the largest deviations are termed "Outlier KVs" (Editor's term), found via sorting. Ablation studies indicate that selectively removing Outlier KVs causes pronounced drops in task accuracy, confirming their significance (Yang et al., 20 Nov 2025).

Automated recognition modules (such as the Outlier KV Recognition Module in FlashCache) prioritize the retention of these KVs under a per-layer budget constraint.

4. Compression Algorithms and Cache Retention Strategies

Frequency-domain KV cache compression encompasses a pipeline comprising:

• Spectral transform: Apply DCT/DFT to project K/V caches to frequency space.
• Spectral selection/pruning: Use ablation studies, energy criteria, or head-specific metrics to select bins or frequency bands to retain. In FAEDKV, per-layer DFT bins are split into contiguous chunks and the most information-rich chunks are identified by observing perplexity drops when ablated (Li et al., 26 Jul 2025).
• Inverse transform: Invert the selected spectrum to reconstruct a compressed cache, or represent the cache in the retained frequency bins.
• Outlier identification: Compute and sort deviations to select critical KVs (FlashCache).
• Dynamic budget allocation: Compute per-layer outlier energy ratios and normalize across layers to allocate retention budgets, maximizing the share of outlier energy preserved (Yang et al., 20 Nov 2025).

EliteKV introduces RoPElite, which restores linearity by keeping only frequency dimensions empirically found to be important for each head, then applies joint low-rank projection (J-LRD) that shares the reduced subspace between keys and values (Zhou et al., 3 Mar 2025).

FAEDKV employs a novel Infinite-Window DFT (IWDFT) to maintain a compressed, frequency-domain cache that incorporates all tokens with equal weight, ensuring unbiased retention and allowing efficient, per-token updates without multiple passes or sliding windows (Li et al., 26 Jul 2025).

5. Computational and Memory Savings

The principal benefit of frequency-domain KV cache compression is a substantial reduction in cache memory and associated compute at each decoding step:

• Memory reduction: For a retention ratio $\rho\in(0,1]$, compressed cache size is $M_{\rm cmp} \approx \rho M_{\rm full}$, with $\sim 70–85\%$ savings typical (Yang et al., 20 Nov 2025, Li et al., 26 Jul 2025, Zhou et al., 3 Mar 2025).
• Runtime speedup: Since cache length is reduced, the dominant attention cost per decoding step drops from $O(N \cdot D)$ to $O(\rho N \cdot D)$, resulting in speedups up to $1/\rho$.
• Overhead: The additional computation for DCT/DFT/IDFT and sorting is $O(D N \log N)$ at most, and negligible relative to attention for large $N \gg D$ (Yang et al., 20 Nov 2025).
• EliteKV: By combining RoPElite and J-LRD, cache size can be reduced by 75% with negligible performance loss, and associated speedups of $1.2\times$–$1.5\times$ are observed (Zhou et al., 3 Mar 2025).

6. Experimental Results and Evaluation

Empirical evaluations demonstrate the efficacy of frequency-domain methods:

Method	Decoding Speedup	KV Memory Reduction	Accuracy Δ vs Full
Full Cache	1.00×	0%	0.0%
LOOK-M	1.30×	60%	–4.2%
MEDA	1.40×	70%	–3.1%
FlashCache	1.69×	80%	–0.3%

(Results on Qwen2.5-VL-7B, ρ=0.2, tasks on MileBench (Yang et al., 20 Nov 2025))

Across six multimodal and language benchmarks, FlashCache matches or exceeds other compression methods, maintaining accuracy drops under 1%. FAEDKV outperforms state-of-the-art eviction-based baselines by up to 22% in tight memory settings, with especially uniform retrieval accuracy throughout the sequence on "Needle-In-A-Haystack" tasks (Li et al., 26 Jul 2025).

EliteKV achieves a 75% reduction in KV cache with less than 1% average performance loss after minimal up-training (0.6% of pre-training tokens), and remains robust across different model scales (Zhou et al., 3 Mar 2025).

7. Practical Integration and Compatibility

Frequency-domain cache compression frameworks impose minimal constraints on model architecture. Notably, methods such as FlashCache are "attention-score-free" and fully compatible with efficient attention kernels, including FlashAttention (Yang et al., 20 Nov 2025).

FAEDKV maintains a position-agnostic, training-free, and unbiased representation, recommending a practical pipeline: (i) one-off frequency ablation, (ii) per-layer cache pruning, and (iii) efficient GPU/CPU mixed-precision implementation (Li et al., 26 Jul 2025). EliteKV further demonstrates adaptability to RoPE-based models via local structure modifications and minor up-training.

A practical implication is that frequency-domain KV cache compression can be realistically deployed to production-scale transformers, providing significant memory and time savings without material accuracy loss, and regardless of input modality or context length.

---

Principal references:

• "Revisiting Multimodal KV Cache Compression: A Frequency-Domain-Guided Outlier-KV-Aware Approach" (Yang et al., 20 Nov 2025)
• "FAEDKV: Infinite-Window Fourier Transform for Unbiased KV Cache Compression" (Li et al., 26 Jul 2025)
• "EliteKV: Scalable KV Cache Compression via RoPE Frequency Selection and Joint Low-Rank Projection" (Zhou et al., 3 Mar 2025)