Frequency-Adaptive Learning Strategy (FaLS)

• FaLS is a methodology that decomposes input signals into distinct frequency bands to emphasize key spectral components.
• It employs spectral decomposition, energy-based frequency selection, and frequency-specific processing to achieve state-of-the-art results in forecasting, continual, and federated learning tasks.
• FaLS offers significant efficiency gains, robust noise suppression, and reduced computational costs across diverse applications from imaging to physical network optimization.

A frequency-adaptive learning strategy (FaLS) is a family of methods in machine learning, signal processing, and physical adaptive networks that explicitly decomposes, selects, or adapts computation with respect to frequency (spectral) properties of input signals. Rather than treating data as a single homogeneous entity, FaLS architectures process, select, or optimize different frequency bands (e.g., low vs. high) with distinct mechanisms, adaptively emphasizing dominant or informative spectral components in learning and inference. Variants of FaLS span time-series forecasting (e.g., energy-based frequency selection), continual learning (split-stream frequency replay), adaptive filtering (meta-learned frequency-domain updates), physical network optimization (multi-frequency physical signaling), federated learning (communication frequency adaptation), multi-scale deep nets (Fourier-based parameter adjustment), and domain-specific imaging (frequency-decomposed denoising). Across domains, FaLS techniques have demonstrated state-of-the-art performance, substantial efficiency gains, and robust generalization in the presence of noise, nonstationarity, and resource constraints.

1. Core Principles and Taxonomy of FaLS

FaLS encompasses a spectrum of methodologies unified by targeted spectral adaptation. The essential principles are:

• Spectral Decomposition: Explicit transformation (e.g., FFT, DWT, Haar wavelets) or learned decomposition of inputs into frequency bands or spectral content.
• Frequency Selection/Attenuation: Adaptive selection, denoising, or removal of irrelevant/noisy frequencies based on spectral energy or posterior error estimates.
• Frequency-specific Processing: Distinct modules, parameter regimes, or architectures for low- and high-frequency subbands or components.
• Frequency-aligned Supervision or Losses: Objective functions (e.g., spectral-domain L1 or hybrid losses) that explicitly evaluate or enforce fidelity in the frequency domain.
• Adaptive Feedback: Learning rules or mechanisms that adjust how frequencies are processed based on a posteriori spectra, usage statistics, or system response.
• Resource-aware Adaptation: In federated or continual learning, "frequency" can refer to the rate of computation or communication, not only spectral content.

Major technical variants include:

• Deterministic spectral selection in time-series forecasting (Wu et al., 1 Aug 2025, Huang et al., 2024);
• Dual-branch frequency-disentangled continual/replay learning (Liu et al., 28 Mar 2025, Seo et al., 2024);
• Meta-learning with adaptive, cross-frequency optimizer controllers (Wu et al., 2022);
• Local frequency-multiplexed physical learning in nonlinear resistor networks (Anisetti et al., 2022);
• Communication frequency adaptation in distributed/federated learning (Tariq et al., 27 Sep 2025);
• Multi-scale, frequency-adaptive architectures for denoising and restoration (Ma et al., 8 Nov 2025).

2. Frequency Selection and Spectral Denoising in Prediction Tasks

A canonical instance is the KFS time-series forecasting architecture (Wu et al., 1 Aug 2025), which implements FaLS via the FreK module:

• Workflow:
  • Input $X \in \mathbb{R}^{L \times C}$ is multiscale-decomposed via average pooling, preserving timestamps.
  • At each scale, the FreK module computes an FFT, evaluates the per-frequency energy $E_k = |X[k]|^2$, and adaptively selects the smallest top-$K$ frequency bins that collectively retain at least a threshold fraction $\delta$ of spectral energy.
  • All other frequency bins are zeroed, and the truncated spectrum is inverted to the time domain, yielding a denoised, dominant-frequency representation.
• Processing:
  • The energy-selected signal is subsequently embedded and encoded with two-layer Group-Rational Kolmogorov-Arnold Networks (KAN), which offer learnable parametric nonlinearities.
  • Feature mixing fuses temporal representations with time-aligned embeddings for final fusion and prediction.
• Loss Function:
  • Combines time-domain MSE ($L_{MSE}$) and a spectral-domain alignment loss on the top $K_f$ frequencies ($L_F$), with a hybrid weight $\alpha$.

Empirical validation across ETT, Weather, and Electricity datasets shows that KFS with FaLS consistently achieves state-of-the-art MSE and MAE (best or second-best in 24/24 settings), with 1–4% relative error reductions over strong baselines and superior GPU efficiency (Wu et al., 1 Aug 2025).

3. FaLS for Continual and Resource-Constrained Learning

FaLS is widely used to improve efficiency and accuracy in continual/rehearsal-based learning:

• Frequency-Decomposing Two-Stream Architectures:
  • FDINet (Liu et al., 28 Mar 2025) applies a one-level Haar DWT to separate images/features into low-frequency (global structure) and high-frequency (edges/details) subbands.
  • Each band is processed by a lightweight, half-width ResNet-18 variant (L-Net, H-Net), reducing overall backbone parameters by 78%. Intermediate features at each network stage are mutually integrated (additive aggregation).
  • This design enables buffer compression, reduces computational cost (94% fewer FLOPs), and leverages the biological analogy to magnocellular (global/low frequency) and parvocellular (detail/high frequency) vision.
  • On Split CIFAR-10, FDINet + CLS-ER attains 7.49pp higher Class-IL accuracy and 5x speedup (Liu et al., 28 Mar 2025).
• Frequency-Adaptive Retrieval in Buffer Sampling:
  • Similarity-Aware Retrieval (SAR) (Seo et al., 2024) tracks per-sample discounted usage-frequencies $c_i$ in a memory buffer.
  • For each training step, SAR computes an "effective use-frequency" $\hat c_i$ for each buffer element, augmenting $c_i$ with a class-similarity-weighted sum over class-aggregate frequencies.
  • The sampling probability is $p_i \propto \exp(-\hat c_i / T)$, where $T$ tunes the exploration-exploitation balance.
  • SAR achieves up to 3x compute-efficiency gains and 2–5pp higher accuracy under fixed compute/memory budgets on CIFAR-10/100 and ImageNet (Seo et al., 2024).

4. Meta-Learning and Adaptive Filtering with Higher-Order Frequency Dependencies

FaLS methods in adaptive filtering and meta-learning exploit cross-frequency structure:

• Meta-Adaptive Filters with Block/Banded Grouping:
  • The FaLS meta-learning adaptive filter framework (Wu et al., 2022) replaces hand-derived per-frequency update rules (LMS, RLS) with a small neural controller (complex-valued 2-layer GRU).
  • Rather than acting on each frequency bin independently ("diagonal"), the framework groups adjacent bins into blocks/bands using learnable downsampling (convolutional) and upsampling layers, enabling the controller to observe higher-order frequency dependencies.

  • Inner-loop updates take the form:

    $$w_k[\tau+1] = w_k[\tau] + \Delta w_k[\tau]; \quad (\Delta w_k[\tau], \psi_k[\tau+1]) = g_\phi(\xi_k[\tau], \psi_k[\tau])$$

  • Complexity is tunable via group size $B$; e.g., block-9 grouping reduces FLOPs by $\sim 9\times$ at modest cost.

  • On the AEC Challenge, banded-3 and block-9 models achieve $3–5$ dB higher SERLE and $+2$ dB SI-SDR over Kalman and baseline meta-AF, with real-time CPU operation (Wu et al., 2022).

5. Frequency-Adaptation in Physical and Hardware Networks

In physical adaptive networks, FaLS leverages frequency-division signaling for localized gradient descent:

• Frequency Multiplexing in Nonlinear Circuits:
  • The frequency propagation algorithm (Anisetti et al., 2022) drives a resistive network with simultaneous activation ($\omega_a$) and error ($\omega_e$) sine currents.
  • Each node's voltage trace encodes an activation coefficient ($s^a_i$) and error coefficient ($s^e_i$) that can be locally extracted by Fourier filtering.

  • Local parameter updates follow:

    $$\Delta g_{jk} = -\eta \, (s^a_j - s^a_k)\, (s^e_j - s^e_k)$$

  • This update scheme provably implements stochastic gradient descent, with strict loss decrease under standard small-signal conditions.

  • The scheme generalizes to elastic/flow networks (by frequency-multiplexed drives) and enables fully local, physical implementation without explicit forward/backward computational separation (Anisetti et al., 2022).

• Dynamical Systems with Frequency-Adapted Natural Frequency:

  • In vibrational resonance, a FaLS learning rule in a Duffing oscillator adaptively tunes the natural frequency $\omega(t)$ via:

    $$\dot\omega = k_\omega [A\cos(\Omega_1 t) + B\cos(\Omega_2 t)]$$

  • This adaptation introduces a large DC shift and parametric modulations into $\omega^2(t)$, enabling resonance tracking of a weak low-frequency signal by tuning the oscillator’s eigenfrequency in situ.

  • Compared to Hebbian learning rules, FaLS achieves higher amplification and broader stability during resonance, both in simulation and circuit implementations (Wan et al., 10 Jan 2026).

6. Frequency-Adaptation in Federated and Multi-Scale Deep Learning

Recent advances have expanded FaLS further:

• Communication Frequency Optimization in Federated Learning:
  • FaLS as skip-based communication protocol (Tariq et al., 27 Sep 2025):
  • Each FL client computes an iteration-wise $\ell_2$ norm of its quantized gradient innovation $\Delta Q_m^{(t)}$.
  • If $\|\Delta Q_m^{(t)}\|_2 \geq \epsilon$ or a skip counter $t_m \geq \tau$, it communicates to the server; otherwise, it accumulates skipped rounds.
  • Aggregated updates drive the model as usual.
  • Empirically, this protocol reduces required communication rounds by 30–40%, transmits up to 50% fewer updates, yet matches or outperforms fixed-frequency baselines in accuracy under both IID and non-IID conditions (Tariq et al., 27 Sep 2025).
• Posterior Frequency-Adaptive Parameterization in Multi-Scale DNNs:
  • Frequency-adaptive multi-scale DNNs (FaLS-MscaleDNN) (Huang et al., 2024):
  • The network decomposes the target’s spectrum after initial training, identifies dominant Fourier modes, and adaptively revises its multi-scale feature set to align with observed spectral energy (posterior error estimation).
  • Retraining with the revised frequency set yields up to three orders of magnitude improvement in $L^2$ accuracy on wave and Schrödinger benchmarks (Huang et al., 2024).
• Frequency-Adaptive Heterogeneous Denoising in SAR Imaging:
  • SAR-FAH (Ma et al., 8 Nov 2025) implements wavelet-based subband separation, applying a neural ODE for low-frequency (homogeneous) bands and a deformable-convolution U-Net for high-frequency (edge-rich) bands.
  • Cross-band fusion yields state-of-the-art despeckling on synthetic, natural texture, and real SAR imagery, with the architecture directly leveraging different speckle statistics by frequency (Ma et al., 8 Nov 2025).

7. Impact, Limitations, and Future Directions

Across domains, FaLS achieves notable accuracy, efficiency, and generalization improvements relative to single-frequency or frequency-agnostic baselines. The empirical improvements include substantial reductions in error metrics (e.g., MSE/MAE, SERLE, SI-SDR, PSNR/SSIM), FLOPs, memory, and communication overhead, with robust behavior observed under noise, nonstationarity, and resource constraints (Wu et al., 1 Aug 2025, Liu et al., 28 Mar 2025, Seo et al., 2024, Tariq et al., 27 Sep 2025, Huang et al., 2024, Ma et al., 8 Nov 2025).

Documented limitations and ongoing challenges include the need for domain-specific choice of decomposing transform (e.g., fixed vs. learned wavelets), parameter sensitivity (e.g., threshold tuning, group sizes), and regime-specific instability (e.g., excessive learning rate in physical FaLS, oversmoothing in ODE-based modules). Extensions under consideration include multi-level and higher-order decompositions, adaptive spectral loss penalties, and application to additional physical and information-processing modalities.

FaLS thus constitutes a cross-disciplinary paradigm that leverages the spectral structure of data, informed by machine learning, signal processing, physical system theory, and distributed optimization. Its continued development and principled application remain active areas of investigation on arXiv and in the broader research community.