Dynamic Weight Sensitivity Metric

• Dynamic Weight Sensitivity Metric is a framework that quantifies the impact of perturbations on model outputs and algorithm stability using Taylor-based and integral methods.
• These metrics are applied in robust dynamic programming, post-training quantization, and parameter-efficient adaptation to optimize resource allocation while preserving model fidelity.
• Practical algorithms leveraging these metrics enable adaptive compression and dynamic rank assignment, providing tight bounds on loss changes and improved robustness.

Dynamic weight sensitivity metrics quantify the impact of perturbations to the weights, activations, or input data on the output, loss, or structure of machine learning models or combinatorial algorithms. These metrics underpin a diverse set of frameworks spanning robust dynamic programming (DP), post-training quantization (PTQ) of deep neural networks, and parameter-efficient adaptation strategies. Unifying these lines is the principle of formally assessing how solution properties (accuracy, sparsity, rank, etc.) or model behavior change in response to explicit or implicit weight perturbations, and leveraging these measures for principled algorithmic design, compression, or adaptation.

1. Formal Definitions: Sensitivity in Dynamic Programming and Neural Networks

Dynamic weight sensitivity manifests differently across domains, but its core formalism shares a Taylor-based or perturbation-based lineage.

Average Sensitivity in DP

For a randomized algorithm $\Alg$ operating on input $V$ of size $n$, the average sensitivity is defined as

$\mathsf{AS}(\Alg,V) = \frac1n\sum_{i\in V}\mathrm{EM}\bigl(\Alg(V),\,\Alg(V\setminus\{i\})\bigr)$

where $\mathrm{EM}(\cdot,\cdot)$ is the earth-mover (minimum-matching) distance—often the Hamming distance between solution sets. All randomness is averaged over the algorithm's internal processes (Kumabe et al., 2021). This extends naturally to problems reduced to a chain-finding instance on a DAG, considering deletion of antichain subsets.

Activation Sensitivity in Neural Networks

For a feedforward network layer with weights $W\in\mathbb R^{d_{\rm out}\times d_{\rm in}}$, average activation sensitivity of input channel $c$ is defined as

$$S_c = \mathbb E\left[ \left\|G^\top X_{:,c}\right\|_2^2 \right]$$

where $X\in\mathbb R^{n\times d_{\rm in}}$ are input activations over $n$ calibration samples and $$G = \partial \mathcal L/\partial Y \in \mathbb{R}^{n\times d_{\rm out}}$$ are per-sample loss gradients with respect to output $Y = XW^\top$ (Xu, 15 Jan 2026).

Post-Quantization Integral (PQI)

For quantization perturbations $\delta$ between pre- and post-quantized weights $w$ and $w+\delta$, the PQI sensitivity is defined by integrating loss gradients along the straight line between $w$ and $w+\delta$: $$v_{\rm PQI} = \int_{0}^{1}\nabla F\bigl((1-t)w + t(w+\delta)\bigr)dt \quad,\quad \mathrm{AFPQI}_i = |v_{{\rm PQI}, i}|\,|\delta_i|$$ with the total impact

$\Delta F \approx \sum_i \mathrm{AFPQI}_i$

(Hu et al., 28 Feb 2025).

Hessian-Based Sensitivities

Sensitivity at the parameter level is also characterized via the diagonal of the Hessian: $$\Delta E \approx \frac12\sum_{i} h_{ii} (\delta w_i)^2$$ where $h_{ii}$ are diagonal entries of the Hessian $H$ of the loss $E(w)$ (Zhang et al., 11 Sep 2025).

2. Foundational Role in Algorithmic Stability and Model Compression

Dynamic weight sensitivity plays several foundational roles across fields:

• Evolutionary Stability in DP: In dynamic programming, average sensitivity quantifies the expected "cost" (in solution change) of single-element edits, serving as a proxy for the algorithmic stability required in robust data scenarios such as noisy biological sequence analysis, document comparison, or RNA folding (Kumabe et al., 2021).
• Weight Perturbation in Deep Learning: In neural network PTQ, sensitivity metrics identify the weights or channels whose errors most strongly impact the post-quantization loss, guiding resource allocation (e.g., mixed-precision quantization) or post-processing (e.g., outlier preservation) for minimum accuracy degradation (Xu, 15 Jan 2026, Hu et al., 28 Feb 2025).
• Parameter-Efficient Adaptation: Recent LoRA variants employ second-order sensitivity estimates to optimally allocate low-rank adaptation capacity to weight matrices, overcoming uniform-rank limitations (Zhang et al., 11 Sep 2025).

These roles share a commonality: sensitivity metrics tie quantifiable modelor solution changes to explicit perturbations in structure, data, or parameters, enabling informed trade-offs between fidelity, efficiency, and robustness.

3. Comparative Methodologies: From Taylor Expansions to Path Integrals

Methodological frameworks for dynamic weight sensitivity fall into several classes:

• First-Order Taylor Expansion: Sensitivity is inferred by evaluating directional gradients, e.g., the gradient-norm $\|\nabla_{W_{ij}} \mathcal L\|$ or activation-weighted gradients, capturing only infinitesimal perturbations around a point (Xu, 15 Jan 2026).
• Second-Order (Hessian) Expansion: Incorporates local curvature information via the diagonal Hessian, yielding weight-level terms proportional to $\tfrac12 h_{ii} (\delta w_i)^2$ (Zhang et al., 11 Sep 2025). Classical pruning methods such as Optimal Brain Damage (OBD) and Optimal Brain Surgeon (OBS) utilize diagonal and full Hessian information, respectively.
• Integral/Path Methods: PQI integrates the instantaneous change in gradient over the actual path from the unquantized to the quantized parameter, yielding

$v_{\rm PQI} = \int_0^1 \nabla F(w(t))\,dt$

and addressing the breakdown of local approximations for large perturbations (Hu et al., 28 Feb 2025).

• Earth-Mover/Coupling Distances: In algorithmic settings, the minimum-matching distance between outputs quantifies combinatorial change, accounting for randomization and discrete solution structures (Kumabe et al., 2021).
• Fused Metrics: Higher-level metrics (e.g., in Sensitivity-LoRA) combine global (trace of Hessian), local (top-$k$ or effective rank of Hessian diagonal), or cross-layer aggregation to drive adaptive rank or bit allocation using proportional scaling strategies (Zhang et al., 11 Sep 2025).

The choice of framework dictates both the fidelity of sensitivity estimates (posterior vs. prior; local vs. aggregated) and the computational cost and granularity with which adaptation or quantization decisions can be made.

4. Practical Algorithms and Sensitivity-Driven Pipelines

Multiple pipelines leverage these metrics for algorithmic control or model deployment:

• Stable Dynamic Programming: Recursively stable algorithms for maximum weighted chains in transitive DAGs and their reductions ensure average sensitivity $O(K\,\delta^{-1}\log^3 n)$ for approximate solutions to LIS, interval scheduling, LCS, LPS, knapsack, and RNA folding, where $K$ is the antichain-deletion overlap (Kumabe et al., 2021).
• PTQ Pipelines (ReQuant): PQI-driven post-training quantization leverages a two-stage pipeline:
  1. Self-adaptive outlier selection: Layer-wise allocation of high-precision weights based on AFPQI, using a grid-searched "temperature" to redistribute a global outlier budget.
  2. Step-wise significant-weight detach: Iterative, greedy restoration of significant weights by AFPQI·$|\delta|$, enabling precise control of accuracy-resource trade-off (Hu et al., 28 Feb 2025).

Quantizer	Baseline PPL	+ReQuant PPL	PPL Δ
AWQ (3b)	16.74	15.36	1.38
QTIP (4b)	4.26	2.13	2.13

(Abridged from (Hu et al., 28 Feb 2025), Table 6–8.)

• Dynamic Rank Assignment (Sensitivity-LoRA): The fusion of global and local Hessian-based sensitivities yields blockwise allocation weights $\theta^W$, with proportional Scaled Rank Allocation producing robust, computation-efficient LoRA rank assignments. The pipeline requires only a few calibration batches and achieves rapid convergence of stable rank ordering (Zhang et al., 11 Sep 2025).

5. Limitations, Empirical Comparisons, and Dynamic Allocation

While first- and second-order sensitivity metrics are computationally efficient, empirical analysis demonstrates that their accuracy deteriorates for moderate or large quantization steps due to the limited convergence radius of Taylor expansions. As shown in (Hu et al., 28 Feb 2025), actual changes in loss $\Delta F$ are often under-predicted by an order of magnitude when using these approximations only. PQI, as an integral metric, closes this gap and provides tight empirical upper bounds on loss impact.

For PTQ and PEFT, sensitivity-driven dynamic allocation (e.g., iterative recalculation of $S_c$ post-quantization, or blockwise rank reassignment in LoRA fine-tuning) allows models to adapt to evolving error landscapes or input perturbations. However, such dynamic recalculation incurs additional calibration cost and samples, presenting a trade-off between allocation adaptivity and practical throughput (Xu, 15 Jan 2026, Zhang et al., 11 Sep 2025).

6. Broader Implications and Future Directions

The dynamic weight sensitivity paradigm unifies approaches across discrete algorithms and neural model compression:

• Robustness Guarantees: Stable-on-average DP algorithms constructed via randomized pivoting and recursive decomposition systematically reduce the propagation of small input changes throughout long solution chains, offering provable bounds on solution perturbation (Kumabe et al., 2021).

• Connections to Privacy: Algorithms with low average sensitivity may serve as strong candidates for differentially private releases or confidential model serving (Kumabe et al., 2021).
• Extension Across Architectures: Integral sensitivity metrics generalize to convolutional nets, activation quantization, and quantization-aware training, with only the definition of interpolation paths requiring adaptation (Hu et al., 28 Feb 2025).
• Adaptive Compression: Sensitivity metrics enable highly granular mixed-precision or resource allocation—per weight, per channel, or per block—superseding heuristic or uniform strategies (Zhang et al., 11 Sep 2025).
• Unification of Pruning and Quantization Theory: By situating AWQ, GPTQ, OBD, OBS, and Fisher metrics in a common sensitivity framework, current PTQ methods can be critically compared and potentially fused (Xu, 15 Jan 2026).

Potential research avenues include worst-case (not just average or posterior) sensitivity estimates, continuous-weight or non-discrete input settings, and rapid recalibration for online or streaming adaptation. The sensitivity-driven paradigm is thus a central tool for robust, efficient, and trustworthy algorithm and model design.