MetaGradNorm: Adaptive Multi-Objective Training

• MetaGradNorm is a meta-learning based multi-objective optimization technique that balances training objectives by equalizing gradient norms from contrastive and ordinal losses.
• It employs per-sample dynamic gating and entropy regularization to adapt loss contributions, ensuring robust and stable fine-tuning of models like LLMs.
• Empirical results demonstrate improved F1 scores, reduced variance, and smoother loss curves, highlighting its effectiveness in multi-task deep learning pipelines.

MetaGradNorm is a meta-learning-based multi-objective optimization technique designed to dynamically balance competing training objectives by aligning their gradient norm contributions. It was developed to address challenges in multi-task and multi-loss deep learning settings, such as the structured fine-tuning of LLMs for robust greenwashing detection in ESG claim analysis. In this context, MetaGradNorm governs the interplay between contrastive and ordinal ranking objectives, adapting their relative influence throughout training and thereby improving stability and generalization performance (Braun et al., 29 Jan 2026).

1. Mathematical Foundations of MetaGradNorm

MetaGradNorm operates on the principle of equalizing the $\ell_2$-norms of gradients corresponding to each task-specific objective, ensuring that neither dominates the learning dynamics over the other. Let $\mathcal{L}_{\mathrm{ctr}}^{(i)}$ and $\mathcal{L}_{\mathrm{ord}}^{(i)}$ denote the per-sample contrastive and ordinal ranking losses, respectively. After applying per-sample gating weights $w_{\mathrm{ctr}}^{(i)}$ and $w_{\mathrm{ord}}^{(i)}$, the overall batched, weighted loss is: $$\mathcal{L}(\theta; \alpha) = \frac{1}{B} \sum_{i=1}^B \left[\lambda_{\mathrm{base}} w_{\mathrm{ctr}}^{(i)} \mathcal{L}_{\mathrm{ctr}}^{(i)} + \lambda_{\mathrm{ord}} w_{\mathrm{ord}}^{(i)} \mathcal{L}_{\mathrm{ord}}^{(i)}\right]$$ where $$\alpha = \{\lambda_{\mathrm{base}}, \lambda_{\mathrm{ord}}, T_{\mathrm{ctr}}, T_{\mathrm{ord}}\}$$ are meta-parameters.

The gradient norms for each loss component are: $$G_{\mathrm{ctr}} = \left\| \nabla_\theta \frac{1}{B} \sum_i \lambda_{\mathrm{base}} w_{\mathrm{ctr}}^{(i)} \mathcal{L}_{\mathrm{ctr}}^{(i)} \right\|_2, \quad G_{\mathrm{ord}} = \left\| \nabla_\theta \frac{1}{B} \sum_i \lambda_{\mathrm{ord}} w_{\mathrm{ord}}^{(i)} \mathcal{L}_{\mathrm{ord}}^{(i)} \right\|_2$$ Relative task difficulty is tracked via normalized losses: $$\tilde{L}_k(t) = \frac{L_k(t)}{L_k(0) + \varepsilon}, \quad r_k(t) = \left(\frac{\tilde{L}_k(t)}{\frac{1}{2} \sum_{j \in \{ \mathrm{ctr}, \mathrm{ord} \}} \tilde{L}_j(t)}\right)^{\gamma}$$ with exponent $\gamma > 0$. The target gradient norm for each task is set by $G_k^* = \bar{G} \cdot r_k(t)$, where $$\bar{G} = \frac{1}{2}(G_{\mathrm{ctr}} + G_{\mathrm{ord}})$$.

The meta-objective for updating $\alpha$ is: $$\mathcal{J}(\alpha \mid \theta) = \left|G_{\mathrm{ctr}} - G_{\mathrm{ctr}}^*\right| + \left|G_{\mathrm{ord}} - G_{\mathrm{ord}}^*\right| + \beta \mathcal{R}_{\mathrm{ent}}(\alpha)$$ where

$$\mathcal{R}_{\mathrm{ent}}(\alpha) = -\frac{1}{B}\sum_{i=1}^B \left(w_{\mathrm{ctr}}^{(i)} \log w_{\mathrm{ctr}}^{(i)} + w_{\mathrm{ord}}^{(i)} \log w_{\mathrm{ord}}^{(i)}\right)$$

is an entropy regularizer to encourage non-collapsing weights and $\beta \geq 0$ controls its strength.

Both $\theta$ (model parameters) and $\alpha$ (meta-parameters) are updated with stochastic gradient descent, using softplus reparameterization to ensure positivity of $\lambda$ and $T$: $$\theta \leftarrow \theta - \eta_\theta \nabla_\theta \mathcal{L}(\theta; \alpha); \quad \alpha \leftarrow \alpha - \eta_\alpha \nabla_\alpha \mathcal{J}(\alpha \mid \theta)$$

2. Integration into Multi-Loss Training Pipelines

MetaGradNorm is embedded at the core of the fine-tuning loop for LLMs enhanced with LoRA adapters. At each mini-batch, the following steps are performed:

1. Extract normalized embeddings from the model.
2. Construct positive and negative sets for contrastive and ordinal tasks.
3. Compute per-sample contrastive and ordinal losses.
4. Calculate gating weights via a softmax over normalized-loss-to-temperature scores for each sample.
5. Aggregate the weighted losses and update model parameters.
6. Compute individual task gradient norms.
7. Derive loss normalizations and difficulty ratios.
8. Compute target gradient norms and meta-objective, including entropy regularization.
9. Update meta-parameters.

The workflow ensures real-time adaptive reweighting between loss components, allowing for dynamic adjustment based on learning progress (Braun et al., 29 Jan 2026).

3. Key Extensions and Modifications

Several refinements distinguish this implementation of MetaGradNorm from original approaches:

• Per-sample dynamic gating: Weights $w_{\mathrm{ctr}}^{(i)}$ and $w_{\mathrm{ord}}^{(i)}$ are computed per sample via a softmax, as opposed to static task weights in vanilla GradNorm. This facilitates granular, instance-specific emphasis on harder objectives.
• Entropy regularization: The term $\beta \mathcal{R}_{\mathrm{ent}}(\alpha)$ discourages degenerate solutions where all weight concentrates on one loss, maintaining balanced learning.
• Softplus reparameterization: Ensures internal meta-parameters remain strictly positive and differentiable, improving optimization stability.
• Joint meta-learning of $\{\lambda, T\}$ and model parameters: Online updates adapt both scaling and gating temperatures in tandem with model weights.

These variations enable robust adaptation of gradient resource allocation not only at the batch level but also for individual samples and epochs. The practical outcome is improved convergence, consistent learning dynamics, and reduced requirement for manual hyperparameter tuning.

4. Hyperparameter Choices and Tuning Strategies

Hyperparameter selection follows a staged regime:

• LoRA adaptation: Learning rate $\eta_{\mathrm{FT}}=3 \times 10^{-5}$, rank $r=8$, scale $\alpha=16$.
• Contrastive loss: Learning rate $\eta_{\mathrm{ctr}}=1 \times 10^{-4}$; temperature $\tau=0.07$; 2–3 positive/negative samples per anchor.
• Ordinal loss: Margin $m_0$ explored in $\{0.05, 0.10, 0.15\}$ (optimized as $m_0=0.05$).
• Gating temperatures: For T5, $(T_{\mathrm{ctr}}, T_{\mathrm{ord}}) = (5.8, 1.0)$; for 7B parameter LLMs, $(13, 1)$.
• Loss-scalers: T5 uses $$(\lambda_{\mathrm{base}}, \lambda_{\mathrm{ord}}) = (1.0, 1.4)$$; LLMs use $(1.0, 2.5)$.
• MetaGradNorm-specific: Trade-off exponent $\gamma=0.5$, entropy weight $\beta=0.01$, and meta-learning rate $\eta_\alpha=10^{-3}$.

All major hyperparameters are tuned offline on a held-out fold and then fixed for consistency across experimental runs and model types.

5. Empirical Effects on Stability and Performance

Empirical ablation demonstrates that augmenting contrastive + ordinal + gating + fixed scaling with MetaGradNorm yields:

• F1 score improvement: LLaMA-3-8B sees mean seen F1 improvement from 0.613 to 0.623 and unseen from 0.396 to 0.411.
• Variance reduction: Standard deviation of unseen F1 across epochs is reduced (from ±1.2 pp to ±0.8 pp).
• Gradient-norm consistency: Relative disparity $$\left|\frac{G_{\mathrm{ctr}}}{G_{\mathrm{ord}}} - 1\right|$$ after 10 epochs falls below 10% (compared to >25% without MetaGradNorm).
• Seen–unseen gap: For T5, full-framework F1 increases from 0.7172 to 0.7240, while shrinking the seen–unseen gap $\Delta$ by ≈0.3 pp.
• Loss curve smoothness: Training curves with MetaGradNorm show monotonic, stable improvement in both objectives, and suppress transient dominance of either loss, facilitating more stable convergence.

A plausible implication is that MetaGradNorm significantly improves the stability and robustness of multi-loss LLM training pipelines and lessens the necessity for exhaustive manual task weight search (Braun et al., 29 Jan 2026).

6. Context and Related Methodologies

MetaGradNorm extends the principles of vanilla GradNorm through enhanced sample-level adaptivity, entropy regularization, and meta-learning of task weights and gating temperatures. Its design specifically targets the needs of LLM-based pipelines where multi-objective optimization is paramount, such as extracting nuanced semantic distinctions in contexts susceptible to ambiguity and disclosure noise, e.g., ESG claim validation.

Comparative empirical findings indicate that MetaGradNorm-equipped frameworks exhibit a less pronounced trade-off between representation rigidity and generalization under cross-category evaluation scenarios. This suggests broader applicability for other regimes where multiple objectives must be jointly optimized without manual heuristics for weighting.

Component	MetaGradNorm Approach	Vanilla GradNorm
Task Weighting	Per-sample, dynamic (via softmax)	Global, static
Entropy Regularization	Present ($\beta$-weighted)	Usually absent
Meta-Learning Updates	Jointly for model and meta-params	Typically only task wts
Hyperparameter Tuning	Automated/staged	Manual

The progressive refinement in MetaGradNorm methodology enables effective deployment of scalable, parameter-efficient multi-objective training in contemporary LLM implementations (Braun et al., 29 Jan 2026).