NAG-Zero in Optimization & Graph Modeling

• NAG-Zero is a dual framework that includes an accelerated gradient method for optimization and a native graph reasoning module for neural models, each eliminating the need for external tuning parameters.
• In optimization, NAG-Zero employs a plain Nesterov update with Lyapunov energy analysis to establish global R-linear convergence without requiring the strong convexity parameter and extends to composite objectives.
• For neural models, NAG-Zero uses frozen Transformer weights with graph token embeddings and topology-aware self-attention to achieve zero-interference graph reasoning and competitive performance.

NAG-Zero denotes two distinct yet influential frameworks in recent research: one in optimization theory, representing a Nesterov-style accelerated gradient method that achieves global R-linear convergence without requiring the strong convexity parameter, and another in neural network modeling, providing a zero-interference method for native graph reasoning in LLMs. Each instantiation—while independent in domain—shares an emphasis on eliminating the need for explicit external knowledge: either bypassing strong convexity parameters (in the optimization context) or avoiding changes to a LLM’s core weights (in the neural context). These threads are summarized respectively from (Bao et al., 2023) and (Gong et al., 30 Jan 2026).

1. Optimization Framework: NAG-Zero Algorithm

NAG-Zero, as introduced by Bao, Chen, and Li, is a variant of the classical Nesterov accelerated gradient (NAG) method tailored for strongly convex and smooth objectives $f \in F_{\mu,L}$, where neither the strong convexity constant $\mu$ nor the extrapolation parameter is required in the iterative update. The paradigm is:

$$\begin{aligned} &\text{Given } x_0 = y_0,\ &\text{For } k = 0,1,2, \ldots:\ &\qquad x_{k+1} = y_k - s \nabla f(y_k) \ &\qquad y_{k+1} = x_{k+1} + \beta_{k+1}(x_{k+1}-x_k) \end{aligned}$$

with fixed step size $s \in (0, 1/L]$, and $\beta_{k+1} = (t_{k+1} - 1)/t_{k+2}$ for a sequence $\{t_k\}$ such as $t_{k+1} = (1+\sqrt{1+4t_k^2})/2$ (the classical Nesterov recursion). Notably, all coefficients can be computed absent any knowledge of $\mu$. This method coincides with the “plain NAG” update presumed optimal for general convex functions, and evidence now establishes that this same protocol achieves global R-linear convergence even in the strongly convex case (Bao et al., 2023).

2. Lyapunov Sequences and Energy Analysis

The theoretical advance underpinning NAG-Zero’s convergence is the construction of Lyapunov (energy) sequences. For $s < 1/L$, an explicit energy is given by

$$E_k = s(t_{k+1} - 1)t_{k+1}\bigl(f(x_k) - f^*\bigr) + \tfrac{1}{2} \| (t_{k+1}-1)(y_k - x_k) + (y_k - x^*) \|^2,$$

and for $s=1/L$, by

$$H_k = \lambda\left[f(x_k)-f^*\right] + \tfrac{1}{2}\|x_k - x_{k-1}\|^2$$

with $\lambda$ determined through a characteristic equation involving $L$ and $\mu$. Precise descent inequalities demonstrate that these quantities decrease Q-linearly or R-linearly, yielding explicit rate constants (e.g., $\rho_k = 1 - 1/\min\{C_k, D_k\}$ for sequences $C_k, D_k>1$).

3. Main Convergence Theorems and Composite Extension

The core result establishes that NAG-Zero achieves global R-linear convergence rates for strongly convex smooth minimization: $$f(x_k) - f^* \leq \left(\prod_{i=0}^{k-1} \rho_i\right)\frac{\|x_0-x^*\|^2}{2s(t_{k+1}-1)t_{k+1}}$$ with $$\rho_k \in (1-\mu s(1-Ls), 1-\mu s(1-Ls)/(1+\max\{\mu/L, 1/8\}))$$.

This result generalizes to accelerated proximal gradient (APG) methods for composite minimization $F(x) = f(x) + g(x)$ by replacing the gradient with the prox-gradient operator $G_s(y)$ in the iterative steps. The same energy analysis applies and yields equivalent R-linear rates.

4. Structural Properties in Neural Modeling (NAG-Zero for Graph Reasoning)

In neural architectures, NAG-Zero is a specific instantiation of the NAG (“Native Architecture for Graphs”) framework designed for text-graph modeling entirely within a LLM’s native computation graph, without invoking an external GNN or altering the backbone parameters (Gong et al., 30 Jan 2026). NAG-Zero is characterized by:

• All pre-trained Transformer weights are frozen.
• Only two new learnable components: special graph token embeddings and layer-wise low-rank gated adapters.
• Adapters are activated exclusively for structural tokens (graph elements); all other tokens pass through the unmodified model path.

This guarantees that linguistic ability for standard text is preserved exactly (“zero-interference”), while allowing the model to propagate graph topology using mechanisms such as topology-aware attention masking and structurally calibrated positional embeddings (synchronized RoPE IDs for graph “hubs”).

5. Topology-Aware Self-Attention and Position Calibration

NAG-Zero leverages a compositional binary mask $M \in \{0,1\}^{|S|\times|S|}$ that augments Transformer self-attention to emulate GNN-style message passing. The mask encodes intra-element causality, inter-element graph edge relationships, global supernode connections, and relevant query-to-graph attention. By enforcing these topological constraints, standard Transformer attention becomes structurally aware. Additionally, rotary position encoding (RoPE) IDs for critical graph “hub” tokens are synchronized, eliminating random linearization artifacts and preventing spurious sequential bias.

6. Empirical Evaluation and Practical Impact

Experimental results on the Qwen3–600M backbone demonstrate that NAG-Zero substantially outperforms traditional dual-path (GNN+LM) models and even parameter-rich alternatives, particularly on synthetic topological reasoning tasks (e.g., Node Count Acc: 99.85%, Edge Count Acc: 93.0%) and competitive semantic graph benchmarks (e.g., ExplaGraphs: 78.16%, WebQSP Hit@1: 41.40%). Notably, this performance is achieved with less than 1% increase in total parameters, and full preservation of linguistic ability for pure-text inputs, confirming the architectural efficiency and utility of NAG-Zero for unified text-graph reasoning.

Method	Node Count Acc	Edge Count Acc	Connected Nodes F1
GraphToken$_{\mathrm{G}}$	82.02%	40.60%	35.62%
GraphToken$_{\mathrm{E}}$	93.26%	59.73%	38.70%
Qwen3-LoRA	100.00%	86.35%	96.85%
NAG-Zero (Ours)	99.85%	93.00%	52.40%

Method	ExplaGraphs Acc	WebQSP Hit@1
GraphToken$_{\mathrm{E}}$	80.51%	42.39%
Qwen3-LoRA	62.09%	44.37%
NAG-Zero (Ours)	78.16%	41.40%

7. Significance and Theoretical Insights

NAG-Zero in optimization represents the first proof of global R-linear convergence for plain Nesterov with unknown strongly-convex parameter, disproving longstanding beliefs derived from ODE analogues. Constants in the rates are explicit, and the technique generalizes to composite (proximal) objectives. In neural text-graph modeling, NAG-Zero validates that graph-structured reasoning can be achieved natively in LLMs—with principled, interpretable modifications, and without trade-offs against pre-trained natural language capability.

NAG-Zero thus establishes new benchmarks for efficiency and integration, both in theoretical optimization and in neural model design for text-graph reasoning (Bao et al., 2023, Gong et al., 30 Jan 2026).