Training-Trajectory-Aware Token Selection (T3S)

• T3S is a paradigm that adapts training by prioritizing tokens based on real-time signals like loss delta and semantic attention.
• It dynamically selects 'yet-to-learn' tokens, focusing gradient updates where they most improve model generalization.
• Empirical results demonstrate improved efficiency and performance in LLM fine-tuning, distillation, and video modeling with reduced computation costs.

Training-Trajectory-Aware Token Selection (T3S) is a paradigm for model efficiency and performance that dynamically selects and weights tokens for training or inference based on the model's own learning trajectory. Its central tenet is that a model's evolving learning dynamics—the "trajectory"—reveal which tokens are crucial for further improvement and which are already mastered or uninformative. T3S has been instantiated across LLM fine-tuning, distillation, video modeling, and video super-resolution, utilizing signals such as retrospective loss changes, semantic attention, and object/motion-centric trajectories to drive token or patch selection. This mechanism improves data efficiency, downstream metrics, and computational/resource allocations across a spectrum of architectures and domains (Qin et al., 21 Oct 2025, Shen et al., 15 Jan 2026, Zheng et al., 29 May 2025, Rai et al., 13 May 2025, Zhu et al., 14 Aug 2025).

1. Theoretical Motivation and Core Concepts

T3S exploits the non-stationary nature of model optimization. Traditional static or loss-only selection often fails to differentiate between tokens that are already overfit, inherently difficult, or semantically unimportant. T3S addresses this by:

• Monitoring the model's training trajectory—i.e., how token-wise metrics (confidence, NLL, embeddings) evolve over optimization steps.
• Focusing gradient and update capacity on "yet-to-learn" tokens that are empirically shown to limit generalization when neglected, while suppressing "imitation anchors" or tokens quickly consolidated by the student model (in distillation) or LLM (in SFT) (Shen et al., 15 Jan 2026).
• Generalizing selection criteria beyond synthetic losses to include semantic signals (e.g., cross-attention to prompts in LLMs (Qin et al., 21 Oct 2025); panoptic sub-object motion in video (Zheng et al., 29 May 2025); motion salience in spatio-temporal video transformers (Rai et al., 13 May 2025); feature-trajectory similarity in VSR (Zhu et al., 14 Aug 2025)).

This trajectory-centric approach provides a continuously self-tuned curriculum, eliminating the need for expensive external reference models or static masking heuristics (Qin et al., 21 Oct 2025, Shen et al., 15 Jan 2026).

2. Mathematical Formulations and Selection Rules

The specific instantiation of T3S depends on the domain, but representative mechanisms include:

A. Self-Modulated Loss Change (LLMs (Qin et al., 21 Oct 2025))

Let $\ell_n(\theta) = -\log P_\theta(x_n|x_{<n})$ be the NLL of token $x_n$. The self-modulated signal is: $$\Delta \ell_n = \ell_n(\theta_\mathrm{prev}) - \ell_n(\theta_\mathrm{curr})$$ This quantifies recent learning progress per token. Selection is normalized and integrated with a semantic attention score $s_n$: $$\widetilde{\Delta\ell_n} = \frac{\Delta\ell_n - \min_k \Delta\ell_k}{\max_k \Delta\ell_k - \min_k \Delta\ell_k},\quad\text{Score}_n = \gamma\widetilde{\Delta\ell_n} + (1-\gamma)s_n$$ Top-$\rho$ tokens by Score$_n$ are chosen for backpropagation.

B. Trajectory-Based Token Masking (Distillation (Shen et al., 15 Jan 2026))

Let $c_t(\theta; x,y) = \log p_\theta(y_t|y_{<t},x)$; compute the trajectory delta: $$\Delta c_t(x,y) = c_t(\theta_b; x,y) - c_t(\theta_0; x,y)$$ Define:

• Imitation-anchors: $t$ with $\Delta c_t>0$
• Yet-to-learn: $t$ with $\Delta c_t < -\tau$ Mask out anchors during loss computation in AR, or guarantee repeated sampling of yet-to-learn tokens in diffusion LMs.

C. Trajectory Semantics in Video (Video Tokenization (Zheng et al., 29 May 2025))

Define a panoptic sub-object trajectory $T_i = \{(M^i_t, c^i_t)\}$ (segment masks and coordinates). Tokenization

$$f^i_t = \frac{\sum_{x,y} M^i_t(x,y) F_t(x,y)}{\sum_{x,y} M^i_t(x,y) + \epsilon}$$

yields appearance/position branches, aggregated via Perceiver-Resampler, producing semantically grounded tokens $\tau_i$.

Tables below summarize the signal types and domains:

Domain	Trajectory Signal	Token Selection Criterion
LLM SFT	Loss delta + attention	High $\Delta\ell_n$ and/or high $s_n$
Distillation	Log-prob delta	Mask out anchors ($\Delta c_t > 0$)
Video Encoding	Sub-object motion trajectories	One token per trajectory
Video SR	Token similarity over flows	Cosine similarity along optical-flow path

3. Algorithmic Workflows and Implementation

An archetypal T3S algorithm for LLM SFT (Qin et al., 21 Oct 2025):

1. At each batch, for each response token $n$, evaluate $\ell_n(\theta_\mathrm{prev})$, $\ell_n(\theta_\mathrm{curr})$
2. Compute and min-max normalize $\Delta\ell_n$
3. Compute semantic attention $s_n$ from attention matrices at a chosen transformer layer
4. Form convex Score$_n$, select top-$\rho$ tokens
5. Mask out the remainder for gradient accumulation
6. Update $\theta_\mathrm{curr}$; update $\theta_\mathrm{prev}$ via EMA as needed

For distillation (Shen et al., 15 Jan 2026):

1. Run a pilot training to find the imitation bottleneck $\theta_b$ (min train accuracy)
2. For all training data, compute $c_t(\theta_0)$ and $c_t(\theta_b)$, derive sets $\mathcal{A}$, $\mathcal{B}$
3. Mask anchors in loss for AR, enforce yet-to-learn for dLLM
4. Continue with masked gradient updates

Video paradigms (Zheng et al., 29 May 2025, Rai et al., 13 May 2025, Zhu et al., 14 Aug 2025) use either deterministic (object tracking, segmentation) or RL-guided (PPO on motion salience) policies to select space-time tokens, often realized as inference-time plug-ins.

4. Empirical Results and Comparative Performance

LLMs (Qin et al., 21 Oct 2025, Shen et al., 15 Jan 2026)

• ssToken with T3S achieves +4.3% accuracy on LLaMA-3.2B, +3.4% on LLaMA-8B, +1.3% on Qwen-7B, +2.1% on Qwen-14B versus full-data fine-tuning (Qin et al., 21 Oct 2025).
• Outperforms baselines such as RHO-1, TokenCleaning, and random token masking, with smaller computational overhead relative to reference-loss methods (15–30% less total training time).
• In distillation, T3S enables Qwen3-8B to surpass DeepSeek-R1 teacher on math reasoning benchmarks and Qwen3-32B to approach Qwen3-235B performance with orders-of-magnitude fewer parameters (Shen et al., 15 Jan 2026). Masking yet-to-learn only destroys learning (28.13 vs. 77.30 AVG), confirming the necessity of trajectory-driven anchor suppression.

Video Models (Zheng et al., 29 May 2025, Zhu et al., 14 Aug 2025, Rai et al., 13 May 2025)

• TrajViT tokenization yields 10× reduction in token count with improved video-text retrieval (+6.0 R@5), better mAP on action detection, and 4.2× faster training, 18× less inference cost on long videos (Zheng et al., 29 May 2025).
• TS-Mamba reports +0.14 dB PSNR margin and 22.7% lower MACs versus next best online VSR competitors, with ablations verifying the role of trajectory-aware loss and scan/shift strategies (Zhu et al., 14 Aug 2025).
• RL-based motion-centric patch selection with trajectory attention (TATS) achieves higher action recognition accuracy at aggressive mask ratios (e.g., 81.75% UCF101, mask 0.95) than AdaMAE and VideoMAE (Rai et al., 13 May 2025).

5. Generalization, Limitations, and Extensions

T3S generalizes across domains:

• Model types: autoregressive LMs, diffusion LMs, spatio-temporal transformers, state space models
• Learning settings: SFT, continual distillation, RLHF, domain adaptation, VSR, masked autoencoding
• Tokenization: semantic/attention signals, motion/object trajectories, model-internal learning signals

Observed limitations:

• Requires in situ monitoring of training accuracy for bottleneck estimation (distillation T3S) (Shen et al., 15 Jan 2026)
• Hyperparameter tuning for thresholds (e.g., $\tau$), tradeoff coefficients ($\gamma$), or attention layer depth (Qin et al., 21 Oct 2025)
• Some reliance on gold labels or trusted verifiers
• Online, adaptive variants (without pre-computation or pilot runs) are the subject of ongoing research

Future work includes hierarchical/online trajectory detection, block-level masking, multi-stage transfer, and cross-model transfer of yet-to-learn token sets (Shen et al., 15 Jan 2026, Qin et al., 21 Oct 2025).

6. Practical Impact and Theoretical Insights

T3S's principal insight is that model improvement is best served by adapting data and loss allocation to the real-time evolution of learning, rather than static or reference-based heuristics. By providing both fine-grained selectivity and interpretability—via semantic attention or tracked object tokens—T3S systematically enhances sample efficiency, model robustness, and computational resource utilization. Its mechanics provide a blueprint for general curriculum learning in high-capacity, data-hungry transformer pipelines. Demonstrated empirical superiority—across LLMs, video transformers, VSR, and SFT—substantiates the paradigm's effectiveness and transferability (Qin et al., 21 Oct 2025, Shen et al., 15 Jan 2026, Zheng et al., 29 May 2025, Zhu et al., 14 Aug 2025, Rai et al., 13 May 2025).