Next Key Point (NKP) in Sequential Prediction

• NKP is a modeling concept that conditions outputs on discrete semantic tokens, capturing high-level intent for improved sequential predictions.
• It reformulates prediction into a two-level hierarchy where intent selection guides local autoregressive generation, enhancing output consistency.
• NKP integration reduces common errors such as drift and duplication, demonstrating substantial performance gains in vessel trajectories and visual detection.

The Next Key Point (NKP) concept refers to the explicit conditioning of model outputs on discrete, semantic-critical points or tokens that encode high-level intent or structural guidance in sequential prediction tasks. NKP has emerged as a powerful architectural and methodological device in both long-horizon trajectory prediction and visual perception frameworks, where modeling future outputs in terms of semantic transitions (rather than raw coordinate regression) yields demonstrable advantages in consistency, token efficiency, and geometric accuracy (Gan et al., 26 Jan 2026, Jiang et al., 14 Oct 2025). NKP may represent conditional waypoints in vessel navigation, or discrete coordinate tokens in image-object detection. By reframing intractable sequential prediction as a two-level hierarchy—intent selection via NKP, then conditioned local autoregression—NKP restricts model output support to feasible, contextually relevant subspaces, improving directional consistency and alignment while mitigating common drift, duplication, and large-box failure scenarios.

1. Formal Definitions and Scope of NKP

NKP is typically instantiated as a discrete latent variable $Z$ encoding semantic intent, navigation transitions, or quantized coordinates. In vessel trajectory modeling, $Z$ defines the equivalence class of all future trajectories sharing a navigational decision, e.g., passage through a port channel or strait. Formally, if $X=(x_1,\dots,x_{T_0})$ is the observed history and $Y=(y_1,\dots,y_T)$ the future sequence, the NKP $Z$ clusters all $Y$ sharing $Z$ as the next intended semantic step (Gan et al., 26 Jan 2026).

In vision tasks, NKP corresponds to a special coordinate token drawn from a quantized vocabulary, $$T_{\text{coord}}=\{\langle 0 \rangle, \dots, \langle 999 \rangle \}$$, which represents positions normalized over the image (Jiang et al., 14 Oct 2025). This tokenization replaces multi-digit output atomization with a compact, self-delimiting scheme.

Application Area	NKP Semantic Definition	Output Token/Label Description
Vessel trajectory	Next navigational intent (port/channel/strait)	Discrete route node labels
Object detection (MLLM)	Next quantized coordinate in image sequence	Special coordinate language token

2. Probabilistic Modeling Frameworks

NKP’s role in probabilistic modeling is to restructure prediction as hierarchical intent selection plus conditioned autoregressive generation. The standard factorization in vessel trajectory prediction is

$p(Y|X) = \sum_z p(Z=z|X) \cdot p(Y|X, Z=z)$

where $p(Z|X)$ is the NKP prior (semantic intent inference) and $p(Y|X, Z)$ generates the output sequence under known intent (Gan et al., 26 Jan 2026). The trajectory component is further factorized as

$p(Y|X, Z) = \prod_{t=1}^T p(y_t | X, Z, y_{<t})$

In visual sequence modeling, the objective is next-token prediction, treating the entire coordinate sequence as language (Jiang et al., 14 Oct 2025). For bounding boxes, output is a sequence of coordinate tokens:

<box_start><x0><y0><x1><y1>, ...<box_end>

For pointing and keypointing, the sequence simply consists of successive $(x, y)$ pairs, each tokenized.

3. Training and Inference Methodologies

NKP-based architectures rely on staged learning paradigms to disentangle intent inference from conditional output modeling. The vessel trajectory framework uses:

• Stage A: Conditional trajectory modeling with an oracle NKP, training $p(Y|X, Z)$ under teacher-forcing with losses for (SOG, COG) and (lat, lon) (Gan et al., 26 Jan 2026).
• Stage B: NKP inference modeling via contrastive fine-tuning, the model learns to encode historical observations $X$ into embeddings clustered by NKP via a contrastive loss,

$$L_{TCL} = \frac{1}{B} \sum_{i=1}^B [ y_{ij}\cdot\max(0, M-\text{sim}(h_i,h_j))^2 + (1-y_{ij})\cdot\text{sim}(h_i,h_j)^2 ]$$

• Stage C: Database voting at inference, the model retrieves reference embeddings and casts votes for $Z$ labels with high similarity.

In Rex-Omni (MLLMs for detection), a two-stage sequence prediction pipeline is employed:

• Supervised Fine-Tuning (SFT): Standard cross-entropy loss on 22M examples for next token prediction.
• Group Relative Policy Optimization (GRPO) RL post-training: Utilizes geometry-aware rewards (IoU, point-in-mask, point-in-box) to penalize duplicate outputs, erroneous coverage, and poorly aligned boxes (Jiang et al., 14 Oct 2025).

4. Architectural Integration of NKP

Integration varies by domain but always places NKP at the critical interaction point between global state encoding and local output generation. In the SKETCH framework for trajectory prediction, the architecture consists of:

• Encoder 1 and MiniMind 1 for historical token input.
• An MLP for NKP coordinate prediction.
• Encoder 2 for embedding the NKP.
• Concatenation (“$H^1 \Vert H^2$”), followed by further decoding in MiniMind 2 and a masked decoder to produce output predictions.
• Conversion of (SOG, COG) into (lat, lon) updates via local-linear motion equations.

In Rex-Omni, all prediction tasks leverage single-token coordinate modeling for each point, obviating the need for separate regression heads. Each segment is delimited by special tokens, and the model outputs coordinate tokens sequentially.

5. NKP’s Effect on Output Consistency and Error Suppression

Explicit conditioning on NKP restricts the support of possible outputs to semantically plausible regions. In vessel forecasting, models with NKP generate globally consistent, smooth turns and proper port entries, while models without NKP drift into straight, east–west sequences, ignoring navigational reality. Empirical ablations demonstrate that correct NKP selection is both necessary and sufficient for robust long-horizon prediction—replacing the predicted $Z$ by the oracle $Z$ yields marginal improvement, but a wrong $Z$ leads to drastic performance collapse (Gan et al., 26 Jan 2026).

In visual perception, the next-point token paradigm mitigates failure modes inherent in teacher-forced regression, including over-generation (duplicate boxes) and collapse to large, imprecise regions. Geometry-aware RL rewards in the second training stage let models learn to suppress duplicate and oversized boxes, yielding improvements in F1 recall and precision on COCO, LVIS, VisDrone, and Dense200 datasets. Single-token coordinates also produce efficient, short output sequences and rapid inference (Jiang et al., 14 Oct 2025).

Scenario	Benefit of NKP Conditioning	Common Failure Mode (no NKP)
Vessel trajectory	Global course fidelity, smooth turns	Drifting or implausible paths
Detection (MLLM, SFT only)	Reduced duplication, improved recall	Duplicate boxes, large coverings
Keypoint prediction	Flexible extension via sequence tokens	Requires separate regression heads

6. Empirical Performance and Generalization Properties

NKP conditioning improves quantitative performance across multiple axes:

• Vessel trajectories: Mean squared position error (MSEP) drops from 1.6 (MP-LSTM) and 0.71 (TrAISformer) to 0.41 (NKP model). Mean curvature error (MSEC) falls by an order of magnitude, and mean Fréchet distance (MFD) reduces from 31.11/19.78 to 7.80, signifying enhanced global-shape matching. On public datasets, NKP-attuned models yield lowest MSEP and MFD, showing generalized spatial robustness (Gan et al., 26 Jan 2026).
• Object detection: RL-trained outputs with NKP display near-elimination of duplicate predictions and improved recall/F1. Token-efficient coordinates support flexible output modalities (pointing, keypointing, OCR, GUI grounding) with performance comparable to or exceeding regression-based counterparts (Jiang et al., 14 Oct 2025).

An ablation analysis emphasizes the criticality of NKP quality. Accurate inference is sufficient for robust performance; misclassification of $Z$ leads to pronounced degradation, particularly in trajectory curvature accuracy (Gan et al., 26 Jan 2026). This suggests that as NKP-based methods become standard, focus must also shift to improving semantic intent inference, not just local autoregressive modeling.

7. Extensions and Broader Implications

NKP—also referred to as Next Point Prediction in vision—generalizes beyond specific application domains, enabling unified treatment of both global intent and local token emission in language-style generative architectures. This paradigm yields broader benefits in model compositionality, open-set generalization, and supports extensible tasks (spatial referring, visual prompting, keypoint annotation) without redesign or bespoke regression heads (Jiang et al., 14 Oct 2025). A plausible implication is that NKP-style conditioning may become a foundational pattern for multimodal sequence prediction tasks with variable output structure and semantic ambiguity.

In summary, explicit NKP modeling restructures sequential prediction problems into tractable, semantically informed hierarchies, materially improving output consistency, efficiency, and adaptability across trajectory, detection, and keypointing tasks.