Trajectory2Task: Unified Data-to-Task Paradigm

• Trajectory2Task is a methodological paradigm that transforms raw trajectory data into task-specific solutions using unified representations and modular pipelines.
• It integrates techniques across trajectory mining, robotic manipulation, vehicle prediction, and tool-calling systems through adaptive optimization and closed-loop verification.
• Key strategies include multi-modal embeddings, mask-and-recover frameworks, and agent-driven synthesis, enabling data-efficient learning and significant performance gains.

Trajectory2Task is a broad methodological paradigm for transforming raw trajectory data—sequences representing motions, tool calls, user actions, or spatial paths—into downstream modeling workflows or control policies that solve specific tasks. Across domains such as trajectory mining, robotic manipulation, vehicle prediction, and tool-calling agent development, Trajectory2Task systems automate the mapping from spatiotemporal inputs to high-performing solutions by leveraging unified representations, modular pipelines, and adaptive optimization strategies.

1. Formal Definitions and Taxonomy

Trajectory2Task comprises a taxonomy of modeling objectives and system abstractions across physical, cognitive, and user-centric domains. In location-based modeling, a trajectory is defined as $T=(p_1, p_2, \ldots, p_n)$ with $p_i = (x_i, y_i, t_i)$ or $p_i = \mathrm{POI}_i$ (Du et al., 2024). The task families include:

• Pattern Mining / Classification: Learn a classifier $f_c: T \rightarrow y$, $y \in \{0, 1, \ldots, C-1\}$. Optimize $\mathcal{L}_{\mathrm{cls}}$ (cross-entropy).
• Future Prediction: Next-location prediction $$f_p: (p_1, \ldots, p_k) \rightarrow \mathrm{POI}_{k+1}$$ minimizing $\mathcal{L}_{\mathrm{nl}}$ (negative log-likelihood); travel-time estimation $f_t: (p_1, \ldots, p_k) \rightarrow \Delta t$ minimizing $\mathcal{L}_{\mathrm{tt}}$ (MAE).
• Generation / Recovery: $f_g: \mathrm{seed} \rightarrow \hat{T}$ optimizing $\mathcal{L}_{\mathrm{gen}}$ (trajectory likelihood).

In user-agent systems, trajectories are action-observation sequences underpinning tool-calling policies. A trajectory $\zeta = \{(o_t, y_t)\}_{t=0}^T$ signifies turn-by-turn agent behavior under latent, non-stationary intent modeled by a POMDP. The framework covers “General,” “Ambiguous,” “Changing,” and “Infeasible” user scenarios (Wang et al., 28 Jan 2026).

For control and robotics, the “task” is typically the realization of a reference trajectory via tracking controllers, where each geometric path constitutes a separate objective (Pereida et al., 2017, Singh et al., 2018, Zhou et al., 1 Oct 2025, Tang et al., 9 Oct 2025).

2. Unified Agentic and Modular Architectures

A central principle is the automation of trajectory modeling via modular workflows and unified interfaces:

• LLM-Driven Agent Workflow: In "TrajAgent," the agentic decomposition includes four sub-agents—LLM_U (task understanding), LLM_P (planning), LLM_E (execution), LLM_S (summary)—with memory, error reflection, and prompt-driven operation (Du et al., 2024).
• Unified Environments (UniEnv): Abstracts datasets, models, and analytics under API contracts, supporting flexible data ingestion (GPS/check-in formats), model registration, training, evaluation, inference, and external tools for hyper-parameter and geo-processing.
• Forward–Backward Synthesis Loops: In tool-calling agent tasks, pipelines first simulate trajectory exploration (generating executable gold traces) and then synthesize user-facing tasks with controlled perturbations (ambiguous, changing, infeasible), ensuring closed-loop verification (Wang et al., 28 Jan 2026).

Algorithms encapsulate the entire mapping process, from parsing natural-language intents to structured execution plans and from trajectory rollouts to evaluation reports.

3. Representation Learning and Task Transfer

Efficient transfer across tasks and domains relies on adaptive, often unified trajectory encoders:

• Multi-Modal Embedding Schemes: TransferTraj encodes each GPS point with spatial, temporal, POI, and road network modalities, feeding them into transformer-based encoders equipped with region-invariant relative embeddings (TRIE) and spatial-context mixture-of-experts (SC-MoE) modules (2505.12672).
• Mask-and-Recover Input–Output Framework: Tasks such as next-point prediction, trajectory recovery, and travel-time estimation are unified under modality/point masking and reconstruction, enabling single-model pre-training and zero task-specific re-training.
• Behavior-Adaptive Patching: Flight2Vec addresses flight data sparsity by adaptively clustering high-angle deviations (“behaviors”) and uniform patching for the remainder, informing representation learning for flight trajectory prediction, recognition, and anomaly detection (Liu et al., 2024).
• Sparse Optical Flow Trajectories: TrajSkill converts human videos into sparse motion trajectories $(x, y, \Delta x, \Delta y)$ capturing essential dynamics, providing an embodiment-invariant representation for manipulation skill transfer (Tang et al., 9 Oct 2025).

4. Optimization, Training, and Data-Efficiency

Optimization strategies vary by task but universally focus on data-efficient learning, robustness, and closed-loop verification:

• Collaborative Learning Schemas: TrajAgent employs a meta-controller (LLM agent) to propose data augmentation and hyper-parameters, with inner-loop gradient descent on model losses and outer-loop policy gradients on agent rewards. Joint optimization consistently outperforms single-mode approaches, yielding gains up to 34.96% (Du et al., 2024).
• Multi-Robot, Multi-Task Transfer Learning: L1 adaptive control aligns robot dynamics to reference models, and iterative learning control constructs affine feedforward maps from trajectories to input sequences; new “tasks” are solved from a single demonstration. Empirically, first-iteration errors decrease by ≈74% without retraining (Pereida et al., 2017).
• Closed-Loop Task Verification: In Trajectory2Task for tool-calling agents, only trajectory–task pairs with valid, policy-compliant traces are retained, supporting direct evaluation and supervised SFT (supervised fine-tuning) for robust generalization (Wang et al., 28 Jan 2026).

5. Task Mapping and Plug-In Strategies

Trajectory2Task supports seamless mapping of real-world tasks:

• Task Descriptor Table (editors' extracted):

Task Type	Input/Condition	Output/Target	Metrics
Next-Location Pred.	$p_1, \ldots, p_k$	$\mathrm{POI}_{k+1}$	Acc@5, Hit@5
Travel-Time Estimation	$p_1, \ldots, p_k$	$\Delta t$	MAE
Generation/Recovery	seed, sparse $T$	reconstructed $\hat{T}$	likelihood, RMSE
Classification/Mining	$T$	categorical $y$	Cross-entropy, AUC
Tool-Calling Agent	obs, user utterances	tool call, NL reply	Pass$^k$, stability

Plug-in strategies are implemented via modular head architectures (MLPs, classifiers, etc.) applied directly to unified trajectory embeddings or contextual latent vectors.

6. Empirical Performance and Benchmarks

Several studies demonstrate the effectiveness of Trajectory2Task approaches:

• TrajAgent: Performance gains from 2.38% to 34.96% over baselines—specific improvements include FSQ next-location Acc@5 rising from 0.1795 to 0.2717 (+51.36%), Porto travel-time MAE dropping from 8.48 to 5.85 (–31.01%), and BGK anomaly AUC increasing from 0.978 to 0.984 (Du et al., 2024).
• TransferTraj: Zero-shot region transfer reduces RMSE for trajectory prediction by 83.7% (e.g., Chengdu→Xi’an: 329.8m vs. baseline 1891m); mask-recovery strategy enables arbitrary downstream tasks without retraining (2505.12672).
• Tool-Calling Agents: SFT on ~3K successful trajectories allows a 4B model to match or exceed baseline 32B performance across ambiguity, changing intent, and infeasibility, with cross-domain generalization (Retail→Airline) outpacing prior models (Wang et al., 28 Jan 2026).
• Skill Transfer: Traj2Action achieves +27% (short-horizon) and +22.25% (long-horizon) gains over vision-language policy baselines in robot manipulation tasks by leveraging 3D trajectory priors (Zhou et al., 1 Oct 2025); TrajSkill improves video fidelity (FVD –39.6%) and cross-embodiment success rates (+16.7%) (Tang et al., 9 Oct 2025).
• Flight2Vec: For 15 min prediction, achieves MAE(lon)=0.0124, MAE(lat)=0.0121, MAE(alt)=7.58, outperforming PatchTST and FlightBERT++; anomaly detection AUC = 0.9050 (Liu et al., 2024).

7. Current Limitations and Forward Directions

Current Trajectory2Task implementations exhibit several constraints:

• Some architectures are domain-specific (e.g., trajectory mining vs. robotic control) and depend on the definition of trajectory “behavior.”
• Orientation and force priors in manipulation scenarios remain a challenge for generalization beyond 3D spatial paths (Zhou et al., 1 Oct 2025).
• Closed-loop dynamic adaptation and real-time disturbance rejection are open engineering challenges.
• Scaling to heterogeneous agents and ultra-high-dimensional datasets (e.g., massive tool-call DAGs) is ongoing.

A plausible implication is the extension of Trajectory2Task principles to broader agentic and data-centric optimization settings, including mixed-modal multi-agent interaction, using unified representation learning and policy adaptation frameworks.