Prompt4RE: Task-Aware Region Embeddings

• The paper introduces a prompt-based approach that injects task-relevant semantic cues into region embeddings, reducing MAE by up to 64.2% on urban prediction tasks.
• The methodology leverages multimodal cross-attention and graph-based prompting to align spatial features with specific task objectives using expert-designed or learnable prompts.
• Empirical results across urban datasets demonstrate significant improvements in accuracy and robustness, validating Prompt4RE as a versatile tool for spatial analysis.

Task-aware prompting for region embeddings (Prompt4RE) is an approach that integrates explicit task semantic information into learned representations of spatial regions, improving their utility for downstream prediction and classification tasks. This paradigm has emerged in response to the limitations of traditional region embedding techniques, which often produce task-agnostic features, limiting adaptability and downstream performance. Prompt4RE typically enhances region embeddings by leveraging prompts—structured semantic cues (from expert knowledge, templates, or learned graph structures)—to align the learned representations with task objectives in either urban spatial analysis or multimodal domains (Jin et al., 2024, Guo et al., 2 Feb 2026).

1. Conceptual Foundation and Motivation

Conventional region representation learning methods commonly rely on two-stage pipelines: first, embeddings are created via graph-based, multimodal, or visual architectures, followed by the application of these representations to supervised tasks (e.g., crime or crash prediction). However, since the embeddings are learned in a task-agnostic manner, there is an inherent mismatch between the extracted spatial/semantic patterns and the requirements of specific downstream objectives. Recent work has identified this as a key bottleneck (Guo et al., 2 Feb 2026). Prompt4RE addresses this gap by introducing a targeted prompt mechanism that explicitly injects task-relevant information—such as region descriptions, context, or query semantics—into the region embedding pipeline, resulting in more discriminative and task-aligned representations.

2. Architectural Formulations

There are two primary architectural instantiations of Prompt4RE:

• Multimodal Cross-Attention Prompting (ToPT framework) (Guo et al., 2 Feb 2026): Region embeddings $E \in \mathbb{R}^{N\times d}$ (where $N$ is the number of regions) are aligned with prompt representations $P\in \mathbb{R}^{L_p\times d}$ extracted from a frozen multimodal LLM (MLLM) given task templates. A multi-head cross-attention mechanism projects both $E$ and $P$ into a shared space through

$$Q^{(i)} = E\,W_Q^{(i)},\quad K^{(i)} = P\,W_K^{(i)},\quad V^{(i)} = P\,W_V^{(i)},$$

and integrates the prompt’s task semantics into each region embedding. The output, $F = [E \,\Vert\, S] \in \mathbb{R}^{N\times 2d}$, concatenates the original regions with their respective “soft prompts,” where $S$ results from residual fusion and projection post-attention.

• Graph-Based Prompting (GURPP framework) (Jin et al., 2024): Here, region embeddings are derived from heterogeneous urban region graphs $G=(V,E,\varphi,\psi)$ and further refined via prompts. Prompts may be:
  • Manually defined: Adjust subgraph structure based on expert knowledge (e.g., up-weighting business node types for crime prediction) before region embedding computation;
  • Learnable trainable prompt graphs: A set of small trainable graphs, with learned node attributes $X^p$, are compared to the region’s induced subgraph $g_r$ via a random-walk kernel $K_p(g_r,G^p_i)$. The vector of kernel similarities is fused with the pre-trained embedding to yield task-aware representations, which are then fine-tuned in a supervised setting.

Both strategies preserve the general spatial and relational semantics while explicitly modulating the representation space to encode task-intent.

3. Prompt Design and Semantic Encoding

Prompt formulation is central to Prompt4RE.

• In multimodal paradigms, prompts may include task-dependent textual templates (“Based on the above, how likely is the crime rate to exceed…?”), region-specific metadata (coordinates, POI categories), and associated images (satellite, street-view). Processed through an MLLM, these yield dense semantic vectors $P$.
• In graph-based paradigms, prompts are over subgraph patterns or synthetic graphs, constructed either by hand (via domain-derived subgraph extractions and type weighting) or learned (optimizing prompt graphs to maximize alignment with task labels). The similarity between region subgraphs and prompt graphs defines the degree of semantic alignment, and downstream prediction heads operate on concatenated or fused features.

This explicit conditioning enables the adaptation of a single representation backbone to a suite of related, but distinct, spatial tasks.

4. Learning Objectives and Training Protocols

Prompt4RE is trained end-to-end with the following regime:

• Frozen Backbone: The spatial encoder (SREL in (Guo et al., 2 Feb 2026), gEnc in (Jin et al., 2024)) and the prompt encoder (MLLM, pre-training graph) are frozen during task-specific prompt tuning and downstream optimization. Only the cross-attention projections, residual connection, prompt graph parameters, and final prediction head are updated.
• Loss Functions: For regression objectives (e.g., crime forecasting), Mean Squared Error (MSE) loss is used,

$$L_{\mathrm{MSE}} = \frac{1}{N}\sum_{i=1}^N (y_i - \hat{y}_i)^2$$

and for categorical settings, cross-entropy loss can be applied.

• Prompt-to-Region Alignment: In learnable prompt settings, only prompt parameters and lightweight fusion layers are trained, ensuring that core spatial/semantic features remain stable and general.

This design provides robust and efficient adaptation while mitigating overfitting to prompt noise or unrelated semantics.

5. Empirical Performance and Ablation Findings

Empirical evidence across urban datasets demonstrates clear advantages of Prompt4RE relative to conventional baselines:

Task & Dataset	Baseline MAE	Prompt4RE MAE	Relative Improvement
Chicago Crime (ToPT)	61.7	49.3	20.1%
Chicago Check-in	1775	634.6	64.2%
NYC Crime (GURPP_T)	72.09	71.48	2.1%
CHI Crash (GURPP_T)	80.36	65.30	17.2%

• Removing Prompt4RE from the ToPT pipeline increased MAE on crime by 26.3%, with analogous drops in prediction performance on other tasks (Guo et al., 2 Feb 2026).
• Task-learnable prompts consistently yielded higher $R^2$ across both NYC and Chicago datasets (Jin et al., 2024).
• Ablations indicate that both semantically meaningful prompts and prompt-to-region alignment (via cross-attention or kernel fusion) are essential; random or unaligned prompts lead to double-digit percentage performance drops.
• Prompt4RE is model-agnostic: swapping LLM/MLLM backbone introduces negligible variance in downstream performance, reaffirming the importance of the prompt/attention alignment over backbone specifics (Guo et al., 2 Feb 2026).

6. Scientific Significance, Limitations, and Extensions

The significance of Prompt4RE lies in its ability to bridge the gap between generic, spatially-informed region embeddings and the explicit requirements of heterogeneous, evolving urban tasks. The approach is flexible: prompt mechanisms can be manual (domain-guided), learned (robust to unknown task structure), multimodal, or purely structural.

Limitations reported include:

• Granularity and Scalability: The extent of regional subgraph granularity or template detail can trade off computational expense versus representational detail (Jin et al., 2024).
• Prompt Generalization: While learnable prompts can generalize across related tasks or geographies, transfer between unrelated domains or very fine semantic tasks may require new prompt training or discovery.
• Adaptation to Streaming or Dynamic Environments: Existing static prompt approaches may not immediately adapt to streaming data or shifting task definitions.

Future directions propose:

• Integration of textual metadata and automatic generation of prompt content (including use of LLM-generated tags).
• Dynamic and continuous prompt learning in response to data streams (e.g., real-time mobility or environmental variables).
• Transfer and adaptation of prompt graphs or templates across spatial domains (e.g., urban to ecological networks), facilitating comparative analytics and robust transfer learning (Jin et al., 2024).

7. Relation to Region-Aware and Scene-Level Prompting

Prompt4RE for region embeddings complements other region prompting paradigms such as Region Prompt Tuning (RPT) for computer vision and scene text detection (Lin et al., 2024). While RPT operates at the level of spatial grid tiling of images—decomposing prompts at the character/token level and enforcing alignment via positional embeddings and bidirectional loss—Prompt4RE adapts these principles to the domain of graph/region representations, emphasizing task-specific semantic conditioning and cross-modal alignment for structured, spatial, and multimodal data. Both share the core principle: increased fine-grained, local alignment between prompt and regional/visual tokens substantially improves downstream discriminative performance.

---

Prompt4RE represents a technical advance in targeted representation learning across urban informatics and spatial data science, offering modular, interpretable, and empirically validated mechanisms for directing generic region embeddings toward specific policy, social, or infrastructural tasks (Jin et al., 2024, Guo et al., 2 Feb 2026).