Atlas-Based Prompts: Multi-Domain Applications

• Atlas-based prompts are structured guidance cues that integrate spatial, semantic, and syntactic priors from anatomical atlases or taxonomy templates to enable one-shot domain adaptation.
• In medical imaging, they generate geometric cues via deformable registration and label warping, markedly improving segmentation performance and Dice scores.
• For NLP, atlas-based prompts embed taxonomy-derived structural, semantic, and syntactic elements to enable clustering and principled prompt refinement, enhancing task outcomes.

Atlas-based prompts are structured guidance signals derived from anatomical atlases or systematic template collections, leveraged as input prompts for foundation models in both computer vision and natural language processing domains. These prompts encode spatial, semantic, or syntactic priors specific to a target context, facilitating domain adaptation, one-shot customization, and task robustness—especially in underrepresented or few-shot settings. In image segmentation, atlas-based prompts originate from deformable registration and label warping; in text prompting, they emerge from a taxonomy-driven embedding of prompts into a geometric “prompt space.” Frameworks such as AtlasSegFM, FeTal-SAM, and PromptPrism demonstrate the principles and impact of atlas-based prompting for high-fidelity, generalizable, and interpretable model behavior across diverse biomedical and NLP benchmarks (Zhang et al., 20 Dec 2025, Jeoung et al., 19 May 2025, Zeng et al., 22 Jan 2026).

1. Mathematical Formulation and Construction of Atlas-Based Prompts

In medical image segmentation, atlas-based prompts are generated by registering a pre-annotated atlas image and its associated label map to a target query image. Let $X_\mathrm{atlas}:\Omega\to\mathbb{R}$ and $Y_\mathrm{atlas}:\Omega\to\{0,1,\dots,C\}$ denote the atlas image and label, with the query image $X_q:\Omega\to\mathbb{R}$, where $\Omega\subset\mathbb{R}^3$ is the image domain. The registration seeks a smooth spatial deformation $\varphi:\Omega\to\Omega$ minimizing

$$T^* = \arg\min_\varphi L_\mathrm{reg}(\varphi; X_\mathrm{atlas}, X_q)$$

with

$$L_\mathrm{reg}(\varphi) = -\mathrm{Sim}(X_\mathrm{atlas}\circ\varphi, X_q) + \lambda R(\varphi),$$

where $\mathrm{Sim}(\cdot,\cdot)$ is a similarity metric (e.g., normalized cross-correlation, mean squared error) and $R(\varphi)$ is a regularity term (e.g., bending energy) (Zhang et al., 20 Dec 2025).

Upon optimization (using, e.g., a U-Net-based VoxelMorph registration network), the warped atlas mask $M_\mathrm{atlas}(u)=Y_\mathrm{atlas}(\varphi(u))$ becomes the geometric prompt. In multi-atlas pipelines, as in FeTal-SAM, several atlas-label pairs $(I_\mathrm{A}^{(k)}, Y_\mathrm{A}^{(k)})$ are rigidly and affinely registered, and their warped labels are processed to provide dense and bounding box prompts for segmentation models (Zeng et al., 22 Jan 2026).

In text prompting, "atlas-based" refers to the systematic embedding of prompts in a taxonomy-derived space. PromptPrism formalizes this by mapping each prompt $P$ into a composite vector $$\phi(P) = [\phi_\mathrm{struct}(P)\|\phi_\mathrm{sem}(P)\|\phi_\mathrm{syn}(P)]$$ that integrates structural (role-based), semantic (component annotation), and syntactic (delimiter, prefix, marker) axes, allowing for clustering and principled prompt comparison (Jeoung et al., 19 May 2025).

2. Prompt Modalities and Encoding Schemes

Atlas-based prompts for segmentation models are converted into standard input formats:

• Click prompt: The centroid of the largest connected component in $M_\mathrm{atlas}$ yields a spatial coordinate $p=\mathrm{centroid}(C_\mathrm{max})$.
• Box prompt: The bounding box around $C_\mathrm{max}$ is defined by $B=[u_\mathrm{min}, u_\mathrm{max}]$.
• Mask prompt: The mask $M_\mathrm{atlas}$ itself, as a dense prompt.

In multi-atlas vision systems such as FeTal-SAM, three registered atlases provide dense label feature maps $F_\mathrm{lab}^{(k)}$ via a U-Net-based encoder and image feature maps $F_\mathrm{img}^{(k)}$ from a ViT-b image encoder. These six feature maps are concatenated and processed with attention and channel-wise fusion mechanisms to produce the final prompt embedding used by the segmentation decoder. Bounding box prompts are averaged across atlases, then encoded by the model's prompt encoder as positional embeddings (Zeng et al., 22 Jan 2026).

PromptPrism adapts the notion of an “atlas” to the NLP domain by classifying prompts along three axes—functional structure (speaker/discourse roles), semantic components (instruction, context, output constraint, etc.), and syntactic patterns (delimiters, special tokens). This enables embedding, clustering, and refinement of prompts for LLMs (Jeoung et al., 19 May 2025).

3. Fusion of Atlas-Based and Foundation Model Outputs

Combining atlas-based priors and model predictions is critical for robust segmentation. AtlasSegFM employs a Kalman-inspired adaptive fusion: $$M_\mathrm{final}(u) = M_\mathrm{fm}(u) + K(u)[M_\mathrm{atlas}(u) - M_\mathrm{fm}(u)]$$ where $K(u)\in[0,1]$ is a spatial gain map learned at test time from the available one-shot support, using a lightweight 3D CNN applied to $[M_\mathrm{atlas}, M_\mathrm{fm}]$, and calibrated by minimizing the Dice loss to the support annotation. This mechanism favors the atlas in uncertain regions, leverages the model where confident, and adapts immediately to new contexts without offline retraining (Zhang et al., 20 Dec 2025).

In FeTal-SAM, 2D masks produced along multiple anatomical planes are fused into the final 3D segmentation via the STAPLE algorithm, which estimates per-voxel label probabilities from the ensemble of candidate segmentations, improving volumetric coherence (Zeng et al., 22 Jan 2026).

4. Implementation: Inference Workflow and Practical Usage

Typical workflows comprise:

• Registration: Rigid + affine alignment (and optionally deformable) of atlas images to the query; low-capacity networks or ANTs-based pipelines are common; per-case time is approximately 1.5 minutes for atlas-query pairs (Zhang et al., 20 Dec 2025).
• Prompt generation: Mask, box, and click extraction from warped atlas labels; feature extraction from registered images and masks (Zhang et al., 20 Dec 2025, Zeng et al., 22 Jan 2026).
• Model inference: Feeding structural prompts into foundation models such as nnInteractive, vesselFM, or Med-SAM; promptable inference in $< 1$s per volume for modern architectures (Zhang et al., 20 Dec 2025).
• Fusion/adaptation: Kalman fusion (AtlasSegFM), attention-fused dense prompts (FeTal-SAM), or taxonomy-guided prompt refinement/selection (PromptPrism) (Zhang et al., 20 Dec 2025, Zeng et al., 22 Jan 2026, Jeoung et al., 19 May 2025).
• Volumetric results: Synthesis across orientations or prompt clusters (e.g., by STAPLE).

Foundation-model pipelines such as AtlasSegFM and FeTal-SAM require no offline retraining for new anatomies; a single annotated support atlas suffices for clinical customization (Zhang et al., 20 Dec 2025, Zeng et al., 22 Jan 2026). PromptPrism provides corresponding toolkit conventions for the structured profiling and optimization of LLM prompts (Jeoung et al., 19 May 2025).

5. Quantitative Performance and Empirical Insights

Atlas-based prompts have demonstrated substantial gains, particularly on small or low-contrast anatomical targets:

Dataset / Structure	Baseline (Dice)	Atlas-based (Dice)	ΔDice
BrainRT organs-at-risk (Zhang et al., 20 Dec 2025)	39.09% (nnInteractive, 5 clicks)	77.07%	+37.98%
Fe-MRA fine vessels (Zhang et al., 20 Dec 2025)	60.31% (vesselFM)	84.42%	+24.11%
Fe-MRA clDice (Zhang et al., 20 Dec 2025)	41.74%	82.99%	+41.25%
Fetal brain MRI dHCP (Zeng et al., 22 Jan 2026)	0.252 (Med-SAM), 0.600 (FT)	0.882 (FeTal-SAM)	+28.2%, +12.6%
Fetal brain MRI CRL (Zeng et al., 22 Jan 2026)	0.393 (Med-SAM), 0.493 (FT)	0.801 (FeTal-SAM)	+40.8%, +30.8%

PromptPrism's atlas-based prompt refinement yields 21–137% generation task improvement on Super-NaturalInstructions via systematic addition of missing semantic components and syntactic normalization (Jeoung et al., 19 May 2025).

A common observation is that atlas-based approaches maintain model flexibility: segmentation of new or custom structures is enabled by supplying corresponding label maps, with no additional model update required. Performance remains robust for well-contrasted anatomy; small, low-contrast structures are limited by registration accuracy and 2D inference constraints (Zeng et al., 22 Jan 2026).

6. Applications, Limitations, and Best Practices

Atlas-based prompt methodologies underpin one-shot customization of foundation segmentation models for rare, delicate, or bespoke anatomical targets, permitting rapid clinical adaptation without retraining. In text domains, embedding prompts in “atlas space” enables principled prompt design, dataset profiling, multi-axis sensitivity analyses, and clustering for robustness and diversity (Jeoung et al., 19 May 2025).

Limitations stem from (1) registration errors—particularly under-prompting when all atlases miss a small structure; (2) reduced performance for subtle, low-contrast regions that lack strong spatial or imaging cues; and (3) slice-by-slice 2D inference failing to fully restore 3D coherence in volumetric segmentations (Zhang et al., 20 Dec 2025, Zeng et al., 22 Jan 2026). Manual intervention—e.g., direct box/click prompts or nonrigid alignment improvements—may be necessary.

Best practices include standardizing prompt component structure, auditing semantic completeness, clustering prompt templates for coverage, and running multi-axis perturbation tests (semantic/syntactic) prior to deployment (Jeoung et al., 19 May 2025). Integrating atlas priors with model-driven refinement consistently outperforms naïve prompting, especially in scenarios characterized by limited training representation or clinical heterogeneity.