CodeCytos: AI-assisted spatial molecular imaging analysis via code-augmented agent action space

Abstract: Conventional tissue image analysis software provides foundational capabilities for cellular analysis, including segmentation, basic morphological feature extraction, and spatial organization analysis. However, these tools often require manual intervention and are not well integrated with code-driven automation, limiting efficiency and scalability for complex spatial tissue studies. In addition, they offer limited flexibility for custom analyses, as they typically support only a fixed set of pre-implemented spatial cellular features. To address these limitations, we propose CodeCytos, a coding-based reasoning agent framework that enables dynamic, programmable interaction with spatial molecular imaging data to improve automation and customization. CodeCytos is designed to streamline the exploration of custom spatial cellular features and adapt to diverse research needs. We demonstrate its utility through case studies on four expert-curated datasets from distinct tissue types: frontal cortex, non-small-cell lung cancer, pancreas, and tonsil. We evaluate CodeCytos under a realistic minimal prompt setting, where bioscientists pose simple questions without task-specific instructions or contextual information about spatial cellular analysis, and benchmark multiple LLM backbones with strong coding capabilities. We further show that incorporating tailored, domain-agnostic few-shot in-context coding-reasoning examples (randomly sampled demonstrations outside the spatial analysis domain) can substantially improve performance without requiring costly, expert-crafted in-domain demonstrations. Overall, CodeCytos outperforms baseline approaches, highlighting the potential of code-action agents to assist with custom feature exploration in spatial molecular imaging and to accelerate biomarker discovery.

• The paper presents a novel code-action LLM agent framework that leverages few-shot coding-reasoning exemplars to enhance spatial molecular imaging analysis.
• It unifies natural language reasoning with Python code generation, enabling dynamic execution and self-correction in segmentation, classification, and spatial queries.
• Quantitative evaluations across multi-tissue benchmarks demonstrate improved success rates and AUP@k compared to traditional tool-augmented LLM approaches.

CodeCytos: Code-Action LLM Agents for Custom Spatial Molecular Imaging

Motivation and System Overview

Conventional spatial molecular imaging pipelines constrain bioscientific analysis due to rigid toolchains, limited customizability, and the dependence on fixed feature sets for cell segmentation, classification, and spatial statistics. CodeCytos proposes a code-action agentic approach that leverages LLMs' programming priors to facilitate dynamic, flexible interaction with spatial omics data. The framework extends the ReAct paradigm with executable code actions via CodeAct, enabling end-to-end reasoning, code synthesis, execution, and environment feedback in multi-turn agent loops.

The agent's action space is unified around Python code generation, allowing arbitrary invocation of segmentation/classification tools and the composition of bespoke spatial queries. This approach eliminates dependence on extensive prompt-engineering or in-domain few-shot demonstrations, and instead exploits general-purpose code-reasoning exemplars to optimize open-source coding LLMs for these tasks.

Figure 1: The CodeCytos workflow and benchmarking overview; usage scenario for bioscientists and radar chart comparison between CodeCytos and tool-augmented LLMs.

Framework Architecture and Methodology

At the core, CodeCytos unifies reasoning (natural language “thought” actions) and acting (Python code emission) in multi-turn trajectories, implemented as iterative cycles: the agent reasons about the next step, generates executable code, observes output or runtime errors, and self-corrects as necessary. Agent context includes user queries, selected domain tools, cellular images, and a full trace of actions and observations. This results in robust, environment-driven self-improvement.

The architecture supports a variety of segmentation/classification backends, ensuring extensibility to new modalities. CodeCytos' open-tool philosophy allows seamless integration with both general-purpose and domain-specific Python packages.

Figure 2: High-level schematic of the CodeCytos agent, integrating multi-turn reasoning, code generation, and observation-driven iteration.

Dataset Construction and Evaluation Protocols

Evaluation is based on an expert-curated, multi-tissue benchmark (Frontal Cortex, NSCLC, Pancreas, Tonsil), with 548 natural-language spatial queries per tissue, paired with 20 field-of-view images, yielding 10,960 total sample points. Queries target a broad taxonomy: NNDs, local neighborhoods, spatial statistics, structure/ROI, and graph topology, with the system evaluated under strict minimal-prompt settings to assess agent autonomy and robustness.

Metrics reported include success rate, pass@k (for $k=5,10,20$), and the summary area under the pass@k curve (AUP@k). Comparisons are made with tool-augmented baselines employing both zero-shot and few-shot chain-of-thought (CoT) strategies using multiple strong coding LLMs: Qwen2.5-Coder, Qwen3-Coder, Facebook-CWM, Devstral-2-123B, Devstral-Small, Kimi-Linear, and GLM-4.5-Air.

Quantitative Results and Category-Wise Analysis

CodeCytos with few-shot, domain-agnostic coding-reasoning exemplars exhibits dominant performance, consistently yielding higher success rates, pass@k, and AUP@k across all tissue types compared to baseline tool-augmented LLMs, even when the exemplars are unrelated to spatial analysis.

Key findings:

• On all four datasets, coding-optimized LLMs (especially Devstral-2-123B and GLM-4.5-Air) show strong performance, but the integration of general code-reasoning exemplars is critical for open-source backbone reliability.
• Performance gains are most prominent in nearest neighbor distance and neighborhood composition queries, while structure/ROI and graph-based features present higher variance across backbones and datasets.
• For most LLMs, domain-agnostic few-shot exemplars produce major improvements in both AUP@k and overall robustness, bridging gaps between open-source and proprietary models.

  Figure 3: AUP@k by category (Frontal Cortex) across tool-augmented and CodeCytos agent variants, showing the consistent boost from few-shot CodeCytos.

  

  Figure 4: AUP@k by category (NSCLC); similar trends with substantial CodeCytos gains.

  

  Figure 5: AUP@k by category (Pancreas), further illustrating the value of the CodeCytos few-shot methodology.

  

  Figure 6: AUP@k by category (Tonsil); CodeCytos leads across feature classes and LLMs.

  

  Figure 7: Performance heatmaps over feature categories and LLM backbones; NNDs are consistently tractable, with coding-centric backbones achieving the highest values.

Pass@k curves plateau by $k \sim 10$, indicating diminishing returns from brute-force sampling and underscoring intrinsic model limitations and the need for improved planning/self-correction.

Figure 8: Frontal Cortex dataset sample; underlying spatial complexity is captured in agent interaction.

Agentic Reasoning and Qualitative Examples

Qualitative case studies demonstrate CodeCytos' ability to autonomously decompose queries, generate relevant code, interpret ambiguous or missing contextual information (e.g., unit inference), and utilize efficient scientific Python primitives (e.g., cdist for distance matrix calculation, networkx for graph analytics). In all cases, code-generation and debugging are transparent, allowing bioscientists or computational experts to validate or augment intermediate outputs.

Figure 9: An NSCLC example where the agent infers missing scale information during VMR computation.

Figure 10: Efficient nearest-neighbor spatial analysis; CodeCytos leverages vectorized computational strategies.

Figure 11: Delaunay graph feature extraction among astrocytes, fully instantiated from a natural language query.

Figure 12: MST-based mean edge length computation with correct graph construction.

Figure 13: Hausdorff distance computation between epithelial and PSC polygons, with correct type inference.

Generalization Beyond Spatial Analysis: M³ToolEval

Ablation on the M³ToolEval multi-tool agent benchmark confirms that the benefit from few-shot, domain-agnostic coding-reasoning exemplars extends to general agentic tool-use scenarios—though the performance gain is more pronounced for complex coding/reasoning than for simpler tool orchestration tasks. CodeCytos' CodeAct-few-shot approach sets new best results on this benchmark for multiple open-source LLMs.

Figure 14: The impact of few-shot coding-reasoning exemplars on M³ToolEval compared to other agent action modes.

Implications and Future Directions

CodeCytos demonstrates that code-action LLM agents—augmented by structured, domain-agnostic, few-shot coding-reasoning demonstrations—can effectively replace or augment bespoke programming and tedious feature engineering in spatial molecular imaging. This directly addresses longstanding friction in multi-modal bioimage analysis workflows, providing bioscientists with a bridge between natural-language hypothesis formulation and precise, programmatically implemented spatial analyses.

From a theoretical perspective, the unified code-action space harnesses the pretrained programming priors in LLMs for dynamic, environment-responsive planning and execution, minimizing dependence on hand-crafted domain knowledge. The system's architecture supports extensibility to new analysis types, imaging modalities, and scientific domains requiring custom tool composition and iterative reasoning.

Opportunities for further development include:

• Enhanced reliability through agentic self-verification, error recovery, and failure-mode targeted fine-tuning.
• Integration with multimodal LLMs for direct image feature extraction, closing the loop between cellular image input and high-level biological insight.
• Extension to reinforcement learning training approaches to further optimize planning, debugging, and resource selection policies.
• Broader benchmarking across additional spatial omics and pathology modalities to validate generalizability.

Conclusion

CodeCytos establishes a robust paradigm for code-driven agentic analysis in spatial molecular imaging, leveraging few-shot code-reasoning exemplars to maximize both customization and automation. The framework provides a scalable, backbone-agnostic solution to the critical challenge of connecting bioscientific experts with the full analytical depth of modern coding LLMs, with far-reaching implications for computational biology and beyond.