SWE-Spot-4B: Repository-Centric Model

• SWE-Spot-4B is a repository-centric language model that internalizes a codebase's unique architecture and behavior for efficient automation.
• It employs a four-unit curriculum—spanning design, implementation, evolutionary replay, and semantic-runtime alignment—to enhance multi-task performance.
• Empirical evaluations show SWE-Spot-4B outperforms larger models in issue resolution and test generation while achieving high sample and token efficiency.

SWE-Spot-4B is a 4-billion-parameter repository-specialized LLM introduced as part of the SWE-Spot family, designed to realize repository-centric learning (RCL) for efficient, high-fidelity software engineering (SWE) task automation. It exemplifies a paradigm shift from conventional Task-Centric Learning (TCL) to RCL, prioritizing deep parametric internalization of a target codebase's structural and behavioral "physics" over broad, cross-codebase task generalization. SWE-Spot-4B achieves state-of-the-art sample efficiency and competitive performance-to-cost ratios on core repository-level tasks, matching or outperforming much larger open-weight and commercial efficiency-focused models (Peng et al., 29 Jan 2026).

1. Conceptual Foundation: Repository-Centric vs. Task-Centric Learning

RCL targets deep vertical mastery within a single codebase or a small set of related repositories. In contrast to TCL—which samples across many repositories for a particular task T, thereby cultivating surface-level, generalizable skills ($p(a \mid s)$)—RCL intensifies interaction density within a given repository R*, driving the model to absorb its unique architecture, dependency structure, and runtime conventions. The critical theoretical premise is that by saturating the training process with dense, multi-modal agentic trajectories from R*, small LLMs can internalize domain-specific priors that are inaccessible to TCL models, particularly when inference-time retrieval and search cost is constrained (Peng et al., 29 Jan 2026).

The formal objective for RCL is to maximize the log-likelihood of action sequences within expert trajectories $\tau$ sampled from the repository-specific experience set $\mathcal{E}_R$: $$\theta^* = \arg\max_{\theta} \; \mathbb{E}_{\tau \sim \mathcal{E}_R}\left[ \sum_{t=0}^{T-1} \log p_{\theta}(a_t \mid s_t) \right],$$ with $\theta$ parameterizing the model and $(s_t, a_t)$ denoting state-action pairs. When empirically distributed across multiple "experience units", this becomes a weighted multi-task loss (Peng et al., 29 Jan 2026).

2. Four-Unit Repository-Centric Curriculum Design

The Repository-Centric Experience (RCX) decomposes repository mastery into four interacting units, each engineered to elicit distinct forms of software expertise:

1. Software Design (Unit 1): Agentic exploration yields structured design reports on modules—mapping responsibilities, call graphs, and rationale—thus encoding software architecture reasoning.
2. Contextual Implementation (Unit 2): Fill-in-the-middle (FIM) tasks are built such that the agent must synthesize functionality requiring cross-file/static symbol resolution and correct API usage, compelling deep integration with global codebase structure.
3. Evolutionary Replay (Unit 3): Synthetic undo/redo of real historical pull requests injects sequences of bug introduction and repair, capturing evolutionary pressure, debugging routines, and historical workflow.
4. Semantic-Runtime Alignment (Unit 4): Reproduction of tests for real bugs forces grounding of specifications in runtime execution, bridging natural-language intent and concrete code behavior.

Each unit transforms static repository artifacts into interactive, multi-turn trajectories, producing a dense, multi-task training curriculum. Empirical studies demonstrate that ablating any single unit (e.g., omitting evolutionary replay) yields significant multi-task performance drops, confirming the necessity of all four forms of signal (Peng et al., 29 Jan 2026).

3. Architecture, Data, and Fine-Tuning Protocols

SWE-Spot-4B is based on the Qwen3-4B-Instruct-2507 decoder-only Transformer, consisting of approximately 32 layers, model dimension ≈4096, 32 attention heads, and a 48k context window. The RCL fine-tuning protocol proceeds as follows:

• Data Synthesis: For each codebase, 8,000 RCX trajectories (≈2,000 per unit) are synthesized by a strong teacher model (Gemini-2.5-Pro), with train/test splits established by commit date (only pre-2021 data for training).
• Supervised Fine-Tuning (SFT): Models are trained for two epochs at batch size 16, max-sequence length 32,768, using AdamW and cosine learning rate decay ($10^{-5} \to 10^{-6}$), with a brief linear warmup. Inference uses temperature=1.0, top-p=0.8, top-k=20, repetition penalty=1.05.
• Resource Footprint: Training utilizes ms-swift + Megatron on two H200 GPUs (Peng et al., 29 Jan 2026).

4. Empirical Results: Performance, Efficiency, and Sample Efficiency

SWE-Spot-4B establishes new benchmarks for compact, repository-specialized models. Results from comprehensive repository-centric evaluation (RCE) benchmarks include issue resolution (SWE-Bench-Verified), test generation (TDD-Bench-Verified), feature implementation (FEA-Bench), and codebase QA (SWE-QA):

Model	Size	Issue %	Test %	Feat %	Exec Avg %	QA score
GPT-4.1-mini	–	21.79	22.27	5.70	17.85	80.28
Qwen3-Coder-30B	32 B	16.74	11.85	3.29	11.56	65.48
CWM (TCL)	32 B	22.22	17.38	4.17	15.88	73.09
Mini-Coder-4B (TCL)	4 B	18.76	0.63	4.61	8.70	57.30
SWE-Spot-4B (RCL)	4 B	19.34	22.75	5.92	17.12	78.05

Notably, SWE-Spot-4B matches or outperforms larger 32B parameter open-weight models (Qwen3-Coder-30B, CWM) and approaches GPT-4.1-mini's efficiency, with robust multitask capability across issue fixing, test generation, feature implementation, and QA (Peng et al., 29 Jan 2026).

Sample Efficiency: Under equal data budgets, RCL-trained SWE-Spot-4B achieves 19.34% issue resolution with 8k samples vs. 14.86% for the TCL baseline, and 11.01% vs. 4.09% on test generation with 2k samples.

Dialogue and Token Efficiency: Mean turns and tokens per inference are notably lower for RCL (32.06 turns, 29.49k tokens) versus TCL (41.62 turns, 33.19k tokens), indicating shorter, more direct agentic interactions to solution under RCL (Peng et al., 29 Jan 2026).

5. Nature of Learned Knowledge and Transfer Analysis

RCL-trained SWE-Spot-4B exhibits genuine parametric mastery of repository-specific conventions and architectural "physics," not mere surface memorization. Oracle localization (providing perfect file/function hints) does not significantly improve RCL's performance, in contrast to TCL, indicating internalized global priors. LoRA adaptation on RCL is insufficient for full repository knowledge; only full-weight fine-tuning achieves peak performance. Cross-task transfer is strong—improvements in test generation yield commensurate gains in issue fixing, and design reasoning supports QA (Peng et al., 29 Jan 2026).

Inter-repo effects: Joint RCL on multiple repositories can raise pass rates on certain targets (synergy) but diminish others (interference), suggesting that unconstrained scaling of repository diversity is not necessarily optimal—a finding unique to the repository-centric paradigm.

6. Implementation, Limitations, and Best Practices

Production of SWE-Spot-4B repo-experts is feasible in privacy-sensitive, on-premise scenarios:

• Data collection: ~8,000 RCX trajectories per repo.
• Full-weight fine-tuning is required; LoRA is inadequate.
• Long context and dynamic agentic prompting are necessary.
• For cost sensitivity, prioritize the most informative RCX units (e.g., Evolutionary Replay, Contextual Implementation).
• Periodic re-training on new commits supports continual learning.

Limitations include reliance on SFT over teacher data (rather than RL-based policy optimization), open questions around inter-repo transfer, and the overhead of full-weight fine-tuning, motivating future research into improved continual learning and more efficient parametric adaptation (Peng et al., 29 Jan 2026).

7. Significance and Broader Impact

SWE-Spot-4B demonstrates that repository-centric learning enables small models to break the scaling laws observed for task-centric approaches. By emphasizing vertical depth and agentic curriculum construction, SWE-Spot-4B affords privacy-compliant, resource-efficient, and high-fidelity repo-expert agents. This model design is particularly well-suited for enterprise and regulated domains, where both parametric adaptation and inference cost constraints are paramount, and it complements rather than replaces broad-coverage, retrieval-augmented LLMs (Peng et al., 29 Jan 2026).