Self-Enhanced Reasoning Training (SERT)

Updated 4 February 2026

SERT is a technique that surfaces latent multi-step reasoning by mining self-generated logical paths using rule-based filtering and self-training.
It employs a multi-stage pipeline including latent path mining, rationale filtering, and teacher-based distillation to enhance performance.
Empirical results demonstrate that SERT improves accuracy on reasoning benchmarks and reduces repetition in generated outputs.

Self-Enhanced Reasoning Training (SERT) refers to a family of techniques designed to activate, mine, and leverage the latent multi-step reasoning capabilities present but rarely surfaced in LLMs—especially smaller models—by exploiting the models’ own self-generated reasoning traces. By directly training LLMs on their own filtered, self-produced high-quality reasoning paths, SERT enables models to internalize step-wise deduction and amplify the benefits of subsequent teacher-based reasoning distillation. The approach generalizes across architectural scales and learning settings, with empirical gains observed in both zero-shot and fine-tuning regimes for tasks demanding logical or commonsense reasoning.

1. Motivation and Theoretical Foundations

LLMs such as GPT-3.5 manifest strong chain-of-thought (CoT) reasoning, but their high computational costs restrict widespread deployment. Smaller models (e.g., GPT-2) offer efficiency but tend to default to direct answer production rather than multi-step rationalization, particularly in zero-shot settings; this phenomenon is attributed to the low probability mass assigned to step-wise rationales in standard decoding. However, raw stochastic sampling reveals that small models do occasionally generate high-quality, logically coherent reasoning paths—here termed latent reasoning—even in the absence of explicit CoT prompting. These paths are virtually absent under deterministic decoding ( $p(\text{rationale}|q)\ll p(\text{direct answer}|q)$ ) due to their low likelihood.

SERT formalizes the objective of mining this “hidden” reasoning potential. The core insight is that self-training on such rare, self-generated multi-step paths (when appropriately filtered) can shift the model’s output distribution, making explicit reasoning more probable and coherent. SERT thereby bridges the performance gap between black-box reasoning distillation (focusing on teacher-provided outputs) and a model’s own reasoning capabilities (Zhang et al., 18 Feb 2025, Chen et al., 2024).

2. Methodological Framework

SERT is instantiated as a multi-stage self-training and distillation pipeline, exemplified in the setting where a smaller student model is trained to improve reasoning using both its own mined rationales and high-quality chains from a large teacher. The canonical SERT pipeline involves:

Latent Path Mining. For each question–answer (QA) pair, the model is prompted in a zero-shot format (e.g., "Question: q Answer:") and the first decoding step is expanded into the top $K$ alternative tokens ( $K=5$ ), yielding $K$ diverse prompt continuations. For each, $M$ completions ( $M=5$ ) are sampled with nucleus (top-p=0.95) and top-k=10 sampling, producing $K\cdot M$ candidate reasoned completions per question.
Filtering. Candidate rationales are subject to rule-based filters:
- Pattern rejection: Discard outputs that are mere answer stubs or that mimic input format.
- Minimum length: Require at least $L_{min}=25$ tokens.
- Repetition control: Limit bigram repetition to $rep_2(r)\leq0.20$ .
- Perplexity gate: Exclude high-probability answer-like completions by enforcing $ppl(r)\geq\theta_{min}=5$ .
Rationale selection. Among filtered candidates, the one with minimal repetition is retained as the reasoning path $\hat{r}_i$ .
Self-Training Dataset Construction. For each QA pair, a new prompt incorporating the mined rationale is constructed: “Question: $q_i$ Answer: $\hat{r}_i$ So the answer is $a_i$ .”
Model Training. The student model is fine-tuned on these augmented pairs via a cross-entropy loss over the reasoning tokens ( $L_{\text{SERT}}$ ), followed by standard reasoning distillation ( $L_{\text{KD}}$ ) from the teacher. The total loss is $L = L_{\text{SERT}} + \lambda L_{\text{KD}}$ (with $\lambda$ typically set to $1$ for sequential training).

A typical pseudocode representation of the SERT pipeline is as follows (excerpted for technical detail) (Zhang et al., 18 Feb 2025):

for (q, a) in QA_data:
    sampled_rationales = []
    for t in TopKTokens(model, "Question: q Answer:", K=5):
        completions = Sample(model, "Question: q Answer: "+t, M=5)
        sampled_rationales.extend(completions)
    filtered = [r for r in sampled_rationales if pass_filters(r)]
    if filtered:
        r_hat = min(filtered, key=lambda r: bigram_repetition(r))
        AddToDataset(q, a, r_hat)
Train(model, SERT_dataset, L_SERT)
Distill(model, teacher, L_KD)

3. Core Concepts: Latent Reasoning and Self-Rewarding Objectives

The central mechanism of SERT is the explicit mining and reinforcement of latent reasoning paths, which are multi-step logical explanations generated by the model but typically assigned low output probability. This concept parallels recent developments in latent variable modeling for reasoning, such as LaTRO (Chen et al., 2024), where the rationale $z$ is formally treated as a latent variable sampled from $q(z|x)$ and optimized via an ELBO objective:

$\log \pi_\theta(y|x) \geq E_{z \sim q(z|x)}[\log \pi_\theta(y|x \oplus z)] - \beta\,\mathrm{KL}[q(z|x)\|\pi_0(z|x)]$

SERT adopts a procedural (rule-based) filter to realize $q(z|x)$ , while LaTRO introduces “self-rewarding” via an intrinsic reward shaped by model likelihoods, further regularized by divergence from a frozen prior $\pi_0$ .

This line of work demonstrates that pre-trained LLMs possess a reservoir of implicit reasoning steps, which can be surfaced and systematically reinforced via self-guided or variational principles—without external annotations.

4. Empirical Performance

Empirical results demonstrate consistent, statistically significant gains on reasoning-centric tasks when adopting SERT. On StrategyQA (binary QA) and CommonsenseQA (5-way classification), SERT applied to GPT-2-large (774M) yields:

Model	StrategyQA Acc (%)	CommonsenseQA Acc (%)
Finetune	53.57	20.88
SERT	55.75	22.63
RD	50.22	22.93
SERT + RD	57.21	26.03

Key outcomes:

SERT alone delivers a $\sim2$ percentage-point gain on StrategyQA and $\sim1.8$ on CommonsenseQA relative to supervised fine-tuning.
Combining SERT with reasoning distillation (SERT+RD) amplifies the gain to $\sim3.1$ pp on CommonsenseQA over direct distillation.
Output-format fidelity is high (format alignment $\sim1.00$ ), and repetition is suppressed.
Gains are statistically significant at $p<0.01$ by paired bootstrap.

Ablation studies indicate that the sequential application of filtering steps, especially perplexity gating, shifts the distribution of reasoning-quality scores of selected paths from low to high (measured on a $0$–$10$ scale) (Zhang et al., 18 Feb 2025).

SERT provides a conceptual and practical template for broader self-enhanced reasoning approaches. Examples include:

Latent Reasoning Optimization (LaTRO) (Chen et al., 2024): Elevates SERT’s intuition to a differentiable variational latent-variable framework for both rational path generation and answer selection, using policy gradients and self-rewarding, with gains of $+12.5\%$ zero-shot accuracy over strong baselines on GSM8K, and $+9.6\%$ over supervised fine-tuning.
Reasoning-Enhanced Self-Training for Personalized Generation (REST-PG) (Salemi et al., 7 Jan 2025): Applies SERT-style two-stage self-training—with explicit reasoning path generation and EM-style reinforcement—to personalized long-form generation, achieving average relative gains of $14.5\%$ over plain SFT on LongLaMP.
Chain of Self-Correction (CoSC) (Gao et al., 2024): Embeds self-verification and correction loops into LLMs, iteratively generating code-based solutions, verifying outputs, and self-correcting, showing strong gains on mathematical benchmarks and confirming that two-stage self-enhancement pipelines are effective beyond direct QA.

These frameworks reflect a convergent research direction in leveraging model-internal reasoning traces for robust, domain-agnostic capability enhancement.

6. Limitations and Future Directions

SERT is subject to several limitations:

Sampling Overhead: Mining latent reasoning paths is computationally intensive, requiring $K\cdot M$ samples per QA.
Rule-Based Filtering: Current filters are handcrafted; there is scope for learned selectors or reward-based approaches to improve recall and precision.
Domain Specificity: Demonstrated gains are within controlled domains (e.g., commonsense QA); generalization to new domains or cross-task transfer is unstudied.
Infrastructure: Scaling SERT to very large models or to instruction-tuned variants may require algorithmic adaptations.
Exploration–Exploitation Tradeoff: Excessive sampling may fail to discover unseen reasoning modes, while insufficient exploration risks convergence to repeated patterns.

Proposed directions include adaptive sampling budgets, integrating classifier-based (learned) filters, iterative refinement of rationales, and cross-domain generalization studies.

7. Conclusion

Self-Enhanced Reasoning Training (SERT) constitutes a general, empirically validated paradigm for surfacing, reinforcing, and leveraging latent reasoning paths in LLMs. By self-training on rare but high-quality multi-step explanations generated by the model itself, SERT establishes a more reasoning-favorable output distribution in small or mid-sized models, facilitating subsequent teacher-based distillation and yielding improved accuracy, output stability, and reasoning quality. The approach generalizes to diverse settings, with instantiations ranging from latent variable optimization to self-correction and reward-aligned reasoning in both generative and mathematical tasks (Zhang et al., 18 Feb 2025, Chen et al., 2024, Salemi et al., 7 Jan 2025, Gao et al., 2024).