Rethinking Reasoning in LLMs: Neuro-Symbolic Local RetoMaton Beyond ICL and CoT
Abstract: Prompt-based reasoning strategies such as Chain-of-Thought (CoT) and In-Context Learning (ICL) have become widely used for eliciting reasoning capabilities in LLMs. However, these methods rely on fragile, implicit mechanisms often yielding inconsistent outputs across seeds, formats, or minor prompt variations making them fundamentally unreliable for tasks requiring stable, interpretable reasoning. In contrast, automata-based neuro-symbolic frameworks like RetoMaton offer a more structured and trustworthy alternative by grounding retrieval in symbolic memory with deterministic transitions. In this work, we extend RetoMaton by replacing its global datastore with a local, task-adaptive Weighted Finite Automaton (WFA), constructed directly from external domain corpora. This local automaton structure promotes robust, context-aware retrieval while preserving symbolic traceability and low inference overhead. Unlike prompting, which entangles context and memory in opaque ways, our approach leverages the explicit structure of WFAs to provide verifiable and modular retrieval behavior, making it better suited for domain transfer and interoperability. We evaluate this local RetoMaton variant on two pretrained LLMs LLaMA-3.2-1B and Gemma-3-1B-PT across three reasoning tasks: TriviaQA (reading comprehension), GSM8K (multi-step math), and MMLU (domain knowledge). Compared to the base model and prompting-based methods, augmenting these setups with local RetoMaton consistently improves performance while enabling transparent and reproducible retrieval dynamics. Our results highlight a promising shift toward trustworthy, symbolic reasoning in modern LLMs via lightweight, automaton-guided memory.
Summary
- The paper introduces a novel neuro-symbolic framework that integrates symbolic memory retrieval with LLM internal representations.
- The approach uses Weighted Finite Automata for structured, task-specific context retrieval, enhancing reasoning efficiency and interpretability.
- Experimental results show performance gains of 4.48% with LLaMA and 2.78% with Gemma, indicating robust improvements over traditional prompt strategies.
Neuro-Symbolic Local RetoMaton: A Structured Approach to Reasoning in LLMs
Introduction
The paper presents the Neuro-Symbolic Local RetoMaton framework, aimed at enhancing reasoning within LLMs by integrating symbolic processing with neural architectures. The framework proposes a novel approach by moving beyond prompt-based methods like chain-of-thought (CoT) and in-context learning (ICL), which struggle with stability and interpretability across variations. Neuro-symbolic strategies promise structured and interpretable reasoning capabilities, which are crucial for tasks demanding stable logical coherence.
Figure 1: Overview of the Local RetoMaton framework. The system combines a LLM, a symbolic datastore of hidden states and next-token labels, and a transition-structured automaton formed by clustering latent states.
Neuro-Symbolic Framework
The Local RetoMaton is built upon the concept of an automaton-guided retrieval mechanism, leveraging Weighted Finite Automata (WFA) constructed from external domain corpora. This method enhances retrieval precision and context-awareness, ensuring that memory access is structured and transparent. Compared to the traditional prompt strategies, which lack an explicit and modular retrieval structure, this approach efficiently organizes memory paths, favoring domain transfer and adaptability without relying on model fine-tuning.
In the Local RetoMaton, the token prediction process combines symbolic memory with the model's internal representations in an adaptive manner. The setup entails constructing WFAs around clusters of latent states derived from task-specific data, thereby reducing inference overhead while ensuring symbolic traceability.
Experimental Evaluation
The framework was evaluated using pre-trained LLMs, specifically LLaMA-3.2-1B and Gemma-3-1B-PT, across three reasoning tasks: TriviaQA for reading comprehension, GSM8K for multi-step mathematical problem-solving, and MMLU for domain knowledge. The experiments show that the Local RetoMaton variant consistently outperforms baseline models and CoT prompting techniques, offering improved reasoning efficiency and robustness due to its structured retrieval mechanism.
Figure 2: GSM8K Input Format Used for Setting Up the Local RetoMaton with both LLaMa and Gemma models.
The datastore construction focuses on aligning retrieval candidates with the task-specific context, which enhances precision by selecting pertinent contexts over a local neighborhood. Notably, the Local RetoMaton variant achieves performance gains of 4.48% with the LLaMa model and 2.78% with Gemma on the tested NLP tasks, demonstrating its efficacy.
Implications and Future Directions
The Neuro-Symbolic Local RetoMaton framework significantly contributes to the field by providing a trustworthy and actionable reasoning process. Its architecture-agnostic nature allows seamless integration with various LLMs, promoting efficient generalization beyond the training data. This approach successfully bridges the gap between neural and symbolic reasoning, offering promising paths for the development of next-generation AI systems.
Future research directions involve exploring the integration of larger LLMs with neuro-symbolic components, expanding the framework’s application to other NLP tasks, and investigating adaptive mechanisms that dynamically alter symbolic constraints during inference.
Figure 3: Distribution of file sizes for TriviaQA’s query-specific datastores, showing that they are significantly more lightweight than global or domain-aligned indexes.
Conclusion
The Local RetoMaton framework exemplifies a shift towards enhancing LLMs with neuro-symbolic methods, enabling structured and interpretable reasoning. By leveraging task-specific symbolic structures and maintaining memory constraints through WFAs, LLMs can achieve better reasoning performance while providing interpretable and transparent outputs. These advances underscore the potential of neuro-symbolic AI in creating intelligent systems with adaptable, explainable, and reliable capabilities.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Explain it Like I'm 14
Overview
This paper looks at a better way to help LLMs—the kinds of AI that write and answer questions—think more clearly and consistently. Instead of relying only on “prompt tricks” like Chain-of-Thought (CoT) and In-Context Learning (ICL), the authors add a small, smart memory system called a Local RetoMaton. This memory is built from the most relevant information for the task and guides the model step-by-step, like following a map, so its reasoning is more stable and easier to understand.
What questions does the paper ask?
The paper focuses on three simple questions:
- Can we make LLMs reason more reliably than just using prompts like CoT and ICL?
- Can we give LLMs a small, organized memory that helps them pick the right information at the right time?
- Will this memory make models more accurate on tasks like reading comprehension, math problems, and general knowledge?
How does it work? (Simple explanation of the method)
Think of the LLM as a student taking a test:
- With CoT and ICL, you’re giving the student hints in the question itself. That can work, but it’s fragile—small changes in the hints can confuse the student.
- The Local RetoMaton is like giving the student a tiny, well-organized notebook made only from helpful notes for the current subject. The student looks up examples and follows a clear path of steps (like a recipe) to find the correct answer.
Here’s what the Local RetoMaton is made of:
- A “symbolic memory” built from text that’s directly related to the task. For example, for trivia questions, it uses the evidence documents linked to those questions; for math problems, it uses math-related text.
- An automaton (a kind of simple machine or roadmap). Picture intersections (states) connected by roads (transitions). Each road has a “weight” (how useful it is), and the model follows the best path through this map to find the next word or step.
- A nearest-neighbor lookup (like finding the most similar notes in your notebook). The model checks which past examples are closest to what it’s currently trying to do and uses them to guide its next move.
Key idea: The model’s own internal signals (its “embeddings,” like coordinates on a map) are clustered into states. These states and their connections form a small, task-specific roadmap. As the model generates text, it follows this roadmap, making its reasoning traceable and reproducible.
What did they test?
They tried this on two compact LLMs (both around 1 billion parameters):
- LLaMA-3.2-1B
- Gemma-3-1B-PT
And on three types of tasks:
- TriviaQA: Reading comprehension (answering trivia questions based on evidence)
- GSM8K: Multi-step grade-school math problems
- MMLU: General knowledge questions across many subjects
They compared:
- The base model with normal prompting (ICL/CoT)
- A Global RetoMaton (memory built from a broad source like Wikipedia)
- A Domain-Aligned RetoMaton (memory tuned to the topic, like math text for math tasks)
- The Local RetoMaton (memory built from the exact task data or evidence)
Main findings and why they matter
- The Local RetoMaton consistently improved accuracy over just prompting and over the global/domain versions.
- Average gains: about 4.48% with LLaMA and 2.78% with Gemma across the three tasks.
- It made the model’s reasoning more transparent. You can trace which “paths” in the memory the model followed to produce each step, making it easier to understand and debug.
- It was more robust. Small changes to prompts didn’t break the reasoning as easily because the model was guided by a stable, structured memory.
- It was efficient. Since the local memory is smaller and focused, it’s faster to use and doesn’t require fine-tuning the model’s weights.
- It reduced “noise” in retrieval. Using nearby, relevant notes (local memory) led to better-calibrated predictions than searching everything (global memory).
In short: The Local RetoMaton helps LLMs be more accurate and more trustworthy, especially on tasks that need careful, step-by-step reasoning.
What’s the impact?
- More reliable AI: Models can reason in a way that’s easier to verify, which is important for math, science, and fact-based tasks.
- Better for smaller models: This approach can boost compact models without expensive retraining, making AI more accessible.
- Easy to adapt: You can swap in new, task-specific memory without changing the model itself, which helps with domain transfer (moving between subjects).
- More transparency: Being able to trace the model’s “thinking path” means better debugging, safety, and trust.
Looking ahead, the authors suggest testing this with larger models, more tasks (like fact checking and summarization), and different architectures. Their main message: combining neural networks with simple, rule-like structures (neuro-symbolic AI) can make AI reasoning clearer, stronger, and more dependable.
Collections
Sign up for free to add this paper to one or more collections.