State Machine of Thoughts: Leveraging Past Reasoning Trajectories for Enhancing Problem Solving
Abstract: Current LLM-based agents reason within an exploration-evaluation framework, navigating problem-solving processes in a tree-like manner. However, these methods often neglect successful reasoning trajectories once a problem is resolved, leading to inefficient use of these trajectories for future analogous problems. To address this inefficiency, we adopt a state machine to record experience derived from previous reasoning trajectories. Within the state machine, states represent decomposed sub-problems, while state transitions reflect the dependencies among sub-problems. The state machine records both successful and failed trajectories. Utilizing the experience from the state machine, our proposed State Machine of Thoughts (SMoT) selects the most optimal sub-solutions and avoids incorrect ones. Our experiments show that SMoT can significantly improve problem-solving abilities in two exploration-intensive problems: the 24-point game and a taxi navigation reinforcement learning game.
- Graph of Thoughts: Solving Elaborate Problems with Large Language Models. arXiv:2308.09687Â [cs.CL]
- Language Models are Few-Shot Learners. In NeurIPS.
- Sparks of Artificial General Intelligence: Early experiments with GPT-4. arXiv:2303.12712Â [cs.CL]
- Dynamic Planning with a LLM. arXiv:2308.06391Â [cs.CL]
- OpenAGI: When LLM Meets Domain Experts. NeurIPS (2023).
- Hierarchical finite state machines with multiple concurrency models. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 18, 6 (June 1999), 742–760.
- Reasoning with Language Model is Planning with World Model. arXiv:2305.14992Â [cs.CL]
- LLM+P: Empowering Large Language Models with Optimal Planning Proficiency. arXiv:2304.11477Â [cs.AI]
- Self-Refine: Iterative Refinement with Self-Feedback. arXiv:2303.17651Â [cs.CL]
- Training language models to follow instructions with human feedback. arXiv:2203.02155Â [cs.CL]
- SayPlan: Grounding Large Language Models using 3D Scene Graphs for Scalable Robot Task Planning. In CoRL.
- Reflexion: Language Agents with Verbal Reinforcement Learning. In NeurIPS.
- LLaMA: Open and Efficient Foundation Language Models. arXiv:2302.13971Â [cs.CL]
- Voyager: An Open-Ended Embodied Agent with Large Language Models. arXiv:2305.16291Â [cs.AI]
- Self-Consistency Improves Chain of Thought Reasoning in Language Models. In ICLR.
- Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. In NeurIPS.
- AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation Framework. arXiv:2308.08155
- Self-Evaluation Guided Beam Search for Reasoning. arXiv:2305.00633
- Mihalis Yannakakis. 2000. Hierarchical State Machines. In Theoretical Computer Science: Exploring New Frontiers of Theoretical Informatics. 315–330.
- Tree of Thoughts: Deliberate Problem Solving with Large Language Models. arXiv:2305.10601
- ReAct: Synergizing Reasoning and Acting in Language Models. In ICLR.
- Building Cooperative Embodied Agents Modularly with Large Language Models. arXiv:2307.02485Â [cs.AI]
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.