Introducing MAPO: Momentum-Aided Gradient Descent Prompt Optimization
Abstract: Momentum-Aided Prompt Optimization (MAPO) enhances the efficiency and efficacy of prompt optimization for LLMs. Building on ProTeGi, MAPO uses positive natural language "gradients" and a momentum-based extension to refine prompts effectively. By tracking gradient history, MAPO avoids local minima and oscillations. It also utilizes beam search and an Upper Confidence Bound (UCB) algorithm for balanced candidate expansion and selection. Benchmark testing shows that MAPO achieves faster convergence time with fewer API calls and higher F1 scores than ProTeGi, proving it as a robust and scalable solution for automated prompt engineering in LLMs.
- RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning. arXiv.org.
- PromptBreeder: Self-Referential Self-Improvement via Prompt Evolution. arXiv.org.
- Prompt Optimization with Human Feedback. arXiv.org.
- Eureka: Human-Level Reward Design via Coding Large Language Models. arXiv.org.
- ETHOS: A Multi-Label Hate Speech Detection Dataset. Complex & Intelligent Systems, 8(6):4663–4678.
- OpenAI. 2022. Introducing ChatGPT. Accessed: 2024-10-19.
- Automatic Prompt Optimization with “Gradient Descent” and Beam Search. arXiv.org.
- A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications. arXiv.org.
- The Prompt Report: A Systematic Survey of Prompting Techniques. arXiv.org.
- AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts. arXiv.org.
- Automatic Prompt Augmentation and Selection with Chain-of-Thought from Labeled Data. arXiv.org.
- William Yang Wang. 2017. “Liar, Liar Pants on Fire”: A New Benchmark Dataset for Fake News Detection. arXiv.org.
- Large Language Models as Optimizers. arXiv.org.
- Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generation. arXiv.org.
- TEMPERA: Test-Time Prompting via Reinforcement Learning. arXiv.org.
- Large Language Models are Human-Level Prompt Engineers. arXiv.org.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.