Open-Ended QA Prompts

• Open-ended QA prompts are flexible instructions designed to elicit detailed, multi-step reasoning and synthesis from LLMs across various domains.
• They employ methods such as chain-of-thought, functional decomposition, and self-prompting to improve answer quality and ensure robust performance.
• They drive tangible improvements in benchmarks for medical, educational, and multimodal tasks through iterative, automated prompt optimization techniques.

Open-ended question-answering prompts are sequences of instructions or demonstrations presented to LLMs to elicit unrestricted, natural-language answers, often with an emphasis on explaining reasoning, generating diverse content, or handling complex, underspecified tasks. Unlike close-ended prompts (e.g., multiple-choice, span selection), open-ended prompts are designed to maximize the model’s capacity for synthesis, justification, abstraction, and multi-step reasoning across a variety of domains.

1. Taxonomy and Foundational Datasets

Open-ended QA prompts are structurally varied according to task demands and evaluation context. Instances include medical clinical reasoning ("What is the most likely diagnosis?"), visual question answering ("Why is the woman holding the umbrella?"), and educational question generation or assessment.

Foundational datasets include:

• MEDQA-OPEN: Derived from the MedQA-USMLE MCQ corpus, with each item rewritten into an open-ended, optionless vignette with stepwise clinician-vetted reasoning and diagnosis. Records include the question (Q), a chain-of-thought (CoT) reasoning (R), and a final answer (A), with all R+A pairs checked by at least two medical experts for plausibility and factual accuracy. Train/dev/test splits mirror the original, with ≈10,000/1,500/1,223 examples (Nachane et al., 2024).
• CUS-QA: Grounded in multilingual regional knowledge (Czech, Slovak, Ukrainian), with instructions designed for “trivia contestant” style concise factual answers, both textual and visually grounded, evaluated under strict correctness, coherence, and cross-lingual fidelity (Libovický et al., 30 Jul 2025).
• OpenCQA: Answers open-ended questions about data visualizations, requiring multi-modal context comprehension and explanatory text output, annotated with decontextualized standalone answers (Kantharaj et al., 2022).

These datasets reflect the span from tightly reasoned medical diagnosis, to factual recall, to explanatory, multimodal abstraction.

2. Prompt Engineering Methodologies

Several distinctive methodologies have been demonstrated for the construction and optimization of open-ended QA prompts:

• Chain-of-Thought (CoT) Prompts: Exemplified by CLINICR (“Chain-of-Thought LInear INcremental CLInical Reasoning”), which mimics clinical thought via numbered, categorically labeled steps: “1. Initial Impression... 2. Differential... 3. Next Step... 4. Refined Plan... Final answer:”. Empirical evidence shows that five well-chosen few-shot CLINICR exemplars robustly induce stepwise logic in LLMs and outperform generic 5-shot CoT prompts (Nachane et al., 2024).
• Functional Decomposition: AMA’s two-step “question()” and “answer()” chains systematize the task by first rephrasing an input as a model-friendly question, then generating an open-ended answer. Each chain is flexible in style (yes/no, Wh-, cloze) and demonstrations, supporting prompt set diversity (Arora et al., 2022).
• Multi-stage/Multi-task Prompting: In educational question generation, multi-stage CoT-inspired pipelines extract concepts, propose question stems, and iteratively refine these into high-complexity, reasoning-driven questions, with prompts selected for desired Bloom-level depth (Maity et al., 2024).
• Self-prompting/Prompt Mining: In visual QA, a set of evidence Q&A pairs are synthesized via a visual question generation model from auto-tagged image entities. These are then fed as compact prompts through a visual-aware prompting module, fusing language and vision embeddings for final answer prediction (Wang et al., 2024).

3. Evolutionary and Automated Prompt Optimization

Prompt optimization for open-ended QA tasks increasingly relies on black-box, data-driven strategies:

• DEEVO: DEbate-driven EVOlutionary optimization evolves prompts via structured LLM-judged debates, guided by Elo ratings. Prompts are recombined using intelligent crossover (semantic block transfer) and mutation (clarify, expand, prune), preserving diversity and driving population-level improvement without direct reliance on task-specific metrics (Nair et al., 30 May 2025).
• PromptQuine: Provides an in-context evolutionary search where prompt “genomes” (fixed in-context demonstrations) are randomly pruned into minimal—often “gibberish”—subsequences. Fitness is scored by task-specific proxies (e.g., classification reward, joint style-content-fluency scores), revealing that non-human-readable prompts frequently outperform hand-crafted baselines, especially when token positions provide strong model hooks (Wang et al., 22 Jun 2025).
• Self-Prompting Framework: Generates synthetic QA pairs, explanations, and supporting passages by recursively prompting the LLM, then curates in-context exemplars via clustering/retrieval to maximize coverage and diversity (Li et al., 2022).

This suggests that prompt optimality increasingly emerges from automated discovery and population-level competition, displacing hand-crafted, interpretable templates where model-facing fitness dominates over human comprehension.

4. Domain-Specific Open-Ended Prompting: Clinical, Educational, Multimodal

Open-ended prompting requires adaptation to domain conventions and data modalities:

• Medical QA: CLINICR’s linear incremental reasoning, coupled with MCQ-CLINICR and MCQ-ELIMINATIVE extensions, first generates differential diagnoses (candidates) then reasons eliminatively to select the best. Final decision is optionally verified by a lightweight learned reward model that improves expert-agreement from ~80% to ~86%, demonstrating the value of discriminative-verification stages after generative candidate expansion (Nachane et al., 2024).
• Education and Assessment: Open-ended prompt engineering is realized through both explicit (long/short prompts, context cues) and multi-stage CoT-inspired pipelines for question generation. Evaluation is hybrid: automatic metrics (ROUGE, BLEU, distinct-n, Bloom-level classification) and expert Likert-scale scoring of fluency, relevance, depth, and originality. Longer, more content-driven prompts yield higher-depth questions, yet models remain dependent on human oversight for curriculum alignment (Maity et al., 2024).
• Multimodal QA: OpenCQA and Q&A Prompts methods use multimodal input encoding (chart images, visual objects, OCR metadata) and multi-hop or evidence-aggregating Q&A pairs. Vision-language pretraining (VL-T5), image tagging, and cross-modal fusion modules are crucial for non-textual settings, with open-ended prompts tailored to extract, summarize, or analytically compare visual features (Kantharaj et al., 2022, Wang et al., 2024).

5. Evaluation, Human-in-the-Loop, and Practical Recommendations

Open-ended QA presents unique evaluation and deployment challenges:

• Metrics: Common automatic metrics include accuracy, precision, recall, F1 (when ground truth is known), ROUGE/BLEU/BLEURT/BERTScore (text similarity and semantics), content selection scores, and LLM-as-judge correctness heuristics. In medical and educational settings, Likert-scale expert assessment is requisite for reasoning and factuality validation (Nachane et al., 2024, Maity et al., 2024, Kantharaj et al., 2022).
• Human-in-the-Loop: Iterative prompt refinement through expert vetting, criterion-driven rubrics (hidden from answerers to preserve informational asymmetry), and feedback cycles are standard in education and high-stakes QA (Matelsky et al., 2023). Inter-annotator agreement datasets inform reward models and human calibration.
• Best Practices: Key guidelines repeatedly substantiated include:
  • Encode domain workflows as discrete, self-contained reasoning steps.
  • Always prefer explicit, descriptive, open-ended prompts (“What is the most likely...?”) to option- or label-constrained formats.
  • Combine generative and discriminative stages for candidate exploration and rigorous selection.
  • Leverage few-shot CoT exemplars for stepwise logic; five exemplars frequently suffice.
  • Use greedy decoding over sampling for faithful stepwise reasoning reproduction.
  • In low-data or non-English regimes, minimalist, system-style zero-shot prompts (“You are an experienced trivia contestant...”) have shown robust cross-lingual fidelity, but are susceptible to hallucination and lack answer grounding (Libovický et al., 30 Jul 2025).

6. Empirical Results and Comparative Analyses

Empirical results consistently support the superiority of open-ended prompts—especially CoT and multi-stage pipelines—over restrictive or mere classification templates:

• CLINICR outperforms state-of-the-art 5-shot CoT prompts in open-ended medical QA, with response verification improving expert-agreement rates by 6 percentage points (Nachane et al., 2024).
• AMA-style open QA prompt sets aggregated with weak supervision deliver an average absolute lift of +10.2% over few-shot baselines across LLM families, surpassing GPT-3 175B in 15 out of 20 tasks using only a 6B model (Arora et al., 2022).
• Self-prompting gives +15 EM gains over direct prompt baselines in zero-shot open-domain QA (Li et al., 2022).
• In visual explanatory QA, Q&A Prompts contribute +5–7% absolute gains on OK-VQA and A-OKVQA benchmarks (Wang et al., 2024).

Performance in multimodal and regional knowledge domains is more uneven; concise "trivia contestant" prompts yield high metric-human correlation for short named-entity answers, but longer, more reasoning-intensive outputs remain bottlenecked by hallucination, limited grounding, and lack of robust automatic metrics (Libovický et al., 30 Jul 2025, Kantharaj et al., 2022).

7. Open Challenges and Future Directions

Open-ended QA prompt research continues to grapple with:

• Evaluation Limitations: Standard n-gram and embedding-based metrics (BLEU, ROUGE, BERTScore) are often insufficient for longer, more abstractive, or visually-grounded responses. Human-in-the-loop and LLM-as-judge metrics yield higher system-level reliability, but answer-level alignment in visual or longer-context settings is weak (Kantharaj et al., 2022, Libovický et al., 30 Jul 2025).
• Prompt Interpretability vs. Fitness: Evolutionary search uncovers effective non-interpretable ("gibberish") prompts, refuting the dogma that prompts must be human-readable. Optimization in model-latent token space may outpace manual engineering, especially as LLM architectures evolve (Wang et al., 22 Jun 2025).
• Hallucinations and Grounding: Hallucination in longer answers and lack of explicit provenance constraints persist as challenges, particularly where prompts lack explicit instruction to avoid non-factual completions.
• Transfer and Robustness: Prompt effectiveness varies by model scale, domain, language, and shot setting. Zero-shot, minimalist prompts show surprising cross-lingual robustness, but for highly specialized reasoning, few-shot, workflow-aligned, or automated evolved prompts deliver more consistent gains.

A plausible implication is further progress will depend on hybrid strategies: automated prompt population search, dynamic prompt chaining, recall-grounded LLM-as-judge scoring, and ultimately integration of programmatic or symbolic reasoning modules for explicit step-level verification and interpretability.