Robust Knowledge Extraction from Large Language Models using Social Choice Theory

Abstract: Large-LLMs can support a wide range of applications like conversational agents, creative writing or general query answering. However, they are ill-suited for query answering in high-stake domains like medicine because they are typically not robust - even the same query can result in different answers when prompted multiple times. In order to improve the robustness of LLM queries, we propose using ranking queries repeatedly and to aggregate the queries using methods from social choice theory. We study ranking queries in diagnostic settings like medical and fault diagnosis and discuss how the Partial Borda Choice function from the literature can be applied to merge multiple query results. We discuss some additional interesting properties in our setting and evaluate the robustness of our approach empirically.

• The paper presents a novel application of social choice theory using the Partial Borda Choice function to aggregate multiple LLM responses for improved consistency.
• It demonstrates enhanced diagnostic accuracy by significantly reducing output variability in critical fields like medical and technical fault diagnosis.
• The study underscores the importance of data quality and domain-specific model tuning for boosting the reliability of LLM-driven decision-making.

Understanding Robustness in AI Language Comprehension

When it comes to AI, particularly LLMs, the promise of providing accurate answers to a variety of questions has opened doors to applications that seemed like science fiction not too long ago. From facilitating conversations to offering assistance in creative writing, these models appear to be a silver bullet. However, there’s a caveat: the robustness of LLM outputs—particularly their consistency—can be problematic, especially in domains where accuracy is paramount, such as medicine and engineering diagnostics.

The Challenge of Varying Answers

A key limitation when using LLMs for applications such as conversational agents or query answering systems is their inconsistency; the same inquiry can yield different results upon multiple prompts. This shortcoming is even more pronounced because LLMs, by design, generate answers regardless of whether they truly "understand" the topic—sometimes leading to what's known as "hallucination" of responses. Moreover, subtle variations in question phrasing or incorporating irrelevant information can skew the outcomes significantly. This variance casts doubt on the reliability of these models in situations where precision is vital.

A Novel Approach to Consistency

To tackle this issue, researchers have proposed a method inspired by social choice theory—a discipline that interprets individual preferences and combines them into a collective choice. To apply this to LLMs, the concept is straightforward: ask the same query multiple times and then use a social choice technique called the Partial Borda Choice function to merge the multiple query results into a single, more reliable answer. This function scores the recurring answers based on their frequency and order of occurrence, leading to a final ranking that represents a collective preference from repeated prompts. For instance, if a particular answer arises relatively consistently, it will score higher, indicating stronger confidence compared to more sporadic responses.

Experimentation and Validation

The approach has been empirically tested with a focus on diagnostic settings, such as medical and technical fault diagnosis, where the causes of particular conditions need to be determined. Here, a query outlining a set of symptoms would be processed multiple times to yield a variety of potential causes, which are then aggregated using the Partial Borda Choice function. The experimental results demonstrated that this method notably improved answer robustness against query repetition and minor syntactic changes, as compared to traditional singular query responses or simpler aggregation strategies.

The Importance of Data Quality and Model Tailoring

The method's effectiveness is influenced not just by the voting system it leverages, but also by the quality of data on which the LLM is trained. While the technique shows promise even with mixed-quality data sources such as the internet, its reliability would be further enhanced if applied to domain-specific models that have been trained on high-quality, peer-reviewed data. In practice, however, the financial and computational resources required for such fine-tuning may be substantial.

Conclusion and Future Directions

This study affirms the potential of social choice theory as a bridge to a more reliable AI-driven decision-making process. By aggregating answers and thus reducing unpredictable variance, LLMs can step closer to becoming dependable assistants in critical domains. Looking forward, expanding this research to counter other types of uncertainty—caused by injected noise and adversarial attacks—could bolster the robustness of LLMs even further. As LLMs continue to evolve, so too must the methods of interpretation and validation to ensure that they can serve as trusted resources in decision-making processes.