How Private are Language Models in Abstractive Summarization?

Abstract: In sensitive domains such as medical and legal, protecting sensitive information is critical, with protective laws strictly prohibiting the disclosure of personal data. This poses challenges for sharing valuable data such as medical reports and legal cases summaries. While LMs have shown strong performance in text summarization, it is still an open question to what extent they can provide privacy-preserving summaries from non-private source documents. In this paper, we perform a comprehensive study of privacy risks in LM-based summarization across two closed- and four open-weight models of different sizes and families. We experiment with both prompting and fine-tuning strategies for privacy-preservation across a range of summarization datasets including medical and legal domains. Our quantitative and qualitative analysis, including human evaluation, shows that LMs frequently leak personally identifiable information in their summaries, in contrast to human-generated privacy-preserving summaries, which demonstrate significantly higher privacy protection levels. These findings highlight a substantial gap between current LM capabilities and expert human expert performance in privacy-sensitive summarization tasks.

• The paper demonstrates that language models often leak personal data during summarization, especially in zero-shot scenarios.
• It reveals that 1-shot prompting and instruction fine-tuning significantly enhance both privacy preservation and summary quality.
• Comparative analysis shows closed-source models generally outperform open-weight ones unless fine-tuned for privacy.

Privacy Implications of LLMs in Abstractive Summarization

The paper "How Private are LLMs in Abstractive Summarization?" addresses the critical concern of privacy in text summarization tasks, especially within sensitive domains like medicine and law. This work stands apart by shifting focus from the conventional investigation of privacy risks stemming from training data to examining how LLMs (LMs) perform in maintaining privacy when summarizing non-private source documents.

Study Design and Methodology

The authors conducted an extensive analysis involving both closed- (GPT-4o and Claude Sonnet 3.5) and open-weight models (including Llama-3.1 and Mistral), employing various sizes and architectures. The investigation was carried out across diverse datasets spanning medicine, law, and general news, with particular emphasis on sensitive text—medical records and legal documents. The methodology was comprehensive, incorporating both prompting variations (0-shot, 1-shot, and anonymize techniques) and instruction fine-tuning (IFT) of open-weight models. Evaluation metrics included both qualitative assessments (human evaluation) and quantitative measures (ROUGE, BERTScore, and privacy-specific metrics like Leaked Documents Ratio (LDR) and Private Token Ratio (PTR)).

Key Findings

1. Privacy Leakage Prevalence: The study finds that LMs frequently fail to prevent PII leakage, even when explicitly instructed to avoid it. This is especially prominent in zero-shot scenarios, where models generally exhibit higher PTR values. Smaller models, notably open-weight ones, are particularly vulnerable compared to their larger counterparts.
2. Impact of Prompting and Fine-Tuning: The utilization of 1-shot prompts showed marked improvements in both privacy preservation and summarization quality, demonstrating the additional context's effectiveness. Furthermore, privacy and utility were enhanced significantly through instruction fine-tuning, allowing open-weight models to reach and even surpass closed-source models in some tasks, highlighting IFT's potential in training models on specific privacy-preserving behaviors.
3. Model Comparison: Closed-source models generally outperformed open-weight models, particularly in raw ability to generate high-quality summaries without leaking PII. However, when enhanced with IFT, open-weight models such as IFT-Llama-3.1-70B showed substantial improvement, matching the closed-source models' performance in terms of privacy metrics.
4. Challenges in Broader Domains: The study notes that maintaining privacy in less structured domains, such as news, presents additional challenges. This is possibly due to the difficulty in discerning relevant from irrelevant PII without specific domain guidelines.

Implications and Future Work

This research highlights the ongoing privacy challenges implicit in deploying LMs for abstractive summarization, especially in privacy-sensitive areas. The findings underscore the necessity to enhance LM architectures further or develop more sophisticated anonymization strategies to ensure robust privacy-preserving capabilities. From a practical perspective, the study suggests leveraging instruction fine-tuning as an effective approach for training models to adhere to domain-specific privacy needs.

Future investigations could benefit from exploring multimodal summarization tasks, incorporating images or structured data that could further complicate privacy preservation. Additionally, extending the research to include larger datasets and real-world application scenarios can shed light on how well these models perform outside of controlled experimental environments. There is also merit in examining dynamic, user-interactive settings where privacy risks can be more pronounced due to unscripted exchanges.

In conclusion, while significant strides have been made in improving LLM privacy, ongoing efforts will be crucial in ensuring these technologies can be widely—yet safely—adopted across various domains.