STRUM-LLM: Attributed and Structured Contrastive Summarization
Abstract: Users often struggle with decision-making between two options (A vs B), as it usually requires time-consuming research across multiple web pages. We propose STRUM-LLM that addresses this challenge by generating attributed, structured, and helpful contrastive summaries that highlight key differences between the two options. STRUM-LLM identifies helpful contrast: the specific attributes along which the two options differ significantly and which are most likely to influence the user's decision. Our technique is domain-agnostic, and does not require any human-labeled data or fixed attribute list as supervision. STRUM-LLM attributes all extractions back to the input sources along with textual evidence, and it does not have a limit on the length of input sources that it can process. STRUM-LLM Distilled has 100x more throughput than the models with comparable performance while being 10x smaller. In this paper, we provide extensive evaluations for our method and lay out future directions for our currently deployed system.
- Ojas Ahuja et al. 2022. ASPECTNEWS: Aspect-oriented summarization of news documents. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6494–6506.
- Reinald Kim Amplayo et al. 2021. Aspect-controllable opinion summarization. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6578–6593.
- Stefanos Angelidis et al. 2020. Extractive opinion summarization in quantized transformer spaces. Transactions of the Association for Computational Linguistics, 9:277–293.
- Palm 2 technical report. ArXiv, abs/2305.10403.
- Constitutional ai: Harmlessness from ai feedback. ArXiv, abs/2212.08073.
- Longformer: The long-document transformer. ArXiv, abs/2004.05150.
- Propsegment: A large-scale corpus for proposition-level segmentation and entailment recognition. In Annual Meeting of the Association for Computational Linguistics.
- Strum: Extractive aspect-based contrastive summarization. Companion Proceedings of the ACM Web Conference 2023.
- Hayate Iso et al. 2022. Comparative opinion summarization via collaborative decoding. In Findings of the Association for Computational Linguistics: ACL 2022, pages 3307–3324.
- Kevin Lerman and Ryan McDonald. 2009. Contrastive summarization: An experiment with consumer reviews. In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Companion Volume: Short Papers. Association for Computational Linguistics.
- Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Annual Meeting of the Association for Computational Linguistics.
- Aurko Roy et al. 2018. Theory and experiments on vector quantized autoencoders. ArXiv, abs/1805.11063.
- Self-critiquing models for assisting human evaluators. ArXiv, abs/2206.05802.
- Contrastive text summarization: a survey. International Journal of Data Science and Analytics.
- Luke Vilnis et al. 2022. Impakt: A dataset for open-schema knowledge base construction. ArXiv, abs/2212.10770.
- Is chatgpt a good nlg evaluator? a preliminary study. ArXiv, abs/2303.04048.
- Bertscore: Evaluating text generation with bert. ArXiv, abs/1904.09675.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.