Sparks: Inspiration for Science Writing using Language Models

Abstract: Large-scale LLMs are rapidly improving, performing well on a wide variety of tasks with little to no customization. In this work we investigate how LLMs can support science writing, a challenging writing task that is both open-ended and highly constrained. We present a system for generating "sparks", sentences related to a scientific concept intended to inspire writers. We find that our sparks are more coherent and diverse than a competitive LLM baseline, and approach a human-created gold standard. In a study with 13 PhD students writing on topics of their own selection, we find three main use cases of sparks: aiding with crafting detailed sentences, providing interesting angles to engage readers, and demonstrating common reader perspectives. We also report on the various reasons sparks were considered unhelpful, and discuss how we might improve LLMs as writing support tools.