When Text or Speech Is Superior for LLM-Visualization Interaction

Determine the tasks, contexts, and user characteristics under which textual or spoken natural language constitutes a superior input modality for interacting with data visualizations via large language models, compared to direct manipulation or other modalities.

Background

The survey discusses how LLMs reintroduce textual and spoken modalities for interacting with visualizations but notes uncertainty about when these modalities outperform alternatives. This uncertainty persists despite improvements that LLMs bring to language-based interaction.

Clarifying the relative advantages of text versus speech for LLM-mediated visualization is foundational for designing effective multimodal systems and for matching interaction techniques to user needs, tasks, and environments.