LLM-for-X: Application-agnostic Integration of Large Language Models to Support Personal Writing Workflows

Abstract: To enhance productivity and to streamline workflows, there is a growing trend to embed LLM functionality into applications, from browser-based web apps to native apps that run on personal computers. Here, we introduce LLM-for-X, a system-wide shortcut layer that seamlessly augments any application with LLM services through a lightweight popup dialog. Our native layer seamlessly connects front-end applications to popular LLM backends, such as ChatGPT and Gemini, using their uniform chat front-ends as the programming interface or their custom API calls. We demonstrate the benefits of LLM-for-X across a wide variety of applications, including Microsoft Office, VSCode, and Adobe Acrobat as well as popular web apps such as Overleaf. In our evaluation, we compared LLM-for-X with ChatGPT's web interface in a series of tasks, showing that our approach can provide users with quick, efficient, and easy-to-use LLM assistance without context switching to support writing and reading tasks that is agnostic of the specific application.

• The paper presents LLM-for-X, a shortcut mechanism that integrates LLMs directly into various applications, reducing context switching and increasing productivity.
• It utilizes an OS-level service, native app interfaces, and browser extensions to interact with LLM backends via chat interfaces or API calls.
• User studies revealed a 40% faster editing speed and improved usability, demonstrating reduced cognitive load compared to conventional interfaces.

System Overview and Implementation

The paper "LLM-for-X: Application-agnostic Integration of LLMs to Support Personal Writing Workflows" introduces a novel shortcut mechanism, LLM-for-X, which aims to seamlessly integrate LLMs into a wide range of applications to enhance task productivity without context switching. This native layer directly connects applications like Microsoft Office, VSCode, and Adobe Acrobat with LLM backends including ChatGPT and Gemini, enabling streamlined workflows and improved task efficiency.

Figure 1: LLM-for-X walk-through. (a) Iterating on LLM responses, (b) Pasting responses as 'insert below' vs. 'replacing' with diff view, (c) Direct in-place pasting without preview, and (d) Selecting and querying for information retrieval.

LLM-for-X Architecture

LLM-for-X operates through a lightweight, application-agnostic shortcut layer that interacts with LLM backends either through uniform chat interfaces or specific API calls. The technology is implemented as an operating system-level service that provides a pop-up UI for real-time querying and response insertion. The following implementation components are crucial:

1. OS-level Background Service: Listens for global keyboard shortcuts across applications to activate LLM integration.
2. Native App Interface via Accessibility API: Extracts and inserts text selections using Windows UI Automation APIs.
3. Browser Extension: Facilitates interaction within web applications, detecting and manipulating DOM elements to handle text selections and responses.
4. Direct LLM Interaction: Communicates with LLM backends either by emulating user input in chat interfaces or direct API call integration.

This architecture allows users to maintain focus on their tasks within apps while leveraging LLM functionalities such as text manipulation and information retrieval efficiently.

User Study and Findings

To assess the effectiveness of LLM-for-X, the authors conducted a controlled study involving 14 participants. The study focused on tasks related to writing, reading, and coding, comparing LLM-for-X with ChatGPT's web interface on metrics of task completion time, usability, and perceived workload.

Key Findings

1. Task Completion Time: LLM-for-X led to significant reductions in completion time, especially for editing tasks, where users completed tasks 40% faster on average compared to using the ChatGPT interface.

   Figure 2: Effect of Interface on task completion time [sec].

2. System Usability Scale: Participants rated LLM-for-X higher in terms of usability (SUS score), indicating a preference for operations that do not require context switching.
3. NASA Task Load Index: LLM-for-X was perceived to be less demanding, particularly in terms of ease of use, as reflected in the NASA TLX scores.

   Figure 3: Effect of Interface on SUS and NASA TLX scores.

4. User Preferences: Feedback indicated a preference for LLM-for-X's seamless integration within applications, reducing distractions through context switches, despite some participants expressing a preference for ChatGPT's conversational interface.

Implications and Future Work

The LLM-for-X framework advocates for minimizing context-switching interruptions, proven to enhance user productivity in writing and editing tasks. This research supports the transition toward more integrated AI tools where core functions can assist users directly within the application environment without the need for multiple, app-specific subscriptions.

Further Research Directions

• Task-Specific Integration: Expand LLM-for-X capabilities to include multimedia and more complex task-specific contexts, like programming IDEs and creative software environments.
• User Interface Evolution: Enhance UI design based on user-specific contexts and adaptive prompts to increase the efficiency of user interactions.
• Extension of API Support: Incorporate additional LLM chat interfaces and APIs to extend service applicability and responsiveness, accommodating broader user needs.

Conclusion

LLM-for-X represents a significant shift in LLM-assisted workflows, focusing on reducing operational frictions associated with context-switching and subscriptions to multiple LLM services. It is positioned to enhance productivity across diverse application domains by offering users a simplified yet powerful interface for interacting with LLMs directly within their native application ecosystem. The study demonstrates its efficacy in fostering efficient task completion while maintaining high usability and user satisfaction, paving the way for more integrated AI solutions in professional settings.