Game Plot Design with an LLM-powered Assistant: An Empirical Study with Game Designers

Abstract: We introduce GamePlot, an LLM-powered assistant that supports game designers in crafting immersive narratives for turn-based games, and allows them to test these games through a collaborative game play and refine the plot throughout the process. Our user study with 14 game designers shows high levels of both satisfaction with the generated game plots and sense of ownership over the narratives, but also reconfirms that LLM are limited in their ability to generate complex and truly innovative content. We also show that diverse user populations have different expectations from AI assistants, and encourage researchers to study how tailoring assistants to diverse user groups could potentially lead to increased job satisfaction and greater creativity and innovation over time.

• The paper introduces GamePlot, an LLM-powered tool combining a design room for creative input with a game room for collaborative testing.
• The paper’s empirical study with 14 designers shows high satisfaction and effective integration of AI in facilitating narrative development.
• The paper highlights divergent role dynamics, with developers delegating narrative tasks while writers maintain active oversight to ensure story coherence.

Game Plot Design with an LLM-powered Assistant: An Empirical Study with Game Designers

The paper "Game Plot Design with an LLM-powered Assistant: An Empirical Study with Game Designers" addresses the integration of LLMs into the creative workflows of game narrative design, introducing a bespoke tool, GamePlot, designed for this purpose. GamePlot serves as an LLM-based application aimed to aid game designers in constructing complex narratives, particularly for turn-based games. The tool not only offers narrative generation and refinement capabilities but also facilitates real-time, collaborative gameplay testing to iteratively enhance narrative coherence based on in-game feedback.

Core Components

GamePlot consists of two core components: the design room and the game room.

• Design Room: Provides an interface where designers can begin building their game stories. Designers can input an opening story, set instructions for LLM behavior, and establish initial game and player turns. As designers interact with GamePlot, the tool provides suggestions for narrative elements, including character expressions and actions. The designers maintain creative control with the ability to modify and guide the content generated by the LLM.
• Game Room: Allows for collaborative testing sessions where both designers and players can engage in gameplay influenced by the LLM. A particularly notable feature is the "Wizard of Oz" function, which allows designers to manually control non-player character (NPC) actions to guide the narrative in real-time.

Empirical Findings

The presented study involved 14 game developers and narrative designers using GamePlot. Findings highlighted several key outcomes:

• User Satisfaction: Participants reported high satisfaction levels with the generated plots and appreciated the ease of integrating AI to enhance storytelling. The ability to tweak narratives in real-time was especially well-received, highlighting a sense of ownership retained by designers over the story despite AI involvement.
• Diverse Role Dynamics: Participants' feedback indicated a variance in how they valued the AI's contributions based on their roles. Game developers were more inclined to offload the narrative construction to the AI, while narrative writers preferred to oversee and shape the creative process actively.
• Challenges Faced: Despite positive feedback, limitations were acknowledged, including the AI’s occasional failure to produce highly innovative or complex narratives. Concerns were also raised regarding the AI’s ability to maintain narrative coherence and tension without extensive human guidance.

Implications and Future Directions

The results of the study suggest several implications for the practical and theoretical development of AI in game design:

• Customized AI Tools: Future research could benefit from tailoring AI tools to meet the diverse needs of different user groups. Such customization could enhance job satisfaction and the creative process.
• Improving Narrative Complexity: While LLMs demonstrate remarkable potential in automating routine narrative tasks, advancements are needed in generating nuanced, unpredictable stories.
• Human-AI Collaboration: By emphasizing the need for effective human-AI collaboration, this research contributes to the broader dialogue on AI as a tool for augmenting rather than replacing human creativity. The strong sense of control and ownership experienced by users could potentially improve narrative quality and innovation over time, provided that AI tools continue to evolve alongside creative needs.

Conclusion

Ultimately, the integration of GamePlot in game design highlights how AI can become a valuable collaborator in the creative process. The study underscores both the current limitations and the potential for growth in using LLMs to support narrative development. As AI technologies advance, developing tools that allow for flexible, user-centered collaboration will remain crucial in realizing the full potential of AI-Augmented narrative design environments.