Exploring Creativity in Human-Human-LLM Collaborative Software Design

Abstract: While the use of LLMs in programming has been extensively studied, there is limited understanding of how LLMs support collaborative work where creativity plays a central role. Software design, as a collaborative and creative activity, provides a valuable context for exploring the influence of LLMs on creativity. This study investigates how and where creativity naturally emerges when software designers collaborate with an LLM during a design task. In a laboratory setting simulating a workplace environment, 18 pairs of software professionals with design experience were asked to complete a design task. Each pair had 90 minutes to produce a software design based on a set of requirements, with optional access to a custom LLM interface. Pairs were not primed to be creative. We find that creativity was present in all pairs in design processes, with 13 producing design documents containing creativity. We primarily attribute creativity to the human designers, driven by traits such as prior experience, empathy, and the use of analogies. The LLM contributed by producing novel ideas and elaborating human ideas. However, in some cases, the LLM appeared to hinder creativity by suggesting complex solutions or adding to unproductive digressions. LLMs can support creativity in collaborative software design, but human insights remain central. To effectively augment human creativity, designers must be intentional in their engagement with LLMs.

• The paper shows that human-led creativity predominantly drives design innovations even with LLM support.
• It uses rigorous qualitative analysis of 18 expert pairs to reveal the significant impact of analogical transfer and prompt engineering.
• Results indicate that LLMs offer solution-focused automation rather than transformative ideation, emphasizing the need for critical human input.

Creativity Dynamics in Human-Human-LLM Collaborative Software Design

Study Context and Methodology

The paper "Exploring Creativity in Human-Human-LLM Collaborative Software Design" (2604.24373) investigates the role of LLMs in collaborative software design tasks, emphasizing creativity emergence and attribution. Eighteen pairs of experienced software professionals participated in a simulated remote workplace laboratory study, tasked with producing a software design document within 90 minutes based on a provided PRD. Participants could optionally access a custom OpenAI ChatGPT 3.5 API interface, with usage patterns ranging from non-use through partial query/support, to exclusive LLM-based artifact generation.

A rigorous qualitative coding protocol was applied to identify creative events and elements, leveraging Boden’s "new, surprising, and valuable" definition for product creativity and Dorst and Cross's "creative events" lens for process creativity. Triangulation across session transcripts, artifact logs, and LLM prompt/responses enabled source attribution: human-initiated, joint human-LLM, and pure LLM-provided.

Process-Level Creativity: Human Dominance and LLM Influence

Across all pairs, creativity surfaced robustly in the design process. Human designers resorted to mechanisms such as reuse of libraries/components (OAuth, Google Auth), scope simplification via assumptions/decisions, and active analogical transfer from mobile applications (Google Maps, Yelp, Lyft). The LLM's involvement was primarily solution-centric rather than problem reframing; it contributed by automating routine artifact creation and occasionally proposing alternative or extended solution elements.

Three vignettes highlighted typical creative events:

• Human-driven reuse resolved requirement ambiguity with OAuth integration.
• Analogical transfer from Google Maps led to simplified data model decision.
• LLM-generated schema prompted complication identification, triggering resolution via analogy.

Notably, analogies facilitated insight in 14/18 pairs’ processes, but only one pair’s artifact exhibited direct analogical-inspired creativity. This underscores the gap between ideation in collaborative dialogue and artifact-level manifestation.

The LLM, instead of reshaping the design problem space, provided solution proposals, sometimes anchoring or inducing digressions. LLM-fueled creative leaps occurred when its output sparked critical review or elaboration, but creative events relating to problem conceptualization originated exclusively from human designers.

Product-Level Creativity: Attribution, Variability, and LLM Impact

Although process creativity was universal, only 13/18 design artifacts contained at least one creative element (novel, surprising, and valuable relative to PRD). These ranged from technical enhancements (scalability, performance) to expanded feature sets (bike park occupancy tracking, content moderation, external police report analysis).

Figure 1: Number of creative elements included in the design document of each pair, categorized into source (human or LLM), with color denoting LLM usage intensity.

Twenty distinct creative elements were catalogued (most common: scalability/performance, bike park occupancy), with heterogeneous distribution across pairs. Human-initiated creativity (prior knowledge/experience, empathy, ambiguity exploitation, underlying goal focus) accounted for 15/29 elements. Joint sources (LLM sparks/elaborates) accounted for 5 elements, while LLM-provided features contributed 9 elements.

Pairs providing expansive, less-constrained prompts to the LLM elicited more novel suggestions; by contrast, pairs copying verbatim requirements into LLM prompts obtained less creative outputs, echoing prior findings on requirements fixation hampering creativity. Importantly, the presence of LLM support did not guarantee artifact creativity—5/16 pairs using the LLM produced no creative additions, while both non-LLM pairs generated creative artifacts.

Practical and Theoretical Implications

The findings reinforce the centrality of human-led creativity in collaborative software design, even amid LLM augmentation. LLMs automate routine aspects and occasionally suggest new features, but rarely introduce fundamentally surprising or transformative elements independent of designer input. The "commoditization" of LLM-generated creativity (content moderation, accessibility, etc.) reflects both model limitations and prompt scope constraints.

The indirect benefit of LLMs—automating rote design work and freeing cognitive bandwidth—was observed to facilitate deeper problem exploration. However, uncritical engagement risks solution anchoring and digressive distractions, reducing creative productivity.

For practitioners, maximizing creative augmentation with LLMs requires:

• Using open-ended prompts to promote unconstrained ideation.
• Deliberately considering multiple alternatives before selection to counter anchoring.
• Time-boxing LLM interactions to mitigate digression-induced productivity loss.
• Utilizing LLMs primarily for boilerplate and routine artifact automation, reserving problem definition and solution exploration for human teams.

From a theoretical standpoint, the study challenges assumptions about LLMs as creativity catalysts: their direct contributions are frequently non-transformative, and optimal integration demands intentional interaction modalities. Creative outputs remain predominantly at the "little-c" and "pro-c" levels—routine or domain-validated novelty—rather than the "Big-C" innovation defined by Boden or Kaufman and Beghetto.

Future Directions in AI-Augmented Design Creativity

Promising research avenues include real-world deployment and longitudinal tracking of LLM-augmented design teams, examining agency and creativity over iterative cycles. Design of LLM-based creativity support tools—purpose-built to encourage problem space exploration, alternative generation, and reflection—holds potential to amplify human creativity without undermining autonomy. Further exploration of requirements framing, prompt engineering, and user agency are needed to ensure meaningful creative augmentation, rather than commodity feature expansion or homogeneity.

Conclusion

This study demonstrates that collaborative creativity in software design remains decisively human-centric, with LLMs offering limited, solution-focused augmentation rather than problem reframing or ideation leadership. To leverage LLMs for maximal creative impact, practitioners must hone prompt engineering and retain critical engagement throughout the design cycle. Future research and tool development should prioritize amplification of human-led creativity, safeguarding agency and innovation within AI-augmented collaborative workflows.