Pragmos: A Process Agentic Modeling System

Abstract: The advent of LLMs has significantly transformed tasks across Software Engineering. In the context of Business Process Management, LLMs are now being explored as tools to derive process models directly from textual descriptions. Existing approaches range from chatbot-driven systems that assist with iterative, text-based modeling to fully automated end-to-end modeling assistants. However, we argue that process modeling is inherently complex and cannot be effectively addressed through black-box solutions. Instead, we envision modeling as an open-ended conversational activity, best supported by an interactive, iterative process involving both humans and LLM. In our approach, the modeling task is decomposed into smaller, manageable steps. Each step results in intermediate artifacts and explicitly documents the rationale behind each modeling decision. During this process, we incrementally uncover simple behavioral relations that guide the construction of the model. Given the current limitations of LLMs in reasoning about complex dependencies, we complement them with specialized tools developed in the field to structure process models based on behavioral relations. This hybrid approach enables the generation of sound, yet comprehensible models that evolve through transparent and explainable steps. In this paper, we present our research agenda and introduce Pragmos, a prototype system that operationalizes this vision. Pragmos demonstrates how LLMs can collaborate with human users as both domain and modeling experts to co-create evolving process models through a structured and explainable workflow.

• The paper demonstrates that Pragmos significantly improves process modeling transparency by combining LLM-driven extraction with human oversight.
• It decomposes process extraction into sequential tasks—execution path, relation discovery, concurrency identification, and loop detection—to enable robust BPMN synthesis.
• The evaluation on 45 process descriptions confirms structural correctness, with promising results alongside challenges in large-context scenarios, paving the way for scalability enhancements.

Pragmos: Agentic Process Modeling via Human-LLM Collaboration

Motivation and Problem Statement

The derivation of process models from unstructured textual descriptions remains a persistent challenge in Business Process Management (BPM). Manual modeling is cognitively taxing, subject to syntactic and semantic errors, and highly dependent on the expertise of analysts in both domain and process notation. The advent of LLMs initiated new possibilities for automating this transformation, promising to alleviate portions of the manual burden. However, existing LLM-based systems have largely adopted black-box architectures or rely on technical intermediate artifacts, impeding the interpretability and controllability essential for robust modeling in complex settings.

Conceptual Architecture and Methodology

The paper presents Pragmos, a prototype system premised on an agentic, iterative approach explicitly leveraging both LLM-driven knowledge extraction and established behavioral structuring techniques. Process modeling with Pragmos is decomposed into a sequence of manageable subtasks, each producing intuitive, intermediate artifacts—namely, execution paths, behavioral relations, and modular decompositions. This transparency empowers human analysts to scrutinize, validate, and modify intermediate outputs, providing direct control at every stage of model synthesis.

Pragmos operationalizes the following workflow:

1. Execution Path Extraction: Given a process description, the LLM enumerates all plausible execution traces, avoiding multiple loop iterations per trace. This step grounds subsequent analyses in artifacts familiar to analysts and analogous to process mining logs.
2. Behavioral Relation Discovery: From the set of execution paths, the system infers causality, conflict, and potential concurrency relations among activities, building an ordering relations graph (ORG).
3. Concurrency Identification: Pragmos optionally separates the explicit detection of concurrency, querying the LLM to nominate activity pairs potentially executable in parallel, refining the ORG in light of these findings.
4. Loop Detection: The system then queries loop structures, identifying blocks of activities corresponding to repeatable process fragments, reconciling cases where short loops may masquerade as concurrency.
5. Structural Refinement: Further steps address block abstraction, relabeling to disentangle ambiguous activity instances in loops, and the handling of control-flow entanglements (e.g., optional/skipped activities, while-loops versus repeat-loops) via symbolic alignment of execution paths to generated models.

These steps are grounded in the theory of modular decomposition for directed graphs, ultimately enabling the synthesis of process models in BPMN notation from acyclic ORGs, as well as adaptations for structured and certain unstructured constructs.

Comparative Analysis to Prior Work

Existing tools such as ProMoAI and BPMN-Chatbot operationalize LLMs for process extraction but exhibit critical limitations:

• Opacity of Decisions: ProMoAI’s intermediate logic is encapsulated in technical code or formal representations (e.g., POWL) inaccessible to most analysts, precluding meaningful intervention during synthesis.
• One-Step Generation Limitations: Prompt-based approaches, including those used in SAP Signavio’s AI-assistant, often rely on monolithic prompts, handling only simple processes and lacking transparency of intermediate reasoning.
• Lack of Structural Guarantees: It remains uncertain whether generated models are sound or free from behavioral anomalies, and downstream error correction is cumbersome.

Pragmos advances the state of the art by (1) promoting explainable, stepwise modeling grounded in behavioral relations, and (2) instantiating a conversational, analyst-in-the-loop paradigm, reconciling the gap between algorithmic process discovery and human interpretability/control.

Evaluation and Results

Pragmos was evaluated using the PET dataset, comprising 45 natural language process descriptions. The prototype, implemented in Python and interfacing with various LLM APIs (OpenAI, Gemini, Gemma, GPT-OSS), successfully synthesized process models reflecting the intended process logic for the majority of cases. The notable exception involved overly large descriptions exceeding the LLM context window, which constrain the effectiveness of modular decomposition and highlight a current technical limitation.

Importantly, the qualitative assessment corroborates that Pragmos exposes intermediate artifacts at each modeling phase, facilitating fine-grained analyst oversight and editability. The approach was demonstrated to generate structurally correct BPMN diagrams across a representative range of process constructs, including conditional branching, concurrency, and repetitive loops.

Implications and Future Directions

Pragmos introduces an agentic modeling paradigm poised to reshape the integration of LLMs in BPM by foregrounding transparency, explainability, and human agency. The separation of concerns—delegating elementary reasoning to LLMs while utilizing specialized process structuring tools—represents a pragmatic synthesis leveraging the current strengths and compensating for the reasoning weaknesses of LLMs. This approach aligns with the emerging consensus in the AI and BPM communities regarding the inadequacy of black-box LLM pipelines for complex modeling.

Practical implications include enabling less technically specialized analysts to directly participate in model derivation, which can accelerate organizational process documentation and adaptation cycles. Theoretical implications pertain to the formalization of hybrid, agentic workflows that tightly couple data-driven reasoning, domain expertise, and modular decomposition.

Next steps include:

• Context Window Scalability: Overcoming LLM context limitations for extremely large process descriptions, potentially through iterative summarization or hierarchical modeling schemes.
• Empirical Human-Subject Validation: Conducting user studies to quantitatively assess improvements in analyst productivity, model correctness, and cognitive load reduction.
• Formal Guarantees: Further integrating behavioral conformance checking and model verification to provide correctness guarantees beyond structural soundness.

Conclusion

Pragmos exemplifies a systematic, agentic framework for LLM-assisted process modeling, differentiating itself by its transparent, stepwise methodology and deep integration with established behavioral structuring techniques. The empirical evaluation supports strong claims regarding its interpretability and ability to produce sound process models suitable for BPM practice. The architecture provides a blueprint for future AI-assisted modeling systems, advocating for explainable, collaborative workflows as the standard for integrating generative AI into critical organizational engineering tasks.