LLM-based Generation of Semantically Diverse and Realistic Domain Model Instances

Abstract: LLMs have been recently proposed for supporting domain modeling tasks mostly related to the completion of partial models by recommending additional model elements. However, there are many more modeling tasks, one of them being the instantiation of domain models to represent concrete domain objects. While there is considerable work supporting the generation of structurally valid instantiations, there are still open challenges to incorporating real-world semantics by having realistic values contained in instances and ensuring the generation of semantically diverse models. Only then will such generated models become human-understandable and helpful in educational or data-driven research contexts. To tackle these challenges, this paper presents an approach that employs LLMs and two prompting strategies in combination with existing model validation tools for instantiating semantically realistic and diverse domain models expressed as UML class diagrams. We have applied our approach to models used in education and available in the literature from different domains and evaluated the generated instances in terms of syntactic correctness, model conformance, semantic correctness, and diversity of the generated values. The results show that the generated instances are mostly syntactically correct, that they conform to the domain model, and that there are only a few semantic errors. Moreover, the generated instance values are semantically diverse, i.e., concrete realistic examples in line with the domain and the combination of the values within one model are semantically coherent.

• The paper introduces a framework employing LLMs to generate semantically diverse and realistic object diagrams for UML class models.
• It compares instruction-based and chain-of-thought prompting strategies, achieving high syntactic validity and value-level semantic diversity.
• Empirical results demonstrate practical applications in testing, educational tools, and refining domain specifications.

LLM-driven Instance Generation for Semantically Rich Domain Models

Introduction

The paper "LLM-based Generation of Semantically Diverse and Realistic Domain Model Instances" (2604.10350) addresses the generation of semantically coherent, realistic, and diverse instance models—object diagrams—for domain models, primarily UML class diagrams, via LLMs. Historically, instance model generation has emphasized structural validity with limited focus on value-level semantics and meaningful diversity. This work systematically examines and benchmarks LLM-based prompting strategies for domain model instantiation, presenting both methodological innovations and empirical evaluation across multiple domains.

Problem Formalization and Motivation

Domain modeling in software engineering routinely uses class diagrams to encode structural types, relationships, attributes, and constraints intrinsic to a target domain. Object diagrams then instantiate these abstractions, representing specific scenarios or test cases. While automated tools, e.g., graph solvers and constraint-based generators, robustly produce structurally conformant instances, such approaches seldom ensure semantic plausibility or value diversity. The paper defines "realistic" models as those where attribute values are contextually meaningful and statistically representative, and further requires semantic diversity across instances.

Figure 1: Class diagram for a banking domain (top) and its object diagram instance (bottom), illustrating instantiation tasks and the kinds of real-world semantics necessary for instructional or test scenarios.

The motivation extends to scenarios in model-driven engineering education, validation of type-level abstractions, testing and amplifying model-based code generators, and bootstrapping large, realistic model repositories for data-driven or learning-based model analysis.

LLM-based Instance Generation Strategies

The core technical contribution is a framework leveraging LLMs to instantiate class diagrams expressed in the UML-based Specification Environment (USE) tool syntax, targeting two primary prompting strategies: (1) instruction-based prompting with exemplar (one-shot) syntax and (2) a chain-of-thought (CoT) decomposition.

Figure 2: Overview of the two instance generation strategies—top: instruction-based one-shot prompting; bottom: chain-of-thought multi-stage prompting where intermediate reasoning and scenario categorization enable finer-grained control over instance diversity.

Instruction Learning Strategy

The instruction-based approach supplies the LLM with explicit task directives, a domain model as textual input, and a single instance "shot" demonstrating the expected object diagram syntax. Sequential prompts request the model to generate multiple, semantically and structurally diverse instances, while automated checks using USE enforce both syntactic and conformance validity. Errors are automatically fed back with USE error messages for iterative correction. Diversity is enhanced by serially prompting the LLM to avoid repetitions.

Chain-of-Thought Learning Strategy

The CoT approach decomposes the instantiation process into explicit reasoning steps. The LLM first analyzes the domain model and generates detailed, category-based instance descriptions (e.g., base, complex, boundary, edge, and over-constrained scenarios). Each scenario is then instantiated in the required syntax, again iteratively corrected and validated. This method aims to probe the domain's solution space more comprehensively, especially at boundaries or constraint edges, and to increase both semantic and structural diversity.

Experimental Protocol and Evaluation Metrics

For empirical analysis, the authors curated a diverse dataset of ten domain models from educational and literature sources. The evaluation protocol systematically varies instance generation strategy, with fixed numbers of instances and controlled correction cycles. Metrics encompass:

• Well-formedness: syntactic correctness per USE parsing.
• Conformance: satisfaction of domain model typing, multiplicity, and constraints.
• Semantic correctness: contextually appropriate values, tested via API, RegEx, and human assessment.
• Semantic diversity: intra- and inter-instance diversity, computed via exact matches and normalized Levenshtein distance across numeric and string attributes.

Empirical Results

Both strategies successfully generated hundreds of syntactically correct and conformant instances per domain, with the instruction strategy achieving zero syntax or conformance errors. The CoT method exhibited few errors even for large, complex models (e.g., 9 syntax errors in 3618 model elements for Restaurant, 28 in 3784 for Football).

Semantic checks showed high performance on well-formedness for structured data (addresses, phone numbers, license plates), but attribute values matching real entities (e.g., actual addresses, real IBANs) were less frequently produced, likely due to privacy or model generality. Temporal constraints and complex model invariants were also largely respected.

Both approaches yielded highly diverse instances: diversity metrics were consistently close to unity (0.87–0.99) for both numeric and string attributes, demonstrating effective coverage of the instance space.

Figure 3: Two generated Bank domain model instances—left: Instruction Learning approach instance with distinct IBAN/BIC values; right: Chain-of-Thought boundary scenario instance fulfilling boundary-age constraints and exploring high/extreme attribute values.

Figure 4: Additional Bank instances—left: CoT edge scenario with unusual but valid attribute values (e.g., very high ages); right: CoT over-constraint scenario intentionally provoking model constraint violations to explore model boundaries.

A notable qualitative result is that the CoT strategy, through categorical scenario decomposition, facilitated modelers in identifying both boundary cases and over-constrained model definitions—useful for domain model refinement.

Related Work and Distinctiveness

Prior work in instance generation has focused on structural diversity via constraint solvers, search-based techniques, and random/MS strategies [BurguenoCCG19]. More recent advances have employed ML for code or schema synthesis, but few have targeted the synthesis of semantically plausible instances at the value level. Ontology instantiation has seen analogous challenges, with most emphasis on type-level semantics rather than object-level realism [CiattoAMO24, BenevidesGBA10]. This paper uniquely foregrounds LLMs' capability in capturing not only syntactic and structural validity but also domain-consistent semantics and value-level diversity.

Implications and Future Prospects

From a practical perspective, the demonstrated approach has direct applicability in generating large, semantically meaningful object diagram corpora for testing, tool evaluation, curriculum development, and benchmarking model-driven engineering systems. The techniques also provide a foundation for building adaptive, LLM-powered modeling assistants, which can suggest contextually-appropriate object instances, support model completion, and facilitate understanding of domain-specific instance spaces.

Theoretically, the findings motivate further exploration of prompt engineering for controlled semantic value generation, more robust feedback loops for constraint satisfaction, and the integration of external ontological or real-world data sources. The distinction between "realistic" (contextually plausible, representative) and "real" (veridical, externally verifiable) values exposes a key limitation and opportunity for more advanced hybrid approaches.

As more extensive domain-model instance repositories become available, supervised or reinforcement learning methods may further improve both the granularity and breadth of instance plausibility, enabling effective fine-tuning of LLMs for domain modeling tasks [Jeong-LLM-Finetuning.24]. Moreover, the approach is agnostic to the underlying LLM, suggesting continued performance improvements as LLM capabilities advance.

Conclusion

This work establishes that LLM-driven prompting strategies—both instruction-based and chain-of-thought—are capable of generating not just syntactically and structurally valid model instances, but also semantically realistic and highly diverse object diagrams for a wide range of domain models. Empirical results support strong correctness and diversity, with the CoT strategy showing particular strength in exploring extreme and boundary scenarios. The practical utility spans education, software testing, and model repository generation, while the theoretical insights frame a research agenda at the intersection of AI, model-driven engineering, and automated knowledge instantiation (2604.10350).