Self-Improving CAD Generation Agents with Finite Element Analysis as Feedback

Abstract: Computer-aided design (CAD) is the backbone of modern industrial design, yet learned CAD generators still fall short of real engineering pipelines: they neither iterate like engineers nor evaluate what engineering requires. Prior work has treated CAD generation as two disjoint steps, part synthesis and assembly, where the former is graded by proximity to a gold reference and the latter, when handled at all, is reduced to a separate constraint solving step. In this work, we introduce a more industry-native task formulation that requires a model to produce a fully assembled multi-part STEP file from a free-form engineering brief, which is then validated via finite element analysis (FEA). FEA validation reveals that Codex (GPT-5.5) and Claude Code (Opus-4.7) agents do not produce a single strict-passing artifact in the main first-attempt sweep, with the best configuration meeting only about 20% of typed requirements on average. Moreover, we introduce two additional supervision signals, a novel text-only blueprint schema and a 21-view image renderer that aids the agent's visual inspection, that better align the generation loop with how engineers iterate in practice. On S2O and Fusion360, the same feedback tools improve geometric reconstruction, with GPT-5.5/xhigh rising from 0.444 to 0.592 Box-IoU on S2O and from 0.397 to 0.505 on Fusion360. Together these signals move CAD programs toward artifacts that are not only visually plausible but also checked against physical and structural requirements.

• The paper redefines CAD generation by integrating structured blueprinting and FEA-driven feedback to satisfy strict engineering requirements.
• It details a novel pipeline that combines design synthesis, multi-view visual inspection, and deterministic repair mechanisms using FEA analysis.
• Experimental results demonstrate that iterative feedback significantly enhances artifact validity, with a 7x compute increase improving strict-pass outcomes.

Self-Improving CAD Generation Agents with FEA-Grounded Feedback

Problem Formulation and Motivation

Learned CAD generation systems have made tangible progress in mapping text or structured prompts into geometric artifacts, but the prevailing evaluation methodologies—geometric proximity to reference meshes, visual plausibility, or raw manifold validity—do not reflect the physical and functional criteria that govern engineering acceptance. This paper redefines agentic CAD generation as the process of producing fully-assembled, physically-realizable STEP artifacts from free-form engineering briefs, judged strictly through finite element analysis (FEA). Unlike previous benchmarks that reward reference matching, the Hephaestus-CCX benchmark introduced here directly queries whether CAD artifacts satisfy explicit, typed requirements reflecting the engineered system’s operational constraints.

This reframing exposes the critical gap: current LLM-based and VLM-based CAD agents routinely synthesize plausible geometry that nonetheless fails strict FEA-based validation, especially in multi-part assemblies subject to interface, load path, and selector constraints. This observation motivates two additional forms of feedback in the agent design loop: a structured blueprinting stage enforcing design intent and dimensional claims, and a rich multiview visual inspection suite far exceeding common render-based checks.

CAD-Agent Pipeline Architecture

The pipeline decomposes agent and controller responsibilities. The agent is tasked with design synthesis, high-level planning, and program repair, with the controller entrusted with deterministic code execution, validation, and integration of simulation tools and renderers.

The pipeline begins with a free-form engineering brief, which is optionally expanded by the agent into a structured blueprint YAML (schema-v4) that encodes part decomposition, parametric envelopes, interfaces, materials, load selectors, and functional acceptance claims. This early structuring constrains design, makes interfaces explicit, and lays the basis for auditable FEA. The agent then synthesizes CadQuery code, executed deterministically into a STEP assembly.

Figure 1: Overview of the pipeline encapsulating blueprinting, decomposition, STEP assembly synthesis, and iterative tool-driven revision.

A critical innovation is the rich-view visual inspection module, which supplies the agent with 21 static renders per artifact, including axis-aligned, close-up, isometric, and x-ray views, ensuring complete auditability of both external and internal features.

Figure 2: A grouped subset of the 21-view render set, highlighting both conventional and deep inspection modalities necessary for internal geometry validation.

The loop is closed with the FEA module (CalculiX), which parses the generated STEP, applies structured loading per spec, runs discretized physics, and checks all numeric criteria (stress, displacement, buckling, vibration, structural/dimensional/thermal constraints).

Hephaestus-CCX Benchmark and Problem Regimes

Hephaestus-CCX constitutes a 50-case curation drawn from a 466-case pool spanning thirteen engineering domains. It uniquely integrates real-world, parametric pass/fail requirements—extracted from engineering standards, supplier datasheets, patents, and intercollegiate competitions—precluding any reference-based evaluation leakage or superficial geometric scoring. Each case explicitly defines the set of FEA-checkable metrics and thresholds—over 2800 requirements in the pool—enabling fine-grained measurement of progress and agent weaknesses.

Figure 3: Per-domain distribution ensures both coverage and diversity, supporting robust sampling for benchmark construction.

Experimental Results and Feedback Efficacy

Baseline Model Behavior

A wide range of SOTA code agents (GPT-5.5, Claude Opus-4.7, Sonnet-4.6, DeepSeek, Kimi) were evaluated using the described pipeline. The major finding: no model produced a single strict-passing artifact on first attempt, and only a single strict pass emerged after one retry in 400+ multi-part submissions. Mean per-case requirement pass averaged around 25–32% for single-parts and 10–16% for multi-parts—well short of engineering acceptance.

Feedback-Driven Self-Improvement

Iterative feedback is shown to be the only reliable mechanism for tangible quality gain. Structured blueprinting and rich-view inspection, especially when combined, consistently raise geometric and physical contract satisfaction across S2O, Fusion 360, and Hephaestus-CCX. On S2O, blueprinting boosts GPT-5.5/xhigh’s Box-IoU from 0.444 to 0.592; on Fusion 360, 0.397 to 0.505. On Hephaestus-CCX, repeated FEA-driven repair lifts mean requirement pass from 38.8% to 60.5% in Codex GPT-5.5/high, yielding 9/50 strict-passing artifacts after eleven feedback cycles—nearly a 7x compute increase per artifact compared to the two-attempt baseline.

Figure 4: Test-time compute spending, when realized as structured feedback and repair, steadily increases compliance with engineering requirements and partial-credit scores.

Detailed ablations reveal that repair success is non-monotonic with raw reasoning scale (e.g., “high” beating “xhigh” on many metrics), but feedback application consistently yields strong, compounding returns. Notably, the largest quality jumps correspond with the agent gaining access to more interpretable and concrete feedback (detailed FEA failures, selector-specific margin reports).

Figure 5: Model-specific plots indicating that feedback-driven repair is orthogonal to first-shot artifact quality.

Repair Regimes

Qualitative analysis of successful strict-pass repairs exposes four main mechanisms:

• Structural retuning: Redesigns that materially improve load paths, stiffness, or reduce mass without directly changing the visible silhouette.
• Mesh-stable simplification: Removal/simplification of fragile or over-complex geometries to enhance meshability and structural performance.
• Checker-contract patches: Addition of missing metadata, selector bindings, or mass aliases necessary for harness interpretation, not solely for physics.
• Hidden mass-property fixes: Corrections (e.g., areal density reduction) that cannot be exposed by rendered views, requiring parametric/contract-based inspection.

Figure 6: Physical design changes enabling compliance with load-bearing specifications.

Figure 7: Simplification and property correction that affect measure signals but not easily visible geometry.

Figure 8: Contract-repair passes where all relevant physical properties were already valid, but selector or metadata binding errors prohibited acceptance.

Theoretical and Practical Implications

This work establishes that, on authentic engineering tasks, CAD agent evaluation must tightly integrate geometric, topological, and FEA-level functional validation. SOTA LLM and VLM agents remain incapable of generating complex, strictly-valid engineering assemblies in closed-loop scenarios without structured, repeated, simulation-grounded feedback. Visual plausibility and naive reference similarity are revealed to be inadequate surrogates for true engineering value.

The agentic pipeline here identifies critical bottlenecks—parametric planning, internal property binding, and feedback leverage—and delivers actionable evidence that feedback-driven, multi-iteration loops can convert increased test-time compute into concrete engineering gains, as long as the feedback is delivered in interpretable, actionable, domain-specific formats.

Practically, these results imply that any move toward AI-augmented engineering design automation must prioritize feedback harness construction over further scaling in parameter count or context window alone. The pipeline and benchmark presented are a foundation for systematic study of agentic failure and repair strategies, enabling reproducible research and cross-agent comparison.

Conclusions

This paper conclusively demonstrates that strict, FEA-based engineering validation is necessary to measure agentic CAD quality. The integration of blueprint planning, comprehensive visual feedback, and FEA-driven repair within a tightly controlled pipeline exposes fundamental weak points in current LLM-based CAD generation. Iterative feedback—not simply higher model capacity—is essential to achievement of engineering-valid artifacts.

The Hephaestus-CCX benchmark and accompanying harness will anchor future research into agentic, simulation-grounded design. The long-run implication is that practical AI for engineering domains will hinge not on pure generative prowess but on continual interaction with powerful domain validators and interpretable, structured feedback loops.