3DMorph: Single-Image-Guided Local 3D Shape Editing and Morphing

Abstract: Despite recent progress in 3D generation, intuitive editing of existing shapes remains limited. Unlike images, which benefit from well-established inpainting tools, general 3D objects such as meshes still lack simple and effective methods for local shape editing. Existing approaches are often global, domain-specific, require complex user interaction, or focus on appearance (color and texture) rather than geometry. We introduce 3DMorph, a training-free framework for single-image-guided local 3D shape editing and morphing. Given an edited image showing a desired shape modification, our method automatically localizes the relevant 3D region and transfers 2D modifications to 3D while preserving unmodified areas. 3DMorph also enables intermediate shape generation between the original and edited objects, facilitating design exploration. To benchmark editing quality, we introduce Delta3D, an image-guided local 3D editing benchmark with paired ground-truth edits. Experimental results show that 3DMorph translates intuitive 2D edits into 3D, outperforming state-of-the-art generative and editing methods.

• The paper introduces a training-free pipeline that lifts local 2D edits to precise 3D mesh modifications.
• Methodology employs SSIM-based localization and voxel-latent hybridization to preserve unedited geometry.
• Results show a 19% reduction in Chamfer Distance and enhanced context preservation compared to baselines.

Single-Image-Guided Local 3D Shape Editing with 3DMorph

Introduction and Motivation

Precise local editing of 3D mesh geometry is a persistent bottleneck in 3D design and engineering pipelines. Existing tools for 3D shape manipulation—especially for meshes without parametric or CAD provenance—either lack intuitive interfaces or require extensive retraining and do not support robust, geometry-preserving local edits. Unlike 2D image editing, which has seen significant advances (e.g., inpainting, sketch-based modifications), 3D shape editing remains constrained by global operations, domain-specific tools, or a predominant focus on visual appearance rather than geometry. "3DMorph: Single-Image-Guided Local 3D Shape Editing and Morphing" (2606.07115) directly addresses these limitations with a training-free approach that lifts local 2D image edits into high-fidelity, localized 3D mesh modifications, retaining global geometric context and supporting morph exploration.

Figure 1: Overview of the 3DMorph pipeline. A user modifies a rendered view of a mesh (e.g., through inpainting or sketching); the edit is automatically localized and lifted into structured 3D latent space, modifying only the affected region and preserving the remainder near-perfectly.

Methodological Framework

Problem Definition

Let $O$ denote the original 3D mesh and $I$ its rendered image. An edited image $\tilde{I}$, produced through standard 2D manipulation (manual or with generative inpainting models), specifies an intended local 3D edit. The central challenge is to transfer the modification in $\tilde{I}$ back to $O$ in 3D, yielding an edited mesh $\tilde{O}$ that:

• Reproduces the edit with geometric fidelity in the localized region;
• Preserves unchanged structure and fine detail elsewhere;
• Enables the synthesis of continuous interpolations ("morphs") between $O$ and $\tilde{O}$.

This is achieved without retraining for new edits, leveraging an architecture built on structured latent representations (SLAT) compatible with recent 3D generative frameworks, notably Trellis.

Pipeline Overview

The 3DMorph pipeline consists of the following core phases:

1. Edit Region Localization: Using SSIM-based 2D differencing between $I$ and $\tilde{I}$, the system detects modified image regions. Via camera calibration and geometric projection, these are mapped into an axis-aligned 3D bounding box in the voxel grid of $I$0.
2. Voxel-Feature Hybridization: The original mesh is voxelized (structure) and multi-view rendered (latents). Within the predicted edit region, the local voxel structure and latent features are re-synthesized using generative flow transformers conditioned on $I$1; outside the box, the original are strictly preserved.
3. Decoding: The hybrid SLAT (edited inside, preserved outside) is fed to a mesh decoder, yielding the output mesh $I$2.
4. Morph Trajectory Generation (Optional): By sequentially activating batches of voxels/latents within the edit region along a spatial ordering, intermediate meshes interpolating between $I$3 and $I$4 can be synthesized for visualization and exploration.

A key technical innovation over prior approaches—including Trellis with flow-consistent conditioning—is the use of a strictly enforced hard constraint on unmodified voxels during early-stage structure prediction, bolstering both geometric context preservation and edit locality.

Figure 3: Comparison of architectures. Vanilla Trellis denoises globally, risking drift; 3DMorph restricts stochasticity to the localized region, integrating prior geometry for robust preservation.

Quantitative and Qualitative Evaluation

Delta3D Benchmark

To address the lack of geometric ground-truth data for local edits, the authors introduce Delta3D—a benchmark with paired mesh and image data derived from curated CAD assembly manipulations. Each instance includes a source mesh, an edited image, and a ground-truth 3D mesh representing the target edit.

Metric Suite

Performance evaluation employs a suite of geometry-centric metrics (Chamfer Distance, Hausdorff Distance, IoU, Dice) and structural image similarity metrics (MS-SSIM, FSIM, GMSD). These are computed both globally and within the predicted edit region.

Results Overview

3DMorph achieves the strongest geometric fidelity and structural preservation among tested methods:

• Global CD: 0.0105 vs. 0.0129 (Trellis-RePaint), with a 19% reduction.
• IoU: 88.2% (3DMorph) vs. 85.2% (Trellis-RePaint).
• Even when compared to multi-view guided models (Trellis-MV with 12 views), single-image 3DMorph surpasses in both geometric and visual metrics.

Distribution analysis confirms the robustness of 3DMorph not only in mean values but across the sample set, showing fewer catastrophic failures and tighter performance variance.

Figure 5: Distributions of geometric metrics across benchmark samples. 3DMorph demonstrates superior central tendency and reduced tail risk compared to alternatives.

Ablation studies reveal that the bounding box localization module is highly effective: with ground-truth edit localization, 3DMorph further improves (e.g., CD drops to 0.0089), but even with predicted boxes, the method nearly matches the quality of strong baselines given perfect information.

Qualitative Results

• Global context is exceptionally preserved, with modifications sharply localized and fine detail undisturbed outside the edit region.
• Competing methods (especially single-image generative models and Trellis-RePaint) frequently inject unwarranted edits, global drift, or misinterpret geometric intent—failures not observed for 3DMorph.
• The morphing module enables explicit visualization of continuous intermediate shapes, useful for design space exploration and objective evaluation in engineering contexts.

  Figure 2: Output diversity (successes and failure cases). Limitation examples trace to SLAT resolution and ambiguous edits due to constrained single-box localization.

Architectural and Algorithmic Distinctions

3DMorph leverages structured latent space architectures from modern 3D diffusion models, but diverges from prior art by:

• Imposing strict context consistency outside the edit region, achieved by hard voxel constraints before latent-stage blending. This is in contrast to (and demonstrably superior over) purely flow-based context injection as in RePaint/Trellis-RePaint.
• Modularizing the editing pipeline to allow for reusability, extensibility (e.g., with future higher-res decoders or improved bounding-box prediction), and practical user input through standard image editing tooling.

Efficient bounding box prediction is central to 3DMorph and largely responsible for its ability to confine edits and avoid geometric drift. The current implementation projects 2D SSIM-based edit regions into 3D using calibrated camera parameters, resulting in editable region mappings that, while slightly conservative (IoGT 58%), are sufficiently tight for most real-world edits.

Limitations and Forward-Looking Implications

The primary limitation stems from the bounding box prediction and SLAT representation:

• SLAT resolution limits the granularity of permissible edits, especially for high-detail or multi-part modifications not suitably captured by axis-aligned boxes.
• Failure cases primarily arise when edit regions are ambiguous, highly occluded, or multi-regional—a constraint inherent in the current projection-then-box-prediction approach.

Anticipated improvements include:

• Higher-resolution SLAT decoders: Expected to further improve geometric fidelity and the sharpness of fine detail, as predicted by recent advancements (e.g., Trellis2).
• Multi-box or segmentation-based edit localization: Allowing more complex, non-contiguous edits.
• Enhanced user interfaces leveraging generative inpainting for intuitive 2D specification of 3D changes.

The practical ramifications are significant: 3DMorph establishes a paradigm where routine, high-fidelity mesh editing becomes as accessible as contemporary image editing, without retraining or parametric models, and with robust guarantees on global geometry preservation. This addresses a critical bottleneck in data-driven design, rapid prototyping, and simulation-based engineering analysis.

Conclusion

3DMorph demonstrates state-of-the-art local 3D mesh editing guided by single-image modifications, outperforming both generative and editing-based baselines in geometric (and visual) accuracy, context preservation, and robustness on the Delta3D benchmark. The modular, training-free architecture enables direct applicability in CAD-centric workflows, setting a foundation for future advances in interactive, user-guided 3D editing pipelines. Prospective directions include domain-general bounding box prediction, handling of disjoint edit regions, and infusion of denser latent representations for unmatched geometric detail.