GeoCode: Interpretable Shape Programs

Abstract: The task of crafting procedural programs capable of generating structurally valid 3D shapes easily and intuitively remains an elusive goal in computer vision and graphics. Within the graphics community, generating procedural 3D models has shifted to using node graph systems. They allow the artist to create complex shapes and animations through visual programming. Being a high-level design tool, they made procedural 3D modeling more accessible. However, crafting those node graphs demands expertise and training. We present GeoCode, a novel framework designed to extend an existing node graph system and significantly lower the bar for the creation of new procedural 3D shape programs. Our approach meticulously balances expressiveness and generalization for part-based shapes. We propose a curated set of new geometric building blocks that are expressive and reusable across domains. We showcase three innovative and expressive programs developed through our technique and geometric building blocks. Our programs enforce intricate rules, empowering users to execute intuitive high-level parameter edits that seamlessly propagate throughout the entire shape at a lower level while maintaining its validity. To evaluate the user-friendliness of our geometric building blocks among non-experts, we conducted a user study that demonstrates their ease of use and highlights their applicability across diverse domains. Empirical evidence shows the superior accuracy of GeoCode in inferring and recovering 3D shapes compared to an existing competitor. Furthermore, our method demonstrates superior expressiveness compared to alternatives that utilize coarse primitives. Notably, we illustrate the ability to execute controllable local and global shape manipulations.

• The paper presents an interpretable procedural program for 3D shape synthesis that enables precise and controllable shape manipulation.
• It employs a directed acyclic graph to integrate continuous, discrete, and binary parameters, ensuring coherent structural adjustments.
• Experimental results show superior performance over existing methods using Chamfer Distance, confirming its robustness and generalizability.

GeoCode: Interpretable Shape Programs

Introduction

The paper "GeoCode: Interpretable Shape Programs" (2212.11715) introduces a novel approach to the problem of 3D shape synthesis, focusing on creating an interpretable and editable parameter space that facilitates intuitive shape manipulation. The key challenge addressed by GeoCode is the dual requirement of modeling both continuous and discrete variations in 3D shapes, transcending the limitations of current methods which either lack intuitive interpretability or produce coarse geometric outputs. GeoCode capitalizes on a procedural methodology, effectively combining complex rules to enable high-level edits that seamlessly translate into refined, high-quality 3D shapes. This system bridges the gap between abstract parameter spaces and tangible geometric outputs, making it feasible to perform controlled local and global shape modifications.

Methodology

GeoCode employs a procedural program constructed as a directed acyclic graph (DAG), where nodes represent operations ranging from basic mathematical functions to mesh transformations. This setup allows for modularity and flexibility in defining shape components and their interrelations. Human-interpretable parameters are used to control the attributes of these nodes, enabling direct manipulation of geometry through discrete, binary, and continuous inputs.

Figure 1: System overview. GeoCode learns to map a point cloud or a sketch input to an intuitively editable parameter space. The input passes through the corresponding encoder to obtain an embedding vector which is then fed to a set of decoders that predict the interpretable parameters. The program enforces a set of rules that, given a parameter representation, produces a high-quality shape by construction.

The program uniquely supports structural interactions and alterations, ensuring that changes in one component are coherently propagated through the model, maintaining structural integrity and semantic consistency. For instance, altering the roundness of a chair's seat autonomously adjusts other vertices to preserve overall coherence.

Experimental Results

The effectiveness of GeoCode is validated through extensive experiments demonstrating its capabilities in accurate shape recovery and generalization, even with out-of-distribution datasets. The shapes generated by GeoCode maintain structural validity across various tests, outperforming existing techniques such as StructureNet and ShapeAssembly in both quantitative and qualitative assessments. A notable aspect of the paper is the use of Chamfer Distance as a metric to benchmark shape reconstruction accuracy, indicating that GeoCode achieves superior performance in preserving the fidelity of reconstructed shapes.

Figure 2: Shape gallery. Showing reconstructed shapes on our test set. Our procedural program produces high-quality geometry from a 3D point cloud or a 2D sketch and contains consistent part segmentation information across the resulting shapes.

Implications and Future Directions

GeoCode's ability to generalize and maintain robustness to various perturbations in input data—such as noise in point clouds or variability in sketch styles—highlights its potential utility in real-world applications, particularly in computer graphics and virtual reality. The system's inherent interpretability and flexibility open avenues for further exploration in more complex scenarios, including 3D scene representation and interactive shape design.

The authors propose future enhancements that could incorporate additional attributes like UV texture maps and material properties, thus enriching the procedural model's versatility and applicability in broader contexts. Given the promising results demonstrated, GeoCode sets a foundational precedent for the integration of interpretable geometry manipulation within traditional 3D modeling workflows.

Conclusion

GeoCode represents a significant advancement in the field of 3D shape synthesis, providing a sophisticated yet accessible means of shape manipulation through an interpretable parameter space. This methodological breakthrough paves the way for enhanced user interaction with 3D models, offering precise control over shape characteristics and ensuring stable and coherent geometric outputs. By establishing a robust framework for interpretable shape processing, GeoCode holds substantial potential for practical deployment across various disciplines requiring detailed and customizable shape creation.