Reconstruction of Convex Polytope Compositions from 3D Point-clouds

Abstract: Reconstructing a composition (union) of convex polytopes that perfectly fits the corresponding input point-cloud is a hard optimization problem with interesting applications in reverse engineering and rigid body dynamics simulations. We propose a pipeline that first extracts a set of planes, then partitions the input point-cloud into weakly convex clusters and finally generates a set of convex polytopes as the intersection of fitted planes for each partition. Finding the best-fitting convex polytopes is formulated as a combinatorial optimization problem over the set of fitted planes and is solved using an Evolutionary Algorithm. For convex clustering, we employ two different methods and detail their strengths and weaknesses in a thorough evaluation based on multiple input data-sets.

• The paper introduces a novel unsupervised pipeline that reconstructs 3D point-clouds into convex polytope compositions using an evolutionary algorithm.
• It combines geometric heuristics like neural pre-segmentation, DBSCAN and RANSAC-based plane fitting, and both LoS and WCSEG clustering to manage noisy, unstructured data.
• Experimental results demonstrate high-fidelity shape reconstruction on complex models with applications in CAD and physics simulations, despite scalability challenges.

Reconstruction of Convex Polytope Compositions from 3D Point-clouds

Introduction and Problem Statement

The paper addresses the problem of reconstructing a solid object from 3D point-cloud data by representing the object as a union of convex polytopes. Applications for this task are situated in reverse engineering (e.g., the CAD reconstruction of buildings), and in real-time physics simulations, where convex decomposition simplifies collision detection. The underlying challenge is two-fold: detecting and fitting planes to noisy, unstructured point data, and optimally partitioning these planes into convex polytopes when the number of polytopes is unknown. This reconstruction is posed as a combinatorial optimization problem with a solution based on an Evolutionary Algorithm (EA).

Figure 1: The proposed polytope reconstruction pipeline.

Methodology Overview

The proposed pipeline comprises four principal stages: plane extraction, point-cloud structuring, weakly convex clustering, and polytope generation.

Plane Extraction

A neural network pre-segmentation is employed to estimate primitive types per point in the cloud. This is followed by DBSCAN-based clustering and a RANSAC-based fitting procedure for each cluster, resulting in robust plane extraction. Planes are assigned to surface points via a mapping, accommodating noise and outlier scenarios.

Point-cloud Structuring

Structured point sets are created by projecting points onto plane-local occupancy grids, forming a set $O_s$ of planar points. A plane-neighborhood graph is constructed, encoding adjacency relationships between planes based on shared neighbors in the $k$-NN graph. This representation facilitates downstream clustering and geometric reasoning and removes noise and outliers from consideration.

Weakly Convex Clustering

The partition of the point-cloud into (weakly) convex regions is critical. Two methods are evaluated:

• Line-of-Sight (LoS) Clustering: A visibility graph is constructed, where edges connect mutually visible pairs of points. Spectral clustering is performed on this graph’s affinity matrix, with the number of clusters selected via a quality metric that weighs intra- and inter-cluster visibility.

  Figure 2: Visualization of convex clusters using exemplar points, their line-of-sight graphs, and surface approximations.

• Weakly Convex Segmentation (WCSEG): Over-segmentation by normal-based region growing is merged by visibility and then refined by volumetric dissimilarity based on the Shape Diameter Function (SDF).

Figure 3: Results of intermediate steps in WCSEG: over-segmentation (left) and after merging (right).

Figure 4: Clustering results on models M1, M2, M4, and M7: LoS (top), WCSEG (bottom).

Cluster assignment aligns structured points with clusters using nearest neighbor queries to ensure consistency regardless of the clustering subsampling involved.

Polytope Generation

For each cluster, an EA searches for the best combination of the associated planes to form one or more convex polytopes. The target volumes are defined via a signed distance field, created using Poisson surface reconstruction on the cluster’s points. The EA optimizes an explicit objective function combining (1) fit to point data, (2) compatibility of polytope interiors with the target volume, and (3) a penalty for the number of polytopes.

Polytopes are generated in the H-representation, and the Double Description method enables efficient conversion to V-representation for volumetric reasoning. Mutation, crossover, and elite selection operators are implemented to drive optimization. Post-processing filters out low-quality, duplicate, or fully-contained polytopes to ensure a parsimonious decomposition.

Figure 5: Example of plane extraction results on a 3D point-cloud.

Experimental Results

The pipeline is evaluated on seven models with varying complexity and point counts (25k–50k points). Clustering outcomes are highly input-dependent:

• The LoS method achieves perfect cluster recovery for models with strong convexity and well-separated features (e.g., M3, M4), but fails for cases with intricate geometric details (e.g., M1, M7), where WCSEG performs better.
• Both methods perform well when polytopes are spatially distinct (M6).

For all models, the reconstructed polytopes accurately capture the input geometry, except for minor errors constrained to cases where the clustering step undersegments complex structures (notably in M2’s roof). The pipeline demonstrates robustness even with noisy or sparse input.

Pipeline step timings were measured: Polytope Generation (EA) dominates runtime, especially when the search space is large due to incomplete clustering. Plane extraction and structuring are orders of magnitude faster (sub-4s even for the largest models).

Discussion and Implications

Key claims:

• The approach is capable of reconstructing arbitrary compositions of convex polytopes from unstructured point-clouds, outperforming prior methods limited to simpler primitives (e.g., cuboids).
• The combination of strong clustering and combinatorial optimization produces high-fidelity, compact representations with direct application in CAD and physics simulation workflows.

Contrasting prior art:

Unlike learning-based primitive fitting approaches (e.g., CvxNet, BSP-Net), which either require category-level priors, input images, or assume a fixed number of primitives, the proposed method is unsupervised, directly operates on raw point-clouds, and dynamically determines both partitioning and polytope count.

Scalability limitations:

EA-based combinatorial optimization scales poorly with the number of planes and clusters, motivating future research in more scalable combinatorial solvers or tighter coupling between clustering and polytope generation. Weaknesses in clustering accuracy directly degrade geometric reconstruction quality, especially for non-convex regions.

Theoretical implications:

The results in this paper suggest that hybrid pipelines leveraging geometric heuristics for initial model structuring and learning-free, evolutionary optimization for primitive assembly are competitive with more supervised, learning-centric frameworks in unconstrained 3D reconstruction.

Practical implications:

The method’s robustness to noise and outliers, and its ability to handle disconnected components, makes it suitable for real-world scanning scenarios, reverse engineering, and geometric preprocessing for dynamic simulations.

Future Work

Potential directions include:

• Improving clustering scalability and robustness using advanced geometric graph techniques or hybrid learning approaches.
• Integrating with generalized CSG recovery pipelines to broaden the scope beyond convex decompositions.
• Exploring differentiable optimization or reinforcement learning as alternatives to classic EA for the combinatorial search.

Conclusion

This paper presents a comprehensive pipeline for reconstructing arbitrary compositions of convex polytopes from dense, unstructured 3D point-clouds. The coupling of RANSAC-based plane extraction, visibility-driven or volumetric clustering, and evolutionary combinatorial optimization yields effective, general polytope decompositions. Continued research into scalable optimization and adaptive clustering mechanisms will further extend the applicability of this approach within geometric modeling and simulation applications.