All summaries · 2026-10-12

A Real-World CAD Segmentation Benchmark: Comparing Rule-Based and Learning-Based Approaches through Hole Detection

Kai Schütte, Edgar Heinert, Matthias Rottmann

arXiv:2610.13466 · abstract · pdf · summarized by meta/muse-spark-1.3-contributor on 2026-10-12

The one-paragraph version

Most CAD segmentation research trains and tests on synthetic parts where labels come for free from the modeling history. This paper asks whether that transfers to a messy real engineering job: finding holes that matter for fasteners like screws, rivets, clips and bolts. The authors hand-label a new real-world benchmark of 2,202 parts from engineering assemblies, build two hand-designed rule-based hole finders, and compare them against two learned models, UV-Net and AAGNet. When trained on plenty of their own data, the learned models win on accuracy. When tested on a different dataset style, MFCAD++, or when training data is scarce, the rule-based methods are far more stable. In short: learning needs data and matching conventions, rules need no data and travel better, with one rule method favoring finding everything and the other favoring being right when it fires.

The problem they're solving, and why it matters

Computer-aided design models, meaning the precise 3D part definitions engineers use for manufacturing, are usually stored as boundary representations. A boundary representation, often shortened to B-Rep, describes a solid by its skin: faces which are surface patches, edges where faces meet, and vertices where edges meet.

Existing public datasets for teaching computers to label those faces — for example MFCAD, MFCAD++, Fusion 360 Gallery, and CADSynth — are mostly synthetic or scraped, with labels generated automatically from how the shape was constructed. That is cheap and scalable, but it misses functional meaning.

The authors focus on fastener-relevant holes: cavity surfaces where a fastener would actually go in an assembly, such as the screw holes in a car body shown in their Figure 1. This is hard because:

Without a manually labeled engineering benchmark, we do not know if modern learning methods actually help automated fastener placement and assembly planning.

The key idea, in plain words

Compare the two philosophies head-to-head on the same real task, with careful tuning for both.

On one side: hand-written geometry rules that need no training. One searches directly in the part topology for loops that look like hole mouths. The other renders the part from many viewpoints like a vision system and checks which visual loops correspond to real cavities.

On the other side: neural networks that learn from examples. UV-Net, meaning a network that samples each face's own parametric surface grid plus its edges and then passes messages to neighbors, and AAGNet, meaning an attributed adjacency graph network that adds explicit face and edge attributes and looks one hop farther.

To make the comparison fair, the authors create a shared language and shared test: a face adjacency graph, meaning a graph where each node is a face and two nodes are linked if those faces share an edge, plus a human-labeled through-hole dataset from real assemblies and both per-face and per-hole scores.

The punchline they test is a trade-off, not a single winner: data-driven flexibility when you have matching data, versus training-free stability when you do not.

How it actually works, step by step

1. Common representation. All methods use the face adjacency graph described above. This captures only which faces touch, so it is naturally unchanged by moving, rotating, or scaling the part.

2. Graph-wire-search, called GWS. Think of looking for the rim of a hole, then growing inward.

3. Multiview rendering, called MVR. Think of photographing the part from all sides, then verifying with topology.

4. Learning baselines. UV-Net samples each face's parametric domain into a regular grid carrying position, normals and trimming, samples each edge along its curve, encodes them with convolutional networks, then aggregates two hops of neighbors plus a global shape vector to predict hole versus non-hole per face. AAGNet does similar but includes explicit B-Rep attributes and uses three message-passing hops, so a larger local neighborhood.

5. Dataset and labeling. Parts come from leaf components of assemblies in the 3D Assembly Repository. Multi-solid parts are excluded because UV-Net as used cannot handle them. To avoid duplicates, parts with identical face-type and edge-count statistics are deduplicated. Annotators label only cavity faces relevant to fastener placement, plus standard cylindrical holes between two perpendicular planar faces even without visible fastener context. This means some geometrically valid holes stay unlabeled and will count as false positives, which the authors state explicitly.

6. Evaluation. They score both per-face, meaning did we label the right surface patches, and per-hole, meaning did we find whole tunnels. Per-hole overlap is area-weighted intersection-over-union, meaning shared area divided by total area covered by label plus prediction, with a very strict threshold of 0.99 for the main tuning to simulate full automation without cleanup.

What they showed — results, honestly, with limitations

The dataset itself shows the gap: 2,202 models, 197,003 faces, only 7.3% hole faces, 6,864 holes, about 3 holes per model, with far more faces per part on average at 89.5 and more diverse surface types including splines and extrusions than MFCAD, MFCAD++, Fusion Gallery or CADSynth.

When trained and tested on their own benchmark, learning wins. Per-hole, UV-Net reaches about 0.934 recall and 0.896 precision, AAGNet about 0.944 recall and 0.810 precision. Recall here means fraction of true holes found; precision means fraction of predictions that were correct. GWS gets very high recall at 0.927 but low precision at 0.578 — it finds almost everything but over-fires. MVR is opposite at 0.728 recall and 0.870 precision — more cautious but more correct.

Cross-dataset tells a different story. Trained on the real benchmark and tested on 200 filtered MFCAD++ parts, AAGNet almost collapses to 0.013 recall, UV-Net drops to 0.468 recall but keeps 0.933 precision, while GWS stays at 0.889 recall and 0.805 precision and MVR at 0.617 recall and 0.987 precision. The authors attribute learning drops partly to modeling conventions: their data often splits a cylindrical hole into multiple faces, while MFCAD++ often uses a single face, plus different local context. MVR mainly fails on long holes never viewed along their axis.

With varying training size, rules are flat because they need no training, while UV-Net and AAGNet start worse and eventually surpass rules when given enough examples, though with instability — the paper notes a drop between 1,200 and 1,600 points. Varying the overlap threshold shows holes tend to be found almost completely or missed entirely, so strict thresholds lose little recall.

Limitations the authors acknowledge or that are clear from the text: aesthetic or aerodynamic parts are largely absent; only through-holes are labeled; multi-solid parts are excluded; unlabeled but geometrically plausible holes penalize GWS in precision, as shown in qualitative examples where yellow marks detections without labels; the MFCAD++ test is a curated 200-part subset without colliding holes or faulty labels; and class-weight and oversampling tuning for the learners showed no clear trend, with run-to-run variance comparable to parameter effects.

Why this might matter to me

For spatial reasoning in LLMs and VLMs: this is a clean test of functional geometry versus pure shape. A torus has a geometric hole but not a fastener hole. Success requires linking local surface evidence to assembly intent, the same kind of context-dependent spatial judgment you want in vision-language models that answer where does this screw go.

For world models: the hollowness check and entrance-exit wire logic are tiny physics-like priors — cavities spread normals differently, tunnels have two mouths. That suggests a useful middle ground between pure learned dynamics and hand-coded simulators: topology plus a little geometry as an invariant world prior that survives domain shift.

For text-to-CAD and parametric geometry: the failure mode around one-face versus multi-face cylinders is directly relevant. If a generator emits the same functional hole with different face splits or construction history, downstream segmentation should be stable. Training only on auto-labeled synthetic histories will bake in a particular modeling style. Their deduplication by face-type counts and their emphasis on multi-face cylinders is a practical warning for dataset design.

For 3D generation and scene representation: MVR shows that colored-face rendering plus multi-view consistency can propose precise B-Rep labels without learning, at the cost of many views and missed grazing angles. That points to hybrid pipelines where neural fields or Gaussian views propose candidates and exact B-Rep topology verifies them, and where per-hole area-weighted scores are more meaningful than per-point accuracy for usable parts.

Terms worth knowing