All summaries · 2026-09-29

ConCAD: Constraint-Aware Image-to-CAD Generation with Dual-Granularity Rewards

Chenxi Zhai, Xi Cheng, Hang Cheng, et al.

arXiv:2609.34494 · abstract · pdf · summarized by meta/muse-spark-1.3-contributor on 2026-09-29

The one-paragraph version

ConCAD tackles image-to-CAD: given a single rendered picture of a mechanical part, generate an executable CadQuery program that rebuilds the part as an editable 3D solid, not just a look-alike mesh. The authors argue that current systems chase the wrong target — whether the code runs and whether volumes overlap — while missing design intent, the hidden relations like “these two holes are coaxial” or “these faces are symmetric” that make CAD editable. Their fix is to train a vision-language model with reinforcement learning using two complementary rewards: one that reads the code text before it ever runs to check for plausible engineering structure, and one that runs the code and measures how well the resulting solids overlap in 3D after canonical alignment. To prove they capture intent, they introduce a new evaluation that does not look at code at all, but measures constraints directly on the final solid geometry. On DeepCAD and Zero2CAD benchmarks they report the best volume overlap, lowest surface error, zero failures on DeepCAD, and highest constraint satisfaction.

The problem they're solving, and why it matters

Reverse-engineering a parametric CAD program from an image is central to reverse engineering, editable asset creation, and design automation. Recent work uses vision-language models, meaning large models that take images plus text and output text, here outputting CAD code, to inherit strong programming priors instead of using a custom decoder for CAD command tokens.

The problem is how we train and judge these models. Training by next-token imitation, where the model is taught to predict the next word in the ground-truth program, copies surface syntax but does not understand that changing a sketch plane or breaking concentricity breaks the design. Evaluation by execution validity, meaning did the program run without crashing, plus Chamfer Distance, meaning average closest-point distance between sampled surface points, plus volumetric Intersection-over-Union, meaning volume of overlap divided by volume of union, also misses the point. Two solids can have 90%+ overlap yet one has a hole drilled off-center or a bracket no longer mirrored.

Source-code similarity is not a fix either, because the same final shape can be written in wildly different ways in CadQuery. What matters for an engineer is the boundary representation, meaning the true solid defined by faces, edges, vertices and how they connect, and whether its relational logic holds. Without that, generated CAD is dead geometry: it looks right but cannot be edited reliably.

The key idea, in plain words

Separate learning signals from truth measurement.

For learning, give the model dense feedback at two levels that complement each other. Think of an apprentice drafter: first you praise them for using the right vocabulary and operations — “good, you made two cylindrical bosses and mirrored them” — even if the drawing still fails to build. That is the code-level constraint reward. Then, once the design actually builds into a solid, you grade precise 3D fit — “after centering and rotating to line up, how much volume overlaps?” That is the execution-level geometric reward. A simple pass-or-fail gate for “did it build a non-empty solid?” underpins both to prevent cheating.

For truth, do not trust the code text. Measure intent directly on the built solids by extracting geometry relations like coaxiality, meaning axes share the same line, concentricity, tangency, parallelism, perpendicularity, and symmetry from the boundary representation itself. That is their new metric, B-rep Geometric Constraint Satisfaction Rate, abbreviated G-CSR. It is deliberately not used as a training reward, so a high score suggests the model really learned structure rather than gaming the metric.

The training algorithm that ties this together is Group Relative Policy Optimization, abbreviated GRPO. Instead of needing a separate critic model to estimate value, for each image the policy samples a small group of candidate programs, scores them, and learns to prefer the better ones within that group relative to the group average.

How it actually works, step by step

Start with a base model, Qwen3-VL-2B-Instruct, a 2-billion-parameter open vision-language model, and first do supervised fine-tuning, meaning one epoch of next-token prediction on 147k DeepCAD pairs and separately on Zero2CAD-100K, to get a reference policy that knows CadQuery syntax.

Then run reinforcement learning. For each input image, sample a group of, in this paper, 4 candidate programs at sampling temperature 0.8, meaning fairly diverse exploration. Each candidate gets a total score that is a weighted sum of three parts, with weights 0.1, 0.3, 1.0 in the paper.

First, the Foundational Execution Validity Gate. Run the code in an isolated subprocess with a strict timeout using the OpenCASCADE kernel, meaning the industrial geometry engine that actually builds topology and exports a mesh. If it succeeds and volume is greater than zero, the gate is 1, otherwise 0. This blocks reward hacking toward non-building gibberish.

Second, Granularity 1: code-level constraint reward. Without running the kernel, a rule-based pattern matcher using regular expressions compares predicted code to ground-truth code across an eight-relation engineering set covering datums and positions, feature types and operation distributions, numerical parameters and proportions, and mirror and pattern configurations. In words: count how many expected structural ingredients appear, divided by how many were expected. For example, if the true program has two concentric holes plus a mirror, and yours has one hole and no mirror, you get partial credit. This gives learning signal even when nothing executes.

Third, Granularity 2: execution-level geometric reward. If the gate is 1, center each solid at its center of mass and scale by an inertia-derived radius, meaning a size measure derived from the moment-of-inertia tensor that captures how mass spreads, to remove scale and position ambiguity from a single view. Then align principal axes, meaning the natural long-medium-short directions from inertia, trying the identity plus four proper sign-flips to resolve that you cannot tell plus from minus on an axis. Compute volumetric overlap by Monte Carlo sampling with 8,192 points, and take the best overlap over those alignments. If nothing executed, this reward is 0.

Advantages within the group are standardized — subtract group mean and divide by group standard deviation — and the policy is updated with a clipped surrogate objective plus a penalty for drifting too far from the reference policy. The penalty is measured by KL divergence, meaning a statistical distance between new and old behavior. The authors also filter DeepCAD reinforcement data to a constraint-rich subset to avoid wasting updates on trivial primitives like lone cubes where rewards saturate.

For evaluation, G-CSR works as follows: normalize and align predicted and reference solids as above, match faces and edges by greedy bipartite matching under predefined tolerances, meaning pair closest compatible primitives, then for each active constraint type compute fraction satisfied and average across types. The paper does not publish the exact tolerance numbers or full constraint list, so independent replication would need those details.

What they showed — results, honestly, with limitations

On the DeepCAD test split shared with CAD-Coder, ConCAD goes from its own supervised start of 0.717 volumetric IoU, 0.011 Chamfer Distance, 1% Invalidity Ratio, meaning share of unexecutable or empty outputs, and 0.903 G-CSR, to 0.734 IoU, 0.009 Chamfer Distance, 0% Invalidity Ratio, and 0.913 G-CSR. That beats specialized baselines CAD-Coder at 0.662 IoU and ReCAD at 0.723 IoU, 0.011 Chamfer Distance, 0.906 G-CSR. General models without CAD tuning fail: vanilla Qwen3-VL-2B has 98% invalidity, while the paper’s reported proprietary models GPT-5.6-Sol and DeepSeek-V4-Pro mostly run but lag far on IoU at 0.599 and 0.466.

On Zero2CAD single-view, a harder set, ConCAD improves its supervised start from 0.443 to 0.507 IoU, 0.016 to 0.013 Chamfer Distance, and 0.768 to 0.781 G-CSR, versus the Zero2CAD baseline at 0.357 IoU and 0.535 G-CSR. Invalidity stays at 23%, showing single-view ambiguity is far from solved.

Ablations are revealing. Geometry-only training drops to 0.700 IoU versus 0.717 start, because sparse 3D feedback alone gives plateaus. Code-only drops to 0.654 IoU but keeps G-CSR at 0.908, because it learns patterns without spatial grounding. Using G-CSR itself as a training reward only reaches 0.702 IoU, because if execution fails the metric gives no signal and credit assignment collapses. The combined version is best on both shape and constraints. Weighting matters: the code weight of 0.3 is best; 0.5 overemphasizes syntax and IoU falls to 0.721.

Limitations: gains on DeepCAD are modest in absolute IoU, and the base failure rate was already low. Tolerances, full eight-pattern definitions, and inertia-normalization stability on symmetric or thin parts are unclear or unsupported in the text. There is no human study that edited outputs to prove editability, and results are shown only on a 2B backbone and single views, not drawings, point clouds, or text prompts.

Why this might matter to me

For spatial reasoning in VLMs: this is a concrete case where token imitation is not enough and where a cheap text-level structural proxy plus a grounded 3D check teaches axis awareness, symmetry, and alignment better than rendered-image similarity. The group-relative trick is attractive because unexecutable but structurally sensible attempts still get differential credit, which may help exploration in other spatial code tasks.

For world models: the inertia-normalized alignment with sign-flip search is a lightweight canonicalization — center, scale by spread, try axis flips — that could serve as a reward or state-alignment step for predicting stable object frames from single views, without a full pose estimator.

For text-to-CAD and parametric geometry: the lesson is to treat constraint recovery as first-class, not as a byproduct of overlap. Even if you work from text rather than images, the eight-way code matcher idea and the B-rep-only G-CSR evaluation offer a template: supervise on syntax patterns that are execution-free, but evaluate on extracted solid relations.

For 3D generation and scene representation: ConCAD argues for programmatic, editable representations over meshes or neural fields when downstream editing matters, and shows why pure Chamfer Distance and IoU can mislead. If you build scenes from parts, adding a B-rep constraint check could catch volumetrically similar but relationally wrong assemblies, like non-coaxial stacks or broken mirrors, that visual metrics miss.

Terms worth knowing