The one-paragraph version
MIRAGE-CAD is a system that turns four different kinds of observations of a mechanical part — a written description, a rendered picture, a 3D point cloud, or STEP geometry — into a working parametric CAD program written in Python. It does this in two steps with an explicit middle step: first predict a short readable construction plan that lists parameters and operations like sketch, extrude, hole pattern or fillet, then translate that plan into code that an OpenCASCADE geometric kernel can run to build a solid and export STEP. On 2,500 held-out parts per input type, it builds a solid 55.4% to 70.0% of the time and exports STEP 52.3% to 66.2% of the time, with no lookup from training at test time. The careful finding is not just that it works, but what the middle step does and does not do: having some construction signal helps a lot, but that signal does not have to be written as text to get good shape accuracy, and being able to run is very different from getting the shape right or being usefully editable.
The problem they're solving, and why it matters
A history-based parametric model is more than a final shape. It is a recipe: what sketches were drawn, what was extruded or cut, in what order, with what sizes left as adjustable knobs.
That recipe matters because it decides what happens when you later edit. The paper gives a concrete example: a cylinder could come from extruding a circle, revolving a rectangle, or cutting a hole in a block. All three can look identical today, but turn one knob and they behave completely differently.
The hard part is that shape does not pin down recipe. The same solid can be made many valid ways, so recovering construction from observation is fundamentally ambiguous. The sensible goal, the author argues, is not to find the one true original history, but to find a plausible construction that matches what you saw and stays useful for editing.
Most prior work predicts that construction as its final answer, either as a fixed list of CAD commands or as executable code. This paper asks a different question: what do you gain if you force generation to go through an explicit construction representation before you write code, and can you cleanly separate whether code runs, whether geometry matches, and whether parameters actually do anything?
The key idea, in plain words
Put a shared construction bottleneck in the middle, and keep it readable and measurable.
Think of it like this. Different witnesses describe the same object in different languages: one writes words, one shows a photo, one gives a bag of 3D points, one gives precise boundary geometry. MIRAGE-CAD first translates each witness into the same internal hunch about how the thing was built. That hunch lives in a shared space anchored on construction, not on appearance.
From that hunch it writes a short text plan, like an ingredient list with named sizes and ordered steps. Only then does it write full Python code, with imports, variable names and function calls.
The clever experimental twist is that the image path is a strict bottleneck: the code writer sees only the text plan, nothing else from the image. For text, point cloud and STEP, the code writer also gets a small extra observation note, for example the original wording or kernel-derived size statistics. The author is explicit about this asymmetry, because it limits what you can claim the plan alone caused.
A readable plan is useful even if it does not improve shape accuracy by itself, because you can inspect it, score its operation list against a reference, and ask whether plan agreement predicts that the final program will run.
How it actually works, step by step
Start with one input at a time. No fusion of multiple inputs is tried.
Encode the query. Each modality has its own encoder that outputs a normalized embedding, meaning a fixed-length vector of length 512 scaled to length one so similarity is just a dot product. In plain terms: text uses a frozen DistilBERT language model averaged over tokens; image uses a frozen CLIP vision model on one 224 by 224 isometric render; point cloud uses a small PointNet on 1,024 centered and scale-normalized points, so absolute size is deliberately removed here; STEP uses mostly a 50-number global descriptor of counts, bounding-box extents, area, volume and face and edge statistics, not a learned face-by-face graph. The paper is clear that the fancy per-face and per-edge streams are effectively empty in this corpus.
Align to construction. A separate encoder reads the reference Construction Intermediate Representation, which is a text document with a header, a list of named parameters with ranges, an ordered list of feature records with 44 possible operation tokens like OP_HOLE_PATTERN or OP_EXTRUDE, and an end marker. Parameters can be referenced symbolically, for example as @hole_radius, rather than hard-coding numbers. Training pulls the four query embeddings toward the embedding of the correct construction using a contrastive loss.
Predict construction without seeing the answer. At test time the reference plan is gone. A small modality-specific network called a prior predicts where the query should sit in construction space.
Turn that continuous hunch into a text plan. A prefix adapter converts the predicted vector into four continuous prompt vectors that steer a shared plan decoder based on Qwen2.5-Coder-1.5B. This is like prefix tuning, except the prefix is predicted from the input rather than fixed. The decoder then writes the text plan.
Turn the plan into code. A second adapter on the same base model reads the plan plus the allowed observation block and writes a Python script that must define a top-level variable named part. Three fixed string rewrites fix common alias mistakes, then the Flluma/OpenCASCADE kernel tries five gates in order: parse as Python, execute, build a solid, pass kernel validity, export STEP.
Training runs in stages: align representations, fit priors, train plan decoder first on STEP latents then continue on all four, train code decoder first on reference plans then continue on a 70% reference plus 30% predicted-plan mix to reduce exposure mismatch. Everything reported is single-seed on a single 16GB GPU.
What they showed — results, honestly, with limitations
Construction help is real, but text itself is not the magic.
Removing the construction pathway on STEP input drops build success from 70.0% to 35.4% and STEP export from 66.2% to 34.6%, even though plain Python syntax actually goes up from 94.5% to 97.6%. So the gain is in buildable models, not just parseable code.
But a decoder fed the continuous construction vector directly, with no text plan, also builds strongly: 86.2% in a one-epoch run and 93.6% with longer training. On the same 500 STEP samples with matched exposure, text-plan versus continuous conditioning shows no detected material loss in per-part shape fidelity. The paired F-score difference is 0.0040 with interval spanning zero and inside the author's empirical practical margin, and similarly for Chamfer distance. The small 4.2-point build edge for text is treated cautiously because it is single-seed with slightly different update counts and different interfaces.
What text buys is diagnosability. Operation-set and operation-order agreement between generated and reference plans predicts build and export success with ranking scores around 0.73, versus about 0.48, near chance, for a single-number latent similarity. Corrupting the construction prefix wrecks plan agreement and multiplies Chamfer error from 2.84 to 22.11 square millimeters while leaving build rate statistically unchanged. In short, a wrong recipe can still run.
Observation channels carry scale. Suppressing the extra observation at inference leaves STEP build at 58.0% versus 71.4% and text roughly flat, but median produced-to-target size ratio collapses from 1.000 to about 0.64 for STEP and 0.62 for text. The construction hunch gives structure; the observation note gives absolute metric size.
Retrieval is deceptively strong because the procedural corpus repeats operation skeletons. Nearest-neighbor plans reach 95.0% to 99.5% build in-distribution and still 97.6% to 100% when whole template families are held out, with median operation-set match of exactly 1.0. Yet on jointly scorable held-out parts, generation is more faithful, median F-score 0.162 versus 0.111 for retrieval, while retrieval wins on all-input average because it scores more often. Coverage and fidelity answer different questions.
The boundary is external point clouds. On 400 human-authored Fusion 360 parts, the autonomous MIRAGE point-cloud path yields measurable geometry for 226 of 400 versus 392 for released CAD-Recode, with paired median Chamfer 55.78 versus 0.117 and paired volumetric overlap 0.0604 versus 0.9226. Retrieval recovers coverage to 392 of 400 but not accuracy. Parametrically, 99.8% of 1,608 edits to exposed knobs still rebuild and 82.2% visibly change shape, but only 29.3% of geometric numbers on average are actually reachable through declared parameters.
Limits the author states plainly: single training run so seed variation is unknown, procedural not human histories, encoders differ in capacity so modalities cannot be ranked fairly, point-cloud normalization removes scale by design, and only single solids are handled, not assemblies or full sketch constraints.
Why this might matter to me
For spatial reasoning in language and vision models, this is a clean test of whether a model truly internalizes construction structure versus parroting runnable syntax. The prefix-shuffle and observation-suppression experiments separate those channels: structure without scale still runs but lands at the wrong size; scale without correct structure runs but lands at the wrong shape. That dissociation is directly relevant if you probe world models for metric versus topological understanding.
For world models and 3D generation, the lesson is to stop collapsing run, match, and edit into one score. MIRAGE-CAD insists on three axes — executable validity as gates, geometric fidelity conditional on scoring plus separate coverage, and parametric responsiveness plus reachability — and shows they diverge in practice. That evaluation pattern ports well to any text-to-3D or image-to-3D pipeline.
For text-to-CAD and parametric geometry, the practical takeaways are that a shared construction anchor can serve four modalities with one plan and one code decoder, that training the code writer on the noisy plans it will actually see matters more than adding updates, and that parameter reachability, not just rebuild survival, is the current bottleneck. If you care about editable scenes rather than frozen meshes, future work the paper points to — construction-space novelty splits, richer B-Rep encoders, scale-aware point encoders, and evaluation against sets of valid histories rather than one reference — is where to push.
Terms worth knowing
Construction Intermediate Representation: the middle text document with parameters and ordered operations. It records intended structure, not Python implementation details like imports or variable names.
Construction latent: a continuous 512-dimensional vector hunch about construction. It has no words, only position in a shared space where similar constructions sit near each other.
Prefix adapter: a small network that turns that vector into continuous prompt vectors prepended to the language model. It conditions generation without using vocabulary tokens.
Plan decoder and code decoder: two differently adapted versions of the same code language model. One writes the construction plan, the other writes executable Python.
Executable validity gates: five sequential checks — parses, executes, builds a solid, passes kernel validation, exports STEP. Passing says the program is admissible, not that the shape is correct.
Chamfer distance, F-score, IoU: three shape-match measures used here. Chamfer averages nearest-point squared distances between sampled surfaces; F-score at 1% counts what fraction of points match within 1% of the reference size; volumetric IoU measures overlap of solid volumes after normalization.
Parametric responsiveness and reachability: responsiveness asks whether tweaking a declared knob keeps the program valid and visibly changes geometry; reachability asks what fraction of all geometric numbers in the code are even controlled by such knobs.
Retrieval baselines: methods that copy the plan from the closest training example instead of generating from the query. Strong here largely because different template families reuse the same operation skeletons.