The one-paragraph version
JoinABLe is a system that learns how two mechanical CAD parts should be joined together, the way a designer would join them with a joint constraint in Fusion 360. Given two parts in isolation, with no object name, no part labels, and no human pointing where to connect, it predicts which surface patches should line up — for example, that this bolt shaft should go into that hole — and then searches for how to slide and twist them into a snug, non-overlapping pose. It does this by treating each part as a graph of its faces and edges, learning embeddings for those pieces, and scoring every possible cross-part pairing as a candidate joint. Trained only on the partial, messy joint selections left behind by real designers, it reaches 79.53% top-1 joint-axis accuracy, essentially matching a human CAD expert at 80% on the same task, and it holds up much better than heuristics or point-cloud methods when there is no obvious hole to match.
The problem they're solving, and why it matters
Physical products — furniture, engines, electronics — are assemblies of many 3D parts. In mechanical CAD, designers do not just drop each part at global coordinates. They define what is called a joint, meaning a relative constraint between a pair of parts that says how they are aligned and how they are allowed to move.
A concrete example helps: a bolt constrained to a hole, then a nut constrained to the bolt. The designer picks a face or edge on each part, the software derives an axis from that pick, and the two axes are brought together.
This is powerful because it preserves what designers call design intent: if you change a dimension later, the relationships hold. But it is also slow. The paper cites that roughly one third of CAD time is assembly work, and many real assemblies end up with missing or half-defined joints.
Prior learning work mostly takes what the authors call a top-down approach: look at a whole set of parts from a known object class like chairs, and predict an absolute position for each part. That relies on heavy annotation — object class labels, part class labels, consistent orientation, manual segmentation — which real manufacturing CAD does not have. Real parts are segmented by how they will be made, not by semantic category.
JoinABLe asks a bottom-up question: given just a pair of parts, can we automatically assemble them locally, without knowing the global design, without class labels, and without human guidance? If you can solve the pair case reliably and output an editable parametric joint, you have a building block for assembly synthesis, robotic assembly and disassembly, motion prediction, and reuse of existing components.
The key idea, in plain words
Use the weak supervision already sitting inside parametric CAD files.
When a designer creates a joint, they leave a trace: they selected this circular edge on Part A and that cylindrical face on Part B, and the file stores the resulting axes and pose. Those traces are incomplete — designers skip joints, and they only create the joints needed for one specific design, not every plausible connection. But they are free labels from domain experts.
The key move is to reframe joint creation as picking discrete entities, not regressing numbers in space. A part in what is called boundary representation already has a finite list of faces and edges. So instead of predicting a 3D point and direction from scratch, which can end up floating slightly off the geometry, predict which face on Part 1 goes with which face or edge on Part 2. Once you know the pair, you can look up the exact geometry to get a clean axis.
To do that, the authors build a graph for each part, learn a local neighborhood-aware description for every face and edge, and then score all cross-part pairs. That scoring is what they call link prediction, a term borrowed from social networks meaning predicting which nodes should be connected. The highest-scoring pair gives you the axes to align; a separate search step then finds the slide, twist, and flip that makes the parts touch without interpenetrating.
How it actually works, step by step
Start with input. The input is two parts in boundary representation format. Boundary representation, usually shortened to B-Rep, means a solid is described as a watertight collection of trimmed parametric surfaces — planes, cylinders, cones, spheres, and so on — bounded by parametric curves like lines, arcs, and circles, all connected by topology.
Step 1: Turn each part into a graph. Here graph means nodes plus connections. Each node is one B-Rep face or one B-Rep edge. Two nodes are connected if those faces or edges touch each other in the solid. Node features are simple facts readily available in the CAD kernel: for a face, what kind of surface it is and whether it is reversed; for an edge, what kind of curve it is, how long it is, and whether it is reversed. The paper tested richer features like face area and dihedral angle but dropped them because they hurt validation performance.
Step 2: Encode with a shared network. The encoder, meaning the network that turns raw features into learned vectors, is Siamese-style, meaning the same weights are used for both parts. Faces go through one small neural network, edges through another, then the results are concatenated. Then a two-layer message-passing network, specifically a Graph Attention Network version 2, lets each face or edge gather context from its neighbors. The output is an embedding, meaning a learned numeric description, 384 numbers long, for every face and edge.
Step 3: Predict the joint axis by scoring pairs. Form what they call a joint connectivity graph, which in words is simply every possible pairing of a node from Part 1 with a node from Part 2. If Part 1 has n entities and Part 2 has m entities, that is n times m candidates, visualized as a matrix. For each pair, concatenate the two embeddings and pass them through a three-layer network to get a single score for joint or not-joint.
Training has to cope with extreme imbalance and what is called positive-and-unlabeled learning. Positive-and-unlabeled learning means you know some positives are true joints, but the rest are not confirmed negatives — some could be valid joints the designer just never made. A gear with thousands of entities might have only one labeled joint. The authors handle this three ways. First, joint consolidation, meaning grouping all known joints between the same pair of part shapes into one joint set so the network does not see contradictory labels. Second, joint equivalents, meaning identifying neighboring faces or edges that share the same axis as the labeled pick — for example the cylinder wall around a labeled circular lip — and counting them as correct at evaluation time because they define a collinear axis. Third, unambiguous evaluation sets, meaning removing geometrically similar but unlabeled sibling entities, like a second identical hole, from validation and test so accuracy is not penalized for guessing a plausible but unlabeled hole.
The loss, meaning the training objective, has two parts. In words: first, a global cross-entropy that pushes true joints to score higher than all other pairs at once; second, a symmetric row-and-column version that contrasts joints within each row and column to focus on harder near-misses in the sparse matrix.
Step 4: Search for joint pose. Pose here means the final rigid placement: where the parts sit in space. The network gives you axes, which you align. Three secondary parameters remain: offset, meaning distance to slide along the axis; rotation, meaning twist around the axis; and flip, meaning which direction the axis points. They optimize these with a derivative-free optimizer called Nelder-Mead, guided by a cost that rewards large contact area, meaning surfaces touching, and penalizes overlap volume, meaning interpenetration. In words, pull parts together until they touch a lot but do not sink into each other. At test time they try the top 50 axis predictions and keep the pose with lowest cost.
What they showed — results, honestly, with limitations
They also release the Fusion 360 Gallery assembly dataset: about 8,251 assemblies with 154,468 parts, plus joint data with 32,148 joints between 23,029 different parts, grouped into about 19,156 joint sets. It includes joints, contacts, holes, materials, and hierarchy. It is explicitly weak supervision, not exhaustive annotation.
For joint-axis prediction, on their cleaned test set, their graph method gets 79.53% top-1 accuracy. For comparison: a human CAD expert given 100 randomly rotated pairs with no assembly context gets 80%; a hand-crafted heuristic that matches similar entity types and radii gets 71.39%; a UV-grid plus CNN variant gets 65.21%; random guessing over B-Rep entities gets 21.55%; and adapted point-cloud joint methods get about 10.59% for dense regression and 4.28% for discrete classification, despite using more parameters. Their model uses about 1.3 million parameters, fewer than the point-cloud and grid baselines.
The gap widens where heuristics struggle. On samples without any holes — no obvious fastener-to-hole cue — they get 76.59% versus 64.97% for the heuristic, an 11.62 point lead. On samples with holes they get 80.15% versus 72.74%. The authors are clear this is the important test: matching bolts to same-radius holes is relatively easy; flat-to-flat, shaft-to-face, or unusual pairings are not.
For joint-pose prediction, measured by average chamfer distance, meaning average nearest-point distance between predicted and ground-truth assemblies where lower is better, network alignment alone gets 0.0627, adding search improves to 0.0580, and a top-down pose-regression baseline gets 0.0700. Search helps fix overlap and sometimes recovers from a wrong axis pick. Qualitatively, some “errors” look plausible — for example a belt buckle in a different open-closed state — because ground truth stores only discrete designer states, not all valid continuous states. So chamfer distance here is approximate, as the authors state.
Limitations to take seriously: only rigid joints between pairs are studied, not flexible joints or full multi-part planning; reliance on B-Rep data means it is untested beyond mechanical CAD; no physics or overlap-avoidance loss during training; large graphs over 950 combined nodes are skipped in training due to memory; and the multi-part demo assumes a known assembly sequence from the dataset and can compound early mistakes. The paper also notes that one related concurrent dataset, AutoMate, is larger, though theirs adds hierarchy, contacts, consolidated joints, and cleaned evaluation splits.
Why this might matter to me
For spatial reasoning in vision-language models, this is a reality check and a resource: even a human expert is only 80% correct when placing two isolated parts without object context, and performance collapses on parts without holes. That suggests future assistants need to reason about local geometric affordances — coaxiality, matching radii, edge length as a proxy for size — not just recognize object categories. The sibling-hole ambiguity is a clean example of referential uncertainty language models will face when asked “which hole?”
For world models, the joint-plus-search split is instructive. The network proposes symbolic attachments, then a contact-versus-penetration cost grounded in signed distance fields verifies the pose. That is a small world model loop: propose a constraint, roll out the geometric consequence, score physical plausibility. Extending that cost to stability, gravity, or articulation limits is a natural next step.
For text-to-CAD and parametric geometry, the lesson is to predict over discrete B-Rep topology rather than regressing raw coordinates. Because predictions index actual faces and edges, the output converts directly back into editable joints with origin and direction derived from surface type — cylinder axis, circle center plus normal, plane centroid plus normal. That preserves editability and design intent, which pure mesh or point-cloud generation loses. Language-conditioned joint suggestion — “fasten these with a rigid joint” — could plug straight into this link-prediction head.
For 3D generation and scene representation, the bottom-up versus top-down contrast matters. Top-down generators often leave small misalignments because they predict absolute poses. Bottom-up assembly predicts relative constraints first, which snap cleanly. A promising hybrid, which the authors sketch, is top-down for coarse layout and object composition plus bottom-up for precise contacts — essentially scene graphs where edges are parametric joints, not just proximity.
Terms worth knowing
- B-Rep / boundary representation: the industry-standard way to store a solid as parametric faces bounded by edges and vertices, plus how they connect. The source of exact geometry for joints.
- Joint: a designer-defined relative constraint between two part occurrences that sets degrees of freedom and rest pose, for example rigid, revolute, slider, cylindrical, pin-slot, planar, or ball.
- Joint axis: the per-part origin point plus direction vector derived from the selected face or edge, for example a circle gives its center and normal. Aligning the two axes is the first step of assembly.
- Joint pose: the final rigid placement after alignment, here controlled by offset along the axis, rotation about the axis, and flip of direction.
- Top-down vs. bottom-up assembly: top-down predicts global positions for all parts at once, often using class priors; bottom-up predicts local pairwise connections and builds up.
- Weak supervision: training from byproduct labels — here designer picks in CAD files — that are incomplete and not created for machine learning.
- Positive-and-unlabeled learning: a setting where labeled positives are known to be true but unlabeled examples might be hidden positives or true negatives.
- Joint set, equivalent, sibling: a joint set groups all known joints for the same part pair; an equivalent shares the same axis as the label and counts as correct; a sibling is a geometrically similar but unlabeled entity, like a duplicate hole, removed from clean test sets to avoid unfair penalties.
- Link prediction: framing the task as scoring which nodes across two graphs should be linked — here which face-edge pair forms the joint.
- Chamfer distance: average nearest-point distance between two point-sampled shapes, used here as an approximate pose error because multiple poses can be valid.