The one-paragraph version
DrawingsDreamer is a single language-model system that writes engineering drawings directly as text. Instead of generating pixels, it generates the vector instructions behind four standard views of the same object — the Front, Right, and Top orthographic views plus one Isometric 3D-looking view — using only lines and curves. To do this, it turns continuous drawing coordinates into discrete number tokens, puts the numbers first and the label for what they mean afterward, and trains one small tuned language model to handle many jobs, from filling in a missing curve to inventing all four views from scratch. Trained on drawings projected from about 178,000 CAD models, it is reported to beat general models like GPT-4o, an SVG model called OmniSVG, and a view-synthesis model called Zero123 on both code correctness and visual similarity, though it still sometimes miscounts holes and distorts very thin shapes.
The problem they're solving, and why it matters
Scalable Vector Graphics, meaning a drawing format that stores shapes as instructions like move, line, and curve rather than as pixels, are central to modern Computer-Aided Design, meaning computer software for designing manufacturable objects. An engineering drawing is not a pretty illustration. It is a manufacturing contract: line endpoints must meet exactly, holes must appear in the same place in every view, and sizes must stay consistent across views.
The authors argue that existing tools miss this need. Optimization-based vectorizers that iteratively tweak curves to match an image are slow and produce tangled, uneditable anchor points. Recent autoregressive models, meaning models that predict one token at a time, and Large Language Model pipelines, meaning systems built on large text-prediction models, have scaled well for icons, fonts, and art but are trained on artistic data with short contexts. They do not enforce long-range coordinate agreement or cross-view alignment. At the same time, most Computer-Aided Design generation research builds 3D solids, meshes, or boundary representations directly, meaning the watertight mathematical surface of a solid, and leaves the task of generating the 2D multi-view drawings themselves largely unexplored. DrawingsDreamer aims to fill that gap with a unified, editable, vector-first generator.
The key idea, in plain words
Treat the whole set of engineering views as a foreign language to be written out, not as pictures to be painted.
Imagine describing a rectangle to a friend by reading out numbers first and then saying what they were for: “24 10 end of move, 104 10 end of line.” That is the postfix idea — coordinates before the label. The intuition is that the model should settle where points are in space before it commits to calling them a move, line, or curve, which stabilizes long strings of numbers.
Then teach in stages, like school. First have the student repair tiny local damage, such as one missing loop inside a view, to learn clean syntax and closed shapes. Next have them complete missing whole views when most of the object is already given, to learn that the Front, Top, Right, and Isometric views must agree. Finally ask them to translate between Isometric and orthographic views or invent everything from nothing, which requires global 3D reasoning. One model sees all these instruction types, with unconditional generation kept always present as an anchor so it does not forget the overall distribution of drawings.
Crucially, there is no image encoder. The authors deliberately avoid rendering pixels and feeding them through a vision network, relying only on discrete coordinate tokens and the language model’s sequence reasoning.
How it actually works, step by step
Data creation comes first. Starting from a CAD program, meaning a script that builds a solid, they reconstruct the 3D solid, render the three orthographic projections plus one isometric projection, and discard cases with missing or empty views. All drawings are placed on a large normalized canvas with size 1000. The three orthographic views share one scale factor so their relative sizes stay truthful to each other, while the isometric view is scaled on its own because its projected size is different. Closed contours are grouped into loops, meaning closed rings like the outline of a plate or the circle of a hole, and sorted into a consistent spatial order to remove arbitrary export ordering. Only three drawing actions are kept: move to start a new subpath, line to draw straight, and cubic Bézier curve to draw smooth curves with two control points. The close-path command is not given its own token; it is rewritten as lines back to the start.
Next is quantization and serialization, meaning turning smooth positions into a limited vocabulary. Each continuous coordinate is mapped into 256 integer bins, essentially rounding position onto a coarse grid to make it language-model friendly. Coordinates are grouped by small labels like end of move, end of line, and end of curve. Those primitives are grouped into paths closed by an end-of-path marker. Paths are grouped into a full view closed by a view marker such as end of front. The four view strings are stored in the order Front, Right, Top, Isometric.
Then comes multitask prompting. Every training example is instruction plus context plus target. For example, no context and all four views as target means unconditional generation; isometric only as context and the three orthographic views as target means isometric-to-orthographic; the reverse means orthographic-to-isometric; isometric plus one orthographic view as context and the other two orthographic views as target means partial completion; and all available paths minus one hidden path as context with that hidden path as target means path-level masking for local repair.
Training uses a pretrained Llama-family causal language model, specifically LLaMA-3.2-3B, with Low-Rank Adaptation, meaning only a small set of added adapter weights is trained while the base model stays mostly frozen. Here the rank is 8 and scaling factor alpha is 32. Loss is computed only on the target tokens, so instructions and context do not directly create gradients but condition the prediction. Sampling follows the microscopic-to-macroscopic curriculum described above: early emphasis on masked-path repair, middle emphasis on partial view completion, late emphasis on full translation, with unconditional generation active throughout. The paper does not publish the exact sampling weights or transition schedule, so that detail is unclear.
What they showed — results, honestly, with limitations
The evaluation covers syntactic accuracy, visual similarity after rasterizing vectors to 256 by 256 images, and geometric error. Command accuracy means whether the predicted action type matches at each position. Parameter accuracy means whether numbers match within a tolerance. Visual metrics are peak signal-to-noise ratio, structural similarity, and learned perceptual similarity, which respectively capture pixel error, structural preservation, and perceptual closeness. Chamfer Distance, meaning average nearest-point mismatch between two sets of drawing pixels, captures geometric misalignment. For unconditional generation they report distribution metrics: Jensen-Shannon Divergence for overall occupancy similarity, Coverage for diversity, and Maximum Mean Discrepancy for average best-match quality.
Reported numbers are strong. For isometric-to-orthographic, DrawingsDreamer reaches about 89.8 percent command accuracy and 80.8 percent parameter accuracy, versus about 46.4 and 35.5 percent for five-shot GPT-4o, with much higher visual scores and much lower Chamfer Distance. For orthographic-to-isometric it reaches about 73.5 percent command and 68.4 percent parameter accuracy versus about 11.3 and 56.7 percent for GPT-4o. It also outscores Zero123 and OmniSVG on image metrics. For partial completion given isometric plus front to predict top and right, it reaches about 92.8 percent command and 82.5 percent parameter accuracy versus 65.5 and 30.1 percent for GPT-4o. For unconditional generation it roughly doubles Coverage and halves divergence measures compared with GPT-4o.
Ablations support the design: removing path masking, switching postfix back to conventional prefix ordering where the label comes first, or using random task sampling instead of the curriculum all degrade accuracy or geometry. Unified multitask training generally beats training separate specialists. A larger 7-billion-parameter variant further improves coordinate accuracy, but the appendix numbers for the 3-billion baseline do not exactly match the main table in one column, so small cross-table comparisons should be treated cautiously.
Limitations are stated plainly. The model sometimes hallucinates topology, such as missing or adding a hole in a repeated pattern, likely because pure attention has no explicit counter. It also struggles with flat, thin, or elongated parts, where quantization drift causes wrong lengths, thicknesses, or radii and breaks cross-view alignment. Evaluation is only on drawings derived from DeepCAD with a 90/5/5 split, so generalization to real industrial drawing styles, dimension annotations, tolerances, and messier topology is unsupported. Claims about reproducibility and statistical stability are asserted but error bars and exact curriculum weights are not shown.
Why this might matter to me
For spatial reasoning in LLMs and VLMs: this is evidence that a text-only language model, with no raster encoder, can learn nontrivial projective consistency if the representation and curriculum force it. It suggests that weakness on spatial tasks may come less from lack of vision and more from prefix ordering, arbitrary path order, and training that jumps straight to hard global inference.
For world models: the setup is a compact, exact world model test. Given one view, predict the other views under deterministic geometry. Success requires an internal notion of a single 3D object seen from fixed cameras, and failure — miscounted holes, drifting thin parts — reveals where implicit memory fails without explicit object slots or geometric constraints.
For text-to-CAD and parametric geometry: the output is already parametric and editable — moves, lines, and cubic curves with quantized coordinates — rather than pixels to be vectorized later. That maps directly to sketch-and-extrude workflows and to constraint inference, though the current vocabulary is tiny and has no dimensions, tolerances, or construction history.
For 3D generation and scene representation: the paper treats multi-view 2D drawings as a complete 3D description, an alternative to meshes, neural fields, or boundary representations. If vector views can be generated consistently, they could become a lightweight, interpretable bridge to 3D reconstruction, but the work does not yet show that its drawings reliably convert back into valid solids.
Terms worth knowing
- Scalable Vector Graphics: a drawing stored as instructions for shapes rather than colored pixels, so it stays sharp at any size and remains editable.
- Orthographic vs. isometric views: orthographic views are straight-on projections like Front, Top, and Right with no perspective distortion; isometric is a 3D-like angled view that shows three faces at once.
- Postfix tokenization: writing numbers before their label, for example coordinates then end-of-curve, rather than label before numbers.
- Quantization: rounding continuous coordinates onto a fixed grid, here 256 bins, so a language model can treat positions as vocabulary items.
- Loop / path / view hierarchy: points form primitives, primitives form a continuous path, paths form closed loops where possible, loops form one view, views form the full drawing.
- Low-Rank Adaptation: a parameter-efficient tuning method that freezes most of a large model and trains small added matrices.
- Curriculum learning: ordering training from easy local tasks to hard global tasks rather than sampling uniformly.
- Command vs. parameter accuracy: whether the model predicts the right action type versus whether it predicts the right numbers within tolerance.
- Chamfer Distance: a geometric error measured as average nearest-point distance in both directions between predicted and true drawing pixels; lower is better.
- DeepCAD: a large dataset of CAD construction sequences used here as source solids for rendering multi-view drawings.