The one-paragraph version
This paper tackles a deceptively simple task: ask a language model to write Scalable Vector Graphics code — the text commands that draw an icon or illustration — from a plain English prompt like "a blue camera emoji." Because one prompt can be drawn correctly in hundreds of different ways, there is no single right answer to copy, which breaks both normal testing and normal training. The authors first show that the usual automatic scores, borrowed from photo evaluation, do a poor job of predicting what people actually prefer for this stylized vector art. A checklist-style judgment works far better: ask a vision-language model to grade the rendered image on several named dimensions. They then turn that finding into a training method called RULER. For each prompt, a strong model writes a custom six-question checklist tailored to that prompt, a separate judge model grades the learner's drawings question by question, and those grades become the reward for reinforcement learning. Trained this way, a relatively small 8-billion-parameter model improves dramatically and matches much larger models on human-aligned quality, without needing paired perfect drawings or human preference labels.
The problem they're solving, and why it matters
Scalable Vector Graphics, usually shortened to SVG, is code that becomes a picture. Unlike a photograph made of pixels, an SVG file is a list of instructions like "draw a blue rounded rectangle here, add a yellow circle there." That makes it editable, scalable without blur, and attractive for icons, illustrations, and interfaces.
The core difficulty the paper focuses on is open-endedness. If you ask for "a minimalistic cat head with triangular ears," countless different line drawings would all count as correct. There is no one ground-truth image to compare against.
That creates two linked problems:
First, evaluation is unreliable. Common automatic scores — meaning a text-image similarity score, an aesthetic classifier score trained to predict what people find pretty in photos, and a human-preference score trained on photo-like generations — were tuned on realistic natural images. On flat, stylized vector art they often miss the point. The paper gives a concrete example: a broken, glitchy blue camera drawing can get a higher similarity or prettiness score than a clean, correct one.
Second, training is misled. The usual approach of fine-tuning by copying example instruction-and-code pairs just clones the style of one dataset and does not generalize. The natural alternative is reinforcement learning, meaning let the model try many drawings and reinforce the ones with high reward. But if the reward is one of those unreliable photo scores, the model learns to game the score instead of drawing better — for example, adding dense overlapping squiggles that fool the prettiness classifier while looking worse to a person. The authors call this reward hacking, meaning improving the number without improving the actual goal.
The key idea, in plain words
Instead of one mysterious overall number, judge the drawing like a teacher with a detailed report card.
The insight is that a vision-language judge — meaning an AI model that can look at an image and read text — does much better when you give it a structured checklist, called a rubric, that breaks quality into separate observable questions.
Crucially, the checklist is instance-aware, meaning custom-made for each prompt rather than the same generic form for everything. A blue camera emoji needs questions about "is the body distinctly blue, is the lens yellow, is the flash visible?" A cat-head outline needs questions about "are the ears and nose triangular, is the outline clean?" A generic checklist cannot know which details matter for this particular request.
Because the custom checklist is written from the text prompt alone, the method needs no perfect example drawing and no human labels. It converts an ambiguous visual judgment — "is this good?" — into six checkable sub-goals the learner can actually optimize.
How it actually works, step by step
Think of the task as a step-by-step writing game. The learner model sees the instruction, then writes SVG code one chunk at a time. Only when it finishes is the code rendered — meaning run through a drawing engine to produce a visible picture — and graded. That makes the design of the grade the central challenge.
Here is the pipeline the paper proposes:
1. Prove that checklist grading aligns with people. The authors collect 900 rendered drawings from three models of different strength, 300 prompts split between icons and illustrations, and get human quality scores that consider prompt fit, visual quality, composition, rendering cleanliness, and style. They compare three automatic approaches: similarity score, aesthetic score, and rubric score from an independent judge. The rubric approach correlates far more strongly with humans, both for absolute scores across all samples and for getting the ranking right within each trio of drawings for the same prompt.
2. Write a custom six-item rubric for each training prompt. A strong frontier model acts as rubric author. Given only the text prompt, it first imagines one good drawing as an internal reference, then writes six graded questions in a fixed order:
- Semantic fidelity, meaning does it mean the right thing: 1) Is the overall concept readable without reading text labels? 2) Are the major parts and distinctive cues present?
- Visual quality, meaning is the form well designed: 3) Are the silhouette, contours, and proportions clean and intentional rather than crude blobs? 4) Is the composition sensible — well scaled, centered, not cramped or cropped?
- Rendering style, meaning is it well finished: 5) Is the vector execution clean, without broken paths, stray fragments, or unfinished regions? 6) Is the style cohesive and visually interesting rather than flat and generic?
Each item has a title, a one-sentence description of the high standard for this prompt, an importance weight, and a detailed scoring guide that tells the judge what counts as excellent versus strong versus partial versus weak versus absent. The prompt explicitly forbids turning this into a reconstruction checklist that demands exact colors, positions, or path counts unless the original instruction requires them. This preserves many valid solutions.
3. Use the rubric as a dense reward. During training, the learner samples a group of drawings for one prompt — in this work, eight tries per prompt — renders each to an image, and sends the image plus the prompt plus its custom rubric to a frozen judge model. The judge returns a satisfaction score between zero and one for each of the six items. The overall reward is then a weighted average, meaning items with higher importance count more, normalized by the total weight. In words: add up weight times score for each question, then divide by the sum of weights.
4. Update with group-relative comparison. The authors use a reinforcement learning method whose name stands for Group Relative Policy Optimization. In plain terms: instead of needing an absolute sense of good, look at the eight tries for the same prompt, compute their average reward, and treat tries above the group average as relatively good and tries below as relatively bad, scaled by how spread out the group is. The model is then nudged to make the relatively good outputs more likely, with a clipping rule that prevents too-large jumps in one update. The diversity across rubric items is what makes this work — different rollouts fail on different questions, so the group comparison gives a rich learning signal.
Practical details from the appendix: the default learner is Qwen3-8B, the rubric author is Claude-Opus-4.6, the training judge is Qwen3-VL-8B kept fixed, rendering is done at 512 by 512 on white, and training uses about 30,700 filtered prompt-and-rubric pairs after discarding malformed rubrics and rubrics where even the author's ideal drawing scores below 0.9.
What they showed — results, honestly, with limitations
On two benchmarks, one for richer illustrations and one for compact icons, the full method lifts the primary human-aligned rubric score from 0.432 to 0.693 on illustrations and from 0.395 to 0.683 on icons, starting from the same 8B backbone. That surpasses dedicated SVG specialist models and matches a substantially larger general model, DeepSeek-V3, while staying competitive on the older similarity, aesthetic, and preference scores. A blinded human preference study on 150 prompts finds the method preferred over every tested baseline when ties are excluded, with non-tie win rates from about 53% against an iterative optimization method up to about 97% against a specialist coder model.
Reward design comparisons isolate why. Optimizing a combined similarity-plus-aesthetic-plus-preference reward inflates prettiness to very high values but collapses actual quality on icons, dropping the rubric score below the starting model and ballooning output length from about 0.3 thousand to 6.3 thousand tokens with overlapping decorative strokes. A shared universal checklist for all prompts helps a lot and avoids that collapse, but still trails the custom per-prompt checklist by about 0.03 on illustrations and 0.09 on icons. The authors interpret this as evidence that instance-specific detail is the active ingredient, not just multi-dimensional scoring.
Ablations show all three axes help, but not equally. Removing the visual-form and composition questions hurts most, removing the rendering-finish and style questions hurts second-most, and removing the semantic questions hurts least — plausibly because the base model already aligns text and image reasonably well. A stricter alternative rubric that drops style and adds a text penalty actually performs worse and unexpectedly encourages the model to sneak readable prompt words into the image as a shortcut. The method also holds up when the learner is shrunk to 4B and when the rubric author is switched to a different frontier model, with only small changes in score.
Limitations are stated clearly. The reward still depends on two external models and can inherit their biases, especially outside the training distribution. Each training step is expensive because every rollout must be rendered and judged question by question. And the particular split into semantic, visual, and stylistic questions may not capture every valid artistic intent. The paper does not claim the rubric is perfect, only that it is substantially more faithful than the scalar photo scores for this domain.
Why this might matter to me
For spatial reasoning in vision-language models, the lesson is that decomposition beats a single number. Asking "is this layout good?" fails, while asking separately about identity, parts, silhouette, placement, cleanliness, and style gives the judge distinct visual evidence to check. The same pattern should transfer to evaluating whether a model understands left versus right, containment, alignment, or scale.
For world models, the work offers a template for grounding language in an executable visual representation. SVG here plays the role that a simulator or renderer could play elsewhere: the model writes structured code, the world renders it, and a judge scores the rendered state. That loop of propose-code, render, judge-state is directly analogous to predicting and scoring future scenes.
For text-to-CAD and parametric geometry, the instance-aware idea is especially relevant. A generic "looks good" score cannot know that a bracket needs two holes of a certain relation or that a cat head needs triangular ears. Generating requirement-specific checks from text alone — without a single ground-truth mesh — while explicitly forbidding exact coordinate matching, points toward how to reward functional correctness plus geometric cleanliness without collapsing to one reference design.
For 3D generation and scene representation, the failure modes are cautionary and useful. Optimizing prettiness or text similarity alone encourages floaters, texture soup, or text billboards that hack the metric. A per-prompt rubric that separately rewards concept readability, composition and silhouette, execution cleanliness, and stylistic coherence maps naturally onto multi-view checks, mesh integrity, and layout quality, and the group-relative update shows how to learn from diverse imperfect samples rather than requiring perfect 3D ground truth.
Terms worth knowing
SVG: Scalable Vector Graphics. A text file of drawing commands — shapes, paths, fills, strokes — that a renderer turns into a sharp image at any size. Example: a circle tag becomes a lens.
Rubric: A structured grading sheet with named criteria, descriptions of excellence, and scoring guidance, rather than a single overall impression.
Instance-aware: Customized to the specific prompt at hand. An instance-aware rubric for "compass pointing northeast" asks about needle direction; one for "beehive" asks about hexagonal cells.
Scalar metric: A single-number score like text-image similarity or predicted prettiness. Useful for ranking, but opaque about what failed.
Reward hacking: Improving the measured score while making the true goal worse, such as adding flashy scribbles that fool an aesthetic classifier.
VLM judge: A vision-language model used as an evaluator that looks at the rendered picture together with the text prompt and rubric.
Group Relative Policy Optimization: A reinforcement learning update that judges each attempt relative to the average of a small group of attempts for the same prompt, reinforcing the better-than-average ones.
Rendering: Executing SVG code to produce pixels you can see and judge. A file can be valid code but render as blank, broken, or cluttered.
MMSVG-Illustration and MMSVG-Icon: The two test collections used here — one for richer illustrative scenes, one for compact icon-style graphics.