All summaries · 2026-10-05

Windfoil: Closed-Form Coverage for Real-Time and Differentiable Vector Graphics

Matt DesLauriers

arXiv:2610.02468 · abstract · pdf · summarized by meta/muse-spark-1.3-contributor on 2026-10-05

The one-paragraph version

Windfoil is a new way to turn vector shapes into pixels, and to fit vector shapes by gradient descent, using one shared math trick. The trick is to compute, in closed form with no sampling, the average inside-or-outside score of a shape over each pixel's little square. In plain terms: instead of asking "is the pixel center inside the curve?" which gives jagged edges, it asks "what fraction of this pixel square is covered by the curve?" and it answers that exactly from the curve equations. The paper implements this in WebGPU so it runs in a browser on a consumer laptop as well as on a server, for both real-time display and differentiable optimization. It reports closer match to an ideal box-filtered image than Skia or Slug at similar speed, and much faster per-step image fitting than DiffVG or Bézier Splatting at equal or better quality, scaling to tens of thousands of shapes.

The problem they're solving, and why it matters

Vector graphics — meaning text, icons, emojis, user-interface elements, illustrations defined as curves rather than a fixed grid of pixels — must be rasterized, meaning converted to pixels for display, and ideally with smooth antialiased edges rather than stair-steps. The standard test for inside-or-outside is the winding number, meaning roughly how many times the shape boundary winds around a point, counted with direction. A fill rule, meaning a rule like nonzero or even-odd that turns that count into inside versus outside, then decides coverage.

The difficulty is twofold. For display, you want crisp, resolution-independent edges at 60 frames per second on a GPU, without blurring, shimmering, or expensive tricks like multisampling, meaning taking many samples per pixel, or flattening curves into many short straight lines, or baking distance fields into textures that only work over a limited zoom range. For machine learning, you want differentiable rendering, meaning a renderer that not only makes an image but also tells you how to nudge each curve control point, color, or opacity to make the image closer to a target. But inside-or-outside is naturally not differentiable: moving a curve a tiny bit usually does not change a pixel center from inside to outside, so the derivative is zero almost everywhere.

Existing systems split these worlds. Production renderers like Skia use one coverage model for display, while differentiable renderers like DiffVG use Monte Carlo sampling, meaning random samples to estimate coverage and gradients, or smooth approximations that do not match what you will finally see. Windfoil aims to use one exact, shared calculation for both display and optimization.

The key idea, in plain words

Imagine a pixel as a tiny square window. Instead of testing only its center, average the winding number over the whole square, then decide coverage. That average is called the box-filtered winding, where box filter means a simple average with equal weight over a rectangle.

The insight is that this area average does not require sampling the interior. By Green's theorem, meaning a math tool that turns an area integral into a walk around its boundary, you can compute it only from where the curves cross that pixel square. Think of sweeping a horizontal strip across the pixel: each right-going curve adds the width from the left edge to the curve, each left-going curve subtracts it. Add those strips up and divide by pixel area, and you have the exact average winding.

Because that calculation is a continuous formula in the curve positions, it is also differentiable in closed form when the filter has positive width: as an edge slides across a pixel, coverage changes smoothly from 0 to 1, so there is a useful slope to follow with gradient descent. One formula therefore serves both crisp display and gradient-based fitting.

How it actually works, step by step

Shapes are represented as closed directed contours of quadratic Bézier curves, where quadratic Bézier curve means a curve segment defined by two endpoints and one middle control point that pulls the curve, and closed contour means the segments loop back to the start with a direction that defines positive versus negative winding.

On the CPU as preprocessing, each quadratic is split at its horizontal and vertical extremes into at most three monotone pieces, where monotone means always moving forward in x and y without turning back, so it can cross a pixel edge at most once. This makes the per-pixel math simple and bounded.

On the GPU per pixel, the system clips each nearby monotone piece to the pixel's vertical range, solves quadratic equations to find where it crosses the left and right edges of the pixel square, and integrates the inside portion exactly. A part fully left contributes nothing, a part fully right contributes full width times height spanned, a part inside contributes a polynomial integrated in closed form. Sum those, divide by pixel area to get the averaged winding, then apply the fill rule: nonzero, meaning any non-zero average counts as covered up to 1, or even-odd, meaning coverage is distance to the nearest even integer. Then multiply by opacity and composite in painter's order with source-over blending, meaning drawing back-to-front with transparency like stacked paint.

Two backends accelerate this. For display, shapes are drawn as instanced quads, meaning rectangles that bound each shape and trigger fragment shader work only inside, padded for antialiasing. Curve data is shared across instances, helpful for repeated text glyphs. To avoid checking every curve, monotone pieces are filed into horizontal row bands, meaning buckets spanning vertical ranges; a fragment reads only bands touched by its pixel. Unlike Slug's dual horizontal-plus-vertical approach, one horizontal axis suffices because the integral sweeps horizontally. Bands with more than four pieces are sorted right-to-left so traversal can stop early; horizontal pieces contribute zero and are skipped. A minification guard switches tiny shapes, roughly under four pixels in bounding box, to a precomputed ink profile.

For differentiation, a uniform bound of three slots per curve keeps GPU work regular. The scene is divided into tiles, meaning small image blocks each with a list of potentially contributing shapes in draw order. A forward compute pass renders and, for image fitting, computes mean-squared error to a target. A backward pass is an explicit analytic vector-Jacobian product, meaning hand-written code that propagates the image error backward through compositing, opacity, fill rule, and winding integral to control points, then a final pullback maps to user parameters like center or radius before an Adam optimizer step, where Adam means a standard adaptive gradient descent method. The paper also demonstrates non-pixel losses by sending the rendered image to a Python server that returns a CLIP loss gradient, where CLIP means a vision-language model that scores image-text similarity.

What they showed — results, honestly, with limitations

For correctness, against a supersampled box-filter reference computed at 24x resolution, the paper reports mean absolute coverage error of 1.2 x 10^-4 for Windfoil, in the range of the reference's own sampling error, versus 1.1 x 10^-3 for Slug and 1.6 x 10^-3 for Skia, about 9x and 13x larger. For speed versus Slug, Windfoil is reported 1.2 to 2.6x faster under magnification and 5 to 6x faster under strong minification thanks to the ink profile, but Slug leads by 1.2 to 2.8x in the middle zoom range. Both stay in 60 fps budget on an Apple M2 MacBook Air for thousands of glyphs. The paper is explicit that the comparison disables its minification guard and uses its own WebGPU port of Slug.

For differentiable image fitting, on the 24-image Kodak set at 768x512 with 512 shapes and 800 steps, mean PSNR, meaning a logarithmic image quality score where higher is better, is 26.60 dB for Windfoil, 26.41 dB for DiffVG, and 22.80 dB for Bézier Splatting, with mean step times 3.2 ms versus 336 ms versus 29 ms on an NVIDIA RTX 2000 Ada. Median parity speed-up, meaning baseline total time divided by Windfoil time to first reach that baseline's final quality, is reported as 243x over DiffVG on 20 of 24 images where it reaches parity, and 196x over Bézier Splatting on all 24. Gaps grow with shape count; at 4,096 shapes Windfoil reaches 26.77 dB versus 21.31 and 22.61 dB, and at 50,000 shapes it reaches 25.46 dB where baselines were not tested. Each benchmark was run once with the same seed, so variation is unclear.

Limitations are stated plainly. Applying the fill rule after averaging is exact along isolated edges but not at self-crossings, overlapping contours, or multiple transitions in one pixel — the paper shows a bright vertical seam rendered as a hole where winding cancels. The box is axis-aligned, so exact under translation and scale but approximate under rotation, shear, or perspective. Each shape is filtered before compositing, not the final composite. Wider tent and cubic B-spline filters use quadrature approximations, not closed form, and cost more. Only closed quadratics are supported directly; strokes, cubics, gradients, textures, and clipping masks need conversion or are unsupported in the differentiable path. Gradients use fixed-point integer atomics with quantisation and overflow risk.

Why this might matter to me

Your interests center on machines that reason about space, geometry, and scenes, not just pixels.

For spatial reasoning in LLMs and VLMs and for world models, Windfoil offers a compact, editable, resolution-independent scene representation with exact gradients. A vision-language loss like CLIP can directly nudge parametric curves — the paper shows text-to-sketch scenes of 256 translucent rectangles and tests up to 50,000 to 100,000 shapes — which suggests a path from "image that looks like a prompt" to "structured drawing that means the prompt," preserving part identity, occlusion order, and inside-outside topology that pixel diffusion loses.

For text-to-CAD and parametric geometry, the formulation is philosophically close: low-dimensional control points plus a differentiable forward model plus constraints. The monotone splitting, row-band acceleration, and coarse-to-fine annealing from wide to narrow filter are reusable patterns for fitting precise outlines, fonts, floorplans, or sketch extrusions without sampling noise. The stated limitation about fill-after-filter at self-intersections is directly relevant to CAD, where overlapping loops and boolean operations are common and must be handled exactly.

For 3D generation and scene representation, think of Windfoil as the 2D analogue of Gaussian splatting or analytic volumetric rendering: a primitive that is fast to rasterize, smooth to optimize, and identical at train and display time. That shared coverage model avoids the common mismatch where you optimize a soft proxy but display a hard mesh. Extending the box-filtered winding idea to perspective-correct footprints, occlusion-aware filtering after compositing, or differentiable stroke-to-fill conversion could inform differentiable vector, curve, or wire-art priors for 3D, as in multi-view wire art cited by the paper.

Terms worth knowing