All summaries · 2026-10-28

Zooming without Zooming v2: From Seeing Details to Connecting Evidence for Generalizable Multimodal Perception

Lai Wei, Yatai Li, Liangbo He, et al.

arXiv:2610.37037 · abstract · pdf · summarized by meta/muse-spark-1.3-contributor on 2026-10-28

The one-paragraph version

This paper argues that many hard vision-language jobs are really the same job in disguise: connecting evidence — finding small visual facts, attaching each fact to the right object, figuring out how they relate, and combining them into an answer. To teach that skill cheaply, the authors build EVIDENCECONNECT, a fully automatic way to make single-image questions that force a model to compare, add up, or chain clues across 3 to 7 related regions. They make the questions using enlarged, neatly arranged crops called privileged views, then keep a question only if two different model families agree on the answer and if hiding any region actually breaks the answer. They then train Qwen models with reinforcement learning on 20.4K such multiple-choice questions, creating Zooming without Zooming v2 (ZwZ2). Even though training never sees multi-image dialogs or robot environments, ZwZ2 improves on all 25 tested benchmarks at 4B, 9B, and 27B scales, including fine-grained seeing, spatial reasoning, long-context memory, embodied reasoning, and agent grounding.

The problem they're solving, and why it matters

Before a multimodal model can reason or act, it has to perceive correctly what is there. The paper focuses on two perception bottlenecks that unlock embodied AI and multimodal agents:

Spatial perception means recovering how things sit relative to each other, including from someone else's viewpoint. Example: is the mug to the left of the laptop from the robot's camera, or from the person's chair? Robots need this for navigation and grasping.

Multimodal long-context understanding means keeping track of evidence scattered across many images and conversation turns. Example: you showed the agent five kitchen photos earlier — which object appeared in all of them? Agents need this to retrieve past observations and resolve references like "that same plug."

Both are starved for good training data. Hand-labeling precise spatial relations across multiple objects is expensive and error-prone, and automatic labelers often hallucinate relations. Diverse, still-answerable long multi-image questions are also hard to invent automatically. Data for embodied trajectories or interactive agents is harder still. The result is no scalable, clean, checkable supervision for exactly the skills that matter most downstream.

The key idea, in plain words

The authors' insight is that fine-grained seeing, spatial reasoning, and long-context memory share one atomic step: connecting evidence.

In plain words, that means: 1. spot the relevant small facts, 2. bind each fact to the correct entity so you do not mix up two similar objects, 3. recover the link between them — spatial, numerical, or referential, 4. compose them into the final answer.

A boarding-pass example from the paper makes this concrete: read the flight number T218 on a boarding pass, use it to find the assigned gate C6, then look at gate C6 to name its service icon, a plug. You cannot answer by looking at one patch alone; you must follow the chain T218 → C6 → plug.

Crucially, it is much easier to synthesize and verify this generic connecting skill from a single rich image than to synthesize full spatial or long-context benchmarks. If you strengthen the shared skill, the authors hypothesize, gains should transfer outward — even to multi-image and agent tasks you never trained on directly. A bonus: since step 1 is itself detail perception, fine-grained perception should improve too.

How it actually works, step by step

EVIDENCECONNECT turns one high-resolution image into a verified multi-region question. Think of it as a careful assembly line:

1. Find candidate regions. Starting from detailed SA-1B photos, a strong vision-language model, meaning a model that takes images plus text as input, judges if the image can support a 3-to-7 region question. If yes, it proposes up to 18 boxes with short descriptions, favoring related groups over isolated eye-catchers. Too tiny, too huge, or heavily overlapping boxes are thrown out.

2. Group regions into a coherent task. A text-only model groups 3 to 7 regions that could support one question. Each group gets a task family that guides writing, such as counting, anomaly or extremum finding within a closed set, pairing, OCR verification, spatial chain, or viewpoint change.

3. Build a privileged evidence layout. This is the "zooming without zooming" trick. The system crops each selected region at native resolution with a small margin, then lays those crops on a compact canvas that preserves left-right and top-bottom order but deletes irrelevant background. Each crop gets a distinct outline color. This enlarged view, called the privileged view, is used only to create and check questions — the learner never sees it.

4. Write compositional questions. A generator model sees only that privileged layout plus blueprints, meaning templates for reasoning patterns. It must write questions where every region has a role: a value to aggregate, a member of a complete candidate set to compare, or a clue a later step depends on. For instance: sum counts across regions, find the outlier among all outlined tools, or fix a viewpoint then interpret directions. The generator also outputs a reasoning certificate, a short step list linking regions to answer, and what the answer would be if each region were removed.

Rule filters reject malformed ideas: unused regions, disconnected reasoning steps, leaked answers, subjective wording, bad arithmetic, or answers that would survive a region removal.

5. Verify with two independent checks. First, cross-model answer anchoring: two models from different families answer from the privileged view several times without seeing the proposed answer. The question is kept only if a clear majority in each family agrees, and that agreed answer becomes the reference. Second, adaptive drop-one testing: mask one region at a time with a gray block and ask if the answer is still recoverable. If two or more regions turn out to be unnecessary, the question is discarded as having decorative regions or shortcuts.

6. Convert for training. Each surviving question becomes four-option multiple choice to allow rule-based rewards. By default the learner, called the policy, sees the full original image with colored outlines to avoid ambiguous references like "that cup." For simple counting questions, the authors sometimes try to rewrite to natural language without outlines, but only keep that version if both model families audit it as uniquely referring, still multi-region, and still yielding the same answer. Finally, overly easy examples that a small screening model always gets right are dropped. That leaves 20.4K examples, mostly 3-4 regions, with about 44% rated hard or very hard.

Training uses reinforcement learning with verifiable rewards, meaning learning by trial and error where correctness can be checked automatically, specifically GRPO, a variant that compares groups of sampled answers. All three base models — Qwen3.5-4B, Qwen3.5-9B, Qwen3.6-27B — train in non-thinking mode for two epochs with no supervised fine-tuning and no benchmark-specific formats.

What they showed — results, honestly, with limitations

The headline is breadth: ZwZ2 beats its own base model on all 25 benchmarks at every scale. Overall unweighted averages rise from 49.41 to 54.79 at 4B, 51.63 to 56.59 at 9B, and 55.52 to 60.51 at 27B. Concrete jumps include ViewSpatial-Bench 37.43 to 45.26 and MileBench 60.08 to 70.08 at 4B, and ERQA up 10.5 points at 27B. The 27B model also leads the three tested frontier models overall at 60.51%, versus 54.02%, 57.24%, and 55.47% for the GPT, GLM, and DeepSeek variants — though the paper notes those frontier inputs were resolution-limited by their platforms.

Notably, single-image training transfers outward: long-context average up about 5.6 points at 4B, AgentVista agent benchmark up 4.8 to 6.2 points, plus consistent gains on GUI grounding, embodied spatial reasoning, and general visual reasoning. Case studies show why: better enforcement of "in all images," better binding of text to the queried archway rather than a nearby fountain, and complete counting of a 2-by-5 tablet pack instead of undercounting 6 versus 10.

Against other synthetic data trained on the same 4B base, EVIDENCECONNECT wins clearly. Four prior sets actually lower the average below the base, improving MileBench but hurting detail and counting tasks. The strongest baseline, ZwZ with 73K single-region zoom questions, is beaten by the 20.4K multi-region set on 10 of 11 head-to-head benchmarks, with the biggest gap on MindCube-Tiny where single-region training hurt but multi-region helped. This supports the claim that connecting across regions matters, not just seeing details.

Ablations are honest about uneven gains. Drop-one verification adds +1.63 points on average, helping O3-Bench and CountQA a lot but hurting ERQA by 4.0 points in that 7K comparison. Privileged-view synthesis adds +1.27 points on average over using original images, but loses on Chartography and ERQA. More data helps but with diminishing returns: 47.16 base to 52.65 with 7K to 53.86 with 20.4K. Long-context gains persist to 256K tokens on MemLens but shrink at the very longest contexts, which makes sense because training uses short single-image contexts. Attention analysis on TreeBench shows the trained model's most-attended image tokens overlap human-annotated evidence slightly more in most layers, but this measures focus, not correctness.

The authors list two limits: no temporal or cross-frame modeling, so video is future work; and testing only up to 27B and only as post-training, so larger-model or pre-training effects remain unknown.

Why this might matter to me

For your interests — spatial reasoning in LLMs/VLMs, world models, text-to-CAD and parametric geometry, and 3D generation — the transfer story is the hook.

Spatial reasoning and world models live or die on binding and viewpoint handling: which face belongs to which part, left from whose frame, what stays identical across views. EVIDENCECONNECT explicitly trains select-then-report, viewpoint-transform, and transitivity patterns, and it rewards proving completeness over a closed set rather than settling for majority overlap. That discipline maps directly to maintaining object identity across frames, checking consistency across rendered views, and composing relations like "hole coaxial with boss on the far side."

For text-to-CAD and parametric geometry, the aggregation and extremum tasks are suggestive analogues: counting instances, summing visible numbers, finding the outlier dimension, pairing mating features. The paper shows that forcing a model to account for every outlined instance before aggregating fixes undercounting — exactly the failure mode that breaks a CAD sketch with missing constraints or duplicated extrusions. The unmarked-rewrite idea also mirrors a CAD need: moving from color-highlighted references to uniquely grounded natural descriptions without losing the multi-part dependency.

For 3D generation and scene representation, the practical lesson is data efficiency: you may not need expensive multi-view or 3D-annotated trajectories to get better multi-view behavior. Cheap, well-audited single-image compositional tasks with agreement checks and ablation tests already improve cross-image category consistency and evidence-focused attention. The limitation matters here too — single-image data does not teach long-range retrieval or true temporal dynamics — so the natural extension for you would be EVIDENCECONNECT-style groups spread across views, video frames, or CAD operation histories, where dropping a view or a feature should break the answer.

Terms worth knowing

Vision-language model (VLM): a model that takes images and text together to answer questions or act. Qwen3.5 and Qwen3.6 are the base VLMs fine-tuned here.

Connecting evidence: the paper's named shared skill — spotting relevant visual facts, binding each to the right entity, recovering their relations, and composing an answer.

EVIDENCECONNECT: the authors' automatic pipeline for making multi-region, compositional questions from a single image, with built-in quality audits.

Privileged evidence layout / privileged view: enlarged native-resolution crops of just the selected regions, arranged to preserve relative order. Used only to write and verify questions, never shown to the learner, so the learner must still do hard perception on the full image.

Policy view: what the learner actually sees during training — usually the full image with colored outlines marking the regions in question.

Cross-model answer anchoring: keeping a reference answer only if two models from different families independently converge on it.

Drop-one / region-ablation test: masking one region at a time to check necessity; if the answer survives without it, that region was not truly needed.

RLVR and GRPO: reinforcement learning with verifiable rewards, and group relative policy optimization, the specific RL algorithm used. Here reward is simple: did the predicted multiple-choice letter match the verified answer?