All summaries · 2026-09-25

Beyond Spatial Benchmarks: From Spatial Reasoning to Navigation

Xun Huang, Shijia Zhao, Rongsheng Qu, et al.

arXiv:2609.29934 · abstract · pdf · summarized by meta/muse-spark-1.3-contributor on 2026-09-25

The one-paragraph version

This paper asks a blunt question: does getting better at spatial puzzles on benchmarks make an embodied agent better at finding things? The answer they find is no — at least not the way benchmarks are currently done. Models fine-tuned to answer generic spatial questions actually got worse at navigation. What did help was spatial reasoning tied directly to the navigation goal and to the phase of the task. The authors build on that insight with a new dataset called SPATIAL-NAV-100K, a two-stage training recipe that first learns shared spatial-navigation skills and then splits into specialists for searching versus approaching, and a distillation method that lets a fast student act without spelling out reasoning at test time. Their 8-billion-parameter model reaches competitive success on ObjectNav with far less training compute and low latency.

The problem they're solving, and why it matters

The downstream task here is Object Goal Navigation, usually shortened to ObjectNav. That means: start somewhere you have never seen, given only a target category like “find a sofa,” and walk through a house using what you see until you get close enough to stop.

This matters because most spatial reasoning research tests something different: isolated question answering from a picture or video, like “how far is A from B?” or “what would this look like from another viewpoint?” Those tests measure what a model can infer, not whether it can use that inference over hundreds of steps to decide where to go next.

The authors show this gap is real. They took five existing spatial specialists — models already tuned for spatial benchmarks — and dropped them into two navigation frameworks, called VLMNav and WMNav. In every case, the specialist did worse than the plain base model it started from. In other words, benchmark-oriented tuning did not transfer, and often hurt. The problem, they argue, is that generic facts like “the chair is left of the table” do not tell you which hallway is likely to lead to a sofa, whether you can walk through a gap, or whether you are close enough to stop.

The key idea, in plain words

Think of finding a sofa as needing different kinds of spatial thinking at different times.

Early on, when you are exploring, you need to ask: which direction points more toward where a sofa would be, which rooms connect to which, and can I actually squeeze through that doorway? Later, when you think you are close, you need to ask: can I actually see the sofa, and am I within stopping distance?

The paper groups those needs into five abilities: Relation for direction relative to the goal, Topology for connectivity and reachability, Affordance for traversability and what you can do, Visibility for what can be seen from here, and Geometry for shape, size and distance.

The key idea is threefold. First, only teach spatial reasoning that is coupled to the goal — not generic spatial description. Second, teach different abilities to different phases: searching versus approaching. Third, teach spatial reasoning alongside action decisions as a parallel helper, not as a long chain of thought the agent must speak out loud before every move, because that chain is slow and lets errors pile up.

How it actually works, step by step

First, they diagnose what helps. Using the same model backbone and data but varying only the annotation style, they find that goal-linked spatial descriptions improve navigation, while appearance descriptions like color and material, generic spatial descriptions without a goal, and goal mentions without spatial grounding do not generalize.

Next, they split the policy in two. In plain words, at each step the agent makes two separate choices. An Approach chooser looks at the raw image and goal and picks between two high-level options: STOP or CONTINUE. If it says CONTINUE, an Exploration chooser looks at the same image overlaid with candidate travel directions and picks which candidate to follow. Candidates themselves come from a fixed proposer that uses onboard depth sensing to suggest walkable directions; the learned model never sees maps, graphs, or privileged simulator state.

Then they build SPATIAL-NAV-100K. They roll out navigation trajectories in HM3D training houses, sample diverse states limited per episode and per scene, and ask a large annotator model to write structured spatial notes. Each note, in plain terms, contains a claim about one ability, the visible evidence for it, why it matters for the goal without naming an action, and what is uncertain or occluded. They also create a second kind of note called a Spatial2Nav Bridge, which explicitly states how visible evidence bears on the goal. Decision labels are kept in completely separate records, so the model never learns to condition its action on text it just generated. The final count is 112,886 supervision records, not unique images, because one image can yield several records.

Training happens in two stages. Stage 1 learns a shared foundation by training on both decision targets and spatial-plus-bridge targets with separate losses. Stage 2 starts from that foundation and creates two experts: an Explore expert trained on Relation, Topology and Affordance, and an Approach expert trained on Relation, Visibility and Geometry.

Finally, they add Spatial-inspired Navigation Policy Distillation, shortened to Spatial-NPD. Here a frozen teacher sees both the normal navigation input plus the training-only spatial note, and outputs a preference distribution over only the legal actions — which candidate, or STOP versus CONTINUE. A student that sees only the navigation input is trained to match that distribution using a standard measure of difference between distributions, plus its normal action loss. At deployment the student acts directly with no spatial text and no extra prior.

What they showed — results, honestly, with limitations

On the main ObjectNav sets, their model based on Qwen3-VL-8B, averaged over four seeds, reaches Success Rate and Success weighted by Path Length — where Success Rate means fraction of episodes that end within 1.0 meter of the goal, and Success weighted by Path Length additionally penalizes long wandering paths — of 77.4 and 35.4 on HM3D-v0.2, 60.2 and 30.5 on HM3D-v0.1, and 47.9 and 20.6 on MP3D, which was never used for training. They report about 45 A100 GPU-hours for policy training and 148 milliseconds per action, which is favorable compared to cited systems needing thousands of GPU-hours or seconds per decision through closed models.

Ablations support each piece. Adding joint decision-spatial training, then phase experts, then shared foundation, then distillation each adds at least about 2.9 points of Success Rate. Forcing the model to output chain-of-thought reasoning at every step drops performance by 8.0 and 12.7 points and pushes latency to 929 milliseconds. Distillation only helps when the teacher gets the correctly aligned spatial note; shuffled notes give no gain or hurt on unseen houses. The same pipeline also lifts a different model family, InternVL3-8B, by large margins.

Honestly, transfer back to generic benchmarks is mixed: navigation-tuned training helps some spatial benchmarks, leaves VSI-Bench nearly flat, and lowers OmniSpatial. Limitations stated by the authors are important: all results are in simulation with a fixed candidate proposer and controller, no real-robot testing, no manipulation or instruction-following tasks, and all spatial notes come from a single annotator model, so its errors and biases can propagate despite filtering and a reported 98.3% mean manual score on a 500-sample check.

Why this might matter to me

For spatial reasoning in LLMs and VLMs, the lesson is that “more spatial QA” is not the same as “more spatially useful.” What transferred here was goal-grounded, phase-specific supervision kept as an auxiliary loss, not generic relation labeling or verbose reasoning at inference. That is a directly reusable pattern for training assistants that must act, not just answer.

For world models, the teacher-student trick is notable. The teacher gets rich spatial context to shape a distribution over next moves, then the student learns the better move distribution without needing the context or a map at runtime. It is a way to get some benefits of lookahead and grounding while keeping a fast, structure-free policy — similar in spirit to distilling a world model’s preferences into a reactive controller.

For text-to-CAD and parametric geometry, and for 3D generation and scene representation, the phase split between Topology plus Affordance versus Visibility plus Geometry is suggestive. Searching benefits from connectivity and traversability — which rooms link, where you can pass — while finalizing benefits from precise visibility and distance. That mirrors a CAD or scene workflow where coarse topological validity comes before exact geometric stopping criteria. It also shows you can get strong embodied behavior without building an explicit map or scene graph at test time, which raises the question of when explicit 3D representations are worth their cost versus when distilled spatial priors are enough.

Terms worth knowing

ObjectNav: the task of navigating an unseen indoor scene to find an instance of a named object category and stop near it.

Success Rate, or SR: percentage of episodes where the agent stops within the success radius, here 1.0 meter.

Success weighted by Path Length, or SPL: like Success Rate but scaled down if the agent took a much longer path than needed; measures efficiency as well as success.

Relation, Topology, Affordance, Visibility, Geometry: the paper’s five spatial abilities. Relation is which direction is better toward the goal. Topology is what areas connect. Affordance is can I pass or interact here. Visibility is is the goal visible from here. Geometry is am I close enough in size and distance to stop.

Exploration versus Approach: the two navigation phases studied. Exploration searches for promising openings. Approach verifies the target and decides when to stop.

Spatial-NPD: the distillation method where a teacher that sees extra spatial notes teaches a student that sees only normal navigation inputs, so the student needs no explicit reasoning at inference.