The one-paragraph version
MindTopo is a new benchmark that asks whether today's foundation models — large language and vision-language models — actually understand topological space, meaning spatial relations that stay the same when you bend, stretch, or twist something without cutting or gluing it. The authors ground this in cognitive science, especially the idea from Jean Piaget that children grasp these qualitative relations before precise distances and angles. They build 11,030 procedurally generated puzzles across 13 task types, organized into five properties — continuity, separation, order, enclosure, and knots — and each property is tested at two levels: reasoning, where the model answers a question about a rendered image, and planning, where the model must act step-by-step in an interactive environment to build or change the structure. Across 14 multimodal models, every model does better at answering than at acting, and even the best model scores about 61% overall compared to about 98% for humans. Extra training and adding image or video generators help with single-image answers but do not fix the core gap: models can often recognize a topological relation but cannot reliably preserve it across a sequence of actions.
The problem they're solving, and why it matters
Most tests of spatial reasoning in AI focus on metric ideas: how far apart things are, what direction they point, how big they are, what 3D shape they have from a viewpoint. The paper argues this misses a more basic layer.
Topological relations are those that survive continuous deformation. A concrete example the authors use: take a closed loop of rope, bend it, coil it, stretch it. Distances and angles change wildly, but whether it is knotted stays the same unless you cut it or pass one part through another. Similarly, whether two points in a maze connect, whether sheep are inside or outside a fence, or whether beads appear in a certain order along a string does not depend on exact measurements.
Cognitive science treats these relations as foundational. Piaget argued topological understanding comes before Euclidean and projective understanding in child development, and vision research found adults extract things like connectedness and presence of holes very early. Yet foundation-model benchmarks largely ignore topology, except for isolated studies of mazes or knots. The authors ask directly: can current models both perceive topological structure and plan actions that transform it correctly?
The key idea, in plain words
The key idea is to separate two different cognitive demands and test them side by side for the same kind of spatial intuition.
The first demand is reasoning: look at one or more rendered scenes and identify a relation, or predict what would happen if you made a stated change. For example: which labeled points connect to point A in a maze, how many through-holes are in a board, or which rings would become free if you cut the red ring.
The second demand is planning: act as a closed-loop agent. That phrase means the model sees a rendered observation, chooses an action, the simulator updates the world, then the model sees the new observation and chooses again. For example: rotate pipes to connect them all to a source, draw a single stroke that separates colors, place blocks to trap a moving cat, or move rope endpoints to remove crossings.
By matching reasoning and planning tasks at the level of the same five properties, the paper can test transfer: does recognizing that a loop is knotted help you figure out how to untangle it? Their answer is mostly no.
How it actually works, step by step
First, they define five properties, using Piaget's classification plus formal topology to decide how to generate ground truth:
Continuity means whether something forms an unbroken whole, like whether two maze points link by a path. Separation means splitting a scene into distinct connected parts, like finding a valid furniture subassembly or drawing a boundary that separates colors. Order means the sequence along a path or boundary, like listing bead colors along a twisted string or tracking points through origami folds. Enclosure means inside versus outside created by a closed boundary, including holes, like counting fenced sheep or through-holes in a solid. Knots means entanglement that persists under deformation, like whether a loop is truly knotted or two loops are linked.
Second, they build 13 task types from those properties. Eight are reasoning with about 1,000 questions each: 2D Maze, 3D Maze, Assembly, Bead, Origami Point, Sheep, Hole, and Knots. Five are planning with 600 episodes each: Pipe for continuity, One Stroke for separation, Swap puzzle for order, Chat Noir — block cells on a hex grid to encircle a moving cat — for enclosure, and Untangle ropes for knots.
Third, everything is generated automatically with fixed seeds. A scene generator samples a state and renders it, often with Three.js or a simulator like ProcTHOR for 3D rooms or an origami simulator. Then a program computes the answer from the underlying state — by graph search for maze connectivity, geometric membership for inside-outside, construction metadata for assemblies and knots, or simulator checks for planning success. Difficulty is controlled by parameters like grid size, wall count, number of beads or ropes, and crossing count, yielding easy, medium, and hard tiers. Invalid or ambiguous scenes are filtered out.
Fourth, evaluation is strict. Reasoning is scored by exact match for numbers and choices, and by set equality — order does not matter — for unordered lists. Planning is scored only if the executed actions reach the goal state within a step budget. There is no partial credit for a bead list of the wrong length or a nearly closed fence.
What they showed — results, honestly, with limitations
The headline gap is large and consistent. The best model, GPT-5.6-Sol, reaches about 67% on reasoning but drops to about 53% on planning by their macro averages, far below about 98% human performance measured on 11,008 examples by professional annotators. Other frontier models fall harder: as reported, Gemini-3.1-Pro drops from about 52% to about 19%, GPT-5.5 from about 54% to about 24%. Open-weight models often score near zero on Pipe, One Stroke, and Untangle.
No model dominates everywhere. GPT-5.6-Sol leads on most properties, but Gemini-3.1-Pro leads on knots. Even within a property, the reasoning leader and planning leader can differ. Swap, a discrete sliding-block permutation task where GPT-5.6-Sol reaches 87.3%, looks much easier for models than Pipe at best 50.5%, One Stroke at best 35%, and Untangle at best 29.2%. The authors caution they have not separated whether this is due to state representation, difficulty, or action horizon.
Training helps answering more than acting. Starting from Qwen3-VL-2B-Instruct, supervised fine-tuning followed by reinforcement learning with Group Relative Policy Optimization raised a nine-task average from 8.0% to 31.4%, but reasoning rose to about 51.5% while planning rose only to about 6.3%. Pipe stayed at 0%. Transfer to held-out tasks was uneven: some gains on Bead and Knots, none on several others.
Error analysis on two models found 58% of reasoning errors were perception-grounding failures — misreading what is visible — while planning errors shifted to choosing poor legal actions at 43% and violating dynamics at 25%.
Probing with generators did not solve it. Adding GPT-Image-2 image prediction or video models like Wan2.2-I2V-A14B could keep local cues and sometimes reach a plausible final image, but audits found dynamics violations in 116 of 119 videos and no process-valid success — meaning a correct-looking endpoint after an invalid path — across 1,560 rollouts.
Limitations the authors state plainly: all scenes are synthetic renders, not real photos; only three video generators were tested as planners; five primitives do not cover all topology; and failure audits cover hundreds, not thousands, of generated images and videos.
Why this might matter to me
For spatial reasoning in LLMs and VLMs, this suggests current scores overstate competence. Models may exploit local visual cues or answer priors — the paper shows Sheep retains much of its score even without the image — while failing on trajectory-wide invariants that require tracking state across steps. That points to evaluation that checks intermediate states, not just final answers.
For world models, the video result is cautionary. A generator that produces a plausible final frame of an untangled rope or connected pipe network is not the same as a world model that preserves topology across transitions. The authors' strict check — every frame must preserve state and every transition must be legal — is a useful template for testing whether a predictive model could be trusted for lookahead.
For text-to-CAD and parametric geometry, the separation, order, and enclosure tasks map closely to real operations: partitioning a solid into connected subassemblies, maintaining order along a curve under deformation, and judging inside-outside and through-holes versus shallow pits. The finding that exact hole counting collapses under distractors and that training does not transfer systematically suggests a need for representations that explicitly encode connectivity and boundary graphs rather than pixels alone.
For 3D generation and scene representation, the knot and continuity results highlight a blind spot: preserving crossing order, linking, and path-connectedness under novel views and edits. If a scene representation cannot guarantee that a rearrangement preserves these invariants, downstream editing and simulation will silently break physical plausibility.
Terms worth knowing
Topological intuition: the paper's term for the ability to perceive, infer about, and operate on relations that remain unchanged under bending and stretching without cutting, joining, or passing through.
Continuity: whether a path or surface forms an unbroken whole. Example: two maze points are in the same component if you can travel between them without crossing a wall.
Separation: the complement of continuity — how a scene divides into distinct connected parts. Example: which furniture parts form a separable subassembly.
Order: the sequence in which marked elements appear along a path or boundary. Example: bead colors encountered when traversing a string from a marked start in a marked direction.
Enclosure: inside-outside division created by a closed boundary, including holes. Example: sheep trapped by a fence, or through-holes that pass fully through a board.
Knots and links: entanglement of loops in 3D that persists unless you cut. Example: a true knot versus a visually tangled but unknotted loop; two rings that are linked versus merely touching. The Untangle task uses a simpler proxy — zero crossings in the top-down projection — not full 3D knot equivalence.
Reasoning versus planning: reasoning means answering a visual question in one shot; planning means acting as a closed-loop agent where a policy — the rule the model plus any helpers use to pick actions from history and current observation — selects actions and the environment updates state until success or budget exhaustion.
Ambient isotopy: the formal idea behind "deform without cutting." Two structures count as equivalent if one can be continuously morphed into the other inside the surrounding space while carrying marked points along.
Procedural generation with ground truth: scenes are sampled by code with known parameters, so labels come from the generator state rather than human labeling, allowing controllable difficulty and exact scoring.