All summaries · 2026-09-23

Benchmarking World Models for Continual Learning on Compositional Tasks

Haoyu Zhou, Joe Watson, Anson Lei, et al.

arXiv:2609.22055 · abstract · pdf · summarized by meta/muse-spark-1.3-contributor on 2026-09-23

The one-paragraph version

This paper asks a sharper question about world models — internal simulators that learn how actions change what the robot sees — than most robot benchmarks do: when a robot meets tasks one after another, does it actually reuse what it already learned, or does it just learn fast from scratch while forgetting the past? To isolate reuse, the authors build curricula where a robot first learns simple primitive skills, then faces a final composition task made only by recombining those primitives. They split that recombination by the two inputs a world model cares about — what it does versus what it sees — and they split the model itself into a shared, reusable dynamics core versus per-task decision parts that are allowed to reset. Tested on standard world models with standard anti-forgetting tricks, no method solves reuse without forgetting; a modular mixture-of-experts model almost eliminates forgetting but only when given a privileged, pre-trained and frozen visual encoder, and even then it does not learn the final task faster than a non-modular baseline.

The problem they're solving, and why it matters

The setting is continual learning, meaning an agent meets tasks in sequence and after each new task it is still expected to do well on the old ones, without being allowed to keep practicing them. A world model seems ideal for this, because unlike a task-specific policy, the physics of pushing, grasping or opening should carry over.

The problem is that existing robot continual-learning benchmarks, such as Continual World built on Meta-World tasks or LIBERO with its spatial and object shifts, tangle two different abilities. When a new task arrives it contains both unfamiliar new content and familiar recurring content. So a good score could mean genuine reuse of old mechanisms, or just fast learning of something new, or just not forgetting. You cannot tell which.

That matters because the authors argue, following work on causal and mechanistic models, that efficient adaptation in the physical world should come mostly from reuse: the world is made of a small set of repeating mechanisms. If we cannot measure reuse separately, we cannot tell whether our world models are really building reusable understanding or just overwriting themselves each time.

The key idea, in plain words

Make the final exam pure reuse. Give the model the ingredients first, then test whether it can cook with them.

Concretely, each curriculum ends with a composition task that is deliberately built by recombining the primitive tasks that came just before it. If the model learns that last task faster than a model starting from scratch, that speedup — what the field calls forward transfer — can only come from reuse, because there is nothing fundamentally new to learn.

Then, to diagnose where reuse breaks, factor that recombination along the two channels a world model is conditioned on. Think of it as: changing what the arm must do while keeping what it sees fixed, versus keeping the motion similar while changing what it sees, versus changing both at once.

Finally, be principled about what should be reused. The authors separate the world model into a task-agnostic backbone — the part that compresses images and proprioception into a compact latent state and predicts how that state evolves — versus task-specific heads — the parts that predict reward, value and action for one particular task definition. Only the backbone is trained continually with anti-forgetting methods; the heads are freshly created for each task and then frozen. Monolithic backbones, meaning one big dynamics network, are then compared to a modular backbone whose dynamics are an explicit collection of reusable experts.

How it actually works, step by step

The six curricula. All built in Meta-World simulation. For example:

Shared robot interface. Observations at each step are stacked images from three cameras — top view, front view and gripper view — plus a 7-number body sense of end-effector pose, velocity and gripper status. Actions are relative 3D hand displacement plus gripper open-close. Rewards are rescaled to encourage minimum-time solutions with an absorbing success state.

Training protocol. For each task in order: create fresh heads for this task, train backbone continually with the chosen continual-learning rule while training heads freely, then freeze those heads. After each stage, evaluate all tasks seen so far using the current backbone plus their frozen heads. Returns are normalized so failure is 0 and a scripted oracle is 100.

Models and continual-learning rules. Baselines are DreamerV3, which carries recurrence and reconstructs images with a decoder, and TD-MPC2, which is decoder-free and shapes its compact state only through consistency, reward and value learning, plus Prismatic World Model, called PWM, which extends TD-MPC2 with multiple dynamics experts combined by a learned router. On the monolithic models the authors try naive fine-tuning with no protection, experience replay keeping 5% of old transitions, elastic weight consolidation that penalizes moving important weights measured by Fisher information, and PackNet that prunes and freezes 25% of free weights per task. For PWM they make a continual variant that adds 3 new frozen-after-training experts per task while the router can see all experts so far.

A diagnostic to isolate modularity. Because PWM’s visual encoder would otherwise keep drifting, they pretrain an encoder as an autoencoder on expert demos of all tasks, freeze it, and give the exact same frozen encoder to monolithic TD-MPC2. Any remaining gap can then be attributed more cleanly to modular dynamics, at the cost of giving both models privileged access to the full distribution — the authors are explicit this is a diagnostic, not a deployable result.

Metrics. Backward transfer, meaning how much old tasks degrade after learning new ones where lower is better, and forward transfer, meaning how much faster the final composition task is learned compared to from scratch where higher is better and measured only on that last task.

What they showed — results, honestly, with limitations

DreamerV3 beats TD-MPC2 on both axes. The authors attribute this to DreamerV3’s reconstruction objective forcing its latent state to keep broad scene detail beyond the immediate task, whereas TD-MPC2’s decoder-free latent keeps only what current reward and value need.

Conventional methods trade reuse against forgetting. Fine-tuning learns the composition well but forgets most. Replay keeps what fine-tuning reuses, giving the best forward transfer with roughly half the forgetting, but drift only slows and the buffer grows forever. Elastic weight consolidation is weakest here — worse than scratch on the composition while still forgetting more than PackNet — because in the authors’ reading it freezes old weights without providing free capacity to recombine them. PackNet comes closest: by freezing one sub-network per task but letting later tasks read all earlier sub-networks, it drives forgetting near zero with competitive forward transfer, but forward transfer still trails replay and capacity shrinks along the sequence while requiring the curriculum length in advance.

Explicit modularity pushes forgetting near zero without hurting adaptation, but with a big caveat. Averaged over six suites under the frozen-encoder diagnostic, modular PWM reaches about 3.9 backward transfer versus about 35.0 for monolithic TD-MPC2, while forward transfer is roughly tied at about 36 versus about 38. In other words, freezing old experts plus adding new ones plus learning a router to combine them stops overwriting while preserving adaptation, and unlike PackNet it adds capacity per task. However, without the privileged frozen encoder PWM is only on par with TD-MPC2, and it does not improve forward transfer beyond the monolithic baseline. The paper is clear this is evidence for what reusable design can buy, not a solved state of the art.

Difficulty rises from action to perception to full composition. Forgetting worsens monotonically along that order for both backbones, and while DreamerV3 holds forward transfer in a narrow band, TD-MPC2 loses it entirely once perception shifts — consistent with the idea that a perception shift hits the encoder and then propagates as a drifting input to dynamics, whereas an action shift under familiar vision is easier.

Does the router actually reuse? Sometimes yes. In Reach, Grasp and BinPnP, routing weight shifts toward overlapping prior tasks — one BinPnP example boosts an overlapping prior to 1.2 times its original weight while cutting a non-overlapping one to 0.57 times — and the top expert per primitive stays top after composition in 13 of 14 cases. But in temporally extended tasks like stacking blocks or open-then-place, where different primitives are needed at different phases, the task-fixed router does not show the expected pattern. Ablations support functional reuse: prior experts alone keep much of the return on several suites, new experts alone do poorly, and uniform weighting without learned routing does poorly.

Limitations, stated by the authors. Simulation only, online model-based reinforcement learning needing on the order of hundreds of thousands to millions of steps per task, dense rewards and automatic resets and oracle normalization that are simulator affordances, single model size and expert count without controlled parameter or compute budgets, and no real-robot test. Porting this to hardware, they say, will need offline-pretrained backbones and learned success detection.

Why this might matter to me

Given your interests in spatial reasoning in language and vision-language models, in world models, and in text-to-CAD, parametric geometry and 3D generation:

This benchmark gives you a clean way to test whether a vision-language policy or world model truly understands space as reusable structure. A model that can pick a red block and a blue block but cannot stack them without relearning vision from scratch does not have compositional spatial reasoning — it has memorized views. The action-versus-perception split maps directly to failure modes you care about: action-composition is like recombining parametric operations such as extrude then fillet, while perception-composition is like the same operation on new geometry or appearance.

The backbone-versus-heads separation is a template for text-to-CAD and 3D generation. You want a stable geometric or physical core — contact, occlusion, support, part attachment — with lightweight task or prompt-specific heads on top. Their finding that decoder-free latents discard scene detail that later reuse needs is a caution for planning-oriented 3D representations: optimizing only for immediate reward or reconstruction-free consistency may throw away the very shape information needed for the next edit.

Finally, the modular result plus its frozen-encoder caveat points to where to invest: learned routers over frozen skill or geometry experts, state-dependent rather than task-fixed routing for multi-phase assemblies, and stable pretrained visual backbones in the spirit of DINOv2-style features the authors cite, so new CAD operations or scene edits compose without drifting the underlying representation.

Terms worth knowing