All summaries · 2026-09-29

GameBoyWorlds: A Testbed for Self-Improvement in Embodied Video Games

Dhananjay Ashok, Adam Shen, Aslan Huo Feng, et al.

arXiv:2609.32093 · abstract · pdf · summarized by meta/muse-spark-1.3-contributor on 2026-09-29

The one-paragraph version

GameBoyWorlds is a new test for whether game-playing AI can teach itself without human help. It has two parts. The first part, called Execution, asks an agent to freely explore older GameBoy training games with no demonstrations, manuals, or scores, then complete 500 short, hand-made tasks phrased in everyday language — things like “Activate the switch using a bomb” or “Fish for a Pokémon” — in both familiar and unseen sequels. The second part, called Playthrough, asks an agent to play two little-documented fan-made Pokémon games from start to finish, earning eight gym badges and beating the final tournament, so it cannot cheat by remembering walkthroughs from its training data. The authors find that leading vision-and-language models, meaning AI that takes both images and text as input, complete less than half the short tasks on their own, that standard teach-yourself methods like learning an internal simulator or inventing your own practice tasks actually make things worse, and that even a purpose-built long-term player with maps and memory cannot get its first Pokémon in the fan games.

The problem they're solving, and why it matters

Most capable game agents today still depend on people. That dependence takes the form of hand-written scores that tell the agent what is good, recordings of human play to copy, or large collections of human control demonstrations. That works, but it does not scale to every new home, robot, or game, and it leaves open a basic question: can an embodied agent, meaning an AI that must look at surroundings and press buttons to act over time, keep getting better just from its own experience?

Video games are a natural place to study this, because a child learning Pokémon learns movement, then remembers maps, then learns what items do, all while playing. Existing game tests for AI do not measure this well. Some only test final performance without allowing learning. Others train with scores taken from the test game itself. And popular games like Pokémon Red are heavily contaminated, meaning their maps, tricks, and story steps already appear many times in the text the models were trained on, so a model can look smart by recalling rather than learning. GameBoyWorlds is designed to close that gap by forbidding outside help during learning and by testing in games the models have likely not memorized.

The key idea, in plain words

Give the agent nothing but the screen and the buttons, let it play around on its own in a training game, then see if that self-collected experience helps it do new jobs in a related test game — or keep learning while playing a long adventure it has never seen.

Self-improvement here has a strict meaning the paper enforces: only data the agent itself generated by interacting counts. No human playbacks, no stronger teacher model giving answers, no wiki or guidebook, and no task-specific score. Any internal score used to encourage exploration must be task-and-game-agnostic, meaning the same rule would apply to any game, like “seek images you have not seen before.” Training is also restricted to the first game in each series, so the agent must learn lessons that carry over, for example from Pokémon Red to Pokémon Crystal.

The concrete example the authors return to is Bomberman Quest. To “Activate the switch using a bomb,” the agent must see the switch on screen, walk around walls to reach it, use the game-specific action of leaving a bomb on the right square, then walk far enough away before it explodes. No manual tells it any of that.

How it actually works, step by step

The testbed itself is built on an emulator, meaning software that mimics a GameBoy, wrapped in a standard research interface. An action, meaning a button choice, is one of up, down, left, right, A, B, or Start, held briefly then released. An observation, meaning what the agent sees, is the final gray image, 144 pixels tall by 160 wide. Every test starts by reloading an exact saved snapshot, so results are reproducible. Success is checked by picture matching: the authors record a small screen region that uniquely appears on success, like a dialogue box saying an item was received, and declare success when new frames closely match that stored picture.

For out-of-the-box testing the authors settle on a two-level helper. A supervisor, meaning a planner that does not press buttons itself, breaks a task like “Pick up a sickle” into standalone visual steps, hands one step at a time to an executor, meaning the button-presser, checks whether the screen shows the step is done, writes a hint naming buttons if it failed, and rewrites the plan at most twice if the plan itself looks wrong. The executor presses one button at a time and writes a one-sentence note about what visibly changed, which serves as short-term memory.

For learning, they try three routes. First, world modeling, meaning learning an internal simulator that predicts the next screen from recent screens and a button, then picking the button whose predicted screen looks best. Their version first compresses images with a small image coder, then learns to predict the compressed code for the next image.

Second, autonomous skill discovery in the Proposer-Agent-Evaluator style. The model looks at a training screen, invents tasks it thinks are doable, tries them for up to 50 steps, judges itself, practices the ones it thinks it passed from slightly shuffled starts, then fine-tunes a vision-language-action model, meaning the same vision-and-language model now trained to output actions, on those practice runs.

Third, their new curiosity-to-guide pipeline. A simple trial-and-error learner is rewarded only for visual and text novelty, meaning how different the current screen or dialogue text is from what it has already seen in this round, measured by comparing randomly projected image codes with cosine similarity, meaning a score for how alike two directions point. Later rounds start with memories of earlier rounds pre-filled, so they are only rewarded for finding something new. The authors then group high-novelty clips by similar final images, ask a vision-and-language model to name the multi-step job in each group and throw away noise, distill concrete non-obvious tips like “to inspect an item in Déjà Vu, first select the eye icon then click,” merge tips into a per-game guidebook, and at test time retrieve only entries judged relevant to the current task and screen.

The Playthrough side uses two fan games, Pokémon Brown built on Red and Pokémon Prism built on Crystal, with new regions called Rijon and Naljo, new creature types like Wood and Gas-plus-Sound, and even sections where you walk as your own Pokémon. Progress is tracked by receiving each gym badge message and finally the champion message. Their long-horizon agent adds tile-by-tile screen labeling into categories like ground, obstacle, entrance, or non-player character, a walkability map that turns walking into route search rather than button counting, seven fixed action routines for moving, interacting, fighting, menus, and dialogue, plus a goal tree for what to do next, a knowledge tree for facts, a location store that builds a map only from crossings it actually made, and a temporary notepad distilled into permanent memory when goals close.

What they showed — results, honestly, with limitations

Out of the box, frontier models struggle. The best average reported is Gemini-3.6-Flash at 49.3% of the 500 tasks, with GPT-5-Mini, Claude-4.5-Haiku, and open models Gemma-4-31B and Qwen3-VL-32B lower overall. The point-and-click Déjà Vu games are hardest, with no model above about 22%, while Bomberman Pocket is easiest, with one model reaching 84%. By task type, menu operation is relatively better at 66%, while interaction at 50%, navigation at 49%, and combat at 54% lag. Hand-checked failure patterns for the leading model are dominated by navigation at 30%, indecisiveness at 23%, and perception at 17%, plus repeats, poor coordination between planner and button-presser, and missing game rules.

None of the teach-yourself methods reliably helps. World modeling lowers average completion by about 3.4 percentage points, which the authors attribute to blurry, inaccurate next-screen predictions. Skill discovery collapses from about 40.7% to 22.6% on average, with Pokémon Crystal falling from 36% to 4%, Pokémon Red from 38.8% to 8.2%, and Zelda: Oracle of Seasons from 40% to 14%, worse on unseen test games, likely because the model wrongly judges its own practice as successful and then copies those mistakes.

The curiosity-to-guide approach is the only partial win, rising from 40.7% to 41.5% on average. The paper describes gains of up to roughly 22% or 10% on select games, giving Bomberman Pocket moving from 72% to 88% and Zelda: Oracle of Seasons from 40% to 44% and even help on the unseen Sword of Hope 2 from 66% to 72%, but also steep drops like Déjà Vu 1 down 29%, Déjà Vu 2 down 25%, and Pokémon Crystal down 17%. The exact percentage wording versus point differences is a little unclear in the text, so treat the headline as “helps in places, hurts in others.”

For long play, contamination checks show models answer 92% to 100% of progression questions correctly for official Red and Crystal but only 8% to 29% for Brown and Prism. In actual play, the same sophisticated agent makes rapid early progress in Red — leaving home, triggering the professor scene, picking a starter, beating a rival — but in Brown it wanders the starting town talking to townspeople, never heads south to the neighboring town to get its first Pokémon, and similarly stalls in Prism. Limitations the authors note: success can only be checked when success shows a distinct screen, which favors tasks with immediate visual feedback; the remaining games they implemented are left for future work; and many scaffolding choices did not clearly beat simpler ones in ablations.

Why this might matter to me

For spatial reasoning in vision-and-language models, this is a clean stress test of grounding, alignment, and pathfinding under pixel-perfect constraints. Failures are not abstract: standing one tile off, facing right instead of up, or misreading which shelf cell holds the sickle breaks the task. The Playthrough agent’s split is instructive — let the model name what and where in words, then let classic search find the route on a walkability map — rather than asking the model to count steps.

For world models, the negative result matters. A small latent predictor in the style of Dreamer-style control, meaning learning behavior by imagining futures in a compressed space, is not enough here where placing a bomb rewrites which squares are walkable under a fuse deadline, or where a menu cursor wraps around. Prediction quality, not planning cleverness, is the bottleneck.

For text-to-CAD and parametric geometry, the Sword of Hope and menu-heavy tasks echo precise, stateful editing: move a cursor in a command grid, open a submenu and cancel without committing, buy exactly one item. The guidebook idea — distilling surprising runs into reusable, checkable tips with preconditions and visual stop conditions — is close in spirit to design copilots that must learn tool semantics from interaction rather than manuals.

For 3D generation and scene representation, watch their two representations: a reusable tile vocabulary that labels each distinct 8-by-8 appearance once and reuses it everywhere, and a purely topological location graph that records that place A connects to place B and how to go back, with no global coordinates. That is enough for short tasks but leaves long exploration fragile: if the map only contains crossings actually made, and the agent never notices an unexplored exit, it never leaves town. That points to frontier-aware mapping and persistent visual memory as the next step.

Terms worth knowing