The one-paragraph version
Vision-Language Models often explain their answers with fluent stories that sound right but may not reflect what they actually saw. This paper introduces a test that calls that bluff. The method, called Explanation-Driven Counterfactual Testing, first asks a model to answer a visual question and explain itself in plain language, then pulls out the visual details it claimed to use, edits just those details in the image with a generative image editor, and asks the model again. If the model said it saw a tennis racket because of its strings, the test replaces the strings with a solid paddle surface and checks whether the new answer and new explanation stay consistent with what is now visible. Using this protocol the authors build EDCT-Bench, 300 carefully checked tests drawn from everyday knowledge questions, driving scenes, and 3D spatial puzzles. Every model they tried, including strong proprietary systems, frequently failed, holding onto old answers or repeating old reasons after the evidence was removed. A small follow-up suggests these edited images are hard, useful training examples, but does not yet prove they make models more faithful.
The problem they're solving, and why it matters
The paper draws a sharp line between an explanation that is plausible and one that is faithful. Plausible means it sounds reasonable to a human listener. Faithful means it accurately points to the visual evidence that actually supports the answer.
That gap matters because Vision-Language Models, meaning AI systems that take in an image plus a text question and produce a text answer, are being proposed for high-stakes uses like driving assistance and medical support. In those settings a convincing but disconnected story can hide real problems: the model might ignore what it cites, hallucinate details that are not there, or give the right answer for the wrong reason.
Existing checks often ask people or automated judges how nice an explanation sounds. The authors argue that is not enough. You need to intervene. Change the very thing the model said it relied on, keep everything else the same, and see whether the model notices. If it does not, you have behavioral evidence of unfaithfulness, even if you cannot see inside the model.
The key idea, in plain words
Let the model choose its own test. Instead of editing random parts of an image, edit exactly what this particular model said mattered.
A concrete example from the paper makes this clear. Asked what is in a woman's hand, a model answers a tennis racket and explains that it sees a long handle and a large oval head strung with strings. The test then keeps the woman, pose, and background but changes that stringed surface into a solid, smooth paddle surface. When asked again, the model still says tennis racket and still talks about strings. That unchanged answer and recycled explanation are inconsistent with the new image, which no longer has strings. The failure suggests the model is leaning on prior expectations or ignoring the edited region, not truly grounding its words in pixels.
Two important nuances keep this fair. First, not every edit has to flip the answer. If other visible clues still support the same answer, staying with that answer is allowed, but the explanation must stop citing what was removed. Second, passing does not prove the explanation mirrors internal processing; failing is what is informative, because it falsifies the claim that the explanation was grounded.
How it actually works, step by step
The pipeline has four stages, orchestrated as an automated workflow.
1. Get a baseline and pull out cited concepts. The target model is shown an original image and question to get its baseline answer. In the same conversation it is asked why, meaning what visual features led to the answer. A separate large language model then reads that explanation and extracts a short list of testable visual concepts. A visual concept here means either a specific attribute of an object, like the red color of a car or the oval shape of a ball, or the object itself if no attribute is named, like car or ball. The authors test at most two concepts per case, ranked by importance.
2. Generate a minimal counterfactual edit. Counterfactual here means an alternate version of the image where one cited detail is deliberately changed. A text-guided image editor, in this work FLUX.2 Max and Gemini 3 Pro Image Preview, is given a structured instruction with four parts: what to keep unchanged, what to change, how to render it realistically with matching light and perspective, and what constraints to avoid. Edits are applied as a chain, so a second edit builds on the image already changed by the first.
3. Verify the edit is localized and valid. Generative editors can spill changes into the background or warp the whole scene, so every edit is checked automatically. The system parses the edit type and target object, finds that object with an open-vocabulary detector called Moondream3 and segments it precisely with Segment Anything Model v2, then slightly expands that mask to allow a boundary margin. It aligns the edited image back to the original to discount small global shifts using feature matching and optical-flow warping. It then measures change inside the mask versus outside it, using both pixel color and gradient differences and a structural similarity measure that captures texture and structure. An edit passes only if there is enough change inside and little change outside. Failed edits get one retry; if they still fail they are excluded.
4. Re-query and score consistency. The same target model is asked the same question on the edited image to get a new answer and new explanation. A language-model judge then assigns two binary judgments in words first. Answer Consistency asks whether the new answer makes sense given the intended change, allowing an unchanged answer when untouched evidence still supports it. Explanation Faithfulness asks whether the new explanation is grounded in evidence that is still valid, failing if it keeps citing a removed or altered feature. The binary Counterfactual Consistency Score for one concept is the product of those two, meaning both must pass to get credit, and the score for an explanation is the average across its concepts. The authors also ask the judge for a holistic rubric score from 1 meaning total failure to 5 meaning high faithfulness, later normalized to a zero-to-one range.
What they showed — results, honestly, with limitations
On 300 curated samples — 160 from OK-VQA for everyday knowledge-intensive questions, 80 from DriveLM for front-camera driving reasoning, and 60 from 3DSRBench for 3D orientation and reference frames — all six tested models showed large gaps.
The best model, Gemini-2.5-Flash, reached only about 0.60 on both the binary and rubric-graded consistency scores. The others fell roughly between 0.44 and 0.57 binary, specifically Pixtral-12B around 0.47, Gemma3-27B around 0.44, Qwen3-VL-30B-A3B around 0.47, Qwen3-VL-8B around 0.51, and GLM-4.6V-106B around 0.57. Answer consistency was generally higher than explanation faithfulness, meaning models sometimes updated the answer while still citing invalid evidence. Capability did not guarantee faithfulness, and bigger was not always better: the smaller Qwen3-VL-8B descriptively outscored its larger Mixture-of-Experts sibling on average, though confidence intervals overlapped. The 3D spatial subset was hardest for every model, with binary scores as low as about 0.22 to 0.43.
Checks support reliability but with caveats. On 128 human-checked interventions, annotators agreed with each other on 88% of the strict 100 cases scoreable by both, and the automated score matched their shared judgment in about 94% of those agreed cases. Swapping the extractor-judge or image editor shifted scores but preserved the overall conclusion that gaps are large; the choice of judge mattered more than the choice of editor. Five repeated runs were stable in aggregate.
The training claim is explicitly preliminary. On 48 driving images about wheel-axle counting, the same Qwen3-VL-8B model got 69.0% right on originals but only 34.5% on counterfactuals, with about four times higher initial training loss, and attention maps looked more focused on axles for counterfactuals and after fine-tuning. The authors state plainly this shows the examples are hard and provide a strong learning signal, not that fine-tuning improves held-out faithfulness, which would need a larger controlled study with post-tuning consistency scores. Other limits they note: because each test comes from that model's own explanation, difficulty can vary by model, and the benchmark should be read as a domain-sensitive stress test, not a universal leaderboard.
Why this might matter to me
For spatial reasoning in Vision-Language Models, the starkest signal is that 3D spatial explanations failed most. That suggests models can produce fluent talk about left versus right, front versus behind, or orientation-dependent frames while not tracking the geometry they cite. If you build evaluations, consider adopting this explanation-first intervention: let your model explain a spatial judgment, then flip exactly the relation it named and require it to revise both answer and reason.
For world models, which need to maintain a coherent internal picture of a scene over actions and time, the valid-invariance idea is directly reusable. A good world model should know when an edit removes the sole support for a belief versus when redundant cues remain. Testing whether explanations shift to remaining valid cues after an intervention is a practical probe of that belief updating.
For text-to-CAD and parametric geometry, where a word like fillet, coaxial, or symmetric must map to an exact parameter or constraint, plausibility is especially dangerous. The paper's lesson is to tie every cited term to an editable visual or geometric feature and demand that removing it changes the description. You could mirror their verification stage in CAD: check that an edit is localized to the named feature and that downstream descriptions stop referencing it.
For 3D generation and scene representation, the localized verification pipeline is a template. Their combination of detection plus precise segmentation, image alignment to discount global drift, and inside-versus-outside change measurement offers a way to audit whether a generative edit actually did what the prompt claimed without corrupting the rest of the scene. Their finding that judge choice moves scores more than editor choice is also a warning to report your evaluator configuration and validate against human judgments on your own 3D data.
Terms worth knowing
- Vision-Language Model: A system that takes an image and a text question together and generates a text answer, often with reasoning.
- Natural Language Explanation: A short plain-text account of what visual features led to the answer, requested after the answer in the same conversation.
- Faithfulness versus plausibility: Plausibility is whether an explanation sounds reasonable. Faithfulness is whether it reflects evidence that actually supports the prediction in the image.
- Counterfactual image: An edited version of the original where a specific cited detail is added, modified, or removed while the rest of the scene is preserved, to test what would happen if that evidence were different.
- Visual concept: The testable detail pulled from the explanation, either an attribute of an object such as red color or oval shape, or the object itself such as car or ball.
- Localized edit verification: The automated check that the change happened mostly inside the target object's mask and minimally outside it, after compensating for small global misalignments.
- Answer Consistency: The judge's check that the new answer is logically consistent with the edited scene, allowing the answer to stay the same when other untouched evidence still suffices.
- Explanation Faithfulness: The judge's check that the new explanation does not keep relying on a removed or altered feature and instead cites updated or remaining valid evidence.
- Counterfactual Consistency Score: The overall faithfulness measure. The binary version requires both answer and explanation to pass and averages across concepts. The rubric-graded version gives partial credit on a 1-to-5 scale for how well answer and explanation reflect the new visual reality.
- EDCT-Bench: The authors' set of 300 verified tests spanning OK-VQA for everyday knowledge, DriveLM for safety-critical driving, and 3DSRBench for 3D spatial reasoning.
- LLM judge: A separate large language model used to extract concepts, write edit instructions, and score consistency, whose wording and choice can shift absolute scores.