PROOF, NOT PROMISES
Do not approve a result because of the model that made it, the prompt behind it, or the message saying it worked. Approve it because of what you can see: what changed, what stayed the same, and what is broken.
A model’s reputation tells you what to try. Only the output in front of you tells you whether it worked. Every case below is a real failure from this pipeline, and every one of them reported success first.
gemma4 advertises vision in its own capability list: ['completion','vision','audio','tools','thinking']. Asked to describe a clip, it answered fluently and immediately.
It described “a person’s arm moving through lush, green foliage”. Then, on a second run, “wild birds, likely raptors… a natural woodland environment”. The clip was a snow leopard walking a snow-covered mountain ridge. Two confident answers, both invented, neither flagged as uncertain.
A capability flag is a claim by the vendor, not a measurement. Fluency is not accuracy — a model that hallucinates does not sound unsure while doing it.
Extract one frame, look at it yourself, and compare. It costs ten seconds and it is the only test that works. qwen3.6 passed the same test on the same frames; gemma4 failed it twice.
THE DECISION RULE
Before you run anything, name the one thing that must be true for the result to be usable. Then inspect the output for that, at the size it will actually be seen.
- For a location: atmosphere, material texture, and any object that has to stay put between shots.
- For a character: identity, skin, pose, and gradients — and compare against the source, not against your memory of it.
- For audio: what the words actually are, not how long the file is.
- For a render: the property you care about, measured on the output and compared to the input.
Approve the result that proves the must-preserve requirement and leaves only failures you can afford to rerun or repair. A passing result on the last shot tells you nothing about this one — check it fresh.
Every case on this page cost between twenty minutes and an hour to find, and each one was invisible from a success message.