The Best AI Models for Video Understanding, Bench-Tested (July 2026)

Seed 2.1 Pro (ByteDance) posted the highest overall score in our July 2026 video-understanding bench: 89.0/100 on basic understanding and 89.4/100 on the transition-analysis special, across ten film clips and five other video categories. But the most useful finding isn’t the ranking — it’s that the six models tested don’t watch video the same way at all. Kimi 2.7 Code (Moonshot AI) fuses frames into motion-aware blocks before its language model ever sees them; Seed reads timestamped frames and reassembles time in context; GPT-5.5 (OpenAI) and Claude Opus 4.8 (Anthropic) worked agent-style from extracted frames plus transcripts; and Gemini 3.1 Pro (Google) — the only native audio-plus-video model in the set — finished last this round at 75.5/100. Pick by task, not by leaderboard.

Scores at a glance

All scores on one normalized 100-point scale. How to read them: these are rubric-scored indexes, useful for deciding which transcripts to read first — they are not ground-truth-validated measurements. The limitations section below is part of the result.

Which model for which video task

One warning that falls out of the architecture section below: if your only path to a model is an agent harness that samples video into screenshots — the default for coding agents like Claude Code or Codex today — you are running at reduced capability. Static frames survive that path; pacing, motion quality, and cut timing do not.

Three ways a model can “watch” a video

The scores make more sense once you know what each model actually receives. Three distinct pipelines showed up in this bench, and each one decides in advance what the model can and cannot see.

Frame extraction (GPT-5.5 and Claude Opus 4.8, agent-style). The video is sampled into a stack of independent stills, usually with a separate audio transcript. Per-frame detail survives well; everything that happens between frames — a car accelerating, a camera pushing in, an edit rhythm suddenly doubling — has to be imagined by the language model. Good for static checks; structurally blind to motion.

Fused clip encoding (Kimi 2.7 Code). Short runs of consecutive frames are packed into spatio-temporal blocks before the language model sees them, so the vision encoder itself understands “a hand is rising, the light is dimming, the camera is pulling back.” Motion arrives pre-understood, which is why Kimi’s short-clip action and transition readings feel like a director’s notes. The cost: per-frame detail gets diluted in the packing.

Timestamped sampling (Seed 2.1 Pro and Seed 2.0 Lite). Every sampled frame keeps single-image fidelity and carries its timestamp, with sampling density adapting to the task. The language model reassembles motion by reasoning over the timeline. This scales to long videos, meeting recordings, and screen captures with small UI text — while dense, fast action still has to be stitched together by the model.

Native audio+video (Gemini 3.1 Pro) is the fourth path: sound and picture enter the model together, no transcript step. In principle that’s the strongest position; in this round, low-FPS visual sampling blind spots and loosely controlled output granularity kept its scores at the bottom of the table. Architecture opens a ceiling — it doesn’t guarantee the floor.

One 30-second clip, six readings

The film set’s first clip is the climactic mansion shootout from Scarface — about 30 seconds of build-up, a grenade blast, then spray-fire montage. It’s a good separator because it mixes hard cuts, in-scene smoke occlusion, and real transitions, and models that can’t tell those apart give themselves away. Every model answered the same “basic conclusion” field. Excerpts (the bench ran in Chinese; quotes below are translated, except Gemini’s, which replied in English on its own):

Kimi 2.7 Code: “The edit builds in three beats — line, pause, eruption: a tracking long take establishes pressure, cross-cut shot/reverse-shots form the standoff, a signature line triggers the grenade blast, and the sequence collapses into rapid, fragmented spray-fire montage. In about 30 seconds the audiovisual language compresses Tony’s mania, isolation, and violent aesthetics.”

Claude Opus 4.8: “Violent climax of a classic crime film (visually consistent with the mansion-shootout ending of Scarface; Chinese subtitles present). The 30s structure is a textbook build-and-release: the first ~22s use long takes and slow cuts to establish dread; at 22.7s it switches to high-frequency hard cuts, syncing muzzle flashes with the edit. The editing logic serves the emotion curve, not information.”

GPT-5.5: “Interior gunfight/standoff; a suited man in a luxury office returns fire after being ambushed. The edit maintains spatial continuity through the room’s main axis, the doorway attack, and smoke occlusion.”

Gemini 3.1 Pro: “Iconic action sequence utilizing rapid hard cuts, extreme sound design, and dynamic camera angles to convey chaotic violence and character mania.”

Three things worth noticing, because they generalize across the whole bench. First, the specificity gradient: Kimi and Claude give time-anchored, falsifiable structure (“at 22.7s”, “three beats”); Gemini’s line is accurate but contains nothing you could check. Second, hedging discipline: Claude was the only model that declined to positively identify the film, marking it “visually consistent with” Scarface instead of asserting it — mid-pack on score, best in the set at knowing what it didn’t know. Third, verbosity is not quality: across the multi-type sets Gemini wrote some of the longest field responses while scoring lowest. Length is not evidence; timestamps are.

Model by model

Seed 2.1 Pro — 89.0 basic / 89.4 transitions

The strongest overall this round. Type recognition, timeline reconstruction, and visual-detail coverage led the set, and the timestamped pipeline shows in how often its claims arrive with usable time references. Double-check before relying on it: strong recognition did not always mean reliable audio-visual claims, long outputs occasionally truncated, and fine cut-point assertions still need evidence review.

Kimi 2.7 Code — 86.9 basic / 83.7 transitions

The best motion intuition in the set. Fused clip encoding is visible in the transcripts: action beats, force, and cut rhythm on short clips read the way an editor would describe them, and its judgments about what a clip is for were consistently usable. Watch for: implicit cuts hidden inside high-motion sequences, small on-screen text, and external-knowledge claims that need verification.

GPT-5.5 — 86.8 basic / 84.3 transitions

Explanation-first. The strongest at organizing theme, editing intent, and audio-visual relations into prose you could paste into a report. Frame-level cut timing needs tool assistance, it goes conservative when the audio track is quiet, and its plausible extrapolations are good enough that you have to make sure they stay labeled as extrapolations.

Claude Opus 4.8 — 85.1 basic / 84.2 transitions

Ran agent-style — extracted frames plus audio transcription — and turned that constraint into the cleanest evidence discipline in the bench: tool-cited claims, explicit uncertainty fields, hedged identifications. No native hearing, so speech analysis depends on the transcript, and motion detail is bounded by frame sampling.

Seed 2.0 Lite — 84.0 basic / 89.5 transitions

The surprise of the round: a lightweight tier that posted the best transition-special score in the set and multi-type field completeness close to its bigger sibling. As a cost/quality control group it earned a permanent slot. Film-set basic understanding trails 2.1 Pro, and complex audio-visual explanations need review.

Gemini 3.1 Pro — 75.5 basic / 65.3 transitions

The only native audio+video model in the set finished last after score normalization onto the shared 100-point scale. The bench notes point at low-FPS visual sampling blind spots and output granularity that needed heavy prompt constraint — depth was not the problem; its responses were among the longest. Treat this as a this-round result for this rubric, not a verdict on native-AV architecture.

What these scores can’t tell you

This section is part of the result, not a disclaimer.

• The scores are a reading index, not a ruling. Field responses were scored against a shared weighted rubric and normalized to 100 points. No human-labeled ground truth sat underneath this round — so “89.0 vs 86.9” is a strong signal about which transcripts read better, not a calibrated capability gap.
• No cut-point ground truth yet. Transition claims were judged on evidence quality and coherence, not against a human-labeled list of true cuts. A model can sound right about an edit it misplaced.
• Pipelines differ by design. Each model ran its native path (frames, fused blocks, timestamps, native AV). That’s deliberate — it’s what you actually get as a user — but it means sampling differences are baked into the numbers.
• The next round closes these gaps: unified 8 fps sampling across models, uniform audio transcription, human-labeled cut points, and one scoring scale — upgrading the bench from “can it analyze” to “can it be verified.” This page will be updated when that lands.

How this bench was run

Run by AgentsBench in July 2026. Six checkpoints as tested: Kimi 2.7-code, Seed 2.1 Pro, Seed 2.0 Lite, Claude Opus 4.8, Gemini 3.1 Pro, GPT-5.5.

Material. Ten film clips chosen to separate models — an interior-gunfight climax (Scarface finale), a fast travel montage, a black-and-white rain battle, a graphic product ad, the bone-to-spacecraft match cut, a desert/snow car commercial, a long-take spatial reveal, a setup-to-eruption gunfight, a CG siege battle, and an in-car dialogue reversal — plus five more categories: narrative, explainer, AI-generated video, UI/demo recordings, and talking-head.

Fields. On the film set, nine base fields (summary, timeline, transition rhythm, audio analysis, audio-visual relations, edit-point reasoning, shot language, visual language, open questions) plus seven transition-special fields; roughly ten fields per category elsewhere — UI clips get step breakdowns and interface-state analysis, talking-head clips get subtitle recognition and speech-to-visual correspondence, AI-generated clips get character/scene/style-consistency and artifact-drift checks.

Scoring. Every field response scored against a shared rubric with per-field weights, all models normalized to one 100-point scale. Reference evidence for judging: a 2 fps frame timeline and an audio waveform per clip. Prompts and rubric ran in Chinese; excerpts above are translated (Gemini’s reply was in English as produced). The reading order we used — transcripts first, scores second — is the one we recommend.

Why this bench exists at all: video-understanding models are the perception layer of AI video agents — the thing that lets an agent watch, QA, and iterate on its own output instead of shipping blind. How we grade and label claims across this site is documented in the methodology. Spotted an error in a quote, a score, or a claim? Send the source to the contact address in the footer — corrections get dated and logged.

Frequently asked questions