BVB: Benchmarking Agentic Video Understanding
via Programmatic Reconstruction in Blender

Yolo Y. Tang¹Daiki Shimada²Jiayue Meng¹Jing Bi¹Pinxin Liu¹Yicheng Wang³Yunzhong Xiao³Zhangyun Tan¹Zeliang Zhang¹Chao Huang¹Susan Liang¹Qianxiang Shen⁴Luchuan Song¹Ali Vosoughi¹Mingqian Feng¹Melika Filvantorkaman¹Chenliang Xu¹

¹ University of Rochester² Sony Group Corporation³ Carnegie Mellon University⁴ University of Washington

Read the paper arXiv Code Explore results

“If an agent truly understands a video, it can reconstruct it programmatically.”

Real videoARKitScenes · 42446049

Blender reconstructionGPT-6 Astra · high

Matched relative video times. Each full clip is shown over a 30-second preview; scores are for this scene. Examples are drawn from the paper.

The paper

Abstract

Multimodal agents can create complex videos in software such as Blender by coding without relying on diffusion models. Yet video understanding benchmarks still evaluate models mainly through question answering. If an agent truly understands a video, it can reconstruct it programmatically. We introduce BVB, Blender-VideoBench, a benchmark that tests this ability by asking agents to reconstruct real-world videos as animated Blender scenes. To ensure fair comparison, each agent programs the reconstruction through a lightweight harness, Mini-BVB, in an identical sandbox under a shared cost limit. The benchmark renders each reconstruction from its animated camera and evaluates it on two axes: (1) Dual VQA measures how many spatiotemporal facts the reconstruction preserves. (2) Latent Similarity measures how closely the reconstruction matches the source video perceptually. Our overall score, a square-root mean, favors balanced performance. We evaluate 51 configurations from 10 model families and analyze semantic retention, perceptual similarity, reasoning effort, and cost. The best model reaches 88.6 Latent Similarity but retains only 53.7% of the source-correct spatiotemporal answers. Additional reasoning improves visual similarity but does not close this gap in factual accuracy. In a blind study with 15 raters and five configurations, Latent Similarity correlates strongly with human preference. These results show that programmatic reconstruction is a viable test of agentic video understanding, and that semantic retention remains the main challenge.

01 / Full results

51 configurations.
Two ways to understand.

We evaluate frontier multimodal models as zero-shot reconstruction agents, without fine-tuning. Both axes are reported beside Overall.

51 configurations · ranked by OverallDownload results

All 51 BVB configurations. Scores on a 0–100 scale. Overall is the square-root mean of Dual VQA and Latent Similarity. Cost and runtime are per-scene means.
Rank	Model & Reasoning Effort	Harness
1	GPT-6 Astrahigh	mini-BVB	70.07	53.7	88.6	87.3	90.0	1.258	14.00
2	GPT-5.6 Solxhigh	mini-BVB	67.49	51.7	85.4	84.0	86.7	0.778	4.96
3	Grok-4.6xhigh	mini-BVB	67.17	52.1	84.2	82.9	85.5	1.682	35.51
4	Qwen3.8-Maxhigh	mini-BVB	66.24	52.7	81.3	79.9	82.8	0.599	12.89
5	Claude Opus 5high	mini-BVB	66.21	52.6	81.4	79.8	82.9	2.157	7.97
6	GPT-5.6 Solhigh	mini-BVB	65.73	49.8	83.9	82.5	85.3	0.419	2.21
7	GPT-5.6 Terraxhigh	mini-BVB	64.79	48.9	82.9	81.6	84.3	0.235	4.76
8	GPT-5.6 Terrahigh	mini-BVB	64.76	49.2	82.4	81.0	83.9	0.122	3.37
9	GPT-5.5high	mini-BVB	64.53	49.3	81.8	80.4	83.2	0.956	6.77
10	GPT-5.6 Sol	mini-BVB	64.09	49.5	80.6	79.1	82.0	0.212	1.20
11	GPT-5.6 Solmedium	mini-BVB	64.01	48.5	81.6	80.1	83.1	0.271	1.60
12	GLM-5.3-Flashxhighopen-weight	mini-BVB	63.96	50.8	78.7	77.0	80.3	0.024	6.41
13	GPT-5.6 Lunahigh	mini-BVB	63.58	48.2	81.1	79.6	82.7	0.080	1.02
14	Muse Spark 1.3xhigh	mini-BVB	63.40	48.9	79.7	78.3	81.2	0.229	6.65
15	GPT-5.6 Sollow	mini-BVB	63.23	49.0	79.3	77.8	80.8	0.200	1.04
16	Grok-4.5high	mini-BVB	62.81	48.4	79.0	77.6	80.4	0.300	2.16
17	GPT-5.5low	mini-BVB	62.80	49.6	77.5	76.0	79.0	0.320	1.71
18	GPT-5.5medium	mini-BVB	62.56	46.9	80.5	79.1	81.9	0.562	4.14
19	GPT-5.6 Terramedium	mini-BVB	62.43	48.2	78.5	77.0	80.1	0.149	8.42
20	Claude Sonnet 4.6	mini-BVB	62.37	50.6	75.3	73.8	76.9	0.432	2.92
21	Gemini 3.1 Prohigh	mini-BVB	62.23	51.1	74.5	72.9	76.1	0.667	6.18
22	Gemini 3.8 Flashhigh	mini-BVB	62.08	48.4	77.4	76.0	78.8	0.628	10.63
23	GPT-5.6 Lunamedium	mini-BVB	61.62	46.5	78.8	77.2	80.5	0.047	0.57
24	Claude Sonnet 5	mini-BVB	61.24	49.2	74.6	72.9	76.3	0.644	3.15
25	GPT-5.2high	mini-BVB	61.08	47.9	75.8	74.4	77.2	0.578	7.45
26	GPT-5.6 Terralow	mini-BVB	61.07	47.1	76.8	75.2	78.4	0.146	5.82
27	Claude Sonnet 5high	mini-BVB	60.63	48.3	74.4	72.7	76.1	0.421	3.08
28	Claude Sonnet 4.6high	mini-BVB	60.53	47.7	74.9	73.4	76.5	0.747	22.86
29	GPT-5.6 Lunalow	mini-BVB	60.33	46.8	75.6	73.9	77.3	0.027	0.35
30	Claude Opus 4.7	mini-BVB	60.32	49.2	72.6	70.9	74.3	0.369	1.53
31	Claude Opus 4.6	mini-BVB	60.23	47.5	74.5	73.0	75.9	0.672	2.33
32	Muse Spark 1.2xhigh	mini-BVB	59.90	46.2	75.4	74.1	76.7	0.235	6.65
33	Claude Opus 4.7high	mini-BVB	59.17	48.3	71.1	69.3	72.9	0.351	1.52
34	Claude Opus 4.8	mini-BVB	59.07	47.2	72.2	70.4	74.1	0.325	1.06
35	Gemini 3 Flashhigh	mini-BVB	58.91	49.6	69.0	67.3	70.7	0.083	1.96
36	Claude Opus 4.8high	mini-BVB	58.46	46.4	71.9	69.9	73.9	0.460	1.52
37	GPT-5.2	mini-BVB	57.95	46.7	70.4	68.7	72.2	0.120	1.25
38	GPT-5.5	mini-BVB	57.03	42.6	73.6	71.8	75.4	0.163	0.75
39	GPT-5.4	mini-BVB	56.80	46.6	68.0	66.3	69.6	0.203	1.07
40	GPT-5.4 Mini	mini-BVB	56.51	46.5	67.5	65.7	69.4	0.034	0.53
41	GLM-5V-Turbo	mini-BVB	56.47	46.8	67.0	65.4	68.7	0.276	6.16
42	Kimi-K2.5open-weight	mini-BVB	55.91	44.0	69.2	67.7	70.8	0.150	6.24
43	MiniMax-M3open-weight	mini-BVB	55.29	45.6	65.9	64.3	67.6	0.058	3.06
44	Grok-4.3high	mini-BVB	54.08	44.7	64.3	62.8	65.9	0.101	1.69
45	Qwen3.5-122Bopen-weight	mini-BVB	53.50	44.1	63.8	62.0	65.6	0.225	5.16
46	Qwen3.5-397Bopen-weight	mini-BVB	52.75	43.0	63.5	61.9	65.2	0.179	13.99
47	Seed-2.0-Mini	mini-BVB	52.24	43.4	61.9	59.8	64.0	0.028	4.25
48	Qwen3.6-Plus	mini-BVB	51.28	41.4	62.2	60.8	63.7	0.170	11.73
49	Grok-4.3	mini-BVB	50.59	44.4	57.2	55.4	59.0	0.047	0.54
50	Seed-2.0-Lite	mini-BVB	49.47	41.3	58.4	56.2	60.6	0.039	1.73
51	Qwen3.5-27Bopen-weight	mini-BVB	47.50	38.6	57.3	55.8	58.9	0.073	6.08

All 288 test scenes are included. Failed reconstructions score zero on both axes. Rank follows the selected metric across the full pool; filtering does not renumber it. Tied displayed scores are ordered using full precision.

Task-level Dual VQA breakdown 8 spatiotemporal tasks

Retention is conditioned on the questions the judge answers correctly on the source. Relative direction combines the easy, medium, and hard subsets.

Dual VQA task scores for each configuration in the current filters and ranking.
Model & Reasoning Effort	Harness	Object count	Absolute distance	Object size	Room size	Relative distance	Relative direction	Route planning	Appearance order
GPT-6 Astrahigh	mini-BVB	48.5	33.5	73.7	33.1	49.0	50.0	54.8	36.0
GPT-5.6 Solxhigh	mini-BVB	38.0	39.9	69.0	53.1	43.6	50.7	47.3	16.0
Grok-4.6xhigh	mini-BVB	40.5	42.2	66.2	46.9	42.7	52.5	59.1	24.0
Qwen3.8-Maxhigh	mini-BVB	42.5	37.6	65.8	45.4	38.6	56.1	72.0	30.0
Claude Opus 5high	mini-BVB	42.0	46.2	66.4	43.8	44.0	52.0	65.6	16.0
GPT-5.6 Solhigh	mini-BVB	34.5	37.6	65.0	40.8	37.8	52.2	63.4	26.0
GPT-5.6 Terraxhigh	mini-BVB	33.0	30.6	65.8	43.1	40.2	50.7	62.4	12.0
GPT-5.6 Terrahigh	mini-BVB	37.5	34.1	66.2	45.4	30.3	52.5	63.4	16.0
GPT-5.5high	mini-BVB	34.5	34.7	65.8	46.2	37.8	50.5	63.4	12.0
GPT-5.6 Sol	mini-BVB	31.0	37.0	67.1	37.7	40.2	50.7	61.3	22.0
GPT-5.6 Solmedium	mini-BVB	35.0	40.5	64.3	33.8	41.1	48.0	61.3	18.0
GLM-5.3-Flashxhighopen-weight	mini-BVB	35.0	38.2	64.5	46.2	46.9	51.7	54.8	28.0
GPT-5.6 Lunahigh	mini-BVB	28.5	36.4	65.2	34.6	37.3	54.4	54.8	10.0
Muse Spark 1.3xhigh	mini-BVB	35.0	36.4	64.1	35.4	40.2	51.2	62.4	20.0
GPT-5.6 Sollow	mini-BVB	28.5	41.0	63.0	35.4	39.4	54.4	62.4	22.0
Grok-4.5high	mini-BVB	35.0	39.3	64.1	42.3	39.8	47.3	59.1	14.0
GPT-5.5low	mini-BVB	35.0	33.5	61.8	42.3	44.8	54.4	55.9	26.0
GPT-5.5medium	mini-BVB	25.5	35.8	63.9	41.5	37.8	50.0	51.6	12.0
GPT-5.6 Terramedium	mini-BVB	36.0	34.1	62.4	39.2	43.2	47.3	55.9	34.0
Claude Sonnet 4.6	mini-BVB	30.5	37.0	63.9	43.8	43.2	57.4	62.4	14.0
Gemini 3.1 Prohigh	mini-BVB	30.0	44.5	65.2	43.1	41.1	55.1	63.4	20.0
Gemini 3.8 Flashhigh	mini-BVB	41.5	35.8	65.4	37.7	42.7	44.9	51.6	18.0
GPT-5.6 Lunamedium	mini-BVB	23.0	31.8	62.2	35.4	41.9	52.5	53.8	14.0
Claude Sonnet 5	mini-BVB	30.0	39.9	62.4	36.2	40.7	54.9	64.5	18.0
GPT-5.2high	mini-BVB	39.0	38.2	60.9	46.2	32.4	51.2	59.1	12.0
GPT-5.6 Terralow	mini-BVB	27.0	36.4	62.4	38.5	40.2	49.0	57.0	24.0
Claude Sonnet 5high	mini-BVB	29.0	37.0	65.4	28.5	39.8	54.7	54.8	10.0
Claude Sonnet 4.6high	mini-BVB	35.5	32.4	62.8	34.6	36.9	52.9	58.1	12.0
GPT-5.6 Lunalow	mini-BVB	20.5	35.8	60.5	34.6	43.2	51.2	67.7	18.0
Claude Opus 4.7	mini-BVB	36.0	41.0	61.5	34.6	42.3	54.9	57.0	8.0
Claude Opus 4.6	mini-BVB	28.0	39.3	63.7	30.0	46.1	48.0	50.5	24.0
Muse Spark 1.2xhigh	mini-BVB	33.5	32.4	62.8	36.9	34.0	47.5	62.4	10.0
Claude Opus 4.7high	mini-BVB	33.5	42.2	61.5	35.4	38.2	53.7	55.9	14.0
Claude Opus 4.8	mini-BVB	30.5	37.0	62.0	35.4	41.9	48.8	57.0	18.0
Gemini 3 Flashhigh	mini-BVB	31.5	38.7	66.4	40.8	39.0	52.5	58.1	18.0
Claude Opus 4.8high	mini-BVB	29.0	33.5	60.9	30.0	41.1	51.7	54.8	16.0
GPT-5.2	mini-BVB	30.0	41.0	60.9	31.5	40.2	50.7	50.5	12.0
GPT-5.5	mini-BVB	21.5	24.3	59.8	30.8	34.9	46.3	59.1	14.0
GPT-5.4	mini-BVB	23.5	41.6	64.8	43.1	32.8	48.5	50.5	16.0
GPT-5.4 Mini	mini-BVB	19.5	45.1	60.9	23.8	40.7	52.2	63.4	14.0
GLM-5V-Turbo	mini-BVB	25.5	42.2	61.5	36.9	32.8	52.9	59.1	12.0
Kimi-K2.5open-weight	mini-BVB	24.0	32.9	59.2	33.8	36.9	47.5	58.1	6.0
MiniMax-M3open-weight	mini-BVB	20.0	35.8	61.5	30.8	40.2	49.5	62.4	14.0
Grok-4.3high	mini-BVB	22.0	36.4	57.5	32.3	41.1	51.0	52.7	12.0
Qwen3.5-122Bopen-weight	mini-BVB	16.5	32.9	63.9	25.4	29.0	52.9	57.0	8.0
Qwen3.5-397Bopen-weight	mini-BVB	16.0	32.9	60.9	29.2	38.2	45.1	57.0	10.0
Seed-2.0-Mini	mini-BVB	17.0	31.8	61.5	20.8	31.1	52.9	61.3	4.0
Qwen3.6-Plus	mini-BVB	20.5	31.8	54.1	30.0	29.9	48.5	61.3	12.0
Grok-4.3	mini-BVB	13.0	29.5	59.0	30.0	40.2	52.5	68.8	12.0
Seed-2.0-Lite	mini-BVB	10.5	33.5	57.9	30.0	32.0	48.5	52.7	8.0
Qwen3.5-27Bopen-weight	mini-BVB	13.5	27.2	54.5	26.2	30.7	45.3	49.5	4.0

02 / The benchmark

From video to an
editable world.

BVB asks multimodal agents to reconstruct real-world videos as animated Blender scenes. Every model faces the same sandbox, prompt, and cost ceiling.

288real indoor videos

5,130spatiotemporal questions

51agent configurations

10model families

01 · Observe

Real indoor video

Video frames alone. No depth maps, segmentation, or 3D ground truth.

ARKitScenes · ScanNet · ScanNet++

02 · Reconstruct

Mini-BVB Harness

The agent inspects frames, writes Python, and runs Blender in a fresh sandbox.

frames + bash · $3 per scene

03 · Render

Animated Blender scene

Geometry built from primitives, with an animated camera along the source trajectory.

result.blend · Blender 4.2

Controlled comparison

External asset libraries are disallowed. Stage 2 scores the file and never re-enters the agent loop. If an agent fails to produce a renderable file, that scene scores zero on both axes.

DVSemantic retention

Dual VQA

Measures how many spatiotemporal facts the reconstruction preserves.

DV = |C_s ∩ C_r||C_s|

A VLM judge answers the same questions on source and rendered videos. Retention is scored only on questions answered correctly on the source.

Judge: GPT-5.4 mini · 16 frames per video

LSPerceptual similarity

Latent Similarity

Measures how closely the reconstruction matches the source video perceptually.

LS = Layout + Motion2

A frozen V-JEPA encoder compares spatial arrangement and temporal dynamics separately, then averages them.

V-JEPA 2.1 ViT-G · 64 frames per video

Two complementary axes

Strong performance on both.

We combine both axes into a square-root mean. More uneven performance receives a larger penalty.

Overall = (√DV + √LS2)²Both axes on a 0–100 scale

03 / What current agents reveal

Looking right is not
the same as being right.

The best model reaches 88.6 Latent Similarity but retains only 53.7% of the source-correct answers, so nearly half of verifiable semantic content is still lost after reconstruction.

Best configuration · GPT-6 Astra high

Visually plausible.
Semantically incomplete.

Current agents therefore build reconstructions that look right but get many facts wrong.

Dual VQA53.7%

Latent Similarity88.6

Overall70.07Rank 1 of 51

What does the frontier cost?

Cost and quality do not scale together.

The paper's cost frontier across 51 configurations, showing Overall score versus mean per-scene cost.

Mean Stage-1 spend per scene · logarithmic cost axis · higher score is better.
Dashed line: best score available at each cost. Select a point to inspect a configuration.

Selected configuration

GPT-6 Astra high

Overall: 70.07
Dual VQA: 53.7
Latent Similarity: 88.6
Cost / scene: $1.258

96%

of the best Overall at $0.778 per scene.
GPT-5.6 Sol · xhigh

91%

of the best Overall at $0.024 per scene.
GLM-5.3-Flash · xhigh

Semantic retention

Whole-scene facts are harder.

Object size and route planning have the highest mean retention, while appearance order and object count are lowest.

Object size63.0%

Route planning58.5%

Relative direction51.0%

Relative distance39.0%

Room size36.5%

Absolute distance36.5%

Object count29.3%

Appearance order16.1%

Mean retention across all 51 configurations.

Full task profiles from the paper

Reasoning effort

More reasoning, similar factual accuracy.

Additional reasoning improves visual similarity but does not close this gap in factual accuracy.

Dual VQA Latent Similarity

GPT-5.6 Sol · provider reasoning settings.
Longer runtime does not mean higher Overall.

All effort ladders from the paper

Use BVB

Build on the benchmark.

BVB provides a reproducible evaluation for agentic video understanding, built on real videos, shared budgets, and released editable artifacts.

Get the code Read the paper arXiv

Citation

@misc{tang2026bvb,
  title = {BVB: Benchmarking Agentic Video Understanding via Programmatic Reconstruction in Blender},
  author = {Yolo Y. Tang and
    Daiki Shimada and
    Jiayue Meng and
    Jing Bi and
    Pinxin Liu and
    Yicheng Wang and
    Yunzhong Xiao and
    Zhangyun Tan and
    Zeliang Zhang and
    Chao Huang and
    Susan Liang and
    Qianxiang Shen and
    Luchuan Song and
    Ali Vosoughi and
    Mingqian Feng and
    Melika Filvantorkaman and
    Chenliang Xu},
  year = {2026},
  eprint = {2609.15478},
  archivePrefix = {arXiv},
  primaryClass = {cs.CV},
  url = {https://arxiv.org/abs/2609.15478}
}