BVB: Benchmarking Agentic Video Understanding
via Programmatic Reconstruction in Blender

If an agent truly understands a video, it can reconstruct it programmatically.

Source video ARKitScenes 42446049
Blender reconstruction

Dual VQA 62.5Latent Similarity 90.0

0.0 / 30 s

Both previews are time-scaled to 30 seconds, so one point on the timeline shows the same relative moment in each video. Scores are for this scene and model.

Overview

Video understanding benchmarks usually ask a model questions about a video and grade the answers. BVB tests understanding through reconstruction instead. An agent watches a real indoor video and writes Python code for Blender that reconstructs it as an animated 3D scene, with a camera that follows the path of the original video. A faithful reconstruction requires the agent to understand the layout of the room, the objects in it, and how the view changes over time.

Every agent works under the same conditions. It sees only the video frames, runs code in the same sandbox through the Mini-BVB harness, and has the same cost limit for each scene. The saved scene is rendered and compared with the source video on two axes. Dual VQA measures whether the reconstruction keeps the spatiotemporal facts that a VLM judge can answer from the source video. Latent Similarity measures how closely the reconstruction matches the source in layout and motion. Overall combines the two and is high only when both are high.

The leaderboard covers 71 configurations from 14 providers. It is updated as new models are evaluated.

Leaderboard

Each configuration is one model at one reasoning effort setting. Baseline runs use the Mini-BVB harness with no fine-tuning. Scores are on a 0 to 100 scale. Entries marked reported link to the provider’s published results; unreported metrics are shown as n/a.

Overall score of the configuration with the highest Overall for each model. Higher is better.

BVB leaderboard. Scores on a 0 to 100 scale. Cost is the mean agent spend per scene in US dollars.
Rank Model and reasoning effort
1Claude Fable 5.2 xhigh76.4466.587.1$2.608
2GPT-6 Astra xhigh76.3364.788.9$1.718
3GPT-6.1 Sol xhigh75.4162.489.7$0.500
4GPT-6.1 Terra xhigh75.1663.188.2$0.296
5GPT-6 Astra high73.9560.688.6$1.258
6GPT-6 Sol xhigh73.4162.285.6$0.397
7Gemini 4 Argon high72.6961.285.2$0.313
8Claude Opus 5.5 xhigh72.3961.384.5$1.950
9Claude Sonnet 5.5 xhigh72.2761.583.9$1.183
10GPT-5.6 Sol xhigh71.7859.485.4$0.778
11Inkling 2 Small max71.5659.984.2$0.257
12Grok-4.6 xhigh70.8658.784.2$1.682
13GPT-6 Luna xhigh70.3258.683.1$0.011
14Grok-4.7 xhigh70.0758.882.4$1.482
15DeepSeek V4.1 Flash 1017 max open-weight70.0258.782.4$0.037
16Claude Opus 5 high69.5358.681.4$2.157
17Qwen3.8-Max high69.3958.481.3$0.599
18GPT-5.6 Sol high69.2155.983.9$0.419
19Glimpse max68.8658.080.7$0.365
20GPT-5.6 Terra high68.7656.382.4$0.122
21GPT-5.5 high68.5556.581.8$0.956
22Claude Haiku 5.5 xhigh68.3757.580.1$0.027
23GPT-5.6 Terra xhigh68.3755.282.9$0.235
24Inkling 2 Mini open-weight reported68.3557.779.9$0.162
25Claude Opus 5.5 high68.1558.678.5$0.493
26GPT-5.6 Sol none68.0856.680.6$0.212
27GLM-5.3-Flash xhigh open-weight67.9158.078.7$0.024
28GPT-5.6 Sol medium67.9055.581.6$0.271
29Muse Wonder xhigh67.4257.777.9$1.326
30Muse Spark 1.3 xhigh67.1355.679.7$0.229
31DeepSeek V4.1 Flash max open-weight66.8257.876.4$0.018
32GPT-5.5 medium66.7054.280.5$0.562
33GPT-5.6 Luna high66.6953.781.1$0.080
34Grok-4.5 high66.4254.979.0$0.300
35GPT-5.6 Sol low66.2854.479.3$0.200
36GPT-5.6 Terra medium66.2755.078.5$0.149
37Gemini 3.8 Flash high65.9255.477.4$0.628
38GPT-5.5 low65.6354.777.5$0.320
39GPT-5.6 Luna medium65.5953.678.8$0.047
40Claude Sonnet 4.6 none65.2956.075.3$0.432
41GPT-5.2 high65.1855.475.8$0.578
42GPT-5.6 Terra low64.6753.676.8$0.146
43Claude Sonnet 5 none64.6455.474.6$0.644
44Gemini 3.1 Pro high64.6255.574.5$0.667
45Claude Sonnet 5 high64.1654.774.4$0.421
46Claude Opus 4.6 none64.0954.574.5$0.672
47Claude Sonnet 4.6 high64.0554.074.9$0.747
48Claude Opus 4.7 none63.5755.172.6$0.369
49Muse Spark 1.2 xhigh63.4052.475.4$0.235
50GPT-5.6 Luna low63.2452.075.6$0.027
51Claude Opus 4.8 none62.5553.672.2$0.325
52Claude Opus 4.8 high62.2153.271.9$0.460
53Gemini 3 Flash high62.1455.769.0$0.083
54Claude Opus 4.7 high62.0853.771.1$0.351
55GPT-5.2 none61.8753.970.4$0.120
56GPT-5.4 none60.5353.568.0$0.203
57GPT-5.5 none60.3448.473.6$0.163
58GPT-5.4 Mini none60.1653.267.5$0.034
59Kimi-K2.5 none open-weight59.6450.869.2$0.150
60GLM-5V-Turbo none59.0051.567.0$0.276
61Inkling Small xhigh59.0051.567.0n/a
62MiniMax-M3 none open-weight58.1850.965.9$0.058
63Qwen3.5-122B none open-weight56.8550.363.8$0.225
64Grok-4.3 high56.5849.364.3$0.101
65Qwen3.5-397B none open-weight56.3249.563.5$0.179
66Seed-2.0-Mini none55.8750.261.9$0.028
67Qwen3.6-Plus none53.9446.262.2$0.170
68Grok-4.3 none52.3747.857.2$0.047
69Nemotron 3.5 Super VL max open-weight52.1147.157.4$0.122
70Seed-2.0-Lite none51.9045.858.4$0.039
71Qwen3.5-27B none open-weight50.7444.557.3$0.073
Leaderboard updated November 20, 2026 Download CSV

For the 288-scene baseline, Latent Similarity is the mean over scenes, and Dual VQA pools the 1,962 questions GPT-5.4 Mini answers correctly on the source videos. A scene with no renderable file scores zero on both axes. Rank follows the sorted column over all configurations and does not change when you filter the table. Ties in displayed scores are broken by the unrounded values.

Layout, Motion, and Dual VQA by task

Layout and Motion are the two parts of Latent Similarity. The task columns split Dual VQA by question type. Relative direction combines the easy, medium, and hard subsets.

Layout, Motion, and Dual VQA task scores for each configuration in the current filters and ranking.
Latent Similarity parts Dual VQA by task
Model and reasoning effort Layout Motion Object
count
Absolute
distance
Object
size
Room
size
Relative
distance
Relative
direction
Route
planning
Appearance
order
Claude Fable 5.2 xhigh85.988.347.056.676.143.859.077.381.865.7
GPT-6 Astra xhigh87.690.251.052.676.546.248.177.575.860.1
GPT-6.1 Sol xhigh88.491.052.045.172.945.451.973.872.756.8
GPT-6.1 Terra xhigh86.989.545.546.874.151.552.772.681.860.6
GPT-6 Astra high87.390.048.533.573.733.150.274.875.857.7
GPT-6 Sol xhigh84.287.046.048.073.159.248.172.674.255.4
Gemini 4 Argon high83.986.543.546.270.538.553.674.674.259.2
Claude Opus 5.5 xhigh83.285.740.545.173.746.951.072.975.856.3
Claude Sonnet 5.5 xhigh82.685.345.048.670.344.648.176.580.355.9
GPT-5.6 Sol xhigh84.086.738.039.969.053.148.571.980.356.8
Inkling 2 Small max82.785.7n/an/an/an/an/an/an/an/a
Grok-4.6 xhigh82.985.540.542.266.246.946.473.678.856.3
GPT-6 Luna xhigh81.684.732.044.568.441.544.474.878.859.2
Grok-4.7 xhigh81.083.842.547.469.542.347.769.975.852.1
DeepSeek V4.1 Flash 1017 max open-weight80.884.034.540.570.953.846.969.978.854.0
Claude Opus 5 high79.882.942.046.266.443.846.071.678.856.8
Qwen3.8-Max high79.982.842.537.665.845.445.672.474.262.4
GPT-5.6 Sol high82.585.334.537.665.040.846.971.981.848.8
Glimpse max79.282.237.045.766.742.343.571.677.359.2
GPT-5.6 Terra high81.083.937.534.166.245.438.973.374.255.4
GPT-5.5 high80.483.234.534.765.846.245.274.371.251.6
Claude Haiku 5.5 xhigh78.681.733.545.767.738.547.770.477.356.3
GPT-5.6 Terra xhigh81.684.333.030.665.843.141.470.977.355.4
Inkling 2 Mini open-weight reported78.481.4n/an/an/an/an/an/an/an/a
Claude Opus 5.5 high77.179.835.045.168.644.646.074.374.254.0
GPT-5.6 Sol none79.182.031.037.067.137.749.475.672.748.8
GLM-5.3-Flash xhigh open-weight77.080.335.038.264.546.245.676.371.261.0
GPT-5.6 Sol medium80.183.135.040.564.333.838.973.374.256.3
Muse Wonder xhigh76.379.532.546.267.946.946.070.775.854.5
Muse Spark 1.3 xhigh78.381.235.036.464.135.445.670.272.759.6
DeepSeek V4.1 Flash max open-weight74.878.129.051.467.943.845.273.369.754.5
GPT-5.5 medium79.181.925.535.863.941.542.768.781.855.9
GPT-5.6 Luna high79.682.728.536.465.234.636.872.174.251.2
Grok-4.5 high77.680.435.039.364.142.344.868.275.850.2
GPT-5.6 Sol low77.880.828.541.063.035.443.570.278.854.5
GPT-5.6 Terra medium77.080.136.034.162.439.247.770.769.754.9
Gemini 3.8 Flash high76.078.841.535.865.437.745.268.768.251.6
GPT-5.5 low76.079.035.033.561.842.342.371.677.354.9
GPT-5.6 Luna medium77.280.523.031.862.235.442.774.172.756.3
Claude Sonnet 4.6 none73.876.930.537.063.943.844.474.168.257.3
GPT-5.2 high74.477.239.038.260.946.237.772.672.757.7
GPT-5.6 Terra low75.278.427.036.462.438.542.770.975.851.6
Claude Sonnet 5 none72.976.330.039.962.436.246.471.977.357.7
Gemini 3.1 Pro high72.976.130.044.565.243.140.269.775.854.9
Claude Sonnet 5 high72.776.129.037.065.428.540.673.174.256.8
Claude Opus 4.6 none73.075.928.039.363.730.045.271.978.853.1
Claude Sonnet 4.6 high73.476.535.532.462.834.639.773.672.751.6
Claude Opus 4.7 none70.974.336.041.061.534.643.172.475.854.9
Muse Spark 1.2 xhigh74.176.733.532.462.836.940.665.571.252.6
GPT-5.6 Luna low73.977.320.535.860.534.643.168.075.855.9
Claude Opus 4.8 none70.474.130.537.062.035.444.870.474.249.8
Claude Opus 4.8 high69.973.929.033.560.930.043.168.775.861.5
Gemini 3 Flash high67.370.731.538.766.440.838.573.877.352.1
Claude Opus 4.7 high69.372.933.542.261.535.439.768.269.756.3
GPT-5.2 none68.772.230.041.060.931.540.671.969.758.2
GPT-5.4 none66.369.623.541.664.843.137.268.962.155.4
GPT-5.5 none71.875.421.524.359.830.837.767.569.744.6
GPT-5.4 Mini none65.769.419.545.160.923.849.469.977.354.9
Kimi-K2.5 none open-weight67.770.824.032.959.233.841.466.762.155.9
GLM-5V-Turbo none65.468.725.542.261.536.937.767.577.344.1
Inkling Small xhighn/an/an/an/an/an/an/an/an/an/a
MiniMax-M3 none open-weight64.367.620.035.861.530.841.467.574.249.8
Qwen3.5-122B none open-weight62.065.616.532.963.925.433.572.668.247.9
Grok-4.3 high62.865.922.036.457.532.337.769.962.145.1
Qwen3.5-397B none open-weight61.965.216.032.960.929.243.566.765.247.4
Seed-2.0-Mini none59.864.017.031.861.520.838.573.172.747.9
Qwen3.6-Plus none60.863.720.531.854.130.034.760.468.251.2
Grok-4.3 none55.459.013.029.559.030.036.868.063.646.5
Nemotron 3.5 Super VL max open-weight55.759.022.534.756.825.440.662.354.545.1
Seed-2.0-Lite none56.260.610.533.557.930.034.365.360.639.0
Qwen3.5-27B none open-weight55.858.913.527.254.526.235.661.969.743.2

Benchmark

BVB uses the 288 real indoor videos and 5,130 spatiotemporal questions of VSI-Bench, drawn from ARKitScenes, ScanNet, and ScanNet++. The agent sees only video frames. It gets no depth maps, segmentation, or 3D ground truth.

Every agent runs through the same harness, Mini-BVB, with the same system prompt. The harness has two tools. frames returns video frames by timestamp or uniform count, and bash runs commands, including Python for Blender, in a fresh Docker sandbox. The agent decides how much of the video to view and how long to work, and the only limit is a cost limit of $3 per scene. External asset libraries are disallowed, so geometry is built from primitives and basic Blender operations. The saved scene is rendered from its animated camera and scored without re-entering the agent loop.

Videos
288 from ARKitScenes, ScanNet, and ScanNet++
Questions
5,130 in 8 task types
Agent input
Video frames only
Tools
frames and bash
Sandbox
Docker with Python, Blender 4.2, and FFmpeg
Cost limit
$3 per scene
Output
result.blend, rendered with EEVEE
Stage 1. The model reads frames from input.mp4 and runs bash, Python, Blender, and FFmpeg in a Docker sandbox through the Mini-BVB harness. The cost limit is 3 dollars per scene. The model writes a .blend file. Stage 2. EEVEE renders the scene. Dual VQA asks a VLM the same question on the source video and on the rendered video. Latent Similarity compares frozen V-JEPA features of the two videos for layout and motion.
BVB agent runs and evaluation. A model alternates between viewing source frames and running Python code in a Docker sandbox under a cost limit, then saves an animated Blender scene. Dual VQA measures how well the rendered video retains answers the judge gets right on the source, and Latent Similarity compares layout and motion using frozen V-JEPA features.

Results

Latent Similarity is higher than Dual VQA for all 71 configurations with both metrics available, by 9.4 to 28.0 points. The highest Dual VQA among these configurations is 66.5, so each loses at least 33.5% of the source-correct answers in its reconstructions.

Score and cost

The horizontal axis is the mean agent cost per scene on a log scale. Select a point, or use the arrow keys, to see its scores. A thin line then joins the reasoning effort settings of the same model.

Full results and analysis are in the paper.

Citation

@misc{tang2026bvb,
  title = {BVB: Benchmarking Agentic Video Understanding via Programmatic Reconstruction in Blender},
  author = {Yolo Y. Tang and Daiki Shimada and Jiayue Meng and Jing Bi and Pinxin Liu and Yicheng Wang and Yunzhong Xiao and Zhangyun Tan and Zeliang Zhang and Chao Huang and Susan Liang and Qianxiang Shen and Luchuan Song and Ali Vosoughi and Mingqian Feng and Melika Filvantorkaman and Chenliang Xu},
  year = {2026},
  eprint = {2609.15478},
  archivePrefix = {arXiv},
  primaryClass = {cs.CV},
  url = {https://arxiv.org/abs/2609.15478}
}