The benchmark at a glance
How well frontier VLMs understand driving scenes
Our scores in the paper's Table 10 layout. details
Paper, Table 10 as published (2025)
The paper's Table 10, as published in 2025. details
Tasks by audit result and saturation
Tasks by audit finding and saturation. details
With and without bounding boxes
TOTAL with boxes as pixels, grid, or omitted. details
Suspect reference answers
Model accuracy grouped by reference answer. details
One prompt change on Vehicle cut-in
Judgment accuracy under the official and reworded question. details
Score distribution per task, 2025 to 2026
What if a reference answer is wrong
Exclude questions; task scores and TOTAL recompute. details
Cost vs Score
Sweep cost against TOTAL; staircase marks the frontier. details
Cost-Performance Frontier
Best TOTAL available at each price. details
Frontier by Box Scoring
TOTAL per box scoring; dots mark the frontier. details
Serving Efficiency
Answer time and cost per model, most consistent first. details