An AI-made pinball table needs more than a convincing screenshot: someone has to play it. Artificial Analysis’s new AA-3D benchmark asks people to explore interactive scenes before choosing between them. Introduced on 12 October, it evaluates 21 models building browser experiences from written briefs, shifting attention from whether code runs to which result people prefer.
Based on 6 reviewed sources · See sources
The judge gets to play
The test includes 55 briefs, from a spinning black hole to a playable pinball table. Models use three.js, a library for browser-based 3D graphics. Each gets one attempt in the same offline environment, with tools for modelling, physics and browser testing, and a limit of three hours or 250 agent turns.
The evaluator introduces a blind human comparison of agent-built interactive scenes, with shared tasks and operating limits.
View the original post on X ↗Under AA’s methodology, scenes must run and pass review before entering blind comparisons. Recruited panelists spend at least 30 seconds with each result before choosing. Model names stay hidden; sessions that fail attention checks are discarded. The score reflects those preferences, not an accuracy percentage or proof that every requested interaction works. Public visitors can compare scenes too, but their votes do not change the leaderboard.
Keep the uncertainty beside the score
Claude Sonnet 5.5 has the highest launch rating, but AA describes four leading models as statistically tied: each of the other three has a 95% confidence interval overlapping Sonnet’s. These intervals show statistical uncertainty around the ratings. The leaderboard currently matches the launch scores below; Muse Wonder is absent, so this is no test of Meta’s newest model.
The leading scores, with their uncertainty
AA-3D v1.0 · leading ratings and 95% intervals checked 13 October 2026
| Model | Configuration | Elo rating | 95% interval |
|---|---|---|---|
| Claude Sonnet 5.5 | Maximum effort, default fallback | 1194 | 1160–1228 |
| Claude Opus 5.5 | Maximum effort, default fallback | 1190 | 1155–1225 |
| GPT-6 Astra | Maximum effort | 1165 | 1131–1199 |
| Claude Fable 5.1 | Maximum effort, default fallback | 1131 | 1098–1165 |
A scene has a bill, as well as a score
AA reports average token costs from $0.20 per scene for GPT-6 Luna to $32.08 for Claude Fable 5.1. Luna ranks eighth; the cheapest is not the highest-rated. These are bills for the tested runs, with their token use and cache pricing, rather than a budget for a finished client project.
A separate project illustrates why visual rankings need their own context. Software engineer Nicola Manzini, who shared his scenes in a quote-post, runs threejseval. Its published methodology permits multiple attempts and mostly emphasises composition over deep interaction. It uses public votes and a different Elo scale. Combining its scores with AA-3D’s would erase those differences.
The useful next step is to inspect the scenes against the brief you actually need to fulfil. Our Muse Wonder analysis provides another concrete example of benchmark gains and longer creative builds telling different stories. A preference board can help narrow a shortlist; it cannot make the final creative decision for you.
Sources
- AA-3D v1.0 launch results
Artificial Analysis · Research ·
- AA-3D generation and judging methodology
Artificial Analysis · Documentation
- AA-3D leaderboard and inspectable scenes
Artificial Analysis · Research
- Original AA-3D announcement
Artificial Analysis on X · Announcement ·
- Separate Three.js benchmark methodology
threejseval · Nicola Manzini · Documentation ·
- Nicola Manzini’s separate scene project
Nicola Manzini on X · Research
Written with AI assistance from the sources above. Analysis and practical implications are our interpretation. Our editorial approach.
