The blind AI model test

One prompt. Every model.
Every answer, named.

We run one identical task through 13 AI models — 8 local open-weight models on our own Brain cluster and 5 cloud flagships. Every raw answer sits on the page under the name of the model that produced it, together with rubric scores against a documented gold answer. Blind refers to the scoring, not to what you get to see: the review was made without knowing which model wrote which answer. No synthetic benchmarks, no cherry-picking — the failures stay on the page too.

Open a round → How it works

179rounds published, each with every raw answer

13models compared — 8 local open-weight, 5 cloud

8disciplines, from logic traps to worked physics problems

How a blind round works

STEP 01

One identical prompt

Every model in the field gets exactly the same task through the same API harness — same wording, one shot, no retries. Timeouts and errors count as a DNF and stay on the page.

STEP 02

Scored without names

The answers are shuffled and reviewed the way a blind tasting works: on substance alone, without knowing the author. The scores you see were made under those conditions.

STEP 03

Published with names

The result names every model: who wrote what, the blind rubric score against a gold answer, response times, and the measured runtime for code rounds — including every failure.

Start here — a classic reasoning trap

Knowledge & Logic · blind round

R31 · Bat and ball

A bat and a ball cost 1.10 together, the bat costs 1.00 more than the ball — the question almost everyone answers wrong on instinct. Eight models took it blind. Can you tell the careful reasoners from the pattern-matchers without seeing the names?

Take R31 blind → Or try R42 · “All but nine” →

Latest blind rounds R31–R180

Round	Task	Category
R180	Fix the broken references	editing-correction
R179	Unify tense and perspective	editing-correction
R178	Remove redundancy without losing facts	editing-correction
R177	Make the numbers consistent	editing-correction
R176	Fix exactly the errors, change nothing else	editing-correction
R175	Noise flowfield painting additive particle trails	Code
R174	Recursive fractal tree growing branch by branch	Code
R173	Metaball lava lamp with heating and cooling blobs	Code
R172	Raindrops sliding down a window pane	Code
R171	Fireworks bursting into fading sparks	Code
R170	Compress a how-to into exactly 5 numbered steps	summarization
R169	Neutral summary of a biased product rant	summarization
R168	Extract action items as a checklist	summarization
R167	One-sentence TL;DR under 25 words	summarization
R166	Faithful 3-sentence summary of a chaotic meeting transcript	summarization
R165	Recursive backtracker carving a maze, then solving it	Code
R164	Boids flocking with separation, alignment, cohesion and a predator	Code
R163	Prim's algorithm growing a minimum spanning tree	Code
R162	Breadth-first search flooding a random grid maze	Code
R161	Selection sort vs shell sort racing side by side	Code
R160	Critique a generic slogan in 2 sentences + give 2 better alternatives	Creative & Marketing
R159	Micro-story: exactly 50 words (±5) with 3 mandatory elements	Creative & Marketing
R158	5 email subject lines under 50 chars, no clickbait, no ALL-CAPS	Creative & Marketing
R157	Rewrite the same product text for two different audiences, facts unchanged	Creative & Marketing
R156	Invent 3 product names + taglines for a fictional object (no superlatives)	Creative & Marketing
R155	Linked list with animated pointer relinking	Code
R154	Ring buffer with producer and consumer at different rates	Code
R153	Binary min-heap with bubble-up and sink-down	Code
R152	Hash table with visible collision chains	Code
R151	Binary search tree growing node by node	Code
R150	Diff two near-identical config blocks	Structure & Data
R149	Extract meeting invite into a fixed JSON schema	Structure & Data
R148	Table QA: sum, filter, max over order data	Structure & Data
R147	CSV with embedded comma to JSON array	Structure & Data
R146	Repair broken JSON (4 precise errors)	Structure & Data
R145	False-friend minefield German into English	Language & Translation
R144	English apology into idiomatic Sie-German	Language & Translation
R143	Bureaucratic German into plain English	Language & Translation
R142	Register shift: casual complaint into formal	Language & Translation
R141	Idiom-dense German into natural English	Language & Translation
R140	Water sloshing in a tilting container	Code
R139	Small n-body system orbiting a central star	Code
R138	Two-source wave interference pattern	Code
R137	Cloth mesh hanging and rippling in the wind	Code
R136	Double pendulum tracing a chaotic path	Code
R135	Conditional dice probability	Knowledge & Logic
R134	Smallest number from three remainders	Knowledge & Logic
R133	Three friends, three pets	Knowledge & Logic
R132	Fermi estimate: a lifetime of drinking water	Knowledge & Logic
R131	Weekday riddle from two clues	Knowledge & Logic
R105	Brian's Brain	Code
R104	Abelian sandpile toppling	Code
R103	Wireworld with a clock and a diode	Code
R102	Rule 110 growing line by line	Code
R101	Langton's ant building a highway	Code
R130	Radix sort ordering by one digit at a time	Code
R129	Heap sort sifting the maximum to the back	Code
R128	Merge sort weaving runs together	Code
R127	Quicksort partitioning around glowing pivots	Code
R126	Bubble sort sweeping bars into order	Code
R125	Voronoi diagram growing by multi-source flood fill	Code
R124	Dijkstra's shortest path over weighted terrain	Code
R123	Breadth-first flood fill spreading through a region	Code
R122	Randomized maze generation via recursive backtracking	Code
R121	A* pathfinding search across an obstacle grid	Code
R120	Scanline fill of a self-intersecting polygon	Code
R119	Floyd-Steinberg dithering a gradient	Code
R118	Sutherland-Hodgman polygon clipping	Code
R117	De Casteljau construction of a Bezier curve	Code
R116	Bresenham circles and lines, pixel by pixel	Code
R115	Crowd leaving through a bottleneck	Code
R114	Traffic jam from nothing	Code
R113	Predator and prey on a shared field	Code
R112	Physarum slime-mould network	Code
R111	Ant colony laying pheromone trails	Code
R110	Penrose-style aperiodic tiling growing	Code
R109	Moire from two rotating rasters	Code
R108	Interlocking star pattern on a grid	Code
R107	Hilbert curve drawing itself	Code
R106	Truchet tiles rearranging	Code
R100	Sankey flow with travelling particles	Code
R99	Three sorting algorithms racing	Code
R98	Squarified treemap of changing data	Code
R97	Circle packing without overlap	Code
R96	Force-directed graph settling	Code
R95	Shaded sphere by painter's algorithm	Code
R94	Raycast maze walkthrough	Code
R93	Rotating wireframe height field	Code
R92	Rotating wireframe solids with depth sorting	Code
R91	Starfield tunnel in perspective	Code
R90	Poisson-disk sampling, Bridson's algorithm	Code
R89	Marching squares on a moving field	Code
R88	L-system tree growing in wind	Code
R87	Gray-Scott reaction-diffusion	Code
R86	Scrolling terrain from value noise	Code
R85	Travelling salesman improved by 2-opt	Code
R84	Voronoi cells with moving seeds	Code
R83	Fourier epicycles tracing a shape	Code
R82	Conway's Game of Life with seeded patterns	Code
R81	Flocking birds with three rules	Code
R80	Aggregation by continuous drift	Code
R79	Aggregation on a lattice	Code
R78	Crowd leaving a room by steering	Code
R77	Crowd leaving a room, cell by cell	Code
R76	Falling sand as free particles	Code
R75	Falling sand on a cell grid	Code
R74	Quadtree collision detection	Code
R73	KMP string search with prefix table	Code
R72	Maze generation by recursive backtracking	Code
R71	Flood fill on a pixel grid	Code
R70	Binary search, step by step	Code
R69	Replanning when the map changes	Code
R68	Edit-distance DP matrix with backtrace	Code
R67	Sieve of Eratosthenes on a number grid	Code
R66	AVL tree rebalancing with rotations	Code
R65	Delaunay triangulation, triangle by triangle	Code
R64	N-Queens solved by backtracking	Code
R63	Kruskal's MST with union-find	Code
R62	Huffman tree from letter frequencies	Code
R61	Convex hull built step by step	Code
R60	Dijkstra vs A* on a weighted grid	Code
R59	Billiard break with spin and friction	Physics Lab
R58	2D wave propagation with reflection	Physics Lab
R57	Hanging rope with Verlet integration	Physics Lab
R56	Double-slit interference, particle by particle	Physics Lab
R55	The double-slit experiment and wave-particle duality (conceptual)	Physics
R54	Time dilation: why do moving clocks run slow? (conceptual)	Physics
R53	Why is the sky blue? (conceptual)	Physics
R52	Fermi estimate: piano tuners in a 10-million-person city	Physics
R51	Heat required to warm water (specific heat capacity)	Physics
R50	Thin lens equation: image distance and magnification	Physics
R49	Ohm's law: resistance and power dissipation	Physics
R48	Low-orbit satellite: speed and period (orbital estimate)	Physics
R47	Unit conversion: km/h to m/s and mph	Physics
R46	Energy conservation on a frictionless incline	Physics
R45	Idiom-dense German to natural English	Language & Translation
R44	Appointment extraction to strict JSON	Structure & Data
R43	Five machines (rate-reasoning trap)	Knowledge & Logic
R42	All but nine (word-problem trap)	Knowledge & Logic
R41	Prose → valid JSON	Structure & Data
R40	isPalindrome (ignore case & non-alphanumerics)	Code
R39	40-word product blurb with 3 required words	Creative & Marketing
R38	Register shift: formal → casual (German)	Language & Translation
R37	False friend: “control” EN→DE	Language & Translation
R36	Continue the sequence (doubling gaps)	Knowledge & Logic
R35	Minutes in a (non-leap) year — show the math	Knowledge & Logic
R34	Constrained taglines (word ban + length cap)	Creative & Marketing
R33	Idioms that break literal translation (EN→DE)	Language & Translation
R32	Four houses, four clues (constraint puzzle)	Knowledge & Logic
R31	Bat and ball (the classic reasoning trap)	Knowledge & Logic

Earlier rounds R1–R30 are in the archive — they predate the blind-first layout and show model names directly.

Standings & scoreboards

We don't publish a single magic ranking number. Scored rounds carry blind rubric scores (0–10 against a gold answer, DNFs included); creative and style rounds stay deliberately unscored — read them yourself. The full picture lives on the overview page.

All rounds, categories & model roster →

Eight disciplines

🧠

Knowledge & Logic

Factual knowledge without hallucinations and clean step-by-step reasoning on classic logic riddles and traps.

🔭

Physics

Worked physics with checkable numbers: mechanics, orbits, circuits, optics, heat, Fermi estimates — plus conceptual questions on relativity and quanta.

💻

Code

Programming in a small space: correct one-liners, CSS that runs, and algorithm races rendered in a real browser.

🌐

Language & Translation

Language feel in German and English: translation, idioms, false friends, register and audience fit.

🧩

Structure & Data

Turning messy text into exact formats: schema-true JSON, classification, table understanding, format repair.

✨

Creative & Marketing

Copy under constraints: slogans, micro-copy, product names and image prompts with market fit.

⚛️

Physics Lab

Compact canvas physics sims — gravity, chaos, orbits — written one-shot and verified in a headless browser.

🖼️

Image Models

Four FLUX image engines, identical prompts, judged by pixels.

The field — 13 models

Eight local open-weight models running on our own hardware, five cloud flagships over their official APIs. Every profile links to all of that model's answers.

gemma4:26b qwen3-coder:30b qwen3:14b qwen3.5:9b qwen3:8b llama3.1:8b deepseek-r1:14b mistral-small:24b Claude Opus 4.8 Claude Sonnet 5 Claude Haiku 4.5 Claude Fable 5 GPT-5.5 (codex)

Why blind?

Model names carry expectations — a famous logo makes an answer look smarter than it is. Blind comparison removes that halo. Our method, its limits, and how scoring works are documented openly on the method page.