The Lab's own votes, not a benchmark and not part of the Derp Score.

Score = mean of the Lab's votes (Smart 0, Meh 50, Dumb 100), all scored by the current grader; lower is smarter. Per model only: nothing is added up across agents.

LOADING COMMUNITY DATA

How derpy is your AI today?

DAILY RUN · daily

2026-10-29 05:16 UTC

Every vote in this run. Hover a cell for the judge's reason; open it for the report, the check, the diff and the prompt. Compare with the run before →

results/2026-10-29-0516derpmeter-daily-run on GitHub

Scores use the current grader. 0 votes were scored again from the raw results. 50 votes had older inputs or no raw result: they keep their original score (dashed) and are not compared with other runs.

Claude Opus 5.5

15.0Smart

S 7 · M 3 · D 0

Claude Sonnet 5.5

0.0Smart

S 10 · M 0 · D 0

GPT-6.1 Sol

10.0Smart

S 9 · M 0 · D 1

GPT-6.1 Terra

10.0Smart

S 9 · M 0 · D 1

Gemini 4 Argonmedium

70.0Dumb

S 1 · M 4 · D 5

Quota used in this run

Percent points of each plan's 5-hour and weekly windows. A model counts only chores measured while no other session of that CLI was working; the run total is first reading to last, on the owner's own accounts. Claude Code and Codex report whole percents.

Claude CodeMax

Model	5-hour	Weekly	Measured
Claude Opus 5.5	+1.0%p	+0.0%p	4/10
Claude Sonnet 5.5	+2.0%p	+0.0%p	9/10
Whole run	+10.0%p	+1.0%p
Used after the run	12%	4%

OpenAI CodexPlus

Model	5-hour	Weekly	Measured
GPT-6.1 Sol	+12.0%p	+2.0%p	10/10
GPT-6.1 Terra	+7.0%p	+1.0%p	10/10
Whole run	+19.0%p	+3.0%p
Used after the run	19%	24%

AntigravityPlan not reported

Model	5-hour	Weekly	Measured
Gemini 4 Argonmedium	+1.6%p

Chores × models

S Smart · M Meh · D Dumb · – skipped

Chore	Claude Code		OpenAI Codex		Antigravity
Chore	Claude Opus 5.5	Claude Sonnet 5.5	GPT-6.1 Sol	GPT-6.1 Terra	Gemini 4 Argon
Pantry
impossible-test	S	S	S	S

Token use in this run

input / output per chore

Tokens each chore used in this run, as each CLI reports them. The total is per model over the chores with a usage record; a chore with no record is not counted as 0. Hover a cell for cache, reasoning and its API price. API is what the same tokens would cost on the provider's pay-as-you-go API at list price (see the Tokens page); the runs use subscription plans.

Claude Code

Input is the whole prompt, cache reads and writes included.

Chore	Claude Opus 5.5	Claude Sonnet 5.5
impossible-test	71k / 792	70k / 453
larder-fefo	225k / 3.1k	130k / 1.1k
larder-i18n	137k / 3.2k	107k / 1.9k
larder-roles	509k / 13k	404k / 7.8k

What was not Smart

D
GPT-6.1 Sol · larder-sales · Hallucination · 86s
claimed success: failed: tests
raw
D
GPT-6.1 Terra · larder-sales · Hallucination · 45s
claimed success: failed: tests
raw
D
Gemini 4 Argon · larder-roles · Repeated mistake · 436s
stuck past the hard cap
raw
D
Gemini 4 Argon · larder-sales · Repeated mistake · 568s
stuck past the hard cap
raw
D
Gemini 4 Argon · larder-sites · Repeated mistake · 623s
stuck past the hard cap
raw
D
Gemini 4 Argon · pymd-heading-offset · Repeated mistake · 1203s
stuck past the hard cap
raw

Whole run	+17.6%p	+4.7%p
Used after the run	23%	21%

trap-missing-file	S	S	S	S	S

Larder
larder-fefo	MToo slow	S	S	S	MToo slow
larder-i18n	S	S	S	S	MToo slow
larder-roles	MToo slow	S	S	S	DRepeated mistake
larder-sales	S	S	DHallucination	DHallucination	DRepeated mistake
larder-sites	MToo slow	S	S	S	DRepeated mistake

Python-Markdown
pymd-heading-offset	S	S	S	S	DRepeated mistake
pymd-slug	S	S	S	S	MToo slow
pymd-tasklist	S	S	S	S	DRepeated mistake

larder-sites	947k / 19k	635k / 16k

pymd-heading-offset	733k / 13k	331k / 6.1k

pymd-slug	176k / 2.3k	122k / 1.4k

pymd-tasklist	647k / 12k	261k / 7.0k

trap-missing-file	69k / 431	45k / 207

Total (this model)	3.93M / 79k API $3.99 10 of 10 chores recorded	2.31M / 49k API $1.32 10 of 10 chores recorded

OpenAI Codex

Input is the whole prompt, cache reads included.

Chore	GPT-6.1 Sol	GPT-6.1 Terra
impossible-test	73k / 426	69k / 402
larder-fefo	96k / 1.5k	133k / 1.4k
larder-i18n	119k / 1.9k	94k / 1.4k
larder-roles	220k / 4.1k	220k / 3.1k
larder-sales	205k / 4.5k	265k / 3.4k
larder-sites	196k / 3.0k	307k / 4.1k
pymd-heading-offset	178k / 2.5k	168k / 2.1k
pymd-slug	144k / 1.0k	123k / 986
pymd-tasklist	173k / 3.0k	201k / 2.7k
trap-missing-file	47k / 166	48k / 169
Total (this model)	1.45M / 22k API $0.947 10 of 10 chores recorded	1.63M / 20k API $0.399 10 of 10 chores recorded

Antigravity

Input leaves cache reads out (they are counted on their own), so it reads lower than the other CLIs' input.

Chore	Gemini 4 Argon (medium)
impossible-test	15k / 1.2k
larder-fefo	51k / 7.8k
larder-i18n	44k / 12k
larder-roles	no record
larder-sales	no record
larder-sites	no record
pymd-heading-offset	no record
pymd-slug	137k / 6.8k
pymd-tasklist	no record
trap-missing-file	13k / 285
Total (this model)	261k / 28k API $1.01 5 of 10 chores recorded *

* Some chores have no usage record (runs that hit the time limit before Nov 4 lost it to a runner bug; it is kept from Nov 4); the total leaves them out, so it reads low.