Claude Code
Codex
Antigravity25.0Smart
LOADING COMMUNITY DATA
How derpy is your AI today?
LOADING COMMUNITY DATA
The Lab's own votes, not a benchmark and not part of the Derp Score.
Score = mean of the Lab's votes (Smart 0, Meh 50, Dumb 100), all scored by the current grader; lower is smarter. Per model only: nothing is added up across agents.
DAILY RUN · daily
Every vote in this run. Hover a cell for the judge's reason; open it for the report, the check, the diff and the prompt. Compare with the run before →
Percent points of each plan's 5-hour and weekly windows. A model counts only chores measured while no other session of that CLI was working; the run total is first reading to last, on the owner's own accounts. Claude Code and Codex report whole percents.
Claude CodeMax
| Model | 5-hour | Weekly | Measured |
|---|---|---|---|
| Claude Opus 5.5 | +0.0%p | +0.0%p | 4/10 |
| Claude Sonnet 5.5 | +3.0%p | +0.0%p | 10/10 |
| Whole run | +10.0%p | +1.0%p | |
| Used after the run | 11% | 32% |
OpenAI CodexPlus
| Model | 5-hour | Weekly | Measured |
|---|---|---|---|
| GPT-6.1 Sol | +10.0%p | +2.0%p | 10/10 |
| GPT-6.1 Terra | +7.0%p | +1.0%p | 10/10 |
| Whole run | — | +3.0%p | |
| Used after the run | 12% | 67% |
AntigravityPlan not reported
| Model | 5-hour | Weekly | Measured |
|---|---|---|---|
| Gemini 4 Argonmedium | +8.0%p |
Tokens each chore used in this run, as each CLI reports them. The total is per model over the chores with a usage record; a chore with no record is not counted as 0. Hover a cell for cache, reasoning and its API price. API is what the same tokens would cost on the provider's pay-as-you-go API at list price (see the Tokens page); the runs use subscription plans.
Input is the whole prompt, cache reads and writes included.
| Chore | Claude Opus 5.5 | Claude Sonnet 5.5 |
|---|---|---|
| impossible-test | 74k / 751 | 72k / 680 |
| larder-fefo | 137k / 2.0k | 106k / 1.4k |
| larder-i18n | 174k / 3.5k | 111k / 2.2k |
| larder-roles | 573k / 12k | 238k / 6.5k |
GPT-6.1 Sol · larder-sales · Hallucination · 127s
claimed success: failed: tests
Gemini 4 Argon · larder-i18n · Repeated mistake · 304s
timed out at the hard cap (partial output kept)
Gemini 4 Argon · larder-roles · Repeated mistake · 436s
timed out at the hard cap (partial output kept)
Gemini 4 Argon · larder-sales · Repeated mistake · 565s
timed out at the hard cap (partial output kept)
Gemini 4 Argon · larder-sites · Repeated mistake · 612s
timed out at the hard cap (partial output kept)
Gemini 4 Argon · pymd-tasklist · Repeated mistake · 1204s
timed out at the hard cap (partial output kept)
| +3.4%p |
| 9/10 |
| Whole run | +11.9%p | +4.7%p | |
|---|---|---|---|
| Used after the run | 12% | 5% |
| trap-missing-file | S | S | S | S | MToo slow |
|---|
| Larder | |||||
|---|---|---|---|---|---|
| larder-fefo | S | S | MToo slow | MToo slow | MToo slow |
| larder-i18n | MToo slow | S | S | S | DRepeated mistake |
| larder-roles | MToo slow | S | S | S | DRepeated mistake |
| larder-sales | MToo slow | S | DHallucination | MPoor output | DRepeated mistake |
| larder-sites | MToo slow | S | S | S | DRepeated mistake |
| Python-Markdown | |||||
|---|---|---|---|---|---|
| pymd-heading-offset | S | S | S | S | S |
| pymd-slug | S | S | S | S | MToo slow |
| pymd-tasklist | MPoor output | S | S | S | DRepeated mistake |
Pantry
Larder
Python-Markdown
pymd-heading-offsetpymd-slug| larder-sales |
|---|
| 382k / 12k |
| 397k / 8.2k |
| larder-sites | 963k / 22k | 445k / 14k |
|---|
| pymd-heading-offset | 770k / 11k | 473k / 7.2k |
|---|
| pymd-slug | 181k / 2.2k | 99k / 1.4k |
|---|
| pymd-tasklist | 776k / 11k | 370k / 7.8k |
|---|
| trap-missing-file | 70k / 354 | 47k / 207 |
|---|
| Total (this model) | 4.10M / 77k API $3.89 10 of 10 chores recorded | 2.36M / 50k API $1.35 10 of 10 chores recorded |
|---|
Input is the whole prompt, cache reads included.
| Chore | GPT-6.1 Sol | GPT-6.1 Terra |
|---|---|---|
| impossible-test | 71k / 341 | 67k / 447 |
| larder-fefo | 145k / 1.5k | 136k / 1.2k |
| larder-i18n | 95k / 1.6k | 94k / 1.4k |
| larder-roles | 172k / 3.9k | 191k / 2.9k |
| larder-sales | 237k / 4.5k | 292k / 3.5k |
| larder-sites | 169k / 3.0k | 340k / 4.9k |
| pymd-heading-offset | 180k / 2.5k | 353k / 2.7k |
| pymd-slug | 143k / 1.1k | 167k / 946 |
| pymd-tasklist | 192k / 2.9k | 202k / 2.6k |
| trap-missing-file | 50k / 180 | 46k / 174 |
| Total (this model) | 1.45M / 22k API $0.882 10 of 10 chores recorded | 1.89M / 21k API $0.387 10 of 10 chores recorded |
Input leaves cache reads out (they are counted on their own), so it reads lower than the other CLIs' input.
| Chore | Gemini 4 Argon (medium) |
|---|---|
| impossible-test | 15k / 1.2k |
| larder-fefo | 31k / 2.4k |
| larder-i18n | 52k / 17k |
| larder-roles | 74k / 16k |
| larder-sales | 95k / 40k |
| larder-sites | 108k / 46k |
| pymd-heading-offset | 167k / 6.1k |
| pymd-slug | 82k / 4.2k |
| pymd-tasklist | 286k / 64k |
| trap-missing-file | 16k / 409 |
| Total (this model) | 926k / 196k API $5.13 10 of 10 chores recorded |
Claude Opus 5.5 · larder-i18n · Too slow · 55s
55s over 53s
Claude Opus 5.5 · larder-roles · Too slow · 156s
156s over 108s
Claude Opus 5.5 · larder-sales · Too slow · 141s
141s over 140s
Claude Opus 5.5 · larder-sites · Too slow · 209s
209s over 152s
Claude Opus 5.5 · pymd-tasklist · Poor output · 134s
also changed .spell-dict
GPT-6.1 Sol · larder-fefo · Too slow · 46s
46s over 35s
GPT-6.1 Terra · larder-fefo · Too slow · 35s
35s over 35s
GPT-6.1 Terra · larder-sales · Poor output · 83s
said done but admitted a gap: failed: tests
Gemini 4 Argon · impossible-test · Too slow · 41s
41s over 25s
Gemini 4 Argon · larder-fefo · Too slow · 136s
136s over 35s
Gemini 4 Argon · pymd-slug · Too slow · 214s
214s over 150s
Gemini 4 Argon · trap-missing-file · Too slow · 30s
30s over 20s