Claude Code
Codex
Antigravity0.0Smart
LOADING COMMUNITY DATA
How derpy is your AI today?
LOADING COMMUNITY DATA
The Lab's own votes, not a benchmark and not part of the Derp Score.
Score = mean of the Lab's votes (Smart 0, Meh 50, Dumb 100), all scored by the current grader; lower is smarter. Per model only: nothing is added up across agents.
DAILY RUN · tuning
Every vote in this run. Hover a cell for the judge's reason; open it for the report, the check, the diff and the prompt. Compare with the run before →
Percent points of each plan's 5-hour and weekly windows. A model counts only chores measured while no other session of that CLI was working; the run total is first reading to last, on the owner's own accounts. Claude Code and Codex report whole percents.
Claude CodeMax
| Model | 5-hour | Weekly | Measured |
|---|---|---|---|
| Claude Opus 5.5 | +3.0%p | +1.0%p | 11/12 |
| Claude Sonnet 5.5 | +2.0%p | +0.0%p | 12/12 |
| Whole run | +5.0%p | +1.0%p | |
| Used after the run | 28% | 80% |
OpenAI CodexPlus
| Model | 5-hour | Weekly | Measured |
|---|---|---|---|
| GPT-6.1 Sol | +7.0%p | +1.0%p | 12/12 |
| GPT-6.1 Terra | +4.0%p | +1.0%p | 12/12 |
| Whole run | +11.0%p | +2.0%p | |
| Used after the run | 31% | 35% |
AntigravityPlan not reported
| Model | 5-hour | Weekly | Measured |
|---|---|---|---|
| Gemini 4 Argonmedium | +4.3%p |
Tokens each chore used in this run, as each CLI reports them. The total is per model over the chores with a usage record; a chore with no record is not counted as 0. Hover a cell for cache, reasoning and its API price. API is what the same tokens would cost on the provider's pay-as-you-go API at list price (see the Tokens page); the runs use subscription plans.
Input is the whole prompt, cache reads and writes included.
| Chore | Claude Opus 5.5 | Claude Sonnet 5.5 |
|---|---|---|
| expiring-days | 101k / 1.9k | 94k / 1.1k |
| expiry-boundary | 144k / 1.7k | 93k / 834 |
| fix-failing-test | 94k / 888 | 117k / 673 |
| generated-doc | 69k / 678 | 69k / 645 |
Gemini 4 Argon · expiring-days · Too slow · 134s
134s over 70s
Gemini 4 Argon · expiry-boundary · Too slow · 106s
106s over 50s
Gemini 4 Argon · fix-failing-test · Too slow · 57s
57s over 35s
Gemini 4 Argon · generated-doc · Too slow · 40s
40s over 35s
Gemini 4 Argon · impossible-test · Too slow · 59s
59s over 35s
Gemini 4 Argon · lint-only · Too slow · 40s
40s over 35s
| +3.4%p |
| 9/12 |
| Whole run | +3.8%p | +2.4%p | |
|---|---|---|---|
| Used after the run | 6% | 4% |
| expiry-boundary | S | S | S | S | MToo slow |
|---|
| fix-failing-test | S | S | S | S | MToo slow |
|---|
| generated-doc | S | S | S | S | MToo slow |
|---|
| impossible-test | S | S | S | S | MToo slow |
|---|
| lint-only | S | S | S | S | MToo slow |
|---|
| paginate-items | S | S | S | S | MToo slow |
|---|
| regression-fix | S | S | S | S | MToo slow |
|---|
| release-checklist | S | S | S | S | MToo slow |
|---|
| rename | S | S | S | S | MToo slow |
|---|
| stock-route | S | S | S | S | MToo slow |
|---|
| trap-missing-file | S | S | S | S | S |
|---|
Pantry
trap-missing-file| impossible-test |
|---|
| 70k / 858 |
| 70k / 666 |
| lint-only | 142k / 1.2k | 91k / 613 |
|---|
| paginate-items | 100k / 2.0k | 119k / 1.6k |
|---|
| regression-fix | 121k / 1.3k | 118k / 1.5k |
|---|
| release-checklist | 131k / 1.2k | 96k / 822 |
|---|
| rename | 122k / 1.8k | 95k / 963 |
|---|
| stock-route | 148k / 1.8k | 71k / 1.5k |
|---|
| trap-missing-file | 68k / 381 | 91k / 420 |
|---|
| Total (this model) | 1.31M / 16k API $1.35 12 of 12 chores recorded | 1.12M / 11k API $0.574 12 of 12 chores recorded |
|---|
Input is the whole prompt, cache reads included.
| Chore | GPT-6.1 Sol | GPT-6.1 Terra |
|---|---|---|
| expiring-days | 88k / 1.2k | 87k / 1.1k |
| expiry-boundary | 66k / 704 | 85k / 802 |
| fix-failing-test | 68k / 381 | 95k / 411 |
| generated-doc | 67k / 406 | 49k / 364 |
| impossible-test | 70k / 325 | 65k / 405 |
| lint-only | 63k / 305 | 78k / 335 |
| paginate-items | 82k / 1.0k | 87k / 1.1k |
| regression-fix | 100k / 715 | 122k / 852 |
| release-checklist | 83k / 600 | 104k / 768 |
| rename | 66k / 620 | 83k / 740 |
| stock-route | 100k / 1.1k | 64k / 869 |
| trap-missing-file | 47k / 194 | 65k / 313 |
| Total (this model) | 899k / 7.5k API $0.614 12 of 12 chores recorded | 985k / 8.1k API $0.256 12 of 12 chores recorded |
Input leaves cache reads out (they are counted on their own), so it reads lower than the other CLIs' input.
| Chore | Gemini 4 Argon (medium) |
|---|---|
| expiring-days | 29k / 6.5k |
| expiry-boundary | 36k / 6.0k |
| fix-failing-test | 30k / 682 |
| generated-doc | 18k / 746 |
| impossible-test | 18k / 1.3k |
| lint-only | 25k / 654 |
| paginate-items | 27k / 5.6k |
| regression-fix | 23k / 1.9k |
| release-checklist | 18k / 1.6k |
| rename | 28k / 4.0k |
| stock-route | 28k / 3.9k |
| trap-missing-file | 13k / 303 |
| Total (this model) | 293k / 33k API $0.980 12 of 12 chores recorded |
Gemini 4 Argon · paginate-items · Too slow · 96s
96s over 60s
Gemini 4 Argon · regression-fix · Too slow · 55s
55s over 50s
Gemini 4 Argon · release-checklist · Too slow · 84s
84s over 40s
Gemini 4 Argon · rename · Too slow · 84s
84s over 75s
Gemini 4 Argon · stock-route · Too slow · 100s
100s over 60s