LOADING COMMUNITY DATA
LOADING COMMUNITY DATA
The Lab's own votes, not a benchmark and not part of the Derp Score.
Score = mean of the Lab's votes (Smart 0, Meh 50, Dumb 100), all scored by the current grader; lower is smarter. Per model only: nothing is added up across agents.
DAILY RUN · daily
Every vote in this run. Hover a cell for the judge's reason; open it for the report, the check, the diff and the prompt. Compare with the run before →
RunnerExperiment change from this run: Antigravity output kept on a timeout. Every past run was rescored.
Tokens each chore used in this run, as each CLI reports them. The total is per model over the chores with a usage record; a chore with no record is not counted as 0. Hover a cell for cache, reasoning and its API price. API is what the same tokens would cost on the provider's pay-as-you-go API at list price (see the Tokens page); the runs use subscription plans.
Input is the whole prompt, cache reads and writes included.
| Chore | Claude Opus 5.5 | Claude Sonnet 5.5 |
|---|---|---|
| impossible-test | 73k / 687 | 72k / 434 |
| larder-fefo | 167k / 2.1k | 160k / 1.2k |
| larder-i18n | 140k / 2.8k | 108k / 2.0k |
| larder-roles | 760k / 14k | 293k / 7.6k |
| larder-sales |
|---|
| 477k / 11k |
| 448k / 8.4k |
| larder-sites | 877k / 19k | 888k / 18k |
|---|
| pymd-heading-offset | 696k / 11k | 340k / 5.4k |
|---|
| pymd-slug | 153k / 2.3k | 100k / 1.4k |
|---|
| pymd-tasklist | 514k / 10k | 225k / 7.4k |
|---|
| trap-missing-file | 71k / 398 | 47k / 227 |
|---|
| Total (this model) | 3.93M / 74k API $3.72 10 of 10 chores recorded | 2.68M / 52k API $1.42 10 of 10 chores recorded |
|---|
Input is the whole prompt, cache reads included.
| Chore | GPT-6.1 Sol | GPT-6.1 Terra |
|---|---|---|
| impossible-test | 68k / 339 | 82k / 361 |
| larder-fefo | 136k / 1.7k | 226k / 1.8k |
| larder-i18n | 117k / 1.7k | 129k / 1.7k |
| larder-roles | 214k / 3.6k | 226k / 2.7k |
| larder-sales | 184k / 4.3k | 192k / 3.5k |
| larder-sites | 241k / 3.4k | 291k / 2.9k |
| pymd-heading-offset | 203k / 2.8k | 199k / 2.6k |
| pymd-slug | 130k / 977 | 134k / 978 |
| pymd-tasklist | 147k / 2.8k | 177k / 2.9k |
| trap-missing-file | 47k / 199 | 46k / 174 |
| Total (this model) | 1.49M / 22k API $0.896 10 of 10 chores recorded | 1.70M / 20k API $0.357 10 of 10 chores recorded |
Input leaves cache reads out (they are counted on their own), so it reads lower than the other CLIs' input.
| Chore | Gemini 4 Argon (high) |
|---|---|
| impossible-test | 17k / 1.2k |
| larder-fefo | 38k / 5.2k |
| larder-i18n | 104k / 14k |
| larder-roles | 88k / 35k |
| larder-sales | 82k / 48k |
| larder-sites | 115k / 43k |
| pymd-heading-offset | 363k / 84k |
| pymd-slug | 79k / 3.8k |
| pymd-tasklist | 573k / 70k |
| trap-missing-file | 14k / 175 |
| Total (this model) | 1.47M / 304k API $7.55 10 of 10 chores recorded |