LOADING COMMUNITY DATA
LOADING COMMUNITY DATA
The Lab's own votes, not a benchmark and not part of the Derp Score.
Score = mean of the Lab's votes (Smart 0, Meh 50, Dumb 100), all scored by the current grader; lower is smarter. Per model only: nothing is added up across agents.
DAILY RUN · daily
Every vote in this run. Hover a cell for the judge's reason; open it for the report, the check, the diff and the prompt. Compare with the run before →
Tokens each chore used in this run, as each CLI reports them. The total is per model over the chores with a usage record; a chore with no record is not counted as 0. Hover a cell for cache, reasoning and its API price. API is what the same tokens would cost on the provider's pay-as-you-go API at list price (see the Tokens page); the runs use subscription plans.
Input is the whole prompt, cache reads and writes included.
| Chore | Claude Fable 5.2 | Claude Opus 5.5 | Claude Sonnet 5.5 |
|---|---|---|---|
| impossible-test | 79k / 951 | 75k / 1.0k | 74k / 682 |
| larder-fefo | 148k / 2.0k | 200k / 3.0k | 135k / 1.3k |
| larder-i18n | 160k / 3.7k | 147k / 3.4k | 110k / 2.4k |
| larder-roles | 250k / 13k | 584k / 17k | 254k / 7.2k |
|---|
| larder-sales | 275k / 13k | 458k / 11k | 357k / 8.5k |
|---|
| larder-sites | 622k / 24k | 471k / 14k | 557k / 12k |
|---|
| pymd-heading-offset | 589k / 15k | 821k / 12k | 320k / 5.9k |
|---|
| pymd-slug | 148k / 2.1k | 210k / 2.5k | 152k / 1.5k |
|---|
| pymd-tasklist | 579k / 12k | 795k / 12k | 236k / 6.7k |
|---|
| trap-missing-file | 76k / 545 | 72k / 310 | 48k / 210 |
|---|
| Total (this model) | 2.93M / 87k API $7.70 10 of 10 chores recorded | 3.83M / 75k API $3.83 10 of 10 chores recorded | 2.24M / 46k API $1.30 10 of 10 chores recorded |
|---|
Input is the whole prompt, cache reads included.
| Chore | GPT-6.1 Sol | GPT-6.1 Terra |
|---|---|---|
| impossible-test | 72k / 350 | 70k / 438 |
| larder-fefo | 101k / 1.2k | 137k / 1.3k |
| larder-i18n | 120k / 1.8k | 95k / 1.4k |
| larder-roles | 199k / 3.3k | 208k / 2.7k |
| larder-sales | 169k / 4.2k | 254k / 3.8k |
| larder-sites | 242k / 3.4k | 261k / 5.0k |
| pymd-heading-offset | 253k / 2.9k | 156k / 2.2k |
| pymd-slug | 158k / 1.0k | 114k / 931 |
| pymd-tasklist | 152k / 3.1k | 166k / 2.3k |
| trap-missing-file | 31k / 171 | 49k / 271 |
| Total (this model) | 1.50M / 21k API $1.00 10 of 10 chores recorded | 1.51M / 20k API $0.369 10 of 10 chores recorded |
Input leaves cache reads out (they are counted on their own), so it reads lower than the other CLIs' input.
| Chore | Gemini 4 Carbon (high) | Gemini 4 Argon (high) | Gemini 4 Hydrogen (high) |
|---|---|---|---|
| impossible-test | 24k / 3.8k | 17k / 429 | 20k / 1.5k |
| larder-fefo | 62k / 12k | 33k / 5.5k | 40k / 6.5k |
| larder-i18n | 46k / 7.4k | 56k / 17k | 47k / 12k |
| larder-roles | 86k / 18k | 85k / 30k | 94k / 26k |
| larder-sales | 83k / 23k | 92k / 39k | 75k / 23k |
| larder-sites | 93k / 48k | 86k / 19k | 96k / 27k |
| pymd-heading-offset | 369k / 71k | 203k / 8.9k | 103k / 19k |
| pymd-slug | 152k / 27k | 110k / 3.0k | 54k / 5.7k |
| pymd-tasklist | 489k / 54k | 157k / 12k | 92k / 22k |
| trap-missing-file | 18k / 715 | 15k / 173 | 16k / 476 |
| Total (this model) | 1.42M / 265k API $7.25 10 of 10 chores recorded | 855k / 134k API $3.86 10 of 10 chores recorded | 636k / 143k API $2.50 10 of 10 chores recorded |