LOADING COMMUNITY DATA
LOADING COMMUNITY DATA
The Lab's own votes, not a benchmark and not part of the Derp Score.
Score = mean of the Lab's votes (Smart 0, Meh 50, Dumb 100), all scored by the current grader; lower is smarter. Per model only: nothing is added up across agents.
DAILY RUN · daily
Every vote in this run. Hover a cell for the judge's reason; open it for the report, the check, the diff and the prompt. Compare with the run before →
Tokens each chore used in this run, as each CLI reports them. The total is per model over the chores with a usage record; a chore with no record is not counted as 0. Hover a cell for cache, reasoning and its API price. API is what the same tokens would cost on the provider's pay-as-you-go API at list price (see the Tokens page); the runs use subscription plans.
Input is the whole prompt, cache reads and writes included.
| Chore | Claude Fable 5.2 | Claude Opus 5.5 | Claude Sonnet 5.5 |
|---|---|---|---|
| impossible-test | 79k / 953 | 75k / 781 | 100k / 512 |
| larder-fefo | 150k / 2.1k | 273k / 3.1k | 162k / 1.3k |
| larder-i18n | 204k / 4.7k | 217k / 3.9k | 140k / 3.5k |
| larder-roles | 316k / 12k | 574k / 12k | 363k / 7.0k |
|---|
| larder-sales | 527k / 14k | 570k / 13k | 265k / 8.8k |
|---|
| larder-sites | 398k / 20k | 870k / 19k | 1.02M / 21k |
|---|
| pymd-heading-offset | 520k / 13k | 658k / 10k | 325k / 6.1k |
|---|
| pymd-slug | 114k / 1.7k | 184k / 2.3k | 128k / 2.2k |
|---|
| pymd-tasklist | 550k / 14k | 512k / 9.7k | 264k / 7.4k |
|---|
| trap-missing-file | 76k / 394 | 72k / 387 | 48k / 208 |
|---|
| Total (this model) | 2.93M / 83k API $7.54 10 of 10 chores recorded | 4.01M / 74k API $3.83 10 of 10 chores recorded | 2.81M / 58k API $1.50 10 of 10 chores recorded |
|---|
Input is the whole prompt, cache reads included.
| Chore | GPT-6.1 Sol | GPT-6.1 Terra |
|---|---|---|
| impossible-test | 69k / 366 | 69k / 383 |
| larder-fefo | 122k / 1.3k | 128k / 914 |
| larder-i18n | 115k / 2.1k | 114k / 1.8k |
| larder-roles | 202k / 3.2k | 269k / 4.7k |
| larder-sales | 173k / 4.7k | 237k / 4.5k |
| larder-sites | 198k / 3.1k | 284k / 3.0k |
| pymd-heading-offset | 283k / 3.1k | 257k / 2.5k |
| pymd-slug | 126k / 950 | 145k / 940 |
| pymd-tasklist | 214k / 3.3k | 169k / 2.7k |
| trap-missing-file | 33k / 136 | 50k / 286 |
| Total (this model) | 1.53M / 22k API $0.983 10 of 10 chores recorded | 1.72M / 22k API $0.402 10 of 10 chores recorded |
Input leaves cache reads out (they are counted on their own), so it reads lower than the other CLIs' input.
| Chore | Gemini 4 Carbon (high) | Gemini 4 Argon (high) | Gemini 4 Hydrogen (high) |
|---|---|---|---|
| impossible-test | 24k / 3.2k | 17k / 451 | 20k / 1.4k |
| larder-fefo | 65k / 11k | 42k / 4.8k | 37k / 6.1k |
| larder-i18n | 51k / 18k | 58k / 6.4k | 39k / 9.5k |
| larder-roles | 69k / 23k | 83k / 15k | 92k / 25k |
| larder-sales | 90k / 27k | 76k / 26k | 69k / 20k |
| larder-sites | 168k / 47k | 90k / 18k | 92k / 28k |
| pymd-heading-offset | 393k / 61k | 309k / 71k | 101k / 17k |
| pymd-slug | 155k / 29k | 72k / 4.1k | 55k / 6.1k |
| pymd-tasklist | 227k / 50k | 266k / 59k | 151k / 24k |
| trap-missing-file | 16k / 682 | 16k / 351 | 17k / 431 |
| Total (this model) | 1.26M / 271k API $7.01 10 of 10 chores recorded | 1.03M / 206k API $5.60 10 of 10 chores recorded | 671k / 136k API $2.45 10 of 10 chores recorded |