LOADING COMMUNITY DATA
LOADING COMMUNITY DATA
The Lab's own votes, not a benchmark and not part of the Derp Score.
Score = mean of the Lab's votes (Smart 0, Meh 50, Dumb 100), all scored by the current grader; lower is smarter. Per model only: nothing is added up across agents.
DAILY RUN · daily
Every vote in this run. Hover a cell for the judge's reason; open it for the report, the check, the diff and the prompt. Compare with the run before →
ModelExperiment change from this run: Claude Fable 5.2 added.
Tokens each chore used in this run, as each CLI reports them. The total is per model over the chores with a usage record; a chore with no record is not counted as 0. Hover a cell for cache, reasoning and its API price. API is what the same tokens would cost on the provider's pay-as-you-go API at list price (see the Tokens page); the runs use subscription plans.
Input is the whole prompt, cache reads and writes included.
| Chore | Claude Fable 5.2 | Claude Opus 5.5 | Claude Sonnet 5.5 |
|---|---|---|---|
| impossible-test | 107k / 1.0k | 75k / 877 | 75k / 617 |
| larder-fefo | 152k / 2.4k | 206k / 2.3k | 161k / 1.4k |
| larder-i18n | 166k / 4.7k | 218k / 4.1k | 143k / 2.5k |
| larder-roles | 196k / 9.8k | 599k / 13k | 271k / 8.9k |
|---|
| larder-sales | 220k / 15k | 672k / 12k | 261k / 7.5k |
|---|
| larder-sites | 474k / 22k | 1.02M / 20k | 751k / 19k |
|---|
| pymd-heading-offset | 357k / 10k | 705k / 12k | 289k / 5.8k |
|---|
| pymd-slug | 153k / 2.0k | 186k / 2.2k | 156k / 2.1k |
|---|
| pymd-tasklist | 393k / 14k | 807k / 13k | 231k / 6.4k |
|---|
| trap-missing-file | 76k / 495 | 72k / 388 | 48k / 208 |
|---|
| Total (this model) | 2.29M / 81k API $7.29 10 of 10 chores recorded | 4.56M / 80k API $4.14 10 of 10 chores recorded | 2.39M / 54k API $1.42 10 of 10 chores recorded |
|---|
Input is the whole prompt, cache reads included.
| Chore | GPT-6.1 Sol | GPT-6.1 Terra |
|---|---|---|
| impossible-test | 72k / 332 | 70k / 431 |
| larder-fefo | 132k / 1.5k | 133k / 1.3k |
| larder-i18n | 102k / 1.9k | 113k / 1.5k |
| larder-roles | 237k / 3.8k | 193k / 2.9k |
| larder-sales | 282k / 5.2k | 246k / 4.7k |
| larder-sites | 180k / 2.9k | 352k / 5.7k |
| pymd-heading-offset | 242k / 2.6k | 234k / 2.4k |
| pymd-slug | 140k / 968 | 164k / 1.2k |
| pymd-tasklist | 212k / 3.2k | 151k / 2.5k |
| trap-missing-file | 33k / 158 | 49k / 190 |
| Total (this model) | 1.63M / 22k API $0.903 10 of 10 chores recorded | 1.71M / 23k API $0.407 10 of 10 chores recorded |
Input leaves cache reads out (they are counted on their own), so it reads lower than the other CLIs' input.
| Chore | Gemini 4 Carbon (high) | Gemini 4 Argon (high) | Gemini 4 Hydrogen (high) |
|---|---|---|---|
| impossible-test | 21k / 3.2k | 16k / 1.1k | 20k / 1.5k |
| larder-fefo | 57k / 12k | 48k / 5.6k | 36k / 6.3k |
| larder-i18n | 48k / 16k | 46k / 11k | 41k / 11k |
| larder-roles | 75k / 26k | 74k / 23k | 93k / 25k |
| larder-sales | 92k / 29k | 69k / 31k | 64k / 21k |
| larder-sites | 82k / 17k | 97k / 39k | 87k / 26k |
| pymd-heading-offset | 493k / 52k | 334k / 74k | 90k / 18k |
| pymd-slug | 148k / 34k | 76k / 1.7k | 51k / 5.2k |
| pymd-tasklist | 252k / 47k | 214k / 45k | 96k / 20k |
| trap-missing-file | 16k / 598 | 18k / 462 | 17k / 418 |
| Total (this model) | 1.28M / 238k API $6.77 10 of 10 chores recorded | 991k / 232k API $5.55 10 of 10 chores recorded | 595k / 134k API $2.33 10 of 10 chores recorded |