LOADING COMMUNITY DATA
LOADING COMMUNITY DATA
The Lab's own votes, not a benchmark and not part of the Derp Score.
Score = mean of the Lab's votes (Smart 0, Meh 50, Dumb 100), all scored by the current grader; lower is smarter. Per model only: nothing is added up across agents.
DAILY RUN · daily
Every vote in this run. Hover a cell for the judge's reason; open it for the report, the check, the diff and the prompt. Compare with the run before →
ChoresExperiment change from this run: Pantry chores retired.
ChoresExperiment change from this run: Three Toonify chores with follow-up requests. Every past run was rescored.
Tokens each chore used in this run, as each CLI reports them. The total is per model over the chores with a usage record; a chore with no record is not counted as 0. Hover a cell for cache, reasoning and its API price. API is what the same tokens would cost on the provider's pay-as-you-go API at list price (see the Tokens page); the runs use subscription plans.
Input is the whole prompt, cache reads and writes included.
| Chore | Claude Fable 5.2 | Claude Opus 5.5 | Claude Sonnet 5.5 |
|---|---|---|---|
| larder-fefo | 150k / 2.0k | 168k / 1.7k | 222k / 1.8k |
| larder-i18n | 169k / 5.3k | 157k / 3.8k | 143k / 2.3k |
| larder-roles | 366k / 12k | 671k / 14k | 262k / 7.2k |
| larder-sales | 318k / 12k | 377k / 9.9k | 263k / 7.5k |
|---|
| larder-sites | 462k / 21k | 853k / 16k | 330k / 16k |
|---|
| pymd-heading-offset | 402k / 12k | 748k / 12k | 425k / 6.6k |
|---|
| pymd-slug | 113k / 1.9k | 187k / 2.0k | 127k / 1.2k |
|---|
| pymd-tasklist | 388k / 12k | 406k / 9.0k | 379k / 9.0k |
|---|
| toon-episode-copy | 1.75M / 25k | 2.68M / 26k | 694k / 14k |
|---|
| toon-failed-charges | 2.18M / 32k | 4.50M / 37k | 2.14M / 21k |
|---|
| toon-image-fetch | 1.28M / 20k | 2.04M / 22k | 1.33M / 14k |
|---|
| Total (this model) | 7.58M / 155k API $13.85 11 of 11 chores recorded | 12.79M / 154k API $8.34 11 of 11 chores recorded | 6.32M / 101k API $2.67 11 of 11 chores recorded |
|---|
Input is the whole prompt, cache reads included.
| Chore | GPT-6.1 Sol | GPT-6.1 Terra |
|---|---|---|
| larder-fefo | 99k / 1.3k | 153k / 1.4k |
| larder-i18n | 115k / 1.8k | 90k / 1.3k |
| larder-roles | 237k / 4.3k | 211k / 3.3k |
| larder-sales | 170k / 4.2k | 222k / 3.4k |
| larder-sites | 209k / 3.2k | 289k / 4.0k |
| pymd-heading-offset | 229k / 2.8k | 258k / 2.3k |
| pymd-slug | 180k / 1.1k | 115k / 1.0k |
| pymd-tasklist | 142k / 3.0k | 147k / 2.4k |
| toon-episode-copy | 2.31M / 24k | 2.18M / 17k |
| toon-failed-charges | 3.39M / 27k | 1.53M / 8.2k |
| toon-image-fetch | 955k / 13k | 1.72M / 20k |
| Total (this model) | 8.04M / 85k API $2.99 11 of 11 chores recorded | 6.91M / 65k API $1.26 11 of 11 chores recorded |
Input leaves cache reads out (they are counted on their own), so it reads lower than the other CLIs' input.
| Chore | Gemini 4 Carbon (high) | Gemini 4 Argon (high) |
|---|---|---|
| larder-fefo | 58k / 12k | 38k / 3.2k |
| larder-i18n | 52k / 15k | 50k / 14k |
| larder-roles | 88k / 29k | 68k / 12k |
| larder-sales | 77k / 33k | 74k / 28k |
| larder-sites | 180k / 51k | 93k / 27k |
| pymd-heading-offset | 269k / 51k | 197k / 8.1k |
| pymd-slug | 170k / 27k | 78k / 2.2k |
| pymd-tasklist | 183k / 30k | 413k / 67k |
| toon-episode-copy | 941k / 268k | 666k / 153k |
| toon-failed-charges | — | 1.12M / 343k |
| toon-image-fetch | — | 796k / 220k |
| Total (this model) | 2.02M / 516k API $13.93 9 of 9 chores recorded | 3.60M / 878k API $21.89 11 of 11 chores recorded |