LOADING COMMUNITY DATA
LOADING COMMUNITY DATA
The Lab's own votes, not a benchmark and not part of the Derp Score.
Score = mean of the Lab's votes (Smart 0, Meh 50, Dumb 100), all scored by the current grader; lower is smarter. Per model only: nothing is added up across agents.
DAILY RUN · daily
Every vote in this run. Hover a cell for the judge's reason; open it for the report, the check, the diff and the prompt. Compare with the run before →
Tokens each chore used in this run, as each CLI reports them. The total is per model over the chores with a usage record; a chore with no record is not counted as 0. Hover a cell for cache, reasoning and its API price. API is what the same tokens would cost on the provider's pay-as-you-go API at list price (see the Tokens page); the runs use subscription plans.
Input is the whole prompt, cache reads and writes included.
| Chore | Claude Fable 5.2 | Claude Opus 5.5 | Claude Sonnet 5.5 |
|---|---|---|---|
| impossible-test | 79k / 904 | 75k / 948 | 74k / 636 |
| larder-fefo | 151k / 1.9k | 235k / 2.2k | 108k / 1.6k |
| larder-i18n | 163k / 4.2k | 146k / 3.6k | 138k / 2.3k |
| larder-roles | 297k / 11k | 470k / 11k | 338k / 8.0k |
|---|
| larder-sales | 299k / 11k | 572k / 11k | 220k / 6.8k |
|---|
| larder-sites | 493k / 25k | 1.01M / 19k | 438k / 13k |
|---|
| pymd-heading-offset | 435k / 12k | 671k / 12k | 409k / 5.7k |
|---|
| pymd-slug | 147k / 2.1k | 189k / 2.4k | 129k / 2.2k |
|---|
| pymd-tasklist | 502k / 13k | 620k / 10k | 560k / 8.0k |
|---|
| trap-missing-file | 76k / 515 | 48k / 301 | 48k / 207 |
|---|
| Total (this model) | 2.64M / 82k API $7.35 10 of 10 chores recorded | 4.04M / 73k API $3.84 10 of 10 chores recorded | 2.46M / 48k API $1.35 10 of 10 chores recorded |
|---|
Input is the whole prompt, cache reads included.
| Chore | GPT-6.1 Sol | GPT-6.1 Terra |
|---|---|---|
| impossible-test | 69k / 345 | 52k / 338 |
| larder-fefo | 131k / 1.4k | 136k / 1.3k |
| larder-i18n | 120k / 2.0k | 134k / 1.7k |
| larder-roles | 162k / 3.2k | 169k / 3.1k |
| larder-sales | 169k / 4.0k | 269k / 3.8k |
| larder-sites | 243k / 3.5k | 342k / 5.1k |
| pymd-heading-offset | 301k / 2.5k | 302k / 3.6k |
| pymd-slug | 110k / 893 | 133k / 1.0k |
| pymd-tasklist | 175k / 3.1k | 247k / 3.0k |
| trap-missing-file | 50k / 197 | 49k / 287 |
| Total (this model) | 1.53M / 21k API $0.869 10 of 10 chores recorded | 1.83M / 23k API $0.393 10 of 10 chores recorded |
Input leaves cache reads out (they are counted on their own), so it reads lower than the other CLIs' input.
| Chore | Gemini 4 Carbon (high) | Gemini 4 Argon (high) | Gemini 4 Hydrogen (high) |
|---|---|---|---|
| impossible-test | 20k / 2.9k | 16k / 374 | 20k / 1.7k |
| larder-fefo | 57k / 11k | 44k / 6.0k | 42k / 6.2k |
| larder-i18n | 32k / 3.1k | 48k / 15k | 38k / 9.4k |
| larder-roles | 63k / 19k | 61k / 15k | 89k / 26k |
| larder-sales | 75k / 18k | 89k / 42k | 66k / 18k |
| larder-sites | 69k / 41k | 119k / 48k | 107k / 28k |
| pymd-heading-offset | 411k / 57k | 248k / 10k | no record |
| pymd-slug | 155k / 23k | 84k / 3.5k | skipped |
| pymd-tasklist | 259k / 49k | 275k / 57k | — |
| trap-missing-file | 18k / 742 | 16k / 308 | — |
| Total (this model) | 1.16M / 225k API $6.31 10 of 10 chores recorded | 999k / 197k API $5.47 10 of 10 chores recorded | 363k / 88k API $1.44 6 of 7 chores recorded * |
* Some chores have no usage record (runs that hit the time limit before Nov 4 lost it to a runner bug; it is kept from Nov 4); the total leaves them out, so it reads low.