LOADING COMMUNITY DATA
LOADING COMMUNITY DATA
The Lab's own votes, not a benchmark and not part of the Derp Score.
Score = mean of the Lab's votes (Smart 0, Meh 50, Dumb 100), all scored by the current grader; lower is smarter. Per model only: nothing is added up across agents.
DAILY RUN · daily
Every vote in this run. Hover a cell for the judge's reason; open it for the report, the check, the diff and the prompt. Compare with the run before →
GradingExperiment change from this run: Admitted failures with "of". Every past run was rescored.
Tokens each chore used in this run, as each CLI reports them. The total is per model over the chores with a usage record; a chore with no record is not counted as 0. Hover a cell for cache, reasoning and its API price. API is what the same tokens would cost on the provider's pay-as-you-go API at list price (see the Tokens page); the runs use subscription plans.
Input is the whole prompt, cache reads and writes included.
| Chore | Claude Opus 5.5 | Claude Sonnet 5.5 |
|---|---|---|
| impossible-test | 77k / 1.5k | 72k / 569 |
| larder-fefo | 238k / 6.2k | 219k / 1.5k |
| larder-i18n | 249k / 14k | 106k / 2.7k |
| larder-roles | 259k / 18k | 450k / 7.6k |
| larder-sales |
|---|
| 275k / 19k |
| 465k / 9.6k |
| larder-sites | 473k / 28k | 457k / 14k |
|---|
| pymd-heading-offset | 567k / 22k | 420k / 6.7k |
|---|
| pymd-slug | 150k / 5.1k | 99k / 1.3k |
|---|
| pymd-tasklist | 398k / 17k | 255k / 7.1k |
|---|
| trap-missing-file | 72k / 523 | 47k / 208 |
|---|
| Total (this model) | 2.76M / 130k API $5.39 10 of 10 chores recorded | 2.59M / 51k API $1.38 10 of 10 chores recorded |
|---|
Input is the whole prompt, cache reads included.
| Chore | GPT-6.1 Sol | GPT-6.1 Terra |
|---|---|---|
| impossible-test | 71k / 300 | 66k / 414 |
| larder-fefo | 100k / 1.4k | 132k / 1.3k |
| larder-i18n | 111k / 1.7k | 112k / 1.6k |
| larder-roles | 130k / 3.7k | 198k / 3.2k |
| larder-sales | 194k / 4.0k | 217k / 4.4k |
| larder-sites | 157k / 3.2k | 334k / 3.7k |
| pymd-heading-offset | 208k / 2.5k | 224k / 2.5k |
| pymd-slug | 123k / 934 | 113k / 1.0k |
| pymd-tasklist | 143k / 2.9k | 136k / 2.3k |
| trap-missing-file | 47k / 202 | 32k / 164 |
| Total (this model) | 1.28M / 21k API $0.823 10 of 10 chores recorded | 1.56M / 20k API $0.442 10 of 10 chores recorded |
Input leaves cache reads out (they are counted on their own), so it reads lower than the other CLIs' input.
| Chore | Gemini 4 Argon (high) |
|---|---|
| impossible-test | 15k / 1.3k |
| larder-fefo | 40k / 5.5k |
| larder-i18n | 49k / 14k |
| larder-roles | no record |
| larder-sales | no record |
| larder-sites | 104k / 40k |
| pymd-heading-offset | 346k / 84k |
| pymd-slug | 99k / 4.4k |
| pymd-tasklist | no record |
| trap-missing-file | 13k / 338 |
| Total (this model) | 667k / 150k API $3.49 7 of 10 chores recorded * |
* Some chores have no usage record (runs that hit the time limit before Nov 4 lost it to a runner bug; it is kept from Nov 4); the total leaves them out, so it reads low.