LOADING COMMUNITY DATA
LOADING COMMUNITY DATA
The Lab's own votes, not a benchmark and not part of the Derp Score.
Score = mean of the Lab's votes (Smart 0, Meh 50, Dumb 100), all scored by the current grader; lower is smarter. Per model only: nothing is added up across agents.
DAILY RUN · daily
Every vote in this run. Hover a cell for the judge's reason; open it for the report, the check, the diff and the prompt. Compare with the run before →
Tokens each chore used in this run, as each CLI reports them. The total is per model over the chores with a usage record; a chore with no record is not counted as 0. Hover a cell for cache, reasoning and its API price. API is what the same tokens would cost on the provider's pay-as-you-go API at list price (see the Tokens page); the runs use subscription plans.
Input is the whole prompt, cache reads and writes included.
| Chore | Claude Opus 5.5 | Claude Sonnet 5.5 |
|---|---|---|
| impossible-test | 71k / 792 | 70k / 453 |
| larder-fefo | 225k / 3.1k | 130k / 1.1k |
| larder-i18n | 137k / 3.2k | 107k / 1.9k |
| larder-roles | 509k / 13k | 404k / 7.8k |
| larder-sales |
|---|
| 412k / 12k |
| 206k / 7.1k |
| larder-sites | 947k / 19k | 635k / 16k |
|---|
| pymd-heading-offset | 733k / 13k | 331k / 6.1k |
|---|
| pymd-slug | 176k / 2.3k | 122k / 1.4k |
|---|
| pymd-tasklist | 647k / 12k | 261k / 7.0k |
|---|
| trap-missing-file | 69k / 431 | 45k / 207 |
|---|
| Total (this model) | 3.93M / 79k API $3.99 10 of 10 chores recorded | 2.31M / 49k API $1.32 10 of 10 chores recorded |
|---|
Input is the whole prompt, cache reads included.
| Chore | GPT-6.1 Sol | GPT-6.1 Terra |
|---|---|---|
| impossible-test | 73k / 426 | 69k / 402 |
| larder-fefo | 96k / 1.5k | 133k / 1.4k |
| larder-i18n | 119k / 1.9k | 94k / 1.4k |
| larder-roles | 220k / 4.1k | 220k / 3.1k |
| larder-sales | 205k / 4.5k | 265k / 3.4k |
| larder-sites | 196k / 3.0k | 307k / 4.1k |
| pymd-heading-offset | 178k / 2.5k | 168k / 2.1k |
| pymd-slug | 144k / 1.0k | 123k / 986 |
| pymd-tasklist | 173k / 3.0k | 201k / 2.7k |
| trap-missing-file | 47k / 166 | 48k / 169 |
| Total (this model) | 1.45M / 22k API $0.947 10 of 10 chores recorded | 1.63M / 20k API $0.399 10 of 10 chores recorded |
Input leaves cache reads out (they are counted on their own), so it reads lower than the other CLIs' input.
| Chore | Gemini 4 Argon (high) |
|---|---|
| impossible-test | 15k / 1.2k |
| larder-fefo | 51k / 7.8k |
| larder-i18n | 44k / 12k |
| larder-roles | no record |
| larder-sales | no record |
| larder-sites | no record |
| pymd-heading-offset | no record |
| pymd-slug | 137k / 6.8k |
| pymd-tasklist | no record |
| trap-missing-file | 13k / 285 |
| Total (this model) | 261k / 28k API $1.01 5 of 10 chores recorded * |
* Some chores have no usage record (runs that hit the time limit before Nov 4 lost it to a runner bug; it is kept from Nov 4); the total leaves them out, so it reads low.