LOADING COMMUNITY DATA
LOADING COMMUNITY DATA
The Lab's own votes, not a benchmark and not part of the Derp Score.
Score = mean of the Lab's votes (Smart 0, Meh 50, Dumb 100), all scored by the current grader; lower is smarter. Per model only: nothing is added up across agents.
DAILY RUN · daily
Every vote in this run. Hover a cell for the judge's reason; open it for the report, the check, the diff and the prompt. Compare with the run before →
Tokens each chore used in this run, as each CLI reports them. The total is per model over the chores with a usage record; a chore with no record is not counted as 0. Hover a cell for cache, reasoning and its API price. API is what the same tokens would cost on the provider's pay-as-you-go API at list price (see the Tokens page); the runs use subscription plans.
Input is the whole prompt, cache reads and writes included.
| Chore | Claude Opus 5.5 | Claude Sonnet 5.5 |
|---|---|---|
| impossible-test | 73k / 968 | 73k / 431 |
| larder-fefo | 236k / 3.3k | 134k / 1.1k |
| larder-i18n | 174k / 3.3k | 108k / 2.0k |
| larder-roles | 527k / 13k | 446k / 7.6k |
| larder-sales |
|---|
| 428k / 11k |
| 347k / 7.8k |
| larder-sites | 1.15M / 24k | 695k / 19k |
|---|
| pymd-heading-offset | 603k / 12k | 305k / 5.4k |
|---|
| pymd-slug | 182k / 2.1k | 182k / 2.5k |
|---|
| pymd-tasklist | 524k / 9.7k | 522k / 8.3k |
|---|
| trap-missing-file | 71k / 445 | 47k / 206 |
|---|
| Total (this model) | 3.97M / 79k API $3.98 10 of 10 chores recorded | 2.86M / 55k API $1.47 10 of 10 chores recorded |
|---|
Input is the whole prompt, cache reads included.
| Chore | GPT-6.1 Sol | GPT-6.1 Terra |
|---|---|---|
| impossible-test | 72k / 427 | 70k / 474 |
| larder-fefo | 99k / 1.3k | 156k / 1.3k |
| larder-i18n | 114k / 1.8k | 130k / 2.0k |
| larder-roles | 204k / 4.0k | 194k / 3.0k |
| larder-sales | 215k / 4.2k | 190k / 3.6k |
| larder-sites | 222k / 5.0k | 330k / 4.9k |
| pymd-heading-offset | 174k / 2.3k | 241k / 2.4k |
| pymd-slug | 155k / 1.1k | 119k / 976 |
| pymd-tasklist | 167k / 3.2k | 144k / 2.6k |
| trap-missing-file | 47k / 184 | 49k / 273 |
| Total (this model) | 1.47M / 23k API $0.996 10 of 10 chores recorded | 1.62M / 21k API $0.404 10 of 10 chores recorded |
Input leaves cache reads out (they are counted on their own), so it reads lower than the other CLIs' input.
| Chore | Gemini 4 Argon (high) |
|---|---|
| impossible-test | 13k / 460 |
| larder-fefo | 38k / 5.8k |
| larder-i18n | 53k / 11k |
| larder-roles | 93k / 33k |
| larder-sales | 177k / 49k |
| larder-sites | 124k / 44k |
| pymd-heading-offset | 436k / 92k |
| pymd-slug | 104k / 5.4k |
| pymd-tasklist | 313k / 77k |
| trap-missing-file | 14k / 266 |
| Total (this model) | 1.37M / 319k API $7.85 10 of 10 chores recorded |