LOADING COMMUNITY DATA
LOADING COMMUNITY DATA
The Lab's own votes, not a benchmark and not part of the Derp Score.
Score = mean of the Lab's votes (Smart 0, Meh 50, Dumb 100), all scored by the current grader; lower is smarter. Per model only: nothing is added up across agents.
DAILY RUN · daily
Every vote in this run. Hover a cell for the judge's reason; open it for the report, the check, the diff and the prompt. Compare with the run before →
Tokens each chore used in this run, as each CLI reports them. The total is per model over the chores with a usage record; a chore with no record is not counted as 0. Hover a cell for cache, reasoning and its API price. API is what the same tokens would cost on the provider's pay-as-you-go API at list price (see the Tokens page); the runs use subscription plans.
Input is the whole prompt, cache reads and writes included.
| Chore | Claude Opus 5.5 | Claude Sonnet 5.5 |
|---|---|---|
| impossible-test | 74k / 1.1k | 73k / 624 |
| larder-fefo | 202k / 2.5k | 256k / 2.2k |
| larder-i18n | 182k / 3.6k | 141k / 3.3k |
| larder-roles | 525k / 13k | 289k / 7.5k |
| larder-sales |
|---|
| 382k / 11k |
| 264k / 8.0k |
| larder-sites | 1.03M / 19k | 443k / 9.1k |
|---|
| pymd-heading-offset | 566k / 9.9k | 398k / 6.2k |
|---|
| pymd-slug | 126k / 2.2k | 151k / 1.9k |
|---|
| pymd-tasklist | 542k / 11k | 303k / 6.7k |
|---|
| trap-missing-file | 47k / 354 | 47k / 207 |
|---|
| Total (this model) | 3.67M / 73k API $3.69 10 of 10 chores recorded | 2.36M / 46k API $1.29 10 of 10 chores recorded |
|---|
Input is the whole prompt, cache reads included.
| Chore | GPT-6.1 Sol | GPT-6.1 Terra |
|---|---|---|
| impossible-test | 73k / 381 | 70k / 424 |
| larder-fefo | 131k / 1.3k | 116k / 1.4k |
| larder-i18n | 121k / 1.7k | 98k / 1.4k |
| larder-roles | 183k / 3.5k | 184k / 2.8k |
| larder-sales | 186k / 4.2k | 297k / 4.0k |
| larder-sites | 248k / 3.5k | 285k / 5.2k |
| pymd-heading-offset | 170k / 2.5k | 256k / 2.3k |
| pymd-slug | 181k / 1.1k | 136k / 1.0k |
| pymd-tasklist | 170k / 3.0k | 207k / 2.6k |
| trap-missing-file | 50k / 185 | 49k / 279 |
| Total (this model) | 1.51M / 21k API $0.893 10 of 10 chores recorded | 1.70M / 21k API $0.411 10 of 10 chores recorded |
Input leaves cache reads out (they are counted on their own), so it reads lower than the other CLIs' input.
| Chore | Gemini 4 Argon (high) |
|---|---|
| impossible-test | 17k / 1.2k |
| larder-fefo | 36k / 3.2k |
| larder-i18n | 47k / 6.1k |
| larder-roles | 68k / 13k |
| larder-sales | 88k / 40k |
| larder-sites | 104k / 43k |
| pymd-heading-offset | no record |
| pymd-slug | 93k / 6.0k |
| pymd-tasklist | 415k / 61k |
| trap-missing-file | 15k / 251 |
| Total (this model) | 883k / 173k API $4.35 9 of 10 chores recorded * |
* Some chores have no usage record (runs that hit the time limit before Nov 4 lost it to a runner bug; it is kept from Nov 4); the total leaves them out, so it reads low.