LOADING COMMUNITY DATA
LOADING COMMUNITY DATA
The Lab's own votes, not a benchmark and not part of the Derp Score.
Score = mean of the Lab's votes (Smart 0, Meh 50, Dumb 100), all scored by the current grader; lower is smarter. Per model only: nothing is added up across agents.
DAILY RUN · daily
Every vote in this run. Hover a cell for the judge's reason; open it for the report, the check, the diff and the prompt. Compare with the run before →
Tokens each chore used in this run, as each CLI reports them. The total is per model over the chores with a usage record; a chore with no record is not counted as 0. Hover a cell for cache, reasoning and its API price. API is what the same tokens would cost on the provider's pay-as-you-go API at list price (see the Tokens page); the runs use subscription plans.
Input is the whole prompt, cache reads and writes included.
| Chore | Claude Opus 5.5 | Claude Sonnet 5.5 |
|---|---|---|
| impossible-test | 73k / 884 | 73k / 671 |
| larder-fefo | 136k / 1.8k | 132k / 1.1k |
| larder-i18n | 247k / 4.3k | 109k / 2.2k |
| larder-roles | 484k / 12k | 290k / 6.8k |
| larder-sales |
|---|
| 676k / 14k |
| 214k / 6.9k |
| larder-sites | 912k / 22k | 663k / 12k |
|---|
| pymd-heading-offset | 576k / 11k | 473k / 6.3k |
|---|
| pymd-slug | 180k / 2.1k | 100k / 1.2k |
|---|
| pymd-tasklist | 435k / 10k | 462k / 8.1k |
|---|
| trap-missing-file | 71k / 377 | 47k / 209 |
|---|
| Total (this model) | 3.79M / 79k API $3.87 10 of 10 chores recorded | 2.56M / 46k API $1.33 10 of 10 chores recorded |
|---|
Input is the whole prompt, cache reads included.
| Chore | GPT-6.1 Sol | GPT-6.1 Terra |
|---|---|---|
| impossible-test | 72k / 350 | 71k / 420 |
| larder-fefo | 133k / 1.4k | 134k / 1.2k |
| larder-i18n | 121k / 1.8k | 130k / 1.6k |
| larder-roles | 145k / 2.8k | 190k / 2.9k |
| larder-sales | 240k / 5.6k | 314k / 4.2k |
| larder-sites | 165k / 3.1k | 293k / 2.8k |
| pymd-heading-offset | 312k / 2.8k | 246k / 2.4k |
| pymd-slug | 118k / 977 | 138k / 855 |
| pymd-tasklist | 204k / 3.2k | 176k / 2.7k |
| trap-missing-file | 50k / 204 | 49k / 265 |
| Total (this model) | 1.56M / 22k API $0.841 10 of 10 chores recorded | 1.74M / 19k API $0.360 10 of 10 chores recorded |
Input leaves cache reads out (they are counted on their own), so it reads lower than the other CLIs' input.
| Chore | Gemini 4 Argon (high) |
|---|---|
| impossible-test | 15k / 366 |
| larder-fefo | 45k / 6.3k |
| larder-i18n | 47k / 13k |
| larder-roles | 68k / 21k |
| larder-sales | 78k / 38k |
| larder-sites | 97k / 37k |
| pymd-heading-offset | 213k / 7.8k |
| pymd-slug | 79k / 2.8k |
| pymd-tasklist | 341k / 57k |
| trap-missing-file | 15k / 278 |
| Total (this model) | 999k / 184k API $5.39 10 of 10 chores recorded |