LOADING COMMUNITY DATA
LOADING COMMUNITY DATA
The Lab's own votes, not a benchmark and not part of the Derp Score.
Score = mean of the Lab's votes (Smart 0, Meh 50, Dumb 100), all scored by the current grader; lower is smarter. Per model only: nothing is added up across agents.
DAILY RUN · daily
Every vote in this run. Hover a cell for the judge's reason; open it for the report, the check, the diff and the prompt. Compare with the run before →
Tokens each chore used in this run, as each CLI reports them. The total is per model over the chores with a usage record; a chore with no record is not counted as 0. Hover a cell for cache, reasoning and its API price. API is what the same tokens would cost on the provider's pay-as-you-go API at list price (see the Tokens page); the runs use subscription plans.
Input is the whole prompt, cache reads and writes included.
| Chore | Claude Opus 5.5 | Claude Sonnet 5.5 |
|---|---|---|
| impossible-test | 73k / 919 | 72k / 725 |
| larder-fefo | 136k / 1.9k | 189k / 1.6k |
| larder-i18n | 202k / 3.2k | 108k / 2.1k |
| larder-roles | 714k / 14k | 201k / 6.6k |
| larder-sales |
|---|
| 491k / 12k |
| 489k / 8.5k |
| larder-sites | 982k / 25k | 529k / 15k |
|---|
| pymd-heading-offset | 649k / 8.7k | 328k / 6.0k |
|---|
| pymd-slug | 208k / 2.2k | 126k / 1.5k |
|---|
| pymd-tasklist | 624k / 12k | 429k / 7.8k |
|---|
| trap-missing-file | 70k / 269 | 47k / 207 |
|---|
| Total (this model) | 4.15M / 80k API $3.98 10 of 10 chores recorded | 2.52M / 50k API $1.39 10 of 10 chores recorded |
|---|
Input is the whole prompt, cache reads included.
| Chore | GPT-6.1 Sol | GPT-6.1 Terra |
|---|---|---|
| impossible-test | 73k / 359 | 69k / 443 |
| larder-fefo | 99k / 1.4k | 168k / 1.6k |
| larder-i18n | 120k / 1.8k | 111k / 1.7k |
| larder-roles | 144k / 3.2k | 170k / 3.4k |
| larder-sales | 221k / 3.9k | 253k / 4.0k |
| larder-sites | 246k / 3.8k | 263k / 2.7k |
| pymd-heading-offset | 207k / 2.7k | 320k / 2.6k |
| pymd-slug | 118k / 1.1k | 138k / 900 |
| pymd-tasklist | 209k / 2.9k | 224k / 2.9k |
| trap-missing-file | 50k / 197 | 46k / 280 |
| Total (this model) | 1.49M / 21k API $0.838 10 of 10 chores recorded | 1.76M / 21k API $0.398 10 of 10 chores recorded |
Input leaves cache reads out (they are counted on their own), so it reads lower than the other CLIs' input.
| Chore | Gemini 4 Argon (high) |
|---|---|
| impossible-test | 14k / 1.3k |
| larder-fefo | 37k / 5.1k |
| larder-i18n | 42k / 11k |
| larder-roles | 69k / 33k |
| larder-sales | 107k / 45k |
| larder-sites | 78k / 17k |
| pymd-heading-offset | 199k / 6.9k |
| pymd-slug | 82k / 2.4k |
| pymd-tasklist | 415k / 61k |
| trap-missing-file | 16k / 430 |
| Total (this model) | 1.06M / 183k API $5.41 10 of 10 chores recorded |