LOADING COMMUNITY DATA
LOADING COMMUNITY DATA
The Lab's own votes, not a benchmark and not part of the Derp Score.
Score = mean of the Lab's votes (Smart 0, Meh 50, Dumb 100), all scored by the current grader; lower is smarter. Per model only: nothing is added up across agents.
DAILY RUN · daily
Every vote in this run. Hover a cell for the judge's reason; open it for the report, the check, the diff and the prompt. Compare with the run before →
Tokens each chore used in this run, as each CLI reports them. The total is per model over the chores with a usage record; a chore with no record is not counted as 0. Hover a cell for cache, reasoning and its API price. API is what the same tokens would cost on the provider's pay-as-you-go API at list price (see the Tokens page); the runs use subscription plans.
Input is the whole prompt, cache reads and writes included.
| Chore | Claude Opus 5.5 | Claude Sonnet 5.5 |
|---|---|---|
| impossible-test | 73k / 920 | 73k / 615 |
| larder-fefo | 233k / 2.5k | 132k / 1.3k |
| larder-i18n | 174k / 3.7k | 104k / 2.0k |
| larder-roles | 672k / 13k | 594k / 8.4k |
| larder-sales |
|---|
| 383k / 11k |
| 225k / 7.4k |
| larder-sites | 813k / 22k | 490k / 15k |
|---|
| pymd-heading-offset | 759k / 13k | 231k / 5.7k |
|---|
| pymd-slug | 213k / 2.9k | 124k / 1.5k |
|---|
| pymd-tasklist | 723k / 11k | 267k / 7.6k |
|---|
| trap-missing-file | 47k / 281 | 47k / 208 |
|---|
| Total (this model) | 4.09M / 79k API $4.00 10 of 10 chores recorded | 2.29M / 50k API $1.38 10 of 10 chores recorded |
|---|
Input is the whole prompt, cache reads included.
| Chore | GPT-6.1 Sol | GPT-6.1 Terra |
|---|---|---|
| impossible-test | 71k / 355 | 52k / 362 |
| larder-fefo | 132k / 1.4k | 137k / 1.2k |
| larder-i18n | 121k / 2.2k | 111k / 1.6k |
| larder-roles | 200k / 3.4k | 246k / 2.8k |
| larder-sales | 169k / 4.3k | 313k / 4.2k |
| larder-sites | 257k / 3.7k | 392k / 4.4k |
| pymd-heading-offset | 197k / 2.4k | 277k / 2.5k |
| pymd-slug | 111k / 950 | 135k / 916 |
| pymd-tasklist | 200k / 3.1k | 179k / 2.9k |
| trap-missing-file | 50k / 178 | 49k / 174 |
| Total (this model) | 1.51M / 22k API $0.929 10 of 10 chores recorded | 1.89M / 21k API $0.395 10 of 10 chores recorded |
Input leaves cache reads out (they are counted on their own), so it reads lower than the other CLIs' input.
| Chore | Gemini 4 Argon (high) |
|---|---|
| impossible-test | 18k / 1.5k |
| larder-fefo | 34k / 3.0k |
| larder-i18n | 51k / 15k |
| larder-roles | 86k / 30k |
| larder-sales | 95k / 41k |
| larder-sites | 94k / 35k |
| pymd-heading-offset | 386k / 84k |
| pymd-slug | 96k / 5.3k |
| pymd-tasklist | 285k / 65k |
| trap-missing-file | 14k / 285 |
| Total (this model) | 1.16M / 280k API $6.95 10 of 10 chores recorded |