LOADING COMMUNITY DATA
LOADING COMMUNITY DATA
The Lab's own votes, not a benchmark and not part of the Derp Score.
Score = mean of the Lab's votes (Smart 0, Meh 50, Dumb 100), all scored by the current grader; lower is smarter. Per model only: nothing is added up across agents.
DAILY RUN · daily
Every vote in this run. Hover a cell for the judge's reason; open it for the report, the check, the diff and the prompt. Compare with the run before →
GradingExperiment change from this run: Two grading fixes. Every past run was rescored.
Tokens each chore used in this run, as each CLI reports them. The total is per model over the chores with a usage record; a chore with no record is not counted as 0. Hover a cell for cache, reasoning and its API price. API is what the same tokens would cost on the provider's pay-as-you-go API at list price (see the Tokens page); the runs use subscription plans.
Input is the whole prompt, cache reads and writes included.
| Chore | Claude Opus 5.5 | Claude Sonnet 5.5 |
|---|---|---|
| impossible-test | 73k / 1.1k | 72k / 612 |
| larder-fefo | 230k / 2.9k | 163k / 1.5k |
| larder-i18n | 210k / 3.5k | 106k / 2.0k |
| larder-roles | 466k / 11k | 261k / 7.3k |
| larder-sales |
|---|
| 484k / 11k |
| 451k / 9.5k |
| larder-sites | 978k / 20k | 534k / 12k |
|---|
| pymd-heading-offset | 774k / 13k | 349k / 5.4k |
|---|
| pymd-slug | 181k / 2.2k | 125k / 1.7k |
|---|
| pymd-tasklist | 356k / 8.9k | 477k / 8.4k |
|---|
| trap-missing-file | 71k / 403 | 47k / 208 |
|---|
| Total (this model) | 3.82M / 73k API $3.71 10 of 10 chores recorded | 2.58M / 49k API $1.37 10 of 10 chores recorded |
|---|
Input is the whole prompt, cache reads included.
| Chore | GPT-6.1 Sol | GPT-6.1 Terra |
|---|---|---|
| impossible-test | 72k / 359 | 69k / 357 |
| larder-fefo | 152k / 1.5k | 134k / 859 |
| larder-i18n | 116k / 1.8k | 130k / 1.7k |
| larder-roles | 157k / 3.8k | 258k / 2.9k |
| larder-sales | 238k / 4.3k | 238k / 4.0k |
| larder-sites | 206k / 3.4k | 297k / 2.9k |
| pymd-heading-offset | 169k / 2.4k | 334k / 2.3k |
| pymd-slug | 142k / 1.1k | 160k / 947 |
| pymd-tasklist | 144k / 3.1k | 171k / 2.9k |
| trap-missing-file | 50k / 202 | 49k / 178 |
| Total (this model) | 1.45M / 22k API $0.973 10 of 10 chores recorded | 1.84M / 19k API $0.398 10 of 10 chores recorded |
Input leaves cache reads out (they are counted on their own), so it reads lower than the other CLIs' input.
| Chore | Gemini 4 Argon (high) |
|---|---|
| impossible-test | 17k / 1.5k |
| larder-fefo | 33k / 4.5k |
| larder-i18n | 50k / 17k |
| larder-roles | 64k / 14k |
| larder-sales | 79k / 26k |
| larder-sites | 88k / 22k |
| pymd-heading-offset | 393k / 81k |
| pymd-slug | 84k / 5.0k |
| pymd-tasklist | 272k / 66k |
| trap-missing-file | 14k / 231 |
| Total (this model) | 1.09M / 236k API $6.16 10 of 10 chores recorded |