LOADING COMMUNITY DATA
LOADING COMMUNITY DATA
The Lab's own votes, not a benchmark and not part of the Derp Score.
Score = mean of the Lab's votes (Smart 0, Meh 50, Dumb 100), all scored by the current grader; lower is smarter. Per model only: nothing is added up across agents.
DAILY RUN · daily
Every vote in this run. Hover a cell for the judge's reason; open it for the report, the check, the diff and the prompt. Compare with the run before →
GradingExperiment change from this run: One grader for everything compared. Every past run was rescored.
Tokens each chore used in this run, as each CLI reports them. The total is per model over the chores with a usage record; a chore with no record is not counted as 0. Hover a cell for cache, reasoning and its API price. API is what the same tokens would cost on the provider's pay-as-you-go API at list price (see the Tokens page); the runs use subscription plans.
Input is the whole prompt, cache reads and writes included.
| Chore | Claude Opus 5.5 | Claude Sonnet 5.5 |
|---|---|---|
| impossible-test | 71k / 713 | 70k / 687 |
| larder-fefo | 162k / 2.4k | 128k / 1.1k |
| larder-i18n | 176k / 3.3k | 107k / 2.2k |
| larder-roles | 528k / 12k | 236k / 6.8k |
| larder-sales |
|---|
| 483k / 10k |
| 250k / 6.6k |
| larder-sites | 825k / 21k | 799k / 18k |
|---|
| pymd-heading-offset | 846k / 11k | 233k / 5.4k |
|---|
| pymd-slug | 176k / 2.2k | 124k / 1.8k |
|---|
| pymd-tasklist | 594k / 12k | 411k / 11k |
|---|
| trap-missing-file | 68k / 270 | 45k / 206 |
|---|
| Total (this model) | 3.93M / 76k API $3.86 10 of 10 chores recorded | 2.40M / 54k API $1.37 10 of 10 chores recorded |
|---|
Input is the whole prompt, cache reads included.
| Chore | GPT-6.1 Sol | GPT-6.1 Terra |
|---|---|---|
| impossible-test | 70k / 349 | 66k / 375 |
| larder-fefo | 115k / 1.3k | 140k / 1.2k |
| larder-i18n | 115k / 1.8k | 111k / 1.5k |
| larder-roles | 178k / 3.7k | 166k / 2.6k |
| larder-sales | 198k / 4.4k | 181k / 3.6k |
| larder-sites | 162k / 3.3k | 282k / 4.6k |
| pymd-heading-offset | 172k / 2.5k | 151k / 2.0k |
| pymd-slug | 131k / 847 | 121k / 948 |
| pymd-tasklist | 166k / 3.0k | 231k / 3.1k |
| trap-missing-file | 50k / 180 | 49k / 272 |
| Total (this model) | 1.36M / 21k API $0.814 10 of 10 chores recorded | 1.50M / 20k API $0.379 10 of 10 chores recorded |
Input leaves cache reads out (they are counted on their own), so it reads lower than the other CLIs' input.
| Chore | Gemini 4 Argon (high) |
|---|---|
| impossible-test | 14k / 1.2k |
| larder-fefo | 39k / 5.0k |
| larder-i18n | 52k / 16k |
| larder-roles | no record |
| larder-sales | no record |
| larder-sites | 108k / 45k |
| pymd-heading-offset | no record |
| pymd-slug | 185k / 4.6k |
| pymd-tasklist | 300k / 56k |
| trap-missing-file | 13k / 267 |
| Total (this model) | 710k / 127k API $3.93 7 of 10 chores recorded * |
* Some chores have no usage record (runs that hit the time limit before Nov 4 lost it to a runner bug; it is kept from Nov 4); the total leaves them out, so it reads low.