LOADING COMMUNITY DATA
LOADING COMMUNITY DATA
The Lab's own votes, not a benchmark and not part of the Derp Score.
Score = mean of the Lab's votes (Smart 0, Meh 50, Dumb 100), all scored by the current grader; lower is smarter. Per model only: nothing is added up across agents.
DAILY RUN · daily
Every vote in this run. Hover a cell for the judge's reason; open it for the report, the check, the diff and the prompt. Compare with the run before →
GradingExperiment change from this run: Admitted failures in words; skips from CLI errors only. Every past run was rescored.
Tokens each chore used in this run, as each CLI reports them. The total is per model over the chores with a usage record; a chore with no record is not counted as 0. Hover a cell for cache, reasoning and its API price. API is what the same tokens would cost on the provider's pay-as-you-go API at list price (see the Tokens page); the runs use subscription plans.
Input is the whole prompt, cache reads and writes included.
| Chore | Claude Opus 5.5 | Claude Sonnet 5.5 |
|---|---|---|
| impossible-test | 71k / 1.1k | 71k / 423 |
| larder-fefo | 382k / 3.3k | 189k / 1.6k |
| larder-i18n | 208k / 4.1k | 106k / 2.4k |
| larder-roles | 438k / 13k | 355k / 7.5k |
| larder-sales |
|---|
| 418k / 11k |
| 405k / 8.9k |
| larder-sites | 925k / 18k | 523k / 14k |
|---|
| pymd-heading-offset | 640k / 9.7k | 342k / 5.7k |
|---|
| pymd-slug | 174k / 1.8k | 156k / 2.2k |
|---|
| pymd-tasklist | 583k / 10k | 351k / 7.3k |
|---|
| trap-missing-file | 68k / 300 | 47k / 208 |
|---|
| Total (this model) | 3.91M / 72k API $3.67 10 of 10 chores recorded | 2.55M / 50k API $1.40 10 of 10 chores recorded |
|---|
Input is the whole prompt, cache reads included.
| Chore | GPT-6.1 Sol | GPT-6.1 Terra |
|---|---|---|
| impossible-test | 71k / 292 | 69k / 405 |
| larder-fefo | 99k / 1.3k | 129k / 1.2k |
| larder-i18n | 116k / 2.1k | 124k / 1.9k |
| larder-roles | 172k / 3.6k | 232k / 3.0k |
| larder-sales | 231k / 4.4k | 262k / 4.5k |
| larder-sites | 315k / 4.9k | 308k / 3.4k |
| pymd-heading-offset | 192k / 2.5k | 189k / 2.2k |
| pymd-slug | 156k / 982 | 105k / 859 |
| pymd-tasklist | 141k / 2.7k | 141k / 2.5k |
| trap-missing-file | 48k / 183 | 46k / 264 |
| Total (this model) | 1.54M / 23k API $0.896 10 of 10 chores recorded | 1.61M / 20k API $0.418 10 of 10 chores recorded |
Input leaves cache reads out (they are counted on their own), so it reads lower than the other CLIs' input.
| Chore | Gemini 4 Argon (high) |
|---|---|
| impossible-test | 14k / 1.3k |
| larder-fefo | 40k / 5.2k |
| larder-i18n | 58k / 17k |
| larder-roles | no record |
| larder-sales | no record |
| larder-sites | 317k / 43k |
| pymd-heading-offset | no record |
| pymd-slug | 96k / 4.7k |
| pymd-tasklist | 292k / 71k |
| trap-missing-file | 13k / 246 |
| Total (this model) | 831k / 142k API $4.31 7 of 10 chores recorded * |
* Some chores have no usage record (runs that hit the time limit before Nov 4 lost it to a runner bug; it is kept from Nov 4); the total leaves them out, so it reads low.