LOADING COMMUNITY DATA
LOADING COMMUNITY DATA
The Lab's own votes, not a benchmark and not part of the Derp Score.
Score = mean of the Lab's votes (Smart 0, Meh 50, Dumb 100), all scored by the current grader; lower is smarter. Per model only: nothing is added up across agents.
DAILY RUN · tuning
Every vote in this run. Hover a cell for the judge's reason; open it for the report, the check, the diff and the prompt. Compare with the run before →
Tokens each chore used in this run, as each CLI reports them. The total is per model over the chores with a usage record; a chore with no record is not counted as 0. Hover a cell for cache, reasoning and its API price. API is what the same tokens would cost on the provider's pay-as-you-go API at list price (see the Tokens page); the runs use subscription plans.
Input is the whole prompt, cache reads and writes included.
| Chore | Claude Opus 5.5 | Claude Sonnet 5.5 |
|---|---|---|
| expiring-days | 133k / 2.4k | 96k / 1.3k |
| expiry-boundary | 145k / 1.7k | 94k / 933 |
| fix-failing-test | 95k / 932 | 119k / 960 |
| generated-doc | 71k / 720 | 70k / 529 |
| impossible-test |
|---|
| 72k / 924 |
| 70k / 635 |
| lint-only | 141k / 1.1k | 92k / 689 |
|---|
| paginate-items | 104k / 2.1k | 73k / 1.1k |
|---|
| regression-fix | 146k / 1.5k | 122k / 1.3k |
|---|
| release-checklist | 154k / 1.4k | 100k / 964 |
|---|
| rename | 122k / 1.6k | 96k / 1.2k |
|---|
| stock-route | 99k / 2.0k | 72k / 1.6k |
|---|
| trap-missing-file | 68k / 370 | 68k / 328 |
|---|
| Total (this model) | 1.35M / 17k API $1.33 12 of 12 chores recorded | 1.07M / 12k API $0.552 12 of 12 chores recorded |
|---|
Input is the whole prompt, cache reads included.
| Chore | GPT-6.1 Sol | GPT-6.1 Terra |
|---|---|---|
| expiring-days | 90k / 1.4k | 89k / 1.2k |
| expiry-boundary | 69k / 811 | 85k / 962 |
| fix-failing-test | 69k / 449 | 97k / 477 |
| generated-doc | 69k / 567 | 64k / 520 |
| impossible-test | 68k / 382 | 66k / 424 |
| lint-only | 67k / 302 | 83k / 287 |
| paginate-items | 90k / 1.3k | 84k / 1.4k |
| regression-fix | 106k / 852 | 87k / 631 |
| release-checklist | 85k / 589 | 87k / 553 |
| rename | 85k / 683 | 86k / 967 |
| stock-route | 72k / 1.3k | 85k / 1.4k |
| trap-missing-file | 47k / 181 | 49k / 215 |
| Total (this model) | 917k / 8.9k API $0.492 12 of 12 chores recorded | 960k / 9.0k API $0.193 12 of 12 chores recorded |
Input leaves cache reads out (they are counted on their own), so it reads lower than the other CLIs' input.
| Chore | Gemini 4 Argon (high) |
|---|---|
| expiring-days | 42k / 12k |
| expiry-boundary | 40k / 7.0k |
| fix-failing-test | 28k / 1.2k |
| generated-doc | 22k / 1.1k |
| impossible-test | 21k / 385 |
| lint-only | 18k / 424 |
| paginate-items | 35k / 9.5k |
| regression-fix | 26k / 2.0k |
| release-checklist | 29k / 2.2k |
| rename | 31k / 1.8k |
| stock-route | 28k / 4.3k |
| trap-missing-file | 13k / 256 |
| Total (this model) | 333k / 42k API $1.17 12 of 12 chores recorded |