LOADING COMMUNITY DATA
LOADING COMMUNITY DATA
The Lab's own votes, not a benchmark and not part of the Derp Score.
Score = mean of the Lab's votes (Smart 0, Meh 50, Dumb 100), all scored by the current grader; lower is smarter. Per model only: nothing is added up across agents.
DAILY RUN · daily
Every vote in this run. Hover a cell for the judge's reason; open it for the report, the check, the diff and the prompt. Compare with the run before →
Tokens each chore used in this run, as each CLI reports them. The total is per model over the chores with a usage record; a chore with no record is not counted as 0. Hover a cell for cache, reasoning and its API price. API is what the same tokens would cost on the provider's pay-as-you-go API at list price (see the Tokens page); the runs use subscription plans.
Input is the whole prompt, cache reads and writes included.
| Chore | Claude Opus 5.5 | Claude Sonnet 5.5 |
|---|---|---|
| impossible-test | 73k / 1.1k | 48k / 407 |
| larder-fefo | 166k / 2.1k | 135k / 1.3k |
| larder-i18n | 243k / 3.9k | 140k / 2.1k |
| larder-roles | 582k / 12k | 367k / 6.9k |
| larder-sales |
|---|
| 476k / 11k |
| 348k / 7.0k |
| larder-sites | 1.14M / 22k | 697k / 16k |
|---|
| pymd-heading-offset | 585k / 11k | 284k / 5.7k |
|---|
| pymd-slug | 181k / 2.3k | 151k / 1.8k |
|---|
| pymd-tasklist | 971k / 13k | 293k / 8.2k |
|---|
| trap-missing-file | 70k / 299 | 47k / 207 |
|---|
| Total (this model) | 4.49M / 78k API $4.04 10 of 10 chores recorded | 2.51M / 49k API $1.35 10 of 10 chores recorded |
|---|
Input is the whole prompt, cache reads included.
| Chore | GPT-6.1 Sol | GPT-6.1 Terra |
|---|---|---|
| impossible-test | 73k / 368 | 70k / 412 |
| larder-fefo | 159k / 1.3k | 135k / 1.6k |
| larder-i18n | 119k / 1.9k | 116k / 1.6k |
| larder-roles | 239k / 3.8k | 215k / 2.9k |
| larder-sales | 237k / 4.2k | 334k / 4.2k |
| larder-sites | 236k / 4.2k | 335k / 3.6k |
| pymd-heading-offset | 176k / 2.6k | 261k / 2.5k |
| pymd-slug | 163k / 1.0k | 154k / 985 |
| pymd-tasklist | 199k / 3.1k | 182k / 2.6k |
| trap-missing-file | 51k / 226 | 49k / 177 |
| Total (this model) | 1.65M / 23k API $0.903 10 of 10 chores recorded | 1.85M / 21k API $0.362 10 of 10 chores recorded |
Input leaves cache reads out (they are counted on their own), so it reads lower than the other CLIs' input.
| Chore | Gemini 4 Argon (high) |
|---|---|
| impossible-test | 19k / 2.0k |
| larder-fefo | 41k / 6.4k |
| larder-i18n | 46k / 12k |
| larder-roles | 81k / 28k |
| larder-sales | 89k / 32k |
| larder-sites | 94k / 36k |
| pymd-heading-offset | 350k / 82k |
| pymd-slug | 130k / 6.5k |
| pymd-tasklist | 317k / 57k |
| trap-missing-file | 14k / 240 |
| Total (this model) | 1.18M / 263k API $6.98 10 of 10 chores recorded |