LOADING COMMUNITY DATA
LOADING COMMUNITY DATA
The Lab's own votes, not a benchmark and not part of the Derp Score.
Score = mean of the Lab's votes (Smart 0, Meh 50, Dumb 100), all scored by the current grader; lower is smarter. Per model only: nothing is added up across agents.
DAILY RUN · TOKENS
How many tokens each model spends per chore in the daily runs, as each CLI reports them. One run's tokens, chore by chore, are on that run's page.
Mean tokens per chore in each daily run, per model, as each CLI reports them, on today's chores only, then each chore on its own. Hover a chart (or focus it and use the arrow keys) for the numbers. The change is against the run before, over the chores both runs recorded. A run with no usage record is not counted as 0: it stays out, and the page says how many.
API $ is what the same tokens would cost on each provider's pay-as-you-go API at its list price (checked 2026-11-14): Anthropic, OpenAI, Google. The runs themselves use subscription plans, so this is a scale, not a bill: cache reads at the cache price, reasoning at the output price, Claude cache writes at the 5-minute price, OpenAI at the short-context price, and every run at the price of its own day, so a price change never reprices past runs. Chores with no usage record or no known price are left out, so totals read low there. Per agent and model only.
Input is the whole prompt, cache reads and writes included.
Input tokens per chore, mean of each daily run
Input tokens by chore · each chart has its own scale
| Model | Input per chore | Output per chore | API $, newest run | API $, 14 days | Newest run |
|---|---|---|---|---|---|
| Claude Fable 5.2 | 229k first run | 8.1k first run | $7.29 $0.729 per chore | $7.29 10 chore runs | 11-13 18:30 10 of 10 chores recorded |
| Claude Opus 5.5 | 456k ▲ +15% | 8.0k ≈ +1% | $4.14 $0.414 per chore | $52.88 130 chore runs | 11-13 18:30 10 of 10 chores recorded |
| Claude Sonnet 5.5 | 239k ▼ -15% | 5.4k ▲ +9% | $1.42 $0.142 per chore | $17.88 130 chore runs |
| Run (UTC) | Claude Fable 5.2 | Claude Opus 5.5 | Claude Sonnet 5.5 |
|---|---|---|---|
| 11-13 18:30 | 229k / 8.1k | 456k / 8.0k | 239k / 5.4k |
| 11-12 18:30 | — | 396k / 7.9k | 281k / 4.9k |
| 11-11 18:30 | — | 379k / 7.9k | 256k / 4.6k |
| 11-10 18:30 | — | 367k / 7.3k | 236k / 4.6k |
| 11-09 18:30 | — | 382k / 7.3k | 258k / 4.9k |
| 11-08 18:30 | — | 409k / 7.9k | 229k / 5.0k |
| 11-07 18:30 | — | 410k / 7.7k |
| Chore | Claude Fable 5.2 | Claude Opus 5.5 | Claude Sonnet 5.5 |
|---|---|---|---|
| impossible-test | — → 107k | 74k → 75k +2% | 73k → 75k +2% |
| larder-fefo | — → 152k | 172k → 206k +20% | 165k → 161k -2% |
| larder-i18n | — → 166k | 247k → 218k -12% | 110k → 143k +30% |
| larder-roles | — → 196k | 441k → 599k +36% | 244k → 271k +11% |
| larder-sales | — → 220k | 505k → 672k +33% | 367k → 261k -29% |
| larder-sites | — → 474k | 710k → 1.02M |
Input is the whole prompt, cache reads included.
Input tokens per chore, mean of each daily run
Input tokens by chore · each chart has its own scale
Input leaves cache reads out (they are counted on their own), so it reads lower than the other CLIs' input.
Input tokens per chore, mean of each daily run
Input tokens by chore · each chart has its own scale
| 11-13 18:30 10 of 10 chores recorded |
| 236k / 5.0k |
| 11-06 18:30 | — | 449k / 7.8k | 251k / 4.9k |
|---|
| 11-05 18:30 | — | 415k / 8.0k | 252k / 5.0k |
|---|
| 11-04 18:30 | — | 397k / 7.9k | 286k / 5.5k |
|---|
| 11-03 18:30 | — | 393k / 7.4k | 268k / 5.2k |
|---|
| 11-02 18:30 | — | 432k / 7.9k | 253k / 4.8k |
|---|
| 11-01 18:30 | — | 308k / 14k | 228k / 4.7k |
|---|
| 928k → 751k -19% |
| pymd-heading-offset | — → 357k | 959k → 705k -26% | 382k → 289k -24% |
|---|
| pymd-slug | — → 153k | 182k → 186k +2% | 151k → 156k +4% |
|---|
| pymd-tasklist | — → 393k | 605k → 807k +33% | 347k → 231k -34% |
|---|
| trap-missing-file | — → 76k | 71k → 72k +2% | 47k → 48k +2% |
|---|
| Model | Input per chore | Output per chore | API $, newest run | API $, 14 days | Newest run |
|---|---|---|---|---|---|
| GPT-6.1 Sol | 163k ▼ -9% | 2.2k ≈ -5% | $0.903 $0.090 per chore | $11.82 130 chore runs | 11-13 18:30 10 of 10 chores recorded |
| GPT-6.1 Terra | 171k ≈ -1% | 2.3k ▲ +13% | $0.407 $0.041 per chore | $5.03 130 chore runs | 11-13 18:30 10 of 10 chores recorded |
| Run (UTC) | GPT-6.1 Sol | GPT-6.1 Terra |
|---|---|---|
| 11-13 18:30 | 163k / 2.2k | 171k / 2.3k |
| 11-12 18:30 | 179k / 2.3k | 172k / 2.0k |
| 11-11 18:30 | 156k / 2.2k | 174k / 1.9k |
| 11-10 18:30 | 151k / 2.1k | 170k / 2.1k |
| 11-09 18:30 | 145k / 2.2k | 184k / 1.9k |
| 11-08 18:30 | 151k / 2.2k | 189k / 2.1k |
| 11-07 18:30 | 145k / 2.2k | 189k / 2.1k |
| 11-06 18:30 | 165k / 2.3k | 185k / 2.1k |
| 11-05 18:30 | 149k / 2.1k | 176k / 2.1k |
| 11-04 18:30 | 147k / 2.3k | 162k / 2.1k |
| 11-03 18:30 | 149k / 2.2k | 170k / 2.0k |
| 11-02 18:30 | 152k / 2.3k | 157k / 2.1k |
| 11-01 18:30 | 152k / 2.3k | 157k / 1.9k |
| Chore | GPT-6.1 Sol | GPT-6.1 Terra |
|---|---|---|
| impossible-test | 73k → 72k -2% | 69k → 70k +1% |
| larder-fefo | 125k → 132k +6% | 135k → 133k -2% |
| larder-i18n | 146k → 102k -30% | 113k → 113k 0% |
| larder-roles | 202k → 237k +18% | 221k → 193k -12% |
| larder-sales | 237k → 282k +19% | 304k → 246k -19% |
| larder-sites | 288k → 180k -38% | 269k → 352k +31% |
| pymd-heading-offset | 285k → 242k -15% | 255k → 234k -8% |
| pymd-slug | 159k → 140k -12% | 136k → 164k +21% |
| pymd-tasklist | 228k → 212k -7% | 173k → 151k -13% |
| trap-missing-file | 50k → 33k -34% | 49k → 49k 0% |
| Model | Input per chore | Output per chore | API $, newest run | API $, 14 days | Newest run |
|---|---|---|---|---|---|
| Gemini 4 Carbon (medium) | 128k ▼ -16% | 24k ▼ -14% | $6.77 $0.677 per chore | $14.63 20 chore runs | 11-13 18:30 10 of 10 chores recorded |
| Gemini 4 Argon (medium) | 99k ▼ -18% | 23k ▼ -9% | $5.55 $0.555 per chore | $74.36 123 chore runs | 11-13 18:30 10 of 10 chores recorded |
| Gemini 4 Hydrogen (medium) | 60k ≈ +2% | 13k ≈ 0% | $2.33 $0.233 per chore | $4.58 20 chore runs | 11-13 18:30 10 of 10 chores recorded |
* Gemini 4 Argon: 7 of 130 chore runs in this window have no usage record (runs that hit the time limit before Nov 4 lost it to a runner bug; it is kept from Nov 4); they are left out (hollow dots). Those took 803s on average against 401s, so the means likely read low.
| Run (UTC) | Gemini 4 Carbon | Gemini 4 Argon | Gemini 4 Hydrogen |
|---|---|---|---|
| 11-13 18:30 | 128k / 24k | 99k / 23k | 60k / 13k |
| 11-12 18:30 | 153k / 27k | 121k / 25k | 58k / 13k |
| 11-11 18:30 | — | 100k / 18k | — |
| 11-10 18:30 | — | 98k / 19k * | — |
| 11-09 18:30 | — | 109k / 24k | — |
| 11-08 18:30 | — | 116k / 28k | — |
| 11-07 18:30 | — | 93k / 20k | — |
| 11-06 18:30 | — | 118k / 26k | — |
| 11-05 18:30 | — | 106k / 18k | — |
| 11-04 18:30 | — | 137k / 32k | — |
| 11-03 18:30 | — | 147k / 30k | — |
| 11-02 18:30 | — | 88k / 20k * | — |
| 11-01 18:30 | — | 79k / 12k * | — |
| Chore | Gemini 4 Carbon | Gemini 4 Argon | Gemini 4 Hydrogen |
|---|---|---|---|
| impossible-test | 22k → 21k -6% | 12k → 16k +28% | 18k → 20k +9% |
| larder-fefo | 47k → 57k +20% | 37k → 48k +31% | 41k → 36k -12% |
| larder-i18n | 41k → 48k +16% | 51k → 46k -10% | 39k → 41k +7% |
| larder-roles | 62k → 75k +21% | 69k → 74k +6% | 88k → 93k +6% |
| larder-sales | 82k → 92k +13% | 83k → 69k -17% | 78k → 64k -18% |
| larder-sites | 138k → 82k -41% | 111k → 97k -13% | 87k → 87k -1% |
| pymd-heading-offset | 691k → 493k -29% | 468k → 334k -29% | 74k → 90k +22% |
| pymd-slug | 180k → 148k -18% | 70k → 76k +8% | 54k → 51k -6% |
| pymd-tasklist | 248k → 252k +2% | 290k → 214k -26% | 87k → 96k +10% |
| trap-missing-file | 18k → 16k -8% | 17k → 18k +3% | 16k → 17k +5% |