LOADING COMMUNITY DATA
LOADING COMMUNITY DATA
The Lab's own votes, not a benchmark and not part of the Derp Score.
Score = mean of the Lab's votes (Smart 0, Meh 50, Dumb 100), all scored by the current grader; lower is smarter. Per model only: nothing is added up across agents.
DAILY RUN · TOKENS
How many tokens each model spends per chore in the daily runs, as each CLI reports them. One run's tokens, chore by chore, are on that run's page.
Mean tokens per chore in each daily run, per model, as each CLI reports them, on today's chores only, then each chore on its own. Hover a chart (or focus it and use the arrow keys) for the numbers. The change is against the run before, over the chores both runs recorded. A run with no usage record is not counted as 0: it stays out, and the page says how many.
API $ is what the same tokens would cost on each provider's pay-as-you-go API at its list price (checked 2026-11-18): Anthropic, OpenAI, Google. The runs themselves use subscription plans, so this is a scale, not a bill: cache reads at the cache price, reasoning at the output price, Claude cache writes at the 5-minute price, OpenAI at the short-context price, and every run at the price of its own day, so a price change never reprices past runs. Chores with no usage record or no known price are left out, so totals read low there. Per agent and model only.
Input leaves cache reads out (they are counted on their own), so it reads lower than the other CLIs' input.
| Model | Input per chore | Output per chore | API $, newest run | API $, 14 days | Newest run |
|---|---|---|---|---|---|
| Gemini 4 Carbon (high) | 224k ≈ -4% · 8 chores | 57k ▲ +12% · 8 chores | $13.93 $1.55 per chore | $48.51 49 chore runs | 11-17 18:30 9 of 9 chores recorded |
| Gemini 4 Argon (high) | 327k ≈ +4% · 8 chores | 80k ▼ -17% · 8 chores | $21.89 $1.99 per chore | $88.59 106 chore runs | 11-17 18:30 11 of 11 chores recorded |
| Gemini 4 Hydrogen (high) | 69k ≈ -2% · 5 chores |
| 17k ▼ -8% · 5 chores |
| $1.39 $0.278 per chore |
| $10.63 37 chore runs |
| 11-16 18:30 5 of 6 chores recorded * |
* Gemini 4 Argon: 1 of 107 chore runs in this window have no usage record (runs that hit the time limit before Nov 4 lost it to a runner bug; it is kept from Nov 4); they are left out (hollow dots). Those took 1215s on average against 521s, so the means likely read low.
* Gemini 4 Hydrogen: 1 of 38 chore runs in this window have no usage record (runs that hit the time limit before Nov 4 lost it to a runner bug; it is kept from Nov 4); they are left out (hollow dots). Those took 1218s on average against 163s, so the means likely read low.
Input tokens per chore, mean of each daily run
Input tokens by chore · each chart has its own scale
| Run (UTC) | Gemini 4 Carbon | Gemini 4 Argon | Gemini 4 Hydrogen |
|---|---|---|---|
| 11-17 18:30 | 224k / 57k | 327k / 80k | — |
| 11-16 18:30 | 140k / 28k | 121k / 24k | 69k / 17k * |
| 11-15 18:30 | 173k / 33k | 103k / 17k | 75k / 18k |
| 11-14 18:30 | 152k / 33k | 125k / 26k | 79k / 17k |
| 11-13 18:30 | 156k / 29k | 120k / 29k | 70k / 16k |
| 11-12 18:30 | 186k / 34k | 147k / 32k | 69k / 17k |
| 11-11 18:30 | — | 121k / 23k | — |
| 11-10 18:30 | — | 122k / 25k * | — |
| 11-09 18:30 | — | 133k / 29k | — |
| 11-08 18:30 | — | 141k / 35k | — |
| 11-07 18:30 | — | 112k / 24k | — |
| 11-06 18:30 | — | 143k / 33k | — |
| 11-05 18:30 | — | 129k / 23k | — |
| Chore | Gemini 4 Carbon | Gemini 4 Argon | Gemini 4 Hydrogen |
|---|---|---|---|
| larder-fefo | 57k → 58k +2% | 44k → 38k -15% | 40k → 42k +7% |
| larder-i18n | 32k → 52k +62% | 48k → 50k +5% | 47k → 38k -19% |
| larder-roles | 63k → 88k +40% | 61k → 68k +12% | 94k → 89k -5% |
| larder-sales | 75k → 77k +3% | 89k → 74k -17% | 75k → 66k -13% |
| larder-sites | 69k → 180k +162% | 119k → 93k -21% | 96k → 107k +12% |
| pymd-heading-offset | 411k → 269k -35% | 248k → 197k -21% | 103k → no record |
| pymd-slug | 155k → 170k +9% | 84k → 78k -7% | 54k → no record |
| pymd-tasklist | 259k → 183k -29% | 275k → 413k +50% | 92k → no record |
| toon-episode-copy | no record → 941k | no record → 666k | no record → no record |
| toon-failed-charges | no record → no record | no record → 1.12M | no record → no record |
| toon-image-fetch | no record → no record | no record → 796k | no record → no record |