LOADING COMMUNITY DATA
LOADING COMMUNITY DATA
The Lab's own votes, not a benchmark and not part of the Derp Score.
Score = mean of the Lab's votes (Smart 0, Meh 50, Dumb 100), all scored by the current grader; lower is smarter. Per model only: nothing is added up across agents.
DAILY RUN · tuning
Every vote in this run. Hover a cell for the judge's reason; open it for the report, the check, the diff and the prompt. Compare with the run before →
Tokens each chore used in this run, as each CLI reports them. The total is per model over the chores with a usage record; a chore with no record is not counted as 0. Hover a cell for cache, reasoning and its API price. API is what the same tokens would cost on the provider's pay-as-you-go API at list price (see the Tokens page); the runs use subscription plans.
Input is the whole prompt, cache reads and writes included.
| Chore | Claude Opus 5.5 | Claude Sonnet 5.5 |
|---|---|---|
| pymd-heading-offset | 820k / 9.6k | 363k / 5.9k |
| pymd-slug | 201k / 2.4k | 120k / 1.4k |
| pymd-tasklist | 699k / 11k | 429k / 8.1k |
| Total (this model) | 1.72M / 23k API $1.31 3 of 3 chores recorded | 913k / 15kAPI $0.396 |
Input is the whole prompt, cache reads included.
| Chore | GPT-6.1 Sol | GPT-6.1 Terra |
|---|---|---|
| pymd-heading-offset | 265k / 2.5k | 159k / 2.2k |
| pymd-slug | 175k / 1.0k | 114k / 1.1k |
| pymd-tasklist | 193k / 3.4k | 178k / 2.8k |
| Total (this model) | 633k / 7.0k API $0.364 3 of 3 chores recorded | 451k / 6.0k API $0.166 3 of 3 chores recorded |
Input leaves cache reads out (they are counted on their own), so it reads lower than the other CLIs' input.
| Chore | Gemini 4 Argon (high) |
|---|---|
| pymd-heading-offset | no record |
| pymd-slug | 233k / 5.9k |
| pymd-tasklist | no record |
| Total (this model) | 233k / 5.9k API $0.648 1 of 3 chores recorded * |
* Some chores have no usage record (runs that hit the time limit before Nov 4 lost it to a runner bug; it is kept from Nov 4); the total leaves them out, so it reads low.