The Lab's own votes, not a benchmark and not part of the Derp Score.

Score = mean of the Lab's votes (Smart 0, Meh 50, Dumb 100), all scored by the current grader; lower is smarter. Per model only: nothing is added up across agents.

LOADING COMMUNITY DATA

How derpy is your AI today?

DAILY RUN · tuning

2026-10-28 06:58 UTC

Every vote in this run. Hover a cell for the judge's reason; open it for the report, the check, the diff and the prompt. Compare with the run before →

results/2026-10-28-0658derpmeter-daily-run on GitHub

Scores use the current grader. 0 votes were scored again from the raw results. 15 votes had older inputs or no raw result: they keep their original score (dashed) and are not compared with other runs.

Claude Opus 5.5

16.7Smart

S 2 · M 1 · D 0

Claude Sonnet 5.5

33.3Smart

S 2 · M 0 · D 1

GPT-6.1 Sol

50.0Meh

S 0 · M 3 · D 0

GPT-6.1 Terra

16.7Smart

S 2 · M 1 · D 0

Gemini 4 Argonmedium

83.3Dumb

S 0 · M 1 · D 2

Quota used in this run

Percent points of each plan's 5-hour and weekly windows. A model counts only chores measured while no other session of that CLI was working; the run total is first reading to last, on the owner's own accounts. Claude Code and Codex report whole percents.

Claude CodeMax

Model	5-hour	Weekly	Measured
Claude Opus 5.5	+2.0%p	+0.0%p	3/3
Claude Sonnet 5.5	+1.0%p	+0.0%p	3/3
Whole run	+3.0%p	+0.0%p
Used after the run	31%	83%

OpenAI CodexPlus

Model	5-hour	Weekly	Measured
GPT-6.1 Sol	+4.0%p	+1.0%p	3/3
GPT-6.1 Terra	+3.0%p	+0.0%p	3/3
Whole run	+7.0%p	+1.0%p
Used after the run	7%	1%

AntigravityPlan not reported

Model	5-hour	Weekly	Measured
Gemini 4 Argonmedium	—

Chores × models

S Smart · M Meh · D Dumb · – skipped

Chore	Claude Code		OpenAI Codex		Antigravity
Chore	Claude Opus 5.5	Claude Sonnet 5.5	GPT-6.1 Sol	GPT-6.1 Terra	Gemini 4 Argon
Python-Markdown
pymd-heading-offset	S	S	MPoor output	S

Token use in this run

input / output per chore

Tokens each chore used in this run, as each CLI reports them. The total is per model over the chores with a usage record; a chore with no record is not counted as 0. Hover a cell for cache, reasoning and its API price. API is what the same tokens would cost on the provider's pay-as-you-go API at list price (see the Tokens page); the runs use subscription plans.

Claude Code

Input is the whole prompt, cache reads and writes included.

Chore	Claude Opus 5.5	Claude Sonnet 5.5
pymd-heading-offset	820k / 9.6k	363k / 5.9k
pymd-slug	201k / 2.4k	120k / 1.4k
pymd-tasklist	699k / 11k	429k / 8.1k
Total (this model)	1.72M / 23k API $1.31 3 of 3 chores recorded	913k / 15kAPI $0.396

What was not Smart

D
Claude Sonnet 5.5 · pymd-tasklist · Hallucination · 103s
claimed success: failed: suite
raw
D
Gemini 4 Argon · pymd-heading-offset · Ignored instructions · 1205s
no report: failed: shift
raw
D
Gemini 4 Argon · pymd-tasklist · Ignored instructions · 1208s
no report: failed: basic, notTasks
raw
M
Claude Opus 5.5 · pymd-tasklist · Poor output · 104s
partly done: missed lineLength
raw
M
GPT-6.1 Sol · pymd-heading-offset · Poor output · 77s
partly done: missed changelog
raw
M
GPT-6.1 Sol · pymd-slug · Poor output · 53s
also changed docs/extensions/toc.md
raw

Whole run	+11.2%p	+2.5%p
Used after the run	19%	9%

pymd-slug	S	S	MPoor output	MPoor output	MToo slow

pymd-tasklist	MPoor output	DHallucination	MPoor output	S	DIgnored instructions

OpenAI Codex

Input is the whole prompt, cache reads included.

Chore	GPT-6.1 Sol	GPT-6.1 Terra
pymd-heading-offset	265k / 2.5k	159k / 2.2k
pymd-slug	175k / 1.0k	114k / 1.1k
pymd-tasklist	193k / 3.4k	178k / 2.8k
Total (this model)	633k / 7.0k API $0.364 3 of 3 chores recorded	451k / 6.0k API $0.166 3 of 3 chores recorded

Antigravity

Input leaves cache reads out (they are counted on their own), so it reads lower than the other CLIs' input.

Chore	Gemini 4 Argon (medium)
pymd-heading-offset	no record
pymd-slug	233k / 5.9k
pymd-tasklist	no record
Total (this model)	233k / 5.9k API $0.648 1 of 3 chores recorded *

* Some chores have no usage record (runs that hit the time limit before Nov 4 lost it to a runner bug; it is kept from Nov 4); the total leaves them out, so it reads low.