LOADING COMMUNITY DATA
How derpy is your AI today?
LOADING COMMUNITY DATA
The Lab's own votes, not a benchmark and not part of the Derp Score.
Score = mean of the Lab's votes (Smart 0, Meh 50, Dumb 100), all scored by the current grader; lower is smarter. Per model only: nothing is added up across agents.
DAILY RUN · MODELS
Pick a model to see its score, time, tokens and API $ run by run, or add up to three to compare them on one chart. Up is worse on every chart.
Comparing 3 models. × removes one; a name shows that model alone.
| Model | This week | Week before | Votes | Most common reason | Avg time | Tokens in / out | Quota per run (5h · week) |
|---|---|---|---|---|---|---|---|
| Claude Fable 5.2 | 17.0 | — | 50 | Too slow · 15 | 99s | 358k / 9k | 1.50 · 0.14 pt |
| Claude Opus 5.5 | 15.6 | 17.9 | 61 | Too slow · 16 | 99s | 544k / 9k | 0.59 · 0.08 pt |
| Claude Sonnet 5.5 | 6.6 | 2.9 | 61 | Too slow · 7 | 64s | 312k / 6k | 0.38 · 0.04 pt |
| Model | This week | Week before | Votes | Most common reason | Avg time | Tokens in / out | Quota per run (5h · week) |
|---|---|---|---|---|---|---|---|
| GPT-6.1 Sol | 17.2 | 14.3 | 61 | Too slow · 15 | 78s | 263k / 3k | 1.39 · 0.21 pt |
| Measure | GPT-6.1 Terra | Claude Sonnet 5.5 | Gemini 4 Hydrogen |
|---|---|---|---|
| Lab score | 18.2 · mean 12.5▲ +5.7 | 0.0 · mean 4.8≈ 0.0 | 58.3 · mean 37.9▲ +27.1 |
| Time per chore | 100s · mean 70s▲ +45% | 93s · mean 70s▲ +54% | 366s · mean 199s▲ +113% |
| Output tokens per chore | 5.9k · mean 2.8k▲ +108% | 9.1k · mean 6.3k▲ +55% | 17k · mean 17k≈ -1% |
| Reasoning tokens per chore | 308 · mean 110▲ +173% | 1.4k · mean 1.2k▲ +11% | 8.1k · mean 8.6k▼ -13% |
| API $ per chore | $0.114 · mean $0.050▲ +151% | $0.242 · mean $0.165▲ +56% | $0.278 · mean $0.287≈ -8% |
| Not Smart, most common | Poor output 14 · Too slow 11 | Too slow 7 · Poor output 3 | Too slow 17 · Poor output 9 |
Newest value, then the mean over the runs shown. Each column is one model on its own.
| GPT-6.1 Terra | 12.3 | 10.0 | 61 | Poor output · 7 | 66s | 253k / 3k | 0.85 · 0.13 pt |
|---|
| Model | This week | Week before | Votes | Most common reason | Avg time | Tokens in / out | Quota per run (5h · week) |
|---|---|---|---|---|---|---|---|
| Gemini 4 Carbon (high) | 80.0 | — | 60 | Repeated mistake · 36 | 634s | 178k / 36k | 1.06 · 0.40 pt |
| Gemini 4 Argon (high) | 64.8 | 68.6 | 61 | Too slow · 30 | 465s | 142k / 31k | 1.17 · 0.64 pt |
| Gemini 4 Hydrogen (high) | 29.8 | — | 47 (+1 skipped) | Too slow · 17 | 167s | 62k / 14k | 0.39 · 0.20 pt |