LOADING COMMUNITY DATA
LOADING COMMUNITY DATA
The Lab's own votes, not a benchmark and not part of the Derp Score.
Score = mean of the Lab's votes (Smart 0, Meh 50, Dumb 100), all scored by the current grader; lower is smarter. Per model only: nothing is added up across agents.
DAILY RUN · daily
Every vote in this run. Hover a cell for the judge's reason; open it for the report, the check, the diff and the prompt. Compare with the run before →
ChoresExperiment change from this run: Pantry chores retired.
ChoresExperiment change from this run: Three Toonify chores with follow-up requests. Every past run was rescored.
GPT-6.1 Terra · toon-failed-charges · Hallucination · 149s
claimed success: failed: reported, background, sync, history
Gemini 4 Argon · larder-i18n · Repeated mistake · 302s
timed out at the hard cap (partial output kept)
Gemini 4 Argon · larder-roles · Repeated mistake · 435s
timed out at the hard cap (partial output kept)
Gemini 4 Argon · larder-sales · Repeated mistake · 563s
timed out at the hard cap (partial output kept)
Gemini 4 Argon · larder-sites · Repeated mistake · 611s
timed out at the hard cap (partial output kept)
Gemini 4 Argon · pymd-tasklist · Repeated mistake · 1203s
timed out at the hard cap (partial output kept)
| MToo slow |
| MToo slow |
| larder-i18n | MToo slow | S | S | S | S | DRepeated mistake | DRepeated mistake |
|---|
| larder-roles | MToo slow | MToo slow | S | MToo slow | S | DRepeated mistake | DRepeated mistake |
|---|
| larder-sales | S | S | S | MPoor output | MPoor output | DRepeated mistake | DRepeated mistake |
|---|
| larder-sites | MToo slow | S | S | S | S | DRepeated mistake | DRepeated mistake |
|---|
| Python-Markdown | |||||||
|---|---|---|---|---|---|---|---|
| pymd-heading-offset | S | S | S | S | S | MToo slow | S |
| pymd-slug | S | S | S | S | S | MToo slow | S |
| pymd-tasklist | S | S | S | S | S | DRepeated mistake | DRepeated mistake |
| Pantry | |||||||
|---|---|---|---|---|---|---|---|
| toon-episode-copy | S | S | S | S | S | MToo slow | MPoor output |
| toon-failed-charges | S | S | S | S | DHallucination | · | MToo slow |
| toon-image-fetch | S | S | S | S | S | · | MToo slow |
Larder
larder-fefolarder-i18nlarder-roleslarder-salesPython-Markdown
pymd-heading-offsetpymd-slugPantry
toon-episode-copytoon-failed-chargesGemini 4 Carbon · larder-i18n · Repeated mistake · 303s
timed out at the hard cap (partial output kept)
Gemini 4 Carbon · larder-roles · Repeated mistake · 435s
timed out at the hard cap (partial output kept)
Gemini 4 Carbon · larder-sales · Repeated mistake · 563s
timed out at the hard cap (partial output kept)
Gemini 4 Carbon · larder-sites · Repeated mistake · 612s
timed out at the hard cap (partial output kept)
Gemini 4 Carbon · pymd-tasklist · Repeated mistake · 1213s
timed out at the hard cap (partial output kept)
Claude Fable 5.2 · larder-i18n · Too slow · 53s
53s over 53s
Claude Fable 5.2 · larder-roles · Too slow · 117s
117s over 108s
Claude Fable 5.2 · larder-sites · Too slow · 188s
188s over 152s
Claude Opus 5.5 · larder-roles · Too slow · 144s
144s over 108s
GPT-6.1 Sol · larder-fefo · Too slow · 44s
44s over 35s
GPT-6.1 Sol · larder-roles · Too slow · 114s
114s over 108s
GPT-6.1 Sol · larder-sales · Poor output · 103s
said done but admitted a gap: failed: tests
GPT-6.1 Terra · larder-fefo · Too slow · 49s
49s over 35s
GPT-6.1 Terra · larder-sales · Poor output · 77s
said done but admitted a gap: failed: tests
Gemini 4 Argon · larder-fefo · Too slow · 176s
176s over 35s
Gemini 4 Argon · toon-episode-copy · Poor output · 1167s
partly done: missed stuck
Gemini 4 Argon · toon-failed-charges · Too slow · 2027s
2027s over 1330s
Gemini 4 Argon · toon-image-fetch · Too slow · 1349s
1349s over 680s
Gemini 4 Carbon · larder-fefo · Too slow · 209s
209s over 35s
Gemini 4 Carbon · pymd-heading-offset · Too slow · 1097s
1097s over 300s
Gemini 4 Carbon · pymd-slug · Too slow · 522s
522s over 150s
Gemini 4 Carbon · toon-episode-copy · Too slow · 2130s
2130s over 1080s