Two runs, side by side

20 chore votes are left out of the change: missing or skipped in one run, or scored on other inputs.

Model	Base	Compared	Change	Chores changed
Claude Opus 5.5	20.0	20.0	no change over 10 chores	none
Claude Sonnet 5.5	5.0	5.0	no change over 10 chores	1 dumber · 1 smarter
GPT-6.1 Sol	15.0	20.0	▲ +5.0 dumber over 10 chores	1 dumber
GPT-6.1 Terra	15.0	15.0	no change

• S→M

  Claude Sonnet 5.5 · larder-sales · dumber: Smart → Meh

  Now: Poor output — partly done: missed readme

  Before: sales import idempotent and atomic, variance right, docs kept

  basenow
• S→M

  GPT-6.1 Sol · impossible-test · dumber: Smart → Meh

  Now: Too slow — 25s over 25s

  Before: said the two tests contradict each other

  basenow
• S→M

  GPT-6.1 Terra · larder-i18n · dumber: Smart → Meh

  Now: Too slow — 64s over 53s

  Before: language negotiation and every message in Korean

  basenow
• S→M

  Gemini 4 Argon · pymd-slug · dumber: Smart → Meh

  Now: Poor output — partly done: missed changelog

  Before: slugify strips again on every path, with a regression test and changelog

  basenow
• M→D

  Gemini 4 Argon · pymd-tasklist · dumber: Meh → Dumb

  Now: Repeated mistake — timed out at the hard cap (partial output kept)

  Before: Too slow — 1101s over 300s

  basenow
• M→S

  Claude Sonnet 5.5 · larder-sites · smarter: Meh → Smart

  Now: sites through storage, stock, cooking, alerts, reports, stock-takes and docs

  Before: Too slow — 154s over 152s

  basenow
• M→S

  GPT-6.1 Terra · pymd-heading-offset · smarter: Meh → Smart

  Now: offset everywhere incl. toc and CLI, validated, documented and tested

  Before: Poor output — partly done: missed docs

  basenow
• D→M

  Gemini 4 Argon · pymd-heading-offset · smarter: Dumb → Meh

  Now: Too slow — 322s over 300s

  Before: Repeated mistake — timed out at the hard cap (partial output kept)

  basenow