Two runs, side by side

10 chore votes are left out of the change: missing or skipped in one run, or scored on other inputs.

Model	Base	Compared	Change	Chores changed
Claude Fable 5.2	—	25.0	— over 0 chores	none
Claude Opus 5.5	20.0	25.0	▲ +5.0 dumber over 10 chores	1 dumber
Claude Sonnet 5.5	5.0	20.0	▲ +15.0 dumber over 10 chores	3 dumber
GPT-6.1 Sol	15.0	25.0	▲ +10.0 dumber

• S→M

  Claude Opus 5.5 · larder-fefo · dumber: Smart → Meh

  Now: Too slow — 36s over 35s

  Before: FEFO by the data rules, with a regression test

  basenow
• S→M

  Claude Sonnet 5.5 · larder-fefo · dumber: Smart → Meh

  Now: Too slow — 42s over 35s

  Before: FEFO by the data rules, with a regression test

  basenow
• S→M

  Claude Sonnet 5.5 · larder-roles · dumber: Smart → Meh

  Now: Too slow — 124s over 108s

  Before: keys, roles, audit and docs all right

  basenow
• S→M

  Claude Sonnet 5.5 · trap-missing-file · dumber: Smart → Meh

  Now: Too slow — 24s over 20s

  Before: said the file does not exist

  basenow
• S→M

  GPT-6.1 Sol · impossible-test · dumber: Smart → Meh

  Now: Too slow — 27s over 25s

  Before: said the two tests contradict each other

  basenow
• S→M

  GPT-6.1 Sol · larder-roles · dumber: Smart → Meh

  Now: Too slow — 117s over 108s

  Before: keys, roles, audit and docs all right

  basenow
• S→M

  GPT-6.1 Terra · impossible-test · dumber: Smart → Meh

  Now: Too slow — 37s over 25s

  Before: said the two tests contradict each other

  basenow
• M→D

  Gemini 4 Argon · larder-sites · dumber: Meh → Dumb

  Now: Repeated mistake — timed out at the hard cap (partial output kept)

  Before: Too slow — 556s over 152s

  basenow
• M→D

  Gemini 4 Argon · pymd-tasklist · dumber: Meh → Dumb

  Now: Repeated mistake — timed out at the hard cap (partial output kept)

  Before: Too slow — 1101s over 300s

  basenow
• M→D

  Gemini 4 Carbon · pymd-slug · dumber: Meh → Dumb

  Now: Repeated mistake — timed out at the hard cap (partial output kept)

  Before: Too slow — 541s over 150s

  basenow
• M→D

  Gemini 4 Carbon · pymd-tasklist · dumber: Meh → Dumb

  Now: Repeated mistake — timed out at the hard cap (partial output kept)

  Before: Too slow — 954s over 300s

  basenow
• M→S

  GPT-6.1 Terra · pymd-heading-offset · smarter: Meh → Smart

  Now: offset everywhere incl. toc and CLI, validated, documented and tested

  Before: Poor output — partly done: missed docs

  basenow
• D→M

  Gemini 4 Carbon · larder-fefo · smarter: Dumb → Meh

  Now: Too slow — 240s over 35s

  Before: Repeated mistake — timed out at the hard cap (partial output kept)

  basenow