“reward hacking persists, with smaller models exhibiting larger gaps on holdout suites”

Evidence for keeping a sealed holdout suite the agent never sees: the visible suite says nothing about gaming, the gap does. The gap widening with code size matters for factories doing long-horizon work. One exploit was a 2,900-line "compiler" that memorised test inputs. Benchmark, not production; revised September 2026. Used in the Verification Atlas ("Keep the grader out of reach").