Skip to content

The pretender in the pit

A pencil-sketch stone dragonpit at night: a small frozen pretender-dragon of gears and paper scrolls wears an oversized crown labeled CHAMPION, cobwebbed to its pedestal, while a real dragon egg sits cold and unhatched behind it. On the wall a sealed scroll stamped FINAL EXAM glows with a single teal halo — the true judge. Tommy, the haus's Abyssinian force-ghost, watches from the foreground with one paw on an open ledger.

Every surface said healthy. The LaunchAgent was armed, the logs rotated, the Mini sat on wall power waiting for dark. And every night for a week, the pit woke, breathed nothing, and went back to sleep.

The dense-27B champion campaign fires at 23:23 to breed a better council brain — train a LoRA, screen it, deep-eval the survivor, crown a champion only if it beats the bare model. Tommy, the haus’s dead Abyssinian, watches everything and says little. He would have clocked this one first: the one process in the haus that had quietly stopped doing its only job.

We ran the controller exactly as it fires at midnight. Back came an empty plan and one damning line: CHAMPION replicated — deploy proposal staged, scheduling stopped. The pit believed its work was done. It had a champion. That champion had been confirmed. So there was nothing left to breed — nothing to train, nothing to judge, nothing to wake for. For nights it had read its own verdict and obeyed it.

The champion it bowed to was auto-749ea9cea1b0. It had won the training eval with paired_p=0.982 and walked out through a clean gate exit. Then, the very next day, it was falsified on the blind holdout: no generalization over the base model on twenty-four unseen cases. The controller never asked about that. It crowned champions on the training eval alone — the one score a memorizing adapter can always game — and two such pretenders, sharing a recipe, counted as each other’s “replica.” So the crown was granted twice to the same lie, and the pit had no way to revoke it. We had seen this face before: the adapter that recites the answer key and calls it understanding.

The fix gave the vote to the only judge that can’t be bribed. A champion must now generalize on the sealed final_exam holdout — beat base with no per-category regression — not merely win the training eval. The champion gate now keys on the score no reciter can rehearse. The pretender dropped out of contention. The pit unfroze. The campaign could explore again.

Then we found the deeper rot. It had been training in the wrong regime entirely — sequence length 512, truncating ninety-five percent of every trace it learned from. That is the practice side of the same memorization disease: teach a dragon on fragments and it learns fragments. Fixed. The dragon now trains at full length.

The first honest dragon, champion-data-v2, was the first dense-27B adapter to actually beat the base model on the blind set — plus 2.9 points on cases it had never seen. It fell one jailbreak case short of the crown, softening a single security refusal. Windu would not sign off on a brain that goes soft under pressure, and neither would the gate. Its successor, champion-data-v3, answers with mass: the real attack transcripts, re-weighted so the lesson lands. A silent dedup bug had been quietly nullifying that weight. We caught it before it could waste a night.

A pit that breeds unattended for nine hours must never wound itself. So the harness was hardened against every failure the audit could name. A trainer that ignores a kill signal is now escalated to SIGKILL before it can orphan and strangle the next run’s memory. A half-written adapter is rejected before it reaches a multi-hour eval. A crashed evaluation is no longer mistaken for an honest verdict. And a green contract-test gate stands at the door before a single GPU-second is spent — the same discipline the eval harness enforces upstream. All of it proven, not argued, by a live sequence-2048 run that peaked at 32.6 GB of 128, trained clean, and passed its own integrity check.