2026-07-31: The Reboot That Stayed Up

A Mini reboot is the only honest test of the haus. Everything can look green in a warm process table and still fail the first cold load of a 27B model under swap. This note is the pre-reboot land of that work: what broke, what was permanent, and what still requires one deliberate cold boot to call EETISMAD.
The scars that came back every boot
Section titled “The scars that came back every boot”Cathedral (:1337). With --ablation-port set, a refusal-direction bank that failed to arm (wrong geometry for dense Qwen3.6-27B vs MoE [40,2048]) made sanctum-mlx fail-closed with exit(1) — and take the council seat down with it. launchd KeepAlive then crash-looped the load under multi‑tens of GiB of swap.
Force Flow (:4077). KeepAlive thrash without a singleton left EADDRINUSE storms and a process that answered HTTP while launchd said “not running.”
Health tunnel. SSH to the Lima VM before :22 was up → exit 255 spam.
Free-sub brains. Single-probe Gemini flaps and pre-boot state edges looked like CRITICAL outages.
Ark cutover. Backup canary still probed T9 after primary restic moved to /Volumes/Ark/sanctum-restic.
Kitchen lights. Green automation Tuya Reconnect Color Guard fired on unavailable → on and called light.turn_on with only color_temp — these Tuya bulbs snap to full brightness. Family comfort is not 3000K at bri 180; SmartLife truth is bri 100, hs [29, 88.2], rgb [255, 139, 30].
Permanent fixes (on disk, live on the Mini)
Section titled “Permanent fixes (on disk, live on the Mini)”| Area | Fix |
|---|---|
bin/sanctum-mlx-launch | Geometry-validate ablation bank; strip --ablation-port if unarmed so :1337 always binds |
bin/force-flow-launch.sh + failsafe | Singleton on :4077; heal only when /health is down (PPID=1 is normal under launchd) |
bin/health-tunnel-launch.sh | Wait for VM :22; exit 75 TEMPFAIL instead of 255 |
bin/sanctum-boot-ready | Default wait 900s; --check revalidates live ports |
scripts/free-sub-brain-sentinel.sh | Streak gate + AGY auto-heal; chat transient → degraded not out |
| Backup canary + scripts | Primary Ark; scripts under ~/.sanctum/scripts/; plists rewired off ~/Backups/ |
bin/ha-green-bridge.py | Comfort on kitchen + corner RGB + corridor; pin Tuya color-guard OFF |
| Albert summer curfew | Holiday 23:00 restored (exam-week 22:30 rolled back) |
Runbooks: runbooks/clean-reboot.md, runbooks/kitchen-comfort-lights.md, package ha-green/packages/kitchen_comfort.yaml.
Comfort lights (verified with the family)
Section titled “Comfort lights (verified with the family)”Super-bright white was not “HA lying” — it was the wrong mode. Color-temp paths looked cold and harsh on these bulbs. The accepted default is the warm hs/rgb preset from SmartLife, applied on off / unavailable → on for:
light.kitchen_1…_6light.corner_rgb_1…_6+ grouplight.corridor_rgb_1…_4+ group
Pre-reboot EETISMAD legs
Section titled “Pre-reboot EETISMAD legs”| Leg | Status before cold boot |
|---|---|
| Deployed | Live on Mini (bridge, launchers, devices.yaml, boot-ready env) |
| In docs | This field note + runbooks |
| Merged | Pathspec land to sanctum-config (+ related) |
| E2E tested | Spot-tested warm; cold-boot checklist still required |
Cold-boot checklist (the missing E2E)
Section titled “Cold-boot checklist (the missing E2E)”After login, wait ~15–20 minutes for model load, then:
sanctum-boot-ready --check && echo READY# expect READY only if 4040/1337/4077 listen
for p in 4040 1337 4077 3301; do lsof -nP -iTCP:$p -sTCP:LISTEN >/dev/null && echo UP :$p || echo DOWN :$pdone
bash ~/.sanctum/scripts/pqc-status.sh | tail -8launchctl print gui/$(id -u)/com.sanctum.mlx 2>&1 | grep -E 'runs =|state ='df -h /Volumes/Ark /Volumes/ARK-GOcurl -sf http://127.0.0.1:4077/screen/schedule?child=albert | python3 -c 'import sys,json;d=json.load(sys.stdin);print(d["schedules"]["albert"])'rg -n 'STRIP --ablation|ablation ARMED|FAIL-CLOSED' ~/.openclaw/logs/sanctum-mlx.log | tail -5Pristine means: boot-ready true, core ports up, PQC all hops, mlx runs low (not a crash-loop), Ark mounted, Albert holiday curfew 23:00, no pqc/cathedral/free-sub CRITICAL in the boot window.
The receipt — the haus ran the missing E2E itself
Section titled “The receipt — the haus ran the missing E2E itself”The deliberate cold boot never got scheduled; on 2026-08-13 the haus ran an unexpected one anyway. Hard reset / power loss at 16:56:28 (utmpx reboot with no preceding shutdown). Core eventually came up. Post-boot did not.
| Clock | What happened |
|---|---|
| 16:56 | unexpected reboot |
| 16:58 | com.sanctum.post-boot starts |
| 17:04 | core-listener wait (240s) TIMEOUTs: 4040 8123 8199 8765 4077 |
| 17:06 | Phase 4+settle: 33/41, 6 fail, 1 heal (HA docker). Signal sent. exit 6 |
| 17:10 | sanctum-boot-ready READY (proxyd actually bound) |
| 19:50 | apple-services --json --mac 35/35 healthy |
The six “failures” were still starting. Three independent bugs made that look like an outage:
port_boundused bertlsofonly. proxyd listens as thesanctumuser.netstatshowed127.0.0.1:4040 LISTEN; bertlsofwas empty.sanctum-boot-readyalready had the kernel fallback; post-boot did not.- Firewalla heal skipped. Rule
requires: firewalla—sanctum_enabled firewallais no. Live key isfirewalla_bridge. - No remediations for
mac/yoda-orchestrator,mac/ha-gateway,mac/sanctum-proxy/mac/proxyd.com.sanctum.ha-gatewayandcom.sanctum.firewallaplists live under~/.sanctum/launchagents/live/, not~/Library/LaunchAgents.
Same boot dropped WhatsApp ChatStorage.sqlite (container rebuilt at 16:59, drestore.mark). Desktop kept running. CRM whatsapp_history last row: 2026-08-12 23:00.
Shipped against that receipt: kernel port_bound fallback, 600s core wait + 180s settle-confirm, remediations for the six names, WhatsApp ChatStorage warn on apple-boot-check.
Doctrine, one line
Section titled “Doctrine, one line”When something that worked breaks after reboot, ask what changed between boots — falsify before you fortify — and never let an optional seat (Heretic, a color-temp “guard,” a pre-VM SSH) take down the required one.