Document the final stability fixes and the model reset

README: window states and forecast assumptions, Model Error incomplete and
skipped spans, Forecast Error classes and why 2 and 5 minutes, and the
lifecycle tests. docs/MODEL.md: strict inputs, unknown-window assumptions,
the plan-matching table. CHANGELOG gains an Unreleased section (version
unchanged); ROADMAP and AUDIT record the findings, root causes, evidence and
remaining limitations.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
buzeraandClaude Opus 5.5 committed 2026-10-27 04:54:19 +02:00
1 parent 0b82a8e26c
commit c4035951df
5 files changed
+275 -47

No files matched your search

+70
View File
@@ -397,3 +397,73 @@ Remaining risks:
ruff, 159 offline tests, 28 HA Core tests; the offline tests also pass on Python
3.9.6. `sh scripts/check.sh` additionally validates packaging, translations and
the changelog.
## Final stability fixes after 0.4.1 — 2026-10-27
Scope: four issues from an independent review of 0.4.1, plus a user-requested
*Reset learned model* action. Repository code and tests only (HA Core 2025.12.0
and 2026.2.3 harness); no HAOS, device or deployment testing. Still no actuator
code; the static and runtime service-call guards pass. Version unchanged (0.4.1).
Confirmed issues, root causes and fixes:
1. **Model Error scored spans with unknown ventilation inputs** (2cfba02).
Reproduced: a window unknown for a whole 15-minute span gave a Model Error
sample (MAE 0.76 °C). Root cause: the replay used the prediction path, which
drops the ventilation term when the opening is unknown, i.e. assumes closed.
Fix: per zone and step, `inputs_complete` checks every input the zone's model
used (window opening and confirmed recovery; wind, direction and other zones'
windows while a window is open; electrical heat and neighbours for the extended
layer, including the neighbour's own completeness). Incomplete spans are
counted, not scored, with the base model's error kept apart; unreplayable
spans are counted as skipped. Learning also stopped substituting a zero facing
term or zero cross-ventilation input for unknown values.
2. **Window unknown at startup treated as closed** (13cb972). Reproduced
(0 % assumed opening). Root cause: last known states lived only in memory, and
"never observed" fell through to the closed case; a first reading of an open
window was also logged as a confirmed opening. Fix: confirmation only for
witnessed transitions; last known states, open events and recovery saved (also
on every window state change) and restored as unknown; per-window forecast
assumptions (observed / last known open, bounded by the open-window time after
it was last seen / last known closed / unobserved), exposed in Status and on
zone forecasts; durations as ranges when an end was not seen. Chosen over
open/closed scenario pairs to avoid a second plan simulation per poll.
3. **Lifecycle and storage races** (28ff953). Confirmed by reading HA's unload
order: the coordinator's own shutdown runs only after platforms unload, and the
save checked for shutdown before waiting for the storage lock (check-then-act).
Found during testing: unloading cancels a running scheduled update, which
released the lock while the file write continued in a worker thread, so a
deletion could run first and be undone. Fix: closing mark at the start of
unload (restored if unload fails); checks after acquiring the lock, including a
removed-model mark set under the lock by removal; the write runs as a shielded,
HA-tracked task that holds the lock until it finishes; failed setup shuts the
coordinator down; queued relay callbacks are ignored once closing. Mutation
checks: removing the locked check or the shield makes the deletion tests fail;
removing the early closing mark fails the ordering test (the HA-level unload
test cannot distinguish it, as HA's own shutdown also lands before release).
4. **Forecast Error accepted materially different relay behaviour** (0b82a8e).
Reproduced: 12 of 15 planned minutes was scored. Root cause: a per-block
on-share tolerance of 20 %, blind to timing within a block. Fix: deviation =
time in another state than planned, block by block from the relay's switch
times, with unobserved time as possible deviation; matched ≤ 2 min (scored),
approximate ≤ 5 min (separate), counterfactual > 5 min observed, otherwise
incomplete with a reason; forecasts made with assumed inputs are incomplete.
Ledger statistics are versioned; 0.4.1 figures are dropped.
Added: *Reset learned model* (49565d6): unload, strip learned keys from the
stored model under the lock, set up again; configuration, heat-source baselines
and window history kept.
Tests added (by area): Model Error incomplete/skipped (7), plan matching (10),
window startup/restart/mixed/strict inputs (9), coordinator window reload and
uncertain forecast (2), lifecycle offline (5) and in HA (4), reset flow (1).
Remaining limitations: unknown-but-last-closed and never-seen windows still
produce a closed-window forecast (flagged, never scored); the 2 and 5 minute
limits rest on typical learned heat coefficients, not a per-forecast calculation;
spans lost when the replay window restarts are not counted; saved window state can
be up to one save old, which only makes the restored unknown period longer.
**Verification.** `sh scripts/check.sh` passes in both environments (pip check,
ruff, 192 offline tests, 33 HA Core tests, compilation, package/translation/
changelog checks, diff whitespace); the offline tests also pass on Python 3.9.6.