Document the final stability fixes and the model reset

README: window states and forecast assumptions, Model Error incomplete and
skipped spans, Forecast Error classes and why 2 and 5 minutes, and the
lifecycle tests. docs/MODEL.md: strict inputs, unknown-window assumptions,
the plan-matching table. CHANGELOG gains an Unreleased section (version
unchanged); ROADMAP and AUDIT record the findings, root causes, evidence and
remaining limitations.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
buzeraandClaude Opus 5.5 committed 2026-10-27 04:54:19 +02:00
1 parent 0b82a8e26c
commit c4035951df
5 files changed
+275 -47

No files matched your search

+19 -4
View File
@@ -95,6 +95,18 @@ changed during optimization or by an overtaken run; and model accuracy reported
as if it were forecast accuracy. Reload/unload tests now cover repeated reloads
(one relay listener, one polling coordinator). Released as 0.4.1; not yet tested on HAOS.
### Final stability fixes after 0.4.1 — 2026-10-27
An independent review of 0.4.1 reported four problems; three were reproduced and
the fourth (lifecycle races) confirmed by reading Home Assistant's unload order.
Fixed with regression tests (see AUDIT.md): Model Error spans with unknown inputs
are now `incomplete`, not scored; window sensors unavailable at startup are
unknown, with last known states saved and restored as unknown; unload, reload
and removal can no longer publish stale results or recreate a deleted model;
Forecast Error scores only forecasts whose plan the relay matched within 2
minutes. Also added: **Reset learned model** in Configure. Not released or tested
on HAOS yet.
## Milestone 1 — Explainable forecasts and dashboards (target v0.3.0)
**Objective:** Users can verify what the model predicts and whether it is actually accurate.
@@ -103,7 +115,7 @@ as if it were forecast accuracy. Reload/unload tests now cover repeated reloads
- [x] Expose predicted +15/+30/+60/+120 minute temperatures and full proposed ON/OFF schedule. *(0.3.0: `forecast_15min` and `plan_15min` on each room forecast entity.)*
- [ ] Provide an HA dashboard example using native cards or a clearly documented optional chart card (do not bundle a heavyweight frontend unnecessarily).
- [ ] Make clear that the **MPC plan forecast is counterfactual** while Generic Thermostat is controlling the relay; add a separate actual-policy forecast/evaluation pathway. *(0.3.0: plan forecasts labelled counterfactual; evaluation uses the actual relay history. Stabilization: published forecasts are scored only when the relay followed their plan and otherwise counted as `counterfactual`. An actual-policy forecast, simulating Generic Thermostat, is still open.)*
- [ ] Store a bounded history of predictions with snapshots of inputs, proposed plan and eventually observed outcome. *(Stabilization: each published forecast is kept with its start readings, plan, model revision and assumptions until scored, in memory only; the scored statistics persist. No browsable history yet.)*
- [ ] Store a bounded history of predictions with snapshots of inputs, proposed plan and eventually observed outcome. *(Stabilization: each published forecast is kept with its start readings, plan, model revision and assumptions, including any it had to assume, until scored, in memory only; the scored statistics persist, split into matched, approximate, counterfactual and incomplete. No browsable history yet.)*
- [x] Implement out-of-sample/walk-forward MAE and bias at 15, 30, 60 and 120 minutes; include sample counts and confidence/quality labels. *(0.3.0: replay with parameters frozen at the forecast start and the actual relay history; `skill` against persistence. Stabilization: these are now the Model Error sensors, with RMSE, effective sample counts and quality labels; a separate Forecast Error sensor scores published forecasts. No confidence intervals yet.)*
- [ ] Compare against simple benchmarks such as current-temperature persistence and a fixed thermal baseline. *(0.3.0: persistence done; fixed baseline open.)*
- [ ] Add inspectable explanations: expected heat loss, expected heating response, target, room weights and reason for the next proposed action.
@@ -187,9 +199,12 @@ Items 1–3 of the original backlog are done (see AUDIT.md and CHANGELOG.md 0.2.
3. Configure zones, the UPS heat source and the windows on HAOS; after a week of
heating review Model Error `improvement_vs_base` and `quality`, per-zone bias,
coefficient `confidence` (base layer included), `model_changes` after any
reconfiguration, and the Forecast Error counts (how many forecasts were scored
vs counterfactual). Switch off optional terms that do not help. Also try an
unavailable window sensor and a zone edit during HAOS acceptance.
reconfiguration, and the Forecast Error counts (matched, approximate,
counterfactual, incomplete and why). Switch off optional terms that do not
help. During HAOS acceptance also try: a window sensor unavailable across an HA
restart, a zone edit, a reload while the integration is running, *Reset learned
model* and removing a disposable entry (its `.storage/heating_mpc.<entry_id>`
file must be gone afterwards).
4. Milestone 1 (rest): a dashboard example; forecast timestamps/provenance;
explanations for the proposed action; an actual-policy forecast that simulates
Generic Thermostat.