Document the final stability fixes and the model reset

README: window states and forecast assumptions, Model Error incomplete and
skipped spans, Forecast Error classes and why 2 and 5 minutes, and the
lifecycle tests. docs/MODEL.md: strict inputs, unknown-window assumptions,
the plan-matching table. CHANGELOG gains an Unreleased section (version
unchanged); ROADMAP and AUDIT record the findings, root causes, evidence and
remaining limitations.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
buzeraandClaude Opus 5.5 committed 2026-10-27 04:54:19 +02:00
1 parent 0b82a8e26c
commit c4035951df
5 files changed
+275 -47

No files matched your search

+67 -31
View File
@@ -66,12 +66,24 @@ modeled; the persistent load is already part of the zone's learned behavior.
**Windows** need a contact sensor (binary sensor), the zone, whether the sensor
reports *on* when closed (inverted), the direction the outside of the window faces
(degrees clockwise from north; default 345°, editable) and an opening factor
(1.0 normal, lower for tilted, higher for large). Unknown or unavailable states are
treated as unknown, never as closed: a window last seen open counts as possibly
open (the forecast assumes it open, the zone's base model does not learn), and an
opening that ended while the sensor was unknown is logged as unconfirmed, with its
longest possible duration. Recovery after closing is learned only from a confirmed
close.
(1.0 normal, lower for tilted, higher for large).
A window is **open**, **closed** or **unknown** (unknown, unavailable or no state);
unknown never means closed. While any window of a zone is unknown, the zone does
not learn. What forecasts assume for an unknown window:
- last seen open: open, but only until the configured open-window time after it was
last seen (never renewed), then closed and recovering;
- last seen closed, or never seen since setup: no ventilation, but the forecast is
flagged (`uncertain_windows` on the zone's forecast entity, `forecast_assumption`
in Status) and is never used to measure forecast accuracy.
An opening or closing is *confirmed* only when it was seen happen: the previous
reading was the other state, with no unknown period or restart in between. Events
report a duration only when both ends are confirmed, otherwise a minimum and, where
possible, a maximum; recovery is learned only after a confirmed close. The last
known states, an open event and a recovery in progress are saved, so a restart or
reload continues them, but they come back as unknown until the sensor reports.
Model options in **Configure**: how fast extra electrical heat is assumed to fade in
forecasts (half-life, default 30 minutes), how long an open window is assumed to
@@ -186,9 +198,10 @@ the plan is discarded and the next poll computes a fresh one. Useful attributes:
included sources), baseline, variable power above it, thermal response, whether it
has settled.
- `windows` — per window: state (`open`, `closed` or `unknown`), the last known
state and for how long the sensor has been unknown, minutes open, wind direction
relative to the window's facing and the resulting exposure (1 = wind straight at
it).
state and for how long the sensor has been unknown, what forecasts assume
(`forecast_assumption`: `observed`, `last_known_open`, `last_known_closed` or
`unobserved`; `forecast_open_minutes`), minutes open, wind direction relative to
the window's facing and the resulting exposure (1 = wind straight at it).
`ventilation` — per zone: current opening, recovery, temperature before opening,
its contribution in °C/h and the ventilation coefficients. `ventilation_events` —
the last five openings with temperatures before, lowest, at closing and 30
@@ -288,35 +301,57 @@ model is scored on the same spans. Each sensor's state is the mean absolute erro
- `base_model_mae`, `improvement_vs_base` — the base model's error on the same
spans, and the share of it the extra layers remove (0 = no gain, negative = they
make it worse: switch that option off).
- `incomplete` — spans left out because an input the zone's model needed was not
observed at some step: a window state (or the start of a recovery), the wind,
wind direction or another zone's windows while a window was open, or, for the
extended layer, electrical heat or a neighbour (whose own span must be complete
too). The base model needs none of these, so its error on those spans is still
reported as `incomplete_base_model_mae`.
- `skipped` — spans that could not be replayed: the radiator-lag estimate was
uncertain along the way, or the zone had no reading at the end.
- `zones` — the same numbers per zone (not written to the Recorder).
Spans are scored only where the inputs were known: polls 4.5–5.5 minutes apart, a
usable reading at the start and the end, and a well-known radiator-lag estimate
throughout. A missing relay, outdoor temperature or all zones restarts the replay
window, so scores resume 15–120 minutes later. Statistics are kept with the model
and survive restarts; recalibrating a zone or deleting the model resets them.
Spans need polls 4.5–5.5 minutes apart; a missing relay, outdoor temperature or
all zones restarts the replay window, so scores resume 15–120 minutes later (these
gaps are not counted). Statistics are kept with the model and survive restarts;
recalibrating a zone or resetting the learned model clears them.
### Forecast Error
Each time a plan is published, its forecast is recorded as it was: start
temperatures, the trajectory, the plan, the model revision and the assumptions
(outdoor temperature and wind held, electrical heat fading, open windows closing).
When the forecast's horizon comes, it is scored against the reading, but only if
the relay actually followed the plan: in every 15-minute block up to that horizon
the relay must have been in the planned state for at least 12 of the 15 minutes.
Otherwise the forecast is not an error measurement, and is counted instead:
When the forecast's horizon comes, the relay's actual behaviour is compared with
the plan, block by block, from the relay's own switch times: the **deviation** is
how long the relay was in another state than planned, so the same on-time at other
times than planned counts as deviation. Relay time that was not observed counts as
possible deviation. Each forecast is then:
- `counterfactual` — the relay did something else: Generic Thermostat, which
controls it, disagreed with the plan. Expected in observation mode whenever the
two disagree.
- `unverifiable` — the relay history is incomplete for that span.
- `missing` — no poll near the due time, or no reading for a zone.
- `matched` — at most 2 minutes of deviation. Only these make the Forecast Error
figures.
- `approximate` — at most 5 minutes (of which at most 2 unobserved), for example
the thermostat switching a few minutes earlier than planned. Scored separately
in `approximate_stats`.
- `counterfactual` — more than 5 minutes: Generic Thermostat, which controls the
relay, did something else. Not an error of the forecast. Expected in observation
mode whenever the two disagree.
- `incomplete` — not comparable, with `incomplete_reasons`: `relay_history` (too
much relay time unknown), `uncertain_inputs` (the forecast had to assume an
unknown window or electrical heat), `no_poll` (no poll near the due time) or
`reading` (a zone had no reading; counted per zone).
The state is the 60-minute MAE once its quality is at least `indicative`; the
attribute `horizons` (not written to the Recorder) gives MAE, RMSE, bias, the
no-change error, `improvement_vs_persistence`, sample counts, quality, the counts
above and the per-zone figures for each horizon. Statistics cover 7 days and
survive restarts; forecasts not yet due when HA restarts are dropped.
Why these limits: d seconds of boiler heat that the plan did not assume move a room
by roughly *heat* × d / 300, where *heat* is the learned boiler coefficient
(typically 0.05–0.12 °C per 5 minutes with warm radiators). 2 minutes stay below
half a typical 0.1 °C sensor step and absorb timestamp jitter; 5 minutes are about
one step.
The state is the 60-minute MAE of matched forecasts once its quality is at least
`indicative`; the attribute `horizons` (not written to the Recorder) gives MAE,
RMSE, bias, the no-change error, `improvement_vs_persistence`, sample counts,
quality, the counts above, `approximate_stats` and the per-zone figures for each
horizon. Statistics cover 7 days and survive restarts; forecasts not yet due when
HA restarts are dropped.
What neither covers: outdoor temperature changes during the horizon (held
constant, as in the MPC), and periods when the model was still learning (included,
@@ -359,9 +394,10 @@ The brand icons in `custom_components/heating_mpc/brand/` are drawn by
`python scripts/make_brand_icons.py` (uses Pillow from the test requirements);
the package check verifies their sizes and transparency.
HA tests cover setup, config/options forms, migration, reload (a single relay
listener and polling coordinator after repeated reloads) and entity cleanup,
scheduled sampling and poll cancellation, the relay listener, stored model
HA tests cover setup, config/options forms (including the reset), migration,
reload (a single relay listener and polling coordinator after repeated reloads),
unload, reload and removal while an optimization or a save is still running,
entity cleanup, scheduled sampling and poll cancellation, the relay listener, stored model
restoration, retention and deletion, missing inputs, manual off/presets, optimizer
failure and Recorder exclusions. A fixture rejects all HA service calls throughout
each test, and a static test fails if actuator service calls are added to the