Document the final stability fixes and the model reset
README: window states and forecast assumptions, Model Error incomplete and skipped spans, Forecast Error classes and why 2 and 5 minutes, and the lifecycle tests. docs/MODEL.md: strict inputs, unknown-window assumptions, the plan-matching table. CHANGELOG gains an Unreleased section (version unchanged); ROADMAP and AUDIT record the findings, root causes, evidence and remaining limitations. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
0b82a8e26c
commit
c4035951df
5 files changed
+275
-47
No files matched your search
@@ -66,12 +66,24 @@ modeled; the persistent load is already part of the zone's learned behavior.
|
||||
**Windows** need a contact sensor (binary sensor), the zone, whether the sensor
|
||||
reports *on* when closed (inverted), the direction the outside of the window faces
|
||||
(degrees clockwise from north; default 345°, editable) and an opening factor
|
||||
(1.0 normal, lower for tilted, higher for large). Unknown or unavailable states are
|
||||
treated as unknown, never as closed: a window last seen open counts as possibly
|
||||
open (the forecast assumes it open, the zone's base model does not learn), and an
|
||||
opening that ended while the sensor was unknown is logged as unconfirmed, with its
|
||||
longest possible duration. Recovery after closing is learned only from a confirmed
|
||||
close.
|
||||
(1.0 normal, lower for tilted, higher for large).
|
||||
|
||||
A window is **open**, **closed** or **unknown** (unknown, unavailable or no state);
|
||||
unknown never means closed. While any window of a zone is unknown, the zone does
|
||||
not learn. What forecasts assume for an unknown window:
|
||||
|
||||
- last seen open: open, but only until the configured open-window time after it was
|
||||
last seen (never renewed), then closed and recovering;
|
||||
- last seen closed, or never seen since setup: no ventilation, but the forecast is
|
||||
flagged (`uncertain_windows` on the zone's forecast entity, `forecast_assumption`
|
||||
in Status) and is never used to measure forecast accuracy.
|
||||
|
||||
An opening or closing is *confirmed* only when it was seen happen: the previous
|
||||
reading was the other state, with no unknown period or restart in between. Events
|
||||
report a duration only when both ends are confirmed, otherwise a minimum and, where
|
||||
possible, a maximum; recovery is learned only after a confirmed close. The last
|
||||
known states, an open event and a recovery in progress are saved, so a restart or
|
||||
reload continues them, but they come back as unknown until the sensor reports.
|
||||
|
||||
Model options in **Configure**: how fast extra electrical heat is assumed to fade in
|
||||
forecasts (half-life, default 30 minutes), how long an open window is assumed to
|
||||
@@ -186,9 +198,10 @@ the plan is discarded and the next poll computes a fresh one. Useful attributes:
|
||||
included sources), baseline, variable power above it, thermal response, whether it
|
||||
has settled.
|
||||
- `windows` — per window: state (`open`, `closed` or `unknown`), the last known
|
||||
state and for how long the sensor has been unknown, minutes open, wind direction
|
||||
relative to the window's facing and the resulting exposure (1 = wind straight at
|
||||
it).
|
||||
state and for how long the sensor has been unknown, what forecasts assume
|
||||
(`forecast_assumption`: `observed`, `last_known_open`, `last_known_closed` or
|
||||
`unobserved`; `forecast_open_minutes`), minutes open, wind direction relative to
|
||||
the window's facing and the resulting exposure (1 = wind straight at it).
|
||||
`ventilation` — per zone: current opening, recovery, temperature before opening,
|
||||
its contribution in °C/h and the ventilation coefficients. `ventilation_events` —
|
||||
the last five openings with temperatures before, lowest, at closing and 30
|
||||
@@ -288,35 +301,57 @@ model is scored on the same spans. Each sensor's state is the mean absolute erro
|
||||
- `base_model_mae`, `improvement_vs_base` — the base model's error on the same
|
||||
spans, and the share of it the extra layers remove (0 = no gain, negative = they
|
||||
make it worse: switch that option off).
|
||||
- `incomplete` — spans left out because an input the zone's model needed was not
|
||||
observed at some step: a window state (or the start of a recovery), the wind,
|
||||
wind direction or another zone's windows while a window was open, or, for the
|
||||
extended layer, electrical heat or a neighbour (whose own span must be complete
|
||||
too). The base model needs none of these, so its error on those spans is still
|
||||
reported as `incomplete_base_model_mae`.
|
||||
- `skipped` — spans that could not be replayed: the radiator-lag estimate was
|
||||
uncertain along the way, or the zone had no reading at the end.
|
||||
- `zones` — the same numbers per zone (not written to the Recorder).
|
||||
|
||||
Spans are scored only where the inputs were known: polls 4.5–5.5 minutes apart, a
|
||||
usable reading at the start and the end, and a well-known radiator-lag estimate
|
||||
throughout. A missing relay, outdoor temperature or all zones restarts the replay
|
||||
window, so scores resume 15–120 minutes later. Statistics are kept with the model
|
||||
and survive restarts; recalibrating a zone or deleting the model resets them.
|
||||
Spans need polls 4.5–5.5 minutes apart; a missing relay, outdoor temperature or
|
||||
all zones restarts the replay window, so scores resume 15–120 minutes later (these
|
||||
gaps are not counted). Statistics are kept with the model and survive restarts;
|
||||
recalibrating a zone or resetting the learned model clears them.
|
||||
|
||||
### Forecast Error
|
||||
|
||||
Each time a plan is published, its forecast is recorded as it was: start
|
||||
temperatures, the trajectory, the plan, the model revision and the assumptions
|
||||
(outdoor temperature and wind held, electrical heat fading, open windows closing).
|
||||
When the forecast's horizon comes, it is scored against the reading, but only if
|
||||
the relay actually followed the plan: in every 15-minute block up to that horizon
|
||||
the relay must have been in the planned state for at least 12 of the 15 minutes.
|
||||
Otherwise the forecast is not an error measurement, and is counted instead:
|
||||
When the forecast's horizon comes, the relay's actual behaviour is compared with
|
||||
the plan, block by block, from the relay's own switch times: the **deviation** is
|
||||
how long the relay was in another state than planned, so the same on-time at other
|
||||
times than planned counts as deviation. Relay time that was not observed counts as
|
||||
possible deviation. Each forecast is then:
|
||||
|
||||
- `counterfactual` — the relay did something else: Generic Thermostat, which
|
||||
controls it, disagreed with the plan. Expected in observation mode whenever the
|
||||
two disagree.
|
||||
- `unverifiable` — the relay history is incomplete for that span.
|
||||
- `missing` — no poll near the due time, or no reading for a zone.
|
||||
- `matched` — at most 2 minutes of deviation. Only these make the Forecast Error
|
||||
figures.
|
||||
- `approximate` — at most 5 minutes (of which at most 2 unobserved), for example
|
||||
the thermostat switching a few minutes earlier than planned. Scored separately
|
||||
in `approximate_stats`.
|
||||
- `counterfactual` — more than 5 minutes: Generic Thermostat, which controls the
|
||||
relay, did something else. Not an error of the forecast. Expected in observation
|
||||
mode whenever the two disagree.
|
||||
- `incomplete` — not comparable, with `incomplete_reasons`: `relay_history` (too
|
||||
much relay time unknown), `uncertain_inputs` (the forecast had to assume an
|
||||
unknown window or electrical heat), `no_poll` (no poll near the due time) or
|
||||
`reading` (a zone had no reading; counted per zone).
|
||||
|
||||
The state is the 60-minute MAE once its quality is at least `indicative`; the
|
||||
attribute `horizons` (not written to the Recorder) gives MAE, RMSE, bias, the
|
||||
no-change error, `improvement_vs_persistence`, sample counts, quality, the counts
|
||||
above and the per-zone figures for each horizon. Statistics cover 7 days and
|
||||
survive restarts; forecasts not yet due when HA restarts are dropped.
|
||||
Why these limits: d seconds of boiler heat that the plan did not assume move a room
|
||||
by roughly *heat* × d / 300, where *heat* is the learned boiler coefficient
|
||||
(typically 0.05–0.12 °C per 5 minutes with warm radiators). 2 minutes stay below
|
||||
half a typical 0.1 °C sensor step and absorb timestamp jitter; 5 minutes are about
|
||||
one step.
|
||||
|
||||
The state is the 60-minute MAE of matched forecasts once its quality is at least
|
||||
`indicative`; the attribute `horizons` (not written to the Recorder) gives MAE,
|
||||
RMSE, bias, the no-change error, `improvement_vs_persistence`, sample counts,
|
||||
quality, the counts above, `approximate_stats` and the per-zone figures for each
|
||||
horizon. Statistics cover 7 days and survive restarts; forecasts not yet due when
|
||||
HA restarts are dropped.
|
||||
|
||||
What neither covers: outdoor temperature changes during the horizon (held
|
||||
constant, as in the MPC), and periods when the model was still learning (included,
|
||||
@@ -359,9 +394,10 @@ The brand icons in `custom_components/heating_mpc/brand/` are drawn by
|
||||
`python scripts/make_brand_icons.py` (uses Pillow from the test requirements);
|
||||
the package check verifies their sizes and transparency.
|
||||
|
||||
HA tests cover setup, config/options forms, migration, reload (a single relay
|
||||
listener and polling coordinator after repeated reloads) and entity cleanup,
|
||||
scheduled sampling and poll cancellation, the relay listener, stored model
|
||||
HA tests cover setup, config/options forms (including the reset), migration,
|
||||
reload (a single relay listener and polling coordinator after repeated reloads),
|
||||
unload, reload and removal while an optimization or a save is still running,
|
||||
entity cleanup, scheduled sampling and poll cancellation, the relay listener, stored model
|
||||
restoration, retention and deletion, missing inputs, manual off/presets, optimizer
|
||||
failure and Recorder exclusions. A fixture rejects all HA service calls throughout
|
||||
each test, and a static test fails if actuator service calls are added to the
|
||||
|
||||
Reference in new issue
Block a user