Document the final stability fixes and the model reset

README: window states and forecast assumptions, Model Error incomplete and
skipped spans, Forecast Error classes and why 2 and 5 minutes, and the
lifecycle tests. docs/MODEL.md: strict inputs, unknown-window assumptions,
the plan-matching table. CHANGELOG gains an Unreleased section (version
unchanged); ROADMAP and AUDIT record the findings, root causes, evidence and
remaining limitations.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
buzeraandClaude Opus 5.5 committed 2026-10-27 04:54:19 +02:00
1 parent 0b82a8e26c
commit c4035951df
5 files changed
+275 -47

No files matched your search

+47 -12
View File
@@ -98,11 +98,19 @@ known. While a window is open, possibly open (its sensor is unknown and it was l
seen open) or the zone is recovering, the base and extended layers do **not**
learn, so window events cannot inflate `loss` or shift `bias`.
An unknown window state is never read as closed. A window last seen open stays
assumed open in forecasts; one last seen closed contributes no ventilation term.
If a window closes while its sensor was unknown, the event's duration is reported
as a range and the recovery term is not learned from it, because the closing time
is not known.
An unknown window state is never read as closed, and an opening or closing counts
as confirmed only when it was seen: the previous reading was the other state with
no unknown period (or restart) in between. If a window closes while its sensor was
unknown, the event's duration is reported as a range and the recovery term is not
learned from it, because the closing time is not known. Last known states, open
events and recovery are saved and restored across restarts, as unknown until the
sensor reports again.
An input that a term needs is never filled in for learning or for Model Error:
while a window is open, the wind, its direction (for `vent_wind_dir`) and the other
zones' windows (for `vent_cross`) must be known. With every window closed, the
opening terms are 0 and need no weather. Forecasts, which must produce a number,
assume calm air for a missing wind input.
`vent_wind_dir` is bounded by `vent_wind` (|vent_wind_dir| ≤ vent_wind), so the
combined wind term never goes negative: a leeward window still ventilates.
@@ -202,7 +210,10 @@ minutes):
its response time. A gaming PC drawing 600 W now is not assumed to run on for two
hours.
* **Windows:** an open window is assumed to stay open for a configurable time
(default 15 minutes), then close; the zone then recovers.
(default 15 minutes), then close; the zone then recovers. A window whose sensor
is unknown but was last seen open is assumed open only until that time after it
was last seen, never renewed. One last seen closed, or never seen since setup, is
assumed closed, and the forecast is flagged as uncertain (`uncertain_windows`).
* **Relay:** the candidate plan being evaluated.
For the plan search the step is compiled into exact per-zone coefficients (every
@@ -228,13 +239,34 @@ scored on exactly the same spans:
* `base_model_mae`, `improvement_vs_base`: against the base layer alone. A feature
that does not improve this should be switched off (for example the gust term).
A span counts only if every input the zone's model used was observed at every
step (see *Ventilation layer*; for the extended layer also electrical heat and
neighbours, whose own replay must be complete). Otherwise it is counted as
`incomplete`, with the base model's error on it kept apart
(`incomplete_base_model_mae`), since the base model needs only the relay and the
outdoor temperature. Spans that cannot be replayed (uncertain radiator lag, no
reading at the end) are counted as `skipped`.
**Forecast accuracy (Forecast Error sensor)** scores forecasts as published: the
start readings, trajectory, plan, model revision and assumptions of the moment
they were made, nothing replaced with hindsight. Because the plan is the MPC's
proposal while Generic Thermostat controls the relay, a forecast is scored at a
horizon only if the relay was in the planned state for at least 12 minutes of
every 15-minute block up to it; otherwise it is counted as `counterfactual` (or
`unverifiable` when the relay history is incomplete) and never as an error.
proposal while Generic Thermostat controls the relay, a forecast is compared only
when the relay followed it. The deviation is the time, up to the horizon, the
relay spent in another state than planned, compared block by block from the
relay's own switch times; unobserved relay time counts as possible deviation.
| Deviation (worst case) | Class | Used for |
|---|---|---|
| ≤ 2 min | matched | Forecast Error figures |
| ≤ 5 min, ≤ 2 min unobserved | approximate | `approximate_stats` only |
| > 5 min observed | counterfactual | counted, never an error |
| otherwise | incomplete | counted, with a reason |
Forecasts made with an assumed input (an unknown window or electrical heat) are
incomplete as well. The limits follow from the model: d seconds of unplanned
boiler heat change a room by about heat × d / 300 with the learned `heat`
coefficient (typically 0.05–0.12 °C per 5 minutes), so 2 minutes stay below half a
0.1 °C sensor step and 5 minutes are about one step.
Both report how much evidence there is. Forecasts start every 5 minutes, so at
horizon h consecutive forecasts share most of their span, and all zones share one
@@ -267,5 +299,8 @@ the contamination the separate ventilation layer prevents.
variation their split is uncertain even when their combined effect is accurate.
* Forecasts hold weather constant and use one scenario for electrical heat and
windows; no forecast uncertainty band is computed yet.
* Window sensors with a permanently unknown state keep their zone out of learning;
disable such a window.
* Window sensors with a permanently unknown state keep their zone out of learning
and its Model Error spans incomplete; disable such a window.
* A window that is unknown and was last seen closed (or never seen) is assumed
closed in forecasts, flagged as uncertain; no open/closed scenario pair is
computed.