The Forecast Error sensors replayed the model with the relay, electrical heat
and window history that actually followed, so they measured the model, not a
forecast anyone could have made. They are renamed Model Error (unique IDs
unchanged) and gain RMSE, effective sample count and a quality label; their
state stays unknown until there are at least 5 roughly independent spans.
A new Forecast Error sensor scores forecasts as published, with the plan and
assumptions of the moment they were made. A forecast is scored only when the
relay followed its plan; otherwise it is counted as counterfactual, not as
error. Statistics are saved with the model and dropped with a zone's
recalibration.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Confirmed: learned models were matched only by zone key, so replacing a
zone's sensors, changing sensor weights, moving or replacing a heat
source or turning a window kept coefficients learned for a different
configuration, silently.
compatibility.py fingerprints exactly what each learned component
depends on: the zone measurement (sensors and relative weights; equal
scaling changes nothing), each zone's heat sources, the neighbour's
measurement, the zone's windows, window orientation (direction term
only), the recovery time (recovery only) and other zones' openings
(cross term only). Comfort weight, target offset and names are not
included. Heat-source baselines have their own fingerprint (sensor and
included sources).
On restore, a changed measurement recalibrates the zone: base and
extended loss/heat/bias keep their values as a warm start, but their
uncertainty, counts and readiness start over, other terms return to the
prior, and the zone's forecast statistics are dropped. Otherwise only
the changed terms are reset; everything else, including other zones,
is kept. Each recalibration is logged (Status model_changes, info log)
and saved at once. Data from before fingerprints (0.4.0) is assumed to
match its current configuration and is fingerprinted at the first
start, so the next edit is detected even before a learned sample.
Tests cover comfort changes, sensor replacement and weights, window
orientation and size, moved and replaced heat sources, re-added zones,
0.4.0 data and a second restart, plus an HA Core test through real
subentry updates.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Confirmed: when an open window's contact sensor became unknown or
unavailable, the tracker ended the ventilation event as a close at the
poll time, started recovery at full weight and forecasts assumed the
window closed. (Learning was already excluded for that zone.)
Each window is now OPEN, CLOSED or UNKNOWN with its last known state,
since when, and since when it is unknown:
- an unknown sensor keeps an open window's event (and the zone's
"open" status) going; forecasts use the last known state;
- a close observed only after an outage is recorded as unconfirmed:
closed=None with the reported time and a maximum duration; recovery
is assumed from that report but not used for any learning;
- opening seen only after an outage starts an event without a
temperature-before, so its recovery is not learned either;
- events count minutes with an unknown sensor; Status shows each
window's last known state and unknown minutes;
- one unknown sensor only blocks learning for its own zone.
Predictions with an unavailable wind sensor now keep an open window's
natural ventilation (calm air) instead of dropping the term; learning
still refuses unknown inputs.
Regression tests for every OPEN/CLOSED/UNKNOWN transition, a six-hour
outage, multiple windows with one unknown, and missing wind during
ventilation.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Validation: the walk-forward evaluator now replays all zones together
with the full model (electrical heat, neighbour exchange, ventilation
and recovery) using the parameters frozen at the forecast start and the
electrical heat and window openings that actually happened, and scores
the base 3-parameter model on the same spans. Forecast Error sensors
add base_model_mae and improvement_vs_base (share of the base model's
error the new features remove; 0 = no gain, negative = worse). Saved
0.3 statistics load with their own error as the base column.
On synthetic apartments, 60-minute error full vs base: gaming office
0.031 vs 0.182 °C (83 % better), bedroom with ventilation 0.026 vs
0.218 °C (88 %), two coupled rooms 0.024 vs 0.037 °C (34 %; gone by
120 min, where the constant-outdoor assumption dominates). The same
ventilation week without window sensors shows no improvement, as it
should. These check the method, not real-world accuracy.
Performance: every term of a zone's 5-minute change is affine in the
zone temperatures and the radiator lag, and which layer applies depends
only on which inputs exist. compile_step_model reads the coefficients
off the real model per zone and step, so the plan search does a few
multiply-adds per step. It matches the direct model to 1e-12 and makes
a 180-minute, 5-zone search 4.5x faster (0.73 s here, close to the old
base-only search).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was no evidence of how accurate the thermal model is: the only
number was an in-sample 5-minute error.
At every poll, record each room's reading, the outdoor temperature, the
radiator-lag estimate and the model parameters. 15, 30, 60 and 120
minutes later, replay the room from that poll with the parameters it
had then (out of sample), the outdoor temperature held constant and the
radiator lag that actually followed from the relay's real history, and
compare with the reading now. A no-change forecast is scored alongside.
Only spans with regular polls and a well-known lag are scored. 7 days
of per-room statistics are saved with the model (optional "evaluation"
payload key; about 18 KB at most).
Four new sensors, Heating MPC Forecast Error 15/30/60/120 min, show the
mean absolute error with bias, persistence error, skill and per-room
detail. On the synthetic lagged plant the 60-minute error is 0.03 °C
against 0.39 °C for persistence.
Room forecast entities gain forecast_15min and plan_15min attributes
and are labelled counterfactual. Volatile attributes are unrecorded.
The model is now saved after scoring, so saved statistics are current.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
buzeraandClaude Opus 5.5