Reproduced: a forecast planning 15 minutes of heat was scored although the
relay was on for 12 (the 0.4.1 rule allowed 20 % per 15-minute block), and
the same on-time placed elsewhere in a block passed as well.
- The relay's on-time per poll interval, integrated at its own switch
times, is compared with the plan block by block up to each horizon: the
deviation is the time the relay spent in another state than planned.
Unobserved relay time counts as possible deviation.
- Matched (at most 2 minutes, worst case): Forecast Error figures.
Approximate (at most 5 minutes, at most 2 of them unobserved): separate
approximate_stats. Counterfactual (more than 5 minutes seen): counted, not
scored. Incomplete otherwise, by reason: relay history, assumed inputs at
forecast time (an unknown window or electrical heat), no poll near the
due time, or a missing reading.
- 2 minutes of unplanned boiler heat move a room by roughly half a 0.1 °C
sensor step for the learned heat coefficients (0.05-0.12 °C per 5
minutes); 5 minutes by about one step.
- Stored ledger statistics are versioned (2); 0.4.1 statistics used the
looser rule and are dropped.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Reproduced: a window unknown for a whole 15-minute span was replayed as
closed and scored as a Model Error sample (MAE 0.76 °C on one sample).
- A zone's replay is complete only if every step's model inputs were
observed: window openings (and a confirmed recovery), the wind, direction
and other zones' windows that an open window's terms use, and, for the
extended layer, electrical heat and neighbours, whose own replay must be
complete too.
- Incomplete spans are counted, not scored; the base model's error on them
is kept separately (it needs only the relay and outdoor temperature).
Spans that could not be replayed (uncertain radiator lag, no reading at
the end) are counted as skipped. Model Error sensors show incomplete,
skipped and incomplete_base_model_mae; stored data gains an "excluded"
section and older data loads unchanged.
- Window inputs no longer stand in for unknown values: the facing term is
unknown while a window is open and the wind direction is not, and the
cross-ventilation input is unknown while another zone's window is. With
every window closed, opening terms are 0 and need no weather input.
Learning refuses such samples; forecasts still assume calm air.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Forecast Error sensors replayed the model with the relay, electrical heat
and window history that actually followed, so they measured the model, not a
forecast anyone could have made. They are renamed Model Error (unique IDs
unchanged) and gain RMSE, effective sample count and a quality label; their
state stays unknown until there are at least 5 roughly independent spans.
A new Forecast Error sensor scores forecasts as published, with the plan and
assumptions of the moment they were made. A forecast is scored only when the
relay followed its plan; otherwise it is counted as counterfactual, not as
error. Statistics are saved with the model and dropped with a zone's
recalibration.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Validation: the walk-forward evaluator now replays all zones together
with the full model (electrical heat, neighbour exchange, ventilation
and recovery) using the parameters frozen at the forecast start and the
electrical heat and window openings that actually happened, and scores
the base 3-parameter model on the same spans. Forecast Error sensors
add base_model_mae and improvement_vs_base (share of the base model's
error the new features remove; 0 = no gain, negative = worse). Saved
0.3 statistics load with their own error as the base column.
On synthetic apartments, 60-minute error full vs base: gaming office
0.031 vs 0.182 °C (83 % better), bedroom with ventilation 0.026 vs
0.218 °C (88 %), two coupled rooms 0.024 vs 0.037 °C (34 %; gone by
120 min, where the constant-outdoor assumption dominates). The same
ventilation week without window sensors shows no improvement, as it
should. These check the method, not real-world accuracy.
Performance: every term of a zone's 5-minute change is affine in the
zone temperatures and the radiator lag, and which layer applies depends
only on which inputs exist. compile_step_model reads the coefficients
off the real model per zone and step, so the plan search does a few
multiply-adds per step. It matches the direct model to 1e-12 and makes
a 180-minute, 5-zone search 4.5x faster (0.73 s here, close to the old
base-only search).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was no evidence of how accurate the thermal model is: the only
number was an in-sample 5-minute error.
At every poll, record each room's reading, the outdoor temperature, the
radiator-lag estimate and the model parameters. 15, 30, 60 and 120
minutes later, replay the room from that poll with the parameters it
had then (out of sample), the outdoor temperature held constant and the
radiator lag that actually followed from the relay's real history, and
compare with the reading now. A no-change forecast is scored alongside.
Only spans with regular polls and a well-known lag are scored. 7 days
of per-room statistics are saved with the model (optional "evaluation"
payload key; about 18 KB at most).
Four new sensors, Heating MPC Forecast Error 15/30/60/120 min, show the
mean absolute error with bias, persistence error, skill and per-room
detail. On the synthetic lagged plant the 60-minute error is 0.03 °C
against 0.39 °C for persistence.
Room forecast entities gain forecast_15min and plan_15min attributes
and are labelled counterfactual. Volatile attributes are unrecorded.
The model is now saved after scoring, so saved statistics are current.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
buzeraandClaude Opus 5.5