Document the stabilization pass

README: window unknown states, what configuration changes do to the learned
model, inputs_changed tolerances, coefficient confidence, model_changes, and
Model Error vs Forecast Error. docs/MODEL.md: configuration-change table,
confidence semantics, both validation modes. CHANGELOG gains an Unreleased
section (version unchanged); AUDIT and ROADMAP record findings, evidence and
remaining risks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
buzeraandClaude Opus 5.5 committed 2026-10-27 03:55:06 +02:00
1 parent 8eef04c385
commit 35370958fc
5 files changed
+340 -58

No files matched your search

+75
View File
@@ -2,6 +2,81 @@
Engineering findings and test evidence behind each change are in [AUDIT.md](AUDIT.md).
## Unreleased — stability and correctness
Fixes four correctness problems found in 0.4.0 and separates model accuracy from
forecast accuracy. No new product features; the version stays 0.4.0 until
released. Still **observation-only**. Tested in the Home Assistant Core harness
with HA 2025.12.0 and 2026.2.3; not yet on HAOS.
### Upgrade notes
- **Forecast Error 15/30/60/120 min are renamed Model Error.** Unique IDs and,
on existing installs, entity IDs are unchanged, so history continues. Their
state now stays *unknown* until there are at least 5 roughly independent spans
(`quality` is `insufficient`); the numbers stay in the attributes meanwhile.
- **A new Heating MPC Forecast Error sensor** scores forecasts as they were
published. It stays unknown until the relay has followed enough MPC plans; while
Generic Thermostat disagrees with the plan, most forecasts are counted as
`counterfactual` instead of being scored.
- On the first start, the saved model gets configuration fingerprints. Models
learned under 0.4.0 are assumed to match the configuration they are loaded with;
later changes are checked against these fingerprints.
- Zone forecast attribute `coefficients.base` now gives `value` and `confidence`
per coefficient instead of a bare value.
- Downgrading to 0.4.0 keeps the learned models but drops the Model Error
statistics (0.4.0 rejects their new format).
### Fixed
- **Unknown window states no longer pass as closed.** An unavailable window
sensor used to end an open-window event, start recovery and let the forecast
assume a closed window. Now the last known state is kept: a window last seen
open is treated as possibly open (no base learning, forecast assumes open), an
event closed while the sensor was unknown is marked unconfirmed with a bounded
duration, and recovery is learned only after a confirmed close.
- **Learned parameters are no longer reused after incompatible changes.** Each
learned component has a fingerprint of the configuration it was learned under
(see *Persisted data*). Changing a zone's sensors or the ratio of their weights
recalibrates that zone; changing a heat source, neighbour, window, orientation
or recovery time resets only the affected coefficients; comfort weight, target
offset and names never invalidate anything. Every recalibration is listed in the
Status attribute `model_changes`.
- **Plans are no longer published from inputs that changed meanwhile.** Before
and after the plan is computed, the integration compares every input it used
(zone and outdoor temperatures beyond 0.25 °C, power beyond 50 W, wind, window
states, relay and thermostat) and discards the plan if one changed meaningfully
(`inputs_changed`, with `changed_inputs`). An older run finishing late never
overwrites a newer result, nothing is published after unload, and saving and
deleting the model can no longer overlap.
### Changed
- **Model accuracy and forecast accuracy are reported separately.** Model Error
replays the model with the inputs that actually happened (relay, electrical
heat, windows); Forecast Error uses only what was known when the forecast was
published, and scores it only when the relay followed the forecast's plan
(in the planned state for at least 12 minutes of every 15-minute block). Both report MAE, RMSE, bias, the
no-change baseline, the improvement over it, sample counts, effective
(roughly independent) sample counts and a quality label.
- Base-model coefficients (`loss`, `heat`, `bias`) report a confidence from their
posterior variance, like the other layers: it reflects whether the data varied
their inputs, not how many samples there are.
- The zone form's sensor weights are numbered fields with the sensors listed
above them (English and Romanian), and the zone form explains the two kinds of
weight.
### Persisted data
- `schema_version` stays 2. New optional keys: `fingerprint_version`,
`fingerprints`, `model_changes` (last 20), `operational_evaluation` (Forecast
Error statistics, 7 days) and per-source baseline fingerprints in
`heat_sources`. Retired zone models keep their fingerprint.
- Model Error statistics gain RMSE columns; 0.3 and 0.4.0 statistics load (their
RMSE stays empty until new forecasts are scored).
- Pending published forecasts are kept in memory only; a restart drops those not
yet due.
## 0.4.0 — 2026-10-27
Thermal zones, electrical heat sources, windows with ventilation and wind, and