Document the stabilization pass
README: window unknown states, what configuration changes do to the learned model, inputs_changed tolerances, coefficient confidence, model_changes, and Model Error vs Forecast Error. docs/MODEL.md: configuration-change table, confidence semantics, both validation modes. CHANGELOG gains an Unreleased section (version unchanged); AUDIT and ROADMAP record findings, evidence and remaining risks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
8eef04c385
commit
35370958fc
5 files changed
+340
-58
No files matched your search
@@ -2,6 +2,81 @@
|
||||
|
||||
Engineering findings and test evidence behind each change are in [AUDIT.md](AUDIT.md).
|
||||
|
||||
## Unreleased — stability and correctness
|
||||
|
||||
Fixes four correctness problems found in 0.4.0 and separates model accuracy from
|
||||
forecast accuracy. No new product features; the version stays 0.4.0 until
|
||||
released. Still **observation-only**. Tested in the Home Assistant Core harness
|
||||
with HA 2025.12.0 and 2026.2.3; not yet on HAOS.
|
||||
|
||||
### Upgrade notes
|
||||
|
||||
- **Forecast Error 15/30/60/120 min are renamed Model Error.** Unique IDs and,
|
||||
on existing installs, entity IDs are unchanged, so history continues. Their
|
||||
state now stays *unknown* until there are at least 5 roughly independent spans
|
||||
(`quality` is `insufficient`); the numbers stay in the attributes meanwhile.
|
||||
- **A new Heating MPC Forecast Error sensor** scores forecasts as they were
|
||||
published. It stays unknown until the relay has followed enough MPC plans; while
|
||||
Generic Thermostat disagrees with the plan, most forecasts are counted as
|
||||
`counterfactual` instead of being scored.
|
||||
- On the first start, the saved model gets configuration fingerprints. Models
|
||||
learned under 0.4.0 are assumed to match the configuration they are loaded with;
|
||||
later changes are checked against these fingerprints.
|
||||
- Zone forecast attribute `coefficients.base` now gives `value` and `confidence`
|
||||
per coefficient instead of a bare value.
|
||||
- Downgrading to 0.4.0 keeps the learned models but drops the Model Error
|
||||
statistics (0.4.0 rejects their new format).
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Unknown window states no longer pass as closed.** An unavailable window
|
||||
sensor used to end an open-window event, start recovery and let the forecast
|
||||
assume a closed window. Now the last known state is kept: a window last seen
|
||||
open is treated as possibly open (no base learning, forecast assumes open), an
|
||||
event closed while the sensor was unknown is marked unconfirmed with a bounded
|
||||
duration, and recovery is learned only after a confirmed close.
|
||||
- **Learned parameters are no longer reused after incompatible changes.** Each
|
||||
learned component has a fingerprint of the configuration it was learned under
|
||||
(see *Persisted data*). Changing a zone's sensors or the ratio of their weights
|
||||
recalibrates that zone; changing a heat source, neighbour, window, orientation
|
||||
or recovery time resets only the affected coefficients; comfort weight, target
|
||||
offset and names never invalidate anything. Every recalibration is listed in the
|
||||
Status attribute `model_changes`.
|
||||
- **Plans are no longer published from inputs that changed meanwhile.** Before
|
||||
and after the plan is computed, the integration compares every input it used
|
||||
(zone and outdoor temperatures beyond 0.25 °C, power beyond 50 W, wind, window
|
||||
states, relay and thermostat) and discards the plan if one changed meaningfully
|
||||
(`inputs_changed`, with `changed_inputs`). An older run finishing late never
|
||||
overwrites a newer result, nothing is published after unload, and saving and
|
||||
deleting the model can no longer overlap.
|
||||
|
||||
### Changed
|
||||
|
||||
- **Model accuracy and forecast accuracy are reported separately.** Model Error
|
||||
replays the model with the inputs that actually happened (relay, electrical
|
||||
heat, windows); Forecast Error uses only what was known when the forecast was
|
||||
published, and scores it only when the relay followed the forecast's plan
|
||||
(in the planned state for at least 12 minutes of every 15-minute block). Both report MAE, RMSE, bias, the
|
||||
no-change baseline, the improvement over it, sample counts, effective
|
||||
(roughly independent) sample counts and a quality label.
|
||||
- Base-model coefficients (`loss`, `heat`, `bias`) report a confidence from their
|
||||
posterior variance, like the other layers: it reflects whether the data varied
|
||||
their inputs, not how many samples there are.
|
||||
- The zone form's sensor weights are numbered fields with the sensors listed
|
||||
above them (English and Romanian), and the zone form explains the two kinds of
|
||||
weight.
|
||||
|
||||
### Persisted data
|
||||
|
||||
- `schema_version` stays 2. New optional keys: `fingerprint_version`,
|
||||
`fingerprints`, `model_changes` (last 20), `operational_evaluation` (Forecast
|
||||
Error statistics, 7 days) and per-source baseline fingerprints in
|
||||
`heat_sources`. Retired zone models keep their fingerprint.
|
||||
- Model Error statistics gain RMSE columns; 0.3 and 0.4.0 statistics load (their
|
||||
RMSE stays empty until new forecasts are scored).
|
||||
- Pending published forecasts are kept in memory only; a restart drops those not
|
||||
yet due.
|
||||
|
||||
## 0.4.0 — 2026-10-27
|
||||
|
||||
Thermal zones, electrical heat sources, windows with ventilation and wind, and
|
||||
|
||||
Reference in new issue
Block a user