Document the stabilization pass

README: window unknown states, what configuration changes do to the learned
model, inputs_changed tolerances, coefficient confidence, model_changes, and
Model Error vs Forecast Error. docs/MODEL.md: configuration-change table,
confidence semantics, both validation modes. CHANGELOG gains an Unreleased
section (version unchanged); AUDIT and ROADMAP record findings, evidence and
remaining risks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
buzeraandClaude Opus 5.5 committed 2026-10-27 03:55:06 +02:00
1 parent 8eef04c385
commit 35370958fc
5 files changed
+340 -58

No files matched your search

+77
View File
@@ -320,3 +320,80 @@ weather forecasts. Learners of removed zones keep only the base model.
**Verification.** `sh scripts/check.sh` passes in both environments (lint, 125
offline tests, 26 HA Core tests, compilation, package/translation/changelog checks).
The offline tests also pass on Python 3.9.6.
## Stabilization after 0.4.0 — 2026-10-27
Scope: correctness of 0.4.0, no new features, version unchanged. Repository code
and tests only (HA Core 2025.12.0 and 2026.2.3 harness); no HAOS, device or
deployment testing. Still no actuator code; the static and runtime service-call
guards pass.
Confirmed issues and fixes:
1. **Unknown window states read as closed** (2bddc74). Reproduced: an open window
whose sensor became unavailable ended the ventilation event, started recovery
at full weight and let the forecast assume a closed window. Windows are now
tri-state with a remembered last known state: last seen open → possibly open
(base and extended layers do not learn, the forecast assumes open); a close
while unknown is logged as unconfirmed with its longest possible duration;
recovery is learned only after a confirmed close. Seven transition tests
(open/closed/unknown in every order, restart while unknown, polarity).
2. **Learned coefficients reused after incompatible changes** (099c63c, 8eef04c).
By construction: changing a zone's sensors or weight ratio, a heat source, a
window or the recovery time kept every coefficient. New
`compatibility.py` fingerprints each component's configuration; on restore
only changed components are reset (a measurement change recalibrates the zone:
coefficients kept as a starting point, uncertainty, counts and accuracy
statistics reset). Comfort weight, target offset and names never invalidate.
Heat-source baseline samples are dropped when the effective power's definition
changes. 0.4.0 data has no fingerprints, is assumed compatible, and is
fingerprinted at the first start. Resets are logged in `model_changes`.
3. **Plans published from changed inputs** (d73e32a). The pre/post-optimization
check covered only temperatures, relay and thermostat, and an overtaken update
could still publish. The input snapshot now covers zone and outdoor
temperatures, electrical power, wind, windows, relay and thermostat, with
tolerances so sensor noise does not discard every plan; a run counter keeps an
older run from overwriting a newer result; shutdown is checked after saving
and after the worker; a per-entry storage lock serializes save, restore and
delete (also fixing a delete-during-save race that could recreate the file).
4. **Model accuracy reported as forecast accuracy** (26c415f). The Forecast Error
sensors replayed the model with the actual future relay, electrical heat and
windows. They are now **Model Error** (unique IDs unchanged) with RMSE,
effective samples and a quality label, and their state stays unknown below 5
effective samples. A new **Forecast Error** sensor scores published forecasts
with the plan and assumptions of the moment, only when the relay followed the
plan (≥ 12 of every 15 minutes in the planned state); otherwise the forecast is
counted as counterfactual, unverifiable or missing, never as error.
Also reviewed:
- **Confidence** (a44163a). Extended and ventilation coefficients already used
posterior variance. Base coefficients now report it too; a test shows two weeks
of non-exciting data (relay off, steady difference) keep `heat` at 0 and `loss`
and `bias` below 0.8, while four varied days bring all three above 0.8.
Readiness still needs sample counts; it is a gate, not a confidence.
- **Lifecycle** (4911b32). Three reloads leave one relay listener and one polling
coordinator; after unload nothing is recorded or published. The test was
checked to fail with the unsubscribe removed.
- **Sensor weights UI** (9bb64e7, user report). Per-sensor weights existed but
were an unlabelled second step; they are now numbered fields with the sensors
listed, in English and Romanian.
Remaining risks:
- In observation mode the relay follows the MPC plan mainly when Generic
Thermostat happens to agree, so Forecast Error may stay `insufficient` for a
long time, and the scored subset is biased towards those periods.
- `effective_samples` corrects for overlapping horizons and shared zones only; it
is not an autocorrelation estimate. Quality thresholds (5, 30) are judgement.
- Configuration changes made before this update cannot be detected: 0.4.0 models
are fingerprinted as they are. A sensor replaced under the same entity ID, or a
weather provider swap with a systematic offset, does not change any fingerprint.
- Input-change tolerances are fixed constants, not options.
- Published forecasts awaiting their horizon are lost on restart.
- None of this has run on HAOS or with real devices.
**Verification.** In both environments (HA 2025.12.0 and 2026.2.3, Python 3.13):
ruff, 159 offline tests, 28 HA Core tests; the offline tests also pass on Python
3.9.6. `sh scripts/check.sh` additionally validates packaging, translations and
the changelog.