Release 0.3.0
First part of Milestone 1: Forecast Error sensors scoring the thermal model against reality, forecast trajectories, Learning Progress and Last Sample sensors, rejected-sample and poll-outcome counters, plus lint and rollback-contract tests. README explains forecast accuracy, the 30-minute learning wait after a restart and where Developer Tools now lives. ROADMAP marks the items now backed by tests; AUDIT records the evaluation design and evidence. Still observation-only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
d2bd398a0f
commit
e59c08f586
5 files changed
+122
-12
No files matched your search
@@ -225,3 +225,47 @@ frozen at 7) and the bedroom resumed one poll after reporting again.
|
||||
|
||||
**Verification.** `sh scripts/check.sh` passes for HA 2025.12.0 and 2026.2.3 (69
|
||||
offline tests, 18 HA Core tests). The offline tests also pass on Python 3.9.6.
|
||||
|
||||
|
||||
## 0.3.0 — forecast evaluation
|
||||
|
||||
Scope: repository code and tests (HA 2025.12.0 and 2026.2.3 harness); no HAOS.
|
||||
|
||||
Design choices and why:
|
||||
|
||||
- **Score the thermal model, not the MPC plan.** Generic Thermostat controls the
|
||||
relay, so the plan is never executed and cannot be checked. Each forecast is
|
||||
replayed with the radiator-lag values that the relay's actual history produced,
|
||||
which tests the model under known heat input.
|
||||
- **Out of sample.** Each replay uses the parameters stored at the forecast start;
|
||||
later learning never influences the forecast it is scored on.
|
||||
- **Only known inputs.** Spans with irregular polls, an uncertain lag estimate, or a
|
||||
missing relay/outdoor/room reading are not scored, and a missing shared input
|
||||
restarts the window.
|
||||
- **Baseline.** Persistence (no change) is scored on exactly the same spans; `skill`
|
||||
reports the share of its error the model removes.
|
||||
- **Known limits.** The outdoor temperature is held at its start value (as the MPC
|
||||
does); consecutive forecasts overlap, so samples are not independent; the 7-day
|
||||
window includes learning-phase forecasts.
|
||||
|
||||
Evidence: on the synthetic lagged plant (5 simulated days, thermostat cycling,
|
||||
outdoor ±5 °C daily swing) the 60-minute error is 0.03 °C against 0.39 °C for
|
||||
persistence (skill 0.92); the 120-minute error grows to 0.09 °C from the
|
||||
constant-outdoor assumption. A model matching the plant exactly scores zero error;
|
||||
a model without heat gain shows the expected negative bias. These are checks of
|
||||
the evaluation itself, not of real-world accuracy.
|
||||
|
||||
Also in this release: per-room rejected-sample counters and per-outcome poll
|
||||
counters (Milestone 0 "failed fitting counters"), a rollback contract test against
|
||||
the 0.2.1 reader, ruff bug-finding lint, and removal of tracked `.DS_Store` files.
|
||||
|
||||
User feedback during testing: after a reset the Status entity only said
|
||||
`learning`, and with Developer Tools moved under Settings in recent HA versions the
|
||||
attributes were hard to find, so learning activity was invisible for the first ~30
|
||||
minutes. Two sensors now show it directly: Learning Progress (%) and Last Sample
|
||||
(an enum with translated states, including *Waiting for radiators to settle*). A
|
||||
poll with missing shared inputs now reports `waiting_for_inputs` instead of nothing.
|
||||
|
||||
**Verification.** `sh scripts/check.sh` passes in both environments: lint, 80
|
||||
offline tests, 20 HA Core tests, compilation, package checks. The offline tests
|
||||
also pass on Python 3.9.6.
|
||||
Reference in new issue
Block a user