Release 0.3.0

First part of Milestone 1: Forecast Error sensors scoring the thermal
model against reality, forecast trajectories, Learning Progress and Last
Sample sensors, rejected-sample and poll-outcome counters, plus lint
and rollback-contract tests. README explains forecast accuracy, the
30-minute learning wait after a restart and where Developer Tools now
lives. ROADMAP marks the items now backed by tests; AUDIT records the
evaluation design and evidence. Still observation-only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

AUDIT.md

@@ -225,3 +225,47 @@ frozen at 7) and the bedroom resumed one poll after reporting again.
 
 **Verification.** `sh scripts/check.sh` passes for HA 2025.12.0 and 2026.2.3 (69
 offline tests, 18 HA Core tests). The offline tests also pass on Python 3.9.6.
+
+
+## 0.3.0 — forecast evaluation
+
+Scope: repository code and tests (HA 2025.12.0 and 2026.2.3 harness); no HAOS.
+
+Design choices and why:
+
+- **Score the thermal model, not the MPC plan.** Generic Thermostat controls the
+  relay, so the plan is never executed and cannot be checked. Each forecast is
+  replayed with the radiator-lag values that the relay's actual history produced,
+  which tests the model under known heat input.
+- **Out of sample.** Each replay uses the parameters stored at the forecast start;
+  later learning never influences the forecast it is scored on.
+- **Only known inputs.** Spans with irregular polls, an uncertain lag estimate, or a
+  missing relay/outdoor/room reading are not scored, and a missing shared input
+  restarts the window.
+- **Baseline.** Persistence (no change) is scored on exactly the same spans; `skill`
+  reports the share of its error the model removes.
+- **Known limits.** The outdoor temperature is held at its start value (as the MPC
+  does); consecutive forecasts overlap, so samples are not independent; the 7-day
+  window includes learning-phase forecasts.
+
+Evidence: on the synthetic lagged plant (5 simulated days, thermostat cycling,
+outdoor ±5 °C daily swing) the 60-minute error is 0.03 °C against 0.39 °C for
+persistence (skill 0.92); the 120-minute error grows to 0.09 °C from the
+constant-outdoor assumption. A model matching the plant exactly scores zero error;
+a model without heat gain shows the expected negative bias. These are checks of
+the evaluation itself, not of real-world accuracy.
+
+Also in this release: per-room rejected-sample counters and per-outcome poll
+counters (Milestone 0 "failed fitting counters"), a rollback contract test against
+the 0.2.1 reader, ruff bug-finding lint, and removal of tracked `.DS_Store` files.
+
+User feedback during testing: after a reset the Status entity only said
+`learning`, and with Developer Tools moved under Settings in recent HA versions the
+attributes were hard to find, so learning activity was invisible for the first ~30
+minutes. Two sensors now show it directly: Learning Progress (%) and Last Sample
+(an enum with translated states, including *Waiting for radiators to settle*). A
+poll with missing shared inputs now reports `waiting_for_inputs` instead of nothing.
+
+**Verification.** `sh scripts/check.sh` passes in both environments: lint, 80
+offline tests, 20 HA Core tests, compilation, package checks. The offline tests
+also pass on Python 3.9.6.

CHANGELOG.md

@@ -2,6 +2,50 @@
 
 Engineering findings and test evidence behind each change are in [AUDIT.md](AUDIT.md).
 
+## 0.3.0 — 2026-10-26
+
+First part of Milestone 1: the integration now measures how accurate its thermal
+model is. Still **observation-only**; no new options. Upgrading from an earlier
+version: also read the notes of each version in between.
+
+### Added
+
+- Four sensors, **Heating MPC Forecast Error 15/30/60/120 min**: the thermal
+  model's mean absolute error over the last 7 days, scored out of sample against
+  what actually happened, using the relay's real ON/OFF history. Attributes show
+  the bias, the error of a no-change forecast (`persistence_mae`), `skill` and
+  per-room details. See README, "Forecast accuracy".
+- **Heating MPC Learning Progress** (% towards forecasts being used) and **Heating
+  MPC Last Sample** (what the last poll did for learning, with English and Romanian
+  state names), so learning activity is visible without opening attributes.
+- Room forecast entities: `forecast_15min` (temperature every 15 minutes over the
+  horizon) and `plan_15min` (the proposed ON/OFF plan), labelled counterfactual.
+- Status: `rejected_samples` per room in `models`, `sample_results_since_start`
+  counting how each poll was handled, and `sample_result: waiting_for_inputs` when
+  the relay, thermostat, outdoor temperature or every room is missing.
+
+### Changed
+
+- The model is saved after each poll's forecasts are scored, with the forecast
+  statistics (about 8 KB for five rooms, at most 18 KB).
+
+### Upgrade notes
+
+- Six new entities appear (four forecast-error sensors, learning progress, last
+  sample). The error sensors stay unknown until forecasts can be scored: 15 minutes after
+  start for the 15-minute sensor, 2 hours for the 120-minute one. Early numbers
+  include the learning phase and are pessimistic.
+- Rollback to 0.2.x is safe: older versions ignore the new saved keys, and the six
+  new sensors become unavailable (remove them in the entity settings if you roll back
+  permanently).
+
+### Development
+
+- Bug-finding lint (ruff: pyflakes, bugbear) in the checks and CI.
+- Tests for restoring a model saved by 0.2.1 and for 0.2.1 being able to read
+  today's saved model.
+- `.DS_Store` files are ignored and rejected by the package check.
+
 ## 0.2.3 — 2026-10-26
 
 A stale or failed room sensor no longer pauses the whole integration. Still

README.md

@@ -1,4 +1,4 @@
-# Heating MPC 0.2.3 — custom integration for Home Assistant OS
+# Heating MPC 0.3.0 — custom integration for Home Assistant OS
 
 **Experimental, observation-only (shadow mode).** It **never** changes the Generic Thermostat, the physical relay, the boiler's setpoint, `heat/off`, or Comfort/Away/Sleep. No Docker add-on, MQTT or InfluxDB connection is needed.
 
@@ -55,6 +55,12 @@ Entities (entity IDs may vary):
   ON/OFF plan). These assume the MPC's plan is executed; while Generic Thermostat
   controls the relay they are **counterfactual**.
 - `binary_sensor.heating_mpc_proposed_heat` — **hypothetical** heating demand (unavailable until ready).
+- **Heating MPC Learning Progress** — how close the least trained room is to being
+  used for forecasts (100 % = ready); attributes show the sample counts and what is
+  required.
+- **Heating MPC Last Sample** — what the last poll did for learning (for example
+  *Learned*, *Relay switched*, *Waiting for radiators to settle*); its history shows
+  when learning happens.
 - **Heating MPC Forecast Error 15/30/60/120 min** — how far the thermal model's
   forecasts were from what actually happened (see *Forecast accuracy*).
 
@@ -71,6 +77,21 @@ Entities (entity IDs may vary):
 
 ## Diagnostics
 
+Start with **Learning Progress** and **Last Sample**. For details, open the
+**Heating MPC Status** entity's attributes. In recent Home Assistant versions
+Developer Tools is at the bottom of **Settings** (or press Cmd/Ctrl+K and type
+"developer"); its **States** tab lists every attribute.
+
+**Why learning starts about 30 minutes after a restart.** The model learns how
+much the radiators warm each room, so it needs to know how warm the radiators are.
+It estimates this from when the relay last switched (20-minute time constant).
+After a Home Assistant restart that time is unknown, so learning waits until the
+estimate is within 10 % (*Waiting for radiators to settle*), about 32 minutes.
+Forecasts continue meanwhile. Reloading the integration usually does not cause this
+wait, because the relay's last change is still known. *Ready* then needs 200
+accepted samples (about 17 hours) per room, including 30 with the relay ON and 30
+with it OFF, which can take a few days depending on how often the boiler runs.
+
 The **Heating MPC Status** entity's state is one of `waiting_for_sensors`,
 `learning`, `paused` (thermostat not in `heat`), `waiting_for_target`,
 `invalid_weights`, `ready`, `inputs_changed` or `prediction_error`. Proposed heat

ROADMAP.md

@@ -10,7 +10,7 @@ Success means predictable comfort, understandable decisions, safe operation and
 
 ## Current baseline — v0.2.1
 
-> **Current release: 0.2.3** — Milestone 0 reliability work on top of this
+> **Current release: 0.3.0** — Milestone 0 reliability work and the first part of Milestone 1 on top of this
 > baseline; see [CHANGELOG.md](CHANGELOG.md). The lists below describe 0.2.1.
 
 **Implemented (experimental):**
@@ -49,7 +49,7 @@ Current behavior: **shadow mode only**. `generic_thermostat` remains the one and
 - [x] Add input freshness thresholds and diagnostics that identify exactly which source is missing/stale/invalid.
 - [x] Verify that source switching, entity renaming, adding/removing rooms and replacing the relay do not silently reuse incompatible fitted coefficients. *(0.2.2: tests for outdoor-source switching, adding, removing and re-adding rooms, and relay replacement. A renamed sensor is handled as removing one room and adding another; its old model is retained under the old ID.)*
 - [ ] Version persisted model schema; test corrupted/older state and migration/rollback behavior.
-- [ ] Verify calculated sample totals and successful/failed fitting counters; expose reasons why training/prediction is unavailable.
+- [x] Verify calculated sample totals and successful/failed fitting counters; expose reasons why training/prediction is unavailable. *(0.3.0: per-room accepted/ON/OFF/rejected counts, per-outcome poll counters, Learning Progress and Last Sample sensors, Status reasons; covered by tests.)*
 - [ ] Add CI, linting, tests and packaging validation (including HACS metadata/translations).
 - [x] Update installation/release documentation and remove packaged `__pycache__`/`.pyc` files if present. *(0.2.2: CHANGELOG with upgrade/rollback notes; package check rejects tracked bytecode.)*
 
@@ -89,13 +89,13 @@ recording volatile attributes, and tests against HA 2025.12.0 and 2026.2.3.
 
 **Objective:** Users can verify what the model predicts and whether it is actually accurate.
 
-- [ ] Expose trajectories for each room at 5–15-minute resolution for 2–3 hours, with forecast generation timestamp, horizon and input provenance.
-- [ ] Expose predicted +15/+30/+60/+120 minute temperatures and full proposed ON/OFF schedule.
+- [ ] Expose trajectories for each room at 5–15-minute resolution for 2–3 hours, with forecast generation timestamp, horizon and input provenance. *(0.3.0: 15-minute trajectory over the horizon; generation timestamp and provenance still to add.)*
+- [x] Expose predicted +15/+30/+60/+120 minute temperatures and full proposed ON/OFF schedule. *(0.3.0: `forecast_15min` and `plan_15min` on each room forecast entity.)*
 - [ ] Provide an HA dashboard example using native cards or a clearly documented optional chart card (do not bundle a heavyweight frontend unnecessarily).
-- [ ] Make clear that the **MPC plan forecast is counterfactual** while Generic Thermostat is controlling the relay; add a separate actual-policy forecast/evaluation pathway.
+- [ ] Make clear that the **MPC plan forecast is counterfactual** while Generic Thermostat is controlling the relay; add a separate actual-policy forecast/evaluation pathway. *(0.3.0: plan forecasts labelled counterfactual; evaluation uses the actual relay history. An actual-policy forecast, simulating Generic Thermostat, is still open.)*
 - [ ] Store a bounded history of predictions with snapshots of inputs, proposed plan and eventually observed outcome.
-- [ ] Implement out-of-sample/walk-forward MAE and bias at 15, 30, 60 and 120 minutes; include sample counts and confidence/quality labels.
-- [ ] Compare against simple benchmarks such as current-temperature persistence and a fixed thermal baseline.
+- [x] Implement out-of-sample/walk-forward MAE and bias at 15, 30, 60 and 120 minutes; include sample counts and confidence/quality labels. *(0.3.0: Forecast Error sensors; replay with parameters frozen at the forecast start and the actual relay history; `skill` against persistence. No confidence intervals yet.)*
+- [ ] Compare against simple benchmarks such as current-temperature persistence and a fixed thermal baseline. *(0.3.0: persistence done; fixed baseline open.)*
 - [ ] Add inspectable explanations: expected heat loss, expected heating response, target, room weights and reason for the next proposed action.
 
 **Candidate quality gate:** Sustain **about 0.3–0.4°C or better 60-minute MAE** on representative held-out data *under comparable known relay input sequences*, without systematic room bias. This is a starting engineering goal to validate, not a guarantee or an arbitrary gate to waive. Compare to baselines.
@@ -174,9 +174,10 @@ Items 1–3 of the original backlog are done (see AUDIT.md and CHANGELOG.md 0.2.
    room out, or optionally keep its last value for up to 24 hours). Before live
    control, decide a minimum `weight_coverage` and the outdoor-temperature outage
    policy, which still pauses.
-3. Milestone 1: record forecasts with their input snapshot and evaluate them
-   walk-forward at 15/30/60/120 minutes against persistence, replaying the actual
-   relay sequence; then expose trajectories for a dashboard.
+3. Milestone 1 (rest, 0.3.x): let the Forecast Error sensors collect a few weeks
+   of heating data and review per-room bias; a dashboard example; forecast
+   timestamps/provenance; a fixed-parameter baseline; explanations for the proposed
+   action; an actual-policy forecast that simulates Generic Thermostat.
 4. Only then consider richer weather inputs and the live-control **design**; do not
    implement live control without separate authorization.
 

custom_components/heating_mpc/manifest.json

@@ -9,5 +9,5 @@
   "iot_class": "local_polling",
   "requirements": [],
   "single_config_entry": true,
-  "version": "0.2.3"
+  "version": "0.3.0"
 }