heating-mpc-ha/CHANGELOG.md

# Changelog

Engineering findings and test evidence behind each change are in [AUDIT.md](AUDIT.md).

## Unreleased — stability and correctness

Fixes four correctness problems found in 0.4.0 and separates model accuracy from
forecast accuracy. No new product features; the version stays 0.4.0 until
released. Still **observation-only**. Tested in the Home Assistant Core harness
with HA 2025.12.0 and 2026.2.3; not yet on HAOS.

### Upgrade notes

- **Forecast Error 15/30/60/120 min are renamed Model Error.** Unique IDs and,
  on existing installs, entity IDs are unchanged, so history continues. Their
  state now stays *unknown* until there are at least 5 roughly independent spans
  (`quality` is `insufficient`); the numbers stay in the attributes meanwhile.
- **A new Heating MPC Forecast Error sensor** scores forecasts as they were
  published. It stays unknown until the relay has followed enough MPC plans; while
  Generic Thermostat disagrees with the plan, most forecasts are counted as
  `counterfactual` instead of being scored.
- On the first start, the saved model gets configuration fingerprints. Models
  learned under 0.4.0 are assumed to match the configuration they are loaded with;
  later changes are checked against these fingerprints.
- Zone forecast attribute `coefficients.base` now gives `value` and `confidence`
  per coefficient instead of a bare value.
- Downgrading to 0.4.0 keeps the learned models but drops the Model Error
  statistics (0.4.0 rejects their new format).

### Fixed

- **Unknown window states no longer pass as closed.** An unavailable window
  sensor used to end an open-window event, start recovery and let the forecast
  assume a closed window. Now the last known state is kept: a window last seen
  open is treated as possibly open (no base learning, forecast assumes open), an
  event closed while the sensor was unknown is marked unconfirmed with a bounded
  duration, and recovery is learned only after a confirmed close.
- **Learned parameters are no longer reused after incompatible changes.** Each
  learned component has a fingerprint of the configuration it was learned under
  (see *Persisted data*). Changing a zone's sensors or the ratio of their weights
  recalibrates that zone; changing a heat source, neighbour, window, orientation
  or recovery time resets only the affected coefficients; comfort weight, target
  offset and names never invalidate anything. Every recalibration is listed in the
  Status attribute `model_changes`.
- **Plans are no longer published from inputs that changed meanwhile.** Before
  and after the plan is computed, the integration compares every input it used
  (zone and outdoor temperatures beyond 0.25 °C, power beyond 50 W, wind, window
  states, relay and thermostat) and discards the plan if one changed meaningfully
  (`inputs_changed`, with `changed_inputs`). An older run finishing late never
  overwrites a newer result, nothing is published after unload, and saving and
  deleting the model can no longer overlap.

### Changed

- **Model accuracy and forecast accuracy are reported separately.** Model Error
  replays the model with the inputs that actually happened (relay, electrical
  heat, windows); Forecast Error uses only what was known when the forecast was
  published, and scores it only when the relay followed the forecast's plan
  (in the planned state for at least 12 minutes of every 15-minute block). Both report MAE, RMSE, bias, the
  no-change baseline, the improvement over it, sample counts, effective
  (roughly independent) sample counts and a quality label.
- Base-model coefficients (`loss`, `heat`, `bias`) report a confidence from their
  posterior variance, like the other layers: it reflects whether the data varied
  their inputs, not how many samples there are.
- The zone form's sensor weights are numbered fields with the sensors listed
  above them (English and Romanian), and the zone form explains the two kinds of
  weight.

### Persisted data

- `schema_version` stays 2. New optional keys: `fingerprint_version`,
  `fingerprints`, `model_changes` (last 20), `operational_evaluation` (Forecast
  Error statistics, 7 days) and per-source baseline fingerprints in
  `heat_sources`. Retired zone models keep their fingerprint.
- Model Error statistics gain RMSE columns; 0.3 and 0.4.0 statistics load (their
  RMSE stays empty until new forecasts are scored).
- Pending published forecasts are kept in memory only; a restart drops those not
  yet due.

## 0.4.0 — 2026-10-27

Thermal zones, electrical heat sources, windows with ventilation and wind, and
heat exchange between zones (Milestone 2, first part). Still **observation-only**:
no new service calls, the boiler and thermostat are untouched. Tested in the Home
Assistant Core harness with HA 2025.12.0 and 2026.2.3; not yet on HAOS.

### Upgrade notes

- **Back up Home Assistant first.** The config entry moves to version 3: each room
  sensor becomes a one-sensor **zone** with its comfort weight, and the sensor's
  entity ID stays the zone's key, so learned models, forecast entities and
  accuracy statistics carry over unchanged. No model reset is needed.
- **Downgrading needs the backup.** Older versions refuse a version 3 entry.
- Configure no longer asks for room sensors or weights; manage zones with **Add
  zone** and each zone's edit button on the integration page.
- Status attributes renamed: `input_quality.rooms` → `input_quality.zones` (each
  zone lists its sensors), `excluded_rooms` → `excluded_zones`,
  `retired_room_models` → `retired_zone_models`. The Forecast Error attribute
  `rooms` is now `zones`, keyed by zone name. The `invalid_weights` status is gone
  (invalid stored weights fall back to 1.0). Update templates that used these.
- New layers start with no assumed effect and need data: the electrical term needs
  gaming sessions (or other load changes), the ventilation terms need window
  openings. Watch `improvement_vs_base` on the Forecast Error sensors before
  relying on them; switch off optional terms that do not help.

### Added

- **Zones** (config subentries): name, several sensors with sensor weights, comfort
  weight, target offset and adjacent zones. Sensor and comfort weights never change
  learned thermal coefficients.
- **Heat sources**: active power (W/kW; VA and % rejected), zone, heat fraction,
  automatic 24-hour baseline or manual value, "included in" another source (no
  double counting), thermal response time, enable/disable.
- **Windows**: contact sensor (optional inverted polarity; unknown stays unknown),
  zone, facing direction (default 345°), opening factor, enable/disable; a bounded
  log of ventilation events.
- **Wind direction** and **wind gust** weather sources with priorities and fallback;
  knots and ft/s accepted.
- Model options: electrical-heat forecast half-life, assumed open-window time,
  recovery time constant, and wind / gust / cross-ventilation terms.
- Per-zone diagnostics on each zone's forecast entity: current temperature,
  +15/+30/+60/+120-minute forecast, contributions in °C/h, coefficients with
  confidence; Status lists heat sources, windows, ventilation and recent events.
- Forecast Error sensors: `base_model_mae` and `improvement_vs_base`.
- Poll outcomes `sensors_changed` and `ventilation`, with English and Romanian names.

### Changed

- Each zone's model is layered: the unchanged base model (fallback and
  comparison), an extended layer with electrical heat and neighbours, and a
  separate ventilation layer. Zones with an open, unknown or recovering window are
  kept out of base and extended learning.
- Forecasts simulate all zones together, with electrical heat fading to baseline
  and open windows closing after the configured time; the plan search uses an exact
  compiled step and stays fast at 180 minutes.
- Accuracy scoring replays all zones with the actual relay, electrical heat and
  window history, and scores the base model on the same spans.

### Persisted data

- Model payload `schema_version` stays 2 with optional new keys: `zone_models`,
  `ventilation_models`, `heat_sources` (baseline samples) and `ventilation`
  (events). Forecast statistics gain a base-model column; 0.3 statistics load.

### Development

- New pure modules: `learning.py` (named-feature RLS), `zones.py`, `heat_sources.py`,
  `windows.py`, `zone_model.py`; [docs/MODEL.md](docs/MODEL.md) documents the model.
- Synthetic-apartment tests for gaming heat, ventilation with wind and recovery,
  and coupled rooms, each checking recovered coefficients and out-of-sample
  improvement over the base model.

## 0.3.0 — 2026-10-26

First part of Milestone 1: the integration now measures how accurate its thermal
model is. Still **observation-only**; no new options. Upgrading from an earlier
version: also read the notes of each version in between.

### Added

- Four sensors, **Heating MPC Forecast Error 15/30/60/120 min**: the thermal
  model's mean absolute error over the last 7 days, scored out of sample against
  what actually happened, using the relay's real ON/OFF history. Attributes show
  the bias, the error of a no-change forecast (`persistence_mae`), `skill` and
  per-room details. See README, "Forecast accuracy".
- **Heating MPC Learning Progress** (% towards forecasts being used) and **Heating
  MPC Last Sample** (what the last poll did for learning, with English and Romanian
  state names), so learning activity is visible without opening attributes.
- Room forecast entities: `forecast_15min` (temperature every 15 minutes over the
  horizon) and `plan_15min` (the proposed ON/OFF plan), labelled counterfactual.
- Status: `rejected_samples` per room in `models`, `sample_results_since_start`
  counting how each poll was handled, and `sample_result: waiting_for_inputs` when
  the relay, thermostat, outdoor temperature or every room is missing.

### Changed

- The model is saved after each poll's forecasts are scored, with the forecast
  statistics (about 8 KB for five rooms, at most 18 KB).

### Upgrade notes

- Six new entities appear (four forecast-error sensors, learning progress, last
  sample). The error sensors stay unknown until forecasts can be scored: 15 minutes after
  start for the 15-minute sensor, 2 hours for the 120-minute one. Early numbers
  include the learning phase and are pessimistic.
- Rollback to 0.2.x is safe: older versions ignore the new saved keys, and the six
  new sensors become unavailable (remove them in the entity settings if you roll back
  permanently).

### Development

- Bug-finding lint (ruff: pyflakes, bugbear) in the checks and CI.
- Tests for restoring a model saved by 0.2.1 and for 0.2.1 being able to read
  today's saved model.
- `.DS_Store` files are ignored and rejected by the package check.

## 0.2.3 — 2026-10-26

A stale or failed room sensor no longer pauses the whole integration. Still
**observation-only**. Upgrading from 0.2.1 (0.2.2 was not deployed): follow the
0.2.2 upgrade notes below as well.

### Changed

- A room sensor that is stale, unavailable, missing or invalid is now **left out**
  instead of pausing everything: the weighted temperature, learning and forecasts
  continue with the other rooms, whose comfort weights are re-normalized. Learning
  and forecasts pause only if no room is usable, or if the relay, thermostat or
  outdoor temperature is missing. Readiness considers only the rooms used.

### Added

- Checkbox **Keep using the last value of a room sensor that stopped reporting**
  (setup and Configure, default off), for sensors that report only when the value
  changes. A stale sensor's last value is then still used, for up to 24 hours
  without a report. Unavailable or invalid sensors are always left out.
- Status attributes `excluded_rooms` and `weight_coverage` (share of the total
  comfort weight covered by the rooms used), and a per-room `used` flag in
  `input_quality`.
- Integration icon (a radiator with a forecast curve) in **Settings → Devices &
  services**, with a dark-theme variant. Needs Home Assistant 2026.3 or newer;
  older versions show the generic placeholder.

### Upgrade notes

- The new checkbox appears in **Configure**; existing installations start with it
  off. If a room is often listed in `excluded_rooms` while its temperature is
  steady, either raise that room's report-age limit or turn the checkbox on.
- Rollback to 0.2.2 or 0.2.1 is safe; older versions ignore the new setting.

## 0.2.2 — 2026-10-26 (not deployed)

Reliability release (Milestone 0). Still **observation-only**: the integration never
calls the relay, the thermostat or any other service. No new options or UI text;
entity IDs and settings carry over. Tested in a Home Assistant Core test harness
with HA 2025.12.0 and 2026.2.3; not yet tested on HAOS or with real devices.

### Upgrade notes

- Back up Home Assistant, replace `custom_components/heating_mpc`, restart Core.
- Saved models load, but coefficients learned by 0.2.1 were fitted with a broken
  radiator-lag estimate (see Fixed). Either let them re-converge over a few days of
  heating, or reset the model: disable the integration, delete
  `/config/.storage/heating_mpc.<entry_id>`, restart Core, enable it again. Deleting
  the file while the integration runs does not work; it saves the model again.
  Deleting and re-adding the integration also resets it, but loses the settings.
- After each HA restart, learning pauses for about 30 minutes
  (`sample_result: lag_uncertain`) because the radiator state is unknown.
  Forecasts continue.
- After a day of running, compare each sensor's `max_age_seconds_24h` in Status
  with the freshness limits in Configure (see README, "Choosing freshness limits").
- Rollback: restore the previous folder (or the backup) and restart. 0.2.1 reads
  models saved by 0.2.2 and ignores the new keys.

### Fixed

- The radiator-lag estimate jumped to fully hot or fully cold at every relay
  change. In simulation this learned the heating gain about a third too low, and
  forecasts made just after a boiler cycle started from the wrong state.
- The learner's covariance grew without limit during long OFF periods (for
  example a summer), which could destabilize learning when heating resumed and
  eventually make a restart discard the saved model.
- Optional weather sources could not be removed in Configure.
- A finished forecast was discarded, making proposed heat unavailable for five
  minutes, when only a diagnostic weather sensor or the thermostat's current
  temperature changed during the calculation.
- Config entries from 0.1 were not migrated.
- Forecast entities of removed rooms stayed in the entity registry.
- Malformed or physically implausible saved models could crash setup or be used.
- ON/OFF readiness counts included rejected samples and other rooms' history.
- Polls 4–7 minutes apart, or containing a relay change, were learned as normal
  5-minute samples.
- Thermostat targets were read as Celsius on Fahrenheit installations.
- Invalid room weights or optimizer settings could cause crashes.
- 30/45-minute horizons reported their end point as a 60-minute forecast.

### Added

- Room and weather report-age limits in setup and Configure (5–1440 minutes,
  defaults 30 and 120). Stale outdoor sources fall back to the next one; a stale
  room pauses learning and forecasts until it reports again.
- A read-only listener on the relay, so all relay changes between polls are
  timestamped and integrated.
- Models of rooms removed in Configure are kept (up to 24) and restored if the
  room is added back.
- Status attributes: `input_quality` (per-source state and report age, including
  `max_age_seconds_24h`), `heat_lag`, `heat_lag_uncertainty`, `sample_result`,
  `retired_room_models`, and per-room accepted ON/OFF counts in `models`.
- Statuses `invalid_weights` and `inputs_changed`.

### Changed

- Readiness now also requires 30 accepted ON and 30 accepted OFF samples per room.
- Learning skips intervals with a relay change, an outdoor-source switch or an
  uncertain lag estimate, and polls not 270–330 seconds apart.
- `input_quality` and `weather_comparison` are no longer written to the Recorder;
  they change every poll. Other attributes are still recorded.

### Persisted data

- Model payload `schema_version` 2 (0.2.1 wrote no version). Optional keys:
  per-room `on_count`/`off_count` and `retired_learners`. Older readers ignore them.

### Development

- Home Assistant Core test harness for HA 2025.12.0 and 2026.2.3, a Gitea Actions
  matrix over both, package and translation validation, and a static test that
  fails if actuator service calls are added.

## 0.2.1

- Corrects the manifest classification from `helper` to `service`. Existing config
  entries move from Helpers to **Settings → Devices & services → Integrations**
  after replacing the files and restarting Home Assistant Core.
- Deleting the config entry removes its entities and the learned model
  (`.storage/heating_mpc.<entry_id>`). The component folder stays until removed
  manually.
- MPC behavior otherwise unchanged; read-only / shadow-only.