heating-mpc-ha/README.md

# Heating MPC 0.4.2 — custom integration for Home Assistant OS

**Experimental, observation-only (shadow mode).** It **never** changes the Generic Thermostat, the physical relay, the boiler's setpoint, `heat/off`, or Comfort/Away/Sleep. No Docker add-on, MQTT or InfluxDB connection is needed.

This project targets Home Assistant OS on a Raspberry Pi 5, with one existing ON/OFF central-heating relay, `generic_thermostat`, and individual indoor temperature sensors.

## Install / upgrade

1. Create a Home Assistant backup.
2. Copy the entire folder `custom_components/heating_mpc` into `/config/custom_components/heating_mpc` (replace its contents when upgrading; read the upgrade notes in [CHANGELOG.md](CHANGELOG.md) first).
3. Restart Home Assistant **Core**.
4. If new: **Settings → Devices & services → Add integration → Heating MPC**. If installed already: select **Heating MPC → Configure** (Options).
5. Choose the existing thermostat and physical heating switch (read-only), then room temperature sensors. Each sensor becomes a zone; group them later (see below).
6. Choose *ordered lists* of **outdoor sensor entity IDs**. First valid sensor is active; if missing, unavailable, nonnumeric or outside reasonable bounds, the next valid sensor is used automatically.
7. Assign a relative comfort weight (0.1–10) to each selected sensor's zone.
8. Optionally add zones, heat sources and windows from the integration page (next section).

On Home Assistant 2026.3 or newer the integration shows its own icon (a radiator with a forecast curve) in **Devices & services**; older versions show a generic placeholder.

**No entity IDs are hard-coded.** Every sensor, zone, heat source, window and outdoor measurement can be changed later in the UI, without editing YAML or Python.

## Zones, heat sources and windows

On **Settings → Devices & services → Heating MPC**, the buttons **Add zone**, **Add
heat source** and **Add window** create items that can each be edited or deleted
later. **Configure** offers **Settings** (thermostat, relay, weather sources,
report-age limits, MPC settings and model options) and **Reset learned model**.

**Zones** are rooms or areas. Each has a name, one or more temperature sensors, a
comfort weight (how much the zone matters to the MPC), an optional target offset
relative to the thermostat's target, and optional adjacent zones it exchanges heat
with. A zone with several sensors gets a second step, **Sensor weights**: one
numbered field per sensor (listed above the fields), setting how much each counts
in the zone temperature. A sensor belongs to one zone only. Upgrading from 0.3
turns each room sensor into a one-sensor zone with the same comfort weight; to
group two sensors of one room, edit one zone to include both and delete the other.

**What a change does to the learned model.** The zone temperature is defined by
its sensors and the ratio of their weights; changing either recalibrates that
zone (its coefficients are kept as a starting point, but their confidence, sample
counts and accuracy statistics start over, so *Ready* has to be earned again).
Scaling all of a zone's sensor weights by the same factor changes nothing. Adding,
editing or removing a heat source, a neighbour, a window, a window's orientation
or the recovery time resets only the coefficients that depend on it. Comfort
weight, target offset and names never touch the model. Each recalibration is
listed in the Status attribute `model_changes`.

**Reset learned model** (in **Configure**, after ticking the confirmation) forgets
everything learned: every zone's coefficients, including those kept for removed
zones, with their confidence and sample counts, and the Model Error and Forecast
Error statistics. Zones, sensors, weights, heat sources, windows, weather sources
and settings stay as they are, and so do the heat sources' baselines and the window
event log. The integration restarts and learns from the beginning (first sample
after about 30 minutes, *Ready* after a day or more); `model_changes` records the
reset. It never touches the boiler or the thermostat.

**Heat sources** are equipment whose electricity ends up as heat in a zone, such as
a UPS feeding computers. Settings: an active-power sensor (W or kW; VA and % are
rejected), the zone, the share of the power that becomes heat there (1.0 by
default), the baseline (automatic: the low end of the last 24 hours; or a manual
value in W), whether its reading is **already included in** another source (for a
PC measured separately behind a UPS, so it is not counted twice), and the thermal
response time (default 15 minutes). Only the variation above the baseline is
modeled; the persistent load is already part of the zone's learned behavior.

**Windows** need a contact sensor (binary sensor), the zone, whether the sensor
reports *on* when closed (inverted), the direction the outside of the window faces
(degrees clockwise from north; default 345°, editable) and an opening factor
(1.0 normal, lower for tilted, higher for large).

A window is **open**, **closed** or **unknown** (unknown, unavailable or no state);
unknown never means closed. While any window of a zone is unknown, the zone does
not learn. What forecasts assume for an unknown window:

- last seen open: open, but only until the configured open-window time after it was
  last seen (never renewed), then closed and recovering;
- last seen closed, or never seen since setup: no ventilation, but the forecast is
  flagged (`uncertain_windows` on the zone's forecast entity, `forecast_assumption`
  in Status) and is never used to measure forecast accuracy.

An opening or closing is *confirmed* only when it was seen happen: the previous
reading was the other state, with no unknown period or restart in between. Events
report a duration only when both ends are confirmed, otherwise a minimum and, where
possible, a maximum; recovery is learned only after a confirmed close. The last
known states, an open event and a recovery in progress are saved, so a restart or
reload continues them, but they come back as unknown until the sensor reports.

Model options in **Configure**: how fast extra electrical heat is assumed to fade in
forecasts (half-life, default 30 minutes), how long an open window is assumed to
stay open (default 15 minutes), the recovery time constant after closing (default
20 minutes), and whether to use wind, wind gusts and cross-ventilation terms.
See [docs/MODEL.md](docs/MODEL.md) for the equations and what is learned when.

## Reconfigurable weather inputs

| Measurement | Required? | Used in the thermal model? |
|---|---|---|
| Outdoor temperature | **Yes** (1–10 prioritized sensors) | **Yes**: estimates thermal losses; constant over the 2 h forecast window |
| Wind speed | No | Yes, for open windows (ventilation), when enabled |
| Wind direction (degrees the wind comes from) | No | Yes, with the windows' facing direction |
| Wind gust | No | Only if the gust option is enabled |
| Cloud cover | No | No, diagnostics/source comparison only |
| Humidity | No | No, diagnostics/source comparison only |
| Feels-like temperature | No | No, diagnostics/source comparison only |
| Atmospheric pressure | No | No, diagnostics/source comparison only |
| Visibility | No | No, diagnostics/source comparison only |
| Rain / snow | No | No, diagnostics/source comparison only |

Choose multiple sensors for a single variable, e.g. OpenWeatherMap then another provider, in priority order. Switch order in **Configure** to run A/B comparisons on successive days, or simply let the second sensor act as fallback. The active source and all current alternative readings are visible in the **Heating MPC Status** entity attributes (`weather`, `weather_active_sources`, `weather_comparison`). The **MPC Outdoor Temperature** sensor also has an `active_source` attribute and an `alternatives` map.

Measurements are normalized where possible: outdoor temperatures to °C; wind and gusts to m/s (from km/h, mph, kn, ft/s); wind direction to degrees (360 = 0); pressure to hPa; visibility to meters. Unknown units or values outside plausibility bounds are rejected in favor of the next selected sensor. Clouds and humidity are expected in %, rain/snow in mm or mm/h as represented by their upstream sensor. A fallback source may have a different sampling delay; configurable report-age limits now reject stale inputs (see below).

Switching only the **weather-provider sensors** does not reset learned thermal parameters. The update is applied after the integration reloads, and it skips learning one transition when the primary temperature source changes, avoiding a jump in the training series. Nevertheless, two sensors with a persistent offset can bias predictions; compare their readings and validate the model again before relying on them. Replacing the physical heating switch discards incompatible model parameters. Learned zone parameters are kept as long as the zone exists (its key does not change when you edit it).

## Model, sampling and output

- Every 5 minutes: reads sensor, relay, power and window states and learns a per-zone model in layers (full description in [docs/MODEL.md](docs/MODEL.md)):
  - **base**: `dT = loss*(T_out-T) + heat*lag + bias`, always learned, the fallback and comparison;
  - **extended**: adds electrical heat above baseline and exchange with adjacent zones;
  - **ventilation**: learned separately while a window is open or the room recovers, so open windows never look like poor insulation.
- New effects start at zero and report a confidence; nothing is assumed until the data shows it. Relay changes are timestamped by a read-only state listener.
- After 200 samples per zone, including both relay states, runs an ON/OFF MPC search for the next 60–180 minutes (default 120) in 15-minute command blocks, simulating all zones together with assumed electrical heat and window behavior.
- Uses zone comfort weights and target offsets with the **existing** Generic Thermostat's target and heat/off state. **The simulated ON/OFF demand never operates hardware**.
- Persists everything learned in HA's `.storage`. Models of zones removed in the UI are kept (up to 24) and restored if the zone is added back.

Entities (entity IDs may vary):

- `sensor.heating_mpc_status` — learning/ready/paused status and diagnostics, including outside weather, active source, all valid alternative weather values, model coefficients, proposed 15-minute plan.
- `sensor.heating_mpc_weighted_temperature` — comfort-weighted average of the zones, informative only.
- `sensor.mpc_outdoor_temperature` — selected external temp and alternatives.
- One forecast entity per zone (**MPC <zone> 60min Prediction**), state = the zone's
  temperature in 60 minutes. Attributes: `current_temperature`, `forecast` (+15/+30/
  +60/+120 minutes as far as the horizon goes), `forecast_15min`, `plan_15min`,
  `model_in_use`, `contributions_c_per_h` (what drives the zone right now: loss,
  heat, bias, internal, neighbors, ventilation, recovery) and `coefficients` with
  confidence. Plan forecasts assume the MPC's plan is executed; while Generic
  Thermostat controls the relay they are **counterfactual**.
- `binary_sensor.heating_mpc_proposed_heat` — **hypothetical** heating demand (unavailable until ready).
- **Heating MPC Learning Progress** — how close the least trained zone is to being
  used for forecasts (100 % = ready); attributes show the sample counts and what is
  required.
- **Heating MPC Last Sample** — what the last poll did for learning (for example
  *Learned*, *Relay switched*, *Waiting for radiators to settle*); its history shows
  when learning happens.
- **Heating MPC Model Error 15/30/60/120 min** — how well the thermal model
  explains what happened, replayed with the inputs that actually occurred.
- **Heating MPC Forecast Error** — how far published forecasts were from what
  happened, using only what was known when they were made (see *Forecast
  accuracy*).

## Known limitations / safety

- **Do not** connect the proposed binary sensor directly to the heating switch or build an automation from it. A production controller would need watchdog, manual override, safety handling, and an exclusive relay owner with Generic Thermostat fallback.
- Weather variables other than temperature, wind and gusts are **captured but not fitted**. Wind and gusts are used only for open windows. Keep optional model terms only when **Model Error** shows they help (`improvement_vs_base`).
- One temperature per zone (no separate wall mass), configured response times, and single-scenario forecasts of electrical heat and windows. See [docs/MODEL.md](docs/MODEL.md#limitations).
- All rooms share one heating actuator. The model does not command individual radiators.
- No weather forecast: assumes outdoor temperature remains constant during simulated horizon.
- No Recorder/InfluxDB training backfill. It learns only from samples collected after installation.
- The demand switch is not a burner-flame or real gas consumption measurement.
- Not yet tested on HAOS or with real devices. Automated tests run the integration in a Home Assistant Core test harness (HA 2025.12.0 and 2026.2.3) with simulated entities.
- Home Assistant Core APIs and config-flow selectors can change; take a backup before trying this experimental custom integration.

## Diagnostics

Start with **Learning Progress** and **Last Sample**. For details, open the
**Heating MPC Status** entity's attributes. In recent Home Assistant versions
Developer Tools is at the bottom of **Settings** (or press Cmd/Ctrl+K and type
"developer"); its **States** tab lists every attribute.

**Why learning starts about 30 minutes after a restart.** The model learns how
much the radiators warm each room, so it needs to know how warm the radiators are.
It estimates this from when the relay last switched (20-minute time constant).
After a Home Assistant restart that time is unknown, so learning waits until the
estimate is within 10 % (*Waiting for radiators to settle*), about 32 minutes.
Forecasts continue meanwhile. Reloading the integration usually does not cause this
wait, because the relay's last change is still known. *Ready* then needs 200
accepted samples (about 17 hours) per zone, including 30 with the relay ON and 30
with it OFF, which can take a few days depending on how often the boiler runs.

The **Heating MPC Status** entity's state is one of `waiting_for_sensors`,
`no_zones` (add a zone), `learning`, `paused` (thermostat not in `heat`), `waiting_for_target`, `ready`,
`inputs_changed` or `prediction_error`. Proposed heat is available only while it
is `ready`. `inputs_changed` means an input moved meaningfully while the plan was
being computed (beyond 0.25 °C, 50 W, 1 m/s wind, 2 m/s gust or 20° direction, or
any change of a window, the relay or the thermostat); `changed_inputs` lists them,
the plan is discarded and the next poll computes a fresh one. Useful attributes:

- `input_quality` — per zone (with each of its sensors) and per weather source
  (grouped by metric): `status` (`valid`, `missing`, `unknown`, `unavailable`,
  `stale`, `invalid_timestamp`, `future_timestamp` or `invalid_value_or_unit`),
  `age_seconds` (time since the sensor last reported to HA) and
  `max_age_seconds_24h` (largest age seen at polls in the last 24 hours; resets on
  reload and can read up to one poll, 5 minutes, low). Zones and sensors have `used`.
- `excluded_zones` and `weight_coverage` — zones left out of this poll, and the
  share of the total comfort weight still covered by the zones used (1.0 = all).
  `zone_names` maps zone keys to names.
- `heat_sources` — per source: measured and effective power (after removing
  included sources), baseline, variable power above it, thermal response, whether it
  has settled.
- `windows` — per window: state (`open`, `closed` or `unknown`), the last known
  state and for how long the sensor has been unknown, what forecasts assume
  (`forecast_assumption`: `observed`, `last_known_open`, `last_known_closed` or
  `unobserved`; `forecast_open_minutes`), minutes open, wind direction relative to
  the window's facing and the resulting exposure (1 = wind straight at it).
  `ventilation` — per zone: current opening, recovery, temperature before opening,
  its contribution in °C/h and the ventilation coefficients. `ventilation_events` —
  the last five openings with temperatures before, lowest, at closing and 30
  minutes after, and the weather.
- `heat_lag` and `heat_lag_uncertainty` — the radiator-lag estimate (0 cold, 1 fully
  warm) and its error bound. Learning waits while the bound is above 0.1, for
  example for about 30 minutes after an HA restart.
- `sample_result` — why the last poll was or was not learned: `first_sample`,
  `sensors_changed` (a zone's set of usable sensors changed), `learned`,
  `ventilation` (only window-affected zones; given to the ventilation model),
  `partly_rejected` (an outlier in some zone), `interval_out_of_range`,
  `relay_changed`, `outside_source_changed`, `lag_uncertain` or `waiting_for_inputs`.
- `models` — per-zone base coefficients with their `confidence`, accepted sample
  counts and the in-sample 5-minute error (`mae_5m`, **not** a forecast accuracy),
  `model_in_use` and the extended layer's coefficients with confidence. Confidence
  (0 to 1) is the share of a coefficient's information that comes from data rather
  than the starting assumption. It grows only while the data varies that
  coefficient's input: weeks with the relay always off leave `heat` at 0, however
  many samples there are.
- `model_changes` — the last recalibrations: when, which zone, which coefficients
  were reset and why.
- `retired_zone_models` — removed zones whose learned models are kept.

`input_quality`, `weather_comparison`, `heat_sources`, `windows`, `ventilation`,
`ventilation_events` and `model_changes` are not written to the Recorder; the other
attributes are.

### Choosing freshness limits

Room and weather report-age limits are set during setup and in **Configure**
(5–1440 minutes; defaults 30 for rooms and 120 for weather).

A zone sensor that is older than its limit, unavailable, missing or invalid is
**left out**; the zone uses its other sensors (without learning across that change).
A zone with no usable sensor is left out entirely: the weighted temperature,
learning and forecasts continue with the
other zones, whose weights are re-normalized. Status shows it in `excluded_zones`.
Learning and forecasts pause only when no zone is usable, or when the relay,
thermostat or outdoor temperature is missing. A zone that reports again learns
from its second poll onwards, so nothing is learned across its gap.

The checkbox **Keep using the last value of a room sensor that stopped reporting**
changes this for stale sensors: their last value is still used, for up to 24 hours
without a report. Use it for sensors that report only when the value changes.
Unavailable or invalid sensors are always left out, and so is any sensor silent
for more than 24 hours, since it is most likely dead.

A stale outdoor source falls back to the next one in its list; optional weather
metrics never block.

Within the limit (or with the checkbox on), the last reading counts as current,
so a frozen value can be learned for a while. A too-tight limit (with the checkbox
off) drops a room whenever its sensor is quietly between reports. Freshness uses
HA's `last_reported`: the time a sensor last reported to HA, even with an unchanged
value. It is not the age of the physical measurement, so a provider re-publishing
cached data looks fresh, and sensors that report only on change can look stale in
a steady room.

After a day of running, set each limit comfortably above (about 1.5–2×) the largest
`max_age_seconds_24h` among the sensors it covers. The relay and thermostat have no
age limit.

## Forecast accuracy

Two different questions, two kinds of sensor:

- **Model Error** (four sensors, 15/30/60/120 min): *does the thermal model explain
  what happened?* Replayed with the relay, electrical heat and windows that
  actually followed. Useful for judging the model and its options. It is **not**
  a forecast accuracy: a real forecast does not know when the relay will switch or
  a window will open.
- **Forecast Error** (one sensor): *how good were the forecasts as published?*
  Uses only what was known when each forecast was made.

### Model Error

At every poll the integration records each zone's reading, the outdoor
temperature, the radiator-lag estimate and the model's parameters. 15, 30, 60 and
120 minutes later it replays all zones together from that reading using the
parameters it had **then**, the outdoor temperature held at its starting value,
and the relay, electrical heat and window history that actually followed. The base
model is scored on the same spans. Each sensor's state is the mean absolute error
(°C) over the last 7 days, across all zones. Attributes, all in °C except counts:

- `samples` — number of scored forecasts; `effective_samples` — roughly
  independent ones. Forecasts start every 5 minutes, so 60-minute forecasts
  overlap twelvefold, and zones share one boiler, so `effective_samples` is
  `samples × 5 / horizon / zones`.
- `quality` — `insufficient` below 5 effective samples (the state then stays
  unknown; the numbers are still in the attributes), `indicative` below 30, `ok`
  from 30.
- `mae`, `rmse`, `bias` — mean absolute, root-mean-square and mean error
  (negative bias: the model predicts too cold).
- `persistence_mae` — error of simply assuming no change; the baseline to beat.
- `skill` — `1 - mae / persistence_mae`: 1 is perfect, 0 is no better than
  assuming no change, negative is worse. Empty while the temperature did not change.
- `base_model_mae`, `improvement_vs_base` — the base model's error on the same
  spans, and the share of it the extra layers remove (0 = no gain, negative = they
  make it worse: switch that option off).
- `incomplete` — spans left out because an input the zone's model needed was not
  observed at some step: a window state (or the start of a recovery), the wind,
  wind direction or another zone's windows while a window was open, or, for the
  extended layer, electrical heat or a neighbour (whose own span must be complete
  too). The base model needs none of these, so its error on those spans is still
  reported as `incomplete_base_model_mae`.
- `skipped` — spans that could not be replayed: the radiator-lag estimate was
  uncertain along the way, or the zone had no reading at the end.
- `zones` — the same numbers per zone (not written to the Recorder).

Spans need polls 4.5–5.5 minutes apart; a missing relay, outdoor temperature or
all zones restarts the replay window, so scores resume 15–120 minutes later (these
gaps are not counted). Statistics are kept with the model and survive restarts;
recalibrating a zone or resetting the learned model clears them.

### Forecast Error

Each time a plan is published, its forecast is recorded as it was: start
temperatures, the trajectory, the plan, the model revision and the assumptions
(outdoor temperature and wind held, electrical heat fading, open windows closing).
When the forecast's horizon comes, the relay's actual behaviour is compared with
the plan, block by block, from the relay's own switch times: the **deviation** is
how long the relay was in another state than planned, so the same on-time at other
times than planned counts as deviation. Relay time that was not observed counts as
possible deviation. Each forecast is then:

- `matched` — at most 2 minutes of deviation. Only these make the Forecast Error
  figures.
- `approximate` — at most 5 minutes (of which at most 2 unobserved), for example
  the thermostat switching a few minutes earlier than planned. Scored separately
  in `approximate_stats`.
- `counterfactual` — more than 5 minutes: Generic Thermostat, which controls the
  relay, did something else. Not an error of the forecast. Expected in observation
  mode whenever the two disagree.
- `incomplete` — not comparable, with `incomplete_reasons`: `relay_history` (too
  much relay time unknown), `uncertain_inputs` (the forecast had to assume an
  unknown window or electrical heat), `no_poll` (no poll near the due time) or
  `reading` (a zone had no reading; counted per zone).

Why these limits: d seconds of boiler heat that the plan did not assume move a room
by roughly *heat* × d / 300, where *heat* is the learned boiler coefficient
(typically 0.05–0.12 °C per 5 minutes with warm radiators). 2 minutes stay below
half a typical 0.1 °C sensor step and absorb timestamp jitter; 5 minutes are about
one step.

The state is the 60-minute MAE of matched forecasts once its quality is at least
`indicative`; the attribute `horizons` (not written to the Recorder) gives MAE,
RMSE, bias, the no-change error, `improvement_vs_persistence`, sample counts,
quality, the counts above, `approximate_stats` and the per-zone figures for each
horizon. Statistics cover 7 days and survive restarts; forecasts not yet due when
HA restarts are dropped.

What neither covers: outdoor temperature changes during the horizon (held
constant, as in the MPC), and periods when the model was still learning (included,
so early numbers are pessimistic). The roadmap's working target is about
0.3–0.4 °C at 60 minutes with clearly positive skill, before any control is
considered.

## Release notes

See [CHANGELOG.md](CHANGELOG.md) for changes and upgrade notes per version, and
[AUDIT.md](AUDIT.md) for audit findings and test evidence.

## Developer checks and Forgejo/Gitea Actions

The HA Core test suite runs against **Home Assistant 2025.12.0** (the minimum in
`hacs.json`, `requirements-test.txt`) and **2026.2.3** (newest published test
fixtures, `requirements-test-latest.txt`), both on Python 3.13. Use a dedicated
development virtual environment; these test dependencies are not installed with
the integration and do not change its local, standard-library model inference.

```bash
python3.13 -m venv .venv
. .venv/bin/activate
python -m pip install -r requirements-test.txt   # or requirements-test-latest.txt
sh scripts/check.sh
```

The check script runs dependency consistency checks, the offline tests, the HA
Core tests, compilation, package metadata, translation and changelog checks, and
diff whitespace checks. The 2025.12.0 requirements pin a compatible DNS library
because pycares 5 breaks that release's dependency. The script also disables a
transitive HA test dependency's optional metadata download. The offline tests also
run without Home Assistant installed:

```bash
python -m unittest discover -s tests -v
```

The brand icons in `custom_components/heating_mpc/brand/` are drawn by
`python scripts/make_brand_icons.py` (uses Pillow from the test requirements);
the package check verifies their sizes and transparency.

HA tests cover setup, config/options forms (including the reset), migration,
reload (a single relay listener and polling coordinator after repeated reloads),
unload, reload and removal while an optimization or a save is still running,
entity cleanup, scheduled sampling and poll cancellation, the relay listener, stored model
restoration, retention and deletion, missing inputs, manual off/presets, optimizer
failure and Recorder exclusions. A fixture rejects all HA service calls throughout
each test, and a static test fails if actuator service calls are added to the
source. HA Core runs locally with simulated input entities and mocked storage I/O;
this does not validate HAOS, physical devices, disk failures or full process restarts.

`.gitea/workflows/tests.yml` runs the same checks for both HA versions on pushes
and pull requests, on the dedicated `bs-node` Actions runner label
(`bs-node:docker://node:22-bookworm`). Repository Actions must be enabled, and the
runner must support the checkout and setup-uv actions. CI uses pinned uv 0.9.6 to
create a Python 3.13 virtual environment from standalone Python builds;
`setup-python` cannot provision this version in the Debian Bookworm job image.
Action caching is disabled to avoid requiring GitHub cache services on Gitea.
Dependencies, Python builds and action downloads need internet access in CI;
model inference does not. Remote CI execution has not yet been verified; local
validation used macOS.