heating-mpc-ha/README.md

# Heating MPC 0.3.0 — custom integration for Home Assistant OS

**Experimental, observation-only (shadow mode).** It **never** changes the Generic Thermostat, the physical relay, the boiler's setpoint, `heat/off`, or Comfort/Away/Sleep. No Docker add-on, MQTT or InfluxDB connection is needed.

This project targets Home Assistant OS on a Raspberry Pi 5, with one existing ON/OFF central-heating relay, `generic_thermostat`, and individual indoor temperature sensors.

## Install / upgrade

1. Create a Home Assistant backup.
2. Copy the entire folder `custom_components/heating_mpc` into `/config/custom_components/heating_mpc` (replace its contents when upgrading; read the upgrade notes in [CHANGELOG.md](CHANGELOG.md) first).
3. Restart Home Assistant **Core**.
4. If new: **Settings → Devices & services → Add integration → Heating MPC**. If installed already: select **Heating MPC → Configure** (Options).
5. Choose the existing thermostat and physical heating switch (read-only), then individual room sensor entities.
6. Choose *ordered lists* of **outdoor sensor entity IDs**. First valid sensor is active; if missing, unavailable, nonnumeric or outside reasonable bounds, the next valid sensor is used automatically.
7. Assign a relative comfort weight (0.1–10) to each selected room.

On Home Assistant 2026.3 or newer the integration shows its own icon (a radiator with a forecast curve) in **Devices & services**; older versions show a generic placeholder.

**No entity IDs are hard-coded.** Every room and outdoor measurement can be changed later via **Configure**, without editing YAML or Python.

## Reconfigurable weather inputs

| Measurement | Required? | Used in the thermal model? |
|---|---|---|
| Outdoor temperature | **Yes** (1–10 prioritized sensors) | **Yes**: estimates thermal losses; constant over the 2 h forecast window |
| Wind speed | No | No, diagnostics/source comparison only |
| Cloud cover | No | No, diagnostics/source comparison only |
| Humidity | No | No, diagnostics/source comparison only |
| Feels-like temperature | No | No, diagnostics/source comparison only |
| Atmospheric pressure | No | No, diagnostics/source comparison only |
| Visibility | No | No, diagnostics/source comparison only |
| Rain / snow | No | No, diagnostics/source comparison only |

Choose multiple sensors for a single variable, e.g. OpenWeatherMap then another provider, in priority order. Switch order in **Configure** to run A/B comparisons on successive days, or simply let the second sensor act as fallback. The active source and all current alternative readings are visible in the **Heating MPC Status** entity attributes (`weather`, `weather_active_sources`, `weather_comparison`). The **MPC Outdoor Temperature** sensor also has an `active_source` attribute and an `alternatives` map.

Measurements are normalized where possible: outdoor temperatures to °C; wind to m/s; pressure to hPa; visibility to meters. Unknown units or values outside plausibility bounds are rejected in favor of the next selected sensor. Clouds and humidity are expected in %, rain/snow in mm or mm/h as represented by their upstream sensor. A fallback source may have a different sampling delay; configurable report-age limits now reject stale inputs (see below).

Switching only the **weather-provider sensors** does not reset learned thermal parameters. The update is applied after the integration reloads, and it skips learning one transition when the primary temperature source changes, avoiding a jump in the training series. Nevertheless, two sensors with a persistent offset can bias predictions; compare their readings and validate the model again before relying on them. Replacing the physical heating switch discards incompatible model parameters. Existing room-specific parameters are retained if room entity IDs remain unchanged.

## Model, sampling and output

- Every 5 minutes: reads HA sensor states and observed relay state; learns a per-room linear thermal model `dT = a*(T_out-T_room) + b*effective_heating + c`.
- Learns only from valid 5-minute transitions; uses a smoothed heating-demand proxy to approximate radiator lag. Relay changes are timestamped by a read-only state listener, so the lag estimate follows every switch between polls.
- After at least 200 transitions and samples with both heating states, runs an ON/OFF MPC search for the next 60–180 minutes (default 120) in 15-minute command blocks. Protects against rapid simulated toggling.
- Uses room comfort weights and the **existing** Generic Thermostat's target and heat/off state. **The simulated ON/OFF demand never operates hardware**.
- Persists model parameters using HA's `.storage` so HA restarts do not discard the trained data. Models of rooms removed in Configure are kept (up to 24) and restored if the room is added back.

Entities (entity IDs may vary):

- `sensor.heating_mpc_status` — learning/ready/paused status and diagnostics, including outside weather, active source, all valid alternative weather values, model coefficients, proposed 15-minute plan.
- `sensor.heating_mpc_weighted_temperature` — own room-weighted average, informative only.
- `sensor.mpc_outdoor_temperature` — selected external temp and alternatives.
- Per-room predicted temperatures after 60 minutes. Attributes `forecast_15min` (the
  temperature every 15 minutes over the horizon) and `plan_15min` (the proposed
  ON/OFF plan). These assume the MPC's plan is executed; while Generic Thermostat
  controls the relay they are **counterfactual**.
- `binary_sensor.heating_mpc_proposed_heat` — **hypothetical** heating demand (unavailable until ready).
- **Heating MPC Learning Progress** — how close the least trained room is to being
  used for forecasts (100 % = ready); attributes show the sample counts and what is
  required.
- **Heating MPC Last Sample** — what the last poll did for learning (for example
  *Learned*, *Relay switched*, *Waiting for radiators to settle*); its history shows
  when learning happens.
- **Heating MPC Forecast Error 15/30/60/120 min** — how far the thermal model's
  forecasts were from what actually happened (see *Forecast accuracy*).

## Known limitations / safety

- **Do not** connect the proposed binary sensor directly to the heating switch or build an automation from it. A production controller would need watchdog, manual override, safety handling, and an exclusive relay owner with Generic Thermostat fallback.
- The other meteorological variables besides temperature are intentionally **captured but not yet fitted into the thermal model**. This allows source tests without falsely treating weak inputs as validated improvements. Later versions can test whether wind or solar proxy reduces held-out forecast error.
- All rooms share one heating actuator. The model does not command individual radiators.
- No weather forecast: assumes outdoor temperature remains constant during simulated horizon.
- No Recorder/InfluxDB training backfill. It learns only from samples collected after installation.
- The demand switch is not a burner-flame or real gas consumption measurement.
- Not yet tested on HAOS or with real devices. Automated tests run the integration in a Home Assistant Core test harness (HA 2025.12.0 and 2026.2.3) with simulated entities.
- Home Assistant Core APIs and config-flow selectors can change; take a backup before trying this experimental custom integration.

## Diagnostics

Start with **Learning Progress** and **Last Sample**. For details, open the
**Heating MPC Status** entity's attributes. In recent Home Assistant versions
Developer Tools is at the bottom of **Settings** (or press Cmd/Ctrl+K and type
"developer"); its **States** tab lists every attribute.

**Why learning starts about 30 minutes after a restart.** The model learns how
much the radiators warm each room, so it needs to know how warm the radiators are.
It estimates this from when the relay last switched (20-minute time constant).
After a Home Assistant restart that time is unknown, so learning waits until the
estimate is within 10 % (*Waiting for radiators to settle*), about 32 minutes.
Forecasts continue meanwhile. Reloading the integration usually does not cause this
wait, because the relay's last change is still known. *Ready* then needs 200
accepted samples (about 17 hours) per room, including 30 with the relay ON and 30
with it OFF, which can take a few days depending on how often the boiler runs.

The **Heating MPC Status** entity's state is one of `waiting_for_sensors`,
`learning`, `paused` (thermostat not in `heat`), `waiting_for_target`,
`invalid_weights`, `ready`, `inputs_changed` or `prediction_error`. Proposed heat
is available only while it is `ready`. Useful attributes:

- `input_quality` — for each room and each weather source (grouped by metric):
  `status` (`valid`, `missing`, `unknown`, `unavailable`, `stale`,
  `invalid_timestamp`, `future_timestamp` or `invalid_value_or_unit`),
  `age_seconds` (time since the sensor last reported to HA) and
  `max_age_seconds_24h` (largest age seen at polls in the last 24 hours; resets on
  reload and can read up to one poll, 5 minutes, low). Rooms also have `used`:
  whether that room's reading was used in this poll.
- `excluded_rooms` and `weight_coverage` — rooms left out of this poll, and the
  share of the total comfort weight still covered by the rooms used (1.0 = all).
- `heat_lag` and `heat_lag_uncertainty` — the radiator-lag estimate (0 cold, 1 fully
  warm) and its error bound. Learning waits while the bound is above 0.1, for
  example for about 30 minutes after an HA restart.
- `sample_result` — why the last poll was or was not learned: `first_sample`,
  `learned`, `partly_rejected` (an outlier in some room), `interval_out_of_range`,
  `relay_changed`, `outside_source_changed` or `lag_uncertain`.
- `models` — per-room coefficients, accepted sample counts and the in-sample
  5-minute error (`mae_5m`), which is **not** a forecast accuracy.
- `retired_room_models` — removed rooms whose learned models are kept.

`input_quality` and `weather_comparison` change every poll, so they are not
written to the Recorder; the other attributes are.

### Choosing freshness limits

Room and weather report-age limits are set during setup and in **Configure**
(5–1440 minutes; defaults 30 for rooms and 120 for weather).

A room sensor that is older than its limit, unavailable, missing or invalid is
**left out**: the weighted temperature, learning and forecasts continue with the
other rooms, whose weights are re-normalized. Status shows it in `excluded_rooms`.
Learning and forecasts pause only when no room is usable, or when the relay,
thermostat or outdoor temperature is missing. A room that reports again learns
from its second poll onwards, so nothing is learned across its gap.

The checkbox **Keep using the last value of a room sensor that stopped reporting**
changes this for stale sensors: their last value is still used, for up to 24 hours
without a report. Use it for sensors that report only when the value changes.
Unavailable or invalid sensors are always left out, and so is any sensor silent
for more than 24 hours, since it is most likely dead.

A stale outdoor source falls back to the next one in its list; optional weather
metrics never block.

Within the limit (or with the checkbox on), the last reading counts as current,
so a frozen value can be learned for a while. A too-tight limit (with the checkbox
off) drops a room whenever its sensor is quietly between reports. Freshness uses
HA's `last_reported`: the time a sensor last reported to HA, even with an unchanged
value. It is not the age of the physical measurement, so a provider re-publishing
cached data looks fresh, and sensors that report only on change can look stale in
a steady room.

After a day of running, set each limit comfortably above (about 1.5–2×) the largest
`max_age_seconds_24h` among the sensors it covers. The relay and thermostat have no
age limit.

## Forecast accuracy

The four **Forecast Error** sensors measure the thermal model against reality,
continuously and out of sample. At every poll the integration records each room's
reading, the outdoor temperature, the radiator-lag estimate and the model's
parameters. 15, 30, 60 and 120 minutes later it replays that room from the earlier
reading using the parameters it had **then**, the outdoor temperature held at its
starting value, and the radiator heat that actually followed from the relay's real
ON/OFF history. The result is compared with the current reading.

This tests the thermal model, not the MPC plan, because the plan is never
executed. Each sensor's state is the mean absolute error (°C) over the last 7 days,
across all rooms. Attributes, all in °C except counts:

- `samples` — number of scored forecasts. Consecutive forecasts overlap, so they
  are not independent; treat small counts as indicative only.
- `bias` — mean error: negative means the model predicts too cold.
- `persistence_mae` — error of simply assuming no change; the baseline to beat.
- `skill` — `1 - mae / persistence_mae`: 1 is perfect, 0 is no better than
  assuming no change, negative is worse. Empty while the temperature did not change.
- `rooms` — the same numbers per room (not written to the Recorder).

Forecasts are scored only where the inputs were known: polls 4.5–5.5 minutes
apart, a usable reading at the start and the end, and a well-known radiator-lag
estimate throughout. A missing relay, outdoor temperature or all rooms restarts the
replay window, so scores resume 15–120 minutes later. Statistics are kept with the
model and survive restarts; deleting the model resets them.

What it does not cover: outdoor temperature changes during the horizon (held
constant, as in the MPC), and periods when the model was still learning (included,
so early numbers are pessimistic). The roadmap's working target is about
0.3–0.4 °C at 60 minutes with clearly positive skill, before any control is
considered.

## Release notes

See [CHANGELOG.md](CHANGELOG.md) for changes and upgrade notes per version, and
[AUDIT.md](AUDIT.md) for audit findings and test evidence.

## Developer checks and Forgejo/Gitea Actions

The HA Core test suite runs against **Home Assistant 2025.12.0** (the minimum in
`hacs.json`, `requirements-test.txt`) and **2026.2.3** (newest published test
fixtures, `requirements-test-latest.txt`), both on Python 3.13. Use a dedicated
development virtual environment; these test dependencies are not installed with
the integration and do not change its local, standard-library model inference.

```bash
python3.13 -m venv .venv
. .venv/bin/activate
python -m pip install -r requirements-test.txt   # or requirements-test-latest.txt
sh scripts/check.sh
```

The check script runs dependency consistency checks, the offline tests, the HA
Core tests, compilation, package metadata, translation and changelog checks, and
diff whitespace checks. The 2025.12.0 requirements pin a compatible DNS library
because pycares 5 breaks that release's dependency. The script also disables a
transitive HA test dependency's optional metadata download. The offline tests also
run without Home Assistant installed:

```bash
python -m unittest discover -s tests -v
```

The brand icons in `custom_components/heating_mpc/brand/` are drawn by
`python scripts/make_brand_icons.py` (uses Pillow from the test requirements);
the package check verifies their sizes and transparency.

HA tests cover setup, config/options forms, migration, reload and entity cleanup,
scheduled sampling and poll cancellation, the relay listener, stored model
restoration, retention and deletion, missing inputs, manual off/presets, optimizer
failure and Recorder exclusions. A fixture rejects all HA service calls throughout
each test, and a static test fails if actuator service calls are added to the
source. HA Core runs locally with simulated input entities and mocked storage I/O;
this does not validate HAOS, physical devices, disk failures or full process restarts.

`.gitea/workflows/tests.yml` runs the same checks for both HA versions on pushes
and pull requests, on the dedicated `bs-node` Actions runner label
(`bs-node:docker://node:22-bookworm`). Repository Actions must be enabled, and the
runner must support the checkout and setup-uv actions. CI uses pinned uv 0.9.6 to
create a Python 3.13 virtual environment from standalone Python builds;
`setup-python` cannot provision this version in the Debian Bookworm job image.
Action caching is disabled to avoid requiring GitHub cache services on Gitea.
Dependencies, Python builds and action downloads need internet access in CI;
model inference does not. Remote CI execution has not yet been verified; local
validation used macOS.