heating-mpc-ha/docs/MODEL.md

# Heating MPC thermal model

This document describes the model used from version 0.4.0: what it computes, the
units and meaning of every coefficient, when each part learns, what the forecasts
assume, and how the model is validated. It is observational: nothing here controls
the boiler.

## Time step and inputs

The model works in 5-minute steps, the polling interval. For each zone *i*, each
poll provides:

| Symbol | Meaning | Unit | Source |
|---|---|---|---|
| T_i | Zone temperature: sensor-weighted mean of its usable sensors | °C | zone sensors |
| T_out | Outdoor temperature (first fresh, valid source) | °C | weather sources |
| lag | Radiator-lag proxy, 0 (cold) to 1 (fully warm); 20-minute first-order response to the relay, integrated at the relay's actual switch times | – | relay state + read-only listener |
| q_i | Electrical heat above baseline after its thermal response | kW | heat sources |
| O_i | Sum of open windows' opening factors, weighted by the share of the interval they were open | – | window sensors |
| F_i | Same, each window also weighted by cos(angle between wind direction and the window's facing) | – | windows + wind direction |
| v, g | Wind speed and gust speed | m/s | weather sources |
| R_i | Recovery weight after the zone's last window closed: exp(−t / τ_rec), 0 below 0.05 | – | window sensors |
| T_ref,i | Zone temperature at the poll before the window opened | °C | zone sensors |

## Equations

Per 5-minute step, the predicted change of zone *i* is the sum of layers:

```text
ΔT_i = loss    × (T_out − T_i)        envelope heat loss
     + heat    × lag                  boiler, through the radiators
     + bias                           constant gains and losses
     + internal × q_i                 electrical heat above baseline     (extended layer)
     + Σ_j couple:j × (T_j − T_i)     exchange with adjacent zones       (extended layer)
     + vent_natural  × O_i (T_out − T_i)                                 (ventilation layer)
     + vent_wind     × O_i v (T_out − T_i)
     + vent_wind_dir × F_i v (T_out − T_i)
     + vent_gust     × O_i max(0, g − v) (T_out − T_i)                   (optional)
     + vent_cross    × O_i O_other (T_out − T_i)                         (optional)
     + recovery      × R_i (T_ref,i − T_i)
```

All coefficients are in °C per 5 minutes per unit of their term. They are
statistical estimates, not measured physical properties: `heat` is not boiler
efficiency, and a relay that is on does not prove the burner is firing.

In conceptual continuous-time form this corresponds to

```text
C_i dT_i/dt = UA_i (T_out − T_i) + Q_boiler + Q_internal + Q_ventilation + Σ_j K_ij (T_j − T_i) + Q_other
```

divided through by the zone's effective heat capacity C_i and discretized to 5
minutes. The model is first-order (one temperature per zone). Thermal inertia
enters through the radiator lag, the electrical response filter and the recovery
term; a separate wall-mass state is not modeled (see Limitations).

### Base layer

The base layer is the 3-parameter model of earlier versions (loss, heat, bias),
learned per zone. It is always kept: it is the fallback whenever another layer's
inputs are unknown, and the comparison that shows whether the other layers help.

### Extended layer

Present when a zone has an enabled heat source or an adjacent zone. It learns all
base terms plus `internal` and `couple:<zone>` and replaces the base layer in
forecasts once it is as trained as the base layer needs to be (200 samples, 30 with
the relay on, 30 off). If any of its inputs is unknown, the base layer is used.

### Electrical heat

For each enabled source:

```text
effective power  = measured W − measured W of enabled sources included in it   (≥ 0)
baseline         = 20th percentile of the last 24 h of effective power, or the manual value
variable heat    = heat fraction × (effective power − baseline) / 1000          (kW)
thermal response : r ← r + (1 − exp(−5 min / τ)) × (variable heat held during the interval − r)
```

q_i is the sum of the responses of the zone's sources. Centring on the baseline
matters: the persistent load (for example a UPS drawing 200 W all day) is already
part of the bias term, so feeding it again would count it twice. The baseline
needs one hour of data; after startup or an outage a source's response must run
for two time constants before it is used for learning.

Only active power counts (W, kW, mW). VA (apparent power) and % (UPS load) are
rejected. A UPS output sensor does not include the UPS's own losses; if they matter,
measure the input side or lower the heat fraction accordingly.

### Ventilation layer

Present when a zone has an enabled window. Learned separately, from the part of
the measured change the closed-window layers cannot explain, only while a window is
open or the zone is recovering, and only while every window state in the zone is
known. While a window is open, possibly open (its sensor is unknown and it was last
seen open) or the zone is recovering, the base and extended layers do **not**
learn, so window events cannot inflate `loss` or shift `bias`.

An unknown window state is never read as closed, and an opening or closing counts
as confirmed only when it was seen: the previous reading was the other state with
no unknown period (or restart) in between. If a window closes while its sensor was
unknown, the event's duration is reported as a range and the recovery term is not
learned from it, because the closing time is not known. Last known states, open
events and recovery are saved and restored across restarts, as unknown until the
sensor reports again.

An input that a term needs is never filled in for learning or for Model Error:
while a window is open, the wind, its direction (for `vent_wind_dir`) and the other
zones' windows (for `vent_cross`) must be known. With every window closed, the
opening terms are 0 and need no weather. Forecasts, which must produce a number,
assume calm air for a missing wind input.

`vent_wind_dir` is bounded by `vent_wind` (|vent_wind_dir| ≤ vent_wind), so the
combined wind term never goes negative: a leeward window still ventilates.

Wind direction is meteorological: the direction the wind comes **from**, degrees
clockwise from north. A window's azimuth is the direction its outside faces. Wind
from the azimuth blows straight at the window (cos = 1); wind from behind the
building gives −1. Angle differences wrap around north correctly.

## Priors, bounds and confidence

| Coefficient | Prior | Bounds | Prior worth |
|---|---|---|---|
| loss | 0.002 | 0 … 0.04 | as in earlier versions |
| heat | 0.045 | 0 … 0.25 | as in earlier versions |
| bias | 0 | −0.06 … 0.06 | as in earlier versions |
| internal | 0 | 0 … 0.5 | 20 samples at 0.4 kW |
| couple:j | 0 | 0 … 0.05 | 200 samples at 1 °C difference |
| vent_natural | 0 | 0 … 0.2 | 20 samples at opening 1, 15 °C difference |
| vent_wind, vent_wind_dir | 0 | 0 … 0.03 / ±vent_wind | 20 samples at 4 m/s |
| vent_gust | 0 | 0 … 0.03 | 20 samples |
| vent_cross | 0 | 0 … 0.1 | 100 samples |
| recovery | 0 | 0 … 0.5 | 20 samples at 0.5 °C deficit |

New effects start at 0: no effect is assumed until the data shows one. Learning is
recursive least squares with exponential forgetting (about 3.5 days of samples),
a covariance bound that stops windup during long periods without excitation, and
outlier rejection above 0.8 °C per 5 minutes.

Each coefficient reports:

* `confidence`: the share of its information that comes from data rather than the
  prior (0 = prior only), from its posterior variance. It is not a sample count:
  it grows only while the data varies the coefficient's input independently of
  the others, and with forgetting it falls again during long periods without such
  variation. The base layer reports it too (`loss`, `heat`, `bias`);
* `std`: its standard error, using a running estimate of the unexplained noise;
* `samples`: how many accepted samples actually exercised it;
* `identified`: at least 30 exercising samples and confidence ≥ 0.8.

The base layer has no per-coefficient sample counts or standard errors; its
`confidence` alone shows identification. In the tests, two weeks of samples with
the relay always off and a steady indoor-outdoor difference leave `heat` at 0 and
`loss` and `bias` below 0.8, while four days with the relay cycling and the
weather changing bring all three above 0.8.

A coefficient can move without being identified. Example from the tests: two rooms
whose temperature difference stays fairly steady cannot separate exchange between
them from each room's bias; the learner finds a positive coupling but correctly
reports it as not identified.

## Learning conditions

A zone learns from the interval between two polls only when:

* the polls are 4.5–5.5 minutes apart, the relay held one state throughout and the
  radiator-lag estimate is known within 0.1 (for example ~32 minutes after an HA
  restart);
* the outdoor source did not change and the zone used the same sensors at both
  polls;
* for the extended layer, every source in the zone is settled and every adjacent
  zone was usable;
* for the ventilation layer, every window state in the zone is known.

No experiments are run and the boiler is never switched to gather data: everything
is learned passively.

## Configuration changes

Every learned component stores a fingerprint of the configuration it was learned
under (`custom_components/heating_mpc/compatibility.py`). At startup, components
whose fingerprint changed are reset; the rest are kept.

| Change | Effect |
|---|---|
| A zone's sensors, or the ratio of their weights | Zone recalibrated: coefficients kept as a starting point; uncertainty, counts and accuracy statistics reset. Its extended and ventilation layers start over too (their loss, heat and bias values kept as a starting point) |
| Scaling all of a zone's weights equally | Nothing (the normalized measurement is unchanged) |
| A heat source's sensor, heat share, baseline, response time or "included in" | `internal` of its zone; the source's baseline samples if its effective power changed |
| A neighbour's sensors or weights | `couple:<neighbour>` |
| A window's sensor, polarity or opening factor | that zone's ventilation terms; `vent_cross` of the other zones |
| A window's facing direction | `vent_wind_dir` |
| Recovery time constant | `recovery` |
| Comfort weight, target offset, names | Nothing (not physical) |

Models saved before fingerprints existed (0.4.0) are assumed to match the
configuration they are loaded with, and are fingerprinted at once. Every reset is
logged in `model_changes` with its reason.

## Forecasts

Forecasts simulate all zones together, step by step, over the MPC horizon (60–180
minutes):

* **Weather:** held at current values.
* **Electrical heat:** the variable part fades towards baseline with a configurable
  half-life (default 30 minutes; 0 = stops now), and the room keeps following it with
  its response time. A gaming PC drawing 600 W now is not assumed to run on for two
  hours.
* **Windows:** an open window is assumed to stay open for a configurable time
  (default 15 minutes), then close; the zone then recovers. A window whose sensor
  is unknown but was last seen open is assumed open only until that time after it
  was last seen, never renewed. One last seen closed, or never seen since setup, is
  assumed closed, and the forecast is flagged as uncertain (`uncertain_windows`).
* **Relay:** the candidate plan being evaluated.

For the plan search the step is compiled into exact per-zone coefficients (every
term is linear in the temperatures and the lag), which keeps the 180-minute search
fast on a Raspberry Pi.

Plan forecasts are **counterfactual** while Generic Thermostat controls the relay:
they describe what would happen under the MPC's proposed plan.

## Validation

Two separate measurements, because they answer different questions.

**Model accuracy (Model Error sensors)** scores the model, not a forecast. At
every poll the model parameters are frozen; 15, 30, 60 and 120 minutes later all
zones are replayed from that poll using the relay, electrical heat and window
history that actually followed (outdoor temperature held at its start value),
and compared with the readings. The base layer and a no-change forecast are
scored on exactly the same spans:

* `mae`, `rmse`, `bias`: the full model's error;
* `persistence_mae`, `skill`: against assuming no change;
* `base_model_mae`, `improvement_vs_base`: against the base layer alone. A feature
  that does not improve this should be switched off (for example the gust term).

A span counts only if every input the zone's model used was observed at every
step (see *Ventilation layer*; for the extended layer also electrical heat and
neighbours, whose own replay must be complete). Otherwise it is counted as
`incomplete`, with the base model's error on it kept apart
(`incomplete_base_model_mae`), since the base model needs only the relay and the
outdoor temperature. Spans that cannot be replayed (uncertain radiator lag, no
reading at the end) are counted as `skipped`.

**Forecast accuracy (Forecast Error sensor)** scores forecasts as published: the
start readings, trajectory, plan, model revision and assumptions of the moment
they were made, nothing replaced with hindsight. Because the plan is the MPC's
proposal while Generic Thermostat controls the relay, a forecast is compared only
when the relay followed it. The deviation is the time, up to the horizon, the
relay spent in another state than planned, compared block by block from the
relay's own switch times; unobserved relay time counts as possible deviation.

| Deviation (worst case) | Class | Used for |
|---|---|---|
| ≤ 2 min | matched | Forecast Error figures |
| ≤ 5 min, ≤ 2 min unobserved | approximate | `approximate_stats` only |
| > 5 min observed | counterfactual | counted, never an error |
| otherwise | incomplete | counted, with a reason |

Forecasts made with an assumed input (an unknown window or electrical heat) are
incomplete as well. The limits follow from the model: d seconds of unplanned
boiler heat change a room by about heat × d / 300 with the learned `heat`
coefficient (typically 0.05–0.12 °C per 5 minutes), so 2 minutes stay below half a
0.1 °C sensor step and 5 minutes are about one step.

Both report how much evidence there is. Forecasts start every 5 minutes, so at
horizon h consecutive forecasts share most of their span, and all zones share one
boiler: `effective_samples = samples × 5 / h / zones`. Below 5 the quality is
`insufficient` and the sensor's state stays unknown; below 30 it is
`indicative`.

Results on synthetic apartments (the tests; these check the method, not real-world
accuracy):

| Scenario | 60-min error, full | base | no change |
|---|---|---|---|
| Office, UPS 200 W + 400 W gaming sessions, thermostat cycling | 0.031 °C | 0.182 °C | 0.40 °C |
| Bedroom, 15–20 min ventilation every 4 h, wind and recovery | 0.026 °C | 0.218 °C | 0.52 °C |
| Two adjacent rooms, true exchange 0.01 | 0.024 °C | 0.037 °C | 0.29 °C |

In the ventilation scenario, the same data without the window configured produced
a −0.013 °C/5 min cooling bias and an 11 % inflated boiler coefficient, which is
the contamination the separate ventilation layer prevents.

## Limitations

* One temperature per zone: walls and air are not separate states. The recovery
  term approximates the air warming back after ventilation.
* Response times (electrical heat, recovery) are configured, not learned. Compare
  settings with `improvement_vs_base`.
* Coupling between zones heated by one boiler is weakly identifiable; it is shrunk
  towards 0 and usually reported as not identified.
* Natural and wind ventilation terms both scale with the opening; with little wind
  variation their split is uncertain even when their combined effect is accurate.
* Forecasts hold weather constant and use one scenario for electrical heat and
  windows; no forecast uncertainty band is computed yet.
* Window sensors with a permanently unknown state keep their zone out of learning
  and its Model Error spans incomplete; disable such a window.
* A window that is unknown and was last seen closed (or never seen) is assumed
  closed in forecasts, flagged as uncertain; no open/closed scenario pair is
  computed.