Fix shadow observer lifecycle, learning and prediction correctness

AUDIT.md

@@ -0,0 +1,93 @@
+# v0.2.1 source audit — 2026-10-26
+
+Scope: repository implementation, offline tests and documented HA APIs. No HAOS,
+physical relay, thermostat, deployment or gas-consumption testing was performed.
+The manifest remains 0.2.1; these are unreleased reliability fixes, not a release
+acceptance declaration. Milestone 0 remains open.
+
+## Findings and repairs
+
+| Area | Baseline finding | Repair / evidence |
+|---|---|---|
+| Lifecycle, high | Config flow VERSION=2 had no migration hook; the weather helper's legacy fallback alone cannot migrate HA entries. | Migrate v1 data and options independently through HA's update API, retaining options precedence and existing prioritized selections. Reject newer config versions. |
+| Lifecycle, medium | Removed room forecast entities remained in the registry after options reload. | Remove only obsolete room forecast IDs for this integration and entry; retain all other entities and compatible IDs. |
+| Lifecycle, medium | Raised platform setup left the coordinator in hass.data. | Remove that reference on failure. HA's own partially loaded platform cleanup still requires real runtime testing. |
+| Persistence, high | Malformed roots/counters could crash restore; finite but nonphysical coefficients and invalid covariance could be accepted. | Validate shapes, coefficient bounds, symmetric positive-definite covariance, integer counts and finite error totals; reset malformed rooms atomically. |
+| Learning, high | Global ON/OFF counts increased even when fitting rejected outliers and could qualify a newly added room using other rooms' history. | Persist accepted ON/OFF counts per room; readiness requires every room to qualify. Retain compatible legacy coefficients/counts but collect fresh excitation evidence. |
+| Learning, high | A 4–7 minute interval was treated as exactly 5 minutes, including intervals containing relay transitions. | Accept 270–330 seconds, normalize temperature change to five minutes, compute lag using actual elapsed time; reject gaps/backward time, source changes and relay changes since the previous poll (including on/off round trips). |
+| Inputs, high | Thermostat targets were interpreted as Celsius even in a Fahrenheit HA installation. | Normalize using thermostat temperature_unit when present, otherwise HA's configured temperature unit. |
+| Inputs, medium | Invalid weights or malformed unit metadata could cause crashes/nonfinite arithmetic. | Reject invalid weights with unavailable proposal; reject non-string units as unusable sources. |
+| Prediction, high | Inputs could change while saving or optimizing, yet the old plan could be published as ready. | Compare captured HA State snapshots before publishing; discard results when any watched state changes. |
+| Prediction, medium | A 30/45-minute pure optimizer horizon returned its endpoint as a 60-minute forecast. | Return no 60-minute forecast for horizons below 60. UI already limits horizons to at least 60. Replay tests check the actual +60 and endpoint values. |
+| Prediction, medium | NaN weights/lag and negative/nonfinite optimizer parameters were not rejected consistently. | Add finite/range validation and regression tests. |
+
+The initial 10 tests passed. The expanded suite contains 33 offline tests covering
+model/weather logic plus coordinator and integration entry points using small HA
+boundary doubles. The doubles check our calls and state handling; they do not
+exercise HA scheduling, Store internals, entity registration or actual config forms.
+The test service-call spy raises if invoked. Source inspection found no actuator
+service calls, service registration, relay automations or network model inference.
+
+## Remaining gaps, in priority order
+
+1. **Freshness and alignment:** numeric but stale room/outdoor states are still
+   accepted. Define configurable freshness limits and reporting timestamps in the
+   UI, distinguish unchanged values from missing reports, expose rejection reasons,
+   and test fallback/recovery. Missing units still assume canonical units for
+   compatibility. Optional weather source lists lack uniform maximum-size checks.
+2. **Real HA test harness and HAOS acceptance:** config/options form submission,
+   reload listener/poller teardown, disabled entities, partial platform setup failure,
+   actual storage failures, restart/uninstall and registry cleanup need tests against
+   a declared supported HA version. Add CI, translation/schema and packaging checks.
+   Existing en/ro flow strings are unchanged by this patch.
+3. **Sampling and identification:** polling detects a relay change through
+   last_changed and skips that interval; it does not integrate the relay duty cycle.
+   Restart/gaps reset the lag proxy to the current command, which is only an
+   approximation. Use timestamped relay events and aligned sensor observations
+   before claiming robust radiator identification. Rejecting transition intervals
+   can slow learning. The jitter correction is a local linear approximation.
+4. **Persistence and identity:** Store envelope version stays 1; the additive model
+   payload is schema_version=2. Legacy payloads remain readable. Unsupported payload
+   schemas are ignored; comprehensive future-version downgrade protection remains
+   open. Renamed room entity IDs are treated as replacements (fresh model), not
+   automatically matched to entity-registry identity. Same-ID physical sensor
+   relocation and systematic weather provider offsets remain undetected.
+5. **Forecast validity:** forecasts remain counterfactual MPC plans with constant
+   outdoor temperature. Five-minute residual MAE is not held-out multi-horizon
+   accuracy. No actual-policy forecast, recorded trajectories, confidence estimate,
+   identifiability test or persistence benchmark exists. Prediction updates remain
+   five-minute polls; a later manual change does not instantly refresh the diagnostic.
+6. **Resource use:** optimization is offloaded, but no Pi 5 benchmark exists.
+   Model state is written on each eligible interval (roughly 288 writes/day).
+   Rejected-fit reasons and aggregate failure counters are not yet exposed.
+
+## Compatibility and safe trial
+
+Back up HA before installing. Replace the component directory and restart Core;
+keep Generic Thermostat as the only relay writer. Compatible room coefficients and
+forecast unique IDs remain stable. Older saved models need at least 30 newly
+accepted ON and 30 OFF samples per room before becoming ready again, plus the
+existing 200-total-sample condition. No artificial heating cycles are generated.
+
+Observe status, normalized target, per-room model counts and weather provenance.
+Test manual heat/off and preset changes, a missing sensor, an options reload, room
+removal and a restart. The proposed-heat entity must remain diagnostic only. Use a
+disposable config entry for uninstall testing because deletion removes its models.
+For rollback, restore the component and HA backup together; older code ignores the
+new per-room counters and retains its original defects. No installation was done
+as part of this audit.
+
+## Verification commands
+
+- `python3 -m unittest discover -s tests -v` — 33 tests pass (Python 3.9.6).
+- `PYTHONPYCACHEPREFIX=/tmp/heating-mpc-audit-pycache python3 -m compileall -q custom_components/heating_mpc` — passes.
+- `git diff --check` — passes.
+
+The initial unredirected compile check failed because macOS's Python cache path
+was outside the writable sandbox; the temporary-cache rerun succeeded. No full HA
+suite, type checker, linter, HAOS or physical-device checks were run.
+
+API references used to check migration and entity-registry calls:
+[HA config entries](https://developers.home-assistant.io/docs/config_entries_index/),
+[config migration](https://developers.home-assistant.io/docs/config_entries_config_flow_handler/#config-entry-migration),
+[entity registry](https://contributors.home-assistant.io/core/integrations/entity-registry/).

README.md

@@ -73,3 +73,28 @@ python -m unittest discover -s tests -v
 - Corrects the manifest classification from `helper` to `service`. Existing config entries should move from the Helpers listing to **Settings → Devices & services → Integrations** after replacing the files and restarting Home Assistant Core. No deletion/reconfiguration is needed to upgrade.
 - When the integration config entry is deleted from the Integrations page, Home Assistant removes its registered entities, and `async_remove_entry` removes the locally persisted learned-model data (`.storage/heating_mpc.<entry_id>`). The custom component source folder remains on disk until manually removed.
 - MPC behavior is otherwise unchanged and remains read-only / shadow-only.
+
+## Unreleased: v0.2.1 reliability audit
+
+See [AUDIT.md](AUDIT.md) for findings, fixes, test evidence and remaining gaps.
+No active control was added; the manifest version has not been advanced.
+
+- Fix config-entry v1 migration, obsolete room forecast cleanup and coordinator
+  cleanup after a raised platform setup error.
+- Validate saved models atomically and add an explicit payload schema version.
+  Compatible legacy coefficients survive; readiness now requires at least 30 new
+  accepted ON and 30 OFF samples **per room** when loading legacy models without
+  per-room counters. The existing 200-total-sample requirement also applies.
+- Reject intervals containing relay changes or outside-provider changes. Accept
+  only 270–330-second intervals and normalize small timing differences. This can
+  slow learning; it never generates extra heating cycles.
+- Normalize thermostat targets to Celsius; reject invalid weights and optimizer
+  parameters. Status can report `invalid_weights` or `inputs_changed`; neither
+  publishes a proposal. Input changes during calculation discard the prediction.
+- Model diagnostics include each room's accepted total/ON/OFF sample counts.
+  Predictions remain **counterfactual**: the proposed plan is not the actual
+  Generic Thermostat's future command sequence.
+
+Offline lifecycle tests use HA doubles. Freshness policies, actual HA/HAOS lifecycle
+validation and held-out prediction accuracy remain unfinished. Back up before
+trying these changes; follow the safe trial and rollback notes in AUDIT.md.

ROADMAP.md

@@ -39,7 +39,7 @@ Current behavior: **shadow mode only**. `generic_thermostat` remains the one and
 
 **Objective:** Make the existing observer trustworthy before extending it.
 
-- [ ] Perform a source audit of v0.2.1: pure model, optimizer, config/options, coordinator update timing, persistent model and entity lifecycle.
+- [x] Perform a source audit of v0.2.1: pure model, optimizer, config/options, coordinator update timing, persistent model and entity lifecycle.
 - [ ] Add Home Assistant test harness coverage for initial setup, update, restart/reload, options changes, missing sensors, `heat/off`, target changes, teardown and uninstall.
 - [ ] Verify that **zero physical-device service calls** occur in observation mode, including restart and failed update cases.
 - [ ] Establish deterministic handling of 5-minute transitions, polling jitter, timestamp discontinuities, missing data and real switch transitions between samples.
@@ -52,6 +52,20 @@ Current behavior: **shadow mode only**. `generic_thermostat` remains the one and
 
 **Exit criteria:** No control writes; reproducible tests covering HA lifecycle; clean upgrade/removal; clear status during outages; no significant model-data loss or silent corruption on restart.
 
+### Audit update — 2026-10-26
+
+See [AUDIT.md](AUDIT.md). The source audit and focused repairs are complete;
+Milestone 0 acceptance is **not** complete. Offline boundary doubles cover setup,
+reload/unload, migration, removal, corrupt storage, missing inputs and manual off.
+They are not a full HA test harness. Per-room accepted ON/OFF counts, additive
+payload schema v2, conservative relay-transition rejection, jitter normalization,
+Fahrenheit targets and forecast input-change guards are implemented.
+
+Next implement configurable freshness/alignment with diagnostics and a supported
+HA test harness/CI. Then improve event-based sampling and identity/persistence
+migration tests. Only after those reliability checks proceed to Milestone 1's
+recorded forecasts and walk-forward evaluation. Live control remains unauthorized.
+
 ## Milestone 1 — Explainable forecasts and dashboards (target v0.3.0)
 
 **Objective:** Users can verify what the model predicts and whether it is actually accurate.

custom_components/heating_mpc/__init__.py

@@ -5,8 +5,10 @@ from homeassistant.const import Platform
 from homeassistant.config_entries import ConfigEntry
 from homeassistant.core import HomeAssistant
 from homeassistant.helpers.storage import Store
+from homeassistant.helpers import entity_registry as er
 
-from .const import DOMAIN
+from .const import DOMAIN, CONF_OUTSIDE, WEATHER_SOURCES
+from .weather_sources import sources_for
 from .coordinator import HeatingMPCCoordinator
 
 PLATFORMS: list[Platform] = [Platform.SENSOR, Platform.BINARY_SENSOR]
@@ -17,7 +19,20 @@ async def async_setup_entry(hass: HomeAssistant, entry: ConfigEntry) -> bool:
     await coordinator.async_restore_model()
     await coordinator.async_config_entry_first_refresh()
     hass.data.setdefault(DOMAIN, {})[entry.entry_id] = coordinator
+    try:
         await hass.config_entries.async_forward_entry_setups(entry, PLATFORMS)
+    except Exception:
+        hass.data[DOMAIN].pop(entry.entry_id, None)
+        raise
+    # Remove only obsolete room forecasts belonging to this entry/platform.
+    registry = er.async_get(hass)
+    expected = {f"{entry.entry_id}_{room}_predicted_60min" for room in coordinator.rooms}
+    for entity in er.async_entries_for_config_entry(registry, entry.entry_id):
+        if (entity.platform == DOMAIN and
+                entity.unique_id.startswith(f"{entry.entry_id}_sensor.") and
+                entity.unique_id.endswith("_predicted_60min") and
+                entity.unique_id not in expected):
+            registry.async_remove(entity.entity_id)
     entry.async_on_unload(entry.add_update_listener(async_reload_entry))
     return True
 
@@ -37,3 +52,21 @@ async def async_remove_entry(hass: HomeAssistant, entry: ConfigEntry) -> None:
     """Remove the learned model when this integration instance is deleted."""
     store = Store(hass, 1, f"{DOMAIN}.{entry.entry_id}")
     await store.async_remove()
+
+
+async def async_migrate_entry(hass: HomeAssistant, entry: ConfigEntry) -> bool:
+    """Migrate legacy single weather source without changing user selections."""
+    if entry.version > 2:
+        return False
+    if entry.version == 1:
+        def migrate(config):
+            result = dict(config)
+            if CONF_OUTSIDE in result:
+                result[WEATHER_SOURCES["temperature"]] = sources_for(result, "temperature")
+                result.pop(CONF_OUTSIDE)
+            return result
+        hass.config_entries.async_update_entry(
+            entry, data=migrate(entry.data), options=migrate(entry.options),
+            version=2, minor_version=1,
+        )
+    return True

custom_components/heating_mpc/coordinator.py

@@ -4,7 +4,7 @@ from __future__ import annotations
 import logging
 from datetime import datetime, timedelta, timezone
 from functools import partial
-from math import isfinite
+from math import exp, isfinite
 from typing import Any
 
 from homeassistant.core import HomeAssistant
@@ -17,7 +17,7 @@ from .const import (
     CONF_HORIZON, CONF_MIN_ON, CONF_MIN_OFF, CONF_SWITCH_COST, CONF_HEAT_COST,
     CONF_BAND, DEFAULTS,
 )
-from .physics import HEAT_LAG, Learner, optimize
+from .physics import Learner, optimize
 from .weather_sources import choose_weather_sources, sources_for, normalize
 
 _LOGGER = logging.getLogger(__name__)
@@ -40,17 +40,26 @@ class HeatingMPCCoordinator(DataUpdateCoordinator[dict[str, Any]]):
 
     async def async_restore_model(self) -> None:
         stored = await self._store.async_load() or {}
+        if not isinstance(stored, dict) or stored.get("schema_version", 1) not in (1, 2):
+            _LOGGER.warning("Ignoring unsupported Heating MPC model data")
+            return
         # Replacing the actual heating system invalidates heating coefficients.
         # Switching among weather providers does not discard months of learning.
         if stored.get("heater_entity", self.config[CONF_HEATER]) != self.config[CONF_HEATER]:
             _LOGGER.warning("Heating MPC heater changed; starting a fresh thermal model")
             return
         # Models carry room identity, so changing the room list is safe.
-        for room, data in stored.get("learners", {}).items():
+        saved = stored.get("learners", {})
+        if not isinstance(saved, dict):
+            return
+        for room, data in saved.items():
             if room in self.learners:
                 self.learners[room] = Learner.from_dict(data)
-        self._on_samples = max(0, int(stored.get("on_samples", 0)))
-        self._off_samples = max(0, int(stored.get("off_samples", 0)))
+        self._update_counts()
+
+    def _update_counts(self) -> None:
+        self._on_samples = min(m.on_count for m in self.learners.values())
+        self._off_samples = min(m.off_count for m in self.learners.values())
 
     def _value(self, entity_id: str, low: float, high: float) -> float | None:
         state = self.hass.states.get(entity_id)
@@ -71,6 +80,9 @@ class HeatingMPCCoordinator(DataUpdateCoordinator[dict[str, Any]]):
             entity_id for metric in WEATHER_SOURCES
             for entity_id in sources_for(self.config, metric)
         }
+        watched = set(self.rooms) | weather_entity_ids | {
+            self.config[CONF_CLIMATE], self.config[CONF_HEATER]}
+        snapshot = {entity: self.hass.states.get(entity) for entity in watched}
         weather_states = {
             entity_id: (state.state, state.attributes.get("unit_of_measurement"))
             for entity_id in weather_entity_ids
@@ -109,21 +121,21 @@ class HeatingMPCCoordinator(DataUpdateCoordinator[dict[str, Any]]):
         if self._previous is not None:
             dt = (now - self._previous["time"]).total_seconds()
             # Do not fit one transition across a provider/fallback switch.
-            if (240 <= dt <= 420 and
+            if (270 <= dt <= 330 and
+                    heater.last_changed <= self._previous["time"] and
+                    heater_on == self._previous["heater_on"] and
                     self._previous["outside_source"] == active_sources["temperature"]):
-                next_lag = self._heat_lag + HEAT_LAG * (float(self._previous["heater_on"]) - self._heat_lag)
+                next_lag = self._heat_lag + (1 - exp(-dt / 1200)) * (float(self._previous["heater_on"]) - self._heat_lag)
                 for room in self.rooms:
                     self.learners[room].learn(
                         self._previous["temps"][room], vals[room],
-                        self._previous["outside"], next_lag
+                        self._previous["outside"], next_lag, dt, self._previous["heater_on"]
                     )
                 self._heat_lag = next_lag
-                if self._previous["heater_on"]:
-                    self._on_samples += 1
-                else:
-                    self._off_samples += 1
+                self._update_counts()
                 # Persist at each sample so restart does not erase hours of training.
                 await self._store.async_save({
+                    "schema_version": 2,
                     "learners": {room: learner.as_dict() for room, learner in self.learners.items()},
                     "on_samples": self._on_samples, "off_samples": self._off_samples,
                     "heater_entity": self.config[CONF_HEATER],
@@ -138,7 +150,13 @@ class HeatingMPCCoordinator(DataUpdateCoordinator[dict[str, Any]]):
             "outside_source": active_sources["temperature"],
         }
         weights = self.config.get(CONF_WEIGHTS, {})
+        try:
             weights = {r: float(weights.get(r, 1.0)) for r in self.rooms}
+            if any(not isfinite(w) or w <= 0 for w in weights.values()) or not isfinite(sum(weights.values())):
+                raise ValueError
+        except (AttributeError, TypeError, ValueError, OverflowError):
+            status["status"] = "invalid_weights"
+            return status
         weighted_temp = sum(weights[r] * vals[r] for r in self.rooms) / sum(weights.values())
         status.update({
             "weighted_temperature": round(weighted_temp, 3),
@@ -151,6 +169,8 @@ class HeatingMPCCoordinator(DataUpdateCoordinator[dict[str, Any]]):
                 room: {"loss": round(model.theta[0], 6),
                        "heat": round(model.theta[1], 6),
                        "bias": round(model.theta[2], 6),
+                       "samples": model.count,
+                       "on_samples": model.on_count, "off_samples": model.off_count,
                        "mae_5m": round(model.mae, 4) if model.mae is not None else None}
                 for room, model in self.learners.items()
             },
@@ -159,8 +179,10 @@ class HeatingMPCCoordinator(DataUpdateCoordinator[dict[str, Any]]):
             status["status"] = "paused"
             return status
         try:
-            target = float(climate.attributes["temperature"])
-            if not isfinite(target) or not 5 <= target <= 35:
+            target = normalize("temperature", climate.attributes["temperature"],
+                               climate.attributes.get("temperature_unit",
+                                   self.hass.config.units.temperature_unit))
+            if target is None or not 5 <= target <= 35:
                 raise ValueError
         except (TypeError, KeyError, ValueError):
             status["status"] = "waiting_for_target"
@@ -184,7 +206,12 @@ class HeatingMPCCoordinator(DataUpdateCoordinator[dict[str, Any]]):
                 heating_cost=float(self.config[CONF_HEAT_COST]),
                 comfort_band=float(self.config[CONF_BAND]),
             )
+            # HA State objects are immutable snapshots. Discard a result if any
+            # input changed while the worker was calculating the hypothetical plan.
             output = await self.hass.async_add_executor_job(job)
+            if any(self.hass.states.get(entity) is not state for entity, state in snapshot.items()):
+                status["status"] = "inputs_changed"
+                return status
         except (ValueError, RuntimeError, OverflowError) as err:
             _LOGGER.warning("MPC prediction failed: %s", err)
             status["status"] = "prediction_error"

custom_components/heating_mpc/physics.py

@@ -26,25 +26,32 @@ class Learner:
     p: list[list[float]] = field(default_factory=lambda: [[0.0005, 0., 0.], [0., 0.8, 0.], [0., 0., 0.08]])
     count: int = 0
     error_sum: float = 0.0
+    on_count: int = 0
+    off_count: int = 0
 
     def predict_delta(self, temp: float, outside: float, heat: float) -> float:
         a, b, c = self.theta
         return a * (outside - temp) + b * heat + c
 
-    def learn(self, temp_before: float, temp_after: float, outside: float, effective_heat: float) -> None:
+    def learn(self, temp_before: float, temp_after: float, outside: float, effective_heat: float,
+              duration_seconds: float = 300, command: bool | None = None) -> bool:
         if not all(isfinite(x) for x in (temp_before, temp_after, outside, effective_heat)):
-            return
-        y = temp_after - temp_before
+            return False
+        if not 0 <= effective_heat <= 1:
+            return False
+        if not isfinite(duration_seconds) or not 270 <= duration_seconds <= 330:
+            return False
+        y = (temp_after - temp_before) * 300 / duration_seconds
         if abs(y) > 0.8:  # sensor glitch/door opened/restarted device
-            return
+            return False
         x = [outside - temp_before, effective_heat, 1.0]
         predicted = sum(t * v for t, v in zip(self.theta, x))
         residual = y - predicted
         px = [sum(self.p[i][j] * x[j] for j in range(3)) for i in range(3)]
         forgetting = 0.999
         denominator = forgetting + sum(x[i] * px[i] for i in range(3))
-        if denominator <= 0:
-            return
+        if not isfinite(denominator) or denominator <= 0:
+            return False
         k = [value / denominator for value in px]
         self.theta = [t + k[i] * residual for i, t in enumerate(self.theta)]
         for i in range(3):
@@ -58,29 +65,57 @@ class Learner:
         ]
         self.count += 1
         self.error_sum += abs(residual)
+        if command is not None:
+            if command:
+                self.on_count += 1
+            else:
+                self.off_count += 1
+        return True
 
     @property
     def mae(self) -> float | None:
         return self.error_sum / self.count if self.count else None
 
     def as_dict(self) -> dict[str, Any]:
-        return {"theta": self.theta, "p": self.p, "count": self.count, "error_sum": self.error_sum}
+        return {"theta": self.theta, "p": self.p, "count": self.count, "error_sum": self.error_sum,
+                "on_count": self.on_count, "off_count": self.off_count}
 
     @classmethod
     def from_dict(cls, source: dict[str, Any]) -> "Learner":
         obj = cls()
         try:
+            if not isinstance(source, dict):
+                return obj
             theta = [float(v) for v in source["theta"]]
             p = [[float(v) for v in r] for r in source["p"]]
             if len(theta) != 3 or len(p) != 3 or any(len(r) != 3 for r in p):
                 return obj
             if not all(isfinite(v) for v in theta + [v for row in p for v in row]):
                 return obj
-            obj.theta, obj.p = theta, p
-            obj.count = max(0, int(source.get("count", 0)))
-            obj.error_sum = max(0., float(source.get("error_sum", 0)))
-        except (KeyError, TypeError, ValueError):
-            return cls()
+            if not (0 <= theta[0] <= .04 and 0 <= theta[1] <= .25 and -.06 <= theta[2] <= .06):
+                return obj
+            # A covariance must be symmetric positive definite (Cholesky check).
+            lower = [[0.0] * 3 for _ in range(3)]
+            for i in range(3):
+                for j in range(i + 1):
+                    if abs(p[i][j] - p[j][i]) > 1e-8:
+                        return obj
+                    value = p[i][j] - sum(lower[i][k] * lower[j][k] for k in range(j))
+                    if i == j:
+                        if value <= 0 or not isfinite(value):
+                            return obj
+                        lower[i][j] = value ** .5
+                    else:
+                        lower[i][j] = value / lower[j][j]
+            counts = [source.get(key, 0) for key in ("count", "on_count", "off_count")]
+            if any(type(v) is not int or v < 0 for v in counts):
+                return obj
+            count, on_count, off_count = counts
+            error_sum = float(source.get("error_sum", 0))
+            if not isfinite(error_sum) or error_sum < 0 or on_count + off_count > count:
+                return obj
+            return cls(theta, p, count, error_sum, on_count, off_count)
+        except (KeyError, TypeError, ValueError, OverflowError):
             return obj
 
 
@@ -112,9 +147,18 @@ def optimize(
         raise ValueError("Invalid temperature")
     if any(not isfinite(t) for t in temps.values()):
         raise ValueError("Invalid room temperature")
-    if any(float(weights.get(room, 0)) <= 0 for room in temps):
+    if any(not isfinite(float(weights.get(room, 0))) or float(weights.get(room, 0)) <= 0 for room in temps):
         raise ValueError("All rooms need a positive weight")
+    if not isfinite(effective_heat) or not 0 <= effective_heat <= 1:
+        raise ValueError("Invalid heating lag")
+    if any(not isfinite(v) or v < 0 for v in (
+        seconds_since_change, minimum_on_minutes, minimum_off_minutes,
+        switching_cost, heating_cost, comfort_band,
+    )):
+        raise ValueError("Invalid optimizer parameter")
     total_weight = sum(float(weights[room]) for room in temps)
+    if not isfinite(total_weight):
+        raise ValueError("Invalid total weight")
     norm = {room: float(weights[room]) / total_weight for room in temps}
     blocks = horizon_minutes // 15
     nsteps_per_block = 3
@@ -156,7 +200,7 @@ def optimize(
             elapsed_s += 15 * 60
         if feasible and cost < best:
             best = cost
-            result = (bits, at_60 or dict(state), dict(state))
+            result = (bits, at_60 or {}, dict(state))
     if result is None:
         raise RuntimeError("No feasible schedule")
     plan, temps_60m, temps_end = result

custom_components/heating_mpc/weather_sources.py

@@ -44,6 +44,8 @@ def normalize(metric: str, raw: object, unit: str | None = None) -> float | None
         return None
     if not isfinite(value):
         return None
+    if unit is not None and not isinstance(unit, str):
+        return None
     unit = (unit or "").strip().lower().replace(" ", "")
     if metric in ("temperature", "feels_like"):
         if unit in ("°f", "f", "fahrenheit"):

tests/test_coordinator.py

@@ -0,0 +1,273 @@
+"""Offline HA boundary doubles; these do not validate the HA runtime itself."""
+import importlib.util
+import sys
+import types
+import unittest
+from datetime import datetime, timedelta, timezone
+from pathlib import Path
+from unittest.mock import AsyncMock, Mock, patch
+
+ROOT = Path(__file__).resolve().parents[1] / 'custom_components' / 'heating_mpc'
+
+
+class CoordinatorBase:
+    @classmethod
+    def __class_getitem__(cls, item):
+        return cls
+
+    def __init__(self, hass, *args, **kwargs):
+        self.hass = hass
+
+    async def async_config_entry_first_refresh(self):
+        self.data = await self._async_update_data()
+
+
+class Store:
+    def __init__(self, *args):
+        self.async_load = AsyncMock(return_value=None)
+        self.async_save = AsyncMock()
+        self.async_remove = AsyncMock()
+
+
+def load_integration():
+    modules = {name: types.ModuleType(name) for name in (
+        'homeassistant', 'homeassistant.const', 'homeassistant.core',
+        'homeassistant.config_entries', 'homeassistant.helpers',
+        'homeassistant.helpers.storage', 'homeassistant.helpers.update_coordinator',
+        'homeassistant.helpers.entity_registry')}
+    modules['homeassistant.const'].Platform = types.SimpleNamespace(SENSOR='sensor', BINARY_SENSOR='binary_sensor')
+    modules['homeassistant.core'].HomeAssistant = object
+    modules['homeassistant.config_entries'].ConfigEntry = object
+    modules['homeassistant.helpers.storage'].Store = Store
+    modules['homeassistant.helpers.update_coordinator'].DataUpdateCoordinator = CoordinatorBase
+    registry = modules['homeassistant.helpers.entity_registry']
+    registry.async_get = lambda hass: hass.registry
+    registry.async_entries_for_config_entry = lambda reg, entry: reg.entries
+    name = 'audit_heating_mpc'
+    spec = importlib.util.spec_from_file_location(name, ROOT / '__init__.py', submodule_search_locations=[str(ROOT)])
+    module = importlib.util.module_from_spec(spec)
+    sys.modules[name] = module
+    with patch.dict(sys.modules, modules):
+        spec.loader.exec_module(module)
+        return module, sys.modules[name + '.coordinator']
+
+
+integration, coordinator = load_integration()
+
+
+class BoundaryTests(unittest.IsolatedAsyncioTestCase):
+    def setUp(self):
+        self.now = datetime.now(timezone.utc)
+        self.states = {}
+        self.hass = types.SimpleNamespace(
+            data={}, states=types.SimpleNamespace(get=self.states.get),
+            config=types.SimpleNamespace(units=types.SimpleNamespace(temperature_unit='°C')),
+            services=types.SimpleNamespace(async_call=Mock(side_effect=AssertionError('No service calls permitted'))),
+            registry=types.SimpleNamespace(entries=[], async_remove=Mock()),
+            config_entries=types.SimpleNamespace(
+                async_forward_entry_setups=AsyncMock(), async_unload_platforms=AsyncMock(return_value=True),
+                async_reload=AsyncMock(), async_update_entry=Mock()),
+        )
+        async def execute(job):
+            return job()
+        self.hass.async_add_executor_job = execute
+        self.entry = types.SimpleNamespace(entry_id='entry', version=2,
+            data={'room_entities': ['sensor.room'], 'heater_entity': 'switch.boiler',
+                  'climate_entity': 'climate.reference', 'outside_temperature_sources': ['sensor.outside'],
+                  'horizon_minutes': 60}, options={}, async_on_unload=Mock(), add_update_listener=Mock())
+        self.set_state('sensor.room', '20', unit_of_measurement='°C')
+        self.set_state('sensor.outside', '5', unit_of_measurement='°C')
+        self.set_state('switch.boiler', 'off')
+        self.set_state('climate.reference', 'heat', temperature=22)
+        self.c = coordinator.HeatingMPCCoordinator(self.hass, self.entry)
+
+    def set_state(self, entity, value, **attrs):
+        self.states[entity] = types.SimpleNamespace(state=value, attributes=attrs,
+            last_changed=self.now - timedelta(hours=1))
+
+    async def sample(self, seconds=300):
+        with patch.object(coordinator, 'datetime') as clock:
+            clock.now.return_value = self.now
+            data = await self.c._async_update_data()
+        self.now += timedelta(seconds=seconds)
+        return data
+
+    def ready(self):
+        m = self.c.learners['sensor.room']
+        m.count, m.on_count, m.off_count = 200, 100, 100
+        self.c._update_counts()
+
+    async def test_learning_persistence_and_restart(self):
+        await self.sample()
+        await self.sample()
+        self.assertEqual(self.c.learners['sensor.room'].count, 1)
+        payload = self.c._store.async_save.call_args.args[0]
+        self.assertEqual(payload['schema_version'], 2)
+        new = coordinator.HeatingMPCCoordinator(self.hass, self.entry)
+        new._store.async_load.return_value = payload
+        await new.async_restore_model()
+        self.assertEqual(new._off_samples, 1)
+        self.assertIsNone(new._previous)
+
+    async def test_transition_and_round_trip_switch_are_skipped(self):
+        for new_state in ('on', 'off'):
+            self.c._previous = None
+            await self.sample()
+            self.states['switch.boiler'].state = new_state
+            self.states['switch.boiler'].last_changed = self.now - timedelta(seconds=60)
+            await self.sample()
+        self.assertEqual(self.c.learners['sensor.room'].count, 0)
+
+    async def test_gaps_backward_clock_and_excessive_jitter_skipped(self):
+        for seconds in (0, -300, 100, 420, 3600):
+            self.c._previous = None
+            await self.sample(seconds)
+            await self.sample()
+        self.assertEqual(self.c.learners['sensor.room'].count, 0)
+
+    async def test_missing_sensor_breaks_training_continuity(self):
+        await self.sample()
+        self.states['sensor.room'].state = 'unavailable'
+        self.assertEqual((await self.sample())['status'], 'waiting_for_sensors')
+        self.states['sensor.room'].state = '20'
+        await self.sample()
+        self.assertEqual(self.c.learners['sensor.room'].count, 0)
+
+    async def test_weather_fallback_skips_transition(self):
+        await self.sample()
+        self.c.config['outside_temperature_sources'].append('sensor.alternate')
+        self.set_state('sensor.alternate', '7', unit_of_measurement='°C')
+        self.states['sensor.outside'].state = 'unavailable'
+        data = await self.sample()
+        self.assertEqual(data['outside_temperature'], 7)
+        self.assertEqual(data['samples'], 0)
+
+    async def test_outlier_does_not_count_towards_readiness(self):
+        await self.sample()
+        self.states['sensor.room'].state = '25'
+        data = await self.sample()
+        self.assertEqual(data['samples'], 0)
+        self.assertEqual(data['off_samples'], 0)
+
+    async def test_fahrenheit_target_and_manual_off(self):
+        self.ready()
+        self.hass.config.units.temperature_unit = '°F'
+        self.states['climate.reference'].attributes['temperature'] = 71.6
+        data = await self.sample()
+        self.assertEqual(data['target'], 22)
+        self.assertEqual(data['status'], 'ready')
+        self.states['climate.reference'].state = 'off'
+        data = await self.sample()
+        self.assertEqual(data['status'], 'paused')
+        self.assertIsNone(data['proposal'])
+        self.assertEqual(self.states['climate.reference'].attributes['temperature'], 71.6)
+        self.hass.services.async_call.assert_not_called()
+
+    async def test_invalid_target_and_optimizer_failure(self):
+        self.ready()
+        self.states['climate.reference'].attributes['temperature'] = 'nan'
+        self.assertEqual((await self.sample())['status'], 'waiting_for_target')
+        self.states['climate.reference'].attributes['temperature'] = 22
+        with patch.object(coordinator, 'optimize', side_effect=RuntimeError('test')):
+            data = await self.sample()
+        self.assertEqual(data['status'], 'prediction_error')
+        self.assertIsNone(data['proposal'])
+
+    async def test_corrupt_storage_and_legacy_counters(self):
+        for payload in ([], 'broken', {'schema_version': 999}, {'learners': []},
+                        {'on_samples': 'bad', 'off_samples': float('inf')}):
+            self.c._store.async_load.return_value = payload
+            await self.c.async_restore_model()
+            self.assertEqual(self.c._on_samples, 0)
+        old = self.c.learners['sensor.room'].as_dict()
+        old.update(count=500)
+        old.pop('on_count'); old.pop('off_count')
+        self.c._store.async_load.return_value = {'learners': {'sensor.room': old}, 'on_samples': 250}
+        await self.c.async_restore_model()
+        self.assertEqual(self.c.learners['sensor.room'].count, 500)
+        self.assertEqual(self.c._on_samples, 0)
+
+    async def test_replaced_heater_invalidates_model(self):
+        self.c._store.async_load.return_value = {'heater_entity': 'switch.other',
+            'learners': {'sensor.room': {'count': 999}}}
+        await self.c.async_restore_model()
+        self.assertEqual(self.c.learners['sensor.room'].count, 0)
+
+    async def test_setup_reload_unload_and_registry_cleanup(self):
+        obsolete = types.SimpleNamespace(platform='heating_mpc', unique_id='entry_sensor.old_predicted_60min', entity_id='sensor.old_forecast')
+        kept = types.SimpleNamespace(platform='heating_mpc', unique_id='entry_sensor.room_predicted_60min', entity_id='sensor.forecast')
+        self.hass.registry.entries = [obsolete, kept]
+        self.assertTrue(await integration.async_setup_entry(self.hass, self.entry))
+        self.hass.registry.async_remove.assert_called_once_with('sensor.old_forecast')
+        await integration.async_reload_entry(self.hass, self.entry)
+        self.hass.config_entries.async_reload.assert_awaited_once_with('entry')
+        self.hass.config_entries.async_unload_platforms.return_value = False
+        self.assertFalse(await integration.async_unload_entry(self.hass, self.entry))
+        self.assertIn('entry', self.hass.data['heating_mpc'])
+        self.hass.config_entries.async_unload_platforms.return_value = True
+        self.assertTrue(await integration.async_unload_entry(self.hass, self.entry))
+        self.assertNotIn('entry', self.hass.data['heating_mpc'])
+        self.hass.services.async_call.assert_not_called()
+
+    async def test_failed_platform_setup_removes_coordinator(self):
+        self.hass.config_entries.async_forward_entry_setups.side_effect = RuntimeError('test')
+        with self.assertRaises(RuntimeError):
+            await integration.async_setup_entry(self.hass, self.entry)
+        self.assertNotIn('entry', self.hass.data['heating_mpc'])
+
+    async def test_uninstall_removes_storage(self):
+        store = Store()
+        with patch.object(integration, 'Store', return_value=store):
+            await integration.async_remove_entry(self.hass, self.entry)
+        store.async_remove.assert_awaited_once()
+
+    async def test_config_migration_preserves_options_precedence(self):
+        self.entry.version = 1
+        self.entry.data['outside_entity'] = 'sensor.old'
+        self.entry.options = {'outside_entity': 'sensor.override'}
+        self.assertTrue(await integration.async_migrate_entry(self.hass, self.entry))
+        kwargs = self.hass.config_entries.async_update_entry.call_args.kwargs
+        self.assertEqual(kwargs['options']['outside_temperature_sources'], ['sensor.override'])
+        self.assertEqual(kwargs['data']['outside_temperature_sources'], ['sensor.outside'])
+        self.assertNotIn('outside_entity', kwargs['data'])
+        self.assertEqual(kwargs['version'], 2)
+        self.entry.version = 3
+        self.assertFalse(await integration.async_migrate_entry(self.hass, self.entry))
+
+    async def test_changed_inputs_during_optimization_discard_result(self):
+        self.ready()
+        async def changed(job):
+            output = job()
+            self.set_state('climate.reference', 'off', temperature=22)
+            return output
+        self.hass.async_add_executor_job = changed
+        data = await self.sample()
+        self.assertEqual(data['status'], 'inputs_changed')
+        self.assertIsNone(data['proposal'])
+
+    async def test_invalid_weights_do_not_publish_proposal(self):
+        for weight in (0, -1, 'bad', float('nan'), float('inf')):
+            self.c.config['weights'] = {'sensor.room': weight}
+            data = await self.sample()
+            self.assertEqual(data['status'], 'invalid_weights')
+            self.assertIsNone(data['proposal'])
+
+    async def test_added_room_needs_its_own_excitation_evidence(self):
+        model = coordinator.Learner(count=500, on_count=250, off_count=250)
+        self.entry.options['room_entities'] = ['sensor.room', 'sensor.new']
+        self.c = coordinator.HeatingMPCCoordinator(self.hass, self.entry)
+        self.c._store.async_load.return_value = {'learners': {'sensor.room': model.as_dict()}}
+        await self.c.async_restore_model()
+        self.assertEqual(self.c.learners['sensor.room'].count, 500)
+        self.assertEqual(self.c._on_samples, 0)
+        self.assertEqual(self.c._off_samples, 0)
+
+    async def test_changed_inputs_during_save_discard_result(self):
+        self.ready()
+        await self.sample()
+        async def changed(payload):
+            self.set_state('climate.reference', 'off', temperature=22)
+        self.c._store.async_save.side_effect = changed
+        data = await self.sample()
+        self.assertEqual(data['status'], 'inputs_changed')
+        self.assertIsNone(data['proposal'])

tests/test_physics.py

@@ -60,6 +60,55 @@ class ModelTests(unittest.TestCase):
             m.optimize({'r': 20}, 10, 0, 22, {'r': 0}, {'r': m.Learner()},
                        False, 1000, horizon_minutes=60)
 
+    def test_short_horizon_does_not_claim_60_minute_prediction(self):
+        out = m.optimize({'r': 20}, 5, 0, 22, {'r': 1}, {'r': m.Learner()},
+                         False, 3600, horizon_minutes=30)
+        self.assertEqual(out['predicted_60m'], {})
+        self.assertIn('r', out['predicted_end'])
+
+    def test_prediction_matches_replay_at_60_minutes(self):
+        learners = {'r': m.Learner()}
+        out = m.optimize({'r': 20}, 5, .4, 22, {'r': 1}, learners, False, 3600,
+                         horizon_minutes=90)
+        temps, lag = {'r': 20}, .4
+        for index, bit in enumerate(out['plan_15min']):
+            for _ in range(3):
+                temps, lag = m.simulate_step(temps, 5, lag, bit, learners)
+            if index == 3:
+                self.assertEqual(out['predicted_60m']['r'], round(temps['r'], 3))
+        self.assertEqual(out['predicted_end']['r'], round(temps['r'], 3))
+
+    def test_jitter_normalized_and_rejection_counted(self):
+        nominal, delayed = m.Learner(), m.Learner()
+        self.assertTrue(nominal.learn(20, 20.1, 5, .5, command=True))
+        self.assertTrue(delayed.learn(20, 20.11, 5, .5, 330, True))
+        for a, b in zip(nominal.theta, delayed.theta):
+            self.assertAlmostEqual(a, b)
+        self.assertFalse(delayed.learn(20, 25, 5, .5, command=False))
+        self.assertEqual((delayed.count, delayed.on_count, delayed.off_count), (1, 1, 0))
+
+    def test_corrupt_model_restore_is_atomic(self):
+        for changes in ({'theta': [-1, .1, 0]}, {'p': [[-1, 0, 0], [0, 1, 0], [0, 0, 1]]},
+                        {'count': float('inf')}, {'count': '5'}, {'error_sum': float('nan')},
+                        {'off_count': 1}, {'theta': [float('nan'), 0, 0]}, {'p': None}):
+            with self.subTest(changes=changes):
+                data = m.Learner().as_dict()
+                data.update(changes)
+                self.assertEqual(m.Learner.from_dict(data), m.Learner())
+        for bad in (None, [], 'broken'):
+            self.assertEqual(m.Learner.from_dict(bad), m.Learner())
+
+    def test_nonfinite_optimizer_inputs_rejected(self):
+        for key, value in (('weights', {'r': float('nan')}), ('effective_heat', float('nan')),
+                           ('switching_cost', float('inf')), ('seconds_since_change', -1),
+                           ('minimum_on_minutes', -1), ('comfort_band', float('nan'))):
+            args = dict(temps={'r': 20}, outside=5, effective_heat=0, target=22,
+                        weights={'r': 1}, learners={'r': m.Learner()}, is_on=False,
+                        seconds_since_change=3600, horizon_minutes=60)
+            args[key] = value
+            with self.subTest(key=key), self.assertRaises(ValueError):
+                m.optimize(**args)
+
 
 if __name__ == '__main__':
     unittest.main()

tests/test_weather_sources.py

@@ -66,6 +66,7 @@ class WeatherInputs(unittest.TestCase):
             self.assertIsNone(w.normalize("temperature", value, "°C"))
         self.assertIsNone(w.normalize("temperature", 22.0, "km/h"))
         self.assertIsNone(w.normalize("wind", -3, "m/s"))
+        self.assertIsNone(w.normalize("temperature", 20, 42))
         self.assertIsNone(w.normalize("clouds", 150, "%"))