Add sensor freshness checks

AUDIT.md

@@ -91,3 +91,18 @@ API references used to check migration and entity-registry calls:
 [HA config entries](https://developers.home-assistant.io/docs/config_entries_index/),
 [config migration](https://developers.home-assistant.io/docs/config_entries_config_flow_handler/#config-entry-migration),
 [entity registry](https://contributors.home-assistant.io/core/integrations/entity-registry/).
+
+
+## Follow-up: configurable freshness
+
+The freshness portion of finding 1 is now addressed: room/weather report-age
+limits, stale-source fallback, recovery without cross-gap learning and per-source
+quality reasons are implemented. Defaults are 30/120 minutes and both limits are
+editable in setup/Options with English and Romanian labels. This supersedes the
+original audit's statement that stale numeric readings are accepted. Physical
+measurement timestamps, source alignment, cached-provider detection and HAOS
+validation remain open. See README for compatibility and safe trial details.
+
+The suite now has 42 passing offline tests. Compilation with the temporary cache
+and `git diff --check` pass. The API basis is HA's
+[state timestamp documentation](https://www.home-assistant.io/docs/configuration/state_object/).

README.md

@@ -31,7 +31,7 @@ This project targets Home Assistant OS on a Raspberry Pi 5, with one existing ON
 
 Choose multiple sensors for a single variable, e.g. OpenWeatherMap then another provider, in priority order. Switch order in **Configure** to run A/B comparisons on successive days, or simply let the second sensor act as fallback. The active source and all current alternative readings are visible in the **Heating MPC Status** entity attributes (`weather`, `weather_active_sources`, `weather_comparison`). The **MPC Outdoor Temperature** sensor also has an `active_source` attribute and an `alternatives` map.
 
-Measurements are normalized where possible: outdoor temperatures to °C; wind to m/s; pressure to hPa; visibility to meters. Unknown units or values outside plausibility bounds are rejected in favor of the next selected sensor. Clouds and humidity are expected in %, rain/snow in mm or mm/h as represented by their upstream sensor. A fallback source may have a different sampling delay; no stale-data age policy exists yet.
+Measurements are normalized where possible: outdoor temperatures to °C; wind to m/s; pressure to hPa; visibility to meters. Unknown units or values outside plausibility bounds are rejected in favor of the next selected sensor. Clouds and humidity are expected in %, rain/snow in mm or mm/h as represented by their upstream sensor. A fallback source may have a different sampling delay; configurable report-age limits now reject stale inputs (see below).
 
 Switching only the **weather-provider sensors** does not reset learned thermal parameters. The update is applied after the integration reloads, and it skips learning one transition when the primary temperature source changes, avoiding a jump in the training series. Nevertheless, two sensors with a persistent offset can bias predictions; compare their readings and validate the model again before relying on them. Replacing the physical heating switch discards incompatible model parameters. Existing room-specific parameters are retained if room entity IDs remain unchanged.
 
@@ -95,6 +95,34 @@ No active control was added; the manifest version has not been advanced.
   Predictions remain **counterfactual**: the proposed plan is not the actual
   Generic Thermostat's future command sequence.
 
-Offline lifecycle tests use HA doubles. Freshness policies, actual HA/HAOS lifecycle
+Offline lifecycle tests use HA doubles. Actual HA/HAOS lifecycle
 validation and held-out prediction accuracy remain unfinished. Back up before
 trying these changes; follow the safe trial and rollback notes in AUDIT.md.
+
+
+## Unreleased: sensor freshness
+
+Room and weather report-age limits are available during setup and in Options
+(5–1440 minutes). Defaults are 30 minutes for rooms and 120 minutes for weather;
+existing entries use these defaults until changed. Celsius/Fahrenheit handling,
+compatible learned models and entity IDs are unchanged.
+
+Stale room readings or an absence of fresh outdoor temperature sources pause
+learning and predictions. Weather selection skips stale sources; optional weather
+outages do not block predictions. Recovery begins a new sampling interval, so no
+training spans an outage. Status attribute `input_quality` lists each room and each
+weather source by metric, with report age in seconds and a reason: `valid`,
+`missing`, `unknown`, `unavailable`, `stale`, `invalid_timestamp`,
+`future_timestamp` or `invalid_value_or_unit`.
+
+Freshness uses HA `last_reported`, falling back to `last_updated` only on older HA
+versions without that field. It measures HA state reports, not the age of the
+original physical measurement: a provider republishing cached data can look fresh.
+Some integrations report only changed values; choose limits appropriate to their
+reporting behavior. The relay and thermostat are event-driven references and do
+not receive these numeric-sensor age limits. Sensor alignment and relay event
+history remain future work. These checks run on the existing five-minute poll.
+
+After backing up and upgrading, check report ages in Status, adjust limits in
+Options to match your sensors, and verify fallback/recovery with an unavailable
+source. Generic Thermostat remains the sole controller. No HAOS trial was performed.

ROADMAP.md

@@ -43,7 +43,7 @@ Current behavior: **shadow mode only**. `generic_thermostat` remains the one and
 - [ ] Add Home Assistant test harness coverage for initial setup, update, restart/reload, options changes, missing sensors, `heat/off`, target changes, teardown and uninstall.
 - [ ] Verify that **zero physical-device service calls** occur in observation mode, including restart and failed update cases.
 - [ ] Establish deterministic handling of 5-minute transitions, polling jitter, timestamp discontinuities, missing data and real switch transitions between samples.
-- [ ] Add input freshness thresholds and diagnostics that identify exactly which source is missing/stale/invalid.
+- [x] Add input freshness thresholds and diagnostics that identify exactly which source is missing/stale/invalid.
 - [ ] Verify that source switching, entity renaming, adding/removing rooms and replacing the relay do not silently reuse incompatible fitted coefficients.
 - [ ] Version persisted model schema; test corrupted/older state and migration/rollback behavior.
 - [ ] Verify calculated sample totals and successful/failed fitting counters; expose reasons why training/prediction is unavailable.
@@ -61,8 +61,9 @@ They are not a full HA test harness. Per-room accepted ON/OFF counts, additive
 payload schema v2, conservative relay-transition rejection, jitter normalization,
 Fahrenheit targets and forecast input-change guards are implemented.
 
-Next implement configurable freshness/alignment with diagnostics and a supported
+Configurable room/weather report-age limits and per-source quality diagnostics
-HA test harness/CI. Then improve event-based sampling and identity/persistence
+are now implemented with offline coverage. Sensor alignment remains open.
+Next implement a supported HA test harness/CI. Then improve event-based sampling and identity/persistence
 migration tests. Only after those reliability checks proceed to Milestone 1's
 recorded forecasts and walk-forward evaluation. Live control remains unauthorized.
 

custom_components/heating_mpc/config_flow.py

@@ -9,6 +9,7 @@ from homeassistant.core import callback
 from homeassistant.helpers import selector
 
 from .const import (
+    CONF_ROOM_MAX_AGE, CONF_WEATHER_MAX_AGE,
     DOMAIN, CONF_CLIMATE, CONF_HEATER, CONF_OUTSIDE, CONF_ROOMS,
     CONF_WEIGHTS, CONF_HORIZON, CONF_MIN_ON, CONF_MIN_OFF,
     CONF_SWITCH_COST, CONF_HEAT_COST, CONF_BAND, DEFAULTS,
@@ -37,6 +38,9 @@ def _main_schema(config: dict[str, Any] | None = None, detailed: bool = False):
             selector.EntitySelectorConfig(domain="switch")),
         _required(CONF_ROOMS, data): _entity_picker(multiple=True),
     }
+    for key in (CONF_ROOM_MAX_AGE, CONF_WEATHER_MAX_AGE):
+        schema[vol.Required(key, default=data.get(key, DEFAULTS[key]))] = vol.All(
+            vol.Coerce(int), vol.Range(min=5, max=1440))
     if detailed:
         for key, allowed in {
             CONF_HORIZON: vol.In([60, 90, 120, 150, 180]),

custom_components/heating_mpc/const.py

@@ -10,6 +10,8 @@ CONF_MIN_ON = "minimum_on_minutes"
 CONF_MIN_OFF = "minimum_off_minutes"
 CONF_SWITCH_COST = "switching_cost"
 CONF_HEAT_COST = "heating_cost"
+CONF_ROOM_MAX_AGE = "room_max_age_minutes"
+CONF_WEATHER_MAX_AGE = "weather_max_age_minutes"
 CONF_BAND = "comfort_band"
 
 # Each source list is ordered: first valid sensor wins; remaining are fallbacks.
@@ -28,6 +30,8 @@ WEATHER_MAIN = ("temperature", "wind", "clouds")
 WEATHER_EXTRA = ("humidity", "feels_like", "pressure", "visibility", "rain", "snow")
 
 DEFAULTS = {
+    CONF_ROOM_MAX_AGE: 30,
+    CONF_WEATHER_MAX_AGE: 120,
     CONF_HORIZON: 120,
     CONF_MIN_ON: 15,
     CONF_MIN_OFF: 10,

custom_components/heating_mpc/coordinator.py

@@ -13,10 +13,12 @@ from homeassistant.helpers.storage import Store
 from homeassistant.helpers.update_coordinator import DataUpdateCoordinator
 
 from .const import (
+    CONF_ROOM_MAX_AGE, CONF_WEATHER_MAX_AGE,
     DOMAIN, CONF_CLIMATE, CONF_HEATER, CONF_ROOMS, CONF_WEIGHTS, WEATHER_SOURCES,
     CONF_HORIZON, CONF_MIN_ON, CONF_MIN_OFF, CONF_SWITCH_COST, CONF_HEAT_COST,
     CONF_BAND, DEFAULTS,
 )
+from .data_quality import freshness
 from .physics import Learner, optimize
 from .weather_sources import choose_weather_sources, sources_for, normalize
 
@@ -61,14 +63,15 @@ class HeatingMPCCoordinator(DataUpdateCoordinator[dict[str, Any]]):
         self._on_samples = min(m.on_count for m in self.learners.values())
         self._off_samples = min(m.off_count for m in self.learners.values())
 
-    def _value(self, entity_id: str, low: float, high: float) -> float | None:
+    def _sensor_quality(self, entity_id: str, now: datetime, limit: float) -> dict:
         state = self.hass.states.get(entity_id)
-        if state is None or state.state in ("unknown", "unavailable"):
+        if state is None:
-            return None
+            return {"status": "missing", "age_seconds": None}
-        value = normalize("temperature", state.state, state.attributes.get("unit_of_measurement"))
+        if state.state in ("unknown", "unavailable"):
-        if value is None or not low <= value <= high:
+            return {"status": state.state, "age_seconds": None}
-            return None
+        # last_updated is a conservative compatibility fallback for older HA.
-        return value
+        return freshness(getattr(state, "last_reported", getattr(state, "last_updated", None)),
+                         now, limit * 60)
 
     async def _async_update_data(self) -> dict[str, Any]:
         now = datetime.now(timezone.utc)
@@ -83,17 +86,41 @@ class HeatingMPCCoordinator(DataUpdateCoordinator[dict[str, Any]]):
         watched = set(self.rooms) | weather_entity_ids | {
             self.config[CONF_CLIMATE], self.config[CONF_HEATER]}
         snapshot = {entity: self.hass.states.get(entity) for entity in watched}
-        weather_states = {
+        room_quality = {}
-            entity_id: (state.state, state.attributes.get("unit_of_measurement"))
+        temps = {}
-            for entity_id in weather_entity_ids
+        for room in self.rooms:
-            if (state := self.hass.states.get(entity_id)) is not None
+            quality = self._sensor_quality(room, now, self.config[CONF_ROOM_MAX_AGE])
-        }
+            value = None
+            if quality["status"] == "valid":
+                state = snapshot[room]
+                value = normalize("temperature", state.state, state.attributes.get("unit_of_measurement"))
+                if value is None or not 0 <= value <= 45:
+                    quality["status"] = "invalid_value_or_unit"
+                    value = None
+            room_quality[room] = quality
+            temps[room] = value
+        weather_states = {}
+        weather_quality = {}
+        for entity_id in weather_entity_ids:
+            quality = self._sensor_quality(entity_id, now, self.config[CONF_WEATHER_MAX_AGE])
+            weather_quality[entity_id] = quality
+            if quality["status"] == "valid":
+                state = snapshot[entity_id]
+                weather_states[entity_id] = (state.state, state.attributes.get("unit_of_measurement"))
         weather, active_sources, weather_comparisons = choose_weather_sources(
             self.config, weather_states
         )
+        metric_quality = {}
+        for metric in WEATHER_SOURCES:
+            metric_quality[metric] = {}
+            for entity_id in sources_for(self.config, metric):
+                quality = dict(weather_quality[entity_id])
+                if quality["status"] == "valid" and entity_id not in weather_comparisons[metric]:
+                    quality["status"] = "invalid_value_or_unit"
+                metric_quality[metric][entity_id] = quality
         outside = weather["temperature"]
-        temps = {room: self._value(room, 0.0, 45.0) for room in self.rooms}
         status: dict[str, Any] = {
+            "input_quality": {"rooms": room_quality, "weather": metric_quality},
             "status": "waiting_for_sensors", "proposal": None, "prediction_60m": {},
             "temperatures": {r: t for r, t in temps.items() if t is not None},
             "on_samples": self._on_samples, "off_samples": self._off_samples,

custom_components/heating_mpc/data_quality.py

@@ -0,0 +1,18 @@
+"""Pure timestamp validation for reported sensor snapshots."""
+from __future__ import annotations
+
+from datetime import datetime
+from math import isfinite
+
+
+def freshness(reported: datetime | None, now: datetime, max_age_seconds: float) -> dict:
+    """Use report time, not value-change time; reject unknown/future clocks."""
+    if not isfinite(max_age_seconds) or max_age_seconds <= 0:
+        raise ValueError("Freshness limit must be positive and finite")
+    if not isinstance(reported, datetime) or reported.tzinfo is None:
+        return {"status": "invalid_timestamp", "age_seconds": None}
+    age = (now - reported).total_seconds()
+    if age < 0:
+        return {"status": "future_timestamp", "age_seconds": round(age, 1)}
+    return {"status": "stale" if age > max_age_seconds else "valid",
+            "age_seconds": round(age, 1)}

custom_components/heating_mpc/strings.json

@@ -8,7 +8,9 @@
         "data": {
           "climate_entity": "Existing Generic Thermostat",
           "heater_entity": "Physical heating switch (read-only)",
-          "room_entities": "Individual room temperature sensors"
+          "room_entities": "Individual room temperature sensors",
+          "room_max_age_minutes": "Maximum room sensor report age (minutes)",
+          "weather_max_age_minutes": "Maximum weather sensor report age (minutes)"
         }
       },
       "weather": {
@@ -59,7 +61,9 @@
           "minimum_off_minutes": "Minimum OFF time (minutes)",
           "switching_cost": "Switching cost",
           "heating_cost": "Heating runtime cost",
-          "comfort_band": "Comfort tolerance (°C)"
+          "comfort_band": "Comfort tolerance (°C)",
+          "room_max_age_minutes": "Maximum room sensor report age (minutes)",
+          "weather_max_age_minutes": "Maximum weather sensor report age (minutes)"
         }
       },
       "weather": {

custom_components/heating_mpc/translations/en.json

@@ -8,7 +8,9 @@
         "data": {
           "climate_entity": "Existing Generic Thermostat",
           "heater_entity": "Physical heating switch (read-only)",
-          "room_entities": "Individual room temperature sensors"
+          "room_entities": "Individual room temperature sensors",
+          "room_max_age_minutes": "Maximum room sensor report age (minutes)",
+          "weather_max_age_minutes": "Maximum weather sensor report age (minutes)"
         }
       },
       "weather": {
@@ -59,7 +61,9 @@
           "minimum_off_minutes": "Minimum OFF time (minutes)",
           "switching_cost": "Switching cost",
           "heating_cost": "Heating runtime cost",
-          "comfort_band": "Comfort tolerance (°C)"
+          "comfort_band": "Comfort tolerance (°C)",
+          "room_max_age_minutes": "Maximum room sensor report age (minutes)",
+          "weather_max_age_minutes": "Maximum weather sensor report age (minutes)"
         }
       },
       "weather": {

custom_components/heating_mpc/translations/ro.json

@@ -8,7 +8,9 @@
         "data": {
           "climate_entity": "Termostatul Generic Thermostat existent",
           "heater_entity": "Releul fizic (numai citire)",
-          "room_entities": "Senzorii individuali din camere"
+          "room_entities": "Senzorii individuali din camere",
+          "room_max_age_minutes": "Vârsta maximă a raportării senzorilor interiori (minute)",
+          "weather_max_age_minutes": "Vârsta maximă a raportării senzorilor meteo (minute)"
         }
       },
       "weather": {
@@ -59,7 +61,9 @@
           "minimum_off_minutes": "Durată minimă OFF (minute)",
           "switching_cost": "Penalizare comutări",
           "heating_cost": "Penalizare durată încălzire",
-          "comfort_band": "Toleranță confort (°C)"
+          "comfort_band": "Toleranță confort (°C)",
+          "room_max_age_minutes": "Vârsta maximă a raportării senzorilor interiori (minute)",
+          "weather_max_age_minutes": "Vârsta maximă a raportării senzorilor meteo (minute)"
         }
       },
       "weather": {

tests/test_coordinator.py

@@ -83,7 +83,7 @@ class BoundaryTests(unittest.IsolatedAsyncioTestCase):
 
     def set_state(self, entity, value, **attrs):
         self.states[entity] = types.SimpleNamespace(state=value, attributes=attrs,
-            last_changed=self.now - timedelta(hours=1))
+            last_changed=self.now - timedelta(hours=1), last_reported=self.now)
 
     async def sample(self, seconds=300):
         with patch.object(coordinator, 'datetime') as clock:
@@ -271,3 +271,42 @@ class BoundaryTests(unittest.IsolatedAsyncioTestCase):
         data = await self.sample()
         self.assertEqual(data['status'], 'inputs_changed')
         self.assertIsNone(data['proposal'])
+
+    async def test_stale_room_pauses_and_recovery_skips_gap(self):
+        await self.sample()
+        self.states['sensor.room'].last_reported = self.now - timedelta(minutes=31)
+        data = await self.sample()
+        self.assertEqual(data['status'], 'waiting_for_sensors')
+        self.assertEqual(data['input_quality']['rooms']['sensor.room']['status'], 'stale')
+        self.assertIsNone(data['proposal'])
+        self.set_state('sensor.room', '20', unit_of_measurement='°C')
+        data = await self.sample()
+        self.assertEqual(data['samples'], 0)
+        self.assertEqual(data['status'], 'learning')
+
+    async def test_stale_weather_fallback_and_optional_outage(self):
+        self.c.config['outside_temperature_sources'].append('sensor.alternate')
+        self.set_state('sensor.alternate', '7', unit_of_measurement='°C')
+        self.states['sensor.outside'].last_reported = self.now - timedelta(minutes=121)
+        self.c.config['outside_wind_sources'] = ['sensor.missing']
+        data = await self.sample()
+        self.assertEqual(data['outside_temperature'], 7)
+        self.assertEqual(data['status'], 'learning')
+        quality = data['input_quality']['weather']
+        self.assertEqual(quality['temperature']['sensor.outside']['status'], 'stale')
+        self.assertEqual(quality['wind']['sensor.missing']['status'], 'missing')
+
+    async def test_fresh_unchanged_reading_is_valid(self):
+        self.states['sensor.room'].last_changed = self.now - timedelta(days=2)
+        self.assertEqual((await self.sample())['status'], 'learning')
+
+    async def test_configured_age_limit_applied(self):
+        self.c.config['room_max_age_minutes'] = 5
+        self.states['sensor.room'].last_reported = self.now - timedelta(minutes=6)
+        self.assertEqual((await self.sample())['status'], 'waiting_for_sensors')
+
+    async def test_invalid_unit_diagnosed_by_metric(self):
+        self.states['sensor.outside'].attributes['unit_of_measurement'] = 'km/h'
+        data = await self.sample()
+        self.assertEqual(data['input_quality']['weather']['temperature']['sensor.outside']['status'],
+                         'invalid_value_or_unit')

tests/test_data_quality.py

@@ -0,0 +1,40 @@
+"""Timestamp boundaries and UI translation coverage without HA dependencies."""
+import importlib.util
+import json
+import unittest
+from datetime import datetime, timedelta, timezone
+from pathlib import Path
+
+ROOT = Path(__file__).resolve().parents[1] / 'custom_components' / 'heating_mpc'
+spec = importlib.util.spec_from_file_location('quality', ROOT / 'data_quality.py')
+quality = importlib.util.module_from_spec(spec)
+spec.loader.exec_module(quality)
+
+
+class FreshnessTests(unittest.TestCase):
+    def test_age_boundaries(self):
+        now = datetime(2026, 10, 26, tzinfo=timezone.utc)
+        for seconds, expected in ((0, 'valid'), (1800, 'valid'), (1801, 'stale'), (-1, 'future_timestamp')):
+            with self.subTest(seconds=seconds):
+                result = quality.freshness(now - timedelta(seconds=seconds), now, 1800)
+                self.assertEqual(result['status'], expected)
+                self.assertEqual(result['age_seconds'], seconds)
+
+    def test_missing_naive_and_malformed_times(self):
+        now = datetime.now(timezone.utc)
+        for timestamp in (None, 'bad', datetime.now()):
+            self.assertEqual(quality.freshness(timestamp, now, 1800)['status'], 'invalid_timestamp')
+
+    def test_invalid_limits_rejected(self):
+        now = datetime.now(timezone.utc)
+        for limit in (0, -1, float('inf'), float('nan')):
+            with self.assertRaises(ValueError):
+                quality.freshness(now, now, limit)
+
+    def test_freshness_settings_translated(self):
+        for name in ('strings.json', 'translations/en.json', 'translations/ro.json'):
+            data = json.loads((ROOT / name).read_text())
+            for section, step in (('config', 'user'), ('options', 'init')):
+                labels = data[section]['step'][step]['data']
+                for key in ('room_max_age_minutes', 'weather_max_age_minutes'):
+                    self.assertTrue(labels[key])