Document second audit pass

Record findings, evidence, compatibility notes and remaining gaps in
AUDIT.md; add user-facing upgrade notes to README and update test
counts; note progress against Milestone 0 in ROADMAP. No version bump.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

AUDIT.md

@@ -135,3 +135,51 @@ Python 3.13 virtual environment, then runs the same dependency and test checks.
 GitHub cache services are disabled and no Gitea token is supplied to setup-uv.
 The runner label and integration runtime are unchanged. Remote execution of the
 corrected workflow remains to be verified.
+
+
+## Follow-up: second source audit — lag, learner stability, options
+
+Scope as above: repository code, offline tests and the HA Core 2025.12.0 suite.
+No HAOS, physical relay, thermostat or deployment testing was performed. The
+manifest remains 0.2.1 and no actuator code was added.
+
+| Area | Finding | Repair / evidence |
+|---|---|---|
+| Learning and prediction, high | The radiator-lag proxy was reset to the new relay command at every relay change and after any skipped interval: it read 1.0 straight after ON and 0.0 straight after OFF, so the lag model was effectively disabled around every boiler cycle. On a synthetic lagged plant cycled by a hysteresis thermostat, the original code learned a heat gain 34 % low (0.079 vs 0.12) and a loss coefficient 43 % low. The optimizer also started each post-transition plan from the wrong lag. | Lag is tracked at every poll with a valid relay state, independently of room sensors. One change between polls is integrated at its `last_changed` time. Ambiguous cases (on/off round trip between polls, relay outage, restart, clock jump) restart from time-in-state with an explicit error half-width; learning waits until it is at most 0.1 (about 32 minutes after an HA restart). The same synthetic test now learns 0.1195 / 0.00389 (true 0.12 / 0.004). |
+| Learning and persistence, high | RLS covariance windup: with forgetting 0.999 and no heat excitation (summer, long OFF periods) the heat variance grew by 1/0.999 per sample without limit, about 8.7×10¹² after 104 simulated OFF days. The first ON samples afterwards could swing coefficients to their bounds, and accumulated asymmetry (9×10⁻⁹ there) approached the absolute 10⁻⁸ restore tolerance, beyond which a restart would silently reset the model. | Covariance is symmetrized and its diagonal bounded by the prior after every update; row/column scaling preserves positive definiteness. Restore uses a relative round-off tolerance and applies the same bound, so wound-up older models load rather than reset. |
+| Configuration, medium | Optional weather source fields used the current selection as a form default. A cleared field omitted from the submission re-inserted the old sources, and an options key absent from the submission fell back to the original setup data, so optional providers could not be removed in the UI. | Suggested values pre-fill the form instead; omitted optional lists are saved as empty and override older entry data. Covered by an HA Core options-flow test. |
+| Prediction availability, medium | A finished plan was discarded when any watched State object changed during the calculation, including diagnostic-only weather sensors and the thermostat's own `current_temperature`/`hvac_action`. The proposed-heat entity could go unavailable for five minutes without cause. | Compare only values the prediction uses: room and outdoor-temperature states and units, thermostat mode and target, relay state and last change. |
+| Lifecycle, low | The coordinator relied on HA's config-entry ContextVar; HA reports this, though it does not enforce it for custom integrations. | Pass `config_entry` explicitly. |
+| Tests, medium | The shadow-only rule was checked only at runtime on exercised paths. | Static AST tripwire over the integration source: no service calls or registration, no actuator method names or service-name strings, and only sensor/binary_sensor platforms. |
+
+New Status attributes: `heat_lag`, `heat_lag_uncertainty` and `sample_result`
+(`first_sample`, `learned`, `partly_rejected`, `interval_out_of_range`,
+`relay_changed`, `outside_source_changed`, `lag_uncertain`). They explain why the
+sample counters are or are not increasing.
+
+**Compatibility.** Persisted payload schema stays 2; existing models load and get
+the covariance bound on load. Coefficients learned before this fix were fitted
+with the wrong lag. With forgetting 0.999 they should re-converge over roughly
+three to four days of heating data, but sample counts are kept, so `ready` does
+not reflect this. Treat earlier forecasts as uncalibrated. For a clean model,
+delete and re-add the integration (this deletes the learned data).
+
+**Remaining gaps found in this pass**, in addition to those above:
+
+1. Relay changes are still inferred from `last_changed` at each poll. An odd
+   number of changes greater than one between polls is treated as one change.
+   A read-only state-change listener for the relay would remove this.
+2. The lag estimate is not persisted, so an options reload restarts it from
+   time-in-state (exact for a long-steady relay, approximate otherwise).
+3. Learners of rooms removed in Options are dropped at the next save; re-adding a
+   room, or reverting an accidental relay change, starts fresh.
+4. Status attributes change every poll (`age_seconds`, MAE), so Recorder stores a
+   new attribute row every five minutes. Consider `_unrecorded_attributes`.
+5. The HA Core suite pins 2025.12.0; current HA releases are untested.
+
+**Verification.** `sh scripts/check.sh` in a Python 3.13 environment with
+`requirements-test.txt`: pip check, 56 offline tests, 16 HA Core tests,
+compilation, package/translation validation and `git diff --check` all pass.
+`python3 -m unittest discover -s tests` also passes on Python 3.9.6. The new
+synthetic-plant and windup tests were also run against the original code to confirm
+they fail there.

README.md

@@ -128,6 +128,38 @@ Options to match your sensors, and verify fallback/recovery with an unavailable
 source. Generic Thermostat remains the sole controller. No HAOS trial was performed.
 
 
+## Unreleased: radiator-lag tracking and learner stability
+
+See the second-pass section of [AUDIT.md](AUDIT.md). Still shadow-only; no new
+options, translations or entities; the manifest version is unchanged.
+
+- **Radiator lag now follows the relay.** Previously the lag estimate jumped to
+  fully hot or fully cold at each relay change, which biased the learned heating
+  coefficient low and gave wrong starting points to forecasts right after a boiler
+  cycle. A relay change between polls is now integrated at its timestamp.
+- **Learning pauses briefly when the lag is unknown**: after an HA restart, a relay
+  outage, or two relay changes inside one five-minute poll. The Status attribute
+  `heat_lag_uncertainty` shows the error bound; learning resumes once it is at most
+  0.1, about 32 minutes after a restart. Forecasts continue meanwhile.
+- **Long OFF periods no longer destabilize learning.** The learner's covariance is
+  bounded, so a summer without heating cannot make coefficients jump when heating
+  resumes, and older saved models with large covariance are loaded rather than reset.
+- **Optional weather sources can be removed in Configure.** Clearing wind, cloud or
+  any additional metric now saves an empty list instead of restoring the old sources.
+- The proposed-heat entity is no longer marked unavailable when only a
+  diagnostic weather sensor or the thermostat's current temperature changed during
+  the calculation.
+- New Status attributes: `heat_lag`, `heat_lag_uncertainty`, `sample_result` (why the
+  last poll was or was not learned).
+
+**After upgrading:** saved models load unchanged, but coefficients learned before
+this change were fitted with the wrong lag. Expect them to re-converge over a few
+days of heating and judge forecasts only on data after the upgrade, or delete and
+re-add the integration for a clean model (this deletes learned data). After a
+restart, check that `sample_result` moves from `lag_uncertain` to `learned` within
+about 35 minutes. Generic Thermostat remains the sole relay controller. No HAOS
+trial was performed.
+
 ## Developer checks and Forgejo/Gitea Actions
 
 The HA Core test suite targets **Home Assistant 2025.12.0 on Python 3.13**,
@@ -142,7 +174,7 @@ python -m pip install -r requirements-test.txt
 sh scripts/check.sh
 ```
 
-The check script runs dependency consistency checks, 42 offline tests, 14 tests
+The check script runs dependency consistency checks, 56 offline tests, 16 tests
 using the real HA Core/fixture package, compilation, lightweight package metadata
 and translation checks, and diff whitespace checks. A compatible DNS dependency
 is pinned because pycares 5 breaks the older HA dependency's import. The script

ROADMAP.md

@@ -70,6 +70,14 @@ remote runner execution remains unverified. Next improve event-based sampling an
 identity/persistence migration tests, then extend the HA version matrix. Only after those reliability checks proceed to Milestone 1's
 recorded forecasts and walk-forward evaluation. Live control remains unauthorized.
 
+A second audit pass (see AUDIT.md) fixed radiator-lag resets at every relay change,
+RLS covariance windup and a restore tolerance that could reset long-running models,
+optional weather sources that could not be cleared in Options, and spurious
+prediction discards. It added `sample_result`/lag diagnostics and a static
+shadow-only tripwire test. Relay transitions between polls are now integrated at
+their timestamp; multiple changes within one poll still need event-based relay
+history before the transition item below can be checked.
+
 ## Milestone 1 — Explainable forecasts and dashboards (target v0.3.0)
 
 **Objective:** Users can verify what the model predicts and whether it is actually accurate.