Compare commits
7
Commits
87437cbd48
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2d027f367a | ||
|
|
c4035951df | ||
|
|
0b82a8e26c | ||
|
|
2cfba024ff | ||
|
|
13cb972566 | ||
|
|
49565d610a | ||
|
|
28ff953ddd |
No files matched your search
@@ -397,3 +397,73 @@ Remaining risks:
|
|||||||
ruff, 159 offline tests, 28 HA Core tests; the offline tests also pass on Python
|
ruff, 159 offline tests, 28 HA Core tests; the offline tests also pass on Python
|
||||||
3.9.6. `sh scripts/check.sh` additionally validates packaging, translations and
|
3.9.6. `sh scripts/check.sh` additionally validates packaging, translations and
|
||||||
the changelog.
|
the changelog.
|
||||||
|
|
||||||
|
## 0.4.2 — final stability fixes after 0.4.1 — 2026-10-27
|
||||||
|
|
||||||
|
Scope: four issues from an independent review of 0.4.1, plus a user-requested
|
||||||
|
*Reset learned model* action. Repository code and tests only (HA Core 2025.12.0
|
||||||
|
and 2026.2.3 harness); no HAOS, device or deployment testing. Still no actuator
|
||||||
|
code; the static and runtime service-call guards pass. Released as 0.4.2.
|
||||||
|
|
||||||
|
Confirmed issues, root causes and fixes:
|
||||||
|
|
||||||
|
1. **Model Error scored spans with unknown ventilation inputs** (2cfba02).
|
||||||
|
Reproduced: a window unknown for a whole 15-minute span gave a Model Error
|
||||||
|
sample (MAE 0.76 °C). Root cause: the replay used the prediction path, which
|
||||||
|
drops the ventilation term when the opening is unknown, i.e. assumes closed.
|
||||||
|
Fix: per zone and step, `inputs_complete` checks every input the zone's model
|
||||||
|
used (window opening and confirmed recovery; wind, direction and other zones'
|
||||||
|
windows while a window is open; electrical heat and neighbours for the extended
|
||||||
|
layer, including the neighbour's own completeness). Incomplete spans are
|
||||||
|
counted, not scored, with the base model's error kept apart; unreplayable
|
||||||
|
spans are counted as skipped. Learning also stopped substituting a zero facing
|
||||||
|
term or zero cross-ventilation input for unknown values.
|
||||||
|
2. **Window unknown at startup treated as closed** (13cb972). Reproduced
|
||||||
|
(0 % assumed opening). Root cause: last known states lived only in memory, and
|
||||||
|
"never observed" fell through to the closed case; a first reading of an open
|
||||||
|
window was also logged as a confirmed opening. Fix: confirmation only for
|
||||||
|
witnessed transitions; last known states, open events and recovery saved (also
|
||||||
|
on every window state change) and restored as unknown; per-window forecast
|
||||||
|
assumptions (observed / last known open, bounded by the open-window time after
|
||||||
|
it was last seen / last known closed / unobserved), exposed in Status and on
|
||||||
|
zone forecasts; durations as ranges when an end was not seen. Chosen over
|
||||||
|
open/closed scenario pairs to avoid a second plan simulation per poll.
|
||||||
|
3. **Lifecycle and storage races** (28ff953). Confirmed by reading HA's unload
|
||||||
|
order: the coordinator's own shutdown runs only after platforms unload, and the
|
||||||
|
save checked for shutdown before waiting for the storage lock (check-then-act).
|
||||||
|
Found during testing: unloading cancels a running scheduled update, which
|
||||||
|
released the lock while the file write continued in a worker thread, so a
|
||||||
|
deletion could run first and be undone. Fix: closing mark at the start of
|
||||||
|
unload (restored if unload fails); checks after acquiring the lock, including a
|
||||||
|
removed-model mark set under the lock by removal; the write runs as a shielded,
|
||||||
|
HA-tracked task that holds the lock until it finishes; failed setup shuts the
|
||||||
|
coordinator down; queued relay callbacks are ignored once closing. Mutation
|
||||||
|
checks: removing the locked check or the shield makes the deletion tests fail;
|
||||||
|
removing the early closing mark fails the ordering test (the HA-level unload
|
||||||
|
test cannot distinguish it, as HA's own shutdown also lands before release).
|
||||||
|
4. **Forecast Error accepted materially different relay behaviour** (0b82a8e).
|
||||||
|
Reproduced: 12 of 15 planned minutes was scored. Root cause: a per-block
|
||||||
|
on-share tolerance of 20 %, blind to timing within a block. Fix: deviation =
|
||||||
|
time in another state than planned, block by block from the relay's switch
|
||||||
|
times, with unobserved time as possible deviation; matched ≤ 2 min (scored),
|
||||||
|
approximate ≤ 5 min (separate), counterfactual > 5 min observed, otherwise
|
||||||
|
incomplete with a reason; forecasts made with assumed inputs are incomplete.
|
||||||
|
Ledger statistics are versioned; 0.4.1 figures are dropped.
|
||||||
|
|
||||||
|
Added: *Reset learned model* (49565d6): unload, strip learned keys from the
|
||||||
|
stored model under the lock, set up again; configuration, heat-source baselines
|
||||||
|
and window history kept.
|
||||||
|
|
||||||
|
Tests added (by area): Model Error incomplete/skipped (7), plan matching (10),
|
||||||
|
window startup/restart/mixed/strict inputs (9), coordinator window reload and
|
||||||
|
uncertain forecast (2), lifecycle offline (5) and in HA (4), reset flow (1).
|
||||||
|
|
||||||
|
Remaining limitations: unknown-but-last-closed and never-seen windows still
|
||||||
|
produce a closed-window forecast (flagged, never scored); the 2 and 5 minute
|
||||||
|
limits rest on typical learned heat coefficients, not a per-forecast calculation;
|
||||||
|
spans lost when the replay window restarts are not counted; saved window state can
|
||||||
|
be up to one save old, which only makes the restored unknown period longer.
|
||||||
|
|
||||||
|
**Verification.** `sh scripts/check.sh` passes in both environments (pip check,
|
||||||
|
ruff, 192 offline tests, 33 HA Core tests, compilation, package/translation/
|
||||||
|
changelog checks, diff whitespace); the offline tests also pass on Python 3.9.6.
|
||||||
@@ -2,6 +2,78 @@
|
|||||||
|
|
||||||
Engineering findings and test evidence behind each change are in [AUDIT.md](AUDIT.md).
|
Engineering findings and test evidence behind each change are in [AUDIT.md](AUDIT.md).
|
||||||
|
|
||||||
|
## 0.4.2 — 2026-10-27
|
||||||
|
|
||||||
|
Four correctness fixes from an independent review of 0.4.1, and a **Reset learned
|
||||||
|
model** action. Still **observation-only**: no service calls; the boiler and
|
||||||
|
thermostat are untouched.
|
||||||
|
Tested in the Home Assistant Core harness with HA 2025.12.0 and 2026.2.3; not on
|
||||||
|
HAOS.
|
||||||
|
|
||||||
|
### Upgrade notes
|
||||||
|
|
||||||
|
- No reconfiguration needed. Zones, sensor and comfort weights, weather sources,
|
||||||
|
heat sources, windows, learned coefficients and fingerprints carry over; entity
|
||||||
|
IDs are unchanged.
|
||||||
|
- **Configure now opens a menu**: *Settings* (the previous form) or *Reset learned
|
||||||
|
model*.
|
||||||
|
- **Forecast Error statistics restart.** 0.4.1 accepted plans the relay followed
|
||||||
|
within 20 % per block; those figures are not comparable with the new rule and
|
||||||
|
are dropped on the first start (they covered at most a day).
|
||||||
|
- Model Error statistics carry over. Those from 0.4.0/0.4.1 may include a few
|
||||||
|
spans with unknown window states; they age out within 7 days.
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- **Reset learned model** (Configure, with a confirmation): forgets every zone's
|
||||||
|
learned coefficients, models kept for removed zones, and the Model Error and
|
||||||
|
Forecast Error statistics; keeps the configuration, heat-source baselines and the
|
||||||
|
window event log, and logs the reset in `model_changes`.
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
- **Model Error no longer scores spans with unknown inputs.** A window unknown
|
||||||
|
during a span was replayed as closed and scored. Spans where a window state, an
|
||||||
|
open window's weather input, or the extended layer's electrical heat or
|
||||||
|
neighbour was unknown are now counted as `incomplete` (with the base model's
|
||||||
|
error kept apart as `incomplete_base_model_mae`); spans that cannot be replayed
|
||||||
|
are counted as `skipped`.
|
||||||
|
- **Window sensors unavailable at startup are unknown, not closed.** Last known
|
||||||
|
states, open events and recovery are saved and restored as unknown until the
|
||||||
|
sensor reports; openings and closings count as confirmed only when seen; an
|
||||||
|
unknown window last seen open is assumed open only until the open-window time
|
||||||
|
after it was last seen; other unknown windows are flagged as uncertain in
|
||||||
|
forecasts (`uncertain_windows`) and in Status (`forecast_assumption`). Event
|
||||||
|
durations are given as a range when an end was not seen.
|
||||||
|
- **Unload, reload and removal cannot publish or save stale work.** The
|
||||||
|
coordinator is marked closing before platforms unload; the model save checks
|
||||||
|
this, and a removed-model mark, after taking the storage lock and holds the lock
|
||||||
|
until the file write finishes, so a deletion can no longer be undone by a save
|
||||||
|
that was waiting or cancelled mid-write. A failed setup shuts the coordinator
|
||||||
|
down; relay callbacks queued before unload are ignored.
|
||||||
|
- **Forecast Error compares the relay's actual timing with the plan.** The
|
||||||
|
deviation is the time the relay spent in another state than planned (unobserved
|
||||||
|
time counts as possible deviation). Up to 2 minutes: matched and scored; up to 5:
|
||||||
|
approximate, scored separately; more: counterfactual; otherwise incomplete, with
|
||||||
|
a reason (relay history, assumed inputs, no poll at the due time, missing
|
||||||
|
reading). Forecasts made with an unknown window or electrical heat are never
|
||||||
|
scored.
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
|
||||||
|
- Learning no longer substitutes values for an open window: samples where the
|
||||||
|
wind direction (for the directional term) or another zone's windows (for
|
||||||
|
cross-ventilation) were unknown are not learned. With every window closed, the
|
||||||
|
opening terms need no weather input, so recovery can be learned without wind.
|
||||||
|
- The gust option's hint refers to Model Error.
|
||||||
|
|
||||||
|
### Persisted data
|
||||||
|
|
||||||
|
- `schema_version` stays 2. `ventilation` gains `state` (last known window
|
||||||
|
states, open events, recovery); `evaluation` gains `excluded` (incomplete and
|
||||||
|
skipped spans); `operational_evaluation` gains `version` 2 and `approximate`.
|
||||||
|
Missing optional keys load as empty.
|
||||||
|
|
||||||
## 0.4.1 — 2026-10-27
|
## 0.4.1 — 2026-10-27
|
||||||
|
|
||||||
Stability and correctness: fixes four problems found in 0.4.0 and separates model
|
Stability and correctness: fixes four problems found in 0.4.0 and separates model
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
# Heating MPC 0.4.1 — custom integration for Home Assistant OS
|
# Heating MPC 0.4.2 — custom integration for Home Assistant OS
|
||||||
|
|
||||||
**Experimental, observation-only (shadow mode).** It **never** changes the Generic Thermostat, the physical relay, the boiler's setpoint, `heat/off`, or Comfort/Away/Sleep. No Docker add-on, MQTT or InfluxDB connection is needed.
|
**Experimental, observation-only (shadow mode).** It **never** changes the Generic Thermostat, the physical relay, the boiler's setpoint, `heat/off`, or Comfort/Away/Sleep. No Docker add-on, MQTT or InfluxDB connection is needed.
|
||||||
|
|
||||||
@@ -23,8 +23,8 @@ On Home Assistant 2026.3 or newer the integration shows its own icon (a radiator
|
|||||||
|
|
||||||
On **Settings → Devices & services → Heating MPC**, the buttons **Add zone**, **Add
|
On **Settings → Devices & services → Heating MPC**, the buttons **Add zone**, **Add
|
||||||
heat source** and **Add window** create items that can each be edited or deleted
|
heat source** and **Add window** create items that can each be edited or deleted
|
||||||
later. **Configure** holds everything else: thermostat, relay, weather sources,
|
later. **Configure** offers **Settings** (thermostat, relay, weather sources,
|
||||||
report-age limits, MPC settings and model options.
|
report-age limits, MPC settings and model options) and **Reset learned model**.
|
||||||
|
|
||||||
**Zones** are rooms or areas. Each has a name, one or more temperature sensors, a
|
**Zones** are rooms or areas. Each has a name, one or more temperature sensors, a
|
||||||
comfort weight (how much the zone matters to the MPC), an optional target offset
|
comfort weight (how much the zone matters to the MPC), an optional target offset
|
||||||
@@ -45,6 +45,15 @@ or the recovery time resets only the coefficients that depend on it. Comfort
|
|||||||
weight, target offset and names never touch the model. Each recalibration is
|
weight, target offset and names never touch the model. Each recalibration is
|
||||||
listed in the Status attribute `model_changes`.
|
listed in the Status attribute `model_changes`.
|
||||||
|
|
||||||
|
**Reset learned model** (in **Configure**, after ticking the confirmation) forgets
|
||||||
|
everything learned: every zone's coefficients, including those kept for removed
|
||||||
|
zones, with their confidence and sample counts, and the Model Error and Forecast
|
||||||
|
Error statistics. Zones, sensors, weights, heat sources, windows, weather sources
|
||||||
|
and settings stay as they are, and so do the heat sources' baselines and the window
|
||||||
|
event log. The integration restarts and learns from the beginning (first sample
|
||||||
|
after about 30 minutes, *Ready* after a day or more); `model_changes` records the
|
||||||
|
reset. It never touches the boiler or the thermostat.
|
||||||
|
|
||||||
**Heat sources** are equipment whose electricity ends up as heat in a zone, such as
|
**Heat sources** are equipment whose electricity ends up as heat in a zone, such as
|
||||||
a UPS feeding computers. Settings: an active-power sensor (W or kW; VA and % are
|
a UPS feeding computers. Settings: an active-power sensor (W or kW; VA and % are
|
||||||
rejected), the zone, the share of the power that becomes heat there (1.0 by
|
rejected), the zone, the share of the power that becomes heat there (1.0 by
|
||||||
@@ -57,12 +66,24 @@ modeled; the persistent load is already part of the zone's learned behavior.
|
|||||||
**Windows** need a contact sensor (binary sensor), the zone, whether the sensor
|
**Windows** need a contact sensor (binary sensor), the zone, whether the sensor
|
||||||
reports *on* when closed (inverted), the direction the outside of the window faces
|
reports *on* when closed (inverted), the direction the outside of the window faces
|
||||||
(degrees clockwise from north; default 345°, editable) and an opening factor
|
(degrees clockwise from north; default 345°, editable) and an opening factor
|
||||||
(1.0 normal, lower for tilted, higher for large). Unknown or unavailable states are
|
(1.0 normal, lower for tilted, higher for large).
|
||||||
treated as unknown, never as closed: a window last seen open counts as possibly
|
|
||||||
open (the forecast assumes it open, the zone's base model does not learn), and an
|
A window is **open**, **closed** or **unknown** (unknown, unavailable or no state);
|
||||||
opening that ended while the sensor was unknown is logged as unconfirmed, with its
|
unknown never means closed. While any window of a zone is unknown, the zone does
|
||||||
longest possible duration. Recovery after closing is learned only from a confirmed
|
not learn. What forecasts assume for an unknown window:
|
||||||
close.
|
|
||||||
|
- last seen open: open, but only until the configured open-window time after it was
|
||||||
|
last seen (never renewed), then closed and recovering;
|
||||||
|
- last seen closed, or never seen since setup: no ventilation, but the forecast is
|
||||||
|
flagged (`uncertain_windows` on the zone's forecast entity, `forecast_assumption`
|
||||||
|
in Status) and is never used to measure forecast accuracy.
|
||||||
|
|
||||||
|
An opening or closing is *confirmed* only when it was seen happen: the previous
|
||||||
|
reading was the other state, with no unknown period or restart in between. Events
|
||||||
|
report a duration only when both ends are confirmed, otherwise a minimum and, where
|
||||||
|
possible, a maximum; recovery is learned only after a confirmed close. The last
|
||||||
|
known states, an open event and a recovery in progress are saved, so a restart or
|
||||||
|
reload continues them, but they come back as unknown until the sensor reports.
|
||||||
|
|
||||||
Model options in **Configure**: how fast extra electrical heat is assumed to fade in
|
Model options in **Configure**: how fast extra electrical heat is assumed to fade in
|
||||||
forecasts (half-life, default 30 minutes), how long an open window is assumed to
|
forecasts (half-life, default 30 minutes), how long an open window is assumed to
|
||||||
@@ -177,9 +198,10 @@ the plan is discarded and the next poll computes a fresh one. Useful attributes:
|
|||||||
included sources), baseline, variable power above it, thermal response, whether it
|
included sources), baseline, variable power above it, thermal response, whether it
|
||||||
has settled.
|
has settled.
|
||||||
- `windows` — per window: state (`open`, `closed` or `unknown`), the last known
|
- `windows` — per window: state (`open`, `closed` or `unknown`), the last known
|
||||||
state and for how long the sensor has been unknown, minutes open, wind direction
|
state and for how long the sensor has been unknown, what forecasts assume
|
||||||
relative to the window's facing and the resulting exposure (1 = wind straight at
|
(`forecast_assumption`: `observed`, `last_known_open`, `last_known_closed` or
|
||||||
it).
|
`unobserved`; `forecast_open_minutes`), minutes open, wind direction relative to
|
||||||
|
the window's facing and the resulting exposure (1 = wind straight at it).
|
||||||
`ventilation` — per zone: current opening, recovery, temperature before opening,
|
`ventilation` — per zone: current opening, recovery, temperature before opening,
|
||||||
its contribution in °C/h and the ventilation coefficients. `ventilation_events` —
|
its contribution in °C/h and the ventilation coefficients. `ventilation_events` —
|
||||||
the last five openings with temperatures before, lowest, at closing and 30
|
the last five openings with temperatures before, lowest, at closing and 30
|
||||||
@@ -279,35 +301,57 @@ model is scored on the same spans. Each sensor's state is the mean absolute erro
|
|||||||
- `base_model_mae`, `improvement_vs_base` — the base model's error on the same
|
- `base_model_mae`, `improvement_vs_base` — the base model's error on the same
|
||||||
spans, and the share of it the extra layers remove (0 = no gain, negative = they
|
spans, and the share of it the extra layers remove (0 = no gain, negative = they
|
||||||
make it worse: switch that option off).
|
make it worse: switch that option off).
|
||||||
|
- `incomplete` — spans left out because an input the zone's model needed was not
|
||||||
|
observed at some step: a window state (or the start of a recovery), the wind,
|
||||||
|
wind direction or another zone's windows while a window was open, or, for the
|
||||||
|
extended layer, electrical heat or a neighbour (whose own span must be complete
|
||||||
|
too). The base model needs none of these, so its error on those spans is still
|
||||||
|
reported as `incomplete_base_model_mae`.
|
||||||
|
- `skipped` — spans that could not be replayed: the radiator-lag estimate was
|
||||||
|
uncertain along the way, or the zone had no reading at the end.
|
||||||
- `zones` — the same numbers per zone (not written to the Recorder).
|
- `zones` — the same numbers per zone (not written to the Recorder).
|
||||||
|
|
||||||
Spans are scored only where the inputs were known: polls 4.5–5.5 minutes apart, a
|
Spans need polls 4.5–5.5 minutes apart; a missing relay, outdoor temperature or
|
||||||
usable reading at the start and the end, and a well-known radiator-lag estimate
|
all zones restarts the replay window, so scores resume 15–120 minutes later (these
|
||||||
throughout. A missing relay, outdoor temperature or all zones restarts the replay
|
gaps are not counted). Statistics are kept with the model and survive restarts;
|
||||||
window, so scores resume 15–120 minutes later. Statistics are kept with the model
|
recalibrating a zone or resetting the learned model clears them.
|
||||||
and survive restarts; recalibrating a zone or deleting the model resets them.
|
|
||||||
|
|
||||||
### Forecast Error
|
### Forecast Error
|
||||||
|
|
||||||
Each time a plan is published, its forecast is recorded as it was: start
|
Each time a plan is published, its forecast is recorded as it was: start
|
||||||
temperatures, the trajectory, the plan, the model revision and the assumptions
|
temperatures, the trajectory, the plan, the model revision and the assumptions
|
||||||
(outdoor temperature and wind held, electrical heat fading, open windows closing).
|
(outdoor temperature and wind held, electrical heat fading, open windows closing).
|
||||||
When the forecast's horizon comes, it is scored against the reading, but only if
|
When the forecast's horizon comes, the relay's actual behaviour is compared with
|
||||||
the relay actually followed the plan: in every 15-minute block up to that horizon
|
the plan, block by block, from the relay's own switch times: the **deviation** is
|
||||||
the relay must have been in the planned state for at least 12 of the 15 minutes.
|
how long the relay was in another state than planned, so the same on-time at other
|
||||||
Otherwise the forecast is not an error measurement, and is counted instead:
|
times than planned counts as deviation. Relay time that was not observed counts as
|
||||||
|
possible deviation. Each forecast is then:
|
||||||
|
|
||||||
- `counterfactual` — the relay did something else: Generic Thermostat, which
|
- `matched` — at most 2 minutes of deviation. Only these make the Forecast Error
|
||||||
controls it, disagreed with the plan. Expected in observation mode whenever the
|
figures.
|
||||||
two disagree.
|
- `approximate` — at most 5 minutes (of which at most 2 unobserved), for example
|
||||||
- `unverifiable` — the relay history is incomplete for that span.
|
the thermostat switching a few minutes earlier than planned. Scored separately
|
||||||
- `missing` — no poll near the due time, or no reading for a zone.
|
in `approximate_stats`.
|
||||||
|
- `counterfactual` — more than 5 minutes: Generic Thermostat, which controls the
|
||||||
|
relay, did something else. Not an error of the forecast. Expected in observation
|
||||||
|
mode whenever the two disagree.
|
||||||
|
- `incomplete` — not comparable, with `incomplete_reasons`: `relay_history` (too
|
||||||
|
much relay time unknown), `uncertain_inputs` (the forecast had to assume an
|
||||||
|
unknown window or electrical heat), `no_poll` (no poll near the due time) or
|
||||||
|
`reading` (a zone had no reading; counted per zone).
|
||||||
|
|
||||||
The state is the 60-minute MAE once its quality is at least `indicative`; the
|
Why these limits: d seconds of boiler heat that the plan did not assume move a room
|
||||||
attribute `horizons` (not written to the Recorder) gives MAE, RMSE, bias, the
|
by roughly *heat* × d / 300, where *heat* is the learned boiler coefficient
|
||||||
no-change error, `improvement_vs_persistence`, sample counts, quality, the counts
|
(typically 0.05–0.12 °C per 5 minutes with warm radiators). 2 minutes stay below
|
||||||
above and the per-zone figures for each horizon. Statistics cover 7 days and
|
half a typical 0.1 °C sensor step and absorb timestamp jitter; 5 minutes are about
|
||||||
survive restarts; forecasts not yet due when HA restarts are dropped.
|
one step.
|
||||||
|
|
||||||
|
The state is the 60-minute MAE of matched forecasts once its quality is at least
|
||||||
|
`indicative`; the attribute `horizons` (not written to the Recorder) gives MAE,
|
||||||
|
RMSE, bias, the no-change error, `improvement_vs_persistence`, sample counts,
|
||||||
|
quality, the counts above, `approximate_stats` and the per-zone figures for each
|
||||||
|
horizon. Statistics cover 7 days and survive restarts; forecasts not yet due when
|
||||||
|
HA restarts are dropped.
|
||||||
|
|
||||||
What neither covers: outdoor temperature changes during the horizon (held
|
What neither covers: outdoor temperature changes during the horizon (held
|
||||||
constant, as in the MPC), and periods when the model was still learning (included,
|
constant, as in the MPC), and periods when the model was still learning (included,
|
||||||
@@ -350,9 +394,10 @@ The brand icons in `custom_components/heating_mpc/brand/` are drawn by
|
|||||||
`python scripts/make_brand_icons.py` (uses Pillow from the test requirements);
|
`python scripts/make_brand_icons.py` (uses Pillow from the test requirements);
|
||||||
the package check verifies their sizes and transparency.
|
the package check verifies their sizes and transparency.
|
||||||
|
|
||||||
HA tests cover setup, config/options forms, migration, reload (a single relay
|
HA tests cover setup, config/options forms (including the reset), migration,
|
||||||
listener and polling coordinator after repeated reloads) and entity cleanup,
|
reload (a single relay listener and polling coordinator after repeated reloads),
|
||||||
scheduled sampling and poll cancellation, the relay listener, stored model
|
unload, reload and removal while an optimization or a save is still running,
|
||||||
|
entity cleanup, scheduled sampling and poll cancellation, the relay listener, stored model
|
||||||
restoration, retention and deletion, missing inputs, manual off/presets, optimizer
|
restoration, retention and deletion, missing inputs, manual off/presets, optimizer
|
||||||
failure and Recorder exclusions. A fixture rejects all HA service calls throughout
|
failure and Recorder exclusions. A fixture rejects all HA service calls throughout
|
||||||
each test, and a static test fails if actuator service calls are added to the
|
each test, and a static test fails if actuator service calls are added to the
|
||||||
|
|||||||
+19
-4
@@ -95,6 +95,18 @@ changed during optimization or by an overtaken run; and model accuracy reported
|
|||||||
as if it were forecast accuracy. Reload/unload tests now cover repeated reloads
|
as if it were forecast accuracy. Reload/unload tests now cover repeated reloads
|
||||||
(one relay listener, one polling coordinator). Released as 0.4.1; not yet tested on HAOS.
|
(one relay listener, one polling coordinator). Released as 0.4.1; not yet tested on HAOS.
|
||||||
|
|
||||||
|
### Final stability fixes after 0.4.1 — 2026-10-27
|
||||||
|
|
||||||
|
An independent review of 0.4.1 reported four problems; three were reproduced and
|
||||||
|
the fourth (lifecycle races) confirmed by reading Home Assistant's unload order.
|
||||||
|
Fixed with regression tests (see AUDIT.md): Model Error spans with unknown inputs
|
||||||
|
are now `incomplete`, not scored; window sensors unavailable at startup are
|
||||||
|
unknown, with last known states saved and restored as unknown; unload, reload
|
||||||
|
and removal can no longer publish stale results or recreate a deleted model;
|
||||||
|
Forecast Error scores only forecasts whose plan the relay matched within 2
|
||||||
|
minutes. Also added: **Reset learned model** in Configure. Released as 0.4.2; not
|
||||||
|
yet tested on HAOS.
|
||||||
|
|
||||||
## Milestone 1 — Explainable forecasts and dashboards (target v0.3.0)
|
## Milestone 1 — Explainable forecasts and dashboards (target v0.3.0)
|
||||||
|
|
||||||
**Objective:** Users can verify what the model predicts and whether it is actually accurate.
|
**Objective:** Users can verify what the model predicts and whether it is actually accurate.
|
||||||
@@ -103,7 +115,7 @@ as if it were forecast accuracy. Reload/unload tests now cover repeated reloads
|
|||||||
- [x] Expose predicted +15/+30/+60/+120 minute temperatures and full proposed ON/OFF schedule. *(0.3.0: `forecast_15min` and `plan_15min` on each room forecast entity.)*
|
- [x] Expose predicted +15/+30/+60/+120 minute temperatures and full proposed ON/OFF schedule. *(0.3.0: `forecast_15min` and `plan_15min` on each room forecast entity.)*
|
||||||
- [ ] Provide an HA dashboard example using native cards or a clearly documented optional chart card (do not bundle a heavyweight frontend unnecessarily).
|
- [ ] Provide an HA dashboard example using native cards or a clearly documented optional chart card (do not bundle a heavyweight frontend unnecessarily).
|
||||||
- [ ] Make clear that the **MPC plan forecast is counterfactual** while Generic Thermostat is controlling the relay; add a separate actual-policy forecast/evaluation pathway. *(0.3.0: plan forecasts labelled counterfactual; evaluation uses the actual relay history. Stabilization: published forecasts are scored only when the relay followed their plan and otherwise counted as `counterfactual`. An actual-policy forecast, simulating Generic Thermostat, is still open.)*
|
- [ ] Make clear that the **MPC plan forecast is counterfactual** while Generic Thermostat is controlling the relay; add a separate actual-policy forecast/evaluation pathway. *(0.3.0: plan forecasts labelled counterfactual; evaluation uses the actual relay history. Stabilization: published forecasts are scored only when the relay followed their plan and otherwise counted as `counterfactual`. An actual-policy forecast, simulating Generic Thermostat, is still open.)*
|
||||||
- [ ] Store a bounded history of predictions with snapshots of inputs, proposed plan and eventually observed outcome. *(Stabilization: each published forecast is kept with its start readings, plan, model revision and assumptions until scored, in memory only; the scored statistics persist. No browsable history yet.)*
|
- [ ] Store a bounded history of predictions with snapshots of inputs, proposed plan and eventually observed outcome. *(Stabilization: each published forecast is kept with its start readings, plan, model revision and assumptions, including any it had to assume, until scored, in memory only; the scored statistics persist, split into matched, approximate, counterfactual and incomplete. No browsable history yet.)*
|
||||||
- [x] Implement out-of-sample/walk-forward MAE and bias at 15, 30, 60 and 120 minutes; include sample counts and confidence/quality labels. *(0.3.0: replay with parameters frozen at the forecast start and the actual relay history; `skill` against persistence. Stabilization: these are now the Model Error sensors, with RMSE, effective sample counts and quality labels; a separate Forecast Error sensor scores published forecasts. No confidence intervals yet.)*
|
- [x] Implement out-of-sample/walk-forward MAE and bias at 15, 30, 60 and 120 minutes; include sample counts and confidence/quality labels. *(0.3.0: replay with parameters frozen at the forecast start and the actual relay history; `skill` against persistence. Stabilization: these are now the Model Error sensors, with RMSE, effective sample counts and quality labels; a separate Forecast Error sensor scores published forecasts. No confidence intervals yet.)*
|
||||||
- [ ] Compare against simple benchmarks such as current-temperature persistence and a fixed thermal baseline. *(0.3.0: persistence done; fixed baseline open.)*
|
- [ ] Compare against simple benchmarks such as current-temperature persistence and a fixed thermal baseline. *(0.3.0: persistence done; fixed baseline open.)*
|
||||||
- [ ] Add inspectable explanations: expected heat loss, expected heating response, target, room weights and reason for the next proposed action.
|
- [ ] Add inspectable explanations: expected heat loss, expected heating response, target, room weights and reason for the next proposed action.
|
||||||
@@ -187,9 +199,12 @@ Items 1–3 of the original backlog are done (see AUDIT.md and CHANGELOG.md 0.2.
|
|||||||
3. Configure zones, the UPS heat source and the windows on HAOS; after a week of
|
3. Configure zones, the UPS heat source and the windows on HAOS; after a week of
|
||||||
heating review Model Error `improvement_vs_base` and `quality`, per-zone bias,
|
heating review Model Error `improvement_vs_base` and `quality`, per-zone bias,
|
||||||
coefficient `confidence` (base layer included), `model_changes` after any
|
coefficient `confidence` (base layer included), `model_changes` after any
|
||||||
reconfiguration, and the Forecast Error counts (how many forecasts were scored
|
reconfiguration, and the Forecast Error counts (matched, approximate,
|
||||||
vs counterfactual). Switch off optional terms that do not help. Also try an
|
counterfactual, incomplete and why). Switch off optional terms that do not
|
||||||
unavailable window sensor and a zone edit during HAOS acceptance.
|
help. During HAOS acceptance also try: a window sensor unavailable across an HA
|
||||||
|
restart, a zone edit, a reload while the integration is running, *Reset learned
|
||||||
|
model* and removing a disposable entry (its `.storage/heating_mpc.<entry_id>`
|
||||||
|
file must be gone afterwards).
|
||||||
4. Milestone 1 (rest): a dashboard example; forecast timestamps/provenance;
|
4. Milestone 1 (rest): a dashboard example; forecast timestamps/provenance;
|
||||||
explanations for the proposed action; an actual-policy forecast that simulates
|
explanations for the proposed action; an actual-policy forecast that simulates
|
||||||
Generic Thermostat.
|
Generic Thermostat.
|
||||||
|
|||||||
@@ -12,7 +12,7 @@ from homeassistant.helpers.event import async_track_state_change_event
|
|||||||
|
|
||||||
from .const import CONF_HEATER, CONF_ROOMS, CONF_WEIGHTS, DOMAIN, CONF_OUTSIDE, WEATHER_SOURCES
|
from .const import CONF_HEATER, CONF_ROOMS, CONF_WEIGHTS, DOMAIN, CONF_OUTSIDE, WEATHER_SOURCES
|
||||||
from .weather_sources import sources_for
|
from .weather_sources import sources_for
|
||||||
from .coordinator import HeatingMPCCoordinator, storage_lock
|
from .coordinator import HeatingMPCCoordinator, mark_model_removed, storage_lock
|
||||||
from .zones import SUBENTRY_ZONE, ZONE_KEY, ZONE_NAME, legacy_zone_data
|
from .zones import SUBENTRY_ZONE, ZONE_KEY, ZONE_NAME, legacy_zone_data
|
||||||
|
|
||||||
PLATFORMS: list[Platform] = [Platform.SENSOR, Platform.BINARY_SENSOR]
|
PLATFORMS: list[Platform] = [Platform.SENSOR, Platform.BINARY_SENSOR]
|
||||||
@@ -29,9 +29,11 @@ async def async_setup_entry(hass: HomeAssistant, entry: ConfigEntry) -> bool:
|
|||||||
hass.data.setdefault(DOMAIN, {})[entry.entry_id] = coordinator
|
hass.data.setdefault(DOMAIN, {})[entry.entry_id] = coordinator
|
||||||
await hass.config_entries.async_forward_entry_setups(entry, PLATFORMS)
|
await hass.config_entries.async_forward_entry_setups(entry, PLATFORMS)
|
||||||
except BaseException:
|
except BaseException:
|
||||||
# HA skips unload callbacks for unexpected setup errors, so clean up here.
|
# HA skips unload callbacks for unexpected setup errors, so clean up here,
|
||||||
|
# including a refresh scheduled by entities that were already added.
|
||||||
unsub_relay()
|
unsub_relay()
|
||||||
hass.data.get(DOMAIN, {}).pop(entry.entry_id, None)
|
hass.data.get(DOMAIN, {}).pop(entry.entry_id, None)
|
||||||
|
await coordinator.async_shutdown()
|
||||||
raise
|
raise
|
||||||
entry.async_on_unload(unsub_relay)
|
entry.async_on_unload(unsub_relay)
|
||||||
# Remove only obsolete zone forecasts belonging to this entry/platform. Zones
|
# Remove only obsolete zone forecasts belonging to this entry/platform. Zones
|
||||||
@@ -53,17 +55,25 @@ async def async_reload_entry(hass: HomeAssistant, entry: ConfigEntry) -> None:
|
|||||||
|
|
||||||
|
|
||||||
async def async_unload_entry(hass: HomeAssistant, entry: ConfigEntry) -> bool:
|
async def async_unload_entry(hass: HomeAssistant, entry: ConfigEntry) -> bool:
|
||||||
|
coordinator = hass.data.get(DOMAIN, {}).get(entry.entry_id)
|
||||||
|
if coordinator is not None:
|
||||||
|
# Before anything is unloaded: an update still running must not publish or save.
|
||||||
|
coordinator.async_begin_shutdown()
|
||||||
ok = await hass.config_entries.async_unload_platforms(entry, PLATFORMS)
|
ok = await hass.config_entries.async_unload_platforms(entry, PLATFORMS)
|
||||||
if ok:
|
if ok:
|
||||||
hass.data[DOMAIN].pop(entry.entry_id, None)
|
hass.data[DOMAIN].pop(entry.entry_id, None)
|
||||||
|
elif coordinator is not None:
|
||||||
|
coordinator.async_cancel_shutdown()
|
||||||
return ok
|
return ok
|
||||||
|
|
||||||
|
|
||||||
async def async_remove_entry(hass: HomeAssistant, entry: ConfigEntry) -> None:
|
async def async_remove_entry(hass: HomeAssistant, entry: ConfigEntry) -> None:
|
||||||
"""Remove the learned model when this integration instance is deleted."""
|
"""Remove the learned model when this integration instance is deleted."""
|
||||||
store = Store(hass, 1, f"{DOMAIN}.{entry.entry_id}")
|
store = Store(hass, 1, f"{DOMAIN}.{entry.entry_id}")
|
||||||
# Wait for a save still in progress, which would otherwise recreate the file.
|
# Wait for a save still in progress, which would otherwise recreate the file;
|
||||||
|
# a save still waiting for the lock sees the mark and does not write.
|
||||||
async with storage_lock(hass, entry.entry_id):
|
async with storage_lock(hass, entry.entry_id):
|
||||||
|
mark_model_removed(hass, entry.entry_id)
|
||||||
await store.async_remove()
|
await store.async_remove()
|
||||||
hass.data.get(f"{DOMAIN}_storage_locks", {}).pop(entry.entry_id, None)
|
hass.data.get(f"{DOMAIN}_storage_locks", {}).pop(entry.entry_id, None)
|
||||||
|
|
||||||
|
|||||||
@@ -23,6 +23,7 @@ from .heat_sources import (
|
|||||||
SOURCE_KEY, SOURCE_NAME, SOURCE_RESPONSE, SOURCE_SENSOR, SOURCE_ZONE, SUBENTRY_HEAT_SOURCE,
|
SOURCE_KEY, SOURCE_NAME, SOURCE_RESPONSE, SOURCE_SENSOR, SOURCE_ZONE, SUBENTRY_HEAT_SOURCE,
|
||||||
HeatSource, build_sources, validate_source,
|
HeatSource, build_sources, validate_source,
|
||||||
)
|
)
|
||||||
|
from .coordinator import async_reset_learned_model
|
||||||
from .weather_sources import sources_for
|
from .weather_sources import sources_for
|
||||||
from .windows import (
|
from .windows import (
|
||||||
DEFAULT_AZIMUTH, OPENING_RANGE, SUBENTRY_WINDOW, WINDOW_AZIMUTH, WINDOW_ENABLED, WINDOW_INVERTED,
|
DEFAULT_AZIMUTH, OPENING_RANGE, SUBENTRY_WINDOW, WINDOW_AZIMUTH, WINDOW_ENABLED, WINDOW_INVERTED,
|
||||||
@@ -205,6 +206,9 @@ class HeatingMPCConfigFlow(config_entries.ConfigFlow, domain=DOMAIN):
|
|||||||
SUBENTRY_WINDOW: WindowSubentryFlow}
|
SUBENTRY_WINDOW: WindowSubentryFlow}
|
||||||
|
|
||||||
|
|
||||||
|
CONFIRM_RESET = "confirm_reset"
|
||||||
|
|
||||||
|
|
||||||
class HeatingMPCOptionsFlow(config_entries.OptionsFlow):
|
class HeatingMPCOptionsFlow(config_entries.OptionsFlow):
|
||||||
"""Edit sensors, ranked alternative weather providers and MPC parameters."""
|
"""Edit sensors, ranked alternative weather providers and MPC parameters."""
|
||||||
|
|
||||||
@@ -215,13 +219,26 @@ class HeatingMPCOptionsFlow(config_entries.OptionsFlow):
|
|||||||
return [sensor for zone in entry_zones(self.config_entry) for sensor in zone.sensors]
|
return [sensor for zone in entry_zones(self.config_entry) for sensor in zone.sensors]
|
||||||
|
|
||||||
async def async_step_init(self, user_input: dict[str, Any] | None = None):
|
async def async_step_init(self, user_input: dict[str, Any] | None = None):
|
||||||
|
return self.async_show_menu(step_id="init", menu_options=["settings", "reset_model"])
|
||||||
|
|
||||||
|
async def async_step_settings(self, user_input: dict[str, Any] | None = None):
|
||||||
config = {**self.config_entry.data, **self.config_entry.options}
|
config = {**self.config_entry.data, **self.config_entry.options}
|
||||||
if user_input is not None:
|
if user_input is not None:
|
||||||
self._draft = user_input
|
self._draft = user_input
|
||||||
return await self.async_step_weather()
|
return await self.async_step_weather()
|
||||||
return self.async_show_form(step_id="init", data_schema=_main_schema(
|
return self.async_show_form(step_id="settings", data_schema=_main_schema(
|
||||||
config, detailed=True))
|
config, detailed=True))
|
||||||
|
|
||||||
|
async def async_step_reset_model(self, user_input: dict[str, Any] | None = None):
|
||||||
|
"""Forget the learned model after an explicit confirmation; settings stay."""
|
||||||
|
if user_input is not None:
|
||||||
|
if user_input.get(CONFIRM_RESET) is not True:
|
||||||
|
return self.async_abort(reason="model_not_reset")
|
||||||
|
await async_reset_learned_model(self.hass, self.config_entry)
|
||||||
|
return self.async_abort(reason="model_reset")
|
||||||
|
return self.async_show_form(step_id="reset_model", data_schema=vol.Schema(
|
||||||
|
{vol.Required(CONFIRM_RESET, default=False): selector.BooleanSelector()}))
|
||||||
|
|
||||||
async def async_step_weather(self, user_input: dict[str, Any] | None = None):
|
async def async_step_weather(self, user_input: dict[str, Any] | None = None):
|
||||||
config = {**self.config_entry.data, **self.config_entry.options, **self._draft}
|
config = {**self.config_entry.data, **self.config_entry.options, **self._draft}
|
||||||
if user_input is not None:
|
if user_input is not None:
|
||||||
|
|||||||
@@ -56,6 +56,43 @@ def storage_lock(hass: HomeAssistant, entry_id: str) -> asyncio.Lock:
|
|||||||
return hass.data.setdefault(f"{DOMAIN}_storage_locks", {}).setdefault(entry_id, asyncio.Lock())
|
return hass.data.setdefault(f"{DOMAIN}_storage_locks", {}).setdefault(entry_id, asyncio.Lock())
|
||||||
|
|
||||||
|
|
||||||
|
def mark_model_removed(hass: HomeAssistant, entry_id: str) -> None:
|
||||||
|
"""Record, under the storage lock, that the entry's model file was deleted for good."""
|
||||||
|
hass.data.setdefault(f"{DOMAIN}_removed", set()).add(entry_id)
|
||||||
|
|
||||||
|
|
||||||
|
def model_removed(hass: HomeAssistant, entry_id: str) -> bool:
|
||||||
|
return entry_id in hass.data.get(f"{DOMAIN}_removed", ())
|
||||||
|
|
||||||
|
|
||||||
|
# Stored keys holding what was learned. The rest is observation, not model: the
|
||||||
|
# heat sources' baseline samples, the window history and the heater's identity.
|
||||||
|
LEARNED_KEYS = ("learners", "retired_learners", "zone_models", "ventilation_models", "evaluation",
|
||||||
|
"operational_evaluation", "fingerprint_version", "fingerprints", "on_samples", "off_samples")
|
||||||
|
|
||||||
|
|
||||||
|
async def async_reset_learned_model(hass: HomeAssistant, entry: ConfigEntry) -> None:
|
||||||
|
"""Forget everything learned for this entry; the configuration is not touched.
|
||||||
|
|
||||||
|
The entry is unloaded first, so no running update can save the old model
|
||||||
|
afterwards, then set up again from the stripped data. Accuracy statistics go
|
||||||
|
too: they describe the model being discarded.
|
||||||
|
"""
|
||||||
|
await hass.config_entries.async_unload(entry.entry_id)
|
||||||
|
store = Store(hass, 1, f"{DOMAIN}.{entry.entry_id}")
|
||||||
|
async with storage_lock(hass, entry.entry_id):
|
||||||
|
stored = await store.async_load()
|
||||||
|
if isinstance(stored, dict):
|
||||||
|
kept = {key: value for key, value in stored.items() if key not in LEARNED_KEYS}
|
||||||
|
changes = kept.get("model_changes") if isinstance(kept.get("model_changes"), list) else []
|
||||||
|
kept["model_changes"] = [*changes, {
|
||||||
|
"time": datetime.now(timezone.utc).isoformat(timespec="seconds"), "zone": "all",
|
||||||
|
"recalibrated": ["all"], "reasons": ["learned model reset by the user"]}][-MAX_MODEL_CHANGES:]
|
||||||
|
await store.async_save(kept)
|
||||||
|
_LOGGER.info("Heating MPC: learned model reset by the user")
|
||||||
|
await hass.config_entries.async_setup(entry.entry_id)
|
||||||
|
|
||||||
|
|
||||||
def entry_zone_list(entry: ConfigEntry, config: dict[str, Any]) -> list[Zone]:
|
def entry_zone_list(entry: ConfigEntry, config: dict[str, Any]) -> list[Zone]:
|
||||||
"""Zones from the entry's zone subentries; one zone per legacy room sensor otherwise."""
|
"""Zones from the entry's zone subentries; one zone per legacy room sensor otherwise."""
|
||||||
subentries = getattr(entry, "subentries", None) or {}
|
subentries = getattr(entry, "subentries", None) or {}
|
||||||
@@ -148,18 +185,57 @@ class HeatingMPCCoordinator(DataUpdateCoordinator[dict[str, Any]]):
|
|||||||
self._off_samples = 0
|
self._off_samples = 0
|
||||||
# Increases with every update; an older update finishing late never publishes.
|
# Increases with every update; an older update finishing late never publishes.
|
||||||
self._run = 0
|
self._run = 0
|
||||||
|
self._closing = False
|
||||||
|
|
||||||
async def async_restore_model(self) -> None:
|
async def async_restore_model(self) -> None:
|
||||||
async with storage_lock(self.hass, self.entry.entry_id):
|
async with storage_lock(self.hass, self.entry.entry_id):
|
||||||
await self._async_restore_model()
|
await self._async_restore_model()
|
||||||
|
|
||||||
async def _async_save(self, outside_source: str | None) -> None:
|
async def _async_save(self, outside_source: str | None) -> bool:
|
||||||
|
"""Write the model unless unloading began or the model was deleted.
|
||||||
|
|
||||||
|
Shielded: unloading cancels a running scheduled update, but the file write
|
||||||
|
itself continues in a worker thread. The lock must stay held until that
|
||||||
|
write has finished, or a deletion could run first and be undone by it.
|
||||||
|
"""
|
||||||
|
write = self.hass.async_create_task(self._async_write(outside_source), f"{DOMAIN} model save")
|
||||||
|
return await asyncio.shield(write)
|
||||||
|
|
||||||
|
async def _async_write(self, outside_source: str | None) -> bool:
|
||||||
async with storage_lock(self.hass, self.entry.entry_id):
|
async with storage_lock(self.hass, self.entry.entry_id):
|
||||||
|
# Checked after acquiring the lock, not before: a save that waited for
|
||||||
|
# a deletion to finish must not write the file again.
|
||||||
|
if self.closing or model_removed(self.hass, self.entry.entry_id):
|
||||||
|
return False
|
||||||
await self._store.async_save(self._payload(outside_source))
|
await self._store.async_save(self._payload(outside_source))
|
||||||
|
return True
|
||||||
|
|
||||||
|
@property
|
||||||
|
def closing(self) -> bool:
|
||||||
|
"""True from the start of unloading: nothing is published, saved or recorded."""
|
||||||
|
return self._closing or getattr(self, "_shutdown_requested", False)
|
||||||
|
|
||||||
|
@callback
|
||||||
|
def async_begin_shutdown(self) -> None:
|
||||||
|
"""Mark unloading before anything is unloaded.
|
||||||
|
|
||||||
|
Home Assistant's own shutdown of the coordinator runs only after platforms
|
||||||
|
are unloaded; an update still running in that window must not publish or save.
|
||||||
|
"""
|
||||||
|
self._closing = True
|
||||||
|
|
||||||
|
@callback
|
||||||
|
def async_cancel_shutdown(self) -> None:
|
||||||
|
"""Unloading failed and the entry stays loaded: carry on."""
|
||||||
|
self._closing = False
|
||||||
|
|
||||||
|
async def async_shutdown(self) -> None:
|
||||||
|
self._closing = True
|
||||||
|
await super().async_shutdown()
|
||||||
|
|
||||||
def _stale(self, run: int) -> bool:
|
def _stale(self, run: int) -> bool:
|
||||||
"""True when a newer update started or the entry is unloading."""
|
"""True when a newer update started or the entry is unloading."""
|
||||||
return run != self._run or getattr(self, "_shutdown_requested", False)
|
return run != self._run or self.closing
|
||||||
|
|
||||||
async def _async_restore_model(self) -> None:
|
async def _async_restore_model(self) -> None:
|
||||||
stored = await self._store.async_load() or {}
|
stored = await self._store.async_load() or {}
|
||||||
@@ -303,6 +379,8 @@ class HeatingMPCCoordinator(DataUpdateCoordinator[dict[str, Any]]):
|
|||||||
@callback
|
@callback
|
||||||
def async_relay_changed(self, event: Event) -> None:
|
def async_relay_changed(self, event: Event) -> None:
|
||||||
"""Record relay state changes as they happen; read-only state listener."""
|
"""Record relay state changes as they happen; read-only state listener."""
|
||||||
|
if self.closing:
|
||||||
|
return # a callback queued before the listener was removed
|
||||||
old, new = event.data.get("old_state"), event.data.get("new_state")
|
old, new = event.data.get("old_state"), event.data.get("new_state")
|
||||||
if new is None:
|
if new is None:
|
||||||
self.record_relay_state(event.time_fired, "removed")
|
self.record_relay_state(event.time_fired, "removed")
|
||||||
@@ -442,7 +520,9 @@ class HeatingMPCCoordinator(DataUpdateCoordinator[dict[str, Any]]):
|
|||||||
details[key] = {"temperature": round(temp, 3),
|
details[key] = {"temperature": round(temp, 3),
|
||||||
"model_in_use": "extended" if model.uses_extended else "base",
|
"model_in_use": "extended" if model.uses_extended else "base",
|
||||||
"contributions_c_per_h": model.contributions(temp, inputs),
|
"contributions_c_per_h": model.contributions(temp, inputs),
|
||||||
"coefficients": coefficients}
|
"coefficients": coefficients,
|
||||||
|
# Windows whose state the forecast had to assume (sensor unknown).
|
||||||
|
"uncertain_windows": list((vent_inputs.get(key) or {}).get("unknown_windows", []))}
|
||||||
return details
|
return details
|
||||||
|
|
||||||
def _window_diagnostics(self, now: float, weather: dict[str, Any], temps: dict[str, float | None],
|
def _window_diagnostics(self, now: float, weather: dict[str, Any], temps: dict[str, float | None],
|
||||||
@@ -453,11 +533,14 @@ class HeatingMPCCoordinator(DataUpdateCoordinator[dict[str, Any]]):
|
|||||||
is_open, since = info.get("open"), info.get("since")
|
is_open, since = info.get("open"), info.get("since")
|
||||||
direction = weather.get("wind_direction")
|
direction = weather.get("wind_direction")
|
||||||
last_known, unknown_since = info.get("last_known_open"), info.get("unknown_since")
|
last_known, unknown_since = info.get("last_known_open"), info.get("unknown_since")
|
||||||
|
assumed = self._ventilation.assumption(window, float(self.config[CONF_WINDOW_OPEN_MINUTES]), now)
|
||||||
windows[window.name] = {
|
windows[window.name] = {
|
||||||
"zone": self.zones[window.zone].name, "sensor": window.sensor, "enabled": window.enabled,
|
"zone": self.zones[window.zone].name, "sensor": window.sensor, "enabled": window.enabled,
|
||||||
"state": "unknown" if is_open is None else "open" if is_open else "closed",
|
"state": "unknown" if is_open is None else "open" if is_open else "closed",
|
||||||
"last_known_state": None if last_known is None else "open" if last_known else "closed",
|
"last_known_state": None if last_known is None else "open" if last_known else "closed",
|
||||||
"unknown_minutes": round((now - unknown_since) / 60, 1) if unknown_since else None,
|
"unknown_minutes": round((now - unknown_since) / 60, 1) if unknown_since else None,
|
||||||
|
# What forecasts assume: observed, or a bounded guess from the last reading.
|
||||||
|
"forecast_assumption": assumed["basis"], "forecast_open_minutes": assumed["open_minutes"],
|
||||||
"open_minutes": round((now - since) / 60, 1) if is_open and since else None,
|
"open_minutes": round((now - since) / 60, 1) if is_open and since else None,
|
||||||
"azimuth": window.azimuth, "opening_factor": window.opening, "inverted": window.inverted,
|
"azimuth": window.azimuth, "opening_factor": window.opening, "inverted": window.inverted,
|
||||||
"relative_wind_direction": info.get("relative_wind_direction"),
|
"relative_wind_direction": info.get("relative_wind_direction"),
|
||||||
@@ -527,6 +610,8 @@ class HeatingMPCCoordinator(DataUpdateCoordinator[dict[str, Any]]):
|
|||||||
snapshot = {entity: self.hass.states.get(entity)
|
snapshot = {entity: self.hass.states.get(entity)
|
||||||
for entity in set(self.sensors) | weather_entity_ids}
|
for entity in set(self.sensors) | weather_entity_ids}
|
||||||
self._run += 1
|
self._run += 1
|
||||||
|
if self.closing:
|
||||||
|
return self._superseded({"status": "inputs_changed"})
|
||||||
run = self._run
|
run = self._run
|
||||||
inputs_before = self._input_snapshot()
|
inputs_before = self._input_snapshot()
|
||||||
sensor_quality = {}
|
sensor_quality = {}
|
||||||
@@ -719,8 +804,10 @@ class HeatingMPCCoordinator(DataUpdateCoordinator[dict[str, Any]]):
|
|||||||
self.ventilation.get(room))) for room in vals},
|
self.ventilation.get(room))) for room in vals},
|
||||||
inputs={room: {"internal": internal[room], "ventilation": vent_inputs.get(room)} for room in vals})
|
inputs={room: {"internal": internal[room], "ventilation": vent_inputs.get(room)} for room in vals})
|
||||||
status["forecast_accuracy"] = self._evaluator.summary(self.rooms)
|
status["forecast_accuracy"] = self._evaluator.summary(self.rooms)
|
||||||
if learned and not self._stale(run):
|
# Persist at each sample so restart does not erase hours of training, and when
|
||||||
# Persist at each sample so restart does not erase hours of training.
|
# a window's state changes so a restart continues from it.
|
||||||
|
window_changed, self._ventilation.changed = self._ventilation.changed, False
|
||||||
|
if (learned or window_changed) and not self._stale(run):
|
||||||
await self._async_save(active_sources["temperature"])
|
await self._async_save(active_sources["temperature"])
|
||||||
if self._stale(run):
|
if self._stale(run):
|
||||||
return self._superseded(status)
|
return self._superseded(status)
|
||||||
@@ -808,9 +895,16 @@ class HeatingMPCCoordinator(DataUpdateCoordinator[dict[str, Any]]):
|
|||||||
_LOGGER.warning("MPC prediction failed: %s", err)
|
_LOGGER.warning("MPC prediction failed: %s", err)
|
||||||
status["status"] = "prediction_error"
|
status["status"] = "prediction_error"
|
||||||
return status
|
return status
|
||||||
|
# Inputs the forecast had to assume: such forecasts are not compared with reality.
|
||||||
|
uncertain = [f"window: {name}" for room in vals
|
||||||
|
for name in (vent_inputs.get(room) or {}).get("unknown_windows", [])]
|
||||||
|
uncertain += [f"electrical heat: {self.zones[room].name}" for room in vals
|
||||||
|
if internal[room] is None and room in self.extended
|
||||||
|
and ZoneModel(self.learners[room], self.extended[room]).uses_extended]
|
||||||
self._forecasts.record(now.timestamp(), vals, output["trajectory_15min"], output["plan_15min"],
|
self._forecasts.record(now.timestamp(), vals, output["trajectory_15min"], output["plan_15min"],
|
||||||
self._revision(), {
|
self._revision(), {
|
||||||
"strategy": "mpc_plan", "outside": outside, "wind": weather.get("wind"),
|
"strategy": "mpc_plan", "outside": outside, "wind": weather.get("wind"),
|
||||||
|
"uncertain_inputs": uncertain,
|
||||||
"internal_half_life_minutes": float(self.config.get(CONF_INTERNAL_HALF_LIFE, 30)),
|
"internal_half_life_minutes": float(self.config.get(CONF_INTERNAL_HALF_LIFE, 30)),
|
||||||
"window_open_minutes": float(self.config[CONF_WINDOW_OPEN_MINUTES]),
|
"window_open_minutes": float(self.config[CONF_WINDOW_OPEN_MINUTES]),
|
||||||
"internal_kw": {room: internal[room] for room in vals},
|
"internal_kw": {room: internal[room] for room in vals},
|
||||||
|
|||||||
@@ -34,7 +34,7 @@ from math import isfinite, sqrt
|
|||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
from .physics import Learner
|
from .physics import Learner
|
||||||
from .windows import ventilation_vector_for_prediction
|
from .windows import ventilation_known, ventilation_vector_for_prediction
|
||||||
from .zone_model import feature_vector
|
from .zone_model import feature_vector
|
||||||
|
|
||||||
HORIZONS = (15, 30, 60, 120) # minutes
|
HORIZONS = (15, 30, 60, 120) # minutes
|
||||||
@@ -97,19 +97,50 @@ def snapshot_delta(snapshot: dict[str, Any], temp: float, inputs: dict[str, Any]
|
|||||||
return main
|
return main
|
||||||
|
|
||||||
|
|
||||||
def replay_zones(origin: dict[str, Any], path: list[dict[str, Any]]) -> dict[str, float]:
|
def inputs_complete(snapshot: dict[str, Any], temp: float, inputs: dict[str, Any],
|
||||||
"""All zones advanced together with the origin's models and the actual inputs."""
|
incomplete: set[str]) -> bool:
|
||||||
|
"""Whether a step of a zone's model used only observed inputs.
|
||||||
|
|
||||||
|
Not when a window state, or a weather input an open window's terms need, was
|
||||||
|
unknown; nor when the extended layer is in use and its electrical heat or a
|
||||||
|
neighbour is unknown, or that neighbour's own replay is already incomplete.
|
||||||
|
The replay itself still falls back as forecasts do; the result is just not a
|
||||||
|
measurement of the model.
|
||||||
|
"""
|
||||||
|
vent_features = snapshot["ventilation"][0] if snapshot.get("ventilation") else ()
|
||||||
|
if not ventilation_known(vent_features, inputs.get("ventilation")):
|
||||||
|
return False
|
||||||
|
if snapshot["uses_extended"] and snapshot["extended"]:
|
||||||
|
features = snapshot["extended"][0]
|
||||||
|
if feature_vector(features, temp, inputs) is None:
|
||||||
|
return False
|
||||||
|
if any(f.startswith("couple:") and f.split(":", 1)[1] in incomplete for f in features):
|
||||||
|
return False
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
def replay_zones(origin: dict[str, Any], path: list[dict[str, Any]]) -> tuple[dict[str, float], set[str]]:
|
||||||
|
"""All zones advanced together with the origin's models and the actual inputs.
|
||||||
|
|
||||||
|
Returns the replayed temperatures and the zones whose replay needed an input
|
||||||
|
that was not observed at some step (see inputs_complete).
|
||||||
|
"""
|
||||||
state = dict(origin["temps"])
|
state = dict(origin["temps"])
|
||||||
|
incomplete: set[str] = set()
|
||||||
for before, poll in zip(path, path[1:]):
|
for before, poll in zip(path, path[1:]):
|
||||||
seconds = (poll["time"] - before["time"]).total_seconds()
|
seconds = (poll["time"] - before["time"]).total_seconds()
|
||||||
new = {}
|
new = {}
|
||||||
|
unknown = set(incomplete)
|
||||||
for zone, temp in state.items():
|
for zone, temp in state.items():
|
||||||
actual = poll["inputs"].get(zone, {})
|
actual = poll["inputs"].get(zone, {})
|
||||||
inputs = {"outside": origin["outside"], "lag": poll["lag"], "temps": state,
|
inputs = {"outside": origin["outside"], "lag": poll["lag"], "temps": state,
|
||||||
"internal": actual.get("internal"), "ventilation": actual.get("ventilation")}
|
"internal": actual.get("internal"), "ventilation": actual.get("ventilation")}
|
||||||
new[zone] = temp + snapshot_delta(origin["models"][zone], temp, inputs) * seconds / STEP_SECONDS
|
snapshot = origin["models"][zone]
|
||||||
state = new
|
if not inputs_complete(snapshot, temp, inputs, incomplete):
|
||||||
return state
|
unknown.add(zone)
|
||||||
|
new[zone] = temp + snapshot_delta(snapshot, temp, inputs) * seconds / STEP_SECONDS
|
||||||
|
state, incomplete = new, unknown
|
||||||
|
return state, incomplete
|
||||||
|
|
||||||
|
|
||||||
class ForecastEvaluator:
|
class ForecastEvaluator:
|
||||||
@@ -121,6 +152,9 @@ class ForecastEvaluator:
|
|||||||
# UTC day -> horizon -> room -> [count, sum error, sum |error|, sum |persistence error|,
|
# UTC day -> horizon -> room -> [count, sum error, sum |error|, sum |persistence error|,
|
||||||
# sum |base model error|, sum error^2, count with squares (0 for 0.3/0.4 data)]
|
# sum |base model error|, sum error^2, count with squares (0 for 0.3/0.4 data)]
|
||||||
self._days: dict[str, dict[str, dict[str, list[float]]]] = {}
|
self._days: dict[str, dict[str, dict[str, list[float]]]] = {}
|
||||||
|
# UTC day -> horizon -> room -> [incomplete spans, skipped spans, sum |base model error|
|
||||||
|
# over the incomplete ones]. Neither kind enters the statistics above.
|
||||||
|
self._excluded: dict[str, dict[str, dict[str, list[float]]]] = {}
|
||||||
|
|
||||||
def reset_window(self) -> None:
|
def reset_window(self) -> None:
|
||||||
"""Inputs were missing: no forecast may be scored across this poll."""
|
"""Inputs were missing: no forecast may be scored across this poll."""
|
||||||
@@ -147,6 +181,11 @@ class ForecastEvaluator:
|
|||||||
self._score()
|
self._score()
|
||||||
self._prune(time)
|
self._prune(time)
|
||||||
|
|
||||||
|
def _exclude(self, day: str, horizon: int, room: str, kind: int, base_error: float = 0.0) -> None:
|
||||||
|
bucket = self._excluded.setdefault(day, {}).setdefault(str(horizon), {}).setdefault(room, [0, 0, 0.0])
|
||||||
|
bucket[kind] += 1
|
||||||
|
bucket[2] += abs(base_error)
|
||||||
|
|
||||||
def _score(self) -> None:
|
def _score(self) -> None:
|
||||||
history = list(self._history)
|
history = list(self._history)
|
||||||
latest = history[-1]
|
latest = history[-1]
|
||||||
@@ -156,19 +195,27 @@ class ForecastEvaluator:
|
|||||||
if len(history) < steps + 1:
|
if len(history) < steps + 1:
|
||||||
continue
|
continue
|
||||||
path = history[-(steps + 1):]
|
path = history[-(steps + 1):]
|
||||||
|
origin = path[0]
|
||||||
# The actual heat input must be known along the way.
|
# The actual heat input must be known along the way.
|
||||||
if any(poll["lag_uncertainty"] > self._max_lag_uncertainty for poll in path[1:]):
|
if any(poll["lag_uncertainty"] > self._max_lag_uncertainty for poll in path[1:]):
|
||||||
|
for room in origin["temps"]:
|
||||||
|
self._exclude(day, horizon, room, SKIPPED)
|
||||||
continue
|
continue
|
||||||
origin = path[0]
|
|
||||||
inputs = [(poll["lag"], (poll["time"] - before["time"]).total_seconds())
|
inputs = [(poll["lag"], (poll["time"] - before["time"]).total_seconds())
|
||||||
for before, poll in zip(path, path[1:])]
|
for before, poll in zip(path, path[1:])]
|
||||||
full = replay_zones(origin, path)
|
full, incomplete = replay_zones(origin, path)
|
||||||
for room, actual in latest["temps"].items():
|
for room, start in origin["temps"].items():
|
||||||
start, theta = origin["temps"].get(room), origin["theta"].get(room)
|
actual, theta = latest["temps"].get(room), origin["theta"].get(room)
|
||||||
if start is None or theta is None:
|
if actual is None or theta is None:
|
||||||
|
self._exclude(day, horizon, room, SKIPPED) # no reading at the end
|
||||||
continue
|
continue
|
||||||
error = full[room] - actual
|
error = full[room] - actual
|
||||||
base_error = replay(start, origin["outside"], theta, inputs) - actual
|
base_error = replay(start, origin["outside"], theta, inputs) - actual
|
||||||
|
if room in incomplete:
|
||||||
|
# The base model needs only the relay and outdoor temperature: its
|
||||||
|
# error is still a measurement, kept apart from the main figures.
|
||||||
|
self._exclude(day, horizon, room, INCOMPLETE, base_error)
|
||||||
|
continue
|
||||||
bucket = self._days.setdefault(day, {}).setdefault(str(horizon), {}).setdefault(
|
bucket = self._days.setdefault(day, {}).setdefault(str(horizon), {}).setdefault(
|
||||||
room, [0, 0.0, 0.0, 0.0, 0.0, 0.0, 0])
|
room, [0, 0.0, 0.0, 0.0, 0.0, 0.0, 0])
|
||||||
bucket[0] += 1
|
bucket[0] += 1
|
||||||
@@ -181,8 +228,9 @@ class ForecastEvaluator:
|
|||||||
|
|
||||||
def _prune(self, time: datetime) -> None:
|
def _prune(self, time: datetime) -> None:
|
||||||
oldest = (time.date() - timedelta(days=WINDOW_DAYS - 1)).isoformat()
|
oldest = (time.date() - timedelta(days=WINDOW_DAYS - 1)).isoformat()
|
||||||
for day in [day for day in self._days if day < oldest]:
|
for store in (self._days, self._excluded):
|
||||||
del self._days[day]
|
for day in [day for day in store if day < oldest]:
|
||||||
|
del store[day]
|
||||||
|
|
||||||
def summary(self, rooms: list[str]) -> dict[int, dict[str, Any]]:
|
def summary(self, rooms: list[str]) -> dict[int, dict[str, Any]]:
|
||||||
"""Error statistics per horizon, overall and per configured room."""
|
"""Error statistics per horizon, overall and per configured room."""
|
||||||
@@ -196,15 +244,27 @@ class ForecastEvaluator:
|
|||||||
for i, value in enumerate(values):
|
for i, value in enumerate(values):
|
||||||
totals[i] += value
|
totals[i] += value
|
||||||
overall = [sum(values[i] for values in per_room.values()) for i in range(7)]
|
overall = [sum(values[i] for values in per_room.values()) for i in range(7)]
|
||||||
result[horizon] = {**_stats(overall, horizon, len(per_room)),
|
excluded: dict[str, list[float]] = {}
|
||||||
"rooms": {room: _stats(values, horizon) for room, values in per_room.items()}}
|
for day in self._excluded.values():
|
||||||
|
for room, values in day.get(str(horizon), {}).items():
|
||||||
|
if room in rooms:
|
||||||
|
totals = excluded.setdefault(room, [0, 0, 0.0])
|
||||||
|
for i, value in enumerate(values):
|
||||||
|
totals[i] += value
|
||||||
|
all_excluded = [sum(values[i] for values in excluded.values()) for i in range(3)]
|
||||||
|
result[horizon] = {**_stats(overall, horizon, len(per_room)), **_excluded_stats(all_excluded),
|
||||||
|
"rooms": {room: {**_stats(per_room.get(room, [0, 0.0, 0.0, 0.0, 0.0, 0.0, 0]),
|
||||||
|
horizon),
|
||||||
|
**_excluded_stats(excluded.get(room, [0, 0, 0.0]))}
|
||||||
|
for room in rooms if room in per_room or room in excluded}}
|
||||||
return result
|
return result
|
||||||
|
|
||||||
def forget(self, room: str) -> None:
|
def forget(self, room: str) -> None:
|
||||||
"""Drop a zone's statistics, for example when its temperature is measured differently."""
|
"""Drop a zone's statistics, for example when its temperature is measured differently."""
|
||||||
for by_horizon in self._days.values():
|
for store in (self._days, self._excluded):
|
||||||
for by_room in by_horizon.values():
|
for by_horizon in store.values():
|
||||||
by_room.pop(room, None)
|
for by_room in by_horizon.values():
|
||||||
|
by_room.pop(room, None)
|
||||||
for poll in self._history:
|
for poll in self._history:
|
||||||
poll["temps"].pop(room, None)
|
poll["temps"].pop(room, None)
|
||||||
|
|
||||||
@@ -213,11 +273,20 @@ class ForecastEvaluator:
|
|||||||
return {"days": {day: {horizon: {room: [v[0], *(round(x, 6) for x in v[1:6]), v[6]]
|
return {"days": {day: {horizon: {room: [v[0], *(round(x, 6) for x in v[1:6]), v[6]]
|
||||||
for room, v in by_room.items()}
|
for room, v in by_room.items()}
|
||||||
for horizon, by_room in by_horizon.items()}
|
for horizon, by_room in by_horizon.items()}
|
||||||
for day, by_horizon in self._days.items()}}
|
for day, by_horizon in self._days.items()},
|
||||||
|
"excluded": {day: {horizon: {room: [v[0], v[1], round(v[2], 6)] for room, v in by_room.items()}
|
||||||
|
for horizon, by_room in by_horizon.items()}
|
||||||
|
for day, by_horizon in self._excluded.items()}}
|
||||||
|
|
||||||
def load(self, data: Any) -> None:
|
def load(self, data: Any) -> None:
|
||||||
"""Restore stored statistics, keeping only well-formed entries."""
|
"""Restore stored statistics, keeping only well-formed entries."""
|
||||||
self._days = {}
|
self._days, self._excluded = {}, {}
|
||||||
|
excluded = data.get("excluded") if isinstance(data, dict) else None
|
||||||
|
for day, horizon, room, values in _entries(excluded):
|
||||||
|
if (isinstance(values, list) and len(values) == 3 and all(type(v) is int and v >= 0 for v in values[:2])
|
||||||
|
and type(values[2]) in (int, float) and isfinite(values[2]) and values[2] >= 0):
|
||||||
|
self._excluded.setdefault(day, {}).setdefault(horizon, {})[room] = [
|
||||||
|
values[0], values[1], float(values[2])]
|
||||||
days = data.get("days") if isinstance(data, dict) else None
|
days = data.get("days") if isinstance(data, dict) else None
|
||||||
if not isinstance(days, dict):
|
if not isinstance(days, dict):
|
||||||
return
|
return
|
||||||
@@ -243,6 +312,30 @@ class ForecastEvaluator:
|
|||||||
*squares]
|
*squares]
|
||||||
|
|
||||||
|
|
||||||
|
INCOMPLETE, SKIPPED = 0, 1 # columns of the excluded-span counts
|
||||||
|
|
||||||
|
|
||||||
|
def _entries(days: Any):
|
||||||
|
"""(day, horizon, zone, values) of stored per-day statistics with a valid shape."""
|
||||||
|
if not isinstance(days, dict):
|
||||||
|
return
|
||||||
|
horizons = {str(horizon) for horizon in HORIZONS}
|
||||||
|
for day, by_horizon in days.items():
|
||||||
|
if not _valid_day(day) or not isinstance(by_horizon, dict):
|
||||||
|
continue
|
||||||
|
for horizon, by_zone in by_horizon.items():
|
||||||
|
if horizon in horizons and isinstance(by_zone, dict):
|
||||||
|
for zone, values in by_zone.items():
|
||||||
|
if isinstance(zone, str):
|
||||||
|
yield day, horizon, zone, values
|
||||||
|
|
||||||
|
|
||||||
|
def _excluded_stats(values: list[float]) -> dict[str, Any]:
|
||||||
|
incomplete, skipped, base = values
|
||||||
|
return {"incomplete": int(incomplete), "skipped": int(skipped),
|
||||||
|
"incomplete_base_model_mae": round(base / incomplete, 3) if incomplete else None}
|
||||||
|
|
||||||
|
|
||||||
def _valid_bucket(room: Any, values: Any) -> bool:
|
def _valid_bucket(room: Any, values: Any) -> bool:
|
||||||
if not isinstance(room, str) or not isinstance(values, list) or len(values) not in (4, 5, 7):
|
if not isinstance(room, str) or not isinstance(values, list) or len(values) not in (4, 5, 7):
|
||||||
return False
|
return False
|
||||||
@@ -278,11 +371,24 @@ def _stats(values: list[float], horizon: int, zones: int = 1) -> dict[str, Any]:
|
|||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
# A forecast's plan counts as executed when, in every 15-minute block, the share
|
# How far the relay may depart from a forecast's plan (seconds in another state than
|
||||||
# of time the relay was on is within this of what the plan assumed (0 or 1).
|
# planned, up to the horizon) for the forecast to be compared with what happened.
|
||||||
PLAN_TOLERANCE = 0.2
|
# d seconds of boiler heat that the plan did not assume (or the reverse) move the
|
||||||
|
# room by roughly heat x d / 300, where heat is the learned boiler coefficient
|
||||||
|
# (typically 0.05-0.12 °C per 5 minutes with warm radiators). Up to 2 minutes keeps
|
||||||
|
# that below half a typical 0.1 °C sensor step and absorbs timestamp jitter;
|
||||||
|
# matched forecasts make the Forecast Error figures. Up to 5 minutes (one poll) is
|
||||||
|
# about one sensor step: kept apart as approximate. Beyond, the relay did
|
||||||
|
# something else: the forecast is counterfactual and is not scored. Relay time
|
||||||
|
# that was not observed counts as possible deviation, and more than 2 minutes of it
|
||||||
|
# leaves the forecast incomplete rather than approximate.
|
||||||
|
MATCH_SECONDS = 120
|
||||||
|
APPROXIMATE_SECONDS = 300
|
||||||
DUE_TOLERANCE_SECONDS = 150 # half a poll: the poll closest to the forecast's due time
|
DUE_TOLERANCE_SECONDS = 150 # half a poll: the poll closest to the forecast's due time
|
||||||
VERDICTS = ("counterfactual", "unverifiable", "missing")
|
VERDICTS = ("matched", "approximate", "counterfactual", "incomplete")
|
||||||
|
# Why a forecast (or, for a missing reading, a zone's forecast) could not be compared.
|
||||||
|
INCOMPLETE_REASONS = ("relay_history", "uncertain_inputs", "no_poll", "reading")
|
||||||
|
LEDGER_VERSION = 2 # 1 (0.4.1) accepted plans within 20 % per 15-minute block
|
||||||
|
|
||||||
|
|
||||||
class OperationalForecastLedger:
|
class OperationalForecastLedger:
|
||||||
@@ -290,10 +396,10 @@ class OperationalForecastLedger:
|
|||||||
|
|
||||||
Nothing is replaced with hindsight: the start temperatures, trajectory, boiler
|
Nothing is replaced with hindsight: the start temperatures, trajectory, boiler
|
||||||
plan and assumptions are those of the moment the forecast was made. Because the
|
plan and assumptions are those of the moment the forecast was made. Because the
|
||||||
plan is the MPC's proposal while Generic Thermostat controls the relay, a forecast
|
plan is the MPC's proposal while Generic Thermostat controls the relay, a
|
||||||
is only scored when the relay actually followed the plan up to that horizon;
|
forecast is compared only when the relay actually followed that plan (see
|
||||||
otherwise it is counted as counterfactual. Pending forecasts live in memory;
|
MATCH_SECONDS), and not at all when it was made with an input it had to
|
||||||
the statistics are saved.
|
assume. Pending forecasts live in memory; the statistics are saved.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
def __init__(self, saved: Any = None) -> None:
|
def __init__(self, saved: Any = None) -> None:
|
||||||
@@ -301,8 +407,9 @@ class OperationalForecastLedger:
|
|||||||
# (interval end, seconds the relay was on or None if unknown, interval seconds)
|
# (interval end, seconds the relay was on or None if unknown, interval seconds)
|
||||||
self._relay: deque[tuple[float, float | None, float]] = deque(maxlen=48)
|
self._relay: deque[tuple[float, float | None, float]] = deque(maxlen=48)
|
||||||
# UTC day -> horizon -> zone -> [count, sum error, sum |error|, sum error^2, sum |persistence error|]
|
# UTC day -> horizon -> zone -> [count, sum error, sum |error|, sum error^2, sum |persistence error|]
|
||||||
self._days: dict[str, dict[str, dict[str, list[float]]]] = {}
|
self._days: dict[str, dict[str, dict[str, list[float]]]] = {} # matched forecasts
|
||||||
# UTC day -> horizon -> verdict -> count of forecasts not scored, and why
|
self._approximate: dict[str, dict[str, dict[str, list[float]]]] = {}
|
||||||
|
# UTC day -> horizon -> verdict (incomplete ones as "incomplete:<reason>") -> count
|
||||||
self._verdicts: dict[str, dict[str, dict[str, int]]] = {}
|
self._verdicts: dict[str, dict[str, dict[str, int]]] = {}
|
||||||
self.load(saved)
|
self.load(saved)
|
||||||
|
|
||||||
@@ -327,20 +434,18 @@ class OperationalForecastLedger:
|
|||||||
forecast["done"].add(horizon)
|
forecast["done"].add(horizon)
|
||||||
if index >= len(forecast["plan"]):
|
if index >= len(forecast["plan"]):
|
||||||
continue # beyond this forecast's horizon
|
continue # beyond this forecast's horizon
|
||||||
if now > due + DUE_TOLERANCE_SECONDS:
|
verdict = self._classify(forecast, horizon, now)
|
||||||
self._count(day, horizon, "missing") # no poll near the due time
|
self._count(day, horizon, verdict)
|
||||||
continue
|
if verdict not in ("matched", "approximate"):
|
||||||
verdict = self._verify(forecast, horizon)
|
|
||||||
if verdict:
|
|
||||||
self._count(day, horizon, verdict)
|
|
||||||
continue
|
continue
|
||||||
|
store = self._days if verdict == "matched" else self._approximate
|
||||||
for zone, path in forecast["trajectory"].items():
|
for zone, path in forecast["trajectory"].items():
|
||||||
measured, start = actual.get(zone), forecast["start"].get(zone)
|
measured, start = actual.get(zone), forecast["start"].get(zone)
|
||||||
if measured is None or start is None or len(path) <= index:
|
if measured is None or start is None or len(path) <= index:
|
||||||
self._count(day, horizon, "missing")
|
self._count(day, horizon, "incomplete:reading")
|
||||||
continue
|
continue
|
||||||
error = path[index] - measured
|
error = path[index] - measured
|
||||||
bucket = self._days.setdefault(day, {}).setdefault(str(horizon), {}).setdefault(
|
bucket = store.setdefault(day, {}).setdefault(str(horizon), {}).setdefault(
|
||||||
zone, [0, 0.0, 0.0, 0.0, 0.0])
|
zone, [0, 0.0, 0.0, 0.0, 0.0])
|
||||||
bucket[0] += 1
|
bucket[0] += 1
|
||||||
bucket[1] += error
|
bucket[1] += error
|
||||||
@@ -350,38 +455,62 @@ class OperationalForecastLedger:
|
|||||||
if len(forecast["done"]) == len(HORIZONS) or now > forecast["time"] + (HORIZONS[-1] + 5) * 60:
|
if len(forecast["done"]) == len(HORIZONS) or now > forecast["time"] + (HORIZONS[-1] + 5) * 60:
|
||||||
self._pending.remove(forecast)
|
self._pending.remove(forecast)
|
||||||
oldest = (datetime.fromtimestamp(now, timezone.utc).date() - timedelta(days=WINDOW_DAYS - 1)).isoformat()
|
oldest = (datetime.fromtimestamp(now, timezone.utc).date() - timedelta(days=WINDOW_DAYS - 1)).isoformat()
|
||||||
for store in (self._days, self._verdicts):
|
for store in (self._days, self._approximate, self._verdicts):
|
||||||
for old in [d for d in store if d < oldest]:
|
for old in [d for d in store if d < oldest]:
|
||||||
del store[old]
|
del store[old]
|
||||||
|
|
||||||
def _verify(self, forecast: dict[str, Any], horizon: int) -> str | None:
|
def _classify(self, forecast: dict[str, Any], horizon: int, now: float) -> str:
|
||||||
"""None when the relay followed the plan up to `horizon`, else the reason."""
|
if now > forecast["time"] + horizon * 60 + DUE_TOLERANCE_SECONDS:
|
||||||
for block in range(horizon // 15):
|
return "incomplete:no_poll"
|
||||||
start = forecast["time"] + block * 900
|
if forecast["assumptions"].get("uncertain_inputs"):
|
||||||
end = start + 900
|
return "incomplete:uncertain_inputs"
|
||||||
on = covered = 0.0
|
deviation, unobserved = self.deviation(forecast, horizon)
|
||||||
for interval_end, on_seconds, seconds in self._relay:
|
if deviation > APPROXIMATE_SECONDS:
|
||||||
overlap = min(end, interval_end) - max(start, interval_end - seconds)
|
return "counterfactual" # whatever happened in the unobserved part
|
||||||
if overlap <= 0:
|
# Otherwise judged by the worst case: unobserved time may all have deviated.
|
||||||
continue
|
if deviation + unobserved <= MATCH_SECONDS:
|
||||||
if on_seconds is None:
|
return "matched"
|
||||||
return "unverifiable"
|
if deviation + unobserved <= APPROXIMATE_SECONDS and unobserved <= MATCH_SECONDS:
|
||||||
on += on_seconds * overlap / seconds
|
return "approximate"
|
||||||
covered += overlap
|
return "incomplete:relay_history"
|
||||||
if covered < 0.8 * 900:
|
|
||||||
return "unverifiable"
|
def deviation(self, forecast: dict[str, Any], horizon: int) -> tuple[float, float]:
|
||||||
if abs(on / covered - forecast["plan"][block]) > PLAN_TOLERANCE:
|
"""Seconds up to `horizon` the relay was seen in another state than planned,
|
||||||
return "counterfactual"
|
and seconds its state is not known.
|
||||||
return None
|
|
||||||
|
Compared per 15-minute plan block, from the relay's on-time per poll interval,
|
||||||
|
itself integrated at the relay's own switch times. So timing counts: the same
|
||||||
|
total on-time at other times than planned is a deviation, while a switch a few
|
||||||
|
seconds off a block boundary is not.
|
||||||
|
"""
|
||||||
|
start, end = forecast["time"], forecast["time"] + horizon * 60
|
||||||
|
deviation = covered = unobserved = 0.0
|
||||||
|
for interval_end, on_seconds, seconds in self._relay:
|
||||||
|
lo, hi = max(start, interval_end - seconds), min(end, interval_end)
|
||||||
|
if hi <= lo:
|
||||||
|
continue
|
||||||
|
covered += hi - lo
|
||||||
|
if on_seconds is None:
|
||||||
|
unobserved += hi - lo
|
||||||
|
continue
|
||||||
|
share = on_seconds / seconds
|
||||||
|
t = lo
|
||||||
|
while t < hi: # split at plan block boundaries
|
||||||
|
block = min(int((t - start) // 900), len(forecast["plan"]) - 1)
|
||||||
|
part_end = min(hi, start + (block + 1) * 900)
|
||||||
|
deviation += (part_end - t) * (share if not forecast["plan"][block] else 1.0 - share)
|
||||||
|
t = part_end
|
||||||
|
return deviation, unobserved + max(0.0, (end - start) - covered)
|
||||||
|
|
||||||
def _count(self, day: str, horizon: int, verdict: str) -> None:
|
def _count(self, day: str, horizon: int, verdict: str) -> None:
|
||||||
counts = self._verdicts.setdefault(day, {}).setdefault(str(horizon), {})
|
counts = self._verdicts.setdefault(day, {}).setdefault(str(horizon), {})
|
||||||
counts[verdict] = counts.get(verdict, 0) + 1
|
counts[verdict] = counts.get(verdict, 0) + 1
|
||||||
|
|
||||||
def forget(self, zone: str) -> None:
|
def forget(self, zone: str) -> None:
|
||||||
for by_horizon in self._days.values():
|
for store in (self._days, self._approximate):
|
||||||
for by_zone in by_horizon.values():
|
for by_horizon in store.values():
|
||||||
by_zone.pop(zone, None)
|
for by_zone in by_horizon.values():
|
||||||
|
by_zone.pop(zone, None)
|
||||||
for forecast in self._pending:
|
for forecast in self._pending:
|
||||||
forecast["trajectory"].pop(zone, None)
|
forecast["trajectory"].pop(zone, None)
|
||||||
|
|
||||||
@@ -392,58 +521,70 @@ class OperationalForecastLedger:
|
|||||||
def summary(self, zones: list[str]) -> dict[int, dict[str, Any]]:
|
def summary(self, zones: list[str]) -> dict[int, dict[str, Any]]:
|
||||||
result = {}
|
result = {}
|
||||||
for horizon in HORIZONS:
|
for horizon in HORIZONS:
|
||||||
per_zone: dict[str, list[float]] = {}
|
per_zone = _sum_by_zone(self._days, horizon, zones)
|
||||||
verdicts = {verdict: 0 for verdict in VERDICTS}
|
approximate = _sum_by_zone(self._approximate, horizon, zones)
|
||||||
for by_horizon in self._days.values():
|
counts: dict[str, int] = {}
|
||||||
for zone, values in by_horizon.get(str(horizon), {}).items():
|
|
||||||
if zone in zones:
|
|
||||||
totals = per_zone.setdefault(zone, [0, 0.0, 0.0, 0.0, 0.0])
|
|
||||||
for i, value in enumerate(values):
|
|
||||||
totals[i] += value
|
|
||||||
for by_horizon in self._verdicts.values():
|
for by_horizon in self._verdicts.values():
|
||||||
for verdict, count in by_horizon.get(str(horizon), {}).items():
|
for verdict, count in by_horizon.get(str(horizon), {}).items():
|
||||||
verdicts[verdict] = verdicts.get(verdict, 0) + count
|
counts[verdict] = counts.get(verdict, 0) + count
|
||||||
|
reasons = {reason: counts.get(f"incomplete:{reason}", 0) for reason in INCOMPLETE_REASONS}
|
||||||
overall = [sum(values[i] for values in per_zone.values()) for i in range(5)]
|
overall = [sum(values[i] for values in per_zone.values()) for i in range(5)]
|
||||||
result[horizon] = {**_operational_stats(overall, horizon, len(per_zone)), **verdicts,
|
approx_overall = [sum(values[i] for values in approximate.values()) for i in range(5)]
|
||||||
"zones": {zone: _operational_stats(values, horizon)
|
result[horizon] = {
|
||||||
for zone, values in per_zone.items()}}
|
**_operational_stats(overall, horizon, len(per_zone)),
|
||||||
|
# Forecasts by how the relay compared with their plan.
|
||||||
|
"matched": counts.get("matched", 0), "approximate": counts.get("approximate", 0),
|
||||||
|
"counterfactual": counts.get("counterfactual", 0), "incomplete": sum(reasons.values()),
|
||||||
|
"incomplete_reasons": reasons,
|
||||||
|
"approximate_stats": _operational_stats(approx_overall, horizon, len(approximate)),
|
||||||
|
"zones": {zone: _operational_stats(values, horizon) for zone, values in per_zone.items()},
|
||||||
|
}
|
||||||
return result
|
return result
|
||||||
|
|
||||||
def as_dict(self) -> dict[str, Any]:
|
def as_dict(self) -> dict[str, Any]:
|
||||||
return {"days": {day: {h: {zone: [v[0], *(round(x, 6) for x in v[1:])] for zone, v in by_zone.items()}
|
def rounded(store):
|
||||||
for h, by_zone in by_horizon.items()} for day, by_horizon in self._days.items()},
|
return {day: {h: {zone: [v[0], *(round(x, 6) for x in v[1:])] for zone, v in by_zone.items()}
|
||||||
"verdicts": self._verdicts}
|
for h, by_zone in by_horizon.items()} for day, by_horizon in store.items()}
|
||||||
|
return {"version": LEDGER_VERSION, "days": rounded(self._days),
|
||||||
|
"approximate": rounded(self._approximate), "verdicts": self._verdicts}
|
||||||
|
|
||||||
def load(self, data: Any) -> None:
|
def load(self, data: Any) -> None:
|
||||||
self._days, self._verdicts = {}, {}
|
self._days, self._approximate, self._verdicts = {}, {}, {}
|
||||||
if not isinstance(data, dict):
|
# 0.4.1 statistics used a looser plan criterion: they are not comparable.
|
||||||
|
if not isinstance(data, dict) or data.get("version") != LEDGER_VERSION:
|
||||||
return
|
return
|
||||||
horizons = {str(h) for h in HORIZONS}
|
for name, store in (("days", self._days), ("approximate", self._approximate)):
|
||||||
days = data.get("days") if isinstance(data.get("days"), dict) else {}
|
for day, horizon, zone, values in _entries(data.get(name)):
|
||||||
for day, by_horizon in days.items():
|
if (isinstance(values, list) and len(values) == 5 and type(values[0]) is int and values[0] >= 0
|
||||||
if not _valid_day(day) or not isinstance(by_horizon, dict):
|
and all(type(v) in (int, float) and isfinite(v) for v in values[1:])
|
||||||
continue
|
and min(values[2:]) >= 0):
|
||||||
for horizon, by_zone in by_horizon.items():
|
store.setdefault(day, {}).setdefault(horizon, {})[zone] = [
|
||||||
if horizon not in horizons or not isinstance(by_zone, dict):
|
values[0], *(float(v) for v in values[1:])]
|
||||||
continue
|
known = {"matched", "approximate", "counterfactual", *(f"incomplete:{r}" for r in INCOMPLETE_REASONS)}
|
||||||
for zone, values in by_zone.items():
|
|
||||||
if (isinstance(zone, str) and isinstance(values, list) and len(values) == 5
|
|
||||||
and type(values[0]) is int and values[0] >= 0
|
|
||||||
and all(type(v) in (int, float) and isfinite(v) for v in values[1:])
|
|
||||||
and min(values[2:]) >= 0):
|
|
||||||
self._days.setdefault(day, {}).setdefault(horizon, {})[zone] = [
|
|
||||||
values[0], *(float(v) for v in values[1:])]
|
|
||||||
verdicts = data.get("verdicts") if isinstance(data.get("verdicts"), dict) else {}
|
verdicts = data.get("verdicts") if isinstance(data.get("verdicts"), dict) else {}
|
||||||
|
horizons = {str(h) for h in HORIZONS}
|
||||||
for day, by_horizon in verdicts.items():
|
for day, by_horizon in verdicts.items():
|
||||||
if not _valid_day(day) or not isinstance(by_horizon, dict):
|
if not _valid_day(day) or not isinstance(by_horizon, dict):
|
||||||
continue
|
continue
|
||||||
for horizon, counts in by_horizon.items():
|
for horizon, counts in by_horizon.items():
|
||||||
if horizon in horizons and isinstance(counts, dict):
|
if horizon in horizons and isinstance(counts, dict):
|
||||||
clean = {k: v for k, v in counts.items() if k in VERDICTS and type(v) is int and v >= 0}
|
clean = {k: v for k, v in counts.items() if k in known and type(v) is int and v >= 0}
|
||||||
if clean:
|
if clean:
|
||||||
self._verdicts.setdefault(day, {})[horizon] = clean
|
self._verdicts.setdefault(day, {})[horizon] = clean
|
||||||
|
|
||||||
|
|
||||||
|
def _sum_by_zone(store: dict[str, dict[str, dict[str, list[float]]]], horizon: int,
|
||||||
|
zones: list[str]) -> dict[str, list[float]]:
|
||||||
|
totals: dict[str, list[float]] = {}
|
||||||
|
for by_horizon in store.values():
|
||||||
|
for zone, values in by_horizon.get(str(horizon), {}).items():
|
||||||
|
if zone in zones:
|
||||||
|
bucket = totals.setdefault(zone, [0, 0.0, 0.0, 0.0, 0.0])
|
||||||
|
for i, value in enumerate(values):
|
||||||
|
bucket[i] += value
|
||||||
|
return totals
|
||||||
|
|
||||||
|
|
||||||
def _valid_day(day: Any) -> bool:
|
def _valid_day(day: Any) -> bool:
|
||||||
try:
|
try:
|
||||||
datetime.strptime(day, "%Y-%m-%d")
|
datetime.strptime(day, "%Y-%m-%d")
|
||||||
|
|||||||
@@ -9,5 +9,5 @@
|
|||||||
"iot_class": "local_polling",
|
"iot_class": "local_polling",
|
||||||
"requirements": [],
|
"requirements": [],
|
||||||
"single_config_entry": true,
|
"single_config_entry": true,
|
||||||
"version": "0.4.1"
|
"version": "0.4.2"
|
||||||
}
|
}
|
||||||
@@ -103,6 +103,8 @@ class MPCRoomPrediction(CoordinatorEntity, SensorEntity):
|
|||||||
"forecast_15min": trajectory or None,
|
"forecast_15min": trajectory or None,
|
||||||
"plan_15min": data.get("plan_15min"),
|
"plan_15min": data.get("plan_15min"),
|
||||||
"basis": "proposed MPC plan; counterfactual while Generic Thermostat controls the relay",
|
"basis": "proposed MPC plan; counterfactual while Generic Thermostat controls the relay",
|
||||||
|
# Windows with an unknown state: the forecast assumes their state (see Status).
|
||||||
|
"uncertain_windows": details.get("uncertain_windows") or [],
|
||||||
"model_in_use": details.get("model_in_use"),
|
"model_in_use": details.get("model_in_use"),
|
||||||
"contributions_c_per_h": details.get("contributions_c_per_h"),
|
"contributions_c_per_h": details.get("contributions_c_per_h"),
|
||||||
"coefficients": details.get("coefficients"),
|
"coefficients": details.get("coefficients"),
|
||||||
@@ -178,6 +180,11 @@ class MPCForecastError(CoordinatorEntity, SensorEntity):
|
|||||||
"skill": accuracy.get("skill"),
|
"skill": accuracy.get("skill"),
|
||||||
"base_model_mae": accuracy.get("base_model_mae"),
|
"base_model_mae": accuracy.get("base_model_mae"),
|
||||||
"improvement_vs_base": accuracy.get("improvement_vs_base"),
|
"improvement_vs_base": accuracy.get("improvement_vs_base"),
|
||||||
|
# Spans left out of the figures above: an input the model needed was unknown
|
||||||
|
# (the base model's error on them is still measured), or no replay was possible.
|
||||||
|
"incomplete": accuracy.get("incomplete", 0),
|
||||||
|
"incomplete_base_model_mae": accuracy.get("incomplete_base_model_mae"),
|
||||||
|
"skipped": accuracy.get("skipped", 0),
|
||||||
# Per zone, by name.
|
# Per zone, by name.
|
||||||
"zones": {self.coordinator.zones[key].name if key in self.coordinator.zones else key: stats
|
"zones": {self.coordinator.zones[key].name if key in self.coordinator.zones else key: stats
|
||||||
for key, stats in accuracy.get("rooms", {}).items()},
|
for key, stats in accuracy.get("rooms", {}).items()},
|
||||||
@@ -222,7 +229,8 @@ class MPCOperationalForecastError(CoordinatorEntity, SensorEntity):
|
|||||||
horizons[f"{horizon}_min"] = {**{k: v for k, v in stats.items() if k != "zones"},
|
horizons[f"{horizon}_min"] = {**{k: v for k, v in stats.items() if k != "zones"},
|
||||||
"zones": {names.get(z, z): v for z, v in stats.get("zones", {}).items()}}
|
"zones": {names.get(z, z): v for z, v in stats.get("zones", {}).items()}}
|
||||||
return {
|
return {
|
||||||
"kind": "forecast accuracy: published forecasts, scored only where the relay followed the plan",
|
"kind": ("forecast accuracy: published forecasts, scored only where the relay matched the plan "
|
||||||
|
"(at most 2 minutes in another state); approximate matches (up to 5 minutes) apart"),
|
||||||
"horizons": horizons,
|
"horizons": horizons,
|
||||||
"window_days": WINDOW_DAYS,
|
"window_days": WINDOW_DAYS,
|
||||||
"unit_of_attributes": "°C",
|
"unit_of_attributes": "°C",
|
||||||
|
|||||||
@@ -56,6 +56,13 @@
|
|||||||
"options": {
|
"options": {
|
||||||
"step": {
|
"step": {
|
||||||
"init": {
|
"init": {
|
||||||
|
"title": "Heating MPC",
|
||||||
|
"menu_options": {
|
||||||
|
"settings": "Settings",
|
||||||
|
"reset_model": "Reset learned model"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"settings": {
|
||||||
"title": "Heating MPC options",
|
"title": "Heating MPC options",
|
||||||
"description": "Change room sensors and controller settings. This integration remains shadow-only.",
|
"description": "Change room sensors and controller settings. This integration remains shadow-only.",
|
||||||
"data": {
|
"data": {
|
||||||
@@ -83,10 +90,17 @@
|
|||||||
"window_open_minutes": "Forecasts assume an open window closes after this many more minutes, then the room recovers.",
|
"window_open_minutes": "Forecasts assume an open window closes after this many more minutes, then the room recovers.",
|
||||||
"recovery_minutes": "Time constant of the air warming back towards the walls' temperature.",
|
"recovery_minutes": "Time constant of the air warming back towards the walls' temperature.",
|
||||||
"ventilation_wind": "Needs wind sources; direction also needs wind direction sources.",
|
"ventilation_wind": "Needs wind sources; direction also needs wind direction sources.",
|
||||||
"ventilation_gust": "Adds a term for gusts above the mean wind. Keep it only if Forecast Error improves.",
|
"ventilation_gust": "Adds a term for gusts above the mean wind. Keep it only if Model Error improves.",
|
||||||
"ventilation_cross": "Adds a term for windows open in several zones at once. Needs windows in two or more zones."
|
"ventilation_cross": "Adds a term for windows open in several zones at once. Needs windows in two or more zones."
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
"reset_model": {
|
||||||
|
"title": "Reset learned model",
|
||||||
|
"description": "Forgets everything the model has learned: every zone's thermal coefficients (also those kept for removed zones), their confidence and sample counts, and the Model Error and Forecast Error statistics.\n\nKept: zones, sensors and weights, heat sources, windows, weather sources and all settings, plus the heat sources' baselines and the window event log.\n\nThe integration restarts and learns from the beginning: the first sample after about 30 minutes, forecasts after a day or more. This cannot be undone. The boiler and the thermostat are not affected.",
|
||||||
|
"data": {
|
||||||
|
"confirm_reset": "Yes, forget the learned model"
|
||||||
|
}
|
||||||
|
},
|
||||||
"weather": {
|
"weather": {
|
||||||
"title": "Weather source priority",
|
"title": "Weather source priority",
|
||||||
"description": "Choose as many sensors per metric as needed and arrange them in priority order. If the preferred source is unavailable, the next one is used automatically. Outdoor temperature is mandatory; wind and cloud coverage are diagnostic-only in this version.",
|
"description": "Choose as many sensors per metric as needed and arrange them in priority order. If the preferred source is unavailable, the next one is used automatically. Outdoor temperature is mandatory; wind and cloud coverage are diagnostic-only in this version.",
|
||||||
@@ -113,6 +127,10 @@
|
|||||||
},
|
},
|
||||||
"error": {
|
"error": {
|
||||||
"invalid_weather": "Select at least one outdoor temperature sensor, distinct from room sensors."
|
"invalid_weather": "Select at least one outdoor temperature sensor, distinct from room sensors."
|
||||||
|
},
|
||||||
|
"abort": {
|
||||||
|
"model_reset": "The learned model was reset. Learning starts again from the beginning.",
|
||||||
|
"model_not_reset": "Nothing was changed: the reset was not confirmed."
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
"entity": {
|
"entity": {
|
||||||
|
|||||||
@@ -56,6 +56,13 @@
|
|||||||
"options": {
|
"options": {
|
||||||
"step": {
|
"step": {
|
||||||
"init": {
|
"init": {
|
||||||
|
"title": "Heating MPC",
|
||||||
|
"menu_options": {
|
||||||
|
"settings": "Settings",
|
||||||
|
"reset_model": "Reset learned model"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"settings": {
|
||||||
"title": "Heating MPC options",
|
"title": "Heating MPC options",
|
||||||
"description": "Change room sensors and controller settings. This integration remains shadow-only.",
|
"description": "Change room sensors and controller settings. This integration remains shadow-only.",
|
||||||
"data": {
|
"data": {
|
||||||
@@ -83,10 +90,17 @@
|
|||||||
"window_open_minutes": "Forecasts assume an open window closes after this many more minutes, then the room recovers.",
|
"window_open_minutes": "Forecasts assume an open window closes after this many more minutes, then the room recovers.",
|
||||||
"recovery_minutes": "Time constant of the air warming back towards the walls' temperature.",
|
"recovery_minutes": "Time constant of the air warming back towards the walls' temperature.",
|
||||||
"ventilation_wind": "Needs wind sources; direction also needs wind direction sources.",
|
"ventilation_wind": "Needs wind sources; direction also needs wind direction sources.",
|
||||||
"ventilation_gust": "Adds a term for gusts above the mean wind. Keep it only if Forecast Error improves.",
|
"ventilation_gust": "Adds a term for gusts above the mean wind. Keep it only if Model Error improves.",
|
||||||
"ventilation_cross": "Adds a term for windows open in several zones at once. Needs windows in two or more zones."
|
"ventilation_cross": "Adds a term for windows open in several zones at once. Needs windows in two or more zones."
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
"reset_model": {
|
||||||
|
"title": "Reset learned model",
|
||||||
|
"description": "Forgets everything the model has learned: every zone's thermal coefficients (also those kept for removed zones), their confidence and sample counts, and the Model Error and Forecast Error statistics.\n\nKept: zones, sensors and weights, heat sources, windows, weather sources and all settings, plus the heat sources' baselines and the window event log.\n\nThe integration restarts and learns from the beginning: the first sample after about 30 minutes, forecasts after a day or more. This cannot be undone. The boiler and the thermostat are not affected.",
|
||||||
|
"data": {
|
||||||
|
"confirm_reset": "Yes, forget the learned model"
|
||||||
|
}
|
||||||
|
},
|
||||||
"weather": {
|
"weather": {
|
||||||
"title": "Weather source priority",
|
"title": "Weather source priority",
|
||||||
"description": "Choose as many sensors per metric as needed and arrange them in priority order. If the preferred source is unavailable, the next one is used automatically. Outdoor temperature is mandatory; wind and cloud coverage are diagnostic-only in this version.",
|
"description": "Choose as many sensors per metric as needed and arrange them in priority order. If the preferred source is unavailable, the next one is used automatically. Outdoor temperature is mandatory; wind and cloud coverage are diagnostic-only in this version.",
|
||||||
@@ -113,6 +127,10 @@
|
|||||||
},
|
},
|
||||||
"error": {
|
"error": {
|
||||||
"invalid_weather": "Select at least one outdoor temperature sensor, distinct from room sensors."
|
"invalid_weather": "Select at least one outdoor temperature sensor, distinct from room sensors."
|
||||||
|
},
|
||||||
|
"abort": {
|
||||||
|
"model_reset": "The learned model was reset. Learning starts again from the beginning.",
|
||||||
|
"model_not_reset": "Nothing was changed: the reset was not confirmed."
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
"entity": {
|
"entity": {
|
||||||
|
|||||||
@@ -56,6 +56,13 @@
|
|||||||
"options": {
|
"options": {
|
||||||
"step": {
|
"step": {
|
||||||
"init": {
|
"init": {
|
||||||
|
"title": "Heating MPC",
|
||||||
|
"menu_options": {
|
||||||
|
"settings": "Setări",
|
||||||
|
"reset_model": "Resetează modelul învățat"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"settings": {
|
||||||
"title": "Configurare Heating MPC",
|
"title": "Configurare Heating MPC",
|
||||||
"description": "Modifică senzorii, ponderile și optimizatorul. Rămâne activ doar modul de observație.",
|
"description": "Modifică senzorii, ponderile și optimizatorul. Rămâne activ doar modul de observație.",
|
||||||
"data": {
|
"data": {
|
||||||
@@ -83,10 +90,17 @@
|
|||||||
"window_open_minutes": "Prognozele presupun că o fereastră deschisă se închide după atâtea minute, apoi camera revine.",
|
"window_open_minutes": "Prognozele presupun că o fereastră deschisă se închide după atâtea minute, apoi camera revine.",
|
||||||
"recovery_minutes": "Constanta de timp a aerului care se încălzește din nou spre temperatura pereților.",
|
"recovery_minutes": "Constanta de timp a aerului care se încălzește din nou spre temperatura pereților.",
|
||||||
"ventilation_wind": "Necesită surse de vânt; direcția necesită și surse pentru direcția vântului.",
|
"ventilation_wind": "Necesită surse de vânt; direcția necesită și surse pentru direcția vântului.",
|
||||||
"ventilation_gust": "Adaugă un termen pentru rafalele peste vântul mediu. Păstrează-l doar dacă eroarea de prognoză scade.",
|
"ventilation_gust": "Adaugă un termen pentru rafalele peste vântul mediu. Păstrează-l doar dacă Model Error scade.",
|
||||||
"ventilation_cross": "Adaugă un termen pentru ferestre deschise în mai multe zone deodată. Necesită ferestre în cel puțin două zone."
|
"ventilation_cross": "Adaugă un termen pentru ferestre deschise în mai multe zone deodată. Necesită ferestre în cel puțin două zone."
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
"reset_model": {
|
||||||
|
"title": "Resetează modelul învățat",
|
||||||
|
"description": "Șterge tot ce a învățat modelul: coeficienții termici ai fiecărei zone (inclusiv cei păstrați pentru zonele eliminate), încrederea și numărul de eșantioane, precum și statisticile Model Error și Forecast Error.\n\nSe păstrează: zonele, senzorii și ponderile, sursele de căldură, ferestrele, sursele meteo și toate setările, plus nivelurile de bază ale surselor de căldură și jurnalul ferestrelor.\n\nIntegrarea repornește și învață de la început: primul eșantion după aproximativ 30 de minute, prognozele după o zi sau mai mult. Operația nu poate fi anulată. Centrala și termostatul nu sunt afectate.",
|
||||||
|
"data": {
|
||||||
|
"confirm_reset": "Da, șterge modelul învățat"
|
||||||
|
}
|
||||||
|
},
|
||||||
"weather": {
|
"weather": {
|
||||||
"title": "Prioritatea surselor meteo",
|
"title": "Prioritatea surselor meteo",
|
||||||
"description": "Alege câți senzori dorești pentru fiecare variabilă și ordonează-i după prioritate. Când prima sursă este indisponibilă, integrarea încearcă automat următoarea. Temperatura exterioară este obligatorie; vântul și nebulozitatea sunt acum doar diagnostice.",
|
"description": "Alege câți senzori dorești pentru fiecare variabilă și ordonează-i după prioritate. Când prima sursă este indisponibilă, integrarea încearcă automat următoarea. Temperatura exterioară este obligatorie; vântul și nebulozitatea sunt acum doar diagnostice.",
|
||||||
@@ -113,6 +127,10 @@
|
|||||||
},
|
},
|
||||||
"error": {
|
"error": {
|
||||||
"invalid_weather": "Alege cel puțin un senzor de temperatură exterioară, diferit de senzorii camerelor."
|
"invalid_weather": "Alege cel puțin un senzor de temperatură exterioară, diferit de senzorii camerelor."
|
||||||
|
},
|
||||||
|
"abort": {
|
||||||
|
"model_reset": "Modelul învățat a fost resetat. Învățarea reîncepe de la zero.",
|
||||||
|
"model_not_reset": "Nu s-a modificat nimic: resetarea nu a fost confirmată."
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
"entity": {
|
"entity": {
|
||||||
|
|||||||
@@ -150,17 +150,22 @@ def ventilation_vector(features: Iterable[str], temp: float, outside: float,
|
|||||||
inputs: Mapping[str, Any]) -> list[float] | None:
|
inputs: Mapping[str, Any]) -> list[float] | None:
|
||||||
"""Ventilation feature values for one zone and step; None if an input is unknown.
|
"""Ventilation feature values for one zone and step; None if an input is unknown.
|
||||||
|
|
||||||
`inputs`: opening, facing (sum of opening x cos), other_opening, recovery,
|
`inputs`: opening, facing (sum of opening x cos; None when the wind direction is
|
||||||
t_ref (pre-ventilation temperature), wind, gust.
|
unknown), other_opening (None when another zone's windows are unknown),
|
||||||
|
recovery, t_ref (pre-ventilation temperature), wind, gust. With every window
|
||||||
|
closed the opening terms are 0 whatever the weather, so only the opening and,
|
||||||
|
while recovering, the reference temperature must be known.
|
||||||
"""
|
"""
|
||||||
difference = outside - temp
|
difference = outside - temp
|
||||||
|
opening = inputs.get("opening")
|
||||||
|
if opening is None:
|
||||||
|
return None
|
||||||
values = []
|
values = []
|
||||||
for name in features:
|
for name in features:
|
||||||
opening = inputs.get("opening")
|
|
||||||
if opening is None:
|
|
||||||
return None
|
|
||||||
if name == "vent_natural":
|
if name == "vent_natural":
|
||||||
values.append(opening * difference)
|
values.append(opening * difference)
|
||||||
|
elif name in ("vent_wind", "vent_wind_dir", "vent_gust", "vent_cross") and opening == 0:
|
||||||
|
values.append(0.0)
|
||||||
elif name in ("vent_wind", "vent_wind_dir", "vent_gust"):
|
elif name in ("vent_wind", "vent_wind_dir", "vent_gust"):
|
||||||
wind = inputs.get("wind")
|
wind = inputs.get("wind")
|
||||||
if wind is None:
|
if wind is None:
|
||||||
@@ -178,7 +183,10 @@ def ventilation_vector(features: Iterable[str], temp: float, outside: float,
|
|||||||
return None
|
return None
|
||||||
values.append(opening * max(0.0, gust - wind) * difference)
|
values.append(opening * max(0.0, gust - wind) * difference)
|
||||||
elif name == "vent_cross":
|
elif name == "vent_cross":
|
||||||
values.append(opening * (inputs.get("other_opening") or 0.0) * difference)
|
other = inputs.get("other_opening")
|
||||||
|
if other is None:
|
||||||
|
return None
|
||||||
|
values.append(opening * other * difference)
|
||||||
elif name == "recovery":
|
elif name == "recovery":
|
||||||
recovery = inputs.get("recovery") or 0.0
|
recovery = inputs.get("recovery") or 0.0
|
||||||
t_ref = inputs.get("t_ref")
|
t_ref = inputs.get("t_ref")
|
||||||
@@ -203,16 +211,36 @@ def ventilation_vector_for_prediction(features: Iterable[str], temp: float, outs
|
|||||||
if x is None and inputs.get("opening") is not None:
|
if x is None and inputs.get("opening") is not None:
|
||||||
x = ventilation_vector(features, temp, outside, {
|
x = ventilation_vector(features, temp, outside, {
|
||||||
**inputs, "wind": inputs.get("wind") or 0.0, "gust": inputs.get("gust") or inputs.get("wind") or 0.0,
|
**inputs, "wind": inputs.get("wind") or 0.0, "gust": inputs.get("gust") or inputs.get("wind") or 0.0,
|
||||||
"facing": inputs.get("facing") or 0.0})
|
"facing": inputs.get("facing") or 0.0, "other_opening": inputs.get("other_opening") or 0.0})
|
||||||
return x
|
return x
|
||||||
|
|
||||||
|
|
||||||
|
def ventilation_known(features: Iterable[str], inputs: Mapping[str, Any] | None) -> bool:
|
||||||
|
"""True when a zone's ventilation term follows from observed inputs alone.
|
||||||
|
|
||||||
|
False while a window state is unknown or a recovery's start is unconfirmed,
|
||||||
|
or while a window is open and a term it activates lacks its weather input.
|
||||||
|
Zones without windows (`inputs` None or empty) need nothing.
|
||||||
|
"""
|
||||||
|
if not inputs:
|
||||||
|
return True
|
||||||
|
return ventilation_vector(list(features), 0.0, 0.0, inputs) is not None
|
||||||
|
|
||||||
|
|
||||||
def _iso(timestamp: float) -> str:
|
def _iso(timestamp: float) -> str:
|
||||||
return datetime.fromtimestamp(timestamp, timezone.utc).isoformat(timespec="seconds")
|
return datetime.fromtimestamp(timestamp, timezone.utc).isoformat(timespec="seconds")
|
||||||
|
|
||||||
|
|
||||||
class VentilationTracker:
|
class VentilationTracker:
|
||||||
"""Window states, ventilation inputs per zone, recovery after closing, event log."""
|
"""Window states, ventilation inputs per zone, recovery after closing, event log.
|
||||||
|
|
||||||
|
Each window is OPEN, CLOSED or UNKNOWN (unknown, unavailable or missing).
|
||||||
|
Unknown never means closed. The last known state is kept apart from the
|
||||||
|
current one, and an opening or closing counts as confirmed only when it was
|
||||||
|
seen: the previous reading was the other state, with no unknown period in
|
||||||
|
between. Last known states, open events and recovery are saved so a restart
|
||||||
|
or reload continues them; they come back as unknown until the sensor reports.
|
||||||
|
"""
|
||||||
|
|
||||||
def __init__(self, windows: list[Window], recovery_minutes: float = DEFAULT_RECOVERY_MINUTES,
|
def __init__(self, windows: list[Window], recovery_minutes: float = DEFAULT_RECOVERY_MINUTES,
|
||||||
saved: Any = None) -> None:
|
saved: Any = None) -> None:
|
||||||
@@ -223,25 +251,24 @@ class VentilationTracker:
|
|||||||
self._t_ref: dict[str, float | None] = {} # zone -> temperature before it opened
|
self._t_ref: dict[str, float | None] = {} # zone -> temperature before it opened
|
||||||
self._last_temps: dict[str, float | None] = {}
|
self._last_temps: dict[str, float | None] = {}
|
||||||
self._open_event: dict[str, dict[str, Any]] = {}
|
self._open_event: dict[str, dict[str, Any]] = {}
|
||||||
# window -> last known open/closed state, since when, and since when it is unknown.
|
# window -> last known open/closed state, since when, when it was last seen
|
||||||
|
# and since when it is unknown (None while known).
|
||||||
self._memory: dict[str, dict[str, Any]] = {}
|
self._memory: dict[str, dict[str, Any]] = {}
|
||||||
self._recovery_confirmed: dict[str, bool] = {}
|
self._recovery_confirmed: dict[str, bool] = {}
|
||||||
|
self._last_update: float | None = None
|
||||||
|
self._restored_at: float | None = None
|
||||||
|
self.changed = False # something worth saving happened since the last save
|
||||||
saved = saved if isinstance(saved, Mapping) else {}
|
saved = saved if isinstance(saved, Mapping) else {}
|
||||||
events = saved.get("events") if isinstance(saved.get("events"), list) else []
|
events = saved.get("events") if isinstance(saved.get("events"), list) else []
|
||||||
self.events: deque[dict[str, Any]] = deque(
|
self.events: deque[dict[str, Any]] = deque(
|
||||||
(e for e in events if isinstance(e, dict)), maxlen=MAX_EVENTS)
|
(e for e in events if isinstance(e, dict)), maxlen=MAX_EVENTS)
|
||||||
self.window_state: dict[str, dict[str, Any]] = {}
|
self.window_state: dict[str, dict[str, Any]] = {}
|
||||||
self.zone_state: dict[str, dict[str, Any]] = {}
|
self.zone_state: dict[str, dict[str, Any]] = {}
|
||||||
|
self._restore(saved.get("state"))
|
||||||
|
|
||||||
def update(self, now: float, states: Mapping[str, tuple[str | None, float | None]],
|
def update(self, now: float, states: Mapping[str, tuple[str | None, float | None]],
|
||||||
temps: Mapping[str, float | None], weather: Mapping[str, float | None]) -> None:
|
temps: Mapping[str, float | None], weather: Mapping[str, float | None]) -> None:
|
||||||
"""One poll. `states`: window key -> (binary sensor state, last_changed timestamp).
|
"""One poll. `states`: window key -> (binary sensor state, last_changed timestamp)."""
|
||||||
|
|
||||||
Each window is OPEN, CLOSED or UNKNOWN (unknown/unavailable/missing). Unknown
|
|
||||||
never means closed: the last known state is kept, an open window's event
|
|
||||||
stays open, and a close that happened while the sensor was unknown is
|
|
||||||
recorded as unconfirmed.
|
|
||||||
"""
|
|
||||||
self.window_state = {}
|
self.window_state = {}
|
||||||
zones: dict[str, dict[str, Any]] = {}
|
zones: dict[str, dict[str, Any]] = {}
|
||||||
for window in self.windows:
|
for window in self.windows:
|
||||||
@@ -254,11 +281,12 @@ class VentilationTracker:
|
|||||||
if previous and changed is not None and 0 < now - previous[0] <= 900:
|
if previous and changed is not None and 0 < now - previous[0] <= 900:
|
||||||
fraction = open_fraction(previous[1], is_open, changed, previous[0], now)
|
fraction = open_fraction(previous[1], is_open, changed, previous[0], now)
|
||||||
self._previous[window.key] = (now, is_open)
|
self._previous[window.key] = (now, is_open)
|
||||||
memory = self._memory.setdefault(window.key, {"last_known": None, "since": None, "unknown_since": None})
|
memory = self._memory.setdefault(window.key, {"last_known": None, "since": None, "seen": None,
|
||||||
|
"unknown_since": None})
|
||||||
zone = zones.setdefault(window.zone, {
|
zone = zones.setdefault(window.zone, {
|
||||||
"opening": 0.0, "facing_unit": 0.0, "known": True, "open_now": False, "possibly_open": False,
|
"opening": 0.0, "facing_unit": 0.0, "known": True, "open_now": False, "possibly_open": False,
|
||||||
"open_windows": [], "last_close": None, "close_confirmed": True, "first_open": None,
|
"open_windows": [], "last_close": None, "close_confirmed": True, "first_open": None,
|
||||||
"open_confirmed": True, "unknown_windows": []})
|
"open_confirmed": True, "unknown_windows": [], "facing_known": True})
|
||||||
if fraction is None:
|
if fraction is None:
|
||||||
zone["known"] = False
|
zone["known"] = False
|
||||||
else:
|
else:
|
||||||
@@ -266,21 +294,26 @@ class VentilationTracker:
|
|||||||
zone["facing_unit"] += window.opening * fraction * (
|
zone["facing_unit"] += window.opening * fraction * (
|
||||||
facing_cos(weather["wind_direction"], window.azimuth)
|
facing_cos(weather["wind_direction"], window.azimuth)
|
||||||
if weather.get("wind_direction") is not None else 0.0)
|
if weather.get("wind_direction") is not None else 0.0)
|
||||||
|
if fraction > 0 and weather.get("wind_direction") is None:
|
||||||
|
zone["facing_known"] = False
|
||||||
if is_open is None:
|
if is_open is None:
|
||||||
if memory["unknown_since"] is None:
|
if memory["unknown_since"] is None:
|
||||||
memory["unknown_since"] = previous[0] if previous and previous[1] is not None else now
|
memory["unknown_since"] = memory["seen"] if memory["seen"] is not None else now
|
||||||
|
self.changed = True
|
||||||
zone["unknown_windows"].append(window.name)
|
zone["unknown_windows"].append(window.name)
|
||||||
if memory["last_known"]:
|
if memory["last_known"]:
|
||||||
zone["possibly_open"] = True # last seen open: still treated as open
|
zone["possibly_open"] = True # last seen open: still treated as open
|
||||||
else:
|
else:
|
||||||
after_outage = memory["unknown_since"] is not None
|
if memory["last_known"] is not is_open: # a change, or the first reading
|
||||||
if memory["last_known"] is not is_open:
|
|
||||||
memory["since"] = changed
|
memory["since"] = changed
|
||||||
if is_open and after_outage:
|
# Seen only if the previous reading was the other state, with no gap.
|
||||||
zone["open_confirmed"] = False # opened at some point during the outage
|
witnessed = memory["last_known"] is not None and memory["unknown_since"] is None
|
||||||
if is_open is False and memory["last_known"] and after_outage:
|
if not witnessed:
|
||||||
zone["close_confirmed"] = False # closed at some point during the outage
|
zone["open_confirmed" if is_open else "close_confirmed"] = False
|
||||||
memory["last_known"], memory["unknown_since"] = is_open, None
|
self.changed = True
|
||||||
|
elif memory["unknown_since"] is not None:
|
||||||
|
self.changed = True # the same state again after a gap
|
||||||
|
memory.update(last_known=is_open, seen=now, unknown_since=None)
|
||||||
if is_open:
|
if is_open:
|
||||||
zone["open_now"] = True
|
zone["open_now"] = True
|
||||||
zone["open_windows"].append(window.name)
|
zone["open_windows"].append(window.name)
|
||||||
@@ -290,13 +323,21 @@ class VentilationTracker:
|
|||||||
zone["last_close"] = max(zone["last_close"] or 0.0, changed)
|
zone["last_close"] = max(zone["last_close"] or 0.0, changed)
|
||||||
self.window_state[window.key] = {
|
self.window_state[window.key] = {
|
||||||
"open": is_open, "since": changed, "last_known_open": memory["last_known"],
|
"open": is_open, "since": changed, "last_known_open": memory["last_known"],
|
||||||
"last_known_since": memory["since"], "unknown_since": memory["unknown_since"],
|
"last_known_since": memory["since"], "seen": memory["seen"],
|
||||||
|
"unknown_since": memory["unknown_since"],
|
||||||
"relative_wind_direction": (round(angle_difference(weather["wind_direction"], window.azimuth), 1)
|
"relative_wind_direction": (round(angle_difference(weather["wind_direction"], window.azimuth), 1)
|
||||||
if weather.get("wind_direction") is not None else None)}
|
if weather.get("wind_direction") is not None else None)}
|
||||||
|
# Time the integration was not running counts as unknown for events it interrupted.
|
||||||
|
gap = now - self._restored_at if self._restored_at is not None else 0.0
|
||||||
|
self._restored_at = None
|
||||||
for key, zone in zones.items():
|
for key, zone in zones.items():
|
||||||
|
if gap > 330 and key in self._open_event:
|
||||||
|
event = self._open_event[key]
|
||||||
|
event["sensor_unknown_minutes"] = round(event.get("sensor_unknown_minutes", 0.0) + gap / 60, 1)
|
||||||
self._update_zone(now, key, zone, temps.get(key), weather, zones)
|
self._update_zone(now, key, zone, temps.get(key), weather, zones)
|
||||||
self.zone_state = zones
|
self.zone_state = zones
|
||||||
self._last_temps = dict(temps)
|
self._last_temps = dict(temps)
|
||||||
|
self._last_update = now
|
||||||
|
|
||||||
def _update_zone(self, now: float, key: str, zone: dict[str, Any], temp: float | None,
|
def _update_zone(self, now: float, key: str, zone: dict[str, Any], temp: float | None,
|
||||||
weather: Mapping[str, float | None], zones: Mapping[str, dict[str, Any]]) -> None:
|
weather: Mapping[str, float | None], zones: Mapping[str, dict[str, Any]]) -> None:
|
||||||
@@ -314,32 +355,41 @@ class VentilationTracker:
|
|||||||
"wind_direction": weather.get("wind_direction"), "wind_gust": weather.get("wind_gust"),
|
"wind_direction": weather.get("wind_direction"), "wind_gust": weather.get("wind_gust"),
|
||||||
"windows": list(zone["open_windows"]),
|
"windows": list(zone["open_windows"]),
|
||||||
"other_open_windows": [n for k, z in zones.items() if k != key for n in z["open_windows"]],
|
"other_open_windows": [n for k, z in zones.items() if k != key for n in z["open_windows"]],
|
||||||
"lowest_temperature": temp, "sensor_unknown_minutes": 0.0,
|
"lowest_temperature": temp, "sensor_unknown_minutes": 0.0, "last_seen_open": _iso(now),
|
||||||
}
|
}
|
||||||
if not zone["open_confirmed"]:
|
if not zone["open_confirmed"]:
|
||||||
self._t_ref[key] = None # the room may already have cooled before we saw it
|
self._t_ref[key] = None # the room may already have cooled before we saw it
|
||||||
|
self.changed = True
|
||||||
elif ventilating and was_open:
|
elif ventilating and was_open:
|
||||||
event = self._open_event[key]
|
event = self._open_event[key]
|
||||||
if temp is not None:
|
if temp is not None:
|
||||||
event["lowest_temperature"] = min(t for t in (event["lowest_temperature"], temp) if t is not None)
|
event["lowest_temperature"] = min(t for t in (event["lowest_temperature"], temp) if t is not None)
|
||||||
if zone["unknown_windows"]:
|
if zone["unknown_windows"]:
|
||||||
event["sensor_unknown_minutes"] = round(event["sensor_unknown_minutes"] + 5.0, 1)
|
event["sensor_unknown_minutes"] = round(event["sensor_unknown_minutes"] + 5.0, 1)
|
||||||
|
if zone["open_now"]:
|
||||||
|
event["last_seen_open"] = _iso(now)
|
||||||
elif was_open and not ventilating:
|
elif was_open and not ventilating:
|
||||||
event = self._open_event.pop(key)
|
event = self._open_event.pop(key)
|
||||||
if zone["close_confirmed"]:
|
opened = _parse(event["opened"]) # the confirmed opening, or when it was first seen open
|
||||||
closed = zone["last_close"] or now
|
seen_open = _parse(event.get("last_seen_open", event["opened"]))
|
||||||
event.update(closed=_iso(closed), closed_confirmed=True,
|
closed = zone["last_close"] or now if zone["close_confirmed"] else now
|
||||||
duration_minutes=round((closed - _parse(event["opened"])) / 60, 1))
|
confirmed = zone["close_confirmed"] and event.get("opened_confirmed", True)
|
||||||
else:
|
# Uncertain periods give a range instead of a duration.
|
||||||
# Closed while the sensor was unknown: no confirmed time, only a range.
|
event.update(
|
||||||
closed = now
|
closed=_iso(closed) if zone["close_confirmed"] else None,
|
||||||
event.update(closed=None, closed_confirmed=False, closed_reported=_iso(now),
|
closed_confirmed=zone["close_confirmed"],
|
||||||
duration_minutes=None,
|
duration_minutes=round((closed - opened) / 60, 1) if confirmed else None,
|
||||||
duration_max_minutes=round((now - _parse(event["opened"])) / 60, 1))
|
duration_min_minutes=round(((closed if zone["close_confirmed"] else seen_open) - opened) / 60, 1),
|
||||||
|
duration_max_minutes=(round((now - opened) / 60, 1)
|
||||||
|
if not zone["close_confirmed"] and event.get("opened_confirmed", True)
|
||||||
|
else None))
|
||||||
|
if not zone["close_confirmed"]:
|
||||||
|
event["closed_reported"] = _iso(now) # closed at some point before this
|
||||||
self._closed_at[key] = closed
|
self._closed_at[key] = closed
|
||||||
self._recovery_confirmed[key] = zone["close_confirmed"] and event.get("opened_confirmed", True)
|
self._recovery_confirmed[key] = confirmed
|
||||||
event["temperature_at_close"] = temp
|
event["temperature_at_close"] = temp
|
||||||
self.events.append(event)
|
self.events.append(event)
|
||||||
|
self.changed = True
|
||||||
elif not ventilating and key in self._closed_at and temp is not None:
|
elif not ventilating and key in self._closed_at and temp is not None:
|
||||||
# Fill in the temperature half an hour after closing, for recovery review.
|
# Fill in the temperature half an hour after closing, for recovery review.
|
||||||
for event in reversed(self.events):
|
for event in reversed(self.events):
|
||||||
@@ -354,7 +404,8 @@ class VentilationTracker:
|
|||||||
recovery = 0.0
|
recovery = 0.0
|
||||||
zone.update(recovery=recovery, t_ref=self._t_ref.get(key),
|
zone.update(recovery=recovery, t_ref=self._t_ref.get(key),
|
||||||
recovery_confirmed=self._recovery_confirmed.get(key, True),
|
recovery_confirmed=self._recovery_confirmed.get(key, True),
|
||||||
other_opening=sum(z["opening"] for k, z in zones.items() if k != key and z["known"]))
|
other_opening=sum(z["opening"] for k, z in zones.items() if k != key and z["known"]),
|
||||||
|
others_known=all(z["known"] for k, z in zones.items() if k != key))
|
||||||
# Learning of the closed-window model is unsafe while anything ventilation-related goes on.
|
# Learning of the closed-window model is unsafe while anything ventilation-related goes on.
|
||||||
zone["affected"] = (not zone["known"]) or zone["opening"] > 0 or ventilating or recovery > 0
|
zone["affected"] = (not zone["known"]) or zone["opening"] > 0 or ventilating or recovery > 0
|
||||||
|
|
||||||
@@ -368,49 +419,139 @@ class VentilationTracker:
|
|||||||
if zone is None:
|
if zone is None:
|
||||||
return None
|
return None
|
||||||
known = zone["known"] and (zone["recovery"] == 0 or zone["recovery_confirmed"])
|
known = zone["known"] and (zone["recovery"] == 0 or zone["recovery_confirmed"])
|
||||||
return {"opening": zone["opening"] if known else None, "facing": zone["facing_unit"],
|
return {"opening": zone["opening"] if known else None,
|
||||||
"other_opening": zone["other_opening"], "recovery": zone["recovery"],
|
"facing": zone["facing_unit"] if zone["facing_known"] else None,
|
||||||
|
"other_opening": zone["other_opening"] if zone["others_known"] else None,
|
||||||
|
"recovery": zone["recovery"],
|
||||||
"t_ref": zone["t_ref"], "wind": weather.get("wind"), "gust": weather.get("wind_gust"),
|
"t_ref": zone["t_ref"], "wind": weather.get("wind"), "gust": weather.get("wind_gust"),
|
||||||
"affected": zone["affected"], "known": known,
|
"affected": zone["affected"], "known": known,
|
||||||
"unknown_windows": list(zone["unknown_windows"])}
|
"unknown_windows": list(zone["unknown_windows"])}
|
||||||
|
|
||||||
|
def assumption(self, window: Window, open_minutes: float, now: float) -> dict[str, Any]:
|
||||||
|
"""What forecasts assume for one window, and on what basis.
|
||||||
|
|
||||||
|
Observed open: open for `open_minutes` more. Unknown but last seen open: open
|
||||||
|
until `open_minutes` after it was last seen, never renewed, so an unavailable
|
||||||
|
sensor does not keep a window open forever. Unknown otherwise: no ventilation,
|
||||||
|
marked uncertain (`last_known_closed`, or `unobserved` with no reading at all).
|
||||||
|
"""
|
||||||
|
state = self.window_state.get(window.key, {})
|
||||||
|
if state.get("open"):
|
||||||
|
return {"basis": "observed", "open_minutes": float(open_minutes), "uncertain": False}
|
||||||
|
if state.get("open") is False:
|
||||||
|
return {"basis": "observed", "open_minutes": 0.0, "uncertain": False}
|
||||||
|
if state.get("last_known_open"):
|
||||||
|
seen = state.get("seen")
|
||||||
|
left = open_minutes - (now - seen) / 60 if seen is not None else 0.0
|
||||||
|
return {"basis": "last_known_open", "open_minutes": round(max(0.0, left), 1), "uncertain": True,
|
||||||
|
"assumed_closed_at": seen + open_minutes * 60 if seen is not None else now}
|
||||||
|
basis = "last_known_closed" if state.get("last_known_open") is False else "unobserved"
|
||||||
|
return {"basis": basis, "open_minutes": 0.0, "uncertain": True}
|
||||||
|
|
||||||
def forecast(self, key: str, steps: int, open_minutes: float, weather: Mapping[str, float | None],
|
def forecast(self, key: str, steps: int, open_minutes: float, weather: Mapping[str, float | None],
|
||||||
now: float) -> list[dict[str, Any]]:
|
now: float) -> list[dict[str, Any]]:
|
||||||
"""Assumed ventilation inputs per forecast step.
|
"""Assumed ventilation inputs per forecast step (see `assumption`).
|
||||||
|
|
||||||
Windows open now (or unknown but last seen open) are assumed to close after
|
Each window open now, or assumed open, closes after its assumed time; once
|
||||||
`open_minutes` (at least one step); after that the zone recovers. Weather
|
the last one has closed the zone recovers. Weather is held at current values.
|
||||||
is held at current values.
|
|
||||||
"""
|
"""
|
||||||
zone = self.zone_state.get(key)
|
zone = self.zone_state.get(key)
|
||||||
if zone is None:
|
if zone is None:
|
||||||
return [{} for _ in range(steps)]
|
return [{} for _ in range(steps)]
|
||||||
windows = [w for w in self.windows if w.zone == key and w.enabled
|
windows = [w for w in self.windows if w.zone == key and w.enabled]
|
||||||
and (self.window_state.get(w.key, {}).get("open")
|
assumed = {w.key: self.assumption(w, open_minutes, now) for w in windows}
|
||||||
or (self.window_state.get(w.key, {}).get("open") is None
|
# Steps each window stays open: at least one for an observed open window.
|
||||||
and self.window_state.get(w.key, {}).get("last_known_open")))]
|
open_steps = {w.key: (max(1, round(open_minutes / 5)) if self.window_state.get(w.key, {}).get("open")
|
||||||
opening = sum(w.opening for w in windows)
|
else round(assumed[w.key]["open_minutes"] / 5)) for w in windows}
|
||||||
facing = sum(w.opening * facing_cos(weather["wind_direction"], w.azimuth) for w in windows) \
|
direction = weather.get("wind_direction")
|
||||||
if weather.get("wind_direction") is not None else 0.0
|
last_open = max(open_steps.values(), default=0)
|
||||||
open_steps = max(1, round(open_minutes / 5)) if windows else 0
|
if last_open:
|
||||||
closed_at = self._closed_at.get(key)
|
start = now + last_open * 300.0
|
||||||
|
else:
|
||||||
|
# Recovery from the last close: observed, or assumed for a window last seen open.
|
||||||
|
closes = [a["assumed_closed_at"] for a in assumed.values() if "assumed_closed_at" in a]
|
||||||
|
closed_at = self._closed_at.get(key)
|
||||||
|
start = max([t for t in (closed_at, *closes) if t is not None], default=None)
|
||||||
result = []
|
result = []
|
||||||
for k in range(steps):
|
for k in range(steps):
|
||||||
t = now + (k + 1) * 300.0
|
t = now + (k + 1) * 300.0
|
||||||
if k < open_steps:
|
open_now = [w for w in windows if k < open_steps[w.key]]
|
||||||
result.append({"opening": opening, "facing": facing, "recovery": 0.0, "t_ref": zone["t_ref"],
|
if open_now:
|
||||||
"other_opening": zone["other_opening"], "wind": weather.get("wind") or 0.0,
|
result.append({
|
||||||
"gust": weather.get("wind_gust") or weather.get("wind") or 0.0})
|
"opening": sum(w.opening for w in open_now),
|
||||||
|
"facing": (sum(w.opening * facing_cos(direction, w.azimuth) for w in open_now)
|
||||||
|
if direction is not None else 0.0),
|
||||||
|
"recovery": 0.0, "t_ref": zone["t_ref"], "other_opening": zone["other_opening"],
|
||||||
|
"wind": weather.get("wind") or 0.0,
|
||||||
|
"gust": weather.get("wind_gust") or weather.get("wind") or 0.0})
|
||||||
continue
|
continue
|
||||||
start = now + open_steps * 300.0 if windows else closed_at
|
recovery = exp(-(t - start) / self.tau) if start is not None and t > start else 0.0
|
||||||
recovery = exp(-(t - start) / self.tau) if start is not None else 0.0
|
|
||||||
result.append({"opening": 0.0, "facing": 0.0, "other_opening": 0.0, "wind": 0.0, "gust": 0.0,
|
result.append({"opening": 0.0, "facing": 0.0, "other_opening": 0.0, "wind": 0.0, "gust": 0.0,
|
||||||
"recovery": recovery if recovery >= RECOVERY_THRESHOLD else 0.0,
|
"recovery": recovery if recovery >= RECOVERY_THRESHOLD else 0.0,
|
||||||
"t_ref": zone["t_ref"]})
|
"t_ref": zone["t_ref"]})
|
||||||
return result
|
return result
|
||||||
|
|
||||||
def as_dict(self) -> dict[str, Any]:
|
def as_dict(self) -> dict[str, Any]:
|
||||||
return {"events": list(self.events)}
|
"""Events, plus the state a restart or reload needs to continue them."""
|
||||||
|
windows = {w.key: w for w in self.windows if w.enabled}
|
||||||
|
state = {
|
||||||
|
"time": self._last_update,
|
||||||
|
"windows": {key: {"sensor": windows[key].sensor, "inverted": windows[key].inverted,
|
||||||
|
"last_known": m["last_known"], "since": m["since"], "seen": m["seen"]}
|
||||||
|
for key, m in self._memory.items()
|
||||||
|
if key in windows and m["last_known"] is not None and m["seen"] is not None},
|
||||||
|
"zones": {zone: {"open_event": self._open_event.get(zone), "closed_at": self._closed_at.get(zone),
|
||||||
|
"recovery_confirmed": self._recovery_confirmed.get(zone, True),
|
||||||
|
"t_ref": self._t_ref.get(zone)}
|
||||||
|
for zone in {w.zone for w in windows.values()}
|
||||||
|
if zone in self._open_event or zone in self._closed_at},
|
||||||
|
}
|
||||||
|
return {"events": list(self.events), "state": state}
|
||||||
|
|
||||||
|
def _restore(self, state: Any) -> None:
|
||||||
|
"""Continue from saved state. Nothing restored is live: every window is unknown
|
||||||
|
from when it was last seen until its sensor reports again."""
|
||||||
|
if not isinstance(state, Mapping):
|
||||||
|
return
|
||||||
|
windows = {w.key: w for w in self.windows if w.enabled}
|
||||||
|
saved_windows = state.get("windows") if isinstance(state.get("windows"), Mapping) else {}
|
||||||
|
for key, item in saved_windows.items():
|
||||||
|
window = windows.get(key)
|
||||||
|
if (window is None or not isinstance(item, Mapping) or item.get("sensor") != window.sensor
|
||||||
|
or item.get("inverted") is not window.inverted or not isinstance(item.get("last_known"), bool)
|
||||||
|
or not _finite(item.get("seen"))):
|
||||||
|
continue # another sensor now, or polarity flipped: nothing carries over
|
||||||
|
seen = float(item["seen"])
|
||||||
|
self._memory[key] = {"last_known": item["last_known"], "seen": seen, "unknown_since": seen,
|
||||||
|
"since": float(item["since"]) if _finite(item.get("since")) else None}
|
||||||
|
remembered_open = {windows[key].zone for key, m in self._memory.items() if m["last_known"]}
|
||||||
|
saved_zones = state.get("zones") if isinstance(state.get("zones"), Mapping) else {}
|
||||||
|
for zone, item in saved_zones.items():
|
||||||
|
if zone not in {w.zone for w in windows.values()} or not isinstance(item, Mapping):
|
||||||
|
continue
|
||||||
|
event = item.get("open_event")
|
||||||
|
# An open event continues only while one of its windows is remembered open.
|
||||||
|
if isinstance(event, dict) and zone in remembered_open and _valid_iso(event.get("opened")):
|
||||||
|
self._open_event[zone] = dict(event)
|
||||||
|
if _finite(item.get("closed_at")):
|
||||||
|
self._closed_at[zone] = float(item["closed_at"])
|
||||||
|
self._recovery_confirmed[zone] = item.get("recovery_confirmed") is True
|
||||||
|
if _finite(item.get("t_ref")):
|
||||||
|
self._t_ref[zone] = float(item["t_ref"])
|
||||||
|
if self._open_event and _finite(state.get("time")):
|
||||||
|
self._restored_at = float(state["time"])
|
||||||
|
|
||||||
|
|
||||||
|
def _finite(value: Any) -> bool:
|
||||||
|
return isinstance(value, (int, float)) and not isinstance(value, bool) and isfinite(value)
|
||||||
|
|
||||||
|
|
||||||
|
def _valid_iso(value: Any) -> bool:
|
||||||
|
try:
|
||||||
|
datetime.fromisoformat(value)
|
||||||
|
return True
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
def _parse(iso: str) -> float:
|
def _parse(iso: str) -> float:
|
||||||
|
|||||||
+47
-12
@@ -98,11 +98,19 @@ known. While a window is open, possibly open (its sensor is unknown and it was l
|
|||||||
seen open) or the zone is recovering, the base and extended layers do **not**
|
seen open) or the zone is recovering, the base and extended layers do **not**
|
||||||
learn, so window events cannot inflate `loss` or shift `bias`.
|
learn, so window events cannot inflate `loss` or shift `bias`.
|
||||||
|
|
||||||
An unknown window state is never read as closed. A window last seen open stays
|
An unknown window state is never read as closed, and an opening or closing counts
|
||||||
assumed open in forecasts; one last seen closed contributes no ventilation term.
|
as confirmed only when it was seen: the previous reading was the other state with
|
||||||
If a window closes while its sensor was unknown, the event's duration is reported
|
no unknown period (or restart) in between. If a window closes while its sensor was
|
||||||
as a range and the recovery term is not learned from it, because the closing time
|
unknown, the event's duration is reported as a range and the recovery term is not
|
||||||
is not known.
|
learned from it, because the closing time is not known. Last known states, open
|
||||||
|
events and recovery are saved and restored across restarts, as unknown until the
|
||||||
|
sensor reports again.
|
||||||
|
|
||||||
|
An input that a term needs is never filled in for learning or for Model Error:
|
||||||
|
while a window is open, the wind, its direction (for `vent_wind_dir`) and the other
|
||||||
|
zones' windows (for `vent_cross`) must be known. With every window closed, the
|
||||||
|
opening terms are 0 and need no weather. Forecasts, which must produce a number,
|
||||||
|
assume calm air for a missing wind input.
|
||||||
|
|
||||||
`vent_wind_dir` is bounded by `vent_wind` (|vent_wind_dir| ≤ vent_wind), so the
|
`vent_wind_dir` is bounded by `vent_wind` (|vent_wind_dir| ≤ vent_wind), so the
|
||||||
combined wind term never goes negative: a leeward window still ventilates.
|
combined wind term never goes negative: a leeward window still ventilates.
|
||||||
@@ -202,7 +210,10 @@ minutes):
|
|||||||
its response time. A gaming PC drawing 600 W now is not assumed to run on for two
|
its response time. A gaming PC drawing 600 W now is not assumed to run on for two
|
||||||
hours.
|
hours.
|
||||||
* **Windows:** an open window is assumed to stay open for a configurable time
|
* **Windows:** an open window is assumed to stay open for a configurable time
|
||||||
(default 15 minutes), then close; the zone then recovers.
|
(default 15 minutes), then close; the zone then recovers. A window whose sensor
|
||||||
|
is unknown but was last seen open is assumed open only until that time after it
|
||||||
|
was last seen, never renewed. One last seen closed, or never seen since setup, is
|
||||||
|
assumed closed, and the forecast is flagged as uncertain (`uncertain_windows`).
|
||||||
* **Relay:** the candidate plan being evaluated.
|
* **Relay:** the candidate plan being evaluated.
|
||||||
|
|
||||||
For the plan search the step is compiled into exact per-zone coefficients (every
|
For the plan search the step is compiled into exact per-zone coefficients (every
|
||||||
@@ -228,13 +239,34 @@ scored on exactly the same spans:
|
|||||||
* `base_model_mae`, `improvement_vs_base`: against the base layer alone. A feature
|
* `base_model_mae`, `improvement_vs_base`: against the base layer alone. A feature
|
||||||
that does not improve this should be switched off (for example the gust term).
|
that does not improve this should be switched off (for example the gust term).
|
||||||
|
|
||||||
|
A span counts only if every input the zone's model used was observed at every
|
||||||
|
step (see *Ventilation layer*; for the extended layer also electrical heat and
|
||||||
|
neighbours, whose own replay must be complete). Otherwise it is counted as
|
||||||
|
`incomplete`, with the base model's error on it kept apart
|
||||||
|
(`incomplete_base_model_mae`), since the base model needs only the relay and the
|
||||||
|
outdoor temperature. Spans that cannot be replayed (uncertain radiator lag, no
|
||||||
|
reading at the end) are counted as `skipped`.
|
||||||
|
|
||||||
**Forecast accuracy (Forecast Error sensor)** scores forecasts as published: the
|
**Forecast accuracy (Forecast Error sensor)** scores forecasts as published: the
|
||||||
start readings, trajectory, plan, model revision and assumptions of the moment
|
start readings, trajectory, plan, model revision and assumptions of the moment
|
||||||
they were made, nothing replaced with hindsight. Because the plan is the MPC's
|
they were made, nothing replaced with hindsight. Because the plan is the MPC's
|
||||||
proposal while Generic Thermostat controls the relay, a forecast is scored at a
|
proposal while Generic Thermostat controls the relay, a forecast is compared only
|
||||||
horizon only if the relay was in the planned state for at least 12 minutes of
|
when the relay followed it. The deviation is the time, up to the horizon, the
|
||||||
every 15-minute block up to it; otherwise it is counted as `counterfactual` (or
|
relay spent in another state than planned, compared block by block from the
|
||||||
`unverifiable` when the relay history is incomplete) and never as an error.
|
relay's own switch times; unobserved relay time counts as possible deviation.
|
||||||
|
|
||||||
|
| Deviation (worst case) | Class | Used for |
|
||||||
|
|---|---|---|
|
||||||
|
| ≤ 2 min | matched | Forecast Error figures |
|
||||||
|
| ≤ 5 min, ≤ 2 min unobserved | approximate | `approximate_stats` only |
|
||||||
|
| > 5 min observed | counterfactual | counted, never an error |
|
||||||
|
| otherwise | incomplete | counted, with a reason |
|
||||||
|
|
||||||
|
Forecasts made with an assumed input (an unknown window or electrical heat) are
|
||||||
|
incomplete as well. The limits follow from the model: d seconds of unplanned
|
||||||
|
boiler heat change a room by about heat × d / 300 with the learned `heat`
|
||||||
|
coefficient (typically 0.05–0.12 °C per 5 minutes), so 2 minutes stay below half a
|
||||||
|
0.1 °C sensor step and 5 minutes are about one step.
|
||||||
|
|
||||||
Both report how much evidence there is. Forecasts start every 5 minutes, so at
|
Both report how much evidence there is. Forecasts start every 5 minutes, so at
|
||||||
horizon h consecutive forecasts share most of their span, and all zones share one
|
horizon h consecutive forecasts share most of their span, and all zones share one
|
||||||
@@ -267,5 +299,8 @@ the contamination the separate ventilation layer prevents.
|
|||||||
variation their split is uncertain even when their combined effect is accurate.
|
variation their split is uncertain even when their combined effect is accurate.
|
||||||
* Forecasts hold weather constant and use one scenario for electrical heat and
|
* Forecasts hold weather constant and use one scenario for electrical heat and
|
||||||
windows; no forecast uncertainty band is computed yet.
|
windows; no forecast uncertainty band is computed yet.
|
||||||
* Window sensors with a permanently unknown state keep their zone out of learning;
|
* Window sensors with a permanently unknown state keep their zone out of learning
|
||||||
disable such a window.
|
and its Model Error spans incomplete; disable such a window.
|
||||||
|
* A window that is unknown and was last seen closed (or never seen) is assumed
|
||||||
|
closed in forecasts, flagged as uncertain; no open/closed scenario pair is
|
||||||
|
computed.
|
||||||
@@ -0,0 +1,134 @@
|
|||||||
|
"""Unload, reload and removal while work is still in flight, in the real HA lifecycle."""
|
||||||
|
import asyncio
|
||||||
|
import threading
|
||||||
|
from datetime import timedelta
|
||||||
|
from unittest.mock import patch
|
||||||
|
|
||||||
|
from homeassistant.helpers import entity_registry as er
|
||||||
|
from homeassistant.helpers.storage import Store
|
||||||
|
from homeassistant.util import dt as dt_util
|
||||||
|
from pytest_homeassistant_custom_component.common import async_fire_time_changed
|
||||||
|
|
||||||
|
from custom_components.heating_mpc import coordinator as coordinator_module
|
||||||
|
from custom_components.heating_mpc.const import DOMAIN
|
||||||
|
|
||||||
|
REAL_OPTIMIZE = coordinator_module.optimize
|
||||||
|
|
||||||
|
|
||||||
|
async def setup(hass, entry):
|
||||||
|
entry.add_to_hass(hass)
|
||||||
|
assert await hass.config_entries.async_setup(entry.entry_id)
|
||||||
|
await hass.async_block_till_done()
|
||||||
|
return hass.data[DOMAIN][entry.entry_id]
|
||||||
|
|
||||||
|
|
||||||
|
def trained(coordinator):
|
||||||
|
model = coordinator.learners["sensor.room"]
|
||||||
|
model.count, model.on_count, model.off_count = 250, 125, 125
|
||||||
|
coordinator._update_counts()
|
||||||
|
|
||||||
|
|
||||||
|
class BlockedOptimizer:
|
||||||
|
"""Holds the optimizer in its worker thread until released."""
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
self.started, self.release = threading.Event(), threading.Event()
|
||||||
|
|
||||||
|
def __call__(self, *args, **kwargs):
|
||||||
|
self.started.set()
|
||||||
|
self.release.wait(5)
|
||||||
|
return REAL_OPTIMIZE(*args, **kwargs)
|
||||||
|
|
||||||
|
async def wait_started(self):
|
||||||
|
# Polled on the event loop: the test executor may run one job at a time.
|
||||||
|
for _ in range(500):
|
||||||
|
if self.started.is_set():
|
||||||
|
return True
|
||||||
|
await asyncio.sleep(0.01)
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def status_id(hass, entry):
|
||||||
|
return er.async_get(hass).async_get_entity_id("sensor", DOMAIN, f"{entry.entry_id}_status")
|
||||||
|
|
||||||
|
|
||||||
|
async def test_unload_during_optimization_publishes_nothing(hass, entry, inputs):
|
||||||
|
coordinator = await setup(hass, entry)
|
||||||
|
trained(coordinator)
|
||||||
|
shown = coordinator.data
|
||||||
|
blocked = BlockedOptimizer()
|
||||||
|
with patch.object(coordinator_module, "optimize", blocked):
|
||||||
|
refresh = hass.async_create_task(coordinator.async_refresh())
|
||||||
|
assert await blocked.wait_started()
|
||||||
|
assert await hass.config_entries.async_unload(entry.entry_id)
|
||||||
|
state_after_unload = hass.states.get(status_id(hass, entry))
|
||||||
|
blocked.release.set()
|
||||||
|
await refresh
|
||||||
|
await hass.async_block_till_done()
|
||||||
|
# The plan finished after unloading began: kept out of the data and the states.
|
||||||
|
assert coordinator.data is shown
|
||||||
|
assert coordinator.data["status"] == "learning"
|
||||||
|
assert hass.states.get(status_id(hass, entry)) == state_after_unload
|
||||||
|
|
||||||
|
|
||||||
|
async def test_reload_during_optimization_shows_only_the_new_coordinator(hass, entry, inputs):
|
||||||
|
old = await setup(hass, entry)
|
||||||
|
trained(old) # only in memory: the reloaded coordinator is not ready
|
||||||
|
blocked = BlockedOptimizer()
|
||||||
|
with patch.object(coordinator_module, "optimize", blocked):
|
||||||
|
refresh = hass.async_create_task(old.async_refresh())
|
||||||
|
assert await blocked.wait_started()
|
||||||
|
assert await hass.config_entries.async_reload(entry.entry_id)
|
||||||
|
await hass.async_block_till_done()
|
||||||
|
new = hass.data[DOMAIN][entry.entry_id]
|
||||||
|
blocked.release.set()
|
||||||
|
await refresh
|
||||||
|
await hass.async_block_till_done()
|
||||||
|
assert new is not old and old.closing
|
||||||
|
assert old.data["status"] != "ready"
|
||||||
|
assert hass.states.get(status_id(hass, entry)).state == new.data["status"] == "learning"
|
||||||
|
|
||||||
|
|
||||||
|
async def test_removal_during_a_save_leaves_no_model_behind(hass, entry, inputs, freezer):
|
||||||
|
coordinator = await setup(hass, entry)
|
||||||
|
started, release = asyncio.Event(), asyncio.Event()
|
||||||
|
write = coordinator._store.async_save
|
||||||
|
|
||||||
|
async def slow_save(data):
|
||||||
|
started.set()
|
||||||
|
await release.wait()
|
||||||
|
await write(data)
|
||||||
|
coordinator._store.async_save = slow_save
|
||||||
|
future = dt_util.utcnow() + timedelta(minutes=5, seconds=1)
|
||||||
|
freezer.move_to(future)
|
||||||
|
async_fire_time_changed(hass, future) # a learning poll, which saves
|
||||||
|
await asyncio.wait_for(started.wait(), 5)
|
||||||
|
removal = hass.async_create_task(hass.config_entries.async_remove(entry.entry_id))
|
||||||
|
for _ in range(5):
|
||||||
|
await asyncio.sleep(0)
|
||||||
|
assert not removal.done() # deletion waits for the save in progress
|
||||||
|
release.set()
|
||||||
|
assert await removal
|
||||||
|
await hass.async_block_till_done()
|
||||||
|
assert await Store(hass, 1, f"{DOMAIN}.{entry.entry_id}").async_load() is None
|
||||||
|
|
||||||
|
|
||||||
|
async def test_repeated_setup_and_unload_leave_nothing_running(hass, entry, inputs, freezer):
|
||||||
|
entry.add_to_hass(hass)
|
||||||
|
coordinators = []
|
||||||
|
for _ in range(3):
|
||||||
|
assert await hass.config_entries.async_setup(entry.entry_id)
|
||||||
|
await hass.async_block_till_done()
|
||||||
|
coordinators.append(hass.data[DOMAIN][entry.entry_id])
|
||||||
|
assert await hass.config_entries.async_unload(entry.entry_id)
|
||||||
|
await hass.async_block_till_done()
|
||||||
|
assert entry.entry_id not in hass.data[DOMAIN]
|
||||||
|
assert all(c.closing and c._unsub_refresh is None and not c._listeners for c in coordinators)
|
||||||
|
counts = [c.learners["sensor.room"].count for c in coordinators]
|
||||||
|
hass.states.async_set("switch.boiler", "on")
|
||||||
|
future = dt_util.utcnow() + timedelta(minutes=10)
|
||||||
|
freezer.move_to(future)
|
||||||
|
async_fire_time_changed(hass, future)
|
||||||
|
await hass.async_block_till_done()
|
||||||
|
assert not any(c._relay_events for c in coordinators)
|
||||||
|
assert [c.learners["sensor.room"].count for c in coordinators] == counts
|
||||||
@@ -21,6 +21,55 @@ async def setup_entry(hass, entry):
|
|||||||
return hass.data[DOMAIN][entry.entry_id]
|
return hass.data[DOMAIN][entry.entry_id]
|
||||||
|
|
||||||
|
|
||||||
|
async def open_settings(hass, entry):
|
||||||
|
"""Configure opens a menu; Settings is the first choice."""
|
||||||
|
result = await hass.config_entries.options.async_init(entry.entry_id)
|
||||||
|
assert result["type"] is FlowResultType.MENU
|
||||||
|
assert result["menu_options"] == ["settings", "reset_model"]
|
||||||
|
return await hass.config_entries.options.async_configure(result["flow_id"], {"next_step_id": "settings"})
|
||||||
|
|
||||||
|
|
||||||
|
async def open_reset(hass, entry):
|
||||||
|
result = await hass.config_entries.options.async_init(entry.entry_id)
|
||||||
|
return await hass.config_entries.options.async_configure(result["flow_id"], {"next_step_id": "reset_model"})
|
||||||
|
|
||||||
|
|
||||||
|
async def test_reset_learned_model_keeps_configuration_and_observations(hass, entry, inputs, hass_storage):
|
||||||
|
coordinator = await setup_entry(hass, entry)
|
||||||
|
model = coordinator.learners["sensor.room"]
|
||||||
|
model.count, model.on_count, model.off_count = 250, 125, 125
|
||||||
|
payload = coordinator._payload("sensor.outside")
|
||||||
|
payload["retired_learners"] = {"sensor.gone": {**model.as_dict()}}
|
||||||
|
payload["ventilation"] = {"events": [{"zone": "sensor.room", "opened": "2026-10-27T10:00:00+00:00"}]}
|
||||||
|
payload["heat_sources"] = {"samples": {"ups": [[1.0, 200.0]]}}
|
||||||
|
await coordinator._store.async_save(payload)
|
||||||
|
assert await hass.config_entries.async_reload(entry.entry_id)
|
||||||
|
await hass.async_block_till_done()
|
||||||
|
assert hass.data[DOMAIN][entry.entry_id].data["status"] == "ready"
|
||||||
|
before = (dict(entry.data), dict(entry.options), dict(entry.subentries))
|
||||||
|
# Not confirmed: nothing happens.
|
||||||
|
result = await open_reset(hass, entry)
|
||||||
|
assert (result["type"], result["step_id"]) == (FlowResultType.FORM, "reset_model")
|
||||||
|
result = await hass.config_entries.options.async_configure(result["flow_id"], {"confirm_reset": False})
|
||||||
|
assert (result["type"], result["reason"]) == (FlowResultType.ABORT, "model_not_reset")
|
||||||
|
assert hass.data[DOMAIN][entry.entry_id].learners["sensor.room"].count == 250
|
||||||
|
# Confirmed: a fresh model, the same configuration.
|
||||||
|
result = await open_reset(hass, entry)
|
||||||
|
result = await hass.config_entries.options.async_configure(result["flow_id"], {"confirm_reset": True})
|
||||||
|
assert (result["type"], result["reason"]) == (FlowResultType.ABORT, "model_reset")
|
||||||
|
await hass.async_block_till_done()
|
||||||
|
fresh = hass.data[DOMAIN][entry.entry_id]
|
||||||
|
assert fresh is not coordinator and fresh.learners["sensor.room"].count == 0
|
||||||
|
assert fresh.data["status"] == "learning" and not fresh._retired
|
||||||
|
assert (dict(entry.data), dict(entry.options), dict(entry.subentries)) == before
|
||||||
|
stored = hass_storage[f"{DOMAIN}.{entry.entry_id}"]["data"]
|
||||||
|
assert not {"learners", "retired_learners", "evaluation", "operational_evaluation", "fingerprints"} & set(stored)
|
||||||
|
assert stored["ventilation"]["events"][0]["opened"] == "2026-10-27T10:00:00+00:00"
|
||||||
|
assert stored["heat_sources"] == {"samples": {"ups": [[1.0, 200.0]]}}
|
||||||
|
assert stored["model_changes"][-1]["reasons"] == ["learned model reset by the user"]
|
||||||
|
assert fresh.data["model_changes"][-1]["recalibrated"] == ["all"]
|
||||||
|
|
||||||
|
|
||||||
async def test_config_flow_creates_one_zone_per_sensor(hass, inputs):
|
async def test_config_flow_creates_one_zone_per_sensor(hass, inputs):
|
||||||
hass.states.async_set("sensor.bedroom", "19", {"unit_of_measurement": "°C", "friendly_name": "Bedroom"})
|
hass.states.async_set("sensor.bedroom", "19", {"unit_of_measurement": "°C", "friendly_name": "Bedroom"})
|
||||||
result = await hass.config_entries.flow.async_init(DOMAIN, context={"source": SOURCE_USER})
|
result = await hass.config_entries.flow.async_init(DOMAIN, context={"source": SOURCE_USER})
|
||||||
@@ -51,7 +100,7 @@ async def test_config_flow_creates_one_zone_per_sensor(hass, inputs):
|
|||||||
|
|
||||||
async def test_options_flow_keeps_settings_without_rooms(hass, entry, inputs):
|
async def test_options_flow_keeps_settings_without_rooms(hass, entry, inputs):
|
||||||
await setup_entry(hass, entry)
|
await setup_entry(hass, entry)
|
||||||
result = await hass.config_entries.options.async_init(entry.entry_id)
|
result = await open_settings(hass, entry)
|
||||||
assert "room_entities" not in {str(key) for key in result["data_schema"].schema}
|
assert "room_entities" not in {str(key) for key in result["data_schema"].schema}
|
||||||
main = result["data_schema"]({**MAIN, "room_max_age_minutes": 45, "room_stale_use_last_value": True})
|
main = result["data_schema"]({**MAIN, "room_max_age_minutes": 45, "room_stale_use_last_value": True})
|
||||||
result = await hass.config_entries.options.async_configure(result["flow_id"], main)
|
result = await hass.config_entries.options.async_configure(result["flow_id"], main)
|
||||||
@@ -65,14 +114,14 @@ async def test_options_flow_keeps_settings_without_rooms(hass, entry, inputs):
|
|||||||
assert coordinator.config["room_max_age_minutes"] == 45
|
assert coordinator.config["room_max_age_minutes"] == 45
|
||||||
assert coordinator.config["room_stale_use_last_value"] is True
|
assert coordinator.config["room_stale_use_last_value"] is True
|
||||||
assert coordinator.rooms == ["sensor.room"] # zones are untouched by Configure
|
assert coordinator.rooms == ["sensor.room"] # zones are untouched by Configure
|
||||||
result = await hass.config_entries.options.async_init(entry.entry_id)
|
result = await open_settings(hass, entry)
|
||||||
assert result["data_schema"](MAIN)["room_stale_use_last_value"] is True
|
assert result["data_schema"](MAIN)["room_stale_use_last_value"] is True
|
||||||
hass.config_entries.options.async_abort(result["flow_id"])
|
hass.config_entries.options.async_abort(result["flow_id"])
|
||||||
|
|
||||||
|
|
||||||
async def test_weather_source_may_not_be_a_zone_sensor(hass, entry, inputs):
|
async def test_weather_source_may_not_be_a_zone_sensor(hass, entry, inputs):
|
||||||
await setup_entry(hass, entry)
|
await setup_entry(hass, entry)
|
||||||
result = await hass.config_entries.options.async_init(entry.entry_id)
|
result = await open_settings(hass, entry)
|
||||||
result = await hass.config_entries.options.async_configure(result["flow_id"], result["data_schema"](MAIN))
|
result = await hass.config_entries.options.async_configure(result["flow_id"], result["data_schema"](MAIN))
|
||||||
result = await hass.config_entries.options.async_configure(
|
result = await hass.config_entries.options.async_configure(
|
||||||
result["flow_id"], result["data_schema"]({"outside_temperature_sources": ["sensor.room"]}))
|
result["flow_id"], result["data_schema"]({"outside_temperature_sources": ["sensor.room"]}))
|
||||||
@@ -172,7 +221,7 @@ async def test_options_flow_clears_optional_weather_sources(hass, inputs):
|
|||||||
"weights": {"sensor.room": 1.0}})
|
"weights": {"sensor.room": 1.0}})
|
||||||
await setup_entry(hass, entry)
|
await setup_entry(hass, entry)
|
||||||
flow = hass.config_entries.options
|
flow = hass.config_entries.options
|
||||||
result = await flow.async_init(entry.entry_id)
|
result = await open_settings(hass, entry)
|
||||||
result = await flow.async_configure(result["flow_id"], result["data_schema"](MAIN))
|
result = await flow.async_configure(result["flow_id"], result["data_schema"](MAIN))
|
||||||
markers = {str(key): key for key in result["data_schema"].schema}
|
markers = {str(key): key for key in result["data_schema"].schema}
|
||||||
assert markers["outside_wind_sources"].description == {"suggested_value": ["sensor.wind"]}
|
assert markers["outside_wind_sources"].description == {"suggested_value": ["sensor.wind"]}
|
||||||
@@ -245,7 +294,7 @@ async def test_window_subentry_flow_and_ventilation_options(hass, entry, inputs)
|
|||||||
("sensor.gust", "9", "m/s")):
|
("sensor.gust", "9", "m/s")):
|
||||||
hass.states.async_set(sensor, value, {"unit_of_measurement": unit})
|
hass.states.async_set(sensor, value, {"unit_of_measurement": unit})
|
||||||
options = hass.config_entries.options
|
options = hass.config_entries.options
|
||||||
result = await options.async_init(entry.entry_id)
|
result = await open_settings(hass, entry)
|
||||||
result = await options.async_configure(result["flow_id"], result["data_schema"](
|
result = await options.async_configure(result["flow_id"], result["data_schema"](
|
||||||
{**MAIN, "ventilation_gust": True, "ventilation_cross": True}))
|
{**MAIN, "ventilation_gust": True, "ventilation_cross": True}))
|
||||||
result = await options.async_configure(result["flow_id"], result["data_schema"]({
|
result = await options.async_configure(result["flow_id"], result["data_schema"]({
|
||||||
|
|||||||
@@ -302,6 +302,7 @@ async def test_forecast_error_sensors_score_scheduled_polls(hass, entry, inputs,
|
|||||||
assert state.attributes["mae"] == accuracy["mae"] > 0
|
assert state.attributes["mae"] == accuracy["mae"] > 0
|
||||||
assert state.attributes["quality"] == "insufficient"
|
assert state.attributes["quality"] == "insufficient"
|
||||||
assert state.attributes["samples"] == 1
|
assert state.attributes["samples"] == 1
|
||||||
|
assert (state.attributes["incomplete"], state.attributes["skipped"]) == (0, 0)
|
||||||
assert state.attributes["zones"]["room"]["samples"] == 1 # by zone name
|
assert state.attributes["zones"]["room"]["samples"] == 1 # by zone name
|
||||||
assert state.attributes["unit_of_measurement"] == "°C"
|
assert state.attributes["unit_of_measurement"] == "°C"
|
||||||
assert state.attributes["friendly_name"] == "Heating MPC Model Error 15 min"
|
assert state.attributes["friendly_name"] == "Heating MPC Model Error 15 min"
|
||||||
|
|||||||
+133
-3
@@ -24,6 +24,9 @@ class CoordinatorBase:
|
|||||||
async def async_config_entry_first_refresh(self):
|
async def async_config_entry_first_refresh(self):
|
||||||
self.data = await self._async_update_data()
|
self.data = await self._async_update_data()
|
||||||
|
|
||||||
|
async def async_shutdown(self):
|
||||||
|
self._shutdown_requested = True
|
||||||
|
|
||||||
|
|
||||||
class Store:
|
class Store:
|
||||||
def __init__(self, *args):
|
def __init__(self, *args):
|
||||||
@@ -72,6 +75,7 @@ class BoundaryTests(unittest.IsolatedAsyncioTestCase):
|
|||||||
config=types.SimpleNamespace(units=types.SimpleNamespace(temperature_unit='°C')),
|
config=types.SimpleNamespace(units=types.SimpleNamespace(temperature_unit='°C')),
|
||||||
services=types.SimpleNamespace(async_call=Mock(side_effect=AssertionError('No service calls permitted'))),
|
services=types.SimpleNamespace(async_call=Mock(side_effect=AssertionError('No service calls permitted'))),
|
||||||
track_state=Mock(return_value=Mock(name='unsub_relay')),
|
track_state=Mock(return_value=Mock(name='unsub_relay')),
|
||||||
|
async_create_task=lambda coro, name=None, eager_start=True: asyncio.ensure_future(coro),
|
||||||
registry=types.SimpleNamespace(entries=[], async_remove=Mock()),
|
registry=types.SimpleNamespace(entries=[], async_remove=Mock()),
|
||||||
config_entries=types.SimpleNamespace(
|
config_entries=types.SimpleNamespace(
|
||||||
async_forward_entry_setups=AsyncMock(), async_unload_platforms=AsyncMock(return_value=True),
|
async_forward_entry_setups=AsyncMock(), async_unload_platforms=AsyncMock(return_value=True),
|
||||||
@@ -1215,6 +1219,35 @@ class BoundaryTests(unittest.IsolatedAsyncioTestCase):
|
|||||||
base = self.c.learners['office']
|
base = self.c.learners['office']
|
||||||
base.count, base.on_count, base.off_count = 200, 100, 100
|
base.count, base.on_count, base.off_count = 200, 100, 100
|
||||||
|
|
||||||
|
async def test_reload_with_an_unavailable_window_keeps_its_last_known_state_uncertain(self):
|
||||||
|
self.ready_office()
|
||||||
|
await self.sample()
|
||||||
|
self.fresh('sensor.room', 'sensor.outside', 'sensor.ups_power')
|
||||||
|
window = self.states['binary_sensor.office_window']
|
||||||
|
window.state, window.last_changed = 'on', self.now - timedelta(seconds=100)
|
||||||
|
self.c._store.async_save.reset_mock()
|
||||||
|
await self.sample()
|
||||||
|
self.c._store.async_save.assert_awaited() # a window change is saved even without learning
|
||||||
|
payload = self.c._store.async_save.await_args.args[0]
|
||||||
|
# Reload: a new coordinator restores the saved state; the sensor is now unavailable.
|
||||||
|
window.state = 'unavailable'
|
||||||
|
self.fresh('sensor.room', 'sensor.outside', 'sensor.ups_power')
|
||||||
|
successor = coordinator.HeatingMPCCoordinator(self.hass, self.entry)
|
||||||
|
successor._store.async_load.return_value = payload
|
||||||
|
await successor.async_restore_model()
|
||||||
|
base = successor.learners['office']
|
||||||
|
base.count, base.on_count, base.off_count = 200, 100, 100
|
||||||
|
successor._update_counts()
|
||||||
|
self.c = successor
|
||||||
|
data = await self.sample()
|
||||||
|
info = data['windows']['Window']
|
||||||
|
self.assertEqual((info['state'], info['last_known_state'], info['forecast_assumption']),
|
||||||
|
('unknown', 'open', 'last_known_open'))
|
||||||
|
self.assertGreater(info['forecast_open_minutes'], 0)
|
||||||
|
self.assertEqual(data['zone_details']['office']['uncertain_windows'], ['Window'])
|
||||||
|
self.assertTrue(data['ventilation']['Office']['affected'])
|
||||||
|
self.assertEqual(data['sample_result'], 'first_sample') # nothing learned across the reload
|
||||||
|
|
||||||
async def change_during_optimization(self, change):
|
async def change_during_optimization(self, change):
|
||||||
async def worker(job):
|
async def worker(job):
|
||||||
output = job()
|
output = job()
|
||||||
@@ -1257,7 +1290,11 @@ class BoundaryTests(unittest.IsolatedAsyncioTestCase):
|
|||||||
with patch.object(coordinator, 'datetime') as clock:
|
with patch.object(coordinator, 'datetime') as clock:
|
||||||
clock.now.return_value = self.now
|
clock.now.return_value = self.now
|
||||||
older = asyncio.ensure_future(self.c._async_update_data())
|
older = asyncio.ensure_future(self.c._async_update_data())
|
||||||
await asyncio.sleep(0)
|
for _ in range(50): # until the older run is inside the optimizer (it may save first)
|
||||||
|
if getattr(self, '_first_started', False):
|
||||||
|
break
|
||||||
|
await asyncio.sleep(0)
|
||||||
|
self.assertTrue(self._first_started)
|
||||||
clock.now.return_value = self.now + timedelta(seconds=60)
|
clock.now.return_value = self.now + timedelta(seconds=60)
|
||||||
newer = await self.c._async_update_data()
|
newer = await self.c._async_update_data()
|
||||||
self.c.data = newer # what DataUpdateCoordinator publishes
|
self.c.data = newer # what DataUpdateCoordinator publishes
|
||||||
@@ -1292,8 +1329,24 @@ class BoundaryTests(unittest.IsolatedAsyncioTestCase):
|
|||||||
# Longer horizons are not due yet.
|
# Longer horizons are not due yet.
|
||||||
self.assertEqual(data['operational_forecast'][30]['samples'], 0)
|
self.assertEqual(data['operational_forecast'][30]['samples'], 0)
|
||||||
stored = self.c._payload('sensor.outside')['operational_evaluation']
|
stored = self.c._payload('sensor.outside')['operational_evaluation']
|
||||||
self.assertEqual(sum(sum(v.get('15', {}).values()) for v in stored['verdicts'].values()),
|
counts = {}
|
||||||
0 if follows else 1)
|
for by_horizon in stored['verdicts'].values():
|
||||||
|
for name, count in by_horizon.get('15', {}).items():
|
||||||
|
counts[name] = counts.get(name, 0) + count
|
||||||
|
self.assertEqual(counts, {'matched': 1} if follows else {'counterfactual': 1})
|
||||||
|
|
||||||
|
async def test_a_forecast_made_with_an_unknown_window_is_never_scored(self):
|
||||||
|
self.ready_office()
|
||||||
|
self.set_state('sensor.room', '23.5', unit_of_measurement='°C') # plan: off, as the relay
|
||||||
|
self.states['binary_sensor.office_window'].state = 'unavailable'
|
||||||
|
first = await self.sample()
|
||||||
|
self.assertEqual(first['status'], 'ready')
|
||||||
|
self.assertEqual(self.c._forecasts._pending[-1]['assumptions']['uncertain_inputs'], ['window: Window'])
|
||||||
|
for _ in range(3):
|
||||||
|
self.fresh('sensor.room', 'sensor.outside', 'sensor.ups_power')
|
||||||
|
data = await self.sample()
|
||||||
|
fifteen = data['operational_forecast'][15]
|
||||||
|
self.assertEqual((fifteen['samples'], fifteen['incomplete_reasons']['uncertain_inputs']), (0, 1))
|
||||||
|
|
||||||
async def test_unloading_during_save_or_optimization_publishes_nothing_new(self):
|
async def test_unloading_during_save_or_optimization_publishes_nothing_new(self):
|
||||||
self.ready_office()
|
self.ready_office()
|
||||||
@@ -1308,6 +1361,83 @@ class BoundaryTests(unittest.IsolatedAsyncioTestCase):
|
|||||||
self.assertIs(await self.sample(), previous)
|
self.assertIs(await self.sample(), previous)
|
||||||
worker.assert_not_called()
|
worker.assert_not_called()
|
||||||
|
|
||||||
|
async def test_unload_marks_shutdown_before_platforms_unload(self):
|
||||||
|
await integration.async_setup_entry(self.hass, self.entry)
|
||||||
|
c = self.hass.data['heating_mpc']['entry']
|
||||||
|
seen = []
|
||||||
|
async def unload_platforms(entry, platforms):
|
||||||
|
seen.append(c.closing)
|
||||||
|
return True
|
||||||
|
self.hass.config_entries.async_unload_platforms.side_effect = unload_platforms
|
||||||
|
self.assertTrue(await integration.async_unload_entry(self.hass, self.entry))
|
||||||
|
self.assertEqual(seen, [True])
|
||||||
|
# A failed unload leaves the entry loaded, so the coordinator carries on.
|
||||||
|
await integration.async_setup_entry(self.hass, self.entry)
|
||||||
|
c = self.hass.data['heating_mpc']['entry']
|
||||||
|
self.hass.config_entries.async_unload_platforms.side_effect = None
|
||||||
|
self.hass.config_entries.async_unload_platforms.return_value = False
|
||||||
|
self.assertFalse(await integration.async_unload_entry(self.hass, self.entry))
|
||||||
|
self.assertFalse(c.closing)
|
||||||
|
|
||||||
|
async def test_nothing_is_learned_saved_or_recorded_once_unloading_began(self):
|
||||||
|
self.ready_office()
|
||||||
|
await self.sample()
|
||||||
|
self.c._store.async_save.reset_mock() # the window's first reading is saved
|
||||||
|
shown = self.c.data = {'status': 'learning', 'proposal': None}
|
||||||
|
self.fresh('sensor.room', 'sensor.outside', 'sensor.ups_power')
|
||||||
|
self.c.async_begin_shutdown()
|
||||||
|
count = self.c.learners['office'].count
|
||||||
|
self.assertIs(await self.sample(), shown)
|
||||||
|
self.assertEqual(self.c.learners['office'].count, count)
|
||||||
|
self.c._store.async_save.assert_not_awaited()
|
||||||
|
self.assertFalse(await self.c._async_save('sensor.outside'))
|
||||||
|
# A relay callback queued before the listener was removed is ignored.
|
||||||
|
new = types.SimpleNamespace(state='on', last_changed=self.now)
|
||||||
|
self.c.async_relay_changed(types.SimpleNamespace(
|
||||||
|
data={'old_state': types.SimpleNamespace(state='off'), 'new_state': new}, time_fired=self.now))
|
||||||
|
self.assertFalse(self.c._relay_events)
|
||||||
|
|
||||||
|
async def test_failed_setup_shuts_the_coordinator_down(self):
|
||||||
|
self.hass.config_entries.async_forward_entry_setups.side_effect = RuntimeError('test')
|
||||||
|
with patch.object(coordinator.HeatingMPCCoordinator, 'async_shutdown', autospec=True) as shutdown:
|
||||||
|
with self.assertRaises(RuntimeError):
|
||||||
|
await integration.async_setup_entry(self.hass, self.entry)
|
||||||
|
shutdown.assert_awaited_once()
|
||||||
|
|
||||||
|
async def test_simultaneous_saves_never_overlap(self):
|
||||||
|
self.ready_office()
|
||||||
|
order = []
|
||||||
|
async def slow_save(payload):
|
||||||
|
order.append('start')
|
||||||
|
await asyncio.sleep(0)
|
||||||
|
await asyncio.sleep(0)
|
||||||
|
order.append('end')
|
||||||
|
self.c._store.async_save.side_effect = slow_save
|
||||||
|
results = await asyncio.gather(*(self.c._async_save('sensor.outside') for _ in range(3)))
|
||||||
|
self.assertEqual(results, [True, True, True])
|
||||||
|
self.assertEqual(order, ['start', 'end'] * 3)
|
||||||
|
|
||||||
|
async def test_a_save_waiting_for_a_deletion_does_not_recreate_the_model(self):
|
||||||
|
self.ready_office()
|
||||||
|
order, release = [], asyncio.Event()
|
||||||
|
store = Store()
|
||||||
|
async def slow_remove():
|
||||||
|
order.append('delete started')
|
||||||
|
await release.wait()
|
||||||
|
order.append('deleted')
|
||||||
|
store.async_remove.side_effect = slow_remove
|
||||||
|
with patch.object(integration, 'Store', return_value=store):
|
||||||
|
removal = asyncio.ensure_future(integration.async_remove_entry(self.hass, self.entry))
|
||||||
|
await asyncio.sleep(0)
|
||||||
|
saving = asyncio.ensure_future(self.c._async_save('sensor.outside'))
|
||||||
|
await asyncio.sleep(0)
|
||||||
|
self.assertEqual(order, ['delete started']) # the save is waiting for the lock
|
||||||
|
release.set()
|
||||||
|
await removal
|
||||||
|
self.assertFalse(await saving)
|
||||||
|
self.assertEqual(order, ['delete started', 'deleted'])
|
||||||
|
self.c._store.async_save.assert_not_awaited()
|
||||||
|
|
||||||
async def test_restore_waits_for_a_save_still_in_progress(self):
|
async def test_restore_waits_for_a_save_still_in_progress(self):
|
||||||
self.ready_office()
|
self.ready_office()
|
||||||
order = []
|
order = []
|
||||||
|
|||||||
@@ -65,7 +65,12 @@ class FreshnessTests(unittest.TestCase):
|
|||||||
def test_freshness_settings_translated(self):
|
def test_freshness_settings_translated(self):
|
||||||
for name in ('strings.json', 'translations/en.json', 'translations/ro.json'):
|
for name in ('strings.json', 'translations/en.json', 'translations/ro.json'):
|
||||||
data = json.loads((ROOT / name).read_text())
|
data = json.loads((ROOT / name).read_text())
|
||||||
for section, step in (('config', 'user'), ('options', 'init')):
|
# Configure opens a menu: settings, or resetting the learned model.
|
||||||
|
options = data['options']
|
||||||
|
self.assertEqual(set(options['step']['init']['menu_options']), {'settings', 'reset_model'})
|
||||||
|
self.assertTrue(options['step']['reset_model']['data']['confirm_reset'])
|
||||||
|
self.assertEqual(set(options['abort']), {'model_reset', 'model_not_reset'})
|
||||||
|
for section, step in (('config', 'user'), ('options', 'settings')):
|
||||||
labels = data[section]['step'][step]['data']
|
labels = data[section]['step'][step]['data']
|
||||||
for key in ('room_max_age_minutes', 'weather_max_age_minutes', 'room_stale_use_last_value'):
|
for key in ('room_max_age_minutes', 'weather_max_age_minutes', 'room_stale_use_last_value'):
|
||||||
self.assertTrue(labels[key])
|
self.assertTrue(labels[key])
|
||||||
|
|||||||
+214
-9
@@ -116,15 +116,22 @@ class EvaluationTests(unittest.TestCase):
|
|||||||
day: {'45': {'sensor.room': [1, 0.0, 0.1, 0.1]},
|
day: {'45': {'sensor.room': [1, 0.0, 0.1, 0.1]},
|
||||||
'15': {'sensor.a': [1, 0.0, -0.1, 0.1], 'sensor.b': [True, 0, 0, 0],
|
'15': {'sensor.a': [1, 0.0, -0.1, 0.1], 'sensor.b': [True, 0, 0, 0],
|
||||||
'sensor.c': [1, float('nan'), 0, 0], 'sensor.d': [2, 0.1, 0.2, 0.4],
|
'sensor.c': [1, float('nan'), 0, 0], 'sensor.d': [2, 0.1, 0.2, 0.4],
|
||||||
'sensor.e': [2, 0.1, 0.2, 0.4, 0.3], 'sensor.f': [2, 0.1, 0.2, 0.4, -1]}}}})
|
'sensor.e': [2, 0.1, 0.2, 0.4, 0.3], 'sensor.f': [2, 0.1, 0.2, 0.4, -1]}}},
|
||||||
|
'excluded': {day: {'15': {'sensor.d': [1, 2, 0.3], 'sensor.x': [1, -2, 0.3], 'sensor.y': [1, 2]},
|
||||||
|
'45': {'sensor.d': [1, 2, 0.3]}}}})
|
||||||
# A 0.3 bucket has no base-model column; its model was the base model then.
|
# A 0.3 bucket has no base-model column; its model was the base model then.
|
||||||
# Neither 0.3 nor 0.4 buckets have squared errors: they do not count for RMSE.
|
# Neither 0.3 nor 0.4 buckets have squared errors: they do not count for RMSE.
|
||||||
|
# 0.4 data has no excluded-span counts.
|
||||||
self.assertEqual(restored.as_dict(), {'days': {day: {'15': {
|
self.assertEqual(restored.as_dict(), {'days': {day: {'15': {
|
||||||
'sensor.d': [2, 0.1, 0.2, 0.4, 0.2, 0.0, 0], 'sensor.e': [2, 0.1, 0.2, 0.4, 0.3, 0.0, 0]}}}})
|
'sensor.d': [2, 0.1, 0.2, 0.4, 0.2, 0.0, 0], 'sensor.e': [2, 0.1, 0.2, 0.4, 0.3, 0.0, 0]}}},
|
||||||
|
'excluded': {day: {'15': {'sensor.d': [1, 2, 0.3]}}}})
|
||||||
self.assertIsNone(restored.summary(['sensor.d'])[15]['rmse'])
|
self.assertIsNone(restored.summary(['sensor.d'])[15]['rmse'])
|
||||||
|
self.assertEqual({k: restored.summary(['sensor.d'])[15][k] for k in ('incomplete', 'skipped',
|
||||||
|
'incomplete_base_model_mae')},
|
||||||
|
{'incomplete': 1, 'skipped': 2, 'incomplete_base_model_mae': 0.3})
|
||||||
for garbage in (None, [], {'days': []}):
|
for garbage in (None, [], {'days': []}):
|
||||||
restored.load(garbage)
|
restored.load(garbage)
|
||||||
self.assertEqual(restored.as_dict(), {'days': {}})
|
self.assertEqual(restored.as_dict(), {'days': {}, 'excluded': {}})
|
||||||
|
|
||||||
|
|
||||||
T0 = START.timestamp()
|
T0 = START.timestamp()
|
||||||
@@ -152,7 +159,7 @@ class OperationalForecastTests(unittest.TestCase):
|
|||||||
for horizon, actual in ((15, 19.9), (30, 19.8), (60, 19.6), (120, 19.2)):
|
for horizon, actual in ((15, 19.9), (30, 19.8), (60, 19.6), (120, 19.2)):
|
||||||
with self.subTest(horizon=horizon):
|
with self.subTest(horizon=horizon):
|
||||||
s = stats[horizon]
|
s = stats[horizon]
|
||||||
self.assertEqual((s['samples'], s['counterfactual'], s['unverifiable']), (1, 0, 0))
|
self.assertEqual((s['samples'], s['matched'], s['counterfactual'], s['incomplete']), (1, 1, 0, 0))
|
||||||
self.assertAlmostEqual(s['mae'], 0, places=3)
|
self.assertAlmostEqual(s['mae'], 0, places=3)
|
||||||
self.assertAlmostEqual(s['persistence_mae'], 20.0 - actual, places=3)
|
self.assertAlmostEqual(s['persistence_mae'], 20.0 - actual, places=3)
|
||||||
self.assertEqual(s['improvement_vs_persistence'], 1.0)
|
self.assertEqual(s['improvement_vs_persistence'], 1.0)
|
||||||
@@ -172,12 +179,14 @@ class OperationalForecastTests(unittest.TestCase):
|
|||||||
|
|
||||||
def test_unknown_relay_history_or_missing_observations(self):
|
def test_unknown_relay_history_or_missing_observations(self):
|
||||||
stats = run_ledger(self.PLAN_OFF, lambda k: False, lambda k: 20.0, self.TRAJECTORY, unknown_relay_at=2)
|
stats = run_ledger(self.PLAN_OFF, lambda k: False, lambda k: 20.0, self.TRAJECTORY, unknown_relay_at=2)
|
||||||
self.assertEqual((stats[15]['samples'], stats[15]['unverifiable'], stats[60]['unverifiable']), (0, 1, 1))
|
self.assertEqual((stats[15]['samples'], stats[15]['incomplete_reasons']['relay_history'],
|
||||||
|
stats[60]['incomplete']), (0, 1, 1))
|
||||||
missing = run_ledger(self.PLAN_OFF, lambda k: False, lambda k: None if k == 12 else 20.0, self.TRAJECTORY)
|
missing = run_ledger(self.PLAN_OFF, lambda k: False, lambda k: None if k == 12 else 20.0, self.TRAJECTORY)
|
||||||
self.assertEqual((missing[60]['samples'], missing[60]['missing'], missing[30]['samples']), (0, 1, 1))
|
self.assertEqual((missing[60]['samples'], missing[60]['incomplete_reasons']['reading'],
|
||||||
|
missing[30]['samples']), (0, 1, 1))
|
||||||
# No poll near the due time (polls 11 to 13 missing): the 60-minute point cannot be scored.
|
# No poll near the due time (polls 11 to 13 missing): the 60-minute point cannot be scored.
|
||||||
gap = run_ledger(self.PLAN_OFF, lambda k: False, lambda k: 20.0, self.TRAJECTORY, skip=(11, 12, 13))
|
gap = run_ledger(self.PLAN_OFF, lambda k: False, lambda k: 20.0, self.TRAJECTORY, skip=(11, 12, 13))
|
||||||
self.assertEqual((gap[60]['samples'], gap[60]['missing']), (0, 1))
|
self.assertEqual((gap[60]['samples'], gap[60]['incomplete_reasons']['no_poll']), (0, 1))
|
||||||
|
|
||||||
def test_due_time_alignment(self):
|
def test_due_time_alignment(self):
|
||||||
ledger = ev.OperationalForecastLedger()
|
ledger = ev.OperationalForecastLedger()
|
||||||
@@ -200,9 +209,205 @@ class OperationalForecastTests(unittest.TestCase):
|
|||||||
self.assertEqual(restored.summary(['r'])[15]['counterfactual'], 1)
|
self.assertEqual(restored.summary(['r'])[15]['counterfactual'], 1)
|
||||||
restored.forget('r')
|
restored.forget('r')
|
||||||
self.assertEqual(restored.summary(['r'])[15]['samples'], 0)
|
self.assertEqual(restored.summary(['r'])[15]['samples'], 0)
|
||||||
for garbage in (None, [], {'days': {'x': {}}, 'verdicts': {'2026-10-27': {'15': {'bogus': 3}}}}):
|
empty = {'version': 2, 'days': {}, 'approximate': {}, 'verdicts': {}}
|
||||||
self.assertEqual(ev.OperationalForecastLedger(garbage).as_dict(), {'days': {}, 'verdicts': {}})
|
for garbage in (None, [], {'version': 2, 'days': {'x': {}}, 'verdicts': {'2026-10-27': {'15': {'bogus': 3}}}}):
|
||||||
|
self.assertEqual(ev.OperationalForecastLedger(garbage).as_dict(), empty)
|
||||||
|
# 0.4.1 statistics accepted plans within 20 % per block: not comparable, dropped.
|
||||||
|
old = {**stats_ledger.as_dict(), 'version': None}
|
||||||
|
old.pop('version')
|
||||||
|
self.assertEqual(ev.OperationalForecastLedger(old).as_dict(), empty)
|
||||||
|
|
||||||
|
|
||||||
if __name__ == '__main__':
|
if __name__ == '__main__':
|
||||||
unittest.main()
|
unittest.main()
|
||||||
|
|
||||||
|
|
||||||
|
BASE = (0.004, 0.12, 0.0005)
|
||||||
|
|
||||||
|
|
||||||
|
def ventilated(observed, actual=None, wind=4.0, features=('vent_natural',), zones=('r',), internal=None,
|
||||||
|
extended=None, lag_uncertain_at=(), missing_end_at=()):
|
||||||
|
"""Zones cooled by real ventilation; `observed[k]` is what the window sensor said
|
||||||
|
for the interval ending at poll k (None: unknown). Returns the summary at 15 min."""
|
||||||
|
actual = actual if actual is not None else [1.0 if o is None else o for o in observed]
|
||||||
|
snapshot = {'base': BASE, 'uses_extended': extended is not None, 'extended': extended,
|
||||||
|
'ventilation': (list(features), (0.02,) * len(features))}
|
||||||
|
evaluator, temps = ev.ForecastEvaluator(0.1), {zone: 20.0 for zone in zones}
|
||||||
|
for k, seen in enumerate(observed):
|
||||||
|
if k:
|
||||||
|
temps = {zone: t + 0.004 * (5 - t) + 0.12 * 0.5 + 0.0005
|
||||||
|
+ 0.02 * actual[k] * (5 - t) * (1 + (wind or 0) * ('vent_wind' in features))
|
||||||
|
for zone, t in temps.items()}
|
||||||
|
vent = {'opening': seen, 'facing': 0.0, 'other_opening': 0.0, 'recovery': 0.0, 't_ref': None,
|
||||||
|
'wind': wind}
|
||||||
|
readings = {z: t for z, t in temps.items() if not (k in missing_end_at and z == zones[0])}
|
||||||
|
evaluator.observe(START + timedelta(seconds=300 * k), readings, 5.0, 0.5,
|
||||||
|
0.5 if k in lag_uncertain_at else 0.0, {z: BASE for z in zones},
|
||||||
|
models={z: snapshot for z in zones},
|
||||||
|
inputs={z: {'internal': internal, 'ventilation': vent if z == zones[0] else
|
||||||
|
{**vent, 'opening': actual[k]}} for z in zones})
|
||||||
|
return evaluator.summary(list(zones))[15]
|
||||||
|
|
||||||
|
|
||||||
|
class IncompleteInputTests(unittest.TestCase):
|
||||||
|
"""Unknown ventilation or extended inputs never become Model Error samples."""
|
||||||
|
|
||||||
|
def test_window_unknown_for_the_whole_span_is_incomplete(self):
|
||||||
|
stats = ventilated([0.0, None, None, None])
|
||||||
|
self.assertEqual((stats['samples'], stats['mae'], stats['incomplete']), (0, None, 1))
|
||||||
|
# The base model needs no window input: its error is still kept, apart.
|
||||||
|
self.assertGreater(stats['incomplete_base_model_mae'], 0.1)
|
||||||
|
|
||||||
|
def test_window_unknown_halfway_or_recovering_during_the_span(self):
|
||||||
|
self.assertEqual(ventilated([0.0, 1.0, None, 1.0])['incomplete'], 1) # unavailable halfway
|
||||||
|
stats = ventilated([0.0, None, 1.0, 1.0, 1.0, 1.0, 1.0]) # back after one poll
|
||||||
|
# Spans ending at polls 3-6: only the first contains the unknown interval.
|
||||||
|
self.assertEqual((stats['incomplete'], stats['samples']), (1, 3))
|
||||||
|
self.assertAlmostEqual(stats['mae'], 0, places=6) # the known spans are exact
|
||||||
|
|
||||||
|
def test_one_zone_unknown_leaves_the_other_zone_scored(self):
|
||||||
|
stats = ventilated([0.0, None, None, None], zones=('a', 'b'))
|
||||||
|
self.assertEqual((stats['rooms']['a']['incomplete'], stats['rooms']['a']['samples']), (1, 0))
|
||||||
|
self.assertEqual((stats['rooms']['b']['incomplete'], stats['rooms']['b']['samples']), (0, 1))
|
||||||
|
|
||||||
|
def test_missing_wind_matters_only_while_a_window_is_open(self):
|
||||||
|
features = ('vent_natural', 'vent_wind')
|
||||||
|
open_span = ventilated([0.0, 1.0, 1.0, 1.0], wind=None, features=features)
|
||||||
|
closed_span = ventilated([0.0, 0.0, 0.0, 0.0], wind=None, features=features)
|
||||||
|
self.assertEqual((open_span['incomplete'], open_span['samples']), (1, 0))
|
||||||
|
self.assertEqual((closed_span['incomplete'], closed_span['samples']), (0, 1))
|
||||||
|
self.assertEqual(ventilated([0.0, 1.0, 1.0, 1.0], features=features)['samples'], 1)
|
||||||
|
|
||||||
|
def test_unknown_electrical_heat_makes_the_extended_replay_incomplete(self):
|
||||||
|
extended = (['loss', 'heat', 'bias', 'internal'], BASE + (0.15,))
|
||||||
|
stats = ventilated([0.0] * 4, extended=extended, internal=None)
|
||||||
|
self.assertEqual((stats['incomplete'], stats['samples']), (1, 0))
|
||||||
|
self.assertIsNotNone(stats['incomplete_base_model_mae'])
|
||||||
|
self.assertEqual(ventilated([0.0] * 4, extended=extended, internal=0.0)['samples'], 1)
|
||||||
|
|
||||||
|
def test_a_neighbour_with_unknown_inputs_spoils_a_coupled_zone(self):
|
||||||
|
coupled = {'base': BASE, 'uses_extended': True, 'ventilation': None,
|
||||||
|
'extended': (['loss', 'heat', 'bias', 'couple:a'], BASE + (0.01,))}
|
||||||
|
plain = {'base': BASE, 'uses_extended': False, 'extended': None,
|
||||||
|
'ventilation': (['vent_natural'], (0.02,))}
|
||||||
|
origin = {'temps': {'a': 20.0, 'b': 20.0}, 'outside': 5.0, 'models': {'a': plain, 'b': coupled}}
|
||||||
|
path = [origin] + [{'time': START + timedelta(seconds=300 * k), 'lag': 0.5,
|
||||||
|
'inputs': {'a': {'ventilation': {'opening': None}}, 'b': {}}} for k in range(4)]
|
||||||
|
origin['time'] = START - timedelta(seconds=300)
|
||||||
|
_, incomplete = ev.replay_zones(origin, path)
|
||||||
|
self.assertEqual(incomplete, {'a', 'b'})
|
||||||
|
path = [origin] + [{**poll, 'inputs': {'a': {'ventilation': {'opening': 0.0}}, 'b': {}}} for poll in path[1:]]
|
||||||
|
self.assertEqual(ev.replay_zones(origin, path)[1], set())
|
||||||
|
|
||||||
|
def test_skipped_spans_are_counted_and_samples_stay_correct(self):
|
||||||
|
stats = ventilated([0.0] * 9, lag_uncertain_at=(4,))
|
||||||
|
# Spans ending at 3..8: those crossing poll 4 (ending 4, 5, 6) are skipped.
|
||||||
|
self.assertEqual((stats['samples'], stats['skipped'], stats['incomplete']), (3, 3, 0))
|
||||||
|
stats = ventilated([0.0] * 5, missing_end_at=(4,), zones=('a', 'b'))
|
||||||
|
self.assertEqual((stats['rooms']['a']['samples'], stats['rooms']['a']['skipped']), (1, 1))
|
||||||
|
self.assertEqual(stats['rooms']['b']['samples'], 2)
|
||||||
|
|
||||||
|
|
||||||
|
def ledger_with(plan, on_seconds, temps=None, uncertain=None, polls=None):
|
||||||
|
"""One forecast at T0; `on_seconds[k-1]` is the relay's on-time in the interval ending
|
||||||
|
at poll k (None: unknown). Returns the ledger."""
|
||||||
|
ledger = ev.OperationalForecastLedger()
|
||||||
|
trajectory = [20.0 + 0.1 * (b + 1) for b in range(len(plan))]
|
||||||
|
ledger.observe(T0, {'r': 20.0}, None, None)
|
||||||
|
ledger.record(T0, {'r': 20.0}, {'r': trajectory}, plan, 'rev1',
|
||||||
|
{'strategy': 'mpc_plan', 'uncertain_inputs': uncertain or []})
|
||||||
|
for k in range(1, (polls or len(on_seconds)) + 1):
|
||||||
|
on = on_seconds[k - 1] if k - 1 < len(on_seconds) else 0.0
|
||||||
|
ledger.observe(T0 + 300 * k, {'r': (temps or (lambda k: 20.0 + k / 30))(k)}, on, 300.0)
|
||||||
|
return ledger
|
||||||
|
|
||||||
|
|
||||||
|
def verdict(ledger, horizon):
|
||||||
|
stats = ledger.summary(['r'])[horizon]
|
||||||
|
for name in ('matched', 'approximate', 'counterfactual'):
|
||||||
|
if stats[name]:
|
||||||
|
return name
|
||||||
|
return next(f'incomplete:{r}' for r, n in stats['incomplete_reasons'].items() if n)
|
||||||
|
|
||||||
|
|
||||||
|
class PlanMatchTests(unittest.TestCase):
|
||||||
|
"""Forecasts are compared with reality only when the relay did what their plan assumed."""
|
||||||
|
|
||||||
|
ON, OFF = 300.0, 0.0
|
||||||
|
|
||||||
|
def test_exact_match_is_scored(self):
|
||||||
|
ledger = ledger_with([1, 1, 0, 0], [self.ON] * 6 + [self.OFF] * 6)
|
||||||
|
self.assertEqual([verdict(ledger, h) for h in (15, 30, 60)], ['matched'] * 3)
|
||||||
|
self.assertEqual(ledger.deviation({'time': T0, 'plan': [1, 1, 0, 0]}, 60), (0.0, 0.0))
|
||||||
|
stats = ledger.summary(['r'])[60]
|
||||||
|
self.assertEqual((stats['samples'], stats['approximate_stats']['samples']), (1, 0))
|
||||||
|
|
||||||
|
def test_small_transition_jitter_still_matches(self):
|
||||||
|
# Planned off at 30 minutes; the relay switched off 40 s later.
|
||||||
|
ledger = ledger_with([1, 1, 0, 0], [self.ON] * 6 + [40.0] + [self.OFF] * 5)
|
||||||
|
self.assertEqual(ledger.deviation({'time': T0, 'plan': [1, 1, 0, 0]}, 60), (40.0, 0.0))
|
||||||
|
self.assertEqual(verdict(ledger, 60), 'matched')
|
||||||
|
|
||||||
|
def test_twelve_of_fifteen_planned_minutes_is_only_approximate(self):
|
||||||
|
ledger = ledger_with([1, 0], [self.ON, self.ON, 120.0, self.OFF, self.OFF, self.OFF])
|
||||||
|
self.assertEqual(verdict(ledger, 15), 'approximate')
|
||||||
|
stats = ledger.summary(['r'])[15]
|
||||||
|
# Not in the main figures; kept apart.
|
||||||
|
self.assertEqual((stats['samples'], stats['mae'], stats['approximate']), (0, None, 1))
|
||||||
|
self.assertEqual(stats['approximate_stats']['samples'], 1)
|
||||||
|
|
||||||
|
def test_same_on_time_at_another_time_is_counterfactual(self):
|
||||||
|
# 15 minutes on as planned, but in the second block instead of the first.
|
||||||
|
ledger = ledger_with([1, 0, 0, 0], [self.OFF] * 3 + [self.ON] * 3 + [self.OFF] * 6)
|
||||||
|
self.assertEqual(ledger.deviation({'time': T0, 'plan': [1, 0, 0, 0]}, 30), (1800.0, 0.0))
|
||||||
|
self.assertEqual([verdict(ledger, h) for h in (15, 30, 60)], ['counterfactual'] * 3)
|
||||||
|
|
||||||
|
def test_early_shutdown_and_unexpected_activation(self):
|
||||||
|
early = ledger_with([1, 1, 1, 1], [self.ON] * 4 + [self.OFF] * 8) # off after 20 of 60 minutes
|
||||||
|
self.assertEqual([verdict(early, h) for h in (15, 30, 60)], ['matched', 'counterfactual', 'counterfactual'])
|
||||||
|
brief = ledger_with([0, 0, 0, 0], [self.OFF] * 4 + [240.0] + [self.OFF] * 7) # 4 minutes on
|
||||||
|
self.assertEqual([verdict(brief, h) for h in (15, 30)], ['matched', 'approximate'])
|
||||||
|
long = ledger_with([0, 0, 0, 0], [self.OFF] * 4 + [self.ON, self.ON] + [self.OFF] * 6)
|
||||||
|
self.assertEqual(verdict(long, 30), 'counterfactual')
|
||||||
|
|
||||||
|
def test_several_cycles_followed_closely_match(self):
|
||||||
|
plan = [1, 0, 1, 0, 1, 0, 1, 0]
|
||||||
|
relay = []
|
||||||
|
for block in plan:
|
||||||
|
relay += [self.ON] * 3 if block else [30.0, self.OFF, self.OFF] # each off 30 s late
|
||||||
|
ledger = ledger_with(plan, relay)
|
||||||
|
self.assertEqual(ledger.deviation({'time': T0, 'plan': plan}, 120)[0], 120.0) # four transitions
|
||||||
|
self.assertEqual(verdict(ledger, 60), 'matched')
|
||||||
|
self.assertEqual(verdict(ledger, 120), 'matched')
|
||||||
|
|
||||||
|
def test_missing_relay_history_is_incomplete_not_matched(self):
|
||||||
|
ledger = ledger_with([0, 0, 0, 0], [self.OFF, None] + [self.OFF] * 10)
|
||||||
|
self.assertEqual([verdict(ledger, h) for h in (15, 60)], ['incomplete:relay_history'] * 2)
|
||||||
|
# A plan already contradicted is counterfactual whatever the gap hides.
|
||||||
|
ledger = ledger_with([1, 1, 1, 1], [None] + [self.OFF] * 11)
|
||||||
|
self.assertEqual(verdict(ledger, 30), 'counterfactual')
|
||||||
|
|
||||||
|
def test_thermostat_disagreeing_is_counterfactual_never_an_error(self):
|
||||||
|
ledger = ledger_with([1] * 4, [self.OFF] * 12, temps=lambda k: 20.0)
|
||||||
|
stats = ledger.summary(['r'])
|
||||||
|
self.assertEqual([stats[h]['counterfactual'] for h in (15, 30, 60)], [1, 1, 1])
|
||||||
|
self.assertEqual([stats[h]['samples'] for h in (15, 30, 60)], [0, 0, 0])
|
||||||
|
|
||||||
|
def test_forecasts_made_with_assumed_inputs_are_incomplete(self):
|
||||||
|
ledger = ledger_with([0] * 4, [self.OFF] * 12, uncertain=['window: Bedroom'])
|
||||||
|
self.assertEqual(verdict(ledger, 15), 'incomplete:uncertain_inputs')
|
||||||
|
self.assertEqual(ledger.summary(['r'])[15]['samples'], 0)
|
||||||
|
|
||||||
|
def test_counts_over_several_forecasts(self):
|
||||||
|
ledger = ev.OperationalForecastLedger()
|
||||||
|
ledger.observe(T0, {'r': 20.0}, None, None)
|
||||||
|
for k, plan in enumerate(([0, 0], [1, 1], [0, 0])): # one forecast per poll, relay always off
|
||||||
|
ledger.record(T0 + 300 * k, {'r': 20.0}, {'r': [20.0, 20.0]}, plan, 'rev1', {})
|
||||||
|
ledger.observe(T0 + 300 * (k + 1), {'r': 20.0}, 0.0, 300.0)
|
||||||
|
for k in range(4, 12):
|
||||||
|
ledger.observe(T0 + 300 * k, {'r': 20.0}, 0.0, 300.0)
|
||||||
|
stats = ledger.summary(['r'])[15]
|
||||||
|
self.assertEqual((stats['matched'], stats['counterfactual'], stats['samples'], stats['incomplete']),
|
||||||
|
(2, 1, 2, 0))
|
||||||
|
restored = ev.OperationalForecastLedger(ledger.as_dict())
|
||||||
|
self.assertEqual(restored.summary(['r']), ledger.summary(['r']))
|
||||||
@@ -253,3 +253,165 @@ class UnknownStateTests(unittest.TestCase):
|
|||||||
self.assertTrue(tracker.events[-1]['closed_confirmed'])
|
self.assertTrue(tracker.events[-1]['closed_confirmed'])
|
||||||
inputs = tracker.zone_inputs('bed', WEATHER)
|
inputs = tracker.zone_inputs('bed', WEATHER)
|
||||||
self.assertEqual((inputs['known'], inputs['unknown_windows']), (False, ['B']))
|
self.assertEqual((inputs['known'], inputs['unknown_windows']), (False, ['B']))
|
||||||
|
|
||||||
|
|
||||||
|
class StartupAndRestartTests(unittest.TestCase):
|
||||||
|
"""Unknown at startup is not closed; saved state comes back as unknown, never live."""
|
||||||
|
|
||||||
|
WINDOWS = wm.build_windows([(window('w'), None)], ['bed'])
|
||||||
|
|
||||||
|
def setUp(self):
|
||||||
|
self.t = 2_000_000.0
|
||||||
|
|
||||||
|
def poll(self, tracker, state, changed_ago=100_000, temp=20.0):
|
||||||
|
tracker.update(self.t, {'w': (state, self.t - changed_ago)}, {'bed': temp}, WEATHER)
|
||||||
|
result = (tracker.zone_inputs('bed', WEATHER), tracker.forecast('bed', 6, 15, WEATHER, self.t),
|
||||||
|
tracker.assumption(self.WINDOWS[0], 15, self.t))
|
||||||
|
self.t += 300
|
||||||
|
return result
|
||||||
|
|
||||||
|
def saved_with(self, *states):
|
||||||
|
tracker = wm.VentilationTracker(self.WINDOWS)
|
||||||
|
for state in states:
|
||||||
|
self.poll(tracker, state, changed_ago=200 if state == 'on' else 100_000, temp=21.0)
|
||||||
|
return tracker, tracker.as_dict()
|
||||||
|
|
||||||
|
def test_unavailable_from_startup_is_unobserved_not_closed(self):
|
||||||
|
tracker = wm.VentilationTracker(self.WINDOWS)
|
||||||
|
for _ in range(3):
|
||||||
|
inputs, forecast, assumed = self.poll(tracker, 'unavailable')
|
||||||
|
self.assertEqual(assumed, {'basis': 'unobserved', 'open_minutes': 0.0, 'uncertain': True})
|
||||||
|
self.assertEqual((inputs['known'], inputs['affected'], inputs['unknown_windows']), (False, True, ['W']))
|
||||||
|
self.assertEqual(forecast[0]['opening'], 0.0) # no ventilation assumed, but flagged uncertain
|
||||||
|
self.assertEqual((len(tracker.events), tracker._open_event), (0, {}))
|
||||||
|
|
||||||
|
def test_first_reading_open_is_an_unconfirmed_opening(self):
|
||||||
|
tracker = wm.VentilationTracker(self.WINDOWS)
|
||||||
|
self.poll(tracker, 'on', changed_ago=30) # we never saw it closed
|
||||||
|
event = tracker._open_event['bed']
|
||||||
|
self.assertEqual((event['opened_confirmed'], event['temperature_before']), (False, None))
|
||||||
|
self.poll(tracker, 'on')
|
||||||
|
self.poll(tracker, 'off', changed_ago=60)
|
||||||
|
event = tracker.events[-1]
|
||||||
|
self.assertTrue(event['closed_confirmed'])
|
||||||
|
# Opened at some unknown time before it was first seen: a minimum duration only.
|
||||||
|
self.assertEqual((event['duration_minutes'], event['duration_max_minutes']), (None, None))
|
||||||
|
self.assertEqual(event['duration_min_minutes'], 9.5)
|
||||||
|
|
||||||
|
def test_restart_with_saved_open_window_and_sensor_unavailable(self):
|
||||||
|
before, saved = self.saved_with('off', 'off', 'on', 'on')
|
||||||
|
self.assertEqual(saved['state']['windows']['w']['last_known'], True)
|
||||||
|
self.assertIn('bed', saved['state']['zones'])
|
||||||
|
self.t += 120 # restart takes two minutes
|
||||||
|
tracker = wm.VentilationTracker(self.WINDOWS, saved=saved)
|
||||||
|
self.assertEqual(tracker.window_state, {}) # nothing is live before the first reading
|
||||||
|
inputs, forecast, assumed = self.poll(tracker, 'unavailable')
|
||||||
|
state = tracker.window_state['w']
|
||||||
|
self.assertEqual((state['open'], state['last_known_open']), (None, True))
|
||||||
|
self.assertEqual(state['unknown_since'], saved['state']['windows']['w']['seen'])
|
||||||
|
# Assumed open only until 15 minutes after it was last seen (7 minutes ago).
|
||||||
|
self.assertEqual((assumed['basis'], assumed['open_minutes'], assumed['uncertain']),
|
||||||
|
('last_known_open', 8.0, True))
|
||||||
|
self.assertEqual([s['opening'] for s in forecast[:3]], [1.0, 1.0, 0.0])
|
||||||
|
self.assertEqual((inputs['known'], inputs['affected']), (False, True))
|
||||||
|
# The event from before the restart continues; the downtime counts as unknown.
|
||||||
|
self.assertEqual(len(tracker.events), 0)
|
||||||
|
event = tracker._open_event['bed']
|
||||||
|
self.assertEqual((event['opened'], event['opened_confirmed']), (before._open_event['bed']['opened'], True))
|
||||||
|
self.assertEqual(event['sensor_unknown_minutes'], 12.0) # 7 minutes down + one unknown poll
|
||||||
|
# Not assumed open forever: past the bound the forecast assumes it closed.
|
||||||
|
for _ in range(3):
|
||||||
|
_, forecast, assumed = self.poll(tracker, 'unavailable')
|
||||||
|
self.assertEqual((assumed['open_minutes'], forecast[0]['opening']), (0.0, 0.0))
|
||||||
|
self.assertGreater(forecast[0]['recovery'], 0.0)
|
||||||
|
# Back as closed: closed at an unknown time, so an unconfirmed close and recovery.
|
||||||
|
inputs, _, _ = self.poll(tracker, 'off', changed_ago=30)
|
||||||
|
event = tracker.events[-1]
|
||||||
|
self.assertEqual((event['closed_confirmed'], event['duration_minutes']), (False, None))
|
||||||
|
self.assertEqual(event['duration_min_minutes'], 8.3) # opened 3.3 min before a poll, seen open 5 min later
|
||||||
|
self.assertEqual((inputs['known'], inputs['affected']), (False, True))
|
||||||
|
|
||||||
|
def test_restart_with_saved_closed_window(self):
|
||||||
|
_, saved = self.saved_with('off', 'off')
|
||||||
|
tracker = wm.VentilationTracker(self.WINDOWS, saved=saved)
|
||||||
|
inputs, forecast, assumed = self.poll(tracker, 'unavailable')
|
||||||
|
self.assertEqual((assumed['basis'], assumed['uncertain'], forecast[0]['opening']),
|
||||||
|
('last_known_closed', True, 0.0))
|
||||||
|
self.assertEqual((inputs['known'], len(tracker.events)), (False, 0))
|
||||||
|
# It reports open: opened during the unknown period, so not a confirmed opening.
|
||||||
|
self.poll(tracker, 'on', changed_ago=60)
|
||||||
|
self.assertFalse(tracker._open_event['bed']['opened_confirmed'])
|
||||||
|
|
||||||
|
def test_restart_with_the_same_state_again_is_a_known_state(self):
|
||||||
|
_, saved = self.saved_with('off', 'off')
|
||||||
|
tracker = wm.VentilationTracker(self.WINDOWS, saved=saved)
|
||||||
|
self.poll(tracker, 'off') # first interval after restart: unknown
|
||||||
|
inputs, _, assumed = self.poll(tracker, 'off')
|
||||||
|
self.assertEqual((inputs['known'], inputs['affected'], assumed['basis']), (True, False, 'observed'))
|
||||||
|
|
||||||
|
def test_saved_state_of_another_sensor_or_polarity_is_ignored(self):
|
||||||
|
_, saved = self.saved_with('off', 'off', 'on', 'on')
|
||||||
|
for changed in ({'sensor': 'binary_sensor.other'}, {'inverted': True}):
|
||||||
|
with self.subTest(changed=changed):
|
||||||
|
windows = wm.build_windows([(window('w', **changed), None)], ['bed'])
|
||||||
|
tracker = wm.VentilationTracker(windows, saved=saved)
|
||||||
|
self.assertEqual((tracker._memory, tracker._open_event), ({}, {}))
|
||||||
|
corrupt = {'events': [], 'state': {'windows': {'w': {'sensor': 'binary_sensor.w', 'last_known': 'yes',
|
||||||
|
'seen': 'x'}}, 'zones': {'bed': 5}}}
|
||||||
|
self.assertEqual(wm.VentilationTracker(self.WINDOWS, saved=corrupt)._memory, {})
|
||||||
|
|
||||||
|
def test_mixed_windows_each_get_their_own_assumption(self):
|
||||||
|
windows = wm.build_windows([(window('a'), None), (window('b', opening_factor=0.5), None),
|
||||||
|
(window('c', zone='living'), None)], ['bed', 'living'])
|
||||||
|
tracker = wm.VentilationTracker(windows)
|
||||||
|
t = 0.0
|
||||||
|
for a, b, c in (('off', 'off', 'off'), ('off', 'on', 'off'), ('on', 'on', 'unavailable'),
|
||||||
|
('on', 'unavailable', 'unavailable'), ('on', 'unavailable', 'unavailable'),
|
||||||
|
('on', 'unavailable', 'unavailable'), ('on', 'unavailable', 'unavailable')):
|
||||||
|
t += 300
|
||||||
|
tracker.update(t, {'a': (a, t - 200), 'b': (b, t - 200), 'c': (c, t - 200)},
|
||||||
|
{'bed': 20.0, 'living': 21.0}, WEATHER)
|
||||||
|
# b was last seen open 20 minutes ago: no longer assumed open; a is observed open.
|
||||||
|
self.assertEqual(tracker.assumption(windows[1], 15, t)['open_minutes'], 0.0)
|
||||||
|
self.assertEqual(tracker.forecast('bed', 4, 15, WEATHER, t)[0]['opening'], 1.0)
|
||||||
|
self.assertEqual(tracker.assumption(windows[2], 15, t)['basis'], 'last_known_closed')
|
||||||
|
# An unknown window in one zone does not stop another zone with known windows.
|
||||||
|
tracker2 = wm.VentilationTracker(windows)
|
||||||
|
for t in (300.0, 600.0):
|
||||||
|
tracker2.update(t, {'a': ('off', 0), 'b': ('off', 0), 'c': ('unavailable', 0)},
|
||||||
|
{'bed': 20.0, 'living': 21.0}, WEATHER)
|
||||||
|
self.assertEqual((tracker2.zone_inputs('bed', WEATHER)['known'],
|
||||||
|
tracker2.zone_inputs('living', WEATHER)['known']), (True, False))
|
||||||
|
|
||||||
|
def test_changes_worth_saving_are_flagged(self):
|
||||||
|
tracker = wm.VentilationTracker(self.WINDOWS)
|
||||||
|
self.poll(tracker, 'off')
|
||||||
|
self.assertTrue(tracker.changed) # first reading
|
||||||
|
tracker.changed = False
|
||||||
|
self.poll(tracker, 'off')
|
||||||
|
self.assertFalse(tracker.changed) # nothing new
|
||||||
|
self.poll(tracker, 'unavailable')
|
||||||
|
self.assertTrue(tracker.changed)
|
||||||
|
|
||||||
|
|
||||||
|
class StrictInputTests(unittest.TestCase):
|
||||||
|
"""Weather inputs of an open window are required, never assumed, outside forecasts."""
|
||||||
|
|
||||||
|
def test_unknown_direction_or_other_zone_while_open(self):
|
||||||
|
windows = wm.build_windows([(window('a'), None), (window('c', zone='living'), None)], ['bed', 'living'])
|
||||||
|
tracker = wm.VentilationTracker(windows)
|
||||||
|
calm = {**WEATHER, 'wind_direction': None}
|
||||||
|
for t, a, c in ((300.0, 'off', 'off'), (600.0, 'on', 'unavailable'), (900.0, 'on', 'unavailable')):
|
||||||
|
tracker.update(t, {'a': (a, 450.0), 'c': (c, 450.0)}, {'bed': 20.0, 'living': 21.0}, calm)
|
||||||
|
bed = tracker.zone_inputs('bed', calm)
|
||||||
|
self.assertEqual((bed['opening'], bed['facing'], bed['other_opening']), (1.0, None, None))
|
||||||
|
features = ['vent_natural', 'vent_wind', 'vent_wind_dir', 'vent_cross']
|
||||||
|
self.assertFalse(wm.ventilation_known(features, bed))
|
||||||
|
self.assertTrue(wm.ventilation_known(['vent_natural', 'vent_wind'], bed))
|
||||||
|
self.assertIsNone(wm.ventilation_vector(features, 20.0, 2.0, bed)) # learning refuses
|
||||||
|
self.assertIsNotNone(wm.ventilation_vector_for_prediction(features, 20.0, 2.0, bed))
|
||||||
|
# Closed, every opening term is 0 whatever the weather: nothing is missing.
|
||||||
|
closed = {**bed, 'opening': 0.0, 'facing': 0.0, 'wind': None, 'other_opening': None}
|
||||||
|
self.assertEqual(wm.ventilation_vector(features, 20.0, 2.0, closed), [0.0, 0.0, 0.0, 0.0])
|
||||||
|
self.assertTrue(wm.ventilation_known(features, closed))
|
||||||
|
self.assertTrue(wm.ventilation_known(features, None)) # no windows
|
||||||
Reference in new issue
Block a user