Document the final stability fixes and the model reset
README: window states and forecast assumptions, Model Error incomplete and skipped spans, Forecast Error classes and why 2 and 5 minutes, and the lifecycle tests. docs/MODEL.md: strict inputs, unknown-window assumptions, the plan-matching table. CHANGELOG gains an Unreleased section (version unchanged); ROADMAP and AUDIT record the findings, root causes, evidence and remaining limitations. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
0b82a8e26c
commit
c4035951df
5 files changed
+275
-47
No files matched your search
@@ -397,3 +397,73 @@ Remaining risks:
|
||||
ruff, 159 offline tests, 28 HA Core tests; the offline tests also pass on Python
|
||||
3.9.6. `sh scripts/check.sh` additionally validates packaging, translations and
|
||||
the changelog.
|
||||
|
||||
## Final stability fixes after 0.4.1 — 2026-10-27
|
||||
|
||||
Scope: four issues from an independent review of 0.4.1, plus a user-requested
|
||||
*Reset learned model* action. Repository code and tests only (HA Core 2025.12.0
|
||||
and 2026.2.3 harness); no HAOS, device or deployment testing. Still no actuator
|
||||
code; the static and runtime service-call guards pass. Version unchanged (0.4.1).
|
||||
|
||||
Confirmed issues, root causes and fixes:
|
||||
|
||||
1. **Model Error scored spans with unknown ventilation inputs** (2cfba02).
|
||||
Reproduced: a window unknown for a whole 15-minute span gave a Model Error
|
||||
sample (MAE 0.76 °C). Root cause: the replay used the prediction path, which
|
||||
drops the ventilation term when the opening is unknown, i.e. assumes closed.
|
||||
Fix: per zone and step, `inputs_complete` checks every input the zone's model
|
||||
used (window opening and confirmed recovery; wind, direction and other zones'
|
||||
windows while a window is open; electrical heat and neighbours for the extended
|
||||
layer, including the neighbour's own completeness). Incomplete spans are
|
||||
counted, not scored, with the base model's error kept apart; unreplayable
|
||||
spans are counted as skipped. Learning also stopped substituting a zero facing
|
||||
term or zero cross-ventilation input for unknown values.
|
||||
2. **Window unknown at startup treated as closed** (13cb972). Reproduced
|
||||
(0 % assumed opening). Root cause: last known states lived only in memory, and
|
||||
"never observed" fell through to the closed case; a first reading of an open
|
||||
window was also logged as a confirmed opening. Fix: confirmation only for
|
||||
witnessed transitions; last known states, open events and recovery saved (also
|
||||
on every window state change) and restored as unknown; per-window forecast
|
||||
assumptions (observed / last known open, bounded by the open-window time after
|
||||
it was last seen / last known closed / unobserved), exposed in Status and on
|
||||
zone forecasts; durations as ranges when an end was not seen. Chosen over
|
||||
open/closed scenario pairs to avoid a second plan simulation per poll.
|
||||
3. **Lifecycle and storage races** (28ff953). Confirmed by reading HA's unload
|
||||
order: the coordinator's own shutdown runs only after platforms unload, and the
|
||||
save checked for shutdown before waiting for the storage lock (check-then-act).
|
||||
Found during testing: unloading cancels a running scheduled update, which
|
||||
released the lock while the file write continued in a worker thread, so a
|
||||
deletion could run first and be undone. Fix: closing mark at the start of
|
||||
unload (restored if unload fails); checks after acquiring the lock, including a
|
||||
removed-model mark set under the lock by removal; the write runs as a shielded,
|
||||
HA-tracked task that holds the lock until it finishes; failed setup shuts the
|
||||
coordinator down; queued relay callbacks are ignored once closing. Mutation
|
||||
checks: removing the locked check or the shield makes the deletion tests fail;
|
||||
removing the early closing mark fails the ordering test (the HA-level unload
|
||||
test cannot distinguish it, as HA's own shutdown also lands before release).
|
||||
4. **Forecast Error accepted materially different relay behaviour** (0b82a8e).
|
||||
Reproduced: 12 of 15 planned minutes was scored. Root cause: a per-block
|
||||
on-share tolerance of 20 %, blind to timing within a block. Fix: deviation =
|
||||
time in another state than planned, block by block from the relay's switch
|
||||
times, with unobserved time as possible deviation; matched ≤ 2 min (scored),
|
||||
approximate ≤ 5 min (separate), counterfactual > 5 min observed, otherwise
|
||||
incomplete with a reason; forecasts made with assumed inputs are incomplete.
|
||||
Ledger statistics are versioned; 0.4.1 figures are dropped.
|
||||
|
||||
Added: *Reset learned model* (49565d6): unload, strip learned keys from the
|
||||
stored model under the lock, set up again; configuration, heat-source baselines
|
||||
and window history kept.
|
||||
|
||||
Tests added (by area): Model Error incomplete/skipped (7), plan matching (10),
|
||||
window startup/restart/mixed/strict inputs (9), coordinator window reload and
|
||||
uncertain forecast (2), lifecycle offline (5) and in HA (4), reset flow (1).
|
||||
|
||||
Remaining limitations: unknown-but-last-closed and never-seen windows still
|
||||
produce a closed-window forecast (flagged, never scored); the 2 and 5 minute
|
||||
limits rest on typical learned heat coefficients, not a per-forecast calculation;
|
||||
spans lost when the replay window restarts are not counted; saved window state can
|
||||
be up to one save old, which only makes the restored unknown period longer.
|
||||
|
||||
**Verification.** `sh scripts/check.sh` passes in both environments (pip check,
|
||||
ruff, 192 offline tests, 33 HA Core tests, compilation, package/translation/
|
||||
changelog checks, diff whitespace); the offline tests also pass on Python 3.9.6.
|
||||
@@ -2,6 +2,78 @@
|
||||
|
||||
Engineering findings and test evidence behind each change are in [AUDIT.md](AUDIT.md).
|
||||
|
||||
## Unreleased — final stability fixes after 0.4.1
|
||||
|
||||
Four correctness fixes from an independent review of 0.4.1, and a **Reset learned
|
||||
model** action you asked for. The version stays 0.4.1 until released. Still
|
||||
**observation-only**: no service calls; the boiler and thermostat are untouched.
|
||||
Tested in the Home Assistant Core harness with HA 2025.12.0 and 2026.2.3; not on
|
||||
HAOS.
|
||||
|
||||
### Upgrade notes
|
||||
|
||||
- No reconfiguration needed. Zones, sensor and comfort weights, weather sources,
|
||||
heat sources, windows, learned coefficients and fingerprints carry over; entity
|
||||
IDs are unchanged.
|
||||
- **Configure now opens a menu**: *Settings* (the previous form) or *Reset learned
|
||||
model*.
|
||||
- **Forecast Error statistics restart.** 0.4.1 accepted plans the relay followed
|
||||
within 20 % per block; those figures are not comparable with the new rule and
|
||||
are dropped on the first start (they covered at most a day).
|
||||
- Model Error statistics carry over. Those from 0.4.0/0.4.1 may include a few
|
||||
spans with unknown window states; they age out within 7 days.
|
||||
|
||||
### Added
|
||||
|
||||
- **Reset learned model** (Configure, with a confirmation): forgets every zone's
|
||||
learned coefficients, models kept for removed zones, and the Model Error and
|
||||
Forecast Error statistics; keeps the configuration, heat-source baselines and the
|
||||
window event log, and logs the reset in `model_changes`.
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Model Error no longer scores spans with unknown inputs.** A window unknown
|
||||
during a span was replayed as closed and scored. Spans where a window state, an
|
||||
open window's weather input, or the extended layer's electrical heat or
|
||||
neighbour was unknown are now counted as `incomplete` (with the base model's
|
||||
error kept apart as `incomplete_base_model_mae`); spans that cannot be replayed
|
||||
are counted as `skipped`.
|
||||
- **Window sensors unavailable at startup are unknown, not closed.** Last known
|
||||
states, open events and recovery are saved and restored as unknown until the
|
||||
sensor reports; openings and closings count as confirmed only when seen; an
|
||||
unknown window last seen open is assumed open only until the open-window time
|
||||
after it was last seen; other unknown windows are flagged as uncertain in
|
||||
forecasts (`uncertain_windows`) and in Status (`forecast_assumption`). Event
|
||||
durations are given as a range when an end was not seen.
|
||||
- **Unload, reload and removal cannot publish or save stale work.** The
|
||||
coordinator is marked closing before platforms unload; the model save checks
|
||||
this, and a removed-model mark, after taking the storage lock and holds the lock
|
||||
until the file write finishes, so a deletion can no longer be undone by a save
|
||||
that was waiting or cancelled mid-write. A failed setup shuts the coordinator
|
||||
down; relay callbacks queued before unload are ignored.
|
||||
- **Forecast Error compares the relay's actual timing with the plan.** The
|
||||
deviation is the time the relay spent in another state than planned (unobserved
|
||||
time counts as possible deviation). Up to 2 minutes: matched and scored; up to 5:
|
||||
approximate, scored separately; more: counterfactual; otherwise incomplete, with
|
||||
a reason (relay history, assumed inputs, no poll at the due time, missing
|
||||
reading). Forecasts made with an unknown window or electrical heat are never
|
||||
scored.
|
||||
|
||||
### Changed
|
||||
|
||||
- Learning no longer substitutes values for an open window: samples where the
|
||||
wind direction (for the directional term) or another zone's windows (for
|
||||
cross-ventilation) were unknown are not learned. With every window closed, the
|
||||
opening terms need no weather input, so recovery can be learned without wind.
|
||||
- The gust option's hint refers to Model Error.
|
||||
|
||||
### Persisted data
|
||||
|
||||
- `schema_version` stays 2. `ventilation` gains `state` (last known window
|
||||
states, open events, recovery); `evaluation` gains `excluded` (incomplete and
|
||||
skipped spans); `operational_evaluation` gains `version` 2 and `approximate`.
|
||||
Missing optional keys load as empty.
|
||||
|
||||
## 0.4.1 — 2026-10-27
|
||||
|
||||
Stability and correctness: fixes four problems found in 0.4.0 and separates model
|
||||
|
||||
@@ -66,12 +66,24 @@ modeled; the persistent load is already part of the zone's learned behavior.
|
||||
**Windows** need a contact sensor (binary sensor), the zone, whether the sensor
|
||||
reports *on* when closed (inverted), the direction the outside of the window faces
|
||||
(degrees clockwise from north; default 345°, editable) and an opening factor
|
||||
(1.0 normal, lower for tilted, higher for large). Unknown or unavailable states are
|
||||
treated as unknown, never as closed: a window last seen open counts as possibly
|
||||
open (the forecast assumes it open, the zone's base model does not learn), and an
|
||||
opening that ended while the sensor was unknown is logged as unconfirmed, with its
|
||||
longest possible duration. Recovery after closing is learned only from a confirmed
|
||||
close.
|
||||
(1.0 normal, lower for tilted, higher for large).
|
||||
|
||||
A window is **open**, **closed** or **unknown** (unknown, unavailable or no state);
|
||||
unknown never means closed. While any window of a zone is unknown, the zone does
|
||||
not learn. What forecasts assume for an unknown window:
|
||||
|
||||
- last seen open: open, but only until the configured open-window time after it was
|
||||
last seen (never renewed), then closed and recovering;
|
||||
- last seen closed, or never seen since setup: no ventilation, but the forecast is
|
||||
flagged (`uncertain_windows` on the zone's forecast entity, `forecast_assumption`
|
||||
in Status) and is never used to measure forecast accuracy.
|
||||
|
||||
An opening or closing is *confirmed* only when it was seen happen: the previous
|
||||
reading was the other state, with no unknown period or restart in between. Events
|
||||
report a duration only when both ends are confirmed, otherwise a minimum and, where
|
||||
possible, a maximum; recovery is learned only after a confirmed close. The last
|
||||
known states, an open event and a recovery in progress are saved, so a restart or
|
||||
reload continues them, but they come back as unknown until the sensor reports.
|
||||
|
||||
Model options in **Configure**: how fast extra electrical heat is assumed to fade in
|
||||
forecasts (half-life, default 30 minutes), how long an open window is assumed to
|
||||
@@ -186,9 +198,10 @@ the plan is discarded and the next poll computes a fresh one. Useful attributes:
|
||||
included sources), baseline, variable power above it, thermal response, whether it
|
||||
has settled.
|
||||
- `windows` — per window: state (`open`, `closed` or `unknown`), the last known
|
||||
state and for how long the sensor has been unknown, minutes open, wind direction
|
||||
relative to the window's facing and the resulting exposure (1 = wind straight at
|
||||
it).
|
||||
state and for how long the sensor has been unknown, what forecasts assume
|
||||
(`forecast_assumption`: `observed`, `last_known_open`, `last_known_closed` or
|
||||
`unobserved`; `forecast_open_minutes`), minutes open, wind direction relative to
|
||||
the window's facing and the resulting exposure (1 = wind straight at it).
|
||||
`ventilation` — per zone: current opening, recovery, temperature before opening,
|
||||
its contribution in °C/h and the ventilation coefficients. `ventilation_events` —
|
||||
the last five openings with temperatures before, lowest, at closing and 30
|
||||
@@ -288,35 +301,57 @@ model is scored on the same spans. Each sensor's state is the mean absolute erro
|
||||
- `base_model_mae`, `improvement_vs_base` — the base model's error on the same
|
||||
spans, and the share of it the extra layers remove (0 = no gain, negative = they
|
||||
make it worse: switch that option off).
|
||||
- `incomplete` — spans left out because an input the zone's model needed was not
|
||||
observed at some step: a window state (or the start of a recovery), the wind,
|
||||
wind direction or another zone's windows while a window was open, or, for the
|
||||
extended layer, electrical heat or a neighbour (whose own span must be complete
|
||||
too). The base model needs none of these, so its error on those spans is still
|
||||
reported as `incomplete_base_model_mae`.
|
||||
- `skipped` — spans that could not be replayed: the radiator-lag estimate was
|
||||
uncertain along the way, or the zone had no reading at the end.
|
||||
- `zones` — the same numbers per zone (not written to the Recorder).
|
||||
|
||||
Spans are scored only where the inputs were known: polls 4.5–5.5 minutes apart, a
|
||||
usable reading at the start and the end, and a well-known radiator-lag estimate
|
||||
throughout. A missing relay, outdoor temperature or all zones restarts the replay
|
||||
window, so scores resume 15–120 minutes later. Statistics are kept with the model
|
||||
and survive restarts; recalibrating a zone or deleting the model resets them.
|
||||
Spans need polls 4.5–5.5 minutes apart; a missing relay, outdoor temperature or
|
||||
all zones restarts the replay window, so scores resume 15–120 minutes later (these
|
||||
gaps are not counted). Statistics are kept with the model and survive restarts;
|
||||
recalibrating a zone or resetting the learned model clears them.
|
||||
|
||||
### Forecast Error
|
||||
|
||||
Each time a plan is published, its forecast is recorded as it was: start
|
||||
temperatures, the trajectory, the plan, the model revision and the assumptions
|
||||
(outdoor temperature and wind held, electrical heat fading, open windows closing).
|
||||
When the forecast's horizon comes, it is scored against the reading, but only if
|
||||
the relay actually followed the plan: in every 15-minute block up to that horizon
|
||||
the relay must have been in the planned state for at least 12 of the 15 minutes.
|
||||
Otherwise the forecast is not an error measurement, and is counted instead:
|
||||
When the forecast's horizon comes, the relay's actual behaviour is compared with
|
||||
the plan, block by block, from the relay's own switch times: the **deviation** is
|
||||
how long the relay was in another state than planned, so the same on-time at other
|
||||
times than planned counts as deviation. Relay time that was not observed counts as
|
||||
possible deviation. Each forecast is then:
|
||||
|
||||
- `counterfactual` — the relay did something else: Generic Thermostat, which
|
||||
controls it, disagreed with the plan. Expected in observation mode whenever the
|
||||
two disagree.
|
||||
- `unverifiable` — the relay history is incomplete for that span.
|
||||
- `missing` — no poll near the due time, or no reading for a zone.
|
||||
- `matched` — at most 2 minutes of deviation. Only these make the Forecast Error
|
||||
figures.
|
||||
- `approximate` — at most 5 minutes (of which at most 2 unobserved), for example
|
||||
the thermostat switching a few minutes earlier than planned. Scored separately
|
||||
in `approximate_stats`.
|
||||
- `counterfactual` — more than 5 minutes: Generic Thermostat, which controls the
|
||||
relay, did something else. Not an error of the forecast. Expected in observation
|
||||
mode whenever the two disagree.
|
||||
- `incomplete` — not comparable, with `incomplete_reasons`: `relay_history` (too
|
||||
much relay time unknown), `uncertain_inputs` (the forecast had to assume an
|
||||
unknown window or electrical heat), `no_poll` (no poll near the due time) or
|
||||
`reading` (a zone had no reading; counted per zone).
|
||||
|
||||
The state is the 60-minute MAE once its quality is at least `indicative`; the
|
||||
attribute `horizons` (not written to the Recorder) gives MAE, RMSE, bias, the
|
||||
no-change error, `improvement_vs_persistence`, sample counts, quality, the counts
|
||||
above and the per-zone figures for each horizon. Statistics cover 7 days and
|
||||
survive restarts; forecasts not yet due when HA restarts are dropped.
|
||||
Why these limits: d seconds of boiler heat that the plan did not assume move a room
|
||||
by roughly *heat* × d / 300, where *heat* is the learned boiler coefficient
|
||||
(typically 0.05–0.12 °C per 5 minutes with warm radiators). 2 minutes stay below
|
||||
half a typical 0.1 °C sensor step and absorb timestamp jitter; 5 minutes are about
|
||||
one step.
|
||||
|
||||
The state is the 60-minute MAE of matched forecasts once its quality is at least
|
||||
`indicative`; the attribute `horizons` (not written to the Recorder) gives MAE,
|
||||
RMSE, bias, the no-change error, `improvement_vs_persistence`, sample counts,
|
||||
quality, the counts above, `approximate_stats` and the per-zone figures for each
|
||||
horizon. Statistics cover 7 days and survive restarts; forecasts not yet due when
|
||||
HA restarts are dropped.
|
||||
|
||||
What neither covers: outdoor temperature changes during the horizon (held
|
||||
constant, as in the MPC), and periods when the model was still learning (included,
|
||||
@@ -359,9 +394,10 @@ The brand icons in `custom_components/heating_mpc/brand/` are drawn by
|
||||
`python scripts/make_brand_icons.py` (uses Pillow from the test requirements);
|
||||
the package check verifies their sizes and transparency.
|
||||
|
||||
HA tests cover setup, config/options forms, migration, reload (a single relay
|
||||
listener and polling coordinator after repeated reloads) and entity cleanup,
|
||||
scheduled sampling and poll cancellation, the relay listener, stored model
|
||||
HA tests cover setup, config/options forms (including the reset), migration,
|
||||
reload (a single relay listener and polling coordinator after repeated reloads),
|
||||
unload, reload and removal while an optimization or a save is still running,
|
||||
entity cleanup, scheduled sampling and poll cancellation, the relay listener, stored model
|
||||
restoration, retention and deletion, missing inputs, manual off/presets, optimizer
|
||||
failure and Recorder exclusions. A fixture rejects all HA service calls throughout
|
||||
each test, and a static test fails if actuator service calls are added to the
|
||||
|
||||
+19
-4
@@ -95,6 +95,18 @@ changed during optimization or by an overtaken run; and model accuracy reported
|
||||
as if it were forecast accuracy. Reload/unload tests now cover repeated reloads
|
||||
(one relay listener, one polling coordinator). Released as 0.4.1; not yet tested on HAOS.
|
||||
|
||||
### Final stability fixes after 0.4.1 — 2026-10-27
|
||||
|
||||
An independent review of 0.4.1 reported four problems; three were reproduced and
|
||||
the fourth (lifecycle races) confirmed by reading Home Assistant's unload order.
|
||||
Fixed with regression tests (see AUDIT.md): Model Error spans with unknown inputs
|
||||
are now `incomplete`, not scored; window sensors unavailable at startup are
|
||||
unknown, with last known states saved and restored as unknown; unload, reload
|
||||
and removal can no longer publish stale results or recreate a deleted model;
|
||||
Forecast Error scores only forecasts whose plan the relay matched within 2
|
||||
minutes. Also added: **Reset learned model** in Configure. Not released or tested
|
||||
on HAOS yet.
|
||||
|
||||
## Milestone 1 — Explainable forecasts and dashboards (target v0.3.0)
|
||||
|
||||
**Objective:** Users can verify what the model predicts and whether it is actually accurate.
|
||||
@@ -103,7 +115,7 @@ as if it were forecast accuracy. Reload/unload tests now cover repeated reloads
|
||||
- [x] Expose predicted +15/+30/+60/+120 minute temperatures and full proposed ON/OFF schedule. *(0.3.0: `forecast_15min` and `plan_15min` on each room forecast entity.)*
|
||||
- [ ] Provide an HA dashboard example using native cards or a clearly documented optional chart card (do not bundle a heavyweight frontend unnecessarily).
|
||||
- [ ] Make clear that the **MPC plan forecast is counterfactual** while Generic Thermostat is controlling the relay; add a separate actual-policy forecast/evaluation pathway. *(0.3.0: plan forecasts labelled counterfactual; evaluation uses the actual relay history. Stabilization: published forecasts are scored only when the relay followed their plan and otherwise counted as `counterfactual`. An actual-policy forecast, simulating Generic Thermostat, is still open.)*
|
||||
- [ ] Store a bounded history of predictions with snapshots of inputs, proposed plan and eventually observed outcome. *(Stabilization: each published forecast is kept with its start readings, plan, model revision and assumptions until scored, in memory only; the scored statistics persist. No browsable history yet.)*
|
||||
- [ ] Store a bounded history of predictions with snapshots of inputs, proposed plan and eventually observed outcome. *(Stabilization: each published forecast is kept with its start readings, plan, model revision and assumptions, including any it had to assume, until scored, in memory only; the scored statistics persist, split into matched, approximate, counterfactual and incomplete. No browsable history yet.)*
|
||||
- [x] Implement out-of-sample/walk-forward MAE and bias at 15, 30, 60 and 120 minutes; include sample counts and confidence/quality labels. *(0.3.0: replay with parameters frozen at the forecast start and the actual relay history; `skill` against persistence. Stabilization: these are now the Model Error sensors, with RMSE, effective sample counts and quality labels; a separate Forecast Error sensor scores published forecasts. No confidence intervals yet.)*
|
||||
- [ ] Compare against simple benchmarks such as current-temperature persistence and a fixed thermal baseline. *(0.3.0: persistence done; fixed baseline open.)*
|
||||
- [ ] Add inspectable explanations: expected heat loss, expected heating response, target, room weights and reason for the next proposed action.
|
||||
@@ -187,9 +199,12 @@ Items 1–3 of the original backlog are done (see AUDIT.md and CHANGELOG.md 0.2.
|
||||
3. Configure zones, the UPS heat source and the windows on HAOS; after a week of
|
||||
heating review Model Error `improvement_vs_base` and `quality`, per-zone bias,
|
||||
coefficient `confidence` (base layer included), `model_changes` after any
|
||||
reconfiguration, and the Forecast Error counts (how many forecasts were scored
|
||||
vs counterfactual). Switch off optional terms that do not help. Also try an
|
||||
unavailable window sensor and a zone edit during HAOS acceptance.
|
||||
reconfiguration, and the Forecast Error counts (matched, approximate,
|
||||
counterfactual, incomplete and why). Switch off optional terms that do not
|
||||
help. During HAOS acceptance also try: a window sensor unavailable across an HA
|
||||
restart, a zone edit, a reload while the integration is running, *Reset learned
|
||||
model* and removing a disposable entry (its `.storage/heating_mpc.<entry_id>`
|
||||
file must be gone afterwards).
|
||||
4. Milestone 1 (rest): a dashboard example; forecast timestamps/provenance;
|
||||
explanations for the proposed action; an actual-policy forecast that simulates
|
||||
Generic Thermostat.
|
||||
|
||||
+47
-12
@@ -98,11 +98,19 @@ known. While a window is open, possibly open (its sensor is unknown and it was l
|
||||
seen open) or the zone is recovering, the base and extended layers do **not**
|
||||
learn, so window events cannot inflate `loss` or shift `bias`.
|
||||
|
||||
An unknown window state is never read as closed. A window last seen open stays
|
||||
assumed open in forecasts; one last seen closed contributes no ventilation term.
|
||||
If a window closes while its sensor was unknown, the event's duration is reported
|
||||
as a range and the recovery term is not learned from it, because the closing time
|
||||
is not known.
|
||||
An unknown window state is never read as closed, and an opening or closing counts
|
||||
as confirmed only when it was seen: the previous reading was the other state with
|
||||
no unknown period (or restart) in between. If a window closes while its sensor was
|
||||
unknown, the event's duration is reported as a range and the recovery term is not
|
||||
learned from it, because the closing time is not known. Last known states, open
|
||||
events and recovery are saved and restored across restarts, as unknown until the
|
||||
sensor reports again.
|
||||
|
||||
An input that a term needs is never filled in for learning or for Model Error:
|
||||
while a window is open, the wind, its direction (for `vent_wind_dir`) and the other
|
||||
zones' windows (for `vent_cross`) must be known. With every window closed, the
|
||||
opening terms are 0 and need no weather. Forecasts, which must produce a number,
|
||||
assume calm air for a missing wind input.
|
||||
|
||||
`vent_wind_dir` is bounded by `vent_wind` (|vent_wind_dir| ≤ vent_wind), so the
|
||||
combined wind term never goes negative: a leeward window still ventilates.
|
||||
@@ -202,7 +210,10 @@ minutes):
|
||||
its response time. A gaming PC drawing 600 W now is not assumed to run on for two
|
||||
hours.
|
||||
* **Windows:** an open window is assumed to stay open for a configurable time
|
||||
(default 15 minutes), then close; the zone then recovers.
|
||||
(default 15 minutes), then close; the zone then recovers. A window whose sensor
|
||||
is unknown but was last seen open is assumed open only until that time after it
|
||||
was last seen, never renewed. One last seen closed, or never seen since setup, is
|
||||
assumed closed, and the forecast is flagged as uncertain (`uncertain_windows`).
|
||||
* **Relay:** the candidate plan being evaluated.
|
||||
|
||||
For the plan search the step is compiled into exact per-zone coefficients (every
|
||||
@@ -228,13 +239,34 @@ scored on exactly the same spans:
|
||||
* `base_model_mae`, `improvement_vs_base`: against the base layer alone. A feature
|
||||
that does not improve this should be switched off (for example the gust term).
|
||||
|
||||
A span counts only if every input the zone's model used was observed at every
|
||||
step (see *Ventilation layer*; for the extended layer also electrical heat and
|
||||
neighbours, whose own replay must be complete). Otherwise it is counted as
|
||||
`incomplete`, with the base model's error on it kept apart
|
||||
(`incomplete_base_model_mae`), since the base model needs only the relay and the
|
||||
outdoor temperature. Spans that cannot be replayed (uncertain radiator lag, no
|
||||
reading at the end) are counted as `skipped`.
|
||||
|
||||
**Forecast accuracy (Forecast Error sensor)** scores forecasts as published: the
|
||||
start readings, trajectory, plan, model revision and assumptions of the moment
|
||||
they were made, nothing replaced with hindsight. Because the plan is the MPC's
|
||||
proposal while Generic Thermostat controls the relay, a forecast is scored at a
|
||||
horizon only if the relay was in the planned state for at least 12 minutes of
|
||||
every 15-minute block up to it; otherwise it is counted as `counterfactual` (or
|
||||
`unverifiable` when the relay history is incomplete) and never as an error.
|
||||
proposal while Generic Thermostat controls the relay, a forecast is compared only
|
||||
when the relay followed it. The deviation is the time, up to the horizon, the
|
||||
relay spent in another state than planned, compared block by block from the
|
||||
relay's own switch times; unobserved relay time counts as possible deviation.
|
||||
|
||||
| Deviation (worst case) | Class | Used for |
|
||||
|---|---|---|
|
||||
| ≤ 2 min | matched | Forecast Error figures |
|
||||
| ≤ 5 min, ≤ 2 min unobserved | approximate | `approximate_stats` only |
|
||||
| > 5 min observed | counterfactual | counted, never an error |
|
||||
| otherwise | incomplete | counted, with a reason |
|
||||
|
||||
Forecasts made with an assumed input (an unknown window or electrical heat) are
|
||||
incomplete as well. The limits follow from the model: d seconds of unplanned
|
||||
boiler heat change a room by about heat × d / 300 with the learned `heat`
|
||||
coefficient (typically 0.05–0.12 °C per 5 minutes), so 2 minutes stay below half a
|
||||
0.1 °C sensor step and 5 minutes are about one step.
|
||||
|
||||
Both report how much evidence there is. Forecasts start every 5 minutes, so at
|
||||
horizon h consecutive forecasts share most of their span, and all zones share one
|
||||
@@ -267,5 +299,8 @@ the contamination the separate ventilation layer prevents.
|
||||
variation their split is uncertain even when their combined effect is accurate.
|
||||
* Forecasts hold weather constant and use one scenario for electrical heat and
|
||||
windows; no forecast uncertainty band is computed yet.
|
||||
* Window sensors with a permanently unknown state keep their zone out of learning;
|
||||
disable such a window.
|
||||
* Window sensors with a permanently unknown state keep their zone out of learning
|
||||
and its Model Error spans incomplete; disable such a window.
|
||||
* A window that is unknown and was last seen closed (or never seen) is assumed
|
||||
closed in forecasts, flagged as uncertain; no open/closed scenario pair is
|
||||
computed.
|
||||
Reference in new issue
Block a user