Score a forecast only when the relay matched its plan in time, not just in share

Reproduced: a forecast planning 15 minutes of heat was scored although the
relay was on for 12 (the 0.4.1 rule allowed 20 % per 15-minute block), and
the same on-time placed elsewhere in a block passed as well.

- The relay's on-time per poll interval, integrated at its own switch
  times, is compared with the plan block by block up to each horizon: the
  deviation is the time the relay spent in another state than planned.
  Unobserved relay time counts as possible deviation.
- Matched (at most 2 minutes, worst case): Forecast Error figures.
  Approximate (at most 5 minutes, at most 2 of them unobserved): separate
  approximate_stats. Counterfactual (more than 5 minutes seen): counted, not
  scored. Incomplete otherwise, by reason: relay history, assumed inputs at
  forecast time (an unknown window or electrical heat), no poll near the
  due time, or a missing reading.
- 2 minutes of unplanned boiler heat move a room by roughly half a 0.1 °C
  sensor step for the learned heat coefficients (0.05-0.12 °C per 5
  minutes); 5 minutes by about one step.
- Stored ledger statistics are versioned (2); 0.4.1 statistics used the
  looser rule and are dropped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
buzeraandClaude Opus 5.5 committed 2026-10-27 04:50:58 +02:00
1 parent 2cfba024ff
commit 0b82a8e26c
5 files changed
+267 -83

No files matched your search

+18 -2
View File
@@ -1329,8 +1329,24 @@ class BoundaryTests(unittest.IsolatedAsyncioTestCase):
# Longer horizons are not due yet.
self.assertEqual(data['operational_forecast'][30]['samples'], 0)
stored = self.c._payload('sensor.outside')['operational_evaluation']
self.assertEqual(sum(sum(v.get('15', {}).values()) for v in stored['verdicts'].values()),
0 if follows else 1)
counts = {}
for by_horizon in stored['verdicts'].values():
for name, count in by_horizon.get('15', {}).items():
counts[name] = counts.get(name, 0) + count
self.assertEqual(counts, {'matched': 1} if follows else {'counterfactual': 1})
async def test_a_forecast_made_with_an_unknown_window_is_never_scored(self):
self.ready_office()
self.set_state('sensor.room', '23.5', unit_of_measurement='°C') # plan: off, as the relay
self.states['binary_sensor.office_window'].state = 'unavailable'
first = await self.sample()
self.assertEqual(first['status'], 'ready')
self.assertEqual(self.c._forecasts._pending[-1]['assumptions']['uncertain_inputs'], ['window: Window'])
for _ in range(3):
self.fresh('sensor.room', 'sensor.outside', 'sensor.ups_power')
data = await self.sample()
fifteen = data['operational_forecast'][15]
self.assertEqual((fifteen['samples'], fifteen['incomplete_reasons']['uncertain_inputs']), (0, 1))
async def test_unloading_during_save_or_optimization_publishes_nothing_new(self):
self.ready_office()