fix(load_ml): guard against zero-filled exogenous inputs, and report rollout accuracy

[springfall2008]

What

Rates and temperature are 3 of the 5 input channels to the ML load model — 864 of 1446 features. Where a forward value was missing, predict() substituted 0.0, feeding 0 p/kWh and 0 °C into a network trained on ~33.6 p/kWh and ~7 °C. Nothing logged it.

This is a startup and failure-path guard, not a steady-state bug. A healthy system supplies the whole rollout: rate_replicate() extends rates past the forecast horizon, publish_rates() emits them out to minutes_now + forecast_minutes + 24h, and minute_data() returns them as forward keys covering all 576 steps. I verified that last step rather than assuming it. The gap only opens when the source entity is unavailable — before the first plan cycle publishes it, or after a failed rate fetch.

This carries the last known value forward there instead, and logs when it engages. PV keeps its 0.0 default deliberately: past the solar forecast there is no generation to assume, and holding a daytime value overnight would be worse.

Why it matters despite being an edge case

Measured by replaying a user's saved predbat_ml_model.npz against their own history:

Forward exog data	+8h MAE (before)	+8h MAE (after)	48h rollout peak (before → after)
fully supplied	0.28 kWh	0.28 kWh	10.5 kW → 10.5 kW
absent	2.43 kWh	0.40 kWh	1.9 kW → 9.9 kW

Their true peak is 10.5 kW. With the inputs zeroed the rollout flattens to 1.9 kW — the household's large daily event vanishes entirely, and the plan is built on it. The fully-supplied case is byte-identical before and after, so there is no behaviour change when data is present.

Also: multi-step accuracy is now reported

validation_mae / mae_kwh is teacher-forced one-step-ahead and stays small however badly the rollout behaves — it reads 0.0065 kWh on the model above. There was no published signal for multi-step quality at all.

_ar_rollout_diagnostic now also scores the day-of-week daily-pattern baseline it is blended against, and rollout_mae_kwh / pattern_mae_kwh are persisted with the model and published on sensor.<prefix>_load_ml_stats. If pattern_mae_kwh is the lower of the two, the network is not beating a simple historical average at range. Both are reporting only and do not change the forecast.

This is arguably the more useful half. The model retrains every two hours, and nothing currently reveals whether a given retrain produced a good long-range rollout or a collapsed one.

Relationship to #4673

Raised while investigating #4673, deliberately not marked as fixing it. Replaying that reporter's model with correctly supplied inputs reproduces no anomaly (+8h MAE 0.28 kWh), so this is not the cause of their chart and their report stays open.

Testing

New apps/predbat/tests/test_load_ml_rollout.py, registered as load_ml_rollout (fast, no training). Four tests: rate features are never zero-filled, the diagnostic reports the pattern baseline, the scores survive a save/load roundtrip, and the stats sensor exposes them.

Full suite: 264 tests pass. Pre-commit clean.

🤖 Generated with Claude Code

[Copilot]

🟡 Changes recommended

There are correctness issues in the new exogenous “ran out” counter/logging logic and in metadata persistence of validation_mae that should be fixed before approval.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR improves the ML load model’s long-horizon autoregressive rollout by preventing missing forward rate/temperature forecasts from being implicitly treated as zero, and adds visibility into multi-step rollout quality versus the built-in daily-pattern baseline.

Changes:

• Carry forward last-known temperature/import/export rate values when forward forecasts run out during rollout (PV remains zero-filled by design).
• Persist and publish additional diagnostic metrics (rollout_mae_kwh, pattern_mae_kwh) alongside the existing teacher-forced MAE.
• Add regression tests and documentation covering rollout accuracy and the new diagnostic signals.

File summaries

File	Description
docs/load-ml.md	Documents forward exogenous fallback behavior and adds a “Rollout Accuracy” section explaining new metrics.
apps/predbat/unit_test.py	Registers the new load_ml_rollout test suite.
apps/predbat/tests/test_load_ml_rollout.py	Adds regression tests for exogenous fallback and rollout-vs-pattern diagnostics/persistence/publication.
apps/predbat/load_predictor.py	Implements exogenous carry-forward, adds rollout/pattern holdout metrics, persists them in model save/load, and logs diagnostic comparison.
apps/predbat/load_ml_component.py	Publishes rollout_mae_kwh and pattern_mae_kwh on the load ML stats sensor.

Review details

• Files reviewed: 5/5 changed files
• Comments generated: 2
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.