Take model tuning from config too, not from the saved state

The companion to the Kalman fix. RecursiveLeastSquares.from_dict and
AdaptiveConformal.from_dict restore lambda, delta, alpha, gamma and the
conformal window alongside their data, and load_dict replaces the
config-built heads with those, so every one of those knobs was immutable on
any station that already had state. Editing config.yaml and restarting looks
exactly like a change with no effect, which is the failure mode that cost real
time on the Kalman side of this before it was found.

Only the estimate is state now. Weights, covariances and conformal scores are
restored; tuning is re-applied from config on every load. The conformal deques
are rebuilt when the configured window changes, preserving their contents.

load_dict also skips heads for a target or horizon this build no longer has,
rather than resurrecting them from a stale file.

Found while implementing a damped-trend ensemble member, which was then
abandoned: see the following note.
This commit is contained in:
2026-08-19 19:31:03 +01:00
parent 3f41881c64
commit 41a46ae14a
3 changed files with 88 additions and 1 deletions
+23 -1
View File
@@ -151,6 +151,7 @@ class NowcastEnsemble:
cfg_model):
self.targets = tuple(targets)
self.horizons = tuple(int(h) for h in horizons_s)
self.cfg = cfg_model
self.grid_s = int(cfg_model.grid_s)
self.scaler = Standardiser(N_FEATURES)
self.heads: Dict[Tuple[str, int], ForecastHead] = {
@@ -306,6 +307,27 @@ class NowcastEnsemble:
self.scaler = Standardiser.from_dict(s["scaler"])
for hs in s["heads"]:
head = ForecastHead.from_dict(hs)
self.heads[(head.target, head.horizon_s)] = head
key = (head.target, head.horizon_s)
if key not in self.heads:
continue # a target or horizon this build no longer has
self.heads[key] = head
self.trained_rows = s.get("trained_rows", 0)
self.refit_phase = int(s.get("refit_phase", 0))
self._apply_config_tuning()
def _apply_config_tuning(self) -> None:
"""Tuning comes from config; only the estimate comes from the file.
Every from_dict below this point restores its knobs alongside its data:
lambda, delta and p_max in the RLS, alpha, gamma and the window in the
conformal calibrator. So each of those was immutable in the field. Edit
config.yaml, restart, and the state file quietly puts the old value
back, which looks exactly like a change that had no effect. The same
defect cost a Kalman retune here before it was found.
"""
c = self.cfg
for head in self.heads.values():
head.model.lam = float(c.rls_forgetting)
head.model.delta = float(c.rls_delta)
head.conformal.retune(c.conformal_alpha, c.conformal_gamma,
c.conformal_window)
+14
View File
@@ -183,6 +183,20 @@ class AdaptiveConformal:
self.alpha = float(np.clip(self.alpha + self.gamma * (self.alpha_target - err),
0.005, 0.75))
def retune(self, alpha: float, gamma: float, window: int) -> None:
"""Re-apply configured tuning, keeping the observed scores.
from_dict restores alpha_target, gamma and the window alongside the
data, so editing any of them in config.yaml did nothing on a station
that already had state: the file put the old values straight back.
"""
self.alpha_target = float(alpha)
self.gamma = float(gamma)
window = int(window)
if self.scores.maxlen != window:
self.scores = deque(self.scores, maxlen=window)
self.hits = deque(self.hits, maxlen=window)
@property
def empirical_coverage(self) -> float:
return float(np.mean(self.hits)) if self.hits else float("nan")