mirror of
https://github.com/lynchaos/ashvale-station.git
synced 2026-09-12 20:52:23 +00:00
Fix the runaway forecasts: refits accumulated, and annual terms fitted too early
Reported from a real station after 1.5 days: a six hour temperature forecast of 53 C in a 24 C room, and 9 C at one day, both carrying a plus or minus of 0.43. Confidently wrong is the one failure this project is supposed to refuse. Root cause. fit() replayed history into the live RLS on every retrain tick and never reset, so 453 grid rows had produced 64,676 updates in a day and a half. RLS with forgetting reads every update as fresh evidence, so the model believed it had a hundred times the data it had: P collapsed, in-sample error looked excellent, and the weights drifted without bound in directions the data never excited. Measured: cond(P) 3.1e9 and ||theta|| 1680 against a median |theta| of 1.67. A refit now starts from the prior, which makes retraining idempotent. Across 25 refits on the real data ||theta|| holds at 11.35, drifting 0.03, where before it grew without limit. The two largest weights were sin_doy and cos_doy at +1174 and +1191. Annual harmonics were in the design matrix from the first sample, where they are near-constant, near-collinear with each other and with the bias, and a rank-deficient regressor is what RLS answers with enormous cancelling weights. They are now held at zero until the record spans the same 120 days the climatology fit already requires, because a day and a half of data says nothing whatsoever about the season. Also raised the standardiser's variance floor from 1e-8, which only caught a bit-exactly constant column, to 1e-3. A feature that merely barely moves was being divided by its own noise. The conformal calibrators and Hedge weights are deliberately not reset by a refit: those are earned from scored forecasts, not from this regression. Backtest unchanged within noise, coverage still 89 to 91 across all 18 heads. Four regression tests added, including that refitting the same history twice must give the same model.
This commit is contained in:
+21
-2
@@ -47,8 +47,9 @@ class RecursiveLeastSquares:
|
||||
self.d = int(n_features)
|
||||
self.lam = float(forgetting)
|
||||
self.p_max = float(p_max)
|
||||
self.delta = float(delta) # kept so a refit can return to the prior
|
||||
self.theta = np.zeros(self.d)
|
||||
self.P = np.eye(self.d) * float(delta)
|
||||
self.P = np.eye(self.d) * self.delta
|
||||
self.n_updates = 0
|
||||
self.ewma_sq_error = 0.0
|
||||
|
||||
@@ -102,14 +103,32 @@ class RecursiveLeastSquares:
|
||||
def noise_var(self) -> float:
|
||||
return float(max(self.ewma_sq_error, 1e-9))
|
||||
|
||||
def reset(self) -> None:
|
||||
"""Return to the prior, keeping the configuration.
|
||||
|
||||
A batch refit has to start from here rather than continuing, because
|
||||
replaying the same history into a live filter is not the same as seeing
|
||||
new data. RLS with forgetting treats every update as fresh evidence, so
|
||||
feeding it the same rows on each retrain tick makes it believe it has
|
||||
many times the data it has: P collapses, and the weights in directions
|
||||
the data never excites drift without anything to pull them back.
|
||||
"""
|
||||
self.theta = np.zeros(self.d)
|
||||
self.P = np.eye(self.d) * self.delta
|
||||
self.n_updates = 0
|
||||
self.ewma_sq_error = 0.0
|
||||
|
||||
def to_dict(self) -> Dict:
|
||||
return {"d": self.d, "lam": self.lam, "p_max": self.p_max,
|
||||
"delta": self.delta,
|
||||
"theta": self.theta.tolist(), "P": self.P.tolist(),
|
||||
"n": self.n_updates, "ewma": self.ewma_sq_error}
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, s: Dict) -> "RecursiveLeastSquares":
|
||||
m = cls(s["d"], s["lam"], 1.0, s.get("p_max", 1e6))
|
||||
# delta must survive the round trip or a refit after a restart would
|
||||
# return to the wrong prior.
|
||||
m = cls(s["d"], s["lam"], s.get("delta", 100.0), s.get("p_max", 1e6))
|
||||
m.theta = np.array(s["theta"], dtype=float)
|
||||
m.P = np.array(s["P"], dtype=float)
|
||||
m.n_updates = s.get("n", 0)
|
||||
|
||||
Reference in New Issue
Block a user