kemal/ashvale-station
1
0
Fork 0
mirror of https://github.com/lynchaos/ashvale-station.git synced 2026-09-12 12:47:49 +00:00
Code Issues Packages Projects Releases Wiki Activity

Stats for Nerds tab, KaTeX methods, weather icons, outlook to top

Browse Source 
New Stats for Nerds tab over a new read-only /api/nerd endpoint: Kalman NIS
and covariance per signal, both compensators, all 18 RLS heads with trace(P)
against the cap, |theta|, EWMA RMSE, conformal alpha against target, realised
coverage and ensemble weights, plus per-head feature attribution over the 33
standardised weights, the Mahalanobis and Page-Hinkley detector state,
climatology harmonics and precipitation coefficients.

Methods overhaul: KaTeX now renders the equations. They were previously passed
through .replace(/[{}\\]/g,' '), which stripped every brace and backslash and
turned real mathematics into mush. Stages 2, 3, 5, 6 and 7 gained full
derivations (RLS normal equations and the trace cap, Joseph-form Kalman with
NIS, adaptive conformal with its coverage limit, ridge harmonic regression with
anomaly decay) and a per-symbol legend rendered inline.

Conditions ahead gains weather icons chosen from measured cloud index, solar
elevation and temperature rather than the barometric class alone, so a fine
barometer under overcast draws a cloud and after sunset draws a moon. Snow is
selected on temperature.

Seven day outlook moves to the top of Live, directly under the nav. Tab renamed
Models and Calibration.

Verified in Chromium at 1600x900: Live, History, Models and Nerd all report
zero scrollbars, zero clipping, no page scroll, zero console errors. Methods
keeps its documented prose scroller. Backtest numerically unchanged.
This commit is contained in:
kemal
2026-08-15 22:00:27 +01:00
parent e27a4b41c8
commit bb9f0a588f
6 changed files with 466 additions and 34 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/ISSUE_TEMPLATE/forecast_quality.md
+1 -1
View File
@@ -11,7 +11,7 @@ labels: forecasting
```
```
**Scorecard from the Models and calibration tab** (or `GET /api/scorecard`)
**Scorecard from the Models and Calibration tab** (or `GET /api/scorecard`)
```json
```
README.md
+5 -4
View File
@@ -160,15 +160,16 @@ permanently.
## The dashboard
Four tabs, one viewport, no scrolling on desktop. Below 1024 px the constraint is
released, because pinning four panels into a phone viewport produces unreadable
Five tabs, one viewport, no scrolling on desktop. Below 1024 px the constraint is
released, because pinning five panels into a phone viewport produces unreadable
eight-pixel type.
| Tab | Answers |
| --- | --- |
| **Live** | What is it doing now, what it expects next, how sure it is, and the week ahead |
| **Live** | The week ahead, current readings, the forecast with its band, and conditions |
| **History** | What did it do, over any timeframe you ask for |
| **Models and calibration** | Has the model earned its confidence, and the calibration inputs |
| **Models and Calibration** | Has the model earned its confidence, and the calibration inputs |
| **Stats for Nerds** | Every internal the estimator and the 18 learners are carrying |
| **Methods** | How the whole thing is wired, and how each stage fails |
Live carries the current readings, the observed-and-forecast chart with its 90%
ashvale/api.py
+102
View File
@@ -34,6 +34,7 @@ from pydantic import BaseModel, Field
from .config import CONFIG
from .dashboard import DASHBOARD_HTML
from .features import FEATURE_NAMES
from .led import LedDisplay
from .methods import describe
from .station import Station
@@ -351,6 +352,107 @@ def models() -> Dict:
})
@app.get("/api/nerd")
def nerd() -> Dict:
"""Every internal number the estimator and the learners are carrying.
Deliberately read-only and computed from live objects rather than stored, so
it cannot drift from what the station is actually using. Everything here is
cheap: no matrix inversions, no queries beyond what the caller already pays
for. `theta` is returned per head so the UI can show which of the 33 features
each horizon actually leans on, which is the single most revealing view of
what the model has learned.
"""
st = _st()
tr = st.tracker
filters = {}
for name, kf in tr.filters.items():
P = np.asarray(kf.P, dtype=float)
filters[name] = {
"level": float(kf.x[0]), "rate_per_h": float(kf.x[1]) * 3600.0,
"nis": float(kf.nis),
"p_level": float(P[0, 0]), "p_rate": float(P[1, 1]),
"p_cross": float(P[0, 1]),
"sigma_level": float(np.sqrt(max(P[0, 0], 0.0))),
"q": float(kf.q), "r": float(kf.r),
"initialised": bool(kf.initialised),
}
heads = []
for (target, h), head in sorted(st.nowcast.heads.items()):
m = head.model
P = np.asarray(m.P, dtype=float)
theta = np.asarray(m.theta, dtype=float)
heads.append({
"target": target, "horizon_s": h,
"n_updates": int(m.n_updates),
"trace_p": float(np.trace(P)),
"theta_norm": float(np.linalg.norm(theta)),
"rmse_ewma": float(np.sqrt(max(m.ewma_sq_error, 0.0))),
"lam": float(m.lam), "p_max": float(m.p_max),
"eff_memory": float(1.0 / max(1.0 - m.lam, 1e-9)),
"alpha": float(head.conformal.alpha),
"alpha_target": float(head.conformal.alpha_target),
"coverage": (float(head.conformal.empirical_coverage)
if np.isfinite(head.conformal.empirical_coverage) else None),
"halfwidth": (float(head.conformal.quantile())
if np.isfinite(head.conformal.quantile()) else None),
"weights": {k: float(v) for k, v in
zip(("persistence", "climatology", "learned"), head.weights)},
"theta": [round(float(v), 6) for v in theta],
})
mono = st.monitor
nov = getattr(mono, "novelty", None)
ph = getattr(mono, "drift", None)
monitoring = {
"novelty": {
"d2": float(getattr(nov, "last_d2", 0.0)) if nov is not None else None,
"threshold": float(getattr(nov, "threshold", 0.0)) if nov is not None else None,
"n": int(getattr(nov, "n", 0)) if nov is not None else None,
"dims": int(getattr(nov, "d", 0)) if nov is not None else None,
"z": [round(float(v), 4) for v in np.asarray(getattr(nov, "z", []), dtype=float)]
if nov is not None else [],
},
"drift": {
"m_pos": float(getattr(ph, "m_pos", 0.0)) if ph is not None else None,
"m_neg": float(getattr(ph, "m_neg", 0.0)) if ph is not None else None,
"mean": float(getattr(ph, "mean", 0.0)) if ph is not None else None,
"n": int(getattr(ph, "n", 0)) if ph is not None else None,
"alarms": int(getattr(ph, "n_alarms", 0)) if ph is not None else None,
"delta": float(getattr(ph, "delta", 0.0)) if ph is not None else None,
},
}
return _clean({
"feature_names": list(FEATURE_NAMES),
"filters": filters,
"compensators": {
"thermal": tr.compensator.to_dict(),
"humidity": tr.hum_compensator.to_dict(),
},
"heads": heads,
"climatology": {
"ready": st.climatology.ready,
"annual_terms": st.climatology.use_annual,
"history_days": round(st.climatology.n_days, 3),
"diurnal_harmonics": st.climatology.kd,
"annual_harmonics": st.climatology.ka,
"ridge": st.climatology.ridge,
"residual_std": st.climatology.resid_std,
"n_coefficients": {k: len(v) for k, v in st.climatology.coef.items()},
},
"precipitation": {
"coefficients": st.precip.coefficients(),
"strong_labels": st.precip.n_strong,
"weak_labels": st.precip.n_weak,
"logloss_ewma": st.precip.ewma_logloss,
},
"monitoring": monitoring,
})
@app.get("/api/scorecard")
def scorecard() -> Dict:
st = _st()
ashvale/dashboard.py
+267 -14
View File
@@ -12,7 +12,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
"""The dashboard: four tabs, one viewport, no scrolling.
"""The dashboard: five tabs, one viewport, no scrolling.
Layout contract. The page is a fixed three-row grid pinned to the
viewport height: header, tab bar, then a content region that takes the
@@ -49,6 +49,8 @@ DASHBOARD_HTML = r"""
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Ashvale Station</title>
<script src="https://cdn.tailwindcss.com"></script>
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/[email protected]/dist/katex.min.css">
<script defer src="https://cdn.jsdelivr.net/npm/[email protected]/dist/katex.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/[email protected]/dist/chart.umd.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/[email protected]/hammer.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/[email protected]/dist/chartjs-plugin-zoom.min.js"></script>
@@ -120,14 +122,26 @@ DASHBOARD_HTML = r"""
<nav role="tablist" class="glass rounded-2xl p-1.5 flex gap-1.5 overflow-x-auto">
<button role="tab" data-tab="live" aria-selected="true" class="tabbtn shrink-0 px-4 py-2 rounded-xl text-xs font-semibold text-slate-400 border border-transparent hover:text-slate-200">Live</button>
<button role="tab" data-tab="history" aria-selected="false" class="tabbtn shrink-0 px-4 py-2 rounded-xl text-xs font-semibold text-slate-400 border border-transparent hover:text-slate-200">History</button>
<button role="tab" data-tab="models" aria-selected="false" class="tabbtn shrink-0 px-4 py-2 rounded-xl text-xs font-semibold text-slate-400 border border-transparent hover:text-slate-200">Models and calibration</button>
<button role="tab" data-tab="models" aria-selected="false" class="tabbtn shrink-0 px-4 py-2 rounded-xl text-xs font-semibold text-slate-400 border border-transparent hover:text-slate-200">Models and Calibration</button>
<button role="tab" data-tab="nerd" aria-selected="false" class="tabbtn shrink-0 px-4 py-2 rounded-xl text-xs font-semibold text-slate-400 border border-transparent hover:text-slate-200">Stats for Nerds</button>
<button role="tab" data-tab="methods" aria-selected="false" class="tabbtn shrink-0 px-4 py-2 rounded-xl text-xs font-semibold text-slate-400 border border-transparent hover:text-slate-200">Methods</button>
</nav>
<main class="min-h-0">
<!-- ---------------- LIVE ---------------- -->
<section id="pane-live" class="pane active h-full min-h-0 gap-3 grid-cols-1 lg:grid-cols-4 lg:grid-rows-[auto_1fr_auto_auto]">
<section id="pane-live" class="pane active h-full min-h-0 gap-3 grid-cols-1 lg:grid-cols-4 lg:grid-rows-[auto_auto_1fr_auto]">
<div class="glass rounded-2xl p-4 lg:col-span-4 shrink-0">
<div class="flex items-center justify-between mb-2">
<div>
<h2 class="text-sm font-bold">Seven day outlook</h2>
<p class="text-[10px] text-slate-500 font-mono">climatology plus decaying anomaly, not a synoptic forecast</p>
</div>
<span id="o-badge" class="px-2 py-0.5 rounded-lg bg-amber-500/10 text-amber-300 border border-amber-500/20 text-[9px] font-mono uppercase font-semibold">warming up</span>
</div>
<div id="o-strip" class="grid grid-cols-4 sm:grid-cols-7 gap-2"></div>
</div>
<div class="glass rounded-2xl p-4 flex flex-col justify-between">
<div class="flex items-center justify-between text-[10px] font-semibold uppercase tracking-wider text-amber-400">
@@ -218,7 +232,9 @@ DASHBOARD_HTML = r"""
</div>
<div class="flex-1 min-h-0 space-y-2 pr-1">
<div class="text-center">
<div id="c-icon" class="flex justify-center mb-0.5"></div>
<div id="c-label" class="text-base font-extrabold leading-tight">--</div>
<div id="c-sky" class="text-[10px] text-slate-400 font-medium">--</div>
<div class="text-[10px] text-slate-500 font-mono mt-0.5">Z=<span id="c-z">-</span> &middot; <span id="c-trend">-</span></div>
</div>
<div>
@@ -245,16 +261,6 @@ DASHBOARD_HTML = r"""
</div>
</div>
<div class="glass rounded-2xl p-4 lg:col-span-4 shrink-0">
<div class="flex items-center justify-between mb-2">
<div>
<h2 class="text-sm font-bold">Seven day outlook</h2>
<p class="text-[10px] text-slate-500 font-mono">climatology plus decaying anomaly, not a synoptic forecast</p>
</div>
<span id="o-badge" class="px-2 py-0.5 rounded-lg bg-amber-500/10 text-amber-300 border border-amber-500/20 text-[9px] font-mono uppercase font-semibold">warming up</span>
</div>
<div id="o-strip" class="grid grid-cols-4 sm:grid-cols-7 gap-2"></div>
</div>
<div class="glass rounded-2xl px-4 py-2.5 lg:col-span-4 grid grid-cols-3 sm:grid-cols-5 lg:grid-cols-10 gap-x-4 gap-y-1.5 font-mono text-[10px]">
<div><div class="text-slate-600 uppercase">wet bulb</div><div id="d-wb" class="text-slate-200 font-semibold">--</div></div>
@@ -394,6 +400,58 @@ DASHBOARD_HTML = r"""
</div>
</section>
<!-- ---------------- STATS FOR NERDS ---------------- -->
<!-- Everything the estimator and the 18 learners are actually carrying, read
straight off the live objects. No internal scrollers: the head bank is a
fixed 18 rows and the attribution list is capped at what fits. -->
<section id="pane-nerd" class="pane h-full min-h-0 gap-3 grid-cols-1 lg:grid-cols-4 lg:grid-rows-[auto_1fr]">
<div class="glass rounded-2xl p-3 lg:col-span-3">
<div class="flex items-baseline justify-between mb-1.5">
<h2 class="text-sm font-bold">Kalman bank</h2>
<span class="text-[9px] font-mono text-slate-600 uppercase">NIS near 1 means honestly tuned</span>
</div>
<div id="n-filters" class="grid grid-cols-1 sm:grid-cols-3 gap-2 font-mono text-[10px]"></div>
</div>
<div class="glass rounded-2xl p-3">
<h2 class="text-sm font-bold mb-1.5">Compensators</h2>
<div id="n-comp" class="font-mono text-[10px] space-y-1"></div>
</div>
<div class="glass rounded-2xl p-3 lg:col-span-2 flex flex-col min-h-0">
<div class="flex items-baseline justify-between mb-1.5 shrink-0">
<h2 class="text-sm font-bold">Learner bank</h2>
<span class="text-[9px] font-mono text-slate-600 uppercase">18 independent RLS heads</span>
</div>
<div id="n-heads" class="flex-1 min-h-0"></div>
</div>
<div class="glass rounded-2xl p-3 flex flex-col min-h-0">
<div class="flex items-baseline justify-between mb-1.5 shrink-0">
<h2 class="text-sm font-bold">Feature attribution</h2>
<select id="n-head-sel" class="bg-slate-900/90 border border-slate-800 text-slate-300 text-[10px] font-mono rounded px-1.5 py-0.5"></select>
</div>
<p class="text-[9px] text-slate-600 font-mono mb-1 shrink-0">largest |&theta;| after standardisation, so these are comparable</p>
<div id="n-theta" class="flex-1 min-h-0 font-mono text-[10px] space-y-0.5"></div>
</div>
<div class="glass rounded-2xl p-3 flex flex-col min-h-0 gap-2">
<div>
<h2 class="text-sm font-bold mb-1">Detectors</h2>
<div id="n-mon" class="font-mono text-[10px] space-y-1"></div>
</div>
<div>
<h2 class="text-sm font-bold mb-1">Climatology</h2>
<div id="n-clim" class="font-mono text-[10px] space-y-0.5"></div>
</div>
<div>
<h2 class="text-sm font-bold mb-1">Precipitation</h2>
<div id="n-precip" class="font-mono text-[10px] space-y-0.5"></div>
</div>
</div>
</section>
<!-- ---------------- METHODS ---------------- -->
<section id="pane-methods" class="pane h-full min-h-0 gap-3 grid-cols-1 lg:grid-cols-5">
<div class="glass rounded-2xl p-4 lg:col-span-2 flex flex-col min-h-0">
@@ -414,6 +472,21 @@ DASHBOARD_HTML = r"""
<script>
const el = (id) => document.getElementById(id);
let lastDerived = {};
const esc = (t) => String(t).replace(/&/g,'&amp;').replace(/"/g,'&quot;')
.replace(/</g,'&lt;').replace(/>/g,'&gt;');
// KaTeX renders after the pane is populated. If the CDN is unreachable the
// raw TeX stays visible, which is ugly but still readable, rather than blank.
function typeset(root) {
if (typeof katex === 'undefined') return;
(root||document).querySelectorAll('.tex[data-tex],.tex-inline[data-tex]').forEach(n => {
if (n.dataset.done) return;
const display = n.classList.contains('tex');
try { katex.render(n.dataset.tex, n, {displayMode:display, throwOnError:false,
output:'html', trust:false}); n.dataset.done='1'; }
catch (e) { n.textContent = n.dataset.tex; }
});
}
const fmt = (v,d=1) => (v===null||v===undefined||Number.isNaN(v)) ? '--' : Number(v).toFixed(d);
const SEV = { info:'text-slate-400', warn:'text-amber-300', error:'text-rose-300' };
const tsFmt = (t) => new Date(t*1000).toLocaleString([], {month:'short',day:'numeric',hour:'2-digit',minute:'2-digit'});
@@ -482,6 +555,9 @@ function applyTelemetry(d) {
setFlash('l-lux', d.color ? String(d.color.clear) : '--');
const rt = d.rates||{}, dv = d.derived||{};
// Kept module-level so the precipitation panel can pick an icon from what
// the sensors actually see, not just from the barometric class.
lastDerived = Object.assign({}, dv, {temperature: d.temperature});
el('l-temp-rate').innerText = (rt.temperature_c_per_h>=0?'+':'')+fmt(rt.temperature_c_per_h,2)+' \u00b0C/h';
el('l-press-rate').innerText = (rt.pressure_hpa_per_h>=0?'+':'')+fmt(rt.pressure_hpa_per_h,2)+' hPa/h';
el('l-temp-raw').innerText = fmt(d.temperature_raw,1)+' / '+fmt(d.cpu_temp,0)+'\u00b0';
@@ -515,6 +591,12 @@ function applyTelemetry(d) {
function applyPrecip(p) {
if (!p || p.rain_probability===undefined) return;
el('c-label').innerText = p.label||'--';
// The barometer gives the class; the light sensor, sun angle and thermometer
// decide which glyph honestly represents it.
const wx = wxPick(p.condition, lastDerived.cloud_index, lastDerived.solar_elevation,
lastDerived.temperature, p.rain_probability);
el('c-icon').innerHTML = wxSvg(wx[0], 56);
el('c-sky').innerText = wx[1];
el('c-z').innerText = p.zambretti_z!==undefined ? p.zambretti_z : '-';
el('c-trend').innerText = p.pressure_characteristic||'-';
if (p.tendency!==undefined) el('c-tend').innerText = (p.tendency>=0?'+':'')+fmt(p.tendency,2)+' hPa/h';
@@ -878,6 +960,168 @@ el('m-hcalrst').addEventListener('click', async () => {
el('m-hcalstat').innerHTML = 'Reset to prior offset = <span class="text-cyan-300">'+r.offset+'%</span>';
});
/* ---------------- STATS FOR NERDS ---------------- */
let nerdDoc = null, nerdHead = null;
const HL = (h) => h<3600 ? (h/60)+'m' : h<86400 ? (h/3600)+'h' : (h/86400)+'d';
function bar(frac, colour) {
const w = Math.max(0, Math.min(1, frac||0))*100;
return '<div class="h-1 bg-slate-950 rounded-full overflow-hidden border border-slate-800/70">'+
'<div class="h-full" style="width:'+w+'%;background:'+colour+'"></div></div>';
}
async function loadNerd() {
const d = await fetch('/api/nerd').then(r=>r.json());
nerdDoc = d;
const COL = {temperature:'#f59e0b', humidity:'#06b6d4', pressure:'#a78bfa'};
el('n-filters').innerHTML = Object.keys(d.filters||{}).map(k=>{
const f = d.filters[k];
// NIS is chi-square(1) distributed when consistent, so 1 is the target and
// the bar is scaled to 3 as a "clearly wrong" ceiling.
return '<div class="bg-slate-900/50 rounded-lg border border-slate-800/70 p-2">'+
'<div class="flex justify-between mb-1"><span style="color:'+COL[k]+'">'+k+'</span>'+
'<span class="text-slate-500">'+(f.initialised?'':'warming')+'</span></div>'+
'<div class="flex justify-between text-slate-500">NIS<span class="text-slate-200 font-bold">'+fmt(f.nis,3)+'</span></div>'+
bar(f.nis/3, COL[k])+
'<div class="flex justify-between text-slate-500 mt-1">&sigma; level<span class="text-slate-300">'+fmt(f.sigma_level,4)+'</span></div>'+
'<div class="flex justify-between text-slate-500">P rate<span class="text-slate-300">'+f.p_rate.toExponential(2)+'</span></div>'+
'<div class="flex justify-between text-slate-600">q / r<span>'+f.q.toExponential(1)+' / '+fmt(f.r,3)+'</span></div>'+
'</div>';
}).join('');
const th = (d.compensators||{}).thermal||{}, hu = (d.compensators||{}).humidity||{};
el('n-comp').innerHTML =
'<div class="flex justify-between text-slate-500">k<span class="text-emerald-300 font-bold">'+fmt(th.k,4)+'</span></div>'+
'<div class="flex justify-between text-slate-600">P / n<span>'+fmt(th.P,3)+' / '+(th.n||0)+'</span></div>'+
'<div class="flex justify-between text-slate-600">clamp<span>'+fmt(th.k_min,2)+' .. '+fmt(th.k_max,2)+'</span></div>'+
'<div class="border-t border-slate-800 my-1"></div>'+
'<div class="flex justify-between text-slate-500">RH offset<span class="text-cyan-300 font-bold">'+fmt(hu.offset,2)+'%</span></div>'+
'<div class="flex justify-between text-slate-600">P / n<span>'+fmt(hu.P,3)+' / '+(hu.n||0)+'</span></div>'+
'<div class="flex justify-between text-slate-600">psychrometric<span>'+(hu.psychrometric?'on':'off')+'</span></div>';
const heads = d.heads||[];
el('n-heads').innerHTML = heads.length ? '<table class="w-full font-mono text-[10px]">'+
'<thead class="text-slate-600 uppercase text-[9px]"><tr class="border-b border-slate-800">'+
'<th class="text-left py-0.5">head</th><th class="text-right">n</th><th class="text-right">tr P</th>'+
'<th class="text-right">|&theta;|</th><th class="text-right">rmse</th><th class="text-right">&alpha;</th>'+
'<th class="text-right">cov</th><th class="text-right">&plusmn;</th><th class="text-right pl-2">p/c/l</th></tr></thead><tbody>'+
heads.map(h=>{
const sat = h.trace_p/h.p_max;
const cov = h.coverage===null||h.coverage===undefined ? '-' : Math.round(h.coverage*100)+'%';
const covCls = (h.coverage!==null && Math.abs(h.coverage-(1-h.alpha_target))<0.03) ? 'text-emerald-300' : 'text-amber-300';
const w = h.weights||{};
return '<tr class="border-b border-slate-800/40">'+
'<td class="py-0.5 text-slate-400">'+h.target.slice(0,4)+' <span class="text-slate-600">'+HL(h.horizon_s)+'</span></td>'+
'<td class="text-right text-slate-600">'+h.n_updates+'</td>'+
'<td class="text-right '+(sat>0.95?'text-rose-300':'text-slate-400')+'">'+h.trace_p.toExponential(1)+'</td>'+
'<td class="text-right text-slate-300">'+fmt(h.theta_norm,1)+'</td>'+
'<td class="text-right text-slate-300">'+fmt(h.rmse_ewma,3)+'</td>'+
'<td class="text-right text-indigo-300">'+fmt(h.alpha,3)+'</td>'+
'<td class="text-right '+covCls+'">'+cov+'</td>'+
'<td class="text-right text-slate-500">'+(h.halfwidth===null?'-':fmt(h.halfwidth,2))+'</td>'+
'<td class="text-right text-slate-600 pl-2">'+Math.round((w.persistence||0)*100)+'/'+
Math.round((w.climatology||0)*100)+'/'+Math.round((w.learned||0)*100)+'</td></tr>';
}).join('')+'</tbody></table>'
: '<p class="text-[10px] text-slate-600 font-mono">No heads trained yet.</p>';
const sel = el('n-head-sel');
if (sel.options.length !== heads.length) {
sel.innerHTML = heads.map((h,i)=>'<option value="'+i+'">'+h.target.slice(0,4)+' '+HL(h.horizon_s)+'</option>').join('');
}
if (nerdHead===null || nerdHead>=heads.length) nerdHead = 0;
sel.value = String(nerdHead);
drawTheta();
const mo = d.monitoring||{}, nv = mo.novelty||{}, dr = mo.drift||{};
el('n-mon').innerHTML =
'<div class="flex justify-between text-slate-500">Mahalanobis d&sup2;<span class="text-slate-200 font-bold">'+fmt(nv.d2,2)+'</span></div>'+
bar((nv.d2||0)/(nv.threshold||12), '#f59e0b')+
'<div class="flex justify-between text-slate-600">threshold / dims<span>'+fmt(nv.threshold,0)+' / '+(nv.dims||0)+'</span></div>'+
'<div class="flex justify-between text-slate-500 mt-1">Page-Hinkley m&#8314;<span class="text-slate-200 font-bold">'+fmt(dr.m_pos,3)+'</span></div>'+
'<div class="flex justify-between text-slate-600">m&#8315; / alarms<span>'+fmt(dr.m_neg,3)+' / '+(dr.alarms||0)+'</span></div>';
const cl = d.climatology||{};
el('n-clim').innerHTML =
'<div class="flex justify-between text-slate-500">history<span class="text-slate-300">'+fmt(cl.history_days,2)+' d</span></div>'+
'<div class="flex justify-between text-slate-500">harmonics<span class="text-slate-300">'+(cl.diurnal_harmonics||0)+' diurnal, '+
(cl.annual_terms?(cl.annual_harmonics||0):0)+' annual</span></div>'+
Object.keys(cl.residual_std||{}).map(k=>
'<div class="flex justify-between text-slate-600">&sigma; '+k.slice(0,4)+'<span>'+fmt(cl.residual_std[k],3)+'</span></div>').join('');
const pr = d.precipitation||{};
const co = (pr.coefficients||[]).slice().sort((a,b)=>Math.abs(b.weight)-Math.abs(a.weight)).slice(0,4);
el('n-precip').innerHTML =
'<div class="flex justify-between text-slate-500">labels<span class="text-slate-300">'+(pr.strong_labels||0)+' strong, '+(pr.weak_labels||0)+' weak</span></div>'+
'<div class="flex justify-between text-slate-500">logloss<span class="text-slate-300">'+fmt(pr.logloss_ewma,4)+'</span></div>'+
co.map(c=>'<div class="flex justify-between text-slate-600">'+String(c.feature||'?').slice(0,16)+'<span class="'+(c.weight>=0?'text-emerald-400':'text-rose-400')+'">'+
(c.weight>=0?'+':'')+fmt(c.weight,3)+'</span></div>').join('');
}
function drawTheta() {
if (!nerdDoc || !nerdDoc.heads || !nerdDoc.heads.length) return;
const h = nerdDoc.heads[nerdHead], names = nerdDoc.feature_names||[];
const pairs = (h.theta||[]).map((v,i)=>({n:names[i]||('f'+i), v:v}))
.sort((a,b)=>Math.abs(b.v)-Math.abs(a.v)).slice(0,11);
const mx = Math.max.apply(null, pairs.map(p=>Math.abs(p.v)).concat([1e-9]));
el('n-theta').innerHTML = pairs.map(p=>
'<div class="flex items-center gap-1.5">'+
'<span class="text-slate-500 truncate" style="width:46%">'+p.n+'</span>'+
'<span class="flex-1">'+bar(Math.abs(p.v)/mx, p.v>=0?'#34d399':'#fb7185')+'</span>'+
'<span class="'+(p.v>=0?'text-emerald-400':'text-rose-400')+'" style="width:22%;text-align:right">'+
(p.v>=0?'+':'')+fmt(p.v,3)+'</span></div>').join('');
}
loaders.nerd = loadNerd;
/* ---------------- WEATHER ICONS ----------------
Inline SVG, no icon font and no extra request. The glyph is not chosen from
the Zambretti class alone: the station also measures cloud index from the
light sensor and knows the solar elevation, so a "fine" barometer under a
thick overcast still draws a cloud, and after sunset it draws a moon rather
than a sun. Snow is picked on temperature, not on the barometer. */
const WX = {
sun: '<circle cx="32" cy="32" r="13" fill="#fbbf24"/>'+
'<g stroke="#fbbf24" stroke-width="4" stroke-linecap="round">'+
'<path d="M32 6v8M32 50v8M6 32h8M50 32h8M13 13l6 6M45 45l6 6M51 13l-6 6M19 45l-6 6"/></g>',
moon: '<path d="M40 10a22 22 0 1 0 14 40A24 24 0 0 1 40 10z" fill="#e2e8f0"/>'+
'<circle cx="20" cy="16" r="1.8" fill="#94a3b8"/><circle cx="14" cy="26" r="1.2" fill="#94a3b8"/>',
cloud: '<path d="M20 46a11 11 0 0 1 1-22 15 15 0 0 1 28 4 9 9 0 0 1-2 18z" fill="#94a3b8"/>',
partsun: '<circle cx="22" cy="22" r="9" fill="#fbbf24"/>'+
'<g stroke="#fbbf24" stroke-width="3" stroke-linecap="round">'+
'<path d="M22 4v6M4 22h6M9 9l4 4M35 9l-4 4"/></g>'+
'<path d="M26 50a10 10 0 0 1 1-20 13 13 0 0 1 25 4 8 8 0 0 1-2 16z" fill="#cbd5e1"/>',
partmoon: '<path d="M30 8a15 15 0 1 0 10 27A16 16 0 0 1 30 8z" fill="#e2e8f0"/>'+
'<path d="M26 52a10 10 0 0 1 1-20 13 13 0 0 1 25 4 8 8 0 0 1-2 16z" fill="#cbd5e1"/>',
rain: '<path d="M20 40a11 11 0 0 1 1-22 15 15 0 0 1 28 4 9 9 0 0 1-2 18z" fill="#94a3b8"/>'+
'<g stroke="#38bdf8" stroke-width="3.5" stroke-linecap="round">'+
'<path d="M22 46l-3 9M33 46l-3 9M44 46l-3 9"/></g>',
showers: '<path d="M20 40a11 11 0 0 1 1-22 15 15 0 0 1 28 4 9 9 0 0 1-2 18z" fill="#a3adbb"/>'+
'<g stroke="#38bdf8" stroke-width="3.5" stroke-linecap="round">'+
'<path d="M26 46l-2 7M39 46l-2 7"/></g>',
storm: '<path d="M20 38a11 11 0 0 1 1-22 15 15 0 0 1 28 4 9 9 0 0 1-2 18z" fill="#64748b"/>'+
'<path d="M34 36l-10 15h7l-3 12 12-17h-7l4-10z" fill="#fbbf24"/>',
snow: '<path d="M20 40a11 11 0 0 1 1-22 15 15 0 0 1 28 4 9 9 0 0 1-2 18z" fill="#cbd5e1"/>'+
'<g stroke="#e0f2fe" stroke-width="3" stroke-linecap="round">'+
'<path d="M22 48v8M18 52h8M33 48v8M29 52h8M44 48v8M40 52h8"/></g>',
};
function wxPick(condition, cloud, elevation, tempC, rainProb) {
const night = (elevation !== undefined && elevation !== null && elevation < -1);
const c = (cloud === undefined || cloud === null) ? 0.5 : cloud;
if (condition === 'stormy') return ['storm', 'Thunderstorms'];
// Snow is a temperature question, not a barometric one.
if (tempC !== undefined && tempC !== null && tempC <= 1.5 && rainProb > 0.35)
return ['snow', 'Snow possible'];
if (condition === 'wet') return ['rain', 'Wet and windy'];
if (condition === 'rain') return ['rain', 'Rain likely'];
if (condition === 'unsettled') return ['showers', 'Showers around'];
if (condition === 'changeable')
return c > 0.7 ? ['cloud','Cloudy'] : [night?'partmoon':'partsun', 'Partly cloudy'];
// settled / fine / fair: defer to what the light sensor actually sees.
if (c < 0.25) return [night?'moon':'sun', night ? 'Clear skies' : 'Sunny, open skies'];
if (c < 0.65) return [night?'partmoon':'partsun', 'Partly cloudy'];
return ['cloud', 'Cloudy'];
}
function wxSvg(name, size) {
return '<svg viewBox="0 0 64 64" width="'+size+'" height="'+size+'" aria-hidden="true">'+(WX[name]||WX.cloud)+'</svg>';
}
/* ---------------- METHODS ---------------- */
let methodsDoc = null, methodSel = 'acquire';
const STAGE_COLOUR = { acquire:'#94a3b8', compensate:'#34d399', kalman:'#f59e0b', features:'#06b6d4',
@@ -930,7 +1174,13 @@ function drawStage() {
'<div class="text-slate-600 uppercase text-[9px]">produces</div><div class="text-slate-300 mt-0.5">'+s.produces+'</div></div></div>'+
(s.math ? '<div class="bg-slate-950/60 rounded-lg border border-slate-800/70 px-3 py-2.5 overflow-x-auto">'+
'<div class="text-[9px] text-slate-600 font-mono uppercase mb-1">core relation</div>'+
'<div class="text-[11px] font-mono text-indigo-300">'+s.math.replace(/[{}\\]/g,' ').replace(/\s+/g,' ')+'</div></div>' : '')+
(Array.isArray(s.math)?s.math:[s.math]).map(m=>'<div class="tex" data-tex="'+esc(m)+'"></div>').join('')+'</div>' : '')+
(s.symbols ? '<div><div class="text-[9px] text-slate-600 font-mono uppercase mb-1">symbols</div>'+
'<div class="grid grid-cols-1 sm:grid-cols-2 gap-x-3 gap-y-1">'+
Object.keys(s.symbols).map(k=>'<div class="flex gap-2 items-baseline">'+
'<span class="tex-inline shrink-0" data-tex="'+esc(k)+'"></span>'+
'<span class="text-[11px] text-slate-500 leading-snug">'+s.symbols[k]+'</span></div>').join('')+
'</div></div>' : '')+
'<div><div class="text-[9px] text-slate-600 font-mono uppercase mb-1">why it is done this way</div>'+
'<p class="text-[12px] text-slate-300 leading-relaxed">'+s.why+'</p></div>'+
'<div class="border-l-2 pl-3" style="border-color:'+c+'66">'+
@@ -951,6 +1201,7 @@ function drawStage() {
'<div class="text-[9px] text-slate-600 font-mono uppercase mb-1.5">glossary</div>'+
methodsDoc.glossary.map(g=>'<div class="mb-2"><span class="text-[11px] font-semibold text-slate-300">'+g.term+'</span>'+
'<p class="text-[11px] text-slate-500 leading-relaxed">'+g.definition+'</p></div>').join('')+'</div>' : '');
typeset(el('me-body'));
}
loaders.methods = loadMethods;
@@ -960,7 +1211,9 @@ loadForecast();
loadOutlook();
fetch('/api/status').then(r=>r.json()).then(s => { el('hd-days').innerText = fmt(s.history_days,2); });
setInterval(() => { if (activeTab==='live') { loadForecast(); loadOutlook(); } }, 60000);
el('n-head-sel').addEventListener('change', e => { nerdHead = Number(e.target.value); drawTheta(); });
setInterval(() => { if (activeTab==='models') loadModels(); }, 60000);
setInterval(() => { if (activeTab==='nerd') loadNerd(); }, 30000);
setInterval(() => { if (activeTab==='history') loadHistory(); }, 300000);
</script>
</body>
ashvale/methods.py
+90 -14
View File
@@ -73,9 +73,22 @@ def pipeline(cfg) -> List[Dict[str, Any]]:
"failure": "A mistyped reference drives k to its clamp and stays there "
"across restarts, because state persists. The reset button "
"on the Models tab exists for exactly that.",
"math": r"k_{t} = k_{t-1} + \frac{P\varphi}{\lambda + \varphi P \varphi}"
r"\left[(T_{raw} - T_{ref}) - k_{t-1}\varphi\right],"
r"\quad \varphi = T_{cpu} - T_{raw}",
"math": [
r"T = T_{raw} - k\,(T_{cpu} - T_{raw}), \qquad k \ge 0",
r"\varphi = \max(T_{cpu} - T_{raw},\,0), \qquad "
r"y = T_{raw} - T_{ref}",
r"g = \frac{P\varphi}{\lambda + \varphi^{2} P}, \qquad "
r"k_t = \operatorname{clip}\!\big(k_{t-1} + g\,(y - k_{t-1}\varphi),\;"
r"k_{\min},\,k_{\max}\big)",
r"P_t = \frac{P_{t-1} - g\,\varphi\,P_{t-1}}{\lambda}",
],
"symbols": {
r"k": "self-heating coefficient, the one estimated parameter",
r"\varphi": "regressor: the CPU-to-sensor gradient, floored at zero",
r"P": "scalar parameter variance; large means uncertain, so large steps",
r"\lambda": "forgetting factor, 0.98. Old calibrations decay",
r"g": "RLS gain. Note it is the Kalman gain for a one-dimensional state",
},
"params": {"current k": f"{s.cpu_heat_k:g} (prior)",
"clamp": f"{s.cpu_heat_k_min:g} to {s.cpu_heat_k_max:g}"},
},
@@ -96,9 +109,26 @@ def pipeline(cfg) -> List[Dict[str, Any]]:
"failure": "Process noise too low and the filter lags real weather; too "
"high and you have an expensive passthrough. The innovation "
"statistic is logged so you can tell which.",
"math": r"x = \begin{bmatrix} \text{level} \\ \text{rate} \end{bmatrix},"
r"\quad Q = q\begin{bmatrix} \Delta t^3/3 & \Delta t^2/2 \\"
r"\Delta t^2/2 & \Delta t \end{bmatrix}",
"math": [
r"x = \begin{bmatrix} \text{level} \\ \text{rate} \end{bmatrix}, \qquad "
r"F = \begin{bmatrix} 1 & \Delta t \\ 0 & 1 \end{bmatrix}, \qquad "
r"H = \begin{bmatrix} 1 & 0 \end{bmatrix}",
r"Q = q\begin{bmatrix} \Delta t^{3}/3 & \Delta t^{2}/2 \\"
r"\Delta t^{2}/2 & \Delta t \end{bmatrix}"
r"\qquad\text{(continuous white-noise acceleration)}",
r"x^{-}_t = Fx_{t-1}, \qquad P^{-}_t = FP_{t-1}F^{\top} + Q",
r"y = z - Hx^{-}_t, \qquad S = HP^{-}_tH^{\top} + r, \qquad "
r"K = P^{-}_tH^{\top}S^{-1}",
r"P_t = (I - KH)P^{-}_t(I - KH)^{\top} + KrK^{\top}"
r"\qquad\text{(Joseph form, stays positive semi-definite)}",
r"\text{NIS} = y^{\top}S^{-1}y \;\approx\; 1 \text{ when the filter is tuned}",
],
"symbols": {
r"q": "process noise density. The only knob that really matters",
r"r": "measurement noise variance, from the sensor datasheet",
r"S": "innovation covariance: how surprised the filter expects to be",
r"\text{NIS}": "normalised innovation squared. Above 1 means overconfident and lagging",
},
"params": {"q temperature": f"{s.kalman_q_temp:g}",
"r temperature": f"{s.kalman_r_temp:g}",
"q pressure": f"{s.kalman_q_press:g}"},
@@ -147,9 +177,28 @@ def pipeline(cfg) -> List[Dict[str, Any]]:
"through quiet nights when the regressor barely moves, and "
"the model then detonates at sunrise. The trace is capped. "
"This is the most common way a field RLS deployment dies.",
"math": r"P_t = \frac{1}{\lambda}\left(P_{t-1} - "
r"\frac{P_{t-1}x x^{\top}P_{t-1}}{\lambda + x^{\top}P_{t-1}x}"
r"\right)",
"math": [
r"\hat{\theta} = \arg\min_{\theta}\; \sum_{i=1}^{t}"
r"\lambda^{\,t-i}\big(y_i - \theta^{\top}x_i\big)^{2}"
r"\qquad\text{(exponentially weighted least squares)}",
r"g_t = \frac{P_{t-1}x_t}{\lambda + x_t^{\top}P_{t-1}x_t}, \qquad "
r"\theta_t = \theta_{t-1} + g_t\big(y_t - \theta_{t-1}^{\top}x_t\big)",
r"P_t = \frac{1}{\lambda}\Big(P_{t-1} - g_t x_t^{\top} P_{t-1}\Big), "
r"\qquad P_t \leftarrow \tfrac{1}{2}\big(P_t + P_t^{\top}\big)",
r"\operatorname{tr}(P_t) > P_{\max} \;\Longrightarrow\; "
r"P_t \leftarrow P_t\,\frac{P_{\max}}{\operatorname{tr}(P_t)}"
r"\qquad\text{(the guard that stops covariance blow-up)}",
r"N_{\text{eff}} = \frac{1}{1-\lambda}"
r"\qquad\text{effective memory in samples}",
r"\hat{y}_{t+h} = y_t + \theta_h^{\top}x_t"
r"\qquad\text{each head predicts a delta, not a level}",
],
"symbols": {
r"\theta": "33 weights, one bank per (target, horizon): 18 banks",
r"P": "parameter covariance. Its trace is the total uncertainty",
r"\lambda": "forgetting factor 0.9985, about 55 hours of memory",
r"P_{\max}": "trace cap. Without it, quiet nights inflate P until sunrise detonates the model",
},
"params": {"forgetting": f"{m.rls_forgetting:g}",
"effective memory": _memory(m.rls_forgetting, m.grid_s),
"members": ", ".join(MEMBERS)},
@@ -171,8 +220,21 @@ def pipeline(cfg) -> List[Dict[str, Any]]:
"does underneath.",
"failure": "If coverage sits far from target, the feedback rate is "
"wrong, not the model. Both are shown on the Models tab.",
"math": r"\alpha_{t+1} = \alpha_t + \gamma\left(\alpha^{*} - "
r"\mathbb{1}[y_t \notin C_t]\right)",
"math": [
r"C_t = \big[\hat{y}_t - q_{1-\alpha_t},\; \hat{y}_t + q_{1-\alpha_t}\big], "
r"\qquad q_{1-\alpha} = \operatorname{Quantile}_{1-\alpha}\big(|e_i|\big)",
r"\alpha_{t+1} = \operatorname{clip}\Big(\alpha_t + \gamma\big(\alpha^{*} - "
r"\mathbb{1}\left[y_t \notin C_t\right]\big),\; 0.005,\; 0.75\Big)",
r"\frac{1}{T}\sum_{t=1}^{T}\mathbb{1}\left[y_t \in C_t\right] "
r"\;\xrightarrow[T\to\infty]{}\; 1-\alpha^{*}"
r"\qquad\text{without assuming exchangeability}",
],
"symbols": {
r"\alpha^{*}": "target miss rate, 0.10 for a 90% band",
r"\alpha_t": "working miss rate. It moves; the target does not",
r"\gamma": "adaptation rate. Larger reacts faster and wanders more",
r"\mathbb{1}[\cdot]": "1 when the truth fell outside the band, else 0",
},
"params": {"target coverage": f"{int((1 - m.conformal_alpha) * 100)}%",
"gamma": f"{m.conformal_gamma:g}",
"window": f"{m.conformal_window} residuals"},
@@ -196,9 +258,23 @@ def pipeline(cfg) -> List[Dict[str, Any]]:
"a 365-day sine to three weeks of data produces a "
"magnificent extrapolation straight off the edge of the "
"physical world.",
"math": r"y \sim \beta_0 + \beta_1 t + \sum_{k=1}^{3}"
r"\left[a_k\sin\tfrac{2\pi k t}{\text{day}} + "
r"b_k\cos\tfrac{2\pi k t}{\text{day}}\right] + \text{annual}",
"math": [
r"y(t) \approx \beta_0 + \beta_1 t + \sum_{k=1}^{K_d}"
r"\left[a_k\sin\frac{2\pi k t}{\tau_d} + b_k\cos\frac{2\pi k t}{\tau_d}\right]"
r" + \sum_{j=1}^{K_a}\left[c_j\sin\frac{2\pi j t}{\tau_a} + "
r"d_j\cos\frac{2\pi j t}{\tau_a}\right]",
r"\hat{\beta} = \big(X^{\top}X + \rho I\big)^{-1}X^{\top}y"
r"\qquad\text{(ridge, because harmonics get collinear on short records)}",
r"\hat{y}(t+h) = \underbrace{\mu(t+h)}_{\text{harmonic fit}} + "
r"\underbrace{\big(y(t)-\mu(t)\big)}_{\text{today's anomaly}}\cdot"
r"\,2^{-h/h_{1/2}}",
],
"symbols": {
r"\tau_d,\ \tau_a": "one day and one tropical year",
r"K_a": "annual harmonics, held at zero below 120 days of history",
r"\rho": "ridge penalty",
r"h_{1/2}": "anomaly half-life. Today's departure decays toward climatology",
},
"params": {"diurnal harmonics": "3", "annual harmonics": "2",
"anomaly half-life": "30 h"},
},
docs/DESIGN.md
+1 -1
View File
@@ -323,7 +323,7 @@ consecutive readings are the only tell.
| Symptom | Knob | Direction |
|---|---|---|
| Temperature reads consistently high | Calibrate from the Models and calibration tab, or `sensor.cpu_heat_k` | Raise |
| Temperature reads consistently high | Calibrate from the Models and Calibration tab, or `sensor.cpu_heat_k` | Raise |
| Humidity reads consistently off | Calibrate against a reference hygrometer, or `sensor.hum_offset` | Either |
| Readings over-smoothed, lag real change | `sensor.kalman_q_temp` | Raise |
| Rates look noisy | `sensor.kalman_q_*` down, or `kalman_r_*` up | |