mirror of
https://github.com/lynchaos/ashvale-station.git
synced 2026-09-12 20:52:23 +00:00
b3193cc151b24c979c21b2ffac4af1ad24075798
34
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 kemal
|
b3193cc151 |
Earn the blend weights and the intervals from forecasts, not from refits
fit() called learn(), and learn() updated three things: the RLS, the conformal
calibrator and the Hedge weights. Only the first belongs to a refit. The comment
above that loop already said so, and was wrong about what the code did.
Measured on 8.2 days of the live station, the 15 minute head had taken 977,078
Hedge updates from 758 distinct supervised pairs, a factor of 1,289, and the
1 day head 296,715 from 12 pairs, a factor of 24,726. A refit is not an outcome.
It is the same week of weather being read again, once every seven minutes.
Hedge is multiplicative, so an edge far too small to be real compounds to
certainty: twelve of twelve temperature and humidity heads had collapsed onto
climatology at a weight of 0.991 or above, while their own member_mae said the
members were within a few percent of each other. The ACI integrator moves by
gamma per observation, so it had likewise pinned against its clips, leaving the
6 hour temperature band (1.571 C) narrower than the 3 hour one (2.258 C), and
pressure at 1 day covering 3 of 7 with alpha jammed at the 0.005 floor.
Three changes, because fixing only the first would freeze the weights forever:
- fit() calls refit_step(), which touches the regression and nothing else.
The climatology and setpoint members were evaluated in that loop purely to
feed the Hedge update, so fit() no longer needs a climatology or a
setpoint_fn at all.
- verify() feeds observe_outcome() with the member predictions the forecast
was actually blended from. These are now written to the forecasts table at
issue time, because the learned member cannot be recovered afterwards: the
RLS has moved on.
- a matured forecast teaches exactly once. It stays readable for an hour so
the scorecard can aggregate a rolling window, which meant verify() was
feeding the calibrator the same outcome about twelve times.
The Hedge weights additionally decline an outcome that overlaps the last one
they took, which is the stride rule from fit() applied on the scoring side.
Forecasts are issued every retrain tick, so at the 1 day horizon roughly two
hundred a day resolve against very nearly the same outcome. The conformal window
absorbs that, a quantile over duplicated scores being merely overconfident about
its sample size, but exponentiated gradient cannot.
Walk-forward over the full 8.2 day record, against the current code:
mean MAE 0.856 (0.938 over the second half alone)
heads improved 17/18
beats persistence 7/18 -> 12/18
coverage |dev from .90| 0.188 -> 0.060, second half 0.112 -> 0.041
Decimating the conformal feed as well was measured and rejected. It reads better
(coverage |dev| 0.023) and is not: three heads fall below MIN_SCORES, drop to
1.645*sigma, and "cover" with a median band of +/- 107% relative humidity. At
h/4 and h/8 it never starves and lands within noise of not decimating at all, so
the simpler rule wins. Honest regressions: humidity at 12 hours is 19% worse,
and pressure past 6 hours is still under-covered, because at 1 day the point
forecast is genuinely poor and ACI can only widen so far.
A state file from before this change has its weights, member_mae, n_scored and
alpha reset on load. They are products of the replay, they are not evidence, and
they do not decay on their own: Hedge needs about twenty independent outcomes to
climb off its 1e-4 floor and the 1 d head sees one a day. The conformal scores
are kept, being residuals of roughly the right size, and the window refreshes
within about two days.
Schema migration verified against a pristine copy of the live database: 1,437
forecast rows preserved, five columns added, idempotent across restarts.
|
||
 kemalandClaude Opus 5
|
8515f4fb03 |
Treat the board being moved as a regime change, from the fused IMU attitude
The accelerometer and gyroscope were logged and never used. They measure nothing about weather, but they do measure the one thing about this station that nothing else can see: whether the sensor is still where it was. Measured over four and a half days on the real station, four genuine movements each stepped the temperature by a median of 1.02 C, against an ordinary fifteen minute change of 0.107 C with a 95th percentile of 0.841. A move therefore lands past the 95th percentile of normal variation. The heads carry about 55 hours of memory, so an undeclared move contaminates two days of training with a discontinuity they will try to fit rather than ignore. This now gets the same treatment set_environment gives a window being opened, because it is the same event: the coupling between the sensor and what it is measuring changed, and nothing in the data says so. Three choices in here were made by measurement, and the obvious one was wrong. Raw accelerometer looks like the natural input and is not. Over the same record a gravity-vector detector fires 112 times against this one's 4, because RTIMULib's gyro fusion removes exactly the desk vibration a bare accelerometer picks up. The fused pitch and roll have a p99 sample-to-sample noise of 0.0001 degrees, so a one degree trigger carries four decades of headroom. Yaw and compass are excluded. They are the only attitude outputs that depend on the magnetometer, and indoors the magnetometer is measuring the building. RTIMULib restarts its fusion from a default attitude when SenseHat is reconstructed, which put an 18 degree step in the record on every one of this station's seven service restarts. Without a settle window every deploy would queue a retrain. 300 seconds rather than 180: one artifact appeared three minutes after a restart, still converging. Replayed against the full record the detector finds 4 genuine movements and leaks 0 artifacts. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
kemalandClaude Opus 5
|
d5b3cdad7c |
Joystick labelling, a clock guard, and throttle logging
Three things the hardware offers that the code ignored. The joystick has never had a line of code. Left records a dry label, right a wet one, middle cycles the LED scene, and a full-panel flash acknowledges the press because a headless box gives no other sign and a button you cannot tell worked gets pressed twice. Precipitation is the weakest head in the bank and strong labels are its binding constraint: this station has 80 of them against thousands of proxy ones, entirely because the only label control lives in a web page, and a web page is not where anyone is standing when it starts raining. The board has no RTC, so a power cut without a network gives a clock somewhere in 1970 on the next boot. Solar elevation, the diurnal harmonics and a sample's position on the 5-minute grid then all lie with complete confidence, and unlike a gap in the record the damage cannot be identified afterwards. train() now refuses a clock below 2025 or one that has stepped behind the newest stored row, and logs the refusal rather than training on fiction. Undervoltage and thermal capping both shift the SoC temperature, which is the regressor in the self-heating compensation, so a weak power supply presents as an unexplained temperature bias rather than as anything resembling a power problem. get_throttled is now sampled hourly and logged when set. Measured and deliberately not done: colour features. r, g and b are logged and 74% of rows carry usable colour, but adding blue/red, green/red and saturation made MAE 1.50% worse and helped in only 13 of 72 cases. Three more regressors on a 33-feature model whose longest horizon trains on 13 independent pairs is straightforwardly overfitting. That also prompted a sweep of the RLS prior and forgetting factor in both directions; delta = 100 with lambda = 0.9985 is a local optimum on both axes, so neither moved. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
kemalandClaude Opus 5
|
2b2a3b7741 |
Learner hygiene: Hedge loss scale, look-ahead residual, conformal minimum
Three changes to ForecastHead and the conformal calibrator, each measured walk-forward on real data over seven train splits. Hedge normalised its losses by the current sample's worst loss, so on a quiet step where every member agreed to within 0.01 C whichever happened to be worst still took the full exp(-eta) penalty, exactly as if it had been wrong by 5 C. The regret bound assumes a fixed loss range, not a per-sample one, and the symptom was weights that jumped around with no relation to horizon. Normalising by the running member MAE instead is worth 1.81% of MAE, better on 106 of 126 heads, coverage unchanged. The residual handed to the conformal calibrator was computed after this sample's loss had already moved the weights, so it was better than anything the forecaster could produce and the intervals were calibrated about 2% too narrow. Coverage survived only because ACI notices the extra misses and reopens the band, a correction that should never have been needed. Scoring the blend with the pre-update weights leaves MAE untouched, as it must, and widens the intervals 2% to the honest width. The conformal quantile refused to produce a band below 20 scores. That number is arbitrary: the (1-alpha) empirical quantile is the ceil((k+1)(1-alpha))-th of k order statistics, so alpha = 0.10 needs 9. The 20 became actively harmful in the previous commit but one, because striding pairs by the horizon leaves a long-horizon head about 13 scores per refit. Twelve of eighteen heads therefore fell through to 1.645*sigma with sigma from an unconstrained x'Px, giving bands of +/- 45 C and +/- 115% relative humidity on a young station. Those cover, by being absurd, which is why the backtest never flagged them: a long walk-forward passes 20 scores early and never looks back. After the change all eighteen heads have a band from the first fit, +/- 3.1 C and +/- 7.7% in the same place. Also measured and deliberately not done: adding the Kalman level variance to the predictive spread. It moves sigma by 0.06% at the shortest horizon and 0.00% everywhere else, so the plumbing to carry it through three files buys nothing. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
kemalandClaude Opus 5
|
0672ee80ec |
Take model tuning from config too, not from the saved state
The companion to the Kalman fix. RecursiveLeastSquares.from_dict and AdaptiveConformal.from_dict restore lambda, delta, alpha, gamma and the conformal window alongside their data, and load_dict replaces the config-built heads with those, so every one of those knobs was immutable on any station that already had state. Editing config.yaml and restarting looks exactly like a change with no effect, which is the failure mode that cost real time on the Kalman side of this before it was found. Only the estimate is state now. Weights, covariances and conformal scores are restored; tuning is re-applied from config on every load. The conformal deques are rebuilt when the configured window changes, preserving their contents. load_dict also skips heads for a target or horizon this build no longer has, rather than resurrecting them from a stale file. Found while implementing a damped-trend ensemble member, which was then abandoned: see the following note. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
kemalandClaude Opus 5
|
2e6d84f17d |
Take Kalman tuning from config, not from the saved state
q and r were written into station_state.json and restored over the configured values, so tuning was effectively immutable in the field. This was found the expensive way: the retune in the previous commit was deployed, the service restarted cleanly, and the filters carried on with q = 2e-6 because that is what the state file said. Measured median rate afterwards was 14.4 C/h against 12.4 before, which is to say nothing happened. Only the estimate is state. x, P and initialised are restored; q and r now come from config every time. P may be momentarily inconsistent with a changed q, which costs a few hundred samples of reconvergence and is far cheaper than a configuration change that appears to work and does not. load_dict also now skips filters this build no longer has, rather than resurrecting them from an older state file. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
kemalandClaude Opus 5
|
6501652e83 |
Match the simulator's barometer noise to a real LPS25HB
The generator added 0.05 hPa of measurement noise where a real board shows 0.0224 hPa as the sd of the change between 30 s samples. Twice the noise flatters any smoother tested against it and understates the skill available at short lead, which is where the pressure heads were already weakest. Correcting my own earlier claim: I reported the simulator's pressure as 60x too noisy. That figure came from comparing the old local database, whose rows are hourly-tier aggregates spanning 251 days, against 30 s rows from the station. Generated like for like at matched cadence the gap is 1.9x, not 60x, and the simulator was never the blocker on pressure work that I described. A gap remains after this change: press_slp still steps 0.0415 hPa per 30 s against the station's 0.0224. That residue is the synoptic OU process moving faster than Cambridge did over these four days, which is a weather-realism question rather than a sensor one. Four days of one room is not enough to retune a synoptic model against, so it is left alone and recorded here. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
kemalandClaude Opus 5
|
f48cd61c10 |
Retune the Kalman process noise, and fuse the two thermometers
Two changes to the same signal path, one large and one small. The large one: all three filters were tuned to track one to three decades faster than their signals move. In a still room the temperature filter reported a median rate of 12.4 C/h while the air moved 0.4 C/h, and it overshot a real -36 C/h event by 77%. Sweeping q against the RMSE of the reported rate versus the true rate, using noise measured on the board (temperature 0.088 C, pressure 0.022 hPa, humidity 0.40 %): temperature 6.45 -> 0.37 C/h RMSE 2e-6 -> 1e-9 pressure 2.15 -> 0.24 hPa/h RMSE 1e-5 -> 1e-8 humidity 27.94 -> 3.55 %/h RMSE 5e-5 -> 2e-8 Tracking does not suffer. Lag against a genuine 2 C/h ramp is 0.003 C at both the old and new values, and the peak response to a five-minute event moves closer to the truth rather than further from it, because the overshoot goes away. What is given up is response to sub-minute transients, which for a station forecasting fifteen minutes to a day ahead is noise to reject. This matters most for pressure, whose tendency drives the precipitation forecast, and which was the worst tuned of the three. config.yaml shadowed kalman_q_temp, so editing the dataclass alone changed nothing. All six values are now listed there with that hazard spelled out, because a silent shadow cost real time here. The small one: temp_raw was the plain average of two thermometers whose white-noise sds differ by 7x (LPS25HB 0.007 C, HTS221 0.049 C), which throws the quiet one away. Inverse-variance weighting cuts the raw noise 3.5x. The trap is that the chips do not agree. They sit at different distances from the SoC and stand about 1.3 C apart, so weighting by variance alone drags temp_raw 0.48 C onto the LPS25HB, which after the 1.55x gain of the inverse compensator is 0.75 C of silent bias on every reading, since k was fitted against the mean of the two. The gradient is therefore tracked and removed before weighting and only the deviations are fused: measured mean shift 0.0001 C, noise still 3.5x lower. The tracked gradient is retained because it is a second observation of self-heating. Also corrected: the earlier claim that the HTS221 was the quieter channel was wrong, taken from twelve samples at a cadence slow enough that real drift dominated. At 0.5 s over 120 samples the LPS25HB is quieter by 7x and takes 98% of the weight. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
kemalandClaude Opus 5
|
f873b0b09d |
Stride training pairs by the horizon instead of by the grid row
fit() trained every head on every consecutive grid row. At the 1 d horizon on
a 5-minute grid adjacent pairs share 287 of their 288 samples, so the filter
was handed the same outcome 288 times and RLS with forgetting read each one as
fresh evidence:
horizon steps overlap independent events in a 400-score window
15m 3 66.7% 133.3
1h 12 91.7% 33.3
3h 36 97.2% 11.1
6h 72 98.6% 5.6
12h 144 99.3% 2.8
1d 288 99.7% 1.4
The day-ahead head was therefore fitted on roughly two independent outcomes by
a filter carrying 667 updates of memory, and its interval was a 90th percentile
of a sample of size one.
This is not a compute shortcut that trades accuracy for speed. Measured
walk-forward on four days of real station data and averaged over five train
splits, striding improves every horizon past fifteen minutes:
15m +0.6% 1h -12.2% 3h -31.7% 6h -33.3% 12h -39.5% 1d -14.4%
with coverage unchanged at 87 to 92%, and the fit 11.6x faster. The redundancy
was not merely wasted work, it was collapsing P onto the one direction the
repeated sample excited.
The stride phase rotates each refit and is persisted, so a long-lived station
eventually trains on every offset rather than seeing one sample in 288 forever,
and a restart does not pin it to phase 0. A floor relaxes the stride when a
long horizon on a short record would otherwise yield one or two pairs; 12 was
chosen by sweeping it across five splits rather than picked.
Single-split runs showed 10 to 17% regressions at the 1 d horizon that moved
with the parameter. Averaging over five splits removed them, which is the
expected result for a head fitted and scored on under two independent
outcomes. That horizon cannot be evaluated on a four-day record and was not
tuned against.
Incidentally, this also retires the parallel-retrain idea: the Pi's 42 s
retrain becomes a few seconds, and multiprocessing inside a 280 MB cap buys
nothing for a job that short.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
kemalandClaude Opus 5
|
154071a791 |
Log both Sense HAT thermometers, and migrate schemas that predate them
The board carries two independent thermometers and the code averaged them into temp_raw without ever recording either. Measured over 12 samples on a real station: HTS221 30.973 C at sd 0.060, LPS25HB 29.810 C at sd 0.443, a standing gradient of 1.163 C with the SoC at 44.55 C. Two things follow from that and neither is possible without the raw channels. A plain average of a quiet sensor and one seven times noisier lands at sd 0.223 where inverse-variance weighting reaches 0.060, and the gradient between two chips at different distances from the SoC is a second observation of self-heating that could identify the compensator's k with no reference thermometer. Both need history, and history cannot be backfilled, so the columns land on their own ahead of the work that consumes them. CREATE TABLE IF NOT EXISTS is a no-op against a table that already exists, so adding to COLUMNS would have reached a fresh install and silently missed every station already running, then surfaced as an OperationalError inside insert_telemetry. That sits on the sample loop, so it takes a station down rather than leaving a gap. Store now reconciles the table against COLUMNS on open, which makes every future column addition safe rather than just this one. The simulator gains the same two channels, with couplings solved so their forward models average to exactly the k = 0.55 the compensator is tuned against. Aggregate behaviour is unchanged; only the per-channel detail is new. Simulated temp_raw noise does rise from 0.05 to 0.223, which is not a regression but the end of an over-optimistic figure: it was modelling the quiet sensor and calling it the average. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
kemal
|
485affe956 |
Fix the runaway forecasts: refits accumulated, and annual terms fitted too early
Reported from a real station after 1.5 days: a six hour temperature forecast of 53 C in a 24 C room, and 9 C at one day, both carrying a plus or minus of 0.43. Confidently wrong is the one failure this project is supposed to refuse. Root cause. fit() replayed history into the live RLS on every retrain tick and never reset, so 453 grid rows had produced 64,676 updates in a day and a half. RLS with forgetting reads every update as fresh evidence, so the model believed it had a hundred times the data it had: P collapsed, in-sample error looked excellent, and the weights drifted without bound in directions the data never excited. Measured: cond(P) 3.1e9 and ||theta|| 1680 against a median |theta| of 1.67. A refit now starts from the prior, which makes retraining idempotent. Across 25 refits on the real data ||theta|| holds at 11.35, drifting 0.03, where before it grew without limit. The two largest weights were sin_doy and cos_doy at +1174 and +1191. Annual harmonics were in the design matrix from the first sample, where they are near-constant, near-collinear with each other and with the bias, and a rank-deficient regressor is what RLS answers with enormous cancelling weights. They are now held at zero until the record spans the same 120 days the climatology fit already requires, because a day and a half of data says nothing whatsoever about the season. Also raised the standardiser's variance floor from 1e-8, which only caught a bit-exactly constant column, to 1e-3. A feature that merely barely moves was being divided by its own noise. The conformal calibrators and Hedge weights are deliberately not reset by a refit: those are earned from scored forecasts, not from this regression. Backtest unchanged within noise, coverage still 89 to 91 across all 18 heads. Four regression tests added, including that refitting the same history twice must give the same model. |
||
|
kemal
|
bd233d7b27 |
Fix image build: register static qemu with the F flag
Second failure was debootstrap dying with 'E: Unable to execute target architecture'. qemu-user-binfmt registers a dynamically linked emulator, which cannot run once debootstrap chroots into a rootfs that has no loader for it. The fix is the static binary registered with the F flag, which preloads the interpreter so it survives the chroot. Registration runs after apt, because installing binfmt-support re-registers the dynamic handlers over the top, and the workflow now asserts the F flag is present rather than discovering its absence an hour into a build. |
||
|
kemal
|
4eefd8816b |
Fix image build: pi-gen wants qemu-user-binfmt, not qemu-user-static
The first CI build failed in under a second with 'Required dependencies not installed: qemu-user-binfmt'. pi-gen checks for the binfmt handler rather than the emulator binary, and I had guessed the package name. The list now comes from pi-gen's own depends file at the pinned commit, and the workflow asserts the aarch64 handler is registered before spending an hour discovering it is not. |
||
|
kemal
|
34f62222e1 |
Prebuilt Pi image and a one-line installer
Two routes onto a Pi. The image for a blank card, the script for a board that already works, which is most of them. deploy/install.sh installs apt dependencies, clones to /opt/ashvale, builds a venv with --system-site-packages, enables I2C and installs a systemd unit. Tested end to end on the real Zero 2 W by installing a second instance on port 8099 alongside the live station: both served, and the live station was untouched throughout. That test earned its keep twice. The installer first declared success while the service was crash-looping on a port clash, because systemd reports active for the instant between exec and the first failed bind; it now polls the HTTP endpoint instead, which is the only check that means anything. And my first attempt to verify that fix was itself worthless, because curl on 127.0.0.1:8000 was answered by the live station rather than the instance under test. deploy/pi-image is a pi-gen stage on Raspberry Pi OS Lite, Trixie, arm64, which is exactly what the board runs. Built by .github/workflows/image.yml against a pinned pi-gen commit, so the artifact does not move when an upstream branch does, and published to Releases where the 2 GB asset limit comfortably fits a Lite image. The image ships no password, no WiFi and no SSH host keys. Baked host keys would give every person who flashed it the same identity and make them trivially impersonable on their own network. Coordinates default to Greenwich at 0 m, wrong for everybody on purpose, because a plausible wrong altitude quietly biases the sea-level reduction on every row. The source is copied through a .gitignore filter rather than a hand-written exclude list, and that is a security property rather than tidiness: the hand-written list I wrote first missed HANDOVER.md, which is gitignored precisely because it contains LAN addresses and SSH details. Verified: 69 files, no state, no local notes, all essentials present. |
||
|
kemal
|
372ea4f067 |
Self-correcting layout density, and no-cache on the dashboard
I claimed 20/20 clean and was over-stating it: that was a fixed list of viewports, and a fixed list cannot cover a user's zoom level, a larger default font, or simply more accumulated history than a panel was designed around. All three change how much space a card needs. I could not reproduce the reported clipping at any width or height I tried, so rather than keep guessing at viewports I made the failure impossible by construction. The page now measures itself after every layout change and on resize: if any card's content escapes it, density escalates to compact, and if that still is not enough the height lock is released and the page scrolls. Content is never silently clipped, whatever the viewport. Scrolling beats hiding. The escalation is exercised, not dead code: across 34 viewports it used compact on 3 and scroll on 2, and at 125% zoom it used both. Also added Cache-Control: no-cache to the dashboard route. The page is generated from live config and changes every deploy, but carried no cache headers, so a browser could hold an old copy indefinitely and show layout bugs that were already fixed. That is a plausible reason a fix can look like it did not land. Verified: 34 viewports from 1920x1080 to 320x568, six tabs each, 204 tab renders, zero card overflow and zero horizontal overflow; plus five zoom levels from 100% to 200%. |
||
|
kemal
|
ca1d6ba528 |
Fix cards whose content overflowed them, and the audit that missed it
Reported on a 13 inch screen: the Detectors column and the Learner bank spilled past their cards. My earlier sweep called those tabs clean because it only checked content escaping the viewport, never content escaping its own card, and the cards are overflow-hidden so it was invisible rather than obviously broken. The worst was not either of those. The Station log was discarding up to 299 px of entries behind overflow-hidden with no indication, at every viewport. It now binary-searches the number of rows that fit and says how many are not shown. The count is searched rather than divided out of an assumed row height, because a long detail line wraps to two lines and no constant knows that. It runs from a ResizeObserver rather than once: measuring at call time reads a stale clientHeight, since the pane has only just become visible and Chart.js resizes its siblings 40 ms later, which fitted 13 rows into a box that holds 10. Also: denser learner and attribution rows on short screens, three precipitation coefficients instead of four, and a lower floor on the tendency chart. Two mistakes of mine on the way, both worth recording. I declared a second const rows inside loadModels, which is a parse error that killed the entire dashboard script; the audit reported it as a flood of sparkline layout faults because it was not listening for page errors, and it now fails loudly on them. And the min-height override did nothing at first because the Tailwind CDN injects its sheet after this style block, so min-h-[42px] won at equal specificity. Verified: 20 theme x viewport combinations, 120 tab renders, from 1920x1080 to 375x667 in both themes. Zero card overflow, zero horizontal overflow, zero console errors. |
||
|
kemal
|
aea740e85b |
Light and dark themes with a three-way selector
Auto (follow the system), light, or dark. Auto is a real preference rather than the absence of one, so it tracks prefers-color-scheme live rather than only at load. Stored in localStorage, not the station's settings overlay: a theme belongs to the screen you are reading on, not to the weather station. Control in the header for one click and in Settings for the explicit three-way choice. The markup is dark-first Tailwind utilities. Adding a dark: variant to several hundred class attributes would have been a large diff with a large blast radius, so light is an overlay that remaps the slate scale and the accent hues under [data-theme=light]. The dark path is byte-identical: nothing is re-specified unless the attribute is set. Chart.js keeps its own copy of every colour and cannot follow a CSS class change, so the charts read the same tokens from the computed style and are re-rendered on switch. Three things the screenshots and measurements caught that reading would not: - Accent text is the 200-400 shades, picked to glow on black. On white they wash out. Remapped to the 600-700 shade of the same hue so the colour coding survives. - My first remap listed exact class names and silently missed every opacity variant, because text-amber-200/60 is a different class. Measured at 1.02:1. Now matched on the hue prefix. - The value-changed flash is a pale indigo that dropped a headline reading to 2.06:1 on white for half a second, which is precisely when you are looking at it. Now theme-aware. Contrast measured with alpha properly composited to the page ground: light 212 nodes below WCAG AA against dark's 208, so light is no worse than the dark theme it mirrors. The remainder is the design's deliberately quiet tertiary text, present in both. Layout re-verified: six tabs across three viewports in both themes, twelve combinations, zero clipping and zero console errors. |
||
|
kemal
|
5fe309a8a6 |
Responsive pass: fix the squeezed chart, mobile navigation, and overflow
Three real faults, all measured rather than guessed. The forecast chart was unreadable on a 13 inch laptop. A 1280x800 screen has desktop width but 280 fewer vertical pixels, and the fixed rows (stat cards, outlook, diagnostics) took that entirely out of the one row meant to flex. Measured: 404 px tall at 1920x1080, 124 px at 1280x800, 92 px at 1024x768. Height-aware media queries now compact the furniture instead, and the chart is 201 px at 1280x800. Mobile navigation did not exist. At 375 px all six tabs were clipped with no affordance they were there, so five of six sections were unreachable. Below 1024 px the tab row is now a native select on its own full-width line, showing the current section rather than a bare chevron. Horizontal overflow on phones: the shell measured 452 px inside a 375 px viewport. Flex and grid children default to min-width:auto, so a long subtitle refused to shrink and dragged the layout wider than the screen. Also restructured the conditions column. It was a fixed stack that simply overflowed its own card on short screens, and shrinking each piece by media query chased the symptom; it is now a flex column with exactly one flexible child, so the tendency chart absorbs the slack and the column fits at any height. And the no-scroll contract gained a height floor: below 700 px tall the page scrolls, because at 1024x600 the chart had collapsed to 1 px and a scrollbar is the better answer. Verified across ten viewports from 1920x1080 to 375x667, six tabs each: 60 combinations, zero clipping, zero overflow, zero console errors. |
||
|
kemal
|
4cca40388f |
Heated environment: a thermostat member in the forecast ensemble
A room held at a setpoint is a different process from one left to drift. It is
a closed loop, and persistence, the baseline everything here is scored against,
is the wrong statement about it: the truth is not that it stays where it is, it
is that it returns to the setpoint.
So site.heating adds a fourth ensemble member, first order because that is what
a controlled system is:
dT_set(h) = (T_set - T_now) * (1 - exp(-h / tau))
Humidity follows and is the part that is easy to get wrong. Heating adds no
moisture, so vapour pressure is conserved and not relative humidity:
RH(h) = RH_now * es(T_now) / es(T_now + dT_set(h))
Warm the air and RH falls although nothing was dried, which is why a heated
house in winter is dry. The test asserts the dew point is unchanged to 1e-6.
Pressure gets zero: a thermostat cannot move the synoptic field.
Offered, not imposed. Hedge scores this member on realised error like any
other, so a wrong tau or a stale setpoint costs accuracy and gets down-weighted
rather than quietly biasing every forecast. Verified: on history with no
heating the ensemble assigned it weight 0.000. With heating off it returns zero
and is identical to persistence.
Going from three members to four means old saved heads must migrate.
from_dict reinitialises weights and member_mae. I missed member_mae first time
and it did not fail on load, it failed later inside learn() on a broadcast
error, which is a much worse place to find out; the migration test now covers
both and calls learn() to prove it.
Settings tab gains the toggle, setpoint and time constant. Turning heating on
or off is treated as a regime change like a door: discontinuity marker plus a
queued retrain.
|
||
|
kemal
|
498b3f6e38 |
Settings tab
Everything that was previously a curl command now has a surface: surroundings, site geometry, the matrix, the psychrometric flag, and the maintenance actions. Changes persist to data/state/settings.json, not config.yaml. That file is hand-annotated and hand-edited per station, and rewriting it from an API would destroy the comments and risk clobbering something the owner set. The overlay is applied last in load_config, so a dashboard change beats both the file and the environment, and deleting the overlay reverts everything. Written atomically via a temp file so a crash cannot truncate it. Every field applies live. A settings page that needs a restart is one people stop trusting, so site geometry is re-read per sample, the compensator flag is set on the live object, and the display picks up its rate the next frame. Two deliberate frictions. Selecting a surroundings pill only stages it: nothing is recorded until you press the button, because that writes a discontinuity marker and queues a retrain. And changing altitude or the psychrometric flag says outright that the stored history is now inconsistent and offers the re-derive, rather than leaving a silent mismatch. Verified in a browser: pills stage and apply, the toggle round-trips, re-derive ran 6201 rows in 0.29 s from the button, all six tabs report zero scrollbars and zero clipping, zero console errors. |
||
|
kemal
|
50b29f8077 |
Readout scene, environment regime tracking, and a Kalman cadence bug in recompute
recompute replayed the Kalman over stored rows at their own spacing while q stays tuned for the live 2 s cadence. Q scales with dt^3, so at the 30 s persist interval the process noise was 3375x too large and the filter tracked noise instead of smoothing: it wrote indoor temperature rates of +/-20 C/h into the history. This is the exact trap DESIGN.md section 2 documents for simulate.py, which does scale q, and I walked into it anyway. Now rescaled per step, because tiering means the stored cadence is not constant. Mean |rate| on the real board dropped to 2.73 C/h; what remains above 10 is the filter's warm-up transient in the first four samples, which is honest. Readout scene puts the actual numbers between the animations: temperature, humidity, sea-level pressure and the signed three hour forecast, each in its channel colour, scrolling. Text is drawn whole-pixel on purpose. Everything else here is sub-pixel and that is what makes it look good, but splitting a 3 px glyph across two columns halves its peak and smears it illegible. Crisp beats smooth when the thing has to be read. site.environment and site.enclosure record where the sensor lives and what has changed around it, with POST /api/environment to change them at runtime. This is not cosmetic: closing a door changes how strongly the sensor couples to outside, which is a regime change in the process the heads are fitting, and at lambda 0.9985 they carry about 55 hours of memory. Left alone they keep predicting the old room for two days. Page-Hinkley would notice eventually but needs matured forecasts to do it, which at the long horizons is the same two days. So the endpoint marks a discontinuity and queues a retrain. |
||
|
kemal
|
23cd76c96e |
Fix: the weather glyphs never appeared, and day/night was inverted
Two bugs, both found by checking the real board rather than the test harness. The glyphs never showed. _pick_glyph gated each one behind narrow conditions and returned None otherwise, so on the Pi's actual state (27.3 C, rain probability 0.024, condition settled, solar elevation -20.3) nothing qualified and the panel silently fell back to the ambient scenes every time. A forecast symbol is the default, not an exception, so it now always returns one of the three: cold wins, then wet, then fair. Day and night were inverted. night = _smoothstep(2.0, -8.0, elev) passes a descending range, and _smoothstep treated edge1 <= edge0 as a degenerate step returning the opposite of the intent, so the panel drew a moon at midday and a sun at midnight. Caught by rendering it and looking, not by reading it. _smoothstep now handles descending ranges, and only the degenerate equal-edge case takes the step branch. The fair-weather glyph also needed to survive after dark or it vanishes for half of every day, which is how it went missing in the first place. Same geometry, cool palette, rays drawn in to a halo. |
||
|
kemal
|
3c99cd53e3 |
Weather glyphs: sun, umbrella, snowflake, switched by the data
Three references were requested as 8x8 animations. Copying their frames does not work and I measured it rather than asserting it: at 8x8 the sun is a 2025:1 area reduction and its rays vanish, the umbrella loses canopy and handle, and the snowflake averages into the background. Downsampled they move 0.0037, 0.0175 and 0.0027 per frame against 0.0177 for the aurora already on the panel, so frame-copying would have been a downgrade. The sun source is only 3 frames and the umbrella 4. These are hand-drawn at 8x8 instead, taking the palette and subject from the references, which also keeps three artists' frames out of an Apache-2.0 repo. Transitions are now the data. _pick_glyph reads rain probability, Kalman temperature, solar elevation and cloud index and selects sun, umbrella or snowflake; a change preempts whatever is on screen and crossfades immediately, so the panel dissolves because the weather moved, not because a timer expired. Between changes the informational scenes still rotate. Verified switching live: sunny -> sun, rain forecast -> umbrella, temperature to 0.4 C -> snowflake, clearing -> sun. Getting them to read took two failed passes, both recorded in comments. First version blew the canopy to white and fused the snowflake into a blob, because seventeen arc samples over ten pixels overlap 1.7 deep. Dropping alpha made them muddy instead. The fix was sampling density, not brightness. Profiled again since these share the board: the glyphs first cost 11 to 13% of a core. Making plot() write scalar components rather than a 3-vector slice, and expressing the sun's eight-fold rays as one angular field instead of 56 splats, took the sun from 275 to 43 us and the worst scene overall from 13.2% to 8.0%. |
||
|
kemal
|
db0f877052 |
Fix flaky row-count assertion in the recompute tests
Equality on the row count raced the live sample loop under TestClient, which legitimately inserts rows mid-test. Now asserts no rows are lost, which is the property that matters. Run three times to confirm it is stable. |
||
|
kemal
|
e05667d75b |
Make the matrix frame rate configurable
24 fps costs about 11% of one core on a Zero 2 W, measured on the board. That is a reasonable default for something you look at, but it is a decorative load sharing a 512 MB machine with the forecaster, so it should be the owner's choice. server.led_fps is clamped to 4..30. Particle fall speed now divides by the configured rate rather than the module constant, so rain falls at the same real-world speed whatever the frame rate, instead of slowing down when you turn the frame rate down. |
||
|
kemal
|
30944c2978 |
Rewrite the LED matrix as an animated instrument
The old display drew static glyphs, held them, and cut to the next, which looks like a microwave clock. This is a continuous 24 fps renderer. Three things do most of the work. Gamma, because LED duty cycle is linear and perception is not, so ungamma'd gradients band and dim colours vanish. Sub-pixel rendering, so a dot at x=3.4 lights two pixels and motion glides rather than steps. Crossfades, so scenes dissolve over 1.3 s and nothing ever cuts. Added temporal dithering after finding the framebuffer is RGB565: 32 levels of red and blue, which after gamma leaves very few steps exactly where an aurora and a star field live. A Bayer pattern rotated each frame alternates between adjacent hardware levels, measured landing on 1.75, 4.31 and 8.06 where the panel can only display integers. The panel is also dimmed by measured lux on a log curve, so at night it is a glow rather than a searchlight. Five scenes, each a reading rather than decoration. Aurora: hue is temperature, curtain drift direction is pressure tendency, contrast is humidity. Solar sky: sun at its true azimuth and elevation over a dawn/day/dusk gradient, becoming a twinkling star field and moon after sunset. Precipitation: drop count from rain probability, snow below 1.5 C with sideways sway, lightning with exponential afterglow when stormy. Forecast ribbon: six horizons scrolling, height is the predicted delta, pale caps are the conformal half-width so uncertainty is visible. Barometer: a breathing ring whose period is the tendency. Profiled because it shares a 512 MB board with the station. The first ribbon cost 330 us a frame, about 16% of a core scaled to a Zero 2 W; vectorising it into fields rather than 84 sub-pixel splats brought the worst scene to 7.6%. Verified 23.6 fps sustained with zero malformed frames. |
||
|
kemal
|
e767ca2105 |
Move tabs into the header, fix Conditions ahead overflow
The tab bar was its own grid row costing about 70 px of vertical space on every tab to hold five buttons, which is a poor trade on a layout that refuses to scroll. The tablist now sits in the header between the title and the status block, so the shell drops from three rows to two. That space goes to the pressure tendency chart, which was 56 px and had no room for Chart.js to lay out its tick row: measured, the caption sat 2 px below its own card. The chart is now 80 px with explicit layout padding, and the column has 11 px of slack instead of overflowing. Also made the tendency x-axis adaptive. Hour-only labels collapsed to three identical ticks on a short window, which is what a young station always has. Below a six hour span the label now carries minutes. Verified at 1600x900: Live, History, Models and Nerd all zero scrollbars, zero clipping, zero console errors. Tabs stay reachable and the header stays one row at 1280 and 1024 wide. Methods improved incidentally, from 68 px of overflow to 4. |
||
|
kemal
|
98210bff8f |
Six enhancements: recompute, markers, vendoring, tests, nerd stats, DS18B20
1. POST /api/recompute re-derives every compensated column from the untouched raw values, removing the step a calibration otherwise leaves through the history. Possible because temp_raw, cpu_temp and hum are never overwritten. Idempotent by construction and tested per row: 0 of 6051 rows change on a second run. 6069 rows in 0.25 s here, so a few seconds on the Pi. 2. Calibration now emits a 'discontinuity' event alongside the calibration log, so downstream views can find the boundary without parsing prose. 3. Vendored Tailwind, Chart.js, hammer, the zoom plugin, KaTeX with its 20 woff2 faces, and both Google fonts into ashvale/static, served by the station. 1.4 MB. Verified with every non-localhost request aborted in the browser: zero external requests, equations still render, fonts still load. The dashboard no longer needs internet. 4. 54 pytest cases over the pure numerics: physics closed forms and round trips, both compensator inverse properties, the Kalman covariance invariants and NIS consistency, the RLS trace cap under a deliberately unexcited regressor, conformal coverage, and the Zambretti ordering. Wired into CI after the seed step so the recompute cases have history. Writing them caught my own sign error on the conformal update: a hit raises alpha and narrows the band, which reads backwards until you follow it through. 5. Stats for Nerds gains the condition number of each head's covariance, a standardised innovation histogram per Kalman filter from a bounded 600 sample ring buffer, and a reliability strip of realised against nominal coverage. All arithmetic on data already in memory. 6. OutdoorProbe reads a DS18B20 over the kernel 1-Wire driver, no new dependency. Polled on its own slower cadence because the sensor blocks for up to 750 ms during conversion, which would eat a third of the 2 s sample budget. Rejects the 85000 power-on sentinel and out-of-range values, and reports age so a dead probe cannot masquerade as fresh. |
||
|
kemal
|
9d2e884b74 |
Fix shutdown hang and blank equations when KaTeX is unavailable
Shutdown hang, the cause of every restart taking systemd's full 90 s timeout and ending in SIGKILL: /api/stream looped forever with no disconnect or shutdown check, so an open dashboard was an in-flight request that never completed and uvicorn's graceful shutdown waited on it. Reproduced cleanly: with no stream client the process stopped in 2 s, with one open client it was still running after 15 s. Fixed by bounding timeout_graceful_shutdown, and by having the generator exit on client disconnect and on a shutdown event. Now 7 s with a client attached. Equations rendered as blank boxes whenever the KaTeX CDN was unreachable, which is a real case for a Pi on wifi. The elements carried the TeX only in a data attribute, so with no KaTeX there was nothing to display. The source is now written into the element as text first and KaTeX replaces it, so it degrades to readable TeX. Verified by aborting the katex request. A window load handler re-runs typesetting for a slow CDN. The code comment claiming this fallback already existed was wrong and is corrected. |
||
|
kemal
|
bb9f0a588f |
Stats for Nerds tab, KaTeX methods, weather icons, outlook to top
New Stats for Nerds tab over a new read-only /api/nerd endpoint: Kalman NIS
and covariance per signal, both compensators, all 18 RLS heads with trace(P)
against the cap, |theta|, EWMA RMSE, conformal alpha against target, realised
coverage and ensemble weights, plus per-head feature attribution over the 33
standardised weights, the Mahalanobis and Page-Hinkley detector state,
climatology harmonics and precipitation coefficients.
Methods overhaul: KaTeX now renders the equations. They were previously passed
through .replace(/[{}\\]/g,' '), which stripped every brace and backslash and
turned real mathematics into mush. Stages 2, 3, 5, 6 and 7 gained full
derivations (RLS normal equations and the trace cap, Joseph-form Kalman with
NIS, adaptive conformal with its coverage limit, ridge harmonic regression with
anomaly decay) and a per-symbol legend rendered inline.
Conditions ahead gains weather icons chosen from measured cloud index, solar
elevation and temperature rather than the barometric class alone, so a fine
barometer under overcast draws a cloud and after sunset draws a moon. Snow is
selected on temperature.
Seven day outlook moves to the top of Live, directly under the nav. Tab renamed
Models and Calibration.
Verified in Chromium at 1600x900: Live, History, Models and Nerd all report
zero scrollbars, zero clipping, no page scroll, zero console errors. Methods
keeps its documented prose scroller. Backtest numerically unchanged.
|
||
|
kemal
|
e27a4b41c8 |
Outlook to Live, humidity calibration, Models tab rebuilt without scrollers
Seven day outlook moves from History to Live, which now runs four rows. Conditions ahead tightened so the Live column no longer needs a scroller. Adds HumidityCompensator: an additive RH offset estimated by one-step RLS from a trusted hygrometer, clamped to +/-35%, persisted, exposed at POST /api/calibrate/humidity and on the renamed Models and calibration tab. It also implements the psychrometric term (RH moved from element temperature onto air temperature via conserved vapour pressure) but leaves it OFF by default. The thermal argument predicts a hot element reads low; measured against a reference hygrometer this board read 75.4% where the truth was 50.4%, so it reads HIGH and that correction would push it the wrong way. When the flag is enabled, simulate.py applies the exact inverse, per the simulator/compensator trap in DESIGN.md section 2. Models pane rebuilt: the scorecard is one column per target so all 18 heads are visible, and no panel on the tab uses an internal scroller. Verified in Chromium at 1600x900: Live, History and Models all report zero scrollbars, zero clipping, no page scroll, zero console errors. Backtest is numerically identical to the previous commit, confirming the humidity work is a no-op while the flag is off. |
||
|
kemal
|
49c0aee2e1 |
Merge Forecast into Live, four tabs; state solo-project policy
The Live tab now carries the observed-and-forecast chart (a superset of the old rolling window, which plotted the same observed series without the prediction) and the precipitation panel. Estimator internals moves to Models, beside the calibration input that sets the coefficient it reports. The seven day outlook moves to History. Verified in Chromium at 1600x900: all four tabs scrollHeight 900 against innerHeight 900, zero clipped elements outside internal scrollers, zero console errors. CONTRIBUTING.md now states plainly that this is a solo project: bug reports welcome, pull requests unlikely to be merged, fork it instead. No other developer was ever named anywhere in the repository. |
||
|
kemal
|
dbbea1a95f | chore: enforce ruff config and fix lint findings | ||
|
kemal
|
06ce53bc44 | Initial release: Ashvale Station 1.0.0 v1.0.0 |