4 Commits

Include renames

Author	SHA1	Message	Date
kemalandClaude Opus 5.5	268457ba8e	P3: vectorised nearest-neighbour search; task-driven cloud trials ... - fit.py: exact and binned nearest-neighbour searches use one distance matrix per searched bin (first-minimum rule kept); fit 2.6 s -> 1.4 s per GSA sample, still bit-identical to the oracle - scripts/validate_p3_large.py: refits 1.2.0 and 1.1.2 transforms for every vendor GTC and reports ULP distances per field - cloud trial runs scripts/cloud/tasks/<TASK>.sh (TASK=p3 default; p1p2 = previous validation + ingest) Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01814GKVzZVL3FuF3DeiZXhv	2026-09-27 22:57:03 +01:00
kemalandClaude Opus 5.5	314fdc40d1	P3: normalization fit (1.2.0 and 1.1.2), apply, polar, simurg normalize ... - signal/normalization/numerics.py: vendor percentile, trimmean, MAD sigma (old/new), bisquare robust fits, float32/float64 typing and sequential sums reproduced exactly (ported from gtc2vcf, MIT) - signal/normalization/fit.py: outlier removal, offset, rotation, shear, scale, nn12/rr12, single-channel model for small Infinium I bins, exact and binned nearest-neighbour sweeps; vendor quirks marked QUIRK - signal/normalization/apply.py: affine transform and theta/R - pipeline/normalize.py + CLI `simurg normalize` - oracle GTCs rebuilt with -ffp-contract=off (aarch64 gcc fuses a*b+c by default, shifting the last bits); the old FMA oracle differed by <=4 ULP Result: transforms bit-identical to the oracle for all 36 GSA samples x 48 bins, for both normalization versions (3456/3456 bins, 0 ULP). Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01814GKVzZVL3FuF3DeiZXhv	2026-09-27 22:51:06 +01:00
kemalandClaude Opus 5.5	a4faead6d7	P2 (WIP): raw X/Y assembly, canonical Parquet model, simurg ingest ... - signal/raw_xy.py: vectorised Infinium I/II raw X/Y and control assembly (plan §7.1) with a per-chip address-map cache; IN-027 coverage gate - core/schemas.py: Arrow schemas (loci, sample_intensity, controls), deterministic atomic Parquet writing - pipeline/ingest.py + CLI: discovery, sheet, IN-010 naming, overwrite check before any work, spawn-based workers, per-sample isolation, --resume by input hashes, run manifest, exit codes 0/3/4 - matches gtc2vcf raw X/Y and controls exactly on 18 GSA samples; outputs byte-identical across thread counts and layouts Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01814GKVzZVL3FuF3DeiZXhv	2026-09-27 20:54:06 +01:00
kemalandClaude Opus 5.5	4357d3deef	P1: file parsers, sample sheet, discovery, CLI inspect ... Readers for IDAT v3, BPM (locus entries v4-v8), CSV manifest, EGT (data blocks 5-9) and GTC v3-v5 (plus a layout-preserving GTC writer); sample-sheet validation (IN-001..IN-014), IDAT discovery and pairing (IN-020..IN-022), message catalogue and exit codes (CLI-014/016), JSON Lines logging (CLI-007/008), atomic writes (CLI-011), run manifest, and `simurg inspect`. Validation (docs/validation/P1.md): GTC, BPM and EGT readers agree exactly with Illumina BeadArrayFiles on GSA v4 / GDA v1 files; GTCs round-trip byte-identically; BPM and CSV manifests are consistent. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01814GKVzZVL3FuF3DeiZXhv	2026-09-27 16:56:39 +01:00