6 Commits

Include renames

Author	SHA1	Message	Date
kemalandClaude Opus 5.5	60a7ae5ada	cloud: optional AZ for spot launches ... Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01814GKVzZVL3FuF3DeiZXhv	2026-09-28 17:44:26 +01:00
kemalandClaude Opus 5.5	70c36446a0	P8: GSE165845 validation scripts (prepare, §16.5 metrics) and cloud task ... scripts/p8/prepare.py lays out the 360 GEO IDATs under their header barcode/position; scripts/p8/validate_p8.py computes concordance, NRD, no-call rate, per-class and per-MAF sensitivity/specificity/precision/F1 vs NYGC 30x and GIAB HG001, replicate concordance, systematic/allele-swap site errors, per-chip batch table and duplicate detection. Cloud userdata can skip the Omni2.5 archive (OMNI=0). Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01814GKVzZVL3FuF3DeiZXhv	2026-09-28 17:43:31 +01:00
kemalandClaude Opus 5.5	56d250c621	P5: cloud task for cohort QC validation; configurable gp3 throughput; batched contamination map ... - scripts/cloud/tasks/p5.sh: P4 preset re-check, genotype (v5) with QC, validate_p5_large.py - scripts/validate_p5_large.py: identify samples via het pattern vs 1000G VCF; sex, parent-offspring, Mendel, contamination/control distributions - trial.sh: DISK_MBPS / DISK_IOPS - qc/contamination: bounded map batches (executor.map submits eagerly) Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01814GKVzZVL3FuF3DeiZXhv	2026-09-28 09:10:02 +01:00
kemalandClaude Opus 5.5	268457ba8e	P3: vectorised nearest-neighbour search; task-driven cloud trials ... - fit.py: exact and binned nearest-neighbour searches use one distance matrix per searched bin (first-minimum rule kept); fit 2.6 s -> 1.4 s per GSA sample, still bit-identical to the oracle - scripts/validate_p3_large.py: refits 1.2.0 and 1.1.2 transforms for every vendor GTC and reports ULP distances per field - cloud trial runs scripts/cloud/tasks/<TASK>.sh (TASK=p3 default; p1p2 = previous validation + ingest) Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01814GKVzZVL3FuF3DeiZXhv	2026-09-27 22:57:03 +01:00
kemalandClaude Opus 5.5	2e75345a30	P2 validation: oracle test and large-scale raw X/Y check ... - tests/oracle/test_p2_raw_xy.py: IDAT-assembled raw X/Y and controls equal the arrays in oracle GTCs - validate_p1_large.py: phase-P2 comparison against vendor GTCs with diagnostics (channel swap, bead-count disagreements) - cloud trial: also runs `simurg ingest` on all samples (time/memory), disk 250 GB Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01814GKVzZVL3FuF3DeiZXhv	2026-09-27 20:57:42 +01:00
kemalandClaude Opus 5.5	a9aa6dc080	Add self-destructing AWS trial and large-scale P1 validation script ... scripts/cloud/trial.sh launches one spot instance that validates the parsers on the 1000 Genomes Omni2.5 set (2141 samples, vendor GTCs) streamed from the public S3 bucket, uploads the summary and terminates itself. Cost guards: spot price cap, hard time limit (shutdown = terminate), EXIT-trap shutdown, CloudWatch idle alarm that terminates from outside, disk deleted with the instance, monthly budget alert, and an `audit` command listing anything still billable. scripts/validate_p1_large.py compares every GTC with BeadArrayFiles, checks byte-identical round-trips, IDAT pairs and BPM/EGT/CSV. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01814GKVzZVL3FuF3DeiZXhv	2026-09-27 19:30:27 +01:00