kemal/SimurgArray
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
91 Commits 1 Branch 3 Tags
9149ef328a5dc85511a432b10e409accc5401cf5
Commit Graph
5 Commits
Author SHA1 Message Date
kemalandClaude Opus 5.5 70c36446a0 P8: GSE165845 validation scripts (prepare, §16.5 metrics) and cloud task
ci / test (3.12) (push) Failing after 8s
Details
ci / test (3.13) (push) Failing after 10s
Details 
scripts/p8/prepare.py lays out the 360 GEO IDATs under their header barcode/position;
scripts/p8/validate_p8.py computes concordance, NRD, no-call rate, per-class and per-MAF
sensitivity/specificity/precision/F1 vs NYGC 30x and GIAB HG001, replicate concordance,
systematic/allele-swap site errors, per-chip batch table and duplicate detection.
Cloud userdata can skip the Omni2.5 archive (OMNI=0).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01814GKVzZVL3FuF3DeiZXhv
 2026-09-28 17:43:31 +01:00
kemalandClaude Opus 5.5 268457ba8e P3: vectorised nearest-neighbour search; task-driven cloud trials
ci / test (3.12) (push) Failing after 8s
Details
ci / test (3.13) (push) Failing after 10s
Details 
- fit.py: exact and binned nearest-neighbour searches use one distance
  matrix per searched bin (first-minimum rule kept); fit 2.6 s -> 1.4 s
  per GSA sample, still bit-identical to the oracle
- scripts/validate_p3_large.py: refits 1.2.0 and 1.1.2 transforms for
  every vendor GTC and reports ULP distances per field
- cloud trial runs scripts/cloud/tasks/<TASK>.sh (TASK=p3 default;
  p1p2 = previous validation + ingest)

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01814GKVzZVL3FuF3DeiZXhv
 2026-09-27 22:57:03 +01:00
kemalandClaude Opus 5.5 2e75345a30 P2 validation: oracle test and large-scale raw X/Y check
ci / test (3.12) (push) Failing after 8s
Details
ci / test (3.13) (push) Failing after 10s
Details 
- tests/oracle/test_p2_raw_xy.py: IDAT-assembled raw X/Y and controls
  equal the arrays in oracle GTCs
- validate_p1_large.py: phase-P2 comparison against vendor GTCs with
  diagnostics (channel swap, bead-count disagreements)
- cloud trial: also runs `simurg ingest` on all samples (time/memory),
  disk 250 GB

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01814GKVzZVL3FuF3DeiZXhv
 2026-09-27 20:57:42 +01:00
kemalandClaude Opus 5.5 f52313fc93 cloud trial: set HOME/PATH for cloud-init, unit tests non-fatal
ci / test (3.12) (push) Failing after 8s
Details
ci / test (3.13) (push) Failing after 10s
Details 
The first run stopped after 1 minute (HOME unbound under set -u) and
terminated itself as designed.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01814GKVzZVL3FuF3DeiZXhv
 2026-09-27 19:36:49 +01:00
kemalandClaude Opus 5.5 a9aa6dc080 Add self-destructing AWS trial and large-scale P1 validation script
ci / test (3.12) (push) Failing after 9s
Details
ci / test (3.13) (push) Failing after 13s
Details 
scripts/cloud/trial.sh launches one spot instance that validates the
parsers on the 1000 Genomes Omni2.5 set (2141 samples, vendor GTCs)
streamed from the public S3 bucket, uploads the summary and terminates
itself. Cost guards: spot price cap, hard time limit (shutdown =
terminate), EXIT-trap shutdown, CloudWatch idle alarm that terminates
from outside, disk deleted with the instance, monthly budget alert, and
an `audit` command listing anything still billable.

scripts/validate_p1_large.py compares every GTC with BeadArrayFiles,
checks byte-identical round-trips, IDAT pairs and BPM/EGT/CSV.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01814GKVzZVL3FuF3DeiZXhv
 2026-09-27 19:30:27 +01:00