kemal/SimurgArray
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
34 Commits 1 Branch 3 Tags
c97aac1a5a58b70b9b5bbd1d13c8314dc66701db
Commit Graph
7 Commits
Author SHA1 Message Date
kemalandClaude Opus 5.5 c97aac1a5a P9 cloud task: install gcc (linker for the Rust build); guide: kernel installation
ci / test (3.12) (push) Failing after 8s
Details
ci / test (3.13) (push) Failing after 10s
Details 
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01814GKVzZVL3FuF3DeiZXhv
 2026-09-28 21:35:42 +01:00
kemalandClaude Opus 5.5 5bdd41b048 P9 (WIP): compiled kernels (rust/simurg-core, PyO3) with bit-identical pure-Python fallback
ci / test (3.12) (push) Failing after 9s
Details
ci / test (3.13) (push) Failing after 11s
Details 
Rust ports of the hot paths found by profiling: fused per-locus GenCall calling
and scoring, BAF/LRR, libm maps with CPython math semantics, both nearest-neighbour
searches of the normalization fit, and the BPM LocusEntry parser. Address
coverage is memoised per IDAT layout. SIMURG_PURE_PYTHON=1 forces the pure paths.
On the Pi: genotype_sample 3.4 s -> 1.1 s, 36-sample GSA run 131 s -> 46 s, BPM
read 4x faster; 36 GSA + 6 Omni2.5 GTCs and QC tables byte-identical to the pure
path; tests/unit/test_core_kernels.py checks every kernel on edge cases.
bench/profile_genotype.py, bench/scaling.py, cloud task p9 (thread scaling).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01814GKVzZVL3FuF3DeiZXhv
 2026-09-28 21:28:55 +01:00
kemalandClaude Opus 5.5 9f36f6443b P8 cloud task: stream NYGC VCFs through bcftools instead of downloading
ci / test (3.12) (push) Failing after 8s
Details
ci / test (3.13) (push) Failing after 10s
Details 
The download-only phase (EBI ~15 MB/s) left the CPU idle long enough for the
idle alarm to terminate the first run. Streams are now cut to the manifest
positions (±100 bp) and the 120 individuals while genotyping runs.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01814GKVzZVL3FuF3DeiZXhv
 2026-09-28 19:06:36 +01:00
kemalandClaude Opus 5.5 70c36446a0 P8: GSE165845 validation scripts (prepare, §16.5 metrics) and cloud task
ci / test (3.12) (push) Failing after 8s
Details
ci / test (3.13) (push) Failing after 10s
Details 
scripts/p8/prepare.py lays out the 360 GEO IDATs under their header barcode/position;
scripts/p8/validate_p8.py computes concordance, NRD, no-call rate, per-class and per-MAF
sensitivity/specificity/precision/F1 vs NYGC 30x and GIAB HG001, replicate concordance,
systematic/allele-swap site errors, per-chip batch table and duplicate detection.
Cloud userdata can skip the Omni2.5 archive (OMNI=0).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01814GKVzZVL3FuF3DeiZXhv
 2026-09-28 17:43:31 +01:00
kemalandClaude Opus 5.5 56d250c621 P5: cloud task for cohort QC validation; configurable gp3 throughput; batched contamination map
ci / test (3.12) (push) Failing after 9s
Details
ci / test (3.13) (push) Failing after 10s
Details 
- scripts/cloud/tasks/p5.sh: P4 preset re-check, genotype (v5) with QC, validate_p5_large.py
- scripts/validate_p5_large.py: identify samples via het pattern vs 1000G VCF; sex, parent-offspring,
  Mendel, contamination/control distributions
- trial.sh: DISK_MBPS / DISK_IOPS
- qc/contamination: bounded map batches (executor.map submits eagerly)

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01814GKVzZVL3FuF3DeiZXhv
 2026-09-28 09:10:02 +01:00
kemalandClaude Opus 5.5 eb19ac7f91 P4: unit tests for calling/BAF/LRR/sex/CLI; vendor validation script
ci / test (3.12) (push) Failing after 9s
Details
ci / test (3.13) (push) Failing after 9s
Details 
- tests/unit/test_genotype.py: fuzzy memberships incl. NaN rules, calls
  at cluster centres, intensity threshold, BAF/LRR interpolation and
  NaN pattern, LogR-dev locus set, sex rules v1/v2, TOP-strand base
  calls, invariant dates, end-to-end `simurg genotype` (summary schema,
  pass-through columns, thread/layout determinism, overwrite refusal)
- tests/oracle/test_p4_genotype.py: 36 GSA samples x 2 versions
  byte-identical to the oracle
- scripts/validate_p4_large.py + cloud task p4: compare every vendor
  GTC field by field and as whole files; reports which sex rule matches

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01814GKVzZVL3FuF3DeiZXhv
 2026-09-28 00:10:47 +01:00
kemalandClaude Opus 5.5 268457ba8e P3: vectorised nearest-neighbour search; task-driven cloud trials
ci / test (3.12) (push) Failing after 8s
Details
ci / test (3.13) (push) Failing after 10s
Details 
- fit.py: exact and binned nearest-neighbour searches use one distance
  matrix per searched bin (first-minimum rule kept); fit 2.6 s -> 1.4 s
  per GSA sample, still bit-identical to the oracle
- scripts/validate_p3_large.py: refits 1.2.0 and 1.1.2 transforms for
  every vendor GTC and reports ULP distances per field
- cloud trial runs scripts/cloud/tasks/<TASK>.sh (TASK=p3 default;
  p1p2 = previous validation + ingest)

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01814GKVzZVL3FuF3DeiZXhv
 2026-09-27 22:57:03 +01:00