kemal/SimurgArray
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
54 Commits 1 Branch 3 Tags
65502eb8e425937d5d610ba7f8691431583f75a9
Commit Graph
7 Commits
Author SHA1 Message Date
kemalandClaude Opus 5.5 92416397fe P10 (WIP): release build script, Dockerfile, conda recipes, fresh-VM acceptance task
ci / test (3.12) (push) Failing after 8s
Details
ci / test (3.13) (push) Failing after 10s
Details 
scripts/build_dist.sh: sdist + pure wheel + manylinux_2_28 kernel wheels (maturin --zig,
both architectures). Dockerfile (multi-stage, kernels included, non-root user).
packaging/conda/{simurg,simurg-core}. scripts/p10/digest.py: platform-independent content
digests of GTC/VCF/summary outputs. Cloud task p10 installs the artifacts on a fresh
Ubuntu VM and runs the tutorial with pip, Docker and conda installs. trial.sh picks the
arm64/amd64 image from the instance type and can run instances in parallel (PARALLEL=1).
Licence files ship in both wheels; Rust crate notices added.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01814GKVzZVL3FuF3DeiZXhv
 2026-09-28 22:41:56 +01:00
kemalandClaude Opus 5.5 60a7ae5ada cloud: optional AZ for spot launches
ci / test (3.12) (push) Failing after 9s
Details
ci / test (3.13) (push) Failing after 9s
Details 
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01814GKVzZVL3FuF3DeiZXhv
 2026-09-28 17:44:26 +01:00
kemalandClaude Opus 5.5 70c36446a0 P8: GSE165845 validation scripts (prepare, §16.5 metrics) and cloud task
ci / test (3.12) (push) Failing after 8s
Details
ci / test (3.13) (push) Failing after 10s
Details 
scripts/p8/prepare.py lays out the 360 GEO IDATs under their header barcode/position;
scripts/p8/validate_p8.py computes concordance, NRD, no-call rate, per-class and per-MAF
sensitivity/specificity/precision/F1 vs NYGC 30x and GIAB HG001, replicate concordance,
systematic/allele-swap site errors, per-chip batch table and duplicate detection.
Cloud userdata can skip the Omni2.5 archive (OMNI=0).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01814GKVzZVL3FuF3DeiZXhv
 2026-09-28 17:43:31 +01:00
kemalandClaude Opus 5.5 56d250c621 P5: cloud task for cohort QC validation; configurable gp3 throughput; batched contamination map
ci / test (3.12) (push) Failing after 9s
Details
ci / test (3.13) (push) Failing after 10s
Details 
- scripts/cloud/tasks/p5.sh: P4 preset re-check, genotype (v5) with QC, validate_p5_large.py
- scripts/validate_p5_large.py: identify samples via het pattern vs 1000G VCF; sex, parent-offspring,
  Mendel, contamination/control distributions
- trial.sh: DISK_MBPS / DISK_IOPS
- qc/contamination: bounded map batches (executor.map submits eagerly)

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01814GKVzZVL3FuF3DeiZXhv
 2026-09-28 09:10:02 +01:00
kemalandClaude Opus 5.5 268457ba8e P3: vectorised nearest-neighbour search; task-driven cloud trials
ci / test (3.12) (push) Failing after 8s
Details
ci / test (3.13) (push) Failing after 10s
Details 
- fit.py: exact and binned nearest-neighbour searches use one distance
  matrix per searched bin (first-minimum rule kept); fit 2.6 s -> 1.4 s
  per GSA sample, still bit-identical to the oracle
- scripts/validate_p3_large.py: refits 1.2.0 and 1.1.2 transforms for
  every vendor GTC and reports ULP distances per field
- cloud trial runs scripts/cloud/tasks/<TASK>.sh (TASK=p3 default;
  p1p2 = previous validation + ingest)

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01814GKVzZVL3FuF3DeiZXhv
 2026-09-27 22:57:03 +01:00
kemalandClaude Opus 5.5 2e75345a30 P2 validation: oracle test and large-scale raw X/Y check
ci / test (3.12) (push) Failing after 8s
Details
ci / test (3.13) (push) Failing after 10s
Details 
- tests/oracle/test_p2_raw_xy.py: IDAT-assembled raw X/Y and controls
  equal the arrays in oracle GTCs
- validate_p1_large.py: phase-P2 comparison against vendor GTCs with
  diagnostics (channel swap, bead-count disagreements)
- cloud trial: also runs `simurg ingest` on all samples (time/memory),
  disk 250 GB

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01814GKVzZVL3FuF3DeiZXhv
 2026-09-27 20:57:42 +01:00
kemalandClaude Opus 5.5 a9aa6dc080 Add self-destructing AWS trial and large-scale P1 validation script
ci / test (3.12) (push) Failing after 9s
Details
ci / test (3.13) (push) Failing after 13s
Details 
scripts/cloud/trial.sh launches one spot instance that validates the
parsers on the 1000 Genomes Omni2.5 set (2141 samples, vendor GTCs)
streamed from the public S3 bucket, uploads the summary and terminates
itself. Cost guards: spot price cap, hard time limit (shutdown =
terminate), EXIT-trap shutdown, CloudWatch idle alarm that terminates
from outside, disk deleted with the instance, monthly budget alert, and
an `audit` command listing anything still billable.

scripts/validate_p1_large.py compares every GTC with BeadArrayFiles,
checks byte-identical round-trips, IDAT pairs and BPM/EGT/CSV.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01814GKVzZVL3FuF3DeiZXhv
 2026-09-27 19:30:27 +01:00