P3: vectorised nearest-neighbour search; task-driven cloud trials
ci / test (3.12) (push) Failing after 8s
ci / test (3.13) (push) Failing after 10s

- fit.py: exact and binned nearest-neighbour searches use one distance
  matrix per searched bin (first-minimum rule kept); fit 2.6 s -> 1.4 s
  per GSA sample, still bit-identical to the oracle
- scripts/validate_p3_large.py: refits 1.2.0 and 1.1.2 transforms for
  every vendor GTC and reports ULP distances per field
- cloud trial runs scripts/cloud/tasks/<TASK>.sh (TASK=p3 default;
  p1p2 = previous validation + ingest)

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01814GKVzZVL3FuF3DeiZXhv
This commit is contained in:
kemalandClaude Opus 5.5 committed 2026-09-27 22:57:03 +01:00
1 parent 314fdc40d1
commit 268457ba8e
6 files changed
+197 -44

No files matched your search

+12
View File
@@ -0,0 +1,12 @@
#!/bin/bash
# Phase P1/P2 validation on the 1000 Genomes Omni2.5 set + `simurg ingest` at scale.
# Args: <data dir with bpm/egt/csv and data/> <results dir>
set -uo pipefail
D=$1; OUT=$2
.venv/bin/python scripts/validate_p1_large.py --root $D/data --bpm $D/HumanOmni2.5-4v1-Multi_B.bpm \
--egt $D/HumanOmni2.5-4v1-Multi_B.egt --csv $D/HumanOmni2.5-4v1_B.csv --out $OUT/summary.json \
--workers "$(nproc)" || echo "validation reported differences (see summary.json)"
/usr/bin/time -v .venv/bin/simurg ingest --bpm $D/HumanOmni2.5-4v1-Multi_B.bpm --idats $D/data \
--out /work/ingest --threads "$(nproc)" --quiet 2> $OUT/ingest_time.txt || echo "ingest exit code $?"
cp /work/ingest/ingest_summary.tsv $OUT/ingest_summary.tsv 2>/dev/null || true
cp /work/ingest/run_manifest.json $OUT/ingest_run_manifest.json 2>/dev/null || true
+8
View File
@@ -0,0 +1,8 @@
#!/bin/bash
# Phase P3: refit normalization transforms (1.2.0 and 1.1.2) for every vendor GTC and compare bit by bit.
# Args: <data dir with bpm and data/> <results dir>
set -uo pipefail
D=$1; OUT=$2
/usr/bin/time -v .venv/bin/python scripts/validate_p3_large.py --root $D/data --bpm $D/HumanOmni2.5-4v1-Multi_B.bpm \
--out $OUT/summary.json --workers "$(nproc)" 2> $OUT/time.txt || echo "P3 validation reported differences"
tail -25 $OUT/time.txt