kemal
e518ea03d5
Readers for IDAT v3, BPM (locus entries v4-v8), CSV manifest, EGT (data blocks 5-9) and GTC v3-v5 (plus a layout-preserving GTC writer); sample-sheet validation (IN-001..IN-014), IDAT discovery and pairing (IN-020..IN-022), message catalogue and exit codes (CLI-014/016), JSON Lines logging (CLI-007/008), atomic writes (CLI-011), run manifest, and `simurg inspect`. Validation (docs/validation/P1.md): GTC, BPM and EGT readers agree exactly with Illumina BeadArrayFiles on GSA v4 / GDA v1 files; GTCs round-trip byte-identically; BPM and CSV manifests are consistent.
59 lines
4.0 KiB
Markdown
59 lines
4.0 KiB
Markdown
# Phase P0/P1 validation report
|
||
|
||
Date: 2026-09-27 · Machine: Raspberry Pi 5 (aarch64, 4 cores, 8 GB) · Python 3.13, NumPy 2.2.
|
||
Commands: `pytest tests/unit` (every CI run), `pytest tests/oracle -m "not slow"` and `-m slow` (needs
|
||
`scripts/fetch_data.sh p0`, `scripts/bootstrap_refs.sh`, `scripts/make_oracle_gtcs.sh`).
|
||
|
||
## Scope delivered
|
||
|
||
| Plan §18 P1 item | Module | Status |
|
||
|---|---|---|
|
||
| Little-endian binary + .NET string codec | `formats/binutil.py` | done |
|
||
| IDAT v3 reader (gzip, metadata, RunInfo → ISO date, encrypted detection) | `formats/idat.py` | done |
|
||
| BPM reader (v3–5, locus entries v4/6/7/8, norm-bin rule incl. mod-256 wrap) | `formats/bpm.py` | done |
|
||
| CSV manifest reader + BPM↔CSV consistency (IN-042) | `formats/csv_manifest.py` | done |
|
||
| EGT reader (data-block variants, count cross-check) | `formats/egt.py` | done |
|
||
| GTC v3–v5 reader and writer (layout-preserving) | `formats/gtc.py` | done |
|
||
| Sample sheet (IN-001 … IN-014) | `formats/samplesheet.py` | done |
|
||
| IDAT discovery, pairing, duplicates, input-set rule (IN-020 … IN-022, CLI-006) | `formats/discovery.py` | done |
|
||
| Message catalogue, exit codes, JSON logging, rate limiting | `core/errors.py`, `core/log.py` | done |
|
||
| Atomic writes, overwrite policy, run manifest | `core/atomic.py`, `core/provenance.py` | done (used from P4 on) |
|
||
| `simurg version`, `simurg inspect [--json]` | `cli/main.py` | done |
|
||
| Synthetic IDAT/BPM/EGT writers for tests | `synth/files.py` | done |
|
||
|
||
## Acceptance results (plan §18 P1 gate)
|
||
|
||
| Criterion | Data | Result |
|
||
|---|---|---|
|
||
| GTC fields equal to BeadArrayFiles (23 fields incl. raw X/Y, genotypes, scores, BAF, LRR, base calls, sample stats, percentiles, 48 normalization transforms) | 18 GSA v4 GTCs from gtc2vcf `idat2gtc` (preset: Array Analysis CLI) | **EXACT, 18/18** |
|
||
| GTC read → write byte-identical | same 18 files; synthetic v3/v5 incl. non-canonical layouts; 25 random property cases | **EXACT** |
|
||
| BPM equal to BeadArrayFiles (names, SNP, Chr, MapInfo, AddressA, assay type, normalization lookups, RefStrand, manifest name, control config) | GSA v4 PGx BPM (683 054 loci) | **EXACT** |
|
||
| EGT equal to BeadArrayFiles (N, θ/R mean/SD ×3, intensity threshold, GenTrain score, cluster separation, address) | GSA v4 (683 054 records) and GDA v1 (1 904 599 records), ~5 000 evenly spaced records each | **EXACT** |
|
||
| BPM ↔ CSV consistency (IN-042: names, counts, Chr, MapInfo, SNP, AddressA/B) | GSA v4 | **0 differences** |
|
||
| All IDATs parse and pair; Red/Grn address sets equal; same address set on every chip | 72 GSA IDATs (36 samples, 2 chips) | **pass** |
|
||
| Manifest addresses present in IDATs | GSA v4 | **100 %** |
|
||
| Bioconductor fixtures: 381 079 / 622 399 addresses (plain + gzip); 2 encrypted files → `E_IDAT_ENCRYPTED` | IlluminaDataTestFiles 1.50.0 | **pass** |
|
||
| Malformed inputs never raise uncaught exceptions (truncation, bad magic, bad TOC offsets, bad string lengths, duplicates) | synthetic matrix (EDGE-002, EDGE-004) | **pass** |
|
||
| Sample-sheet matrix EDGE-007 (27 cases) incl. 10 000 rows < 2 s (PERF-007) | synthetic | **pass** |
|
||
| Locale independence of `inspect --json` (C, tr_TR, de_DE) | synthetic | **pass** |
|
||
|
||
## Performance on the Pi (single thread, pure Python/NumPy)
|
||
|
||
| Operation | Time | Peak RSS |
|
||
|---|---|---|
|
||
| IDAT read (734 656 addresses) | 3 ms | – |
|
||
| GTC read (683 054 loci) | 34 ms | – |
|
||
| EGT read GSA / GDA | 2.2 s / 6.1 s | – |
|
||
| CSV manifest read GSA | 12.5 s | 713 MB |
|
||
| BPM read GSA | 17.8 s | 665 MB |
|
||
|
||
BPM/CSV parsing is the only slow step; it is a one-off per run and moves to Rust in P9 (or is cached).
|
||
|
||
## Not yet covered (needs the larger machine / later phases)
|
||
|
||
- GEO sweep of 254 genotyping IDAT series and the 1000 Genomes Omni2.5 set (plan §18 P1 wording) — P1 is
|
||
validated on P0 data only; rerun `tests/oracle` after `scripts/fetch_data.sh p1`.
|
||
- GTCs written by vendor software (only gtc2vcf-written GTCs are available locally); the 1000 Genomes Broad set
|
||
contains vendor GTCs.
|
||
- IN-027 (IDAT ↔ manifest coverage gate) is P2; the coverage computation is already exercised by a test.
|