kemal
80b93e9f45
- Omni2.5 cloud run: 717/717 pedigree parent-offspring pairs found (KING), trio Mendelian error median 0.031 %, sex vs pedigree 99.6 %, P4 preset re-check 2138/2141 byte-identical (K-015 closed) - qc/controls: skip bead addresses absent from the IDAT; control missing only if none is on the chip - QC-008 amended: signal / zero background = +Inf (passes), 0/0 = NaN (flagged); Inf in TSV/report - plotly bundle stored gzip-compressed (repo grep stays clean: the DOM event name contains the forbidden substring) - docs: validation P5 part 2, summaries, known_unknowns K-021
4.0 KiB
Phase P5 validation — QC
Status: passed — part 1 (local, GSA v4 PGx demo, 36 samples) and part 2 (1000 Genomes Omni2.5, 2141 samples, AWS run
run-20260928T081042Z, 72 min, ≈ $0.37). Summary: P5_omni2141_summary.json.
Part 1 — local, against open-source references (tests/oracle/test_p5_qc.py)
| Check | Reference | Result |
|---|---|---|
| BAF-regression contamination, 36 samples, all 683 054 loci as sites | gtc2vcf bcftools +BAFregress (MIT), built by scripts/make_oracle_gtcs.sh |
36/36 identical at the plugin's printed precision (%.4f), homozygous-site counts identical (K-017: float threshold 0.1f and klib quickselect median reproduced) |
| Locus summary, 683 054 loci | BeadArrayFiles examples/locus_summary.py (BSD-2), scripts/make_oracle_qc.sh |
every column bit-identical except ChiTest_P100 (≤ 3.6e-15 relative, K-018) |
| Control metrics | QC-004 formulas | unit tests with hand-computed values; GSA demo values plausible (staining, extension, NP > 5; hyb > 1) |
| Thresholds / profiles / status / flags | QC-020 … QC-033 | unit tests for every QC-021 error code, precedence, aliases, profiles, both status axes |
| KING-robust kinship, Mendelian errors | Manichaikul 2010 | simulated trio: duplicate φ = 0.50, parent–child φ ≈ 0.25, unrelated ≈ 0, 0 Mendelian errors in the true trio |
| End to end | simurg qc controls, simurg genotype with QC |
control tables identical between the two paths; QC columns filled |
GSA demo observations: all 36 samples PASS (autosomal call rate ≥ 0.997, LogR dev 0.13–0.17), control status CLEAR; contamination 0.0012–0.0040; replicate pairs (φ ≈ 0.50) and first-degree relatives (φ ≈ 0.25) are present (K-019).
Part 2 — cohort scale, 1000 Genomes Omni2.5 (2141 samples)
simurg genotype --preset autocall-1.6.2 --gtc-version 5 --qc-profile genotyping on all 2141 IDAT pairs, with control
tables and cohort QC (contamination, KING relatedness), then scripts/validate_p5_large.py against the public
1000G pedigree (integrated_call_samples.20130502.ALL.ped) and genotype VCF. The GTCs carry lab sample names, so every
sample was identified by its heterozygosity pattern over 20 000 common SNPs (orientation-free).
| Check | Result |
|---|---|
| Sample identification | 2139/2141 unique (concordance 1.0); 2 ambiguous (margin < 1e-4) |
| Parent–offspring pairs of the pedigree detected as first degree | 717/717; KING φ median 0.247 (min 0.192, max 0.259) |
| Mendelian error rate, 335 complete trios, autosomal sites | median 0.031 %, max 0.108 % |
| Sex estimate vs pedigree | 2088 match, 9 mismatch (99.6 %), 44 without pedigree sex |
| Close pairs (≥ second degree) not explained by parent/sibling records | 69 (cryptic relatives, mostly MXL/ASW/PUR/CHS second degree — known in 1000G) |
| Contamination (BAF regression, autosomal sites) | median 0.0002; 15 samples > 0.02 (max 0.075, HG00707) |
| Autosomal call rate | median 0.9966, min 0.9746 (2 below 0.98) |
| LRR SD | median 0.187, p90 0.219 → 602 samples above 0.20 (functional FAIL 604): a property of this legacy data set; recalibrate with qc suggest-thresholds |
| Runtime / memory | genotype + control tables + cohort QC: 34 min, 31 GB peak RSS (16 vCPU); validation 5 min |
Control metrics exposed two issues, fixed after the run (K-021): absent second bead-pool addresses poisoned the bead
means (1794 false staining flags), and background-subtracted Biotin (Bgnd) = 0 made staining green signal/0. Recomputed
from the run's controls_raw.tsv: control status 2139 CLEAR, 2 FLAGGED (DNP (High) red = 0 — genuine).
P4 re-check with --preset autocall-1.6.2 (same run)
2138/2141 GTCs byte-identical to the vendor's; the 3 others are the K-009 samples. GC10/GC50/#calls, which
differed in all 2141 before the preset, now match (P4_preset_omni2141_summary.json).
Not done
Sanger Omni2.5-8 BAF/LRR vs the GenomeStudio FinalReport: the 1000G Sanger folder has IDATs and the report but no cluster (EGT) file or BPM, so BAF/LRR cannot be recomputed.