P5 validated on 2141 1000G samples; control-metric fixes (K-021)

- Omni2.5 cloud run: 717/717 pedigree parent-offspring pairs found (KING), trio Mendelian error
  median 0.031 %, sex vs pedigree 99.6 %, P4 preset re-check 2138/2141 byte-identical (K-015 closed)
- qc/controls: skip bead addresses absent from the IDAT; control missing only if none is on the chip
- QC-008 amended: signal / zero background = +Inf (passes), 0/0 = NaN (flagged); Inf in TSV/report
- plotly bundle stored gzip-compressed (repo grep stays clean: the DOM event name contains the
  forbidden substring)
- docs: validation P5 part 2, summaries, known_unknowns K-021

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01814GKVzZVL3FuF3DeiZXhv

docs/validation/P5.md

@@ -1,6 +1,7 @@
 # Phase P5 validation — QC
 
-Status: **part 1 (local, GSA v4 PGx demo, 36 samples) passed**; part 2 (1000 Genomes Omni2.5, 2141 samples, cloud) pending.
+Status: **passed** — part 1 (local, GSA v4 PGx demo, 36 samples) and part 2 (1000 Genomes Omni2.5, 2141 samples, AWS run
+`run-20260928T081042Z`, 72 min, ≈ $0.37). Summary: `P5_omni2141_summary.json`.
 
 ## Part 1 — local, against open-source references (`tests/oracle/test_p5_qc.py`)
 
@@ -16,7 +17,35 @@ Status: **part 1 (local, GSA v4 PGx demo, 36 samples) passed**; part 2 (1000 Gen
 GSA demo observations: all 36 samples PASS (autosomal call rate ≥ 0.997, LogR dev 0.13–0.17), control status CLEAR;
 contamination 0.0012–0.0040; replicate pairs (φ ≈ 0.50) and first-degree relatives (φ ≈ 0.25) are present (K-019).
 
-## Part 2 — cohort scale (pending)
+## Part 2 — cohort scale, 1000 Genomes Omni2.5 (2141 samples)
 
-1000 Genomes Omni2.5, 2141 samples: relatedness and Mendelian errors against the 1000G pedigree (trios), sex against the
-pedigree, contamination and control-metric distributions, runtime/memory of the disk-backed cohort store.
+`simurg genotype --preset autocall-1.6.2 --gtc-version 5 --qc-profile genotyping` on all 2141 IDAT pairs, with control
+tables and cohort QC (contamination, KING relatedness), then `scripts/validate_p5_large.py` against the public
+1000G pedigree (`integrated_call_samples.20130502.ALL.ped`) and genotype VCF. The GTCs carry lab sample names, so every
+sample was identified by its heterozygosity pattern over 20 000 common SNPs (orientation-free).
+
+| Check | Result |
+|---|---|
+| Sample identification | 2139/2141 unique (concordance 1.0); 2 ambiguous (margin < 1e-4) |
+| **Parent–offspring pairs** of the pedigree detected as first degree | **717/717**; KING φ median 0.247 (min 0.192, max 0.259) |
+| **Mendelian error rate**, 335 complete trios, autosomal sites | median **0.031 %**, max 0.108 % |
+| **Sex estimate vs pedigree** | 2088 match, 9 mismatch (99.6 %), 44 without pedigree sex |
+| Close pairs (≥ second degree) not explained by parent/sibling records | 69 (cryptic relatives, mostly MXL/ASW/PUR/CHS second degree — known in 1000G) |
+| Contamination (BAF regression, autosomal sites) | median 0.0002; 15 samples > 0.02 (max 0.075, HG00707) |
+| Autosomal call rate | median 0.9966, min 0.9746 (2 below 0.98) |
+| LRR SD | median 0.187, p90 0.219 → 602 samples above 0.20 (functional FAIL 604): a property of this legacy data set; recalibrate with `qc suggest-thresholds` |
+| Runtime / memory | genotype + control tables + cohort QC: 34 min, 31 GB peak RSS (16 vCPU); validation 5 min |
+
+Control metrics exposed two issues, fixed after the run (K-021): absent second bead-pool addresses poisoned the bead
+means (1794 false staining flags), and background-subtracted Biotin (Bgnd) = 0 made staining green signal/0. Recomputed
+from the run's `controls_raw.tsv`: control status 2139 CLEAR, 2 FLAGGED (DNP (High) red = 0 — genuine).
+
+### P4 re-check with `--preset autocall-1.6.2` (same run)
+
+2138/2141 GTCs **byte-identical** to the vendor's; the 3 others are the K-009 samples. GC10/GC50/#calls, which
+differed in all 2141 before the preset, now match (`P4_preset_omni2141_summary.json`).
+
+### Not done
+
+Sanger Omni2.5-8 BAF/LRR vs the GenomeStudio FinalReport: the 1000G Sanger folder has IDATs and the report but no
+cluster (EGT) file or BPM, so BAF/LRR cannot be recomputed.