diff --git a/docs/known_unknowns.md b/docs/known_unknowns.md index 1eda868..626a2dc 100644 --- a/docs/known_unknowns.md +++ b/docs/known_unknowns.md @@ -4,7 +4,7 @@ Plan §20 lists the open questions; this file records what the implementation ha | ID | Date | Topic | Finding | Status | |---|---|---|---|---| -| K-027 | 2026-09-29 | CN/LOH (v2) | CYTO-014 noise gate: the summary `logr_dev` also measures cluster-file mismatch (GSE165845: 0.246, all samples `SampleQuality`) while the LRR used after the batch reference has SD ≈ 0.1; the gate now uses the SD of the normalised LRR (the JSON still reports the summary value). Spec: built-in mask/cytobands from UCSC; the annotation bundle uses UCSC ncbiRefSeq instead of RefSeq GFF + Ensembl GTF + HGNC. Real-data result: constitutional DEL/DUP calls 74–77 % in NYGC truth, mosaic calls mostly false positives. | Open: mosaic calling needs rework (`docs/validation/V2_cn.md`). | +| K-027 | 2026-09-29 | CN/LOH (v2) | CYTO-014 noise gate: the summary `logr_dev` also measures cluster-file mismatch (GSE165845: 0.246, all samples `SampleQuality`) while the LRR used after the batch reference has SD ≈ 0.1; the gate now uses the SD of the normalised LRR (the JSON still reports the summary value). Spec: built-in mask/cytobands from UCSC; the annotation bundle uses UCSC ncbiRefSeq instead of RefSeq GFF + Ensembl GTF + HGNC. Real-data result: constitutional DEL/DUP calls 74–77 % in NYGC truth, mosaic calls mostly false positives. Pi tuning: mosaic prior / switch distance barely matter; CYTO-041 (noise-normalised BAF split, complete-loss promotion) cuts PASS mosaic calls by ~30 % without losing implanted ≥ 15 % mosaics. Two limits remain: (1) without phasing, < 20 % mosaic losses are not separable from regional BAF noise; (2) common CNVs (carrier frequency > 20 %) are almost never detected (1/236 DEL) because the median-LRR batch reference absorbs them; rare DEL (< 5 %) 38 %. | Open: rare-event sensitivity as primary metric; phasing-based mosaic BAF (`docs/validation/V2_cn.md`). | | K-026 | 2026-09-29 | Methylation QC (v2) | Spec deviations: vendor EPIC manifests put a 5th field on some control lines (≥ 4 fields accepted); detection p follows sesame's pOOBAH (1 − max(F(M), F(U))) rather than the ambiguous "1 − ECDF(M + U)"; minfi-style `Sentrix_ID`/`Sentrix_Position` sheet keys accepted. Against sesame 1.28.1 on minfiDataEPIC: β r = 0.99998 (max \|Δβ\| 0.011), detection calls 99.1 % identical, noob signals within 0.5 % (median) / 5 % (max). The systematic remainder is unexplained; it is not the negative controls (sesame did not attach the supplied control table). | Open; `docs/validation/V2_methylation.md`. Next step: dump sesame's background vectors and noob parameters to find the difference. | | K-025 | 2026-09-28 | Scaling limits (P9) | With the Rust kernels a Graviton3 core genotypes 78–80 GSA samples/min. At > 8 workers a 125 MB/s gp3 volume caps the run at ~620 samples/min (GTC ≈ 13 MB each); on a RAM disk efficiency is 0.77 at 16 and 0.48 at 32 workers for 360 samples, explained by a fixed start-up cost (≈ 2.5 s + 0.2 s per worker). Linear scaling for large batches is inferred from that model, not measured. | `docs/validation/P9.md`; measure a ≥ 5000-sample batch when one is at hand. | | K-024 | 2026-09-28 | GSE165845 (P8) | GSA-MD v1 chips have no public cluster file; the GSA v1 C1 EGT with the C2 build-38 BPM covers 99.996 % of bead addresses and gives median LogR deviation 0.246 (cluster mismatch, not noise). GSM5051309 (NA20362 rep 2) is a failed array (call rate 0.762, concordance 0.336 with its own replicates). 27 510 array records have no NYGC 30x record and are scored as hom-ref; 1444 of them are systematically called alt. | Recorded in `docs/validation/P8.md`; results reported with and without these. | diff --git a/docs/validation/V2_cn.md b/docs/validation/V2_cn.md index df69fff..e3ac20d 100644 --- a/docs/validation/V2_cn.md +++ b/docs/validation/V2_cn.md @@ -53,11 +53,34 @@ BAF-supported "mosaic" events are 0–2 % in truth and rarely reproduced: shifte mismatched cluster file mimic allelic imbalance). Small BAF-supported mosaic gains are often real germline duplications labelled mosaic (43 % in truth). +## Pi tuning rounds (2026-09-29), 40 arrays (20 individuals × replicates 1, 2), reference = replicate 3 of all 120 + +`scripts/v2/cn_tune.sh TAG [options]`; truth as above, QUAL ≥ 20, PASS only (after the K-027 noise-gate fix). + +| Round | DEL calls / in truth | DUP | mosaic loss PASS / in truth | mosaic gain PASS / in truth | LOH replicated | +|---|---|---|---|---|---| +| base (per-probe reference het BAF spread in the emissions) | 22 / 100 % | 3 / 67 % | 29 / 11 % | 24 / 38 % | 80 % | +| `--mosaic-prior 0.0005` | 25 / 96 % | 4 / 75 % | 27 / 12 % | 22 / 36 % | 80 % | +| `--mosaic-prior 0.0005 --switch-distance 1e8` | 24 / 96 % | 4 / 75 % | 25 / 14 % | 23 / 35 % | 80 % | +| CYTO-041 (noise-normalised BAF split, complete-loss promotion) | 27 / 93 % | 4 / 75 % | **19** / 5 % | **19** / 42 % | 80 % | + +* The mosaic prior and switch distance barely matter. CYTO-041 moves heterozygote-free "mosaic" losses (compressed + constitutional deletions) to DEL (`HIGHFRACTION`) and sends noise-driven mosaic calls to `LowFraction` + (MDEL 29 → 19 PASS, MDUP 24 → 19). On the implanted set (`data/v2cn/syn3`) MF estimates stay unbiased + (15 % → 0.16, 20 % → 0.21, 30 % → 0.30), detection is unchanged (126/216) and the CYTO-022 gate still passes. +* Remaining mosaic-loss false positives are long, low-amplitude LRR dips (−0.05 … −0.1) with a heterozygous BAF + spread only 1.0–1.3 × the probe's reference spread, or short events with < 20 heterozygous probes (MF from LRR). + Without phasing a < 20 % mosaic loss cannot be separated from regional BAF noise (K-027). +* **Sensitivity is limited by common CNVs.** Detected truth DEL by carrier frequency among the 120 individuals: + < 5 %: 24/64 (38 %), 5–20 %: 1/20, 20–50 %: 1/80, > 50 %: 0/156 (DUP 10/66, 0/2, 1/24, 0/40). The batch + reference is the per-probe median LRR, so an event carried by most reference samples is normalised away (and the + cluster file, trained on the population, already absorbs part of it). Array CN calling targets rare events; the + headline sensitivity figures above are dominated by common polymorphic deletions. + ## Next steps -1. Mosaic calling: require consistent BAF imbalance with a per-probe reference of heterozygous BAF spread, raise the - evidence for mosaic states (prior/size), and relabel compressed germline gains/losses as constitutional; tune on - GSE165845 and re-measure (replicate reproducibility is the cheapest check). -2. Sensitivity for 25–500 kb events: lower switch cost for short events with strong LRR, per-probe LRR SD from the - reference. -3. Re-run the truth comparison; document the final numbers. +1. Report sensitivity for rare events (carrier frequency < 5 %) as the primary metric in `cn_validate_truth.py`. +2. Mosaic losses: phasing-based BAF (MoChA-style) is the principled fix for low fractions; until then consider a + minimum heterozygous-probe count for mosaic PASS calls. +3. Sensitivity for 25–500 kb events: per-probe LRR SD from the reference. +4. A final full-cohort run (cloud, ~$0.10, needs approval) once the above settle.