CN: final full-cohort cloud validation (GSE165845, CYTO-041/042 defaults)

360 arrays vs NYGC 30x SV: all samples OK (K-027 gate fix), PASS precision DEL 44 % / DUP 57 % (first run 39 % / 34 % with filters ignored), 61 % without the one excess_events array; rare DEL sensitivity 30 %; LOH replicated 85 %. m7g.4xlarge spot, 17 min. docs/validation/V2_cn.md, K-027.
2026-09-29 23:04:16 +01:00
1 parent 7c57eb5586
commit 08e67cff29
@@ -83,9 +83,33 @@ duplications labelled mosaic (43 % in truth).
  cluster file, trained on the population, already absorbs part of it). Array CN calling targets rare events; the
  headline sensitivity figures above are dominated by common polymorphic deletions.

+## Final full-cohort run (cloud, 2026-09-29, commit ccf7f4f, CYTO-041/042 defaults)
+
+The run used m7g.4xlarge spot: 17 min in total (`cn build-model` 1 min 17 s, 6.0 GB; `cn call` of 360 samples 7 min 44 s, 1.7 GB).
+Truth and matching are the same as above. PASS calls only, QUAL ≥ 20, no filters ignored
+(`V2_cn_gse165845_final.json`).
+
+* **Sample QC.** All 360 samples are `OK`; the first cloud run flagged every sample `SampleQuality` (the K-027 gate
+  fix). 14 samples carry `high_noise` and one carries `excess_events`.
+* **The `excess_events` array.** 203610890020_R06C01 (NA19923) has 238 DEL and 688 mosaic calls, and none of its
+  147 CNV calls is in truth. It is a bad array, and it is the only one flagged.
+* **Per-sample call counts (median).** 1 DEL, 0 DUP, 4 mosaic.
+
+| | DEL | DUP |
+|---|---|---|
+| precision, PASS calls | 213/480 = **44 %** (first run, filters ignored: 39 %) | 142/249 = **57 %** (first run, filters ignored: 34 %) |
+| precision, CNV calls without R06C01 | 355/582 = **61 %** (DEL + DUP) | |
+| sensitivity 25–50 / 50–100 / 100–500 / 500–1000 kb | 5 / 7 / 8 / 9 % | 0 / 3 / 10 / 14 % |
+| sensitivity by carrier frequency < 5 % / 5–20 % / ≥ 20 % | **30 %** / 5 % / 0.05 % | 13 % / 21 % / 0.6 % |
+
+* **Replicate reproducibility.** CNV 35 %, LOH 85 %.
+* **What CYTO-041/042 changed.** Precision is clearly higher and sensitivity is unchanged. The low overall
+  sensitivity is the common-CNV effect described above, not missed rare events.
+* **Rare deletions.** 30 % of rare (< 5 %) truth deletions are found, the same as in the Pi tuning.
+
 ## Next steps

 1. Mosaic losses below ~20 %: phasing-based BAF (MoChA-style, MIT) is the principled fix.
 2. Rare-event sensitivity (currently DEL 30 %, DUP 18 % for events ≥ 25 kb with ≥ 10 probes): exclude reference
   samples carrying the event when computing the per-probe median (iterative reference), size-aware transitions.
-3. A final full-cohort run (cloud, ~$0.10, needs approval).
+3. ~~A final full-cohort run~~ — done (section above).
@@ -0,0 +1,123 @@
+{
+ "samples": 360,
+ "min_qual": 20,
+ "ignored_filters": [],
+ "sensitivity": {
+  "DEL": {
+   "100-500kb": {
+    "found": 128,
+    "truth": 1554,
+    "rate": 0.0824
+   },
+   "1000-infkb": {
+    "found": 0,
+    "truth": 15,
+    "rate": 0.0
+   },
+   "25-50kb": {
+    "found": 12,
+    "truth": 225,
+    "rate": 0.0533
+   },
+   "50-100kb": {
+    "found": 59,
+    "truth": 894,
+    "rate": 0.066
+   },
+   "500-1000kb": {
+    "found": 16,
+    "truth": 180,
+    "rate": 0.0889
+   }
+  },
+  "DUP": {
+   "100-500kb": {
+    "found": 109,
+    "truth": 1059,
+    "rate": 0.1029
+   },
+   "1000-infkb": {
+    "found": 9,
+    "truth": 21,
+    "rate": 0.4286
+   },
+   "25-50kb": {
+    "found": 0,
+    "truth": 198,
+    "rate": 0.0
+   },
+   "50-100kb": {
+    "found": 13,
+    "truth": 462,
+    "rate": 0.0281
+   },
+   "500-1000kb": {
+    "found": 11,
+    "truth": 78,
+    "rate": 0.141
+   }
+  }
+ },
+ "sensitivity_by_carrier_frequency": {
+  "DEL": {
+   "5-20%": {
+    "found": 15,
+    "truth": 285,
+    "rate": 0.0526
+   },
+   "common>=20%": {
+    "found": 1,
+    "truth": 1923,
+    "rate": 0.0005
+   },
+   "rare<5%": {
+    "found": 199,
+    "truth": 660,
+    "rate": 0.3015
+   }
+  },
+  "DUP": {
+   "5-20%": {
+    "found": 21,
+    "truth": 102,
+    "rate": 0.2059
+   },
+   "common>=20%": {
+    "found": 5,
+    "truth": 855,
+    "rate": 0.0058
+   },
+   "rare<5%": {
+    "found": 116,
+    "truth": 861,
+    "rate": 0.1347
+   }
+  }
+ },
+ "precision": {
+  "DEL": {
+   "in_truth": 213,
+   "calls": 480,
+   "rate": 0.4437
+  },
+  "DUP": {
+   "in_truth": 142,
+   "calls": 249,
+   "rate": 0.5703
+  }
+ },
+ "replicate_reproducibility": {
+  "CNV": {
+   "matched": 512,
+   "comparisons": 1458,
+   "rate": 0.3512
+  },
+  "LOH": {
+   "matched": 356,
+   "comparisons": 420,
+   "rate": 0.8476
+  }
+ },
+ "calls_per_sample_median": 1.0,
+ "loh_calls_per_sample_median": 0.0
+}