SimurgArray/docs/validation/V2_cn.md

# v2 module validation — genome-wide CNV / LOH / mosaicism (`simurg cn`) — interim

Date: 2026-09-29. Status: **not accepted yet** — constitutional calling works, mosaic calling produces mostly false
positives on the real-data set; see "Next steps". Spec: CYTO-001 … CYTO-040.

## Delivered

`simurg cn build-model` (GC 1 kb / 100 kb / 1 Mb, PFB from the cluster file, sex-aware batch reference from normal
GTCs), `simurg cn call` (GC-wave correction, smoothing, HMM with constitutional, gain-LOH, mosaic and haploid
states, events, post-filters, thresholds, centromere mask, CNV VCF + tabix, segments, summary, sidecar),
`simurg cn annotate` (JSON report, ISCN 2020, cytobands, RefSeq genes), `scripts/build_annotation_bundle.sh`.
Unit tests: `tests/unit/test_cn.py` (11).

## Synthetic implants (CYTO-022), GSA v4 demo, 22 synthetic samples, 216 events

Reference model from 30 samples, events implanted into the other 6 (`scripts/v2/cn_synthetic_eval.py`,
`V2_cn_synthetic_gsa36.json`), 4 replicates per cell:

| Event | 1 Mb | 5 Mb | 20 Mb |
|---|---|---|---|
| constitutional DEL / DUP | 4/4 · 2/4 | 4/4 · 4/4 | 4/4 · 4/4 |
| constitutional LOH (threshold 3 Mb) | 0/4 | 3/4 | 4/4 |
| mosaic loss / gain / CN-LOH, 15 % | 3/4 · 1/4 · 0/4 | **4/4 · 4/4 · 4/4** | 4/4 · 4/4 · 4/4 |
| mosaic, 5–10 % | mostly missed | mostly missed | 0–3/4 |

The CYTO-022 gate (≥ 90 % at 15 % / 5 Mb) is met on implanted data (12/12, small n).

## Real data: GSE165845 (360 GSA-MD arrays, 120 individuals × 3) vs NYGC 30x SV callset (freeze V3)

Cloud run 2026-09-29 (m7g.4xlarge, `cn call` 360 samples in 7 min 40 s; reference from 120 arrays of the same set).
Truth: non-reference DEL/DUP ≥ 25 kb of the same individual, ≥ 10 array probes; match = reciprocal overlap ≥ 50 %.

* Every sample was flagged `SampleQuality` (summary `logr_dev` median 0.246 — the GSA v1 C1 cluster file does not
  match the GSA-MD chips, P8) and 46 fell into whole-chromosome-only mode, although the LRR used for calling has a
  robust SD of 0.099 after the batch reference. The gate now uses the SD of the normalised LRR (K-027). The numbers
  below ignore `SampleQuality` (`--ignore-filter`), evaluated from the run's sidecar.

| QUAL ≥ 20 | DEL | DUP |
|---|---|---|
| sensitivity 25–50 / 50–100 / 100–500 / 500–1000 kb | 7 / 8 / 8 / 7 % | 1 / 3 / 11 / 13 % |
| precision (all calls of the class) | 39 % | 34 % |

| Call type (QUAL ≥ 20) | calls | in NYGC truth | seen in another replicate |
|---|---|---|---|
| constitutional DEL | 227 | **77 %** | 50 % |
| constitutional DUP | 23 | **74 %** | 30 % |
| LOH | — | — | 74 % |
| mosaic loss | 399 | 12 % | 12 % |
| mosaic gain | 567 | 23 % | 23 % |

Constitutional calls are mostly true; the mosaic states generate most calls and most false positives (large
BAF-supported "mosaic" events are 0–2 % in truth and rarely reproduced: shifted heterozygous BAF bands of the
mismatched cluster file mimic allelic imbalance). Small BAF-supported mosaic gains are often real germline
duplications labelled mosaic (43 % in truth).

## Pi tuning rounds (2026-09-29), 40 arrays (20 individuals × replicates 1, 2), reference = replicate 3 of all 120

`scripts/v2/cn_tune.sh TAG [options]`; truth as above, QUAL ≥ 20, PASS only (after the K-027 noise-gate fix).

| Round | DEL calls / in truth | DUP | mosaic loss PASS / in truth | mosaic gain PASS / in truth | LOH replicated |
|---|---|---|---|---|---|
| base (per-probe reference het BAF spread in the emissions) | 22 / 100 % | 3 / 67 % | 29 / 11 % | 24 / 38 % | 80 % |
| `--mosaic-prior 0.0005` | 25 / 96 % | 4 / 75 % | 27 / 12 % | 22 / 36 % | 80 % |
| `--mosaic-prior 0.0005 --switch-distance 1e8` | 24 / 96 % | 4 / 75 % | 25 / 14 % | 23 / 35 % | 80 % |
| CYTO-041 (noise-normalised BAF split, complete-loss promotion) | 27 / 93 % | 4 / 75 % | 19 / 5 % | 19 / 42 % | 80 % |
| + CYTO-042 (per-probe LRR noise from the reference; **default**) | 20 / **100 %** | 4 / **100 %** | **11** / 9 % | **13** / **62 %** | 80 % |

* The mosaic prior and switch distance barely matter. CYTO-041 moves heterozygote-free "mosaic" losses (compressed
  constitutional deletions) to DEL (`HIGHFRACTION`) and sends noise-driven mosaic calls to `LowFraction`
  (MDEL 29 → 19 PASS, MDUP 24 → 19). On the implanted set (`data/v2cn/syn3`) MF estimates stay unbiased
  (15 % → 0.16, 20 % → 0.21, 30 % → 0.30), detection is unchanged (126/216) and the CYTO-022 gate still passes.
* Remaining mosaic-loss false positives are long, low-amplitude LRR dips (−0.05 … −0.1) with a heterozygous BAF
  spread only 1.0–1.3 × the probe's reference spread, or short events with < 20 heterozygous probes (MF from LRR).
  Without phasing a < 20 % mosaic loss cannot be separated from regional BAF noise (K-027).
* CYTO-042 (per-probe emission SD = sample SD × the probe's reference LRR spread / median spread, clipped to
  [0.5, 3]) halves the mosaic calls and makes constitutional calls fully precise, at the cost of 5 rare deletions
  (rare DEL 24/64 → 19/64; rare DUP 10/68 → 12/68). Synthetic set with the rebuilt model (`data/v2cn/syn4`): calls
  564 → 402, detection 126 → 124/216, the CYTO-022 gate (15 % / 5 Mb: 4/4 loss, gain, CN-LOH) still passes.
  Disable with `--no-probe-lrr-sd`.
* **Sensitivity is limited by common CNVs.** Detected truth DEL by carrier frequency among the 120 individuals:
  < 5 %: 24/64 (38 %), 5–20 %: 1/20, 20–50 %: 1/80, > 50 %: 0/156 (DUP 10/66, 0/2, 1/24, 0/40). The batch
  reference is the per-probe median LRR, so an event carried by most reference samples is normalised away (and the
  cluster file, trained on the population, already absorbs part of it). Array CN calling targets rare events; the
  headline sensitivity figures above are dominated by common polymorphic deletions.

## Final full-cohort run (cloud, 2026-09-29, commit ccf7f4f, CYTO-041/042 defaults)

The run used m7g.4xlarge spot: 17 min in total (`cn build-model` 1 min 17 s, 6.0 GB; `cn call` of 360 samples 7 min 44 s, 1.7 GB).
Truth and matching are the same as above. PASS calls only, QUAL ≥ 20, no filters ignored
(`V2_cn_gse165845_final.json`).

* **Sample QC.** All 360 samples are `OK`; the first cloud run flagged every sample `SampleQuality` (the K-027 gate
  fix). 14 samples carry `high_noise` and one carries `excess_events`.
* **The `excess_events` array.** 203610890020_R06C01 (NA19923) has 238 DEL and 688 mosaic calls, and none of its
  147 CNV calls is in truth. It is a bad array, and it is the only one flagged.
* **Per-sample call counts (median).** 1 DEL, 0 DUP, 4 mosaic.

| | DEL | DUP |
|---|---|---|
| precision, PASS calls | 213/480 = **44 %** (first run, filters ignored: 39 %) | 142/249 = **57 %** (first run, filters ignored: 34 %) |
| precision, CNV calls without R06C01 | 355/582 = **61 %** (DEL + DUP) | |
| sensitivity 25–50 / 50–100 / 100–500 / 500–1000 kb | 5 / 7 / 8 / 9 % | 0 / 3 / 10 / 14 % |
| sensitivity by carrier frequency < 5 % / 5–20 % / ≥ 20 % | **30 %** / 5 % / 0.05 % | 13 % / 21 % / 0.6 % |

* **Replicate reproducibility.** CNV 35 %, LOH 85 %.
* **What CYTO-041/042 changed.** Precision is clearly higher and sensitivity is unchanged. The low overall
  sensitivity is the common-CNV effect described above, not missed rare events.
* **Rare deletions.** 30 % of rare (< 5 %) truth deletions are found, the same as in the Pi tuning.

## Next steps

1. Mosaic losses below ~20 %: phasing-based BAF (MoChA-style, MIT) is the principled fix.
2. Rare-event sensitivity (currently DEL 30 %, DUP 18 % for events ≥ 25 kb with ≥ 10 probes): exclude reference
   samples carrying the event when computing the per-probe median (iterative reference), size-aware transitions.
3. ~~A final full-cohort run~~ — done (section above).