v2 CN: real-data validation vs NYGC 30x SVs (interim), noise gate on normalised LRR (K-027)
GSE165845 360 arrays: constitutional DEL/DUP calls 77/74 % in the NYGC SV truth, LOH reproduced across replicates 74 %; mosaic calls are mostly false positives (12-23 %), sensitivity for 25 kb-1 Mb events 7-13 % at QUAL >= 20. All samples had been flagged SampleQuality by the summary logr_dev (cluster-file mismatch); the gate now uses the SD of the LRR after the batch reference. Validation script: --ignore-filter.
This commit is contained in:
kemal committed
1 parent
726fe045f5
commit
a5728d84e3
6 files changed
+252
-3
No files matched your search
@@ -0,0 +1,63 @@
|
||||
# v2 module validation — genome-wide CNV / LOH / mosaicism (`simurg cn`) — interim
|
||||
|
||||
Date: 2026-09-29. Status: **not accepted yet** — constitutional calling works, mosaic calling produces mostly false
|
||||
positives on the real-data set; see "Next steps". Spec: CYTO-001 … CYTO-040.
|
||||
|
||||
## Delivered
|
||||
|
||||
`simurg cn build-model` (GC 1 kb / 100 kb / 1 Mb, PFB from the cluster file, sex-aware batch reference from normal
|
||||
GTCs), `simurg cn call` (GC-wave correction, smoothing, HMM with constitutional, gain-LOH, mosaic and haploid
|
||||
states, events, post-filters, thresholds, centromere mask, CNV VCF + tabix, segments, summary, sidecar),
|
||||
`simurg cn annotate` (JSON report, ISCN 2020, cytobands, RefSeq genes), `scripts/build_annotation_bundle.sh`.
|
||||
Unit tests: `tests/unit/test_cn.py` (11).
|
||||
|
||||
## Synthetic implants (CYTO-022), GSA v4 demo, 22 synthetic samples, 216 events
|
||||
|
||||
Reference model from 30 samples, events implanted into the other 6 (`scripts/v2/cn_synthetic_eval.py`,
|
||||
`V2_cn_synthetic_gsa36.json`), 4 replicates per cell:
|
||||
|
||||
| Event | 1 Mb | 5 Mb | 20 Mb |
|
||||
|---|---|---|---|
|
||||
| constitutional DEL / DUP | 4/4 · 2/4 | 4/4 · 4/4 | 4/4 · 4/4 |
|
||||
| constitutional LOH (threshold 3 Mb) | 0/4 | 3/4 | 4/4 |
|
||||
| mosaic loss / gain / CN-LOH, 15 % | 3/4 · 1/4 · 0/4 | **4/4 · 4/4 · 4/4** | 4/4 · 4/4 · 4/4 |
|
||||
| mosaic, 5–10 % | mostly missed | mostly missed | 0–3/4 |
|
||||
|
||||
The CYTO-022 gate (≥ 90 % at 15 % / 5 Mb) is met on implanted data (12/12, small n).
|
||||
|
||||
## Real data: GSE165845 (360 GSA-MD arrays, 120 individuals × 3) vs NYGC 30x SV callset (freeze V3)
|
||||
|
||||
Cloud run 2026-09-29 (m7g.4xlarge, `cn call` 360 samples in 7 min 40 s; reference from 120 arrays of the same set).
|
||||
Truth: non-reference DEL/DUP ≥ 25 kb of the same individual, ≥ 10 array probes; match = reciprocal overlap ≥ 50 %.
|
||||
|
||||
* Every sample was flagged `SampleQuality` (summary `logr_dev` median 0.246 — the GSA v1 C1 cluster file does not
|
||||
match the GSA-MD chips, P8) and 46 fell into whole-chromosome-only mode, although the LRR used for calling has a
|
||||
robust SD of 0.099 after the batch reference. The gate now uses the SD of the normalised LRR (K-027). The numbers
|
||||
below ignore `SampleQuality` (`--ignore-filter`), evaluated from the run's sidecar.
|
||||
|
||||
| QUAL ≥ 20 | DEL | DUP |
|
||||
|---|---|---|
|
||||
| sensitivity 25–50 / 50–100 / 100–500 / 500–1000 kb | 7 / 8 / 8 / 7 % | 1 / 3 / 11 / 13 % |
|
||||
| precision (all calls of the class) | 39 % | 34 % |
|
||||
|
||||
| Call type (QUAL ≥ 20) | calls | in NYGC truth | seen in another replicate |
|
||||
|---|---|---|---|
|
||||
| constitutional DEL | 227 | **77 %** | 50 % |
|
||||
| constitutional DUP | 23 | **74 %** | 30 % |
|
||||
| LOH | — | — | 74 % |
|
||||
| mosaic loss | 399 | 12 % | 12 % |
|
||||
| mosaic gain | 567 | 23 % | 23 % |
|
||||
|
||||
Constitutional calls are mostly true; the mosaic states generate most calls and most false positives (large
|
||||
BAF-supported "mosaic" events are 0–2 % in truth and rarely reproduced: shifted heterozygous BAF bands of the
|
||||
mismatched cluster file mimic allelic imbalance). Small BAF-supported mosaic gains are often real germline
|
||||
duplications labelled mosaic (43 % in truth).
|
||||
|
||||
## Next steps
|
||||
|
||||
1. Mosaic calling: require consistent BAF imbalance with a per-probe reference of heterozygous BAF spread, raise the
|
||||
evidence for mosaic states (prior/size), and relabel compressed germline gains/losses as constitutional; tune on
|
||||
GSE165845 and re-measure (replicate reproducibility is the cheapest check).
|
||||
2. Sensitivity for 25–500 kb events: lower switch cost for short events with strong LRR, per-probe LRR SD from the
|
||||
reference.
|
||||
3. Re-run the truth comparison; document the final numbers.
|
||||
@@ -0,0 +1,89 @@
|
||||
{
|
||||
"samples": 360,
|
||||
"min_qual": 10.0,
|
||||
"ignored_filters": [
|
||||
"SampleQuality"
|
||||
],
|
||||
"sensitivity": {
|
||||
"DEL": {
|
||||
"100-500kb": {
|
||||
"found": 230,
|
||||
"truth": 1554,
|
||||
"rate": 0.148
|
||||
},
|
||||
"1000-infkb": {
|
||||
"found": 0,
|
||||
"truth": 15,
|
||||
"rate": 0.0
|
||||
},
|
||||
"25-50kb": {
|
||||
"found": 54,
|
||||
"truth": 225,
|
||||
"rate": 0.24
|
||||
},
|
||||
"50-100kb": {
|
||||
"found": 132,
|
||||
"truth": 894,
|
||||
"rate": 0.1477
|
||||
},
|
||||
"500-1000kb": {
|
||||
"found": 51,
|
||||
"truth": 180,
|
||||
"rate": 0.2833
|
||||
}
|
||||
},
|
||||
"DUP": {
|
||||
"100-500kb": {
|
||||
"found": 279,
|
||||
"truth": 1059,
|
||||
"rate": 0.2635
|
||||
},
|
||||
"1000-infkb": {
|
||||
"found": 9,
|
||||
"truth": 21,
|
||||
"rate": 0.4286
|
||||
},
|
||||
"25-50kb": {
|
||||
"found": 4,
|
||||
"truth": 198,
|
||||
"rate": 0.0202
|
||||
},
|
||||
"50-100kb": {
|
||||
"found": 55,
|
||||
"truth": 462,
|
||||
"rate": 0.119
|
||||
},
|
||||
"500-1000kb": {
|
||||
"found": 19,
|
||||
"truth": 78,
|
||||
"rate": 0.2436
|
||||
}
|
||||
}
|
||||
},
|
||||
"precision": {
|
||||
"DEL": {
|
||||
"in_truth": 473,
|
||||
"calls": 1814,
|
||||
"rate": 0.2607
|
||||
},
|
||||
"DUP": {
|
||||
"in_truth": 367,
|
||||
"calls": 1494,
|
||||
"rate": 0.2456
|
||||
}
|
||||
},
|
||||
"replicate_reproducibility": {
|
||||
"CNV": {
|
||||
"matched": 2132,
|
||||
"comparisons": 6616,
|
||||
"rate": 0.3222
|
||||
},
|
||||
"LOH": {
|
||||
"matched": 400,
|
||||
"comparisons": 532,
|
||||
"rate": 0.7519
|
||||
}
|
||||
},
|
||||
"calls_per_sample_median": 6.0,
|
||||
"loh_calls_per_sample_median": 0.0
|
||||
}
|
||||
@@ -0,0 +1,89 @@
|
||||
{
|
||||
"samples": 360,
|
||||
"min_qual": 20.0,
|
||||
"ignored_filters": [
|
||||
"SampleQuality"
|
||||
],
|
||||
"sensitivity": {
|
||||
"DEL": {
|
||||
"100-500kb": {
|
||||
"found": 126,
|
||||
"truth": 1554,
|
||||
"rate": 0.0811
|
||||
},
|
||||
"1000-infkb": {
|
||||
"found": 0,
|
||||
"truth": 15,
|
||||
"rate": 0.0
|
||||
},
|
||||
"25-50kb": {
|
||||
"found": 16,
|
||||
"truth": 225,
|
||||
"rate": 0.0711
|
||||
},
|
||||
"50-100kb": {
|
||||
"found": 70,
|
||||
"truth": 894,
|
||||
"rate": 0.0783
|
||||
},
|
||||
"500-1000kb": {
|
||||
"found": 13,
|
||||
"truth": 180,
|
||||
"rate": 0.0722
|
||||
}
|
||||
},
|
||||
"DUP": {
|
||||
"100-500kb": {
|
||||
"found": 117,
|
||||
"truth": 1059,
|
||||
"rate": 0.1105
|
||||
},
|
||||
"1000-infkb": {
|
||||
"found": 6,
|
||||
"truth": 21,
|
||||
"rate": 0.2857
|
||||
},
|
||||
"25-50kb": {
|
||||
"found": 1,
|
||||
"truth": 198,
|
||||
"rate": 0.0051
|
||||
},
|
||||
"50-100kb": {
|
||||
"found": 15,
|
||||
"truth": 462,
|
||||
"rate": 0.0325
|
||||
},
|
||||
"500-1000kb": {
|
||||
"found": 10,
|
||||
"truth": 78,
|
||||
"rate": 0.1282
|
||||
}
|
||||
}
|
||||
},
|
||||
"precision": {
|
||||
"DEL": {
|
||||
"in_truth": 222,
|
||||
"calls": 571,
|
||||
"rate": 0.3888
|
||||
},
|
||||
"DUP": {
|
||||
"in_truth": 149,
|
||||
"calls": 436,
|
||||
"rate": 0.3417
|
||||
}
|
||||
},
|
||||
"replicate_reproducibility": {
|
||||
"CNV": {
|
||||
"matched": 554,
|
||||
"comparisons": 2014,
|
||||
"rate": 0.2751
|
||||
},
|
||||
"LOH": {
|
||||
"matched": 286,
|
||||
"comparisons": 394,
|
||||
"rate": 0.7259
|
||||
}
|
||||
},
|
||||
"calls_per_sample_median": 2.0,
|
||||
"loh_calls_per_sample_median": 0.0
|
||||
}
|
||||
Reference in new issue
Block a user