PGx CN: pgx copy-number train|call (CLI-031/032, PCN-001..017, IN-050), region BED v1, demo identities (K-028)

- simurg/pgx/cn.py: per-region CN mixture, mu_c = delta + gamma*log2((c+1)/3), free lower CN-0 component,
  Hardy-Weinberg-constrained weights over haplotype copy numbers, EM from a grid of label starts.
- pipeline/pgx_cn.py: model file (.simurg-pgxcn), batch rules (PCN-004), per-sample CN VCF + tbi, batch and
  sample summaries.
- The GSA v4 PGx demo arrays are 30 CEPH/1000G individuals (scripts/pgx/demo36_identity.tsv); against NYGC 30x
  SV RD_CN 224/240 region calls agree (GSTM1, UGT2B17 30/30), in-sample (docs/validation/V2_pgx_cn.md).

docs/known_unknowns.md

@@ -4,6 +4,7 @@ Plan §20 lists the open questions; this file records what the implementation ha
 
 | ID | Date | Topic | Finding | Status |
 |---|---|---|---|---|
+| K-028 | 2026-09-29 | PGx CN (v2) | Deviations from PCN-009: the per-probe reference is the training cohort's median stored in the model (a calling-batch median is not CN 2 where deletions are common, e.g. GSTM1 null ≈ 50 %); no GC-correction step; the CN mixture is ours: µ_c = δ + γ·log2((c+1)/3) per region with a free lower CN-0 component and Hardy–Weinberg-constrained weights (probe compression differs by region: γ ≈ 0.19 CYP2D6.p5 … 0.56 GSTT1). Interim validation on the GSA v4 PGx demo — whose 36 arrays turned out to be 30 CEPH/1000G individuals — against NYGC 30x SV RD_CN: 224/240 region calls (GSTM1, UGT2B17 30/30), but in-sample (trained on the same arrays, < 96 samples, priors chosen with this truth in view). | Open: validate on an independent GSA-PGx cohort; CYP2D6 via GeT-RM star alleles (`docs/validation/V2_pgx_cn.md`). |
 | K-027 | 2026-09-29 | CN/LOH (v2) | CYTO-014 noise gate: the summary `logr_dev` also measures cluster-file mismatch (GSE165845: 0.246, all samples `SampleQuality`) while the LRR used after the batch reference has SD ≈ 0.1; the gate now uses the SD of the normalised LRR (the JSON still reports the summary value). Spec: built-in mask/cytobands from UCSC; the annotation bundle uses UCSC ncbiRefSeq instead of RefSeq GFF + Ensembl GTF + HGNC. Real-data result: constitutional DEL/DUP calls 74–77 % in NYGC truth, mosaic calls mostly false positives. Pi tuning: mosaic prior / switch distance barely matter; CYTO-041 (noise-normalised BAF split, complete-loss promotion) cuts PASS mosaic calls by ~30 % without losing implanted ≥ 15 % mosaics. Two limits remain: (1) without phasing, < 20 % mosaic losses are not separable from regional BAF noise; (2) common CNVs (carrier frequency > 20 %) are almost never detected (1/236 DEL) because the median-LRR batch reference absorbs them; rare DEL (< 5 %) 38 %. | Open: rare-event sensitivity as primary metric; phasing-based mosaic BAF (`docs/validation/V2_cn.md`). |
 | K-026 | 2026-09-29 | Methylation QC (v2) | Spec deviations: vendor EPIC manifests put a 5th field on some control lines (≥ 4 fields accepted); detection p follows sesame's pOOBAH (1 − max(F(M), F(U))) rather than the ambiguous "1 − ECDF(M + U)"; minfi-style `Sentrix_ID`/`Sentrix_Position` sheet keys accepted. Against sesame 1.28.1 on minfiDataEPIC: β r = 0.99998 (max \|Δβ\| 0.011), detection calls 99.1 % identical, noob signals within 0.5 % (median) / 5 % (max). The systematic remainder is unexplained; it is not the negative controls (sesame did not attach the supplied control table). | Open; `docs/validation/V2_methylation.md`. Next step: dump sesame's background vectors and noob parameters to find the difference. |
 | K-025 | 2026-09-28 | Scaling limits (P9) | With the Rust kernels a Graviton3 core genotypes 78–80 GSA samples/min. At > 8 workers a 125 MB/s gp3 volume caps the run at ~620 samples/min (GTC ≈ 13 MB each); on a RAM disk efficiency is 0.77 at 16 and 0.48 at 32 workers for 360 samples, explained by a fixed start-up cost (≈ 2.5 s + 0.2 s per worker). Linear scaling for large batches is inferred from that model, not measured. | `docs/validation/P9.md`; measure a ≥ 5000-sample batch when one is at hand. |

docs/messages.md

@@ -36,11 +36,15 @@ Generated from `src/simurg/core/errors.py` — do not edit by hand.
 | `E_METH_NO_CONTROLS` | error | Missing control probes in manifest | MET-002 |
 | `E_METH_NO_IDATS` | error | No IDATs found | MET-004 |
 | `E_METH_PREP_FAILED` | error | Prep failed (sample) | MET-010 |
+| `E_MODEL_CLUSTER_MISMATCH` | error | Input GTCs were called with another cluster file than the model | IN-050 |
 | `E_MODEL_MANIFEST_MISMATCH` | error | Model was built for a different manifest probe set | IN-051 |
 | `E_MODEL_VERSION` | error | Model schema is too old; rebuild it | IN-051 |
 | `E_NO_INPUT` | error | Neither an input folder nor a sample sheet was given | CLI-006 |
 | `E_NO_INPUT_FOLDER` | error | Sample sheet has no RootFolder and no folder was given | CLI-006 |
 | `E_OUTPUT_EXISTS` | error | Outputs already exist; use --force or --resume | CLI-010 |
+| `E_PGX_BATCH_TOO_SMALL` | error | Too few (passing) samples in the batch for PGx CN calling | PCN-004 |
+| `E_PGX_CN_FAILED` | error | PGx CN calling failed for the sample | PCN-002 |
+| `E_PGX_TRAIN_TOO_SMALL` | error | Too few (passing) samples to train a PGx CN model | PCN-005 |
 | `E_QC_CONFIG_EXT` | error | QC config file must be .yaml or .yml | QC-021 |
 | `E_QC_CONFIG_MISSING` | error | QC config file does not exist | QC-021 |
 | `E_QC_CONFIG_PARSE` | error | QC config file is not valid YAML | QC-021 |
@@ -92,7 +96,12 @@ Generated from `src/simurg/core/errors.py` — do not edit by hand.
 | `W_METH_MISSING_CONTROL_TYPES` | warning | Control types missing from the manifest; metric set to NA | MET-009 |
 | `W_METH_MISSING_SAMPLE` | warning | Missing sample: IDATs absent; the sample fails, the run continues | MET-015 |
 | `W_METH_NO_FEATURES` | warning | No features, skipping PCA plot | MET-013 |
+| `W_MODEL_CLUSTER_MISMATCH` | warning | Input GTCs were called with another cluster file than the model | IN-050 |
 | `W_NO_BGZIP` | warning | Uncompressed VCF written; downstream commands need bgzip + index | OUT-010 |
+| `W_PGX_BATCH_BELOW_RECOMMENDED` | warning | Fewer passing samples than recommended | PCN-004 |
+| `W_PGX_BATCH_COMPOSITION` | warning | Batch spans several lots/batches or > 14 days of scans | PCN-006 |
+| `W_PGX_NO_ALT_CONTIG` | warning | FASTA lacks a contig carrying a PGx region; REF written as N | IN-062 |
+| `W_PGX_REGION_FEW_PROBES` | warning | PGx CN region has fewer than 10 unmasked probes | PCN-008 |
 | `W_QC_LABEL_SANITIZED` | warning | Dataset label was sanitized | QC-042 |
 | `W_QC_NO_BAF` | warning | GTC has no B-allele frequencies; contamination not estimated | QC-064 |
 | `W_QC_PEDIGREE` | warning | Parent1/Parent2/Replicate value does not name a sample in the set | IN-018 |

docs/validation/V2_pgx_cn.md

@@ -0,0 +1,48 @@
+# V2 — Targeted PGx copy number (PCN-001 … PCN-017)
+
+Status: **implemented, interim validation** (2026-09-29). `simurg pgx copy-number train|call` (CLI-031/032), model
+file `.simurg-pgxcn` (IN-050), region BED v1 built from UCSC ncbiRefSeq (`scripts/pgx/build_cn_regions.py`,
+packaged as `simurg/pgx/data/pgx_cn_regions.GRCh38.v1.bed`), per-sample `*.pgx_cn.vcf.gz` + `.tbi`,
+`pgx_cn_batch_summary.tsv`, `pgx_cn_sample_summary.tsv`. Unit tests: `tests/unit/test_pgx_cn.py`.
+
+## Data
+
+* **Arrays:** the GSA v4 PGx demo (`idats_Demo_36`, 36 arrays), genotyped by SimurgArray. Its CNV probes cover every
+  region (51–357 probes; GSTT1: 160 probes on `chr22_KI270879v1_alt` in the GRCh38 manifest).
+* **Identities:** the demo arrays are 30 CEPH / 1000 Genomes individuals (6 replicate arrays). Identified by
+  genotype concordance with the 3202-sample 1000G 30x panel on 4647 chr22 SNPs: best match 0.992–0.996, next best
+  ≤ 0.91 (relatives) — `scripts/pgx/demo36_identity.tsv`.
+* **Truth:** NYGC 30x SV callset (1KGP_3202 GATK-SV/svtools freeze V3), RD_CN per individual
+  (`scripts/pgx/validate_cn_truth.py`).
+* The GSE165845 GSA-MD v1 arrays (360 arrays, CN/LOH validation) have 0–9 probes per PGx region: not usable here.
+
+## Result (30 individuals, model trained on the same 36 arrays with `--min-samples 30`)
+
+| Region | Concordance with NYGC RD_CN | Mismatches (truth → call) |
+|---|---|---|
+| GSTM1 (whole gene) | **30/30** | – |
+| UGT2B17 (whole gene) | **30/30** | – |
+| SULT1A1 | 29/30 | 2→1 |
+| CYP2A6.exon.1 | 29/30 | 3→2 |
+| CYP2A6.intron.7 | 28/30 | 3→2, 1→2 |
+| CYP2D6.p5 | 28/30 | 4→3, 2→1 |
+| CYP2D6.intron.2 | 25/30 | 3→2 ×3, 4→3, 2→1 |
+| CYP2D6.exon.9 | 25/30 | 3→2 ×3, 4→2, 2→1 |
+| **all** | **224/240 (93.3 %)** | |
+
+GSTT1 (no SV truth on the alt contig): CN 0/1/2 = 5/16/9, in Hardy–Weinberg proportions (null allele 0.43; CEU
+literature ≈ 0.4–0.5). Details: `V2_pgx_cn_demo36.json`.
+
+* CYP2D6 truth is one gene-level record; the exon-9 / intron-2 "misses" at truth CN 3 are expected for tandem
+  structures such as `*68+*4` or `*36+*10`, where not every sub-region is duplicated. Star-allele truth (GeT-RM)
+  is the right yardstick and follows with the star-allele caller.
+* PCN-017 target (≥ 95 % per region) is met for GSTM1, UGT2B17, SULT1A1, CYP2A6.exon.1; not (yet) shown for the
+  CYP2D6 sub-regions.
+
+## Caveats (K-028)
+
+* Trained and evaluated on the same 30 individuals (the spec requires ≥ 96 training samples; the priors were
+  chosen while looking at this truth). This is an optimistic, in-sample figure until an independent cohort is
+  available.
+* Deviations from PCN-009: reference from the training cohort (not the calling batch); no GC-correction step; CN
+  model = compressed-intensity template + HWE-constrained weights (`simurg/pgx/cn.py`).

docs/validation/V2_pgx_cn_demo36.json

@@ -0,0 +1,89 @@
+{
+ "individuals": 30,
+ "regions": {
+  "GSTM1": {
+   "record": "HGSV_9691",
+   "concordant": 30,
+   "n": 30,
+   "rate": 1.0,
+   "truth_to_call_mismatches": {}
+  },
+  "UGT2B17": {
+   "record": "HGSV_63272",
+   "concordant": 30,
+   "n": 30,
+   "rate": 1.0,
+   "truth_to_call_mismatches": {}
+  },
+  "SULT1A1": {
+   "record": "HGSV_207805",
+   "concordant": 29,
+   "n": 30,
+   "rate": 0.9667,
+   "truth_to_call_mismatches": {
+    "2->1": 1
+   }
+  },
+  "CYP2A6.intron.7": {
+   "record": "HGSV_232737",
+   "concordant": 28,
+   "n": 30,
+   "rate": 0.9333,
+   "truth_to_call_mismatches": {
+    "3->2": 1,
+    "1->2": 1
+   }
+  },
+  "CYP2A6.exon.1": {
+   "record": "HGSV_232737",
+   "concordant": 29,
+   "n": 30,
+   "rate": 0.9667,
+   "truth_to_call_mismatches": {
+    "3->2": 1
+   }
+  },
+  "CYP2D6.exon.9": {
+   "record": "HGSV_247317",
+   "concordant": 25,
+   "n": 30,
+   "rate": 0.8333,
+   "truth_to_call_mismatches": {
+    "4->2": 1,
+    "3->2": 3,
+    "2->1": 1
+   }
+  },
+  "CYP2D6.intron.2": {
+   "record": "HGSV_247317",
+   "concordant": 25,
+   "n": 30,
+   "rate": 0.8333,
+   "truth_to_call_mismatches": {
+    "4->3": 1,
+    "3->2": 3,
+    "2->1": 1
+   }
+  },
+  "CYP2D6.p5": {
+   "record": "HGSV_247317",
+   "concordant": 28,
+   "n": 30,
+   "rate": 0.9333,
+   "truth_to_call_mismatches": {
+    "4->3": 1,
+    "2->1": 1
+   }
+  }
+ },
+ "overall": {
+  "concordant": 224,
+  "n": 240,
+  "rate": 0.9333
+ },
+ "gstt1_cn_distribution": {
+  "0": 5,
+  "1": 16,
+  "2": 9
+ }
+}