Release 2.0.0: v2 modules (ArrayTrain, CN/LOH, PGx, methylation QC)

User guide: ArrayTrain section, v2 commands in the table, v2 accuracy rows,
updated limitations. README status for v2.0.0. Version 2.0.0 (Python package);
the Rust kernels are unchanged and stay simurg_core 1.1.0. uv.lock committed
(uv lock --check clean).

Full suite on the Pi, file by file under the memory guard: 29 files exit 0
(unit: 282 passed), ruff and mypy clean.

.dockerignore

@@ -0,0 +1,14 @@
+.git
+.venv
+data
+refs
+dist
+results
+_phase0_archive
+**/__pycache__
+**/target
+*.idat
+*.idat.gz
+*.bpm
+*.egt
+*.gtc

.github/workflows/ci.yml

@@ -23,3 +23,9 @@ jobs:
         run: uv run mypy
       - name: Unit tests (no downloaded data needed)
         run: uv run pytest tests/unit -q
+      - name: Build the compiled kernels (rust/simurg-core)
+        run: |
+          curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal
+          PATH="$HOME/.cargo/bin:$PATH" uv pip install ./rust/simurg-core
+      - name: Unit tests with the compiled kernels (must equal the pure-Python results)
+        run: uv run pytest tests/unit -q

.gitignore

@@ -1,5 +1,5 @@
 # vendor / large data — never commit
-data/
+/data/
 refs/
 _phase0_archive/
 *.idat

CHANGELOG.md

@@ -1,5 +1,48 @@
 # Changelog
 
+## 2.0.0 — 2026-09-30 (v2 modules)
+
+The Python package is 2.0.0. The optional Rust kernels are unchanged and stay `simurg_core` 1.1.0.
+
+**Methylation control QC** (`simurg methyl qc`, `simurg methyl convert-manifest`; MET-001 … MET-018): methylation
+CSV manifests, the 21 control metrics, pOOBAH detection p-values, noob, linear/nonlinear dye bias, β and M-values,
+summary tsv/xlsx/html with a β PCA, per-sample tables; methylation folders load in `simurg qc report`;
+`qc controls --array-type methylation`. Checked against sesame (`docs/validation/V2_methylation.md`, K-026).
+
+**Copy number, LOH and mosaicism** (`simurg cn build-model|call|annotate`; CYTO-001 … CYTO-042): GC-wave
+correction, sex-aware batch reference, HMM with constitutional, LOH and mosaic states, event statistics and
+filters, CNV VCF + segments + sidecar, ISCN 2020 annotation from UCSC tables; noise-normalised mosaic fractions and
+complete-loss promotion (CYTO-041), per-probe LRR noise (CYTO-042). Validated on implanted events and on 360
+GSE165845 arrays against the NYGC 30x SV callset (`docs/validation/V2_cn.md`, K-027).
+
+**Pharmacogenomics** (`simurg pgx db build|coverage`, `pgx copy-number train|call`, `pgx star-allele call`,
+`pgx phenotype`; PCN-001 … PCN-017, PGX-001 … PGX-035): targeted CN for CYP2D6 sub-regions, CYP2A6, GSTM1, GSTT1,
+UGT2B17, SULT1A1, CYP2E1 (HWE-constrained compression mixture); a deterministic PGx DB from CPIC, PharmVar and
+ClinPGx (70 genes); solver-free diplotype calling with array-equivalence classes, CYP2D6 structures (*5, xN,
+*68+*4, *36+*10, *13), G6PD, HLA tags, variant-based and *0 genes; probe masks from 1000G 30x concordance; CPIC and
+DPWG phenotypes; per-sample JSON and the TSV set. The GSA v4 PGx demo arrays are 30 CEPH/1000G individuals: region
+CN 224/240 vs NYGC SV, gene calls 154/168 vs GeT-RM (`docs/validation/V2_pgx_cn.md`, `V2_pgx_star.md`, K-028, K-029).
+
+**ArrayTrain** (`simurg train`; plan §12): trains EGT cluster files (v3, dbv9) from a cohort's GTCs with sex-aware
+sample selection, BIC-selected 1/2/3-cluster θ mixtures, HWE-guided labelling, and missing clusters imputed from
+cross-locus relations. On GSE165845 (chr22 vs 1000G 30x) it gives 99.44 % concordance at a 99.61 % call rate;
+the vendor EGT gives 99.46 % at 99.15 % (`docs/validation/V2_arraytrain.md`, K-030).
+
+## 1.1.0 — 2026-09-28 (P9, P10)
+
+**Packaging (P10).** `scripts/build_dist.sh` (sdist, pure wheel, manylinux_2_28 kernel wheels for x86_64 and
+aarch64), `Dockerfile`, conda recipes (`packaging/conda/`). Verified on fresh x86_64 and aarch64 VMs: pip, Docker
+and conda installs give identical outputs (`docs/validation/P10.md`). A closed stdout pipe now exits 141 silently
+instead of reporting an internal error.
+
+**Performance (P9).**
+
+Optional compiled kernels `rust/simurg-core` (PyO3): fused GenCall calling and BAF/LRR, libm maps with CPython
+semantics, normalization nearest-neighbour searches, BPM parser. Byte-identical outputs to the pure-Python paths
+(`SIMURG_PURE_PYTHON=1`); 78–80 GSA samples/min per Graviton3 core (pure Python 30), manifest loading 4× faster.
+Address coverage memoised per IDAT layout; worker state pickled once. `bench/profile_genotype.py`,
+`bench/scaling.py`. Details and scaling limits: `docs/validation/P9.md`.
+
 ## 1.0.0 — 2026-09-28
 
 First release: the v1.0 goals of the master plan (§1.1), built and validated phase by phase (P0–P8).

Dockerfile

@@ -0,0 +1,28 @@
+# SimurgArray container: the `simurg` CLI with the compiled kernels.
+#   docker build -t simurg:2.0.0 .
+#   docker run --rm -v "$PWD":/work simurg:2.0.0 genotype --bpm M.bpm --egt C.egt --idats idats --out run/gtc
+# Inputs and outputs live in the mounted /work folder; the image holds no data.
+ARG PYTHON=3.12
+
+FROM python:${PYTHON}-slim-bookworm AS build
+RUN apt-get update -q && apt-get install -y -q --no-install-recommends curl gcc libc6-dev \
+    && rm -rf /var/lib/apt/lists/*
+RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal
+ENV PATH=/root/.cargo/bin:$PATH
+WORKDIR /src
+COPY pyproject.toml README.md LICENSE THIRD_PARTY_NOTICES.md ./
+COPY src ./src
+COPY rust/simurg-core ./rust/simurg-core
+RUN pip wheel --no-cache-dir --no-deps -w /wheels . ./rust/simurg-core
+
+FROM python:${PYTHON}-slim-bookworm
+LABEL org.opencontainers.image.title="SimurgArray" \
+      org.opencontainers.image.description="Research toolkit for Illumina Infinium genotyping BeadChip data" \
+      org.opencontainers.image.licenses="MIT"
+COPY --from=build /wheels /wheels
+RUN pip install --no-cache-dir /wheels/*.whl && rm -rf /wheels \
+    && useradd --create-home --uid 1000 simurg
+USER simurg
+WORKDIR /work
+ENTRYPOINT ["simurg"]
+CMD ["--help"]

README.md

@@ -1,7 +1,8 @@
 # SimurgArray
 
 Personal research toolkit for Illumina Infinium genotyping BeadChip data
-(IDAT → normalization → genotype calling → BAF/LRR → QC → VCF/PLINK; later CN/LOH, PGx, cluster training).
+(IDAT → normalization → genotype calling → BAF/LRR → QC → VCF/PLINK; CN/LOH/mosaicism, PGx, cluster training,
+methylation control QC).
 
 - **SIMURGARRAY_MASTER_PLAN.md** — architecture, algorithms, roadmap
 - **SIMURGARRAY_FUNCTIONAL_SPEC.md** — 416 itemised requirements (CLI, inputs, outputs, QC, CN, PGx, methylation, edge cases)
@@ -9,9 +10,10 @@ Personal research toolkit for Illumina Infinium genotyping BeadChip data
 
 ## Status
 
-**v1.0.0** (2026-09-28) — see `CHANGELOG.md`. IDAT → normalization → GenCall-equivalent genotype calling → GTC v5 →
-sample/control QC with an HTML report → plus-strand VCF/BCF → PLINK 1/2, bedGraph, final report; plus a
-differential-testing harness. Validation (`docs/validation/`):
+**v2.0.0** (2026-09-30) — see `CHANGELOG.md`. v1: IDAT → normalization → GenCall-equivalent genotype calling →
+GTC v5 → sample/control QC with an HTML report → plus-strand VCF/BCF → PLINK 1/2, bedGraph, final report; plus a
+differential-testing harness. v2 adds cluster training (ArrayTrain), genome-wide CN/LOH/mosaic calling,
+pharmacogenomics (targeted CN, star alleles, phenotypes) and methylation control QC. Validation (`docs/validation/`):
 
 | Check | Result |
 |---|---|
@@ -19,11 +21,17 @@ differential-testing harness. Validation (`docs/validation/`):
 | GSA demo vs gtc2vcf, GTCtoVCF, plink2 | zero unexplained differences |
 | 1000 Genomes pedigrees (Omni2.5) | 717/717 parent–offspring pairs; Mendel error median 0.031 % |
 | 360 GSA arrays (GSE165845) vs NYGC 30x WGS / GIAB HG001 | concordance 0.9934 / 0.9985; replicates 0.9966 |
+| ArrayTrain EGT, GSE165845 chr22 vs 1000G 30x | 99.44 % concordance (vendor EGT 99.46 %), call rate 99.61 % |
+| CN, 360 GSE165845 arrays vs NYGC 30x SV | PASS precision DEL 44 % / DUP 57 % |
+| PGx, GSA v4 demo vs GeT-RM | 154/168 gene calls |
+| Methylation QC vs sesame | β r = 0.99998 |
 
 Commands: `simurg inspect`, `ingest`, `normalize`, `genotype`, `qc controls|samples|contamination|relatedness|loci|report|suggest-thresholds`,
-`gtc-to-vcf`, `export plink|bedgraph|finalreport|locus-summary`, `compare`. **User guide: `docs/user_guide.md`**;
-worked example: `examples/gsa_demo.sh`. Next (plan §18): P9 performance (compiled core), P10 packaging, then v2
-modules (cluster training, CN/LOH, PGx).
+`gtc-to-vcf`, `export plink|bedgraph|finalreport|locus-summary`, `compare`, `train`, `cn build-model|call|annotate`,
+`pgx db build|coverage|copy-number|star-allele|phenotype`, `methyl qc|convert-manifest`. **User guide:
+`docs/user_guide.md`**; worked example: `examples/gsa_demo.sh`. Optional Rust kernels (byte-identical outputs,
+~80 GSA samples/min per server core, `docs/validation/P9.md`); wheels, Docker image and conda recipes
+(`docs/validation/P10.md`).
 
 ## Quick start
 

SIMURGARRAY_FUNCTIONAL_SPEC.md

@@ -2435,6 +2435,23 @@ about **algorithms or numerics**, the plan wins. Each such conflict found during
 
   Each yields valid, possibly empty, VCF and JSON output (EDGE-024). (src: E-35)
 
+- **CYTO-041** `§10.3 · v2` **MF refinements [OURS].** Two refinements of CYTO-018, from real-data validation
+  (docs/validation/V2_cn.md, K-027):
+  - *noise-corrected split:* per-probe expected noise σ_i = the reference's heterozygous BAF spread of the probe ×
+    the sample's scale (median |BAF − 0.5| / spread over autosomal heterozygous calls, ÷ 0.6745; the sample's
+    heterozygous BAF SD without a reference spread). Over the event's heterozygous calls, m = median(|BAF − 0.5| /
+    σ_i); the split in σ units δ solves Φ(m − δ) − Φ(−m − δ) = ½ (δ = 0 when m ≤ 0.6745) and d = median(σ_i)·δ.
+    Without this, noise alone gave MF ≈ 0.1–0.3 and genomic waves were called as mosaic losses.
+  - *complete loss:* a mosaic loss whose called genotypes hold fewer than a quarter of the heterozygotes the PFB
+    predicts (Σ 2p(1−p) ≥ 3) has MF = 1. With the default `--max-mosaic-fraction 1.0` it is therefore promoted
+    (CYTO-020) to a constitutional DEL with `HIGHFRACTION`. This catches constitutional deletions whose LRR is
+    compressed towards the mosaic states.
+
+- **CYTO-042** `§10.2 · v2` **Per-probe LRR noise [OURS].** A model built with `--reference-gtcs` stores each
+  probe's robust LRR SD across the reference (1.4826·MAD after removing each sample's median and the probe's
+  reference median). `cn call` scales the sample's LRR emission SD per probe by that spread relative to the autosomal
+  median spread, clipped to [0.5, 3]. Without the column, or with `--no-probe-lrr-sd`, every probe uses the sample SD.
+
 ---
 
 ## 7. Targeted PGx copy number (v2)
@@ -2742,7 +2759,10 @@ about **algorithms or numerics**, the plan wins. Each such conflict found during
 
   In a region whose CN is n ≠ 2, the variant copy number `k ∈ {0..n}` is the maximum-likelihood value under the
   observed BAF ~ N(k/n, σ²), with σ estimated per batch from AB clusters. It is reported as `variant_copy_number`.
-  GS enters as a per-call error rate: ε(GS) = 0.5·(1 − GS)², clipped to [1e-4, 0.25] [OURS]. (src: B-30, E-41)
+  GS enters as a per-call error rate: ε = 1e-3 for GS ≥ 0.15 and 0.05 below [OURS, calibrated: after probe masking
+  (PGX-010), 0 of 9 740 GSA v4 PGx calls at DB variants disagreed with 1000G 30x in any GS bin, so errors are
+  probe-specific and belong in the mask; the earlier ε(GS) = 0.5·(1 − GS)² made correct low-GS calls such as
+  CYP2C9 rs1799853 at GS 0.16 lose to the prior; K-029]. (src: B-30, E-41)
 
 - **PGX-013** `§11 · v2` **Candidate enumeration [DIFF: solver-free].**
   - For each gene, SimurgArray enumerates diplotypes from the gene's haplotypes: star alleles, `*0`, duplications
@@ -3710,6 +3730,8 @@ Generated from the item tags (`build.py`). Phases: P0–P10 as in plan §18; `v2
 | CYTO-038 | Model building | §10 | v2 | A-G52 |
 | CYTO-039 | Batch scale | §10, §19 | v2 | D-9, E-35 |
 | CYTO-040 | Degenerate samples | §10 | v2 | E-35 |
+| CYTO-041 | MF refinements | §10.3 | v2 | K-027 |
+| CYTO-042 | Per-probe LRR noise | §10.2 | v2 | K-027 |
 | PCN-001 | Region catalogue | §11 | v2 | B-23, C-G25, D-10 |
 | PCN-002 | Genotype every region [EMU] | §11 | v2 | B-23 |
 | PCN-003 | Inputs | §11 | v2 | A-G37 |

THIRD_PARTY_NOTICES.md

@@ -12,7 +12,13 @@ used only as external subprocess oracles and are not copied or linked.
 | [klib `ksort.h`](https://github.com/attractivechaos/klib) as shipped in htslib — Copyright (c) 2008, 2012-2013, 2017-2019 Genome Research Ltd | MIT | Quickselect `ks_ksmall` partition order, ported to `simurg/qc/contamination.py` (`ks_ksmall`) to reproduce BAFregress medians |
 | [Illumina GTCtoVCF](https://github.com/Illumina/GTCtoVCF) — Copyright Illumina, Inc. | Apache-2.0 | Indel source-sequence handling ported to `simurg/harmonize/sites.py` (design-strand orientation, left shift, IUPAC flank matching, deletion/insertion context score); multi-probe consistency idea in `simurg/harmonize/combine.py`; test oracle (run under Python 3 with two local compatibility patches, never shipped) |
 | [Picard `picard.arrays`](https://github.com/broadinstitute/picard) | MIT | Second format reference |
+| [PyO3](https://github.com/PyO3/pyo3), [rust-numpy](https://github.com/PyO3/rust-numpy), [ndarray](https://github.com/rust-ndarray/ndarray) and their Rust dependencies | MIT / Apache-2.0 / BSD-2-Clause | Compiled into the optional `simurg_core` extension (`rust/simurg-core`); list with `cargo tree` |
+| [UCSC Genome Browser](https://genome.ucsc.edu/) `cytoBand` tables (hg38, hg19) and `ncbiRefSeq` (RefSeq transcripts, NCBI) | Free for all uses (UCSC data terms; RefSeq is public domain) | `simurg/cn/data/cytoBand_GRCh3{7,8}.txt.gz` and the derived centromere/acrocentric mask BEDs are shipped; `scripts/build_annotation_bundle.sh` downloads cytoBand + ncbiRefSeq for gene lists; `simurg/pgx/data/pgx_cn_regions.GRCh38.v1.bed` is derived from ncbiRefSeq exon coordinates (`scripts/pgx/build_cn_regions.py`) |
 | [illuminaio](https://github.com/HenrikBengtsson/illuminaio) | GPL-2 | Documentation of IDAT field codes only; oracle via subprocess |
+| [CPIC](https://cpicpgx.org) database via the CPIC API (allele definitions, allele function, diplotype→phenotype tables) | CC0 1.0 | downloaded by `scripts/pgx/fetch_sources.sh` into `data/` (never committed); `simurg pgx db build` derives the local PGx DB from it; source version and file hashes are recorded in the DB `meta` table |
+| [PharmVar](https://www.pharmvar.org) haplotype definitions (Gaedigk et al. 2018 CPT 103:399) | PharmVar terms of use (attribution) | per-gene zips downloaded by `scripts/pgx/fetch_sources.sh` into `data/` (never committed); `simurg pgx db build` derives core-allele definitions for genes CPIC does not define |
+| [GeT-RM](https://www.cdc.gov/lab-quality/php/get-rm/reference-materials.html) consolidated PGx/HLA table (CDC) | US government work | validation truth only (`scripts/pgx/getrm_truth.py`), not redistributed |
+| [ClinPGx](https://www.clinpgx.org) (formerly PharmGKB) clinical-annotation variants | CC BY-SA 4.0 (attribution: ClinPGx, https://www.clinpgx.org/page/citingClinpgx) | `clinicalVariants.zip` downloaded by `scripts/pgx/fetch_sources.sh` into `data/` (never committed); rsIDs become genotype-only variant-based genes in the local PGx DB |
 
 ## BSD-2-Clause (BeadArrayFiles, verbatim from its README)
 

bench/profile_genotype.py

@@ -0,0 +1,83 @@
+#!/usr/bin/env python3
+"""Per-stage timing of `simurg genotype` for one sample (plan §19 measurements).
+
+python bench/profile_genotype.py BPM EGT GRN_IDAT [--repeat N]
+"""
+
+from __future__ import annotations
+
+import argparse
+import datetime
+import time
+from dataclasses import replace
+from pathlib import Path
+
+import numpy as np
+
+from simurg.formats.bpm import read_bpm
+from simurg.formats.egt import read_egt
+from simurg.formats.gtc import write_gtc
+from simurg.formats.idat import read_idat
+from simurg.genotype.baf_lrr import baf_lrr, logr_dev
+from simurg.genotype.clusters import align_clusters
+from simurg.genotype.gencall import call_genotypes
+from simurg.genotype.sample import genotype_sample, manifest_context, params_for_preset
+from simurg.genotype.sex import estimate_sex
+from simurg.signal.normalization.apply import normalize, polar
+from simurg.signal.normalization.fit import fit_transforms
+from simurg.signal.raw_xy import LocusMapCache, assemble_raw_xy
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("bpm")
+    ap.add_argument("egt")
+    ap.add_argument("grn")
+    ap.add_argument("--repeat", type=int, default=3)
+    ap.add_argument("--out", default="/tmp/profile.gtc")
+    a = ap.parse_args()
+    m = read_bpm(a.bpm)
+    ctx = manifest_context(m, align_clusters(read_egt(a.egt), m))
+    lite = m.lite()
+    ctx = replace(ctx, manifest=lite, cache=LocusMapCache(lite))
+    params = params_for_preset("current")
+    red_path = a.grn.replace("_Grn", "_Red")
+    t: dict[str, list[float]] = {}
+
+    def lap(name: str, t0: float) -> float:
+        now = time.perf_counter()
+        t.setdefault(name, []).append(now - t0)
+        return now
+
+    for _ in range(a.repeat):
+        t0 = time.perf_counter()
+        grn, red = read_idat(a.grn), read_idat(red_path)
+        t0 = lap("read_idat x2", t0)
+        raw = assemble_raw_xy(grn, red, lite, cache=ctx.cache)
+        t0 = lap("assemble_raw_xy", t0)
+        xf = fit_transforms(raw.raw_x, raw.raw_y, ctx.norm_bins, params.norm_version)
+        t0 = lap("fit_transforms", t0)
+        nx, ny = normalize(raw.raw_x, raw.raw_y, ctx.lookup, xf)
+        th, r = polar(nx, ny)
+        t0 = lap("normalize+polar", t0)
+        z = (raw.raw_x == 0) & (raw.raw_y == 0)
+        thc = np.where(z, np.float32(np.nan), th).astype(np.float32)
+        rc = np.where(z, np.float32(np.nan), r).astype(np.float32)
+        calls = call_genotypes(thc, rc, z, ctx.clusters, lite.intensity_only, params.cutoff)
+        t0 = lap("call_genotypes", t0)
+        estimate_sex(calls.genotypes, r, ctx.clusters.gentrain, ctx.clusters.present, ctx.sex_classes, params.sex)
+        t0 = lap("estimate_sex", t0)
+        _, lrr = baf_lrr(thc, rc, z, ctx.clusters, lite.intensity_only)
+        logr_dev(lrr, ctx.autosomal, lite.intensity_only, ctx.clusters.present)
+        t0 = lap("baf_lrr+logr_dev", t0)
+        g = genotype_sample(grn, red, ctx, params, datetime.datetime(2026, 1, 1))
+        t0 = lap("genotype_sample (whole)", t0)
+        write_gtc(g, Path(a.out))
+        t0 = lap("write_gtc", t0)
+    print(f"{'stage':28s} {'median s':>9s}")
+    for k, v in t.items():
+        print(f"{k:28s} {sorted(v)[len(v) // 2]:9.3f}")
+
+
+if __name__ == "__main__":
+    main()

bench/scaling.py

@@ -0,0 +1,114 @@
+#!/usr/bin/env python3
+"""Thread scaling of ``simurg genotype`` (plan §18 P9 gate: ≥ 20 samples/min/core on GSA-size arrays, linear
+scaling to 32 threads).
+
+For every worker count the same N samples are genotyped (``--no-cohort-qc``); the per-sample phase is timed from
+the run log ("genotyping … sample(s)" → "done"), so manifest loading is reported separately and does not blur
+the scaling. Writes a JSON summary.
+
+    python bench/scaling.py --bpm M.bpm --egt C.egt --idats DIR --out OUT --threads 1,2,4,8 [--samples N]
+"""
+
+from __future__ import annotations
+
+import argparse
+import datetime as dt
+import json
+import os
+import shutil
+import subprocess
+import sys
+import time
+from pathlib import Path
+
+
+def phase_seconds(log: Path) -> tuple[float, float]:
+    """(manifest+setup seconds, per-sample phase seconds) from a genotype run log."""
+    ts: dict[str, dt.datetime] = {}
+    for line in log.read_text().splitlines():
+        rec = json.loads(line)
+        msg = rec.get("msg") or ""
+        t = dt.datetime.fromisoformat(rec["ts"].replace("Z", "+00:00"))
+        ts.setdefault("first", t)
+        if msg.startswith("genotyping "):
+            ts["start"] = t
+        elif msg.startswith("done:"):
+            ts["done"] = t
+    return (ts["start"] - ts["first"]).total_seconds(), (ts["done"] - ts["start"]).total_seconds()
+
+
+def subset(idats: Path, n: int, dest: Path) -> Path:
+    grn = sorted(p for p in idats.rglob("*_Grn.idat*"))
+    if n <= 0 or n >= len(grn):
+        return idats
+    dest.mkdir(parents=True, exist_ok=True)
+    for g in grn[:n]:
+        d = dest / g.parent.name
+        d.mkdir(exist_ok=True)
+        for p in (g, g.with_name(g.name.replace("_Grn.", "_Red."))):
+            link = d / p.name
+            if not link.exists():
+                link.symlink_to(p.resolve())
+    return dest
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    ap.add_argument("--bpm", required=True)
+    ap.add_argument("--egt", required=True)
+    ap.add_argument("--idats", required=True, type=Path)
+    ap.add_argument("--out", required=True, type=Path)
+    ap.add_argument("--threads", default="1,2,4")
+    ap.add_argument("--samples", type=int, default=0, help="use the first N samples (0 = all)")
+    ap.add_argument("--simurg", default=str(Path(sys.executable).with_name("simurg")))
+    ap.add_argument("--pure-python", action="store_true", help="run with SIMURG_PURE_PYTHON=1")
+    ap.add_argument("--work", type=Path, help="folder for the GTC outputs (e.g. a tmpfs); default: --out")
+    ap.add_argument("--clean", action="store_true", help="delete each run's GTCs after timing it")
+    a = ap.parse_args()
+    a.out.mkdir(parents=True, exist_ok=True)
+    idats = subset(a.idats, a.samples, a.out / "idats")
+    n = len(list(idats.rglob("*_Grn.idat*")))
+    env = {**os.environ, "SIMURG_PURE_PYTHON": "1" if a.pure_python else "0"}
+    runs = []
+    for t in (int(x) for x in a.threads.split(",")):
+        out = (a.work or a.out) / f"t{t}"
+        t0 = time.time()
+        cmd = [a.simurg, "genotype", "--bpm", a.bpm, "--egt", a.egt, "--idats", str(idats), "--out", str(out),
+               "--threads", str(t), "--no-cohort-qc", "--autocall-date", "1/1/2026 12:00:00 AM",
+               "--quiet", "--force"]  # fmt: skip
+        rc = subprocess.run(cmd, env=env).returncode
+        wall = time.time() - t0
+        setup, phase = phase_seconds(out / "logs" / "genotype.log")
+        if a.clean:
+            shutil.rmtree(out / "samples", ignore_errors=True)
+        runs.append(
+            {
+                "threads": t,
+                "rc": rc,
+                "wall_s": round(wall, 1),
+                "setup_s": round(setup, 1),
+                "per_sample_phase_s": round(phase, 2),
+                "samples_per_min": round(60 * n / phase, 1) if phase else None,
+                "samples_per_min_per_core": round(60 * n / phase / t, 1) if phase else None,
+            }
+        )
+        print(json.dumps(runs[-1]), flush=True)
+    base = runs[0]["per_sample_phase_s"] * runs[0]["threads"]
+    for r in runs:
+        r["speedup_vs_first"] = round(base / r["per_sample_phase_s"], 2) if r["per_sample_phase_s"] else None
+        r["efficiency"] = round(r["speedup_vs_first"] / r["threads"], 3) if r["speedup_vs_first"] else None
+    from simurg._core import backend
+
+    summary = {
+        "samples": n,
+        "backend": "python" if a.pure_python else backend(),
+        "cpu_count": os.cpu_count(),
+        "machine": os.uname().machine,
+        "runs": runs,
+    }
+    (a.out / "scaling.json").write_text(json.dumps(summary, indent=1))
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

docs/known_unknowns.md

@@ -4,6 +4,12 @@ Plan §20 lists the open questions; this file records what the implementation ha
 
 | ID | Date | Topic | Finding | Status |
 |---|---|---|---|---|
+| K-030 | 2026-09-29 | ArrayTrain (v2) | The clustering is ours, not the vendor's (plan §12): unbounded 1/2/3-component θ mixtures chosen by BIC, with components labelled by HWE goodness of fit plus weak position and gap priors. Fixed θ windows (v1) mislabelled compressed and asymmetric loci. On GSE165845 (train on replicate 3, evaluate on replicates 1–2, chr22 vs 1000G 30x) the result is 99.44 % concordance, 1.30 % NRD and 99.61 % call rate, against 99.46 % / 1.23 % / 99.15 % with the vendor EGT. The parameters were tuned on this evaluation (a flat optimum). GenTrain and cluster-separation scores are ours, so GenCall scores are not comparable with vendor-file scores. | Open: second cohort/chip; score calibration (`docs/validation/V2_arraytrain.md`). |
+| K-029 | 2026-09-29 | PGx star alleles (v2) | (1) PGX-012 error model: ε(GS) = 0.5·(1 − GS)² was replaced by ε = 1e-3 (GS ≥ 0.15) / 0.05 — after probe masking 0 of 9 740 GSA v4 PGx calls disagreed with 1000G 30x at any GS, and the old model made correct low-GS calls lose to the prior (spec amended). (2) PGX-010 masks derived from 14 individuals (spec: ≥ 100); a variant absent from the filtered 1000G panel counts as hom-ref only if rare (CYP2B6 rs2279343 is filtered out, not absent). (3) CPIC core-allele definitions use IUPAC wildcards; alleles without definitions (NUDT15 *2, structural CYP2D6) are not callable from SNVs. (4) GeT-RM uses legacy UGT1A1/SLCO1B1 names. Result vs GeT-RM (16 CEPH): 100/109 gene calls, CYP2D6/2C19/2C9 41/45. (5) DPWG: no public machine-readable genotype→phenotype mapping; CPIC mappings are used for genes CPIC covers (as ClinPGx does), DPWG-only genes (CYP1A2, CYP3A4, F5) get no phenotype. (6) GeT-RM definitions are older than CPIC/PharmVar for UGT1A1, SLCO1B1, CYP1A2 and CYP2C8 (e.g. PharmVar CYP2C8 *19 = rs2071426 alone). Result vs GeT-RM (16 CEPH, 21 genes): 154/168 comparable gene calls. Independent check on GSE165845 (GSA v1 C2, 120 individuals × 3, WGS-derived truth, 2-fold masks): 95.7 % array-resolvable concordance, 67.9 % top-solution (the chip lacks many defining variants), replicate concordance 99.2 % without the failed array. | Open: KNMP DPWG tables if they become available (`docs/validation/V2_pgx_star.md`). |
+| K-028 | 2026-09-29 | PGx CN (v2) | Deviations from PCN-009: the per-probe reference is the training cohort's median stored in the model (a calling-batch median is not CN 2 where deletions are common, e.g. GSTM1 null ≈ 50 %); no GC-correction step; the CN mixture is ours: µ_c = δ + γ·log2((c+1)/3) per region with a free lower CN-0 component and Hardy–Weinberg-constrained weights (probe compression differs by region: γ ≈ 0.19 CYP2D6.p5 … 0.56 GSTT1). Interim validation on the GSA v4 PGx demo — whose 36 arrays turned out to be 30 CEPH/1000G individuals — against NYGC 30x SV RD_CN: 224/240 region calls (GSTM1, UGT2B17 30/30), but in-sample (trained on the same arrays, < 96 samples, priors chosen with this truth in view). | Open: validate on an independent GSA-PGx cohort; CYP2D6 via GeT-RM star alleles (`docs/validation/V2_pgx_cn.md`). |
+| K-027 | 2026-09-29 | CN/LOH (v2) | CYTO-014 noise gate: the summary `logr_dev` also measures cluster-file mismatch (GSE165845: 0.246, all samples `SampleQuality`) while the LRR used after the batch reference has SD ≈ 0.1; the gate now uses the SD of the normalised LRR (the JSON still reports the summary value). Spec: built-in mask/cytobands from UCSC; the annotation bundle uses UCSC ncbiRefSeq instead of RefSeq GFF + Ensembl GTF + HGNC. Real-data result: constitutional DEL/DUP calls 74–77 % in NYGC truth, mosaic calls mostly false positives. Pi tuning: mosaic prior / switch distance barely matter; CYTO-041 (noise-normalised BAF split, complete-loss promotion) cuts PASS mosaic calls by ~30 % without losing implanted ≥ 15 % mosaics. Two limits remain: (1) without phasing, < 20 % mosaic losses are not separable from regional BAF noise; (2) common CNVs (carrier frequency > 20 %) are almost never detected (1/236 DEL) because the median-LRR batch reference absorbs them; rare DEL (< 5 %) 38 %. Final cloud run (360 arrays, CYTO-041/042 defaults): all samples `OK`; PASS precision DEL 44 %, DUP 57 % (61 % for CNV calls without one bad array); rare DEL sensitivity 30 %. | Open: rare-event sensitivity as primary metric; phasing-based mosaic BAF (`docs/validation/V2_cn.md`). |
+| K-026 | 2026-09-29 | Methylation QC (v2) | Spec deviations: vendor EPIC manifests put a 5th field on some control lines (≥ 4 fields accepted); detection p follows sesame's pOOBAH (1 − max(F(M), F(U))) rather than the ambiguous "1 − ECDF(M + U)"; minfi-style `Sentrix_ID`/`Sentrix_Position` sheet keys accepted. Against sesame 1.28.1 on minfiDataEPIC: β r = 0.99998 (max \|Δβ\| 0.011), detection calls 99.1 % identical, noob signals within 0.5 % (median) / 5 % (max). The systematic remainder is unexplained; it is not the negative controls (sesame did not attach the supplied control table). | Open; `docs/validation/V2_methylation.md`. Next step: dump sesame's background vectors and noob parameters to find the difference. |
+| K-025 | 2026-09-28 | Scaling limits (P9) | With the Rust kernels a Graviton3 core genotypes 78–80 GSA samples/min. At > 8 workers a 125 MB/s gp3 volume caps the run at ~620 samples/min (GTC ≈ 13 MB each); on a RAM disk efficiency is 0.77 at 16 and 0.48 at 32 workers for 360 samples, explained by a fixed start-up cost (≈ 2.5 s + 0.2 s per worker). Linear scaling for large batches is inferred from that model, not measured. | `docs/validation/P9.md`; measure a ≥ 5000-sample batch when one is at hand. |
 | K-024 | 2026-09-28 | GSE165845 (P8) | GSA-MD v1 chips have no public cluster file; the GSA v1 C1 EGT with the C2 build-38 BPM covers 99.996 % of bead addresses and gives median LogR deviation 0.246 (cluster mismatch, not noise). GSM5051309 (NA20362 rep 2) is a failed array (call rate 0.762, concordance 0.336 with its own replicates). 27 510 array records have no NYGC 30x record and are scored as hom-ref; 1444 of them are systematically called alt. | Recorded in `docs/validation/P8.md`; results reported with and without these. |
 | K-023 | 2026-09-28 | P6 optional outputs | Implemented and validated the same day: `--info-fields` (with `--egt`), `--extra-format`, `--merge-snv-indel`, `--auxiliary-loci`, PLINK 2 `.pgen`. Two decisions: OUT-035 rule (d) uses the MNV's last plus-strand base for minus-strand designs (spec amended); `--extra-format` writes NORMX/NORMY missing where the raw intensities are both 0, and THETA/R missing where both normalized intensities are 0 (148 values on 3 GSA samples; gtc2vcf writes 0). | resolved |
 | K-022 | 2026-09-28 | VCF oracles' limits (GSA v4 PGx, GRCh38) | gtc2vcf leaves 144 indel probes unresolved (`REF=I`/`D`) that SimurgArray resolves from `SourceSeq`; both leave 49 indels unresolved. GTCtoVCF (python-2.7 era) runs under Python 3 only with two local patches (pyvcf3 `Format` field; skip instead of crash on 3 CYP2D6 indels whose `SourceSeq` has the dash on the right, `[ALLELE/-]`, which SimurgArray handles), and needs the alt-contig loci filtered when the FASTA has no alts. Neither oracle applies ploidy (males' X/Y/MT hets stay diploid). | documented |

docs/messages.md

@@ -8,6 +8,10 @@ Generated from `src/simurg/core/errors.py` — do not edit by hand.
 | `E_BPM_CSV_MISMATCH` | error | BPM and CSV manifests describe different loci | IN-042 |
 | `E_BPM_FORMAT` | error | BPM manifest is malformed or unsupported | IN-040 |
 | `E_BUILD_MISMATCH` | error | Manifest genome build differs from the FASTA build | IN-063 |
+| `E_CN_FAILED` | error | CN calling failed for the sample | CYTO-040 |
+| `E_CN_SEX_UNRESOLVED` | error | Sample sex unresolved and --unknown-sex-policy error | CYTO-024 |
+| `E_CN_SIDECAR_MISSING` | error | cn_call_manifest.json not found in the cn call output | CYTO-032 |
+| `E_CN_VCF_UNINDEXED` | error | CNV VCFs must be bgzipped and indexed for annotate | CYTO-032 |
 | `E_CONFIG_KEY` | error | Unknown key in --config YAML | CLI-019 |
 | `E_CONFIG_PARSE` | error | --config file is not valid YAML | CLI-019 |
 | `E_CSV_MANIFEST_FORMAT` | error | CSV manifest is malformed | IN-041 |
@@ -26,9 +30,27 @@ Generated from `src/simurg/core/errors.py` — do not edit by hand.
 | `E_IDAT_MANIFEST_MISMATCH` | error | IDATs and manifest belong to different products | IN-027 |
 | `E_IDAT_PAIR_MISMATCH` | error | Red and Grn IDATs of a sample do not match | IN-023 |
 | `E_INTERNAL` | error | Internal error (bug); rerun with --debug and report | CLI-014 |
+| `E_METH_CONTROLS_FORMAT` | error | Controls not formatted correctly. Must have 4 columns (Address,Type,Color_Channel,Name) | MET-002 |
+| `E_METH_MISSING_COLUMNS` | error | Missing required columns in the methylation manifest | MET-002 |
+| `E_METH_NO_ASSAY` | error | Probe section not found | MET-002 |
+| `E_METH_NO_CONTROLS` | error | Missing control probes in manifest | MET-002 |
+| `E_METH_NO_IDATS` | error | No IDATs found | MET-004 |
+| `E_METH_PREP_FAILED` | error | Prep failed (sample) | MET-010 |
+| `E_MODEL_CLUSTER_MISMATCH` | error | Input GTCs were called with another cluster file than the model | IN-050 |
+| `E_MODEL_MANIFEST_MISMATCH` | error | Model was built for a different manifest probe set | IN-051 |
+| `E_MODEL_VERSION` | error | Model schema is too old; rebuild it | IN-051 |
 | `E_NO_INPUT` | error | Neither an input folder nor a sample sheet was given | CLI-006 |
 | `E_NO_INPUT_FOLDER` | error | Sample sheet has no RootFolder and no folder was given | CLI-006 |
 | `E_OUTPUT_EXISTS` | error | Outputs already exist; use --force or --resume | CLI-010 |
+| `E_PGX_BATCH_TOO_SMALL` | error | Too few (passing) samples in the batch for PGx CN calling | PCN-004 |
+| `E_PGX_CN_FAILED` | error | PGx CN calling failed for the sample | PCN-002 |
+| `E_PGX_CN_MISSING` | error | A sample has no PGx CN VCF (give --no-cn to run without) | PGX-011 |
+| `E_PGX_GUIDELINE` | error | Guideline tables not present in the PGx DB | PGX-025 |
+| `E_PGX_PLATFORM_UNKNOWN` | error | PGx CN platform (LCG/EX) unknown: give --csv or --platform | CLI-032 |
+| `E_PGX_SOURCE` | error | PGx source tables missing; run scripts/pgx/fetch_sources.sh | PGX-001 |
+| `E_PGX_TRAIN_TOO_SMALL` | error | Too few (passing) samples to train a PGx CN model | PCN-005 |
+| `E_PGX_VCF_UNINDEXED` | error | SNV VCFs for PGx must be bgzipped and indexed | PGX-011 |
+| `E_PGX_VCF_UNSUITABLE` | error | SNV VCF unsuitable for PGx (squashing/locus filters/foreign) | PGX-011 |
 | `E_QC_CONFIG_EXT` | error | QC config file must be .yaml or .yml | QC-021 |
 | `E_QC_CONFIG_MISSING` | error | QC config file does not exist | QC-021 |
 | `E_QC_CONFIG_PARSE` | error | QC config file is not valid YAML | QC-021 |
@@ -37,7 +59,6 @@ Generated from `src/simurg/core/errors.py` — do not edit by hand.
 | `E_QC_FORMAT` | error | Invalid --output-format value | QC-063 |
 | `E_QC_LABEL_COUNT` | error | Number of labels does not match number of datasets | QC-042 |
 | `E_QC_LABEL_DUPLICATE` | error | Dataset labels are not unique | QC-042 |
-| `E_QC_METHYLATION` | error | Methylation control metrics are not available yet (v2) | QC-006 |
 | `E_QC_MIXED_TYPES` | error | Genotyping and methylation datasets cannot be mixed | QC-043 |
 | `E_QC_NO_DATASETS` | error | No QC dataset folder found | QC-041 |
 | `E_QC_PROFILE` | error | Unknown QC profile | QC-024 |
@@ -54,7 +75,12 @@ Generated from `src/simurg/core/errors.py` — do not edit by hand.
 | `E_SHEET_NO_KEY_COLUMNS` | error | Sample sheet lacks SentrixBarcode_A/SentrixPosition_A | IN-002 |
 | `E_SHEET_RESERVED` | error | Sample sheet column collides with an output column | IN-005 |
 | `E_SHEET_TOO_LONG` | error | Sample sheet name or value longer than 500 characters | IN-006 |
+| `E_TRAIN_TOO_SMALL` | error | Too few GTCs to train a cluster file | plan §12 |
+| `I_CN_SEX_RESOLVED` | info | Sample sex updated from U by the caller | CYTO-024 |
 | `I_UNLISTED_IDATS` | info | IDAT pairs not listed in the sample sheet are ignored | IN-016 |
+| `W_CN_NO_FASTA` | warning | No FASTA: VCF REF padding bases written as N | CYTO-029 |
+| `W_CN_PROBE_FLOOR` | warning | Minimum probe count below 10 raised to 10 | CYTO-008 |
+| `W_CN_THRESHOLD_EXCEEDS_GENOME` | warning | Size threshold exceeds the effective genome; no events | CYTO-009 |
 | `W_CTRL_DEFS_DIFFER` | warning | BPM and CSV control definitions differ; the BPM wins | QC-001 |
 | `W_CTRL_MISSING` | warning | Controls needed by a metric are absent; metric not evaluable | QC-007 |
 | `W_CTRL_ZERO_DIVISOR` | warning | Control metric has a zero denominator (NaN) | QC-008 |
@@ -73,7 +99,17 @@ Generated from `src/simurg/core/errors.py` — do not edit by hand.
 | `W_MERGE_ALL_NC` | warning | Every probe at a merged site is a no-call | OUT-031 |
 | `W_MERGE_AMBIGUOUS` | warning | Probes at a site allow several genotypes; set missing | OUT-031 |
 | `W_MERGE_INCONSISTENT` | warning | Probes at a site disagree; genotype set missing | OUT-031 |
+| `W_METH_MISSING_CONTROL_NAMES` | warning | Control names missing from the manifest; metric set to NA | MET-009 |
+| `W_METH_MISSING_CONTROL_TYPES` | warning | Control types missing from the manifest; metric set to NA | MET-009 |
+| `W_METH_MISSING_SAMPLE` | warning | Missing sample: IDATs absent; the sample fails, the run continues | MET-015 |
+| `W_METH_NO_FEATURES` | warning | No features, skipping PCA plot | MET-013 |
+| `W_MODEL_CLUSTER_MISMATCH` | warning | Input GTCs were called with another cluster file than the model | IN-050 |
 | `W_NO_BGZIP` | warning | Uncompressed VCF written; downstream commands need bgzip + index | OUT-010 |
+| `W_PGX_BATCH_BELOW_RECOMMENDED` | warning | Fewer passing samples than recommended | PCN-004 |
+| `W_PGX_BATCH_COMPOSITION` | warning | Batch spans several lots/batches or > 14 days of scans | PCN-006 |
+| `W_PGX_NO_ALT_CONTIG` | warning | FASTA lacks a contig carrying a PGx region; REF written as N | IN-062 |
+| `W_PGX_NO_PROBE_MAP` | warning | The PGx DB has no probe map for this manifest | PGX-035 |
+| `W_PGX_REGION_FEW_PROBES` | warning | PGx CN region has fewer than 10 unmasked probes | PCN-008 |
 | `W_QC_LABEL_SANITIZED` | warning | Dataset label was sanitized | QC-042 |
 | `W_QC_NO_BAF` | warning | GTC has no B-allele frequencies; contamination not estimated | QC-064 |
 | `W_QC_PEDIGREE` | warning | Parent1/Parent2/Replicate value does not name a sample in the set | IN-018 |

docs/user_guide.md

@@ -1,4 +1,4 @@
-# SimurgArray user guide (v1.0)
+# SimurgArray user guide (v2.0)
 
 SimurgArray turns Illumina Infinium genotyping BeadChip scans (IDAT files) into genotype calls, QC tables and
 reports, VCF/BCF and PLINK files. It is a personal, non-commercial **research** tool — not for clinical or
@@ -6,6 +6,8 @@ diagnostic use. Behaviour is specified item by item in `SIMURGARRAY_FUNCTIONAL_S
 cited below); the algorithms and their sources are in `SIMURGARRAY_MASTER_PLAN.md`.
 
 Contents: [Install](#install) · [Inputs](#inputs) · [A complete run](#a-complete-run) · [Commands](#commands) ·
+[Methylation](#methylation-arrays) · [CN/LOH](#copy-number-loh-and-mosaicism) · [PGx](#pharmacogenomics) ·
+[Cluster training](#cluster-training-arraytrain) ·
 [Presets and reproducibility](#presets-and-reproducibility) · [QC](#quality-control) · [Outputs](#outputs) ·
 [Accuracy](#accuracy) · [Limitations](#limitations) · [Troubleshooting](#troubleshooting)
 
@@ -19,6 +21,40 @@ uv venv && uv pip install -e .          # add ".[dev]" for tests and linters
 .venv/bin/simurg --version
 ```
 
+**Optional compiled kernels** (≈ 3–4× faster genotyping, 4× faster manifest loading; results are byte-identical):
+
+```bash
+curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal   # Rust (needs a C linker, e.g. gcc)
+uv pip install ./rust/simurg-core
+.venv/bin/python -c "from simurg._core import backend; print(backend())"                   # rust 1.1.0
+```
+
+Without them the same code runs in pure Python/NumPy; `SIMURG_PURE_PYTHON=1` forces that path.
+
+**From release files** (`bash scripts/build_dist.sh` writes them to `dist/`: a pure wheel, an sdist and
+`manylinux_2_28` kernel wheels for x86_64 and aarch64, glibc ≥ 2.28):
+
+```bash
+python3 -m venv simurg-env && simurg-env/bin/pip install simurg-2.0.0-py3-none-any.whl \
+    simurg_core-1.1.0-cp312-abi3-manylinux_2_28_$(uname -m).whl        # the second wheel is optional
+```
+
+**Docker** (image ≈ 550 MB, kernels included, runs as a non-root user; data stay in the mounted folder):
+
+```bash
+docker build -t simurg:2.0.0 .
+docker run --rm --user "$(id -u):$(id -g)" -v "$PWD":/work simurg:2.0.0 \
+    genotype --bpm M.bpm --egt C.egt --idats idats --out run/gtc
+```
+
+**conda** (recipes in `packaging/conda/`):
+
+```bash
+conda build -c conda-forge --output-folder conda-bld packaging/conda/simurg-core packaging/conda/simurg
+conda create -n simurg -c ./conda-bld -c conda-forge simurg simurg-core
+```
+
+All four installs give identical results on x86_64 and aarch64 (`docs/validation/P10.md`).
 Only pure-Python dependencies are needed (numpy, pyarrow, typer, rich, pyyaml, openpyxl). VCF/BCF, tabix/CSI
 indexes and PLINK files are written by SimurgArray itself; bcftools/plink2 are not required (they are used only
 as test oracles).
@@ -74,11 +110,122 @@ simurg export finalreport --bpm M.bpm --csv M.csv --egt C.egt --gtcs run/gtc --o
 | `simurg qc suggest-thresholds` | data-driven threshold YAML from a cohort's `controls_qc.tsv` |
 | `simurg gtc-to-vcf` | harmonised VCF/BCF. Options: `--merge`, `--unsquash-duplicates`, `--merge-snv-indel`, `--auxiliary-loci`, `--info-fields`, `--extra-format`, `--filter-gs`, `--drop-ambiguous`, `--ploidy-policy sex-aware\|diploid`, `--include-loci/--exclude-loci` |
 | `simurg export plink\|bedgraph\|finalreport\|locus-summary` | downstream formats |
+| `simurg methyl qc` / `methyl convert-manifest` | methylation-array control QC, detection p-values, β/M values (see below) |
+| `simurg cn build-model\|call\|annotate` | genome-wide CNV / LOH / mosaic calling from GTCs (see below) |
+| `simurg pgx db build\|coverage`, `pgx copy-number train\|call`, `pgx star-allele call`, `pgx phenotype` | pharmacogenomics (see below) |
+| `simurg train` | ArrayTrain: an EGT cluster file from a cohort's GTCs (see below) |
 | `simurg compare A B` | field-level differential test of GTCs or VCFs (files or folders paired by name): EXACT / NEAR / DISCREPANCY / UNKNOWN with an HTML dashboard; `--fail-on-discrepancy` exits 3 |
 
 Global logging options on every command: `--quiet`, `--log-level`, `--log-json` (JSON Lines on stderr), `--debug`.
 Run `simurg COMMAND --help` for the full list.
 
+## Methylation arrays
+
+`simurg methyl qc` runs control-probe QC and a signal summary for Infinium methylation arrays (EPIC v1/v2, MSA,
+custom designs) from IDATs and the methylation CSV manifest (spec MET-001 … MET-018). It is a QC tool: there is no
+differential-methylation analysis.
+
+```bash
+simurg methyl qc --manifest infinium-methylationepic-v-1-0-b5-manifest-file.csv --idats idats/ \
+    --sheet SampleSheet.csv --out run/methyl --profile auto
+```
+
+* 21 control metrics (staining, extension, hybridization, target removal I/II, bisulfite conversion I/II with their
+  background ratios, specificity I/II, non-polymorphic, restoration); thresholds from the `methylation_epic` /
+  `methylation_msa` profiles (`auto` picks by manifest name), `--config` or `--threshold KEY=VALUE`.
+* Per sample: detection p-values (pOOBAH: out-of-band Infinium I signal and negative controls), noob background
+  correction, dye-bias correction (`--dye-bias linear|nonlinear`), β = M/(M+U+α) (`--beta-offset`, default 0) and
+  M-values.
+* `pass_qc` = every enabled, evaluable metric passes; `--min-prop-passing X` also requires that fraction of probes
+  to pass detection (`--pval-threshold`, default 0.05).
+* Outputs: `methyl_qc_summary.tsv/.xlsx/.html` (detection histogram and a β PCA coloured by `Sample_Group`),
+  `samples/<id>/<id>_cgs.tsv.gz` (probe_id, beta, m_value, detection_p), `<id>_ctrl.tsv.gz`, `pcs.tsv.gz`,
+  `probes.tsv`/`controls.tsv` (the manifest in the open-source toolkits' layout; also written by
+  `simurg methyl convert-manifest`), and `controls_raw.tsv`/`controls_qc.tsv`, so `simurg qc report` accepts the folder.
+* Sample sheets may use `Sentrix_ID`/`Sentrix_Position` (minfi/sesame convention) as key columns.
+* `simurg qc controls --array-type methylation --csv MANIFEST.csv` computes only the control tables.
+
+## Copy number, LOH and mosaicism
+
+Genome-wide CNV / LOH / mosaic calling from GTCs (spec CYTO-001 … CYTO-042; validation in
+`docs/validation/V2_cn.md`).
+
+```bash
+# once per manifest + batch: GC covariates, population BAF, and a batch reference from ≥ 24 normal GTCs
+simurg cn build-model --bpm GSA.bpm --fasta GRCh38.fa --egt GSA.egt --reference-gtcs normals/ --out gsa.simurg-cyto
+simurg cn call --bpm GSA.bpm --model gsa.simurg-cyto --gtcs run/ --fasta GRCh38.fa --out run/cn
+simurg cn annotate --vcfs run/cn --annotation-bundle simurg-cyto-annot-GRCh38-….zip   # ISCN, cytobands, genes
+```
+
+* Per sample `<id>.cyto.cnv.vcf.gz` (+ `.tbi`) with DEL/DUP/LOH/mosaic calls, `<id>.cn_segments.parquet`, and
+  `cn_sample_summary.tsv` / `cn_call_manifest.json` for the batch. `scripts/build_annotation_bundle.sh` builds the
+  annotation bundle from UCSC tables.
+* Mosaic fractions are estimated from the noise-normalised BAF split (CYTO-041); losses without heterozygotes are
+  promoted to constitutional deletions. Per-probe LRR noise from the reference is on by default
+  (`--no-probe-lrr-sd` turns it off).
+* Array CN calling finds rare events; CNVs common in the reference batch are normalised away (K-027).
+
+## Pharmacogenomics
+
+Targeted copy number, star alleles and phenotypes (spec PCN-001 … PCN-017, PGX-001 … PGX-035; validation in
+`docs/validation/V2_pgx_cn.md` and `V2_pgx_star.md`). GRCh38 only.
+
+```bash
+bash scripts/pgx/fetch_sources.sh                      # CPIC (CC0), PharmVar, ClinPGx → data/pgx/sources/
+simurg pgx db build --sources data/pgx/sources --fasta GRCh38.fa --manifests GSA-PGx.bpm --csv GSA-PGx.csv \
+    [--mask GSA-PGx.mask.tsv] --out pgxdb/              # → pgx_db-1-<sha12>.sqlite
+simurg pgx coverage --bpm GSA-PGx.bpm --db pgxdb/pgx_db-….sqlite     # which alleles this array can call
+simurg pgx copy-number train --gtcs cohort/ --bpm GSA-PGx.bpm --csv GSA-PGx.csv --fasta GRCh38.fa --out cnmodel/
+simurg pgx copy-number call --gtcs run/ --model cnmodel/GSA-PGx.simurg-pgxcn --fasta GRCh38.fa --out run/vcf
+simurg gtc-to-vcf --gtcs run/ --bpm GSA-PGx.bpm --csv GSA-PGx.csv --fasta GRCh38.fa --out run/vcf
+simurg pgx star-allele call --vcfs run/vcf --db pgxdb/pgx_db-….sqlite --out run/pgx
+simurg pgx phenotype --calls run/pgx --db pgxdb/pgx_db-….sqlite [--guideline cpic|dpwg]
+```
+
+* **Targeted CN** for CYP2D6 (5′ region, intron 2, exon 9), CYP2A6, GSTM1, GSTT1 (alt contig), UGT2B17, SULT1A1 and
+  CYP2E1: one record per region in `<id>.pgx_cn.vcf.gz`. Training needs ≥ 96 samples (`--min-samples` lowers it
+  for research use) and the platform from `--csv` or `--platform`.
+* **Star alleles**: CPIC star-allele genes, PharmVar genes CPIC does not define (CYP1A2, CYP2A6, CYP2A13, CYP2C8,
+  CYP3A4), G6PD, HLA-A/B tag variants, genotype-only variant-based genes (CPIC and ClinPGx), and `*0` genotypes
+  for GSTM1/GSTT1/UGT2B17. CYP2D6 structures (`*5`, `xN`, `*68+*4`, `*36+*10`, `*13`) come from the sub-region CNs.
+* **Probe mask** (PGX-010): `scripts/pgx/derive_masks.py` compares array calls with 1000 Genomes 30x for the batch's
+  1000G individuals and masks probes below 95 % concordance.
+* **Outputs**: `diplotype_summary.tsv`, `supporting_variants.tsv`, `unresolved_variants.tsv`,
+  `variants_not_covered.tsv`, `pgx_calls.json`; after `phenotype`, `<id>.pgx.json` (`simurg.pgx/1`) and
+  `diplotype_annotations.tsv`. Array-equivalent alleles, no-call impact and alternatives are listed per gene.
+* **DPWG**: genes CPIC covers use the CPIC mappings (as ClinPGx does); DPWG-only genes (CYP1A2, CYP3A4, F5) get no
+  phenotype (K-029).
+
+## Cluster training (ArrayTrain)
+
+`simurg train` builds an EGT cluster file (v3, data block v9) from a cohort's GTCs, so calling does not depend on a
+vendor cluster file (plan §12; validation in `docs/validation/V2_arraytrain.md`).
+
+```bash
+simurg genotype --bpm GSA.bpm --egt any.egt --idats cohort_idats/ --out cohort_gtc   # any EGT: training reads raw X/Y
+simurg train --gtcs cohort_gtc --bpm GSA.bpm --out mycohort.egt                      # ≥ 100 GTCs (--min-samples)
+simurg genotype --bpm GSA.bpm --egt mycohort.egt --idats new_idats/ --out run/       # call with the trained file
+```
+
+* Training uses each GTC's raw intensities and normalisation transforms, not its genotypes. The cluster file the
+  cohort was first called with therefore does not matter.
+* Samples are chosen by sex: X loci are trained from females, Y from males, and MT is haploid. The sex is the GTC's
+  `gender` field.
+* At most `--max-samples` (2000) GTCs are used, as a seeded random subset.
+* **Per locus:**
+  1. 1-, 2- and 3-cluster θ mixtures are fitted; BIC picks one.
+  2. Clusters are labelled AA/AB/BB by Hardy–Weinberg fit plus weak position priors, so compressed or asymmetric
+     loci are handled.
+  3. Missing clusters are imputed from relations learned across well-clustered loci of the same assay type.
+* **Outputs:** `<out>.egt` and `<out>.egt.json`. The JSON gives the counts of 3-, 2- and 1-cluster loci and the
+  median training score.
+* GenTrain and cluster-separation scores in a trained file are SimurgArray's own. GenCall scores are therefore not
+  comparable with calls made with a vendor cluster file, so re-derive any GenCall-score thresholds (K-030).
+* **Accuracy.** GSE165845, trained on 120 arrays and evaluated on 240 other arrays of the same individuals, chr22
+  against 1000G 30x. The trained file gives 99.44 % concordance at a 99.61 % call rate; the vendor file gives
+  99.46 % at 99.15 %.
+* **Time.** 120 GSA samples take about 70 min on a Raspberry Pi 5, with a 2 GB peak.
+
 ## Presets and reproducibility
 
 * `--preset current` (default): normalization 1.2.0, sex rule 2, GTC v5 — the current vendor software behaviour.
@@ -139,23 +286,30 @@ Validated phase by phase (reports in `docs/validation/`):
 | Pedigree QC, 2141 Omni2.5 samples | 717/717 parent–offspring pairs; trio Mendel error median 0.031 % |
 | 360 GSA arrays (GSE165845) vs NYGC 30x WGS | genotype concordance 0.9934; replicate concordance 0.9966 |
 | NA12878 × 3 vs GIAB HG001 v4.2.1 | concordance 0.9985; non-reference F1 0.997 |
+| Methylation QC vs sesame (minfiDataEPIC) | β r = 0.99998 |
+| CN, 360 GSE165845 arrays vs NYGC 30x SV | PASS precision DEL 44 % / DUP 57 %; rare-DEL sensitivity 30 % |
+| PGx, GSA v4 demo vs GeT-RM (16 CEPH) | 154/168 gene calls; GSE165845 array-resolvable 95.7 % |
+| ArrayTrain, GSE165845 chr22 vs 1000G 30x | 99.44 % concordance (vendor EGT 99.46 %) at 99.61 % call rate |
 
 A nightly differential run (`scripts/nightly.py`) re-checks these against the oracles.
 
 ## Limitations
 
-* Calls need an existing EGT cluster file; cluster training (v2), copy number / LOH / mosaicism and PGx
-  diplotyping are not in v1.0.
+* ArrayTrain was validated on one chip and cohort (GSA v1, GSE165845). CN calling finds rare events, but common
+  CNVs are normalised away, and mosaic losses below ~20 % are not separable from noise (K-027). PGx is limited
+  to what the array's probes can resolve: `simurg pgx coverage` lists it. All v2 modules are research-grade.
 * The largest cohort tested is 2141 samples (plan goal ≥ 10 000). Memory grows with samples × loci in the cohort
   QC steps; on an 8 GB machine keep `--threads` low (≈ 2–3 GB per worker for GSA/Omni-size arrays).
-* Speed is that of NumPy: genotyping a GSA sample takes ~3 CPU-seconds on a server core (360 samples in 2 min on
-  16 cores) and ~25 s on a Raspberry Pi 5 core; a compiled core is planned (P9).
-* Methylation arrays: control-probe QC only.
+* Speed: with the compiled kernels a GSA sample takes ~1.1 s on a Raspberry Pi 5 core (pure Python ~3.4 s); see
+  `docs/validation/P9.md` for server numbers and thread scaling. Loading a manifest/cluster file is a fixed cost
+  per run (GSA ~5 s, Omni2.5 ~15 s on the Pi).
+* Methylation arrays: control-probe QC, detection and β values only (no differential analysis).
 
 ## Troubleshooting
 
 Errors and warnings carry a stable code (`E_…`, `W_…`) — all are listed in `docs/messages.md` with their spec item.
 Add `--debug` for a traceback or `--log-json` for machine-readable logs. Exit codes (CLI-014): 0 success,
 1 internal error, 2 usage or pre-flight validation error (nothing processed), 3 some samples failed (or
-`compare --fail-on-discrepancy` found discrepancies), 4 no sample succeeded, 130/143 interrupted. Findings about vendor behaviour that SimurgArray reproduces
+`compare --fail-on-discrepancy` found discrepancies), 4 no sample succeeded, 130/143 interrupted, 141 the reader of
+stdout went away (e.g. `simurg inspect … | head`; nothing is reported). Findings about vendor behaviour that SimurgArray reproduces
 (or deliberately does not) are in `docs/known_unknowns.md`.

docs/validation/P10.md

@@ -0,0 +1,43 @@
+# Phase P10 validation — packaging and fresh-machine acceptance
+
+Date: 2026-09-28. Artifacts built and tested on fresh AWS Ubuntu 24.04 VMs (4 vCPU): aarch64 c7g.xlarge and x86_64
+m6a.xlarge / c6a.xlarge (one x86 spot instance was reclaimed by AWS after 4 min and relaunched). Cloud cost ≈ $0.10.
+
+## Artifacts
+
+| Artifact | How | Notes |
+|---|---|---|
+| `simurg-1.0.0-py3-none-any.whl`, `simurg-1.0.0.tar.gz` | `uv build` (hatchling) | QC report assets, profiles, `LICENSE` and `THIRD_PARTY_NOTICES.md` inside |
+| `simurg_core-1.0.0-cp312-abi3-manylinux_2_28_{aarch64,x86_64}.whl` | `maturin --zig --compatibility manylinux_2_28` (both built on either machine) | one wheel per architecture for every Python ≥ 3.12; glibc ≥ 2.28 |
+| Docker image (`Dockerfile`) | multi-stage, `python:3.12-slim-bookworm`, non-root user | 545 MB (x86_64), 551 MB (aarch64) |
+| conda packages (`packaging/conda/`) | `conda build -c conda-forge` | `simurg` (noarch) + `simurg-core` (Rust compiler from conda-forge) |
+
+`scripts/build_dist.sh` builds the wheels and sdist and writes `dist/SHA256SUMS`. A wheel built on the Pi is not
+portable (glibc 2.41 symbols such as `exp@GLIBC_2.29`), hence the zig-linked manylinux build.
+
+## Acceptance (plan §18 P10: clean install + tutorial works on a fresh VM)
+
+`scripts/cloud/tasks/p10.sh` (and `p10t.sh`, the same without Docker/conda) on each architecture:
+
+| Step | aarch64 | x86_64 |
+|---|---|---|
+| Build sdist, pure wheel, both kernel wheels | ok | ok |
+| `pip install` into a new venv of the system Python 3.12.3 (with / without kernels) | ok / ok | ok / ok |
+| Unit tests against the installed package (225) | pass | pass |
+| Tutorial `examples/gsa_demo.sh` on 6 demo samples, pip install with kernels | ok (506 s) | ok (521 s) |
+| Tutorial, pure-Python install | ok (539 s) | ok (494 s) |
+| Docker build + `genotype` + `gtc-to-vcf` | ok | ok |
+| conda build + install + `genotype` | ok | ok |
+
+Content digests (`scripts/p10/digest.py`: every numeric GTC field and sample statistic with NaNs canonicalised, all
+VCF data lines) of the 6 GTCs and 6 VCFs are **identical across all of them**: pip with kernels, pip pure, Docker and
+conda, on aarch64 and on x86_64, and equal to the Raspberry Pi's own run of the same tutorial. The x86_64 results come
+from AMD EPYC (m6a/c6a) machines; bit-exactness had so far only been shown on ARM.
+
+## Found and fixed
+
+* `simurg inspect … | head` made the CLI report `E_INTERNAL: BrokenPipeError` and exit 1, which stopped the tutorial
+  under `set -o pipefail` (it passed on the Pi only by timing luck). A closed stdout now exits with 141 (like a tool
+  killed by SIGPIPE) and prints nothing; the tutorial reads with `sed` instead of `head`; unit test added.
+* The `fast` extra (`simurg[fast]` → `simurg-core`) was dropped: `uv` must resolve every extra, and `simurg-core` is
+  not on a package index. The kernels are installed as their own wheel.

docs/validation/P9.md

@@ -0,0 +1,82 @@
+# Phase P9 validation — performance (compiled kernels)
+
+Date: 2026-09-28. Development and identity checks on the Raspberry Pi 5; scaling on AWS c7g.8xlarge (32 Graviton3
+cores, 64 GB) with GSE165845 (360 GSA-MD arrays, GSA-24v1-0 C2 manifest). Cloud cost ≈ $0.25 (three short spot runs,
+one of which stopped after 2 min because the image lacked a C linker).
+
+## What was built
+
+Profiling first (`bench/profile_genotype.py`): one GSA sample took 3.4 s on a Pi core, 63 % of it in calling — the
+bit-exact port evaluates `exp`/`log10`/`pow` element-wise through CPython's `math` (≈ 6.8 M scalar calls per sample) —
+and 24 % in the normalization fit's nearest-neighbour searches. Reading a manifest cost 13 s (GSA) to 53 s (Omni2.5)
+per command. The IDAT parser was already negligible (0.02 s), so it was not ported.
+
+`rust/simurg-core` (PyO3 + maturin, optional; imported as `simurg_core`, used through `simurg._core`):
+
+| Kernel | Replaces | Notes |
+|---|---|---|
+| `call_genotypes` | `genotype.gencall.call_genotypes` | one fused pass per locus: flair SDs, the six likelihood cases, posteriors, sub-scores, score map; every intermediate in the reference's f32/f64 type and order |
+| `baf_lrr` | `genotype.baf_lrr.baf_lrr` | canonical NaN 0xFFC00000 kept |
+| `exp`, `log10`, `log2`, `pow` | `gencall._libm` | glibc libm (the functions CPython's `math` calls) with CPython's error semantics (NaN passthrough, +NaN on domain error, `log10(0)` = −inf) |
+| `closest_binned`, `closest_exact` | the two nearest-neighbour searches of `normalization.fit` | float32 distances, first minimum wins, NaN wins as in `numpy.argmin` |
+| `bpm_locus_entries` | `formats.bpm._parse_locus_entries` | same columns, string interning and error messages |
+
+No floating-point contraction: Rust never fuses `a*b+c` (cf. K-012). Without the extension, or with
+`SIMURG_PURE_PYTHON=1`, the pure paths run unchanged. Also: address coverage is memoised per IDAT layout (it was
+recomputed per sample), and the worker state is pickled once instead of once per worker.
+
+## Acceptance (plan §18 P9)
+
+### Outputs identical to the Python reference
+
+| Check | Result |
+|---|---|
+| `tests/unit/test_core_kernels.py` (19 tests): every kernel vs its pure path on edge values (±0, ±inf, NaN payloads and signs, subnormals, overflow, ties, empty bins, NaN clusters, zero SDs, empty clusters, missing thresholds), bit patterns compared | pass (Pi; on Graviton3 the run log does not show the count, the 360-GTC identity below covers that machine) |
+| 36 GSA v4 demo samples (Pi) | 36/36 GTCs byte-identical; `sample_summary.tsv`, `controls_qc.tsv`, `relatedness.tsv` identical |
+| 6 Omni2.5 samples, `--preset autocall-1.6.2` (Pi) | 6/6 GTCs and all tables identical |
+| 360 GSE165845 samples (Graviton3) | **360/360 GTCs byte-identical**, sample summary identical |
+| 5 manifests (GSA v1 C1/C2, GSA v4, GDA v1, Omni2.5) parsed by both BPM parsers | every column identical |
+| Oracle tests P1, P3, P4 (vs gtc2vcf/vendor) | pass with the kernels |
+
+### Throughput: ≥ 20 samples/min/core (GSA-size arrays)
+
+| Machine, 1 worker | Pure Python | Kernels |
+|---|---|---|
+| Graviton3 (360 samples, per-sample phase) | 29.6 samples/min | **78–80 samples/min** (0.75 s per sample) |
+| Raspberry Pi 5 (`genotype_sample` only) | 3.4 s per sample | 1.1 s per sample (≈ 55/min) |
+
+Manifest loading: GSA 13 s → 3 s, Omni2.5 53 s → 13 s, GDA 40 s → 10 s (Pi). A 36-sample GSA run on the Pi
+(3 workers, with controls and cohort QC) went from 131 s to 45 s.
+
+### Scaling to 32 threads
+
+360 samples, `simurg genotype --no-cohort-qc`, per-sample phase from the run log (`bench/scaling.py`):
+
+| Workers | GTCs on the gp3 disk (125 MB/s) | | GTCs on a RAM disk | |
+|---|---|---|---|---|
+| | samples/min | efficiency | samples/min | efficiency |
+| 1 | 78 | 1.00 | 80 | 1.00 |
+| 4 | 290 | 0.93 | 305 | 0.95 |
+| 8 | 546 | 0.87 | 578 | 0.90 |
+| 16 | 618 | 0.49 | 979 | 0.77 |
+| 24 | — | — | 1186 | 0.62 |
+| 32 | 619 | 0.25 | 1226 | 0.48 |
+
+* On the default gp3 disk the run is **write-bound** above 8 workers: one GSA GTC is ~13 MB, so ~620 samples/min is
+  ~140 MB/s, the volume's throughput. The pure-Python path hit the same ceiling (550 samples/min at 32 workers).
+  Faster storage (gp3 with more MB/s, local NVMe, tmpfs) or fewer outputs is needed to go beyond it.
+* On a RAM disk the remaining gap is a **fixed start-up cost that grows with the worker count**: measured phase minus
+  ideal (0.75 s × 360 / workers) is 3.4 s at 4 workers, 3.6 s at 8, 5.2 s at 16, 7.0 s at 24 and 9.2 s at 32
+  (≈ 2.5 s + 0.2 s per worker: process spawn, imports, manifest context). It does not depend on the number of samples,
+  so with 360 samples (11 per worker at 32 workers) it dominates. **Inferred, not measured:** for a 10 000-sample batch
+  at 32 workers the same overhead gives an efficiency of ≈ 0.96. The plan's "linear scaling to 32 threads" is
+  therefore met for large batches on fast storage, and not for short batches or on a 125 MB/s disk.
+* Pickling the worker state once (instead of per worker) did not change start-up on the Pi (4–8 workers: 1.5–2.6 s
+  either way); it is kept because it removes serial work from the parent.
+
+## Notes
+
+* Remaining per-sample time with the kernels (Pi, 1.1 s): normalization fit 0.53 s (many small robust fits per bin),
+  calling 0.24 s, normalize/polar 0.08 s, raw X/Y 0.06 s. A full Rust normalization fit is the next step if needed.
+* Pi thread scaling is poor (36 samples, 3 workers ≈ 1.7× one core): the Pi 5's memory bandwidth, not the code.
+* CI (`.github/workflows/ci.yml`) builds the extension and runs the unit tests with and without it.

docs/validation/P9_identity_360.json

@@ -0,0 +1 @@
+{"gtcs": 360, "differ": 0, "sample_summary": "identical"}

docs/validation/P9_scaling_gp3.json

@@ -0,0 +1,74 @@
+{
+ "samples": 360,
+ "backend": "rust 1.0.0",
+ "cpu_count": 32,
+ "machine": "aarch64",
+ "runs": [
+  {
+   "threads": 1,
+   "rc": 0,
+   "wall_s": 299.1,
+   "setup_s": 6.9,
+   "per_sample_phase_s": 276.63,
+   "samples_per_min": 78.1,
+   "samples_per_min_per_core": 78.1,
+   "speedup_vs_first": 1.0,
+   "efficiency": 1.0
+  },
+  {
+   "threads": 2,
+   "rc": 0,
+   "wall_s": 166.5,
+   "setup_s": 7.1,
+   "per_sample_phase_s": 143.78,
+   "samples_per_min": 150.2,
+   "samples_per_min_per_core": 75.1,
+   "speedup_vs_first": 1.92,
+   "efficiency": 0.96
+  },
+  {
+   "threads": 4,
+   "rc": 0,
+   "wall_s": 96.9,
+   "setup_s": 6.8,
+   "per_sample_phase_s": 74.58,
+   "samples_per_min": 289.6,
+   "samples_per_min_per_core": 72.4,
+   "speedup_vs_first": 3.71,
+   "efficiency": 0.927
+  },
+  {
+   "threads": 8,
+   "rc": 0,
+   "wall_s": 61.9,
+   "setup_s": 6.8,
+   "per_sample_phase_s": 39.56,
+   "samples_per_min": 546.0,
+   "samples_per_min_per_core": 68.3,
+   "speedup_vs_first": 6.99,
+   "efficiency": 0.874
+  },
+  {
+   "threads": 16,
+   "rc": 0,
+   "wall_s": 57.4,
+   "setup_s": 6.9,
+   "per_sample_phase_s": 34.98,
+   "samples_per_min": 617.5,
+   "samples_per_min_per_core": 38.6,
+   "speedup_vs_first": 7.91,
+   "efficiency": 0.494
+  },
+  {
+   "threads": 32,
+   "rc": 0,
+   "wall_s": 57.4,
+   "setup_s": 6.9,
+   "per_sample_phase_s": 34.91,
+   "samples_per_min": 618.8,
+   "samples_per_min_per_core": 19.3,
+   "speedup_vs_first": 7.92,
+   "efficiency": 0.247
+  }
+ ]
+}

docs/validation/P9_scaling_ramdisk.json

@@ -0,0 +1,74 @@
+{
+ "samples": 360,
+ "backend": "rust 1.0.0",
+ "cpu_count": 32,
+ "machine": "aarch64",
+ "runs": [
+  {
+   "threads": 1,
+   "rc": 0,
+   "wall_s": 292.6,
+   "setup_s": 6.8,
+   "per_sample_phase_s": 270.27,
+   "samples_per_min": 79.9,
+   "samples_per_min_per_core": 79.9,
+   "speedup_vs_first": 1.0,
+   "efficiency": 1.0
+  },
+  {
+   "threads": 4,
+   "rc": 0,
+   "wall_s": 93.3,
+   "setup_s": 6.8,
+   "per_sample_phase_s": 70.94,
+   "samples_per_min": 304.5,
+   "samples_per_min_per_core": 76.1,
+   "speedup_vs_first": 3.81,
+   "efficiency": 0.953
+  },
+  {
+   "threads": 8,
+   "rc": 0,
+   "wall_s": 59.7,
+   "setup_s": 6.8,
+   "per_sample_phase_s": 37.37,
+   "samples_per_min": 578.0,
+   "samples_per_min_per_core": 72.2,
+   "speedup_vs_first": 7.23,
+   "efficiency": 0.904
+  },
+  {
+   "threads": 16,
+   "rc": 0,
+   "wall_s": 44.4,
+   "setup_s": 6.8,
+   "per_sample_phase_s": 22.06,
+   "samples_per_min": 979.3,
+   "samples_per_min_per_core": 61.2,
+   "speedup_vs_first": 12.25,
+   "efficiency": 0.766
+  },
+  {
+   "threads": 24,
+   "rc": 0,
+   "wall_s": 40.6,
+   "setup_s": 6.8,
+   "per_sample_phase_s": 18.22,
+   "samples_per_min": 1185.6,
+   "samples_per_min_per_core": 49.4,
+   "speedup_vs_first": 14.83,
+   "efficiency": 0.618
+  },
+  {
+   "threads": 32,
+   "rc": 0,
+   "wall_s": 40.0,
+   "setup_s": 6.8,
+   "per_sample_phase_s": 17.62,
+   "samples_per_min": 1225.7,
+   "samples_per_min_per_core": 38.3,
+   "speedup_vs_first": 15.34,
+   "efficiency": 0.479
+  }
+ ]
+}

docs/validation/V2_arraytrain.md

@@ -0,0 +1,86 @@
+# V2 — ArrayTrain (plan §12)
+
+`simurg train` builds an EGT cluster file from a cohort's GTCs. It does not reproduce the vendor's clustering, so the
+benchmark is genotype accuracy against sequencing truth: an EGT trained by SimurgArray versus the vendor's published
+cluster file, used by the same GenCall emulation on the same arrays.
+
+## Set-up
+
+- **Data.** GSE165845: 120 1000 Genomes individuals, each on 3 GSA v1 arrays. The C2 build-38 BPM is used; the
+  public GSA v1 C1 cluster file is the vendor baseline (K-024).
+- **Training.** Replicate 3, 120 GTCs (59 F, 60 M, 1 U), all 618 540 loci. Pi 5, 69 min, peak 1.9 GB.
+- **Evaluation.** Replicates 1–2 (240 arrays) go from IDAT through `simurg genotype` with the trained EGT, then
+  `simurg gtc-to-vcf`. The same arrays are also genotyped with the vendor EGT.
+- **Truth.** The NYGC 30x phased 1000G panel on chr22: 8278 array SNV sites (`scripts/train/concordance_chr.py`).
+  Concordance counts called genotypes only. NRD is the non-reference discordance.
+
+The evaluation individuals are also in the training set, but on different arrays. The cluster file never sees their
+truth; the same holds for the vendor file.
+
+## Results (240 arrays, chr22)
+
+| Cluster file | Call rate | Concordance | NRD |
+|---|---|---|---|
+| ArrayTrain v1 (fixed θ windows) | 99.47 % | 99.27 % | 1.89 % |
+| **ArrayTrain v2 (BIC + HWE labelling)** | **99.61 %** | **99.44 %** | **1.30 %** |
+| Vendor GSA v1 C1 EGT | 99.15 % | 99.46 % | 1.23 % |
+
+`V2_arraytrain_gse165845.json` holds the numbers. v2 calls 0.46 % more genotypes than the vendor file at nearly the
+same concordance (−0.02 points).
+
+## What changed from v1, and why
+
+The v1 training fitted one three-component θ mixture. Its means were bounded to fixed windows (AA ≤ 0.4,
+0.2 ≤ AB ≤ 0.8, BB ≥ 0.6), and close clusters were merged by a fixed θ gap.
+
+A per-locus diagnosis on chr22 compared our trained clusters, the vendor clusters and the truth for 80 arrays:
+
+- **Errors concentrate in few loci.** The top 20 loci held 43 % of the errors.
+- **2-cluster loci were the problem.** There, v1 made 1418 errors against the vendor's 186.
+- **The cause was the fixed θ windows.** Many loci are compressed or asymmetric in θ; for example the vendor file
+  has AA 0.05 / AB 0.27 / BB 0.45, or AA 0.37 / AB 0.68 / BB 0.96. With fixed windows such loci were mislabelled:
+  rs2003814 (truth 14 AB + 66 BB) was clustered as AA + AB.
+
+v2 (`src/simurg/train/clusters.py`) replaces this with:
+
+1. **Unbounded 1-, 2- and 3-component mixtures** on θ, plus a uniform outlier component. Each is fitted by EM from
+   several starts: canonical positions and θ quantiles.
+   - An SD floor of 0.02 inside the EM stops a component collapsing onto the θ = 0 / 1 spikes.
+   - BIC chooses among admissible fits. A fit is admissible if every component has ≥ max(3, 1 %) of the samples and
+     adjacent components are ≥ 1.5 standardised gaps apart.
+2. **Labelling of the θ-ordered components** by the best non-decreasing map onto AA/AB/BB. A map's score adds up:
+   - the HWE goodness of fit of the implied genotype counts, with an inbreeding allowance F = 0.1. This is the fit
+     relative to the saturated multinomial, so monomorphic labellings are not favoured automatically;
+   - weak priors on the cluster positions: AA ~ N(0.03, 0.2), AB ~ N(0.5, 0.25), BB ~ N(0.97, 0.2);
+   - a weak prior on the θ gaps between genotype groups: 0.42 ± 0.12 per step.
+
+   Adjacent components may share a genotype (a split cluster) only when their standardised gap is < 3.5.
+
+Fast chr22 loop (train on replicate 3, call 80 evaluation arrays, score against truth):
+
+| Step | Errors (80 arrays) |
+|---|---|
+| v1 | 2562 |
+| Unbounded mixtures + HWE labelling of 2-cluster fits | 1793 |
+| + MIN_SEP 1.5 and position prior | 1611 |
+| + general labelling with split-cluster merges (HWE goodness of fit) | 1380 |
+| Vendor EGT | 1237 |
+
+- **Parameter sweep.** MERGE_SEP {2.5, 3.5, 5} × HWE F {0.05, 0.1, 0.2} × gap SD {0.12, 0.2}. Every setting
+  lands at 99.70–99.79 % accuracy; MERGE_SEP 5 is clearly worse. The chosen values are in the middle of a flat
+  optimum.
+- **Where the errors are now.** By our cluster count, v2's errors are 123 at 1-cluster loci (vendor 157), 217 at
+  2-cluster loci (vendor 113) and 1040 at 3-cluster loci (vendor 967).
+- **Many 3-cluster errors are shared with the vendor.** Both files see three clear clusters where the truth is
+  monomorphic, e.g. rs191067607 and rs5758036. These are probe or truth problems, not clustering errors.
+
+## Limits (K-030)
+
+- Only one cohort and one chromosome were used for accuracy. The tuning used this evaluation, so the numbers are
+  optimistic for other chips and cohorts.
+- 120 training arrays is close to the 100-sample minimum. Rare-allele clusters are imputed from cross-locus
+  relations, not observed.
+- The trained EGT scores (GenTrain, cluster separation) are ours. GenCall scores from a trained file are not
+  comparable with scores from vendor files. At a matched call rate, GenCall score ranks v1's errors worse than the
+  vendor's.
+- Training takes 69 min for 120 × 618 540 on the Pi (8 EM runs per locus); a Rust kernel would help at cohort scale.

docs/validation/V2_arraytrain_gse165845.json

@@ -0,0 +1,28 @@
+{
+ "dataset": "GSE165845 GSA v1 (C2 build-38 BPM); train = replicate 3 (120 arrays), evaluate = replicates 1-2 (240 arrays)",
+ "truth": "1000G NYGC 30x phased panel, chr22 SNVs on the array",
+ "egt_v1_fixed_windows": {
+  "chrom": "22",
+  "samples": 240,
+  "truth_sites": 8278,
+  "call_rate": 0.9947053434807118,
+  "concordance": 0.9926765907658587,
+  "nrd": 0.018946074088334416
+ },
+ "egt_v2_bic_hwe": {
+  "chrom": "22",
+  "samples": 240,
+  "truth_sites": 8278,
+  "call_rate": 0.9960623540307643,
+  "concordance": 0.9943676222304063,
+  "nrd": 0.013034595450012981
+ },
+ "vendor_gsa_v1_c1_egt": {
+  "chrom": "22",
+  "samples": 240,
+  "truth_sites": 8278,
+  "call_rate": 0.9915201940887493,
+  "concordance": 0.9945781643910259,
+  "nrd": 0.012251034586575862
+ }
+}

docs/validation/V2_cn.md

@@ -0,0 +1,115 @@
+# v2 module validation — genome-wide CNV / LOH / mosaicism (`simurg cn`) — interim
+
+Date: 2026-09-29. Status: **not accepted yet** — constitutional calling works, mosaic calling produces mostly false
+positives on the real-data set; see "Next steps". Spec: CYTO-001 … CYTO-040.
+
+## Delivered
+
+`simurg cn build-model` (GC 1 kb / 100 kb / 1 Mb, PFB from the cluster file, sex-aware batch reference from normal
+GTCs), `simurg cn call` (GC-wave correction, smoothing, HMM with constitutional, gain-LOH, mosaic and haploid
+states, events, post-filters, thresholds, centromere mask, CNV VCF + tabix, segments, summary, sidecar),
+`simurg cn annotate` (JSON report, ISCN 2020, cytobands, RefSeq genes), `scripts/build_annotation_bundle.sh`.
+Unit tests: `tests/unit/test_cn.py` (11).
+
+## Synthetic implants (CYTO-022), GSA v4 demo, 22 synthetic samples, 216 events
+
+Reference model from 30 samples, events implanted into the other 6 (`scripts/v2/cn_synthetic_eval.py`,
+`V2_cn_synthetic_gsa36.json`), 4 replicates per cell:
+
+| Event | 1 Mb | 5 Mb | 20 Mb |
+|---|---|---|---|
+| constitutional DEL / DUP | 4/4 · 2/4 | 4/4 · 4/4 | 4/4 · 4/4 |
+| constitutional LOH (threshold 3 Mb) | 0/4 | 3/4 | 4/4 |
+| mosaic loss / gain / CN-LOH, 15 % | 3/4 · 1/4 · 0/4 | **4/4 · 4/4 · 4/4** | 4/4 · 4/4 · 4/4 |
+| mosaic, 5–10 % | mostly missed | mostly missed | 0–3/4 |
+
+The CYTO-022 gate (≥ 90 % at 15 % / 5 Mb) is met on implanted data (12/12, small n).
+
+## Real data: GSE165845 (360 GSA-MD arrays, 120 individuals × 3) vs NYGC 30x SV callset (freeze V3)
+
+Cloud run 2026-09-29 (m7g.4xlarge, `cn call` 360 samples in 7 min 40 s; reference from 120 arrays of the same set).
+Truth: non-reference DEL/DUP ≥ 25 kb of the same individual, ≥ 10 array probes; match = reciprocal overlap ≥ 50 %.
+
+* Every sample was flagged `SampleQuality` (summary `logr_dev` median 0.246 — the GSA v1 C1 cluster file does not
+  match the GSA-MD chips, P8) and 46 fell into whole-chromosome-only mode, although the LRR used for calling has a
+  robust SD of 0.099 after the batch reference. The gate now uses the SD of the normalised LRR (K-027). The numbers
+  below ignore `SampleQuality` (`--ignore-filter`), evaluated from the run's sidecar.
+
+| QUAL ≥ 20 | DEL | DUP |
+|---|---|---|
+| sensitivity 25–50 / 50–100 / 100–500 / 500–1000 kb | 7 / 8 / 8 / 7 % | 1 / 3 / 11 / 13 % |
+| precision (all calls of the class) | 39 % | 34 % |
+
+| Call type (QUAL ≥ 20) | calls | in NYGC truth | seen in another replicate |
+|---|---|---|---|
+| constitutional DEL | 227 | **77 %** | 50 % |
+| constitutional DUP | 23 | **74 %** | 30 % |
+| LOH | — | — | 74 % |
+| mosaic loss | 399 | 12 % | 12 % |
+| mosaic gain | 567 | 23 % | 23 % |
+
+Constitutional calls are mostly true; the mosaic states generate most calls and most false positives (large
+BAF-supported "mosaic" events are 0–2 % in truth and rarely reproduced: shifted heterozygous BAF bands of the
+mismatched cluster file mimic allelic imbalance). Small BAF-supported mosaic gains are often real germline
+duplications labelled mosaic (43 % in truth).
+
+## Pi tuning rounds (2026-09-29), 40 arrays (20 individuals × replicates 1, 2), reference = replicate 3 of all 120
+
+`scripts/v2/cn_tune.sh TAG [options]`; truth as above, QUAL ≥ 20, PASS only (after the K-027 noise-gate fix).
+
+| Round | DEL calls / in truth | DUP | mosaic loss PASS / in truth | mosaic gain PASS / in truth | LOH replicated |
+|---|---|---|---|---|---|
+| base (per-probe reference het BAF spread in the emissions) | 22 / 100 % | 3 / 67 % | 29 / 11 % | 24 / 38 % | 80 % |
+| `--mosaic-prior 0.0005` | 25 / 96 % | 4 / 75 % | 27 / 12 % | 22 / 36 % | 80 % |
+| `--mosaic-prior 0.0005 --switch-distance 1e8` | 24 / 96 % | 4 / 75 % | 25 / 14 % | 23 / 35 % | 80 % |
+| CYTO-041 (noise-normalised BAF split, complete-loss promotion) | 27 / 93 % | 4 / 75 % | 19 / 5 % | 19 / 42 % | 80 % |
+| + CYTO-042 (per-probe LRR noise from the reference; **default**) | 20 / **100 %** | 4 / **100 %** | **11** / 9 % | **13** / **62 %** | 80 % |
+
+* The mosaic prior and switch distance barely matter. CYTO-041 moves heterozygote-free "mosaic" losses (compressed
+  constitutional deletions) to DEL (`HIGHFRACTION`) and sends noise-driven mosaic calls to `LowFraction`
+  (MDEL 29 → 19 PASS, MDUP 24 → 19). On the implanted set (`data/v2cn/syn3`) MF estimates stay unbiased
+  (15 % → 0.16, 20 % → 0.21, 30 % → 0.30), detection is unchanged (126/216) and the CYTO-022 gate still passes.
+* Remaining mosaic-loss false positives are long, low-amplitude LRR dips (−0.05 … −0.1) with a heterozygous BAF
+  spread only 1.0–1.3 × the probe's reference spread, or short events with < 20 heterozygous probes (MF from LRR).
+  Without phasing a < 20 % mosaic loss cannot be separated from regional BAF noise (K-027).
+* CYTO-042 (per-probe emission SD = sample SD × the probe's reference LRR spread / median spread, clipped to
+  [0.5, 3]) halves the mosaic calls and makes constitutional calls fully precise, at the cost of 5 rare deletions
+  (rare DEL 24/64 → 19/64; rare DUP 10/68 → 12/68). Synthetic set with the rebuilt model (`data/v2cn/syn4`): calls
+  564 → 402, detection 126 → 124/216, the CYTO-022 gate (15 % / 5 Mb: 4/4 loss, gain, CN-LOH) still passes.
+  Disable with `--no-probe-lrr-sd`.
+* **Sensitivity is limited by common CNVs.** Detected truth DEL by carrier frequency among the 120 individuals:
+  < 5 %: 24/64 (38 %), 5–20 %: 1/20, 20–50 %: 1/80, > 50 %: 0/156 (DUP 10/66, 0/2, 1/24, 0/40). The batch
+  reference is the per-probe median LRR, so an event carried by most reference samples is normalised away (and the
+  cluster file, trained on the population, already absorbs part of it). Array CN calling targets rare events; the
+  headline sensitivity figures above are dominated by common polymorphic deletions.
+
+## Final full-cohort run (cloud, 2026-09-29, commit ccf7f4f, CYTO-041/042 defaults)
+
+The run used m7g.4xlarge spot: 17 min in total (`cn build-model` 1 min 17 s, 6.0 GB; `cn call` of 360 samples 7 min 44 s, 1.7 GB).
+Truth and matching are the same as above. PASS calls only, QUAL ≥ 20, no filters ignored
+(`V2_cn_gse165845_final.json`).
+
+* **Sample QC.** All 360 samples are `OK`; the first cloud run flagged every sample `SampleQuality` (the K-027 gate
+  fix). 14 samples carry `high_noise` and one carries `excess_events`.
+* **The `excess_events` array.** 203610890020_R06C01 (NA19923) has 238 DEL and 688 mosaic calls, and none of its
+  147 CNV calls is in truth. It is a bad array, and it is the only one flagged.
+* **Per-sample call counts (median).** 1 DEL, 0 DUP, 4 mosaic.
+
+| | DEL | DUP |
+|---|---|---|
+| precision, PASS calls | 213/480 = **44 %** (first run, filters ignored: 39 %) | 142/249 = **57 %** (first run, filters ignored: 34 %) |
+| precision, CNV calls without R06C01 | 355/582 = **61 %** (DEL + DUP) | |
+| sensitivity 25–50 / 50–100 / 100–500 / 500–1000 kb | 5 / 7 / 8 / 9 % | 0 / 3 / 10 / 14 % |
+| sensitivity by carrier frequency < 5 % / 5–20 % / ≥ 20 % | **30 %** / 5 % / 0.05 % | 13 % / 21 % / 0.6 % |
+
+* **Replicate reproducibility.** CNV 35 %, LOH 85 %.
+* **What CYTO-041/042 changed.** Precision is clearly higher and sensitivity is unchanged. The low overall
+  sensitivity is the common-CNV effect described above, not missed rare events.
+* **Rare deletions.** 30 % of rare (< 5 %) truth deletions are found, the same as in the Pi tuning.
+
+## Next steps
+
+1. Mosaic losses below ~20 %: phasing-based BAF (MoChA-style, MIT) is the principled fix.
+2. Rare-event sensitivity (currently DEL 30 %, DUP 18 % for events ≥ 25 kb with ≥ 10 probes): exclude reference
+   samples carrying the event when computing the per-probe median (iterative reference), size-aware transitions.
+3. ~~A final full-cohort run~~ — done (section above).

docs/validation/V2_cn_gse165845_final.json

@@ -0,0 +1,123 @@
+{
+ "samples": 360,
+ "min_qual": 20,
+ "ignored_filters": [],
+ "sensitivity": {
+  "DEL": {
+   "100-500kb": {
+    "found": 128,
+    "truth": 1554,
+    "rate": 0.0824
+   },
+   "1000-infkb": {
+    "found": 0,
+    "truth": 15,
+    "rate": 0.0
+   },
+   "25-50kb": {
+    "found": 12,
+    "truth": 225,
+    "rate": 0.0533
+   },
+   "50-100kb": {
+    "found": 59,
+    "truth": 894,
+    "rate": 0.066
+   },
+   "500-1000kb": {
+    "found": 16,
+    "truth": 180,
+    "rate": 0.0889
+   }
+  },
+  "DUP": {
+   "100-500kb": {
+    "found": 109,
+    "truth": 1059,
+    "rate": 0.1029
+   },
+   "1000-infkb": {
+    "found": 9,
+    "truth": 21,
+    "rate": 0.4286
+   },
+   "25-50kb": {
+    "found": 0,
+    "truth": 198,
+    "rate": 0.0
+   },
+   "50-100kb": {
+    "found": 13,
+    "truth": 462,
+    "rate": 0.0281
+   },
+   "500-1000kb": {
+    "found": 11,
+    "truth": 78,
+    "rate": 0.141
+   }
+  }
+ },
+ "sensitivity_by_carrier_frequency": {
+  "DEL": {
+   "5-20%": {
+    "found": 15,
+    "truth": 285,
+    "rate": 0.0526
+   },
+   "common>=20%": {
+    "found": 1,
+    "truth": 1923,
+    "rate": 0.0005
+   },
+   "rare<5%": {
+    "found": 199,
+    "truth": 660,
+    "rate": 0.3015
+   }
+  },
+  "DUP": {
+   "5-20%": {
+    "found": 21,
+    "truth": 102,
+    "rate": 0.2059
+   },
+   "common>=20%": {
+    "found": 5,
+    "truth": 855,
+    "rate": 0.0058
+   },
+   "rare<5%": {
+    "found": 116,
+    "truth": 861,
+    "rate": 0.1347
+   }
+  }
+ },
+ "precision": {
+  "DEL": {
+   "in_truth": 213,
+   "calls": 480,
+   "rate": 0.4437
+  },
+  "DUP": {
+   "in_truth": 142,
+   "calls": 249,
+   "rate": 0.5703
+  }
+ },
+ "replicate_reproducibility": {
+  "CNV": {
+   "matched": 512,
+   "comparisons": 1458,
+   "rate": 0.3512
+  },
+  "LOH": {
+   "matched": 356,
+   "comparisons": 420,
+   "rate": 0.8476
+  }
+ },
+ "calls_per_sample_median": 1.0,
+ "loh_calls_per_sample_median": 0.0
+}

docs/validation/V2_cn_gse165845_q10.json

@@ -0,0 +1,89 @@
+{
+ "samples": 360,
+ "min_qual": 10.0,
+ "ignored_filters": [
+  "SampleQuality"
+ ],
+ "sensitivity": {
+  "DEL": {
+   "100-500kb": {
+    "found": 230,
+    "truth": 1554,
+    "rate": 0.148
+   },
+   "1000-infkb": {
+    "found": 0,
+    "truth": 15,
+    "rate": 0.0
+   },
+   "25-50kb": {
+    "found": 54,
+    "truth": 225,
+    "rate": 0.24
+   },
+   "50-100kb": {
+    "found": 132,
+    "truth": 894,
+    "rate": 0.1477
+   },
+   "500-1000kb": {
+    "found": 51,
+    "truth": 180,
+    "rate": 0.2833
+   }
+  },
+  "DUP": {
+   "100-500kb": {
+    "found": 279,
+    "truth": 1059,
+    "rate": 0.2635
+   },
+   "1000-infkb": {
+    "found": 9,
+    "truth": 21,
+    "rate": 0.4286
+   },
+   "25-50kb": {
+    "found": 4,
+    "truth": 198,
+    "rate": 0.0202
+   },
+   "50-100kb": {
+    "found": 55,
+    "truth": 462,
+    "rate": 0.119
+   },
+   "500-1000kb": {
+    "found": 19,
+    "truth": 78,
+    "rate": 0.2436
+   }
+  }
+ },
+ "precision": {
+  "DEL": {
+   "in_truth": 473,
+   "calls": 1814,
+   "rate": 0.2607
+  },
+  "DUP": {
+   "in_truth": 367,
+   "calls": 1494,
+   "rate": 0.2456
+  }
+ },
+ "replicate_reproducibility": {
+  "CNV": {
+   "matched": 2132,
+   "comparisons": 6616,
+   "rate": 0.3222
+  },
+  "LOH": {
+   "matched": 400,
+   "comparisons": 532,
+   "rate": 0.7519
+  }
+ },
+ "calls_per_sample_median": 6.0,
+ "loh_calls_per_sample_median": 0.0
+}

docs/validation/V2_cn_gse165845_q20.json

@@ -0,0 +1,89 @@
+{
+ "samples": 360,
+ "min_qual": 20.0,
+ "ignored_filters": [
+  "SampleQuality"
+ ],
+ "sensitivity": {
+  "DEL": {
+   "100-500kb": {
+    "found": 126,
+    "truth": 1554,
+    "rate": 0.0811
+   },
+   "1000-infkb": {
+    "found": 0,
+    "truth": 15,
+    "rate": 0.0
+   },
+   "25-50kb": {
+    "found": 16,
+    "truth": 225,
+    "rate": 0.0711
+   },
+   "50-100kb": {
+    "found": 70,
+    "truth": 894,
+    "rate": 0.0783
+   },
+   "500-1000kb": {
+    "found": 13,
+    "truth": 180,
+    "rate": 0.0722
+   }
+  },
+  "DUP": {
+   "100-500kb": {
+    "found": 117,
+    "truth": 1059,
+    "rate": 0.1105
+   },
+   "1000-infkb": {
+    "found": 6,
+    "truth": 21,
+    "rate": 0.2857
+   },
+   "25-50kb": {
+    "found": 1,
+    "truth": 198,
+    "rate": 0.0051
+   },
+   "50-100kb": {
+    "found": 15,
+    "truth": 462,
+    "rate": 0.0325
+   },
+   "500-1000kb": {
+    "found": 10,
+    "truth": 78,
+    "rate": 0.1282
+   }
+  }
+ },
+ "precision": {
+  "DEL": {
+   "in_truth": 222,
+   "calls": 571,
+   "rate": 0.3888
+  },
+  "DUP": {
+   "in_truth": 149,
+   "calls": 436,
+   "rate": 0.3417
+  }
+ },
+ "replicate_reproducibility": {
+  "CNV": {
+   "matched": 554,
+   "comparisons": 2014,
+   "rate": 0.2751
+  },
+  "LOH": {
+   "matched": 286,
+   "comparisons": 394,
+   "rate": 0.7259
+  }
+ },
+ "calls_per_sample_median": 2.0,
+ "loh_calls_per_sample_median": 0.0
+}

docs/validation/V2_cn_synthetic_gsa36.json

@@ -0,0 +1,97 @@
+{
+ "events": 216,
+ "detected": 126,
+ "table": {
+  "MDEL f=0.10": {
+   "5 Mb": "1/4",
+   "20 Mb": "3/4",
+   "1 Mb": "1/4"
+  },
+  "MLOH f=0.10": {
+   "5 Mb": "0/4",
+   "20 Mb": "0/4",
+   "1 Mb": "0/4"
+  },
+  "MDUP f=0.20": {
+   "1 Mb": "4/4",
+   "20 Mb": "4/4",
+   "5 Mb": "3/4"
+  },
+  "MLOH f=0.15": {
+   "1 Mb": "0/4",
+   "5 Mb": "4/4",
+   "20 Mb": "4/4"
+  },
+  "MLOH f=0.30": {
+   "1 Mb": "0/4",
+   "20 Mb": "4/4",
+   "5 Mb": "2/4"
+  },
+  "DEL": {
+   "1 Mb": "4/4",
+   "5 Mb": "4/4",
+   "20 Mb": "4/4"
+  },
+  "MLOH f=0.20": {
+   "1 Mb": "0/4",
+   "5 Mb": "4/4",
+   "20 Mb": "4/4"
+  },
+  "MLOH f=0.05": {
+   "1 Mb": "0/4",
+   "5 Mb": "0/4",
+   "20 Mb": "0/4"
+  },
+  "MDEL f=0.20": {
+   "1 Mb": "4/4",
+   "20 Mb": "4/4",
+   "5 Mb": "3/4"
+  },
+  "MDEL f=0.30": {
+   "20 Mb": "4/4",
+   "5 Mb": "4/4",
+   "1 Mb": "4/4"
+  },
+  "MDUP f=0.10": {
+   "20 Mb": "2/4",
+   "1 Mb": "0/4",
+   "5 Mb": "2/4"
+  },
+  "MDUP f=0.30": {
+   "5 Mb": "4/4",
+   "20 Mb": "4/4",
+   "1 Mb": "4/4"
+  },
+  "DUP": {
+   "1 Mb": "2/4",
+   "20 Mb": "4/4",
+   "5 Mb": "4/4"
+  },
+  "MDEL f=0.05": {
+   "5 Mb": "0/4",
+   "20 Mb": "0/4",
+   "1 Mb": "0/4"
+  },
+  "LOH": {
+   "5 Mb": "3/4",
+   "20 Mb": "4/4",
+   "1 Mb": "0/4"
+  },
+  "MDUP f=0.15": {
+   "1 Mb": "1/4",
+   "5 Mb": "4/4",
+   "20 Mb": "4/4"
+  },
+  "MDEL f=0.15": {
+   "20 Mb": "4/4",
+   "5 Mb": "4/4",
+   "1 Mb": "3/4"
+  },
+  "MDUP f=0.05": {
+   "5 Mb": "0/4",
+   "1 Mb": "0/4",
+   "20 Mb": "0/4"
+  }
+ },
+ "calls_total": 564
+}

docs/validation/V2_methylation.md

@@ -0,0 +1,53 @@
+# v2 module validation — methylation control QC (`simurg methyl qc`)
+
+Date: 2026-09-29. Development on the Raspberry Pi; oracle runs on AWS (c7g.xlarge, five short runs, ≈ $0.025).
+Spec: MET-001 … MET-018 (and QC-004/QC-006, QC-062).
+
+## Data
+
+* **minfiDataEPIC** (Bioconductor, Artistic-2.0): 3 EPIC v1 arrays of NA12878 (IDATs + minfi-style sample sheet).
+* **Infinium MethylationEPIC v1.0 B5 manifest (CSV)**: 865 918 probes (142 158 Infinium I: 92 212 red, 49 946 green),
+  635 control beads. Read in 8.6 s on the Pi.
+
+## Delivered
+
+`simurg methyl qc`, `simurg methyl convert-manifest`, `simurg qc controls --array-type methylation`, methylation
+datasets in `simurg qc report`; modules `formats/methyl_manifest.py`, `signal/methyl.py`, `pipeline/methyl.py`;
+the 21 methylation metrics in `qc/controls.py`. User guide: "Methylation arrays".
+
+## Results on minfiDataEPIC
+
+| Check | Result |
+|---|---|
+| Run time, 3 samples, 3 workers (Pi) | 25 s (2.9 s signal processing per sample) |
+| 21 control metrics | all evaluable, all pass the `methylation_epic` profile (e.g. staining green 57, BS conversion I green 13, BS conversion II 5.3, specificity II 24) |
+| Probes passing detection (p < 0.05, analytical CG + SNP probes) | 90.9 %, 91.1 %, 91.3 % |
+| β of the three NA12878 replicates | Pearson r 0.997 – 0.9974 between replicates; bimodal distribution |
+| Sheet `Sample_Group` → PCA colours; outputs of MET-013 | present; folder loads in `qc report` |
+
+## Against the sesame oracle (sesame 1.28.1, same probe set via `probes.tsv`/`controls.tsv`)
+
+`scripts/v2/sesame_oracle.R` + `scripts/v2/methyl_oracle_compare.py`, bead-count masking off on both sides (sesame's
+`min_beads` masks every Infinium II probe because their M address is NA), 865 918 probes per sample:
+
+| Quantity | Agreement (3 samples) |
+|---|---|
+| β values | Pearson r 0.99998; \|Δβ\| median 0.0009, 99th percentile 0.008, max 0.011 |
+| noob-corrected M/U signals | relative difference median 0.5 %, max 5 % |
+| Detection calls at p < 0.05 | 99.0 – 99.1 % identical; passing fraction 91.0 – 91.4 % (simurg) vs 92.0 – 92.3 % (sesame) |
+| Detection p-values | not identical (median \|Δp\| 0.0008) |
+
+The remaining difference is systematic (same size in all three samples) and is not the negative controls: sesame did
+not attach the supplied control table (`negControls` failed, so its background is out-of-band Infinium I signal only),
+and simurg without negative controls is not closer (noob difference 0.56 % instead of 0.53 %). The exact cause is
+open (K-026). For QC use — pass/fail per sample, detection rate, β for a PCA — the agreement is sufficient; simurg's
+methylation numerics are documented as its own [OURS] implementation of the published methods (MET-010), not as a
+bit-exact port.
+
+## Spec deviations (K-026)
+
+* MET-002: the vendor EPIC manifest has a 5th field (`AVG`) on some `NEGATIVE` control lines; ≥ 4 fields are
+  accepted and the first four used. Fewer than 4 still gives `E_METH_CONTROLS_FORMAT`.
+* MET-010: the detection p-value follows sesame's pOOBAH (p = 1 − max(F(M), F(U)), each in its own channel) instead
+  of the ambiguous "1 − ECDF(M + U)", so an open-source oracle exists.
+* MET-004: `Sentrix_ID`/`Sentrix_Position` (minfi/sesame sheets) accepted as a third key-column pair.

docs/validation/V2_methylation_vs_sesame.json

@@ -0,0 +1,263 @@
+{
+ "200144450018_R04C01": {
+  "probes": 865918,
+  "masked_sesame": 0,
+  "masked_simurg": 0,
+  "combine_neg=True": {
+   "noob_fit": {
+    "green": {
+     "mu": 883.7232985218219,
+     "sigma": 533.736,
+     "alpha": 6615.19430861235
+    },
+    "red": {
+     "mu": 1475.9648221678526,
+     "sigma": 974.0681999999999,
+     "alpha": 7235.1788325587695
+    }
+   },
+   "pval_abs_diff": {
+    "median": 0.0007849171113696351,
+    "p99": 0.012655374352560589,
+    "max": 0.7810294625483152
+   },
+   "pval_identical_fraction": 2.7716250268501175e-05,
+   "detection_call_agreement_0.05": 0.9904228806884716,
+   "prop_passing": {
+    "simurg": 0.9104377127786546,
+    "sesame": 0.9199708917641769
+   },
+   "noob_m_rel_diff": {
+    "median": 0.005309072922507809,
+    "p99": 0.03942181326074589,
+    "max": 0.04794913709182962
+   },
+   "noob_u_rel_diff": {
+    "median": 0.005799391773898118,
+    "p99": 0.03459884716823513,
+    "max": 0.04794913709182962
+   },
+   "beta_abs_diff": {
+    "median": 0.0008897385559986926,
+    "p99": 0.008521359578262878,
+    "max": 0.011219185321570957
+   },
+   "beta_corr": 0.9999813374332573
+  },
+  "combine_neg=False": {
+   "noob_fit": {
+    "green": {
+     "mu": 885.3040134669028,
+     "sigma": 533.736,
+     "alpha": 6613.613593667269
+    },
+    "red": {
+     "mu": 1481.7310124080875,
+     "sigma": 975.5508,
+     "alpha": 7229.412642318534
+    }
+   },
+   "pval_abs_diff": {
+    "median": 0.0007957874497818184,
+    "p99": 0.013082543985806876,
+    "max": 0.7841804034247452
+   },
+   "pval_identical_fraction": 2.7716250268501175e-05,
+   "detection_call_agreement_0.05": 0.9901872925611894,
+   "prop_passing": {
+    "simurg": 0.9102024830066768,
+    "sesame": 0.9199708917641769
+   },
+   "noob_m_rel_diff": {
+    "median": 0.005602741250468986,
+    "p99": 0.038392170387569056,
+    "max": 0.047002982560608926
+   },
+   "noob_u_rel_diff": {
+    "median": 0.0065460727630588,
+    "p99": 0.03442105845709709,
+    "max": 0.047002982560608926
+   },
+   "beta_abs_diff": {
+    "median": 0.0009450303433635281,
+    "p99": 0.008636600261016411,
+    "max": 0.011550874711670933
+   },
+   "beta_corr": 0.9999799503820251
+  }
+ },
+ "200144450019_R07C01": {
+  "probes": 865918,
+  "masked_sesame": 0,
+  "masked_simurg": 0,
+  "combine_neg=True": {
+   "noob_fit": {
+    "green": {
+     "mu": 1004.7491370858492,
+     "sigma": 606.3833999999999,
+     "alpha": 7726.238894681398
+    },
+    "red": {
+     "mu": 1645.4509315964758,
+     "sigma": 1097.124,
+     "alpha": 8498.773085184836
+    }
+   },
+   "pval_abs_diff": {
+    "median": 0.000796427075933526,
+    "p99": 0.012417625677686978,
+    "max": 0.8027931613655367
+   },
+   "pval_identical_fraction": 5.65873442981899e-05,
+   "detection_call_agreement_0.05": 0.9906884947535448,
+   "prop_passing": {
+    "simurg": 0.912474825779329,
+    "sesame": 0.9217102015559928
+   },
+   "noob_m_rel_diff": {
+    "median": 0.005314312414259365,
+    "p99": 0.04273539399348046,
+    "max": 0.051907861845576456
+   },
+   "noob_u_rel_diff": {
+    "median": 0.005468488622733155,
+    "p99": 0.03355605525549436,
+    "max": 0.051907861845576456
+   },
+   "beta_abs_diff": {
+    "median": 0.0008606416392141769,
+    "p99": 0.008394710099924632,
+    "max": 0.010860663350041877
+   },
+   "beta_corr": 0.9999812358896786
+  },
+  "combine_neg=False": {
+   "noob_fit": {
+    "green": {
+     "mu": 1006.5499088107555,
+     "sigma": 606.3833999999999,
+     "alpha": 7724.4381229564915
+    },
+    "red": {
+     "mu": 1651.850010672488,
+     "sigma": 1098.6065999999998,
+     "alpha": 8492.374006108823
+    }
+   },
+   "pval_abs_diff": {
+    "median": 0.0008019741538315371,
+    "p99": 0.012866482361703901,
+    "max": 0.8059396632834097
+   },
+   "pval_identical_fraction": 5.65873442981899e-05,
+   "detection_call_agreement_0.05": 0.9904552163137849,
+   "prop_passing": {
+    "simurg": 0.9122453898440994,
+    "sesame": 0.9217102015559928
+   },
+   "noob_m_rel_diff": {
+    "median": 0.005610084745351353,
+    "p99": 0.04163859014373106,
+    "max": 0.05094825885592487
+   },
+   "noob_u_rel_diff": {
+    "median": 0.006166215991548127,
+    "p99": 0.033209584923356225,
+    "max": 0.05094825885592487
+   },
+   "beta_abs_diff": {
+    "median": 0.0009157086256231864,
+    "p99": 0.008483978716612176,
+    "max": 0.011139138206961108
+   },
+   "beta_corr": 0.999980025075608
+  }
+ },
+ "200144450021_R05C01": {
+  "probes": 865918,
+  "masked_sesame": 0,
+  "masked_simurg": 0,
+  "combine_neg=True": {
+   "noob_fit": {
+    "green": {
+     "mu": 1075.2185580103821,
+     "sigma": 618.2442,
+     "alpha": 7644.388814794943
+    },
+    "red": {
+     "mu": 1583.358547469605,
+     "sigma": 1043.7504,
+     "alpha": 7872.027822306209
+    }
+   },
+   "pval_abs_diff": {
+    "median": 0.0008097160213839151,
+    "p99": 0.012557480308668073,
+    "max": 0.7179490998567224
+   },
+   "pval_identical_fraction": 8.083906328312843e-05,
+   "detection_call_agreement_0.05": 0.9906931141285895,
+   "prop_passing": {
+    "simurg": 0.9142593274977809,
+    "sesame": 0.9234993383438433
+   },
+   "noob_m_rel_diff": {
+    "median": 0.005312767934935867,
+    "p99": 0.0416027184427129,
+    "max": 0.05163890421954045
+   },
+   "noob_u_rel_diff": {
+    "median": 0.005627520758233918,
+    "p99": 0.033403350351144646,
+    "max": 0.05163890421954045
+   },
+   "beta_abs_diff": {
+    "median": 0.000886446134372592,
+    "p99": 0.008067365546443757,
+    "max": 0.01057770765422894
+   },
+   "beta_corr": 0.9999820250823297
+  },
+  "combine_neg=False": {
+   "noob_fit": {
+    "green": {
+     "mu": 1077.1430682973066,
+     "sigma": 618.2442,
+     "alpha": 7642.464304508018
+    },
+    "red": {
+     "mu": 1589.590396661226,
+     "sigma": 1045.233,
+     "alpha": 7865.7959731145875
+    }
+   },
+   "pval_abs_diff": {
+    "median": 0.0008198122345790535,
+    "p99": 0.013020971906371236,
+    "max": 0.7195549302039842
+   },
+   "pval_identical_fraction": 8.083906328312843e-05,
+   "detection_call_agreement_0.05": 0.9905060294392771,
+   "prop_passing": {
+    "simurg": 0.9140716071871386,
+    "sesame": 0.9234993383438433
+   },
+   "noob_m_rel_diff": {
+    "median": 0.005623618294636237,
+    "p99": 0.04045226512957654,
+    "max": 0.05065151257916101
+   },
+   "noob_u_rel_diff": {
+    "median": 0.006360115250782237,
+    "p99": 0.03307310255358611,
+    "max": 0.05065151257916101
+   },
+   "beta_abs_diff": {
+    "median": 0.0009431797599571898,
+    "p99": 0.008174273334787542,
+    "max": 0.010885211471086453
+   },
+   "beta_corr": 0.9999807797628828
+  }
+ }
+}

docs/validation/V2_pgx_cn.md

@@ -0,0 +1,61 @@
+# V2 — Targeted PGx copy number (PCN-001 … PCN-017)
+
+Status: **implemented, interim validation** (2026-09-29). `simurg pgx copy-number train|call` (CLI-031/032), model
+file `.simurg-pgxcn` (IN-050), region BED v1 built from UCSC ncbiRefSeq (`scripts/pgx/build_cn_regions.py`,
+packaged as `simurg/pgx/data/pgx_cn_regions.GRCh38.v1.bed`), per-sample `*.pgx_cn.vcf.gz` + `.tbi`,
+`pgx_cn_batch_summary.tsv`, `pgx_cn_sample_summary.tsv`. Unit tests: `tests/unit/test_pgx_cn.py`.
+
+## Data
+
+* **Arrays:** the GSA v4 PGx demo (`idats_Demo_36`, 36 arrays), genotyped by SimurgArray. Its CNV probes cover every
+  region (51–357 probes; GSTT1: 160 probes on `chr22_KI270879v1_alt` in the GRCh38 manifest).
+* **Identities:** the demo arrays are 30 CEPH / 1000 Genomes individuals (6 replicate arrays). Identified by
+  genotype concordance with the 3202-sample 1000G 30x panel on 4647 chr22 SNPs: best match 0.992–0.996, next best
+  ≤ 0.91 (relatives) — `scripts/pgx/demo36_identity.tsv`.
+* **Truth:** NYGC 30x SV callset (1KGP_3202 GATK-SV/svtools freeze V3), RD_CN per individual
+  (`scripts/pgx/validate_cn_truth.py`).
+* The GSE165845 GSA-MD v1 arrays (360 arrays, CN/LOH validation) have 0–9 probes per PGx region: not usable here.
+
+## Result (30 individuals, model trained on the same 36 arrays with `--min-samples 30`)
+
+| Region | Concordance with NYGC RD_CN | Mismatches (truth → call) |
+|---|---|---|
+| GSTM1 (whole gene) | **30/30** | – |
+| UGT2B17 (whole gene) | **30/30** | – |
+| SULT1A1 | 29/30 | 2→1 |
+| CYP2A6.exon.1 | 29/30 | 3→2 |
+| CYP2A6.intron.7 | 28/30 | 3→2, 1→2 |
+| CYP2D6.p5 | 28/30 | 4→3, 2→1 |
+| CYP2D6.intron.2 | 25/30 | 3→2 ×3, 4→3, 2→1 |
+| CYP2D6.exon.9 | 25/30 | 3→2 ×3, 4→2, 2→1 |
+| **all** | **224/240 (93.3 %)** | |
+
+GSTT1 (no SV truth on the alt contig): CN 0/1/2 = 5/16/9, in Hardy–Weinberg proportions (null allele 0.43; CEU
+literature ≈ 0.4–0.5). Details: `V2_pgx_cn_demo36.json`.
+
+* CYP2D6 truth is one gene-level record; the exon-9 / intron-2 "misses" at truth CN 3 are expected for tandem
+  structures such as `*68+*4` or `*36+*10`, where not every sub-region is duplicated. Star-allele truth (GeT-RM)
+  is the right yardstick and follows with the star-allele caller.
+* PCN-017 target (≥ 95 % per region) is met for GSTM1, UGT2B17, SULT1A1, CYP2A6.exon.1; not (yet) shown for the
+  CYP2D6 sub-regions.
+
+## Additions
+
+* PCN-006: a warning when the batch's IDAT scan dates span more than 14 days (no sample-sheet input in `pgx
+  copy-number call`, so the lot/batch-column check does not apply).
+* PCN-010: `pgx_cn_baf_evidence.tsv` — per region the log-likelihood of CN 0 … 5 from the informative SNP BAFs
+  (binomial weights from the training cohort's B-allele frequency); `--bafs-in-cn` adds it to the CN posterior
+  (experimental, off by default). On the demo it agrees poorly with the LRR calls (e.g. GSTM1 2/36): the SNPs in
+  these regions do not separate the CN states here, so it stays informational.
+* PCN-009 / EDGE-026: training excludes samples whose genome-wide level **or** noise is an outlier (robust |z| > 5);
+  `test_outlier_robustness` injects 10 % noisy, shifted arrays and requires ≥ 99 % unchanged calls for the others.
+* CLI-032: the platform (LCG/EX) comes from the CSV manifest's `Assay Format` (`--csv`), or `--platform`; the demo
+  model records `EX`.
+
+## Caveats (K-028)
+
+* Trained and evaluated on the same 30 individuals (the spec requires ≥ 96 training samples; the priors were
+  chosen while looking at this truth). This is an optimistic, in-sample figure until an independent cohort is
+  available.
+* Deviations from PCN-009: reference from the training cohort (not the calling batch); no GC-correction step; CN
+  model = compressed-intensity template + HWE-constrained weights (`simurg/pgx/cn.py`).

docs/validation/V2_pgx_cn_demo36.json

@@ -0,0 +1,89 @@
+{
+ "individuals": 30,
+ "regions": {
+  "GSTM1": {
+   "record": "HGSV_9691",
+   "concordant": 30,
+   "n": 30,
+   "rate": 1.0,
+   "truth_to_call_mismatches": {}
+  },
+  "UGT2B17": {
+   "record": "HGSV_63272",
+   "concordant": 30,
+   "n": 30,
+   "rate": 1.0,
+   "truth_to_call_mismatches": {}
+  },
+  "SULT1A1": {
+   "record": "HGSV_207805",
+   "concordant": 29,
+   "n": 30,
+   "rate": 0.9667,
+   "truth_to_call_mismatches": {
+    "2->1": 1
+   }
+  },
+  "CYP2A6.intron.7": {
+   "record": "HGSV_232737",
+   "concordant": 28,
+   "n": 30,
+   "rate": 0.9333,
+   "truth_to_call_mismatches": {
+    "3->2": 1,
+    "1->2": 1
+   }
+  },
+  "CYP2A6.exon.1": {
+   "record": "HGSV_232737",
+   "concordant": 29,
+   "n": 30,
+   "rate": 0.9667,
+   "truth_to_call_mismatches": {
+    "3->2": 1
+   }
+  },
+  "CYP2D6.exon.9": {
+   "record": "HGSV_247317",
+   "concordant": 25,
+   "n": 30,
+   "rate": 0.8333,
+   "truth_to_call_mismatches": {
+    "4->2": 1,
+    "3->2": 3,
+    "2->1": 1
+   }
+  },
+  "CYP2D6.intron.2": {
+   "record": "HGSV_247317",
+   "concordant": 25,
+   "n": 30,
+   "rate": 0.8333,
+   "truth_to_call_mismatches": {
+    "4->3": 1,
+    "3->2": 3,
+    "2->1": 1
+   }
+  },
+  "CYP2D6.p5": {
+   "record": "HGSV_247317",
+   "concordant": 28,
+   "n": 30,
+   "rate": 0.9333,
+   "truth_to_call_mismatches": {
+    "4->3": 1,
+    "2->1": 1
+   }
+  }
+ },
+ "overall": {
+  "concordant": 224,
+  "n": 240,
+  "rate": 0.9333
+ },
+ "gstt1_cn_distribution": {
+  "0": 5,
+  "1": 16,
+  "2": 9
+ }
+}

docs/validation/V2_pgx_star.md

@@ -0,0 +1,154 @@
+# V2 — PGx DB and star-allele calling (PGX-001 … PGX-021)
+
+Status: **implemented for CPIC star-allele genes, interim validation** (2026-09-29).
+
+* `scripts/pgx/fetch_sources.sh` → CPIC API tables (CC0, data version 2026-08-03).
+* `simurg pgx db build` → deterministic SQLite: 22 genes, 1354 alleles, 1897 normalised variant rows; GSA v4 PGx
+  probe map 5002 links; two builds byte-identical.
+* `simurg pgx star-allele call` → `diplotype_summary.tsv`, `pgx_calls.json` (11 star-allele genes).
+* `simurg pgx phenotype` (alias `annotate`) → per-sample `<sample>.pgx.json` (`simurg.pgx/1`) and
+  `diplotype_annotations.tsv` with `#` metadata lines (PGX-024 … PGX-028); CPIC activity scores (xN = N × activity,
+  *5/*0 = 0) and function-count lookups; disagreeing feasible solutions → `Indeterminate` plus per-solution
+  phenotypes. `tests/unit/test_pgx_phenotype.py`.
+* Unit tests: `tests/unit/test_pgx_db.py`, `tests/unit/test_pgx_star.py`.
+
+## Data and truth
+
+* Arrays: the GSA v4 PGx demo (36 arrays = 30 CEPH/1000G individuals, `scripts/pgx/demo36_identity.tsv`), SNV VCFs
+  and PGx CN VCFs from SimurgArray.
+* Star-allele truth: GeT-RM consolidated PGx/HLA table (CDC, 2025 V4) — 16 of the 30 individuals
+  (`scripts/pgx/getrm_truth.py`).
+* Probe mask (PGX-010): array vs 1000G 30x (NYGC phased panel, fetched per gene over HTTPS) for the **other 14
+  individuals** (`scripts/pgx/derive_masks.py`), so the mask is independent of the GeT-RM validation set: 8 of 658
+  assessed records (23 probe names) below 95 % concordance — e.g. CYP2C19 rs367543002 and CYP2D6 rs1135836 called
+  homozygous ALT in every sample, the UGT1A1 `C>CATAT` TA-repeat probe. Regenerate with the command in the script
+  header (the mask lists vendor probe names and is not committed).
+
+## Result (16 individuals; replicate arrays counted once)
+
+| Gene | Top solution = GeT-RM | Truth among array-equivalent feasible solutions |
+|---|---|---|
+| CYP2C9 | **15/15** | 15/15 |
+| CYP2C19 | **14/15** | 15/15 |
+| CYP2D6 | **13/15** | **15/15** |
+| CYP3A5 | 15/15 | 15/15 |
+| TPMT | 15/16 | 15/16 |
+| CYP2B6 | 12/14 | 12/14 |
+| CYP4F2 | 8/9 | 8/9 |
+| NAT2 | 8/9 | 8/9 |
+| NUDT15 | 1/1 | 1/1 |
+| VKORC1 (rs9923231, c.-1639G>A) | 13/15 | 13/15 |
+| GSTM1 (CN: DEL/DEL vs present) | 9/9 | 9/9 |
+| GSTT1 (CN 0/1/2) | 7/9 | 7/9 |
+| UGT2B17 (CN) | 2/2 | 2/2 |
+| CYP3A4 (PharmVar) | **15/15** | 15/15 |
+| CYP2A6 (PharmVar; *4 deletion from CN) | 7/9 | 8/9 |
+| **total** | **154/168 (91.7 %)** | **158/168 (94.0 %)** |
+
+PGX-032 target (≥ 90 % for CYP2D6/CYP2C19/CYP2C9 against truth): **42/45 = 93 %** (top solution), 45/45 within
+the array-equivalent feasible solutions.
+
+CYP2D6 structures (PGX-013) come from the three sub-region CNs (`simurg.pgx.star.cyp2d6_structure`): NA12878
+`*3/*68+*4` and NA11832 `*1/*68+*4` are called exactly as GeT-RM reports them; one sample is called `*36+*10/*4`
+(p5 = intron 2 = 3, exon 9 = 2; no GeT-RM truth). The two remaining top-solution misses are *1 vs *2, which this
+array cannot separate (both within the feasible set).
+Details and every discordant call: `V2_pgx_star_getrm16.json`.
+
+Not in the total (GeT-RM uses older names or definitions than CPIC/PharmVar today): UGT1A1 (GeT-RM `*60`;
+0/9 top), SLCO1B1 (`*1B/*17/*21` vs CPIC `*37/*15/*20`; 9/15), CYP1A2 (lettered `*1F`; 0/9) and CYP2C8 (PharmVar now
+defines rs2071426 alone as `*19`; the array calls it concordantly with 1000G, GeT-RM reports those carriers as
+`*1`; 8/15).
+
+Remaining discordances: CYP2D6 *2 vs *1 where the array cannot separate them (prior picks *1), CYP2C19 *8, TPMT *21, CYP2B6 *27, CYP4F2 *2 — rare
+alleles whose defining variants are absent or masked on this array.
+
+## Error model (K-029)
+
+After masking, 0 of 9 740 array calls at DB variants disagreed with 1000G 30x in any GenCall-score bin (0.15–1.0),
+so the per-call error is 1e-3 (GS ≥ 0.15) and errors are handled by the probe mask. The first model,
+ε = 0.5·(1 − GS)², let correct low-GS calls lose to the prior (CYP2C9 7/15 → 15/15 after the change).
+
+## Caveats
+
+* 16 individuals, one array type, European (CEPH) only; masks from 14 individuals (spec: ≥ 100 samples).
+* CPIC now defines the GRCh38 reference as its own allele where it differs from *1 (CYP2C19 *38): a sample
+  heterozygous at the *1-defining rs3758581 fits *1/*38 best (raw score 0.98), but *38 has no CPIC frequency, so the
+  prior keeps *1/*1 on top — which is what legacy GeT-RM reports.
+* `simurg pgx coverage` (PGX-009/035) on GSA v4 PGx with the mask: 438 alleles callable, 68 array-equivalent to
+  another allele, 88 not covered, 4 masked. Against the reference's known limitations: CYP4F2 *2 not covered (reported
+  as *1) and *4 ≡ *3 — as the reference; UGT1A1 *28 callable (its C>CAT probe matches 1000G) and *37 masked (the
+  C>CATAT probe fails); CYP2D6 *40/*58 and NUDT15 *6/*9 are separable here (CPIC core definitions differ from the
+  reference's).
+* Detail outputs (PGX-016 … PGX-020, PGX-027/028, PGX-031): `supporting_variants.tsv`, `unresolved_variants.tsv`,
+  `variants_not_covered.tsv`, per-allele supporting/unresolved sites for **both** alleles of every candidate,
+  `variant_change` vs the top solution (`distant_alternative` above 3 changes), `unexplained_variants`, full HGVS on
+  GRCh38 RefSeq accessions.
+* All CPIC gene kinds are called: star alleles, G6PD (named haplotypes; males hemizygous from the VCF ploidy),
+  variant-based genes (ABCG2, CACNA1S, CFTR, DPYD, IFNL3, MT-RNR1, RYR1, VKORC1; genotype as `rs… REF/ALT` or
+  `Reference`, phenotype diplotype from the two lowest-activity carried variants), HLA-A/B tag variants (PGX-007,
+  positions resolved from the manifest's probes, flag `tag_caveat`), and GSTM1/GSTT1/UGT2B17 `*0` genotypes from
+  the targeted CN (PGX-006).
+* PharmVar genes without CPIC definitions (CYP1A2, CYP2A6, CYP2A13, CYP2C8, CYP3A4; PharmVar 6.2.29): core alleles
+  from the GRCh38 haplotype VCFs, `*1` = reference, prior reference 0.8 (no population frequencies); genotype only
+  (no CPIC phenotype). The probe mask was re-derived with these variants (one more record: CYP2A6 19:40843845 C>A,
+  concordance 0.47, which had produced false *5 calls).
+* ClinPGx clinical-annotation variants (CC BY-SA 4.0; levels 1A–2B, and level 3 for the PGX-008 parity genes) add
+  genotype-only variant-based genes — OPRM1, COMT, DRD2, ANKK1, MTHFR, F5, … (43 genes, 171 of 187 rsIDs resolved on
+  GSA v4 PGx from the manifest's probes). The DB now holds 70 genes (16 star-allele, G6PD, 2 HLA tag, 51
+  variant-based; spec parity list: 58). `pgx coverage`: 1225 alleles callable, 93 array-equivalent, 437 not
+  covered, 5 masked.
+* `--guideline dpwg` (PGX-025): ClinPGx curates DPWG guidelines with the CPIC allele-function and phenotype mappings
+  for genes CPIC covers, so those genes reuse the CPIC tables (flag `dpwg_via_cpic_mapping`); DPWG-only genes
+  (CYP1A2, CYP3A4, F5) have no public machine-readable mapping and get phenotype `null` with the flag
+  `dpwg_mapping_not_available` (K-029).
+
+## Independent validation: GSE165845 (GSA v1 C2, 120 1000 Genomes individuals × 3 arrays)
+
+A second cohort, independent of the demo used for development (2026-09-29, all on the Pi, $0):
+
+* **Arrays:** the 360 GSE165845 GTCs (GSA-MD v1 chips, GSA v1 C2 manifest; K-024) → SNV VCFs; no targeted CN probes
+  on this chip, so star alleles are called with `--no-cn`. The chip carries little PGx content: 292 probe-to-variant
+  links (GSA v4 PGx: 5 280); `pgx coverage`: 495 alleles callable, 136 array-equivalent, 1 129 not covered — e.g.
+  no CYP2D6 *2/*4/*10 probes (rs16947, rs3892097, rs1065852 are absent from the manifest).
+* **Truth:** star alleles derived from the 1000G 30x phased panel with the same DB definitions
+  (`scripts/pgx/wgs_star_truth.py`; SNV/indel level — CYP2D6 *5, xN and hybrids invisible; CYP2B6 left out because
+  its common core variant rs2279343 is filtered from the panel). The derivation was checked against GeT-RM on the
+  demo's CEPH individuals (CYP2D6 14/15, CYP2C9/CYP3A4/CYP3A5 15/15, TPMT 16/16; the rest are nomenclature changes).
+* **Masks:** 2-fold — individuals split in halves; each half is called with the mask derived from the other half
+  (A → 0 probes masked, B → 1). After masking the calls agree with 1000G at 99 % or better in every GS bin.
+* **Metrics:** *top* = the top solution equals the WGS diplotype; *array-resolvable* = both diplotypes projected
+  onto the alleles this array can tell apart (`pgx coverage`: callable → itself, equivalent_to:X → X, not covered →
+  the reference's class) — the fair measure for a chip that lacks a gene's defining variants.
+
+| Gene | Top = WGS | Truth within feasible equivalents | Array-resolvable |
+|---|---|---|---|
+| CYP1A2 | 18/120 | 72/120 | 120/120 |
+| CYP2A13 | 97/120 | 119/120 | 117/120 |
+| CYP2A6 | 73/120 | 112/120 | 117/120 |
+| CYP2C19 | 100/120 | 116/120 | 110/120 |
+| CYP2C8 | 114/120 | 120/120 | 115/120 |
+| CYP2C9 | 95/120 | 108/120 | 116/120 |
+| CYP2D6 (SNV level) | 32/120 | 54/120 | 111/120 |
+| CYP3A4 | 112/120 | 117/120 | 117/120 |
+| CYP3A5 | 120/120 | 120/120 | 120/120 |
+| CYP4F2 | 43/120 | 86/120 | 96/120 |
+| NAT2 | 95/120 | 105/120 | 120/120 |
+| NUDT15 | 120/120 | 120/120 | 120/120 |
+| SLCO1B1 | 31/120 | 87/120 | 109/120 |
+| TPMT | 107/120 | 115/120 | 102/103 |
+| UGT1A1 | 66/120 | 120/120 | 47/47 |
+| **total** | **1223/1800 (67.9 %)** | | **1637/1710 (95.7 %)** |
+
+* Top-solution concordance is low where the chip lacks the defining variants: the call is then the most probable
+  member of an equivalence class the array cannot split (CYP2D6 *1/*2/*10, SLCO1B1 *1/*37, CYP1A2), so the prior
+  decides. Projected onto what the array can see, 95.7 % agree (half A 815/854, half B 822/856).
+* CYP4F2 (96/120) is the weakest: the array genotypes at its covered sites match 1000G exactly (rs3093200 180/180),
+  so the residue is definitional — *5 and *23 share the only covered variant, and the projection cannot separate
+  alleles whose other defining variants are off-array.
+* **GeT-RM** (7 of the 120 individuals, older consensus): 28/35 top, array-resolvable 29/30.
+* **Replicates:** the top genotype of all 70 genes agrees between replicate arrays of one individual in 24 916 of
+  25 200 pairs (98.9 %); 87 of the 284 discordant pairs come from NA20362's failed replicate (call rate 0.76, K-024);
+  without it 99.2 %.
+
+Files: `docs/validation/V2_pgx_star_gse165845.json`; scripts `wgs_star_truth.py`, `validate_star_getrm.py`
+(`--all-genes --individuals --coverage`), `replicate_concordance.py`.

docs/validation/V2_pgx_star_getrm16.json

@@ -0,0 +1,429 @@
+{
+ "individuals": 30,
+ "genes": {
+  "CYP1A2": {
+   "n": 9,
+   "top": 0,
+   "equivalent": 0
+  },
+  "CYP2A6": {
+   "n": 9,
+   "top": 7,
+   "equivalent": 8
+  },
+  "CYP2B6": {
+   "n": 14,
+   "top": 12,
+   "equivalent": 12,
+   "not_comparable": 1
+  },
+  "CYP2C19": {
+   "n": 15,
+   "top": 14,
+   "equivalent": 15
+  },
+  "CYP2C8": {
+   "n": 15,
+   "top": 8,
+   "equivalent": 9
+  },
+  "CYP2C9": {
+   "n": 15,
+   "top": 15,
+   "equivalent": 15
+  },
+  "CYP2D6": {
+   "n": 15,
+   "top": 13,
+   "equivalent": 15
+  },
+  "CYP3A4": {
+   "n": 15,
+   "top": 15,
+   "equivalent": 15
+  },
+  "CYP3A5": {
+   "n": 15,
+   "top": 15,
+   "equivalent": 15
+  },
+  "CYP4F2": {
+   "n": 9,
+   "top": 8,
+   "equivalent": 8
+  },
+  "GSTM1": {
+   "n": 9,
+   "top": 9,
+   "equivalent": 9
+  },
+  "GSTT1": {
+   "n": 9,
+   "top": 7,
+   "equivalent": 7
+  },
+  "NAT2": {
+   "n": 9,
+   "top": 8,
+   "equivalent": 8
+  },
+  "NUDT15": {
+   "n": 1,
+   "top": 1,
+   "equivalent": 1
+  },
+  "SLCO1B1": {
+   "n": 15,
+   "top": 9,
+   "equivalent": 9
+  },
+  "TPMT": {
+   "n": 16,
+   "top": 15,
+   "equivalent": 15
+  },
+  "UGT1A1": {
+   "n": 9,
+   "top": 0,
+   "equivalent": 1
+  },
+  "UGT2B17": {
+   "n": 2,
+   "top": 2,
+   "equivalent": 2
+  },
+  "VKORC1": {
+   "n": 15,
+   "top": 13,
+   "equivalent": 13
+  }
+ },
+ "legacy_nomenclature_genes": [
+  "CYP1A2",
+  "CYP2C8",
+  "SLCO1B1",
+  "UGT1A1"
+ ],
+ "total": {
+  "n": 168,
+  "top": 154,
+  "equivalent": 158
+ },
+ "discordant": [
+  {
+   "individual": "NA12156",
+   "gene": "CYP1A2",
+   "truth": "*1A/*1F",
+   "call": "*1/*15",
+   "within_equivalents": false
+  },
+  {
+   "individual": "NA12156",
+   "gene": "CYP2C8",
+   "truth": "*1/*15",
+   "call": "*1/*1",
+   "within_equivalents": true
+  },
+  {
+   "individual": "NA12156",
+   "gene": "CYP4F2",
+   "truth": "(*2)/*3",
+   "call": "*1/*3",
+   "within_equivalents": false
+  },
+  {
+   "individual": "NA12156",
+   "gene": "SLCO1B1",
+   "truth": "*1/*21",
+   "call": "*1/*37",
+   "within_equivalents": false
+  },
+  {
+   "individual": "NA12156",
+   "gene": "UGT1A1",
+   "truth": "*60/*28",
+   "call": "*28/*80",
+   "within_equivalents": false
+  },
+  {
+   "individual": "NA12044",
+   "gene": "TPMT",
+   "truth": "*1/*21",
+   "call": "*1/*1",
+   "within_equivalents": false
+  },
+  {
+   "individual": "NA07019",
+   "gene": "CYP1A2",
+   "truth": "*1A/*1F",
+   "call": "*1/*15",
+   "within_equivalents": false
+  },
+  {
+   "individual": "NA07019",
+   "gene": "CYP2C8",
+   "truth": "*1/*1",
+   "call": "*1/*19",
+   "within_equivalents": false
+  },
+  {
+   "individual": "NA07019",
+   "gene": "UGT1A1",
+   "truth": "*1/*60",
+   "call": "*1/*1",
+   "within_equivalents": false
+  },
+  {
+   "individual": "NA06991",
+   "gene": "CYP1A2",
+   "truth": "*1F/*1F",
+   "call": "*15/*24",
+   "within_equivalents": false
+  },
+  {
+   "individual": "NA06991",
+   "gene": "UGT1A1",
+   "truth": "*60/*60",
+   "call": "*1/*1",
+   "within_equivalents": false
+  },
+  {
+   "individual": "NA06991",
+   "gene": "VKORC1",
+   "truth": "AA",
+   "call": "rs9923231 T/T",
+   "within_equivalents": false
+  },
+  {
+   "individual": "NA12878",
+   "gene": "CYP1A2",
+   "truth": "*1F/*1F",
+   "call": "*15/*24",
+   "within_equivalents": false
+  },
+  {
+   "individual": "NA12878",
+   "gene": "UGT1A1",
+   "truth": "*60/*28",
+   "call": "*28/*80",
+   "within_equivalents": false
+  },
+  {
+   "individual": "NA11993",
+   "gene": "CYP1A2",
+   "truth": "*1F/*1F",
+   "call": "*15/*24",
+   "within_equivalents": false
+  },
+  {
+   "individual": "NA11993",
+   "gene": "CYP2A6",
+   "truth": "*9/*17",
+   "call": "*15/*17",
+   "within_equivalents": true
+  },
+  {
+   "individual": "NA11993",
+   "gene": "SLCO1B1",
+   "truth": "*1/*17",
+   "call": "*1/*15",
+   "within_equivalents": false
+  },
+  {
+   "individual": "NA11993",
+   "gene": "UGT1A1",
+   "truth": "(*28 + *60)/*60",
+   "call": "*28/*80",
+   "within_equivalents": false
+  },
+  {
+   "individual": "NA12003",
+   "gene": "CYP1A2",
+   "truth": "*1A/*1F",
+   "call": "*15/*24",
+   "within_equivalents": false
+  },
+  {
+   "individual": "NA12003",
+   "gene": "CYP2A6",
+   "truth": "*1/(*8)",
+   "call": "*1/*1",
+   "within_equivalents": false
+  },
+  {
+   "individual": "NA12003",
+   "gene": "UGT1A1",
+   "truth": "*60/*28",
+   "call": "*28/*80",
+   "within_equivalents": false
+  },
+  {
+   "individual": "NA12003",
+   "gene": "VKORC1",
+   "truth": "AA",
+   "call": "rs9923231 T/T",
+   "within_equivalents": false
+  },
+  {
+   "individual": "NA12003",
+   "gene": "GSTT1",
+   "truth": "(*A/*A)",
+   "call": "*1/*0",
+   "within_equivalents": false
+  },
+  {
+   "individual": "NA07029",
+   "gene": "CYP1A2",
+   "truth": "*1A/*1F",
+   "call": "*1/*15",
+   "within_equivalents": false
+  },
+  {
+   "individual": "NA07029",
+   "gene": "CYP2B6",
+   "truth": "*6/(*27)",
+   "call": "*1/*6",
+   "within_equivalents": false
+  },
+  {
+   "individual": "NA07029",
+   "gene": "CYP2C19",
+   "truth": "*8/*17",
+   "call": "*1/*17",
+   "within_equivalents": true
+  },
+  {
+   "individual": "NA07029",
+   "gene": "CYP2C8",
+   "truth": "*1/*3",
+   "call": "*19/*3",
+   "within_equivalents": false
+  },
+  {
+   "individual": "NA07029",
+   "gene": "UGT1A1",
+   "truth": "*1/*1",
+   "call": "*1/*28",
+   "within_equivalents": true
+  },
+  {
+   "individual": "NA12892",
+   "gene": "CYP2B6",
+   "truth": "*1/*1",
+   "call": "*1/*5",
+   "within_equivalents": false
+  },
+  {
+   "individual": "NA10846",
+   "gene": "CYP2C8",
+   "truth": "*1/*1",
+   "call": "*1/*19",
+   "within_equivalents": false
+  },
+  {
+   "individual": "NA07000",
+   "gene": "CYP2C8",
+   "truth": "*1/*1",
+   "call": "*1/*19",
+   "within_equivalents": false
+  },
+  {
+   "individual": "NA11832",
+   "gene": "CYP1A2",
+   "truth": "*1F/*1F",
+   "call": "*15/*24",
+   "within_equivalents": false
+  },
+  {
+   "individual": "NA11832",
+   "gene": "NAT2",
+   "truth": "*5/(*12)",
+   "call": "*1/*5",
+   "within_equivalents": false
+  },
+  {
+   "individual": "NA11832",
+   "gene": "SLCO1B1",
+   "truth": "*1/*1",
+   "call": "*14/*20",
+   "within_equivalents": false
+  },
+  {
+   "individual": "NA11832",
+   "gene": "UGT1A1",
+   "truth": "(*28 + *60)/*60",
+   "call": "*28/*80",
+   "within_equivalents": false
+  },
+  {
+   "individual": "NA10838",
+   "gene": "CYP2D6",
+   "truth": "*2/*4",
+   "call": "*1/*4;*2/*4",
+   "within_equivalents": true
+  },
+  {
+   "individual": "NA10838",
+   "gene": "SLCO1B1",
+   "truth": "*1/*1",
+   "call": "*1/*14",
+   "within_equivalents": false
+  },
+  {
+   "individual": "NA12813",
+   "gene": "CYP1A2",
+   "truth": "*1F/*1L",
+   "call": "*15/*24",
+   "within_equivalents": false
+  },
+  {
+   "individual": "NA12813",
+   "gene": "CYP2C8",
+   "truth": "*1/*1",
+   "call": "*1/*19",
+   "within_equivalents": false
+  },
+  {
+   "individual": "NA12813",
+   "gene": "CYP2D6",
+   "truth": "*2/*4",
+   "call": "*1/*4;*2/*4",
+   "within_equivalents": true
+  },
+  {
+   "individual": "NA12813",
+   "gene": "SLCO1B1",
+   "truth": "*1/*21",
+   "call": "*1/*20",
+   "within_equivalents": false
+  },
+  {
+   "individual": "NA12813",
+   "gene": "UGT1A1",
+   "truth": "(*28 + *60)/(*28 + *60)",
+   "call": "*80/*80",
+   "within_equivalents": false
+  },
+  {
+   "individual": "NA12813",
+   "gene": "GSTT1",
+   "truth": "(*A/*A)",
+   "call": "*1/*0",
+   "within_equivalents": false
+  },
+  {
+   "individual": "NA11881",
+   "gene": "CYP2C8",
+   "truth": "*1/*1",
+   "call": "*1/*19",
+   "within_equivalents": false
+  },
+  {
+   "individual": "NA11881",
+   "gene": "SLCO1B1",
+   "truth": "*1/(*15)",
+   "call": "*14/*15",
+   "within_equivalents": false
+  }
+ ]
+}

examples/gsa_demo.sh

@@ -30,7 +30,7 @@ mkdir -p "$OUT"
 # Optional subset: symlink the first N IDAT pairs into $OUT/idats (same <barcode>/<barcode>_<pos>_<ch>.idat layout).
 if [[ $N -gt 0 ]]; then
   rm -rf "$OUT/idats"
-  find "$IDATS" -name '*_Grn.idat' | sort | head -n "$N" | while read -r g; do
+  find "$IDATS" -name '*_Grn.idat' | sort | sed -n "1,${N}p" | while read -r g; do
     b=$(basename "$(dirname "$g")"); mkdir -p "$OUT/idats/$b"
     ln -s "$(realpath "$g")" "$OUT/idats/$b/"; ln -s "$(realpath "${g%_Grn.idat}_Red.idat")" "$OUT/idats/$b/"
   done
@@ -38,12 +38,12 @@ if [[ $N -gt 0 ]]; then
 fi
 
 echo "== 1. inspect the inputs"
-$SIMURG inspect "$BPM" "$EGT" | head -20
+$SIMURG inspect "$BPM" "$EGT" | sed -n 1,20p   # sed reads to the end: no SIGPIPE under pipefail
 
 echo "== 2. genotype: GTC v5 per sample, sample_summary.tsv, control-probe QC, contamination, relatedness"
 $SIMURG genotype --bpm "$BPM" --egt "$EGT" --csv "$CSV" --idats "$IDATS" --out "$OUT/gtc" \
   --qc-profile genotyping --threads "$THREADS" --force
-column -t -s$'\t' "$OUT/gtc/sample_summary.tsv" | cut -c1-150 | head -8
+column -t -s$'\t' "$OUT/gtc/sample_summary.tsv" | cut -c1-150 | sed -n 1,8p
 
 echo "== 3. HTML QC report + QC table"
 $SIMURG qc report --data "$OUT/gtc" --out "$OUT/qc_report" --output-format both

packaging/conda/simurg-core/meta.yaml

@@ -0,0 +1,40 @@
+# conda recipe for the optional compiled kernels of SimurgArray (Rust, PyO3; abi3 for Python >= 3.12).
+#   conda build packaging/conda/simurg-core -c conda-forge
+{% set version = load_file_regex(load_file="Cargo.toml", regex_pattern='version = "([^"]+)"', from_recipe_dir=False).group(1) %}
+
+package:
+  name: simurg-core
+  version: {{ version }}
+
+source:
+  path: ../../../rust/simurg-core
+
+build:
+  number: 0
+  skip: true  # [win]
+  script: {{ PYTHON }} -m pip install . --no-deps -vv
+
+requirements:
+  build:
+    - {{ compiler('c') }}
+    - {{ compiler('rust') }}
+  host:
+    - python >=3.12
+    - pip
+    - maturin >=1.7,<2
+  run:
+    - python >=3.12
+    - numpy >=1.26
+
+test:
+  imports:
+    - simurg_core
+  commands:
+    - python -c "import simurg_core, numpy as np; assert simurg_core.log10(np.array([0.0]))[0] == -np.inf"
+
+about:
+  summary: Optional compiled kernels of SimurgArray (results identical to the pure-Python paths)
+  license: MIT
+  license_file:
+    - LICENSE
+    - NOTICE

packaging/conda/simurg/meta.yaml

@@ -0,0 +1,48 @@
+# conda recipe for SimurgArray (pure Python). Build from a checkout:
+#   conda build packaging/conda/simurg -c conda-forge
+{% set version = load_file_regex(load_file="src/simurg/__init__.py", regex_pattern='__version__ = "([^"]+)"', from_recipe_dir=False).group(1) %}
+
+package:
+  name: simurg
+  version: {{ version }}
+
+source:
+  path: ../../..
+
+build:
+  noarch: python
+  number: 0
+  script: {{ PYTHON }} -m pip install . --no-deps --no-build-isolation -vv
+  entry_points:
+    - simurg = simurg.cli.main:app
+
+requirements:
+  host:
+    - python >=3.12
+    - pip
+    - hatchling >=1.27
+  run:
+    - python >=3.12
+    - numpy >=1.26
+    - typer >=0.12
+    - rich >=13
+    - pyyaml >=6
+    - pyarrow >=15
+    - openpyxl >=3.1
+  run_constrained:
+    - simurg-core =={{ version }}
+
+test:
+  imports:
+    - simurg
+    - simurg.cli.main
+  commands:
+    - simurg --version
+    - simurg --help
+
+about:
+  summary: Research toolkit for Illumina Infinium genotyping BeadChip data (IDAT to GTC, QC, VCF, PLINK)
+  license: MIT
+  license_file:
+    - LICENSE
+    - THIRD_PARTY_NOTICES.md

pyproject.toml

@@ -1,14 +1,25 @@
 [build-system]
-requires = ["hatchling>=1.24"]
+requires = ["hatchling>=1.27"]
 build-backend = "hatchling.build"
 
 [project]
 name = "simurg"
 description = "SimurgArray: open research toolkit for Illumina Infinium genotyping BeadChip data"
 readme = "README.md"
-license = { text = "MIT" }
+license = "MIT"
+license-files = ["LICENSE", "THIRD_PARTY_NOTICES.md"]
 requires-python = ">=3.12"
 authors = [{ name = "Kemal Yaylali" }]
+keywords = ["genotyping", "microarray", "infinium", "idat", "gtc", "vcf", "bioinformatics"]
+classifiers = [
+    "Development Status :: 5 - Production/Stable",
+    "Intended Audience :: Science/Research",
+    "Operating System :: POSIX :: Linux",
+    "Operating System :: MacOS",
+    "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
+    "Topic :: Scientific/Engineering :: Bio-Informatics",
+]
 dynamic = ["version"]
 dependencies = [
     "numpy>=1.26",   # all numerics

rust/simurg-core/.gitignore

@@ -0,0 +1 @@
+target/

rust/simurg-core/Cargo.lock

@@ -0,0 +1,228 @@
+# This file is automatically @generated by Cargo.
+# It is not intended for manual editing.
+version = 4
+
+[[package]]
+name = "autocfg"
+version = "1.5.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "f2032f911046de80f0a198e0901378627c33f59ea0ac00e363d481118bd70a53"
+
+[[package]]
+name = "heck"
+version = "0.5.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea"
+
+[[package]]
+name = "libc"
+version = "0.2.189"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "3eaf3ede3fee6db1a4c2ee091bf8a8b4dccdc6d17f656fb07896ee72867612f2"
+
+[[package]]
+name = "matrixmultiply"
+version = "0.3.11"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "3f607c237553f086e7043417a51df26b2eb899d3caff94e6a67592ff992fedc7"
+dependencies = [
+ "autocfg",
+ "rawpointer",
+]
+
+[[package]]
+name = "ndarray"
+version = "0.17.2"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "520080814a7a6b4a6e9070823bb24b4531daac8c4627e08ba5de8c5ef2f2752d"
+dependencies = [
+ "matrixmultiply",
+ "num-complex",
+ "num-integer",
+ "num-traits",
+ "portable-atomic",
+ "portable-atomic-util",
+ "rawpointer",
+]
+
+[[package]]
+name = "num-complex"
+version = "0.4.6"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "73f88a1307638156682bada9d7604135552957b7818057dcef22705b4d509495"
+dependencies = [
+ "num-traits",
+]
+
+[[package]]
+name = "num-integer"
+version = "0.1.47"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "7ce2d95d4b3734dc35aa2f45e1aa22cd416814592a4f9d9205e11affd5b8e10b"
+dependencies = [
+ "num-traits",
+]
+
+[[package]]
+name = "num-traits"
+version = "0.2.19"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "071dfc062690e90b734c0b2273ce72ad0ffa95f0c74596bc250dcfd960262841"
+dependencies = [
+ "autocfg",
+]
+
+[[package]]
+name = "numpy"
+version = "0.29.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "6a5b15d63a5ff39e378daed0e1340d3a5964703ea9712eb09a0dc66fade996f4"
+dependencies = [
+ "libc",
+ "ndarray",
+ "num-complex",
+ "num-integer",
+ "num-traits",
+ "pyo3",
+ "pyo3-build-config",
+ "rustc-hash",
+]
+
+[[package]]
+name = "once_cell"
+version = "1.21.4"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50"
+
+[[package]]
+name = "portable-atomic"
+version = "1.15.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "05c8b63e8d9609db387f0324918f81d68fe27748f084ef092fb35954d0539a85"
+
+[[package]]
+name = "portable-atomic-util"
+version = "0.2.8"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "10ab3eb7f3becc3a1cbc4f2c6f20267996cfc1a6467a873763411b136a122715"
+dependencies = [
+ "portable-atomic",
+]
+
+[[package]]
+name = "proc-macro2"
+version = "1.0.107"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "985e7ec9bb745e6ce6535b544d84d6cd6f7ad8bd711c398938ae983b91a766d9"
+dependencies = [
+ "unicode-ident",
+]
+
+[[package]]
+name = "pyo3"
+version = "0.29.2"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "4688ddedf473e32662b9b067670129a8afb8c18e351482c70d62ba4a88171e8b"
+dependencies = [
+ "libc",
+ "once_cell",
+ "portable-atomic",
+ "pyo3-build-config",
+ "pyo3-ffi",
+ "pyo3-macros",
+]
+
+[[package]]
+name = "pyo3-build-config"
+version = "0.29.2"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "f41027e41b4bd03f6e60f9f417fe24a6341a6bb744edd62b6f709f2a52ea30e9"
+dependencies = [
+ "target-lexicon",
+]
+
+[[package]]
+name = "pyo3-ffi"
+version = "0.29.2"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "e591a95526fead067432c3b3a33fc74770b87b1e04e73671090d9c2055a2b327"
+dependencies = [
+ "libc",
+ "pyo3-build-config",
+]
+
+[[package]]
+name = "pyo3-macros"
+version = "0.29.2"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "73225868fc1cd84eef2c3c230ddb91273bf1de46aeb8a4248da76d32a0924a1c"
+dependencies = [
+ "proc-macro2",
+ "pyo3-macros-backend",
+ "quote",
+ "syn",
+]
+
+[[package]]
+name = "pyo3-macros-backend"
+version = "0.29.2"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "571575aa3749fa6216757dd47d2a3e7ef360f329a40f0666a9fbd14889024952"
+dependencies = [
+ "heck",
+ "proc-macro2",
+ "quote",
+ "syn",
+]
+
+[[package]]
+name = "quote"
+version = "1.0.47"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "1fbf4db142a473a8d80c26bbf18454ed458bf8d26c8219c331daecfdbd079001"
+dependencies = [
+ "proc-macro2",
+]
+
+[[package]]
+name = "rawpointer"
+version = "0.2.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "60a357793950651c4ed0f3f52338f53b2f809f32d83a07f72909fa13e4c6c1e3"
+
+[[package]]
+name = "rustc-hash"
+version = "2.1.3"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "6b1e7f9a428571be2dc5bc0505c13fb6bf936822b894ec87abf8a08a4e51742d"
+
+[[package]]
+name = "simurg-core"
+version = "1.1.0"
+dependencies = [
+ "numpy",
+ "pyo3",
+]
+
+[[package]]
+name = "syn"
+version = "2.0.119"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "872831b642d1a07999a962a351ed35b955ea2cfc8f3862091e2a240a84f17297"
+dependencies = [
+ "proc-macro2",
+ "quote",
+ "unicode-ident",
+]
+
+[[package]]
+name = "target-lexicon"
+version = "0.13.5"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "adb6935a6f5c20170eeceb1a3835a49e12e19d792f6dd344ccc76a985ca5a6ca"
+
+[[package]]
+name = "unicode-ident"
+version = "1.0.26"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d245f478577f809a851594d02313b640fb437e0bb33866753cff937863096954"

rust/simurg-core/Cargo.toml

@@ -0,0 +1,20 @@
+[package]
+name = "simurg-core"
+version = "1.1.0"
+edition = "2021"
+license = "MIT"
+description = "Compiled kernels for SimurgArray (plan §18 P9)"
+publish = false
+
+[lib]
+name = "simurg_core"
+crate-type = ["cdylib"]
+
+[dependencies]
+numpy = "0.29.0"
+pyo3 = { version = "0.29", features = ["extension-module", "abi3-py312"] }
+
+[profile.release]
+opt-level = 3
+lto = true
+codegen-units = 1

rust/simurg-core/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Kemal Yaylali
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

rust/simurg-core/NOTICE

@@ -0,0 +1,6 @@
+simurg-core: optional compiled kernels of SimurgArray (MIT, see LICENSE).
+
+The kernels port numerics that SimurgArray's pure-Python modules port from gtc2vcf (MIT, (C) Giulio Genovese) and
+Illumina BeadArrayFiles (BSD-2-Clause, (C) 2020 Illumina); full notices: THIRD_PARTY_NOTICES.md of SimurgArray.
+The binary statically links PyO3, rust-numpy, ndarray and their dependencies (MIT / Apache-2.0 / BSD-2-Clause);
+`cargo tree` lists them and each crate's licence ships with its source on crates.io.

rust/simurg-core/pyproject.toml

@@ -0,0 +1,15 @@
+[build-system]
+requires = ["maturin>=1.7,<2"]
+build-backend = "maturin"
+
+[project]
+name = "simurg-core"
+description = "Compiled kernels for SimurgArray (optional; the pure-Python paths give identical results)"
+requires-python = ">=3.12"
+license = "MIT"
+license-files = ["LICENSE", "NOTICE"]
+dependencies = ["numpy>=1.26"]
+dynamic = ["version"]
+
+[tool.maturin]
+features = ["pyo3/extension-module"]

rust/simurg-core/src/bpm.rs

@@ -0,0 +1,275 @@
+//! BPM LocusEntry records — port of `simurg.formats.bpm._parse_locus_entries` (format: Illumina BeadArrayFiles
+//! `BeadPoolManifest.py`, BSD-2). Same field order, checks and error messages; strings of ≤ 64 bytes are shared
+//! Python objects, as in the Python parser.
+
+use std::collections::HashMap;
+
+use numpy::{IntoPyArray, PyArray1, PyArray2};
+use pyo3::exceptions::PyValueError;
+use pyo3::prelude::*;
+use pyo3::types::{PyList, PyString};
+
+pub const STR_COLS: usize = 16;
+// output column order (matches the Python `str_cols` list)
+const ILMN: usize = 0;
+const SNP: usize = 1;
+const CHR: usize = 2;
+const ISTR: usize = 3;
+const SSTR: usize = 4;
+const RSTR: usize = 5;
+const PLOIDY: usize = 6;
+const SPECIES: usize = 7;
+const BUILD: usize = 8;
+const SOURCE: usize = 9;
+const SVER: usize = 10;
+const CSTR: usize = 11;
+const SSEQ: usize = 12;
+const TOP: usize = 13;
+const PA: usize = 14;
+const PB: usize = 15;
+
+struct Cursor<'a> {
+    b: &'a [u8],
+    pos: usize,
+}
+
+fn err(msg: impl Into<String>) -> PyErr {
+    PyValueError::new_err(msg.into())
+}
+
+impl<'a> Cursor<'a> {
+    fn string(&mut self) -> PyResult<&'a [u8]> {
+        let size = self.b.len();
+        let mut p = self.pos;
+        if p >= size {
+            return Err(err("unexpected end of file in locus entry"));
+        }
+        let mut byte = self.b[p];
+        p += 1;
+        let mut ln = (byte & 0x7f) as usize;
+        let mut shift = 7;
+        while byte & 0x80 != 0 {
+            if p >= size || shift > 28 {
+                return Err(err("bad string length in locus entry"));
+            }
+            byte = self.b[p];
+            p += 1;
+            ln |= ((byte & 0x7f) as usize) << shift;
+            shift += 7;
+        }
+        let end = p + ln;
+        if end > size {
+            return Err(err("string runs past end of file in locus entry"));
+        }
+        self.pos = end;
+        Ok(&self.b[p..end])
+    }
+    fn i32(&mut self, what: &str) -> PyResult<i32> {
+        if self.pos + 4 > self.b.len() {
+            return Err(err(format!("unexpected end of file at {what}")));
+        }
+        let v = i32::from_le_bytes(self.b[self.pos..self.pos + 4].try_into().unwrap());
+        self.pos += 4;
+        Ok(v)
+    }
+}
+
+type Parsed<'py> = (
+    Vec<Bound<'py, PyList>>,
+    Bound<'py, PyArray1<i64>>,
+    Bound<'py, PyArray1<i32>>,
+    Bound<'py, PyArray1<i32>>,
+    Bound<'py, PyArray1<i16>>,
+    Bound<'py, PyArray1<i8>>,
+    Bound<'py, PyArray1<bool>>,
+    Bound<'py, PyArray1<i8>>,
+    Option<Bound<'py, PyArray2<f32>>>,
+    usize,
+);
+
+/// Parse `n` LocusEntry records starting at byte 0 of `buf`. Returns the 16 string columns, map_infos,
+/// address_a, address_b, assay types (−1 = infer from sequences, entry version < 6), expected clusters,
+/// intensity-only flags, entry versions, base fractions (None unless some entry is v7+) and the bytes consumed.
+/// Errors are ValueError with the Python parser's message.
+#[pyfunction]
+pub fn bpm_locus_entries<'py>(py: Python<'py>, buf: &[u8], n: usize, names: Vec<String>) -> PyResult<Parsed<'py>> {
+    let name_to_idx: HashMap<&str, usize> = names.iter().enumerate().map(|(i, s)| (s.as_str(), i)).collect();
+    let mut cols: Vec<Vec<Option<Py<PyString>>>> = (0..STR_COLS).map(|_| (0..n).map(|_| None).collect()).collect();
+    let mut intern: HashMap<&[u8], Py<PyString>> = HashMap::new();
+    let mut map_infos = vec![0i64; n];
+    let mut address_a = vec![0i32; n];
+    let mut address_b = vec![0i32; n];
+    let mut assay = vec![0i16; n];
+    let mut exp_clusters = vec![0i8; n];
+    let mut int_only = vec![false; n];
+    let mut versions = vec![0i8; n];
+    let mut fracs = vec![0f32; 4 * n];
+    let mut seen = vec![false; n];
+    let mut any_v7 = false;
+    let mut c = Cursor { b: buf, pos: 0 };
+
+    let to_py = |raw: &'_ [u8], intern: &mut HashMap<&[u8], Py<PyString>>| -> Py<PyString> {
+        if let Some(s) = intern.get(raw) {
+            return s.clone_ref(py);
+        }
+        let s = PyString::new(py, &String::from_utf8_lossy(raw)).unbind();
+        s
+    };
+
+    for _ in 0..n {
+        let ver = c.i32("locus entry")?;
+        if ![4, 6, 7, 8].contains(&ver) {
+            return Err(err(format!("unsupported locus entry version {ver}")));
+        }
+        let ilmn_id = c.string()?;
+        let name = c.string()?;
+        for _ in 0..3 {
+            c.string()?;
+        }
+        c.i32("locus index")?;
+        c.string()?;
+        let ilmn_strand = c.string()?;
+        let snp = c.string()?;
+        let chrom = c.string()?;
+        let ploidy = c.string()?;
+        let species = c.string()?;
+        let map_info = c.string()?;
+        let top_seq = c.string()?;
+        let cust_strand = c.string()?;
+        if c.pos + 8 > buf.len() {
+            return Err(err("unexpected end of file at addresses"));
+        }
+        let a_addr = c.i32("addresses")?;
+        let b_addr = c.i32("addresses")?;
+        let probe_a = c.string()?;
+        let probe_b = c.string()?;
+        let build = c.string()?;
+        let source = c.string()?;
+        let source_ver = c.string()?;
+        let source_strand = c.string()?;
+        let source_seq = c.string()?;
+        let (mut at, mut exp_cl, mut io) = (-1i16, 0i8, 0u8);
+        if ver >= 6 {
+            if c.pos + 4 > buf.len() {
+                return Err(err("unexpected end of file at assay type"));
+            }
+            exp_cl = buf[c.pos + 1] as i8;
+            io = buf[c.pos + 2];
+            at = buf[c.pos + 3] as i16;
+            c.pos += 4;
+            if at > 2 {
+                return Err(err(format!("invalid assay type {at}")));
+            }
+            if (b_addr == 0) != (at == 0) {
+                return Err(err("assay type is inconsistent with address B"));
+            }
+        }
+        let mut f4: Option<[f32; 4]> = None;
+        if ver >= 7 {
+            if c.pos + 16 > buf.len() {
+                return Err(err("unexpected end of file at base fractions"));
+            }
+            let mut f = [0f32; 4];
+            for (k, v) in f.iter_mut().enumerate() {
+                *v = f32::from_le_bytes(buf[c.pos + 4 * k..c.pos + 4 * k + 4].try_into().unwrap());
+            }
+            f4 = Some(f);
+            c.pos += 16;
+        }
+        let ref_strand: &[u8] = if ver >= 8 { c.string()? } else { b"" };
+
+        let name_s = String::from_utf8_lossy(name);
+        let idx = match name_to_idx.get(name_s.as_ref()) {
+            Some(&i) => i,
+            None => return Err(err(format!("locus entry {} not in the names table", py_repr(&name_s)))),
+        };
+        if seen[idx] {
+            return Err(err(format!("locus entry {} appears twice", py_repr(&name_s))));
+        }
+        seen[idx] = true;
+        for (col, raw) in [
+            (ILMN, ilmn_id),
+            (SNP, snp),
+            (CHR, chrom),
+            (ISTR, ilmn_strand),
+            (SSTR, source_strand),
+            (RSTR, ref_strand),
+            (PLOIDY, ploidy),
+            (SPECIES, species),
+            (BUILD, build),
+            (SOURCE, source),
+            (SVER, source_ver),
+            (CSTR, cust_strand),
+            (SSEQ, source_seq),
+            (TOP, top_seq),
+            (PA, probe_a),
+            (PB, probe_b),
+        ] {
+            let s = to_py(raw, &mut intern);
+            if raw.len() <= 64 && !intern.contains_key(raw) {
+                intern.insert(raw, s.clone_ref(py));
+            }
+            cols[col][idx] = Some(s);
+        }
+        map_infos[idx] = if map_info.is_empty() {
+            0
+        } else {
+            let t = String::from_utf8_lossy(map_info);
+            match t.trim().parse::<i64>() {
+                Ok(v) => v,
+                Err(_) => {
+                    return Err(err(format!(
+                        "non-integer MapInfo {} for {}",
+                        py_repr(&t),
+                        py_repr(&name_s)
+                    )))
+                }
+            }
+        };
+        address_a[idx] = a_addr;
+        address_b[idx] = b_addr;
+        assay[idx] = at;
+        exp_clusters[idx] = exp_cl;
+        int_only[idx] = io != 0;
+        versions[idx] = ver as i8;
+        if let Some(f) = f4 {
+            fracs[4 * idx..4 * idx + 4].copy_from_slice(&f);
+            any_v7 = true;
+        }
+    }
+    let missing = seen.iter().filter(|s| !**s).count();
+    if missing > 0 {
+        return Err(err(format!("{missing} loci in the names table have no locus entry")));
+    }
+    let empty = PyString::new(py, "").unbind();
+    let lists = cols
+        .into_iter()
+        .map(|col| PyList::new(py, col.into_iter().map(|s| s.unwrap_or_else(|| empty.clone_ref(py)))))
+        .collect::<PyResult<Vec<_>>>()?;
+    let fr = if any_v7 {
+        Some(numpy::ndarray::Array2::from_shape_vec((n, 4), fracs).map_err(|e| err(e.to_string()))?.into_pyarray(py))
+    } else {
+        None
+    };
+    Ok((
+        lists,
+        map_infos.into_pyarray(py),
+        address_a.into_pyarray(py),
+        address_b.into_pyarray(py),
+        assay.into_pyarray(py),
+        exp_clusters.into_pyarray(py),
+        int_only.into_pyarray(py),
+        versions.into_pyarray(py),
+        fr,
+        c.pos,
+    ))
+}
+
+/// Python `repr()` of a str for error messages (single quotes unless the text contains one).
+fn py_repr(s: &str) -> String {
+    if s.contains('\'') && !s.contains('"') {
+        format!("\"{s}\"")
+    } else {
+        format!("'{}'", s.replace('\\', "\\\\").replace('\'', "\\'"))
+    }
+}

rust/simurg-core/src/gencall.rs

@@ -0,0 +1,357 @@
+//! Fused per-locus ports of `simurg.genotype.gencall.call_genotypes` and `simurg.genotype.baf_lrr.baf_lrr`
+//! (which port gtc2vcf `gen_std_flair`/`modilik`/`compute_score_call_prelim`/`gencall_score_map`/`make_calls`
+//! and `compute_baf_lrr`, MIT, © Giulio Genovese). Every intermediate keeps the NumPy reference's type (f32 or
+//! f64) and evaluation order; NumPy's `fmin`/`fmax` are `f32::min`/`max`; libm calls use CPython `math`
+//! semantics. One pass per locus instead of dozens of full-array temporaries.
+
+use numpy::{IntoPyArray, PyArray1, PyReadonlyArray1};
+use pyo3::exceptions::PyValueError;
+use pyo3::prelude::*;
+
+use crate::{math_exp, math_log, math_pow};
+
+const M_2_SQRTPI: f64 = 1.12837916709551257390;
+const M_SQRT1_2: f64 = 0.707106781186547524401;
+const NEG_NAN32: f32 = f32::from_bits(0xffc0_0000);
+
+#[inline]
+fn smf(x: f32, a: f32, b: f32) -> f32 {
+    let mid = (a + b) / 2.0;
+    if a >= b {
+        return if x >= mid { 1.0 } else { 0.0 };
+    }
+    let u = (x - a) / (b - a);
+    let v = (x - b) / (b - a);
+    if x <= a {
+        0.0
+    } else if a < x && x <= mid {
+        2.0 * (u * u)
+    } else if mid < x && x <= b {
+        1.0 - 2.0 * (v * v)
+    } else {
+        1.0
+    }
+}
+
+#[inline]
+fn zmf(x: f32, a: f32, b: f32) -> f32 {
+    let mid = (a + b) / 2.0;
+    let u = (x - a) / (b - a);
+    let v = (x - b) / (b - a);
+    if x <= a {
+        1.0
+    } else if a < x && x <= mid {
+        1.0 - 2.0 * (u * u)
+    } else if mid < x && x <= b {
+        2.0 * (v * v)
+    } else {
+        0.0
+    }
+}
+
+/// Normal pdf pieces of `_pdf`: (full, peak, bad) for value x, mean mu, sd sigma.
+#[inline]
+fn pdf(x: f32, mu: f32, sigma: f32) -> (f64, f64, bool) {
+    let z = ((x - mu) / sigma) as f64;
+    let e = math_exp(-0.5 * (z * z));
+    let sd = sigma as f64;
+    let full = e * 0.5 * M_2_SQRTPI * M_SQRT1_2 / sd;
+    let peak = (0.5 * M_2_SQRTPI * M_SQRT1_2) / sd;
+    (full, peak, sigma <= 0.0)
+}
+
+#[derive(Clone, Copy)]
+enum Side {
+    Left,
+    Right,
+    Full,
+}
+
+#[inline]
+fn get(p: (f64, f64, bool), x: f32, mu: f32, side: Side) -> f64 {
+    let (full, peak, bad) = p;
+    if bad {
+        return f64::NAN;
+    }
+    match side {
+        Side::Left => {
+            if x < mu {
+                peak
+            } else {
+                full
+            }
+        }
+        Side::Right => {
+            if x > mu {
+                peak
+            } else {
+                full
+            }
+        }
+        Side::Full => full,
+    }
+}
+
+fn check(len: usize, n: usize, what: &str) -> PyResult<()> {
+    if len != n {
+        return Err(PyValueError::new_err(format!("gencall: {what} has length {len}, expected {n}")));
+    }
+    Ok(())
+}
+
+/// Genotypes (0 NC, 1 AA, 2 AB, 3 BB) and GenCall scores; (n, 3) cluster arrays are passed flattened (C order).
+#[pyfunction]
+#[allow(clippy::too_many_arguments)]
+pub fn call_genotypes<'py>(
+    py: Python<'py>,
+    theta: PyReadonlyArray1<'py, f32>,
+    r: PyReadonlyArray1<'py, f32>,
+    raw_zero: PyReadonlyArray1<'py, bool>,
+    present: PyReadonlyArray1<'py, bool>,
+    n_cl: PyReadonlyArray1<'py, i32>,
+    theta_mean: PyReadonlyArray1<'py, f32>,
+    theta_dev: PyReadonlyArray1<'py, f32>,
+    r_mean: PyReadonlyArray1<'py, f32>,
+    r_dev: PyReadonlyArray1<'py, f32>,
+    intensity_threshold: PyReadonlyArray1<'py, f32>,
+    gentrain: PyReadonlyArray1<'py, f32>,
+    intensity_only: PyReadonlyArray1<'py, bool>,
+    cutoff: f32,
+) -> PyResult<(Bound<'py, PyArray1<u8>>, Bound<'py, PyArray1<f32>>)> {
+    let (th, rr, rz, pres) = (theta.as_slice()?, r.as_slice()?, raw_zero.as_slice()?, present.as_slice()?);
+    let (ncl, tmv, tdv, rmv, rdv) =
+        (n_cl.as_slice()?, theta_mean.as_slice()?, theta_dev.as_slice()?, r_mean.as_slice()?, r_dev.as_slice()?);
+    let (ithr, gt, io) = (intensity_threshold.as_slice()?, gentrain.as_slice()?, intensity_only.as_slice()?);
+    let n = th.len();
+    for (l, w) in [(rr.len(), "r"), (rz.len(), "raw_zero"), (pres.len(), "present"), (ithr.len(), "threshold"),
+                   (gt.len(), "gentrain"), (io.len(), "intensity_only")] {
+        check(l, n, w)?;
+    }
+    for (l, w) in [(ncl.len(), "n"), (tmv.len(), "theta_mean"), (tdv.len(), "theta_dev"), (rmv.len(), "r_mean"),
+                   (rdv.len(), "r_dev")] {
+        check(l, 3 * n, w)?;
+    }
+    let tiny_d = f64::MIN_POSITIVE * f64::EPSILON;
+    let third: f32 = 1.0 / 3.0;
+    let mut geno = vec![0u8; n];
+    let mut scores = vec![0f32; n];
+
+    for i in 0..n {
+        let t = th[i];
+        let r = rr[i];
+        let tm = [tmv[3 * i], tmv[3 * i + 1], tmv[3 * i + 2]];
+        let td = [tdv[3 * i], tdv[3 * i + 1], tdv[3 * i + 2]];
+        let rm = [rmv[3 * i], rmv[3 * i + 1], rmv[3 * i + 2]];
+        let rd = [rdv[3 * i], rdv[3 * i + 1], rdv[3 * i + 2]];
+        let nc = [ncl[3 * i], ncl[3 * i + 1], ncl[3 * i + 2]];
+
+        // gen_std_flair
+        let z1 = 0.5f32 * (tm[1] - tm[0]) / (td[1] + td[0]);
+        let z2 = 0.5f32 * (tm[2] - tm[1]) / (td[2] + td[1]);
+        let mz = z1.min(z2);
+        let alpha = (third * mz).max(1.0);
+        let beta = (third * mz).min(1.0);
+        let eta = alpha * beta;
+        let mut ftd = [0f32; 3];
+        for k in 0..3 {
+            let v = td[k] * eta;
+            ftd[k] = if v < 0.02 { 0.02 } else { v };
+        }
+        let med = td[0].min(td[1]).max(td[0].max(td[1]).min(td[2]));
+        let min_r = if 0.2f32 < med { med } else { 0.2 };
+        let mut frd = [0f32; 3];
+        for k in 0..3 {
+            let mrk0 = rm[k] / 7.0;
+            let mrk = if mrk0 < min_r { min_r } else { mrk0 };
+            frd[k] = if rd[k] < mrk { mrk } else { rd[k] };
+        }
+
+        // modilik
+        let d: [f64; 3] = std::array::from_fn(|k| 100.0 * ((t - tm[k]).abs() as f64) + ((r - rm[k]).abs() as f64));
+        let (daa, dab, dbb) = (d[0], d[1], d[2]);
+        let ok_t = !t.is_nan();
+        let pt: [(f64, f64, bool); 3] = std::array::from_fn(|k| pdf(t, tm[k], ftd[k]));
+        let pr: [(f64, f64, bool); 3] = std::array::from_fn(|k| pdf(r, rm[k], frd[k]));
+        let pdf2 = |k: usize, st: Side, sr: Side| get(pt[k], t, tm[k], st) * get(pr[k], r, rm[k], sr);
+        use Side::{Full, Left, Right};
+        let mut lik = [0f64; 3];
+        let mut covered = false;
+        if ok_t {
+            let cases: [(bool, [(usize, Side, Side); 2]); 6] = [
+                (daa <= dbb && dab <= dbb && rm[0] <= rm[1], [(0, Left, Left), (1, Right, Right)]),
+                (daa <= dab && dbb <= dab && rm[0] <= rm[2], [(0, Left, Left), (2, Right, Right)]),
+                (dab <= daa && dbb <= daa && rm[1] <= rm[2], [(1, Left, Left), (2, Right, Right)]),
+                (daa <= dbb && dab <= dbb && rm[0] > rm[1], [(0, Left, Right), (1, Right, Left)]),
+                (dab <= daa && dbb <= daa && rm[1] > rm[2], [(1, Left, Right), (2, Right, Left)]),
+                (daa <= dab && dbb <= dab && rm[0] > rm[2], [(0, Left, Right), (2, Right, Left)]),
+            ];
+            for (cond, assigns) in cases {
+                if cond {
+                    for (k, st, sr) in assigns {
+                        lik[k] = pdf2(k, st, sr);
+                    }
+                    covered = true;
+                }
+            }
+        }
+        if !covered {
+            for (k, l) in lik.iter_mut().enumerate() {
+                *l = pdf2(k, Full, Full);
+            }
+        }
+        for l in lik.iter_mut() {
+            if l.is_nan() {
+                *l = 0.0;
+            }
+        }
+        if !ok_t {
+            lik = [f64::NAN; 3];
+        }
+
+        // priors and posteriors
+        let ntot = nc[0] as i64 + nc[1] as i64 + nc[2] as i64;
+        let p: [f64; 3] = std::array::from_fn(|k| (((nc[k] as f32) / (ntot as f32)) as f64 + 1.0) / 4.0);
+        let lp: [f64; 3] = std::array::from_fn(|k| lik[k] * p[k]);
+        let ev = (lp[0] + lp[1]) + lp[2];
+        let (aa, ab, bb) = (lp[0] / ev, lp[1] / ev, lp[2] / ev);
+        let mut iap: u8 = if aa >= ab && aa >= bb {
+            1
+        } else if ab >= aa && ab >= bb {
+            2
+        } else if bb >= aa && bb >= ab {
+            3
+        } else {
+            0
+        };
+        let mut mx = if aa > ab { aa } else { ab };
+        let mut scnd = if aa > ab { ab } else { aa };
+        let b_gt_mx = bb > mx;
+        scnd = if b_gt_mx { mx } else if bb > scnd { bb } else { scnd };
+        mx = if b_gt_mx { bb } else { mx };
+        let ap_lod = math_log(mx / (tiny_d + scnd), true);
+        let score_ap = smf(ap_lod as f32, 0.0, 2.0);
+
+        // sub-scores
+        let score_r1 = smf(r, 0.0, 0.1);
+        let nclus = nc.iter().filter(|&&v| v > 0).count();
+        let misclust: f32 = if nclus == 1 && nc[1] == 0 {
+            0.7
+        } else if nclus != 3 {
+            0.95
+        } else {
+            1.0
+        };
+        let g = (iap as i64 - 1).clamp(0, 2) as usize;
+        let (gm_t, gd_t, gm_r, gd_r) = (tm[g], ftd[g], rm[g], frd[g]);
+        let st_aa = zmf(t, gm_t + 2.0 * gd_t, gm_t + 6.0 * gd_t);
+        let st_ab = zmf(((t - gm_t) / gd_t).abs(), 2.0, 6.0);
+        let st_bb = smf(t, gm_t - 6.0 * gd_t, gm_t - 2.0 * gd_t);
+        let mut score_t = match iap {
+            1 => st_aa,
+            2 => st_ab,
+            3 => st_bb,
+            _ => 1.0,
+        };
+        let r2 = smf(r, 0.0, gm_r / 10.0);
+        let r3 = smf(r, gm_r - 6.0 * gd_r, gm_r - 2.0 * gd_r);
+        let r4 = zmf(r, gm_r + 6.0 * gd_r, gm_r + 12.0 * gd_r);
+        let mut score_r = if iap > 0 { ((score_r1 * r2) * r3) * r4 } else { 0.0 };
+        let mut score_n: f32 = 1.0;
+        if gt[i].is_nan() {
+            score_t = f32::NAN;
+            score_r = f32::NAN;
+        }
+        if !ok_t {
+            score_t = f32::NAN;
+            score_r = f32::NAN;
+            score_n = f32::NAN;
+        }
+        let mut prelim = ((((score_ap * gt[i]) * score_t) * score_r) * score_n) * misclust;
+        if r < ithr[i] {
+            prelim = f32::NAN;
+            iap = 0;
+        }
+
+        // gencall_score_map
+        let x = prelim as f64;
+        let q = (x - 1.08) / 0.504;
+        let gbell = 1.0 / (1.0 + math_pow(q * q, 1.71));
+        let mut score = (math_pow(x, 0.35) * gbell) as f32;
+        if score.is_nan() {
+            score = 0.0;
+        }
+        if rz[i] || !pres[i] {
+            score = 0.0;
+            iap = 0;
+        }
+        geno[i] = if score < cutoff || io[i] { 0 } else { iap };
+        scores[i] = score;
+    }
+    Ok((geno.into_pyarray(py), scores.into_pyarray(py)))
+}
+
+/// BAF and LRR (float32, canonical NaN 0xFFC00000 where undefined).
+#[pyfunction]
+#[allow(clippy::too_many_arguments)]
+pub fn baf_lrr<'py>(
+    py: Python<'py>,
+    theta: PyReadonlyArray1<'py, f32>,
+    r: PyReadonlyArray1<'py, f32>,
+    raw_zero: PyReadonlyArray1<'py, bool>,
+    present: PyReadonlyArray1<'py, bool>,
+    theta_mean: PyReadonlyArray1<'py, f32>,
+    r_mean: PyReadonlyArray1<'py, f32>,
+    mean_r: PyReadonlyArray1<'py, f32>,
+    intensity_only: PyReadonlyArray1<'py, bool>,
+) -> PyResult<(Bound<'py, PyArray1<f32>>, Bound<'py, PyArray1<f32>>)> {
+    let (th, rr, rz, pres) = (theta.as_slice()?, r.as_slice()?, raw_zero.as_slice()?, present.as_slice()?);
+    let (tmv, rmv, mr, io) = (theta_mean.as_slice()?, r_mean.as_slice()?, mean_r.as_slice()?, intensity_only.as_slice()?);
+    let n = th.len();
+    for (l, w) in [(rr.len(), "r"), (rz.len(), "raw_zero"), (pres.len(), "present"), (mr.len(), "mean_r"), (io.len(), "intensity_only")] {
+        check(l, n, w)?;
+    }
+    check(tmv.len(), 3 * n, "theta_mean")?;
+    check(rmv.len(), 3 * n, "r_mean")?;
+    let mut baf = vec![0f32; n];
+    let mut lrr = vec![0f32; n];
+    for i in 0..n {
+        let (t, r) = (th[i], rr[i]);
+        let (at, bt, ct) = (tmv[3 * i], tmv[3 * i + 1], tmv[3 * i + 2]);
+        let (ar, br, cr) = (rmv[3 * i], rmv[3 * i + 1], rmv[3 * i + 2]);
+        let (eq, lo, hi) = (t == bt, t < bt, t > bt);
+        let (r_ref, mut b) = if eq {
+            (br, 0.5f32)
+        } else if lo {
+            (ar + ((t - at) * (ar - br)) / (at - bt), ((t - at) / (bt - at)) * 0.5)
+        } else if hi {
+            (br + ((t - bt) * (cr - br)) / (ct - bt), 0.5 + ((t - bt) / (ct - bt)) * 0.5)
+        } else {
+            (f32::NAN, NEG_NAN32)
+        };
+        let denom = if io[i] { mr[i] } else { r_ref };
+        let ratio = (r / denom) as f64;
+        let mut l = math_log(ratio, false) as f32;
+        if r == 0.0 {
+            l = -f32::MAX;
+        }
+        if !(eq || lo || hi) {
+            l = NEG_NAN32;
+        }
+        b = if b < 0.0 {
+            0.0
+        } else if b > 1.0 {
+            1.0
+        } else {
+            b
+        };
+        if rz[i] || !pres[i] || b.is_nan() {
+            b = NEG_NAN32;
+        }
+        if rz[i] || !pres[i] || l.is_nan() {
+            l = NEG_NAN32;
+        }
+        baf[i] = b;
+        lrr[i] = l;
+    }
+    Ok((baf.into_pyarray(py), lrr.into_pyarray(py)))
+}

rust/simurg-core/src/lib.rs

@@ -0,0 +1,226 @@
+//! Compiled kernels for SimurgArray (plan §18 P9, §19).
+//!
+//! Every kernel is a drop-in replacement for a pure-Python/NumPy reference in the `simurg` package and returns
+//! bit-identical results (checked by `tests/unit/test_core_kernels.py`):
+//!
+//! * `exp`, `log10`, `log2`, `pow` — element-wise glibc libm calls with the error semantics of CPython's `math`
+//!   module as used by `simurg.genotype.gencall._c_call` (NaN input returned as is, a domain error gives the
+//!   canonical +NaN, `log10(0)` = −inf, overflow = +inf). The libm symbols are the ones CPython's `math` calls, so
+//!   results equal the C reference implementations (gtc2vcf, MIT) that the Python port follows.
+//! * `closest_binned` — the per-bin candidate scan of the vendor's binned nearest-neighbour search
+//!   (`simurg.signal.normalization.fit._closest_binned`): float32 squared distances, first minimum wins.
+//!
+//! * `call_genotypes`, `baf_lrr` — fused per-locus GenCall calling/scoring and BAF/LRR (`gencall.rs`).
+//! * `bpm_locus_entries` — the BPM LocusEntry parser (`simurg.formats.bpm._parse_locus_entries`).
+//!
+//! No floating-point contraction (FMA) happens: Rust never fuses `a*b+c` implicitly.
+
+mod bpm;
+mod gencall;
+
+use numpy::{IntoPyArray, PyArray1, PyReadonlyArray1};
+use pyo3::exceptions::PyValueError;
+use pyo3::prelude::*;
+
+#[link(name = "m")]
+unsafe extern "C" {
+    #[link_name = "exp"]
+    fn c_exp(x: f64) -> f64;
+    #[link_name = "log10"]
+    fn c_log10(x: f64) -> f64;
+    #[link_name = "log2"]
+    fn c_log2(x: f64) -> f64;
+    #[link_name = "pow"]
+    fn c_pow(x: f64, y: f64) -> f64;
+}
+
+/// CPython's `float('nan')`.
+const PY_NAN: f64 = f64::from_bits(0x7ff8_0000_0000_0000);
+
+#[inline]
+pub(crate) fn math_exp(v: f64) -> f64 {
+    if v.is_nan() {
+        return v;
+    }
+    let r = unsafe { c_exp(v) };
+    if r.is_nan() { PY_NAN } else if r.is_infinite() && v.is_finite() { f64::INFINITY } else { r }
+}
+
+/// CPython `m_log10`/`m_log2` + `_c_call`: x > 0 → libm; 0 → −inf (log10) or NaN (log2); x < 0, −inf → NaN.
+#[inline]
+pub(crate) fn math_log(v: f64, ten: bool) -> f64 {
+    if v.is_nan() {
+        return v;
+    }
+    if v.is_infinite() {
+        return if v > 0.0 { v } else { PY_NAN };
+    }
+    if v > 0.0 {
+        return unsafe { if ten { c_log10(v) } else { c_log2(v) } };
+    }
+    if v == 0.0 && ten { f64::NEG_INFINITY } else { PY_NAN }
+}
+
+/// CPython `math.pow(v, e)` for a finite exponent e > 0 that is not an integer, + `_c_call`.
+#[inline]
+pub(crate) fn math_pow(v: f64, e: f64) -> f64 {
+    if v.is_nan() {
+        return v;
+    }
+    if v.is_infinite() {
+        return f64::INFINITY;
+    }
+    if v == 0.0 {
+        return 0.0;
+    }
+    let r = unsafe { c_pow(v, e) };
+    if r.is_nan() { PY_NAN } else if r.is_infinite() { f64::INFINITY } else { r }
+}
+
+fn map_f64<'py>(py: Python<'py>, x: PyReadonlyArray1<'py, f64>, f: impl Fn(f64) -> f64) -> PyResult<Bound<'py, PyArray1<f64>>> {
+    let s = x.as_slice()?;
+    let out: Vec<f64> = s.iter().map(|&v| f(v)).collect();
+    Ok(out.into_pyarray(py))
+}
+
+#[pyfunction]
+fn exp<'py>(py: Python<'py>, x: PyReadonlyArray1<'py, f64>) -> PyResult<Bound<'py, PyArray1<f64>>> {
+    map_f64(py, x, math_exp)
+}
+
+#[pyfunction]
+fn log10<'py>(py: Python<'py>, x: PyReadonlyArray1<'py, f64>) -> PyResult<Bound<'py, PyArray1<f64>>> {
+    map_f64(py, x, |v| math_log(v, true))
+}
+
+#[pyfunction]
+fn log2<'py>(py: Python<'py>, x: PyReadonlyArray1<'py, f64>) -> PyResult<Bound<'py, PyArray1<f64>>> {
+    map_f64(py, x, |v| math_log(v, false))
+}
+
+#[pyfunction]
+fn pow<'py>(py: Python<'py>, x: PyReadonlyArray1<'py, f64>, e: f64) -> PyResult<Bound<'py, PyArray1<f64>>> {
+    if !(e.is_finite() && e > 0.0 && e.fract() != 0.0) {
+        return Err(PyValueError::new_err("pow kernel: exponent must be finite, positive and not an integer"));
+    }
+    map_f64(py, x, |v| math_pow(v, e))
+}
+
+/// For each sweep point `p` searching bin `choice[p]` (0..11 regular, 12..22 shifted; any other value: none), the
+/// index of the closest candidate among `eligible[i]` with `b1[i] == choice` (regular) or `b2[i] == choice − 12`
+/// (shifted), in ascending order; squared distance in float32; −1 if no candidate or the best is not < 1e20.
+#[pyfunction]
+#[allow(clippy::too_many_arguments)]
+fn closest_binned<'py>(
+    py: Python<'py>,
+    raw_x: PyReadonlyArray1<'py, f32>,
+    raw_y: PyReadonlyArray1<'py, f32>,
+    axis_x: PyReadonlyArray1<'py, f32>,
+    axis_y: PyReadonlyArray1<'py, f32>,
+    choice: PyReadonlyArray1<'py, i64>,
+    eligible: PyReadonlyArray1<'py, i64>,
+    b1: PyReadonlyArray1<'py, i64>,
+    b2: PyReadonlyArray1<'py, i64>,
+) -> PyResult<Bound<'py, PyArray1<i64>>> {
+    let (rx, ry) = (raw_x.as_slice()?, raw_y.as_slice()?);
+    let (ax, ay, ch) = (axis_x.as_slice()?, axis_y.as_slice()?, choice.as_slice()?);
+    let (el, b1, b2) = (eligible.as_slice()?, b1.as_slice()?, b2.as_slice()?);
+    if ax.len() != ay.len() || ax.len() != ch.len() || el.len() != b1.len() || el.len() != b2.len() || rx.len() != ry.len() {
+        return Err(PyValueError::new_err("closest_binned: length mismatch"));
+    }
+    let mut groups: Vec<Vec<usize>> = vec![Vec::new(); 23];
+    for (i, &e) in el.iter().enumerate() {
+        let e = usize::try_from(e).map_err(|_| PyValueError::new_err("closest_binned: negative index"))?;
+        if e >= rx.len() {
+            return Err(PyValueError::new_err("closest_binned: index out of range"));
+        }
+        if (0..12).contains(&b1[i]) {
+            groups[b1[i] as usize].push(e);
+        }
+        if (0..11).contains(&b2[i]) {
+            groups[12 + b2[i] as usize].push(e);
+        }
+    }
+    let out: Vec<i64> = (0..ax.len())
+        .map(|p| {
+            let c = ch[p];
+            if !(0..23).contains(&c) {
+                return -1;
+            }
+            let cand = &groups[c as usize];
+            let mut best_j = usize::MAX;
+            let mut best = f32::INFINITY;
+            for (j, &e) in cand.iter().enumerate() {
+                let dx = rx[e] - ax[p];
+                let dy = ry[e] - ay[p];
+                let d = dx * dx + dy * dy;
+                // numpy argmin: the first NaN wins, otherwise the first strict minimum
+                if d.is_nan() {
+                    best_j = j;
+                    best = d;
+                    break;
+                }
+                if best_j == usize::MAX || d < best {
+                    best_j = j;
+                    best = d;
+                }
+            }
+            if best_j == usize::MAX || !((best as f64) < 1e20) { -1 } else { cand[best_j] as i64 }
+        })
+        .collect();
+    Ok(out.into_pyarray(py))
+}
+
+/// Exact nearest neighbour (`simurg.signal.normalization.fit._closest_slow`): for each sweep point (x, y) the
+/// index of the reference point with the smallest float32 squared distance `(x-xr)²+(y-yr)²`; first minimum wins,
+/// a NaN distance wins as in `numpy.argmin`. `xref` must not be empty.
+#[pyfunction]
+fn closest_exact<'py>(
+    py: Python<'py>,
+    xref: PyReadonlyArray1<'py, f32>,
+    yref: PyReadonlyArray1<'py, f32>,
+    x: PyReadonlyArray1<'py, f32>,
+    y: PyReadonlyArray1<'py, f32>,
+) -> PyResult<Bound<'py, PyArray1<i64>>> {
+    let (xr, yr, xs, ys) = (xref.as_slice()?, yref.as_slice()?, x.as_slice()?, y.as_slice()?);
+    if xr.is_empty() || xr.len() != yr.len() || xs.len() != ys.len() {
+        return Err(PyValueError::new_err("closest_exact: empty reference or length mismatch"));
+    }
+    let out: Vec<i64> = xs
+        .iter()
+        .zip(ys)
+        .map(|(&px, &py_)| {
+            let mut best_j = 0usize;
+            let mut best = f32::INFINITY;
+            for (j, (&rx, &ry)) in xr.iter().zip(yr).enumerate() {
+                let dx = px - rx;
+                let dy = py_ - ry;
+                let d = dx * dx + dy * dy;
+                if d.is_nan() {
+                    return j as i64;
+                }
+                if j == 0 || d < best {
+                    best_j = j;
+                    best = d;
+                }
+            }
+            best_j as i64
+        })
+        .collect();
+    Ok(out.into_pyarray(py))
+}
+
+#[pymodule]
+fn simurg_core(m: &Bound<'_, PyModule>) -> PyResult<()> {
+    m.add("__version__", env!("CARGO_PKG_VERSION"))?;
+    m.add_function(wrap_pyfunction!(exp, m)?)?;
+    m.add_function(wrap_pyfunction!(log10, m)?)?;
+    m.add_function(wrap_pyfunction!(log2, m)?)?;
+    m.add_function(wrap_pyfunction!(pow, m)?)?;
+    m.add_function(wrap_pyfunction!(closest_binned, m)?)?;
+    m.add_function(wrap_pyfunction!(closest_exact, m)?)?;
+    m.add_function(wrap_pyfunction!(bpm::bpm_locus_entries, m)?)?;
+    m.add_function(wrap_pyfunction!(gencall::call_genotypes, m)?)?;
+    m.add_function(wrap_pyfunction!(gencall::baf_lrr, m)?)?;
+    Ok(())
+}

scripts/build_annotation_bundle.sh

@@ -0,0 +1,32 @@
+#!/usr/bin/env bash
+# Build the cyto annotation bundle (spec CYTO-036, IN-054) from public UCSC tables:
+#   cytoBand (ISCN bands) and ncbiRefSeq (RefSeq transcripts with gene symbols, from NCBI RefSeq).
+# Output: simurg-cyto-annot-<build>-<date>-<sha12>.zip with cytoBand.txt.gz, genes.tsv.gz and manifest.json.
+#   bash scripts/build_annotation_bundle.sh GRCh38|GRCh37 [OUT_DIR]
+set -euo pipefail
+BUILD=${1:?usage: build_annotation_bundle.sh GRCh38|GRCh37 [OUT_DIR]}
+OUT=${2:-data/annotation}
+case $BUILD in GRCh38) UCSC=hg38;; GRCh37) UCSC=hg19;; *) echo "unknown build $BUILD"; exit 2;; esac
+BASE=https://hgdownload.soe.ucsc.edu/goldenPath/$UCSC/database
+W=$(mktemp -d); trap 'rm -rf "$W"' EXIT
+mkdir -p "$OUT"
+for t in cytoBand ncbiRefSeq; do curl -sSfL -o "$W/$t.txt.gz" "$BASE/$t.txt.gz"; done
+curl -sSfL -o "$W/ncbiRefSeq.sql" "$BASE/ncbiRefSeq.sql" || true
+# ncbiRefSeq columns: bin name chrom strand txStart txEnd cdsStart cdsEnd exonCount exonStarts exonEnds score name2 ...
+zcat "$W/ncbiRefSeq.txt.gz" | awk -F'\t' 'BEGIN{OFS="\t"; print "chrom","start","end","gene","transcript"}
+  { c=$3; sub(/^chr/,"",c); if (c ~ /_/) next; print c, $5+1, $6, $13, $2 }' | gzip -9 > "$W/genes.tsv.gz"
+DATE=$(date -u +%Y%m%d)
+python3 - "$W" "$BUILD" "$UCSC" "$DATE" <<'PY'
+import hashlib, json, sys, pathlib
+w, build, ucsc, date = pathlib.Path(sys.argv[1]), *sys.argv[2:]
+files = {n: hashlib.sha256((w / n).read_bytes()).hexdigest() for n in ("cytoBand.txt.gz", "genes.tsv.gz")}
+m = {"schema": "simurg.cyto_annotation/1", "build": build, "created": date,
+     "sources": [{"name": "UCSC cytoBand", "version": ucsc, "release_date": None},
+                 {"name": "RefSeq (UCSC ncbiRefSeq)", "version": ucsc, "release_date": None}],
+     "files": files}
+(w / "manifest.json").write_text(json.dumps(m, indent=1))
+PY
+( cd "$W" && zip -q -X bundle.zip manifest.json cytoBand.txt.gz genes.tsv.gz )
+SHA=$(sha256sum "$W/bundle.zip" | cut -c1-12)
+cp "$W/bundle.zip" "$OUT/simurg-cyto-annot-$BUILD-$DATE-$SHA.zip"
+ls -la "$OUT/simurg-cyto-annot-$BUILD-$DATE-$SHA.zip"

scripts/build_dist.sh

@@ -0,0 +1,22 @@
+#!/usr/bin/env bash
+# Build the release artifacts into dist/:
+#   simurg-<v>-py3-none-any.whl + simurg-<v>.tar.gz           (pure Python)
+#   simurg_core-<v>-cp312-abi3-manylinux_2_28_{aarch64,x86_64}.whl   (compiled kernels, Python >= 3.12)
+# Portable Linux wheels are linked against glibc 2.28 with `maturin --zig` (cross-compiles the other architecture).
+# Needs: uv, a Rust toolchain (rustup), ~60 MB for zig on first use.
+#   bash scripts/build_dist.sh [aarch64|x86_64|all]      (default: all)
+set -euo pipefail
+cd "$(dirname "$0")/.."
+export PATH="$HOME/.cargo/bin:$PATH"
+WHICH=${1:-all}
+mkdir -p dist
+uv build -o dist                                   # sdist + pure wheel of simurg
+MATURIN="uvx --from maturin[zig]>=1.7,<2 maturin"
+for arch in aarch64 x86_64; do
+  [[ $WHICH == all || $WHICH == "$arch" ]] || continue
+  rustup target add "$arch-unknown-linux-gnu" > /dev/null
+  $MATURIN build --release --zig --compatibility manylinux_2_28 --target "$arch-unknown-linux-gnu" \
+    -m rust/simurg-core/Cargo.toml -o dist
+done
+( cd dist && sha256sum ./*.whl ./*.tar.gz > SHA256SUMS )
+ls -la dist

scripts/cloud/tasks/cnv.sh

@@ -0,0 +1,60 @@
+#!/bin/bash
+# v2 CN/LOH validation on GSE165845 (360 GSA arrays, 120 1000 Genomes individuals x 3): simurg cn call vs the NYGC
+# 30x structural-variant callset (1KGP_3202 GATK-SV/svtools freeze V3) and between replicates.
+# Args: <unused data dir>  <results dir>. Run with OMNI=0.
+set -uo pipefail
+OUT=$2
+W=/work/cnv; R=$OUT/cnv; mkdir -p $W $R
+N=$(nproc)
+export DEBIAN_FRONTEND=noninteractive
+apt-get install -y -q bcftools tabix gcc > /dev/null
+curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal > /dev/null 2>&1
+export PATH="$HOME/.cargo/bin:$PATH"
+VIRTUAL_ENV=$PWD/.venv uv pip install -q ./rust/simurg-core || echo "!! kernels not built (pure Python)"
+
+SV=https://ftp.1000genomes.ebi.ac.uk/vol1/ftp/data_collections/1000G_2504_high_coverage/working/20210124.SV_Illumina_Integration/1KGP_3202.gatksv_svtools_novelins.freeze_V3.wAF.vcf.gz
+( curl -sSfL --retry 5 -o $W/sv.vcf.gz $SV && curl -sSfL --retry 5 -o $W/sv.vcf.gz.tbi $SV.tbi \
+  && echo "SV callset $(du -h $W/sv.vcf.gz | cut -f1) at $(date -u +%T)" ) > $W/sv.log 2>&1 &
+SV_PID=$!
+( bash scripts/fetch_data.sh p6 ) > $W/fasta.log 2>&1 &
+FASTA_PID=$!
+
+echo "=== data $(date -u +%T) ==="
+GSA=https://webdata.illumina.com/downloads/productfiles/global-screening-array/v1-0
+mkdir -p $W/gsa && cd $W/gsa
+for z in infinium-global-screening-array-24-v1-0-c2-manifest-file-bpm-build38.zip \
+         infinium-global-screening-array-v1-0-c1-cluster-file.zip; do
+  curl -sSfL --retry 5 -O "$GSA/$z" && unzip -oq $z
+done
+cd - > /dev/null
+BPM=$W/gsa/GSA-24v1-0_C2.bpm; EGT=$W/gsa/GSA-24v1-0_C1_ClusterFile.egt
+.venv/bin/python scripts/p8/prepare.py --tsv scripts/p8/gse165845_samples.tsv --dl $W/geo --out $W/idats --workers 24 \
+  | tail -2 || exit 1
+
+echo "=== genotype $(date -u +%T) ==="
+.venv/bin/simurg genotype --bpm $BPM --egt $EGT --idats $W/idats --out $W/gt --threads $N --quiet
+cp $W/idats/samples_map.tsv $W/gt/sample_summary.tsv $R/
+
+echo "=== model $(date -u +%T) ==="
+wait $FASTA_PID; tail -1 $W/fasta.log
+/usr/bin/time -v .venv/bin/simurg cn build-model --bpm $BPM --fasta data/p6/GRCh38.primary.fa --egt $EGT \
+  --reference-gtcs $W/gt --max-reference 120 --out $W/gsa1.simurg-cyto 2> $R/model_time.txt | grep -v "GC windows"
+grep -E "Elapsed|Maximum resident" $R/model_time.txt
+
+echo "=== cn call $(date -u +%T) ==="
+/usr/bin/time -v .venv/bin/simurg cn call --bpm $BPM --model $W/gsa1.simurg-cyto --gtcs $W/gt \
+  --fasta data/p6/GRCh38.primary.fa --out $W/cn --threads $N --quiet 2> $R/call_time.txt
+grep -E "Elapsed|Maximum resident" $R/call_time.txt
+.venv/bin/simurg cn annotate --vcfs $W/cn --quiet || echo "annotate failed"
+cp $W/cn/cn_sample_summary.tsv $W/cn/cn_call_manifest.json $R/
+
+echo "=== truth $(date -u +%T) ==="
+wait $SV_PID; cat $W/sv.log
+cut -f3 $W/idats/samples_map.tsv | tail -n +2 | sort -u > $W/inds.txt
+bcftools view -S $W/inds.txt --force-samples -i 'INFO/SVTYPE="DEL" || INFO/SVTYPE="DUP"' -Ou $W/sv.vcf.gz \
+  | bcftools query -i 'GT="alt"' -f '[%CHROM\t%POS\t%INFO/END\t%INFO/SVTYPE\t%SAMPLE\n]' > $W/truth.tsv
+wc -l < $W/truth.tsv
+.venv/bin/python scripts/v2/cn_validate_truth.py --calls $W/cn --map $W/idats/samples_map.tsv --truth $W/truth.tsv \
+  --bpm $BPM --out $R/validation
+gzip -c $W/truth.tsv > $R/truth.tsv.gz
+echo "=== task done $(date -u +%T) ==="

scripts/cloud/tasks/meth.sh

@@ -0,0 +1,33 @@
+#!/bin/bash
+# v2 methylation QC: numerics vs the sesame oracle (R, MIT) on the public minfiDataEPIC set (3 x NA12878, EPIC v1).
+# Args: <unused data dir>  <results dir>. Run with OMNI=0.
+set -uo pipefail
+OUT=$2
+R=$OUT/meth; W=/work/meth; mkdir -p $R $W
+ARCH=$(uname -m)
+echo "=== data $(date -u +%T) ==="
+cd $W
+curl -sSfL -O https://bioconductor.org/packages/release/data/experiment/src/contrib/minfiDataEPIC_1.38.0.tar.gz \
+  && tar -xzf minfiDataEPIC_1.38.0.tar.gz minfiDataEPIC/inst/extdata
+curl -sSfL -O https://webdata.illumina.com/downloads/productfiles/methylationEPIC/infinium-methylationepic-v-1-0-b5-manifest-file-csv.zip \
+  && unzip -oq infinium-methylationepic-v-1-0-b5-manifest-file-csv.zip
+cd - > /dev/null
+MAN=$W/infinium-methylationepic-v-1-0-b5-manifest-file.csv
+E=$W/minfiDataEPIC/inst/extdata
+PREFIXES="$E/200144450018/200144450018_R04C01 $E/200144450019/200144450019_R07C01 $E/200144450021/200144450021_R05C01"
+.venv/bin/simurg methyl convert-manifest --manifest $MAN --out $W/mft --quiet
+
+echo "=== sesame (conda) $(date -u +%T) ==="
+curl -sSfL -o $W/mf.sh "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-$ARCH.sh" \
+  && bash $W/mf.sh -b -p $W/conda > /dev/null
+$W/conda/bin/mamba create -y -q -p $W/renv -c conda-forge -c bioconda r-base bioconductor-sesame > $R/conda.log 2>&1 \
+  || { echo "!! sesame install failed"; tail -30 $R/conda.log; }
+$W/renv/bin/Rscript scripts/v2/sesame_oracle.R $W/mft/probes.tsv $W/mft/controls.tsv $W/sesame $PREFIXES \
+  > $R/sesame.log 2>&1 || echo "!! sesame run failed"
+tail -40 $R/sesame.log
+
+echo "=== compare $(date -u +%T) ==="
+.venv/bin/python scripts/v2/methyl_oracle_compare.py --manifest $MAN --sesame $W/sesame --out $R/compare.json $PREFIXES
+.venv/bin/simurg methyl qc --manifest $MAN --idats $E --sheet $E/SampleSheet.csv --out $W/qc --threads 0 --quiet
+cp $W/qc/methyl_qc_summary.tsv $W/qc/controls_qc.tsv $R/
+echo "=== task done $(date -u +%T) ==="

scripts/cloud/tasks/p10.sh

@@ -0,0 +1,125 @@
+#!/bin/bash
+# Phase P10 acceptance on a fresh Ubuntu 24.04 VM (plan §18: "clean install + tutorial works on fresh VM"):
+#   1. build the release artifacts (scripts/build_dist.sh: sdist, pure wheel, manylinux_2_28 kernel wheels for both
+#      architectures);
+#   2. install them with plain pip into new venvs of the system Python (with and without the kernels) and run the
+#      unit tests against the installed package;
+#   3. fetch the public demo data and run the tutorial (examples/gsa_demo.sh, 6 samples) with each install;
+#   4. build the Docker image and the conda packages, run genotype + gtc-to-vcf with them;
+#   5. content digests of every output (scripts/p10/digest.py) must agree across all installs.
+# Args: <unused data dir>  <results dir>. Run with OMNI=0.
+set -uo pipefail
+OUT=$2
+R=$OUT/p10; W=/work/p10; mkdir -p $R $W
+N=$(nproc)
+ARCH=$(uname -m)
+declare -A STEP
+step() { STEP[$1]=$2; echo "--- $1: $2"; }
+
+export DEBIAN_FRONTEND=noninteractive
+apt-get install -y -q gcc docker.io bsdextrautils > /dev/null
+curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal > /dev/null 2>&1
+export PATH="$HOME/.cargo/bin:$PATH"
+
+echo "=== 1. build artifacts $(date -u +%T) ==="
+if bash scripts/build_dist.sh all > $R/build_dist.log 2>&1; then step build ok; else step build FAILED; tail -30 $R/build_dist.log; fi
+cp dist/*.whl dist/*.tar.gz dist/SHA256SUMS $R/ 2>/dev/null
+ls -la dist
+
+echo "=== 2. pip install into fresh venvs $(date -u +%T) ==="
+PY=/usr/bin/python3
+$PY --version
+CORE=$(ls dist/simurg_core-*manylinux_2_28_$ARCH.whl 2>/dev/null | head -1)
+$PY -m venv $W/fresh && $W/fresh/bin/pip install -q dist/simurg-*-py3-none-any.whl "$CORE" \
+  && step pip_install ok || step pip_install FAILED
+$PY -m venv $W/pure && $W/pure/bin/pip install -q dist/simurg-*-py3-none-any.whl \
+  && step pip_install_pure ok || step pip_install_pure FAILED
+$W/fresh/bin/python -c "from simurg._core import backend; print('fresh:', backend())"
+$W/pure/bin/python -c "from simurg._core import backend; print('pure:', backend())"
+$W/fresh/bin/simurg --version
+$W/fresh/bin/pip install -q pytest hypothesis
+# tests import the installed package (the checkout keeps its sources under src/, not on sys.path)
+( cd $W && cp -r /work/repo/tests /work/repo/docs . && $W/fresh/bin/python -m pytest tests/unit -q -p no:cacheprovider \
+    -k "not selftest_in_browser" > $R/unit_tests_installed.txt 2>&1 ) && step unit_tests_installed ok \
+  || { step unit_tests_installed FAILED; tail -20 $R/unit_tests_installed.txt; }
+tail -3 $R/unit_tests_installed.txt
+
+echo "=== 3. tutorial $(date -u +%T) ==="
+bash scripts/fetch_data.sh p0 > $W/fetch_p0.log 2>&1 && bash scripts/fetch_data.sh p6 > $W/fetch_p6.log 2>&1 \
+  && step fetch_data ok || { step fetch_data FAILED; tail -5 $W/fetch_p0.log $W/fetch_p6.log; }
+for v in fresh pure; do
+  t0=$SECONDS
+  SIMURG=$W/$v/bin/simurg THREADS=$N bash examples/gsa_demo.sh $W/tut_$v 6 > $R/tutorial_$v.log 2>&1 \
+    && step tutorial_$v "ok ($((SECONDS - t0)) s)" || { step tutorial_$v FAILED; tail -20 $R/tutorial_$v.log; }
+  $W/fresh/bin/python scripts/p10/digest.py $W/tut_$v/gtc --out $R/digest_gtc_$v.json
+  $W/fresh/bin/python scripts/p10/digest.py $W/tut_$v/vcf --out $R/digest_vcf_$v.json
+done
+
+echo "=== 4a. docker $(date -u +%T) ==="
+if [[ "${SKIP_DOCKER:-0}" != "1" ]]; then
+D=data/p0/gsa_v4_pgx
+BPM=$D/GSA-PGx-48v4-0_20079540_E2_bpm/GSA-PGx-48v4-0_20079540_E2.bpm
+CSV=$D/GSA-PGx-48v4-0_20079540_E2_csv/GSA-PGx-48v4-0_20079540_E2.csv
+EGT=$D/GSA-PGx-48v4-0_20079540_E2_ClusterFile/GSA-PGx-48v4-0_20079540_E2_ClusterFile.egt
+FASTA=data/p6/GRCh38.primary.fa
+if docker build -q -t simurg:test . > $R/docker_build.log 2>&1; then
+  step docker_build ok
+  docker image ls simurg:test --format '{{.Size}}' | sed 's/^/image size: /'
+  DR="docker run --rm --user 0:0 -v /work:/work -w /work/repo simurg:test"
+  $DR --version
+  $DR genotype --bpm $BPM --egt $EGT --csv $CSV --idats $W/tut_fresh/idats --out $W/docker/gtc \
+      --qc-profile genotyping --threads $N --quiet --force \
+    && $DR gtc-to-vcf --bpm $BPM --csv $CSV --fasta $FASTA --gtcs $W/docker/gtc --out $W/docker/vcf --threads $N \
+      --quiet --force && step docker_run ok || step docker_run FAILED
+  $W/fresh/bin/python scripts/p10/digest.py $W/docker/gtc --out $R/digest_gtc_docker.json
+  $W/fresh/bin/python scripts/p10/digest.py $W/docker/vcf --out $R/digest_vcf_docker.json
+else
+  step docker_build FAILED; tail -30 $R/docker_build.log
+fi
+fi
+
+echo "=== 4b. conda $(date -u +%T) ==="
+if [[ "${SKIP_CONDA:-0}" != "1" ]]; then
+curl -sSfL -o $W/miniforge.sh "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-$ARCH.sh" \
+  && bash $W/miniforge.sh -b -p $W/conda > /dev/null
+C=$W/conda/bin
+if $C/conda install -y -q -n base conda-build > $R/conda.log 2>&1 \
+   && $C/conda build -c conda-forge --output-folder $W/condabld packaging/conda/simurg-core packaging/conda/simurg \
+      >> $R/conda.log 2>&1; then
+  step conda_build ok
+  $C/conda create -y -q -p $W/cenv -c $W/condabld -c conda-forge simurg simurg-core >> $R/conda.log 2>&1 \
+    && $W/cenv/bin/simurg --version \
+    && $W/cenv/bin/python -c "from simurg._core import backend; print('conda:', backend())" \
+    && $W/cenv/bin/simurg genotype --bpm $BPM --egt $EGT --idats $W/tut_fresh/idats --out $W/conda_run/gtc \
+         --threads $N --quiet --force \
+    && step conda_run ok || { step conda_run FAILED; tail -20 $R/conda.log; }
+  $W/fresh/bin/python scripts/p10/digest.py $W/conda_run/gtc --out $R/digest_gtc_conda.json
+else
+  step conda_build FAILED; tail -40 $R/conda.log
+fi
+fi
+
+echo "=== 5. digests $(date -u +%T) ==="
+$W/fresh/bin/python - "$R" <<'PY' | tee $R/agreement.txt
+import json, sys
+from pathlib import Path
+r = Path(sys.argv[1])
+for kind in ("gtc", "vcf"):
+    runs = {p.stem.split("_", 2)[2]: json.loads(p.read_text())[kind] for p in sorted(r.glob(f"digest_{kind}_*.json"))}
+    if not runs:
+        continue
+    ref = runs.get("fresh", {})
+    if not ref:
+        print(f"{kind}: no reference (the pip-installed tutorial produced no outputs)")
+        continue
+    for name, d in runs.items():
+        if name != "fresh":
+            print(f"{kind}: {name} vs fresh: {'IDENTICAL' if d == ref else 'DIFFERENT'} ({len(d)} vs {len(ref)} files)")
+PY
+{
+  echo "{"; echo "  \"arch\": \"$ARCH\","
+  for k in "${!STEP[@]}"; do echo "  \"$k\": \"${STEP[$k]}\","; done
+  echo "  \"python\": \"$($PY --version)\""; echo "}"
+} > $R/steps.json
+cat $R/steps.json
+echo "=== task done $(date -u +%T) ==="

scripts/cloud/tasks/p10t.sh

@@ -0,0 +1,3 @@
+#!/bin/bash
+# P10 acceptance without the Docker and conda parts (already verified): build, pip installs, tutorial, digests.
+SKIP_DOCKER=1 SKIP_CONDA=1 bash "$(dirname "$0")/p10.sh" "$@"

scripts/cloud/tasks/p9.sh

@@ -0,0 +1,54 @@
+#!/bin/bash
+# Phase P9 performance gate on a 32-vCPU Graviton instance, GSE165845 GSA arrays (360 samples):
+#   1. build the compiled kernels (rust/simurg-core) into the venv;
+#   2. thread scaling 1 → 32 with the kernels (bench/scaling.py, per-sample phase from the run log);
+#   3. pure-Python reference at 1 and 32 threads on a subset, and byte-identity of all 360 GTCs (kernels vs pure).
+# Args: <unused data dir>  <results dir>. Run with OMNI=0.
+set -uo pipefail
+OUT=$2
+W=/work/p9; R=$OUT/p9; mkdir -p $W $R
+N=$(nproc)
+
+echo "=== build simurg-core $(date -u +%T) ==="
+DEBIAN_FRONTEND=noninteractive apt-get install -y -q gcc > /dev/null  # Rust links through the C toolchain
+curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal > /dev/null 2>&1
+export PATH="$HOME/.cargo/bin:$PATH"
+rustc --version
+VIRTUAL_ENV=$PWD/.venv uv pip install -q ./rust/simurg-core || { echo "!! simurg-core build failed"; exit 1; }
+.venv/bin/python -c "from simurg._core import backend; print('backend', backend())"
+.venv/bin/python -m pytest tests/unit/test_core_kernels.py -q -p no:cacheprovider 2>&1 | tail -2
+
+echo "=== data $(date -u +%T) ==="
+GSA=https://webdata.illumina.com/downloads/productfiles/global-screening-array/v1-0
+mkdir -p $W/gsa && cd $W/gsa
+for z in infinium-global-screening-array-24-v1-0-c2-manifest-file-bpm-build38.zip \
+         infinium-global-screening-array-v1-0-c1-cluster-file.zip; do
+  curl -sSfL --retry 5 -O "$GSA/$z" && unzip -oq $z
+done
+cd - > /dev/null
+BPM=$W/gsa/GSA-24v1-0_C2.bpm; EGT=$W/gsa/GSA-24v1-0_C1_ClusterFile.egt
+.venv/bin/python scripts/p8/prepare.py --tsv scripts/p8/gse165845_samples.tsv --dl $W/geo --out $W/idats --workers 24 \
+  || exit 1
+
+echo "=== scaling with kernels $(date -u +%T) ==="
+.venv/bin/python bench/scaling.py --bpm $BPM --egt $EGT --idats $W/idats --out $W/rust \
+  --threads 1,2,4,8,16,$N 2>&1 | grep -v WARNING
+cp $W/rust/scaling.json $R/scaling_rust.json
+
+echo "=== pure Python reference $(date -u +%T) ==="
+.venv/bin/python bench/scaling.py --bpm $BPM --egt $EGT --idats $W/idats --out $W/python --threads 1 --samples 64 \
+  --pure-python 2>&1 | grep -v WARNING
+cp $W/python/scaling.json $R/scaling_python_t1.json
+.venv/bin/python bench/scaling.py --bpm $BPM --egt $EGT --idats $W/idats --out $W/python_all --threads $N \
+  --pure-python 2>&1 | grep -v WARNING
+cp $W/python_all/scaling.json $R/scaling_python_t$N.json
+
+echo "=== identity: 360 GTCs, kernels vs pure Python $(date -u +%T) ==="
+n=0; d=0
+for f in $(cd $W/rust/t$N && find samples -name '*.gtc'); do
+  n=$((n+1)); cmp -s $W/rust/t$N/$f $W/python_all/t$N/$f || { d=$((d+1)); echo "differs: $f"; }
+done
+s=$(cmp -s $W/rust/t$N/sample_summary.tsv $W/python_all/t$N/sample_summary.tsv && echo identical || echo DIFFERS)
+echo "{\"gtcs\": $n, \"differ\": $d, \"sample_summary\": \"$s\"}" | tee $R/identity.json
+lscpu | grep -i "model name\|^CPU(s)" > $R/cpu.txt; free -g >> $R/cpu.txt
+echo "=== task done $(date -u +%T) ==="

scripts/cloud/tasks/p9b.sh

@@ -0,0 +1,29 @@
+#!/bin/bash
+# Phase P9 follow-up: thread scaling with the GTC outputs on a RAM disk (/dev/shm), to separate compute scaling
+# from the gp3 disk's write throughput (125 MB/s baseline; one GSA GTC is ~13 MB). Run with OMNI=0.
+set -uo pipefail
+OUT=$2
+W=/work/p9; R=$OUT/p9b; mkdir -p $W $R
+N=$(nproc)
+DEBIAN_FRONTEND=noninteractive apt-get install -y -q gcc > /dev/null
+curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal > /dev/null 2>&1
+export PATH="$HOME/.cargo/bin:$PATH"
+VIRTUAL_ENV=$PWD/.venv uv pip install -q ./rust/simurg-core || { echo "!! simurg-core build failed"; exit 1; }
+.venv/bin/python -c "from simurg._core import backend; print('backend', backend())"
+GSA=https://webdata.illumina.com/downloads/productfiles/global-screening-array/v1-0
+mkdir -p $W/gsa && cd $W/gsa
+for z in infinium-global-screening-array-24-v1-0-c2-manifest-file-bpm-build38.zip \
+         infinium-global-screening-array-v1-0-c1-cluster-file.zip; do
+  curl -sSfL --retry 5 -O "$GSA/$z" && unzip -oq $z
+done
+cd - > /dev/null
+BPM=$W/gsa/GSA-24v1-0_C2.bpm; EGT=$W/gsa/GSA-24v1-0_C1_ClusterFile.egt
+.venv/bin/python scripts/p8/prepare.py --tsv scripts/p8/gse165845_samples.tsv --dl /dev/shm/geo --out /dev/shm/idats \
+  --workers 24 2>&1 | tail -2 || exit 1
+df -h /dev/shm | tail -1
+echo "=== scaling, outputs on /dev/shm $(date -u +%T) ==="
+.venv/bin/python bench/scaling.py --bpm $BPM --egt $EGT --idats /dev/shm/idats --out $W/shm --work /dev/shm/out \
+  --clean --threads 1,4,8,16,24,$N 2>&1 | grep -v WARNING
+cp $W/shm/scaling.json $R/scaling_rust_shm.json
+lscpu | grep -i "model name\|^CPU(s)" > $R/cpu.txt
+echo "=== task done $(date -u +%T) ==="

scripts/cloud/trial.sh

@@ -14,6 +14,8 @@ set -euo pipefail
 cd "$(dirname "$0")/../.."
 REGION=${REGION:-us-east-1}
 TYPE=${TYPE:-m7g.4xlarge}
+# Graviton families carry a "g" after the generation number (m7g, c7gn, r8g): arm64 image; everything else amd64
+if [[ $TYPE =~ ^[a-z]+[0-9]+g ]]; then ARCH=${ARCH:-arm64}; else ARCH=${ARCH:-amd64}; fi
 MAX_PRICE=${MAX_PRICE:-0.30}        # USD/hour spot cap (on-demand is 0.6528)
 MAX_MINUTES=${MAX_MINUTES:-360}     # hard cap: 6 hours
 DISK_GB=${DISK_GB:-250}
@@ -59,7 +61,7 @@ EOF
 launch)
   running=$($AWS ec2 describe-instances --filters Name=tag:Project,Values=$TAG \
     Name=instance-state-name,Values=pending,running --query 'length(Reservations[].Instances[])')
-  [[ "$running" == "0" ]] || { echo "a trial instance is already running; refusing to start another"; exit 1; }
+  [[ "$running" == "0" || "${PARALLEL:-0}" == "1" ]] || { echo "a trial instance is already running; refusing to start another (PARALLEL=1 overrides)"; exit 1; }
   RUN=run-$(date -u +%Y%m%dT%H%M%SZ)
   echo "-> packing repository at $(git rev-parse --short HEAD) -> s3://$BUCKET/$RUN/"
   git archive --format=tar.gz -o /tmp/simurg-repo.tar.gz HEAD
@@ -69,7 +71,7 @@ launch)
   sed -e "s/__BUCKET__/$BUCKET/" -e "s/__RUN__/$RUN/" -e "s/__MAX_MINUTES__/$MAX_MINUTES/" -e "s/__TASK__/$TASK/g" \
     -e "s/__OMNI__/$OMNI/" \
     scripts/cloud/userdata.sh > /tmp/simurg-userdata.sh
-  AMI=$($AWS ssm get-parameter --name /aws/service/canonical/ubuntu/server/24.04/stable/current/arm64/hvm/ebs-gp3/ami-id \
+  AMI=$($AWS ssm get-parameter --name /aws/service/canonical/ubuntu/server/24.04/stable/current/$ARCH/hvm/ebs-gp3/ami-id \
     --query Parameter.Value --output text)
   echo "-> task: $TASK"
   echo "-> launching $TYPE spot (cap \$$MAX_PRICE/h, hard limit ${MAX_MINUTES} min, ${DISK_GB} GB disk deleted on exit)"

scripts/p10/digest.py

@@ -0,0 +1,86 @@
+#!/usr/bin/env python3
+"""Content digests of SimurgArray outputs, for comparing runs across machines, architectures and installs (P10).
+
+GTC: every numeric field and sample statistic, NaNs canonicalised (their sign/payload is platform-specific, K-014);
+free-text metadata (dates, software version) is left out. VCF(.gz): all data lines, header lines excluded.
+TSV: the listed columns of sample_summary.tsv.
+
+    python scripts/p10/digest.py RUN_DIR [--out digest.json]
+"""
+
+from __future__ import annotations
+
+import argparse
+import gzip
+import hashlib
+import json
+import sys
+from pathlib import Path
+
+import numpy as np
+
+from simurg.formats.gtc import read_gtc
+
+GTC_FIELDS = ("normalization_transforms", "controls_x", "controls_y", "raw_x", "raw_y", "genotypes", "base_calls",
+              "genotype_scores", "b_allele_freqs", "logr_ratios", "percentiles_x", "percentiles_y", "sample_data",
+              "call_rate", "gender", "logr_dev", "gc10")  # fmt: skip
+SUMMARY_COLUMNS = ("sample_id", "status", "autosomal_call_rate", "call_rate", "logr_dev", "sex_est", "gc10",
+                   "gc50", "n_calls", "n_nocalls", "het_rate", "contamination", "functional_status", "control_status",
+                   "flags")  # fmt: skip
+
+
+def _canon(a: np.ndarray) -> bytes:
+    if a.dtype.names:  # structured: canonicalise float members
+        a = a.copy()
+        for n in a.dtype.names:
+            if np.issubdtype(a.dtype[n], np.floating):
+                v = a[n]
+                v[np.isnan(v)] = np.nan
+        return a.tobytes()
+    if np.issubdtype(a.dtype, np.floating):
+        a = np.where(np.isnan(a), np.array(np.nan, a.dtype), a)
+    return np.ascontiguousarray(a).tobytes()
+
+
+def gtc_digest(path: Path) -> dict[str, str]:
+    g = read_gtc(path)
+    out = {}
+    for f in GTC_FIELDS:
+        v = getattr(g, f)
+        b = _canon(np.asarray(v)) if not isinstance(v, bytes) else v
+        out[f] = hashlib.sha256(b).hexdigest()[:16]
+    return out
+
+
+def vcf_digest(path: Path) -> str:
+    h = hashlib.sha256()
+    with gzip.open(path, "rt") if path.suffix == ".gz" else open(path) as fh:
+        for line in fh:
+            if not line.startswith("#"):
+                h.update(line.encode())
+    return h.hexdigest()[:16]
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    ap.add_argument("run", type=Path)
+    ap.add_argument("--out", type=Path)
+    a = ap.parse_args()
+    d: dict[str, object] = {}
+    d["gtc"] = {p.stem: gtc_digest(p) for p in sorted(a.run.rglob("*.gtc"))}
+    d["vcf"] = {p.name: vcf_digest(p) for p in sorted(a.run.rglob("*.vcf.gz"))}
+    summ = {}
+    for p in sorted(a.run.rglob("sample_summary.tsv")):
+        rows = [ln.rstrip("\n").split("\t") for ln in p.open()]
+        idx = [rows[0].index(c) for c in SUMMARY_COLUMNS if c in rows[0]]
+        summ[str(p.relative_to(a.run))] = hashlib.sha256(
+            "\n".join("\t".join(r[i] for i in idx) for r in rows).encode()
+        ).hexdigest()[:16]
+    d["summary"] = summ
+    text = json.dumps(d, indent=1, sort_keys=True)
+    (a.out.write_text(text) if a.out else sys.stdout.write(text + "\n"))
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

scripts/pgx/build_cn_regions.py

@@ -0,0 +1,85 @@
+#!/usr/bin/env python3
+"""Build the PGx CN target-region BED (spec PCN-001) from UCSC ncbiRefSeq (GRCh38).
+
+    python scripts/pgx/build_cn_regions.py ncbiRefSeq.hg38.txt.gz OUT.bed [--version N]
+
+Regions (0-based half-open BED: chrom start0 end region gene), from the MANE/first-listed transcript:
+  CYP2D6.p5        2 kb 5' flank of CYP2D6 (upstream of exon 1; the gene is on the minus strand)
+  CYP2D6.intron.2  CYP2D6 intron 2
+  CYP2D6.exon.9    CYP2D6 exon 9
+  CYP2A6.exon.1    CYP2A6 exon 1 (± 500 bp)
+  CYP2A6.intron.7  CYP2A6 intron 7
+  GSTM1, GSTT1 (on chr22_KI270879v1_alt, IN-062), UGT2B17, SULT1A1, CYP2E1: whole gene.
+Exons and introns are numbered in transcript order (strand-aware). [DB: NCBI RefSeq via UCSC ncbiRefSeq]
+"""
+
+from __future__ import annotations
+
+import argparse
+import gzip
+from pathlib import Path
+
+TRANSCRIPT = {"CYP2D6": "NM_000106", "CYP2A6": "NM_000762", "GSTM1": "NM_000561", "GSTT1": "NM_000853",
+              "UGT2B17": "NM_001077", "SULT1A1": "NM_001055", "CYP2E1": "NM_000773"}  # fmt: skip
+FLANK = 2000
+PAD = 500
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    ap.add_argument("refseq", type=Path)
+    ap.add_argument("out", type=Path)
+    ap.add_argument("--version", type=int, default=1)
+    a = ap.parse_args()
+    tx: dict[str, tuple[str, str, list[int], list[int]]] = {}
+    with gzip.open(a.refseq, "rt") as fh:
+        for line in fh:
+            f = line.rstrip("\n").split("\t")
+            name, chrom, strand, gene = f[1], f[2], f[3], f[12]
+            if gene not in TRANSCRIPT or not name.startswith(TRANSCRIPT[gene] + "."):
+                continue
+            if "_" in chrom and not (gene == "GSTT1" and chrom == "chr22_KI270879v1_alt"):
+                continue  # primary assembly, except GSTT1 which is only on the alt contig
+            starts = [int(x) for x in f[9].rstrip(",").split(",")]
+            ends = [int(x) for x in f[10].rstrip(",").split(",")]
+            tx.setdefault(gene, (chrom, strand, starts, ends))
+    missing = set(TRANSCRIPT) - set(tx)
+    if missing:
+        raise SystemExit(f"transcripts not found: {sorted(missing)}")
+    rows: list[tuple[str, int, int, str, str]] = []
+
+    def exon(gene: str, k: int) -> tuple[int, int]:
+        _, strand, s, e = tx[gene]
+        i = k - 1 if strand == "+" else len(s) - k
+        return s[i], e[i]
+
+    def intron(gene: str, k: int) -> tuple[int, int]:
+        strand = tx[gene][1]
+        a_, b_ = exon(gene, k), exon(gene, k + 1)
+        return (a_[1], b_[0]) if strand == "+" else (b_[1], a_[0])
+
+    for gene in ("CYP2D6", "CYP2A6"):
+        chrom, strand, s, e = tx[gene]
+        if gene == "CYP2D6":
+            p5 = (e[-1], e[-1] + FLANK) if strand == "-" else (s[0] - FLANK, s[0])
+            rows += [(chrom, *p5, "CYP2D6.p5", gene), (chrom, *intron(gene, 2), "CYP2D6.intron.2", gene),
+                     (chrom, *exon(gene, 9), "CYP2D6.exon.9", gene)]  # fmt: skip
+        else:
+            x1 = exon(gene, 1)
+            rows += [(chrom, x1[0] - PAD, x1[1] + PAD, "CYP2A6.exon.1", gene),
+                     (chrom, *intron(gene, 7), "CYP2A6.intron.7", gene)]  # fmt: skip
+    for gene in ("GSTM1", "GSTT1", "UGT2B17", "SULT1A1", "CYP2E1"):
+        chrom, strand, s, e = tx[gene]
+        rows.append((chrom, s[0], e[-1], gene, gene))
+    order = {f"chr{i}": i for i in range(1, 23)}
+    rows.sort(key=lambda r: (order.get(r[0].split("_")[0], 99), r[0], r[1]))
+    with a.out.open("w") as fh:
+        fh.write(f"# simurg PGx CN regions v{a.version}, GRCh38, from UCSC ncbiRefSeq (PCN-001)\n")
+        for r in rows:
+            fh.write("\t".join(map(str, r)) + "\n")
+    print(a.out.read_text())
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

scripts/pgx/demo36_identity.tsv

@@ -0,0 +1,39 @@
+# GSA v4 PGx demo (idats_Demo_36) sample identities: best match among the 3202 1000 Genomes 30x individuals
+# by genotype concordance on 4647 chr22 SNPs with 1000G AF 0.05-0.95 (next best <= 0.91: relatives)
+sample_id	individual	concordance	second_best	second_concordance
+207895980019_R03C01	NA12875	0.9920	NA12865	0.7038
+207895980019_R03C02	NA12156	0.9933	NA10831	0.7021
+207895980019_R04C01	NA12156	0.9931	NA10831	0.7027
+207895980019_R05C01	NA12044	0.9931	NA10857	0.6969
+207895980019_R06C01	NA11992	0.9931	NA10860	0.6848
+207895980019_R07C01	NA06993	0.9940	NA06991	0.7209
+207895980019_R07C03	NA07019	0.9953	NA07056	0.7051
+207895980019_R08C01	NA12239	0.9938	NA10847	0.7022
+207895980019_R09C01	NA12716	0.9944	NA12707	0.6896
+207895980019_R09C03	NA06991	0.9931	NA06993	0.7215
+207895980019_R10C01	NA12878	0.9933	NA12892	0.7022
+207895980019_R11C01	NA12760	0.9948	NA12752	0.7171
+207895980019_R11C02	NA06985	0.9948	NA06991	0.6996
+207895980019_R12C01	NA11993	0.9961	NA10860	0.7168
+207895980019_R12C02	NA12003	0.9935	NA10838	0.6998
+207895980019_R13C03	NA06994	0.9948	NA07029	0.7114
+207895980019_R14C03	NA11993	0.9959	NA10860	0.7163
+207895980019_R15C02	NA12003	0.9931	NA10838	0.6993
+207895980035_R03C03	NA07029	0.9952	NA07000	0.7237
+207895980035_R04C03	NA12892	0.9957	NA12878	0.7005
+207895980035_R05C03	NA12248	0.9950	NA10835	0.7143
+207895980035_R06C03	NA10846	0.9920	NA12145	0.7056
+207895980035_R07C01	NA07000	0.9957	NA07029	0.7231
+207895980035_R07C03	NA12155	0.9935	NA10831	0.6903
+207895980035_R08C01	NA11832	0.9942	NA10855	0.7252
+207895980035_R08C03	NA06993	0.9940	NA06991	0.7205
+207895980035_R09C03	NA10838	0.9948	NA12004	0.7007
+207895980035_R10C01	NA12813	0.9937	NA07045	0.9103
+207895980035_R10C03	NA12249	0.9938	NA10835	0.6858
+207895980035_R11C03	NA12812	0.9942	NA12801	0.7061
+207895980035_R12C03	NA11881	0.9935	NA10859	0.7138
+207895980035_R13C01	NA11995	0.9953	NA10861	0.6857
+207895980035_R13C03	NA12248	0.9950	NA10835	0.7145
+207895980035_R14C01	NA12891	0.9942	NA12878	0.6998
+207895980035_R15C03	NA12043	0.9946	NA10857	0.7162
+207895980042_R15C02	NA11832	0.9942	NA10855	0.7253

scripts/pgx/derive_masks.py

@@ -0,0 +1,163 @@
+#!/usr/bin/env python3
+"""Derive a PGx probe mask from array-vs-WGS concordance (spec PGX-010) for the individuals of a GSA PGx batch.
+
+    python scripts/pgx/derive_masks.py --db DB.sqlite --vcfs VCF_DIR --identity demo36_identity.tsv \
+        --exclude-individuals getrm.txt --manifest NAME --out MASK.tsv [--min-concordance 0.95]
+
+Truth: the 1000 Genomes 30x phased panel (NYGC, 3202 samples), fetched per gene window over HTTPS with pysam (MIT;
+a script-only dependency: ``uv pip install pysam``). For every PGx-DB defining variant the array record(s) at that
+site are compared with the truth alt dosage; records whose concordance is below the threshold (≥ 5 comparisons)
+are masked by their probe names. Individuals listed in ``--exclude-individuals`` (the validation set) are not used,
+so the mask stays independent of the star-allele validation. The spec asks for ≥ 100 samples; this batch has fewer
+(recorded as K-029).
+"""
+
+from __future__ import annotations
+
+import argparse
+import csv
+import gzip
+import sqlite3
+from collections import defaultdict
+from pathlib import Path
+
+PANEL = (
+    "https://ftp.1000genomes.ebi.ac.uk/vol1/ftp/data_collections/1000G_2504_high_coverage/working/"
+    "20220422_3202_phased_SNV_INDEL_SV/1kGP_high_coverage_Illumina.{c}.filtered.SNV_INDEL_SV_phased_panel.vcf.gz"
+)
+
+
+def main() -> int:
+    import pysam
+
+    ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    ap.add_argument("--db", required=True, type=Path)
+    ap.add_argument("--vcfs", required=True, type=Path)
+    ap.add_argument("--identity", required=True, type=Path)
+    ap.add_argument("--exclude-individuals", type=Path)
+    ap.add_argument("--manifest", required=True)
+    ap.add_argument("--min-concordance", type=float, default=0.95)
+    ap.add_argument("--out", required=True, type=Path)
+    ap.add_argument("--cache", type=Path, default=Path("data/pgx/cache"), help="where pysam keeps remote .tbi files")
+    a = ap.parse_args()
+    ident = {}
+    with a.identity.open() as fh:
+        for r in csv.DictReader((ln for ln in fh if not ln.startswith("#")), delimiter="\t"):
+            ident[r["sample_id"]] = r["individual"]
+    excl = set(a.exclude_individuals.read_text().split()) if a.exclude_individuals else set()
+    use = {s: i for s, i in ident.items() if i not in excl}
+    con = sqlite3.connect(a.db)
+    # the highest population frequency among the alleles that carry each variant (CPIC frequencies)
+    vfreq: dict[tuple[str, int, str, str], float] = defaultdict(float)
+    for c, p, r, al, f in con.execute(
+        "SELECT v.chrom, v.pos, v.ref, v.alt, MAX(f.freq) FROM allele_variants v JOIN frequencies f "
+        "ON f.gene = v.gene AND f.allele = v.allele GROUP BY v.chrom, v.pos, v.ref, v.alt"
+    ):
+        vfreq[(c, p, r, al)] = float(f or 0.0)
+    keys = sorted({(c, p, r, al, g) for g, c, p, r, al in con.execute(
+        "SELECT gene, chrom, pos, ref, alt FROM allele_variants")})  # fmt: skip
+    # array dosages per (chrom, pos, ref, alt) and record ID
+    wanted = defaultdict(set)
+    for c, p, *_ in keys:
+        wanted[c].add(p)
+    arr: dict[tuple[str, int, str, str], dict[str, dict[str, int]]] = defaultdict(lambda: defaultdict(dict))
+    gsd: dict[tuple[str, int, str, str, str, str], float] = {}
+    for sid in sorted(use):
+        path = a.vcfs / f"{sid}.snv.vcf.gz"
+        if not path.exists():
+            continue
+        with gzip.open(path, "rt") as fh:
+            for ln in fh:
+                if ln[0] == "#":
+                    continue
+                c, p, rid, ref, alts = ln.split("\t", 5)[:5]
+                if int(p) not in wanted.get(c, ()):
+                    continue
+                sample = ln.rstrip("\n").split("\t")[9].split(":")
+                gt = sample[0].replace("|", "/").split("/")
+                if "." in gt:
+                    continue
+                gs_v = float(sample[1]) if len(sample) > 1 and sample[1] not in (".", "") else float("nan")
+                for ai, alt in enumerate(alts.split(","), start=1):
+                    arr[(c, int(p), ref, alt)][rid][sid] = sum(int(x) == ai for x in gt)
+                    gsd[(c, int(p), ref, alt, rid, sid)] = gs_v
+    # truth per gene window
+    truth: dict[tuple[str, int, str, str], dict[str, int]] = {}
+    by_gene = defaultdict(list)
+    for c, p, r, al, g in keys:
+        by_gene[(g, c)].append((p, r, al))
+    inds = sorted(set(use.values()))
+    import os
+
+    a.out = a.out.resolve()
+    a.cache.mkdir(parents=True, exist_ok=True)
+    os.chdir(a.cache)  # pysam downloads the remote tabix indexes into the working directory
+    for (g, c), vs in sorted(by_gene.items()):
+        core = c.removeprefix("chr")
+        chrom = "chrM" if core in ("MT", "M") else f"chr{core}"
+        if chrom == "chrM":
+            continue
+        try:
+            vf = pysam.VariantFile(PANEL.format(c=chrom))
+        except (OSError, ValueError):
+            print(f"{g} {chrom}: panel file not available, skipped")
+            continue
+        samples = [s for s in inds if s in vf.header.samples]
+        vf.subset_samples(samples)
+        lo, hi = min(p for p, _, _ in vs) - 10, max(p + len(r) for p, r, _ in vs) + 10
+        want = {(p, r, al) for p, r, al in vs}
+        seen = set()
+        for rec in vf.fetch(chrom, lo, hi):
+            for ai, alt in enumerate(rec.alts or (), start=1):
+                k = (rec.pos, rec.ref, alt)
+                if k in want:
+                    truth[(c, *k)] = {s: sum(x == ai for x in rec.samples[s]["GT"] if x is not None) for s in samples}
+                    seen.add(k)
+        for p, r, al in want - seen:
+            # absent from the (filtered) panel: homozygous reference only for a rare variant; a common one was
+            # most likely filtered out (e.g. CYP2B6 rs2279343 next to CYP2B7) and is not assessed
+            if vfreq.get((c, p, r, al), 0.0) < 0.05:
+                truth[(c, p, r, al)] = dict.fromkeys(samples, 0)
+        print(f"{g} {chrom}: {len(want)} variant(s), {len(seen)} in the panel")
+    rows = []
+    stats = [0, 0]
+    bins = [0.0, 0.15, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 1.01]
+    calib = [[0, 0] for _ in bins[:-1]]  # [discordant, n] per GS bin over unmasked records (error-model calibration)
+    for k, recs in sorted(arr.items()):
+        t = truth.get(k)
+        if t is None:
+            continue
+        for rid, dos in sorted(recs.items()):
+            pairs = [(d, t[use[s]]) for s, d in dos.items() if use[s] in t]
+            if len(pairs) < 5:
+                continue
+            conc = sum(x == y for x, y in pairs) / len(pairs)
+            stats[1] += 1
+            if conc >= a.min_concordance:
+                for s_, d in dos.items():
+                    if use[s_] not in t:
+                        continue
+                    gv = gsd.get((*k, rid, s_), float("nan"))
+                    for bi in range(len(bins) - 1):
+                        if bins[bi] <= gv < bins[bi + 1]:
+                            calib[bi][0] += d != t[use[s_]]
+                            calib[bi][1] += 1
+            if conc < a.min_concordance:
+                stats[0] += 1
+                for probe in rid.split(","):
+                    rows.append((probe, f"concordance {conc:.2f} vs 1000G 30x at {k[0]}:{k[1]} {k[2]}>{k[3]} "
+                                        f"(n={len(pairs)})", a.manifest))  # fmt: skip
+    with a.out.open("w") as fh:
+        fh.write(f"# PGx probe mask for {a.manifest} (PGX-010): array vs 1000G 30x, {len(inds)} individuals, "
+                 f"threshold {a.min_concordance}\n")  # fmt: skip
+        for r in sorted(set(rows)):
+            fh.write("\t".join(r) + "\n")
+    print(f"masked {stats[0]} of {stats[1]} assessed record(s); {len(set(rows))} probe name(s) -> {a.out}")
+    for bi, (dsc, n) in enumerate(calib):
+        if n:
+            print(f"GS [{bins[bi]:.2f}, {bins[bi + 1]:.2f}): discordance {dsc}/{n} = {dsc / n:.4f} (unmasked records)")
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

scripts/pgx/fetch_sources.sh

@@ -0,0 +1,75 @@
+#!/usr/bin/env bash
+# Download the public PGx sources for `simurg pgx db build` (spec PGX-001) into a dated folder.
+#   bash scripts/pgx/fetch_sources.sh [OUT_DIR]       (default data/pgx/sources)
+# CPIC (CC0): tables from the CPIC API (https://api.cpicpgx.org, PostgREST) as JSON; the newest change-log date is
+# recorded as the data version. PharmVar (https://www.pharmvar.org; cite Gaedigk et al. 2018 CPT 103:399): per-gene
+# zips (haplotype VCFs, GRCh37/GRCh38) for the star-allele genes CPIC does not define. Never committed (data/ is
+# ignored); the DB records each source's version and file hashes.
+set -euo pipefail
+OUT=${1:-data/pgx/sources}
+DATE=$(date -u +%Y%m%d)
+D="$OUT/cpic-$DATE"
+mkdir -p "$D"
+API=https://api.cpicpgx.org/v1
+for t in gene allele allele_definition allele_location_value sequence_location gene_result gene_result_lookup \
+         gene_result_diplotype population change_log; do
+  curl -sSf --retry 3 -H "Accept: application/json" "$API/$t" -o "$D/$t.json"
+  python3 -c "import json,sys; print(sys.argv[1], len(json.load(open(sys.argv[2]))))" "$t" "$D/$t.json"
+done
+python3 - "$D" <<'PY'
+import hashlib, json, pathlib, sys
+d = pathlib.Path(sys.argv[1])
+log = json.loads((d / "change_log.json").read_text())
+ver = max(e["date"] for e in log if e.get("date"))
+files = {p.name: hashlib.sha256(p.read_bytes()).hexdigest() for p in sorted(d.glob("*.json")) if p.name != "source.json"}
+(d / "source.json").write_text(json.dumps({"name": "CPIC", "version": ver, "retrieved": d.name.split("-")[1],
+                                          "url": "https://api.cpicpgx.org/v1", "license": "CC0-1.0",
+                                          "files": files}, indent=1))
+print("CPIC data version", ver)
+PY
+
+# ---- PharmVar: star-allele genes without CPIC definitions
+PV="$OUT/pharmvar-$DATE"
+mkdir -p "$PV"
+PV_GENES=${PV_GENES:-"CYP1A2 CYP2A6 CYP2A13 CYP2C8 CYP3A4"}  # the PharmVar genes CPIC does not define (2026-09)
+for g in $PV_GENES; do
+  if curl -sSf --retry 3 -o "$PV/$g.zip" \
+       "https://www.pharmvar.org/get-download-file?name=$g&refSeq=ALL&fileType=zip&version=current"; then
+    echo "PharmVar $g $(unzip -l "$PV/$g.zip" | awk 'NR==4{print $4}' | cut -d/ -f1)"
+  else
+    echo "PharmVar $g: not available"; rm -f "$PV/$g.zip"
+  fi
+done
+python3 - "$PV" <<'PY'
+import hashlib, json, pathlib, re, sys, zipfile
+d = pathlib.Path(sys.argv[1])
+vers = set()
+for z in sorted(d.glob("*.zip")):
+    top = zipfile.ZipFile(z).namelist()[0].split("/")[0]
+    m = re.search(r"-(\d+\.\d+\.\d+)$", top)
+    if m:
+        vers.add(m.group(1))
+files = {p.name: hashlib.sha256(p.read_bytes()).hexdigest() for p in sorted(d.glob("*.zip"))}
+(d / "source.json").write_text(json.dumps({"name": "PharmVar", "version": ",".join(sorted(vers)),
+                                          "retrieved": d.name.split("-")[1], "url": "https://www.pharmvar.org",
+                                          "license": "PharmVar terms of use (cite Gaedigk et al. 2018)",
+                                          "files": files}, indent=1))
+print("PharmVar version", ",".join(sorted(vers)))
+PY
+
+# ---- ClinPGx (formerly PharmGKB; CC BY-SA 4.0): clinical-annotation variants (variant-based genes, genotype only)
+CP="$OUT/clinpgx-$DATE"
+mkdir -p "$CP"
+curl -sSfL --retry 3 -o "$CP/clinicalVariants.zip" "https://api.clinpgx.org/v1/download/file/data/clinicalVariants.zip"
+python3 - "$CP" <<'PY'
+import hashlib, json, pathlib, sys, zipfile
+d = pathlib.Path(sys.argv[1])
+z = zipfile.ZipFile(d / "clinicalVariants.zip")
+created = next((n.split("_", 1)[1].rsplit(".", 1)[0] for n in z.namelist() if n.startswith("CREATED_")), "")
+(d / "source.json").write_text(json.dumps({"name": "ClinPGx clinical variants", "version": created,
+                                          "retrieved": d.name.split("-")[1], "url": "https://www.clinpgx.org",
+                                          "license": "CC BY-SA 4.0",
+                                          "files": {"clinicalVariants.zip": hashlib.sha256(
+                                              (d / "clinicalVariants.zip").read_bytes()).hexdigest()}}, indent=1))
+print("ClinPGx clinical variants", created)
+PY

scripts/pgx/getrm_truth.py

@@ -0,0 +1,55 @@
+#!/usr/bin/env python3
+"""Extract GeT-RM consensus diplotypes (CDC Genetic Testing Reference Material program, consolidated PGx/HLA table,
+public) for the individuals of an identity table.
+
+    python scripts/pgx/getrm_truth.py --xlsx Consolidated_PGx-HLA_table_1-22-25-V4.xlsx \
+        --identity scripts/pgx/demo36_identity.tsv --out getrm_truth.tsv
+Writes ``individual, gene, diplotype`` (the table's text, trimmed) for the star-allele genes.
+"""
+
+from __future__ import annotations
+
+import argparse
+import csv
+from pathlib import Path
+
+GENES = ("CYP2B6", "CYP2C9", "CYP2C19", "CYP2D6", "CYP3A4", "CYP3A5", "CYP4F2", "NAT2", "NUDT15", "SLCO1B1", "TPMT",
+         "UGT1A1", "DPYD", "CYP2A6", "CYP1A2", "CYP2C8", "GSTM1", "GSTT1", "UGT2B17")  # fmt: skip
+
+
+def main() -> int:
+    import openpyxl
+
+    ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    ap.add_argument("--xlsx", required=True, type=Path)
+    ap.add_argument("--identity", required=True, type=Path)
+    ap.add_argument("--out", required=True, type=Path)
+    a = ap.parse_args()
+    with a.identity.open() as fh:
+        inds = {r["individual"] for r in csv.DictReader((ln for ln in fh if not ln.startswith("#")), delimiter="\t")}
+    wb = openpyxl.load_workbook(a.xlsx, read_only=True)
+    rows = list(wb[wb.sheetnames[0]].iter_rows(values_only=True))
+    head = [str(h or "").split("\n")[0].strip() for h in rows[1]]
+    col = {g: head.index(g) for g in GENES if g in head}
+    vk = next((i for i, h in enumerate(rows[1]) if str(h or "").startswith("VKORC1") and "rs9923231" in str(h)
+               and "Reference" not in str(h)), None)  # fmt: skip
+    if vk is not None:
+        col["VKORC1"] = vk  # c.-1639G>A genotype, e.g. "GA"
+    out = []
+    for r in rows[2:]:
+        if not r or not r[1] or str(r[1]).strip() not in inds:
+            continue
+        for g, i in col.items():
+            v = r[i]
+            if v is not None and str(v).strip():
+                out.append((str(r[1]).strip(), g, " ".join(str(v).split())))
+    with a.out.open("w") as fh:
+        fh.write("individual\tgene\tdiplotype\n")
+        for t in sorted(out):
+            fh.write("\t".join(t) + "\n")
+    print(f"{len({t[0] for t in out})} individual(s), {len(out)} gene call(s) -> {a.out}")
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

scripts/pgx/replicate_concordance.py

@@ -0,0 +1,53 @@
+#!/usr/bin/env python3
+"""Replicate concordance of `pgx star-allele call`: the top genotype of every gene compared between the replicate
+arrays of one individual (PGX-029 reproducibility on real replicates).
+
+    python scripts/pgx/replicate_concordance.py --calls STAR_DIR/pgx_calls.json --identity identity.tsv --out rep.json
+"""
+
+from __future__ import annotations
+
+import argparse
+import csv
+import itertools
+import json
+from collections import defaultdict
+from pathlib import Path
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    ap.add_argument("--calls", required=True, type=Path)
+    ap.add_argument("--identity", required=True, type=Path)
+    ap.add_argument("--out", required=True, type=Path)
+    a = ap.parse_args()
+    with a.identity.open() as fh:
+        ident = {r["sample_id"]: r["individual"] for r in csv.DictReader((ln for ln in fh if not ln.startswith("#")),
+                                                                           delimiter="\t")}  # fmt: skip
+    calls: dict[str, dict[str, dict[str, str]]] = defaultdict(dict)
+    for s in json.loads(a.calls.read_text())["samples"]:
+        ind = ident.get(s["sample_id"])
+        if ind:
+            calls[ind][s["sample_id"]] = {g["gene"]: (g["candidates"][0]["genotype"] if g["candidates"] else "")
+                                          for g in s["genes"]}  # fmt: skip
+    per_gene: dict[str, list[int]] = defaultdict(lambda: [0, 0])
+    discord = []
+    for ind, reps in sorted(calls.items()):
+        for (_s1, c1), (_s2, c2) in itertools.combinations(sorted(reps.items()), 2):
+            for gene in sorted(set(c1) & set(c2)):
+                ok = c1[gene] == c2[gene]
+                per_gene[gene][0] += ok
+                per_gene[gene][1] += 1
+                if not ok:
+                    discord.append({"individual": ind, "gene": gene, "a": c1[gene], "b": c2[gene]})
+    tot = [sum(v[0] for v in per_gene.values()), sum(v[1] for v in per_gene.values())]
+    out = {"pairs_by_gene": {g: {"concordant": v[0], "pairs": v[1]} for g, v in sorted(per_gene.items())},
+           "total": {"concordant": tot[0], "pairs": tot[1], "rate": round(tot[0] / tot[1], 4) if tot[1] else None},
+           "discordant": discord}  # fmt: skip
+    a.out.write_text(json.dumps(out, indent=1))
+    print(json.dumps(out["total"]), f"{len(discord)} discordant gene pair(s)")
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

scripts/pgx/validate_cn_truth.py

@@ -0,0 +1,74 @@
+#!/usr/bin/env python3
+"""Validate `simurg pgx copy-number call` against the NYGC 30x SV callset (1KGP_3202 freeze V3) — PCN-017.
+
+    python scripts/pgx/validate_cn_truth.py --calls CALL_DIR --identity scripts/pgx/demo36_identity.tsv \
+        --sv 1KGP_3202.sv_freeze_V3.vcf.gz --bcftools BCFTOOLS --out summary.json
+
+Truth per region = RD_CN of one SV record per region (below, chosen as the record spanning the whole region; for
+GSTM1 the read-depth DUP/CNV record, whose RD_CN gives 0/1/2 in Hardy–Weinberg proportions — the DEL record there
+reports CN 1 for 27/30 individuals). CYP2D6 sub-regions share one gene-level record, so exon-9 conversions (*36) and
+2D7-2D6 hybrids (*13, *68) legitimately differ from it. GSTT1 lies on an alt contig with no SV calls: only its CN
+distribution is reported. Replicate arrays of one individual are counted once (the first array).
+"""
+
+from __future__ import annotations
+
+import argparse
+import collections
+import csv
+import json
+import subprocess
+from pathlib import Path
+
+TRUTH_RECORD = {"GSTM1": "HGSV_9691", "UGT2B17": "HGSV_63272", "SULT1A1": "HGSV_207805",
+                "CYP2A6.intron.7": "HGSV_232737", "CYP2A6.exon.1": "HGSV_232737", "CYP2D6.exon.9": "HGSV_247317",
+                "CYP2D6.intron.2": "HGSV_247317", "CYP2D6.p5": "HGSV_247317"}  # fmt: skip
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    ap.add_argument("--calls", required=True, type=Path)
+    ap.add_argument("--identity", required=True, type=Path)
+    ap.add_argument("--sv", required=True)
+    ap.add_argument("--bcftools", default="bcftools")
+    ap.add_argument("--out", required=True, type=Path)
+    a = ap.parse_args()
+    ident = {}
+    with a.identity.open() as fh:
+        for r in csv.DictReader((ln for ln in fh if not ln.startswith("#")), delimiter="\t"):
+            ident[r["sample_id"]] = r["individual"]
+    calls: dict[str, dict[str, str]] = {}
+    with (a.calls / "pgx_cn_sample_summary.tsv").open() as fh:
+        for r in csv.DictReader(fh, delimiter="\t"):
+            ind = ident.get(r["sample_id"])
+            if ind and ind not in calls:
+                calls[ind] = {k[3:]: v for k, v in r.items() if k.startswith("cn_")}
+    ids = sorted(set(TRUTH_RECORD.values()))
+    q = subprocess.run([a.bcftools, "query", "-H", "-s", ",".join(sorted(calls)), "--force-samples", "-i",
+                        "ID=@/dev/stdin", "-f", "%ID[\t%RD_CN]\n", a.sv], input="\n".join(ids), capture_output=True,
+                       text=True, check=True).stdout.splitlines()  # fmt: skip
+    samples = [h.split("]")[1].split(":")[0] for h in q[0].split("\t")[1:]]
+    truth = {ln.split("\t")[0]: dict(zip(samples, ln.split("\t")[1:], strict=True)) for ln in q[1:]}
+    res: dict[str, dict[str, object]] = {}
+    tot = [0, 0]
+    for region, rid in TRUTH_RECORD.items():
+        pairs = [(truth[rid][ind], c[region]) for ind, c in calls.items() if ind in truth.get(rid, {})]
+        pairs = [(t, c) for t, c in pairs if t not in (".", "")]
+        ok = sum(t == c for t, c in pairs)
+        tot[0] += ok
+        tot[1] += len(pairs)
+        res[region] = {"record": rid, "concordant": ok, "n": len(pairs),
+                       "rate": round(ok / len(pairs), 4) if pairs else None,
+                       "truth_to_call_mismatches": dict(collections.Counter(
+                           f"{t}->{c}" for t, c in pairs if t != c))}  # fmt: skip
+    summary = {"individuals": len(calls), "regions": res, "overall": {"concordant": tot[0], "n": tot[1],
+               "rate": round(tot[0] / tot[1], 4) if tot[1] else None},
+               "gstt1_cn_distribution": dict(sorted(collections.Counter(
+                   c.get("GSTT1", ".") for c in calls.values()).items()))}  # fmt: skip
+    a.out.write_text(json.dumps(summary, indent=1))
+    print(json.dumps(summary, indent=1))
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

scripts/pgx/validate_star_getrm.py

@@ -0,0 +1,199 @@
+#!/usr/bin/env python3
+"""Compare `simurg pgx star-allele call` with GeT-RM consensus diplotypes (PGX-032).
+
+    python scripts/pgx/validate_star_getrm.py --calls STAR_DIR/pgx_calls.json --truth getrm_truth.tsv \
+        --identity scripts/pgx/demo36_identity.tsv --out summary.json
+
+GeT-RM notation is normalised per allele: "*2(*35)" / "*2 (*35)" accept *2 or *35; a trailing sub-allele letter
+("*1F") also matches the core allele ("*1") when the call has no letter; "(*68)+*4" tandems and "xN" duplications
+reduce to the core allele (structural forms are reported separately); "(*28 + *60)" keeps the star alleles the DB
+defines. "A/B or C/D" accepts either. UGT1A1 and SLCO1B1 are reported but left out of the total: GeT-RM uses pre-2023
+names (UGT1A1 *60, SLCO1B1 *1B/*17/*21) that the CPIC definitions no longer carry. A sample's gene counts as **top**
+concordant when the top solution matches, and as **equivalent**
+concordant when the truth is among the array-equivalent members of any feasible solution. Replicate arrays of one
+individual are counted once (the first).
+"""
+
+from __future__ import annotations
+
+import argparse
+import csv
+import json
+import re
+from collections import defaultdict
+from pathlib import Path
+
+
+def truth_options(text: str) -> list[set[str]] | None:
+    """Per haplotype, the set of acceptable core alleles; None when the text is not a diplotype."""
+    t = text.replace(" ", "")
+    if "/" not in t:
+        return None
+    out = []
+    for side in t.split("/", 1):
+        opts = set(re.findall(r"\*\d+[A-Z]?", side))
+        if not opts:
+            return None
+        cores = {re.sub(r"x\d+|xN", "", o) for o in opts}
+        cores |= {re.sub(r"[A-Z]$", "", o) for o in cores}
+        if "+" in side:  # tandem / hybrid: the non-parenthesised last unit is the core allele
+            last = re.findall(r"\*\d+[A-Z]?", side.split("+")[-1])
+            cores |= set(last)
+        out.append(cores)
+    return out
+
+
+def _gstm1(t: str) -> int:  # GeT-RM GSTM1: DEL/DEL = no copy; *A/*B assays do not see a heterozygous deletion
+    return 0 if t.replace(" ", "").upper() == "DEL/DEL" else 1
+
+
+def _gstt1(t: str) -> int:  # "0" null, "(*A/0)" one null allele, "(*A/*A)" two copies
+    t = t.replace(" ", "")
+    return 0 if t in ("0", "0/0") else (1 if "/0" in t or "0/" in t else 2)
+
+
+def _ugt2b17(t: str) -> int:  # *2 = whole-gene deletion
+    return 2 - t.count("*2")
+
+
+CN_TRUTH = {"GSTM1": _gstm1, "GSTT1": _gstt1, "UGT2B17": _ugt2b17}  # capped: GSTM1 at 1, the others at 2
+# GeT-RM uses older names or definitions than the current CPIC/PharmVar ones: UGT1A1 *60 and SLCO1B1 *1B/*17/*21
+# (pre-2023), CYP1A2 lettered alleles (*1F …), CYP2C8 *19 (PharmVar now defines rs2071426 alone as *19; GeT-RM *1)
+LEGACY_NOMENCLATURE = {"UGT1A1", "SLCO1B1", "CYP1A2", "CYP2C8"}
+
+
+def truth_alternatives(text: str) -> list[list[set[str]]]:
+    """GeT-RM "A/B or C/D" → one option list per alternative diplotype."""
+    alts = [truth_options(x) for x in re.split(r"\bor\b", text)]
+    return [x for x in alts if x is not None]
+
+
+def matches(call: tuple[str, str], truth: list[list[set[str]]]) -> bool:
+    a, b = (re.sub(r"x\d+$", "", x.split("+")[-1]) for x in call)  # tandem "*68+*4" → core "*4"
+    return any((a in t[0] and b in t[1]) or (a in t[1] and b in t[0]) for t in truth)
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    ap.add_argument("--calls", required=True, type=Path)
+    ap.add_argument("--truth", required=True, type=Path)
+    ap.add_argument("--identity", required=True, type=Path)
+    ap.add_argument("--out", required=True, type=Path)
+    ap.add_argument("--all-genes", action="store_true",
+                    help="truth with the same definitions (wgs_star_truth.py): no legacy-name exclusions")  # fmt: skip
+    ap.add_argument("--individuals", type=Path, help="only these individuals (one per line)")
+    ap.add_argument("--coverage", type=Path, help="pgx_coverage.tsv of the array: also report array-resolvable "
+                    "concordance (both diplotypes projected onto the alleles the array can tell apart)")  # fmt: skip
+    a = ap.parse_args()
+    if a.all_genes:
+        LEGACY_NOMENCLATURE.clear()
+    only = set(a.individuals.read_text().split()) if a.individuals else None
+    proj: dict[tuple[str, str], str] = {}
+    refs: dict[str, str] = {}
+    if a.coverage:
+        with a.coverage.open() as fh:
+            for r in csv.DictReader((ln for ln in fh if not ln.startswith("#")), delimiter="\t"):
+                if r["allele_kind"] == "reference":
+                    refs[r["gene"]] = r["allele"]
+                st = r["status"]
+                proj[(r["gene"], r["allele"])] = st.split(":", 1)[1] if st.startswith("equivalent_to:") else (
+                    r["allele"] if st == "callable" else "")  # fmt: skip
+
+    def project(gene: str, allele: str) -> str:
+        core = re.sub(r"x\d+$|xN$", "", allele.split("+")[-1])
+        v = proj.get((gene, core), core)
+        if v:
+            return v
+        ref = refs.get(gene, core)  # not covered: indistinguishable from the reference's class
+        return proj.get((gene, ref)) or ref
+
+    with a.identity.open() as fh:
+        ident = {r["sample_id"]: r["individual"] for r in csv.DictReader((ln for ln in fh if not ln.startswith("#")),
+                                                                           delimiter="\t")}  # fmt: skip
+    truth: dict[tuple[str, str], str] = {}
+    with a.truth.open() as fh:
+        for r in csv.DictReader(fh, delimiter="\t"):
+            if r.get("flag") == "incomplete" or (only is not None and r["individual"] not in only):
+                continue
+            truth[(r["individual"], r["gene"])] = r["diplotype"]
+    calls = json.loads(a.calls.read_text())
+    per_gene: dict[str, dict[str, int]] = defaultdict(lambda: defaultdict(int))
+    detail = []
+    seen = set()
+    for s in calls["samples"]:
+        ind = ident.get(s["sample_id"])
+        if ind is None or ind in seen:
+            continue
+        seen.add(ind)
+        for g in s["genes"]:
+            t = truth.get((ind, g["gene"]))
+            if t is None:
+                continue
+            st = per_gene[g["gene"]]
+            if g["gene"] in CN_TRUTH:  # whole-gene deletion genes: compare copy-number classes (PGX-006)
+                want_cn = CN_TRUTH[g["gene"]](t)
+                cn = g.get("copy_number")
+                got_cn = None if cn is None else (min(cn, 2) if g["gene"] != "GSTM1" else min(cn, 1))
+                st["n"] += 1
+                ok = want_cn is not None and want_cn == got_cn
+                st["top"] += ok
+                st["equivalent"] += ok
+                if not ok:
+                    detail.append({"individual": ind, "gene": g["gene"], "truth": t, "call": g["genotype"],
+                                   "within_equivalents": False})  # fmt: skip
+                continue
+            if g["gene"] == "VKORC1":  # c.-1639G>A (minus strand) = rs9923231 C>T: count A = count T
+                want = t.count("A")
+                got = g["genotype"].split("/")[-1].count("T") if g["genotype"] != "Reference" else 0
+                st["n"] += 1
+                st["top"] += want == got
+                st["equivalent"] += want == got
+                if want != got:
+                    detail.append({"individual": ind, "gene": "VKORC1", "truth": t, "call": g["genotype"],
+                                   "within_equivalents": False})  # fmt: skip
+                continue
+            opts = truth_alternatives(t)
+            if not opts:
+                st["not_comparable"] += 1
+                continue
+            st["n"] += 1
+            top = g["candidates"][0]["genotype"].split("/") if g["candidates"] else ["?", "?"]
+            ok_top = len(top) == 2 and matches((top[0], top[1]), opts)
+            feas = [c for c in g["candidates"] if c["genotype"] in g["feasible_solutions"]]
+            ok_eq = ok_top or any(matches(tuple(e.split("/")[:2]), opts) for c in feas for e in c["equivalents"]
+                                  if len(e.split("/")) >= 2)  # fmt: skip
+            st["top"] += ok_top
+            st["equivalent"] += ok_eq
+            if a.coverage and len(top) == 2 and len(opts) == 1 and all(len(x) == 1 for x in opts[0]):
+                tt = sorted(project(g["gene"], next(iter(x))) for x in opts[0])
+                cc = sorted(project(g["gene"], x) for x in top)
+                st["array_resolvable_n"] += 1
+                st["array_resolvable"] += tt == cc
+            if not ok_top:
+                detail.append({"individual": ind, "gene": g["gene"], "truth": t, "call": g["genotype"],
+                               "within_equivalents": ok_eq})  # fmt: skip
+    comparable = {k: v for k, v in per_gene.items() if k not in LEGACY_NOMENCLATURE}
+    tot_ar = [
+        sum(v.get("array_resolvable", 0) for v in comparable.values()),
+        sum(v.get("array_resolvable_n", 0) for v in comparable.values()),
+    ]
+    summary = {"individuals": len(seen), "genes": {k: dict(v) for k, v in sorted(per_gene.items())},
+               "array_resolvable_total": {"concordant": tot_ar[0], "n": tot_ar[1]},
+               "legacy_nomenclature_genes": sorted(LEGACY_NOMENCLATURE),
+               "total": {k: sum(v.get(k, 0) for v in comparable.values()) for k in ("n", "top", "equivalent")},
+               "discordant": detail}  # fmt: skip
+    a.out.write_text(json.dumps(summary, indent=1))
+    for g, v in sorted(per_gene.items()):
+        print(f"{g:8s} top {v['top']}/{v['n']}  within equivalents {v['equivalent']}/{v['n']}"
+              + (f"  (not comparable {v['not_comparable']})" if v.get("not_comparable") else "")
+              + (f"  array-resolvable {v.get('array_resolvable', 0)}/{v.get('array_resolvable_n', 0)}"
+                 if a.coverage else "")
+              + ("  [legacy nomenclature]" if g in LEGACY_NOMENCLATURE else ""))  # fmt: skip
+    print("total (excluding legacy-nomenclature genes)", summary["total"])
+    if a.coverage:
+        print("array-resolvable total", summary["array_resolvable_total"])
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

scripts/pgx/wgs_star_truth.py

@@ -0,0 +1,130 @@
+#!/usr/bin/env python3
+"""Star-allele truth from 1000 Genomes 30x phased genotypes with the PGx DB definitions (SNV/indel alleles only).
+
+    python scripts/pgx/wgs_star_truth.py --db DB.sqlite --individuals ids.txt --out wgs_truth.tsv [--genes G,G]
+
+For every star-allele gene of the DB the defining variants are fetched from the NYGC phased panel (pysam over HTTPS,
+one window per gene; ``uv pip install pysam``). Each phased haplotype gets the allele whose defining variants it
+carries completely and that explains most of the haplotype's carried variants — its defining ones plus those at its
+wildcard (IUPAC) positions, so a *4 haplotype carrying the *10 variants stays *4 (ties: more defining variants,
+higher CPIC/PharmVar frequency, natural order); a haplotype carrying none is the reference allele. A variant absent
+from the filtered panel counts as reference only when rare (max allele frequency < 5 %); a gene with a common
+defining variant missing from the panel is written with ``incomplete`` (not comparable). Structural alleles (CYP2D6
+*5, xN, hybrids; CYP2A6 *4) are invisible to this truth: the diplotypes are SNV-level only.
+"""
+
+from __future__ import annotations
+
+import argparse
+import os
+import re
+import sqlite3
+from collections import defaultdict
+from pathlib import Path
+
+PANEL = (
+    "https://ftp.1000genomes.ebi.ac.uk/vol1/ftp/data_collections/1000G_2504_high_coverage/working/"
+    "20220422_3202_phased_SNV_INDEL_SV/1kGP_high_coverage_Illumina.{c}.filtered.SNV_INDEL_SV_phased_panel.vcf.gz"
+)
+
+
+def natural(a: str) -> tuple[object, ...]:
+    return tuple(int(x) if x.isdigit() else x for x in re.split(r"(\d+)", a))
+
+
+def main() -> int:
+    import pysam
+
+    ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    ap.add_argument("--db", required=True, type=Path)
+    ap.add_argument("--individuals", required=True, type=Path)
+    ap.add_argument("--out", required=True, type=Path)
+    ap.add_argument("--genes", default="")
+    ap.add_argument("--cache", type=Path, default=Path("data/pgx/cache"))
+    a = ap.parse_args()
+    inds = sorted(set(a.individuals.read_text().split()))
+    out_path = a.out.resolve()
+    con = sqlite3.connect(a.db)
+    genes = [g for (g,) in con.execute("SELECT gene FROM genes WHERE result_basis = 'star_allele' ORDER BY gene")]
+    if a.genes:
+        genes = [g for g in genes if g in a.genes.split(",")]
+    a.cache.mkdir(parents=True, exist_ok=True)
+    os.chdir(a.cache)
+    rows = []
+    for gene in genes:
+        q_alle = "SELECT allele FROM alleles WHERE gene = ? AND defined = 1 AND structural = 0"
+        alle = [x for (x,) in con.execute(q_alle, (gene,))]
+        q_ref = "SELECT allele FROM alleles WHERE gene = ? AND kind = 'reference'"
+        ref_allele = next((x for (x,) in con.execute(q_ref, (gene,))), "*1")
+        defs: dict[str, set[tuple[int, str, str]]] = defaultdict(set)
+        chrom = None
+        loc_keys: dict[int, set[tuple[int, str, str]]] = defaultdict(set)
+        q_var = "SELECT allele, chrom, pos, ref, alt, location_id FROM allele_variants WHERE gene = ?"
+        for allele, c, p, r, al, lid in con.execute(q_var, (gene,)):
+            defs[allele].add((p, r, al))
+            loc_keys[lid].add((p, r, al))
+            chrom = c
+        wild: dict[str, set[tuple[int, str, str]]] = defaultdict(set)
+        for allele, lid in con.execute("SELECT allele, location_id FROM allele_wildcards WHERE gene = ?", (gene,)):
+            wild[allele] |= loc_keys.get(lid, set())
+        if chrom is None:
+            continue
+        freq: dict[str, float] = defaultdict(float)
+        for allele, f in con.execute("SELECT allele, AVG(freq) FROM frequencies WHERE gene = ? GROUP BY allele",
+                                     (gene,)):  # fmt: skip
+            freq[allele] = float(f or 0)
+        vmax: dict[tuple[int, str, str], float] = defaultdict(float)
+        for allele, vs in defs.items():
+            for v in vs:
+                vmax[v] = max(vmax[v], freq.get(allele, 0.0))
+        keys = set().union(*defs.values()) if defs else set()
+        core = chrom.removeprefix("chr")
+        vf = pysam.VariantFile(PANEL.format(c=f"chr{core}"))
+        present = [s for s in inds if s in vf.header.samples]
+        vf.subset_samples(present)
+        lo = min(p for p, _, _ in keys) - 10
+        hi = max(p + len(r) for p, r, _ in keys) + 10
+        hap_alts: dict[tuple[str, int], set[tuple[int, str, str]]] = defaultdict(set)
+        seen = set()
+        for rec in vf.fetch(f"chr{core}", lo, hi):
+            for ai, alt in enumerate(rec.alts or (), start=1):
+                k = (rec.pos, rec.ref, alt)
+                if k not in keys:
+                    continue
+                seen.add(k)
+                for s in present:
+                    gt = rec.samples[s]["GT"]
+                    for h, x in enumerate(gt[:2]):
+                        if x == ai:
+                            hap_alts[(s, h)].add(k)
+        missing_common = [k for k in keys - seen if vmax[k] >= 0.05]
+        for s in present:
+            haps = []
+            for h in (0, 1):
+                carried = hap_alts[(s, h)]
+                # the allele that explains most carried variants (its defining ones plus those at its wildcard
+                # positions), then the most specific, then the most frequent
+                best = (0, 0, 0.0)
+                pick = ref_allele
+                for x in alle:
+                    d = defs.get(x, set())
+                    if not d or not d <= carried:
+                        continue
+                    score = (len(carried & (d | wild.get(x, set()))), len(d), freq.get(x, 0.0))
+                    if score > best or (score == best and natural(x) < natural(pick)):
+                        best, pick = score, x
+                haps.append(pick)
+            dip = "/".join(sorted(haps, key=natural))
+            rows.append((s, gene, dip, "incomplete" if missing_common else ""))
+        print(f"{gene}: {len(keys)} defining variant(s), {len(seen)} in the panel, "
+              f"{len(missing_common)} common missing", flush=True)  # fmt: skip
+    with out_path.open("w") as fh:
+        fh.write("individual\tgene\tdiplotype\tflag\n")
+        for r in sorted(rows):
+            fh.write("\t".join(r) + "\n")
+    print(f"{len(rows)} gene call(s) -> {out_path}")
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

scripts/pi_run.sh

@@ -0,0 +1,50 @@
+#!/usr/bin/env bash
+# Run a heavy command on the 8 GB Pi so that running out of memory kills only this command, never the desktop session.
+#   MEM=4G MIN_FREE_MB=1200 bash scripts/pi_run.sh <command...>
+# 1. When the memory cgroup is available (cmdline.txt: cgroup_enable=memory cgroup_memory=1), the command runs in a
+#    systemd user scope with MemoryMax=$MEM and no swap: the kernel kills it alone when it exceeds the cap.
+# 2. In every case a guard stops the command's process group when the system's MemAvailable drops below
+#    MIN_FREE_MB (covers other processes filling memory, and systems without the memory cgroup).
+# Prints the peak memory of the command when it ends.
+set -uo pipefail
+MEM=${MEM:-4G}
+MIN_FREE_MB=${MIN_FREE_MB:-1200}
+uid=$(id -u)
+unit="simurg-job-$$-$(date +%s)"
+cg="/sys/fs/cgroup/user.slice/user-$uid.slice/user@$uid.service/app.slice/$unit.scope"
+if grep -qw memory "/sys/fs/cgroup/user.slice/user-$uid.slice/user@$uid.service/cgroup.controllers" 2>/dev/null; then
+  setsid systemd-run --user --scope --unit="$unit" -p MemoryMax="$MEM" -p MemorySwapMax=0 --quiet -- "$@" &
+  mode="cgroup MemoryMax=$MEM"
+else
+  setsid "$@" &
+  mode="no memory cgroup"
+fi
+pid=$!
+peak=0
+anon_peak=0  # process memory without page cache (memory.peak counts cached file pages, e.g. a FASTA being read)
+tripped=0
+while kill -0 "$pid" 2>/dev/null; do
+  if [[ -r $cg/memory.peak ]]; then
+    peak=$(( $(cat "$cg/memory.peak") / 1024 ))
+    anon=$(awk '$1 == "anon" {print int($2 / 1024)}' "$cg/memory.stat" 2>/dev/null || echo 0)
+    (( anon > anon_peak )) && anon_peak=$anon
+  else
+    rss=$(ps -o rss= -g "$pid" 2>/dev/null | awk '{s+=$1} END {print s+0}'); (( rss > peak )) && peak=$rss
+  fi
+  avail=$(awk '/^MemAvailable:/ {print int($2/1024)}' /proc/meminfo)
+  if (( avail < MIN_FREE_MB )); then
+    echo "pi_run: MemAvailable ${avail} MiB < ${MIN_FREE_MB} MiB -- stopping the job" >&2
+    kill -TERM -- "-$pid" 2>/dev/null; sleep 5; kill -KILL -- "-$pid" 2>/dev/null
+    tripped=1
+    break
+  fi
+  sleep 1
+done
+wait "$pid"; rc=$?
+(( tripped )) && rc=137
+(( rc == 137 && ! tripped )) && [[ $mode == cgroup* ]] \
+  && echo "pi_run: the job was killed (SIGKILL); most likely it exceeded MemoryMax=$MEM" >&2
+anon_note=""
+(( anon_peak > 0 )) && anon_note=", process $((anon_peak / 1024)) MiB without page cache"
+echo "pi_run: exit $rc, peak memory $((peak / 1024)) MiB$anon_note ($mode, guard ${MIN_FREE_MB} MiB free)" >&2
+exit $rc

scripts/train/concordance_chr.py

@@ -0,0 +1,91 @@
+#!/usr/bin/env python3
+"""Array genotypes vs 1000 Genomes 30x on one chromosome (ArrayTrain validation, plan §12).
+
+    python scripts/train/concordance_chr.py --vcfs VCF_DIR --identity identity.tsv --truth chr22.vcf.gz \
+        --bcftools BCFTOOLS --chrom 22 --out summary.json [--samples-glob '*.snv.vcf.gz']
+
+Per array sample (identity table: sample_id → individual): biallelic SNVs of the chromosome whose (POS, REF, ALT)
+is a record of the phased 3202-sample panel; genotype concordance and non-reference discordance (NRD = discordant
+/ (discordant + concordant non-ref); plan §16.5) over sites called on the array, plus the array call rate there.
+"""
+
+from __future__ import annotations
+
+import argparse
+import csv
+import json
+import subprocess
+from pathlib import Path
+
+import numpy as np
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    ap.add_argument("--vcfs", required=True, type=Path)
+    ap.add_argument("--identity", required=True, type=Path)
+    ap.add_argument("--truth", required=True, type=Path)
+    ap.add_argument("--bcftools", default="bcftools")
+    ap.add_argument("--chrom", default="22")
+    ap.add_argument("--out", required=True, type=Path)
+    a = ap.parse_args()
+    with a.identity.open() as fh:
+        ident = {r["sample_id"]: r["individual"] for r in csv.DictReader((ln for ln in fh if not ln.startswith("#")),
+                                                                           delimiter="\t")}  # fmt: skip
+    vcfs = {p.name.split(".")[0]: p for p in sorted(a.vcfs.rglob("*.snv.vcf.gz"))}
+    vcfs = {k: v for k, v in vcfs.items() if k in ident}
+    inds = sorted({ident[k] for k in vcfs})
+    # array sites (from the first VCF) → truth genotypes of the individuals at those sites
+    first = next(iter(vcfs.values()))
+    q = subprocess.run([a.bcftools, "query", "-r", a.chrom, "-i", 'TYPE="snp" && N_ALT=1', "-f", "%POS\t%REF\t%ALT\n",
+                        str(first)], capture_output=True, text=True, check=True).stdout  # fmt: skip
+    sites = {(int(p), r, al) for p, r, al in (ln.split("\t") for ln in q.splitlines())}
+    reg = a.out.with_suffix(".sites.tsv")
+    chrom_t = f"chr{a.chrom}"
+    reg.write_text("".join(f"{chrom_t}\t{p}\n" for p in sorted({s[0] for s in sites})))
+    tq = subprocess.run([a.bcftools, "query", "-R", str(reg), "-s", ",".join(inds), "--force-samples", "-i",
+                         'TYPE="snp"', "-f", "%POS\t%REF\t%ALT[\t%GT]\n", str(a.truth)], capture_output=True, text=True,
+                        check=True).stdout  # fmt: skip
+    head = subprocess.run([a.bcftools, "query", "-l", "-s", ",".join(inds), "--force-samples", str(a.truth)],
+                          capture_output=True, text=True, check=True).stdout.split()  # fmt: skip
+    truth: dict[tuple[int, str, str], dict[str, int]] = {}
+    for ln in tq.splitlines():
+        f = ln.split("\t")
+        k = (int(f[0]), f[1], f[2])
+        if k in sites:
+            truth[k] = {
+                s: sum(x == "1" for x in g.replace("|", "/").split("/")) for s, g in zip(head, f[3:], strict=True)
+            }
+    per = []
+    for sid, path in sorted(vcfs.items()):
+        ind = ident[sid]
+        aq = subprocess.run([a.bcftools, "query", "-r", a.chrom, "-f", "%POS\t%REF\t%ALT[\t%GT]\n", str(path)],
+                            capture_output=True, text=True, check=True).stdout  # fmt: skip
+        n = called = conc = nrd_bad = nrd_ok = 0
+        for ln in aq.splitlines():
+            p, r, al, gt = ln.split("\t")
+            k = (int(p), r, al)
+            t = truth.get(k, {}).get(ind)
+            if t is None:
+                continue
+            n += 1
+            if "." in gt:
+                continue
+            called += 1
+            d = sum(x == "1" for x in gt.replace("|", "/").split("/"))
+            conc += d == t
+            if d > 0 or t > 0:
+                nrd_ok += d == t
+                nrd_bad += d != t
+        per.append({"sample": sid, "individual": ind, "sites": n, "call_rate": called / n if n else None,
+                    "concordance": conc / called if called else None,
+                    "nrd": nrd_bad / (nrd_bad + nrd_ok) if nrd_bad + nrd_ok else None})  # fmt: skip
+    summ = {k: float(np.mean([p[k] for p in per if p[k] is not None])) for k in ("call_rate", "concordance", "nrd")}
+    a.out.write_text(json.dumps({"chrom": a.chrom, "samples": len(per), "truth_sites": len(truth), **summ,
+                                 "per_sample": per}, indent=1))  # fmt: skip
+    print(json.dumps({"samples": len(per), "truth_sites": len(truth), **summ}))
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

scripts/v2/cn_synthetic_eval.py

@@ -0,0 +1,147 @@
+#!/usr/bin/env python3
+"""Sensitivity of `simurg cn call` on implanted events (spec CYTO-022, plan §16.4).
+
+Real GTCs of normal samples are copied and events are written into their LRR/BAF:
+  constitutional DEL (CN1: LRR − 0.45, heterozygous BAF → 0/1), DUP (CN3: LRR + 0.3, het → 1/3 or 2/3), LOH (het → 0/1);
+  mosaic loss / gain / CN-LOH with cell fraction f (LRR shifted by f × the constitutional shift, heterozygous BAF
+  moved to 0.5 ± d(f), sign random per probe as in unphased data), keeping each probe's original noise.
+Every synthetic sample gets several events on different chromosomes. `simurg cn call` then runs on them and an
+implanted event counts as detected when a called event of the same class (loss / gain / LOH) overlaps ≥ 50 % of it.
+
+    python scripts/v2/cn_synthetic_eval.py --bpm M.bpm --model MODEL --gtcs BASE_DIR --out OUT [--threads N]
+"""
+
+from __future__ import annotations
+
+import argparse
+import copy
+import gzip
+import json
+import subprocess
+import sys
+from pathlib import Path
+
+import numpy as np
+
+from simurg.formats.bpm import read_bpm
+from simurg.formats.discovery import discover_gtcs
+from simurg.formats.gtc import read_gtc, write_gtc
+
+KINDS = [("DEL", 1.0), ("DUP", 1.0), ("LOH", 1.0)] + [
+    (k, f) for k in ("MDEL", "MDUP", "MLOH") for f in (0.05, 0.10, 0.15, 0.20, 0.30)
+]
+SIZES = (1_000_000, 5_000_000, 20_000_000)
+CLASS = {"DEL": "loss", "MDEL": "loss", "DUP": "gain", "MDUP": "gain", "LOH": "loh", "MLOH": "loh", "GAINLOH": "gain"}
+CHROMS = [str(c) for c in range(1, 21)]  # leave 21, 22, X, Y untouched
+
+
+def implant(lrr: np.ndarray, baf: np.ndarray, idx: np.ndarray, kind: str, f: float, rng: np.random.Generator) -> None:
+    het = (baf[idx] > 0.15) & (baf[idx] < 0.85)
+    h = idx[het]
+    noise = baf[h] - 0.5
+    sign = rng.choice([-1.0, 1.0], len(h))
+    if kind in ("DEL", "LOH"):
+        if kind == "DEL":
+            lrr[idx] -= 0.45
+        baf[h] = np.clip(np.where(sign > 0, 1.0, 0.0) + noise * 0.2, 0, 1)
+    elif kind == "DUP":
+        lrr[idx] += 0.30
+        baf[h] = np.clip(np.where(sign > 0, 2 / 3, 1 / 3) + noise, 0, 1)
+    else:
+        d = {"MDEL": f / (2 * (2 - f)), "MDUP": f / (2 * (2 + f)), "MLOH": f / 2}[kind]
+        lrr[idx] += {"MDEL": -0.45 * f, "MDUP": 0.30 * f, "MLOH": 0.0}[kind]
+        baf[h] = np.clip(0.5 + sign * d + noise, 0, 1)
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    ap.add_argument("--bpm", required=True)
+    ap.add_argument("--model", required=True)
+    ap.add_argument("--gtcs", required=True, type=Path, help="normal base samples (not in the model's reference)")
+    ap.add_argument("--out", required=True, type=Path)
+    ap.add_argument("--threads", type=int, default=0)
+    ap.add_argument("--seed", type=int, default=7)
+    ap.add_argument("--reps", type=int, default=4, help="replicates of every (kind, fraction, size) cell")
+    a = ap.parse_args()
+    rng = np.random.default_rng(a.seed)
+    man = read_bpm(a.bpm)
+    chroms = np.array(man.chroms)
+    pos = man.map_infos.astype(np.int64)
+    base = sorted(discover_gtcs(a.gtcs).values())
+    syn = a.out / "gtcs"
+    syn.mkdir(parents=True, exist_ok=True)
+    events = [(k, f, s) for k, f in KINDS for s in SIZES for _ in range(a.reps)]
+    rng.shuffle(events)
+    truth = []
+    per_sample = len(CHROMS) // 2  # one event per chromosome pair slot, spaced out
+    for n, start in enumerate(range(0, len(events), per_sample)):
+        chunk = events[start : start + per_sample]
+        src = base[n % len(base)]
+        g = copy.deepcopy(read_gtc(src))
+        lrr = g.logr_ratios.astype(np.float64)
+        baf = g.b_allele_freqs.astype(np.float64)
+        chosen = rng.permutation(CHROMS)[: len(chunk)]
+        sid = f"syn{n:03d}"
+        for (kind, f, size), c in zip(chunk, chosen, strict=True):
+            sel = np.flatnonzero((chroms == c) & (pos > 0))
+            lo, hi = int(pos[sel].min()), int(pos[sel].max())
+            s0 = int(rng.integers(lo + 2_000_000, max(lo + 2_000_001, hi - size - 2_000_000)))
+            idx = sel[(pos[sel] >= s0) & (pos[sel] < s0 + size)]
+            implant(lrr, baf, idx, kind, f, rng)
+            truth.append({"sample": sid, "chrom": c, "start": s0, "end": s0 + size - 1, "kind": kind, "fraction": f,
+                          "size": size, "probes": len(idx), "base": src.stem})  # fmt: skip
+        g.logr_ratios = lrr.astype(np.float32)
+        g.b_allele_freqs = baf.astype(np.float32)
+        (syn / sid).mkdir(exist_ok=True)
+        write_gtc(g, syn / sid / f"{sid}.gtc")
+    (a.out / "truth.json").write_text(json.dumps(truth, indent=1))
+    simurg = str(Path(sys.executable).with_name("simurg"))
+    cmd = [simurg, "cn", "call", "--bpm", a.bpm, "--model", a.model, "--gtcs", str(syn), "--out", str(a.out / "call"),
+           "--threads", str(a.threads), "--unknown-sex-policy", "female", "--force", "--quiet"]  # fmt: skip
+    subprocess.run(cmd, check=True)
+    calls: dict[str, list[tuple[str, int, int, str, str]]] = {}
+    for vcf in (a.out / "call").glob("*.cyto.cnv.vcf.gz"):
+        sid = vcf.name.split(".")[0]
+        with gzip.open(vcf, "rt") as fh:
+            lines = [ln for ln in fh if not ln.startswith("#")]
+        for line in lines:
+            c, p, vid, _, _alt, _q, filt, info = line.split("\t")[:8]
+            end = int(next(x for x in info.split(";") if x.startswith("END="))[4:])
+            call = vid.split(":")[0]
+            cls = {"DEL": "loss", "MosaicDEL": "loss", "DUP": "gain", "MosaicDUP": "gain", "AOH": "loh",
+                   "CNLOH": "loh", "GAINLOH": "gain"}[call]  # fmt: skip
+            calls.setdefault(sid, []).append((c, int(p) + 1, end, cls, filt))
+    for t in truth:
+        cover = 0
+        seen = []
+        for c, s, e, cls, filt in calls.get(t["sample"], []):
+            ov = max(0, min(e, t["end"]) - max(s, t["start"]) + 1) if c == t["chrom"] else 0
+            if ov:
+                seen.append(f"{cls}:{filt}:{ov / (t['end'] - t['start'] + 1):.2f}")
+                if cls == CLASS[t["kind"]]:
+                    cover += ov  # fragments of one event add up
+        frac = cover / (t["end"] - t["start"] + 1)
+        t["detected"] = frac >= 0.5
+        t["overlap"] = round(frac, 3)
+        t["overlapping_calls"] = seen
+    cells: dict[str, dict[str, list[bool]]] = {}
+    for t in truth:
+        key = t["kind"] if t["fraction"] == 1.0 else f"{t['kind']} f={t['fraction']:.2f}"
+        cells.setdefault(key, {}).setdefault(f"{t['size'] // 1_000_000} Mb", []).append(t["detected"])
+    table = {k: {s: f"{sum(v)}/{len(v)}" for s, v in d.items()} for k, d in cells.items()}
+    n_calls = sum(len(v) for v in calls.values())
+    implanted_hits = sum(t["detected"] for t in truth)
+    summary = {"events": len(truth), "detected": implanted_hits, "table": table, "calls_total": n_calls}
+    (a.out / "summary.json").write_text(json.dumps(summary, indent=1))
+    (a.out / "truth_detected.json").write_text(json.dumps(truth, indent=1))
+    for k, v in table.items():
+        print(f"{k:14s} " + "  ".join(f"{s}: {v.get(s, '-')}" for s in ("1 Mb", "5 Mb", "20 Mb")))
+    print(f"detected {implanted_hits}/{len(truth)}; {n_calls} calls in {len(calls)} samples")
+    for t in truth:
+        if not t["detected"] and t["kind"] in ("DUP", "DEL", "LOH") and t["size"] >= 5_000_000:
+            print("  missed", t["kind"], t["size"], t["probes"], "probes:", t["overlapping_calls"])
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

scripts/v2/cn_tune.sh

@@ -0,0 +1,29 @@
+#!/usr/bin/env bash
+# CN tuning round on the Pi (GSE165845): reference = replicate-3 arrays of all 120 individuals, test = replicates 1
+# and 2 of the first N individuals; truth = NYGC SV rows (data/v2cn/cloud1/truth.tsv).
+#   bash scripts/v2/cn_tune.sh TAG [extra `simurg cn call` options...]      (env: N=20, MODEL=...)
+set -euo pipefail
+cd "$(dirname "$0")/../.."
+TAG=${1:?tag}; shift
+N=${N:-20}
+G=data/v2cn/gse_gt; M=data/v2cn/gse_idats/samples_map.tsv; W=data/v2cn/tune
+BPM=data/p8/gsa1/GSA-24v1-0_C2.bpm
+mkdir -p $W/ref $W/test
+if [[ ! -f $W/.dirs ]]; then
+  inds=$(tail -n +2 $M | cut -f3 | awk '!s[$0]++' | head -n "$N")
+  tail -n +2 $M | while IFS=$'\t' read -r sid gsm ind rep; do
+    [[ $rep == 3 ]] && ln -sfn "$(realpath $G/samples/$sid)" $W/ref/$sid
+    if [[ $rep != 3 ]] && grep -qx "$ind" <<< "$inds"; then ln -sfn "$(realpath $G/samples/$sid)" $W/test/$sid; fi
+  done
+  touch $W/.dirs
+fi
+MODEL=${MODEL:-$W/ref.simurg-cyto}
+if [[ ! -f $MODEL ]]; then
+  MEM=${MEM:-5G} MIN_FREE_MB=1500 bash scripts/pi_run.sh .venv/bin/simurg cn build-model --bpm $BPM --fasta data/p6/GRCh38.primary.fa \
+    --egt data/p8/gsa1/GSA-24v1-0_C1_ClusterFile.egt --reference-gtcs $W/ref --max-reference 120 --out $MODEL --quiet
+fi
+MEM=${MEM:-5G} MIN_FREE_MB=1500 bash scripts/pi_run.sh .venv/bin/simurg cn call --bpm $BPM --model $MODEL --gtcs $W/test \
+  --summary $G/sample_summary.tsv --out $W/$TAG --threads 3 --force --quiet "$@"
+.venv/bin/python scripts/v2/cn_validate_truth.py --calls $W/$TAG --map $M --truth data/v2cn/cloud1/truth.tsv --bpm $BPM \
+  --min-qual 20 --out $W/$TAG/val > /dev/null
+.venv/bin/python scripts/v2/cn_tune_report.py $W/$TAG $M

scripts/v2/cn_tune_report.py

@@ -0,0 +1,64 @@
+#!/usr/bin/env python3
+"""One-line-per-call-type summary of a tuning round: calls, fraction in NYGC truth, fraction seen in the other
+replicate; plus truth sensitivity by size bin (QUAL >= 20, PASS or SampleQuality)."""
+
+from __future__ import annotations
+
+import csv
+import json
+import sys
+from collections import defaultdict
+from pathlib import Path
+
+
+def ro(a: tuple[int, int], b: tuple[int, int]) -> float:
+    ov = min(a[1], b[1]) - max(a[0], b[0]) + 1
+    return 0.0 if ov <= 0 else min(ov / (a[1] - a[0] + 1), ov / (b[1] - b[0] + 1))
+
+
+def main() -> int:
+    run, mapf = Path(sys.argv[1]), Path(sys.argv[2])
+    with mapf.open() as fh:
+        smap = {r["sample_id"]: r["individual"] for r in csv.DictReader(fh, delimiter="\t")}
+    truth: dict[str, list[tuple[str, int, int, str]]] = defaultdict(list)
+    with open("data/v2cn/cloud1/truth.tsv") as fh:
+        for line in fh:
+            c, s, e, t, i = line.rstrip("\n").split("\t")
+            truth[i].append((c.removeprefix("chr"), int(s), int(e), t))
+    side = json.loads((run / "cn_call_manifest.json").read_text())
+    calls = {s["sample_id"]: [e for e in s["events"] if e["quality"] >= 20 and set(e["filters"]) <= {"PASS",
+             "SampleQuality"}] for s in side["samples"] if s.get("status") == "OK"}  # fmt: skip
+    cls = {"DEL": "DEL", "MDEL": "DEL", "DUP": "DUP", "MDUP": "DUP"}
+    by_ind: dict[str, list[str]] = defaultdict(list)
+    for sid in calls:
+        by_ind[smap[sid]].append(sid)
+    st: dict[str, list[int]] = defaultdict(lambda: [0, 0, 0, 0])
+    for sid, evs in calls.items():
+        ind = smap[sid]
+        others = [calls[x] for x in by_ind[ind] if x != sid]
+        for e in evs:
+            k = e["call"]
+            v = st[k]
+            v[0] += 1
+            if k in cls:
+                v[1] += any(t == cls[k] and c == e["chrom"] and ro((s, en), (e["start"], e["end"])) >= 0.5
+                            for c, s, en, t in truth[ind])  # fmt: skip
+            for oc in others:
+                v[2] += any(o["chrom"] == e["chrom"] and o["call"] == k and ro((o["start"], o["end"]),
+                            (e["start"], e["end"])) >= 0.5 for o in oc)  # fmt: skip
+                v[3] += 1
+    print(f"{run.name}: {len(calls)} samples")
+    for k, v in sorted(st.items()):
+        tr = f"{v[1] / v[0]:.2f}" if k in cls and v[0] else "  - "
+        rp = f"{v[2] / v[3]:.2f}" if v[3] else "  - "
+        print(f"  {k:8s} calls {v[0]:5d}  in-truth {tr}  replicated {rp}")
+    val = json.loads((run / "val" / "summary.json").read_text())
+    for t, bins in val["sensitivity"].items():
+        print(f"  sensitivity {t}: " + ", ".join(f"{b} {x['found']}/{x['truth']}" for b, x in sorted(bins.items())))
+    for t, bins in val.get("sensitivity_by_carrier_frequency", {}).items():
+        print(f"  by frequency {t}: " + ", ".join(f"{b} {x['found']}/{x['truth']}" for b, x in sorted(bins.items())))
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

scripts/v2/cn_validate_truth.py

@@ -0,0 +1,187 @@
+#!/usr/bin/env python3
+"""Validate `simurg cn call` on GSE165845 (120 1000 Genomes individuals × 3 GSA arrays) against the NYGC 30x
+structural-variant callset (1KGP_3202 GATK-SV/svtools freeze V3) and between replicates.
+
+    python scripts/v2/cn_validate_truth.py --calls CN_OUT --map samples_map.tsv --truth truth.tsv --bpm M.bpm --out DIR
+
+``truth.tsv``: ``chrom start end svtype sample`` for non-reference DEL/DUP genotypes of the 120 individuals
+(``bcftools query``; chr prefix removed). Matching: reciprocal overlap ≥ 50 % and the same class (DEL ↔ losses
+incl. mosaic, DUP ↔ gains). Only truth events with ≥ 10 array probes are counted as detectable. Sensitivity is also
+split by the event's carrier frequency among the individuals in ``truth.tsv`` (identical chrom/start/end/type):
+common events are normalised away by a median-LRR batch reference (K-027), so ``rare`` (< 5 %) is the primary metric.
+"""
+
+from __future__ import annotations
+
+import argparse
+import csv
+import json
+import statistics
+from collections import defaultdict
+from pathlib import Path
+
+import numpy as np
+
+from simurg.formats.bpm import read_bpm
+
+FREQ = [(0.0, 0.05, "rare<5%"), (0.05, 0.2, "5-20%"), (0.2, 1.01, "common>=20%")]
+BINS = [(25_000, 50_000), (50_000, 100_000), (100_000, 500_000), (500_000, 1_000_000), (1_000_000, 10**9)]
+CLASS = {"DEL": "DEL", "MDEL": "DEL", "DUP": "DUP", "MDUP": "DUP"}
+
+
+def ro(a: tuple[int, int], b: tuple[int, int]) -> float:
+    ov = min(a[1], b[1]) - max(a[0], b[0]) + 1
+    if ov <= 0:
+        return 0.0
+    return min(ov / (a[1] - a[0] + 1), ov / (b[1] - b[0] + 1))
+
+
+def bin_of(size: int) -> str:
+    for lo, hi in BINS:
+        if lo <= size < hi:
+            return f"{lo // 1000}-{hi // 1000 if hi < 10**9 else 'inf'}kb"
+    return "<25kb"
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    ap.add_argument("--calls", required=True, type=Path)
+    ap.add_argument("--map", required=True, type=Path)
+    ap.add_argument("--truth", required=True, type=Path)
+    ap.add_argument("--bpm", required=True)
+    ap.add_argument("--min-qual", type=float, default=20)
+    ap.add_argument("--ignore-filter", action="append", default=[], help="FILTER values not to exclude (repeatable)")
+    ap.add_argument("--out", required=True, type=Path)
+    a = ap.parse_args()
+    a.out.mkdir(parents=True, exist_ok=True)
+    man = read_bpm(a.bpm)
+    chroms = np.array(["X" if c == "XY" else c for c in man.chroms])
+    pos = man.map_infos.astype(np.int64)
+    probe_pos = {c: np.sort(pos[(chroms == c) & (pos > 0)]) for c in set(chroms.tolist())}
+
+    def n_probes(c: str, s: int, e: int) -> int:
+        p = probe_pos.get(c)
+        return 0 if p is None else int(np.searchsorted(p, e, "right") - np.searchsorted(p, s, "left"))
+
+    with a.map.open() as fh:
+        smap = {r["sample_id"]: r for r in csv.DictReader(fh, delimiter="\t")}
+    side = json.loads((a.calls / "cn_call_manifest.json").read_text())
+    calls: dict[str, list[dict]] = {}
+    for s in side["samples"]:
+        if s.get("status") != "OK":
+            continue
+        ign = set(a.ignore_filter) | {"PASS"}
+        calls[s["sample_id"]] = [e for e in s["events"] if e["quality"] >= a.min_qual and set(e["filters"]) <= ign]
+    truth: dict[str, list[tuple[str, int, int, str]]] = defaultdict(list)
+    carriers: dict[tuple[str, int, int, str], set[str]] = defaultdict(set)
+    with a.truth.open() as fh:
+        for line in fh:
+            c, s_, e_, t, ind = line.rstrip("\n").split("\t")
+            c = c[3:] if c.startswith("chr") else c
+            carriers[(c, int(s_), int(e_), t)].add(ind)
+            if t in ("DEL", "DUP") and int(e_) - int(s_) + 1 >= 25_000:
+                truth[ind].append((c, int(s_), int(e_), t))
+    n_ind = len({i for v in carriers.values() for i in v}) or 1
+
+    def freq_class(key: tuple[str, int, int, str]) -> str:
+        f = len(carriers[key]) / n_ind
+        return next(name for lo, hi, name in FREQ if lo <= f < hi)
+
+    # sensitivity per class and size bin
+    sens: dict[str, dict[str, list[int]]] = defaultdict(lambda: defaultdict(lambda: [0, 0]))
+    sens_freq: dict[str, dict[str, list[int]]] = defaultdict(lambda: defaultdict(lambda: [0, 0]))
+    prec = defaultdict(lambda: [0, 0])
+    per_sample = []
+    for sid, evs in calls.items():
+        ind = smap.get(sid, {}).get("individual")
+        if ind is None:
+            continue
+        tr = [t for t in truth.get(ind, []) if n_probes(t[0], t[1], t[2]) >= 10]
+        hit_t = 0
+        for c, s_, e_, t in tr:
+            ok = any(CLASS.get(e["call"]) == t and e["chrom"] == c and ro((s_, e_), (e["start"], e["end"])) >= 0.5
+                     for e in evs)  # fmt: skip
+            cell = sens[t][bin_of(e_ - s_ + 1)]
+            cell[0] += ok
+            cell[1] += 1
+            fc = sens_freq[t][freq_class((c, s_, e_, t))]
+            fc[0] += ok
+            fc[1] += 1
+            hit_t += ok
+        hit_c = 0
+        cnv_calls = [e for e in evs if e["call"] in CLASS]
+        for e in cnv_calls:
+            ok = any(t == CLASS[e["call"]] and c == e["chrom"] and ro((s_, e_), (e["start"], e["end"])) >= 0.5
+                     for c, s_, e_, t in truth.get(ind, []))  # fmt: skip
+            p = prec[CLASS[e["call"]]]
+            p[0] += ok
+            p[1] += 1
+            hit_c += ok
+        per_sample.append({"sample": sid, "individual": ind, "truth_detectable": len(tr), "truth_found": hit_t,
+                           "cnv_calls": len(cnv_calls), "calls_in_truth": hit_c,
+                           "loh_calls": sum(e["call_type"] == "LOH" for e in evs)})  # fmt: skip
+
+    # replicate reproducibility of CNV and LOH calls
+    by_ind: dict[str, list[str]] = defaultdict(list)
+    for sid in calls:
+        if sid in smap:
+            by_ind[smap[sid]["individual"]].append(sid)
+    rep = {"CNV": [0, 0], "LOH": [0, 0]}
+
+    def kind_of(e: dict) -> str | None:
+        if e["call_type"] == "LOH":
+            return "LOH"
+        return "CNV" if e["call"] in CLASS else None
+
+    def matched(e: dict, others: list[dict], kind: str) -> bool:
+        return any(
+            o["chrom"] == e["chrom"]
+            and kind_of(o) == kind
+            and ro((o["start"], o["end"]), (e["start"], e["end"])) >= 0.5
+            for o in others
+        )
+
+    for sids in by_ind.values():
+        for i, s1 in enumerate(sids):
+            for e in calls[s1]:
+                kind = kind_of(e)
+                if kind is None:
+                    continue
+                for j, s2 in enumerate(sids):
+                    if j != i:
+                        rep[kind][0] += matched(e, calls[s2], kind)
+                        rep[kind][1] += 1
+
+    def rate(v: list[int]) -> float | None:
+        return round(v[0] / v[1], 4) if v[1] else None
+
+    summary = {
+        "samples": len(per_sample),
+        "min_qual": a.min_qual,
+        "ignored_filters": sorted(a.ignore_filter),
+        "sensitivity": {
+            t: {b: {"found": v[0], "truth": v[1], "rate": rate(v)} for b, v in sorted(d.items())}
+            for t, d in sens.items()
+        },
+        "sensitivity_by_carrier_frequency": {
+            t: {b: {"found": v[0], "truth": v[1], "rate": rate(v)} for b, v in sorted(d.items())}
+            for t, d in sens_freq.items()
+        },
+        "precision": {t: {"in_truth": v[0], "calls": v[1], "rate": rate(v)} for t, v in prec.items()},
+        "replicate_reproducibility": {
+            k: {"matched": v[0], "comparisons": v[1], "rate": rate(v)} for k, v in rep.items()
+        },
+        "calls_per_sample_median": statistics.median([p["cnv_calls"] for p in per_sample]) if per_sample else None,
+        "loh_calls_per_sample_median": statistics.median([p["loh_calls"] for p in per_sample]) if per_sample else None,
+    }
+    (a.out / "summary.json").write_text(json.dumps(summary, indent=1))
+    with (a.out / "per_sample.tsv").open("w") as fh:
+        w = csv.DictWriter(fh, fieldnames=list(per_sample[0]) if per_sample else ["sample"], delimiter="\t")
+        w.writeheader()
+        w.writerows(per_sample)
+    print(json.dumps(summary, indent=1))
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

scripts/v2/methyl_oracle_compare.py

@@ -0,0 +1,81 @@
+#!/usr/bin/env python3
+"""Compare `simurg methyl qc` numerics with the sesame oracle (scripts/v2/sesame_oracle.R) on the same probes.
+
+python scripts/v2/methyl_oracle_compare.py --manifest M.csv --sesame DIR --out summary.json PREFIX...
+"""
+
+from __future__ import annotations
+
+import argparse
+import csv
+import json
+from pathlib import Path
+
+import numpy as np
+
+from simurg.formats.idat import read_idat
+from simurg.formats.methyl_manifest import read_methyl_manifest
+from simurg.signal import methyl as sig
+
+
+def _q(a: np.ndarray) -> dict[str, float]:
+    a = a[np.isfinite(a)]
+    if not len(a):
+        return {}
+    return {k: float(np.quantile(a, q)) for k, q in (("median", 0.5), ("p99", 0.99), ("max", 1.0))}
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--manifest", required=True)
+    ap.add_argument("--sesame", required=True, type=Path)
+    ap.add_argument("--out", required=True, type=Path)
+    ap.add_argument("prefixes", nargs="+")
+    a = ap.parse_args()
+    man = read_methyl_manifest(a.manifest)
+    pos = {p: i for i, p in enumerate(man.probe_ids)}
+    res = {}
+    for prefix in a.prefixes:
+        # min_beads=0: the oracle runs without bead-count masking (see sesame_oracle.R)
+        s = sig.assemble(read_idat(prefix + "_Grn.idat"), read_idat(prefix + "_Red.idat"), man, min_beads=0)
+        rows = list(csv.DictReader((a.sesame / (Path(prefix).name + ".tsv")).open(), delimiter="\t"))
+        idx = np.array([pos[r["probe_id"]] for r in rows])
+
+        def col(name: str, rows: list[dict[str, str]] = rows) -> np.ndarray:
+            return np.array([float(r[name]) if r[name] not in ("NA", "") else np.nan for r in rows])
+
+        mask_s = np.array([r["mask"] == "TRUE" for r in rows])
+        variants = {}
+        for neg in (True, False):
+            p = sig.detection_p(s, combine_neg=neg)
+            n, fg, fr = sig.noob(s, combine_neg=neg)
+            variants[neg] = (p, n, fg, fr, sig.beta_values(n))
+        keep = ~mask_s & ~s.masked[idx]
+        sp, sm, su, sb = col("pval"), col("m_noob"), col("u_noob"), col("beta_noob")
+        ana = man.analytical()[idx] & keep
+        out: dict[str, object] = {
+            "probes": len(rows),
+            "masked_sesame": int(mask_s.sum()),
+            "masked_simurg": int(s.masked.sum()),
+        }
+        for neg, (p, n, fg, fr, b) in variants.items():
+            op, om, ou, ob = p[idx], n.m[idx], n.u[idx], b[idx]
+            out[f"combine_neg={neg}"] = {
+                "noob_fit": {"green": vars(fg), "red": vars(fr)},
+                "pval_abs_diff": _q(np.abs(op - sp)[keep]),
+                "pval_identical_fraction": float((op[keep] == sp[keep]).mean()),
+                "detection_call_agreement_0.05": float(((op < 0.05) == (sp < 0.05))[keep].mean()),
+                "prop_passing": {"simurg": float((op[ana] < 0.05).mean()), "sesame": float((sp[ana] < 0.05).mean())},
+                "noob_m_rel_diff": _q(np.abs(om - sm)[keep] / np.maximum(np.abs(sm[keep]), 1)),
+                "noob_u_rel_diff": _q(np.abs(ou - su)[keep] / np.maximum(np.abs(su[keep]), 1)),
+                "beta_abs_diff": _q(np.abs(ob - sb)[keep]),
+                "beta_corr": float(np.corrcoef(ob[keep & np.isfinite(sb)], sb[keep & np.isfinite(sb)])[0, 1]),
+            }
+        res[Path(prefix).name] = out
+    a.out.write_text(json.dumps(res, indent=1))
+    print(json.dumps(res, indent=1))
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

scripts/v2/sesame_oracle.R

@@ -0,0 +1,48 @@
+# Oracle for `simurg methyl qc` numerics: sesame (MIT) pOOBAH, noob and beta values on the same probe set.
+# Usage: Rscript sesame_oracle.R probes.tsv controls.tsv OUTDIR IDAT_PREFIX...
+# probes.tsv / controls.tsv are written by `simurg methyl convert-manifest` (MET-003: sesame manifest layout).
+suppressPackageStartupMessages(library(sesame))
+args <- commandArgs(trailingOnly = TRUE)
+p <- read.delim(args[1], stringsAsFactors = FALSE, na.strings = "NA", colClasses = c("character", "numeric",
+                "numeric", "character"))
+mft <- data.frame(Probe_ID = p$probe_id, M = p$m, U = p$u, col = ifelse(is.na(p$col), "2", p$col),
+                  stringsAsFactors = FALSE)
+mft$col <- factor(mft$col, levels = c("G", "R", "2"))
+ctl <- read.delim(args[2], stringsAsFactors = FALSE)
+controls <- data.frame(Address = as.character(ctl$address), Type = ctl$type, Color_Channel = ctl$color_channel,
+                       Name = ctl$name, stringsAsFactors = FALSE)
+out <- args[3]
+dir.create(out, showWarnings = FALSE, recursive = TRUE)
+cat("sesame", as.character(packageVersion("sesame")), "\n")
+# platform-level annotation (masks, signatures) that sesame looks up even when the manifest is given
+invisible(tryCatch(sesameDataCache("EPIC"), error = function(e) message("cache EPIC: ", conditionMessage(e))))
+invisible(tryCatch(sesameDataCache(c("idatSignature", "KYCG.EPIC.Mask.20220123")),
+                   error = function(e) message("cache: ", conditionMessage(e))))
+inband <- function(s) {
+  m <- ifelse(s$col == "R", s$MR, ifelse(s$col == "G", s$MG, s$UG))
+  u <- ifelse(s$col == "R", s$UR, ifelse(s$col == "G", s$UG, s$UR))
+  list(m = m, u = u)
+}
+for (prefix in args[-(1:3)]) {
+  # no bead-count masking here (sesame's min_beads also masks every Infinium II probe, whose M address is NA);
+  # the comparison script runs simurg with the same setting
+  read <- function() readIDATpair(prefix, platform = "EPIC", manifest = mft, controls = controls)
+  sdf <- tryCatch(read(), error = function(e) {
+    message("first read failed (", conditionMessage(e), "); caching idatSignature and retrying")
+    sesameDataCache("idatSignature")
+    read()
+  })
+  neg <- tryCatch(negControls(sdf), error = function(e) NULL)
+  cat("negative controls found by sesame:", if (is.null(neg)) "error" else nrow(neg), "\n")
+  print(head(sdf[grepl("^ctl", sdf$Probe_ID), ], 3))
+  pv <- pOOBAH(sdf, return.pval = TRUE)
+  sn <- noob(sdf)
+  ib <- inband(sn)
+  b <- getBetas(sn, mask = FALSE)
+  res <- data.frame(probe_id = sdf$Probe_ID, mask = sdf$mask, pval = pv[sdf$Probe_ID],
+                    m_noob = ib$m, u_noob = ib$u, beta_noob = b[sdf$Probe_ID])
+  f <- file.path(out, paste0(basename(prefix), ".tsv"))
+  write.table(res, f, sep = "\t", quote = FALSE, row.names = FALSE)
+  cat(basename(prefix), nrow(res), "probes;", sum(res$mask), "masked\n")
+}
+sessionInfo()

src/simurg/__init__.py

@@ -4,5 +4,5 @@ Specification: SIMURGARRAY_MASTER_PLAN.md (algorithms, roadmap) and
 SIMURGARRAY_FUNCTIONAL_SPEC.md (externally visible behaviour, item IDs such as CLI-021).
 """
 
-__version__ = "1.0.0"
+__version__ = "2.0.0"
 PRODUCT_NAME = "SimurgArray"

src/simurg/_core.py

@@ -0,0 +1,24 @@
+"""Optional compiled kernels (``rust/simurg-core``, plan §18 P9).
+
+``core`` is the ``simurg_core`` extension module when it is installed, else ``None``; every caller keeps a pure
+Python/NumPy path with bit-identical results. ``SIMURG_PURE_PYTHON=1`` forces the pure paths (tests, debugging).
+"""
+
+from __future__ import annotations
+
+import os
+from types import ModuleType
+
+core: ModuleType | None
+if os.environ.get("SIMURG_PURE_PYTHON", "") not in ("", "0"):
+    core = None
+else:
+    try:
+        import simurg_core as core  # type: ignore[no-redef]
+    except ImportError:  # pragma: no cover - depends on the installation
+        core = None
+
+
+def backend() -> str:
+    """``"rust <version>"`` or ``"python"`` (recorded in run manifests)."""
+    return f"rust {core.__version__}" if core is not None else "python"

src/simurg/cli/main.py

@@ -7,6 +7,7 @@ Later phases add the other commands of CLI-001.
 from __future__ import annotations
 
 import json
+import os
 import sys
 from collections.abc import Callable
 from pathlib import Path
@@ -58,6 +59,11 @@ def run_guarded(reporter: Reporter, fn: Callable[[], int]) -> None:
     code = ExitCode.OK
     try:
         code = ExitCode(fn())
+        sys.stdout.flush()  # surface a closed pipe here, not at interpreter exit
+    except BrokenPipeError:
+        # the consumer of stdout stopped reading (`| head`): silence further writes, exit like a SIGPIPE'd tool
+        os.dup2(os.open(os.devnull, os.O_WRONLY), sys.stdout.fileno())
+        code = ExitCode.SIGPIPE
     except SimurgError as e:
         reporter.error(e.code, e.message + (f"\n  hint: {e.hint}" if e.hint else ""), exc=e, file=e.path)
         code = e.exit_code
@@ -660,6 +666,424 @@ def qc_controls_cmd(
     run_guarded(rep, lambda: run_qc_controls(opts, rep))
 
 
+# --------------------------------------------------------------------------------------------------------
+cn_app = typer.Typer(help="Genome-wide copy number, LOH and mosaicism (CYTO-001).")
+app.add_typer(cn_app, name="cn")
+
+
+@cn_app.command("build-model")
+def cn_build_model_cmd(
+    bpm: Annotated[Path, typer.Option("--bpm", help="BPM manifest (required).")],
+    fasta: Annotated[Path, typer.Option("--fasta", help="Reference FASTA of the manifest's build (required).")],
+    out: Annotated[Path, typer.Option("--out", help="Output model file (.simurg-cyto).")],
+    egt: Annotated[Path | None, typer.Option("--egt", help="Cluster file: population B-allele frequencies.")] = None,
+    reference_gtcs: Annotated[
+        Path | None, typer.Option("--reference-gtcs", help="GTCs of normal samples (>= 24) for the batch reference.")
+    ] = None,
+    max_reference: Annotated[int, typer.Option("--max-reference", min=1, help="Random subset size.")] = 100,
+    log_json: LogJson = False,
+    debug: Debug = False,
+    quiet: Quiet = False,
+) -> None:
+    """Cyto model: GC covariates, PFB and an optional batch reference per probe (CYTO-038, IN-051)."""
+    from simurg.pipeline.cn import run_build_model
+
+    rep = _reporter("cn build-model", out.parent, log_json, debug, quiet)
+    run_guarded(rep, lambda: run_build_model(bpm, fasta, out, rep, egt, "auto", reference_gtcs, max_reference))
+
+
+def _kb(v: int) -> int:
+    if not 0 <= v <= 250_000:
+        raise typer.BadParameter("sizes are 0-250000 kb")
+    return v * 1000
+
+
+@cn_app.command("call")
+def cn_call_cmd(
+    bpm: Annotated[Path, typer.Option("--bpm", help="BPM manifest (required).")],
+    model: Annotated[Path, typer.Option("--model", help="Cyto model from cn build-model (required).")],
+    out: Annotated[Path, typer.Option("--out", help="Output folder (required).")],
+    gtcs: Annotated[Path | None, typer.Option("--gtcs", help="Folder searched recursively for GTC files.")] = None,
+    summary: Annotated[
+        Path | None, typer.Option("--summary", help="sample_summary.tsv (default: next to GTCs).")
+    ] = None,
+    fasta: Annotated[Path | None, typer.Option("--fasta", help="FASTA for REF padding bases.")] = None,
+    min_del_probes: Annotated[int, typer.Option("--min-del-probes", min=0, max=250000)] = 10,
+    min_del_size: Annotated[int, typer.Option("--min-del-size", help="kb")] = 25,
+    min_dup_probes: Annotated[int, typer.Option("--min-dup-probes", min=0, max=250000)] = 10,
+    min_dup_size: Annotated[int, typer.Option("--min-dup-size", help="kb")] = 50,
+    min_loh_probes: Annotated[int, typer.Option("--min-loh-probes", min=0, max=250000)] = 500,
+    min_loh_size: Annotated[int, typer.Option("--min-loh-size", help="kb")] = 3000,
+    min_gainloh_probes: Annotated[int, typer.Option("--min-gainloh-probes", min=0, max=250000)] = 300,
+    min_gainloh_size: Annotated[int, typer.Option("--min-gainloh-size", help="kb")] = 500,
+    min_mdel_probes: Annotated[int, typer.Option("--min-mdel-probes", min=0, max=250000)] = 10,
+    min_mdel_size: Annotated[int, typer.Option("--min-mdel-size", help="kb")] = 25,
+    min_mdup_probes: Annotated[int, typer.Option("--min-mdup-probes", min=0, max=250000)] = 10,
+    min_mdup_size: Annotated[int, typer.Option("--min-mdup-size", help="kb")] = 50,
+    min_mloh_probes: Annotated[int, typer.Option("--min-mloh-probes", min=0, max=250000)] = 500,
+    min_mloh_size: Annotated[int, typer.Option("--min-mloh-size", help="kb")] = 3000,
+    smoothing: Annotated[int, typer.Option("--smoothing", min=0, max=1000)] = 5,
+    max_mosaic_fraction: Annotated[float, typer.Option("--max-mosaic-fraction")] = 1.0,
+    min_mosaic_fraction: Annotated[float, typer.Option("--min-mosaic-fraction")] = 0.10,
+    filter_qual: Annotated[int, typer.Option("--filter-qual", min=0, max=60)] = 10,
+    cn_cap: Annotated[int, typer.Option("--cn-cap", min=3, max=8)] = 4,
+    gc_correct: Annotated[bool, typer.Option("--gc-correct/--no-gc-correct")] = True,
+    unknown_sex_policy: Annotated[str, typer.Option("--unknown-sex-policy")] = "skip-sex-chromosomes",
+    gt_mode: Annotated[str, typer.Option("--gt-mode", help="vcf|compat")] = "vcf",
+    svlen_mode: Annotated[str, typer.Option("--svlen-mode", help="signed|positive")] = "signed",
+    mask_bed: Annotated[Path | None, typer.Option("--mask-bed", help="Mask BED (default: built-in).")] = None,
+    no_bgzip: Annotated[bool, typer.Option("--no-bgzip", help="Plain VCF (cn annotate then refuses it).")] = False,
+    mosaic_prior: Annotated[
+        float, typer.Option("--mosaic-prior", help="Advanced: mosaic states' switch mass.")
+    ] = 0.004,
+    switch_distance: Annotated[
+        float, typer.Option("--switch-distance", help="Advanced: HMM switch distance D (bp).")
+    ] = 5e8,
+    probe_lrr_sd: Annotated[
+        bool, typer.Option("--probe-lrr-sd/--no-probe-lrr-sd", help="Advanced: per-probe LRR noise from the model.")
+    ] = True,
+    threads: ThreadsOpt = 0,
+    force: ForceOpt = False,
+    log_json: LogJson = False,
+    debug: Debug = False,
+    quiet: Quiet = False,
+) -> None:
+    """GTCs -> per-sample CNV/LOH/mosaic VCF, segments and summary (CLI-028)."""
+    from simurg.cn.events import Thresholds
+    from simurg.pipeline.cn import CnCallOptions, run_cn_call
+
+    if unknown_sex_policy not in ("skip-sex-chromosomes", "female", "male", "error"):
+        raise typer.BadParameter("--unknown-sex-policy: skip-sex-chromosomes|female|male|error")
+    if gt_mode not in ("vcf", "compat") or svlen_mode not in ("signed", "positive"):
+        raise typer.BadParameter("--gt-mode vcf|compat, --svlen-mode signed|positive")
+    if not 0 < max_mosaic_fraction <= 1 or not 0 <= min_mosaic_fraction < 1:
+        raise typer.BadParameter("mosaic fractions: 0 < max <= 1, 0 <= min < 1")
+    th = Thresholds(
+        probes={"del": min_del_probes, "dup": min_dup_probes, "loh": min_loh_probes, "gainloh": min_gainloh_probes,
+                "mdel": min_mdel_probes, "mdup": min_mdup_probes, "mloh": min_mloh_probes},
+        size_bp={"del": _kb(min_del_size), "dup": _kb(min_dup_size), "loh": _kb(min_loh_size),
+                 "gainloh": _kb(min_gainloh_size), "mdel": _kb(min_mdel_size), "mdup": _kb(min_mdup_size),
+                 "mloh": _kb(min_mloh_size)},
+        max_mosaic_fraction=max_mosaic_fraction, min_mosaic_fraction=min_mosaic_fraction, filter_qual=filter_qual,
+    )  # fmt: skip
+    rep = _reporter("cn call", out, log_json, debug, quiet)
+    opts = CnCallOptions(
+        out=out, bpm=bpm, model=model, gtcs=gtcs, summary=summary, fasta=fasta, thresholds=th, smoothing=smoothing,
+        cn_cap=cn_cap, gc_correct=gc_correct, unknown_sex_policy=unknown_sex_policy, gt_mode=gt_mode,
+        svlen_mode=svlen_mode, mask_bed=mask_bed, bgzip=not no_bgzip, mosaic_prior=mosaic_prior,
+        switch_d=switch_distance, probe_lrr_sd=probe_lrr_sd, threads=threads, force=force,
+        argv=sys.argv[1:],
+    )  # fmt: skip
+    run_guarded(rep, lambda: run_cn_call(opts, rep))
+
+
+@cn_app.command("annotate")
+def cn_annotate_cmd(
+    vcfs: Annotated[Path, typer.Option("--vcfs", help="Output folder of cn call (required).")],
+    bundle: Annotated[
+        Path | None, typer.Option("--annotation-bundle", help="Bundle from scripts/build_annotation_bundle.sh.")
+    ] = None,
+    out: Annotated[Path | None, typer.Option("--out", help="Output folder (default: --vcfs).")] = None,
+    min_qual: Annotated[int, typer.Option("--min-qual", min=0, max=60)] = 20,
+    size_basis: Annotated[str, typer.Option("--size-basis", help="effective|raw")] = "effective",
+    iscn_min_size: Annotated[int, typer.Option("--iscn-min-size", min=0, help="bp")] = 0,
+    force: ForceOpt = False,
+    log_json: LogJson = False,
+    debug: Debug = False,
+    quiet: Quiet = False,
+) -> None:
+    """cn call output -> per-sample JSON report with ISCN 2020 strings, cytobands and genes (CLI-029)."""
+    from simurg.pipeline.cn import CnAnnotateOptions, run_cn_annotate
+
+    if size_basis not in ("effective", "raw"):
+        raise typer.BadParameter("--size-basis effective|raw")
+    rep = _reporter("cn annotate", out or vcfs, log_json, debug, quiet)
+    opts = CnAnnotateOptions(vcfs=vcfs, bundle=bundle, out=out, min_qual=min_qual, size_basis=size_basis,
+                             iscn_min_size=iscn_min_size, force=force)  # fmt: skip
+    run_guarded(rep, lambda: run_cn_annotate(opts, rep))
+
+
+# --------------------------------------------------------------------------------------------------------
+@app.command("train")
+def train_cmd(
+    gtcs: Annotated[Path, typer.Option("--gtcs", help="Folder searched recursively for training GTCs (required).")],
+    bpm: Annotated[Path, typer.Option("--bpm", help="BPM manifest (required).")],
+    out: Annotated[Path, typer.Option("--out", help="Output cluster file (.egt, required).")],
+    min_samples: Annotated[int, typer.Option("--min-samples", min=1)] = 100,
+    max_samples: Annotated[int, typer.Option("--max-samples", min=1)] = 2000,
+    force: ForceOpt = False,
+    log_json: LogJson = False,
+    debug: Debug = False,
+    quiet: Quiet = False,
+) -> None:
+    """ArrayTrain: train an EGT cluster file from a cohort's GTCs (plan §12)."""
+    from simurg.pipeline.train import TrainOptions, run_train
+
+    rep = _reporter("train", out.parent, log_json, debug, quiet)
+    o = TrainOptions(gtcs=gtcs, bpm=bpm, out=out, min_samples=min_samples, max_samples=max_samples, force=force)
+    run_guarded(rep, lambda: run_train(o, rep))
+
+
+# --------------------------------------------------------------------------------------------------------
+pgx_app = typer.Typer(help="Pharmacogenomics: targeted copy number, star alleles, phenotypes (v2).")
+app.add_typer(pgx_app, name="pgx")
+pgx_cn_app = typer.Typer(help="Targeted PGx copy number (PCN-001).")
+pgx_app.add_typer(pgx_cn_app, name="copy-number")
+
+
+pgx_db_app = typer.Typer(help="PGx allele-definition database (PGX-001).")
+pgx_app.add_typer(pgx_db_app, name="db")
+
+
+@pgx_db_app.command("build")
+def pgx_db_build_cmd(
+    sources: Annotated[Path, typer.Option("--sources", help="Folder from scripts/pgx/fetch_sources.sh (required).")],
+    fasta: Annotated[Path, typer.Option("--fasta", help="GRCh38 FASTA for variant normalisation (required).")],
+    out: Annotated[Path, typer.Option("--out", help="Output folder for pgx_db-<schema>-<sha12>.sqlite.")] = Path("."),
+    manifests: Annotated[
+        list[Path] | None, typer.Option("--manifests", help="BPM manifest(s) for the probe map (repeatable).")
+    ] = None,
+    csv: Annotated[list[Path] | None, typer.Option("--csv", help="CSV manifest(s), same order (indels).")] = None,
+    mask: Annotated[list[Path] | None, typer.Option("--mask", help="Probe mask TSV(s) (PGX-010).")] = None,
+    force: ForceOpt = False,
+    log_json: LogJson = False,
+    debug: Debug = False,
+    quiet: Quiet = False,
+) -> None:
+    """Build the PGx DB from public sources (CLI-035, PGX-002)."""
+    from simurg.pipeline.pgx_db import PgxDbOptions, run_db_build
+
+    rep = _reporter("pgx db build", out, log_json, debug, quiet)
+    o = PgxDbOptions(sources=sources, fasta=fasta, out=out, manifests=manifests or [], csvs=csv or [],
+                     masks=mask or [], force=force)  # fmt: skip
+    run_guarded(rep, lambda: run_db_build(o, rep))
+
+
+pgx_star_app = typer.Typer(help="Star-allele calling (PGX-011).")
+pgx_app.add_typer(pgx_star_app, name="star-allele")
+
+
+@pgx_star_app.command("call")
+def pgx_star_call_cmd(
+    vcfs: Annotated[list[Path], typer.Option("--vcfs", help="Folder(s) with SNV + PGx CN VCFs (repeatable).")],
+    db: Annotated[Path, typer.Option("--db", help="PGx DB from pgx db build (required).")],
+    out: Annotated[Path, typer.Option("--out", help="Output folder (required).")],
+    population: Annotated[str, typer.Option("--population", help="Frequency prior population.")] = "global",
+    max_candidates: Annotated[int, typer.Option("--max-candidates", min=1)] = 10,
+    feasible_ratio: Annotated[float, typer.Option("--feasible-ratio", min=0.0, max=1.0)] = 0.5,
+    low_gs: Annotated[float, typer.Option("--low-gs")] = 0.15,
+    no_cn: Annotated[bool, typer.Option("--no-cn", help="Run without PGx CN VCFs.")] = False,
+    allow_foreign_vcf: Annotated[bool, typer.Option("--allow-foreign-vcf")] = False,
+    gene: Annotated[list[str] | None, typer.Option("--gene", help="Restrict to gene(s).")] = None,
+    force: ForceOpt = False,
+    log_json: LogJson = False,
+    debug: Debug = False,
+    quiet: Quiet = False,
+) -> None:
+    """Call star-allele diplotypes (CLI-033, PGX-012 ... PGX-021)."""
+    from simurg.pipeline.pgx_star import StarOptions, run_star_call
+
+    rep = _reporter("pgx star-allele call", out, log_json, debug, quiet)
+    o = StarOptions(vcfs=vcfs, db=db, out=out, population=population, max_candidates=max_candidates,
+                    feasible_ratio=feasible_ratio, low_gs=low_gs, no_cn=no_cn, allow_foreign_vcf=allow_foreign_vcf,
+                    genes=gene or [], force=force)  # fmt: skip
+    run_guarded(rep, lambda: run_star_call(o, rep))
+
+
+def _pgx_phenotype(calls: Path, db: Path, out: Path | None, guideline: str, force: bool, log_json: bool,
+                   debug: bool, quiet: bool, name: str) -> None:  # fmt: skip
+    from simurg.pipeline.pgx_pheno import PhenoOptions, run_phenotype
+
+    if guideline not in ("cpic", "dpwg"):
+        raise typer.BadParameter("--guideline cpic|dpwg")
+    rep = _reporter(name, out or calls, log_json, debug, quiet)
+    o = PhenoOptions(calls=calls, db=db, out=out, guideline=guideline, force=force)
+    run_guarded(rep, lambda: run_phenotype(o, rep))
+
+
+@pgx_app.command("phenotype")
+def pgx_phenotype_cmd(
+    calls: Annotated[Path, typer.Option("--calls", help="Output folder of pgx star-allele call (required).")],
+    db: Annotated[Path, typer.Option("--db", help="PGx DB (required).")],
+    out: Annotated[Path | None, typer.Option("--out", help="Output folder (default: --calls).")] = None,
+    guideline: Annotated[str, typer.Option("--guideline", help="cpic|dpwg")] = "cpic",
+    force: ForceOpt = False,
+    log_json: LogJson = False,
+    debug: Debug = False,
+    quiet: Quiet = False,
+) -> None:
+    """Phenotypes from star-allele calls: per-sample PGx JSON and diplotype_annotations.tsv (CLI-034)."""
+    _pgx_phenotype(calls, db, out, guideline, force, log_json, debug, quiet, "pgx phenotype")
+
+
+@pgx_app.command("annotate", hidden=True)
+def pgx_annotate_cmd(
+    calls: Annotated[Path, typer.Option("--calls")],
+    db: Annotated[Path, typer.Option("--db")],
+    out: Annotated[Path | None, typer.Option("--out")] = None,
+    guideline: Annotated[str, typer.Option("--guideline")] = "cpic",
+    force: ForceOpt = False,
+    log_json: LogJson = False,
+    debug: Debug = False,
+    quiet: Quiet = False,
+) -> None:
+    """Alias of pgx phenotype (CLI-034)."""
+    _pgx_phenotype(calls, db, out, guideline, force, log_json, debug, quiet, "pgx annotate")
+
+
+@pgx_app.command("coverage")
+def pgx_coverage_cmd(
+    bpm: Annotated[Path, typer.Option("--bpm", help="BPM manifest (required).")],
+    db: Annotated[Path, typer.Option("--db", help="PGx DB built with this manifest (required).")],
+    out: Annotated[Path, typer.Option("--out", help="Output folder.")] = Path("."),
+    log_json: LogJson = False,
+    debug: Debug = False,
+    quiet: Quiet = False,
+) -> None:
+    """Which alleles this array can call: pgx_coverage.tsv (PGX-009, PGX-035)."""
+    from simurg.pipeline.pgx_db import run_coverage
+
+    rep = _reporter("pgx coverage", out, log_json, debug, quiet)
+    run_guarded(rep, lambda: run_coverage(bpm, db, out, rep))
+
+
+@pgx_cn_app.command("train")
+def pgx_cn_train_cmd(
+    gtcs: Annotated[Path, typer.Option("--gtcs", help="Folder searched recursively for training GTCs (required).")],
+    bpm: Annotated[Path, typer.Option("--bpm", help="BPM manifest (required).")],
+    fasta: Annotated[Path, typer.Option("--fasta", help="GRCh38 FASTA (required).")],
+    out: Annotated[Path, typer.Option("--out", help="Output folder.")] = Path("."),
+    mask: Annotated[Path | None, typer.Option("--mask", help="Probe mask TSV (IN-052).")] = None,
+    regions: Annotated[Path | None, typer.Option("--regions", help="Region BED (default: built-in v1).")] = None,
+    summary: Annotated[Path | None, typer.Option("--summary", help="sample_summary.tsv (assay control).")] = None,
+    csv: Annotated[Path | None, typer.Option("--csv", help="CSV manifest (Assay Format → platform).")] = None,
+    platform: Annotated[str, typer.Option("--platform", help="auto|LCG|EX")] = "auto",
+    min_samples: Annotated[int, typer.Option("--min-samples", min=1)] = 96,
+    min_passing: Annotated[int, typer.Option("--min-passing", min=1)] = 90,
+    max_samples: Annotated[int, typer.Option("--max-samples", min=1)] = 3000,
+    train_lrr_sd_max: Annotated[float, typer.Option("--train-lrr-sd-max")] = 0.20,
+    train_call_rate_min: Annotated[float, typer.Option("--train-call-rate-min")] = 0.99,
+    train_assay_control_min: Annotated[float, typer.Option("--train-assay-control-min")] = 1.0,
+    force: ForceOpt = False,
+    log_json: LogJson = False,
+    debug: Debug = False,
+    quiet: Quiet = False,
+) -> None:
+    """Train a PGx CN model (CLI-032, PCN-005)."""
+    from simurg.pipeline.pgx_cn import PgxTrainOptions, run_train
+
+    if platform not in ("auto", "LCG", "EX"):
+        raise typer.BadParameter("--platform auto|LCG|EX")
+    rep = _reporter("pgx copy-number train", out, log_json, debug, quiet)
+    o = PgxTrainOptions(gtcs=gtcs, bpm=bpm, fasta=fasta, out=out, mask=mask, regions=regions, summary=summary,
+                        platform=platform, csv=csv, min_samples=min_samples, min_passing=min_passing,
+                        max_samples=max_samples,
+                        lrr_sd_max=train_lrr_sd_max, call_rate_min=train_call_rate_min,
+                        assay_control_min=train_assay_control_min, force=force)  # fmt: skip
+    run_guarded(rep, lambda: run_train(o, rep))
+
+
+@pgx_cn_app.command("call")
+def pgx_cn_call_cmd(
+    gtcs: Annotated[Path, typer.Option("--gtcs", help="Folder searched recursively for GTC files (required).")],
+    model: Annotated[Path, typer.Option("--model", help="PGx CN model (.simurg-pgxcn, required).")],
+    fasta: Annotated[Path, typer.Option("--fasta", help="GRCh38 FASTA with alt contigs (required).")],
+    out: Annotated[Path, typer.Option("--out", help="Output folder (required).")],
+    min_batch: Annotated[int, typer.Option("--min-batch", min=1)] = 24,
+    min_passing: Annotated[int, typer.Option("--min-passing", min=1)] = 22,
+    recommended_passing: Annotated[int, typer.Option("--recommended-passing", min=1)] = 96,
+    lrr_sd_max: Annotated[float, typer.Option("--lrr-sd-max")] = 0.20,
+    cluster_mismatch: Annotated[str, typer.Option("--cluster-mismatch", help="error|warn")] = "error",
+    bafs_in_cn: Annotated[
+        bool, typer.Option("--bafs-in-cn", help="Experimental: add CN-consistent BAF evidence to the CN posterior.")
+    ] = False,
+    no_bgzip: Annotated[bool, typer.Option("--no-bgzip")] = False,
+    force: ForceOpt = False,
+    log_json: LogJson = False,
+    debug: Debug = False,
+    quiet: Quiet = False,
+) -> None:
+    """Call targeted PGx copy number per region (CLI-031, PCN-009)."""
+    from simurg.pipeline.pgx_cn import PgxCallOptions, run_call
+
+    if cluster_mismatch not in ("error", "warn"):
+        raise typer.BadParameter("--cluster-mismatch error|warn")
+    if min_passing > min_batch:
+        raise typer.BadParameter("--min-passing must be <= --min-batch")
+    rep = _reporter("pgx copy-number call", out, log_json, debug, quiet)
+    o = PgxCallOptions(gtcs=gtcs, model=model, fasta=fasta, out=out, min_batch=min_batch, min_passing=min_passing,
+                       recommended_passing=recommended_passing, lrr_sd_max=lrr_sd_max,
+                       cluster_mismatch=cluster_mismatch, bafs_in_cn=bafs_in_cn, bgzip=not no_bgzip,
+                       force=force)  # fmt: skip
+    run_guarded(rep, lambda: run_call(o, rep))
+
+
+# --------------------------------------------------------------------------------------------------------
+methyl_app = typer.Typer(help="Methylation arrays: control QC, detection, beta values (MET-001).")
+app.add_typer(methyl_app, name="methyl")
+
+
+@methyl_app.command("qc")
+def methyl_qc_cmd(
+    manifest: Annotated[Path, typer.Option("--manifest", help="Methylation CSV manifest (required).")],
+    out: Annotated[Path, typer.Option("--out", help="Output folder (required).")],
+    idats: Annotated[Path | None, typer.Option("--idats", help="Folder searched recursively for IDAT pairs.")] = None,
+    sheet: Annotated[Path | None, typer.Option("--sheet", help="Sample sheet (CSV).")] = None,
+    config: Annotated[Path | None, typer.Option("--config", help="QC threshold YAML.")] = None,
+    profile: Annotated[str, typer.Option("--profile", help="QC profile name, 'auto' or 'none'.")] = "auto",
+    threshold: Annotated[list[str] | None, typer.Option("--threshold", help="KEY=VALUE (repeatable).")] = None,
+    bg_offset: Annotated[float | None, typer.Option("--bg-offset", help="Background offset x (3000).")] = None,
+    bs_set: Annotated[str, typer.Option("--bs-conv-red-bg-set", help="U3-U5 (default) or U4-U6.")] = "U3-U5",
+    pval: Annotated[float, typer.Option("--pval-threshold", help="Detection p threshold.")] = 0.05,
+    beta_offset: Annotated[float, typer.Option("--beta-offset", help="alpha in M/(M+U+alpha).")] = 0.0,
+    dye_bias: Annotated[str, typer.Option("--dye-bias", help="linear|nonlinear")] = "linear",
+    min_prop: Annotated[
+        float | None, typer.Option("--min-prop-passing", help="Fail samples below this detection rate.")
+    ] = None,
+    pca_full: Annotated[int, typer.Option("--pca-max-samples-full", min=2)] = 100,
+    pca_probes: Annotated[int, typer.Option("--pca-probes", min=2)] = 10_000,
+    threads: Annotated[int, typer.Option("--threads", min=0, max=1024, help="Worker processes (0 = auto).")] = 0,
+    force: Annotated[bool, typer.Option("--force", help="Replace existing outputs.")] = False,
+    log_json: LogJson = False,
+    debug: Debug = False,
+    quiet: Quiet = False,
+) -> None:
+    """Methylation control QC: 21 control metrics, detection p-values, noob/dye-bias corrected β (MET-001)."""
+    from simurg.pipeline.methyl import MethylQcOptions, run_methyl_qc
+
+    if dye_bias not in ("linear", "nonlinear"):
+        raise typer.BadParameter("--dye-bias must be linear or nonlinear")
+    if bs_set not in ("U3-U5", "U4-U6"):
+        raise typer.BadParameter("--bs-conv-red-bg-set must be U3-U5 or U4-U6")
+    if not 0 < pval <= 1:
+        raise typer.BadParameter("--pval-threshold must be in (0, 1]")
+    rep = _reporter("methyl qc", out, log_json, debug, quiet)
+    opts = MethylQcOptions(
+        out=out, manifest=manifest, idats=idats, sheet=sheet, config=config,
+        profile=None if profile == "none" else profile, thresholds=threshold or [], bg_offset=bg_offset,
+        bs_conv_red_bg_set=bs_set, pval_threshold=pval, beta_offset=beta_offset, dye_bias=dye_bias,
+        min_prop_passing=min_prop, pca_max_samples_full=pca_full, pca_probes=pca_probes, threads=threads,
+        force=force, argv=sys.argv[1:],
+    )  # fmt: skip
+    run_guarded(rep, lambda: run_methyl_qc(opts, rep))
+
+
+@methyl_app.command("convert-manifest")
+def methyl_convert_cmd(
+    manifest: Annotated[Path, typer.Option("--manifest", help="Methylation CSV manifest (required).")],
+    out: Annotated[Path, typer.Option("--out", help="Output folder (required).")],
+    force: Annotated[bool, typer.Option("--force", help="Replace existing outputs.")] = False,
+    log_json: LogJson = False,
+    debug: Debug = False,
+    quiet: Quiet = False,
+) -> None:
+    """Methylation manifest -> probes.tsv and controls.tsv (MET-003)."""
+    from simurg.pipeline.methyl import run_convert_manifest
+
+    rep = _reporter("methyl convert-manifest", out, log_json, debug, quiet)
+    run_guarded(rep, lambda: run_convert_manifest(manifest, out, force, rep))
+
+
 def _qc_cohort(
     command: str,
     out: Path,

src/simurg/cn/annotate.py

@@ -0,0 +1,130 @@
+"""CN event annotation: cytobands, genes, ISCN 2020 strings — spec CYTO-033 … CYTO-035. Pure functions, no I/O.
+
+[DOC: ISCN 2020 (An International System for Human Cytogenomic Nomenclature), microarray nomenclature section;
+UCSC cytoBand tables] Microarray form ``arr[<build>] <chr><band_start>[<band_end>](<S>_<E>)x<CN>`` per event, LOH as
+``x2 hmz`` (gain-LOH ``x3 hmz``), mosaic as ``x<base>~<CN>``; normal results ``arr[<build>](1-22,X)x2`` (female),
+``(1-22)x2,(XY)x1`` (male), ``(1-22)x2`` (sex unresolved). Traditional form ``del(<chr>)(<bands>)`` /
+``dup(<chr>)(<bands>)`` for CNVs only.
+"""
+
+from __future__ import annotations
+
+import bisect
+from collections.abc import Sequence
+from dataclasses import dataclass
+from typing import Any
+
+CHROM_ORDER = {str(i): i for i in range(1, 23)} | {"X": 23, "Y": 24}
+
+
+@dataclass
+class Bands:
+    """1-based inclusive band intervals per chromosome, sorted."""
+
+    starts: dict[str, list[int]]
+    ends: dict[str, list[int]]
+    names: dict[str, list[str]]
+
+    @classmethod
+    def from_ucsc(cls, text: str) -> Bands:
+        rows: dict[str, list[tuple[int, int, str]]] = {}
+        for line in text.splitlines():
+            f = line.split("\t")
+            if len(f) < 4:
+                continue
+            c = f[0][3:] if f[0].lower().startswith("chr") else f[0]
+            if "_" in c:
+                continue
+            rows.setdefault(c, []).append((int(f[1]) + 1, int(f[2]), f[3]))
+        s, e, n = {}, {}, {}
+        for c, v in rows.items():
+            v.sort()
+            s[c], e[c], n[c] = [x[0] for x in v], [x[1] for x in v], [x[2] for x in v]
+        return cls(s, e, n)
+
+    def band(self, chrom: str, pos: int) -> str:
+        st = self.starts.get(chrom)
+        if not st:
+            return ""
+        i = max(0, bisect.bisect_right(st, pos) - 1)
+        return self.names[chrom][i]
+
+
+@dataclass
+class Genes:
+    """Transcripts: per chromosome sorted by start, with the gene label (HGNC symbol or Ensembl/RefSeq ID)."""
+
+    starts: dict[str, list[int]]
+    ends: dict[str, list[int]]
+    labels: dict[str, list[str]]
+    max_len: dict[str, int]
+
+    @classmethod
+    def from_rows(cls, rows: Sequence[tuple[str, int, int, str]]) -> Genes:
+        by: dict[str, list[tuple[int, int, str]]] = {}
+        for c, s, e, lab in rows:
+            by.setdefault(c, []).append((s, e, lab))
+        st, en, lb, ml = {}, {}, {}, {}
+        for c, v in by.items():
+            v.sort()
+            st[c], en[c], lb[c] = [x[0] for x in v], [x[1] for x in v], [x[2] for x in v]
+            ml[c] = max(x[1] - x[0] for x in v)
+        return cls(st, en, lb, ml)
+
+    def overlapping(self, chrom: str, start: int, end: int) -> list[str]:
+        """CYTO-035: every transcript overlapping by ≥ 1 bp; labels deduplicated in transcript-start order."""
+        st = self.starts.get(chrom)
+        if not st:
+            return []
+        lo = bisect.bisect_left(st, start - self.max_len[chrom])
+        hi = bisect.bisect_right(st, end)
+        out: list[str] = []
+        seen: set[str] = set()
+        for i in range(lo, hi):
+            if self.ends[chrom][i] >= start and self.labels[chrom][i] not in seen:
+                seen.add(self.labels[chrom][i])
+                out.append(self.labels[chrom][i])
+        return out
+
+
+def _band_span(bands: Bands, ev: dict[str, Any]) -> tuple[str, str]:
+    return bands.band(ev["chrom"], ev["start"]), bands.band(ev["chrom"], ev["end"])
+
+
+def _chrom_label(c: str) -> str:
+    return c
+
+
+def iscn_microarray_event(ev: dict[str, Any], bands: Bands) -> str:
+    b1, b2 = _band_span(bands, ev)
+    span = b1 if b1 == b2 else f"{b1}{b2}"
+    cn = int(ev["copy_number"])
+    if ev.get("mosaic"):
+        base = int(ev.get("base_cn", 2))
+        copies = f"x{base}~{cn if ev['call'] != 'MLOH' else base}"
+        if ev["call"] == "MLOH":
+            copies += " hmz"
+    elif ev["call_type"] == "LOH":
+        copies = f"x{cn} hmz"
+    else:
+        copies = f"x{cn}"
+    return f"{_chrom_label(ev['chrom'])}{span}({ev['start']}_{ev['end']}){copies}"
+
+
+def iscn_traditional_event(ev: dict[str, Any], bands: Bands) -> str | None:
+    if ev["call_type"] not in ("DEL", "DUP"):
+        return None
+    b1, b2 = _band_span(bands, ev)
+    kind = "del" if ev["call_type"] == "DEL" else "dup"
+    return f"{kind}({ev['chrom']})({b1}{b2})"
+
+
+def iscn_sample(events: list[dict[str, Any]], bands: Bands, build: str, sex: str) -> tuple[str, str]:
+    """(traditional, microarray) strings for the reported events (already filtered)."""
+    evs = sorted(events, key=lambda e: (CHROM_ORDER.get(e["chrom"], 99), e["start"]))
+    if not evs:
+        normal = {"F": "(1-22,X)x2", "M": "(1-22)x2,(XY)x1"}.get(sex, "(1-22)x2")
+        return "", f"arr[{build}]{normal}"
+    micro = f"arr[{build}] " + ",".join(iscn_microarray_event(e, bands) for e in evs)
+    trad = ",".join(t for t in (iscn_traditional_event(e, bands) for e in evs) if t)
+    return trad, micro

src/simurg/cn/data/mask_GRCh37.bed

@@ -0,0 +1,26 @@
+# centromeres, gvar/stalk bands, acrocentric p-arms; derived from UCSC hg19 cytoBand
+1	121500000	142600000
+2	90500000	96800000
+3	87900000	98300000
+4	48200000	52700000
+5	46100000	50700000
+6	58700000	63300000
+7	58000000	61700000
+8	43100000	48100000
+9	47300000	65900000
+10	38000000	42300000
+11	51600000	55700000
+12	33300000	38200000
+13	0	19500000
+14	0	19100000
+15	0	20700000
+16	34600000	47000000
+17	22200000	25800000
+18	15400000	19000000
+19	20000000	32400000
+20	25600000	29400000
+21	0	14300000
+22	0	17900000
+X	58100000	63000000
+Y	11600000	13400000
+Y	28800000	59373566

src/simurg/cn/data/mask_GRCh38.bed

@@ -0,0 +1,26 @@
+# centromeres, gvar/stalk bands, acrocentric p-arms; derived from UCSC hg38 cytoBand
+1	121700000	143200000
+2	91800000	96000000
+3	87800000	98600000
+4	48200000	51800000
+5	46100000	51400000
+6	58500000	62600000
+7	58100000	62100000
+8	43200000	47200000
+9	42200000	61500000
+10	38000000	41600000
+11	51000000	55800000
+12	33200000	37800000
+13	0	18900000
+14	0	18200000
+15	0	20500000
+16	35300000	47000000
+17	22700000	27400000
+18	15400000	21500000
+19	19900000	31900000
+20	25700000	30400000
+21	0	13000000
+22	0	17400000
+X	58100000	63800000
+Y	10300000	10600000
+Y	26600000	57227415

src/simurg/cn/events.py

@@ -0,0 +1,290 @@
+"""CN/LOH event extraction, statistics and filters — spec CYTO-006 … CYTO-021, CYTO-027. Pure functions, no I/O.
+
+From a per-chromosome Viterbi path: runs of non-baseline states become events; statistics per CYTO-027; effective
+size per CYTO-010 (gaps > 150 × the chromosome's median spacing replaced by the median); QUAL from the mean posterior
+of the called state class (CYTO-012); mosaic fraction from the BAF split or, failing that, LRR (CYTO-018/019/042);
+post-filters in the order of CYTO-015; per-type thresholds (CYTO-007/008) and promotion/low fraction (CYTO-020/021).
+"""
+
+from __future__ import annotations
+
+import math
+from dataclasses import dataclass, field
+
+import numpy as np
+import numpy.typing as npt
+
+from simurg.cn.hmm import State
+
+F64 = npt.NDArray[np.float64]
+I64 = npt.NDArray[np.int64]
+GAP_FACTOR = 150
+MIN_MF_PROBES = 20
+LOSS_EXPECTED_HETS = 3.0  # CYTO-041: expected heterozygous calls needed to call a loss complete
+LOSS_HET_RATIO = 0.25  # CYTO-041: observed / expected heterozygous calls below which a loss is complete
+
+
+def complete_loss(geno: npt.NDArray[np.uint8], pfb: F64) -> bool:
+    """CYTO-041 [OURS]: a loss is complete when its called genotypes lack the heterozygotes the PFB predicts
+    (Σ 2p(1 − p) ≥ 3 expected, fewer than a quarter observed); a mosaic loss keeps its heterozygotes."""
+    called = geno > 0
+    p = np.clip(np.where(np.isfinite(pfb), pfb, 0.5), 0.0, 1.0)[called]
+    expected = float((2 * p * (1 - p)).sum())
+    return expected >= LOSS_EXPECTED_HETS and float((geno[called] == 2).sum()) < LOSS_HET_RATIO * expected
+
+
+TYPES = ("del", "dup", "loh", "gainloh", "mdel", "mdup", "mloh")
+DEFAULTS = {"del": (10, 25_000), "dup": (10, 50_000), "loh": (500, 3_000_000), "gainloh": (300, 500_000),
+            "mdel": (10, 25_000), "mdup": (10, 50_000), "mloh": (500, 3_000_000)}  # fmt: skip
+CALL_KEY = {"DEL": "del", "DUP": "dup", "LOH": "loh", "GAINLOH": "gainloh", "MDEL": "mdel", "MDUP": "mdup",
+            "MLOH": "mloh"}  # fmt: skip
+
+
+@dataclass
+class Event:
+    chrom: str
+    start: int  # first affected base S (1-based) = first probe position
+    end: int  # last affected base E = last probe position
+    i0: int  # probe index range in the chromosome arrays [i0, i1)
+    i1: int
+    call: str  # DEL | DUP | LOH | GAINLOH | MDEL | MDUP | MLOH
+    cn: int
+    state: str
+    quality: float = 0.0
+    effective_size: int = 0
+    nr: float = math.nan
+    lrd: float = math.nan
+    baf_dev: float = math.nan
+    percent_het: float = math.nan
+    lrr_median: float = math.nan
+    mosaic_fraction: float | None = None
+    high_fraction: bool = False
+    filters: list[str] = field(default_factory=list)
+    flags: list[str] = field(default_factory=list)
+
+    @property
+    def mosaic(self) -> bool:
+        return self.call.startswith("M")
+
+    @property
+    def probes(self) -> int:
+        return self.i1 - self.i0
+
+    @property
+    def size(self) -> int:
+        return self.end - self.start + 1
+
+    @property
+    def svtype(self) -> str:
+        return {"MDEL": "DEL", "MDUP": "DUP", "MLOH": "LOH", "GAINLOH": "LOH"}.get(self.call, self.call)
+
+    @property
+    def loh_type(self) -> str | None:
+        if self.call == "LOH":
+            return "AOH"
+        if self.call == "GAINLOH":
+            return "AOH"
+        if self.call == "MLOH":
+            return "CNLOH"
+        return None
+
+    @property
+    def id_prefix(self) -> str:
+        return {"DEL": "DEL", "DUP": "DUP", "LOH": "AOH", "GAINLOH": "AOH", "MDEL": "MosaicDEL", "MDUP": "MosaicDUP",
+                "MLOH": "CNLOH"}[self.call]  # fmt: skip
+
+
+def median_spacing(pos: I64) -> float:
+    d = np.diff(np.sort(pos))
+    d = d[d > 0]
+    return float(np.median(d)) if len(d) else 1.0
+
+
+def effective_size(pos: I64, m: float) -> int:
+    """CYTO-010 for the probes of one event (sorted positions)."""
+    if len(pos) < 2:
+        return 1
+    s = np.diff(pos).astype(np.float64)
+    gap = s > GAP_FACTOR * m
+    return int((pos[-1] - pos[0] + 1) - float((s[gap] - m).sum()))
+
+
+def extract(chrom: str, path: npt.NDArray[np.int32], states: list[State], pos: I64) -> list[Event]:
+    """Runs of non-baseline states, merged across runs of the same call type (mosaic fraction steps)."""
+    out: list[Event] = []
+    if len(path) == 0:
+        return out
+    br = np.flatnonzero(np.diff(path) != 0) + 1
+    starts, ends = np.r_[0, br], np.r_[br, len(path)]
+    for s, e in zip(starts.tolist(), ends.tolist(), strict=True):
+        st = states[int(path[s])]
+        if not st.call:
+            continue
+        if out and out[-1].i1 == s and out[-1].call == st.call and out[-1].cn == st.cn:
+            out[-1].i1 = e
+            out[-1].end = int(pos[e - 1])
+            continue
+        out.append(Event(chrom, int(pos[s]), int(pos[e - 1]), s, e, st.call, st.cn, st.name))
+    return out
+
+
+def _phi(x: float) -> float:
+    return 0.5 * (1.0 + math.erf(x / math.sqrt(2.0)))
+
+
+def _split_from_median(m: float) -> float:
+    """δ ≥ 0 with median |N(±δ, 1)| = m (bisection; 0 when m ≤ 0.6745)."""
+    if _phi(m) - _phi(-m) <= 0.5:
+        return 0.0
+    lo, hi = 0.0, m + 10.0
+    for _ in range(60):
+        mid = 0.5 * (lo + hi)
+        if _phi(m - mid) - _phi(-m - mid) > 0.5:
+            lo = mid
+        else:
+            hi = mid
+    return 0.5 * (lo + hi)
+
+
+def _mf_from_baf(call: str, baf: F64, sigma: F64 | float = 0.0) -> float | None:
+    """CYTO-018 with the CYTO-041 noise correction [OURS]. ``baf``: heterozygous-call BAF of the event; ``sigma``:
+    each probe's expected BAF noise (reference heterozygous spread × the sample's scale). With m = median(|BAF − 0.5|
+    / σ), the split δ (in σ units) solves Φ(m − δ) − Φ(−m − δ) = ½ (the median of |N(±δ, 1)|; δ = 0 when
+    m ≤ 0.6745, i.e. no more spread than noise), and d = median(σ)·δ."""
+    sig = np.broadcast_to(np.asarray(sigma, dtype=np.float64), baf.shape)
+    ok = np.isfinite(baf) & (baf > 0.1) & (baf < 0.9) & np.isfinite(sig)
+    b, sg = baf[ok], sig[ok]
+    if len(b) < MIN_MF_PROBES:
+        return None
+    if float(np.median(sg)) > 0:
+        sg = np.maximum(sg, 1e-3)
+        d = float(np.median(sg)) * _split_from_median(float(np.median(np.abs(b - 0.5) / sg)))
+    else:
+        d = float(np.median(np.abs(b - 0.5)))
+    if call == "MDEL":
+        f = 2 * d / (0.5 + d)
+    elif call == "MDUP":
+        f = 2 * d / (0.5 - d) if d < 0.5 else 1.0
+    else:
+        f = 2 * d
+    return float(min(1.0, max(0.0, f)))
+
+
+def _mf_from_lrr(call: str, lrr: F64, gamma: float) -> float | None:
+    x = lrr[np.isfinite(lrr)]
+    if len(x) < MIN_MF_PROBES or gamma <= 0:
+        return None
+    f = abs(2 ** (float(np.median(x)) / gamma) - 1)
+    return float(min(1.0, max(0.0, f)))
+
+
+def annotate_stats(
+    ev: Event, pos: I64, lrr: F64, baf: F64, geno: npt.NDArray[np.uint8], post: F64, states: list[State], m: float,
+    gamma: float, haploid: bool, sigma: F64 | None = None, pfb: F64 | None = None,
+) -> None:  # fmt: skip
+    """CYTO-010/012/018/027 statistics in place (``lrr`` unsmoothed)."""
+    sl = slice(ev.i0, ev.i1)
+    x = lrr[sl]
+    fx = x[np.isfinite(x)]
+    ev.nr = float(np.median(2.0**fx)) if len(fx) else math.nan
+    ev.lrd = float(np.std(fx)) if len(fx) > 1 else math.nan
+    ev.lrr_median = float(np.median(fx)) if len(fx) else math.nan
+    g = geno[sl]
+    called = g > 0
+    ev.percent_het = float(100.0 * (g == 2).sum() / called.sum()) if called.any() else math.nan
+    b = baf[sl][(g == 2) & np.isfinite(baf[sl])]
+    ev.baf_dev = float(np.std(b)) if len(b) > 1 else math.nan
+    ev.effective_size = effective_size(pos[sl], m)
+    cls = [j for j, st in enumerate(states) if st.call == ev.call]
+    p = float(post[sl][:, cls].sum(axis=1).mean()) if cls else 0.0
+    ev.quality = 60.0 if p >= 1.0 else float(min(60, max(0, round(-10 * math.log10(max(1e-300, 1 - p))))))
+    if ev.mosaic:
+        if not haploid and ev.call == "MDEL" and pfb is not None and complete_loss(g, pfb[sl]):
+            ev.mosaic_fraction = 1.0
+            return
+        hb = np.where(g == 2, baf[sl], np.nan) if sigma is not None else baf[sl]
+        mf = None if haploid else _mf_from_baf(ev.call, hb, 0.0 if sigma is None else sigma[sl])
+        ev.mosaic_fraction = mf if mf is not None else _mf_from_lrr(ev.call, x, gamma)
+
+
+@dataclass
+class Thresholds:
+    probes: dict[str, int]
+    size_bp: dict[str, int]
+    max_mosaic_fraction: float = 1.0
+    min_mosaic_fraction: float = 0.10
+    filter_qual: int = 10
+    lrr_direction_min_frac: float = 0.6
+
+    @classmethod
+    def defaults(cls) -> Thresholds:
+        return cls({k: v[0] for k, v in DEFAULTS.items()}, {k: v[1] for k, v in DEFAULTS.items()})
+
+    def as_header(self, extra: dict[str, object]) -> str:
+        parts = []
+        for k in ("del", "dup", "loh", "gainloh", "mdel", "mdup", "mloh"):
+            parts += [f"min_{k}_probes={self.probes[k]}", f"min_{k}_size_bp={self.size_bp[k]}"]
+        parts += [f"{k}={v}" for k, v in extra.items()]
+        return ";".join(parts)
+
+
+def post_filter(events: list[Event], lrr_by_event: list[F64], th: Thresholds, counts: dict[str, int]) -> list[Event]:
+    """CYTO-015 steps 1–3 and CYTO-020 promotion; per-type thresholds are applied by :func:`apply_thresholds`."""
+    kept: list[Event] = []
+    for ev, x in zip(events, lrr_by_event, strict=True):
+        fx = x[np.isfinite(x)]
+        if ev.call in ("DEL", "MDEL") and len(fx) and (fx < 0).mean() < th.lrr_direction_min_frac:
+            counts["lrr_direction"] = counts.get("lrr_direction", 0) + 1
+            continue
+        if ev.call in ("DUP", "MDUP", "GAINLOH") and len(fx) and (fx > 0).mean() < th.lrr_direction_min_frac:
+            counts["lrr_direction"] = counts.get("lrr_direction", 0) + 1
+            continue
+        kept.append(ev)
+    # short LOH between copy-neutral neighbours (no other event adjacent on either side)
+    out: list[Event] = []
+    for i, ev in enumerate(kept):
+        if ev.call in ("LOH", "MLOH") and ev.size < 500_000:
+            left = kept[i - 1] if i > 0 and kept[i - 1].chrom == ev.chrom else None
+            right = kept[i + 1] if i + 1 < len(kept) and kept[i + 1].chrom == ev.chrom else None
+            if not (left and left.i1 == ev.i0) and not (right and right.i0 == ev.i1):
+                counts["short_loh"] = counts.get("short_loh", 0) + 1
+                continue
+        out.append(ev)
+    merged: list[Event] = []
+    for ev in out:
+        prev = merged[-1] if merged else None
+        if (prev and prev.chrom == ev.chrom and prev.call == ev.call and prev.cn == ev.cn
+                and ev.i0 - prev.i1 <= 3 and (ev.start - prev.end) <= 0.1 * (ev.end - prev.start + 1)):  # fmt: skip
+            prev.i1, prev.end = ev.i1, ev.end
+            counts["merged"] = counts.get("merged", 0) + 1
+            continue
+        merged.append(ev)
+    for ev in merged:  # CYTO-020 promotion
+        if ev.mosaic and ev.mosaic_fraction is not None and ev.mosaic_fraction >= th.max_mosaic_fraction:
+            ev.call = {"MDEL": "DEL", "MDUP": "DUP", "MLOH": "LOH"}[ev.call]
+            ev.high_fraction = True
+    return merged
+
+
+def apply_thresholds(events: list[Event], th: Thresholds, counts: dict[str, int]) -> list[Event]:
+    out = []
+    for ev in events:
+        k = CALL_KEY[ev.call]
+        if ev.probes < th.probes[k] or ev.effective_size < th.size_bp[k]:
+            counts["below_threshold"] = counts.get("below_threshold", 0) + 1
+            continue
+        out.append(ev)
+    return out
+
+
+def set_filters(events: list[Event], th: Thresholds, sample_noisy: bool) -> None:
+    """CYTO-013 FILTER values."""
+    for ev in events:
+        f = []
+        if ev.quality < th.filter_qual:
+            f.append(f"Q{th.filter_qual}")
+        if sample_noisy:
+            f.append("SampleQuality")
+        if ev.mosaic and ev.mosaic_fraction is not None and ev.mosaic_fraction < th.min_mosaic_fraction:
+            f.append("LowFraction")
+        ev.filters = f or ["PASS"]

src/simurg/cn/gcwave.py

@@ -0,0 +1,38 @@
+"""GC-wave correction of LRR — spec CYTO-003, plan §10.1. Pure functions, no I/O.
+
+[LIT: Diskin et al. 2008 NAR 36:e126] LRR is regressed on the probe-centred GC fractions (1 kb, 100 kb, 1 Mb) by
+ordinary least squares over autosomal probes with |LRR| < 1 (CNV and outlier probes do not steer the fit); the GC
+component (fitted value minus its mean) is subtracted. [OURS] GCWF is reported as the standard deviation of that
+removed component, i.e. the size of the wave in LRR units.
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+
+import numpy as np
+import numpy.typing as npt
+
+F64 = npt.NDArray[np.float64]
+MIN_PROBES = 1000
+
+
+@dataclass
+class GcFit:
+    applied: bool
+    coef: F64  # intercept + one slope per window
+    gcwf: float
+
+
+def gc_correct(lrr: F64, gc: npt.NDArray[np.float32], use: npt.NDArray[np.bool_]) -> tuple[F64, GcFit]:
+    """Return corrected LRR (NaN stays NaN) and the fit; unchanged LRR when too few probes qualify."""
+    x = np.asarray(gc, dtype=np.float64)
+    ok = use & np.isfinite(lrr) & (np.abs(lrr) < 1.0) & np.isfinite(x).all(axis=1)
+    if ok.sum() < MIN_PROBES:
+        return lrr.copy(), GcFit(False, np.zeros(x.shape[1] + 1), 0.0)
+    xm = x[ok].mean(axis=0)
+    design = np.column_stack([np.ones(int(ok.sum())), x[ok] - xm])
+    coef, *_ = np.linalg.lstsq(design, lrr[ok], rcond=None)
+    fitted = np.where(np.isfinite(x).all(axis=1), (np.nan_to_num(x) - xm) @ coef[1:], 0.0)
+    out = lrr - fitted
+    return out, GcFit(True, coef, float(np.std(fitted[ok])))

src/simurg/cn/hmm.py

@@ -0,0 +1,196 @@
+"""Genome-wide CN/LOH/mosaic hidden Markov model — spec CYTO-004/005/012/018, plan §10.2. Pure functions, no I/O.
+
+[LIT: Wang et al. 2007 Genome Res 17:1665 (PennCNV: LRR/BAF emissions weighted by population B-allele frequency,
+distance-dependent transitions); Loh et al. 2018 Nature 559:350 (MoChA: mosaic BAF split); DOC: cnvPartition
+technical note (reference LRR state means CN0 −5, CN1 −0.45, CN2 0, CN3 0.3, CN4 0.75)] [OURS: state grid, mixture
+weights, switch model]
+
+* Diploid regions: CN0, CN1, CN2, CN2-LOH, CN3, CN3-LOH, CN4 and mosaic loss / gain / CN-LOH at cell fractions
+  0.2, 0.4, 0.6, 0.8. Haploid regions (male X outside PAR, male Y): CN0, CN1, CN2 and mosaic gain/loss relative to
+  the region's median LRR.
+* LRR ~ (1−ε)·N(µ_state, σ) + ε·U(−5, 3); σ = 1.4826·MAD of the sample's autosomal LRR (floor 0.08), scaled per
+  probe by the reference's LRR spread (CYTO-042); CN0 σ = 2.
+  Mosaic means interpolate linearly between the constitutional means by the cell fraction.
+* BAF ~ (1−ε)·Σ_g w_g N(b_g, σ_b) + ε·U(0, 1); genotype weights from the probe's PFB p (binomial in the copy
+  number; LOH: {0: 1−p, 1: p}); mosaic heterozygotes split into the two bands 0.5 ± d(f).
+* Transitions: switch probability 1 − exp(−d / D) for inter-probe distance d, D = 5·10⁸ bp; a switch lands on state
+  j with probability π_j (the same target distribution from every state).
+* Viterbi path for segmentation; forward–backward posteriors for QUAL.
+"""
+
+from __future__ import annotations
+
+import math
+from dataclasses import dataclass, field
+
+import numpy as np
+import numpy.typing as npt
+
+F64 = npt.NDArray[np.float64]
+FRACTIONS = (0.2, 0.4, 0.6, 0.8)
+LRR_MEAN = {0: -5.0, 1: -0.45, 2: 0.0, 3: 0.3, 4: 0.75}
+EPS = 0.01
+MOSAIC_PRIOR = 0.004  # total switch-target mass of the 12 diploid mosaic states
+SWITCH_D = 5e8
+
+
+@dataclass(frozen=True)
+class State:
+    name: str
+    cn: int  # copy number (baseline-relative for haploid regions: 0, 1, 2)
+    loh: bool
+    mosaic: str  # "" | "loss" | "gain" | "cnloh"
+    fraction: float  # cell fraction of mosaic states, 1.0 otherwise
+    call: str  # "" (baseline) | DEL | DUP | LOH | GAINLOH | MDEL | MDUP | MLOH
+    lrr_mu: float
+    baf: tuple[tuple[str, float], ...]  # (genotype-weight key, BAF mean)
+    prior: float
+
+
+def _d_split(kind: str, f: float) -> float:
+    if kind == "loss":
+        return f / (2 * (2 - f))
+    if kind == "gain":
+        return f / (2 * (2 + f))
+    return f / 2
+
+
+def diploid_states(mosaic_prior: float = MOSAIC_PRIOR) -> list[State]:
+    s = [
+        State("CN0", 0, False, "", 1.0, "DEL", LRR_MEAN[0], (), 0.01),
+        State("CN1", 1, False, "", 1.0, "DEL", LRR_MEAN[1], (("h0", 0.0), ("h1", 1.0)), 0.06),
+        State("CN2", 2, False, "", 1.0, "", 0.0, (("aa", 0.0), ("ab", 0.5), ("bb", 1.0)), 0.60),
+        State("CN2LOH", 2, True, "", 1.0, "LOH", 0.0, (("h0", 0.0), ("h1", 1.0)), 0.06),
+        State("CN3", 3, False, "", 1.0, "DUP", LRR_MEAN[3],
+              (("t0", 0.0), ("t1", 1 / 3), ("t2", 2 / 3), ("t3", 1.0)), 0.06),
+        State("CN3LOH", 3, True, "", 1.0, "GAINLOH", LRR_MEAN[3], (("h0", 0.0), ("h1", 1.0)), 0.02),
+        State("CN4", 4, False, "", 1.0, "DUP", LRR_MEAN[4],
+              (("q0", 0.0), ("q1", 0.25), ("q2", 0.5), ("q3", 0.75), ("q4", 1.0)), 0.03),
+    ]  # fmt: skip
+    for kind, call, mu_end in (("loss", "MDEL", LRR_MEAN[1]), ("gain", "MDUP", LRR_MEAN[3]), ("cnloh", "MLOH", 0.0)):
+        for f in FRACTIONS:
+            d = _d_split(kind, f)
+            cn = {"loss": 1, "gain": 3, "cnloh": 2}[kind]
+            bands = (("aa", 0.0), ("ab-", 0.5 - d), ("ab+", 0.5 + d), ("bb", 1.0))
+            s.append(State(f"M{kind}{f:.1f}", cn, kind == "cnloh", kind, f, call, mu_end * f, bands, mosaic_prior / 12))
+    return s
+
+
+def haploid_states(mosaic_prior: float = MOSAIC_PRIOR) -> list[State]:
+    s = [
+        State("H0", 0, False, "", 1.0, "DEL", LRR_MEAN[0], (), 0.02),
+        State("H1", 1, False, "", 1.0, "", 0.0, (("h0", 0.0), ("h1", 1.0)), 0.80),
+        State("H2", 2, False, "", 1.0, "DUP", 0.45, (("h0", 0.0), ("h1", 1.0)), 0.06),
+    ]
+    for f in FRACTIONS:
+        s.append(State(f"Hgain{f:.1f}", 2, False, "gain", f, "MDUP", 0.45 * f, (("h0", 0.0), ("h1", 1.0)),
+                       mosaic_prior / 8))  # fmt: skip
+        s.append(State(f"Hloss{f:.1f}", 0, False, "loss", f, "MDEL", -1.0 * f, (("h0", 0.0), ("h1", 1.0)),
+                       mosaic_prior / 8))  # fmt: skip
+    return s
+
+
+def _geno_weights(key: str, p: F64) -> F64:
+    q = 1 - p
+    table = {
+        "aa": q * q, "ab": 2 * p * q, "bb": p * p, "ab-": p * q, "ab+": p * q,
+        "h0": q, "h1": p,
+        "t0": q**3, "t1": 3 * p * q * q, "t2": 3 * p * p * q, "t3": p**3,
+        "q0": q**4, "q1": 4 * p * q**3, "q2": 6 * p * p * q * q, "q3": 4 * p**3 * q, "q4": p**4,
+    }  # fmt: skip
+    return table[key]
+
+
+def _norm(x: F64, mu: float, sd: float | F64) -> F64:
+    return np.exp(-0.5 * ((x - mu) / sd) ** 2) / (sd * math.sqrt(2 * math.pi))
+
+
+@dataclass
+class Params:
+    lrr_sd: float | F64  # scalar or per probe (CYTO-042)
+    baf_sd: float
+    lrr_offset: float = 0.0  # added to every state mean (haploid regions: the region's median LRR)
+    switch_d: float = SWITCH_D
+    eps: float = EPS
+    extra: dict[str, float] = field(default_factory=dict)
+
+
+def emissions(states: list[State], lrr: F64, baf: F64, pfb: F64, par: Params, het_sd: F64 | None = None) -> F64:
+    """Per-probe, per-state emission likelihood (n × S); missing LRR/BAF contribute 1. ``het_sd``: per-probe spread
+    of heterozygous BAF in the reference; heterozygous components use max(σ_b, het_sd)."""
+    n = len(lrr)
+    out = np.ones((n, len(states)), dtype=np.float64)
+    lok = np.isfinite(lrr)
+    bok = np.isfinite(baf)
+    x = np.where(lok, lrr, 0.0)
+    b = np.clip(np.where(bok, baf, 0.5), 0.0, 1.0)
+    p = np.clip(np.where(np.isfinite(pfb), pfb, 0.5), 0.01, 0.99)
+    sd_het: float | F64 = (
+        np.maximum(par.baf_sd, np.nan_to_num(het_sd, nan=par.baf_sd)) if het_sd is not None else par.baf_sd
+    )
+    for j, st in enumerate(states):
+        sd = 2.0 if st.cn == 0 and not st.mosaic else par.lrr_sd
+        le = (1 - par.eps) * _norm(x, st.lrr_mu + par.lrr_offset, sd) + par.eps / 8.0
+        if st.baf:
+            be = np.zeros(n)
+            for key, mean in st.baf:
+                bsd = par.baf_sd if mean in (0.0, 1.0) or het_sd is None else sd_het
+                be += _geno_weights(key, p) * _norm(b, mean, bsd)
+            be = (1 - par.eps) * be + par.eps
+        else:
+            be = np.ones(n)
+        out[:, j] = np.where(lok, le, 1.0) * np.where(bok, be, 1.0)
+    return out
+
+
+def switch_probs(pos: npt.NDArray[np.int64], switch_d: float = SWITCH_D) -> F64:
+    """Switch probability before each probe (the first probe gets 1: it draws from the prior)."""
+    d = np.diff(np.asarray(pos, dtype=np.float64), prepend=np.nan)
+    p = 1.0 - np.exp(-np.maximum(d, 0.0) / switch_d)
+    p[0] = 1.0
+    return np.clip(np.nan_to_num(p, nan=1.0), 1e-12, 1.0)
+
+
+def forward_backward(e: F64, sw: F64, prior: F64) -> F64:
+    """Posterior state probabilities (n × S) under the switch model."""
+    n, s = e.shape
+    alpha = np.empty((n, s))
+    c = np.empty(n)
+    a = prior * e[0]
+    c[0] = a.sum() or 1e-300
+    alpha[0] = a / c[0]
+    for t in range(1, n):
+        a = ((1 - sw[t]) * alpha[t - 1] + sw[t] * prior) * e[t]
+        c[t] = a.sum() or 1e-300
+        alpha[t] = a / c[t]
+    beta = np.ones(s)
+    post = np.empty((n, s))
+    post[n - 1] = alpha[n - 1]
+    for t in range(n - 1, 0, -1):
+        eb = e[t] * beta
+        beta = ((1 - sw[t]) * eb + sw[t] * float(prior @ eb)) / c[t]
+        g = alpha[t - 1] * beta
+        post[t - 1] = g / (g.sum() or 1e-300)
+    return post
+
+
+def viterbi(e: F64, sw: F64, prior: F64) -> npt.NDArray[np.int32]:
+    """Most likely state path under the switch model (log space)."""
+    n, s = e.shape
+    le = np.log(np.maximum(e, 1e-300))
+    lp = np.log(np.maximum(prior, 1e-300))
+    back = np.empty((n, s), dtype=np.int32)
+    ld = lp + le[0]
+    idx = np.arange(s, dtype=np.int32)
+    for t in range(1, n):
+        stay = np.log(np.maximum(1 - sw[t] + sw[t] * prior, 1e-300)) + ld
+        best = int(np.argmax(ld))
+        jump = math.log(max(sw[t], 1e-300)) + lp + ld[best]
+        take_jump = jump > stay
+        back[t] = np.where(take_jump, best, idx)
+        ld = np.where(take_jump, jump, stay) + le[t]
+    path = np.empty(n, dtype=np.int32)
+    path[-1] = int(np.argmax(ld))
+    for t in range(n - 1, 0, -1):
+        path[t - 1] = back[t, path[t]]
+    return path

src/simurg/cn/model.py

@@ -0,0 +1,107 @@
+"""Cyto model covariates — spec CYTO-038, IN-051 (plan §10.1). Pure functions, no I/O.
+
+GC fraction of windows centred on each probe [LIT: Diskin et al. 2008 NAR 36:e126 (GC-wave), windows 1 kb / 100 kb /
+1 Mb]: (G+C) / (A+C+G+T) over the window, N bases excluded; NaN when the window has no called base. Population
+B-allele frequency per probe (PFB) [LIT: Wang et al. 2007 Genome Res 17:1665 (PennCNV)] from cluster-file counts:
+(n_AB + 2·n_BB) / (2·N); 0.5 when unknown.
+"""
+
+from __future__ import annotations
+
+import numpy as np
+import numpy.typing as npt
+
+GC_WINDOWS = (1_000, 100_000, 1_000_000)
+_C, _G, _A, _T = ord("C"), ord("G"), ord("A"), ord("T")
+
+
+def gc_fractions(
+    seq: npt.NDArray[np.uint8], positions: npt.NDArray[np.int64], windows: tuple[int, ...] = GC_WINDOWS
+) -> npt.NDArray[np.float32]:
+    """GC fraction per (probe, window) for 1-based ``positions`` on one contig (upper-case ASCII ``seq``)."""
+    n = len(seq)
+    gc = np.zeros(n + 1, dtype=np.int32)
+    called = np.zeros(n + 1, dtype=np.int32)
+    step = 1 << 24
+    run_gc = run_called = 0
+    for s in range(0, n, step):  # chunked cumulative sums keep temporaries small
+        chunk = np.asarray(seq[s : s + step])
+        g = (chunk == _C) | (chunk == _G)
+        c = g | (chunk == _A) | (chunk == _T)
+        cg = np.cumsum(g, dtype=np.int32) + run_gc
+        cc = np.cumsum(c, dtype=np.int32) + run_called
+        gc[s + 1 : s + 1 + len(chunk)] = cg
+        called[s + 1 : s + 1 + len(chunk)] = cc
+        run_gc, run_called = int(cg[-1]), int(cc[-1])
+    out = np.full((len(positions), len(windows)), np.nan, dtype=np.float32)
+    centre = np.asarray(positions, dtype=np.int64) - 1  # 0-based
+    for j, w in enumerate(windows):
+        lo = np.clip(centre - w // 2, 0, n)
+        hi = np.clip(centre - w // 2 + w, 0, n)
+        num = gc[hi] - gc[lo]
+        den = called[hi] - called[lo]
+        ok = den > 0
+        out[ok, j] = num[ok] / den[ok]
+    return out
+
+
+def population_bfreq(counts: npt.NDArray[np.int32] | None, n_loci: int) -> npt.NDArray[np.float32]:
+    """PFB from cluster counts (n, 3) = (AA, AB, BB); 0.5 where there are no counts."""
+    if counts is None:
+        return np.full(n_loci, 0.5, dtype=np.float32)
+    c = np.asarray(counts, dtype=np.float64)
+    tot = c.sum(axis=1)
+    with np.errstate(invalid="ignore", divide="ignore"):
+        p = (c[:, 1] + 2 * c[:, 2]) / (2 * tot)
+    return np.where(tot > 0, p, 0.5).astype(np.float32)
+
+
+def reference_stats(
+    lrr: npt.NDArray[np.float32], baf: npt.NDArray[np.float32]
+) -> tuple[npt.NDArray[np.float32], npt.NDArray[np.float32], npt.NDArray[np.int32]]:
+    """Batch reference (plan §10.1) from a samples × probes stack of normal samples: per-probe median LRR, per-probe
+    median heterozygous BAF (samples with 0.2 < BAF < 0.8; 0.5 when none) and the number of such samples."""
+    with np.errstate(all="ignore"):
+        med_lrr = np.nanmedian(np.where(np.isfinite(lrr), lrr, np.nan), axis=0)
+        het = np.where((baf > 0.2) & (baf < 0.8), baf, np.nan)
+        n_het = np.isfinite(het).sum(axis=0).astype(np.int32)
+        het_c = np.where(n_het > 0, np.nanmedian(het, axis=0), 0.5)
+    return (
+        np.nan_to_num(med_lrr, nan=0.0).astype(np.float32),
+        np.nan_to_num(het_c, nan=0.5).astype(np.float32),
+        n_het,
+    )
+
+
+def reference_het_spread(baf: npt.NDArray[np.float32]) -> npt.NDArray[np.float32]:
+    """Per-probe robust SD (1.4826·MAD) of heterozygous BAF (0.2 < BAF < 0.8) across reference samples; NaN when
+    fewer than 5 heterozygous samples."""
+    het = np.where((baf > 0.2) & (baf < 0.8), baf, np.nan)
+    with np.errstate(all="ignore"):
+        import warnings
+
+        with warnings.catch_warnings():
+            warnings.simplefilter("ignore", RuntimeWarning)
+            med = np.nanmedian(het, axis=0)
+            mad = np.nanmedian(np.abs(het - med), axis=0)
+    n = np.isfinite(het).sum(axis=0)
+    return np.where(n >= 5, 1.4826 * mad, np.nan).astype(np.float32)
+
+
+def reference_lrr_spread(lrr: npt.NDArray[np.float32], ref_lrr: npt.NDArray[np.float32]) -> npt.NDArray[np.float32]:
+    """Per-probe robust SD (1.4826·MAD) of LRR across reference samples after removing each sample's median and the
+    probe's reference median [OURS; per-probe noise as in PennCNV's per-marker SD, LIT: Wang et al. 2007]; NaN with
+    fewer than 5 finite values."""
+    import warnings
+
+    out = np.full(lrr.shape[1], np.nan, dtype=np.float32)
+    with warnings.catch_warnings():
+        warnings.simplefilter("ignore", RuntimeWarning)
+        smed = np.nanmedian(np.where(np.isfinite(lrr), lrr, np.nan), axis=1, keepdims=True).astype(np.float32)
+        for j in range(0, lrr.shape[1], 20_000):  # column chunks keep the temporaries small
+            x = np.where(np.isfinite(lrr[:, j : j + 20_000]), lrr[:, j : j + 20_000], np.nan) - smed
+            x -= ref_lrr[None, j : j + 20_000]
+            mad = np.nanmedian(np.abs(x - np.nanmedian(x, axis=0)), axis=0)
+            n = np.isfinite(x).sum(axis=0)
+            out[j : j + 20_000] = np.where(n >= 5, 1.4826 * mad, np.nan)
+    return out

src/simurg/core/errors.py

@@ -23,6 +23,7 @@ class ExitCode(IntEnum):
     PARTIAL = 3  # >= 1 sample failed and >= 1 succeeded
     ALL_FAILED = 4  # run completed but no sample succeeded
     SIGINT = 130
+    SIGPIPE = 141  # the reader of stdout went away (e.g. `simurg inspect … | head`); nothing is reported
     SIGTERM = 143
 
 
@@ -115,7 +116,6 @@ CATALOGUE: dict[str, MessageDef] = dict(
         _m("W_CTRL_MISSING", "warning", "Controls needed by a metric are absent; metric not evaluable", "QC-007"),
         _m("W_CTRL_ZERO_DIVISOR", "warning", "Control metric has a zero denominator (NaN)", "QC-008"),
         _m("W_CTRL_DEFS_DIFFER", "warning", "BPM and CSV control definitions differ; the BPM wins", "QC-001"),
-        _m("E_QC_METHYLATION", "error", "Methylation control metrics are not available yet (v2)", "QC-006"),
         _m("E_QC_NO_DATASETS", "error", "No QC dataset folder found", "QC-041"),
         _m("E_QC_DATASET_INCOMPLETE", "error", "Dataset folder lacks a required file", "QC-041"),
         _m("E_QC_LABEL_COUNT", "error", "Number of labels does not match number of datasets", "QC-042"),
@@ -135,6 +135,81 @@ CATALOGUE: dict[str, MessageDef] = dict(
         _m("W_QC_PEDIGREE", "warning", "Parent1/Parent2/Replicate value does not name a sample in the set", "IN-018"),
         _m("W_QC_NO_BAF", "warning", "GTC has no B-allele frequencies; contamination not estimated", "QC-064"),
         _m("W_QC_TOO_FEW", "warning", "Too few samples or sites for a cohort QC statistic", "QC-026"),
+        # --- cytogenetics CNV/LOH (CYTO-001 .. CYTO-040) -------------------------------------------------
+        _m("E_MODEL_MANIFEST_MISMATCH", "error", "Model was built for a different manifest probe set", "IN-051"),
+        _m("E_MODEL_VERSION", "error", "Model schema is too old; rebuild it", "IN-051"),
+        _m("W_CN_PROBE_FLOOR", "warning", "Minimum probe count below 10 raised to 10", "CYTO-008"),
+        _m(
+            "W_CN_THRESHOLD_EXCEEDS_GENOME",
+            "warning",
+            "Size threshold exceeds the effective genome; no events",
+            "CYTO-009",
+        ),
+        _m("I_CN_SEX_RESOLVED", "info", "Sample sex updated from U by the caller", "CYTO-024"),
+        _m("E_CN_SEX_UNRESOLVED", "error", "Sample sex unresolved and --unknown-sex-policy error", "CYTO-024"),
+        _m("W_CN_NO_FASTA", "warning", "No FASTA: VCF REF padding bases written as N", "CYTO-029"),
+        _m("E_CN_SIDECAR_MISSING", "error", "cn_call_manifest.json not found in the cn call output", "CYTO-032"),
+        _m("E_CN_VCF_UNINDEXED", "error", "CNV VCFs must be bgzipped and indexed for annotate", "CYTO-032"),
+        _m("E_CN_FAILED", "error", "CN calling failed for the sample", "CYTO-040"),
+        # --- targeted PGx copy number (PCN-001 .. PCN-017) ---------------------------------------------------
+        _m("E_PGX_BATCH_TOO_SMALL", "error", "Too few (passing) samples in the batch for PGx CN calling", "PCN-004"),
+        _m("W_PGX_BATCH_BELOW_RECOMMENDED", "warning", "Fewer passing samples than recommended", "PCN-004"),
+        _m("E_PGX_TRAIN_TOO_SMALL", "error", "Too few (passing) samples to train a PGx CN model", "PCN-005"),
+        _m(
+            "E_MODEL_CLUSTER_MISMATCH",
+            "error",
+            "Input GTCs were called with another cluster file than the model",
+            "IN-050",
+        ),
+        _m(
+            "W_MODEL_CLUSTER_MISMATCH",
+            "warning",
+            "Input GTCs were called with another cluster file than the model",
+            "IN-050",
+        ),
+        _m("W_PGX_REGION_FEW_PROBES", "warning", "PGx CN region has fewer than 10 unmasked probes", "PCN-008"),
+        _m("W_PGX_NO_ALT_CONTIG", "warning", "FASTA lacks a contig carrying a PGx region; REF written as N", "IN-062"),
+        _m("W_PGX_BATCH_COMPOSITION", "warning", "Batch spans several lots/batches or > 14 days of scans", "PCN-006"),
+        _m("E_PGX_CN_FAILED", "error", "PGx CN calling failed for the sample", "PCN-002"),
+        _m("E_PGX_CN_MISSING", "error", "A sample has no PGx CN VCF (give --no-cn to run without)", "PGX-011"),
+        _m("E_PGX_VCF_UNINDEXED", "error", "SNV VCFs for PGx must be bgzipped and indexed", "PGX-011"),
+        _m("E_PGX_VCF_UNSUITABLE", "error", "SNV VCF unsuitable for PGx (squashing/locus filters/foreign)", "PGX-011"),
+        _m("E_PGX_GUIDELINE", "error", "Guideline tables not present in the PGx DB", "PGX-025"),
+        _m("W_PGX_NO_PROBE_MAP", "warning", "The PGx DB has no probe map for this manifest", "PGX-035"),
+        _m("E_PGX_PLATFORM_UNKNOWN", "error", "PGx CN platform (LCG/EX) unknown: give --csv or --platform", "CLI-032"),
+        _m("E_TRAIN_TOO_SMALL", "error", "Too few GTCs to train a cluster file", "plan §12"),
+        _m("E_PGX_SOURCE", "error", "PGx source tables missing; run scripts/pgx/fetch_sources.sh", "PGX-001"),
+        # --- methylation control QC (MET-001 .. MET-018) -------------------------------------------------
+        _m("E_METH_NO_ASSAY", "error", "Probe section not found", "MET-002"),
+        _m("E_METH_NO_CONTROLS", "error", "Missing control probes in manifest", "MET-002"),
+        _m("E_METH_MISSING_COLUMNS", "error", "Missing required columns in the methylation manifest", "MET-002"),
+        _m(
+            "E_METH_CONTROLS_FORMAT",
+            "error",
+            "Controls not formatted correctly. Must have 4 columns (Address,Type,Color_Channel,Name)",
+            "MET-002",
+        ),
+        _m("E_METH_NO_IDATS", "error", "No IDATs found", "MET-004"),
+        _m(
+            "W_METH_MISSING_SAMPLE",
+            "warning",
+            "Missing sample: IDATs absent; the sample fails, the run continues",
+            "MET-015",
+        ),
+        _m("E_METH_PREP_FAILED", "error", "Prep failed (sample)", "MET-010"),
+        _m(
+            "W_METH_MISSING_CONTROL_TYPES",
+            "warning",
+            "Control types missing from the manifest; metric set to NA",
+            "MET-009",
+        ),
+        _m(
+            "W_METH_MISSING_CONTROL_NAMES",
+            "warning",
+            "Control names missing from the manifest; metric set to NA",
+            "MET-009",
+        ),
+        _m("W_METH_NO_FEATURES", "warning", "No features, skipping PCA plot", "MET-013"),
     ]
 )
 

src/simurg/formats/bpm.py

@@ -32,6 +32,7 @@ from typing import Any
 import numpy as np
 import numpy.typing as npt
 
+from simurg._core import core
 from simurg.core.errors import FormatError
 from simurg.formats.binutil import BinaryReader, load_bytes
 
@@ -223,7 +224,9 @@ def read_bpm(path: str | Path) -> Manifest:
 
 
 def _parse_locus_entries(r: BinaryReader, n: int, name_to_idx: dict[str, int]) -> dict[str, Any]:
-    """Hot loop: parse ``n`` LocusEntry records (optimised pure Python; Rust in phase P9)."""
+    """Hot loop: parse ``n`` LocusEntry records (compiled in ``simurg_core`` when installed; same results)."""
+    if core is not None:
+        return _parse_locus_entries_core(r, n, name_to_idx)
     view = memoryview(r._buf)
     size = r.size
     pos = r.pos
@@ -416,6 +419,53 @@ def _parse_locus_entries(r: BinaryReader, n: int, name_to_idx: dict[str, int]) -
     }
 
 
+_STR_COLS = (
+    "ilmn_ids",
+    "snps",
+    "chroms",
+    "ilmn_strands",
+    "source_strands",
+    "ref_strands",
+    "ploidies",
+    "species",
+    "genome_builds",
+    "sources",
+    "source_versions",
+    "customer_strands",
+    "source_seqs",
+    "top_genomic_seqs",
+    "allele_a_probe_seqs",
+    "allele_b_probe_seqs",
+)
+
+
+def _parse_locus_entries_core(r: BinaryReader, n: int, name_to_idx: dict[str, int]) -> dict[str, Any]:
+    """``_parse_locus_entries`` through ``simurg_core.bpm_locus_entries`` (identical columns and error messages)."""
+    assert core is not None
+    pos = r.pos
+    try:
+        lists, map_infos, a, b, assay, exp_cl, int_only, versions, fracs, used = core.bpm_locus_entries(
+            bytes(r._view[pos:]), n, list(name_to_idx)
+        )
+    except ValueError as e:
+        raise r.fail(str(e)) from None
+    out: dict[str, Any] = dict(zip(_STR_COLS, lists, strict=True))
+    for i in np.flatnonzero(assay < 0):
+        assay[i] = infer_assay_type(out["allele_a_probe_seqs"][i], out["allele_b_probe_seqs"][i], out["source_seqs"][i])
+    r.pos = pos + used
+    return {
+        **out,
+        "map_infos": map_infos,
+        "address_a": a,
+        "address_b": b,
+        "assay_types": assay.astype(np.uint8),
+        "exp_clusters": exp_cl,
+        "intensity_only": int_only,
+        "entry_versions": versions,
+        "base_fractions": fracs,
+    }
+
+
 _IUPAC = {
     "A": 1,
     "C": 2,

src/simurg/formats/egt.py

@@ -233,3 +233,52 @@ def _parse_body(
         trailer_floats=trailer,
         extra_per_record=extra,
     )
+
+
+def write_egt(
+    path: str | Path,
+    names: list[str],
+    addresses: npt.NDArray[np.int32],
+    n: npt.NDArray[np.int32],
+    theta_mean: npt.NDArray[np.float32],
+    theta_dev: npt.NDArray[np.float32],
+    r_mean: npt.NDArray[np.float32],
+    r_dev: npt.NDArray[np.float32],
+    cluster_sep: npt.NDArray[np.float32],
+    total_score: npt.NDArray[np.float32],
+    manifest_name: str,
+    *,
+    cluster_version: str = "",
+    date_created: str = "",
+    normalization_version: str = "1.2.0",
+) -> None:
+    """Write an EGT v3, data-block version 9 file (plan §12 step 5): 15 extra floats per record (the first is the
+    intensity threshold, written as 0), per-record scores, genotypes, names, addresses, counts and the dbv-9 trailer.
+    Readable by this reader, BeadArrayFiles and gtc2vcf."""
+    from simurg.formats.binutil import BinaryWriter
+
+    k = len(names)
+    w = BinaryWriter()
+    w.i32(3)
+    for s in ("ArrayTrain", cluster_version, cluster_version, normalization_version, date_created):
+        w.string(s)
+    w.u8(1)
+    w.string(manifest_name)
+    w.i32(9)
+    w.string("")
+    w.i32(k)
+    rec = np.zeros(k, dtype=_record_dtype(15))
+    rec["n"], rec["r_dev"], rec["r_mean"] = n, r_dev, r_mean
+    rec["theta_dev"], rec["theta_mean"] = theta_dev, theta_mean
+    w.write(rec.tobytes())
+    sc = np.zeros(k, dtype=_SCORE_DTYPE)
+    sc["cluster_sep"], sc["total_score"], sc["original_score"] = cluster_sep, total_score, total_score
+    w.write(sc.tobytes())
+    for _ in range(k):
+        w.string("")
+    for nm in names:
+        w.string(nm)
+    w.array(np.asarray(addresses, np.int32), "<i4")
+    w.array(np.asarray(n, np.int32).ravel(), "<i4")
+    w.array(np.zeros(k, np.float32), "<f4")
+    Path(path).write_bytes(w.getvalue())

src/simurg/formats/methyl_manifest.py

@@ -0,0 +1,157 @@
+"""Methylation CSV manifest (EPIC v1/v2, MSA and custom designs) — spec MET-002, MET-003; plan §6.3.
+
+Sections: ``[Heading]`` (key,value), ``[Assay]`` (header row + one row per probe; needs ``IlmnID``, ``AddressA_ID``,
+``AddressB_ID``, ``Color_Channel``) and ``[Controls]`` (``Address,Type,Color_Channel,Name``, no header row; the
+vendor files pad lines with empty fields). Infinium I probes have an ``AddressB_ID``; the colour channel is read
+from ``Color_Channel`` (``Red``/``Grn``), Infinium II probes leave both empty.
+[DOC: Illumina methylation manifest column descriptions; layout of the open-source toolkits' manifests (MET-003)]
+"""
+
+from __future__ import annotations
+
+import csv
+from dataclasses import dataclass, field
+from pathlib import Path
+
+import numpy as np
+import numpy.typing as npt
+
+from simurg.core.errors import FormatError
+from simurg.formats.csv_manifest import _open_text
+
+REQUIRED = ("IlmnID", "AddressA_ID", "AddressB_ID", "Color_Channel")
+
+
+@dataclass
+class MethylControl:
+    address: int
+    type: str
+    color_channel: str
+    name: str
+
+    @property
+    def probe_id(self) -> str:
+        return f"ctl_{self.address}"
+
+
+@dataclass
+class MethylManifest:
+    path: str
+    heading: dict[str, str]
+    probe_ids: list[str]
+    u: npt.NDArray[np.int64]  # AddressA_ID
+    m: npt.NDArray[np.int64]  # AddressB_ID; 0 for Infinium II
+    col: npt.NDArray[np.str_]  # "R" / "G" for Infinium I, "" for Infinium II
+    controls: list[MethylControl] = field(default_factory=list)
+
+    @property
+    def num_probes(self) -> int:
+        return len(self.probe_ids)
+
+    @property
+    def type_i(self) -> npt.NDArray[np.bool_]:
+        return np.asarray(self.m != 0, dtype=np.bool_)
+
+    def analytical(self) -> npt.NDArray[np.bool_]:
+        """CG and SNP probes (MET-011): IDs starting with ``cg`` or ``rs``."""
+        return np.array([p[:2] in ("cg", "rs") for p in self.probe_ids], dtype=np.bool_)
+
+
+def _address(s: str) -> int:
+    s = s.strip()
+    return int(s) if s else 0
+
+
+def _channel(s: str, *, infinium_i: bool) -> str:
+    c = s.strip().casefold()
+    if not infinium_i:
+        return ""
+    if c in ("red", "r"):
+        return "R"
+    if c in ("grn", "green", "g"):
+        return "G"
+    return "?"
+
+
+def read_methyl_manifest(path: str | Path) -> MethylManifest:
+    """Parse a methylation CSV manifest (plain or gzipped). Raises the MET-002 errors."""
+    p = Path(path)
+    spath = str(p)
+    heading: dict[str, str] = {}
+    header: list[str] | None = None
+    idx: dict[str, int] = {}
+    ids: list[str] = []
+    u: list[int] = []
+    m: list[int] = []
+    col: list[str] = []
+    controls: list[MethylControl] = []
+    section = None
+    saw_assay = saw_controls = False
+    with _open_text(p) as fh:
+        for row in csv.reader(fh):
+            if not row or not any(c.strip() for c in row):
+                continue
+            first = row[0].strip()
+            if first.startswith("[") and first.endswith("]"):
+                section = first[1:-1].strip().casefold()
+                saw_assay |= section == "assay"
+                saw_controls |= section == "controls"
+                continue
+            if section == "heading":
+                heading[first] = row[1].strip() if len(row) > 1 else ""
+            elif section == "assay":
+                if header is None:
+                    header = [c.strip() for c in row]
+                    idx = {c: i for i, c in enumerate(header)}
+                    miss = [c for c in REQUIRED if c not in idx]
+                    if miss:
+                        msg = f"{spath}: Missing required columns: {', '.join(miss)}"
+                        raise FormatError("E_METH_MISSING_COLUMNS", msg, path=spath)
+                    iu, im, ic, ii = idx["AddressA_ID"], idx["AddressB_ID"], idx["Color_Channel"], idx["IlmnID"]
+                    continue
+                ids.append(row[ii].strip())
+                a, b = _address(row[iu]), _address(row[im]) if im < len(row) else 0
+                u.append(a)
+                m.append(b)
+                col.append(_channel(row[ic] if ic < len(row) else "", infinium_i=b != 0))
+            elif section == "controls":
+                fields = list(row)
+                while fields and not fields[-1].strip():
+                    fields.pop()
+                if len(fields) < 4:  # vendor EPIC files add a 5th field ("AVG") to some NEGATIVE lines (K-026)
+                    raise FormatError(
+                        "E_METH_CONTROLS_FORMAT",
+                        f"{spath}: Controls not formatted correctly. Must have 4 columns "
+                        f"(Address,Type,Color_Channel,Name): {','.join(fields)[:80]!r}",
+                        path=spath,
+                    )
+                addr, typ, colour, name = (f.strip() for f in fields[:4])
+                controls.append(MethylControl(_address(addr), typ, colour, name))
+    if not saw_assay or header is None:
+        raise FormatError("E_METH_NO_ASSAY", f"{spath}: Probe section not found", path=spath)
+    if not saw_controls or not controls:
+        raise FormatError("E_METH_NO_CONTROLS", f"{spath}: Missing control probes in manifest", path=spath)
+    return MethylManifest(
+        path=spath,
+        heading=heading,
+        probe_ids=ids,
+        u=np.asarray(u, dtype=np.int64),
+        m=np.asarray(m, dtype=np.int64),
+        col=np.asarray(col, dtype="<U1"),
+        controls=controls,
+    )
+
+
+def write_probe_tables(man: MethylManifest, out: Path) -> tuple[Path, Path]:
+    """MET-003: ``probes.tsv`` (probe_id, u, m, col; ``NA`` for Infinium II) and ``controls.tsv``."""
+    out.mkdir(parents=True, exist_ok=True)
+    probes, ctl = out / "probes.tsv", out / "controls.tsv"
+    with probes.open("w", encoding="utf-8", newline="\n") as fh:
+        fh.write("probe_id\tu\tm\tcol\n")
+        for pid, a, b, c in zip(man.probe_ids, man.u.tolist(), man.m.tolist(), man.col.tolist(), strict=True):
+            fh.write(f"{pid}\t{a}\t{b if b else 'NA'}\t{c or 'NA'}\n")
+    with ctl.open("w", encoding="utf-8", newline="\n") as fh:
+        fh.write("address\ttype\tcolor_channel\tname\tprobe_id\n")
+        for c in man.controls:
+            fh.write(f"{c.address}\t{c.type}\t{c.color_channel}\t{c.name}\t{c.probe_id}\n")
+    return probes, ctl

src/simurg/formats/samplesheet.py

@@ -15,7 +15,12 @@ from pathlib import Path
 from simurg.core.errors import SimurgError
 
 MAX_LEN = 500
-KEY_PAIRS = (("SentrixBarcode_A", "SentrixPosition_A"), ("beadChipName", "sampleSectionName"))
+# the third pair is the methylation toolkits' convention (minfi/sesame sheets), accepted for MET-004 (K-026)
+KEY_PAIRS = (
+    ("SentrixBarcode_A", "SentrixPosition_A"),
+    ("beadChipName", "sampleSectionName"),
+    ("Sentrix_ID", "Sentrix_Position"),
+)
 RECOGNISED = (
     "Sample_ID",
     "Sample_Name",
@@ -246,7 +251,8 @@ def parse_samplesheet(path: str | Path, *, on_duplicate_sample: str = "error") -
         raise _err(
             "E_SHEET_NO_KEY_COLUMNS",
             spath,
-            "missing SentrixBarcode_A/SentrixPosition_A (or beadChipName/sampleSectionName) columns",
+            "missing SentrixBarcode_A/SentrixPosition_A (or beadChipName/sampleSectionName, "
+            "Sentrix_ID/Sentrix_Position) columns",
         )
 
     recognised = {fold(c) for c in RECOGNISED} | {fold(key[0]), fold(key[1])}

src/simurg/genotype/baf_lrr.py

@@ -11,6 +11,7 @@ import math
 import numpy as np
 import numpy.typing as npt
 
+from simurg._core import core
 from simurg.genotype.clusters import ClusterArrays
 from simurg.genotype.gencall import _libm
 
@@ -22,6 +23,17 @@ NEG_NAN = np.frombuffer(np.uint32(0xFFC00000).tobytes(), dtype=np.float32)[0]  #
 def baf_lrr(
     theta: F32Arr, r: F32Arr, raw_zero: npt.NDArray[np.bool_], c: ClusterArrays, intensity_only: npt.NDArray[np.bool_]
 ) -> tuple[F32Arr, F32Arr]:
+    if core is not None:
+        return core.baf_lrr(  # type: ignore[no-any-return]
+            np.ascontiguousarray(theta, np.float32),
+            np.ascontiguousarray(r, np.float32),
+            np.ascontiguousarray(raw_zero, np.bool_),
+            np.ascontiguousarray(c.present, np.bool_),
+            np.ascontiguousarray(c.theta_mean, np.float32).ravel(),
+            np.ascontiguousarray(c.r_mean, np.float32).ravel(),
+            np.ascontiguousarray(c.mean_r, np.float32),
+            np.ascontiguousarray(intensity_only, np.bool_),
+        )
     t = np.asarray(theta, np.float32)
     rr = np.asarray(r, np.float32)
     at, bt, ct = c.theta_mean[:, 0], c.theta_mean[:, 1], c.theta_mean[:, 2]

src/simurg/genotype/gencall.py

@@ -15,6 +15,7 @@ from dataclasses import dataclass
 import numpy as np
 import numpy.typing as npt
 
+from simurg._core import core
 from simurg.genotype.clusters import ClusterArrays
 
 F32 = np.float32
@@ -42,8 +43,17 @@ def _c_call(fn, v: float) -> float:  # type: ignore[no-untyped-def]
         return math.inf
 
 
+_CORE_FN = {math.exp: "exp", math.log10: "log10", math.log2: "log2"}
+
+
 def _libm(fn, x: F64Arr) -> F64Arr:  # type: ignore[no-untyped-def]
-    """Element-wise libm call (bit-identical to C)."""
+    """Element-wise libm call (bit-identical to C); compiled when ``simurg_core`` is installed."""
+    if core is not None:
+        xa = np.ascontiguousarray(x, dtype=np.float64).ravel()
+        if (e := getattr(fn, "exponent", None)) is not None:
+            return core.pow(xa, e).reshape(np.shape(x))  # type: ignore[no-any-return]
+        if (name := _CORE_FN.get(fn)) is not None:
+            return getattr(core, name)(xa).reshape(np.shape(x))  # type: ignore[no-any-return]
     return np.fromiter((_c_call(fn, v) for v in x.tolist()), dtype=np.float64, count=len(x))
 
 
@@ -51,6 +61,7 @@ def _pow(e: float):  # type: ignore[no-untyped-def]
     def f(v: float) -> float:
         return math.pow(v, e)
 
+    f.exponent = e  # type: ignore[attr-defined]
     return f
 
 
@@ -137,6 +148,23 @@ def call_genotypes(
     cutoff: float = DEFAULT_CUTOFF,
 ) -> Calls:
     """Bayesian calls and GenCall scores for one sample (plan §7.5)."""
+    if core is not None:
+        f32 = [np.ascontiguousarray(a, np.float32).ravel() for a in (c.theta_mean, c.theta_dev, c.r_mean, c.r_dev)]
+        geno, score = core.call_genotypes(
+            np.ascontiguousarray(theta, np.float32),
+            np.ascontiguousarray(r, np.float32),
+            np.ascontiguousarray(raw_zero, np.bool_),
+            np.ascontiguousarray(c.present, np.bool_),
+            np.ascontiguousarray(c.n, np.int32).ravel(),
+            *f32,
+            np.ascontiguousarray(c.intensity_threshold, np.float32),
+            np.ascontiguousarray(c.gentrain, np.float32),
+            np.ascontiguousarray(intensity_only, np.bool_),
+            F32(cutoff),
+        )
+        return Calls(
+            genotypes=geno, scores=score, num_calls=int((geno > 0).sum()), num_intensity_only=int(intensity_only.sum())
+        )
     n = len(theta)
     t = np.asarray(theta, np.float32)
     rr = np.asarray(r, np.float32)

src/simurg/pgx/cn.py

@@ -0,0 +1,232 @@
+"""Targeted PGx copy number — spec PCN-001, PCN-009. Pure functions, no I/O.
+
+Region value x = median over the region's unmasked probes of (sample-centred LRR − the probe's training reference),
+minus the sample's scale (median of the same quantity over non-target autosomal probes).
+
+CN mixture [OURS; array intensity compression as in cnvPartition/PennCNV, LIT: Wang et al. 2007 Genome Res 17:1665]:
+component means µ_c = δ_r + γ_r · log2((c + 1) / 3) for CN c = 0 … 5 — a probe signal proportional to (c + β) with
+a background β = 1 copy-equivalent, compressed by γ_r. δ_r (the region's CN-2 location), γ_r, σ_r and the weights
+π_r are per region; γ_r has a Gaussian prior N(0.25, 0.08²). CN 0 may sit lower than the template (its own shift
+ε₀ ≤ 0 and SD): GSTT1 probes drop by ~2 at CN 0, GSTM1 probes by ~0.3. The weights are constrained to Hardy–Weinberg
+proportions of haplotype copy numbers (0 … 3 copies per haplotype, Dirichlet prior favouring one copy). Fitted by
+EM from a grid of starts (each observed cluster labelled CN 0 … 4, CN 0 on the template or on the lowest values).
+HWE and the template's spacing ratios identify the labels in multi-cluster regions (GSTM1 0/1/2 rather than 1/2/3,
+which would imply unobserved CN-0 samples). On GSA v4 PGx CNV probes γ_r ≈ 0.19–0.56 (K-028).
+"""
+
+from __future__ import annotations
+
+import math
+from dataclasses import dataclass
+
+import numpy as np
+import numpy.typing as npt
+
+F64 = npt.NDArray[np.float64]
+CN_MAX = 5  # "5 or more"
+CNS = np.arange(CN_MAX + 1)
+TEMPLATE = np.log2((CNS + 1) / 3.0)
+HAP_ALPHA = np.array([1.2, 3.0, 1.2, 1.05])  # Dirichlet prior on haplotype copy frequencies f_0..f_3 (f_1 favoured)
+SIGMA_FLOOR = 0.012
+GAMMA_PRIOR = (0.25, 0.08)  # mean, SD
+GAMMA_RANGE = (0.15, 0.80)
+PI_FLOOR = 0.002
+CN0_PRIOR_N = 2.0
+
+
+@dataclass(frozen=True)
+class Region:
+    chrom: str
+    start0: int
+    end: int
+    name: str
+    gene: str
+
+
+def parse_regions(text: str) -> list[Region]:
+    """PCN-001 BED: ``chrom start0 end region gene``; ``#`` comments."""
+    out = []
+    for ln in text.splitlines():
+        if not ln.strip() or ln.startswith("#"):
+            continue
+        c, s, e, r, g = ln.split("\t")[:5]
+        out.append(Region(c, int(s), int(e), r, g))
+    return out
+
+
+@dataclass
+class RegionFit:
+    delta: float
+    gamma: float
+    sigma: float
+    weights: list[float]
+    n: int
+    cn0_shift: float = 0.0  # ≤ 0: extra drop of the CN-0 component below the template (probe-specific background)
+    cn0_sigma: float = 0.0  # SD of the CN-0 component (≥ sigma; 0 = sigma)
+
+    def means(self) -> F64:
+        m = self.delta + self.gamma * TEMPLATE
+        m[0] += self.cn0_shift
+        return m
+
+    def sds(self) -> F64:
+        s = np.full(len(CNS), self.sigma)
+        s[0] = max(self.sigma, self.cn0_sigma)
+        return s
+
+
+def hwe_weights(f: F64) -> F64:
+    """CN weights under Hardy–Weinberg from haplotype copy-number frequencies f_k (k = 0 … 3 copies):
+    π_c = Σ_{i+j=c} f_i f_j, CN ≥ 5 pooled into 5 [LIT: HWE; OURS: use as the CN mixture prior]."""
+    pi = np.zeros(len(CNS))
+    for i in range(len(f)):
+        for j in range(len(f)):
+            pi[min(i + j, CN_MAX)] += f[i] * f[j]
+    return pi
+
+
+def _hap_update(r: F64, f: F64) -> F64:
+    """M-step for f: split each sample's CN responsibilities over the haplotype pairs (i, j) with i + j = c."""
+    counts = HAP_ALPHA - 1.0
+    counts = counts.astype(np.float64).copy()
+    pi = hwe_weights(f)
+    for i in range(len(f)):
+        for j in range(len(f)):
+            c = min(i + j, CN_MAX)
+            share = f[i] * f[j] / max(pi[c], 1e-300)
+            w = float(r[:, c].sum()) * share
+            counts[i] += w
+            counts[j] += w
+    counts = np.maximum(counts, 1e-9)
+    out: F64 = counts / counts.sum()
+    return out
+
+
+def _norm_pdf(x: F64, mu: F64, sd: float | F64) -> F64:
+    sd_ = np.broadcast_to(np.asarray(sd, dtype=np.float64), mu.shape)[None, :]
+    return np.exp(-0.5 * ((x[:, None] - mu[None, :]) / sd_) ** 2) / (sd_ * math.sqrt(2 * math.pi))
+
+
+def _em_region(x: F64, gamma: float, delta: float, iters: int, eps0: float = 0.0) -> tuple[RegionFit, float]:
+    """EM for one region; returns the fit and its penalised log-likelihood (HWE weights from haplotype copy
+    frequencies with a Dirichlet prior, Gaussian prior on γ).
+    CN 0 has its own shift ε₀ ≤ 0 below the template and its own SD, both shrunk towards the template with a prior
+    weight of ``CN0_PRIOR_N`` samples (on some probe sets CN 0 is far lower than the compressed template, GSTT1)."""
+    n = len(x)
+    f = np.array([0.1, 0.8, 0.07, 0.03])
+    pi = hwe_weights(f)
+    sd = max(SIGMA_FLOOR, float(np.std(x)) if n > 1 else 0.05)
+    sd0 = sd
+    if eps0 < 0:  # a start with a far CN-0 cluster: the other components get the spread of the rest
+        rest = x[x > delta + gamma * TEMPLATE[0] + eps0 / 2]
+        sd = max(SIGMA_FLOOR, float(np.std(rest)) if len(rest) > 1 else 0.05)
+        sd0 = sd
+    g0, gs = GAMMA_PRIOR
+    ll = -math.inf
+    for _ in range(iters):
+        mu = delta + gamma * TEMPLATE
+        mu[0] += eps0
+        sds = np.full(len(CNS), sd)
+        sds[0] = sd0
+        dens = pi[None, :] * _norm_pdf(x, mu, sds) + 1e-300
+        tot = dens.sum(axis=1, keepdims=True)
+        ll = float(np.log(tot).sum() + ((HAP_ALPHA - 1) * np.log(np.maximum(f, 1e-12))).sum()
+                   - 0.5 * ((gamma - g0) / gs) ** 2)  # fmt: skip
+        r = dens / tot
+        f = _hap_update(r, f)
+        pi = hwe_weights(f)
+        # δ and γ: weighted least squares of x on (1, t_c) over CN ≥ 1, γ shrunk towards its prior
+        rr = r.copy()
+        rr[:, 0] = 0.0
+        w = rr.sum(axis=0)
+        sw, st, stt = float(w.sum()), float((w * TEMPLATE).sum()), float((w * TEMPLATE**2).sum())
+        sx, stx = float((rr * x[:, None]).sum()), float((rr * x[:, None] * TEMPLATE[None, :]).sum())
+        lam = sd**2 / gs**2
+        det = sw * (stt + lam) - st * st
+        if det > 0:
+            delta = (sx * (stt + lam) - st * (stx + lam * g0)) / det
+            gamma = float(np.clip((sw * (stx + lam * g0) - st * sx) / det, *GAMMA_RANGE))
+        mu = delta + gamma * TEMPLATE
+        n1 = float(w.sum())
+        if n1 > 0:
+            sd = max(SIGMA_FLOOR, math.sqrt(float((rr * (x[:, None] - mu[None, :]) ** 2).sum() / n1)))
+        r0 = r[:, 0]
+        n0 = float(r0.sum())
+        eps0 = min(0.0, float((r0 * (x - mu[0])).sum()) / (n0 + CN0_PRIOR_N))
+        v0 = float((r0 * (x - mu[0] - eps0) ** 2).sum())
+        sd0 = max(sd, math.sqrt((v0 + CN0_PRIOR_N * sd**2) / (n0 + CN0_PRIOR_N)))
+    return RegionFit(float(delta), float(gamma), sd, pi.tolist(), n, float(eps0), float(sd0)), ll
+
+
+def fit_region(x: F64, iters: int = 200) -> RegionFit:
+    """PCN-009 step 4 for one region: the best of EM runs started with each observed cluster labelled CN 0 … 4, each
+    with CN 0 on the template or on the lowest values."""
+    g0 = GAMMA_PRIOR[0]
+    cands = sorted({round(float(q) - g0 * float(t), 4) for q in np.quantile(x, [0.1, 0.3, 0.5, 0.7, 0.9])
+                    for t in TEMPLATE[:5]})  # fmt: skip
+    lo = float(np.quantile(x, 0.02))
+    best: tuple[float, float, float] | None = None
+    for d0 in cands:
+        for e0 in sorted({0.0, round(min(0.0, lo - (d0 + g0 * float(TEMPLATE[0]))), 4)}):
+            _, ll = _em_region(x, g0, d0, 80, e0)
+            if best is None or ll > best[0] + 1e-9:
+                best = (ll, d0, e0)
+    assert best is not None
+    return _em_region(x, g0, best[1], iters, best[2])[0]
+
+
+def fit_mixture(values: dict[str, F64], iters: int = 200) -> dict[str, RegionFit]:
+    """Per-region fits for every region with ≥ 3 finite values."""
+    out = {}
+    for name, x in values.items():
+        x = x[np.isfinite(x)]
+        if len(x) >= 3:
+            out[name] = fit_region(x, iters)
+    return out
+
+
+def posterior(x: float, fit: RegionFit) -> F64:
+    """P(CN = c | x) for c = 0 … 5, with the training weights (floored) as prior."""
+    pi = np.maximum(np.asarray(fit.weights), PI_FLOOR)
+    pi = pi / pi.sum()
+    d = pi * _norm_pdf(np.array([x]), fit.means(), fit.sds())[0]
+    s = float(d.sum())
+    out: F64 = d / s if s > 0 else pi
+    return out
+
+
+def baf_loglik(baf: F64, pfb: F64 | None = None, sd: float = 0.05) -> F64:
+    """PCN-010: per-CN-state log-likelihood of a region's SNP BAFs: for CN c the BAF is k/c with binomial weights
+    C(c, k) p^k (1 − p)^(c − k) from the probe's population B-allele frequency p (0.5 when unknown); CN 0 → uniform
+    [LIT: Wang et al. 2007 (PennCNV BAF emissions); OURS: region-level use]."""
+    ok = np.isfinite(baf)
+    b = baf[ok]
+    p = np.clip(np.nan_to_num(pfb[ok] if pfb is not None else np.full(len(b), 0.5), nan=0.5), 0.01, 0.99)
+    out = np.zeros(len(CNS))
+    if len(b) == 0:
+        return out
+    for c in CNS:
+        if c == 0:
+            out[c] = 0.0  # uniform on [0, 1]
+            continue
+        k = np.arange(c + 1)
+        means = k / c
+        w = np.array([math.comb(c, int(j)) for j in k])[None, :] * p[:, None] ** k[None, :] * (1 - p[:, None]) ** (
+            c - k[None, :])  # fmt: skip
+        dens = np.exp(-0.5 * ((b[:, None] - means[None, :]) / sd) ** 2) / (sd * math.sqrt(2 * math.pi))
+        out[c] = float(np.log(0.98 * (w * dens).sum(axis=1) + 0.02).sum())
+    return out
+
+
+def call_cn(x: float, fit: RegionFit, baf_ll: F64 | None = None) -> tuple[int, int, F64]:
+    """PCN-009 steps 5–6: (CN, QUAL, posterior); ``baf_ll`` (PCN-010, experimental ``--bafs-in-cn``) is added to the
+    log posterior."""
+    p = posterior(x, fit)
+    if baf_ll is not None:
+        lp = np.log(np.maximum(p, 1e-300)) + baf_ll
+        p = np.exp(lp - lp.max())
+        p = p / p.sum()
+    cn = int(np.argmax(p))
+    pmax = float(p[cn])
+    qual = 60 if pmax >= 1.0 else int(min(60, max(0, round(-10 * math.log10(max(1e-300, 1 - pmax))))))
+    return cn, qual, p

src/simurg/pgx/data/pgx_cn_regions.GRCh38.v1.bed

@@ -0,0 +1,11 @@
+# simurg PGx CN regions v1, GRCh38, from UCSC ncbiRefSeq (PCN-001)
+chr1	109687816	109693745	GSTM1	GSTM1
+chr4	68537172	68576322	UGT2B17	UGT2B17
+chr10	133527362	133539123	CYP2E1	CYP2E1
+chr16	28605257	28610044	SULT1A1	SULT1A1
+chr19	40844772	40845293	CYP2A6.intron.7	CYP2A6
+chr19	40849746	40850947	CYP2A6.exon.1	CYP2A6
+chr22	42126498	42126752	CYP2D6.exon.9	CYP2D6
+chr22	42129185	42129737	CYP2D6.intron.2	CYP2D6
+chr22	42130810	42132810	CYP2D6.p5	CYP2D6
+chr22_KI270879v1_alt	270313	278446	GSTT1	GSTT1

src/simurg/pgx/db.py

@@ -0,0 +1,289 @@
+"""PGx allele definitions from public sources → normalised VCF-style variants — spec PGX-001 … PGX-005.
+Pure functions: the reference sequence comes in as a ``fetch(contig, start0, end0)`` callable.
+
+CPIC tables (CC0, https://api.cpicpgx.org) [DB: CPIC allele definition, allele function, diplotype→phenotype]:
+``sequence_location`` gives each defining position as HGVS on the chromosome (``g.``/``m.``); ``allele_location_value``
+gives each allele's value there. Values are literal sequences (``A``, ``AA`` for a dup, ``TA(7)`` for a repeat) or
+``del…``/``ins…`` forms; the reference allele definition carries the reference value. Every value becomes a
+replacement of a reference window, then a left-normalised, anchored VCF (POS, REF, ALT) — the representation the
+SNV VCFs use (IN-046), so definitions and array records compare directly.
+"""
+
+from __future__ import annotations
+
+import re
+from collections.abc import Callable
+from dataclasses import dataclass, field
+from typing import Any
+
+Fetch = Callable[[str, int, int], str]
+HGVS_RE = re.compile(r"^([gm])\.(\d+)(?:_(\d+))?(.*)$")
+REPEAT_RE = re.compile(r"^([ACGT]+)\((\d+)\)$")
+IUPAC_WILDCARD = set("RYSWKMBDHVN")
+# GRCh38 RefSeq chromosome accessions, for full HGVS (PGX-028 "NC_000022.11:g.42126611C>G")
+REFSEQ_GRCH38 = {"1": "NC_000001.11", "2": "NC_000002.12", "3": "NC_000003.12", "4": "NC_000004.12",
+                 "5": "NC_000005.10", "6": "NC_000006.12", "7": "NC_000007.14", "8": "NC_000008.11",
+                 "9": "NC_000009.12", "10": "NC_000010.11", "11": "NC_000011.10", "12": "NC_000012.12",
+                 "13": "NC_000013.11", "14": "NC_000014.9", "15": "NC_000015.10", "16": "NC_000016.10",
+                 "17": "NC_000017.11", "18": "NC_000018.10", "19": "NC_000019.10", "20": "NC_000020.11",
+                 "21": "NC_000021.9", "22": "NC_000022.11", "X": "NC_000023.11", "Y": "NC_000024.10",
+                 "M": "NC_012920.1", "MT": "NC_012920.1"}  # fmt: skip
+
+
+def full_hgvs(chrom: str, hgvs: str) -> str:
+    """``g.…``/``m.…`` → ``NC_…:g.…`` on the GRCh38 chromosome accession (unchanged when unknown)."""
+    acc = REFSEQ_GRCH38.get(chrom.removeprefix("chr"))
+    return (
+        f"{acc}:{hgvs.split(';')[0].strip()}" if acc and not hgvs.startswith("NC_") else hgvs
+    )  # CPIC core-allele definitions: the position does not constrain the allele
+
+
+@dataclass(frozen=True)
+class Window:
+    """Reference interval [start, end] (1-based, inclusive) replaced by allele values; ``insertion`` windows are the
+    single anchor base S of ``g.S_EinsX`` (values are anchor + inserted sequence)."""
+
+    start: int
+    end: int
+    kind: str  # snv | del | ins | dup | repeat | delins
+    unit: str = ""  # repeat unit
+
+
+def parse_location(hgvs: str) -> Window | None:
+    """First HGVS expression of a CPIC ``chromosomelocation`` (several are ``;``-joined for one site)."""
+    first = hgvs.split(";")[0].strip()
+    m = HGVS_RE.match(first)
+    if not m:
+        return None
+    s = int(m.group(2))
+    e = int(m.group(3)) if m.group(3) else s
+    rest = m.group(4)
+    if re.fullmatch(r"[ACGT]>[ACGT]", rest):
+        return Window(s, s, "snv")
+    if rest.startswith("delins"):
+        return Window(s, e, "delins")
+    if rest.startswith("del"):
+        return Window(s, e, "del")
+    if rest.startswith("ins"):
+        return Window(s, s, "ins")
+    if rest.startswith("dup"):
+        return Window(s, e, "dup")
+    rm = re.fullmatch(r"([ACGT]+)\[(\d+)\]", rest)
+    if rm:
+        return Window(s, s, "repeat", rm.group(1))
+    return None
+
+
+def value_sequence(value: str, w: Window, fetch: Fetch, contig: str) -> tuple[int, int, str] | None:
+    """(start, end, sequence) that the value puts in place of the reference window; None = unusable/ambiguous."""
+    v = value.strip()
+    if " or " in v or ";" in v or not v:
+        return None
+    if w.kind == "repeat":
+        m = REPEAT_RE.match(v)
+        unit = w.unit
+        n_ref = 0
+        while fetch(contig, w.start - 1 + n_ref * len(unit), w.start - 1 + (n_ref + 1) * len(unit)) == unit:
+            n_ref += 1
+        end = w.start - 1 + n_ref * len(unit)
+        if m:
+            return w.start, end, m.group(1) * int(m.group(2))
+        if re.fullmatch(r"[ACGT]+", v):
+            return w.start, end, v
+        return None
+    if w.kind == "ins":
+        anchor = fetch(contig, w.start - 1, w.start)
+        if v == "del":
+            return w.start, w.start, anchor
+        if v.startswith("ins"):
+            return w.start, w.start, anchor + v[3:]
+        return None
+    if w.kind == "delins":
+        if v.startswith("del"):
+            return w.start, w.end, fetch(contig, w.start - 1, w.end)
+        if v.startswith("ins"):
+            return w.start, w.end, v[3:]
+        return None
+    if v.startswith("del"):
+        return w.start, w.end, ""
+    if re.fullmatch(r"[ACGT]+", v):
+        return w.start, w.end, v
+    return None
+
+
+def normalize(fetch: Fetch, contig: str, start: int, end: int, alt: str) -> tuple[int, str, str]:
+    """Replacement of reference [start, end] (1-based; end = start − 1 for a pure insertion before start) by ``alt``
+    → left-normalised, anchored (POS, REF, ALT) [LIT: Tan et al. 2015 Bioinformatics 31:2202 (vt normalize)]."""
+    ref = fetch(contig, start - 1, end)
+    pos = start
+    # trim the common suffix, then the common prefix
+    while ref and alt and ref[-1] == alt[-1]:
+        ref, alt = ref[:-1], alt[:-1]
+    while len(ref) > 0 and len(alt) > 0 and ref[0] == alt[0]:
+        ref, alt = ref[1:], alt[1:]
+        pos += 1
+    if ref and alt and len(ref) == len(alt):
+        return pos, ref, alt  # SNV / MNV
+    # left shift the indel while the base before it equals its last base
+    while pos > 1 and (ref or alt):
+        prev = fetch(contig, pos - 2, pos - 1)
+        body = ref or alt
+        if prev != body[-1] or (ref and alt):
+            break
+        if ref:
+            ref = prev + ref[:-1]
+        else:
+            alt = prev + alt[:-1]
+        pos -= 1
+    anchor = fetch(contig, pos - 2, pos - 1)
+    return pos - 1, anchor + ref, anchor + alt
+
+
+@dataclass
+class VariantSite:
+    location_id: int
+    gene: str
+    chrom: str  # contig name as in the FASTA
+    hgvs: str
+    rsid: str | None
+    ref_value: str | None
+    alleles: dict[str, tuple[int, str, str]] = field(default_factory=dict)  # value → normalised (pos, ref, alt)
+
+
+@dataclass
+class AlleleDef:
+    gene: str
+    name: str
+    function: str | None
+    activity: str | None
+    reference: bool
+    structural: bool
+    variants: list[tuple[int, str]] = field(default_factory=list)  # (location_id, value) where ≠ reference
+    frequencies: dict[str, float] = field(default_factory=dict)
+    wildcards: list[int] = field(default_factory=list)  # location ids with an IUPAC value (either base)
+
+    @property
+    def undefined(self) -> bool:
+        """A non-reference allele without any defining variant cannot be called (e.g. CPIC NUDT15 *2)."""
+        return not self.reference and not self.variants
+
+
+def cpic_definitions(
+    t: dict[str, list[dict[str, Any]]], fetch: Fetch, contig_of: Callable[[str], str | None]
+) -> tuple[dict[str, list[AlleleDef]], dict[int, VariantSite], list[str]]:
+    """CPIC tables → (gene → allele definitions, location id → site, problems)."""
+    problems: list[str] = []
+    gene_chr = {g["symbol"]: g.get("chr") for g in t["gene"]}
+    locs = {loc["id"]: loc for loc in t["sequence_location"]}
+    values: dict[int, list[tuple[int, str]]] = {}
+    for v in t["allele_location_value"]:
+        values.setdefault(v["alleledefinitionid"], []).append((v["locationid"], v["variantallele"]))
+    func = {(a["genesymbol"], a["name"]): a for a in t["allele"]}
+    defs: dict[str, list[AlleleDef]] = {}
+    sites: dict[int, VariantSite] = {}
+    ref_value: dict[int, str] = {}
+    for d in sorted(t["allele_definition"], key=lambda d: (d["genesymbol"], d["name"])):
+        if d.get("matchesreferencesequence"):
+            for lid, val in values.get(d["id"], []):
+                ref_value[lid] = val
+    for d in sorted(t["allele_definition"], key=lambda d: (d["genesymbol"], d["name"])):
+        g = d["genesymbol"]
+        a = func.get((g, d["name"]), {})
+        freq = {k: float(v) for k, v in (a.get("frequency") or {}).items() if isinstance(v, int | float)}
+        ref_def, sv = bool(d.get("matchesreferencesequence")), bool(d.get("structuralvariation"))
+        ad = AlleleDef(g, d["name"], a.get("clinicalfunctionalstatus"), a.get("activityvalue"), ref_def, sv, [], freq)
+        for lid, val in sorted(values.get(d["id"], [])):
+            loc = locs.get(lid)
+            if loc is None:
+                continue
+            if lid not in sites:
+                hg = loc.get("chromosomelocation") or ""
+                chrom = "chrM" if hg.startswith("m.") else (gene_chr.get(g) or "")
+                contig = contig_of(chrom) if chrom else None
+                if contig is None:
+                    problems.append(f"{g} {loc['name']}: no contig for {chrom!r}")
+                    continue
+                sites[lid] = VariantSite(lid, g, contig, hg, loc.get("dbsnpid"), ref_value.get(lid))
+            site = sites[lid]
+            if len(val) == 1 and val in IUPAC_WILDCARD:
+                ad.wildcards.append(lid)
+            elif val != site.ref_value:
+                ad.variants.append((lid, val))
+        defs.setdefault(g, []).append(ad)
+    for lid, site in sites.items():
+        w = parse_location(site.hgvs)
+        if w is None:
+            problems.append(f"{site.gene} {site.hgvs}: unparsed location")
+            continue
+        vals = {val for d in defs.get(site.gene, []) for (l2, val) in d.variants if l2 == lid}
+        for val in sorted(vals):
+            seq = value_sequence(val, w, fetch, site.chrom)
+            if seq is None:
+                problems.append(f"{site.gene} {site.hgvs}: value {val!r} not representable")
+                continue
+            s, e, alt = seq
+            if w.kind == "ins":
+                s, e, alt = s + 1, s, alt[1:]
+            site.alleles[val] = normalize(fetch, site.chrom, s, e, alt)
+    return defs, sites, problems
+
+
+def result_basis(gene: str, names: list[str]) -> str:
+    """PGX-002 ``genes.result_basis``."""
+    if gene in ("HLA-A", "HLA-B"):
+        return "tagging_variant"
+    if gene in ("G6PD", "APOE"):
+        return "haplotype_named"
+    if names and all(n.startswith("*") for n in names):
+        return "star_allele"
+    return "variant_based"
+
+
+def pharmvar_core_alleles(gene: str, vcfs: dict[str, str]) -> dict[str, list[tuple[str, int, str, str, str | None]]]:
+    """PharmVar per-allele GRCh38 VCFs (file name → text) → core alleles (``*N``; sub-alleles ``*N.001`` skipped) and
+    their variants (chrom, pos, ref, alt, rsid) as written by PharmVar [DB: PharmVar haplotype VCFs]."""
+    out: dict[str, list[tuple[str, int, str, str, str | None]]] = {}
+    pat = re.compile(rf"^{re.escape(gene)}_(\d+)\.vcf$")
+    for name, text in vcfs.items():
+        m = pat.match(name.rsplit("/", 1)[-1])
+        if not m:
+            continue
+        recs = []
+        for ln in text.splitlines():
+            if not ln or ln.startswith("#"):
+                continue
+            f = ln.split("\t")
+            rsid = f[2] if f[2].startswith("rs") else None
+            for alt in f[4].split(","):
+                recs.append((f[0], int(f[1]), f[3].upper(), alt.upper(), rsid))
+        out[f"*{m.group(1)}"] = recs
+    return out
+
+
+CLINPGX_LEVELS = ("1A", "1B", "2A", "2B")
+# PGX-008 parity list of variant-based genes: level-3 annotations are kept for these too
+PARITY_VARIANT_GENES = {"DPYD", "RYR1", "OPRM1", "CFTR", "DRD2", "COMT", "OPRD1", "CACNA1C", "HTR2A", "OPRK1",
+                        "MT-RNR1", "TNF", "BDNF", "IL6", "HTR2C", "GRIN2B", "MTHFR", "ITGB3", "FKBP5", "ADH1B",
+                        "ANKK1", "HMGCR", "ATM", "KIF6", "F2", "CACNA1S", "LPA", "F13A1", "GRIK1", "GRIK4", "YEATS4",
+                        "ABCG2", "F5", "ALDH2", "ANK3", "VKORC1"}  # fmt: skip
+
+
+def clinpgx_variant_genes(tsv: str, exclude: set[str], levels: tuple[str, ...] = CLINPGX_LEVELS) -> dict[str, set[str]]:
+    """ClinPGx ``clinicalVariants.tsv`` → gene → rsIDs with a clinical annotation at the given evidence levels, for
+    single-gene rows of genes that no allele-definition source covers (level 3 as well for the PGX-008 parity genes)
+    [DB: ClinPGx clinical annotations, CC BY-SA 4.0].
+    Positions and alleles are resolved later from the manifest's probes (genotype-only variant-based genes)."""
+    out: dict[str, set[str]] = {}
+    lines = tsv.splitlines()
+    head = lines[0].split("\t")
+    iv, ig, il = head.index("variant"), head.index("gene"), head.index("level of evidence")
+    for ln in lines[1:]:
+        f = ln.split("\t")
+        if len(f) <= max(iv, ig, il):
+            continue
+        gene, var, lev = f[ig].strip(), f[iv].strip(), f[il].strip()
+        ok_level = lev in levels or (lev == "3" and gene in PARITY_VARIANT_GENES)
+        if "," in gene or not gene or gene in exclude or not ok_level or not re.fullmatch(r"rs\d+", var):
+            continue
+        out.setdefault(gene, set()).add(var)
+    return out

src/simurg/pgx/phenotype.py

@@ -0,0 +1,129 @@
+"""Diplotype → phenotype lookup — spec PGX-024, PGX-025. Pure functions, no I/O.
+
+[DB: CPIC gene_result_lookup — the diplotype's allele functions (or, for activity-score genes, allele activity
+values) as a count key, e.g. {"Normal function": 1, "No function": 1} or {"1.0": 1, "0.25": 1}.] A duplicated allele
+``*Nxk`` contributes k × its activity value (CPIC activity-score rule for gene duplications); a whole-gene deletion
+(``*5``, ``*0``) has activity 0 and "No function". When a key is absent from the table, activity-score genes fall back
+to the total score (the phenotype of the largest tabulated total ≤ the score); anything else is ``Indeterminate`` with
+the flag ``phenotype_not_in_guideline`` (PGX-024).
+"""
+
+from __future__ import annotations
+
+import json
+import re
+from dataclasses import dataclass, field
+
+
+@dataclass
+class GuidelineTable:
+    gene: str
+    by_key: dict[str, tuple[str, str | None]]  # canonical key JSON → (phenotype, total activity)
+    by_total: list[tuple[float, str]]  # (total activity, phenotype), sorted
+    function: dict[str, str | None]  # allele → clinical function
+    activity: dict[str, float | None]  # allele → activity value
+
+    @property
+    def activity_based(self) -> bool:
+        return bool(self.by_total)
+
+
+@dataclass
+class PhenotypeResult:
+    phenotype: str | None
+    activity_score: float | None
+    flags: list[str] = field(default_factory=list)
+
+
+def _num(v: object) -> float | None:
+    try:
+        return float(str(v))
+    except (TypeError, ValueError):
+        return None
+
+
+def build_table(gene: str, rows: list[tuple[str, str | None, str]], alleles: list[tuple[str, str | None, str | None]]
+                ) -> GuidelineTable:  # fmt: skip
+    """``rows``: (lookup_key JSON, total activity, phenotype); ``alleles``: (allele, function, activity)."""
+    by_key: dict[str, tuple[str, str | None]] = {}
+    totals: dict[float, str] = {}
+    for key, total, pheno in rows:
+        k = _canon(json.loads(key))
+        by_key.setdefault(k, (pheno, total))
+        t = _num(total)
+        if t is not None:
+            totals.setdefault(t, pheno)
+    func: dict[str, str | None] = {a: f.lower() for a, f, _ in alleles if f}
+    act = {a: _num(v) for a, _, v in alleles}
+    return GuidelineTable(gene, by_key, sorted(totals.items()), func, act)
+
+
+def _canon(counts: dict[str, int]) -> str:
+    """Case-insensitive lookup key (CPIC mixes "Normal risk …" and "normal risk …", MT-RNR1)."""
+    merged: dict[str, int] = {}
+    for k, v in counts.items():
+        merged[k.lower()] = merged.get(k.lower(), 0) + v
+    return json.dumps(merged, sort_keys=True)
+
+
+def _split(allele: str) -> tuple[str, int]:
+    m = re.fullmatch(r"(.+?)x(\d+|N)", allele)
+    if not m:
+        return allele, 1
+    return m.group(1), (3 if m.group(2) == "N" else int(m.group(2)))
+
+
+def phenotype(t: GuidelineTable, haplotypes: tuple[str, ...]) -> PhenotypeResult:
+    """Phenotype of one diplotype (haplotypes as reported, e.g. ("*2x2", "*4") or ("*1", "*5"))."""
+    deletion = {"*5", "*0"}
+    if t.activity_based:
+        vals: list[float] = []
+        for h in haplotypes:  # a tandem "*36+*10" or "*68+*4" sums its units (CPIC activity-score rule)
+            hap_act = 0.0
+            for unit in h.split("+"):
+                core, n = _split(unit)
+                a = 0.0 if core in deletion else t.activity.get(core)
+                if a is None:
+                    return PhenotypeResult("Indeterminate", None, ["phenotype_not_in_guideline"])
+                hap_act += a * n
+            vals.append(hap_act)
+        counts: dict[str, int] = {}
+        for v in vals:
+            s = f"{v:.1f}" if abs(v * 10 - round(v * 10)) < 1e-9 else f"{v:g}"
+            counts[s] = counts.get(s, 0) + 1
+        key = _canon(counts)
+        total = sum(vals)
+        hit = t.by_key.get(key)
+        if hit is not None:
+            return PhenotypeResult(hit[0], total)
+        below = [p for tot, p in t.by_total if tot <= total + 1e-9]
+        if below:
+            return PhenotypeResult(below[-1], total)
+        return PhenotypeResult("Indeterminate", total, ["phenotype_not_in_guideline"])
+    funcs: dict[str, int] = {}
+    for h in haplotypes:
+        core, n = _split(h)
+        f = "no function" if core in deletion else t.function.get(core)
+        if f is None:
+            return PhenotypeResult("Indeterminate", None, ["phenotype_not_in_guideline"])
+        if n > 1:
+            return PhenotypeResult("Indeterminate", None, ["phenotype_not_in_guideline"])  # no CPIC rule for xN here
+        funcs[f] = funcs.get(f, 0) + 1
+    hit = t.by_key.get(_canon(funcs))
+    if hit is None:
+        return PhenotypeResult("Indeterminate", None, ["phenotype_not_in_guideline"])
+    return PhenotypeResult(hit[0], None)
+
+
+def gene_phenotype(t: GuidelineTable, feasible: list[tuple[str, ...]]) -> tuple[PhenotypeResult, list[str]]:
+    """PGX-025: a phenotype only when every feasible solution agrees; otherwise ``Indeterminate`` plus the
+    per-solution phenotypes."""
+    res = [phenotype(t, s) for s in feasible]
+    if not res:
+        return PhenotypeResult(None, None, ["no_call"]), []
+    phenos = sorted({r.phenotype or "" for r in res})
+    if len(phenos) == 1:
+        acts = {r.activity_score for r in res}
+        return PhenotypeResult(res[0].phenotype, res[0].activity_score if len(acts) == 1 else None,
+                               sorted({f for r in res for f in r.flags})), []  # fmt: skip
+    return PhenotypeResult("Indeterminate", None, ["feasible_solutions_disagree"]), [r.phenotype or "" for r in res]

src/simurg/pgx/star.py

@@ -0,0 +1,243 @@
+"""Star-allele / haplotype calling from array genotypes — spec PGX-012 … PGX-020. Pure functions, no I/O.
+
+Evidence model (PGX-012) [OURS]: each defining variant v of a gene is ``called`` (alt dosage g, GenCall score GS,
+BAF), ``no_call``, ``masked`` or ``not_covered``; only called variants contribute. With diploid CN 2 the genotype
+likelihood is P(g | d) = 1 − ε for the expected alt dosage d and ε/2 otherwise, with ε = 1e-3 for GS ≥ 0.15 and
+0.05 below (calibrated: after probe masking, 0 of 9740 GSA v4 PGx calls at DB variants disagreed with 1000G 30x in any
+GS bin — errors are probe-specific and handled by masks, not by GS; K-029). When the gene's copy number n ≠ 2
+(targeted CN), the BAF likelihood N(b; k/n, σ_b) replaces the
+genotype term (k = expected alt copies). Haplotype positions with an IUPAC value (CPIC core definitions) are wildcards:
+the best-fitting dosage is used.
+
+Candidates (PGX-013) [DIFF: solver-free]: every unordered pair of defined haplotypes, and for n ≠ 2 the structural
+forms of CYP2D6-like genes: n = 1 → one whole-gene deletion haplotype; n = 0 → two; n ≥ 3 → one haplotype duplicated
+(``*1x2``, n − 1 copies). Diplotypes that no called variant separates are array-equivalent (PGX-017) and form one
+solution, named after the member with the highest prior. Scores (PGX-014): raw_score = softmax(LL); score =
+softmax(LL + log π), π = Σ over members of f(h1)·f(h2) (×2 for h1 ≠ h2), f floored at 1e-5.
+"""
+
+from __future__ import annotations
+
+import math
+from dataclasses import dataclass, field
+
+import numpy as np
+import numpy.typing as npt
+
+F64 = npt.NDArray[np.float64]
+FREQ_FLOOR = 1e-5
+CALLED, NO_CALL, MASKED, NOT_COVERED = 0, 1, 2, 3
+STATE_NAMES = {CALLED: "called", NO_CALL: "no_call", MASKED: "masked", NOT_COVERED: "not_on_array"}
+
+
+@dataclass
+class GeneModel:
+    gene: str
+    variants: list[tuple[str, int, str, str]]  # (chrom, pos, ref, alt), one per defining variant
+    variant_ids: list[tuple[str, str | None]]  # (hgvs, rsid)
+    haplotypes: list[str]
+    carries: npt.NDArray[np.int8]  # (H, V): 1 carries alt, 0 reference, −1 wildcard
+    freq: F64  # (H,) population frequency of each haplotype (prior)
+    deletion_allele: str | None = None  # named whole-gene deletion (CYP2D6 *5), or "*0"
+    reference: str = ""
+    basis: str = "star_allele"  # PGX-002 result basis: star_allele | haplotype_named | variant_based | tagging_variant
+    activity: dict[str, float] = field(default_factory=dict)  # allele → activity value (variant-based ordering)
+    function: dict[str, str] = field(default_factory=dict)
+
+
+@dataclass
+class Observations:
+    state: npt.NDArray[np.int8]  # (V,)
+    dosage: npt.NDArray[np.int8]  # alt dosage (0 … ploidy), −1 when not called
+    gs: F64
+    baf: F64
+
+
+@dataclass
+class Solution:
+    haplotypes: tuple[str, ...]  # representative, e.g. ("*1", "*4") or ("*2x2", "*4")
+    ll: float
+    prior: float
+    members: list[tuple[str, ...]] = field(default_factory=list)
+    raw_score: float = 0.0
+    score: float = 0.0
+    copy_number: int = 2
+    copies: tuple[int, ...] = ()  # haplotype index of every gene copy of the representative (SNV model)
+
+    @property
+    def genotype(self) -> str:
+        return "/".join(self.haplotypes)
+
+
+ERR_CALLED = 1e-3
+ERR_LOW_GS = 0.05
+LOW_GS = 0.15
+
+
+def gs_error(gs: F64) -> F64:
+    """PGX-012: per-call error rate from the GenCall score (calibrated, see the module docstring)."""
+    return np.where(np.nan_to_num(gs, nan=0.0) >= LOW_GS, ERR_CALLED, ERR_LOW_GS)
+
+
+def _dosage_ll(obs: Observations, expected: npt.NDArray[np.int16], wild: npt.NDArray[np.bool_]) -> F64:
+    """(D, V) log-likelihood of each candidate's expected dosage (CN 2 genotype model)."""
+    eps = gs_error(obs.gs)[None, :]
+    g = obs.dosage[None, :].astype(np.int16)
+    hit = (expected == g) | wild
+    return np.where(hit, np.log(1 - eps), np.log(eps / 2))
+
+
+def _baf_ll(obs: Observations, frac_lo: F64, frac_hi: F64, sigma: float) -> F64:
+    """(D, V) BAF log-likelihood for expected alt fractions (a wildcard spans [lo, hi]: the closest value is used)."""
+    b = np.nan_to_num(obs.baf, nan=0.5)[None, :]
+    near = np.clip(b, frac_lo, frac_hi)
+    return np.log(0.98 * np.exp(-0.5 * ((b - near) / sigma) ** 2) / (sigma * math.sqrt(2 * math.pi)) + 0.02)
+
+
+def candidates(m: GeneModel, cn: int | None) -> list[tuple[tuple[int, ...], tuple[str, ...]]]:
+    """(haplotype index per gene copy, display names) for every candidate diplotype (PGX-013)."""
+    h = len(m.haplotypes)
+    out: list[tuple[tuple[int, ...], tuple[str, ...]]] = []
+    if cn is None or cn == 2:
+        for i in range(h):
+            for j in range(i, h):
+                out.append(((i, j), (m.haplotypes[i], m.haplotypes[j])))
+        return out
+    dele = m.deletion_allele or "*0"
+    if cn == 0:
+        return [((), (dele, dele))]
+    if cn == 1:
+        return [((i,), (m.haplotypes[i], dele)) for i in range(h)]
+    extra = cn - 2  # copies beyond two: one haplotype duplicated (e.g. CN 3 → *2x2/*4)
+    for i in range(h):
+        for j in range(h):
+            out.append(((i,) * (extra + 1) + (j,), (f"{m.haplotypes[i]}x{extra + 1}", m.haplotypes[j])))
+    return out
+
+
+@dataclass(frozen=True)
+class Cyp2d6Structure:
+    """CYP2D6 copies from the sub-region CNs (PGX-013) [OURS; DOC: PharmVar CYP2D6 structural variation]:
+    ``full`` complete CYP2D6 copies, ``u36`` *36-type units (CYP2D7 exon 9), ``u68`` *68-type units (CYP2D6 exon 1
+    only), ``u13`` *13-type units (CYP2D7 5′ part). CN(p5) = full + u36 + u68, CN(intron 2) = full + u36 + u13,
+    CN(exon 9) = full + u13; with u68 · u13 = 0 this has one solution."""
+
+    full: int
+    u36: int
+    u68: int
+    u13: int
+
+
+def cyp2d6_structure(p5: int | None, intron2: int | None, exon9: int | None) -> Cyp2d6Structure | None:
+    if p5 is None:
+        return None
+    ni = p5 if intron2 is None else intron2
+    n9 = ni if exon9 is None else exon9
+    u36 = max(0, ni - n9)
+    x = p5 - u36 - n9
+    if x >= 0:
+        full, u68, u13 = n9, x, 0
+    else:
+        full, u68, u13 = p5 - u36, 0, -x
+    if full < 0:
+        return Cyp2d6Structure(max(0, p5), 0, 0, 0)
+    return Cyp2d6Structure(full, u36, u68, u13)
+
+
+def structure_candidates(m: GeneModel, st: Cyp2d6Structure) -> list[tuple[tuple[int, ...], tuple[str, ...]]]:
+    """Candidates for a CYP2D6 structure: SNV copies = full + u36 (a *36 unit carries the *10 sequence up to exon 8);
+    *36 units attach to a *10 copy (``*36+*10``), *68 units to *4 when present (``*68+*4``), *13 units are separate
+    haplotypes."""
+    snv_copies = st.full + st.u36
+    base = candidates(m, snv_copies)
+    i10 = m.haplotypes.index("*10") if "*10" in m.haplotypes else None
+    out = []
+    for ix, names in base:
+        names_l = list(names)
+        if st.u36:
+            if i10 is None or ix.count(i10) < st.u36 + 1:  # the *36 unit(s) plus the *10 they precede
+                continue
+            k = next((n for n, x in enumerate(names_l) if x.startswith("*10")), None)
+            if k is None:
+                continue
+            unit = "*36" if st.u36 == 1 else f"*36x{st.u36}"
+            names_l[k] = f"{unit}+*10"
+        if st.u68:
+            unit = "*68" if st.u68 == 1 else f"*68x{st.u68}"
+            k = next((n for n, x in enumerate(names_l) if x.split("+")[-1].split("x")[0] == "*4"), 0)
+            names_l[k] = f"{unit}+{names_l[k]}"
+        if st.u13:
+            if len(names_l) == 2 and names_l[1] == (m.deletion_allele or "*0"):
+                names_l[1] = "*13" if st.u13 == 1 else f"*13x{st.u13}"
+            else:
+                names_l[0] = f"*13+{names_l[0]}"
+        out.append((ix, tuple(names_l)))
+    return out
+
+
+def call_gene(
+    m: GeneModel, obs: Observations, cn: int | None = None, baf_sigma: float = 0.06, ploidy: int = 2,
+    max_candidates: int = 10, feasible_ratio: float = 0.5, structure: Cyp2d6Structure | None = None,
+) -> tuple[list[Solution], list[Solution]]:  # fmt: skip
+    """All scored solutions (ranked) and the feasible ones (PGX-014/015)."""
+    called = obs.state == CALLED
+    haploid: list[tuple[tuple[int, ...], tuple[str, ...]]] = [((i,), (m.haplotypes[i],)) for i in
+                                                               range(len(m.haplotypes))]  # fmt: skip
+    if ploidy == 1:
+        cands = haploid
+    elif structure is not None and (structure.u36 or structure.u68 or structure.u13):
+        cands = structure_candidates(m, structure)
+        cn = structure.full + structure.u36
+    else:
+        cands = candidates(m, cn)
+    if not cands:
+        return [], []
+    carries = m.carries.astype(np.int16)
+    lo = np.zeros((len(cands), carries.shape[1]))
+    hi = np.zeros_like(lo)
+    for k, (ix, _) in enumerate(cands):
+        for i in ix:
+            c = carries[i]
+            lo[k] += np.where(c < 0, 0, c)
+            hi[k] += np.where(c < 0, 1, c)
+    ncp = np.array([max(1, len(ix)) for ix, _ in cands], dtype=np.float64)[:, None]
+    if ploidy == 1 or cn is None or cn == 2:
+        wild = hi != lo
+        ll_v = _dosage_ll(obs, lo.astype(np.int16), wild)
+        if wild.any():  # a wildcard may also take the higher dosage
+            ll_v = np.maximum(ll_v, _dosage_ll(obs, hi.astype(np.int16), wild))
+    else:
+        ll_v = _baf_ll(obs, lo / ncp, hi / ncp, baf_sigma)
+    ll = (ll_v * called[None, :]).sum(axis=1)
+    f = np.maximum(m.freq, FREQ_FLOOR)
+    # array-equivalence classes: identical expected dosages at every called variant (PGX-017)
+    keys: dict[bytes, list[int]] = {}
+    for k in range(len(cands)):
+        sig = np.round(np.concatenate([lo[k, called], hi[k, called]]) / ncp[k, 0], 6).tobytes()
+        keys.setdefault(sig + str(len(cands[k][0])).encode(), []).append(k)
+    sols = []
+    for members in keys.values():
+        pri = []
+        for k in members:
+            ix = cands[k][0]
+            if not ix:
+                pri.append(1e-6)
+                continue
+            p = float(np.prod([f[i] for i in ix]))
+            if len(set(ix)) > 1:
+                p *= 2
+            pri.append(p)
+        best = members[int(np.argmax(pri))]
+        sols.append(Solution(cands[best][1], float(ll[best]), float(sum(pri)), [cands[k][1] for k in members],
+                             copy_number=cn if cn is not None else 2, copies=cands[best][0]))  # fmt: skip
+    lls = np.array([s.ll for s in sols])
+    lp = lls + np.log([s.prior for s in sols])
+    raw = np.exp(lls - lls.max())
+    sc = np.exp(lp - lp.max())
+    raw, sc = raw / raw.sum(), sc / sc.sum()
+    for s, r, v in zip(sols, raw, sc, strict=True):
+        s.raw_score, s.score = float(r), float(v)
+    sols.sort(key=lambda s: (-s.score, -s.raw_score, s.genotype))
+    top = sols[0].score
+    feasible = [s for s in sols if s.score >= feasible_ratio * top]
+    return sols[: max(max_candidates, len(feasible))], feasible

src/simurg/pipeline/genotype.py

@@ -9,6 +9,8 @@ import json
 import math
 import multiprocessing as mp
 import os
+import pickle
+import tempfile
 from concurrent.futures import ProcessPoolExecutor
 from dataclasses import dataclass, field, replace
 from pathlib import Path
@@ -147,6 +149,13 @@ def _init(
     )
 
 
+def _init_from_file(path: str) -> None:
+    """Worker initializer: the shared state is pickled once by the parent (not once per worker, which serialised
+    worker start-up: ~0.25 s per worker for a GSA manifest context) and read by every worker in parallel."""
+    with open(path, "rb") as fh:
+        _init(*pickle.load(fh))
+
+
 def _process(job: Job) -> dict[str, Any]:
     ctx: ManifestContext = _W["ctx"]
     params: GenotypeParams = _W["params"]
@@ -160,7 +169,7 @@ def _process(job: Job) -> dict[str, Any]:
     try:
         grn, red = read_idat(job.grn), read_idat(job.red)
         check_pair(grn, red)
-        _, notes = check_coverage(grn, ctx.manifest)
+        _, notes = check_coverage(grn, ctx.manifest, ctx.cache)
         row["notes"].extend(notes)
         g = genotype_sample(grn, red, ctx, params, _W["started"])
         write_gtc(g, Path(job.gtc_path))
@@ -338,13 +347,17 @@ def run_genotype(opts: GenotypeOptions, rep: Reporter) -> int:
             _init(ctx, params, auto_1_22, probes, started, beads, th.bg_offset)
             results = [_process(j) for j in jobs]
         else:
-            with ProcessPoolExecutor(
-                workers,
-                mp_context=mp.get_context("spawn"),
-                initializer=_init,
-                initargs=(ctx, params, auto_1_22, probes, started, beads, th.bg_offset),
-            ) as ex:
-                results = list(ex.map(_process, jobs))
+            opts.out.mkdir(parents=True, exist_ok=True)
+            fd, state = tempfile.mkstemp(prefix=".simurg-workers-", suffix=".pkl", dir=opts.out)
+            try:
+                with os.fdopen(fd, "wb") as fh:
+                    pickle.dump((ctx, params, auto_1_22, probes, started, beads, th.bg_offset), fh, protocol=5)
+                with ProcessPoolExecutor(
+                    workers, mp_context=mp.get_context("spawn"), initializer=_init_from_file, initargs=(state,)
+                ) as ex:
+                    results = list(ex.map(_process, jobs))
+            finally:
+                Path(state).unlink(missing_ok=True)
     except KeyboardInterrupt:
         rm.status = "interrupted"
         rm.write(opts.out / "run_manifest.json")

src/simurg/pipeline/methyl.py

@@ -0,0 +1,443 @@
+"""``simurg methyl qc`` / ``simurg methyl convert-manifest`` — methylation control QC (spec MET-001 … MET-018).
+
+Per sample (streamed, worker processes): control table, detection p-values, noob + dye-bias corrected β and M-values.
+Per batch: ``methyl_qc_summary.tsv/.xlsx/.html``, ``pcs.tsv.gz``, ``probes.tsv``/``controls.tsv`` and the shared
+``controls_raw.tsv``/``controls_qc.tsv`` (so ``simurg qc report`` accepts the folder).
+"""
+
+from __future__ import annotations
+
+import datetime as _dt
+import gzip
+import html
+import json
+import math
+import multiprocessing as mp
+import os
+import pickle
+import tempfile
+from concurrent.futures import ProcessPoolExecutor
+from dataclasses import dataclass, field
+from pathlib import Path
+from typing import Any
+
+import numpy as np
+import numpy.typing as npt
+
+from simurg.core.atomic import check_outputs_absent
+from simurg.core.errors import ExitCode, SimurgError
+from simurg.core.log import Reporter
+from simurg.core.provenance import RunManifest
+from simurg.formats.idat import check_pair, read_idat
+from simurg.formats.methyl_manifest import MethylManifest, read_methyl_manifest, write_probe_tables
+from simurg.pipeline.qc import ControlSample, idat_metadata, write_control_tables, write_tsv
+from simurg.qc.controls import ControlBead, canonical_id, compute_control_metrics, metrics_for
+from simurg.qc.thresholds import Thresholds, parse_cli_pairs, resolve_thresholds
+from simurg.signal import methyl as sig
+
+SUMMARY = "methyl_qc_summary"
+CONTROL_TYPES = ("STAINING", "EXTENSION", "HYBRIDIZATION", "TARGET REMOVAL", "NON-POLYMORPHIC", "SPECIFICITY I",
+                 "SPECIFICITY II", "BISULFITE CONVERSION I", "BISULFITE CONVERSION II", "NEGATIVE", "NORM_A",
+                 "NORM_C", "NORM_G", "NORM_T")  # fmt: skip
+AVG_COLUMNS = ("avg_green_raw", "avg_red_raw", "avg_green_norm", "avg_red_norm")
+
+
+@dataclass
+class MethylQcOptions:
+    out: Path
+    manifest: Path
+    idats: Path | None = None
+    sheet: Path | None = None
+    config: Path | None = None
+    profile: str | None = "auto"
+    thresholds: list[str] = field(default_factory=list)
+    bg_offset: float | None = None
+    bs_conv_red_bg_set: str = "U3-U5"
+    pval_threshold: float = 0.05
+    beta_offset: float = 0.0
+    dye_bias: str = "linear"
+    min_prop_passing: float | None = None
+    pca_max_samples_full: int = 100
+    pca_probes: int = 10_000
+    seed: int = 42
+    threads: int = 0
+    force: bool = False
+    on_duplicate_idat: str = "error"
+    argv: list[str] = field(default_factory=list)
+
+
+@dataclass
+class _Job:
+    sample_id: str
+    grn: str | None
+    red: str | None
+    sheet_values: dict[str, str]
+    out_dir: str
+
+
+_W: dict[str, Any] = {}
+
+
+def _init_from_file(path: str) -> None:
+    with open(path, "rb") as fh:
+        _W.update(pickle.load(fh))
+
+
+def _write_gz_tsv(path: Path, header: str, lines: list[str]) -> None:
+    tmp = path.with_suffix(path.suffix + ".part")
+    with gzip.open(tmp, "wt", encoding="utf-8", newline="\n", compresslevel=6) as fh:
+        fh.write(header + "\n")
+        fh.write("\n".join(lines))
+        if lines:
+            fh.write("\n")
+    tmp.replace(path)
+
+
+def _g6(v: float) -> str:
+    return "NA" if not math.isfinite(v) else f"{v:.6g}"
+
+
+def _process(job: _Job) -> dict[str, Any]:
+    man: MethylManifest = _W["man"]
+    opts: MethylQcOptions = _W["opts"]
+    beads: list[ControlBead] = _W["beads"]
+    row: dict[str, Any] = {"sample_id": job.sample_id, "status": "OK", "fail_reason": ""}
+    if not job.grn or not job.red:
+        row.update(status="FAILED", fail_reason="W_METH_MISSING_SAMPLE: IDATs absent")
+        return row
+    step = "read"
+    try:
+        grn, red = read_idat(job.grn), read_idat(job.red)
+        check_pair(grn, red)
+        row["meta"] = idat_metadata(grn, red, job.sheet_values)
+        row["sentrix_id"], row["sentrix_position"] = grn.barcode or red.barcode, grn.position or red.position
+        row["scan_time"] = red.scan_time_iso or grn.scan_time_iso
+        step = "signal assembly"
+        raw = sig.assemble(grn, red, man)
+        g, r = raw.ctrl_green, raw.ctrl_red
+        qc = compute_control_metrics(beads, g, r, "methylation", _W["bg_offset"], opts.bs_conv_red_bg_set)
+        row["control"] = ControlSample(job.sample_id, row["meta"], g, r, qc)
+        step = "detection"
+        pval = sig.detection_p(raw)
+        step = "noob"
+        corr, _, _ = sig.noob(raw)
+        step = "dye bias"
+        corr = sig.dye_bias_nonlinear(corr) if opts.dye_bias == "nonlinear" else sig.dye_bias_linear(corr)
+        step = "beta"
+        beta = sig.beta_values(corr, opts.beta_offset)
+        mval = sig.m_values(beta)
+        ana = _W["analytical"]
+        row["n_probes"] = int(ana.sum())
+        row["n_pass_detection"] = int((pval[ana] < opts.pval_threshold).sum())
+        row["prop_probes_passing"] = row["n_pass_detection"] / row["n_probes"] if row["n_probes"] else math.nan
+        row["avg_green_raw"], row["avg_red_raw"] = sig.average_signals(raw, ana)
+        row["avg_green_norm"], row["avg_red_norm"] = sig.average_signals(corr, ana)
+        row["pca"] = beta[_W["pca_idx"]].astype(np.float32)
+        step = "write"
+        d = Path(job.out_dir)
+        d.mkdir(parents=True, exist_ok=True)
+        _write_gz_tsv(
+            d / f"{job.sample_id}_ctrl.tsv.gz",
+            "address\ttype\tcolor_channel\tname\tprobe_id\tgreen\tred",
+            [
+                f"{c.address}\t{c.type}\t{c.color_channel}\t{c.name}\t{c.probe_id}\t{_g6(gv)}\t{_g6(rv)}"
+                for c, gv, rv in zip(man.controls, g.tolist(), r.tolist(), strict=True)
+            ],
+        )
+        _write_gz_tsv(
+            d / f"{job.sample_id}_cgs.tsv.gz",
+            "probe_id\tbeta\tm_value\tdetection_p",
+            [
+                f"{pid}\t{_g6(b)}\t{_g6(mv)}\t{_g6(pv)}"
+                for pid, b, mv, pv in zip(man.probe_ids, beta.tolist(), mval.tolist(), pval.tolist(), strict=True)
+            ],
+        )
+    except Exception as e:  # MET-010: this sample fails, the others continue
+        code = e.code if isinstance(e, SimurgError) else "E_METH_PREP_FAILED"
+        row.update(status="FAILED", fail_reason=f"{code}: Prep failed ({step}): {e}")
+        row.pop("control", None)
+    return row
+
+
+def _pca(mat: npt.NDArray[np.float32]) -> npt.NDArray[np.float64] | None:
+    """First two principal components (samples × 2) of a samples × probes matrix; None when not computable."""
+    ok = np.isfinite(mat).all(axis=0)
+    x = mat[:, ok].astype(np.float64)
+    if x.shape[0] < 2 or x.shape[1] < 2:
+        return None
+    x -= x.mean(axis=0)
+    gram = x @ x.T
+    w, v = np.linalg.eigh(gram)
+    order = np.argsort(w)[::-1][:2]
+    pcs = v[:, order] * np.sqrt(np.maximum(w[order], 0))
+    for j in range(pcs.shape[1]):  # deterministic sign: largest |loading| positive
+        k = int(np.argmax(np.abs(pcs[:, j])))
+        if pcs[k, j] < 0:
+            pcs[:, j] = -pcs[:, j]
+    if pcs.shape[1] < 2:
+        pcs = np.column_stack([pcs, np.zeros(len(pcs))])
+    return pcs
+
+
+def _thresholds(opts: MethylQcOptions, man: MethylManifest) -> Thresholds:
+    cli: dict[str, Any] = parse_cli_pairs(opts.thresholds)
+    if opts.bg_offset is not None:
+        cli["bgOffset"] = opts.bg_offset
+    return resolve_thresholds(
+        config=opts.config,
+        profile=opts.profile,
+        cli=cli,
+        array_type="methylation",
+        manifest_name=man.heading.get("Descriptor File Name", Path(man.path).name),
+    )
+
+
+def run_methyl_qc(opts: MethylQcOptions, rep: Reporter) -> int:
+    from simurg.pipeline.inputs import check_paths, resolve_sample_set, sheet_values
+
+    rm = RunManifest(command="methyl qc", argv=opts.argv)
+    rep.run_id = rm.run_id
+    check_paths([(opts.manifest, "methylation manifest"), (opts.idats, "IDAT folder"), (opts.sheet, "sheet")])
+    try:
+        sset = resolve_sample_set(opts.idats, opts.sheet, opts.on_duplicate_idat, rep, rm)
+    except SimurgError as e:
+        if e.code == "E_NO_INPUT":
+            raise SimurgError("E_METH_NO_IDATS", f"No IDATs found ({e.message})") from None
+        raise
+    outputs = [opts.out / f"{SUMMARY}.{x}" for x in ("tsv", "xlsx", "html")] + [opts.out / "pcs.tsv.gz"]
+    check_outputs_absent(outputs, force=opts.force)
+    rep.info(f"reading methylation manifest {opts.manifest}")
+    man = read_methyl_manifest(opts.manifest)
+    opts.out.mkdir(parents=True, exist_ok=True)
+    for p in write_probe_tables(man, opts.out):
+        rm.add_output(p, "simurg.methyl_manifest/1")
+    th = _thresholds(opts, man)
+    for rec in th.as_records():
+        rm.set_option(f"qc.{rec['key']}", rec["value"], rec["source"])
+    beads = [ControlBead(c.type, c.color_channel, c.name, c.address, canonical_id(c.name)) for c in man.controls]
+    types = {c.type.upper() for c in man.controls}
+    if miss := [t for t in CONTROL_TYPES if t not in types]:
+        rep.warn("W_METH_MISSING_CONTROL_TYPES", f"manifest lacks control type(s): {', '.join(miss)}")
+    analytical = man.analytical()
+    n = len(sset.samples)
+    cand = np.flatnonzero(analytical)
+    if n > opts.pca_max_samples_full and len(cand) > opts.pca_probes:
+        cand = np.sort(np.random.default_rng(opts.seed).choice(cand, opts.pca_probes, replace=False))
+    jobs = []
+    for s in sset.samples:
+        grn = str(s.pair.grn) if s.pair is not None and s.pair.grn is not None else None
+        red = str(s.pair.red) if s.pair is not None and s.pair.red is not None else None
+        if not (grn and red):
+            rep.warn("W_METH_MISSING_SAMPLE", f"Missing sample: {s.sample_id}", sample_id=s.sample_id)
+        jobs.append(_Job(s.sample_id, grn, red, sheet_values(s, sset.sheet), str(opts.out / "samples" / s.sample_id)))
+    state = {"man": man, "opts": opts, "beads": beads, "bg_offset": th.bg_offset, "analytical": analytical,
+             "pca_idx": cand}  # fmt: skip
+    workers = max(1, min(opts.threads or (os.cpu_count() or 1), len(jobs) or 1))
+    rep.info(f"methylation QC of {len(jobs)} sample(s) with {workers} worker(s)")
+    if workers == 1:
+        _W.update(state)
+        try:
+            rows = [_process(j) for j in jobs]
+        finally:
+            _W.clear()
+    else:
+        fd, path = tempfile.mkstemp(prefix=".simurg-workers-", suffix=".pkl", dir=opts.out)
+        try:
+            with os.fdopen(fd, "wb") as fh:
+                pickle.dump(state, fh, protocol=5)
+            ctx = mp.get_context("spawn")
+            with ProcessPoolExecutor(workers, mp_context=ctx, initializer=_init_from_file, initargs=(path,)) as ex:
+                rows = list(ex.map(_process, jobs))
+        finally:
+            Path(path).unlink(missing_ok=True)
+
+    metrics = metrics_for("methylation")
+    col_key = {m.column: m.key for m in metrics}
+    missing_names: dict[str, list[str]] = {}
+    for r in rows:
+        if r["status"] != "OK":
+            rep.error(
+                r["fail_reason"].split(":", 1)[0], f"{r['sample_id']}: {r['fail_reason']}", sample_id=r["sample_id"]
+            )
+            continue
+        q = r["control"].qc
+        missing_names.update(q.missing)
+        if not r["scan_time"]:
+            rep.warn("W_IDAT_NO_SCAN_TIME", f"{r['sample_id']}: No scan time available", sample_id=r["sample_id"])
+        fails = [m.column for m in metrics if m.column not in q.missing and th.get(m.key).fails(q.values[m.column])]
+        if opts.min_prop_passing is not None and not r["prop_probes_passing"] >= opts.min_prop_passing:
+            fails.append("prop_probes_passing")
+        r["fail_codes"] = ";".join(fails)
+        r["na_metrics"] = ";".join(q.na_metrics)
+        r["pass_qc"] = 0 if fails else 1
+        r.update({m.column: q.values[m.column] for m in metrics})
+    if missing_names:
+        ids = sorted({c for v in missing_names.values() for c in v})
+        rep.warn("W_METH_MISSING_CONTROL_NAMES", f"manifest lacks control(s) {', '.join(ids)}; "
+                 f"not evaluable: {', '.join(sorted(missing_names))}")  # fmt: skip
+
+    meta_cols: list[str] = []
+    for j in jobs:
+        for k in j.sheet_values:
+            if k not in meta_cols:
+                meta_cols.append(k)
+    cols = ["sentrix_id", "sentrix_position", "sample_id", *meta_cols, *(m.column for m in metrics), *AVG_COLUMNS,
+            "scan_time", "n_probes", "n_pass_detection", "prop_probes_passing", "pass_qc", "fail_codes",
+            "na_metrics", "status", "fail_reason"]  # fmt: skip
+    out_rows = []
+    for j, r in zip(jobs, rows, strict=True):
+        out_rows.append({**j.sheet_values, **{c: r.get(c, "") for c in cols if c not in j.sheet_values}})
+    summary_tsv = opts.out / f"{SUMMARY}.tsv"
+    write_tsv(summary_tsv, cols, out_rows)
+    rm.add_output(summary_tsv, "simurg.methyl_qc_summary/1")
+    _write_xlsx(opts.out / f"{SUMMARY}.xlsx", cols, out_rows, th, col_key)
+
+    ok_rows = [r for r in rows if r["status"] == "OK"]
+    samples = [r["control"] for r in ok_rows]
+    if samples:
+        for p in write_control_tables(opts.out, samples, beads, th.bg_offset, rep, array_type="methylation"):
+            rm.add_output(p, "simurg.controls/1")
+    pcs = _pca(np.vstack([r["pca"] for r in ok_rows])) if len(ok_rows) >= 2 else None
+    group_col = next((k for k in meta_cols if k.casefold() == "sample_group"), None)
+    groups = {j.sample_id: (j.sheet_values.get(group_col, "") if group_col else "") for j in jobs}
+    pc_rows = []
+    if pcs is None:
+        rep.warn("W_METH_NO_FEATURES", "No features, skipping PCA plot")
+    else:
+        for r, (p1, p2) in zip(ok_rows, pcs.tolist(), strict=True):
+            pc_rows.append({"sample_id": r["sample_id"], "pc1": p1, "pc2": p2,
+                            "sample_group": groups[r["sample_id"]] or "NA"})  # fmt: skip
+    with gzip.open(opts.out / "pcs.tsv.gz", "wt", encoding="utf-8", newline="\n") as fh:
+        fh.write("sample_id\tpc1\tpc2\tsample_group\n")
+        for pr in pc_rows:
+            fh.write(f"{pr['sample_id']}\t{pr['pc1']:.6g}\t{pr['pc2']:.6g}\t{pr['sample_group']}\n")
+    _write_html(opts.out / f"{SUMMARY}.html", out_rows, pc_rows, th, metrics, man, len(cand))
+    n_ok = len(ok_rows)
+    rm.summary = {"samples": len(rows), "prepared": n_ok, "pass_qc": sum(r.get("pass_qc") == 1 for r in ok_rows)}
+    rm.status = "completed"
+    rm.write(opts.out / "run_manifest.json")
+    rep.info(f"done: {n_ok}/{len(rows)} samples prepared, {rm.summary['pass_qc']} pass QC; summary {summary_tsv}")
+    if n_ok == len(rows):
+        return ExitCode.OK
+    return ExitCode.PARTIAL if n_ok else ExitCode.ALL_FAILED
+
+
+def _write_xlsx(
+    path: Path, cols: list[str], rows: list[dict[str, Any]], th: Thresholds, col_key: dict[str, str]
+) -> None:
+    from openpyxl import Workbook
+    from openpyxl.styles import Font, PatternFill
+
+    bad = PatternFill("solid", fgColor="F5C28E")
+    wb = Workbook()
+    ws = wb.active
+    ws.title = "Samples"
+    ws.append(cols)
+    for c in ws[1]:
+        c.font = Font(bold=True)
+    for r in rows:
+        ws.append([_xl(r.get(c)) for c in cols])
+        cur = ws.max_row
+        for j, c in enumerate(cols, start=1):
+            v = r.get(c)
+            na = c in str(r.get("na_metrics", "")).split(";")
+            metric_fails = c in col_key and isinstance(v, float) and not na and th.get(col_key[c]).fails(v)
+            if metric_fails or (c == "pass_qc" and v == 0):
+                ws.cell(cur, j).fill = bad
+    ws.freeze_panes = "D2"
+    ws.auto_filter.ref = ws.dimensions
+    t2 = wb.create_sheet("Thresholds")
+    t2.append(["key", "threshold", "direction", "source"])
+    for k in col_key.values():
+        t = th.get(k)
+        t2.append([k, t.value, t.direction, t.source])
+    tmp = path.with_suffix(".xlsx.part")
+    wb.save(tmp)
+    tmp.replace(path)
+
+
+def _xl(v: Any) -> Any:
+    if isinstance(v, float) and not math.isfinite(v):
+        return "NaN" if math.isnan(v) else ("Inf" if v > 0 else "-Inf")
+    return "" if v is None else v
+
+
+def _write_html(
+    path: Path,
+    rows: list[dict[str, Any]],
+    pcs: list[dict[str, Any]],
+    th: Thresholds,
+    metrics: list[Any],
+    man: MethylManifest,
+    n_pca_probes: int,
+) -> None:
+    """MET-013: self-contained page — summary table, detection histogram, PCA of β."""
+    from simurg.pipeline.qc_report import _plotly
+
+    ok = [r for r in rows if r.get("status") == "OK"]
+    fin = [r for r in ok if math.isfinite(r["prop_probes_passing"])]
+    hist = {
+        "pass": [r["prop_probes_passing"] for r in fin if r.get("pass_qc") == 1],
+        "fail": [r["prop_probes_passing"] for r in fin if r.get("pass_qc") != 1],
+    }
+    data = {
+        "hist": hist,
+        "pcs": pcs,
+        "n_pca_probes": n_pca_probes,
+    }
+    head = ["sample_id", "pass_qc", "prop_probes_passing", "fail_codes", "na_metrics", "scan_time"]
+    trs = "".join("<tr>" + "".join(f"<td>{html.escape(_cell(r.get(c)))}</td>" for c in head) + "</tr>" for r in rows)
+    th_rows = "".join(
+        f"<tr><td>{m.column}</td><td>{m.key}</td><td>{_cell(th.get(m.key).value)}</td></tr>" for m in metrics
+    )
+    generated = _dt.datetime.now(_dt.UTC).strftime("%Y-%m-%d %H:%M UTC")
+    page = f"""<!doctype html><html lang="en"><head><meta charset="utf-8">
+<meta name="viewport" content="width=device-width, initial-scale=1"><title>Methylation QC</title>
+<style>body{{font:14px system-ui,sans-serif;margin:16px;color:#222;background:#fff}}table{{border-collapse:collapse}}
+td,th{{border:1px solid #ccc;padding:3px 6px;text-align:left}}.plots{{display:flex;flex-wrap:wrap;gap:16px}}
+.plot{{width:min(560px,100%);height:380px}}.note{{color:#666}}</style>
+<script>{_plotly()}</script></head><body>
+<h1>Methylation control QC</h1>
+<p class="note">{len(rows)} samples · manifest {html.escape(Path(man.path).name)}
+ · profile {html.escape(th.profile or "none")} · generated {generated} · SimurgArray</p>
+<div class="plots"><div id="hist" class="plot"></div><div id="pca" class="plot"></div></div>
+<h2>Samples</h2><table><tr>{"".join(f"<th>{c}</th>" for c in head)}</tr>{trs}</table>
+<h2>Thresholds</h2><table><tr><th>metric</th><th>key</th><th>threshold</th></tr>{th_rows}</table>
+<script>const D={json.dumps(data, allow_nan=False, default=_json_num)};
+Plotly.newPlot("hist",[{{x:D.hist.pass,type:"histogram",name:"pass",marker:{{color:"#4c78a8"}}}},
+{{x:D.hist.fail,type:"histogram",name:"fail",marker:{{color:"#e45756"}}}}],
+{{title:"Probes passing detection",barmode:"overlay",xaxis:{{title:"proportion"}},yaxis:{{title:"samples"}}}});
+if(D.pcs.length){{const g=[...new Set(D.pcs.map(p=>p.sample_group))];
+Plotly.newPlot("pca",g.map(k=>({{x:D.pcs.filter(p=>p.sample_group===k).map(p=>p.pc1),
+y:D.pcs.filter(p=>p.sample_group===k).map(p=>p.pc2),text:D.pcs.filter(p=>p.sample_group===k).map(p=>p.sample_id),
+mode:"markers",type:"scatter",name:k}})),{{title:"PCA of beta ("+D.n_pca_probes+" candidate probes)",
+xaxis:{{title:"PC1"}},yaxis:{{title:"PC2"}}}});}}
+else{{document.getElementById("pca").textContent="No features, PCA skipped";}}
+</script></body></html>
+"""
+    tmp = path.with_suffix(".html.part")
+    tmp.write_text(page, encoding="utf-8")
+    tmp.replace(path)
+
+
+def _cell(v: Any) -> str:
+    if v is None:
+        return ""
+    if isinstance(v, float):
+        return "NaN" if math.isnan(v) else f"{v:.4g}"
+    return str(v)
+
+
+def _json_num(v: Any) -> Any:
+    if isinstance(v, np.floating | np.integer):
+        return v.item()
+    raise TypeError(type(v))
+
+
+def run_convert_manifest(manifest: Path, out: Path, force: bool, rep: Reporter) -> int:
+    """``simurg methyl convert-manifest`` (MET-003)."""
+    check_outputs_absent([out / "probes.tsv", out / "controls.tsv"], force=force)
+    man = read_methyl_manifest(manifest)
+    probes, ctl = write_probe_tables(man, out)
+    rep.info(f"{man.num_probes} probes -> {probes}; {len(man.controls)} controls -> {ctl}")
+    return ExitCode.OK

src/simurg/pipeline/pgx_cn.py

@@ -0,0 +1,592 @@
+"""``simurg pgx copy-number train|call`` — targeted PGx copy number (spec PCN-001 … PCN-017, CLI-031/032, IN-050).
+
+I/O and orchestration only; the numerics live in ``simurg.pgx.cn``.
+
+Deviations (K-028): the per-probe reference is the training cohort's median, stored in the model (not the calling
+batch's median, which is not CN 2 in regions with common deletions such as GSTM1); no GC correction step (the region
+median over many probes plus the sample scale already removes most of it); GSTT1 needs alt-contig probe coordinates,
+which the GSA v4 PGx GRCh38 manifest carries (160 probes on `chr22_KI270879v1_alt`); manifests without them give
+`NoCall` for GSTT1.
+"""
+
+from __future__ import annotations
+
+import hashlib
+import io
+import json
+import math
+import zipfile
+from dataclasses import dataclass, field
+from datetime import UTC, datetime
+from importlib import resources
+from pathlib import Path
+from typing import Any
+
+import numpy as np
+import numpy.typing as npt
+import pyarrow as pa
+import pyarrow.parquet as pq
+
+from simurg import __version__
+from simurg.core.errors import ExitCode, SimurgError
+from simurg.core.log import Reporter
+from simurg.formats.bpm import read_bpm
+from simurg.pgx.cn import CN_MAX, Region, RegionFit, baf_loglik, call_cn, fit_mixture, parse_regions
+
+MODEL_SCHEMA = 1
+REGIONS_FILE = "pgx_cn_regions.GRCh38.v1.bed"
+N_SCALE_PROBES = 20_000
+MIN_REGION_PROBES = 10  # PCN-008
+MIN_FINITE_FRAC = 0.5  # PCN-002: fewer finite region values → NoCall [OURS]
+F32 = npt.NDArray[np.float32]
+
+
+def default_regions() -> tuple[str, str]:
+    """(file name, text) of the packaged region BED (PCN-001)."""
+    return REGIONS_FILE, resources.files("simurg.pgx.data").joinpath(REGIONS_FILE).read_text(encoding="utf-8")
+
+
+def _core(chrom: str) -> str:
+    c = chrom[3:] if chrom.lower().startswith("chr") else chrom
+    return "X" if c == "XY" else c
+
+
+@dataclass
+class PgxCnModel:
+    header: dict[str, Any]
+    regions: list[Region]
+    regions_text: str
+    probe_idx: npt.NDArray[np.int64]  # manifest indices of all probes used (region probes, then scale probes)
+    probe_region: list[str]  # region name, "" for scale probes
+    probe_ref: F32  # training reference (median sample-centred LRR)
+    fits: dict[str, RegionFit] = field(default_factory=dict)
+    probe_pfb: F32 | None = None  # training B-allele frequency per probe (NaN: no called genotypes), PCN-010
+
+    def region_sel(self, name: str) -> npt.NDArray[np.int64]:
+        return np.flatnonzero(np.asarray(self.probe_region) == name)
+
+
+def region_probes(
+    regions: list[Region], chroms: list[str], pos: npt.NDArray[np.int64], masked: set[int]
+) -> dict[str, npt.NDArray[np.int64]]:
+    """Manifest indices inside each region (1-based position in (start0, end]), mask removed (PCN-008)."""
+    core = np.array([_core(c) for c in chroms])
+    out = {}
+    for r in regions:
+        sel = np.flatnonzero((core == _core(r.chrom)) & (pos > r.start0) & (pos <= r.end))
+        out[r.name] = np.array([i for i in sel.tolist() if i not in masked], dtype=np.int64)
+    return out
+
+
+def scale_probes(
+    chroms: list[str], pos: npt.NDArray[np.int64], exclude: npt.NDArray[np.int64], n: int = N_SCALE_PROBES
+) -> npt.NDArray[np.int64]:
+    """Seeded random autosomal non-target probes for the per-sample scale (PCN-009 step 2)."""
+    core = np.array([_core(c) for c in chroms])
+    ok = np.isin(core, [str(i) for i in range(1, 23)]) & (pos > 0)
+    ok[exclude] = False
+    cand = np.flatnonzero(ok)
+    rng = np.random.default_rng(20260929)
+    return np.sort(rng.choice(cand, min(n, len(cand)), replace=False)).astype(np.int64)
+
+
+def _centred(lrr: npt.NDArray[np.float32], auto_idx: npt.NDArray[np.int64]) -> npt.NDArray[np.float64]:
+    x = lrr.astype(np.float64)
+    x[~np.isfinite(x)] = np.nan
+    med = float(np.nanmedian(x[auto_idx])) if np.isfinite(x[auto_idx]).any() else 0.0
+    return x - med
+
+
+def region_values(d: npt.NDArray[np.float64], model: PgxCnModel) -> dict[str, tuple[float, int]]:
+    """Per region: (x, n finite probes). ``d`` = centred LRR at ``model.probe_idx`` minus the reference."""
+    scale_sel = model.region_sel("")
+    sc = float(np.nanmedian(d[scale_sel])) if np.isfinite(d[scale_sel]).any() else 0.0
+    out = {}
+    for r in model.regions:
+        sel = model.region_sel(r.name)
+        v = d[sel]
+        nf = int(np.isfinite(v).sum())
+        out[r.name] = (float(np.nanmedian(v)) - sc if nf else math.nan, nf)
+    return out
+
+
+# ------------------------------------------------------------------------------------------------ model file
+def write_model(path: Path, m: PgxCnModel) -> None:
+    cols = {"index": m.probe_idx, "region": m.probe_region, "ref": m.probe_ref, "name": m.header.pop("_names")}
+    if m.probe_pfb is not None:
+        cols["pfb"] = m.probe_pfb
+    table = pa.table(cols)
+    buf = io.BytesIO()
+    pq.write_table(table, buf, compression="zstd")
+    mixture = {name: {"delta": f.delta, "gamma": f.gamma, "sigma": f.sigma, "weights": f.weights, "n": f.n,
+                      "cn0_shift": f.cn0_shift, "cn0_sigma": f.cn0_sigma} for name, f in m.fits.items()}  # fmt: skip
+    path.parent.mkdir(parents=True, exist_ok=True)
+    tmp = path.with_suffix(path.suffix + ".part")
+    with zipfile.ZipFile(tmp, "w", compression=zipfile.ZIP_STORED) as z:
+        z.writestr("header.json", json.dumps(m.header, indent=1))
+        z.writestr("probes.parquet", buf.getvalue())
+        z.writestr("regions.bed", m.regions_text)
+        z.writestr("mixture.json", json.dumps(mixture, indent=1))
+    tmp.replace(path)
+
+
+def read_model(path: Path) -> PgxCnModel:
+    """IN-050."""
+    with zipfile.ZipFile(path) as z:
+        header = json.loads(z.read("header.json"))
+        table = pq.read_table(io.BytesIO(z.read("probes.parquet")))
+        regions_text = z.read("regions.bed").decode()
+        mixture = json.loads(z.read("mixture.json"))
+    if int(header.get("schema_version", 0)) < MODEL_SCHEMA:
+        raise SimurgError("E_MODEL_VERSION", f"{path}: PGx CN model schema {header.get('schema_version')} is too old",
+                          hint="retrain it with simurg pgx copy-number train")  # fmt: skip
+    fits = {
+        k: RegionFit(
+            v["delta"],
+            v["gamma"],
+            v["sigma"],
+            list(v["weights"]),
+            int(v["n"]),
+            v.get("cn0_shift", 0.0),
+            v.get("cn0_sigma", 0.0),
+        )
+        for k, v in mixture.items()
+    }
+    return PgxCnModel(header, parse_regions(regions_text), regions_text,
+                      table.column("index").to_numpy().astype(np.int64), table.column("region").to_pylist(),
+                      table.column("ref").to_numpy().astype(np.float32), fits,
+                      table.column("pfb").to_numpy().astype(np.float32)
+                      if "pfb" in table.column_names else None)  # fmt: skip
+
+
+# ------------------------------------------------------------------------------------------------ train
+@dataclass
+class PgxTrainOptions:
+    gtcs: Path
+    bpm: Path
+    fasta: Path
+    out: Path
+    mask: Path | None = None
+    regions: Path | None = None
+    summary: Path | None = None
+    platform: str = "auto"
+    csv: Path | None = None  # CSV manifest: source of the Assay Format heading (CLI-032)
+    min_samples: int = 96
+    min_passing: int = 90
+    max_samples: int = 3000
+    lrr_sd_max: float = 0.20
+    call_rate_min: float = 0.99
+    assay_control_min: float = 1.0
+    force: bool = False
+    argv: list[str] = field(default_factory=list)
+
+
+def read_mask_tsv(path: Path | None) -> tuple[set[str], str]:
+    """IN-052: ``probe_name<TAB>reason``; returns (names, sha256)."""
+    if path is None:
+        return set(), ""
+    data = path.read_bytes()
+    names = set()
+    for ln in data.decode("utf-8").splitlines():
+        if ln.strip() and not ln.startswith("#"):
+            names.add(ln.split("\t")[0].strip())
+    return names, hashlib.sha256(data).hexdigest()
+
+
+def _summary(path: Path | None) -> dict[str, dict[str, str]]:
+    if path is None or not path.exists():
+        return {}
+    lines = path.read_text(encoding="utf-8").splitlines()
+    head = lines[0].split("\t")
+    return {r[0]: dict(zip(head, r, strict=False)) for r in (ln.split("\t") for ln in lines[1:] if ln)}
+
+
+def _assay_control(row: dict[str, str]) -> float:
+    for k, v in row.items():
+        if k.endswith("_norm_r") and k.startswith("TGA_Ctrl"):
+            try:
+                return float(v)
+            except ValueError:
+                return math.nan
+    return math.nan
+
+
+def _probe_set_sha(names: list[str], chroms: list[str], pos: npt.NDArray[np.int64]) -> str:
+    text = "\n".join(f"{n}\t{c}\t{p}" for n, c, p in zip(names, chroms, pos.tolist(), strict=True))
+    return hashlib.sha256(text.encode()).hexdigest()
+
+
+def run_train(o: PgxTrainOptions, rep: Reporter) -> int:
+    """``simurg pgx copy-number train`` (CLI-032, PCN-005)."""
+    from simurg.core.atomic import check_outputs_absent
+    from simurg.formats.discovery import discover_gtcs
+    from simurg.formats.fasta import Genome
+    from simurg.formats.gtc import read_gtc
+    from simurg.pipeline.qc import write_tsv
+
+    man = read_bpm(o.bpm)
+    csv_fmt = o.platform
+    if o.platform == "auto":
+        from simurg.formats.csv_manifest import assay_format, read_csv_manifest
+
+        csv_fmt = assay_format(read_csv_manifest(o.csv)) if o.csv else ""
+        if csv_fmt not in ("LCG", "EX"):
+            raise SimurgError(
+                "E_PGX_PLATFORM_UNKNOWN",
+                "cannot tell the platform (LCG/EX)" + (f": Assay Format {csv_fmt!r}" if csv_fmt else ": no --csv"),
+                hint="give --csv or --platform LCG|EX",
+            )
+    out_model = o.out / f"{o.bpm.stem}.simurg-pgxcn"
+    check_outputs_absent([out_model, o.out / "train_exclusions.tsv"], force=o.force)
+    genome = Genome(o.fasta)
+    if genome.build not in ("GRCh38", "unknown", ""):
+        raise SimurgError("E_BUILD_MISMATCH", f"PGx CN supports GRCh38 only (FASTA is {genome.build})")  # PCN-015
+    reg_name, reg_text = (o.regions.name, o.regions.read_text()) if o.regions else default_regions()
+    regions = parse_regions(reg_text)
+    mask_path = o.mask or (
+        o.bpm.with_name(o.bpm.stem + ".mask.tsv") if (o.bpm.with_name(o.bpm.stem + ".mask.tsv")).exists() else None
+    )
+    mask_names, mask_sha = read_mask_tsv(mask_path)
+    masked = {i for i, n in enumerate(man.names) if n in mask_names}
+    pos = man.map_infos.astype(np.int64)
+    rp = region_probes(regions, list(man.chroms), pos, masked)
+    for r in regions:
+        if len(rp[r.name]) < MIN_REGION_PROBES:
+            rep.warn("W_PGX_REGION_FEW_PROBES", f"region {r.name}: {len(rp[r.name])} unmasked probe(s) on "
+                     f"{man.manifest_name}; it will be NoCall")  # fmt: skip
+    reg_idx = np.concatenate([rp[r.name] for r in regions]) if regions else np.zeros(0, np.int64)
+    sc_idx = scale_probes(list(man.chroms), pos, reg_idx)
+    probe_idx = np.concatenate([reg_idx, sc_idx])
+    probe_region = [r.name for r in regions for _ in range(len(rp[r.name]))] + [""] * len(sc_idx)
+    gtcs = discover_gtcs(o.gtcs)
+    keys = sorted(gtcs)
+    if len(keys) > o.max_samples:
+        rng = np.random.default_rng(42)
+        keys = sorted(rng.choice(keys, o.max_samples, replace=False).tolist())
+        rep.info(f"training on a seeded random subset of {o.max_samples} of {len(gtcs)} samples (PCN-005)")
+    summ = _summary(o.summary or (o.gtcs / "sample_summary.tsv"))
+    rows = np.full((len(keys), len(probe_idx)), np.nan, dtype=np.float32)
+    alt_count = np.zeros(len(probe_idx))
+    n_called = np.zeros(len(probe_idx))
+    exclusions: list[dict[str, str]] = []
+    passing = np.zeros(len(keys), dtype=np.bool_)
+    cluster_files = set()
+    for i, k in enumerate(keys):
+        g = read_gtc(gtcs[k])
+        if len(g.logr_ratios) != man.num_loci:
+            raise SimurgError("E_MODEL_MANIFEST_MISMATCH", f"{gtcs[k]} has {len(g.logr_ratios)} loci, the manifest "
+                              f"{man.num_loci}")  # fmt: skip
+        cluster_files.add(g.cluster_file)
+        rows[i] = _centred(g.logr_ratios, sc_idx)[probe_idx]
+        gt = np.asarray(g.genotypes)[probe_idx]
+        n_called += gt > 0
+        alt_count += np.where(gt == 2, 1, np.where(gt == 3, 2, 0))
+        reasons = []
+        if not g.logr_dev <= o.lrr_sd_max:
+            reasons.append(f"logr_dev {g.logr_dev:.3f} > {o.lrr_sd_max}")
+        if not g.call_rate >= o.call_rate_min:
+            reasons.append(f"call_rate {g.call_rate:.4f} < {o.call_rate_min}")
+        ac = _assay_control(summ.get(k, {}))
+        if math.isfinite(ac) and ac < o.assay_control_min:
+            reasons.append(f"assay_control_norm_r {ac:.3f} < {o.assay_control_min}")
+        if reasons:
+            exclusions.append({"sample_id": k, "reason": "; ".join(reasons)})
+        else:
+            passing[i] = True
+    n_pass = int(passing.sum())
+    if len(keys) < o.min_samples or n_pass < o.min_passing:
+        raise SimurgError("E_PGX_TRAIN_TOO_SMALL", f"{n_pass} of {len(keys)} sample(s) passed QC; training requires "
+                          f"at least {o.min_samples} samples of which {o.min_passing} pass",
+                          hint="lower --min-samples/--min-passing only for research use")  # fmt: skip
+    ref = np.nanmedian(rows[passing], axis=0)
+    d = rows - ref
+    sc_sel = np.flatnonzero(np.asarray(probe_region) == "")
+    sc = np.nanmedian(d[:, sc_sel], axis=1)
+
+    def robust_z(v: npt.NDArray[np.float64]) -> npt.NDArray[np.float64]:
+        med = float(np.nanmedian(v[passing]))
+        mad = float(np.nanmedian(np.abs(v[passing] - med))) * 1.4826 or 1e-9
+        out: npt.NDArray[np.float64] = np.abs(v - med) / mad
+        return out
+
+    # PCN-009 / EDGE-026: outlier samples out of the reference — genome-wide level or genome-wide noise
+    noise = 1.4826 * np.nanmedian(np.abs(d[:, sc_sel] - sc[:, None]), axis=1)
+    z, zn = robust_z(sc), robust_z(noise)
+    use = passing & (z <= 5) & (zn <= 5)
+    for i in np.flatnonzero(passing & ~use).tolist():
+        exclusions.append({"sample_id": keys[i], "reason": f"genome-wide NR outlier (level |z| = {z[i]:.1f}, "
+                           f"noise |z| = {zn[i]:.1f}; limit 5)"})  # fmt: skip
+    if (~use & passing).any():
+        ref = np.nanmedian(rows[use], axis=0)
+        d = rows - ref
+    with np.errstate(invalid="ignore", divide="ignore"):
+        pfb = np.where(n_called > 0, alt_count / (2 * n_called), np.nan).astype(np.float32)
+    model = PgxCnModel({}, regions, reg_text, probe_idx, probe_region, ref.astype(np.float32), probe_pfb=pfb)
+    values = {}
+    for r in regions:
+        if len(rp[r.name]) < MIN_REGION_PROBES:
+            continue
+        values[r.name] = np.array([region_values(d[i], model)[r.name][0] for i in np.flatnonzero(use)])
+    model.fits = fit_mixture(values)
+    model.header = {
+        "schema": "simurg.pgx_cn_model", "schema_version": MODEL_SCHEMA, "software_version": __version__,
+        "manifest_name": man.manifest_name, "manifest_revision": getattr(man, "version", None),
+        "probe_set_sha256": _probe_set_sha(man.names, list(man.chroms), pos),
+        "n_probes": man.num_loci,
+        "cluster_file_name": sorted(cluster_files)[0] if len(cluster_files) == 1 else sorted(cluster_files),
+        "genome_build": "GRCh38", "fasta_contigs_sha256": genome.sha256, "platform": csv_fmt,
+        "mask_sha256": mask_sha, "regions_bed_version": reg_name, "n_training_samples": int(use.sum()),
+        "training_exclusions": len(exclusions), "created": datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%SZ"),
+        "regions_few_probes": [r.name for r in regions if len(rp[r.name]) < MIN_REGION_PROBES],
+        "model": "mu_c = delta_r + gamma_r * log2((c + 1) / 3), c = 0..5 (simurg.pgx.cn)",
+        "_names": [man.names[i] for i in probe_idx.tolist()],
+    }  # fmt: skip
+    o.out.mkdir(parents=True, exist_ok=True)
+    write_model(out_model, model)
+    write_tsv(o.out / "train_exclusions.tsv", ["sample_id", "reason"], exclusions)
+    for name, f in model.fits.items():
+        rep.info(f"{name}: CN2 at {f.delta:+.3f}, gamma {f.gamma:.3f}, sd {f.sigma:.3f}, "
+                 f"weights {' '.join(f'{w:.2f}' for w in f.weights)}")  # fmt: skip
+    rep.info(f"PGx CN model from {int(use.sum())} sample(s) written to {out_model}")
+    return ExitCode.OK
+
+
+# ------------------------------------------------------------------------------------------------ call
+@dataclass
+class PgxCallOptions:
+    gtcs: Path
+    model: Path
+    fasta: Path
+    out: Path
+    min_batch: int = 24
+    min_passing: int = 22
+    recommended_passing: int = 96
+    lrr_sd_max: float = 0.20
+    cluster_mismatch: str = "error"
+    bafs_in_cn: bool = False  # PCN-010 experimental
+    bgzip: bool = True
+    force: bool = False
+    argv: list[str] = field(default_factory=list)
+
+
+def _vcf_header(sample: str, contigs: list[tuple[str, int]], meta: dict[str, str]) -> str:
+    h = ["##fileformat=VCFv4.3", f"##source=SimurgArray {__version__} pgx-cn", f"##reference=file://{meta['fasta']}",
+         "##simurg_genome_build=GRCh38", f"##simurg_manifest={meta['manifest']}",
+         f"##simurg_pgx_cn_model={meta['model']}", f"##simurg_batch={meta['batch']}",
+         f"##OverallPloidy={meta['ploidy']}", "##GCCorrect=false"]  # fmt: skip
+    h += [f"##contig=<ID={c},length={n}>" for c, n in contigs]
+    h += ['##ALT=<ID=CNV,Description="Copy number variable region (targeted genotype)">',
+          '##FILTER=<ID=PASS,Description="All filters passed">',
+          '##FILTER=<ID=Q7,Description="Quality below 7">',
+          '##FILTER=<ID=SampleQuality,Description="Sample LRR SD above 0.2">',
+          '##FILTER=<ID=NoCall,Description="Region could not be assessed">',
+          '##INFO=<ID=END,Number=1,Type=Integer,Description="Last base of the region (1-based, inclusive)">',
+          '##INFO=<ID=SVTYPE,Number=1,Type=String,Description="CNV">',
+          '##INFO=<ID=CNVLEN,Number=1,Type=Integer,Description="Region length in bp">',
+          '##INFO=<ID=PROBE,Number=1,Type=Integer,Description="Number of probes used for the region">',
+          '##INFO=<ID=REGION,Number=1,Type=String,Description="Target region name">',
+          '##INFO=<ID=GENE,Number=1,Type=String,Description="Gene symbol">',
+          '##FORMAT=<ID=CN,Number=1,Type=Integer,Description="Copy number; 5 means 5 or more">',
+          '##FORMAT=<ID=NR,Number=1,Type=Float,Description="Aggregated normalized intensity (about CN/2)">',
+          "\t".join(["#CHROM", "POS", "ID", "REF", "ALT", "QUAL", "FILTER", "INFO", "FORMAT", sample])]  # fmt: skip
+    return "\n".join(h) + "\n"
+
+
+def call_sample_regions(
+    d: npt.NDArray[np.float64],
+    model: PgxCnModel,
+    passing: bool,
+    baf: npt.NDArray[np.float64] | None = None,
+    bafs_in_cn: bool = False,
+) -> list[dict[str, Any]]:
+    """PCN-002/009/010: one result per region; ``baf`` = the sample's BAF at ``model.probe_idx``."""
+    vals = region_values(d, model)
+    out = []
+    for r in model.regions:
+        x, nf = vals[r.name]
+        n_all = len(model.region_sel(r.name))
+        fit = model.fits.get(r.name)
+        filters: list[str] = []
+        bll = None
+        n_snp = 0
+        if baf is not None:
+            sel = model.region_sel(r.name)
+            rb = baf[sel]
+            pf = model.probe_pfb[sel].astype(np.float64) if model.probe_pfb is not None else None
+            if pf is not None:
+                rb = np.where(np.isfinite(pf) & (pf > 0.01) & (pf < 0.99), rb, np.nan)  # informative SNPs only
+            n_snp = int(np.isfinite(rb).sum())
+            bll = baf_loglik(rb, pf)
+        if fit is None or n_all < MIN_REGION_PROBES or nf < MIN_FINITE_FRAC * n_all or not math.isfinite(x):
+            cn, qual, nr = None, 0, math.nan
+            filters.append("NoCall")
+        else:
+            cn, qual, _ = call_cn(x, fit, bll if bafs_in_cn and n_snp else None)
+            # NR ≈ CN / 2: the continuous CN that the region's mixture means imply for x [OURS]
+            nr = max(0.0, 3.0 * 2.0 ** ((x - fit.delta) / fit.gamma) - 1.0) / 2.0 if cn else 0.0
+            if qual < 7:
+                filters.append("Q7")
+        if not passing:
+            filters.append("SampleQuality")
+        out.append({"region": r, "cn": cn, "qual": qual, "nr": nr, "x": x, "probes": nf, "filters": filters or ["PASS"],
+                    "n_snp": n_snp,
+                    "baf_ll": None if bll is None or not n_snp else [round(float(v), 3) for v in bll]})  # fmt: skip
+    return out
+
+
+def _parse_date(text: str) -> Any:
+    from datetime import datetime as _dt
+
+    for fmt in ("%m/%d/%Y %I:%M:%S %p", "%m/%d/%Y %H:%M:%S", "%Y-%m-%dT%H:%M:%S", "%Y-%m-%d %H:%M:%S"):
+        try:
+            return _dt.strptime(text.strip(), fmt)
+        except ValueError:
+            continue
+    return None
+
+
+def run_call(o: PgxCallOptions, rep: Reporter) -> int:
+    """``simurg pgx copy-number call`` (CLI-031)."""
+    from simurg.core.atomic import atomic_write_text, check_outputs_absent
+    from simurg.formats.discovery import discover_gtcs
+    from simurg.formats.fasta import ContigMap, Genome
+    from simurg.formats.gtc import read_gtc
+    from simurg.pipeline.cn import write_vcf
+    from simurg.pipeline.qc import write_tsv
+
+    model = read_model(o.model)
+    genome = Genome(o.fasta)
+    if genome.build not in ("GRCh38", "unknown", ""):
+        raise SimurgError("E_BUILD_MISMATCH", f"PGx CN supports GRCh38 only (FASTA is {genome.build})")
+    cmap = ContigMap([e.name for e in genome.entries], "auto")
+    gtcs = discover_gtcs(o.gtcs)
+    ext = ".pgx_cn.vcf.gz" if o.bgzip else ".pgx_cn.vcf"
+    check_outputs_absent([o.out / f"{k}{ext}" for k in gtcs] + [o.out / "pgx_cn_batch_summary.tsv"], force=o.force)
+    n_probes = int(model.header["n_probes"])
+    mcl = model.header.get("cluster_file_name")
+    # pass 1: batch QC from GTC headers (PCN-004)
+    heads = {}
+    scan_dates: list[Any] = []
+    for k, p in gtcs.items():
+        g = read_gtc(p)
+        heads[k] = (g.logr_dev, g.sample_name or k, g.cluster_file, len(g.logr_ratios))
+        scan_dates.append(_parse_date(g.imaging_date))
+    n_pass = sum(1 for v in heads.values() if v[0] < o.lrr_sd_max)
+    if len(heads) < o.min_batch or n_pass < o.min_passing:
+        raise SimurgError("E_PGX_BATCH_TOO_SMALL", f"{n_pass} sample(s) passed QC. Requires at least "
+                          f"{max(o.min_passing, 1)} samples to proceed.")  # fmt: skip
+    dates = [x for x in scan_dates if x is not None]
+    if dates and (max(dates) - min(dates)).days > 14:  # PCN-006 (sample-sheet lot/batch columns: no sheet input here)
+        span = (max(dates) - min(dates)).days
+        rep.warn("W_PGX_BATCH_COMPOSITION", f"scan dates span {span} days ({min(dates):%Y-%m-%d} … "
+                 f"{max(dates):%Y-%m-%d}); batch statistics may mix processing batches")  # fmt: skip
+    if n_pass < o.recommended_passing:
+        rep.warn("W_PGX_BATCH_BELOW_RECOMMENDED", f"{n_pass} passing sample(s); {o.recommended_passing} recommended")
+    for k, (_, _, cf, nl) in heads.items():
+        if nl != n_probes:
+            raise SimurgError("E_MODEL_MANIFEST_MISMATCH", f"{k} has {nl} loci, the model's manifest {n_probes}")
+        if isinstance(mcl, str) and cf and cf != mcl:
+            msg = f"Cluster file ({cf}) differs from the CN model's cluster file ({mcl})"
+            if o.cluster_mismatch == "error":
+                raise SimurgError("E_MODEL_CLUSTER_MISMATCH", msg, hint="retrain the model (PCN-005)")
+            rep.warn("W_MODEL_CLUSTER_MISMATCH", msg)
+            break
+    contigs: list[tuple[str, int]] = []
+    contig_name: dict[str, str | None] = {}
+    for r in model.regions:
+        if r.chrom in contig_name:
+            continue
+        c = cmap.get(r.chrom)
+        contig_name[r.chrom] = c
+        if c is None:
+            rep.warn("W_PGX_NO_ALT_CONTIG", f"{r.chrom} is not in {o.fasta.name}; REF written as N (IN-062)")
+        contigs.append((r.chrom, genome.contig_length(c) if c else r.end))
+    order = {c: i for i, (c, _) in enumerate(contigs)}
+    model_sha = hashlib.sha256(o.model.read_bytes()).hexdigest()
+    mh = model.header
+    meta = {
+        "fasta": str(o.fasta.resolve()),
+        "manifest": f"{mh['manifest_name']};sha256={mh['probe_set_sha256']}",
+        "model": f"{o.model.name};sha256={model_sha};cluster_file={mcl};regions={mh['regions_bed_version']}",
+        "batch": f"n_samples={len(heads)};n_passing={n_pass};lrr_sd_max={o.lrr_sd_max}",
+    }
+    centre_idx = model.probe_idx[model.region_sel("")]  # the scale probes (autosomal), as in training
+    o.out.mkdir(parents=True, exist_ok=True)
+    baf_rows: list[dict[str, Any]] = []
+    srows, per_region = [], {r.name: {"n_gain": 0, "n_loss": 0, "n_neutral": 0, "n_nocall": 0, "n_filtered": 0}
+                             for r in model.regions}  # fmt: skip
+    failed = 0
+    for k, p in gtcs.items():
+        try:
+            g = read_gtc(p)
+            d = _centred(g.logr_ratios, centre_idx)[model.probe_idx] - model.probe_ref
+            passing = heads[k][0] < o.lrr_sd_max
+            bafs = None
+            if len(
+                g.b_allele_freqs
+            ):  # PCN-010: SNP probes only (called genotypes; intensity-only CNV probes never are)
+                bafs = np.asarray(g.b_allele_freqs, dtype=np.float64)[model.probe_idx]
+                bafs[np.asarray(g.genotypes)[model.probe_idx] == 0] = np.nan
+            res = call_sample_regions(d, model, passing, bafs, o.bafs_in_cn)
+        except SimurgError:
+            raise
+        except Exception as e:  # PCN: one sample never stops the batch
+            failed += 1
+            rep.error("E_PGX_CN_FAILED", f"{k}: {e}", sample_id=k)
+            continue
+        for x in res:
+            if x["baf_ll"] is not None:
+                bl = x["baf_ll"]
+                baf_rows.append({"sample_id": k, "region": x["region"].name, "n_snp": x["n_snp"],
+                                 **{f"ll_cn{c}": bl[c] for c in range(len(bl))},
+                                 "best_cn_by_baf": int(np.argmax(bl[1:]) + 1)})  # fmt: skip
+        called = [(x["region"], x["cn"]) for x in res if x["cn"] is not None]
+        tot = sum(r.end - r.start0 for r, _ in called)
+        ploidy = sum((r.end - r.start0) * min(cn, CN_MAX) for r, cn in called) / tot if tot else math.nan
+        meta_s = {**meta, "ploidy": f"{ploidy:.3f}" if math.isfinite(ploidy) else "."}
+        recs = []
+        for x in sorted(res, key=lambda y: (order[y["region"].chrom], y["region"].start0)):
+            r = x["region"]
+            c = contig_name.get(r.chrom)
+            ref = (genome.fetch(c, r.start0 - 1, r.start0) or "N").upper() if c and r.start0 >= 1 else "N"
+            s, e_ = r.start0 + 1, r.end
+            cn = "." if x["cn"] is None else str(x["cn"])
+            nr = "." if not math.isfinite(x["nr"]) else f"{x['nr']:.6f}"
+            line = "\t".join([r.chrom, str(s - 1), f"CNV:{r.name}:{r.chrom}:{s}-{e_}", ref, "<CNV>", str(x["qual"]),
+                              ";".join(x["filters"]), f"END={e_};SVTYPE=CNV;CNVLEN={e_ - s + 1};PROBE={x['probes']};"
+                              f"REGION={r.name};GENE={r.gene}", "CN:NR", f"{cn}:{nr}"])  # fmt: skip
+            recs.append((r.chrom, s - 2, e_, line))
+            pr = per_region[r.name]
+            if x["cn"] is None:
+                pr["n_nocall"] += 1
+            elif x["cn"] > 2:
+                pr["n_gain"] += 1
+            elif x["cn"] < 2:
+                pr["n_loss"] += 1
+            else:
+                pr["n_neutral"] += 1
+            pr["n_filtered"] += x["filters"] != ["PASS"]
+        write_vcf(o.out / f"{k}{ext}", _vcf_header(heads[k][1], contigs, meta_s), recs, o.bgzip)
+        srows.append({"sample_id": k, "sample_name": heads[k][1], "sample_folder": str(Path(p).parent),
+                      "logr_dev": f"{heads[k][0]:.6f}", "passed_batch_qc": str(passing).lower(),
+                      "used_in_reference": "false",
+                      **{f"cn_{x['region'].name}": ("." if x["cn"] is None else x["cn"]) for x in res},
+                      "flags": ""})  # fmt: skip
+    cols = ["region", "gene", "n_gain", "n_loss", "n_neutral", "n_nocall", "n_filtered"]
+    rows: list[dict[str, Any]] = [
+        {"region": r.name, "gene": r.gene, **per_region[r.name]} for r in sorted(model.regions, key=lambda r: r.name)
+    ]
+    totals = {c: sum(per_region[r.name][c] for r in model.regions) for c in cols[2:]}
+    rows.append({"region": "ALL", "gene": "", **totals})
+    write_tsv(o.out / "pgx_cn_batch_summary.tsv", cols, rows)
+    scols = ["sample_id", "sample_name", "sample_folder", "logr_dev", "passed_batch_qc", "used_in_reference"]
+    scols += [f"cn_{r.name}" for r in model.regions] + ["flags"]
+    write_tsv(o.out / "pgx_cn_sample_summary.tsv", scols, srows)
+    write_tsv(o.out / "pgx_cn_baf_evidence.tsv", ["sample_id", "region", "n_snp"] + [f"ll_cn{c}" for c in range(6)]
+              + ["best_cn_by_baf"], baf_rows)  # PCN-010: for the star-allele caller and review  # fmt: skip
+    run = {"software": f"SimurgArray {__version__}", **meta, "samples": len(gtcs), "failed": failed}
+    atomic_write_text(o.out / "pgx_cn_run.json", json.dumps(run, indent=1))
+    rep.info(f"PGx CN: {len(srows)}/{len(gtcs)} sample(s) called")
+    if failed == 0:
+        return ExitCode.OK
+    return ExitCode.PARTIAL if srows else ExitCode.ALL_FAILED