# SimurgArray

Personal research toolkit for Illumina Infinium genotyping BeadChip data
(IDAT → normalization → genotype calling → BAF/LRR → QC → VCF/PLINK; CN/LOH/mosaicism, PGx, cluster training,
methylation control QC).

- **SIMURGARRAY_MASTER_PLAN.md** — architecture, algorithms, roadmap
- **SIMURGARRAY_FUNCTIONAL_SPEC.md** — 416 itemised requirements (CLI, inputs, outputs, QC, CN, PGx, methylation, edge cases)
- `docs/validation/` — per-phase validation reports · `docs/known_unknowns.md` — findings log · `docs/research/` — background

## Status

**v2.0.0** (2026-09-30) — see `CHANGELOG.md`. v1: IDAT → normalization → GenCall-equivalent genotype calling →
GTC v5 → sample/control QC with an HTML report → plus-strand VCF/BCF → PLINK 1/2, bedGraph, final report; plus a
differential-testing harness. v2 adds cluster training (ArrayTrain), genome-wide CN/LOH/mosaic calling,
pharmacogenomics (targeted CN, star alleles, phenotypes) and methylation control QC. Validation (`docs/validation/`):

| Check | Result |
|---|---|
| 2141 vendor GTCs, 1000 Genomes Omni2.5 (`--preset autocall-1.6.2`) | normalization bit-identical; GTCs byte-identical in 2138/2141 |
| GSA demo vs gtc2vcf, GTCtoVCF, plink2 | zero unexplained differences |
| 1000 Genomes pedigrees (Omni2.5) | 717/717 parent–offspring pairs; Mendel error median 0.031 % |
| 360 GSA arrays (GSE165845) vs NYGC 30x WGS / GIAB HG001 | concordance 0.9934 / 0.9985; replicates 0.9966 |
| ArrayTrain EGT, GSE165845 chr22 vs 1000G 30x | 99.44 % concordance (vendor EGT 99.46 %), call rate 99.61 % |
| CN, 360 GSE165845 arrays vs NYGC 30x SV | PASS precision DEL 44 % / DUP 57 % |
| PGx, GSA v4 demo vs GeT-RM | 154/168 gene calls |
| Methylation QC vs sesame | β r = 0.99998 |

Commands: `simurg inspect`, `ingest`, `normalize`, `genotype`, `qc controls|samples|contamination|relatedness|loci|report|suggest-thresholds`,
`gtc-to-vcf`, `export plink|bedgraph|finalreport|locus-summary`, `compare`, `train`, `cn build-model|call|annotate`,
`pgx db build|coverage|copy-number|star-allele|phenotype`, `methyl qc|convert-manifest`. **User guide:
`docs/user_guide.md`**; worked example: `examples/gsa_demo.sh`. Optional Rust kernels (byte-identical outputs,
~80 GSA samples/min per server core, `docs/validation/P9.md`); wheels, Docker image and conda recipes
(`docs/validation/P10.md`).

## Quick start

```bash
uv venv && uv pip install -e ".[dev]"
uv run pytest tests/unit                  # no downloads needed
bash scripts/bootstrap_refs.sh            # reference open-source repos → refs/ (oracles)
bash scripts/fetch_data.sh p0             # ~1.4 GB of public test data → data/
bash scripts/make_oracle_gtcs.sh          # builds gtc2vcf, writes oracle GTCs
bash scripts/make_oracle_qc.sh            # locus-summary oracle (BeadArrayFiles, separate venv)
bash scripts/fetch_data.sh p6             # GRCh38 FASTA (~0.9 GB download, 3.1 GB unpacked)
bash scripts/make_oracle_vcf.sh           # gtc2vcf oracle VCF for the P6 tests
uv run pytest tests/oracle                # real-data acceptance tests (add -m "not slow" to skip the long ones)
uv run simurg inspect data/p0/gsa_v4_pgx/idats_Demo_36/idats_Demo_36/207895980019/207895980019_R03C01_Grn.idat
bash examples/gsa_demo.sh results/gsa_demo 6   # end-to-end run on 6 demo samples
```

Vendor data files are downloaded for personal research use only and are never committed. See `THIRD_PARTY_NOTICES.md`.