kemal
0b05b0d346
Wheels (pure + manylinux_2_28 kernels), sdist, Docker image and conda packages built and installed on fresh Ubuntu 24.04 machines; 225 unit tests pass against the installed package; the tutorial runs with pip (with and without kernels); GTC/VCF content digests are identical across pip, Docker and conda on both architectures and equal to the Pi's run. docs/validation/P10.md; guide: install from wheels, Docker, conda; exit code 141.
47 lines
2.9 KiB
Markdown
47 lines
2.9 KiB
Markdown
# SimurgArray
|
||
|
||
Personal research toolkit for Illumina Infinium genotyping BeadChip data
|
||
(IDAT → normalization → genotype calling → BAF/LRR → QC → VCF/PLINK; later CN/LOH, PGx, cluster training).
|
||
|
||
- **SIMURGARRAY_MASTER_PLAN.md** — architecture, algorithms, roadmap
|
||
- **SIMURGARRAY_FUNCTIONAL_SPEC.md** — 416 itemised requirements (CLI, inputs, outputs, QC, CN, PGx, methylation, edge cases)
|
||
- `docs/validation/` — per-phase validation reports · `docs/known_unknowns.md` — findings log · `docs/research/` — background
|
||
|
||
## Status
|
||
|
||
**v1.0.0** (2026-09-28) — see `CHANGELOG.md`. IDAT → normalization → GenCall-equivalent genotype calling → GTC v5 →
|
||
sample/control QC with an HTML report → plus-strand VCF/BCF → PLINK 1/2, bedGraph, final report; plus a
|
||
differential-testing harness. Validation (`docs/validation/`):
|
||
|
||
| Check | Result |
|
||
|---|---|
|
||
| 2141 vendor GTCs, 1000 Genomes Omni2.5 (`--preset autocall-1.6.2`) | normalization bit-identical; GTCs byte-identical in 2138/2141 |
|
||
| GSA demo vs gtc2vcf, GTCtoVCF, plink2 | zero unexplained differences |
|
||
| 1000 Genomes pedigrees (Omni2.5) | 717/717 parent–offspring pairs; Mendel error median 0.031 % |
|
||
| 360 GSA arrays (GSE165845) vs NYGC 30x WGS / GIAB HG001 | concordance 0.9934 / 0.9985; replicates 0.9966 |
|
||
|
||
Commands: `simurg inspect`, `ingest`, `normalize`, `genotype`, `qc controls|samples|contamination|relatedness|loci|report|suggest-thresholds`,
|
||
`gtc-to-vcf`, `export plink|bedgraph|finalreport|locus-summary`, `compare`. **User guide: `docs/user_guide.md`**;
|
||
worked example: `examples/gsa_demo.sh`. P9 (performance) is done: optional Rust kernels, byte-identical outputs,
|
||
~80 GSA samples/min per server core (`docs/validation/P9.md`). P10 (packaging) is done: wheels, Docker image and
|
||
conda recipes, verified on fresh x86_64 and aarch64 machines (`docs/validation/P10.md`). Next (plan §18): v2 modules
|
||
(cluster training, CN/LOH, PGx).
|
||
|
||
## Quick start
|
||
|
||
```bash
|
||
uv venv && uv pip install -e ".[dev]"
|
||
uv run pytest tests/unit # no downloads needed
|
||
bash scripts/bootstrap_refs.sh # reference open-source repos → refs/ (oracles)
|
||
bash scripts/fetch_data.sh p0 # ~1.4 GB of public test data → data/
|
||
bash scripts/make_oracle_gtcs.sh # builds gtc2vcf, writes oracle GTCs
|
||
bash scripts/make_oracle_qc.sh # locus-summary oracle (BeadArrayFiles, separate venv)
|
||
bash scripts/fetch_data.sh p6 # GRCh38 FASTA (~0.9 GB download, 3.1 GB unpacked)
|
||
bash scripts/make_oracle_vcf.sh # gtc2vcf oracle VCF for the P6 tests
|
||
uv run pytest tests/oracle # real-data acceptance tests (add -m "not slow" to skip the long ones)
|
||
uv run simurg inspect data/p0/gsa_v4_pgx/idats_Demo_36/idats_Demo_36/207895980019/207895980019_R03C01_Grn.idat
|
||
bash examples/gsa_demo.sh results/gsa_demo 6 # end-to-end run on 6 demo samples
|
||
```
|
||
|
||
Vendor data files are downloaded for personal research use only and are never committed. See `THIRD_PARTY_NOTICES.md`.
|