Commit Graph
8 Commits
Author SHA1 Message Date
kemal 80b93e9f45 P5 validated on 2141 1000G samples; control-metric fixes (K-021)
- Omni2.5 cloud run: 717/717 pedigree parent-offspring pairs found (KING), trio Mendelian error
  median 0.031 %, sex vs pedigree 99.6 %, P4 preset re-check 2138/2141 byte-identical (K-015 closed)
- qc/controls: skip bead addresses absent from the IDAT; control missing only if none is on the chip
- QC-008 amended: signal / zero background = +Inf (passes), 0/0 = NaN (flagged); Inf in TSV/report
- plotly bundle stored gzip-compressed (repo grep stays clean: the DOM event name contains the
  forbidden substring)
- docs: validation P5 part 2, summaries, known_unknowns K-021
2026-09-28 10:25:32 +01:00
kemal a4c9c2d7d8 P5a: QC — control metrics, thresholds, status, contamination, relatedness, locus summary
- qc/controls: canonical control names (QC-002), bkg/offset (QC-003), 13 genotyping metrics (QC-004/005),
  missing/zero-divisor handling (QC-007/008); controls_raw.tsv / controls_qc.tsv (QC-009/011) incl. compat CSVs
- qc/thresholds + profiles: YAML schema, validation codes, precedence, shipped profiles, auto (QC-020..025)
- qc/status: functional PASS/FAIL/NA, control CLEAR/FLAGGED, flags (QC-030..033)
- qc/contamination: BAF regression; bit-compatible with gtc2vcf BAFregress (36/36, K-017), exact mode too
- qc/relatedness: KING-robust, duplicates, Mendelian errors, pedigree from sheet (IN-018)
- qc/locus_summary: OUT-072, bit-identical to BeadArrayFiles except ChiTest (K-018)
- qc/suggest: suggest-thresholds (QC-026)
- pipeline/qc: disk-backed cohort store; CLI qc controls|samples|contamination|relatedness|loci|suggest-thresholds
- genotype: control tables (OUT-070), het_rate, QC columns, cohort QC
- tests: unit + oracle (BAFregress plugin, BeadArrayFiles locus summary)
2026-09-28 08:35:10 +01:00
kemal f469c97917 P4 validated on 2141 vendor GTCs; legacy AutoCall 1.6.2 preset
- vendor comparison (1000 Genomes Omni2.5, AutoCall 1.6.2.2): every
  computed field identical in 2138/2141 samples; the 3 others trace back
  to the single-locus raw-intensity rule (K-009); sex rule 1 matches all
- AutoCall 1.6.2.x leaves GC10/GC50/#calls/#no-calls at 0 in GTC v3:
  new `--preset autocall-1.6.2` (1.1.2 normalization, sex rule 1, GTC v3,
  legacy statistics); the sample summary keeps the real statistics
- NaN sign canonicalised to the x86 pattern (K-014); report P4.md
2026-09-28 01:21:49 +01:00
kemal f7d883826d P3 validated: normalization bit-identical on 2141 vendor GTCs
Normalization 1.1.2 reproduces all 109,191 bins of the 1000 Genomes
Omni2.5 GTCs (AutoCall 1.6.2.2) with 0 ULP difference, reserved fields
included; 1.2.0 and 1.1.2 are bit-identical to the no-FMA gtc2vcf oracle
on 36 GSA samples. AWS run: 33 min, ~$0.15, no resources left.
2026-09-27 23:41:43 +01:00
kemal c8cd2f741c P2 validated on 2141 vendor GTCs (1000 Genomes Omni2.5)
Raw X/Y and controls assembled from IDATs equal the vendor GTC arrays for
2138/2141 samples x 2.45M loci; 3 single-locus differences are recorded
as K-009 (probable per-channel missing rule in AutoCall 1.6.2.2). Channel
order X=Red confirmed (U1). `simurg ingest` processed all 2141 samples in
19 min (I/O-bound). AWS run cost ~$0.34 and left no resources.
2026-09-27 22:19:28 +01:00
kemal 705138d70b P0: project setup (packaging, CI, licences, data/oracle scripts)
- pyproject (uv/hatch, ruff, mypy strict, pytest markers), MIT LICENSE,
  THIRD_PARTY_NOTICES with verbatim BSD-2 (BeadArrayFiles) and MIT (gtc2vcf)
- CI workflow for Gitea/GitHub Actions: lint, type check, unit tests
- fetch_data.sh p0 now fetches the GSA v4 PGx demo IDATs (the GDA "Demo 36"
  bundle is a GenomeStudio project), unpacks archives, fixes pipefail bug
- make_oracle_gtcs.sh builds bcftools + gtc2vcf and writes oracle GTCs
- plan §17: corrected P0 dataset table
2026-09-27 16:56:39 +01:00
kemal 336a6b8226 Add functional specification (416 items) and link it from plan/docs
SIMURGARRAY_FUNCTIONAL_SPEC.md defines externally visible behaviour:
CLI, inputs, genotyping outputs, QC and report, cytogenetics CNV/LOH,
targeted PGx copy number, star alleles, methylation control QC, edge-case
catalogue, performance targets, open questions and a traceability table.
2026-09-27 16:13:40 +01:00
kemal 631c1d030d SimurgArray: master plan, repo scaffolding and data/ref fetch scripts
Phase 0 deliverables: full technical specification and roadmap
(SIMURGARRAY_MASTER_PLAN.md), CLAUDE.md, bootstrap script for reference
open-source repos and tiered dataset fetch script with checksums.
2026-09-27 15:20:06 +01:00