Commit Graph
8 Commits
Author SHA1 Message Date
kemal b012f74016 cloud trial: set HOME/PATH for cloud-init, unit tests non-fatal
The first run stopped after 1 minute (HOME unbound under set -u) and
terminated itself as designed.
2026-09-27 19:36:49 +01:00
kemal e786b6b64b Lint fixes in validate_p1_large.py 2026-09-27 19:30:37 +01:00
kemal 6bea6c4930 Add self-destructing AWS trial and large-scale P1 validation script
scripts/cloud/trial.sh launches one spot instance that validates the
parsers on the 1000 Genomes Omni2.5 set (2141 samples, vendor GTCs)
streamed from the public S3 bucket, uploads the summary and terminates
itself. Cost guards: spot price cap, hard time limit (shutdown =
terminate), EXIT-trap shutdown, CloudWatch idle alarm that terminates
from outside, disk deleted with the instance, monthly budget alert, and
an `audit` command listing anything still billable.

scripts/validate_p1_large.py compares every GTC with BeadArrayFiles,
checks byte-identical round-trips, IDAT pairs and BPM/EGT/CSV.
2026-09-27 19:30:27 +01:00
kemal e518ea03d5 P1: file parsers, sample sheet, discovery, CLI inspect
Readers for IDAT v3, BPM (locus entries v4-v8), CSV manifest, EGT
(data blocks 5-9) and GTC v3-v5 (plus a layout-preserving GTC writer);
sample-sheet validation (IN-001..IN-014), IDAT discovery and pairing
(IN-020..IN-022), message catalogue and exit codes (CLI-014/016),
JSON Lines logging (CLI-007/008), atomic writes (CLI-011), run manifest,
and `simurg inspect`.

Validation (docs/validation/P1.md): GTC, BPM and EGT readers agree
exactly with Illumina BeadArrayFiles on GSA v4 / GDA v1 files; GTCs
round-trip byte-identically; BPM and CSV manifests are consistent.
2026-09-27 16:56:39 +01:00
kemal 705138d70b P0: project setup (packaging, CI, licences, data/oracle scripts)
- pyproject (uv/hatch, ruff, mypy strict, pytest markers), MIT LICENSE,
  THIRD_PARTY_NOTICES with verbatim BSD-2 (BeadArrayFiles) and MIT (gtc2vcf)
- CI workflow for Gitea/GitHub Actions: lint, type check, unit tests
- fetch_data.sh p0 now fetches the GSA v4 PGx demo IDATs (the GDA "Demo 36"
  bundle is a GenomeStudio project), unpacks archives, fixes pipefail bug
- make_oracle_gtcs.sh builds bcftools + gtc2vcf and writes oracle GTCs
- plan §17: corrected P0 dataset table
2026-09-27 16:56:39 +01:00
kemal 854cf95029 Add sanitized Phase-0 research archive under docs/research
Research notes R1-R4, documentation-study files A-E (source of the
functional spec's `src:` tags) and PGx coverage CSVs, rewritten to use
neutral naming and current project decisions (plan §2).
2026-09-27 16:27:34 +01:00
kemal 336a6b8226 Add functional specification (416 items) and link it from plan/docs
SIMURGARRAY_FUNCTIONAL_SPEC.md defines externally visible behaviour:
CLI, inputs, genotyping outputs, QC and report, cytogenetics CNV/LOH,
targeted PGx copy number, star alleles, methylation control QC, edge-case
catalogue, performance targets, open questions and a traceability table.
2026-09-27 16:13:40 +01:00
kemal 631c1d030d SimurgArray: master plan, repo scaffolding and data/ref fetch scripts
Phase 0 deliverables: full technical specification and roadmap
(SIMURGARRAY_MASTER_PLAN.md), CLAUDE.md, bootstrap script for reference
open-source repos and tiered dataset fetch script with checksums.
2026-09-27 15:20:06 +01:00