From bd5d7c0e1082d193f7ed921926f74c87d2101427 Mon Sep 17 00:00:00 2001 From: Kemal Yaylali Date: Mon, 28 Sep 2026 21:35:42 +0100 Subject: [PATCH] P9 cloud task: install gcc (linker for the Rust build); guide: kernel installation --- docs/user_guide.md | 14 ++++++++++++-- scripts/cloud/tasks/p9.sh | 1 + 2 files changed, 13 insertions(+), 2 deletions(-) diff --git a/docs/user_guide.md b/docs/user_guide.md index 3a9d3e9..ef76d02 100644 --- a/docs/user_guide.md +++ b/docs/user_guide.md @@ -19,6 +19,15 @@ uv venv && uv pip install -e . # add ".[dev]" for tests and linters .venv/bin/simurg --version ``` +**Optional compiled kernels** (≈ 3–4× faster genotyping, 4× faster manifest loading; results are byte-identical): + +```bash +curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal # Rust (needs a C linker, e.g. gcc) +uv pip install ./rust/simurg-core +.venv/bin/python -c "from simurg._core import backend; print(backend())" # rust 1.0.0 +``` + +Without them the same code runs in pure Python/NumPy; `SIMURG_PURE_PYTHON=1` forces that path. Only pure-Python dependencies are needed (numpy, pyarrow, typer, rich, pyyaml, openpyxl). VCF/BCF, tabix/CSI indexes and PLINK files are written by SimurgArray itself; bcftools/plink2 are not required (they are used only as test oracles). @@ -148,8 +157,9 @@ A nightly differential run (`scripts/nightly.py`) re-checks these against the or diplotyping are not in v1.0. * The largest cohort tested is 2141 samples (plan goal ≥ 10 000). Memory grows with samples × loci in the cohort QC steps; on an 8 GB machine keep `--threads` low (≈ 2–3 GB per worker for GSA/Omni-size arrays). -* Speed is that of NumPy: genotyping a GSA sample takes ~3 CPU-seconds on a server core (360 samples in 2 min on - 16 cores) and ~25 s on a Raspberry Pi 5 core; a compiled core is planned (P9). +* Speed: with the compiled kernels a GSA sample takes ~1.1 s on a Raspberry Pi 5 core (pure Python ~3.4 s); see + `docs/validation/P9.md` for server numbers and thread scaling. Loading a manifest/cluster file is a fixed cost + per run (GSA ~5 s, Omni2.5 ~15 s on the Pi). * Methylation arrays: control-probe QC only. ## Troubleshooting diff --git a/scripts/cloud/tasks/p9.sh b/scripts/cloud/tasks/p9.sh index d0f2790..eaa8279 100644 --- a/scripts/cloud/tasks/p9.sh +++ b/scripts/cloud/tasks/p9.sh @@ -10,6 +10,7 @@ W=/work/p9; R=$OUT/p9; mkdir -p $W $R N=$(nproc) echo "=== build simurg-core $(date -u +%T) ===" +DEBIAN_FRONTEND=noninteractive apt-get install -y -q gcc > /dev/null # Rust links through the C toolchain curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal > /dev/null 2>&1 export PATH="$HOME/.cargo/bin:$PATH" rustc --version