P9 cloud task: install gcc (linker for the Rust build); guide: kernel installation

This commit is contained in:
kemal committed 2026-09-28 21:35:42 +01:00
1 parent 587fadbaef
commit f0d3739493
2 files changed
+13 -2

No files matched your search

+12 -2
View File
@@ -19,6 +19,15 @@ uv venv && uv pip install -e . # add ".[dev]" for tests and linters
.venv/bin/simurg --version .venv/bin/simurg --version
``` ```
**Optional compiled kernels** (≈ 3–4× faster genotyping, 4× faster manifest loading; results are byte-identical):
```bash
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal # Rust (needs a C linker, e.g. gcc)
uv pip install ./rust/simurg-core
.venv/bin/python -c "from simurg._core import backend; print(backend())" # rust 1.0.0
```
Without them the same code runs in pure Python/NumPy; `SIMURG_PURE_PYTHON=1` forces that path.
Only pure-Python dependencies are needed (numpy, pyarrow, typer, rich, pyyaml, openpyxl). VCF/BCF, tabix/CSI Only pure-Python dependencies are needed (numpy, pyarrow, typer, rich, pyyaml, openpyxl). VCF/BCF, tabix/CSI
indexes and PLINK files are written by SimurgArray itself; bcftools/plink2 are not required (they are used only indexes and PLINK files are written by SimurgArray itself; bcftools/plink2 are not required (they are used only
as test oracles). as test oracles).
@@ -148,8 +157,9 @@ A nightly differential run (`scripts/nightly.py`) re-checks these against the or
diplotyping are not in v1.0. diplotyping are not in v1.0.
* The largest cohort tested is 2141 samples (plan goal ≥ 10 000). Memory grows with samples × loci in the cohort * The largest cohort tested is 2141 samples (plan goal ≥ 10 000). Memory grows with samples × loci in the cohort
QC steps; on an 8 GB machine keep `--threads` low (≈ 2–3 GB per worker for GSA/Omni-size arrays). QC steps; on an 8 GB machine keep `--threads` low (≈ 2–3 GB per worker for GSA/Omni-size arrays).
* Speed is that of NumPy: genotyping a GSA sample takes ~3 CPU-seconds on a server core (360 samples in 2 min on * Speed: with the compiled kernels a GSA sample takes ~1.1 s on a Raspberry Pi 5 core (pure Python ~3.4 s); see
16 cores) and ~25 s on a Raspberry Pi 5 core; a compiled core is planned (P9). `docs/validation/P9.md` for server numbers and thread scaling. Loading a manifest/cluster file is a fixed cost
per run (GSA ~5 s, Omni2.5 ~15 s on the Pi).
* Methylation arrays: control-probe QC only. * Methylation arrays: control-probe QC only.
## Troubleshooting ## Troubleshooting
+1
View File
@@ -10,6 +10,7 @@ W=/work/p9; R=$OUT/p9; mkdir -p $W $R
N=$(nproc) N=$(nproc)
echo "=== build simurg-core $(date -u +%T) ===" echo "=== build simurg-core $(date -u +%T) ==="
DEBIAN_FRONTEND=noninteractive apt-get install -y -q gcc > /dev/null # Rust links through the C toolchain
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal > /dev/null 2>&1 curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal > /dev/null 2>&1
export PATH="$HOME/.cargo/bin:$PATH" export PATH="$HOME/.cargo/bin:$PATH"
rustc --version rustc --version