feat(pipeline): VEP database mode, and a pipeline-specific database URL
Makes a real annotation runnable locally without the 25 GB VEP cache, which is what the demo needs and what a reviewer can reproduce in minutes. - params.vep_database (VEP_DATABASE=true) queries Ensembl's public database instead of a local cache. Slower per variant and fewer fields, so --everything is swapped for the flags the loader actually stores. Its cache placeholder is NO_CACHE, not NO_FILE: Nextflow rejects two staged inputs sharing a filename. - PIPELINE_DATABASE_URL is handed to the pipeline when set. The loader runs inside a container, where the API's own localhost URL would point at the container itself. - README: how to run the UI's annotate button locally against host Nextflow + Docker. Verified end to end on pipeline/tests/data/tiny.vcf: bcftools norm split the multiallelic record, VEP 113 annotated 4 variants live, the loader wrote them and marked the job succeeded, and the UI shows them. The deletion came back as 22:42126611 CT>C with exact VCF alleles, which is the case the audit's ID-tagging fix exists for. Tests: api 51, loader 16, stub run 3/3; ruff, mypy clean.
This commit is contained in:
@@ -43,6 +43,28 @@ make annotate JOB=<job id from the UI> VCF=data/example.vcf.gz
|
||||
make pipeline VCF=data/example.vcf.gz # dry run: annotate without touching the database
|
||||
```
|
||||
|
||||
No cache? `VEP_DATABASE=true` queries Ensembl's public database instead. It is slow per variant
|
||||
and returns fewer fields, but it needs no 25 GB download, which is enough to demonstrate the
|
||||
pipeline on a handful of variants:
|
||||
|
||||
```bash
|
||||
VEP_DATABASE=true make pipeline VCF=pipeline/tests/data/tiny.vcf
|
||||
```
|
||||
|
||||
To make the UI's "Run VEP annotation" button work, run the API on the host (where Nextflow and
|
||||
Docker are) rather than in docker-compose:
|
||||
|
||||
```bash
|
||||
docker compose up -d db
|
||||
cd api && DATABASE_URL=postgresql+asyncpg://rarelens:rarelens@localhost:5432/rarelens \
|
||||
PIPELINE_DATABASE_URL=postgresql+asyncpg://rarelens:[email protected]:5432/rarelens \
|
||||
LOCAL_DATA_ROOT=$PWD/.. VEP_DATABASE=true \
|
||||
uv run --extra dev uvicorn app.main:app --port 8000
|
||||
```
|
||||
|
||||
`PIPELINE_DATABASE_URL` is what the loader container gets: inside it, the API's own `localhost`
|
||||
would be the container itself. `LOCAL_DATA_ROOT` is the directory a sample's `vcf_uri` must sit under.
|
||||
|
||||
To train and register a model (the API scores with `models:/rarelens-pathogenicity@production`):
|
||||
|
||||
```bash
|
||||
|
||||
Reference in New Issue
Block a user