The Nextflow DSL2 pipeline

Three processes, each a container. The same workflow file runs on a laptop, on Google Batch and under Argo Workflows — the executor is a profile, not a rewrite.

NORMALISE

bcftools norm: left-align and split multi-allelics, then set the VCF ID to CHROM_POS_REF_ALT.

bcftools 1.20

→

VEP

Ensembl VEP 113. Cache mode for full annotation, or --database against Ensembl's public server when the 25 GB cache is not available. CADD and AlphaMissense are optional plugins.

4 cpus, 8 GB

→

LOAD_DB

Parses VEP --tab output, inserts variants idempotently (a retried task cannot duplicate), and marks the job succeeded.

psycopg, chunked COPY

Why the ID carries identity

VEP's own Location and Allele columns trim indel alleles and shift positions, so a deletion round-trips as the wrong variant. Carrying CHROM_POS_REF_ALT through the ID field keeps the exact alleles the caller emitted. Verified on a real run: 22:42126611 CT>C survives intact.

Stub blocks keep CI honest

Every process has a stub: block, so CI runs nextflow run main.nf -stub-run on a three-record fixture and checks wiring and channel shapes on every pull request — no containers, no VEP cache, seconds not hours.