feat: redesign around phenotype-driven triage, not variant filtering

A table with filters made the user do the work. Rare disease triage is a different task:
which few variants could explain *this* patient's phenotype, and why. The app now answers
that, and lets a reviewer act on the answer.

Domain
- a case is a proband: a VCF plus the HPO terms observed in the patient (samples -> cases)
- HPO's gene-to-phenotype annotations are loaded as reference data (scripts/load-hpo.py)
- each candidate can be shortlisted or dismissed with a reason and a note

Ranking (app/services/triage.py, 21 tests)
- weighted sum of phenotype match, rarity, consequence severity and the model's score,
  with every component shown next to the candidate
- rarity and consequence filter; phenotype only ranks, because a real diagnosis can sit in
  a gene nobody has annotated yet and filtering on it would hide exactly that case
- ClinVar is deliberately not an input: it appears beside the result as independent
  confirmation, so nothing ranks highly merely because ClinVar already said pathogenic

UI
- the funnel is the headline: variants called -> rare -> coding candidates -> phenotype-matched
- ranked candidates with evidence chips, not a grid of everything; filters are demoted
- a variant panel showing the score breakdown, the matched HPO terms, the raw VEP record and
  links out to Ensembl/gnomAD/ClinVar, with the decision controls
- a printable case report: phenotype, funnel, shortlisted variants with reasons, provenance

API: /cases with phenotypes, /cases/{id}/candidates (funnel + ranked + weights),
/variants/{id}, /variants/{id}/decision, /cases/{id}/report, /phenotypes for the picker.
Scoring moved under the case and now answers 503 with the reason when no model registry is
reachable, instead of a 500.

Verified end to end on a simulated proband (scripts/make-demo-case.sh: real GIAB HG002
background + one real ClinVar 2-star pathogenic NF2 variant). 13 variants called -> 1 coding
candidate, and the planted variant ranks first at 0.80 on phenotype 1.00, rarity 1.00 and
consequence 1.00, with ClinVar agreeing afterwards.

Tests: api 75, ml 18, loader 16, web 27; ruff, mypy, svelte-check, terraform validate, both
kustomize overlays and the Nextflow stub run all clean.
This commit is contained in:
Kemal Yaylali
2026-09-12 08:30:44 +01:00
parent abde5ec6e4
commit 07a01715fd
47 changed files with 2159 additions and 539 deletions
+11 -4
View File
@@ -1,8 +1,11 @@
# rarelens
A small, end-to-end variant interpretation platform for rare genetic disease research.
Scientists upload a VCF, a Nextflow workflow annotates it with Ensembl VEP, a machine
learning model scores each variant, and results are browsable in a web app.
A case is a proband: a VCF plus the patient's phenotype (HPO terms). A Nextflow workflow
annotates the variants with Ensembl VEP, a model scores each one, and the app narrows
thousands of variants to a handful of candidates ranked against that phenotype — each
carrying the evidence for its rank, and each able to be shortlisted or dismissed with a
reason that ends up in a case report.
This repository is a **self-training lab**. It exists so that one engineer can learn, in
public, how a modern life-sciences platform is built end to end: full-stack application,
@@ -28,11 +31,15 @@ clinical tool and makes no diagnostic claims.
```bash
make up # postgres + api + web + mlflow via docker-compose
make migrate # alembic upgrade head
make data # real public data: GIAB HG002 + ClinVar, chr22 (needs bcftools)
make hpo # HPO gene-to-phenotype annotations: what the ranking matches against
make demo-case # a simulated proband: GIAB background + one ClinVar pathogenic variant
make test # api, ml, loader and web tests (no Docker needed for the DB tests)
```
Then open http://localhost:5173.
Then open http://localhost:5173, create a case pointing at `data/proband-simulated.vcf.gz`,
give it the phenotype of the planted disease (for the default NF2 case: bilateral vestibular
schwannoma, sensorineural hearing impairment, tinnitus, meningioma, cataract), and analyse it.
The planted variant should come back ranked first.
The docker-compose API has no Nextflow, so "Run VEP annotation" marks the job failed with the
command to run instead. With Nextflow and Docker on the host, a VEP cache in `pipeline/cache/vep`