feat: redesign around phenotype-driven triage, not variant filtering

A table with filters made the user do the work. Rare disease triage is a different task:
which few variants could explain *this* patient's phenotype, and why. The app now answers
that, and lets a reviewer act on the answer.

Domain
- a case is a proband: a VCF plus the HPO terms observed in the patient (samples -> cases)
- HPO's gene-to-phenotype annotations are loaded as reference data (scripts/load-hpo.py)
- each candidate can be shortlisted or dismissed with a reason and a note

Ranking (app/services/triage.py, 21 tests)
- weighted sum of phenotype match, rarity, consequence severity and the model's score,
  with every component shown next to the candidate
- rarity and consequence filter; phenotype only ranks, because a real diagnosis can sit in
  a gene nobody has annotated yet and filtering on it would hide exactly that case
- ClinVar is deliberately not an input: it appears beside the result as independent
  confirmation, so nothing ranks highly merely because ClinVar already said pathogenic

UI
- the funnel is the headline: variants called -> rare -> coding candidates -> phenotype-matched
- ranked candidates with evidence chips, not a grid of everything; filters are demoted
- a variant panel showing the score breakdown, the matched HPO terms, the raw VEP record and
  links out to Ensembl/gnomAD/ClinVar, with the decision controls
- a printable case report: phenotype, funnel, shortlisted variants with reasons, provenance

API: /cases with phenotypes, /cases/{id}/candidates (funnel + ranked + weights),
/variants/{id}, /variants/{id}/decision, /cases/{id}/report, /phenotypes for the picker.
Scoring moved under the case and now answers 503 with the reason when no model registry is
reachable, instead of a 500.

Verified end to end on a simulated proband (scripts/make-demo-case.sh: real GIAB HG002
background + one real ClinVar 2-star pathogenic NF2 variant). 13 variants called -> 1 coding
candidate, and the planted variant ranks first at 0.80 on phenotype 1.00, rarity 1.00 and
consequence 1.00, with ClinVar agreeing afterwards.

Tests: api 75, ml 18, loader 16, web 27; ruff, mypy, svelte-check, terraform validate, both
kustomize overlays and the Nextflow stub run all clean.
This commit is contained in:
Kemal Yaylali
2026-09-12 08:30:44 +01:00
parent abde5ec6e4
commit 07a01715fd
47 changed files with 2159 additions and 539 deletions
+19
View File
@@ -28,6 +28,7 @@ benchmarks variant callers against, so it is both realistic and unambiguously sh
| **gnomAD** v4 | allele frequency feature and filter | `gs://gcp-public-data--gnomad`, `s3://gnomad-public-us-east-1` | free use, no restriction | Chen et al., *Nature* 625:92100, 2024. [10.1038/s41586-023-06045-0](https://doi.org/10.1038/s41586-023-06045-0); Karczewski et al., *Nature* 581:434443, 2020. [10.1038/s41586-020-2308-7](https://doi.org/10.1038/s41586-020-2308-7) |
| **1000 Genomes** 30x | optional cohort/trio data | EBI FTP, `s3://1000genomes` | fully open, no access restriction | Byrska-Bishop et al., *Cell* 185(18):34263440.e19, 2022. [10.1016/j.cell.2022.08.004](https://doi.org/10.1016/j.cell.2022.08.004) |
| **MANE Select** | one transcript per gene, if transcript choice ever matters | Ensembl/RefSeq | open | Morales et al., *Nature* 604:310315, 2022. [10.1038/s41586-022-04558-8](https://doi.org/10.1038/s41586-022-04558-8) |
| **Human Phenotype Ontology** gene-to-phenotype | what the phenotype half of the ranking matches against (`make hpo`) | `purl.obolibrary.org/obo/hp/hpoa/genes_to_phenotype.txt` | free to use with attribution | Gargano et al., *Nucleic Acids Res* 52(D1):D1333D1346, 2024. [10.1093/nar/gkad1005](https://doi.org/10.1093/nar/gkad1005) |
## Tools and scores
@@ -40,6 +41,24 @@ benchmarks variant callers against, so it is both realistic and unambiguously sh
Neither score is required: `rarelens_ml.features` treats a missing CADD or AlphaMissense value as
NaN and LightGBM handles it, so the pipeline runs without the plugin data.
## The simulated proband
`make demo-case` builds `data/proband-simulated.vcf.gz`: real GIAB HG002 variants as background
plus one real ClinVar 2-star pathogenic variant in a disease gene (NF2 by default, giving
neurofibromatosis type 2). Spiking a known variant into a public genome is how phenotype-driven
triage tools are benchmarked, every input is public, and the file's header says SIMULATED. It is
not a patient, and no part of it is invented: both the background and the planted variant are real
published records.
The point of it is that the case has a right answer, so the ranking can be checked rather than
admired. Give the case the phenotype of the planted disease and the planted variant should rank
first — on phenotype, rarity and consequence, with ClinVar agreeing only afterwards.
**Caveat when running without a VEP cache.** `VEP_DATABASE=true` queries Ensembl's public database
instead of the 25 GB cache. It returns no gnomAD frequencies, so every variant looks absent from
gnomAD and the rarity term stops discriminating. Fine for showing the mechanics; use the cache for
anything you would quote.
## Evaluating the model honestly
The model trains on ClinVar labels and is scored on ClinVar-labelled variants, which is exactly