feat: redesign around phenotype-driven triage, not variant filtering

A table with filters made the user do the work. Rare disease triage is a different task:
which few variants could explain *this* patient's phenotype, and why. The app now answers
that, and lets a reviewer act on the answer.

Domain
- a case is a proband: a VCF plus the HPO terms observed in the patient (samples -> cases)
- HPO's gene-to-phenotype annotations are loaded as reference data (scripts/load-hpo.py)
- each candidate can be shortlisted or dismissed with a reason and a note

Ranking (app/services/triage.py, 21 tests)
- weighted sum of phenotype match, rarity, consequence severity and the model's score,
  with every component shown next to the candidate
- rarity and consequence filter; phenotype only ranks, because a real diagnosis can sit in
  a gene nobody has annotated yet and filtering on it would hide exactly that case
- ClinVar is deliberately not an input: it appears beside the result as independent
  confirmation, so nothing ranks highly merely because ClinVar already said pathogenic

UI
- the funnel is the headline: variants called -> rare -> coding candidates -> phenotype-matched
- ranked candidates with evidence chips, not a grid of everything; filters are demoted
- a variant panel showing the score breakdown, the matched HPO terms, the raw VEP record and
  links out to Ensembl/gnomAD/ClinVar, with the decision controls
- a printable case report: phenotype, funnel, shortlisted variants with reasons, provenance

API: /cases with phenotypes, /cases/{id}/candidates (funnel + ranked + weights),
/variants/{id}, /variants/{id}/decision, /cases/{id}/report, /phenotypes for the picker.
Scoring moved under the case and now answers 503 with the reason when no model registry is
reachable, instead of a 500.

Verified end to end on a simulated proband (scripts/make-demo-case.sh: real GIAB HG002
background + one real ClinVar 2-star pathogenic NF2 variant). 13 variants called -> 1 coding
candidate, and the planted variant ranks first at 0.80 on phenotype 1.00, rarity 1.00 and
consequence 1.00, with ClinVar agreeing afterwards.

Tests: api 75, ml 18, loader 16, web 27; ruff, mypy, svelte-check, terraform validate, both
kustomize overlays and the Nextflow stub run all clean.
This commit is contained in:
Kemal Yaylali
2026-09-12 08:30:44 +01:00
parent abde5ec6e4
commit 07a01715fd
47 changed files with 2159 additions and 539 deletions
+30 -1
View File
@@ -2,8 +2,9 @@
```mermaid
flowchart LR
U[Scientist] -->|browser| W[SvelteKit web]
U[Scientist] -->|phenotype + VCF| W[SvelteKit web]
W -->|REST /api| A[FastAPI]
H[(HPO gene-phenotype annotations)] --> A
A --> P[(PostgreSQL / Cloud SQL)]
A -->|publish vcf-uploaded| Q[Pub/Sub]
Q --> E[Argo Events sensor]
@@ -12,6 +13,7 @@ flowchart LR
B -->|reads VCF, VEP cache| G[(GCS bucket)]
B -->|writes variants, marks job succeeded| P
AW -.->|exit handler marks job failed| P
A -->|rank: phenotype, rarity, consequence, model| C[Ranked candidates -> decisions -> report]
A -->|models:/rarelens-pathogenicity@production| M[MLflow registry]
T[ml/train.py] --> M
GH[GitHub Actions] -->|images via WIF| AR[Artifact Registry]
@@ -19,6 +21,33 @@ flowchart LR
R --> CD[ArgoCD] --> K[GKE Autopilot]
```
## The triage model
A **case** is a proband: a VCF plus the HPO terms observed in that patient. Annotation produces
variants; the model scores them; ranking then answers the only question that matters — which few
variants could explain *this* phenotype.
Rarity (<0.1% in gnomAD) and consequence (HIGH or MODERATE) *filter*, which is the usual first
pass. Phenotype only *ranks*: a real diagnosis can sit in a gene nobody has annotated yet, and
filtering on phenotype would hide exactly that case. The rank is a weighted sum whose parts are
shown next to every candidate (`app/services/triage.py`):
| Component | Weight |
|---|---|
| phenotype terms of this patient annotated to the gene | 0.35 |
| rarity in gnomAD | 0.25 |
| consequence severity | 0.20 |
| model P(pathogenic) | 0.20 |
**ClinVar is deliberately not an input.** It sits beside the result as independent confirmation, so
the demo never ranks a variant highly merely because ClinVar already called it pathogenic. On the
simulated NF2 case the planted variant ranks first on phenotype, rarity and consequence alone, and
ClinVar agrees afterwards.
Each candidate can be shortlisted or dismissed with a reason and a note; the case report is that
decision trail plus the funnel counts and the provenance (VEP version, model version, run time).
There is no authentication, so decisions are shared by everyone who opens the demo.
## Two deployment tracks
The same images and the same pipeline, deployed two ways (`infra/terraform/variables.tf`):