"Variants are unscored" was accurate: nothing was ever trained, so a quarter of every rank was
dead weight and the UI leaked a connection error at the reader.
- scripts/make-training-set.sh derives a training table from ClinVar directly. ClinVar already
carries the molecular consequence, the gene and an allele frequency, which is the feature set
serving sends, so this avoids running VEP over hundreds of thousands of variants. 2-star
records only.
- train.py now holds out whole genes (GroupShuffleSplit). docs/data.md had said to do this since
the data pass; the code was still doing a random split, which is the leak Grimm 2015 describes.
- evaluate() reports missense on its own. On the last run: AUROC 0.986 over 74,239 held-out
variants, but 0.872 over the 13,553 missense ones, and the docs say plainly why even that is
flattered — within missense the only live feature is allele frequency, and ClinVar's benign
calls often use allele frequency as evidence (ACMG BA1/BS1), so the feature partly caused the
label.
- the 503 now names what is missing (model@alias via tracking URI) and leaves the exception in
the server log instead of the UI.
- make training-set / make train; the 58 MB table is gitignored.
Verified end to end: model registered as v2, the simulated NF2 case scores 0.999 on the planted
variant, and it now ranks 1.00 with all four components live.
Tests: api 77, ml 22, loader 16, web 32; ruff, mypy, svelte-check clean.
From clicking through the redesigned UI:
- scoring a case without a model registry painted a red failure across a case that had in
fact analysed fine. It is now a quiet note saying the model term contributes 0, because
scoring is an optional fourth of the rank, not the analysis.
- the MLflow default moves to port 5001. On macOS, AirPlay Receiver owns 5000, which is why
the registry answered "403" rather than refusing the connection; docker-compose publishes
5001 to match.
- a funnel step that kept nothing drew a visible bar. Zero now draws zero.
- "1 candidates".
- the funnel's fixed grid columns forced a horizontal scrollbar on the report.
The report also lists the top undecided candidates now: the first thing anyone opens has no
decisions in it, and "Shortlisted (0)" alone said nothing about what the tool found.
Tests: api 77, web 32; ruff, mypy, svelte-check clean.
A table with filters made the user do the work. Rare disease triage is a different task:
which few variants could explain *this* patient's phenotype, and why. The app now answers
that, and lets a reviewer act on the answer.
Domain
- a case is a proband: a VCF plus the HPO terms observed in the patient (samples -> cases)
- HPO's gene-to-phenotype annotations are loaded as reference data (scripts/load-hpo.py)
- each candidate can be shortlisted or dismissed with a reason and a note
Ranking (app/services/triage.py, 21 tests)
- weighted sum of phenotype match, rarity, consequence severity and the model's score,
with every component shown next to the candidate
- rarity and consequence filter; phenotype only ranks, because a real diagnosis can sit in
a gene nobody has annotated yet and filtering on it would hide exactly that case
- ClinVar is deliberately not an input: it appears beside the result as independent
confirmation, so nothing ranks highly merely because ClinVar already said pathogenic
UI
- the funnel is the headline: variants called -> rare -> coding candidates -> phenotype-matched
- ranked candidates with evidence chips, not a grid of everything; filters are demoted
- a variant panel showing the score breakdown, the matched HPO terms, the raw VEP record and
links out to Ensembl/gnomAD/ClinVar, with the decision controls
- a printable case report: phenotype, funnel, shortlisted variants with reasons, provenance
API: /cases with phenotypes, /cases/{id}/candidates (funnel + ranked + weights),
/variants/{id}, /variants/{id}/decision, /cases/{id}/report, /phenotypes for the picker.
Scoring moved under the case and now answers 503 with the reason when no model registry is
reachable, instead of a 500.
Verified end to end on a simulated proband (scripts/make-demo-case.sh: real GIAB HG002
background + one real ClinVar 2-star pathogenic NF2 variant). 13 variants called -> 1 coding
candidate, and the planted variant ranks first at 0.80 on phenotype 1.00, rarity 1.00 and
consequence 1.00, with ClinVar agreeing afterwards.
Tests: api 75, ml 18, loader 16, web 27; ruff, mypy, svelte-check, terraform validate, both
kustomize overlays and the Nextflow stub run all clean.
Kemal Yaylali