Files

T

Kemal Yaylali 07a01715fd feat: redesign around phenotype-driven triage, not variant filtering

A table with filters made the user do the work. Rare disease triage is a different task:
which few variants could explain *this* patient's phenotype, and why. The app now answers
that, and lets a reviewer act on the answer.

Domain
- a case is a proband: a VCF plus the HPO terms observed in the patient (samples -> cases)
- HPO's gene-to-phenotype annotations are loaded as reference data (scripts/load-hpo.py)
- each candidate can be shortlisted or dismissed with a reason and a note

Ranking (app/services/triage.py, 21 tests)
- weighted sum of phenotype match, rarity, consequence severity and the model's score,
  with every component shown next to the candidate
- rarity and consequence filter; phenotype only ranks, because a real diagnosis can sit in
  a gene nobody has annotated yet and filtering on it would hide exactly that case
- ClinVar is deliberately not an input: it appears beside the result as independent
  confirmation, so nothing ranks highly merely because ClinVar already said pathogenic

UI
- the funnel is the headline: variants called -> rare -> coding candidates -> phenotype-matched
- ranked candidates with evidence chips, not a grid of everything; filters are demoted
- a variant panel showing the score breakdown, the matched HPO terms, the raw VEP record and
  links out to Ensembl/gnomAD/ClinVar, with the decision controls
- a printable case report: phenotype, funnel, shortlisted variants with reasons, provenance

API: /cases with phenotypes, /cases/{id}/candidates (funnel + ranked + weights),
/variants/{id}, /variants/{id}/decision, /cases/{id}/report, /phenotypes for the picker.
Scoring moved under the case and now answers 503 with the reason when no model registry is
reachable, instead of a 500.

Verified end to end on a simulated proband (scripts/make-demo-case.sh: real GIAB HG002
background + one real ClinVar 2-star pathogenic NF2 variant). 13 variants called -> 1 coding
candidate, and the planted variant ranks first at 0.80 on phenotype 1.00, rarity 1.00 and
consequence 1.00, with ClinVar agreeing afterwards.

Tests: api 75, ml 18, loader 16, web 27; ruff, mypy, svelte-check, terraform validate, both
kustomize overlays and the Nextflow stub run all clean.

2026-09-12 08:30:44 +01:00

7.2 KiB

Raw Blame History

Data: what rarelens actually runs on

Everything below is public, peer-reviewed and consented for open redistribution. No patient data, no data access agreement, nothing that needs an application. These are the references to quote when showing the platform to someone.

Citations were verified against PubMed; each row links its DOI.

The demo slice

make data fetches two real files, chromosome 22 only (roughly 100 MB, minutes rather than hours):

File	What it is	Role
`data/example.vcf.gz`	GIAB HG002 (NA24385) v4.2.1 benchmark calls, GRCh38, chr22	the sample a scientist annotates
`data/clinvar.chr22.vcf.gz`	ClinVar, GRCh38, chr22	training labels, and the ClinVar column in the UI

HG002 is the NIST Genome in a Bottle Ashkenazi son, recruited through the Personal Genome Project, which consents participants to unrestricted public release. It is the reference genome the field benchmarks variant callers against, so it is both realistic and unambiguously shareable.

Datasets

Dataset	Used for	Access	Terms	Citation
ClinVar (GRCh38)	pathogenic/benign labels, ClinVar column	`ftp.ncbi.nlm.nih.gov/pub/clinvar/vcf_GRCh38/`	NCBI public domain	Landrum et al., Nucleic Acids Res 48(D1):D835–D844, 2020. 10.1093/nar/gkz972
Genome in a Bottle HG002 v4.2.1	the demo sample	`ftp-trace.ncbi.nlm.nih.gov/ReferenceSamples/giab/`, `s3://giab`	open, no use restriction	Zook et al., Nat Biotechnol 37:561–566, 2019. 10.1038/s41587-019-0074-6
gnomAD v4	allele frequency feature and filter	`gs://gcp-public-data--gnomad`, `s3://gnomad-public-us-east-1`	free use, no restriction	Chen et al., Nature 625:92–100, 2024. 10.1038/s41586-023-06045-0; Karczewski et al., Nature 581:434–443, 2020. 10.1038/s41586-020-2308-7
1000 Genomes 30x	optional cohort/trio data	EBI FTP, `s3://1000genomes`	fully open, no access restriction	Byrska-Bishop et al., Cell 185(18):3426–3440.e19, 2022. 10.1016/j.cell.2022.08.004
MANE Select	one transcript per gene, if transcript choice ever matters	Ensembl/RefSeq	open	Morales et al., Nature 604:310–315, 2022. 10.1038/s41586-022-04558-8
Human Phenotype Ontology gene-to-phenotype	what the phenotype half of the ranking matches against (`make hpo`)	`purl.obolibrary.org/obo/hp/hpoa/genes_to_phenotype.txt`	free to use with attribution	Gargano et al., Nucleic Acids Res 52(D1):D1333–D1346, 2024. 10.1093/nar/gkad1005

Tools and scores

Tool	Role	Terms	Citation
Ensembl VEP 113	annotation (`pipeline/modules/vep.nf`)	Apache 2.0	McLaren et al., Genome Biol 17:122, 2016. 10.1186/s13059-016-0974-4
CADD	`cadd_phred` feature	free for non-commercial use; commercial licence required	Rentzsch et al., Nucleic Acids Res 47(D1):D886–D894, 2019. 10.1093/nar/gky1016; Schubach et al., Nucleic Acids Res 52(D1), 2024. 10.1093/nar/gkad989
AlphaMissense	`am_pathogenicity` feature	predictions moved to CC BY 4.0 in March 2024 (originally CC BY-NC-SA)	Cheng et al., Science 381:eadg7492, 2023. 10.1126/science.adg7492

Neither score is required: rarelens_ml.features treats a missing CADD or AlphaMissense value as NaN and LightGBM handles it, so the pipeline runs without the plugin data.

The simulated proband

make demo-case builds data/proband-simulated.vcf.gz: real GIAB HG002 variants as background plus one real ClinVar 2-star pathogenic variant in a disease gene (NF2 by default, giving neurofibromatosis type 2). Spiking a known variant into a public genome is how phenotype-driven triage tools are benchmarked, every input is public, and the file's header says SIMULATED. It is not a patient, and no part of it is invented: both the background and the planted variant are real published records.

The point of it is that the case has a right answer, so the ranking can be checked rather than admired. Give the case the phenotype of the planted disease and the planted variant should rank first — on phenotype, rarity and consequence, with ClinVar agreeing only afterwards.

Caveat when running without a VEP cache. VEP_DATABASE=true queries Ensembl's public database instead of the 25 GB cache. It returns no gnomAD frequencies, so every variant looks absent from gnomAD and the rarity term stops discriminating. Fine for showing the mechanics; use the cache for anything you would quote.

Evaluating the model honestly

The model trains on ClinVar labels and is scored on ClinVar-labelled variants, which is exactly where published benchmarks go wrong. What to do about it:

Never let the label into the features. CLIN_SIG is excluded by construction; clinvar_sig is stored for display only (rarelens_ml/features.py lists the five feature columns).
Split by gene, not by variant. Random splits put variants from the same gene on both sides, and a model can then score a gene rather than a variant. Grimm et al. showed this inflates reported accuracy for exactly this class of tool: Hum Mutat 36:513–523, 2015. 10.1002/humu.22768
Prefer a time-based holdout. Train on an older ClinVar release (monthly archives live under vcf_GRCh38/archive_2.0/) and test only on variants classified after that date. This is the closest thing to a prospective evaluation available without new patients.
Filter labels by review status. ClinVar's CLNREVSTAT marks how much evidence backs a classification; two-star and above ("multiple submitters, no conflicts") is the usual bar. Known gap: VEP's CLIN_SIG does not carry review status, so this needs ClinVar annotated as a custom field before it can be enforced.
Report against published baselines on the same rows. CADD PHRED and AlphaMissense are already columns in the variant table, so AUROC and AUPRC for the model next to those two, with the variant count, is a fair comparison rather than a number with nothing to beat.
Report AUPRC, not just AUROC. Pathogenic variants are the minority class; AUROC flatters.

What must not be claimed

ACMG/AMP treats computational predictions as supporting evidence only, never sufficient on their own for classifying a variant (Richards et al., Genet Med 17:405–424, 2015. 10.1038/gim.2015.30). rarelens is a learning platform on public data: it makes no diagnostic claim, and the UI shows a score next to the evidence rather than a verdict. For what a real diagnostic pipeline looks like end to end, see the 100,000 Genomes Project rare-disease pilot: Smedley et al., N Engl J Med 385:1868–1880, 2021. 10.1056/NEJMoa2035790

7.2 KiB Raw Blame History Unescape Escape