fix(science): stop scoring evidence that was never looked up

A review of the ranking's arithmetic found four things wrong, all of which made the score look better informed than it was. Measurements below are from this repo, not estimates. **Components now abstain instead of inventing a number.** A run without a VEP cache returns no allele frequencies, and rarity_score(None) read that as "absent from gnomAD, therefore maximally rare" and awarded every variant a free 0.25. jobs.has_frequencies / has_effect_scores record what the run actually produced, absent components are dropped from the weighted mean, and the remaining weights are renormalised so the score keeps its meaning. The UI shows "not looked up" rather than a bar, and the funnel stops calling a step "rare" when nothing was filtered. **Allele frequency is no longer a model feature.** It dominated: the same missense variant scored 0.887 at AF 0 and 0.0003 at AF 0.01. That double- counted, because the ranking already scores frequency explicitly, putting ~45% of every rank on one measurement; and it was circular, because ACMG assigns ClinVar's benign labels using frequency (BA1/BS1). Retraining without it moves missense AUROC from 0.872 to 0.500 — exactly random. The old figure was allele frequency, not variant-effect knowledge. The model therefore abstains unless CADD or AlphaMissense is present, since otherwise it only restates the consequence class. **Phenotype matching is weighted by information content** and HPO annotations are propagated up the ontology. Counting terms alike let "global developmental delay" (IC 0.93) count as much as "dilated left subclavian artery" (IC 7.88). **A real bug in the propagation, found by checking it.** The ancestor walk read a pre-order DFS backwards, which on a DAG lets a term resolve before one of its parents and inherit that parent alone instead of its lineage. It dropped 399 terms out of the phenotype branch, Camptodactyly and Chiari malformation among them. Now a true post-order, tested against a reference transitive closure. The ontology arithmetic moved to rarelens_ml.hpo so it is covered by tests, and rarelens_ml.benchmark measures the whole thing: across 10,178 published cases the causal gene ranks first 45.9-81.0% of the time against 5,269 genes, versus 0.02% for chance. docs/data.md reports that with its contamination (HPO's annotations come from these same case reports), and includes the measurement showing information-content weighting earns its place while propagation does not - kept anyway, for a reason the docs argue rather than assume.
2026-09-12 11:32:46 +01:00
parent 749b0f8214
commit e76ae847a1
37 changed files with 4324 additions and 195 deletions
@@ -18,4 +18,5 @@ mlruns/
 data/*.vcf*
 data/*.tsv
 data/*.case.json
 data/*.zip
 !data/README.md
@@ -1,7 +1,9 @@
-.PHONY: up down clean migrate test lint data hpo demo-case published-case training-set train loader pipeline annotate images kind serverless-deploy serverless-destroy gcp-configure gcp-secrets
+.PHONY: up down clean migrate test lint data hpo demo-case published-case benchmark training-set train loader pipeline annotate images kind serverless-deploy serverless-destroy gcp-configure gcp-secrets
 VCF ?= data/example.vcf.gz
 MLFLOW_URI ?= http://localhost:5001
 DB_CONTAINER ?= rarelens-db-1
 BENCH_DIR ?= data
 TAG ?= latest
 # The loader container reaches docker-compose's Postgres through the host.
 HOST_DB_URL ?= postgresql://rarelens:[email protected]:5432/rarelens
@@ -31,8 +33,8 @@ lint:
 data:  ## download the public demo slice: GIAB HG002 + ClinVar, chr22 (see docs/data.md)
 	scripts/fetch-demo-data.sh
-hpo:  ## load HPO gene-to-phenotype annotations, which the ranking matches against
+hpo:  ## load HPO annotations, propagated up the ontology and weighted by information content
-	scripts/load-hpo.py
+	cd ml && uv run --extra db python ../scripts/load-hpo.py
 demo-case:  ## build the simulated proband: GIAB background + one ClinVar pathogenic variant
 	scripts/make-demo-case.sh
@@ -43,6 +45,17 @@ published-case:  ## build a case from a published patient: a GA4GH phenopacket +
 training-set:  ## build a ClinVar training table, shaped like VEP --tab output
 	scripts/make-training-set.sh
 benchmark:  ## measure the phenotype ranking against every published case (see docs/data.md)
 	docker exec $(DB_CONTAINER) psql -U rarelens -d rarelens -At -F',' \
 	  -c "select gene_symbol, hpo_id from gene_phenotypes" \
 	  | tr ',' '\t' > $(BENCH_DIR)/gene_phenotypes.tsv
 	test -f $(BENCH_DIR)/all_phenopackets.zip || curl -sL -o $(BENCH_DIR)/all_phenopackets.zip \
 	  "$$(curl -s https://api.github.com/repos/monarch-initiative/phenopacket-store/releases/latest \
 	     | sed -n 's/.*"browser_download_url": "\(.*all_phenopackets.zip\)".*/\1/p')"
 	cd ml && uv run --extra dev python -m rarelens_ml.benchmark \
 	  --phenopackets ../$(BENCH_DIR)/all_phenopackets.zip \
 	  --annotations ../$(BENCH_DIR)/gene_phenotypes.tsv
 train: ## train the pathogenicity model and point the production alias at it (needs `make up`)
 	cd ml && MLFLOW_TRACKING_URI=$(MLFLOW_URI) uv run --extra dev \
 		python -m rarelens_ml.train --tsv ../data/clinvar-training.vep.tsv --register
@@ -89,8 +89,22 @@ make training-set   # a ClinVar-derived training table, ~370k labelled variants
 make train          # fits, reports held-out metrics by gene split, moves the production alias
 ```
-What those metrics do and do not mean is in [docs/data.md](docs/data.md); the headline AUROC
+What those metrics do and do not mean is in [docs/data.md](docs/data.md). The short version:
-flatters a model whose strongest feature is the consequence class.
+with allele frequency removed as a feature, the model scores AUROC **0.500 — exactly random — on
 missense variants**, because nothing is left but the consequence class the ranking already uses.
 It therefore abstains from the ranking unless CADD or AlphaMissense scores are available. The
 frequency feature is what made the old 0.872 look respectable, and ACMG assigns ClinVar's benign
 labels using frequency, so the feature had partly caused the label.
 ```bash
 make benchmark      # rank every published case in Phenopacket Store by phenotype alone
 ```
 Across 10,178 published cases the causal gene is ranked first 45.9-81.0% of the time (the range is
 ties; random would be 0.02%). That benchmark is contaminated — HPO's gene annotations come from the
 same case reports — so read it as an upper bound. [docs/data.md](docs/data.md) has the full table,
 including the measurement that says information-content weighting earns its place and ontology
 propagation does not.
 Local Kubernetes: `make kind` builds the images, loads them into a kind cluster and applies
 `infra/k8s/overlays/local`.
@@ -0,0 +1,48 @@
 """Record what an annotation run produced, and how specific each HPO term is
 Two changes, both so the ranking can stop asserting things it has not measured:
 - jobs.has_frequencies / has_effect_scores: whether the run looked up allele frequencies and
  CADD/AlphaMissense at all. Without this the API cannot tell "absent from gnomAD" (strong
  evidence) from "nobody checked gnomAD" (no evidence), and scored both as maximally rare.
 - hpo_terms: each term's information content, so a phenotype match is weighted by how specific
  the matching term is rather than counting every term alike.
 Existing rows default to false, which is correct: every job recorded before this ran used VEP's
 database mode, which returns neither frequencies nor plugin scores.
 Revision ID: b7d4e2f80c31
 Revises: 9a1c2d3e4f50
 """
 from collections.abc import Sequence
 import sqlalchemy as sa
 from alembic import op
 revision: str = "b7d4e2f80c31"
 down_revision: str | None = "9a1c2d3e4f50"
 branch_labels: str | Sequence[str] | None = None
 depends_on: str | Sequence[str] | None = None
 def upgrade() -> None:
    op.add_column(
        "jobs",
        sa.Column("has_frequencies", sa.Boolean(), nullable=False, server_default=sa.false()),
    )
    op.add_column(
        "jobs",
        sa.Column("has_effect_scores", sa.Boolean(), nullable=False, server_default=sa.false()),
    )
    op.create_table(
        "hpo_terms",
        sa.Column("hpo_id", sa.String(length=20), primary_key=True),
        sa.Column("name", sa.String(length=200), nullable=False),
        sa.Column("ic", sa.Float(), nullable=False),
    )
 def downgrade() -> None:
    op.drop_table("hpo_terms")
    op.drop_column("jobs", "has_effect_scores")
    op.drop_column("jobs", "has_frequencies")
@@ -8,6 +8,7 @@ import uuid
 from datetime import datetime
 from sqlalchemy import (
    Boolean,
    DateTime,
    Enum,
    Float,
@@ -63,8 +64,25 @@ class CasePhenotype(Base):
    case: Mapped[Case] = relationship(back_populates="phenotypes")
 class HpoTerm(Base):
    """One HPO term and its information content (scripts/load-hpo.py); read-only reference data.
    ic is -ln(fraction of annotated genes carrying the term), so a term shared by nearly every
    gene is worth almost nothing and a near-pathognomonic one is worth a lot.
    """
    __tablename__ = "hpo_terms"
    hpo_id: Mapped[str] = mapped_column(String(20), primary_key=True)
    name: Mapped[str] = mapped_column(String(200))
    ic: Mapped[float] = mapped_column(Float)
 class GenePhenotype(Base):
-    """HPO's gene-to-phenotype annotations (scripts/load-hpo.py); read-only reference data."""
+    """HPO's gene-to-phenotype annotations (scripts/load-hpo.py); read-only reference data.
    Propagated up the ontology at load time: a gene annotated with a term also carries that term's
    ancestors, so a case term matches a gene annotated with anything more specific.
    """
    __tablename__ = "gene_phenotypes"
    __table_args__ = (UniqueConstraint("gene_symbol", "hpo_id", name="uq_gene_phenotypes_gene_term"),)
@@ -81,6 +99,10 @@ class Job(Base):
    status: Mapped[JobStatus] = mapped_column(Enum(JobStatus), default=JobStatus.queued)
    workflow_ref: Mapped[str | None] = mapped_column(String(200))   # Argo workflow name / nf run id
    vep_version: Mapped[str | None] = mapped_column(String(40))
    # What this run's annotation actually produced. The ranking refuses to score a line of
    # evidence the run never looked up, so it has to be recorded rather than assumed.
    has_frequencies: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
    has_effect_scores: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
    log: Mapped[str | None] = mapped_column(Text)
    created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now())
    finished_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
@@ -24,6 +24,7 @@ from app.schemas import (
    CandidatePage,
    CaseCreate,
    CaseOut,
    EvidenceOut,
    FunnelOut,
    JobOut,
    ProvenanceOut,
@@ -194,7 +195,8 @@ async def list_candidates(
    return CandidatePage(
        # The funnel describes the whole case, not the filtered view.
        funnel=FunnelOut(**asdict(view.funnel)),
-        weights=triage.WEIGHTS,
+        evidence=EvidenceOut(**asdict(view.evidence), missing=view.evidence.missing),
        weights=triage.weights_in_use(view.evidence),
        items=[CandidateOut.from_candidate(c, labels) for c in items[offset : offset + limit]],
        total=len(items),
        limit=limit,
@@ -28,10 +28,15 @@ async def get_variant(variant_id: int, session: SessionDep) -> VariantDetailOut:
        raise HTTPException(404, "variant not found")
    case = await case_view.get_case(session, variant.job.case_id)
    terms = [p.hpo_id for p in case.phenotypes] if case else []
-    gene_terms = await case_view.gene_terms_for(session, {variant.gene} if variant.gene else set())
+    ontology = await case_view.ontology_for(
        session, {variant.gene} if variant.gene else set(), terms
    )
    evidence = triage.Evidence(
        frequencies=variant.job.has_frequencies, effect_scores=variant.job.has_effect_scores
    )
    # evaluate, not rank: the panel must work for a variant that did not make the candidate list.
-    scored = triage.evaluate(variant, terms, gene_terms)
+    scored = triage.evaluate(variant, terms, ontology, evidence)
    labels = {p.hpo_id: p.label for p in case.phenotypes} if case else {}
    base = CandidateOut.from_candidate(scored, labels).model_dump()
    return VariantDetailOut(**base, annotations=variant.annotations or {})
@@ -114,7 +114,9 @@ class DecisionOut(ORMModel):
 class CandidateOut(BaseModel):
    variant: VariantOut
    score: float
-    components: dict[str, float]
+    # A null component means that evidence was never looked up, so it did not enter the score.
    # It is not a zero, and the UI must not draw it as an empty bar.
    components: dict[str, float | None]
    matched_terms: list[PhenotypeTerm]
    scored: bool
    decision: DecisionOut | None = None
@@ -125,7 +127,10 @@ class CandidateOut(BaseModel):
        return cls(
            variant=VariantOut.model_validate(variant),
            score=round(candidate.score, 4),
-            components={name: round(v, 4) for name, v in candidate.components.items()},
+            components={
                name: None if v is None else round(v, 4)
                for name, v in candidate.components.items()
            },
            matched_terms=[
                PhenotypeTerm(hpo_id=term, label=labels.get(term, term))
                for term in candidate.matched_terms
@@ -144,10 +149,21 @@ class FunnelOut(BaseModel):
    rare: int
    candidates: int
    phenotype_matched: int
    frequencies: bool = False  # False: the rare step could not filter, nothing was looked up
 class EvidenceOut(BaseModel):
    """What the annotation run produced, and therefore which components scored at all."""
    frequencies: bool
    effect_scores: bool
    missing: list[str]
 class CandidatePage(BaseModel):
    funnel: FunnelOut
    evidence: EvidenceOut
    # The weights as applied: renormalised over the components that had evidence.
    weights: dict[str, float]
    items: list[CandidateOut]
    total: int
@@ -4,13 +4,14 @@ Everything for one job is loaded at once, which is fine for a gene panel or a ch
 size of case this demo handles. A whole genome would need the narrowing pushed into SQL.
 """
 import uuid
-from dataclasses import dataclass
+from collections.abc import Sequence
 from dataclasses import dataclass, field
 from sqlalchemy import select
 from sqlalchemy.ext.asyncio import AsyncSession
 from sqlalchemy.orm import selectinload
-from app.models import Case, GenePhenotype, Job, JobStatus, Variant
+from app.models import Case, GenePhenotype, HpoTerm, Job, JobStatus, Variant
 from app.services import triage
 EMPTY_FUNNEL = triage.Funnel(total=0, rare=0, candidates=0, phenotype_matched=0)
@@ -22,6 +23,7 @@ class CaseView:
    job: Job | None
    funnel: triage.Funnel
    candidates: list[triage.Candidate]
    evidence: triage.Evidence = field(default_factory=triage.Evidence)
    @property
    def labels(self) -> dict[str, str]:
@@ -43,19 +45,31 @@ async def latest_job(
    return await session.scalar(stmt)
-async def gene_terms_for(session: AsyncSession, genes: set[str]) -> dict[str, set[str]]:
+async def ontology_for(
-    """gene symbol -> the HPO terms annotated to it."""
+    session: AsyncSession, genes: set[str], case_terms: Sequence[str]
 ) -> triage.Ontology:
    """The HPO reference data this case needs: each gene's terms, and each case term's specificity.
    Only the case's own terms need an information content: they are the denominator of the
    phenotype score, and a gene's other annotations never enter it.
    """
    if not genes:
-        return {}
+        return triage.Ontology()
    rows = await session.execute(
        select(GenePhenotype.gene_symbol, GenePhenotype.hpo_id).where(
            GenePhenotype.gene_symbol.in_(genes)
        )
    )
-    index: dict[str, set[str]] = {}
+    gene_terms: dict[str, set[str]] = {}
    for gene, hpo_id in rows:
-        index.setdefault(gene, set()).add(hpo_id)
+        gene_terms.setdefault(gene, set()).add(hpo_id)
-    return index
+    ic: dict[str, float] = {}
    if case_terms:
        weights = await session.execute(
            select(HpoTerm.hpo_id, HpoTerm.ic).where(HpoTerm.hpo_id.in_(case_terms))
        )
        ic = {hpo_id: float(value) for hpo_id, value in weights}
    return triage.Ontology(gene_terms=gene_terms, ic=ic)
 async def build(session: AsyncSession, case: Case) -> CaseView:
@@ -70,10 +84,14 @@ async def build(session: AsyncSession, case: Case) -> CaseView:
        )
    ).all()
    case_terms = [p.hpo_id for p in case.phenotypes]
-    gene_terms = await gene_terms_for(session, {v.gene for v in variants if v.gene})
+    ontology = await ontology_for(session, {v.gene for v in variants if v.gene}, case_terms)
    evidence = triage.Evidence(
        frequencies=job.has_frequencies, effect_scores=job.has_effect_scores
    )
    return CaseView(
        case=case,
        job=job,
-        funnel=triage.funnel(variants, case_terms, gene_terms),
+        funnel=triage.funnel(variants, case_terms, ontology, evidence),
-        candidates=triage.rank(variants, case_terms, gene_terms),
+        candidates=triage.rank(variants, case_terms, ontology, evidence),
        evidence=evidence,
    )
@@ -19,8 +19,9 @@ from sqlalchemy.ext.asyncio import AsyncSession
 from app.config import settings
 from app.models import Prediction, Variant
-# Must match rarelens_ml.features.RAW_COLUMNS.
+# Must match rarelens_ml.features.RAW_COLUMNS. Allele frequency is not among them: the ranking
-RAW_COLUMNS = ["impact", "consequence", "gnomad_af", "cadd_phred", "am_pathogenicity"]
+# scores frequency itself, and feeding it here too counted one measurement twice.
 RAW_COLUMNS = ["impact", "consequence", "cadd_phred", "am_pathogenicity"]
 CHUNK_SIZE = 5000
 _models: dict[str, Any] = {}  # model version -> loaded pyfunc
@@ -53,7 +54,6 @@ def raw_frame(variants: Sequence[Variant]) -> pd.DataFrame:
        {
            "impact": [v.impact for v in variants],
            "consequence": [v.consequence for v in variants],
            "gnomad_af": [v.gnomad_af if v.gnomad_af is not None else float("nan") for v in variants],
            "cadd_phred": [v.annotations.get("CADD_PHRED") for v in variants],
            "am_pathogenicity": [v.annotations.get("am_pathogenicity") for v in variants],
        },
@@ -1,14 +1,27 @@
 """Narrow a case's variants the way a clinical scientist does, and say why.
-The rank is a weighted sum of four parts a reviewer can audit. ClinVar is deliberately not one of
+The rank is a weighted mean of four lines of evidence a reviewer can audit. ClinVar is deliberately
-them: it is shown beside the result as independent confirmation, so a variant never ranks highly
+not one of them: it is shown beside the result as independent confirmation, so a variant never
-merely because ClinVar already called it pathogenic.
+ranks highly merely because ClinVar already called it pathogenic.
 Rarity and consequence *filter* (the usual first pass); phenotype only *ranks*, because a real
 diagnosis can sit in a gene nobody has annotated yet and filtering on it would hide exactly that.
 Two rules keep the number honest:
 **A line of evidence that was never looked up abstains.** It does not score zero, and it certainly
 does not score full marks. Treating "no gnomAD frequency in the annotation run" as "absent from
 gnomAD, therefore maximally rare" awarded every variant a free 0.25, which is a guess wearing the
 costume of a measurement. `Evidence` says what the run actually produced, and the weights
 renormalise over whatever is left, so the score stays on a 0-1 scale and means the same thing.
 **Each line of evidence is counted once.** The model used to take allele frequency as a feature
 while `rarity` scored the same frequency again, so roughly 45% of the rank was one measurement
 double-counted. The model no longer sees frequency (see rarelens_ml.features); it earns its weight
 only when it has something the other three do not already say, which means CADD or AlphaMissense.
 """
 from collections.abc import Mapping, Sequence
-from dataclasses import dataclass
+from dataclasses import dataclass, field
 from app.models import Variant
@@ -19,6 +32,49 @@ CANDIDATE_IMPACTS = frozenset({"HIGH", "MODERATE"})
 IMPACT_SEVERITY = {"HIGH": 1.0, "MODERATE": 0.6, "LOW": 0.2, "MODIFIER": 0.0}
 # Allele frequency ceiling -> score, rarest first.
 RARITY_STEPS = ((0.0, 1.0), (0.0001, 0.8), (0.001, 0.5), (0.01, 0.2))
 # A term HPO has never annotated to any gene cannot match anything, so its information content is
 # unknown. Treating it as maximally specific keeps it in the denominator and depresses every gene
 # equally, which is the neutral choice.
 DEFAULT_IC = 10.0
@dataclass(frozen=True)
 class Evidence:
    """What the annotation run actually produced, and therefore which components may score.
    Decided once per job rather than per variant: components must be in play for every variant in
    a case, or two variants would be scored against different denominators and their ranks would
    not be comparable.
    """
    frequencies: bool = False       # did the run look up allele frequencies at all?
    effect_scores: bool = False     # CADD / AlphaMissense, the only features the model adds
    @property
    def missing(self) -> list[str]:
        absent = []
        if not self.frequencies:
            absent.append("rarity")
        if not self.effect_scores:
            absent.append("model")
        return absent
@dataclass(frozen=True)
 class Ontology:
    """HPO reference data: what each gene is annotated with, and how specific each term is.
    `gene_terms` is expected to be propagated up the ontology by scripts/load-hpo.py, so a case
    term matches a gene annotated with any of its descendants. `ic` is information content,
    -ln(fraction of genes carrying the term): "Bifid uvula" is worth many times "Abnormality of
    the head", which nearly every gene in the corpus carries.
    """
    gene_terms: Mapping[str, set[str]] = field(default_factory=dict)
    ic: Mapping[str, float] = field(default_factory=dict)
    def weight(self, term: str) -> float:
        return self.ic.get(term, DEFAULT_IC)
@dataclass(frozen=True)
@@ -29,19 +85,21 @@ class Funnel:
    rare: int
    candidates: int
    phenotype_matched: int
    frequencies: bool = False  # False means the "rare" step filtered nothing, because it could not
@dataclass(frozen=True)
 class Candidate:
    variant: Variant
    score: float
-    components: dict[str, float]
+    components: dict[str, float | None]  # None: this evidence was not available
    matched_terms: list[str]
    scored: bool
 def rarity_score(af: float | None) -> float:
-    if af is None:  # absent from gnomAD
+    """Only meaningful when frequencies were annotated; None then means absent from gnomAD."""
    if af is None:
        return 1.0
    for ceiling, score in RARITY_STEPS:
        if af <= ceiling:
@@ -54,14 +112,22 @@ def consequence_score(impact: str | None) -> float:
 def phenotype_score(
-    gene: str | None, case_terms: Sequence[str], gene_terms: Mapping[str, set[str]]
+    gene: str | None, case_terms: Sequence[str], ontology: Ontology
 ) -> tuple[float, list[str]]:
-    """What fraction of the patient's terms HPO associates with this gene, and which ones."""
+    """How much of the patient's phenotype HPO associates with this gene, weighted by specificity.
    Information-content-weighted recall: the share of the *total specificity* of the patient's
    terms that this gene accounts for. Plain term counting let a common term like global
    developmental delay count as much as a near-pathognomonic one.
    """
    if not gene or not case_terms:
        return 0.0, []
-    annotated = gene_terms.get(gene, set())
+    annotated = ontology.gene_terms.get(gene, set())
    matched = [term for term in case_terms if term in annotated]
-    return len(matched) / len(case_terms), matched
+    total = sum(ontology.weight(term) for term in case_terms)
    if total <= 0:
        return 0.0, matched
    return sum(ontology.weight(term) for term in matched) / total, matched
 def is_rare(variant: Variant) -> bool:
@@ -72,35 +138,51 @@ def is_candidate(variant: Variant) -> bool:
    return is_rare(variant) and variant.impact in CANDIDATE_IMPACTS
-def funnel(
+def combine(components: Mapping[str, float | None]) -> float:
-    variants: Sequence[Variant], case_terms: Sequence[str], gene_terms: Mapping[str, set[str]]
+    """Weighted mean over the components that have evidence, renormalised to 0-1."""
-) -> Funnel:
+    weight = sum(WEIGHTS[name] for name, value in components.items() if value is not None)
    if weight <= 0:
        return 0.0
    return sum(WEIGHTS[name] * value for name, value in components.items() if value is not None) / weight
 def weights_in_use(evidence: Evidence) -> dict[str, float]:
    """The weights as actually applied, so the UI never shows a bar the score did not use."""
    live = {name: w for name, w in WEIGHTS.items() if name not in evidence.missing}
    total = sum(live.values())
    return {name: round(w / total, 4) for name, w in live.items()} if total else {}
 def funnel(variants: Sequence[Variant], case_terms: Sequence[str], ontology: Ontology,
           evidence: Evidence) -> Funnel:
    rare = [v for v in variants if is_rare(v)]
    candidates = [v for v in rare if v.impact in CANDIDATE_IMPACTS]
-    matched = sum(1 for v in candidates if phenotype_score(v.gene, case_terms, gene_terms)[1])
+    matched = sum(1 for v in candidates if phenotype_score(v.gene, case_terms, ontology)[1])
-    return Funnel(len(variants), len(rare), len(candidates), matched)
+    return Funnel(len(variants), len(rare), len(candidates), matched, evidence.frequencies)
 def evaluate(
-    variant: Variant, case_terms: Sequence[str], gene_terms: Mapping[str, set[str]]
+    variant: Variant, case_terms: Sequence[str], ontology: Ontology, evidence: Evidence
 ) -> Candidate:
    """Score one variant, whether or not it survived the filters."""
-    phenotype, matched = phenotype_score(variant.gene, case_terms, gene_terms)
+    phenotype, matched = phenotype_score(variant.gene, case_terms, ontology)
    prediction = variant.prediction
-    components = {
+    model: float | None = None
    if evidence.effect_scores and prediction is not None:
        model = float(prediction.score)
    components: dict[str, float | None] = {
        "phenotype": phenotype,
-        "rarity": rarity_score(variant.gnomad_af),
+        "rarity": rarity_score(variant.gnomad_af) if evidence.frequencies else None,
        "consequence": consequence_score(variant.impact),
-        "model": float(prediction.score) if prediction is not None else 0.0,
+        "model": model,
    }
-    score = sum(WEIGHTS[name] * value for name, value in components.items())
+    return Candidate(variant, combine(components), components, matched, prediction is not None)
    return Candidate(variant, score, components, matched, prediction is not None)
 def rank(
-    variants: Sequence[Variant], case_terms: Sequence[str], gene_terms: Mapping[str, set[str]]
+    variants: Sequence[Variant], case_terms: Sequence[str], ontology: Ontology, evidence: Evidence
 ) -> list[Candidate]:
-    candidates = [evaluate(v, case_terms, gene_terms) for v in variants if is_candidate(v)]
+    candidates = [evaluate(v, case_terms, ontology, evidence) for v in variants if is_candidate(v)]
    # id breaks ties, so equal scores do not shuffle between requests.
    candidates.sort(key=lambda c: (-c.score, c.variant.id))
    return candidates
@@ -1,8 +1,19 @@
 import uuid
 from typing import Any
 from sqlalchemy.dialects.postgresql import insert
 from app.db import SessionLocal
-from app.models import Case, CasePhenotype, GenePhenotype, Job, JobStatus, Prediction, Variant
+from app.models import (
    Case,
    CasePhenotype,
    GenePhenotype,
    HpoTerm,
    Job,
    JobStatus,
    Prediction,
    Variant,
 )
 VARIANT_DEFAULTS: dict[str, Any] = {
    "chrom": "22", "pos": 1, "ref": "A", "alt": "G", "gene": "NF2",
@@ -17,10 +28,16 @@ async def seed_case(
    gene_terms: dict[str, list[tuple[str, str]]] | None = None,
    status: JobStatus = JobStatus.succeeded,
    name: str | None = None,
    has_frequencies: bool = True,
    has_effect_scores: bool = True,
 ) -> tuple[uuid.UUID, uuid.UUID]:
    """Insert a case, its phenotypes, a job and its variants. Returns (case_id, job_id).
    `variants` entries override VARIANT_DEFAULTS; a "score" key becomes a Prediction.
    The job defaults to a run that looked everything up, so a test says so explicitly when it
    wants the opposite. Every term gets information content 1.0, which makes the phenotype score
    plain term counting unless a test seeds its own weights.
    """
    async with SessionLocal() as s:
        case = Case(
@@ -29,11 +46,30 @@ async def seed_case(
            assembly="GRCh38",
            phenotypes=[CasePhenotype(hpo_id=hpo, label=label) for hpo, label in (phenotypes or [])],
        )
-        job = Job(case=case, status=status, vep_version="113.0")
+        job = Job(
            case=case,
            status=status,
            vep_version="113.0",
            has_frequencies=has_frequencies,
            has_effect_scores=has_effect_scores,
        )
        s.add_all([case, job])
-        for gene, terms in (gene_terms or {}).items():
+        # HPO rows are shared reference data, so two seeds in one test may name the same term.
-            s.add_all(
+        terms_seen: dict[str, str] = dict(phenotypes or [])
-                GenePhenotype(gene_symbol=gene, hpo_id=hpo, hpo_name=label) for hpo, label in terms
+        annotations = [
            {"gene_symbol": gene, "hpo_id": hpo, "hpo_name": label}
            for gene, terms in (gene_terms or {}).items()
            for hpo, label in terms
        ]
        for terms in (gene_terms or {}).values():
            terms_seen.update(terms)
        if annotations:
            await s.execute(insert(GenePhenotype).values(annotations).on_conflict_do_nothing())
        if terms_seen:
            await s.execute(
                insert(HpoTerm)
                .values([{"hpo_id": h, "name": lab, "ic": 1.0} for h, lab in terms_seen.items()])
                .on_conflict_do_nothing()
            )
        for spec in variants or []:
            fields = VARIANT_DEFAULTS | spec
@@ -45,7 +45,9 @@ async def test_the_funnel_shows_the_narrowing(client: AsyncClient) -> None:
    case_id, _ = await a_case_with_candidates()
    page = (await client.get(f"/api/cases/{case_id}/candidates")).json()
    # 4 variants -> 3 rare -> 2 rare and coding -> 1 of those in a phenotype-matched gene
-    assert page["funnel"] == {"total": 4, "rare": 3, "candidates": 2, "phenotype_matched": 1}
+    assert page["funnel"] == {
        "total": 4, "rare": 3, "candidates": 2, "phenotype_matched": 1, "frequencies": True,
    }
@pytest.mark.usefixtures("db")
@@ -1,4 +1,3 @@
 import math
 import uuid
 import numpy as np
@@ -29,7 +28,8 @@ def test_raw_frame_sends_the_model_contract_columns() -> None:
    assert frame["impact"].tolist() == ["HIGH", "LOW"]
    assert frame["cadd_phred"].iloc[0] == "35"
    assert pd.isna(frame["cadd_phred"].iloc[1])
-    assert math.isnan(frame["gnomad_af"].iloc[0])
+    # Frequency is scored by the ranking, auditably; sending it here too counted it twice.
    assert "gnomad_af" not in frame.columns
 class FakeModel:
@@ -4,6 +4,11 @@ import pytest
 from app.models import Prediction, Variant
 from app.services import triage
 # A run with a VEP cache and plugins: every line of evidence was looked up.
 FULL = triage.Evidence(frequencies=True, effect_scores=True)
 # VEP's database mode: no frequencies, no CADD/AlphaMissense.
 DATABASE_ONLY = triage.Evidence(frequencies=False, effect_scores=False)
 def variant(**kw: object) -> Variant:
    fields: dict = {
@@ -19,6 +24,12 @@ def variant(**kw: object) -> Variant:
    return v
 def ontology(gene_terms: dict[str, set[str]], ic: dict[str, float] | None = None) -> triage.Ontology:
    """Equal information content unless a test is specifically about specificity."""
    terms = {t for terms in gene_terms.values() for t in terms}
    return triage.Ontology(gene_terms=gene_terms, ic=ic or dict.fromkeys(terms, 1.0))
 def test_weights_sum_to_one() -> None:
    assert sum(triage.WEIGHTS.values()) == pytest.approx(1.0)
@@ -39,20 +50,40 @@ def test_consequence_severity(impact: str | None, expected: float) -> None:
    assert triage.consequence_score(impact) == expected
-def test_phenotype_match_is_the_fraction_of_the_patients_terms() -> None:
+def test_phenotype_match_is_the_share_of_the_patients_terms() -> None:
    gene_terms = {"NF2": {"HP:0000365", "HP:0009592"}}
    case_terms = ["HP:0000365", "HP:0009592", "HP:0002321", "HP:0000598"]
-    score, matched = triage.phenotype_score("NF2", case_terms, gene_terms)
+    o = triage.Ontology(
-    assert score == 0.5
+        gene_terms={"NF2": {"HP:0000365", "HP:0009592"}}, ic=dict.fromkeys(case_terms, 1.0)
    )
    score, matched = triage.phenotype_score("NF2", case_terms, o)
    assert score == pytest.approx(0.5)  # 2 of 4 terms, all equally specific
    assert matched == ["HP:0000365", "HP:0009592"]
 def test_a_specific_term_outweighs_a_common_one() -> None:
    """Counting terms alike let 'global developmental delay' rival a near-pathognomonic sign."""
    ic = {"HP:0001263": 0.1, "HP:0000193": 6.0}  # developmental delay vs bifid uvula
    case_terms = ["HP:0001263", "HP:0000193"]
    common = triage.phenotype_score("A", case_terms, triage.Ontology({"A": {"HP:0001263"}}, ic))
    specific = triage.phenotype_score("B", case_terms, triage.Ontology({"B": {"HP:0000193"}}, ic))
    assert common[0] == pytest.approx(0.1 / 6.1)
    assert specific[0] == pytest.approx(6.0 / 6.1)
    assert specific[0] > common[0] * 10
 def test_an_unknown_term_is_treated_as_maximally_specific() -> None:
    """It can never match, so it must depress every gene equally rather than be ignored."""
    o = triage.Ontology({"NF2": {"HP:0000365"}}, {"HP:0000365": triage.DEFAULT_IC})
    score, _ = triage.phenotype_score("NF2", ["HP:0000365", "HP:9999999"], o)
    assert score == pytest.approx(0.5)
 def test_phenotype_match_is_zero_for_genes_hpo_has_never_annotated() -> None:
-    assert triage.phenotype_score("NOVEL1", ["HP:0000365"], {}) == (0.0, [])
+    assert triage.phenotype_score("NOVEL1", ["HP:0000365"], ontology({})) == (0.0, [])
 def test_phenotype_match_is_zero_when_no_phenotype_was_entered() -> None:
-    assert triage.phenotype_score("NF2", [], {"NF2": {"HP:0000365"}}) == (0.0, [])
+    assert triage.phenotype_score("NF2", [], ontology({"NF2": {"HP:0000365"}})) == (0.0, [])
 def test_the_funnel_counts_each_narrowing_step() -> None:
@@ -62,18 +93,24 @@ def test_the_funnel_counts_each_narrowing_step() -> None:
        variant(id=3, gnomad_af=0.3, impact="HIGH", gene="NF2"),           # common
        variant(id=4, gnomad_af=None, impact="MODIFIER", gene="NF2"),      # rare, non-coding
    ]
-    funnel = triage.funnel(variants, case_terms=["HP:0000365"], gene_terms={"NF2": {"HP:0000365"}})
+    funnel = triage.funnel(variants, ["HP:0000365"], ontology({"NF2": {"HP:0000365"}}), FULL)
    assert (funnel.total, funnel.rare, funnel.candidates, funnel.phenotype_matched) == (4, 3, 2, 1)
    assert funnel.frequencies is True
 def test_the_funnel_admits_when_the_rare_step_filtered_nothing() -> None:
    variants = [variant(id=1, gnomad_af=None, impact="HIGH")]
    assert triage.funnel(variants, [], ontology({}), DATABASE_ONLY).frequencies is False
 def test_the_diagnosis_outranks_the_noise() -> None:
-    gene_terms = {"NF2": {"HP:0000365", "HP:0009592"}}
+    o = ontology({"NF2": {"HP:0000365", "HP:0009592"}})
    case_terms = ["HP:0000365", "HP:0009592"]
    diagnosis = variant(id=1, gene="NF2", impact="HIGH", gnomad_af=None, score=0.94)
    plausible = variant(id=2, gene="CHEK2", impact="MODERATE", gnomad_af=0.0004, score=0.55)
    noise = variant(id=3, gene="TTN", impact="MODERATE", gnomad_af=0.0009, score=0.10)
-    ranked = triage.rank([noise, plausible, diagnosis], case_terms, gene_terms)
+    ranked = triage.rank([noise, plausible, diagnosis], case_terms, o, FULL)
    assert [c.variant.id for c in ranked] == [1, 2, 3]
    top = ranked[0]
    assert top.matched_terms == case_terms
@@ -81,9 +118,41 @@ def test_the_diagnosis_outranks_the_noise() -> None:
    assert top.score == pytest.approx(0.35 + 0.25 + 0.20 + 0.20 * 0.94)
 def test_evidence_that_was_never_looked_up_abstains_instead_of_scoring_full_marks() -> None:
    """The bug this guards: a database-mode run gave every variant rarity 1.0 for free."""
    v = variant(id=1, gene="NF2", impact="HIGH", gnomad_af=None, score=0.94)
    o = ontology({"NF2": {"HP:0000365"}})
    [candidate] = triage.rank([v], ["HP:0000365"], o, DATABASE_ONLY)
    assert candidate.components["rarity"] is None
    assert candidate.components["model"] is None
    # Only phenotype (0.35) and consequence (0.20) had evidence, renormalised over 0.55.
    assert candidate.score == pytest.approx((0.35 * 1.0 + 0.20 * 1.0) / 0.55)
 def test_the_model_abstains_when_it_has_no_feature_the_ranking_lacks() -> None:
    """Without CADD or AlphaMissense the model only restates the consequence class."""
    v = variant(id=1, gnomad_af=None, score=0.89)
    evidence = triage.Evidence(frequencies=True, effect_scores=False)
    [candidate] = triage.rank([v], [], ontology({}), evidence)
    assert candidate.components["model"] is None
    assert candidate.scored is True  # a prediction exists; it just does not earn a weight
 def test_weights_in_use_renormalise_to_one() -> None:
    full = triage.weights_in_use(FULL)
    assert sum(full.values()) == pytest.approx(1.0)
    assert full == {name: pytest.approx(w) for name, w in triage.WEIGHTS.items()}
    partial = triage.weights_in_use(DATABASE_ONLY)
    assert set(partial) == {"phenotype", "consequence"}
    assert sum(partial.values()) == pytest.approx(1.0)
    assert partial["phenotype"] == pytest.approx(0.35 / 0.55, abs=1e-4)
 def test_an_unscored_variant_still_ranks_and_says_so() -> None:
-    [candidate] = triage.rank([variant(id=1, gnomad_af=None)], [], {})
+    [candidate] = triage.rank([variant(id=1, gnomad_af=None)], [], ontology({}), FULL)
-    assert candidate.components["model"] == 0.0
+    assert candidate.components["model"] is None
    assert candidate.scored is False
@@ -92,10 +161,10 @@ def test_common_and_non_coding_variants_are_not_candidates() -> None:
        variant(id=1, gnomad_af=0.2, impact="HIGH"),
        variant(id=2, gnomad_af=None, impact="MODIFIER"),
    ]
-    assert triage.rank(variants, [], {}) == []
+    assert triage.rank(variants, [], ontology({}), FULL) == []
 def test_ranking_is_deterministic_for_equal_scores() -> None:
    a = variant(id=7, gene="AAA", chrom="1", pos=10, gnomad_af=None)
    b = variant(id=3, gene="BBB", chrom="1", pos=10, gnomad_af=None)
-    assert [c.variant.id for c in triage.rank([a, b], [], {})] == [3, 7]
+    assert [c.variant.id for c in triage.rank([a, b], [], ontology({}), FULL)] == [3, 7]
@@ -29,15 +29,23 @@ variants could explain *this* phenotype.
 Rarity (<0.1% in gnomAD) and consequence (HIGH or MODERATE) *filter*, which is the usual first
 pass. Phenotype only *ranks*: a real diagnosis can sit in a gene nobody has annotated yet, and
-filtering on phenotype would hide exactly that case. The rank is a weighted sum whose parts are
+filtering on phenotype would hide exactly that case. The rank is a weighted mean whose parts are
 shown next to every candidate (`app/services/triage.py`):
-| Component | Weight |
+| Component | Weight | Scores when |
-|---|---|
+|---|---|---|
-| phenotype terms of this patient annotated to the gene | 0.35 |
+| share of this patient's phenotype annotated to the gene, weighted by term specificity | 0.35 | always |
-| rarity in gnomAD | 0.25 |
+| rarity in gnomAD | 0.25 | the run looked up frequencies |
-| consequence severity | 0.20 |
+| consequence severity | 0.20 | always |
-| model P(pathogenic) | 0.20 |
+| model P(pathogenic) | 0.20 | the run has CADD or AlphaMissense |
 **A component with no evidence abstains** rather than scoring zero or, worse, full marks. A run
 without a VEP cache returns no allele frequencies, and scoring every variant 1.0 for rarity because
 nobody consulted gnomAD is a guess wearing the costume of a measurement. The job records what the
 run produced, absent components are dropped, and the remaining weights are renormalised so the
 score still means the same thing. The model abstains without CADD or AlphaMissense for a related
 reason: with only the consequence class it achieves AUROC 0.500 on missense variants, so it would
 be restating the consequence component rather than adding evidence. See docs/data.md.
 **ClinVar is deliberately not an input.** It sits beside the result as independent confirmation, so
 the demo never ranks a variant highly merely because ClinVar already called it pathogenic. On the
@@ -56,9 +56,11 @@ admired. Give the case the phenotype of the planted disease and the planted vari
 first — on phenotype, rarity and consequence, with ClinVar agreeing only afterwards.
 **Caveat when running without a VEP cache.** `VEP_DATABASE=true` queries Ensembl's public database
-instead of the 25 GB cache. It returns no gnomAD frequencies, so every variant looks absent from
+instead of the 25 GB cache. It returns no gnomAD frequencies and no plugin scores, so the rarity
-gnomAD and the rarity term stops discriminating. Fine for showing the mechanics; use the cache for
+and model components have nothing to work with. They abstain: the job records what the run looked
-anything you would quote.
+up (`jobs.has_frequencies`, `jobs.has_effect_scores`), those components are dropped from the score
 rather than given a default, and the remaining weights are renormalised. Fine for showing the
 mechanics; use the cache for anything you would quote.
 ## A published case
@@ -86,31 +88,31 @@ window only 9 fall in coding exons, so a random sample is entirely intronic, the
 filter throws all of it away, and the causal variant ends up the only candidate left — a funnel
 that proves nothing. Real coding variants give the ranking something it has to rank *against*.
-What that run looks like: 21 variants in, 21 "rare" (see the caveat below), **2** surviving the
+What that run looks like: 21 variants in, **2** surviving the consequence filter, **1** matching
-consequence filter, **1** matching the phenotype.
+the phenotype. Run without a VEP cache, so two of the four components have nothing to go on and
 abstain; the remaining weights are renormalised over 0.55.
-| | score | phenotype | rarity | consequence | model |
+| | score | phenotype (0.64) | rarity | consequence (0.36) | model |
 |---|---|---|---|---|---|
-| *TGFBR2* 3:30672252 missense | **0.897** | 1.00 (30/30 terms) | 1.00 | 0.60 | 0.887 |
+| *TGFBR2* 3:30672252 missense | **0.855** | 1.00 (30/30 terms) | not looked up | 0.60 | not looked up |
-| *OSBPL10* 3:31748090 missense | 0.547 | 0.00 | 1.00 | 0.60 | 0.887 |
+| *OSBPL10* 3:31748090 missense | 0.218 | 0.00 | not looked up | 0.60 | not looked up |
 This is the whole argument for phenotype-driven triage in one table. Both are rare missense
-variants; the model scores them **identically**, to three decimal places, because nothing about
+variants, identical on every piece of evidence this run has except one. What separates the
-the variants themselves distinguishes them. What separates the published diagnosis from an
+published diagnosis from an incidental variant in a lipid-transport gene is the patient's
-incidental variant in a lipid-transport gene is the patient's phenotype, and nothing else.
+phenotype, and nothing else. ClinVar's "pathogenic" on the first row is shown afterwards as
-ClinVar's "pathogenic" on the first row is shown afterwards as independent confirmation — it is
+independent confirmation — it is not an input to the rank.
 not an input to the rank.
 Three things to say out loud when showing it:
 - **The phenotype match is partly circular.** HPO's gene-to-phenotype annotations are themselves
  curated from published cases, quite possibly including this one. A 30/30 term match against
  *TGFBR2* is evidence the plumbing works, not evidence the ranking would find a novel gene.
- **Rarity is not doing any work without a VEP cache.** See the caveat above: in database mode
+- **Two of the four components abstained, and that is the honest outcome.** In database mode VEP
-  every variant looks absent from gnomAD, so the funnel's rarity step passes everything and every
+  returns no frequencies and no plugin scores, so rarity and the model have nothing to say.
-  variant scores a full 1.0 on rarity. `--af_gnomade` is rejected outright with `--database`, and
+  `--af_gnomade` is rejected outright with `--database`, and plain `--af` returns nothing even for
-  plain `--af` returns nothing even for common variants — checked against rs429358, roughly 15%
+  common variants — checked against rs429358, roughly 15% globally. An earlier version of this
-  globally. Frequencies need the cache; there is no shortcut.
+  table read 1.00 for rarity on both rows, which was not a measurement: it was the absence of one.
 - **The background is one healthy genome, not a diagnostic exome.** A real case would have
  thousands of rare coding variants to discard, not a handful.
@@ -120,35 +122,90 @@ Other cases work the same way — any phenopacket with GRCh38 coordinates will d
 scripts/make-published-case.py --phenopacket <raw phenopacket-store JSON URL>
 ```
-## The model, and what its numbers mean
+## The model, and why it currently abstains
 `make training-set` builds a training table straight from ClinVar rather than running VEP over
-hundreds of thousands of variants: ClinVar already carries the molecular consequence (`MC`), the
+hundreds of thousands of variants: ClinVar already carries the molecular consequence (`MC`) and the
-gene (`GENEINFO`) and an allele frequency (`AF_EXAC`), which is the feature set serving sends.
+gene (`GENEINFO`). Only 2-star-and-above records are kept. `make train` then fits LightGBM and
-Only 2-star-and-above records are kept. `make train` then fits LightGBM and points the
+points the `production` alias at the new version. 312,025 training and 74,239 held-out variants
-`production` alias at the new version.
+across 7,728 and 1,932 genes, with no gene on both sides.
-The last run: 312,025 training and 74,239 held-out variants across 7,728 and 1,932 genes, with no
+The model used to take allele frequency as a feature. Removing it is the single most informative
-gene on both sides.
+thing in this document, because of what happened to the numbers:
-| | AUROC | AUPRC |
+| | AUROC | AUPRC | missense AUROC | missense AUPRC |
-|---|---|---|
+|---|---|---|---|---|
-| all held-out variants | 0.986 | 0.954 |
+| v2, with gnomAD allele frequency | 0.986 | 0.954 | 0.872 | 0.725 |
-| missense only (13,553) | 0.872 | 0.725 |
+| v3, allele frequency removed | 0.966 | 0.881 | **0.500** | 0.398 |
-Three things to say before anyone quotes the headline number:
+**0.500 on missense is exactly random.** Strip frequency out and the model cannot tell one missense
 variant from another at all, because nothing is left but the consequence class — every missense row
 gets the identical score. So the respectable-looking 0.872 was not variant-effect knowledge. It was
 allele frequency, and ClinVar's benign calls are *made with* allele frequency under ACMG's BA1/BS1
 criteria. The feature had partly caused the label; the model had rediscovered the labelling rule.
-1. **0.986 mostly measures how separable ClinVar's classes are by consequence.** Its pathogenic set
+The remaining 0.966 is the same trick one level up: ClinVar's pathogenic set is largely loss of
-   is largely loss of function and its benign set largely is not, so a model handed the consequence
+function and its benign set largely is not, so a model handed the consequence class separates them
-   class does well without knowing anything hard. That is why the missense row exists: missense is
+without knowing anything hard.
-   where interpretation is actually difficult.
+
-2. **Even 0.872 is flattered by circularity.** Within missense, every row has the same consequence
+Two consequences, both deliberate:
-   and impact and no CADD or AlphaMissense score, so allele frequency is doing nearly all the work
+
-   — and ClinVar's benign calls frequently *use* allele frequency as evidence (ACMG BA1/BS1). The
+1. **Allele frequency is no longer a feature.** The ranking already scores frequency explicitly, as
-   feature partly caused the label.
+   a step function a reviewer can read (`triage.rarity_score`). Feeding it to the model as well put
-3. **It is not comparable to published CADD or AlphaMissense numbers.** Those are trained and
+   roughly 45% of every rank on one measurement counted twice.
-   evaluated on different data. A fair comparison scores the same held-out rows with all three,
+2. **The model abstains unless it has CADD or AlphaMissense.** Without them it can only restate the
-   which needs the plugin data (see above) and is the obvious next step.
+   consequence class, which the ranking already scores — and 0.500 is the measurement saying so.
   Install the plugin data (see data/README.md) and the model earns its 0.20 back; until then it
   contributes nothing, and the UI says so rather than showing a number.
 It is also not comparable to published CADD or AlphaMissense figures, which are trained and
 evaluated on different data. A fair comparison scores the same held-out rows with all three, which
 needs the plugin data and is the obvious next step.
 ## How well does the phenotype ranking actually work?
 `make benchmark` runs the ranking against every case in Phenopacket Store: given a real patient's
 reported terms, where does the gene their authors diagnosed come in a ranking of all 5,269 genes
 HPO annotates? Ties are reported as a range, because term-overlap scoring puts many genes on
 identical scores — optimistic counts a tie as a win, pessimistic counts every tied gene as ahead.
 | cases | | top-1 | top-10 | MRR |
 |---|---|---|---|---|
 | all 10,178 | optimistic | 81.0% | 87.2% | 0.830 |
 | | pessimistic | 45.9% | 69.7% | 0.541 |
 | the 6,485 with ≥6 terms | optimistic | 77.1% | 85.4% | 0.797 |
 | | pessimistic | 59.5% | 81.0% | 0.670 |
 Random guessing would put the right gene first 0.02% of the time, so the phenotype term is doing
 real work. Two caveats, and the first is severe.
 **The benchmark is contaminated.** The median causal gene already carries *every one* of its
 patient's terms, because HPO's gene annotations are curated from these same case reports. This
 measures how well the ranking retrieves a gene HPO has already been told about — an upper bound.
 A prospective number, on a patient whose gene nobody has annotated yet, would be lower, and this
 corpus cannot say by how much.
 **Information content earns its place; propagation does not, measurably.** Both were added to
 replace plain term counting, and the corpus was asked whether they helped. On the 6,485 cases with
 at least six terms, pessimistic figures (the honest end of the range):
 | scoring | top-1 | top-10 | MRR |
 |---|---|---|---|
 | direct annotations, count terms (the original) | 61.8% | 80.1% | 0.682 |
 | direct annotations, weight by information content | **63.6%** | **83.5%** | **0.706** |
 | propagated annotations, count terms | 58.2% | 77.7% | 0.653 |
 | propagated + information content (shipped) | 59.5% | 81.0% | 0.670 |
 Weighting by specificity helps: it breaks ties, which is exactly what it is for. Propagation costs
 about as much as weighting gains, and the shipped combination is a wash against the original — a
 point or two either way inside a contaminated benchmark.
 Propagation is kept anyway, and the reason is worth stating plainly rather than hiding behind the
 numbers. This corpus cannot show what propagation is for: its terms were chosen by the same
 curators whose choices HPO records, so the IDs already line up and exact matching is flattered.
 The app's users pick their own terms from a search box and will not line up that neatly. The
 measurement is here so anyone who disagrees can act on it — the four rows above are one flag and
 one argument to `make benchmark` apart.
 ## Evaluating the model honestly
@@ -156,7 +213,8 @@ The model trains on ClinVar labels and is scored on ClinVar-labelled variants, w
 where published benchmarks go wrong. What to do about it:
 1. **Never let the label into the features.** `CLIN_SIG` is excluded by construction; `clinvar_sig`
-   is stored for display only (`rarelens_ml/features.py` lists the five feature columns).
+   is stored for display only (`rarelens_ml/features.py` lists the feature columns). Allele
   frequency was removed for a related reason: ACMG uses it to *assign* the benign label.
 2. **Split by gene, not by variant.** Random splits put variants from the same gene on both sides,
   and a model can then score a gene rather than a variant. Grimm et al. showed this inflates
   reported accuracy for exactly this class of tool: *Hum Mutat* 36:513–523, 2015.
@@ -11,6 +11,7 @@ dependencies = ["lightgbm>=4.5", "mlflow>=3,<4", "pandas", "scikit-learn"]
 [project.optional-dependencies]
 gpu = ["torch"]   # for the optional deep-learning baseline on GPU
 db = ["sqlalchemy>=2", "psycopg[binary]>=3"]   # scripts/load-hpo.py writes the HPO tables
 dev = ["pytest>=8"]
 [tool.setuptools.packages.find]
@@ -0,0 +1,134 @@
 """Measure the phenotype ranking against every published case in Phenopacket Store.
 One demo case ranking correctly is an anecdote. This asks the only question that matters for a
 phenotype-driven tool: given a real patient's reported terms, where does the gene the authors
 actually diagnosed come in a ranking of every gene HPO annotates?
    python -m rarelens_ml.benchmark --phenopackets all_phenopackets.zip
 **Read the result with the contamination in mind.** HPO's gene-to-phenotype annotations are
 themselves curated from published case reports — quite possibly the very ones being scored here.
 The median causal gene already carries every one of its patient's terms, so this measures how well
 the ranking retrieves a gene HPO has already been told about. It is an upper bound. A prospective
 number, on a patient whose disease gene nobody has annotated yet, would be lower; how much lower
 this corpus cannot say.
 Ties are the other trap. Scoring by term overlap alone puts many genes on identical scores, so the
 honest report is a range: optimistic counts a tie as a win, pessimistic counts every tied gene as
 ranked ahead of the right answer. The truth is between them.
 """
 import argparse
 import json
 import statistics
 import sys
 import zipfile
 from collections import defaultdict
 from collections.abc import Iterator
 from rarelens_ml.hpo import information_content, phenotype_score
 def gene_annotations(path: str) -> dict[str, set[str]]:
    """gene -> HPO terms, from the propagated table scripts/load-hpo.py writes (TSV export)."""
    genes: dict[str, set[str]] = defaultdict(set)
    with open(path) as fh:
        for line in fh:
            gene, _, term = line.rstrip("\n").partition("\t")
            if gene and term:
                genes[gene].add(term)
    return genes
 def cases(path: str) -> Iterator[tuple[str, list[str]]]:
    """(causal gene, observed HPO terms) for each phenopacket with exactly one causal gene."""
    with zipfile.ZipFile(path) as z:
        for entry in z.namelist():
            if not entry.endswith(".json"):
                continue
            try:
                packet = json.loads(z.read(entry))
            except ValueError:
                continue
            causal = {
                g.get("variantInterpretation", {})
                .get("variationDescriptor", {})
                .get("geneContext", {})
                .get("symbol")
                for i in packet.get("interpretations", [])
                for g in i.get("diagnosis", {}).get("genomicInterpretations", [])
            } - {None}
            terms = [
                f["type"]["id"]
                for f in packet.get("phenotypicFeatures", [])
                if not f.get("excluded")
            ]
            if len(causal) == 1 and terms:
                yield causal.pop(), terms
 def rank_of(gene: str, terms: list[str], genes: dict[str, set[str]], ic: dict[str, float],
            default: float) -> tuple[int, int, float]:
    """(optimistic rank, pessimistic rank, the causal gene's own score)."""
    target = phenotype_score(terms, genes[gene], ic, default)
    better = tied = 0
    for other, annotated in genes.items():
        if other == gene:
            continue
        value = phenotype_score(terms, annotated, ic, default)
        if value > target:
            better += 1
        elif value == target:
            tied += 1
    return better + 1, better + tied + 1, target
 def report(ranks: list[tuple[int, int, float]], n_genes: int, min_terms: int = 1) -> str:
    n = len(ranks)
    median_score = statistics.median(t for *_, t in ranks)
    lines = [
        (
            f"{n} published cases with at least {min_terms} HPO term(s), ranked against "
            f"{n_genes} genes (random top-1 would be {1 / n_genes:.2%})"
        ),
        (
            f"the causal gene's own phenotype score: median {median_score:.2f}"
            "  <- 1.00 means HPO already carries every one of the patient's terms for that gene"
        ),
    ]
    for label, column in (("optimistic", 0), ("pessimistic", 1)):
        r = [row[column] for row in ranks]
        lines.append(
            f"  {label:12s} top-1 {sum(x == 1 for x in r) / n:6.1%}   "
            f"top-10 {sum(x <= 10 for x in r) / n:6.1%}   "
            f"MRR {sum(1 / x for x in r) / n:.3f}   median rank {statistics.median(r):.0f}"
        )
    return "\n".join(lines)
 def main() -> None:
    p = argparse.ArgumentParser()
    p.add_argument("--phenopackets", required=True,
                   help="all_phenopackets.zip from a phenopacket-store release")
    p.add_argument("--annotations", required=True,
                   help="TSV of gene<tab>hpo_id, exported from the gene_phenotypes table")
    p.add_argument("--limit", type=int, help="benchmark only the first N cases (a smoke run)")
    p.add_argument("--min-terms", type=int, default=1,
                   help="skip cases with fewer HPO terms; a one-term case can only tie")
    a = p.parse_args()
    genes = gene_annotations(a.annotations)
    ic = information_content(genes)
    default = max(ic.values(), default=1.0)
    ranks = []
    for gene, terms in cases(a.phenopackets):
        if gene in genes and len(terms) >= a.min_terms:
            ranks.append(rank_of(gene, terms, genes, ic, default))
        if a.limit and len(ranks) >= a.limit:
            break
    if not ranks:
        sys.exit("no benchmarkable cases: is --annotations the gene_phenotypes export?")
    print(report(ranks, len(genes), a.min_terms))
 if __name__ == "__main__":
    main()
@@ -2,11 +2,23 @@
 Training imports it, and train.log_and_register ships this package inside the logged pyfunc
 (code_paths), so serving runs exactly this code on the raw columns below.
 **Allele frequency is deliberately not a feature.** It used to be, and it dominated everything:
 the same missense variant scored 0.887 at AF 0 and 0.0003 at AF 0.01, so the model was largely a
 frequency lookup. That caused two problems. It double-counted, because the ranking already scores
 frequency explicitly and auditably in `triage.rarity_score`, putting ~45% of the rank on one
 measurement. And it was circular, because ClinVar's labels are assigned with ACMG criteria that
 call a variant benign *on frequency* (BA1/BS1), so the model was rediscovering the rule used to
 label its own training data — which is most of why the headline AUROC looked so good.
 What is left is the variant's predicted effect: what it does to the protein, and how damaging two
 independent predictors think that is. That is evidence the rest of the ranking does not already
 have, which is the only reason to give the model a weight at all.
 """
 import pandas as pd
 # What serving must send: raw values as stored in the variants table / its annotations.
-RAW_COLUMNS = ["impact", "consequence", "gnomad_af", "cadd_phred", "am_pathogenicity"]
+RAW_COLUMNS = ["impact", "consequence", "cadd_phred", "am_pathogenicity"]
 IMPACT_ORDER = {"MODIFIER": 0, "LOW": 1, "MODERATE": 2, "HIGH": 3}
@@ -14,8 +26,8 @@ IMPACT_ORDER = {"MODIFIER": 0, "LOW": 1, "MODERATE": 2, "HIGH": 3}
 def build(df: pd.DataFrame) -> pd.DataFrame:
    out = pd.DataFrame(index=df.index)
    out["impact_rank"] = df["impact"].map(IMPACT_ORDER).fillna(0).astype(int)
-    # No gnomAD record means the variant was not observed: treat as AF 0.
+    # Left as NaN on purpose: LightGBM handles missing natively, and imputing a number here would
-    out["gnomad_af"] = pd.to_numeric(df["gnomad_af"], errors="coerce").fillna(0.0)
+    # assert a score nobody computed.
    out["cadd_phred"] = pd.to_numeric(df["cadd_phred"], errors="coerce")
    out["am_pathogenicity"] = pd.to_numeric(df["am_pathogenicity"], errors="coerce")
    out["consequence"] = df["consequence"].astype("category")
@@ -0,0 +1,125 @@
 """HPO ontology handling: propagation and information content.
 Shared by scripts/load-hpo.py, which writes the tables the API ranks against, and
 rarelens_ml.benchmark, which scores that ranking. It lives in the package rather than in the
 script so the arithmetic underneath the project's main scientific claim is covered by tests.
 Two ideas, both standard practice and both absent from the first version of the ranking:
 **Propagation.** HPO's gene annotations are direct. A gene linked to "Aortic root aneurysm" is not
 also linked to "Aortic aneurysm", so matching case terms by exact ID missed any patient whose
 description sat one level away from the curator's chosen term. The annotation propagation rule
 says a gene annotated with a term is annotated with all of that term's ancestors; matching then
 works in both directions without the ranking knowing the ontology exists.
 **Information content.** IC(term) = -ln(share of genes carrying it). After propagation almost
 every gene carries "Abnormality of the cardiovascular system", so its IC is near zero, while
 "Dilated left subclavian artery" is worth a great deal. Counting terms alike let a patient's
 "Global developmental delay" count as much as a near-pathognomonic sign.
 """
 import io
 import math
 from collections import defaultdict
 from collections.abc import Iterable
 # Terms outside this branch (inheritance, clinical modifiers, frequency) describe how a disease
 # behaves rather than what is wrong with the patient, and must not count towards a match.
 PHENOTYPIC_ABNORMALITY = "HP:0000118"
 def parse_obo(handle: io.TextIOBase) -> tuple[dict[str, set[str]], dict[str, str]]:
    """Each term's direct parents and its name, from hp.obo. Obsolete terms are dropped."""
    parents: dict[str, set[str]] = {}
    names: dict[str, str] = {}
    term_id: str | None = None
    name: str | None = None
    is_a: set[str] = set()
    obsolete = in_term = False
    def flush() -> None:
        if term_id and not obsolete:
            parents[term_id] = is_a
            names[term_id] = name or term_id
    for raw in handle:
        line = raw.rstrip("\n")
        if line.startswith("["):
            flush()
            term_id, name, is_a, obsolete = None, None, set(), False
            in_term = line == "[Term]"
        elif not in_term:
            continue
        elif line.startswith("id: HP:"):
            term_id = line[4:].strip()
        elif line.startswith("name: "):
            name = line[6:].strip()[:200]
        elif line.startswith("is_a: HP:"):
            is_a.add(line[6:].split("!")[0].strip())
        elif line.startswith("is_obsolete: true"):
            obsolete = True
    flush()
    return parents, names
 def ancestors_of(parents: dict[str, set[str]]) -> dict[str, set[str]]:
    """Every term's ancestors, itself included.
    Iterative, because HPO is deep enough to exhaust the recursion limit, and in true post-order:
    a term is resolved only once every parent is resolved. A pre-order walk read backwards looks
    like it would do, but on a DAG a term can be visited before one of its parents on another
    branch, and then it silently inherits that parent alone instead of the parent's whole
    lineage. That dropped Camptodactyly and Chiari malformation out of the phenotype branch
    entirely, which is what this shape of bug looks like from the outside.
    """
    cache: dict[str, set[str]] = {}
    for start in parents:
        if start in cache:
            continue
        stack: list[tuple[str, bool]] = [(start, False)]
        while stack:
            node, resolved = stack.pop()
            if node in cache:
                continue
            if resolved:
                found = {node}
                for parent in parents.get(node, ()):
                    found |= cache.get(parent, {parent})  # fallback guards against a cycle
                cache[node] = found
            else:
                stack.append((node, True))
                stack.extend((p, False) for p in parents.get(node, ()) if p not in cache)
    return cache
 def propagate(
    direct: Iterable[tuple[str, str]], ancestors: dict[str, set[str]]
 ) -> dict[str, set[str]]:
    """gene -> its annotated terms plus all their ancestors, within the phenotype branch."""
    genes: dict[str, set[str]] = defaultdict(set)
    for gene, term in direct:
        for node in ancestors.get(term, {term}):
            if node != PHENOTYPIC_ABNORMALITY and PHENOTYPIC_ABNORMALITY in ancestors.get(node, ()):
                genes[gene].add(node)
    return dict(genes)
 def information_content(genes: dict[str, set[str]]) -> dict[str, float]:
    """-ln(share of genes carrying the term); 0 for a term every gene has."""
    if not genes:
        return {}
    counts: dict[str, int] = defaultdict(int)
    for terms in genes.values():
        for term in terms:
            counts[term] += 1
    return {term: -math.log(n / len(genes)) for term, n in counts.items()}
 def phenotype_score(
    case_terms: Iterable[str], gene_terms: set[str], ic: dict[str, float], default: float
 ) -> float:
    """Information-content-weighted recall; the same arithmetic as app.services.triage."""
    terms = list(case_terms)
    total = sum(ic.get(t, default) for t in terms)
    if total <= 0:
        return 0.0
    return sum(ic.get(t, default) for t in terms if t in gene_terms) / total
@@ -29,10 +29,7 @@ PARAMS = {
 POS = {"pathogenic", "likely_pathogenic"}
 NEG = {"benign", "likely_benign"}
 # VEP --tab column -> raw feature column (am_pathogenicity already matches).
-VEP_TO_RAW = {
+VEP_TO_RAW = {"IMPACT": "impact", "Consequence": "consequence", "CADD_PHRED": "cadd_phred"}
    "IMPACT": "impact", "Consequence": "consequence", "gnomADe_AF": "gnomad_af",
    "CADD_PHRED": "cadd_phred",
 }
 def label(clin_sig: object) -> int | None:
@@ -0,0 +1,99 @@
 """The benchmark is the project's strongest scientific claim, so its arithmetic is tested."""
 import json
 import math
 import zipfile
 from pathlib import Path
 import pytest
 from rarelens_ml.benchmark import cases, gene_annotations, rank_of, report
 from rarelens_ml.hpo import information_content
 from rarelens_ml.hpo import phenotype_score as score
 def annotations_file(tmp_path: Path, genes: dict[str, list[str]]) -> str:
    path = tmp_path / "gp.tsv"
    path.write_text("".join(f"{g}\t{t}\n" for g, terms in genes.items() for t in terms))
    return str(path)
 def phenopacket(gene: str, terms: list[str], excluded: list[str] | None = None) -> dict:
    return {
        "phenotypicFeatures": [{"type": {"id": t}} for t in terms]
        + [{"type": {"id": t}, "excluded": True} for t in (excluded or [])],
        "interpretations": [
            {
                "diagnosis": {
                    "genomicInterpretations": [
                        {"variantInterpretation": {"variationDescriptor": {
                            "geneContext": {"symbol": gene}}}}
                    ]
                }
            }
        ],
    }
 def store(tmp_path: Path, packets: dict[str, dict]) -> str:
    path = tmp_path / "pps.zip"
    with zipfile.ZipFile(path, "w") as z:
        for name, packet in packets.items():
            z.writestr(name, json.dumps(packet))
    return str(path)
 def test_information_content_makes_a_universal_term_worthless() -> None:
    genes = {"A": {"HP:1", "HP:2"}, "B": {"HP:1"}, "C": {"HP:1"}}
    ic = information_content(genes)
    assert ic["HP:1"] == pytest.approx(0.0)               # every gene has it
    assert ic["HP:2"] == pytest.approx(math.log(3))       # one gene in three
 def test_score_is_recall_weighted_by_specificity() -> None:
    ic = {"HP:1": 0.0, "HP:2": 4.0}
    assert score(["HP:1", "HP:2"], {"HP:2"}, ic, 1.0) == pytest.approx(1.0)
    assert score(["HP:1", "HP:2"], {"HP:1"}, ic, 1.0) == pytest.approx(0.0)
 def test_rank_separates_optimistic_from_pessimistic_on_ties() -> None:
    """Every gene carrying the same term ties; the report must not hide that."""
    genes = {"RIGHT": {"HP:1"}, "TIED": {"HP:1"}, "WRONG": {"HP:9"}}
    ic = {"HP:1": 1.0, "HP:9": 1.0}
    optimistic, pessimistic, target = rank_of("RIGHT", ["HP:1"], genes, ic, 1.0)
    assert (optimistic, pessimistic) == (1, 2)
    assert target == pytest.approx(1.0)
 def test_a_uniquely_matching_gene_ranks_first_either_way() -> None:
    genes = {"RIGHT": {"HP:1", "HP:2"}, "PARTIAL": {"HP:1"}, "WRONG": set()}
    ic = {"HP:1": 1.0, "HP:2": 1.0}
    assert rank_of("RIGHT", ["HP:1", "HP:2"], genes, ic, 1.0)[:2] == (1, 1)
 def test_cases_reads_the_causal_gene_and_drops_excluded_terms(tmp_path: Path) -> None:
    """An excluded feature means the authors looked and did not find it."""
    path = store(tmp_path, {
        "a/one.json": phenopacket("TGFBR2", ["HP:1", "HP:2"], excluded=["HP:3"]),
        "a/notes.txt": {},
    })
    assert list(cases(path)) == [("TGFBR2", ["HP:1", "HP:2"])]
 def test_cases_skips_packets_without_exactly_one_causal_gene(tmp_path: Path) -> None:
    two = phenopacket("A", ["HP:1"])
    two["interpretations"][0]["diagnosis"]["genomicInterpretations"].append(
        {"variantInterpretation": {"variationDescriptor": {"geneContext": {"symbol": "B"}}}}
    )
    path = store(tmp_path, {"two.json": two, "none.json": phenopacket("C", [])})
    assert list(cases(path)) == []
 def test_gene_annotations_reads_the_export(tmp_path: Path) -> None:
    path = annotations_file(tmp_path, {"A": ["HP:1", "HP:2"], "B": ["HP:1"]})
    assert gene_annotations(path) == {"A": {"HP:1", "HP:2"}, "B": {"HP:1"}}
 def test_report_states_the_contamination_and_both_bounds() -> None:
    text = report([(1, 2, 1.0), (1, 1, 1.0), (3, 5, 0.5)], n_genes=100)
    assert "optimistic" in text and "pessimistic" in text
    assert "HPO already carries" in text  # the caveat travels with the number
@@ -9,7 +9,6 @@ def raw(**overrides: list) -> pd.DataFrame:
    base = {
        "impact": ["HIGH", "LOW", None],
        "consequence": ["stop_gained", "synonymous_variant", None],
        "gnomad_af": [None, "0.12", 0.001],
        "cadd_phred": ["35", "2.1", "-"],
        "am_pathogenicity": ["0.98", None, "-"],
    }
@@ -18,13 +17,22 @@ def raw(**overrides: list) -> pd.DataFrame:
 def test_raw_columns_are_the_serving_contract() -> None:
-    assert RAW_COLUMNS == ["impact", "consequence", "gnomad_af", "cadd_phred", "am_pathogenicity"]
+    assert RAW_COLUMNS == ["impact", "consequence", "cadd_phred", "am_pathogenicity"]
 def test_allele_frequency_is_not_a_feature() -> None:
    """It dominated the model and the ranking already scores it, auditably and only once.
    Keeping it here also meant learning ACMG's own frequency-based benign rule from labels that
    rule produced, which is most of why the headline AUROC looked so good.
    """
    assert "gnomad_af" not in RAW_COLUMNS
    assert "gnomad_af" not in build(raw(gnomad_af=[0.0, 0.5, None])).columns
 def test_build_ranks_impact_and_coerces_numbers() -> None:
    out = build(raw())
    assert out["impact_rank"].tolist() == [3, 1, 0]
    assert out["gnomad_af"].tolist() == [0.0, 0.12, 0.001]  # missing AF means absent from gnomAD
    assert out["cadd_phred"].iloc[0] == 35.0
    assert math.isnan(out["cadd_phred"].iloc[2])  # VEP writes "-" for missing
    assert math.isnan(out["am_pathogenicity"].iloc[1])
@@ -0,0 +1,145 @@
 """The ontology arithmetic sits underneath the phenotype half of the ranking, so it is tested."""
 import io
 import math
 import random
 import pytest
 from rarelens_ml.hpo import (
    PHENOTYPIC_ABNORMALITY,
    ancestors_of,
    information_content,
    parse_obo,
    phenotype_score,
    propagate,
 )
 OBO = f"""format-version: 1.2
 [Term]
 id: {PHENOTYPIC_ABNORMALITY}
 name: Phenotypic abnormality
 [Term]
 id: HP:0001
 name: Abnormality of the vasculature
 is_a: {PHENOTYPIC_ABNORMALITY} ! Phenotypic abnormality
 [Term]
 id: HP:0002
 name: Aortic aneurysm
 is_a: HP:0001 ! Abnormality of the vasculature
 [Term]
 id: HP:0003
 name: Aortic root aneurysm
 is_a: HP:0002 ! Aortic aneurysm
 [Term]
 id: HP:0004
 name: Autosomal dominant inheritance
 [Term]
 id: HP:0005
 name: Obsolete thing
 is_a: HP:0001 ! Abnormality of the vasculature
 is_obsolete: true
 """
 def ontology() -> tuple[dict[str, set[str]], dict[str, str]]:
    return parse_obo(io.StringIO(OBO))
 def test_parse_obo_reads_parents_and_drops_obsolete_terms() -> None:
    parents, names = ontology()
    assert parents["HP:0003"] == {"HP:0002"}
    assert names["HP:0002"] == "Aortic aneurysm"
    assert "HP:0005" not in parents
 def test_ancestors_include_the_term_itself_and_the_whole_lineage() -> None:
    ancestors = ancestors_of(ontology()[0])
    assert ancestors["HP:0003"] == {"HP:0003", "HP:0002", "HP:0001", PHENOTYPIC_ABNORMALITY}
    assert ancestors["HP:0004"] == {"HP:0004"}  # its own branch, not under phenotypic abnormality
 def closure(parents: dict[str, set[str]]) -> dict[str, set[str]]:
    """Reference transitive closure by relaxation: obviously correct, too slow for 20k terms."""
    result = {node: {node} | set(ps) for node, ps in parents.items()}
    changed = True
    while changed:
        changed = False
        for node, found in result.items():
            grown = set(found)
            for parent in found - {node}:
                grown |= result.get(parent, {parent})
            if grown != found:
                result[node] = grown
                changed = True
    return result
 def test_ancestors_match_a_reference_closure_on_a_tangled_dag() -> None:
    """The regression this guards cost 399 HPO terms, Camptodactyly and Chiari malformation among
    them: on a DAG a term can be reached before one of its parents, and the old walk then gave it
    that parent alone instead of the parent's whole lineage. It only shows up when a node shares
    ancestors by several routes, so the test needs a genuinely tangled graph rather than a
    hand-drawn diamond.
    """
    rng = random.Random(0)
    nodes = [PHENOTYPIC_ABNORMALITY] + [f"HP:{i:04d}" for i in range(1, 80)]
    parents = {PHENOTYPIC_ABNORMALITY: set()}
    for i, node in enumerate(nodes[1:], start=1):
        # only earlier nodes may be parents, which keeps it acyclic
        parents[node] = set(rng.sample(nodes[:i], k=min(i, rng.randint(1, 3))))
    for _ in range(5):  # dict order decides the traversal, so try several
        shuffled = list(parents.items())
        rng.shuffle(shuffled)
        assert ancestors_of(dict(shuffled)) == closure(dict(shuffled))
 def test_every_descendant_of_the_root_keeps_the_root() -> None:
    """The property that actually matters: losing it drops the term out of the phenotype branch."""
    parents = {
        PHENOTYPIC_ABNORMALITY: set(),
        "HP:P": {PHENOTYPIC_ABNORMALITY},
        "HP:X": {"HP:P"},
        "HP:N": {"HP:P"},
        "HP:A": {"HP:X", "HP:N"},
    }
    ancestors = ancestors_of(parents)
    for term in ("HP:P", "HP:X", "HP:N", "HP:A"):
        assert PHENOTYPIC_ABNORMALITY in ancestors[term], term
 def test_propagation_lets_a_parent_term_match_a_gene_annotated_with_a_child() -> None:
    ancestors = ancestors_of(ontology()[0])
    genes = propagate([("TGFBR2", "HP:0003")], ancestors)
    assert genes["TGFBR2"] == {"HP:0003", "HP:0002", "HP:0001"}
 def test_propagation_drops_the_root_and_anything_outside_the_phenotype_branch() -> None:
    ancestors = ancestors_of(ontology()[0])
    genes = propagate([("A", "HP:0003"), ("A", "HP:0004")], ancestors)
    assert PHENOTYPIC_ABNORMALITY not in genes["A"]  # every gene has it; it carries no information
    assert "HP:0004" not in genes["A"]               # inheritance is not a patient finding
 def test_information_content_is_zero_for_a_term_every_gene_carries() -> None:
    ic = information_content({"A": {"HP:1", "HP:2"}, "B": {"HP:1"}, "C": {"HP:1"}})
    assert ic["HP:1"] == pytest.approx(0.0)
    assert ic["HP:2"] == pytest.approx(math.log(3))
 def test_phenotype_score_weights_by_specificity() -> None:
    ic = {"HP:common": 0.1, "HP:rare": 6.0}
    terms = ["HP:common", "HP:rare"]
    assert phenotype_score(terms, {"HP:rare"}, ic, 1.0) == pytest.approx(6.0 / 6.1)
    assert phenotype_score(terms, {"HP:common"}, ic, 1.0) == pytest.approx(0.1 / 6.1)
 def test_phenotype_score_treats_an_unscored_term_as_maximally_specific() -> None:
    """It can never match, so it must depress every gene equally rather than vanish."""
    assert phenotype_score(["HP:1", "HP:unknown"], {"HP:1"}, {"HP:1": 5.0}, 5.0) == pytest.approx(0.5)
@@ -90,7 +90,25 @@ def to_rows(df: pd.DataFrame, job_id: str) -> list[dict]:
    return rows
-def load(engine: Engine, job_id: str, rows: list[dict], vep: str | None) -> None:
+FREQUENCY_COLUMNS = ("gnomADe_AF", "gnomADg_AF", "AF")
 EFFECT_COLUMNS = ("CADD_PHRED", "am_pathogenicity")
 def sources(df: pd.DataFrame) -> dict[str, bool]:
    """Which lines of evidence this run looked up at all.
    Column *presence*, not a non-empty value: a variant absent from gnomAD is strong evidence of
    rarity, but only when gnomAD was consulted. VEP's database mode emits no frequency column at
    all, and the API must be able to tell the two apart instead of scoring both as maximally rare.
    """
    return {
        "has_frequencies": any(c in df.columns for c in FREQUENCY_COLUMNS),
        "has_effect_scores": any(c in df.columns for c in EFFECT_COLUMNS),
    }
 def load(engine: Engine, job_id: str, rows: list[dict], vep: str | None,
         evidence: dict[str, bool]) -> None:
    with engine.begin() as conn:
        conn.execute(text("DELETE FROM variants WHERE job_id = :id"), {"id": job_id})
        for start in range(0, len(rows), INSERT_CHUNK):
@@ -106,10 +124,12 @@ def load(engine: Engine, job_id: str, rows: list[dict], vep: str | None) -> None
        conn.execute(
            text("""
                UPDATE jobs SET status = 'succeeded', vep_version = :vep, log = NULL,
                                has_frequencies = :has_frequencies,
                                has_effect_scores = :has_effect_scores,
                                finished_at = now()
                WHERE id = :id
            """),
-            {"id": job_id, "vep": vep},
+            {"id": job_id, "vep": vep, **evidence},
        )
@@ -120,14 +140,16 @@ def main() -> None:
    p.add_argument("--dry-run", action="store_true", help="parse only; do not touch the database")
    a = p.parse_args()
-    rows = to_rows(read_vep_tab(a.tsv), job_id=a.job_id)
+    df = read_vep_tab(a.tsv)
    rows = to_rows(df, job_id=a.job_id)
    evidence = sources(df)
    if a.dry_run:
-        print(f"{len(rows)} variants parsed (dry run, no DB)")
+        print(f"{len(rows)} variants parsed (dry run, no DB); {evidence}")
        return
    url = os.environ.get("DATABASE_URL")
    if not url:
        sys.exit("DATABASE_URL is not set")
-    load(engine_for(url), a.job_id, rows, vep=vep_version(a.tsv))
+    load(engine_for(url), a.job_id, rows, vep=vep_version(a.tsv), evidence=evidence)
    print(f"loaded {len(rows)} variants for job {a.job_id}", file=sys.stderr)
@@ -16,6 +16,8 @@ CREATE TABLE jobs (
    id uuid PRIMARY KEY,
    status jobstatus NOT NULL,
    vep_version text,
    has_frequencies boolean NOT NULL DEFAULT false,
    has_effect_scores boolean NOT NULL DEFAULT false,
    log text,
    finished_at timestamptz
 );
@@ -3,7 +3,15 @@ import uuid
 from pathlib import Path
 import pytest
-from load_db import engine_for, load, parse_variant_id, read_vep_tab, to_rows, vep_version
+from load_db import (
    engine_for,
    load,
    parse_variant_id,
    read_vep_tab,
    sources,
    to_rows,
    vep_version,
 )
 from set_job_status import set_status
 from sqlalchemy import text
@@ -11,6 +19,8 @@ HEADER = [
    "Uploaded_variation", "Location", "Allele", "Consequence", "IMPACT", "SYMBOL",
    "HGVSc", "HGVSp", "gnomADe_AF", "CLIN_SIG", "CADD_PHRED",
 ]
 # A run with a VEP cache and plugins: the fixture header carries gnomADe_AF and CADD_PHRED.
 CACHE_RUN = {"has_frequencies": True, "has_effect_scores": True}
 # Longer than the old VARCHAR(120) column.
 LONG_CLIN_SIG = (
    "conflicting_classifications_of_pathogenicity,uncertain_significance,"
@@ -96,18 +106,40 @@ def job_and_count(engine, job_id: str) -> tuple:
 def test_load_is_idempotent_and_marks_job_succeeded(engine, tmp_path: Path) -> None:
    job_id = new_job(engine)
    rows = to_rows(read_vep_tab(vep_tab(tmp_path, ROWS)), job_id=job_id)
-    load(engine, job_id, rows, vep="113.0")
+    load(engine, job_id, rows, vep="113.0", evidence=CACHE_RUN)
-    load(engine, job_id, rows, vep="113.0")  # a retried task must not duplicate variants
+    # a retried task must not duplicate variants
    load(engine, job_id, rows, vep="113.0", evidence=CACHE_RUN)
    assert job_and_count(engine, job_id) == ("succeeded", "113.0", None, 2)
 def test_load_with_no_variants_still_succeeds(engine, tmp_path: Path) -> None:
    job_id = new_job(engine)
    rows = to_rows(read_vep_tab(vep_tab(tmp_path, [])), job_id=job_id)
-    load(engine, job_id, rows, vep="113.0")
+    load(engine, job_id, rows, vep="113.0", evidence=CACHE_RUN)
    assert job_and_count(engine, job_id) == ("succeeded", "113.0", None, 0)
 def test_load_records_which_evidence_the_run_looked_up(engine, tmp_path: Path) -> None:
    """The API must be able to tell "absent from gnomAD" from "nobody asked gnomAD"."""
    job_id = new_job(engine)
    df = read_vep_tab(vep_tab(tmp_path, ROWS))
    load(engine, job_id, to_rows(df, job_id=job_id), vep="113.0", evidence=sources(df))
    with engine.begin() as conn:
        flags = conn.execute(
            text("SELECT has_frequencies, has_effect_scores FROM jobs WHERE id=:id"),
            {"id": job_id},
        ).one()
    assert flags == (True, True)  # this fixture has gnomADe_AF and CADD_PHRED columns
 def test_a_database_mode_run_reports_no_frequencies(engine, tmp_path: Path) -> None:
    """VEP --database emits neither a frequency column nor plugin scores."""
    path = tmp_path / "db.vep.tsv"
    columns = [c for c in HEADER if c not in ("gnomADe_AF", "CADD_PHRED")]
    path.write_text("## ENSEMBL VARIANT EFFECT PREDICTOR v113.0\n#" + "\t".join(columns) + "\n")
    assert sources(read_vep_tab(path)) == {"has_frequencies": False, "has_effect_scores": False}
 def test_set_status_failed_records_the_reason(engine) -> None:
    job_id = new_job(engine)
    set_status(engine, job_id, "failed", "workflow annotate-abc Failed")
@@ -1,8 +1,15 @@
 #!/usr/bin/env python3
-"""Load HPO's gene-to-phenotype annotations into the gene_phenotypes table.
+"""Load HPO into the gene_phenotypes and hpo_terms tables: the reference data the ranking uses.
-This is the reference data the ranking matches a case's phenotype against. Source file:
+Two source files:
-https://purl.obolibrary.org/obo/hp/hpoa/genes_to_phenotype.txt (HPO release, ~20 MB).
+
  https://purl.obolibrary.org/obo/hp/hpoa/genes_to_phenotype.txt   (gene -> term, ~20 MB)
  https://purl.obolibrary.org/obo/hp.obo                           (the ontology itself, ~10 MB)
 The ontology matters because HPO's gene annotations are direct, and a patient may be described one
 level away from whichever term the curator chose. `rarelens_ml.hpo` holds the propagation and
 information-content arithmetic, with the tests, since it is what the phenotype half of the ranking
 rests on. Run through the ml environment, which is where that package lives: `make hpo`.
 Cite the Human Phenotype Ontology when showing results; see docs/data.md.
 """
@@ -13,52 +20,67 @@ import os
 import sys
 import urllib.request
 from rarelens_ml.hpo import ancestors_of, information_content, parse_obo, propagate
 from sqlalchemy import create_engine, text
 from sqlalchemy.engine import make_url
-URL = "https://purl.obolibrary.org/obo/hp/hpoa/genes_to_phenotype.txt"
+GENES_URL = "https://purl.obolibrary.org/obo/hp/hpoa/genes_to_phenotype.txt"
 OBO_URL = "https://purl.obolibrary.org/obo/hp.obo"
-def rows(handle: io.TextIOBase) -> list[tuple[str, str, str]]:
+def fetch(url: str) -> io.TextIOBase:
    print(f"downloading {url}", file=sys.stderr)
    return io.TextIOWrapper(urllib.request.urlopen(url), encoding="utf-8")
 def annotations(handle: io.TextIOBase) -> set[tuple[str, str]]:
    """Unique (gene, term) pairs; the file repeats them once per associated disease."""
-    seen: set[tuple[str, str]] = set()
+    return {
-    out: list[tuple[str, str, str]] = []
+        (row["gene_symbol"][:60], row["hpo_id"][:20])
-    for row in csv.DictReader(handle, delimiter="\t"):
+        for row in csv.DictReader(handle, delimiter="\t")
-        gene, hpo_id, name = row["gene_symbol"], row["hpo_id"], row["hpo_name"]
+        if row.get("gene_symbol") and row.get("hpo_id")
-        if not gene or not hpo_id or (gene, hpo_id) in seen:
+    }
            continue
        seen.add((gene, hpo_id))
        out.append((gene[:60], hpo_id[:20], name[:200]))
    return out
 def main() -> None:
    p = argparse.ArgumentParser()
-    p.add_argument("--url", default=URL)
+    p.add_argument("--url", default=GENES_URL)
-    p.add_argument("--file", help="use a local copy instead of downloading")
+    p.add_argument("--obo", default=OBO_URL)
    p.add_argument("--file", help="use a local genes_to_phenotype.txt instead of downloading")
    p.add_argument("--obo-file", help="use a local hp.obo instead of downloading")
    a = p.parse_args()
    url = os.environ.get("DATABASE_URL")
    if not url:
        sys.exit("DATABASE_URL is not set")
-    if a.file:
+    with (open(a.obo_file) if a.obo_file else fetch(a.obo)) as handle:
-        with open(a.file) as fh:
+        parents, names = parse_obo(handle)
-            annotations = rows(fh)
+    ancestors = ancestors_of(parents)
-    else:
+    with (open(a.file) if a.file else fetch(a.url)) as handle:
-        print(f"downloading {a.url}", file=sys.stderr)
+        direct = annotations(handle)
-        with urllib.request.urlopen(a.url) as response:  # noqa: S310 - fixed HPO release URL
+
-            annotations = rows(io.TextIOWrapper(response, encoding="utf-8"))
+    genes = propagate(direct, ancestors)
-    print(f"{len(annotations)} gene/term pairs", file=sys.stderr)
+    ic = information_content(genes)
    rows = [(gene, term, names.get(term, term)) for gene, terms in genes.items() for term in terms]
    print(
        f"{len(direct)} direct annotations over {len(genes)} genes -> {len(rows)} after "
        f"propagation; {len(ic)} terms with information content",
        file=sys.stderr,
    )
    engine = create_engine(make_url(url).set(drivername="postgresql+psycopg"))
    with engine.begin() as conn:
        conn.execute(text("TRUNCATE gene_phenotypes RESTART IDENTITY"))
        conn.execute(text("TRUNCATE hpo_terms"))
        cursor = conn.connection.cursor()
        with cursor.copy("COPY gene_phenotypes (gene_symbol, hpo_id, hpo_name) FROM STDIN") as copy:
-            for row in annotations:
+            for row in rows:
                copy.write_row(row)
-    print(f"loaded {len(annotations)} annotations", file=sys.stderr)
+        with cursor.copy("COPY hpo_terms (hpo_id, name, ic) FROM STDIN") as copy:
            for term, value in ic.items():
                copy.write_row((term, names.get(term, term), value))
    print(f"loaded {len(rows)} annotations and {len(ic)} terms", file=sys.stderr)
 if __name__ == "__main__":
@@ -100,6 +100,8 @@ td.coord, td.hgvs { font-family: var(--mono); font-size: 0.85rem; }  /* aligned
 .components { border: none; background: none; }
 .components th { font-weight: 400; color: var(--ink-soft); padding: 0.15rem 0.5rem 0.15rem 0; }
 .components td { padding: 0.15rem 0; font-size: 0.82rem; text-align: right; }
 /* A component with no evidence behind it is stated, not drawn as a number. */
 .components tr.abstained th, .components tr.abstained td { color: var(--ink-soft); font-style: italic; }
 .evidence { display: grid; grid-template-columns: 7rem 1fr; gap: 0.2rem 0.5rem; margin: 0; font-size: 0.88rem; }
 .evidence dt { color: var(--ink-soft); }
 .evidence dd { margin: 0; }
@@ -31,17 +31,28 @@ export type Decision = {
 export type Candidate = {
  variant: Variant;
  score: number;
-  components: Record<string, number>;
+  // null means that evidence was never looked up, so it did not enter the score. Not a zero.
  components: Record<string, number | null>;
  matched_terms: PhenotypeTerm[];
  scored: boolean;
  decision: Decision | null;
 };
 export type VariantDetail = Candidate & { annotations: Record<string, string> };
-export type Funnel = { total: number; rare: number; candidates: number; phenotype_matched: number };
+export type Funnel = {
  total: number;
  rare: number;
  candidates: number;
  phenotype_matched: number;
  frequencies: boolean;
 };
 /** What the annotation run produced, and so which components were allowed to score. */
 export type Evidence = { frequencies: boolean; effect_scores: boolean; missing: string[] };
 export type CandidatePage = {
  funnel: Funnel;
  evidence: Evidence;
  weights: Record<string, number>;
  items: Candidate[];
  total: number;
@@ -1,6 +1,23 @@
 import { describe, expect, it } from 'vitest';
-import { barWidth, candidateQuery, evidenceChips, funnelSteps, needsScoring, plural, scoreBarPercent } from './candidates';
+import {
-import type { Candidate, Funnel } from './api';
+  barWidth,
  candidateQuery,
  evidenceChips,
  funnelSteps,
  missingEvidenceNote,
  needsScoring,
  plural,
  scoreBarPercent
 } from './candidates';
 import type { Candidate, Evidence, Funnel } from './api';
 // A run with a VEP cache and plugins; and one in VEP's database mode, which has neither.
 const FULL: Evidence = { frequencies: true, effect_scores: true, missing: [] };
 const DATABASE_ONLY: Evidence = {
  frequencies: false,
  effect_scores: false,
  missing: ['rarity', 'model']
 };
 const candidate = (over: Partial<Candidate> = {}): Candidate => ({
  variant: {
@@ -55,7 +72,9 @@ describe('evidenceChips', () => {
 describe('funnelSteps', () => {
  it('describes each narrowing step in order', () => {
-    const funnel: Funnel = { total: 1284, rare: 41, candidates: 12, phenotype_matched: 6 };
+    const funnel: Funnel = {
      total: 1284, rare: 41, candidates: 12, phenotype_matched: 6, frequencies: true
    };
    expect(funnelSteps(funnel).map((s) => `${s.label} ${s.value}`)).toEqual([
      'variants called 1284',
      'rare (<0.1%) 41',
@@ -63,6 +82,43 @@ describe('funnelSteps', () => {
      'in phenotype-matched genes 6'
    ]);
  });
  it('does not call the step "rare" when no frequency was ever looked up', () => {
    const funnel: Funnel = {
      total: 21, rare: 21, candidates: 2, phenotype_matched: 1, frequencies: false
    };
    expect(funnelSteps(funnel)[1]).toEqual({ label: 'rarity not checked', value: 21 });
  });
 });
 describe('missing evidence', () => {
  it('says nothing when every line of evidence was looked up', () => {
    expect(missingEvidenceNote(FULL)).toBeNull();
  });
  it('names what is missing and why, rather than showing a silent zero', () => {
    const note = missingEvidenceNote(DATABASE_ONLY) ?? '';
    expect(note).toContain('no allele frequencies');
    expect(note).toContain('no CADD or AlphaMissense');
    expect(note).toContain('reweighted');
  });
  it('will not claim a variant is absent from gnomAD when gnomAD was never consulted', () => {
    const chips = evidenceChips(candidate(), 3, DATABASE_ONLY);
    expect(chips.map((c) => c.label)).toContain('frequency not checked');
    expect(chips.map((c) => c.label)).not.toContain('absent from gnomAD');
  });
  it('still reports a real gnomAD absence when frequencies were looked up', () => {
    const chips = evidenceChips(candidate(), 3, FULL);
    expect(chips.map((c) => c.label)).toContain('absent from gnomAD');
  });
  it('does not offer scoring when the model would abstain anyway', () => {
    const items = [candidate({ scored: false })];
    expect(needsScoring(items, FULL)).toBe(true);
    expect(needsScoring(items, DATABASE_ONLY)).toBe(false);
  });
 });
 describe('scoreBarPercent', () => {
@@ -1,4 +1,4 @@
-import type { Candidate, Funnel } from './api';
+import type { Candidate, Evidence, Funnel } from './api';
 export type Chip = {
  label: string;
@@ -20,23 +20,36 @@ const formatAf = (af: number) => af.toExponential(1);
 * The reasons this variant is a candidate, in the order a reviewer reads them. ClinVar comes last
 * and is styled apart, because it confirms the ranking rather than feeding it.
 */
-export function evidenceChips(candidate: Candidate, caseTermCount: number): Chip[] {
+export function evidenceChips(
  candidate: Candidate,
  caseTermCount: number,
  evidence?: Evidence
 ): Chip[] {
  const { variant } = candidate;
  const chips: Chip[] = [
    candidate.matched_terms.length
      ? { label: `${candidate.matched_terms.length}/${caseTermCount} phenotype terms`, tone: 'match' }
-      : { label: 'no phenotype match', tone: 'muted' },
+      : { label: 'no phenotype match', tone: 'muted' }
-    variant.gnomad_af === null
+  ];
-      ? { label: 'absent from gnomAD', tone: 'rare' }
+  // "absent from gnomAD" is a claim about gnomAD. Only make it if gnomAD was actually consulted.
-      : { label: `gnomAD ${formatAf(variant.gnomad_af)}`, tone: 'rare' },
+  if (evidence && !evidence.frequencies) {
-    {
+    chips.push({ label: 'frequency not checked', tone: 'muted' });
  } else if (variant.gnomad_af === null) {
    chips.push({ label: 'absent from gnomAD', tone: 'rare' });
  } else {
    chips.push({ label: `gnomAD ${formatAf(variant.gnomad_af)}`, tone: 'rare' });
  }
  chips.push({
    label: (variant.consequence ?? 'unknown consequence').split('&')[0].replace(/_/g, ' '),
    tone: 'impact'
-    },
+  });
-    candidate.scored && variant.prediction
+  if (evidence && !evidence.effect_scores) {
-      ? { label: `model ${variant.prediction.score.toFixed(2)}`, tone: 'model' }
+    chips.push({ label: 'model not used', tone: 'muted' });
-      : { label: 'unscored', tone: 'muted' }
+  } else if (candidate.scored && variant.prediction) {
-  ];
+    chips.push({ label: `model ${variant.prediction.score.toFixed(2)}`, tone: 'model' });
  } else {
    chips.push({ label: 'unscored', tone: 'muted' });
  }
  if (variant.clinvar_sig) chips.push({ label: `ClinVar: ${variant.clinvar_sig}`, tone: 'clinvar' });
  return chips;
 }
@@ -45,12 +58,22 @@ export function evidenceChips(candidate: Candidate, caseTermCount: number): Chip
 export function funnelSteps(funnel: Funnel): { label: string; value: number }[] {
  return [
    { label: 'variants called', value: funnel.total },
-    { label: 'rare (<0.1%)', value: funnel.rare },
+    // Without frequencies this step cannot filter; saying "rare" would claim it had.
    { label: funnel.frequencies ? 'rare (<0.1%)' : 'rarity not checked', value: funnel.rare },
    { label: 'coding candidates', value: funnel.candidates },
    { label: 'in phenotype-matched genes', value: funnel.phenotype_matched }
  ];
 }
 /** Why a component is absent from the score, in the reviewer's language rather than the schema's. */
 export function missingEvidenceNote(evidence: Evidence): string | null {
  if (!evidence.missing.length) return null;
  const reasons: string[] = [];
  if (!evidence.frequencies) reasons.push('no allele frequencies (VEP ran without its cache)');
  if (!evidence.effect_scores) reasons.push('no CADD or AlphaMissense scores');
  return `This run has ${reasons.join(' and ')}, so ${evidence.missing.join(' and ')} did not score. The remaining evidence was reweighted to make up the difference.`;
 }
 export const scoreBarPercent = (score: number): number =>
  Math.max(0, Math.min(100, Math.round(score * 100)));
@@ -77,5 +100,9 @@ export function barWidth(value: number, total: number): number {
 export const plural = (n: number, noun: string): string => `${n} ${noun}${n === 1 ? '' : 's'}`;
-/** True when a case was analysed before a model existed: scoring can be run on its own. */
+/**
-export const needsScoring = (items: Candidate[]): boolean => items.some((c) => !c.scored);
+ * True when a case was analysed before a model existed and scoring is worth running on its own.
 * Pointless when the model has no feature the ranking lacks, because it would abstain anyway.
 */
 export const needsScoring = (items: Candidate[], evidence?: Evidence): boolean =>
  (!evidence || evidence.effect_scores) && items.some((c) => !c.scored);
@@ -1,5 +1,5 @@
 <script lang="ts">
-  import type { Candidate } from '$lib/api';
+  import type { Candidate, Evidence } from '$lib/api';
  import { evidenceChips, scoreBarPercent } from '$lib/candidates';
  import Chips from './Chips.svelte';
@@ -7,12 +7,14 @@
    candidate,
    rank,
    termCount,
    evidence,
    selected = false,
    onselect
  }: {
    candidate: Candidate;
    rank: number;
    termCount: number;
    evidence?: Evidence;
    selected?: boolean;
    onselect: () => void;
  } = $props();
@@ -27,7 +29,7 @@
    <span class="coord">{v.chrom}:{v.pos} {v.ref}&gt;{v.alt}</span>
    {#if v.hgvsp ?? v.hgvsc}<span class="hgvs">{v.hgvsp ?? v.hgvsc}</span>{/if}
  </span>
-  <Chips chips={evidenceChips(candidate, termCount)} />
+  <Chips chips={evidenceChips(candidate, termCount, evidence)} />
  <span class="rankscore">
    <span class="scorebar"><span style="width: {scoreBarPercent(candidate.score)}%"></span></span>
    <span class="scorenum">{candidate.score.toFixed(2)}</span>
@@ -1,5 +1,5 @@
 <script lang="ts">
-  import { api, type DecisionState, type VariantDetail } from '$lib/api';
+  import { api, type DecisionState, type Evidence, type VariantDetail } from '$lib/api';
  import { evidenceChips } from '$lib/candidates';
  import Chips from './Chips.svelte';
@@ -7,12 +7,14 @@
    detail,
    termCount,
    weights,
    evidence,
    onclose,
    ondecided
  }: {
    detail: VariantDetail;
    termCount: number;
    weights: Record<string, number>;
    evidence?: Evidence;
    onclose: () => void;
    ondecided: (updated: VariantDetail) => void;
  } = $props();
@@ -66,16 +68,20 @@
    <button class="quiet" onclick={onclose} aria-label="Close panel">✕</button>
  </header>
-  <Chips chips={evidenceChips(detail, termCount)} />
+  <Chips chips={evidenceChips(detail, termCount, evidence)} />
  <h4>Why it ranks {detail.score.toFixed(2)}</h4>
  <table class="components">
    <tbody>
      {#each Object.entries(detail.components) as [name, value] (name)}
-        <tr>
+        <tr class:abstained={value === null}>
          <th>{name}</th>
          {#if value === null}
            <td class="coord" colspan="2">not looked up — did not score</td>
          {:else}
            <td class="coord">{value.toFixed(2)} × {(weights[name] ?? 0).toFixed(2)}</td>
            <td class="coord">{(value * (weights[name] ?? 0)).toFixed(3)}</td>
          {/if}
        </tr>
      {/each}
    </tbody>
@@ -3,7 +3,7 @@
  import { api, type Candidate, type CandidatePage, type Case, type Job, type VariantDetail } from '$lib/api';
  import { poll } from '$lib/poll';
  import { formatElapsed, latestStep } from '$lib/progress';
-  import { needsScoring, plural } from '$lib/candidates';
+  import { missingEvidenceNote, needsScoring, plural } from '$lib/candidates';
  import CandidateRow from '$lib/components/CandidateRow.svelte';
  import Funnel from '$lib/components/Funnel.svelte';
  import VariantPanel from '$lib/components/VariantPanel.svelte';
@@ -33,7 +33,8 @@
  const termCount = $derived(kase?.phenotypes.length ?? 0);
  // Annotation is expensive and scoring is not: a case annotated before a model existed should
  // not need a five-minute re-run of VEP to get its score.
-  const unscored = $derived(!!page && page.items.length > 0 && needsScoring(page.items));
+  const unscored = $derived(!!page && page.items.length > 0 && needsScoring(page.items, page.evidence));
  const evidenceNote = $derived(page ? missingEvidenceNote(page.evidence) : null);
  // Tick the elapsed time while a run is in flight; polling refreshes the step itself.
  $effect(() => {
@@ -146,7 +147,10 @@
 {#if error}<p role="alert">{error}</p>{/if}
 {#if scoreNote}
-  <p class="note">Variants are unscored, so the model term contributes 0 to every rank. {scoreNote}</p>
+  <p class="note">{scoreNote}</p>
 {/if}
 {#if evidenceNote}
  <p class="note">{evidenceNote}</p>
 {/if}
 {#if running}
@@ -196,6 +200,7 @@
          {candidate}
          rank={i + 1}
          {termCount}
          evidence={page.evidence}
          selected={detail?.variant.id === candidate.variant.id}
          onselect={() => select(candidate)} />
      {:else}
@@ -207,6 +212,7 @@
        {detail}
        {termCount}
        weights={page.weights}
        evidence={page.evidence}
        onclose={() => (detail = null)}
        ondecided={decided} />
    {/if}