feat: redesign around phenotype-driven triage, not variant filtering

A table with filters made the user do the work. Rare disease triage is a different task:
which few variants could explain *this* patient's phenotype, and why. The app now answers
that, and lets a reviewer act on the answer.

Domain
- a case is a proband: a VCF plus the HPO terms observed in the patient (samples -> cases)
- HPO's gene-to-phenotype annotations are loaded as reference data (scripts/load-hpo.py)
- each candidate can be shortlisted or dismissed with a reason and a note

Ranking (app/services/triage.py, 21 tests)
- weighted sum of phenotype match, rarity, consequence severity and the model's score,
  with every component shown next to the candidate
- rarity and consequence filter; phenotype only ranks, because a real diagnosis can sit in
  a gene nobody has annotated yet and filtering on it would hide exactly that case
- ClinVar is deliberately not an input: it appears beside the result as independent
  confirmation, so nothing ranks highly merely because ClinVar already said pathogenic

UI
- the funnel is the headline: variants called -> rare -> coding candidates -> phenotype-matched
- ranked candidates with evidence chips, not a grid of everything; filters are demoted
- a variant panel showing the score breakdown, the matched HPO terms, the raw VEP record and
  links out to Ensembl/gnomAD/ClinVar, with the decision controls
- a printable case report: phenotype, funnel, shortlisted variants with reasons, provenance

API: /cases with phenotypes, /cases/{id}/candidates (funnel + ranked + weights),
/variants/{id}, /variants/{id}/decision, /cases/{id}/report, /phenotypes for the picker.
Scoring moved under the case and now answers 503 with the reason when no model registry is
reachable, instead of a 500.

Verified end to end on a simulated proband (scripts/make-demo-case.sh: real GIAB HG002
background + one real ClinVar 2-star pathogenic NF2 variant). 13 variants called -> 1 coding
candidate, and the planted variant ranks first at 0.80 on phenotype 1.00, rarity 1.00 and
consequence 1.00, with ClinVar agreeing afterwards.

Tests: api 75, ml 18, loader 16, web 27; ruff, mypy, svelte-check, terraform validate, both
kustomize overlays and the Nextflow stub run all clean.
This commit is contained in:
Kemal Yaylali
2026-09-12 08:30:44 +01:00
parent abde5ec6e4
commit 07a01715fd
47 changed files with 2159 additions and 539 deletions
+57 -33
View File
@@ -1,18 +1,20 @@
<script lang="ts">
import { onMount } from 'svelte';
import { api, type Assembly, type Sample } from '$lib/api';
import { api, type Assembly, type Case, type PhenotypeTerm } from '$lib/api';
import PhenotypePicker from '$lib/components/PhenotypePicker.svelte';
let samples = $state<Sample[]>([]);
let cases = $state<Case[]>([]);
let loaded = $state(false);
let loadError = $state<string | null>(null);
let name = $state('');
let vcfUri = $state('');
let assembly = $state<Assembly>('GRCh38');
let phenotypes = $state<PhenotypeTerm[]>([]);
let error = $state<string | null>(null);
async function refresh() {
try {
samples = await api.samples();
cases = await api.cases();
loadError = null;
} catch (e) {
loadError = (e as Error).message;
@@ -24,44 +26,66 @@
async function add() {
error = null;
try { await api.createSample({ name, vcf_uri: vcfUri, assembly }); name = ''; vcfUri = ''; await refresh(); }
catch (e) { error = (e as Error).message; }
try {
await api.createCase({ name, vcf_uri: vcfUri, assembly, phenotypes });
name = '';
vcfUri = '';
phenotypes = [];
await refresh();
} catch (e) {
error = (e as Error).message;
}
}
const status = (c: Case) => c.latest_job?.status ?? 'not analysed';
</script>
<h1>rarelens</h1>
<p class="lede">Annotate a VCF with Ensembl VEP, score each variant, and browse what came back. Public test data only.</p>
<p class="lede">
Rare disease triage on public data: a proband's variants narrowed against their phenotype, each
candidate carrying the evidence for its rank. Research demo, not a diagnostic tool.
</p>
<h2>Add a sample</h2>
<div style="display:flex; gap:0.5rem; flex-wrap:wrap">
<input placeholder="Sample name" bind:value={name} aria-label="Sample name" />
<input placeholder="gs://bucket/sample.vcf.gz or /data/example.vcf.gz" bind:value={vcfUri} aria-label="VCF path" style="flex:1; min-width: 20rem" />
<select bind:value={assembly} aria-label="Assembly">
<option>GRCh38</option><option>GRCh37</option>
</select>
<button onclick={add} disabled={!name || !vcfUri}>Add sample</button>
<h2>New case</h2>
<div class="newcase">
<div class="row">
<input placeholder="Case name, e.g. PROBAND-01" bind:value={name} aria-label="Case name" />
<input
placeholder="gs://bucket/proband.vcf.gz or /data/proband.vcf.gz"
bind:value={vcfUri}
aria-label="VCF path"
style="flex:1; min-width: 18rem" />
<select bind:value={assembly} aria-label="Assembly">
<option>GRCh38</option><option>GRCh37</option>
</select>
</div>
<PhenotypePicker bind:selected={phenotypes} />
<button onclick={add} disabled={!name || !vcfUri}>Create case</button>
</div>
{#if error}<p role="alert">Could not add the sample: {error}</p>{/if}
{#if error}<p role="alert">Could not create the case: {error}</p>{/if}
<h2>Samples</h2>
<h2>Cases</h2>
{#if loadError}
<p role="alert">Could not load samples: {loadError}. <button class="quiet" onclick={refresh}>Retry</button></p>
<p role="alert">Could not load cases: {loadError}. <button class="quiet" onclick={refresh}>Retry</button></p>
{:else if !loaded}
<p style="color:var(--ink-soft)">Loading samples</p>
{:else if samples.length === 0}
<div class="empty">No samples yet. Add one above to run the annotation pipeline.</div>
<p class="muted">Loading…</p>
{:else if cases.length === 0}
<div class="empty">No cases yet. Create one above, then run the annotation pipeline on it.</div>
{:else}
<table>
<thead><tr><th>Name</th><th>VCF</th><th>Assembly</th><th>Added</th></tr></thead>
<tbody>
{#each samples as s (s.id)}
<tr>
<td><a href="/samples/{s.id}">{s.name}</a></td>
<td class="hgvs">{s.vcf_uri}</td>
<td>{s.assembly}</td>
<td>{new Date(s.created_at).toLocaleDateString('en-GB')}</td>
</tr>
{/each}
</tbody>
</table>
<ul class="caselist">
{#each cases as c (c.id)}
<li>
<a class="casename" href="/cases/{c.id}">{c.name}</a>
<span class="chips">
{#each c.phenotypes.slice(0, 4) as term (term.hpo_id)}
<span class="chip match">{term.label}</span>
{/each}
{#if c.phenotypes.length > 4}<span class="chip muted">+{c.phenotypes.length - 4}</span>{/if}
{#if c.phenotypes.length === 0}<span class="chip muted">no phenotype</span>{/if}
</span>
<span class="status {c.latest_job?.status ?? ''}">{status(c)}</span>
<span class="muted">{c.shortlisted} shortlisted</span>
</li>
{/each}
</ul>
{/if}
+206
View File
@@ -0,0 +1,206 @@
<script lang="ts">
import { onDestroy, onMount } from 'svelte';
import { api, type Candidate, type CandidatePage, type Case, type Job, type VariantDetail } from '$lib/api';
import { poll } from '$lib/poll';
import { formatElapsed, latestStep } from '$lib/progress';
import CandidateRow from '$lib/components/CandidateRow.svelte';
import Funnel from '$lib/components/Funnel.svelte';
import VariantPanel from '$lib/components/VariantPanel.svelte';
const POLL_MS = 3000;
const finished = (j: Job) => j.status === 'succeeded' || j.status === 'failed';
let { data } = $props();
let kase = $state<Case | null>(null);
let job = $state<Job | null>(null);
let page = $state<CandidatePage | null>(null);
let detail = $state<VariantDetail | null>(null);
let gene = $state('');
let impact = $state('');
let maxAf = $state<number | null>(null);
let decisionFilter = $state('');
let busy = $state(false);
let scoring = $state(false);
let error = $state<string | null>(null);
let now = $state(Date.now());
let stopPolling: (() => void) | null = null;
const running = $derived((!!job && !finished(job)) || scoring);
const elapsedMs = $derived(job ? now - Date.parse(job.created_at) : 0);
const step = $derived(scoring ? 'scoring variants with the model' : latestStep(job?.log ?? null));
const termCount = $derived(kase?.phenotypes.length ?? 0);
// Tick the elapsed time while a run is in flight; polling refreshes the step itself.
$effect(() => {
if (!running) return;
const tick = setInterval(() => (now = Date.now()), 1000);
return () => clearInterval(tick);
});
onMount(async () => {
try {
kase = await api.case(data.caseId);
job = kase.latest_job;
if (job && !finished(job)) follow(job);
else await loadCandidates();
} catch (e) {
error = (e as Error).message;
}
});
onDestroy(() => stopPolling?.());
function follow(j: Job) {
stopPolling?.();
stopPolling = poll(() => api.job(j.id), {
intervalMs: POLL_MS,
done: finished,
onValue: (next) => {
job = next;
if (next.status === 'succeeded') scoreThenLoad();
},
onError: (e) => { error = `Lost track of the run: ${(e as Error).message}`; }
});
}
async function analyse() {
busy = true;
error = null;
page = null;
detail = null;
try {
job = await api.annotate(data.caseId);
if (!finished(job)) follow(job);
} catch (e) {
error = (e as Error).message;
} finally {
busy = false;
}
}
// Annotation and scoring are one action to the user; the API keeps them separate.
async function scoreThenLoad() {
scoring = true;
try {
await api.score(data.caseId);
} catch (e) {
error = `Scoring failed: ${(e as Error).message}`;
} finally {
scoring = false;
}
await loadCandidates();
}
async function loadCandidates() {
try {
page = await api.candidates(data.caseId, {
gene,
impact,
maxAf,
state: decisionFilter as never,
limit: 100
});
} catch (e) {
error = (e as Error).message;
}
}
async function select(candidate: Candidate) {
try {
detail = await api.variant(candidate.variant.id);
} catch (e) {
error = (e as Error).message;
}
}
async function decided(updated: VariantDetail) {
detail = updated;
await loadCandidates();
}
</script>
<a href="/">All cases</a>
<div class="casehead">
<h1>{kase?.name ?? 'Case'}</h1>
<span class="chips">
{#each kase?.phenotypes ?? [] as term (term.hpo_id)}
<span class="chip match">{term.label}</span>
{/each}
{#if kase && kase.phenotypes.length === 0}
<span class="chip muted">no phenotype recorded — ranking falls back to rarity and consequence</span>
{/if}
</span>
<p class="muted provenance">
{kase?.assembly}
{#if job?.vep_version}· VEP {job.vep_version}{/if}
{#if page?.items[0]?.variant.prediction}· model {page.items[0].variant.prediction.model_version}{/if}
{#if job?.finished_at}· analysed {new Date(job.finished_at).toLocaleString('en-GB')}{/if}
</p>
</div>
{#if error}<p role="alert">{error}</p>{/if}
{#if running}
<p class="progress" aria-live="polite">
<span class="spinner" aria-hidden="true"></span>
Analysing for {formatElapsed(elapsedMs)}{#if step} · <span class="hgvs">{step}</span>{/if}
</p>
{:else}
<p class="actions">
<button onclick={analyse} disabled={busy}>{job ? 'Re-analyse case' : 'Analyse case'}</button>
{#if page && page.funnel.total > 0}
<a class="reportlink" href="/cases/{data.caseId}/report">Case report →</a>
{/if}
</p>
{/if}
{#if job?.status === 'failed' && job.log}
<pre class="log hgvs">{job.log}</pre>
{/if}
{#if page && page.funnel.total > 0}
<Funnel funnel={page.funnel} />
<div class="filters">
<input placeholder="Gene" bind:value={gene} aria-label="Gene" />
<select bind:value={impact} aria-label="Impact">
<option value="">Any impact</option><option>HIGH</option><option>MODERATE</option>
</select>
<label>Max AF <input type="number" step="0.0001" min="0" max="1" bind:value={maxAf} style="width:6rem" /></label>
<select bind:value={decisionFilter} aria-label="Decision">
<option value="">All decisions</option>
<option value="undecided">Undecided</option>
<option value="shortlisted">Shortlisted</option>
<option value="dismissed">Dismissed</option>
</select>
<button class="quiet" onclick={loadCandidates}>Apply</button>
</div>
<div class="triage">
<div class="candidates">
<p class="muted">{page.total} candidates ranked by phenotype fit, rarity, consequence and model score</p>
{#each page.items as candidate, i (candidate.variant.id)}
<CandidateRow
{candidate}
rank={i + 1}
{termCount}
selected={detail?.variant.id === candidate.variant.id}
onselect={() => select(candidate)} />
{:else}
<div class="empty">No candidates match these filters.</div>
{/each}
</div>
{#if detail && page}
<VariantPanel
{detail}
{termCount}
weights={page.weights}
onclose={() => (detail = null)}
ondecided={decided} />
{/if}
</div>
{:else if page && !running}
<div class="empty">
No results yet. "Analyse case" runs VEP over the VCF, scores each variant, then ranks what is
left against the phenotype.
</div>
{/if}
+2
View File
@@ -0,0 +1,2 @@
import type { PageLoad } from './$types';
export const load: PageLoad = ({ params }) => ({ caseId: params.id });
@@ -0,0 +1,78 @@
<script lang="ts">
import { onMount } from 'svelte';
import { api, type Report } from '$lib/api';
import Funnel from '$lib/components/Funnel.svelte';
import Chips from '$lib/components/Chips.svelte';
import { evidenceChips } from '$lib/candidates';
let { data } = $props();
let report = $state<Report | null>(null);
let error = $state<string | null>(null);
onMount(async () => {
try {
report = await api.report(data.caseId);
} catch (e) {
error = (e as Error).message;
}
});
const termCount = $derived(report?.case.phenotypes.length ?? 0);
</script>
<a class="noprint" href="/cases/{data.caseId}">← Back to triage</a>
{#if error}<p role="alert">{error}</p>{/if}
{#if report}
<article class="report">
<h1>Case report — {report.case.name}</h1>
<p class="muted">
Generated {new Date(report.generated_at).toLocaleString('en-GB')} ·
{report.case.assembly}
{#if report.provenance.vep_version}· VEP {report.provenance.vep_version}{/if}
{#if report.provenance.model_version}· model {report.provenance.model_version}{/if}
</p>
<h2>Phenotype</h2>
<span class="chips">
{#each report.case.phenotypes as term (term.hpo_id)}
<span class="chip match">{term.label} <small class="muted">{term.hpo_id}</small></span>
{:else}
<span class="chip muted">none recorded</span>
{/each}
</span>
<h2>Narrowing</h2>
<Funnel funnel={report.funnel} />
<h2>Shortlisted ({report.shortlisted.length})</h2>
{#each report.shortlisted as item (item.variant.id)}
<div class="reportitem">
<h3>{item.variant.gene ?? 'intergenic'} <span class="coord">{item.variant.chrom}:{item.variant.pos} {item.variant.ref}&gt;{item.variant.alt}</span></h3>
<Chips chips={evidenceChips(item, termCount)} />
<p><strong>Reason:</strong> {item.decision?.reason ?? '—'}</p>
{#if item.decision?.note}<p class="muted">{item.decision.note}</p>{/if}
</div>
{:else}
<p class="muted">Nothing shortlisted yet.</p>
{/each}
{#if report.dismissed.length}
<h2>Dismissed ({report.dismissed.length})</h2>
<ul class="dismissed">
{#each report.dismissed as item (item.variant.id)}
<li>
<strong>{item.variant.gene ?? 'intergenic'}</strong>
<span class="coord">{item.variant.chrom}:{item.variant.pos}</span>
{item.decision?.reason ?? 'no reason given'}
</li>
{/each}
</ul>
{/if}
<p class="disclaimer">
Research demonstration on public data. Computational evidence is supporting only under
ACMG/AMP guidance and this tool makes no diagnostic claim.
</p>
</article>
{/if}
@@ -0,0 +1,2 @@
import type { PageLoad } from './$types';
export const load: PageLoad = ({ params }) => ({ caseId: params.id });
-141
View File
@@ -1,141 +0,0 @@
<script lang="ts">
import { onDestroy, onMount } from 'svelte';
import { api, type Job, type VariantPage } from '$lib/api';
import { poll } from '$lib/poll';
import { formatElapsed, latestStep } from '$lib/progress';
const POLL_MS = 3000;
const finished = (j: Job) => j.status === 'succeeded' || j.status === 'failed';
let { data } = $props();
let job = $state<Job | null>(null);
let page = $state<VariantPage | null>(null);
let gene = $state('');
let impact = $state('');
let maxAf = $state<number | null>(0.01);
let busy = $state(false);
let error = $state<string | null>(null);
let stopPolling: (() => void) | null = null;
let now = $state(Date.now());
const running = $derived(!!job && !finished(job));
const elapsedMs = $derived(job ? now - Date.parse(job.created_at) : 0);
const step = $derived(latestStep(job?.log ?? null));
// Tick the elapsed time while a run is in flight; polling refreshes the step itself.
$effect(() => {
if (!running) return;
const tick = setInterval(() => (now = Date.now()), 1000);
return () => clearInterval(tick);
});
onMount(async () => {
try {
const jobs = await api.jobsForSample(data.sampleId); // newest first
const latest = jobs[0] ?? null;
const lastGood = jobs.find((j) => j.status === 'succeeded') ?? null;
// Follow a run in progress; otherwise show the latest results, else the latest failure.
job = latest && !finished(latest) ? latest : (lastGood ?? latest);
if (job) await follow(job);
} catch (e) {
error = (e as Error).message;
}
});
onDestroy(() => stopPolling?.());
async function follow(j: Job) {
stopPolling?.();
if (j.status === 'succeeded') return loadVariants();
if (finished(j)) return;
stopPolling = poll(() => api.job(j.id), {
intervalMs: POLL_MS,
done: finished,
onValue: (next) => {
job = next;
if (next.status === 'succeeded') loadVariants();
},
onError: (e) => { error = `Lost track of the job: ${(e as Error).message}`; }
});
}
async function runAnnotation() {
busy = true;
error = null;
page = null;
try {
job = await api.annotate(data.sampleId);
await follow(job);
} catch (e) {
error = (e as Error).message;
} finally {
busy = false;
}
}
async function loadVariants() {
if (!job) return;
error = null;
try {
page = await api.variants({ jobId: job.id, gene, impact, maxAf, limit: 100 });
} catch (e) {
error = (e as Error).message;
}
}
const scoreClass = (s: number) => (s >= 0.8 ? 'score high' : s >= 0.5 ? 'score mid' : 'score');
</script>
<a href="/">All samples</a>
<h1>Sample {data.sampleId.slice(0, 8)}</h1>
{#if error}<p role="alert">{error}</p>{/if}
{#if job}
<p>Job <span class="hgvs">{job.id.slice(0, 8)}</span>: <span class="status {job.status}">{job.status}</span></p>
{#if running}
<p class="progress" aria-live="polite">
<span class="spinner" aria-hidden="true"></span>
Annotating for {formatElapsed(elapsedMs)}{#if step} · <span class="hgvs">{step}</span>{/if}
</p>
{/if}
{#if job.status === 'failed' && job.log}
<pre class="hgvs" style="white-space:pre-wrap; background:white; border:1px solid var(--line); padding:0.75rem; max-height:16rem; overflow:auto">{job.log}</pre>
{/if}
{/if}
{#if !job || job.status === 'failed'}
<button onclick={runAnnotation} disabled={busy}>{job ? 'Run VEP annotation again' : 'Run VEP annotation'}</button>
{/if}
{#if job?.status === 'succeeded'}
<h2>Variants</h2>
<div style="display:flex; gap:0.5rem; flex-wrap:wrap; margin-bottom:0.75rem">
<input placeholder="Gene symbol" bind:value={gene} aria-label="Gene" />
<select bind:value={impact} aria-label="Impact">
<option value="">Any impact</option><option>HIGH</option><option>MODERATE</option><option>LOW</option><option>MODIFIER</option>
</select>
<label>Max gnomAD AF <input type="number" step="0.001" min="0" max="1" bind:value={maxAf} style="width:6rem" /></label>
<button class="quiet" onclick={loadVariants}>Apply filters</button>
</div>
{#if page && page.items.length === 0}
<div class="empty">No variants match these filters. Raise the allele frequency cap or clear the gene filter.</div>
{:else if page}
<p style="color:var(--ink-soft)">{page.total} variants, showing {page.items.length}</p>
<table>
<thead><tr><th>Position</th><th>Gene</th><th>Consequence</th><th>HGVS</th><th>gnomAD AF</th><th>ClinVar</th><th>Score</th></tr></thead>
<tbody>
{#each page.items as v (v.id)}
<tr>
<td class="coord">{v.chrom}:{v.pos} {v.ref}>{v.alt}</td>
<td>{v.gene ?? ''}</td>
<td>{v.consequence ?? ''}<br /><small style="color:var(--ink-soft)">{v.impact ?? ''}</small></td>
<td class="hgvs">{v.hgvsp ?? v.hgvsc ?? ''}</td>
<td>{v.gnomad_af?.toExponential(2) ?? 'absent'}</td>
<td>{v.clinvar_sig ?? ''}</td>
<td>{#if v.prediction}<span class={scoreClass(v.prediction.score)}>{v.prediction.score.toFixed(2)}</span>{:else}<span style="color:var(--ink-soft)">unscored</span>{/if}</td>
</tr>
{/each}
</tbody>
</table>
{/if}
{/if}
-2
View File
@@ -1,2 +0,0 @@
import type { PageLoad } from './$types';
export const load: PageLoad = ({ params }) => ({ sampleId: params.id });