kemal/rarelens
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: a technical introduction to the platform, as a LaTeX article

Browse Source 
docs/blog/rarelens.tex builds a 10-page write-up covering the Nextflow DSL2
pipeline, the three execution backends behind one API call, the Argo and
ArgoCD track, Terraform, the external systems integrated, the ranking, the
benchmark and the model.

It is deliberately as much an account of what was wrong as of what works: the
components that scored evidence nobody had looked up, the missense AUROC that
fell from 0.872 to 0.500 once allele frequency was removed, the ontology walk
that silently dropped 399 terms, and the propagation change that measured
slightly worse than what it replaced and was kept anyway with the numbers
published.

Figures come from one headless-browser script. Screenshots are of the running
application; the six diagrams are hand-written HTML styled from the same
palette, rendered by the same script. Keeping both in one place is what stops
the article drifting from the system, and every number in a diagram has to be
changed deliberately.
This commit is contained in:
Kemal Yaylali
2026-09-12 13:10:47 +01:00
parent c25fb53666
commit 1e46fff2ff
25 changed files with 729 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/blog/diagram-sources/model.html
+25
View File
@@ -0,0 +1,25 @@
<!doctype html><meta charset="utf-8"><link rel="stylesheet" href="_base.css">
<style>
.big { font-family:var(--mono); font-size:34px; font-weight:600; letter-spacing:-0.02em; }
.mrow { display:grid; grid-template-columns:1fr 92px 92px 110px 110px; font-size:13px; align-items:center; }
.mrow > div { padding:8px 10px; border-bottom:1px solid var(--line); }
.mhead > div { color:var(--soft); font-weight:600; font-size:11px; text-transform:uppercase; letter-spacing:0.06em; background:var(--paper); }
.num { font-family:var(--mono); text-align:right; }
</style>
<div id="frame">
<h1>The number that changed what the model is allowed to do</h1>
<p class="sub">Held-out evaluation with whole genes held out, never single variants (Grimm <em>et al.</em> 2015): 312,025 training and 74,239 test variants across 7,728 and 1,932 genes, with no gene on both sides.</p>
<div class="mrow mhead"><div>Model</div><div class="num">AUROC</div><div class="num">AUPRC</div><div class="num">missense AUROC</div><div class="num">missense AUPRC</div></div>
<div class="mrow"><div>v2 &mdash; with gnomAD allele frequency as a feature</div><div class="num">0.986</div><div class="num">0.954</div><div class="num">0.872</div><div class="num">0.725</div></div>
<div class="mrow"><div>v3 &mdash; allele frequency removed</div><div class="num">0.966</div><div class="num">0.881</div><div class="num" style="color:var(--plum);font-weight:700">0.500</div><div class="num">0.398</div></div>
<div style="height:20px"></div>
<div class="row">
<div class="box plum" style="flex:0 0 268px; text-align:center; padding:17px"><div class="big" style="color:var(--plum)">0.500</div><div class="d" style="margin-top:5px">AUROC on missense variants once frequency is removed. Exactly random.</div></div>
<div style="width:15px"></div>
<div class="col" style="flex:1">
<div class="box"><div class="t">What that proves</div><div class="d">Strip frequency out and the model cannot tell one missense variant from another at all &mdash; nothing is left but the consequence class, so every missense row scores identically. The respectable-looking 0.872 was never variant-effect knowledge. It was allele frequency.</div></div>
<div class="box amber"><div class="t">And the frequency feature was circular</div><div class="d">ACMG's BA1/BS1 criteria assign ClinVar's <em>benign</em> labels <em>using</em> allele frequency. The feature had partly caused the label, so the model was rediscovering the rule that produced its own training data.</div></div>
<div class="box green"><div class="t">The consequence for the product</div><div class="d">Frequency is no longer a feature &mdash; the ranking already scores it explicitly and auditably, and feeding it to the model as well put ~45% of every rank on one measurement counted twice. The model now abstains unless it has CADD or AlphaMissense, because 0.500 is the measurement saying it has nothing else to add.</div></div>
</div>
</div>
</div>