From 749b0f821481776cfc172372f1bc3c20c322601f Mon Sep 17 00:00:00 2001 From: Kemal Yaylali Date: Sat, 12 Sep 2026 10:52:51 +0100 Subject: [PATCH] docs(readme): correct the parts that no longer match the code Five things had drifted: - The tech table listed GKE and Cloud SQL as the cloud, but the default deployment track has been serverless Cloud Run + Google Batch since docs/cloud.md; GKE and Cloud SQL are behind flags. Cloud Run was missing entirely, including as the Nextflow driver. - The button is "Analyse case", not "Run VEP annotation" (twice). - samples were renamed to cases, so LOCAL_DATA_ROOT holds a case's vcf_uri. - The Data section omitted HPO gene-to-phenotype, which is half the ranking, and the phenopackets behind the published case. - The status list described the sample list and filtered variant table the UI no longer has, and read as pending work when all of it ships. Replaced with what actually works, plus the two known gaps: no allele frequencies without the VEP cache, and no authentication on the deployed demo. --- README.md | 32 +++++++++++++++++++------------- 1 file changed, 19 insertions(+), 13 deletions(-) diff --git a/README.md b/README.md index bd5d2e3..9af1b1f 100644 --- a/README.md +++ b/README.md @@ -21,9 +21,9 @@ clinical tool and makes no diagnostic claims. | Database | PostgreSQL 16 | `docker-compose.yml` | | Frontend | SvelteKit, TypeScript | `web/` | | ML | LightGBM pathogenicity scorer, MLflow registry | `ml/` | -| Orchestration | Argo Workflows + Argo Events (pipeline), Pub/Sub (events) | `infra/argo-workflows/` | +| Orchestration | Cloud Run job (default), Argo Workflows + Argo Events, Pub/Sub | `infra/terraform/`, `infra/argo-workflows/` | | Platform | Kubernetes (Kustomize), ArgoCD (GitOps) | `infra/k8s/`, `infra/argocd/` | -| Cloud | GCP: GKE Autopilot, Cloud SQL, GCS, Batch, Secret Manager, Artifact Registry | `infra/terraform/` | +| Cloud | GCP: Cloud Run, Google Batch, GCS, Secret Manager, Artifact Registry; GKE Autopilot and Cloud SQL behind flags | `infra/terraform/` | | CI/CD | GitHub Actions, Workload Identity Federation | `.github/workflows/` | ## Quick start (local) @@ -51,7 +51,7 @@ the rest of that patient's genome is not public. It writes the phenotype list al so the case can be created exactly as reported. See [docs/data.md](docs/data.md) for the provenance and for what this case does and does not demonstrate. -The docker-compose API has no Nextflow, so "Run VEP annotation" marks the job failed with the +The docker-compose API has no Nextflow, so "Analyse case" marks the job failed with the command to run instead. With Nextflow and Docker on the host, a VEP cache in `pipeline/cache/vep` and a VCF under `data/` (see [data/README.md](data/README.md)): @@ -68,7 +68,7 @@ pipeline on a handful of variants: VEP_DATABASE=true make pipeline VCF=pipeline/tests/data/tiny.vcf ``` -To make the UI's "Run VEP annotation" button work, run the API on the host (where Nextflow and +To make the UI's "Analyse case" button work, run the API on the host (where Nextflow and Docker are) rather than in docker-compose: ```bash @@ -80,7 +80,7 @@ cd api && DATABASE_URL=postgresql+asyncpg://rarelens:rarelens@localhost:5432/rar ``` `PIPELINE_DATABASE_URL` is what the loader container gets: inside it, the API's own `localhost` -would be the container itself. `LOCAL_DATA_ROOT` is the directory a sample's `vcf_uri` must sit under. +would be the container itself. `LOCAL_DATA_ROOT` is the directory a case's `vcf_uri` must sit under. To train and register a model (the API scores with `models:/rarelens-pathogenicity@production`): @@ -133,9 +133,10 @@ Every green CI run on `main` bumps image tags in the gcp overlay and ArgoCD depl ## Data The demo runs on published, openly licensed human data: the NIST Genome in a Bottle HG002 -benchmark genome as the sample, ClinVar for labels, gnomAD for allele frequencies. Sources, -licences, citations and how the model should be evaluated honestly are in -[docs/data.md](docs/data.md). +benchmark genome as the background sample, ClinVar for labels, gnomAD for allele frequencies, the +Human Phenotype Ontology's gene-to-phenotype annotations as what the ranking matches against, and +GA4GH phenopackets curated from case reports for the published case. Sources, licences, citations +and how the model should be evaluated honestly are in [docs/data.md](docs/data.md). ## Architecture @@ -145,13 +146,18 @@ than AWS. ## Status -Work in progress. Milestones, in order: +A self-training lab, built in the open. Working end to end: -1. Skeleton, Postgres, FastAPI, Nextflow VEP annotation on a public VCF, CI green -2. SvelteKit UI: sample list, variant table with filters, job status +1. Postgres, FastAPI, Nextflow VEP annotation on a public VCF, CI green +2. SvelteKit UI: cases, a phenotype-ranked candidate list showing the evidence behind each rank, + shortlist/dismiss decisions, and a case report 3. Kubernetes manifests, kind, Argo Workflows trigger -4. Terraform for GCP, ArgoCD GitOps deploy -5. Pathogenicity model, MLflow registry, prediction endpoint +4. Terraform for GCP: serverless Cloud Run + Batch by default, GKE and ArgoCD behind a flag +5. Pathogenicity model trained on ClinVar, MLflow registry, scoring endpoint +6. A demo case built from a published patient, with the citations behind it + +Known gaps: allele frequencies need the 25 GB VEP cache, because VEP's database mode returns +none, so the rarity term does no work without it; and the deployed demo has no authentication. ## Licence