kemal
5463f489a3
End-to-end variant interpretation platform for rare genetic disease research: SvelteKit UI, FastAPI + PostgreSQL API, Nextflow/Ensembl VEP pipeline, LightGBM pathogenicity scoring with MLflow, K8s/ArgoCD/GCP infrastructure. Public test data only; no clinical claims.
rarelens
A small, end-to-end variant interpretation platform for rare genetic disease research. Scientists upload a VCF, a Nextflow workflow annotates it with Ensembl VEP, a machine learning model scores each variant, and results are browsable in a web app.
This repository is a self-training lab. It exists so that one engineer can learn, in public, how a modern life-sciences platform is built end to end: full-stack application, scientific pipeline, ML serving, and cloud infrastructure, all in one monorepo. It is not a clinical tool and makes no diagnostic claims.
What is in the box
| Layer | Technology | Directory |
|---|---|---|
| Pipeline | Nextflow DSL2, bcftools, Ensembl VEP, Docker | pipeline/ |
| API | FastAPI, Pydantic v2, SQLAlchemy 2.0 (async), Alembic | api/ |
| Database | PostgreSQL 16 | docker-compose.yml |
| Frontend | SvelteKit, TypeScript | web/ |
| ML | LightGBM pathogenicity scorer, MLflow tracking | ml/ |
| Orchestration | Argo Workflows (pipeline), Pub/Sub (events) | infra/argo-workflows/ |
| Platform | Kubernetes (Kustomize), ArgoCD (GitOps) | infra/k8s/, infra/argocd/ |
| Cloud | GCP: GKE Autopilot, Cloud SQL, GCS, Artifact Registry | infra/terraform/ |
| CI/CD | GitHub Actions, Workload Identity Federation | .github/workflows/ |
Quick start (local)
make up # postgres + api + web via docker-compose
make migrate # alembic upgrade head
make pipeline # nextflow run pipeline/main.nf -profile docker --vcf data/example.vcf.gz
make kind # spin up a local kind cluster and apply infra/k8s/overlays/local
Then open http://localhost:5173.
Architecture
See docs/architecture.md for the diagram and the reasoning behind each choice.
Status
Work in progress. Milestones, in order:
- Skeleton, Postgres, FastAPI, Nextflow VEP annotation on a public VCF, CI green
- SvelteKit UI: sample list, variant table with filters, job status
- Kubernetes manifests, kind, Argo Workflows trigger
- Terraform for GCP, ArgoCD GitOps deploy
- Pathogenicity model, MLflow registry, prediction endpoint
Licence
AGPL-3.0. Test data are public (ClinVar, gnomAD subsets); no patient data are used or accepted.