Initial release: rarelens platform skeleton (AGPL-3.0)
ci / api (push) Failing after 10s
ci / terraform (push) Failing after 11s
ci / web (push) Failing after 35s
ci / pipeline (push) Failing after 2m29s
ci / images (api) (push) Skipped
ci / images (ml) (push) Skipped
ci / images (pipeline) (push) Skipped
ci / images (web) (push) Skipped

End-to-end variant interpretation platform for rare genetic disease research:
SvelteKit UI, FastAPI + PostgreSQL API, Nextflow/Ensembl VEP pipeline,
LightGBM pathogenicity scoring with MLflow, K8s/ArgoCD/GCP infrastructure.
Public test data only; no clinical claims.
This commit is contained in:
2026-09-11 16:55:35 +01:00
commit 5463f489a3
74 changed files with 4597 additions and 0 deletions
+54
View File
@@ -0,0 +1,54 @@
# rarelens
A small, end-to-end variant interpretation platform for rare genetic disease research.
Scientists upload a VCF, a Nextflow workflow annotates it with Ensembl VEP, a machine
learning model scores each variant, and results are browsable in a web app.
This repository is a **self-training lab**. It exists so that one engineer can learn, in
public, how a modern life-sciences platform is built end to end: full-stack application,
scientific pipeline, ML serving, and cloud infrastructure, all in one monorepo. It is not a
clinical tool and makes no diagnostic claims.
## What is in the box
| Layer | Technology | Directory |
|------------|--------------------------------------------------------|----------------------|
| Pipeline | Nextflow DSL2, bcftools, Ensembl VEP, Docker | `pipeline/` |
| API | FastAPI, Pydantic v2, SQLAlchemy 2.0 (async), Alembic | `api/` |
| Database | PostgreSQL 16 | `docker-compose.yml` |
| Frontend | SvelteKit, TypeScript | `web/` |
| ML | LightGBM pathogenicity scorer, MLflow tracking | `ml/` |
| Orchestration | Argo Workflows (pipeline), Pub/Sub (events) | `infra/argo-workflows/` |
| Platform | Kubernetes (Kustomize), ArgoCD (GitOps) | `infra/k8s/`, `infra/argocd/` |
| Cloud | GCP: GKE Autopilot, Cloud SQL, GCS, Artifact Registry | `infra/terraform/` |
| CI/CD | GitHub Actions, Workload Identity Federation | `.github/workflows/` |
## Quick start (local)
```bash
make up # postgres + api + web via docker-compose
make migrate # alembic upgrade head
make pipeline # nextflow run pipeline/main.nf -profile docker --vcf data/example.vcf.gz
make kind # spin up a local kind cluster and apply infra/k8s/overlays/local
```
Then open http://localhost:5173.
## Architecture
See [docs/architecture.md](docs/architecture.md) for the diagram and the reasoning behind
each choice.
## Status
Work in progress. Milestones, in order:
1. Skeleton, Postgres, FastAPI, Nextflow VEP annotation on a public VCF, CI green
2. SvelteKit UI: sample list, variant table with filters, job status
3. Kubernetes manifests, kind, Argo Workflows trigger
4. Terraform for GCP, ArgoCD GitOps deploy
5. Pathogenicity model, MLflow registry, prediction endpoint
## Licence
AGPL-3.0. Test data are public (ClinVar, gnomAD subsets); no patient data are used or accepted.