kemal
5463f489a3
ci / api (push) Failing after 10s
ci / terraform (push) Failing after 11s
ci / web (push) Failing after 35s
ci / pipeline (push) Failing after 2m29s
ci / images (api) (push) Skipped
ci / images (ml) (push) Skipped
ci / images (pipeline) (push) Skipped
ci / images (web) (push) Skipped
End-to-end variant interpretation platform for rare genetic disease research: SvelteKit UI, FastAPI + PostgreSQL API, Nextflow/Ensembl VEP pipeline, LightGBM pathogenicity scoring with MLflow, K8s/ArgoCD/GCP infrastructure. Public test data only; no clinical claims.
55 lines
2.5 KiB
Markdown
55 lines
2.5 KiB
Markdown
# rarelens
|
|
|
|
A small, end-to-end variant interpretation platform for rare genetic disease research.
|
|
Scientists upload a VCF, a Nextflow workflow annotates it with Ensembl VEP, a machine
|
|
learning model scores each variant, and results are browsable in a web app.
|
|
|
|
This repository is a **self-training lab**. It exists so that one engineer can learn, in
|
|
public, how a modern life-sciences platform is built end to end: full-stack application,
|
|
scientific pipeline, ML serving, and cloud infrastructure, all in one monorepo. It is not a
|
|
clinical tool and makes no diagnostic claims.
|
|
|
|
## What is in the box
|
|
|
|
| Layer | Technology | Directory |
|
|
|------------|--------------------------------------------------------|----------------------|
|
|
| Pipeline | Nextflow DSL2, bcftools, Ensembl VEP, Docker | `pipeline/` |
|
|
| API | FastAPI, Pydantic v2, SQLAlchemy 2.0 (async), Alembic | `api/` |
|
|
| Database | PostgreSQL 16 | `docker-compose.yml` |
|
|
| Frontend | SvelteKit, TypeScript | `web/` |
|
|
| ML | LightGBM pathogenicity scorer, MLflow tracking | `ml/` |
|
|
| Orchestration | Argo Workflows (pipeline), Pub/Sub (events) | `infra/argo-workflows/` |
|
|
| Platform | Kubernetes (Kustomize), ArgoCD (GitOps) | `infra/k8s/`, `infra/argocd/` |
|
|
| Cloud | GCP: GKE Autopilot, Cloud SQL, GCS, Artifact Registry | `infra/terraform/` |
|
|
| CI/CD | GitHub Actions, Workload Identity Federation | `.github/workflows/` |
|
|
|
|
## Quick start (local)
|
|
|
|
```bash
|
|
make up # postgres + api + web via docker-compose
|
|
make migrate # alembic upgrade head
|
|
make pipeline # nextflow run pipeline/main.nf -profile docker --vcf data/example.vcf.gz
|
|
make kind # spin up a local kind cluster and apply infra/k8s/overlays/local
|
|
```
|
|
|
|
Then open http://localhost:5173.
|
|
|
|
## Architecture
|
|
|
|
See [docs/architecture.md](docs/architecture.md) for the diagram and the reasoning behind
|
|
each choice.
|
|
|
|
## Status
|
|
|
|
Work in progress. Milestones, in order:
|
|
|
|
1. Skeleton, Postgres, FastAPI, Nextflow VEP annotation on a public VCF, CI green
|
|
2. SvelteKit UI: sample list, variant table with filters, job status
|
|
3. Kubernetes manifests, kind, Argo Workflows trigger
|
|
4. Terraform for GCP, ArgoCD GitOps deploy
|
|
5. Pathogenicity model, MLflow registry, prediction endpoint
|
|
|
|
## Licence
|
|
|
|
AGPL-3.0. Test data are public (ClinVar, gnomAD subsets); no patient data are used or accepted.
|