Kubernetes and GitOps
Off by default. terraform apply -var deploy_kubernetes=true turns it on; it exists to
demonstrate the GitOps path and is meant to be destroyed afterwards.
Layout
infra/k8s/base/ api, web, postgres, ingress
infra/k8s/overlays/local/ kind: in-cluster Postgres, NodePort
infra/k8s/overlays/gcp/ Cloud SQL, Workload Identity, image tags
infra/argo-workflows/ annotate WorkflowTemplate, EventSource, Sensor, RBAC
infra/argocd/app.yaml the Application that reconciles the cluster
Locally
make kind # builds the images, loads them into a kind cluster, applies the local overlay
The GitOps loop
- A pull request merges to
main. - CI builds and pushes images to Artifact Registry.
- CI bumps the image tags in
infra/k8s/overlays/gcp. - ArgoCD sees the commit and reconciles the cluster.
Deployment is therefore a commit, and rollback is a revert.
Argo Workflows and Argo Events
annotate.yaml is a WorkflowTemplate running the same containers the Nextflow pipeline uses.
events.yaml holds the Pub/Sub EventSource and the Sensor that triggers the template when the
API publishes a job. rbac.yaml grants the sensor permission to create workflows in the
rarelens namespace and nothing else.
The ConfigMap trap
Kustomize appends a content hash to generated ConfigMap names so that a config change forces a
rollout. That substitution only happens for workloads Kustomize considers in scope. Both overlays
were missing namespace: rarelens, so the hashed name was generated but never substituted, and
pods mounted a ConfigMap name that no longer existed.
The symptom is a pod stuck in CreateContainerConfigError while every manifest reads correctly.
See Gotchas.
rarelens
Understanding it
Working on it
Running it
When it goes wrong