Skip to content

Kubernetes with Helm

The chart is in deploy/helm/necodoc. Its values.yaml documents every setting.

  • Deployments for api (ROLE=api), worker (ROLE=worker) and app (the web app), with Services for api and app.
  • An Ingress: /api to the API, everything else to the app, on one host.
  • A ConfigMap with the non-secret env, and a Secret (or your existing one).
  • A migration Job that runs as a pre-install and pre-upgrade hook, before any new pod starts.
  • PodDisruptionBudgets for api and worker.
  • Optional: a CPU HorizontalPodAutoscaler or a KEDA ScaledObject for workers, and a PersistentVolumeClaim for disk storage.
  • Prometheus metrics on port 9464 of every api and worker pod, with prometheus.io/* annotations; optionally a Prometheus Operator PodMonitor. See Observability.

Pods run as non-root (UID 1000; nginx’s 101 for the app) with a read-only root filesystem, no Linux capabilities and the RuntimeDefault seccomp profile. Chromium doesn’t need extra privileges.

The chart doesn’t install Postgres. Use a managed service or an operator such as CloudNativePG.

Create a Secret with at least DATABASE_URL and BETTER_AUTH_SECRET, plus any storage credentials (see Configuration):

Terminal window
kubectl create namespace necodoc
kubectl -n necodoc create secret generic necodoc-env \
--from-literal=DATABASE_URL='postgres://necodoc:[email protected]:5432/necodoc' \
--from-literal=BETTER_AUTH_SECRET="$(openssl rand -base64 32)"

Then a values file:

my-values.yaml
image: { repository: registry.example.com/necodoc-api, tag: "0.1.0" }
appImage: { repository: registry.example.com/necodoc-app, tag: "0.1.0" }
publicUrl: https://necodoc.example.com
secrets: { existingSecret: necodoc-env }
ingress:
className: nginx
host: necodoc.example.com
tls: [{ secretName: necodoc-tls, hosts: [necodoc.example.com] }]
storage:
driver: s3
s3: { bucket: necodoc-outputs, region: us-east-1 }
Terminal window
helm install necodoc deploy/helm/necodoc -n necodoc -f my-values.yaml

You can put DATABASE_URL and BETTER_AUTH_SECRET under secrets: instead of existingSecret. The chart’s Secret is a Helm hook, so the migration Job can read it on the first install. Because of that, helm uninstall leaves it behind; delete it by hand.

Your controller must allow request bodies of 25 MB (uploads are capped at 20 MB) and upstream reads of RENDER_WAIT_MS plus a margin, 120 seconds with the default (synchronous renders wait up to RENDER_WAIT_MS). Set those through ingress.annotations in your controller’s syntax.

  • disk: the chart creates a PVC (or uses storage.disk.existingClaim) and mounts it at /data/files on every API and worker pod. With more than one pod it must be ReadWriteMany (NFS, EFS, Azure Files, CephFS). It has helm.sh/resource-policy: keep, so uninstalling keeps your PDFs.
  • s3 and azure: no volume. Put keys in the Secret, or leave them out and use workload identity: annotate the chart’s ServiceAccount through serviceAccount.annotations (IRSA’s eks.amazonaws.com/role-arn, or azure.workload.identity/client-id). Azure workload identity also needs the azure.workload.identity/use: "true" pod label, which the chart has no value for: add it with a Helm post-renderer, or use a connection string or account key instead.

Details and examples: Storage.

Set worker.replicas, or turn on one autoscaler:

worker:
autoscaling: { enabled: true, minReplicas: 2, maxReplicas: 10, targetCPUUtilizationPercentage: 70 }

Or scale on the render queue with KEDA (install KEDA first):

worker:
keda: { enabled: true, minReplicas: 1, maxReplicas: 20, jobsPerReplica: 4 }

The ScaledObject counts queued and running jobs on the render queue in pg-boss’s pgboss.job table, using the worker’s DATABASE_URL. The two are exclusive; KEDA creates its own HPA. See Roles and scaling.

API and worker pods have a startup and liveness probe on /api/health and a readiness probe on /api/health/ready. A pod with no database, queue or storage goes unready. See Health checks.

Terminal window
helm upgrade necodoc deploy/helm/necodoc -n necodoc -f my-values.yaml --set image.tag=0.2.0 --set appImage.tag=0.2.0

The migration Job runs first; if it fails, the upgrade stops before any pod changes. Pods restart when the chart’s ConfigMap or Secret changes (checksum annotations); after changing an existing Secret, run kubectl rollout restart. See Upgrades.