OIDC Deploying via K8s

As I switch from GitHub to Forgejo for my various project repositories, I find myself looking for a way to securely deploy new services into my Kubernetes cluster without having to keep updating Kubernetes auth tokens and storing them as a workflow secret.

GitHub Actions and, as of Forgejo v15, Forgejo Actions also supports providing temporary JWT tokens to workflow steps that securely state what repository and other commits details a given build is for. When a workflow is running, it can pass it to Kubernetes and other supported services and authenticate as a secure client with all the identity information of that repository and workflow. This is called OpenID Connect or OIDC.

This is super powerful because can federate identity and minimize the number of secrets required–great the next time a widely used workflow step gets compromised and steps all your secrets. Having fewer secrets helps.

In this post, I’m going to show I configured Nix, Kubernetes, and Forgejo to securely update a deployment.

Don't use nodeName in Kubernetes

In my Kubernetes, I sometimes try to run a pod on a specific worker node. Maybe one of them has a folder that I need or a specific hardware characteristic. Historically, I’ve used Pod spec.nodeName: srv5 However, when that node becomes unavailable, say because it’s run out of disk space and has DiskPressure on it, then Kubernetes will continually try to spin up thousands of pods on it.

A screenshot of CPU usage growing, then Prometheus falls over and can’t scrap anymore.

Replatforming RKE1 to Nix-based K8s - Part 2

In my previous post, I showed how to provision a Kubernetes node in NixOS maintaining compatibility with RKE1 (Rancher Kubernetes Engine v1), but switching to the Kubernetes nixpkg. In this post, I’m going to show how to take an Ubuntu worker and replace it with a NixOS based worker without rebuilding the cluster.