Aflevering 138: We Built Our Own Wormhole to Migrate 150 Kubernetes Clusters

In this episode, recorded live at KubeCon, Ronald and Jan talk with Jannis Relakis and Michael Seiwald-McCarty, both senior platform engineers at Celonis. Celonis manages over 150 Kubernetes clusters across GKE, AKS, and EKS, but it wasn't always that clean. They started with six different Kubernetes flavors, including Gardener, K-Ops, and OpenShift (both Rosa and ARO), spread across multiple cloud providers.

In their KubeCon talk "No Shame in Just Paying," they shared how they tackled this consolidation project: migrating all workloads to three standardized, fully managed Kubernetes distributions. Key topics include their self-built cross-cluster connectivity tool called "Wormhole" (powered by Envoy's dynamic forward proxy), RabbitMQ federation for seamless message queue migration, and how they used Karpenter and Cilium to align node and network management across clouds.

They also get candid about what went wrong: an accidental ArgoCD sync that caused a 10-minute full environment outage, the pain of "snowflake" environments (including one requiring full HIPAA compliance with Istio mTLS), and the constant fight against scope creep that threatened to derail the entire project.

The episode closes with a forward-looking discussion on FinOps, resource rightsizing, the future of VPA, and whether Kubernetes and serverless can ever truly converge.Powered by ACC ICT