Aflevering 136: vLLM, LMD, and the Quest to Build the Linux of AI Inference

vLLM serves as a "Rosetta Stone" between the ever-growing zoo of models (Llama, DeepSeek, Mistral, Qwen, Nvidia Nemotron) and accelerators (Nvidia, AMD, Intel, Google TPUs).

Mohamed and Thierry discuss why OpenStack is still highly relevant at massive scale, especially for organizations requiring multi-tenancy, hardware abstraction, GPU enablement, HPC workloads, and advanced networking performance.

Carl shares how edge computing has matured. What used to require explaining the concept itself has now shifted toward real-world adoption, with use cases across retail, manufacturing, telecom and beyond.

A key topic in this episode is Kube Solo, a lightweight Kubernetes distribution that can run on roughly 200 MB of RAM. The goal is to make Kubernetes usable in environments where traditional clusters are too heavy, such as IoT and Edge.

Many of the changes in this release focus on closing gaps and improving control, such as more fine-grained authorization, better admission control during node startup, and addressing previously existing vulnerabilities.

A key theme is balance: developers need freedom, but within secure guardrails. That’s why pipelines are not immediately blocked on vulnerabilities, but first used to provide visibility and gradually increase maturity.

Instead of reacting to vulnerabilities after they appear, the conversation explores a different mindset. One where security isn’t something you bolt on later, but something you start with. Not “shift left”… but start left.

Veel organisaties draaien al vijf jaar Kubernetes, maar hebben vaak nog maar 5–10% van hun applicatielandschap gemoderniseerd . Volgens Jan ligt het echte probleem niet bij het platform, maar bij legacy-applicaties en de businesscontext eromheen.

Too often, PAM is treated as a compliance checkbox. Something you buy because an auditor, insurer, or regulation tells you to. Maurice explains why that mindset is dangerous