OSS + ELC EU 2026

It's hard to experiment, let alone develop applications, for LLMs when access to GPUs is restricted. But what if you could do your development on CPUs? This session demonstrates how to deploy vLLM on a local Kubernetes cluster using Kind and Podman Desktop, running entirely on a CPU-only laptop.

The presentation covers building a minimal environment, deploying vLLM containers, and running inference against models like Gemma-3 and Qwen2.5. A live benchmark comparison between vLLM and llama.cpp reveals when each runtime excels across different model sizes and hardware profiles. The session then extends the workflow to production-grade serving with KServe, including autoscaling and model lifecycle management.

Attendees will learn how to deploy and test vLLM locally without GPUs or cloud resources, understand when to choose vLLM versus llama.cpp based on model size and hardware, and extend a local setup to production-grade serving with KServe.