OSS + ELC EU 2026

Running LLMs near devices or in constrained edge environments changes the observability problem. A small model can answer successfully while still being unusable: first token arrives too late, prompts burn scarce CPU/RAM budget, output is malformed, or gateway/model-selection errors hide behind green Linux and container metrics.

This session walks through a reproducible, CPU-first Ubuntu demo for resource-constrained self-hosted LLMs. It uses small Ollama models (llama3.2:1b and qwen3.5:2b) on the same CPU workload with MicroK8s, Juju, COS Lite, Tempo, a FastAPI OpenAI-compatible gateway, and Canonical's Charmed OpenTelemetry Collector. The gateway emits OpenTelemetry GenAI spans, metrics, and trace-correlated logs; Prometheus, Loki, Tempo, Grafana, and Alertmanager turn them into an LLM golden-signals view.

Using recorded normal, slow, expensive, error, and low-quality scenarios, attendees will see how time-to-first-token, token usage, traces, alerts, and quality signals reveal whether a local model is useful within tight compute constraints. The model comparison shows why embedded and edge AI decisions need measured latency/cost/quality trade-offs, not just 'the model runs.'