Verified, then logged.
Sun 22 Nov 2026 · 00:33 KST / 15:33 UTC한국어 | ENSearch /

Breaking01 / AI

llm-d 1.0 strengthens zero-downtime and multi-tenant serving

In a data center aisle, color-coded network cables branch from a rack-top switch into dense inference servers while an out-of-focus technician works in the background, seen from behind. AI-generated image

IBM said contributors to llm-d announced version 1.0 of the open-source Kubernetes inference stack on 20 Oct 2026, and the company published a figure showing its router captured 97% of reusable cache in an agentic coding test.

Operations. According to the announcement, version 1.0 strengthens zero-downtime operations, agent-aware serving and multi-tenant quality of service. Flow control balances bursts of requests among tenants, the user groups that share infrastructure. It also pins multi-turn requests to the appropriate infrastructure, which keeps their caches warm.

Cache-aware routing and the separation of reading input from generating responses are techniques the project has covered from the start. The IBM announcement states, "These ideas are not new to version 1.0." The change is the level of integration at which the techniques work together through traffic changes, failures and updates.

Cost. IBM and Red Hat ran 3,000 software-engineering agents concurrently, according to the announcement. Self-hosting cost 5 to 10 times less than the price of serving the same open model as a service. This is also a company-published figure.

IBM's announcement describes llm-d as a sandbox project of the Cloud Native Computing Foundation (CNCF). The stack adds distributed routing and operational capabilities to inference engines such as vLLM and SGLang. Red Hat, IBM and Google co-founded llm-d last year with CoreWeave and NVIDIA.

IBM's announcement did not disclose the models or hardware configurations used in the two tests.