Verified, then logged.
Sun 22 Nov 2026 · 00:20 KST / 15:20 UTC한국어 | ENSearch /

Breaking02 / IT

AWS adds shared cache skill for self-hosted language models

KVTrace estimates the benefits of a shared cache before deployment.

AWS introduced a distributed caching skill for teams that run their own language models on 21 Nov, according to its blog. Cache deployment on Amazon ElastiCache requires Valkey 9.1 or later.

The AWS blog lists support for Kiro, Claude Code and OpenAI Codex. A model generates key-value (KV) state as it processes input. The shared cache lets multiple servers reuse that state.

AWS gives the example of servers with separate caches. If a server takes over a conversation and lacks its cached state, the model processes the earlier input again.

AWS says KVTrace estimates the benefits of a shared cache before deployment at no extra charge. It replays recorded cache activity at different capacities.

Deployment requirements. The AWS blog specifies cache.i8ge.48xlarge nodes for the shared cache. Deployment also requires an Elastic Fabric Adapter (EFA) network interface that supports remote direct memory access (RDMA) writes.

The AWS blog does not disclose the licence terms for the skill itself.