Verified, then logged.
Sun 22 Nov 2026 · 00:20 KST / 15:20 UTC한국어 | ENSearch /

Breaking02 / IT

NVIDIA introduces ReplaySSM for faster hybrid-model decoding

The post examines state storage in models such as Nemotron 3 and the Qwen family, which mix SSM and attention layers.

Inside an open GPU server in a data center, memory modules are packed in tight rows as an out-of-focus gloved hand slots one into place. AI-generated image

NVIDIA introduced ReplaySSM in a developer blog post on 13 Oct 2026, re-examining how state is stored during hybrid-model decoding. The post is aimed at developers working on decoding for these models. Decoding is the step in which a model generates its output.

According to the blog, state-space model (SSM) layers form the backbone of a hybrid model. A small number of attention layers sit between them. An SSM is a sequence model that carries information forward in an internal state. NVIDIA Nemotron 3 and Qwen3.5 to Qwen3.8 use this design. Kimi-K3 and GLM-5.3-Flash are built the same way.

The SSM layer is implemented differently from model to model. The blog gives Mamba-2 and Gated DeltaNet (GDN) as examples. Kimi Delta Attention (KDA) also serves in this position.

The blog explains that SSM layers are designed to make decoding fast. A key-value (KV) cache stores data from earlier tokens for attention layers to reuse. SSM layers do not rely on one that keeps growing. They use a fixed-size state instead.