Breaking01 / AI
SLAM-ASR relies on induction heads for speech transcription
Researchers found that some pairs of attention heads cancelled out noise in the attention signal and helped the model focus on the word it had just generated.

Researchers reported on 19 Nov 2026 that the SLAM-ASR speech recognition model relies on induction heads that copy repeated sequences of text when it transcribes speech.
According to the paper’s abstract, SLAM-ASR passes numerical representations of speech through a small conversion module to a language model. The language model’s weights remain fixed.
The researchers found that the same induction heads that copy repeated sequences of text tokens, or units of text, also help map speech representations to tokens in the transcript.
The researchers found that some attention heads worked in pairs to cancel out noise in the attention signal. The model then focused more strongly on the word it had just generated.
The researchers suggested that induction heads could be used to improve transcription when the audio contains noise.
The abstract did not quantify any improvement in transcription quality on noisy audio.