Decode Before You Hear: Speculative Listening for Chunk-Interleaved Full-Duplex Speech Language Models

Decode Before You Hear: Speculative Listening for Full-Duplex Speech Language Models

Most existing full-duplex speech models alternate between listening and speaking in fixed time chunks, which leaves the load across chunks highly uneven: compute sits idle during "listening", while "speaking" and tool calls miss their deadlines and fall behind because they generate many tokens serially. We observe that with silence as the hypothesized input, the output of 95.3% of chunks is identical to the output produced from the real audio, and 88% of divergences appear at the very first token. Building on this, Speculative Listening uses the idle compute of listening to draft the next chunk before the real audio arrives, using silence and the runner-up first-token candidate. All drafts are then verified together once the audio arrives, which requires no additional training and guarantees lossless output.

Speed: On TickTock-Bench, the eight-task evaluation suite we built, speaking chunks are accelerated by 2.6–2.8×, and tool-call chunks by up to 23.4×.

Latency: Decoding latency before a response begins drops by 73% on average, to 74–241 ms, close to the 200 ms turn-taking gap in human conversation. The maximum lag behind real time is reduced by 94.4%.