Speculative Speculative Decoding: Eliminating the Last Sequential Bottleneck in LLM Inference
How speculating about speculation itself achieves up to 5x faster LLM inference by eliminating the draft model’s idle time during verification, and the three engineering challenges that make it work.