How LLMs Get Faster Without Changing Their Answers | Speculative Decoding
Jia-Bin Huang · 24:34
Speculative decoding speeds up large-language-model generation without changing the target model’s sampling distribution: a cheap drafter proposes several tokens (or features), the large model verifies them in one par...