How draft-model speculative sampling makes LLM inference 2-3x faster without changing the output: why decode is bandwidth-bound, the draft-verify-accept loop, the rejection-sampling step that keeps it lossless, the acceptance-rate math that decides the speedup, EAGLE and Medusa as learned drafters, and the batch-size regime where speculation quietly makes things slower.
Latency
-
Speculative Decoding Explained: Trading Compute for Bandwidth to Decode Faster -
DDR5 and Why Memory Latency Stopped Improving DRAM capacity grew 128x and bandwidth 20x in two decades while latency barely moved 1.3x. Why the time to fetch the first byte has been stuck near 13 nanoseconds since DDR3, what DDR5 actually changed — dual sub-channels, more bank groups, on-die ECC, the PMIC — and why every one of those wins is about bandwidth and reliability, not the latency wall it cannot break.