Fp8

• Jun 22, 2026 KV Cache Engineering: PagedAttention, Continuous Batching, Attention Variants, and the Bandwidth Wall

  The KV cache is the single biggest lever for LLM inference cost and throughput. How it works, why naive allocation wastes 60–80% of GPU memory, how PagedAttention and continuous batching fix that, what prefix caching actually saves, how FP8/INT8 quantization and GQA/MLA shrink the cache, and why long context is fundamentally a memory-bandwidth problem.

• Jun 22, 2026 LLM Quantization Compared: GGUF, AWQ, GPTQ, FP8, and the int4 Cliff

  A practical comparison of the quantization formats that decide whether a model fits your GPU: weight-only versus activation quantization, what calibration data actually buys you, the accuracy cost of int4, and which of GGUF, GPTQ, AWQ, and FP8 makes sense for which workload.