A rigorous look at every major LLM quantization format: how GPTQ's Hessian-guided rounding works, why AWQ's activation-aware scaling beats naive per-channel approaches, when GGUF k-quants are the right choice, where AQLM wins at sub-3-bit, and how MLX handles Apple Silicon. Includes conversion workflows, a perplexity shootout, and a decision guide.
Performance
-
Quantization Deep Dive: GPTQ, AWQ, GGUF, AQLM, and MLX -
Semantic Caching for LLM Applications: Cutting Cost and Latency How embedding-based semantic caching works, why threshold tuning is the hardest part, cache invalidation patterns that prevent staleness, and practical implementations with GPTCache, LiteLLM, and a from-scratch Redis + FastAPI setup.
-
Speculative Decoding: Draft Models, EAGLE, and How to Actually Use It How speculative decoding achieves 2–4x inference speedup without changing model outputs, covering the rejection-sampling proof, EAGLE and EAGLE2 draft strategies, ngram lookahead, and production configuration for vLLM and llama.cpp.
-
GPU Programming Without CUDA: OpenCL, ROCm, Vulkan, Metal, and WebGPU The full landscape of non-NVIDIA GPU compute: OpenCL for portability, ROCm on AMD hardware, Vulkan Compute shaders, Metal for Apple Silicon, and WebGPU for cross-platform GPU work in the browser and beyond.
-
Linux Memory Management Deep Dive A comprehensive guide to Linux virtual memory, huge pages, NUMA topology, OOM killer tuning, /proc/meminfo interpretation, and using perf mem to find memory bottlenecks.