A practical comparison of the quantization formats that decide whether a model fits your GPU: weight-only versus activation quantization, what calibration data actually buys you, the accuracy cost of int4, and which of GGUF, GPTQ, AWQ, and FP8 makes sense for which workload.
Int4
-
LLM Quantization Compared: GGUF, AWQ, GPTQ, FP8, and the int4 Cliff