Smaller weights, faster inference
A trained model stores its weights as 32-bit floats. Quantization re-expresses those weights with far fewer bits — int8 instead of float32 shrinks the model roughly 4× and speeds up inference, because integer math is cheaper and you move a quarter of the bytes through memory. That last part is the real win on edge, mobile, and serverless, where memory bandwidth and cold-start size, not raw FLOPs, are what hurt.
One call in Keras 3
Keras 3 exposes model.quantize("int8") as a direct, in-place conversion — no separate toolkit, no graph export. "int4" (Keras 3.11+) pushes compression to ~8× for the cases that can tolerate it, and type_filter=["Dense"] lets you quantize only the layer types that are safe to squeeze while leaving sensitive ones in full precision.
PTQ vs QAT
What quantize() does is post-training quantization (PTQ): convert the already-trained weights, no retraining. It's nearly free and int8 usually costs under ~0.5% accuracy. When that loss is too much — often at int4 — you move to quantization-aware training (QAT), which simulates the low-precision rounding during training so the model learns to absorb it. Start with PTQ; reach for QAT only when the numbers force you. Either way, benchmark on your own task — the accuracy hit is data-dependent, not a constant.