C.W.K.
Stream
Lesson 04 of 07 · published

Quantization

~8 min · advanced

Level 0Keras Apprentice
0 XP0/97 lessons0/20 achievements
0/120 XP to next level120 XP to go0% complete

Smaller weights, faster inference

A trained model stores its weights as 32-bit floats. Quantization re-expresses those weights with far fewer bits — int8 instead of float32 shrinks the model roughly 4× and speeds up inference, because integer math is cheaper and you move a quarter of the bytes through memory. That last part is the real win on edge, mobile, and serverless, where memory bandwidth and cold-start size, not raw FLOPs, are what hurt.

One call in Keras 3

Keras 3 exposes model.quantize("int8") as a direct, in-place conversion — no separate toolkit, no graph export. "int4" (Keras 3.11+) pushes compression to ~8× for the cases that can tolerate it, and type_filter=["Dense"] lets you quantize only the layer types that are safe to squeeze while leaving sensitive ones in full precision.

PTQ vs QAT

What quantize() does is post-training quantization (PTQ): convert the already-trained weights, no retraining. It's nearly free and int8 usually costs under ~0.5% accuracy. When that loss is too much — often at int4 — you move to quantization-aware training (QAT), which simulates the low-precision rounding during training so the model learns to absorb it. Start with PTQ; reach for QAT only when the numbers force you. Either way, benchmark on your own task — the accuracy hit is data-dependent, not a constant.

Code

In-place PTQ: int8, int4, and selective quantization·python
# Int8 quantization (4x smaller, faster inference)
model.quantize("int8")

# Int4 quantization (8x smaller, Keras 3.11+)
model.quantize("int4")

# Selective quantization (exclude specific layers)
model.quantize("int8", type_filter=["Dense"])  # Only Dense layers

External links

Exercise

Train a small MNIST classifier and record its accuracy and on-disk size. Call model.quantize("int8"), then model.quantize("int4") on a fresh copy. For each, re-measure accuracy and size, and note the accuracy-vs-compression curve. Where does int4 stop being worth it for you?
Hint
Save with model.save() before and after to compare file sizes; evaluate on the same test set each time so the accuracy numbers are comparable.

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign inPlease sign in to comment.

No comments yet — be the first.