TFLiteConverter and Post-Training Quantization

Three ways to shrink a model 4–10×

The tf.lite.TFLiteConverter has three factory methods: from_saved_model (recommended), from_keras_model, and from_concrete_functions. The SavedModel path produces the most complete and optimized conversion.

The three quantization tiers, each smaller and faster than the last:

None (float32 baseline) — direct conversion, no size reduction. Same accuracy as the original.
Dynamic range (optimizations=[tf.lite.Optimize.DEFAULT]) — weights statically quantized to int8, activations quantized at runtime. ~4× smaller, 2–3× faster CPU. No calibration data needed. Easiest first step.
Full integer (int8) — both weights AND activations as int8. Requires a representative dataset (100–200 samples) to calibrate activation ranges. ~4× smaller, 3×+ faster. Required for Edge TPU and microcontrollers.

Code

3 tier quantization·python

import tensorflow as tf
import numpy as np

# Tier 1: baseline (no quantization)
conv1 = tf.lite.TFLiteConverter.from_saved_model('my_saved_model/')
tflite_baseline = conv1.convert()

# Tier 2: dynamic range — easiest, no calibration data
conv2 = tf.lite.TFLiteConverter.from_saved_model('my_saved_model/')
conv2.optimizations = [tf.lite.Optimize.DEFAULT]
tflite_dynamic = conv2.convert()

# Tier 3: full integer — requires calibration dataset
def representative_dataset():
    # Use REAL training data here, NOT random noise
    for sample in train_samples[:200]:
        yield [sample[None].astype(np.float32)]

conv3 = tf.lite.TFLiteConverter.from_saved_model('my_saved_model/')
conv3.optimizations = [tf.lite.Optimize.DEFAULT]
conv3.representative_dataset = representative_dataset
conv3.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
conv3.inference_input_type = tf.int8       # for Edge TPU / MCU
conv3.inference_output_type = tf.int8
tflite_int8 = conv3.convert()

for name, model_bytes in [('baseline', tflite_baseline),
                           ('dynamic', tflite_dynamic),
                           ('int8', tflite_int8)]:
    print(f"{name}: {len(model_bytes) / 1024:.1f} KB")

TFLiteConverter and Post-Training Quantization

Three ways to shrink a model 4–10×

Code

Progress

Comments 0