TFLite Interpreter and Hardware Delegates

Where the model actually runs

The TFLite Interpreter is the runtime that executes .tflite models on device. On Python (useful for testing on desktop before deploying to mobile), use tf.lite.Interpreter.

Delegates let TFLite offload computation to specialized hardware. Each delegate is a plugin — TFLite tries each supported op on the delegate first, falls back to CPU for unsupported ops.

Delegate	Hardware	Platform	Supported quant
GPU Delegate	OpenCL/Metal GPU	Android, iOS, Linux	fp32, fp16, dynamic range
Core ML Delegate	Neural Engine (A12+)	iOS 12+	fp32, fp16
Edge TPU	Coral Edge TPU	Linux/Win/macOS	full int8 only
NNAPI	Neural Proc Unit	Android 8.1–14	varies by device

The delegate choice and quantization choice are coupled. Targeting Coral Edge TPU? You must use full int8 quantization. Targeting iOS Neural Engine? Use fp16. Picking a quantization tier without considering the deployment target wastes work.

Code

Python interpreter — desktop 테스트·python

import tensorflow as tf
import numpy as np

interpreter = tf.lite.Interpreter(model_path='model.tflite')
interpreter.allocate_tensors()

input_details  = interpreter.get_input_details()
output_details = interpreter.get_output_details()

print("Input shape:", input_details[0]['shape'])
print("Input dtype:", input_details[0]['dtype'])

# Run inference
input_data = np.random.rand(1, 224, 224, 3).astype(np.float32)
interpreter.set_tensor(input_details[0]['index'], input_data)
interpreter.invoke()

output = interpreter.get_tensor(output_details[0]['index'])
print("Predictions:", output)

Edge TPU delegate·python

import tflite_runtime.interpreter as tflite

# Coral Edge TPU — requires libedgetpu installed
interpreter = tflite.Interpreter(
    model_path='model_edgetpu.tflite',
    experimental_delegates=[
        tflite.load_delegate('libedgetpu.so.1')      # Linux
        # tflite.load_delegate('libedgetpu.1.dylib')  # macOS
        # tflite.load_delegate('edgetpu.dll')          # Windows
    ]
)
interpreter.allocate_tensors()

TFLite Interpreter and Hardware Delegates

Where the model actually runs

Code

Progress

Comments 0