Edge Deployment — ExecuTorch, CoreML, MLX

Three paths off the GPU

Server inference is one deployment target. Mobile and edge are very different. The PyTorch ecosystem has dedicated tools:

ExecuTorch — PyTorch's mobile / edge runtime. Targets iOS, Android, and microcontrollers. The successor to the old PyTorch Mobile.
CoreML — Apple's on-device ML framework. Maximum performance on iOS / macOS. Convert from PyTorch via the coremltools bridge.
MLX — Apple's native ML framework for Apple Silicon. Built around the unified memory architecture. The right choice when you're squeezing every last bit out of a Mac or iPhone.

The flow

For ExecuTorch and CoreML, the modern path is the same: torch.export → backend-specific lowering. ExecuTorch lowers to a .pte file you ship with your app. CoreML lowers to .mlpackage.

For MLX, you have two options: convert PyTorch weights to MLX format (works for many architectures) or rewrite the model in MLX directly (best perf, but a porting effort).

Code

ExecuTorch — export for mobile·python

# pip install executorch
import torch
from executorch.exir import to_edge

class TinyMLP(torch.nn.Module):
    def __init__(self): super().__init__(); self.fc = torch.nn.Linear(10, 4)
    def forward(self, x): return self.fc(x)

model = TinyMLP().eval()
example = torch.randn(1, 10)

# 1. Export with torch.export
exported = torch.export.export(model, (example,))

# 2. Lower to ExecuTorch's edge IR
edge = to_edge(exported)

# 3. Optimize and serialize
et_program = edge.to_executorch()
with open('/tmp/tiny.pte', 'wb') as f:
    f.write(et_program.buffer)

# .pte ships with your iOS / Android app

CoreML — Apple device deployment·python

# pip install coremltools
import torch
import coremltools as ct

class TinyMLP(torch.nn.Module):
    def __init__(self): super().__init__(); self.fc = torch.nn.Linear(10, 4)
    def forward(self, x): return self.fc(x)

model = TinyMLP().eval()
example = torch.randn(1, 10)

# Trace the model (CoreML's converter still uses tracing under the hood)
traced = torch.jit.trace(model, example)

mlmodel = ct.convert(
    traced,
    inputs=[ct.TensorType(shape=example.shape, name='x')],
    convert_to='mlprogram',                # modern MLProgram format
    minimum_deployment_target=ct.target.macOS14,
)
mlmodel.save('/tmp/tiny.mlpackage')

# Drop the .mlpackage into Xcode and you have a CoreML model

MLX — native Apple Silicon, two paths·python

# pip install mlx mlx-lm
import mlx.core as mx
import mlx.nn as nn

# Path 1: re-implement in MLX (best performance)
class MLXMLP(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc = nn.Linear(10, 4)
    def __call__(self, x):
        return self.fc(x)

model = MLXMLP()
x = mx.random.normal((1, 10))
y = model(x)                                  # eager-style execution
mx.eval(y)                                     # force evaluation
print(y.shape)                                 # (1, 4)

# Path 2: load PyTorch weights into MLX
# Many community projects (mlx_lm) support direct loading of HF checkpoints.
# from mlx_lm import load
# model, tokenizer = load("mlx-community/Llama-3.2-3B-Instruct-4bit")

Picking deployment target by hardware·python

# A quick decision table:
#
# Target               | Recommended path
# --------------------- | ----------------------------------------------------
# iOS / iPadOS         | CoreML (best Apple integration) or ExecuTorch
# Android              | ExecuTorch (with NNAPI / Vulkan delegate)
# macOS (Apple Silicon)| MLX (native) or CoreML
# Linux server (GPU)   | torch.compile + bf16, or vLLM for LLMs
# Linux server (CPU)   | torch.compile + ONNX Runtime, OpenVINO
# NVIDIA Jetson / edge | TensorRT (via ONNX export)
# Browser              | ONNX Runtime Web, transformers.js

Edge Deployment — ExecuTorch, CoreML, MLX

Three paths off the GPU

The flow

Code

External links

Exercise

Progress

Comments 0