Where MLX is heading, as of 2026-05
The previous lessons of this quest are about MLX as it is today. This final lesson is the forecast — the three threads worth watching for what MLX will be in the next year or two. Reading these lets you make better deployment decisions today (knowing what's likely to land) and better learning decisions (knowing where to invest your attention).
JACCL — distributed inference across Thunderbolt-connected Macs
JACCL (pronounced “jackal”) is the communication backend behind mx.distributed for Macs cabled together over Thunderbolt 5. It rides RDMA over Thunderbolt, which Apple shipped in macOS 26.2, and MLX's documentation puts its latency an order of magnitude below the older ring backend. That is what makes tensor parallelism possible across machines: every layer of the model is split between the Macs, so they have to exchange results for every layer of every token.
JACCL works from macOS 26.2 (MLX v0.30.1, December 2025), and MLX v0.30.6 made its bandwidth much faster on macOS 26.3 and later. The rough edges are in the setup, not the API. RDMA has to be switched on with rdma_ctl enable from macOS Recovery on every machine — MLX's docs say it cannot be done remotely, even with sudo — and JACCL accepts only a fully connected mesh, a direct cable between every pair of Macs. The mlx-lm side is already there: model-parallel generation arrived in mlx-lm v0.30.0 with an example launched through mlx.launch --backend jaccl, and v0.30.6 brought distributed inference to its server. mlx-lm splits a model layer by layer when the model family supports tensor parallelism and in stages when it does not, so whether your model can be split well is answered per model family in the mlx-lm release notes.
What pooling buys, and what it doesn't — Cable two 512 GB Studios together and a model too big for either one can load. That settles one question: does it fit? Whether it is usable is a separate question, and pooling does not answer it. Every generated token still has to read the model's active weights out of memory, and once the model is split across machines, the machines also wait on each other over the cable. Splitting each layer across machines can win some speed back; splitting the model into stages mostly buys fit, because a single reply still walks through every stage in turn. Apple's own M5 benchmark draws the underlying line: time-to-first-token is compute-bound, while generating tokens is memory-bandwidth-bound. Loading a model is not the same as using it.
M5 Neural Accelerators — the next hardware step
The M5-class GPUs put a Neural Accelerator in every GPU core: dedicated hardware for matrix multiplication, the operation most machine learning spends its time on. MLX reaches it through the tensor operations and Metal Performance Primitives that Metal 4 introduced, which is why MLX's M5 support (v0.30.0) needs macOS 26.2 or later. MLX is not the only way in. Apple says apps built on Core ML, Metal Performance Shaders and Metal 4 pick up the speedup automatically, and developers can program the accelerators directly with Metal 4's Tensor APIs. What MLX gives you is a way to use them from the same Python code you research and serve with.
Read the headline number carefully. Apple's MLX benchmark on M5 reports up to 4× faster time-to-first-token than M4 — that is prefill, the compute-bound half of inference, where a matrix accelerator helps most. Generating the reply is bounded by memory bandwidth instead, and there the same benchmark shows 19–27% over M4, in line with the M5's 28% higher bandwidth (153 GB/s against 120 GB/s). New matrix hardware speeds up reading the prompt; the reply still waits on memory.
CUDA backend — MLX as a cross-platform option
Since v0.27, MLX has had a CUDA backend on Linux. As of 2026-05 it's incomplete (some op coverage gaps, performance behind the native Apple Silicon path), but it exists and is improving every release. The implications are larger than they look — if you can write MLX-shape code that runs natively on Apple Silicon AND on NVIDIA Linux machines, MLX stops being "the Mac framework" and becomes "the framework that runs on whatever hardware you have."
This won't happen quickly. JAX has had multiple backends for years and the per-backend coverage gaps still bite. But the trajectory is real: MLX's API is good enough that being able to use it on multiple backends is genuinely valuable, and the CUDA work is the first concrete step toward that.
What this all adds up to
MLX in 2026-05 is the framework you reach for on Apple Silicon for ML research, LLM serving, and fine-tuning. By 2027-05 (the realistic window for the threads above to mature), it has a real shot at being:
- The framework you reach for whenever your deployment hardware is Apple Silicon, from a single Mac to a Thunderbolt-connected cluster via JACCL — where the question for a cluster is whether the model is usable, not only whether it loads.
- A direct way into the M5-class Neural Accelerators from research code, alongside Core ML and Metal 4 rather than instead of them.
- A credible cross-platform option for code that needs to run on both Apple Silicon and NVIDIA, replacing some of what PyTorch MPS does today.
None of these are guaranteed. All three are visible enough that planning around them isn't speculative. Watch the release notes for ml-explore/mlx and ml-explore/mlx-lm; the threads above will land there first.