Where MLX is heading, as of 2026-05
The previous lessons of this quest are about MLX as it is today. This final lesson is the forecast — the three threads worth watching for what MLX will be in the next year or two. Reading these lets you make better deployment decisions today (knowing what's likely to land) and better learning decisions (knowing where to invest your attention).
JACCL — distributed inference across Thunderbolt-connected Macs
JACCL (probably the most underhyped MLX development of the past year) is the distributed-computation library that lets multiple Thunderbolt-connected Macs cooperate on a single inference workload. Two M3 Ultra Studios connected via TB5 cables can pool their unified memory and split a frontier-class model across both, sharing kernels via TB-attached compute.
As of 2026-05, JACCL requires macOS 26.3+ and is still firmly in the early-adopter zone. The pieces are there — the wire protocol, the kernel-dispatch layer, the example applications — but it's not yet drop-in for the way PyTorch's distributed training works on NVIDIA. Watch this space; the moment JACCL lands a polished mlx-lm integration is the moment two-Mac inference becomes a routine deployment pattern.
Why it matters for the office Mac fleet — Dad's office cluster of M3 Ultra Studios is exactly the hardware JACCL is designed for. Pooling four 512 GB Studios via Thunderbolt could put 2 TB of unified memory in one logical machine. That's frontier-model territory served from a desk.
M5 Neural Accelerators — the next hardware step
The M5-class chips ship with hardware-accelerated matrix multiplication units (the Neural Accelerators) that MLX is the only software door for. PyTorch MPS doesn't expose them; CoreML uses them but only through its higher-level API. MLX's --q-mode mxfp4 / nvfp4 / mxfp8 formats (you saw these in convert.lesson2) are designed to map directly to these new units.
The 4× TTFT improvement Apple cited at the launch is real on workloads that hit these paths. As more developers' inference paths get optimized for MX-format quantization, the gap between MLX-on-M5 and any other Apple-Silicon-friendly framework will widen.
CUDA backend — MLX as a cross-platform option
Since v0.27, MLX has had a CUDA backend on Linux. As of 2026-05 it's incomplete (some op coverage gaps, performance behind the native Apple Silicon path), but it exists and is improving every release. The implications are larger than they look — if you can write MLX-shape code that runs natively on Apple Silicon AND on NVIDIA Linux machines, MLX stops being "the Mac framework" and becomes "the framework that runs on whatever hardware you have."
This won't happen quickly. JAX has had multiple backends for years and the per-backend coverage gaps still bite. But the trajectory is real: MLX's API is good enough that being able to use it on multiple backends is genuinely valuable, and the CUDA work is the first concrete step toward that.
What this all adds up to
MLX in 2026-05 is the framework you reach for on Apple Silicon for ML research, LLM serving, and fine-tuning. By 2027-05 (the realistic window for the threads above to mature), it has a real shot at being:
- The framework you reach for whenever your deployment hardware is Apple Silicon, regardless of scale (single Mac to TB-connected cluster via JACCL).
- The only software path for getting full performance out of M5+ Neural Accelerators.
- A credible cross-platform option for code that needs to run on both Apple Silicon and NVIDIA, replacing some of what PyTorch MPS does today.
None of these are guaranteed. All three are visible enough that planning around them isn't speculative. Watch the release notes for ml-explore/mlx and ml-explore/mlx-lm; the threads above will land there first.