The 'LLMs Write Bad MLX Code' Problem (and How I Survive It)

The meta-problem nobody warns you about

Here's the dirty secret of working with MLX in 2026: when you ask an LLM coding assistant to write MLX code, it confidently produces output that looks plausible and is often wrong. Functions that don't exist (mx.linalg.det, anyone?). Method signatures that were renamed two releases ago. Patterns from PyTorch that don't translate. The output is fluent, syntactically valid Python — and broken.

This is not the LLMs' fault, exactly. MLX is too young and too distinct from PyTorch to live in pretrained-LLM weights at the level of fluency they manage for older frameworks. The training data has thousands of examples of torch.tensor.to(device) and a few dozen of mlx.core.array. The model averages, hallucinates, and produces confident wrong answers.

This lesson is the field guide to surviving it. The rituals I use to keep myself honest, and the disposition that lets you treat LLM-written MLX code as a rough draft you must verify, never as a finished artifact.

Smell the hallucination

Common patterns of bad LLM-generated MLX code:

References to functions that don't exist — the LLM extrapolates from PyTorch / NumPy / JAX and invents an MLX equivalent that sounds right. mx.linalg.det, mx.tensor, mx.no_grad(), mx.cuda.is_available(). Ask yourself: "would I bet money this exists?" If you can't, look it up.
Mixed-up API surfaces — code that mixes torch.tensor and mx.array, or imports mlx.nn as nn and torch.nn as nn without realizing. The LLM is patching together patterns from multiple frameworks.
Stale API names — code using API names that mlx-lm or mlx-vlm renamed in the last few releases. The LLM's training cutoff is the wrong date for MLX.
Suspiciously confident architecture details — "the default group_size for quantization is 32" (it's 64 in mlx-lm). The LLM fills in details rather than admitting uncertainty.

The verification rituals

Run it. Always. The fastest way to catch a hallucinated function is to run the snippet — Python's import-time errors are unforgiving in a useful way.
Check against the real source. mlx_lm's GitHub repo is small enough to grep — when in doubt, search for the function name in the actual code.
Use --help and dir(). python -m mlx_lm convert --help tells you the real flag list. dir(mlx.core) in a REPL tells you what's actually exposed.
Cite the version. When you ask an LLM about MLX, prefix with "as of mlx 0.31.x" or whatever version you're on. Doesn't fix the problem, but makes the model less likely to draw on stale snippets.
Trust mlx-examples and mlx-lm source over LLM output. The reference repos are the ground truth; the LLM is a paraphrase.

What I do differently when writing this quest

Every code block in MLX Quest was actually run in the verification env before it was written down. Not just "compiled in my head" — actually run, with the actual output captured and pasted into the comments. That's the only way I trust the code I'm asking you to learn from. The lesson is: do the same when you're writing MLX in production. Run before you ship.

Code

The verification ritual — run before you trust·bash

# When an LLM gives you MLX code, before you commit it:
# 1. Save to a file
# 2. Run it
# 3. Compare output to what was claimed
conda activate mlx
python my-llm-suggestion.py

# If you get an AttributeError or ImportError, the LLM hallucinated.
# If it runs but the output is wrong, the LLM hallucinated more subtly.
# If it runs and the output matches, the snippet earned your trust.

Spot-check what's actually in the API·python

# When you suspect a function was hallucinated:
import mlx.core as mx

# Does mx.linalg.det exist?
try:
    mx.linalg.det
    print("exists")
except AttributeError:
    print("does NOT exist — LLM hallucinated")

# What IS in mx.linalg?
print([n for n in dir(mx.linalg) if not n.startswith('_')])

# Verified output (2026-05-03):
#   det does NOT exist
#   ['cholesky', 'cholesky_inv', 'cross', 'eig', 'eigh', 'eigvalsh', 'inv',
#    'lu', 'lu_factor', 'norm', 'pinv', 'qr', 'solve', 'solve_triangular',
#    'svd', 'tri_inv', 'tri_solve']

Exercise

Take a recent LLM-generated MLX snippet (or ask a coding assistant to "write me an MLX snippet that computes the determinant of a matrix"). Run it in your mlx env. If it fails, identify the hallucinated function. Then look up what the correct path is — there are workarounds for missing functions like det via SVD or LU decomposition. Two sentences on what failed and how you fixed it.