Tooling Around MLX — LangChain, LlamaIndex, and Friends

MLX as the engine, other tools as the chassis

MLX-as-a-backend has been wired into the major LLM orchestration frameworks at this point. You can use MLX models from LangChain, LlamaIndex, the OpenAI SDK (via mlx-lm's server, lesson 6 of lm), and a growing list of dev tools. The integrations are usually thin shims that point the framework's existing model abstractions at a local mlx-lm server or directly at mlx_lm.load.

The right question isn't "can I use MLX with X?" — the answer is almost always yes. The right question is "is the integration worth it for my workflow?" For most projects the answer is also yes; the LLM-orchestration frameworks save you hand-rolling things like RAG pipelines, agent loops, and chat-history management. For some projects, calling mlx_lm.generate directly is more honest and easier to debug than dragging a framework into the picture.

The integrations worth knowing

OpenAI SDK — point any OpenAI client (Python, JS, Cursor, Continue, Aider) at the local mlx-lm server. Covered in lm.lesson6. The path of least friction for tools that already speak OpenAI.
LangChain — has a ChatMLX integration that wraps mlx-lm models for use inside LangChain chains, agents, and RAG pipelines. Useful if you're already invested in LangChain; not worth adopting just to talk to MLX.
LlamaIndex — has a local LLM provider that includes mlx-lm. Same shape as the LangChain story: useful inside an existing LlamaIndex codebase, overkill for one-off use.
Ollama (since v0.19, 2026-03) — built on MLX directly. If you want "a model just running" without writing Python at all, Ollama on Mac is the smoothest experience and is fundamentally an MLX wrapper.
LM Studio — desktop GUI that runs mlx-lm models with no code at all. The right tool for non-programmer users on Mac, and a respectable debugging environment when you want a graphical view of what mlx-lm is doing.

The integration that's noticeably absent

As of 2026-05, MLX-native equivalents of training-side ecosystems (DeepSpeed, Accelerate, training-cluster orchestration) are thin to nonexistent. The Mac single-machine deployment story is strong; the multi-machine training story isn't really there yet. JACCL (covered briefly in prod.lesson6) is the most active distributed-training research area but isn't yet drop-in-replacement for the PyTorch distributed-training tooling.

The integration mental model

Treat MLX integrations as adapters that let your existing framework see an MLX model. The MLX model itself is the constant — the same load + generate shape under the hood. The framework wrapper just translates between its own abstractions and MLX's. When debugging an integration issue, the first move is usually to skip the wrapper and call mlx-lm directly to confirm the model works; the bug is usually in the wrapper, not in MLX.

Code

MLX through LangChain (the typical pattern)·python

# Requires `pip install langchain langchain-community`
from langchain_community.llms.mlx_pipeline import MLXPipeline
from langchain.prompts import PromptTemplate
from langchain.chains import LLMChain

llm = MLXPipeline.from_model_id(
    model_id="mlx-community/Llama-3.2-1B-Instruct-4bit",
    pipeline_kwargs={"max_tokens": 100, "temp": 0.7},
)

prompt = PromptTemplate(
    input_variables=["topic"],
    template="Write a one-sentence joke about {topic}.",
)

chain = LLMChain(llm=llm, prompt=prompt)
print(chain.run("MLX"))

MLX through Ollama (zero-Python path)·bash

# Install Ollama on Mac (https://ollama.com)
brew install ollama

# Pull and run a model — Ollama handles the MLX backend transparently
ollama pull llama3.2
ollama run llama3.2

# Or via API:
curl http://localhost:11434/api/generate -d '{"model": "llama3.2", "prompt": "Hello"}'

Exercise

Pick one orchestration framework you'd actually use (LangChain, LlamaIndex, or Ollama) and wire it to an MLX model. Run a single prompt through it and confirm output. Then run the same prompt directly through mlx_lm.generate and compare — same result? Different sampling defaults? Two sentences on whether the framework integration paid for itself in your workflow.