The engine landscape in one screen
| Tool | Role | Best for |
|---|---|---|
| Ollama | Local server with REST API + CLI | App integration, daily driver, fallback path |
| llama.cpp | Foundational C/C++ inference engine | Custom builds, embedded, raw control |
| LM Studio | Desktop GUI app for local models | Exploration, beginners, model comparison |
| mlx-lm | Python API on top of MLX | Apple Silicon power users, custom pipelines |
| vLLM | Throughput-optimized batched server | Multi-user serving, high QPS |
| TGI (Text Generation Inference) | HuggingFace's server, in maintenance mode since December 2025 | Deployments that already run it; HF points new work at vLLM or SGLang |
| HuggingFace Hub | Model registry | Where almost every open model lives |
What this quest emphasizes
Ollama is the spine of this quest because it's the simplest correct path from nothing installed to local model behind a stable HTTP API. Most everything we build later — streaming, tool use, vision, the adapter pattern — is shown against Ollama's API and then mapped to the others. The serving track later demystifies vLLM, llama.cpp server, and TGI as variants of the same idea.
Don't conflate engine with model
The engine runs the model. The model is the weight file (GGUF / MLX / safetensors). You can run the same Qwen 2.5 7B model in Ollama, llama.cpp server, LM Studio or mlx-lm — but not the same file. Ollama and llama.cpp want GGUF, mlx-lm wants an MLX directory, vLLM and TGI want HuggingFace safetensors. Popular models exist in all of them, so switching engines costs a download or a conversion rather than a copy. Pick the engine that fits your operational shape; the model travels, the file usually does not.