Engines and Ecosystem

The engine landscape in one screen

Tool	Role	Best for
Ollama	Local server with REST API + CLI	App integration, daily driver, fallback path
llama.cpp	Foundational C/C++ inference engine	Custom builds, embedded, raw control
LM Studio	Desktop GUI app for local models	Exploration, beginners, model comparison
mlx-lm	Python API on top of MLX	Apple Silicon power users, custom pipelines
vLLM	Throughput-optimized batched server	Multi-user serving, high QPS
TGI (Text Generation Inference)	HuggingFace's production server	HF-native deploys, sharded multi-GPU
HuggingFace Hub	Model registry	Where almost every open model lives

What this quest emphasizes

Ollama is the spine of this quest because it's the simplest correct path from nothing installed to local model behind a stable HTTP API. Most everything we build later — streaming, tool use, vision, the adapter pattern — is shown against Ollama's API and then mapped to the others. The serving track later demystifies vLLM, llama.cpp server, and TGI as variants of the same idea.

Don't conflate engine with model

The engine runs the model. The model is the weight file (GGUF / MLX / safetensors). You can run the same Qwen 2.5 7B model in Ollama, llama.cpp server, LM Studio, or mlx-lm. Pick the engine that fits your operational shape; the model itself is portable.

Code

One model, four engines·bash

# Same Qwen 2.5 7B Instruct, four ways:

# 1. Ollama (background daemon + REST API)
ollama pull qwen2.5:7b
curl http://localhost:11434/api/chat -d '{"model":"qwen2.5:7b","messages":[{"role":"user","content":"hi"}],"stream":false}'

# 2. llama.cpp server (manual GGUF)
# (after building llama.cpp)
./llama-server -m qwen2.5-7b-instruct-q4_k_m.gguf -c 4096

# 3. LM Studio (GUI)
# Pick the model in the LM Studio UI and click "Local server"

# 4. mlx-lm (Apple Silicon, MLX format)
mlx_lm.generate --model mlx-community/Qwen2.5-7B-Instruct-4bit --prompt "hi"

The engine landscape in one screen

What this quest emphasizes

Don't conflate engine with model

Code

External links

Exercise

Progress

Comments 0