Beyond Ollama — Engine Map · Local AI Quest

The four engines that matter beyond Ollama

Engine	Strength	When to reach for it
llama.cpp server	Raw GGUF execution; portable C/C++; low overhead	Embedded, edge, custom builds, when you need control of every flag
vLLM	Throughput; continuous batching; PagedAttention	Multi-user serving; high QPS; sharded multi-GPU
TGI (HF Text Generation Inference)	HuggingFace-native; in maintenance mode since December 2025 — HF now points users at vLLM and SGLang	Existing TGI deployments; new work starts on vLLM or SGLang
LM Studio	GUI; built-in OpenAI-compat server	Exploration, model comparison, demo days

What they share

All four expose an OpenAI-compatible chat completions endpoint at some path (usually /v1/chat/completions). They all support streaming. The differences are operational: throughput model, hardware target, ergonomics. The model travels; the file usually does not. Ollama and llama.cpp want GGUF, vLLM and TGI want HuggingFace safetensors, and MLX wants its own. Popular weights exist in every format, so you re-pull rather than copy — budget a conversion or a download when you switch engines, not a file move.

Should you switch off Ollama?

Probably not, for most projects. Ollama covers ~90% of local serving needs with the lowest setup cost. Switch to llama.cpp server when you need a specific GGUF that Ollama doesn't ship; switch to vLLM when you're serving many concurrent users; and if your team already lives in the HuggingFace ecosystem, reach for vLLM or SGLang rather than TGI — those are what HuggingFace itself now recommends.

EV v3 · 2026-09-18: One sentence here was doing real damage: “the model itself (the weights) is portable across all of them”. Read as written, that says one file works in every engine, and it does not — Ollama and llama.cpp load GGUF, vLLM and TGI load HuggingFace safetensors. Popular models exist in both, which is what makes the switch cheap, but the cost is a re-download or a conversion, not a copy. Separately, TGI: its README has opened by declaring maintenance mode and recommending vLLM or SGLang since 2025-12-11, five months before this quest published — so calling it the HuggingFace-native production server was wrong on day one, not something that aged. Every site that said so moved: this table, the track card, the why-track engine map, and the paragraph below telling you to switch to TGI when your team lives in HuggingFace.

Code

Quick smoke tests across engines·bash

# Ollama (default)
curl -s http://localhost:11434/api/version

# llama.cpp server (assuming you started it on 8080)
curl -s http://localhost:8080/v1/models | python3 -m json.tool

# vLLM (default port 8000)
curl -s http://localhost:8000/v1/models | python3 -m json.tool

# TGI (default port 8080)
curl -s http://localhost:8080/info | python3 -m json.tool

# Drop-in OpenAI client at any of them
python3 - <<'PY'
from openai import OpenAI
client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")
print(client.models.list())
PY

Beyond Ollama — Engine Map

The four engines that matter beyond Ollama

What they share

Should you switch off Ollama?

Code

External links

Exercise

Progress

Comments 0