Performance Tuning

~22 min · ops, performance

Level 0Downloader

0 XP0/41 lessons0/11 achievements

0/120 XP to next level120 XP to go0% complete

The five environment variables that matter

Variable	What it does	Default value
`OLLAMA_FLASH_ATTENTION`	Enables FlashAttention — usually the single biggest perf win	off (set to `1`)
`OLLAMA_KV_CACHE_TYPE`	Quantize KV cache (`q8_0` halves it; `q4_0` quarters it)	fp16
`OLLAMA_NUM_PARALLEL`	Concurrent requests per model	1
`OLLAMA_MAX_LOADED_MODELS`	How many models can be in memory simultaneously	3
`OLLAMA_KEEP_ALIVE`	How long models stay loaded after a request	`5m`

Persisting on macOS

Ollama on macOS is launched by launchd, which doesn't see your shell's export. Set persistent env vars with launchctl setenv, then restart the Ollama service so it picks them up.

Context window math

Each parallel request multiplies KV cache. num_ctx=8192 with NUM_PARALLEL=4 = 32K of effective KV cache. OLLAMA_KV_CACHE_TYPE=q8_0 halves that. The default num_ctx on most models is 4096 — bump to 8192 or 16384 for real use, but watch memory.

Code

Persistent tuning on macOS·bash

# Set the five knobs (persists across restarts via launchd)
launchctl setenv OLLAMA_FLASH_ATTENTION 1
launchctl setenv OLLAMA_KV_CACHE_TYPE q8_0
launchctl setenv OLLAMA_NUM_PARALLEL 2
launchctl setenv OLLAMA_MAX_LOADED_MODELS 3
launchctl setenv OLLAMA_KEEP_ALIVE 30m

# Restart Ollama so the daemon picks up the new env
osascript -e 'tell application "Ollama" to quit'
open -a Ollama

# Verify the daemon sees them
curl -s http://localhost:11434/api/version
ps eauwx | grep -i ollama | head -1   # check env in process listing

Linux (systemd) version·bash

# Edit the systemd unit — the official install puts these here:
sudo systemctl edit ollama.service
# Add under [Service]:
#   Environment="OLLAMA_FLASH_ATTENTION=1"
#   Environment="OLLAMA_KV_CACHE_TYPE=q8_0"
#   Environment="OLLAMA_NUM_PARALLEL=2"
#   Environment="OLLAMA_MAX_LOADED_MODELS=3"
#   Environment="OLLAMA_KEEP_ALIVE=30m"

sudo systemctl daemon-reload
sudo systemctl restart ollama

External links

Exercise

Set the five env vars on your machine (persistent — launchctl setenv or systemd unit edit). Restart Ollama. Time the same prompt before and after on a 7B model. Note the speedup; if it's <10%, your hardware was already saturated and you'd benefit more from a smaller quant or larger model.