Performance Tuning

The five environment variables that matter

Variable	What it does	Default value
`OLLAMA_FLASH_ATTENTION`	Forces Flash Attention on (`1`) or off (`0`). Ollama already uses it automatically wherever the backend and devices support it, so this is an override, not a switch you have to find	automatic
`OLLAMA_KV_CACHE_TYPE`	Quantize KV cache (`q8_0` halves it; `q4_0` quarters it). Needs Flash Attention, which you already have	`f16`
`OLLAMA_NUM_PARALLEL`	Concurrent requests per model	1
`OLLAMA_MAX_LOADED_MODELS`	How many models can be in memory simultaneously	3
`OLLAMA_KEEP_ALIVE`	How long models stay loaded after a request	`5m`

Persisting on macOS

Ollama on macOS is launched by launchd, which doesn't see your shell's export. Set persistent env vars with launchctl setenv, then restart the Ollama service so it picks them up.

Context window math

Each parallel request multiplies KV cache. num_ctx=8192 with NUM_PARALLEL=4 = 32K of effective KV cache. OLLAMA_KV_CACHE_TYPE=q8_0 halves that. The default num_ctx on most models is 4096 — bump to 8192 or 16384 for real use, but watch memory.

EV v3 · 2026-09-18: The table said OLLAMA_FLASH_ATTENTION defaults to off and called it the single biggest performance win, and a callout told you to set it if you did nothing else. Ollama's FAQ is explicit that it “uses Flash Attention automatically when the selected backend and devices support it”; the variable forces it on or off. The advice was aimed at a switch already flipped, so following it changed nothing on most machines. The KV-cache row was wrong in a smaller way: the documented default is f16, not fp16, and f16 / q8_0 / q4_0 are the accepted spellings. The callout now points at the quantized KV cache, which is a real choice you still make. The ops-track quiz that keyed the same claim moved with it.

Code

Persistent tuning on macOS·bash

# Set the five knobs (persists across restarts via launchd)
launchctl setenv OLLAMA_FLASH_ATTENTION 1
launchctl setenv OLLAMA_KV_CACHE_TYPE q8_0
launchctl setenv OLLAMA_NUM_PARALLEL 2
launchctl setenv OLLAMA_MAX_LOADED_MODELS 3
launchctl setenv OLLAMA_KEEP_ALIVE 30m

# Restart Ollama so the daemon picks up the new env
osascript -e 'tell application "Ollama" to quit'
open -a Ollama

# Verify the daemon sees them
curl -s http://localhost:11434/api/version
ps eauwx | grep -i ollama | head -1   # check env in process listing

Linux (systemd) version·bash

# Edit the systemd unit — the official install puts these here:
sudo systemctl edit ollama.service
# Add under [Service]:
#   Environment="OLLAMA_FLASH_ATTENTION=1"
#   Environment="OLLAMA_KV_CACHE_TYPE=q8_0"
#   Environment="OLLAMA_NUM_PARALLEL=2"
#   Environment="OLLAMA_MAX_LOADED_MODELS=3"
#   Environment="OLLAMA_KEEP_ALIVE=30m"

sudo systemctl daemon-reload
sudo systemctl restart ollama

Exercise

Set the five env vars on your machine (persistent — launchctl setenv or a systemd unit edit). Restart Ollama. Now measure the two things these knobs actually move, because none of them speeds up a single warm decode: (1) time the same prompt on a 7B model *cold* and then *warm*, and see how much OLLAMA_KEEP_ALIVE is worth on the second call; (2) note the resident memory in ollama ps with OLLAMA_KV_CACHE_TYPE unset and then at q8_0, at a num_ctx large enough to matter. If a warm decode did get faster, write down what else changed — it was not these variables.