Tokens are scoped, not all-powerful
HF user access tokens come in three scopes: read, write, and fine-grained. Read is enough to download public models, accept gated licenses, and call the Inference API. Write adds repo creation, push, and deletion. Fine-grained tokens (the modern default for service accounts) let you whitelist specific orgs and specific repos — you should be using these in any non-trivial CI setup.
Where the token lives
After huggingface-cli login the token is stored at ~/.cache/huggingface/token (override with HF_HOME). All clients (transformers, datasets, huggingface_hub, accelerate) read it from there. You can override per-process with the HF_TOKEN environment variable; this is the standard pattern in Docker images and CI.
Token rotation, the boring rule
Every token has a created-at timestamp and a last-used timestamp visible in huggingface.co/settings/tokens. If a token has been on a CI runner for > 90 days, rotate it. If a token has read scope and somehow ends up in a public repo, the blast radius is "you can download what they could already download" — but if it's a write token, that's a real incident. Rotate, audit pushes, and consider GitHub's secret scanning push protection.