Tasks, Auto Classes, and from_pretrained()

The Auto* classes are dispatchers

Below the pipeline() abstraction sit the Auto* classes: AutoTokenizer, AutoModel, AutoModelForCausalLM, AutoModelForSequenceClassification, AutoModelForSeq2SeqLM, AutoModelForImageClassification, AutoFeatureExtractor, AutoProcessor, ... — one per task family.

Each Auto class wraps a registry: from a config.json field "model_type": "llama" it dispatches to LlamaTokenizer + LlamaForCausalLM. You almost never instantiate the concrete class directly; you call AutoModelForXxx.from_pretrained(repo_id) and the right class is chosen.

The four loader knobs that matter

torch_dtype — "auto" reads dtype from config; torch.float16/torch.bfloat16 halves memory; torch.float32 only if you need it.
device_map — "auto" shards across visible GPUs/MPS/CPU; pass a dict for explicit placement.
load_in_8bit / load_in_4bit — bitsandbytes quantization at load time. 8-bit is ~half memory; 4-bit is ~quarter. Inference quality cost is usually small for chat tasks.
revision — pin a commit SHA. Same rule as the Hub track.

The same loader works for any model that declares its task via library_name: transformers in the card. That's the whole point of the registry.

Code

AutoTokenizer + AutoModelForCausalLM·python

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

repo = "meta-llama/Llama-3.2-1B-Instruct"  # any 1B-class instruct model

tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(
    repo,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

prompt = tok.apply_chat_template(
    [{"role": "user", "content": "What is the Hugging Face Hub?"}],
    tokenize=False,
    add_generation_prompt=True,
)

inputs = tok(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=120, do_sample=False)
print(tok.decode(out[0], skip_special_tokens=True))

Different head, same recipe (classification)·python

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

repo = "distilbert-base-uncased-finetuned-sst-2-english"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForSequenceClassification.from_pretrained(repo)

inputs = tok("The movie was ok.", return_tensors="pt")
with torch.no_grad():
    logits = model(**inputs).logits
print("predicted label id:", logits.argmax().item())

Exercise

Take any text classifier on the Hub. Load it with AutoModelForSequenceClassification.from_pretrained(...). Compare three settings: float32 default, bfloat16, and 8-bit (bitsandbytes). For each, time the same 100 inputs and measure peak GPU memory with nvidia-smi or torch.cuda.max_memory_allocated().