The effective batch size formula
effective_batch = per_device_batch × gradient_accumulation_steps × num_GPUs
Larger effective batches give smoother, more stable training but use more memory per step. When you cannot fit a large batch in VRAM, use gradient accumulation — sum gradients over multiple small batches before updating weights.
Effective batch guidelines
| Dataset size | Recommended effective batch |
|---|---|
| < 1,000 examples | 4–8 |
| 1,000–10,000 | 16–32 |
| 10,000+ | 32–128 |