C.W.K.
Stream
Lesson 02 of 06 · published

Batch Size & Gradient Accumulation

~18 min · batch-size, gradient-accumulation, effective-batch

Level 0Observer
0 XP0/43 lessons0/11 achievements
0/120 XP to next level120 XP to go0% complete

The effective batch size formula

effective_batch = per_device_batch × gradient_accumulation_steps × num_GPUs

Larger effective batches give smoother, more stable training but use more memory per step. When you cannot fit a large batch in VRAM, use gradient accumulation — sum gradients over multiple small batches before updating weights.

Effective batch guidelines

Dataset sizeRecommended effective batch
< 1,000 examples4–8
1,000–10,00016–32
10,000+32–128

Code

Achieve effective batch size of 32 on a single GPU·python
from trl import SFTConfig

args = SFTConfig(
    per_device_train_batch_size=2,      # small batch fits in VRAM
    gradient_accumulation_steps=16,     # accumulate 16 steps
    # Effective batch: 2 × 16 = 32
)

External links

Exercise

Calculate the right gradient_accumulation_steps to achieve effective batch 16 on three setups: single 24 GB GPU with per_device=4, single 80 GB GPU with per_device=8, and 4× 24 GB GPUs with per_device=2. Verify your math.

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign inPlease sign in to comment.

No comments yet — be the first.