Don't let your GPU sit idle
Models train only as fast as data is delivered. If your GPU is idle waiting for the CPU to load and preprocess the next batch, you're burning expensive compute. tf.data solves this by building data pipelines as lazy, parallelized streams — feeding the GPU before it finishes the previous batch.
The five things tf.data gives you:
- Prefetching — prepare batch N+1 while GPU processes batch N
- Parallelism — preprocess on multiple CPU threads simultaneously
- Caching — avoid re-doing expensive preprocessing every epoch
- Memory efficiency — load only what's needed, when it's needed
- Composability — chain transformations in a readable way
On ImageNet-scale datasets (1.28M images), a naive Python loop leaves the GPU starved. A properly tuned tf.data pipeline keeps the GPU at near 100% utilization. The difference is 5–10× training speed.