AUTOTUNE, Caching, Prefetching

The three magic words for performance

Three mechanisms that turn a naive pipeline into a fast one — AUTOTUNE, caching, prefetching. Use all three by default.

AUTOTUNE (tf.data.AUTOTUNE): instead of hardcoding num_parallel_calls=4, let TF measure throughput at runtime and tune parallelism dynamically. Almost always better than any fixed value.

Caching (.cache()): saves the output of the pipeline at that point. First epoch runs the preprocessing and stores results; subsequent epochs read from cache. Use .cache() in memory if data fits, .cache("/tmp/path") for disk caching of larger sets.

Prefetching (.prefetch(AUTOTUNE)): overlaps CPU data preparation with GPU model computation. Always the LAST step in the pipeline. Adding it to a pipeline that didn't have it often gives 20–50% speedup with zero code complexity — it's free performance.

Cache before augment, not after. Cache holds the cached pipeline output forever. If you cache after augmentation, you cache the same augmentations every epoch — defeating the entire purpose of augmentation. Cache the deterministic preprocessing, augment after.

Code

Cache + augment 올바른 순서·python

import tensorflow as tf

AUTOTUNE = tf.data.AUTOTUNE

# CORRECT: cache deterministic preprocessing, augment after cache
correct = (
    tf.data.Dataset.from_tensor_slices((x, y))
    .map(normalize, num_parallel_calls=AUTOTUNE)   # cheap, deterministic
    .cache()                                         # cache normalized
    .shuffle(10000)
    .batch(128)
    .map(augment, num_parallel_calls=AUTOTUNE)      # fresh per epoch
    .prefetch(AUTOTUNE)
)

# WRONG: caching augmented data — same augmentation every epoch
wrong = (
    tf.data.Dataset.from_tensor_slices((x, y))
    .map(augment)
    .cache()                                         # locks in augmentation
    .batch(128)
)

AUTOTUNE, Caching, Prefetching

The three magic words for performance

Code

Progress

Comments 0