Dataset Basics — Creating and Iterating

~11 min · dataset, from-tensor-slices, iteration

Level 0Level 0

0 XP0/78 lessons0/17 achievements

0/100 XP to next level100 XP to go0% complete

Where pipelines begin

A tf.data.Dataset is an ordered collection of elements. Each element can be a single tensor, a tuple of tensors (typical: features and labels), or a dict of tensors (named features). You create one from in-memory data, from files, or from a generator.

Datasets are lazy — building one doesn't load any data. The pipeline runs only when you iterate over it (during model.fit or a manual loop). .take(n) creates a new Dataset with the first n elements — invaluable for debugging.

Code

Dataset 만들기 — 4가지 source·python

import tensorflow as tf
import numpy as np

# From a Python list
ds_list = tf.data.Dataset.from_tensor_slices([1, 2, 3, 4, 5])

# From NumPy arrays — (features, labels) pairs
x_np = np.random.randn(1000, 32).astype(np.float32)
y_np = np.random.randint(0, 10, size=(1000,))
ds_numpy = tf.data.Dataset.from_tensor_slices((x_np, y_np))

# From a dict — named features
ds_dict = tf.data.Dataset.from_tensor_slices({
    'image':  np.random.randn(100, 28, 28, 1).astype(np.float32),
    'label':  np.random.randint(0, 10, size=(100,)),
    'weight': np.ones(100, dtype=np.float32),
})

# From a list of files (lazy — doesn't read files yet)
ds_files = tf.data.Dataset.list_files("data/images/*.jpg")

Iterate / inspect / take·python

for element in ds_list.take(3):
    print(element.numpy())   # 1, 2, 3

for x, y in ds_numpy.take(2):
    print(f"x shape: {x.shape}, y: {y.numpy()}")

# Convert small dataset to numpy for inspection
first_batch = list(ds_list.take(5).as_numpy_iterator())
print(first_batch)   # [1, 2, 3, 4, 5]

Progress

Progress is local-only — sign in to sync across devices.

← PreviousWhy tf.data Exists Next →Chaining Transformations — map, batch, shuffle, filter

Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)

No comments yet — be the first.