Common Shape Errors and How to Debug Them

The error messages you'll see most

Three TF errors will eat 80% of your debugging time. Recognizing them on sight saves hours.

Error 1: incompatible matmul shapes. tf.matmul(a, b) requires a.shape[-1] == b.shape[-2]. The fix is usually a tf.transpose on one operand. The shorthand tf.matmul(a, b, transpose_b=True) avoids creating an intermediate tensor.

Error 2: forgot the batch dimension. Models trained on (batch, ...) will choke on a single sample with shape (...). tf.expand_dims(x, axis=0) or x[tf.newaxis] fixes it.

Error 3: dtype mismatch. NumPy creates float64 by default; TF expects float32. Cast at the boundary: tf.cast(np_array, tf.float32). This is the most common silent bug in image pipelines that load PNG/JPEG and forget to normalize from uint8 0–255 to float32 0–1.

Code

디버깅 헬퍼 함수·python

import tensorflow as tf

def debug_tensor(t, name="tensor"):
    print(f"{name}: shape={t.shape}, dtype={t.dtype}")
    if t.shape.rank is not None and tf.size(t) <= 20:
        print(f"  values: {t.numpy()}")

x = tf.random.normal([32, 28, 28, 1])
debug_tensor(x, "raw input")

x_flat = tf.reshape(x, [32, -1])
debug_tensor(x_flat, "after flatten")

Cast 패턴 — uint8 image를 float32 0-1로·python

import tensorflow as tf
import numpy as np

# Common pipeline bug: NumPy float64 image into TF float32 model
np_data = np.random.randn(100, 10)         # float64 by default
tf_data = tf.cast(tf.constant(np_data), tf.float32)

# uint8 image → float32 0-1
raw = tf.constant([255, 128, 0], dtype=tf.uint8)
norm = tf.cast(raw, tf.float32) / 255.0    # [1.0, 0.502, 0.0]

Exercise

Take a tensor of shape (32, 28, 28, 1) (a batch of MNIST images). Without looking up the answer, write the code to: (a) normalize from uint8 0-255 to float32 0-1, (b) flatten to (32, 784) for a Dense input, (c) compute matmul against a weight tensor of shape (784, 10). Verify the output is (32, 10).