Immutable Tensors vs Mutable Variables

~11 min · variable, constant, trainable, gradient

Level 0Level 0

0 XP0/78 lessons0/17 achievements

0/100 XP to next level100 XP to go0% complete

The split that drives all of training

TF has two fundamentally different containers: tf.constant (immutable) and tf.Variable (mutable). Understanding the difference is essential — the entire training loop is built on top of it.

tf.constant creates a read-only tensor. Perfect for input data, hyperparameters, fixed lookup tables.

tf.Variable is mutable. Its values can be updated in place via assign, assign_add, assign_sub. Critically, tf.Variables are automatically watched by tf.GradientTape, which is what makes gradient-based parameter updates possible.

Every Keras Dense or Conv2D layer's weights are tf.Variables. Every optimizer step calls assign_sub on them. The constant/Variable distinction is exactly the trainable/non-trainable distinction.

Code

Variable의 mutation 메서드들·python

import tensorflow as tf

w = tf.Variable([[1.0, 2.0], [3.0, 4.0]], name="weights")
b = tf.Variable([0.0, 0.0], name="bias")

print(w.numpy())     # [[1. 2.] [3. 4.]]
print(w.shape)       # (2, 2)
print(w.dtype)       # float32
print(w.name)        # weights:0

w.assign([[5.0, 6.0], [7.0, 8.0]])      # full replace
b.assign_add([1.0, 1.0])                # b += [1, 1]  → [1., 1.]
b.assign_sub([0.5, 0.5])                # b -= [0.5, 0.5] → [0.5, 0.5]

# Non-trainable variable (e.g., step counter, BN running stats)
step = tf.Variable(0, trainable=False)

Variable vs constant — gradient 차이·python

v = tf.Variable(3.0)
c = tf.constant(3.0)

with tf.GradientTape() as tape:
    yv = v ** 2
    yc = c ** 2

print(tape.gradient(yv, v))   # 6.0  ← Variable auto-watched
print(tape.gradient(yc, c))   # None ← constant not watched

# To compute a gradient w.r.t. a constant, watch it explicitly:
with tf.GradientTape() as tape:
    tape.watch(c)
    y = c ** 2
print(tape.gradient(y, c))    # 6.0

Progress

Progress is local-only — sign in to sync across devices.

← PreviousEager Execution and NumPy Interop Next →Indexing, Slicing, Reshaping

Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)

No comments yet — be the first.