GradientTape Basics — Your First Gradient

~13 min · autodiff, gradient-tape, backprop

Level 0Level 0

0 XP0/78 lessons0/17 achievements

0/100 XP to next level100 XP to go0% complete

Where deep learning's magic actually happens

Automatic differentiation is the engine behind training. tf.GradientTape records (on a tape) every operation involving watched tensors during a forward pass. When you call tape.gradient(target, source), it plays the tape backward — applying the chain rule automatically — and returns the gradient.

The default rule: tf.Variables are watched automatically. Plain tensors and tf.constant are not; you must call tape.watch(c) explicitly to compute a gradient w.r.t. them. This default exists because Variables are the trainable parameters — almost always what you want gradients for.

If tape.gradient(loss, weight) returns None, the variable wasn't used in the forward pass — there's no path connecting it to the loss. Either you froze the layer, you used tf.stop_gradient somewhere, or there's a bug in your model code.

Code

첫 gradient·python

import tensorflow as tf

# d(x²)/dx = 2x
x = tf.Variable(3.0)

with tf.GradientTape() as tape:
    y = x ** 2

dy_dx = tape.gradient(y, x)
print(dy_dx)   # 6.0 (= 2 * 3)

여러 변수 한 번에·python

import tensorflow as tf

x = tf.Variable(2.0)
y = tf.Variable(3.0)

with tf.GradientTape() as tape:
    z = x * x + y * y    # z = x² + y²

# Pass a dict or list
grads = tape.gradient(z, {'x': x, 'y': y})
print(grads['x'])    # dz/dx = 2x = 4.0
print(grads['y'])    # dz/dy = 2y = 6.0

watch — constant도 추적하기·python

import tensorflow as tf

c = tf.constant(3.0)

with tf.GradientTape() as tape:
    y = c ** 2
print(tape.gradient(y, c))   # None — constant not watched

with tf.GradientTape() as tape:
    tape.watch(c)            # explicit
    y = c ** 2
print(tape.gradient(y, c))   # 6.0

External links

Gradients and automatic differentiation

Exercise

Compute the gradient of f(x) = sin(x²) at x = 1.0 using GradientTape. Verify by hand using chain rule that the answer matches 2 * cos(1) ≈ 1.0806.

Progress

Progress is local-only — sign in to sync across devices.

← PreviousMatrix Multiplication and Broadcasting Next →Persistent Tapes and Higher-Order Gradients

Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)

No comments yet — be the first.