Grad-CAM — Seeing What CNNs See

Why interpretability matters in production

A CNN that says "cat" with 99% confidence is useless if you don't know which pixels it looked at. Grad-CAM (Gradient-weighted Class Activation Mapping) generates a heatmap showing which image regions most influenced a specific class prediction.

The technique: for a chosen class, compute gradients of that class's logit with respect to the last convolutional layer's feature maps. Average those gradients over spatial dimensions to get per-channel importance weights. Multiply each feature map by its weight and sum — the result is a 2D heatmap, upsampled to the original image size.

This is how you debug "the model is overconfident on cats" — you find that it's looking at the watermark in the corner, not the cat. Same technique is used in medical imaging audits, content moderation explanations, and dataset auditing.

Code

Grad-CAM 구현·python

import tensorflow as tf
import numpy as np

def grad_cam(model, image, class_idx, last_conv_layer_name):
    """Generate Grad-CAM heatmap for the given class."""
    # Sub-model that returns last conv output AND final predictions
    grad_model = tf.keras.Model(
        inputs=[model.inputs],
        outputs=[model.get_layer(last_conv_layer_name).output, model.output]
    )

    with tf.GradientTape() as tape:
        conv_outputs, predictions = grad_model(image)
        class_channel = predictions[:, class_idx]

    # Gradient of class score w.r.t. last conv output
    grads = tape.gradient(class_channel, conv_outputs)

    # Average gradients over spatial dims → channel importance
    pooled_grads = tf.reduce_mean(grads, axis=(0, 1, 2))

    # Weight each feature map by its importance, sum to get heatmap
    conv_outputs = conv_outputs[0]
    heatmap = conv_outputs @ pooled_grads[..., tf.newaxis]
    heatmap = tf.squeeze(heatmap)
    heatmap = tf.maximum(heatmap, 0) / tf.reduce_max(heatmap)
    return heatmap.numpy()

# Usage
model = tf.keras.applications.EfficientNetV2S(weights='imagenet')
img = tf.keras.utils.load_img("cat.jpg", target_size=(224, 224))
img_array = tf.expand_dims(tf.keras.utils.img_to_array(img), 0)
img_array = tf.keras.applications.efficientnet_v2.preprocess_input(img_array)

preds = model.predict(img_array)
top_class = int(tf.argmax(preds[0]))
heatmap = grad_cam(model, img_array, top_class, "top_conv")

Grad-CAM — Seeing What CNNs See

Why interpretability matters in production

Code

External links

Progress

Comments 0