Loss Functions and Metrics — The Right Choice

~11 min · loss, metric, classification, regression

Level 0Level 0

0 XP0/78 lessons0/17 achievements

0/100 XP to next level100 XP to go0% complete

The wrong loss is worse than no model at all

Loss functions define what the model is optimizing. Pick wrong and no architecture tuning compensates. Pick right and even mediocre models converge.

Classification:

Multi-class with integer labels — SparseCategoricalCrossentropy(from_logits=True)
Multi-class with one-hot labels — CategoricalCrossentropy(from_logits=True)
Binary or multi-label (each sample can have multiple labels) — BinaryCrossentropy(from_logits=True)

Regression:

Standard — MeanSquaredError
Outlier-robust — Huber(delta=1.0) — MSE for small errors, MAE for large
Pure L1 — MeanAbsoluteError

Custom loss is a function with signature (y_true, y_pred) -> tensor. Returns a per-sample loss; Keras averages over the batch.

Code

Custom focal loss — class imbalance용·python

import tensorflow as tf
from tensorflow import keras

def focal_loss(y_true, y_pred, gamma=2.0, alpha=0.25):
    """Focal loss for class-imbalanced data (RetinaNet paper)."""
    bce = keras.losses.binary_crossentropy(y_true, y_pred, from_logits=True)
    p_t = tf.exp(-bce)
    return tf.reduce_mean(alpha * (1 - p_t) ** gamma * bce)

model.compile(loss=focal_loss, optimizer='adam')

Multi-output loss + metric 매핑·python

model.compile(
    optimizer='adam',
    loss={'class_out': 'sparse_categorical_crossentropy',
          'bbox_out': 'mse'},
    loss_weights={'class_out': 1.0, 'bbox_out': 0.1},
    metrics={'class_out': ['accuracy'],
             'bbox_out': ['mae']},
)

Progress

Progress is local-only — sign in to sync across devices.

← PreviousOptimizers — Adam, SGD, AdamW, Schedules Next →model.fit() — The Training Loop You Don't Write

Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)

No comments yet — be the first.