The loss is your objective, stated precisely
The loss function is the one number training exists to minimize — it turns "how wrong was that prediction" into a single scalar the optimizer can chase. Pick the wrong one and the model will dutifully optimize the wrong thing, often without throwing an error. So the choice isn't cosmetic; it's a statement of what problem you're actually solving.
Match the loss to the task and the label format together:
| Loss | Task | Labels Format |
|---|---|---|
CategoricalCrossentropy | Multi-class classification | One-hot: [0,0,1,0] |
SparseCategoricalCrossentropy | Multi-class classification | Integer: 2 |
BinaryCrossentropy | Binary / multi-label | 0 or 1 |
MeanSquaredError | Regression | Continuous values |
Huber | Robust regression | Continuous (outlier-resistant) |
The Sparse vs non-Sparse split trips up everyone once: SparseCategoricalCrossentropy wants integer labels like 2, the plain CategoricalCrossentropy wants one-hot vectors like [0,0,1,0]. Feed one the other's format and you get a shape error at best, silently wrong gradients at worst. For regression, reach for Huber over plain MSE when your data has outliers — it behaves like MSE near zero but like MAE in the tails, so a few wild samples don't dominate the gradient.
from_logits=True — the free stability win
Use from_logits=True when your final layer outputs raw scores with no softmax or sigmoid. It's mathematically identical to applying the activation then the loss, but Keras fuses the two into one numerically stable operation — no intermediate probabilities to underflow. The Code section shows both forms; Option B is the one to default to.