The knobs on a layer
A Dense layer takes a handful of constructor arguments, but only two of them you'll touch on almost every layer — units and activation. The rest have well-chosen defaults you should leave alone until you have a concrete reason not to.
| Argument | Type | Description |
|---|---|---|
units | int | Output dimensionality — how many neurons in this layer |
activation | str/fn | Activation function: 'relu', 'sigmoid', 'softmax', 'tanh', None |
input_shape | tuple | Shape of input (only needed for first layer, prefer keras.Input) |
use_bias | bool | Whether to add a bias term (default: True) |
kernel_initializer | str | Weight initialization: 'glorot_uniform' (default), 'he_normal', etc. |
Activation is a decision about the output, not a style choice
The activation on your last layer is dictated by the task, not by taste — get it wrong and your loss function silently receives values it can't interpret:
- ReLU — the default for hidden layers. Simple, fast, and it dodges the vanishing-gradient trap that sigmoid/tanh fall into in deep stacks.
- Sigmoid — output layer for binary classification: squashes to a single probability in [0, 1].
- Softmax — output layer for multi-class classification: a vector of probabilities that sums to 1 across the classes.
- Tanh — output range [-1, 1]. Occasionally useful in hidden layers and common in older RNN cells.
- None / Linear — output layer for regression. No transformation, so the network can emit any real number.
Notice the pattern: hidden layers almost always use ReLU, and the final activation is chosen by what you're predicting. That single rule eliminates a whole class of beginner bugs.