Regularization and Reshaping

Regularization: deliberately handicapping training

A model with enough capacity will happily memorize its training set. Regularization layers fight that by injecting controlled difficulty so the network is forced to learn patterns that generalize instead of lookup tables that don't.

Dropout(0.3) — randomly zeros 30% of units during training only; at evaluation it's automatically a no-op. A starting rate of 0.2–0.5 is typical, tuned against your validation curve.
SpatialDropout1D / SpatialDropout2D — drop entire feature maps rather than scattered elements. Better for conv layers, where neighboring pixels are so correlated that ordinary dropout barely removes information.

Reshaping: moving axes without moving data

Reshaping layers change how a tensor is viewed, not what it contains:

Flatten() — collapses everything after the batch axis to 1D, the classic bridge from conv stack to Dense.
Reshape(target_shape) — an arbitrary reshape; the element count must stay constant.
Permute(dims) — reorders axes, e.g. swapping a channels-first layout to channels-last.

Merging: where multi-branch models join

Functional-API models with parallel branches need a layer to recombine them. Concatenate stacks features along an axis (the branches keep their identity); Add sums them element-wise (the basis of residual connections); Multiply and Average do the obvious. The merge layers are the seams of every non-linear architecture you'll build.

Code

Merge layers for combining parallel branches·python

# Merge two branches
layers.Concatenate()([branch_a, branch_b])  # Concatenate along axis
layers.Add()([branch_a, branch_b])          # Element-wise addition
layers.Multiply()([branch_a, branch_b])     # Element-wise multiplication
layers.Average()([branch_a, branch_b])      # Element-wise average

Exercise

Take a working CNN that overfits (training accuracy far above validation). Add Dropout(0.5) after the Flatten, retrain, and plot both curves with and without it. Then build a tiny two-branch Functional model — two Dense paths merged once with Add and once with Concatenate — and inspect how the merged output shape differs. Confirm the right rate and merge are data-dependent, not assumptions.

Hint

With Add the merged shape equals each branch's shape; with Concatenate the joined axis is the sum of the branch widths. The right dropout rate is whatever closes the train/val gap without hurting val accuracy.