Core Layers

Layers are the atoms. Dense, Conv2D, MultiHeadAttention, LayerNormalization, Dropout, plus the preprocessing layers that bake normalization and tokenization *into the model* so deployment carries them along. This track is the catalog of what Keras gives you out of the box.

What a layer actually is

A Keras layer is a callable object that owns two things: a chunk of state (its trainable weights) and a transformation (its call method). You compose them like Lego — the output tensor of one becomes the input tensor of the next — and Keras tracks the weights, the shapes, and the gradient path for you. Master the handful of core layers below and you can read the architecture of almost any published model.

Layer	Purpose	Key Args
`Dense`	Fully connected	`units`, `activation`
`Embedding`	Integer → dense vector lookup	`input_dim`, `output_dim`, `mask_zero`
`Masking`	Marks timesteps as padding	`mask_value=0.0`
`Lambda`	Wraps arbitrary operation	`function`
`Input`	Placeholder tensor	`shape`, `dtype`, `name`
`EinsumDense`	Generalized dense via einsum	`equation`, `output_shape`

Embedding: the one that trips people up

Most core layers are obvious from their name. Embedding is the exception worth pausing on. It's a learned lookup table: integer token IDs go in, dense float vectors come out, and the table itself is trained by gradient descent alongside the rest of the model. Two args do all the work — input_dim is your vocabulary size (how many distinct tokens exist) and output_dim is the width of each vector. Set mask_zero=True and token 0 is treated as padding, so downstream recurrent and attention layers know to ignore it.

Code

Embedding: integer tokens to dense vectors·python

# Embedding layer: integer tokens → dense vectors
embedding = layers.Embedding(
    input_dim=10000,   # Vocabulary size
    output_dim=128,     # Embedding dimension
    mask_zero=True,     # Treat 0 as padding
)
# Input shape: (batch, seq_len) of integers
# Output shape: (batch, seq_len, 128) of floats

Exercise

Build a small Functional model wiring up Input → Embedding → Flatten → Dense. Use vocab_size=100, embed_dim=8 for the Embedding, and a final Dense(10). Print model.summary() and trace each layer's output shape by hand before you read it off — confirm your mental model of how the dims flow.

Hint

Embedding turns (batch, seq_len) into (batch, seq_len, 8). Flatten collapses everything after the batch axis, so a seq_len of 20 gives Flatten an output of (batch, 160).

What a layer actually is

Embedding: the one that trips people up

Code

External links

Exercise

Progress

Comments 0