Forward Pass: Shooting the Arrow

The Archery Metaphor

A neural network's forward pass is like shooting an arrow. Data enters the input layer, flows through hidden layers, lands as a prediction. The bullseye is the correct answer. The arrow usually misses on the first shot — by exactly the amount we call loss.

The loss function is your ruler for "how badly did you miss?" For regression: mean squared error. For classification: cross-entropy. We met both in the Learning from Data track.

What's Actually Happening

Each layer applies a transformation: $h_{ℓ + 1} = σ (W_{ℓ} h_{ℓ} + b_{ℓ})$ . Multiply by a weight matrix, add a bias, pass through a non-linearity. Stack many of these and you get a deep network.

Input $x$ becomes $h_{1}$ after layer 1.
$h_{1}$ becomes $h_{2}$ after layer 2.
...
The final $h_{L}$ is your prediction $\overset{y}{^}$ .
Compare $\overset{y}{^}$ to true $y$ with the loss function.

That's the forward pass. Pure forward computation, no learning yet.

Forward pass = compute the prediction. Loss = how wrong it is. The backward pass (next lesson) is what makes the learning happen.

Code

One forward pass·python

import torch
import torch.nn as nn
import torch.nn.functional as F

# A 2-layer network — forward pass only
class TinyNet(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(4, 8)
        self.fc2 = nn.Linear(8, 3)

    def forward(self, x):
        h = F.relu(self.fc1(x))      # layer 1 + ReLU
        return self.fc2(h)            # layer 2 → logits

net = TinyNet()
x = torch.randn(1, 4)                # 1 sample, 4 features
y_true = torch.tensor([2])           # true class index

logits = net(x)                       # forward pass
loss = F.cross_entropy(logits, y_true)
print(f"loss: {loss.item():.4f}")

Exercise

Define a 3-layer fully-connected network in PyTorch (input 10 → 64 → 32 → 1). Pass a random input through it. What's the shape of each intermediate layer's output?

Hint

Use nn.Linear and F.relu. Print h.shape after each layer to see the dimensionality flow: (1, 10) → (1, 64) → (1, 32) → (1, 1).

Forward Pass: Shooting the Arrow

The Archery Metaphor

What's Actually Happening

Code

Exercise

Progress

Comments 0