Perceptron Basics

The 1958 model that started everything

Frank Rosenblatt's perceptron (1958) is a single-output binary classifier: y = sign(w · x + b). Rosenblatt proved a beautiful convergence theorem — if the data is linearly separable, his learning rule will find a separating hyperplane in finite steps. The press called it the seed of self-aware machines, which aged about as well as you would expect.

Minsky and Papert's Perceptrons (1969) pointed out that a single perceptron cannot learn XOR — a problem that requires a non-linear decision boundary. The book is often blamed for the AI winter that followed; the actual reason was that nobody yet knew how to train multilayer networks effectively. Backprop solved that almost twenty years later.

Why we still teach this

The perceptron is the simplest example of "linear model with a step at the output." Every modern classifier — softmax, sigmoid, cross-entropy — is a smoother, differentiable cousin. Understanding the perceptron makes the rest of the math feel inevitable rather than mysterious.

Tip: The perceptron's failure on XOR is the canonical motivation for hidden layers and non-linear activations. Be ready to draw it on a whiteboard at any time — it is one of the highest-leverage examples in the whole field.

What the perceptron rule looks like

For each misclassified example, push the weight vector toward the input that should have been positive (or away from the one that should have been negative): w := w + η y x. It is gradient descent in disguise — the gradient of a hinge-shaped loss with a step function on top.

Code

Perceptron from scratch·python

import numpy as np

def perceptron_train(X, y, lr=1.0, epochs=20):
    N, D = X.shape
    w = np.zeros(D); b = 0.0
    for _ in range(epochs):
        wrong = 0
        for xi, yi in zip(X, y):
            if yi * (w @ xi + b) <= 0:
                w += lr * yi * xi
                b += lr * yi
                wrong += 1
        if wrong == 0:
            break
    return w, b

X = np.array([[2, 3], [1, 1], [4, 5], [-1, -2], [-2, -3], [-3, -1]])
y = np.array([+1, +1, +1, -1, -1, -1])
w, b = perceptron_train(X, y)
print("learned w, b:", w, b)
print("predictions:", np.sign(X @ w + b))

The 1958 model that started everything

Why we still teach this

What the perceptron rule looks like

Code

External links

Exercise

Progress

Comments 0