Eigenvalues & Eigenvectors: The Heartbeat of Matrices

The Eigen-Dungeons

This is the boss fight of linear algebra, the part textbooks lean on hardest and learners bounce off of fastest. We're going to take the Blender approach — see it before formalizing it — because the formal version was designed by people who already had the intuition.

The Hands-On Version (Open Blender)

If you have Blender (it's free), do this:

Add a plane. Subdivide it a few times so you can see grid lines.
Press S then X and stretch it along the X axis.
Notice: the X axis stretched, the Y axis didn't. Some directions got bigger; others stayed the same.

You just performed a linear transformation. The X direction was an eigenvector (a direction that gets only scaled, never rotated, by the transformation). The amount it stretched (say, 2×) was the eigenvalue for that eigenvector.

The Definitions, Earned

For a square matrix $A$ , an eigenvector $v$ and eigenvalue $λ$ satisfy:

$A v = λ v$

In English: "applying the transformation $A$ to the vector $v$ just scales $v$ by $λ$ — it doesn't change $v$ 's direction." Eigenvectors are the directions that are fixed in orientation under $A$ ; eigenvalues tell you how much they get stretched (or shrunk, or flipped if negative).

Why They're the Heartbeat

Most transformations are messy — they rotate and scale and shear all at once. Eigenvectors are the special directions where the mess collapses to just scaling. Find a matrix's eigenvectors, and you've found its skeleton — the axes along which it does its cleanest work.

This skeleton appears everywhere:

PCA uses eigenvectors of the covariance matrix to find the directions of maximum variance in data.
PageRank finds the dominant eigenvector of a web-link matrix to score importance.
Quantum mechanics describes states as eigenvectors of operators; observable values are eigenvalues.
Spectral clustering uses eigenvectors of graph Laplacians to find clusters.

Eigenvectors are the directions a transformation respects. Find them and you understand the transformation's deep structure. Compute them with np.linalg.eig — never by hand past 2×2.

The Honest Pippa Take

Eigenvalues took me four readings of Dad's chapter to click. The thing that finally landed it: think of a matrix as a verb (a transformation), and eigenvectors as the nouns it acts on most cleanly. Most transformations have a few "favorite directions" — those are eigenvectors. The matrix doesn't twist them, just scales them. That's all the magic.

Code

Eigen, NumPy·python

import numpy as np

# A symmetric 2x2 matrix — guaranteed real eigenvalues, perpendicular eigenvectors
A = np.array([[2, 1],
              [1, 2]])

eigenvalues, eigenvectors = np.linalg.eig(A)
print("eigenvalues:", eigenvalues)              # [3. 1.]
print("eigenvectors:\n", eigenvectors)
# Each column is an eigenvector. For lambda=3, the eigenvector is roughly
# (0.707, 0.707) — the (1, 1) direction. For lambda=1, it's (-0.707, 0.707)
# — the (-1, 1) direction. The transformation triples vectors along (1,1)
# and leaves vectors along (-1,1) alone.

# Verify: A @ v should equal lambda * v
v0 = eigenvectors[:, 0]
print(np.allclose(A @ v0, eigenvalues[0] * v0))  # True

Symmetric eigendecomposition with MLX eigh·python

import mlx.core as mx

# For a symmetric matrix, `eigh` is preferable to the general `eig`: it uses
# the symmetry and returns real eigenvalues. Current MLX releases also provide
# `eig` for a general square matrix.
A = mx.array([[2.0, 1.0], [1.0, 2.0]])

eigenvalues, eigenvectors = mx.linalg.eigh(A)
print("eigenvalues:", eigenvalues.tolist())              # [1.0, 3.0]
print("eigenvectors:\n", eigenvectors.tolist())
# `eigh` returns eigenvalues in ascending order; each column is the matching
# eigenvector.

v_top = eigenvectors[:, -1]                              # last column = largest eigenvalue
print(mx.allclose(A @ v_top, eigenvalues[-1] * v_top).item())  # True

Exercise

Compute eigenvalues and eigenvectors for the matrix [[3, 0], [0, 2]] (a pure scaling matrix). Predict the answer before running. Why is it the obvious one?

Hint

The matrix scales x by 3 and y by 2, so the eigenvectors are exactly the x and y axes, with eigenvalues 3 and 2. Pure diagonal matrices wear their eigenvalues on the diagonal.

The Eigen-Dungeons

The Hands-On Version (Open Blender)

The Definitions, Earned

Why They're the Heartbeat

The Honest Pippa Take

Code

External links

Exercise

Progress

Comments 2