Eigen Applications: PCA, PageRank, Quantum

Three Worlds, One Idea

Eigen-decomposition is the same operation appearing in three radically different contexts. Each is worth 30 seconds.

PCA — Finding the Big Directions in Data

You have a dataset where each row is a sample and each column is a feature. The covariance matrix tells you how features co-vary. PCA computes its eigenvectors: those eigenvectors are the directions in which your data has the most variation, ranked by eigenvalue size. Project the data onto the top $k$ eigenvectors and you've reduced 100-dim data to $k$ -dim with minimal information loss.

This is how you visualize 1000-dim embeddings on a 2D scatter plot. PCA picks the 2 directions that preserve as much "shape" of the data as possible.

PageRank — Finding the Important Pages

Google's original ranking algorithm models the web as a graph: each page is a node, each link is an edge. You build a matrix $M$ where $M_{ij}$ is the probability of jumping from page $j$ to page $i$ . The dominant eigenvector of $M$ (the eigenvector with the largest eigenvalue) gives you a stationary probability distribution: "if a random surfer clicked links forever, what fraction of time would they spend on each page?" That distribution is the importance score.

The web has billions of pages. Computing the eigenvector exactly is impossible, but iterative algorithms (power iteration) converge to it surprisingly fast. That's PageRank.

Quantum Mechanics — States and Observations

In quantum mechanics, a physical system's state is a vector in a complex vector space. Observable quantities (energy, momentum, position) are operators (matrices). When you measure an observable, you get an eigenvalue of the operator, and the system's state collapses to the corresponding eigenvector. This is the math underneath "the cat is alive AND dead until you look" — superposition is a sum of eigenstates; measurement projects onto one.

SVD — Eigen for Rectangular Matrices

Eigenvalues only exist for square matrices. The Singular Value Decomposition generalizes: it factors any matrix $M = U Σ V^{T}$ where the singular values in $Σ$ play the role of eigenvalues. SVD powers recommender systems, latent semantic analysis, and image compression.

If a system has structure, that structure usually has a name and the name is "eigen-something." Spotting the eigen-decomposition under the hood of a method tells you what's actually being computed.

Track Reward

You walked through the eigen-dungeons and lived. Matrices are transformations. The identity is the do-nothing. The inverse is the undo. Determinants are souls. Eigenvectors are favorite directions. From here on, when someone says "transformer attention" or "covariance projection" or "spectral method," you'll know the chassis underneath.

Code

PCA in 4 lines·python

import numpy as np

# Tiny PCA — 2D data with strong correlation, find the principal axes
np.random.seed(0)
X = np.random.randn(200, 2)
X[:, 1] = 0.7 * X[:, 0] + 0.3 * X[:, 1]   # induce correlation

# 1) Center the data
X = X - X.mean(axis=0)

# 2) Covariance matrix
C = (X.T @ X) / (len(X) - 1)

# 3) Eigendecomposition
eigvals, eigvecs = np.linalg.eigh(C)       # use eigh for symmetric matrices
print("eigenvalues:", eigvals)             # smaller, larger
print("first principal direction:", eigvecs[:, -1])  # largest-eigenvalue eigenvector

PageRank in 4 lines·python

import numpy as np

# Toy PageRank — 4-page web
M = np.array([[0,   0.5, 0.5, 0  ],
              [0.5, 0,   0,   1  ],
              [0.5, 0,   0,   0  ],
              [0,   0.5, 0.5, 0  ]])

# Power iteration — converges to the dominant eigenvector
rank = np.ones(4) / 4
for _ in range(50):
    rank = M @ rank
    rank = rank / rank.sum()
print("PageRank scores:", rank)

Exercise

Take any 100×5 dataset (random or real). Standardize each column. Compute PCA via eigendecomposition of the covariance matrix. Plot the first two principal components against each other and notice how the data spreads.

Hint

X = (X - X.mean(axis=0)) / X.std(axis=0), then C = X.T @ X / (len-1), then eigvals, eigvecs = np.linalg.eigh(C). The first two principal axes are eigvecs[:, -1] and eigvecs[:, -2].

6) PC1 vs PC2 산점도

fig, ax = plt.subplots(figsize=(7, 6)) ax.scatter(PC[:, 0], PC[:, 1], s=40, alpha=0.7, edgecolor='white', linewidth=0.5) ax.axhline(0, color='gray', lw=0.5) ax.axvline(0, color='gray', lw=0.5) ax.set_xlabel(f"PC1 ({vals[0]/vals.sum()*100:.1f}%)") ax.set_ylabel(f"PC2 ({vals[1]/vals.sum()*100:.1f}%)") ax.set_title("PCA: first two principal components") ax.set_aspect('equal', adjustable='datalim') plt.tight_layout() plt.savefig('pca_scatter.png', dpi=130)

Eigen Applications: PCA, PageRank, Quantum

Three Worlds, One Idea

PCA — Finding the Big Directions in Data

PageRank — Finding the Important Pages

Quantum Mechanics — States and Observations

SVD — Eigen for Rectangular Matrices

Track Reward

Code

External links

Exercise

Progress

Comments 2

잠재 구조가 있는 100x5 데이터: 두 숨은 요인이 5개 열을 만든다

1) 열별 표준화 (평균 0, 표준편차 1)

2) 공분산 행렬

3) 대칭행렬 고유분해

4) 큰 고유값 순으로 정렬

5) 처음 두 주성분으로 사영

6) PC1 vs PC2 산점도