Types of Matrices: A Bestiary

Same Object, Different Shapes

Matrices come in flavors. Each flavor is just a constraint on the structure, but the constraints earn each their own name because they unlock specific powers.

Type	Constraint	Why it's useful
Square	same number of rows and columns	only shape that has an inverse, determinant, eigenvalues
Rectangular	m ≠ n	changes dimensionality (e.g. project 1024-dim down to 256-dim)
Diagonal	zero off the diagonal	scaling per axis; multiplication is element-wise on diagonals
Identity $I$	diagonal of 1s, zero elsewhere	multiplicative identity — $I A = A$
Zero	all zeros	additive identity — $0 + A = A$
Symmetric	$A = A^{T}$	often arises in correlation, distance, energy
Sparse	mostly zeros	save memory, speed up multiplication when most cells are dead

Why You Should Care

You'll meet most of these in the wild. Identity matrices show up in matrix inverse formulas. Diagonal matrices appear all through scaling and SVD. Symmetric matrices are the entire reason eigenvalue decomposition is well-behaved in many ML algorithms. Sparse matrices are how massive recommendation systems and graph neural networks fit in RAM at all.

Type hints, mathematical edition. Recognizing a matrix's type tells you which operations are cheap and which break. Don't compute inverses on rectangular matrices (it doesn't exist); don't store sparse matrices densely (you'll OOM); don't assume symmetric when it's not.

Code

Bestiary, four types·python

import numpy as np

zero = np.zeros((3, 3))
identity = np.eye(3)
diagonal = np.diag([2, 5, 7])
print(diagonal)
# [[2 0 0]
#  [0 5 0]
#  [0 0 7]]

# Diagonal matrices scale: D @ v scales v[i] by D[i, i]
v = np.array([1, 1, 1])
print(diagonal @ v)        # [2 5 7]

# Symmetric — auto-symmetric construction
A = np.random.randn(3, 3)
S = (A + A.T) / 2          # any matrix + its transpose is symmetric
print(np.allclose(S, S.T)) # True

Types of Matrices: A Bestiary

Same Object, Different Shapes

Why You Should Care

Code

External links

Exercise

Progress

Comments 2