Vector Operations in Code: The Patterns You'll Reuse Forever

The Five Patterns

Almost every piece of vector code in AI is one of these five patterns. Knowing them by sight saves enormous reading time.

Element-wise op — a + b, a * b, np.exp(a). Same shape in, same shape out.
Reduction — a.sum(), a.mean(), a.max(). Many in, one (or fewer) out. Optionally specify axis=.
Dot product / matmul — a @ b. Two vectors → scalar. Vector and matrix → vector. Two matrices → matrix.
Broadcasting — operating on different-shaped arrays where one is "lifted" to match. matrix + vector often Just Works because of this.
Slicing / fancy indexing — a[2:5], a[a > 0], a[indices]. Pulling subsets without writing loops.

Broadcasting in 30 Seconds

NumPy/PyTorch will let you add a (3,) vector to a (4, 3) matrix — the vector gets broadcast across each row. The rule: dimensions are compatible if they're equal or one of them is 1. Lined up from the right. If the rule fails, you get the legendary shapes ... not aligned error.

Stop writing loops over array elements. If you find yourself writing for i in range(len(arr)), ask: is there a vectorized version? In 95% of cases, yes — and it'll be 10-1000× faster.

Track Reward

You now think in vectors. Magnitude × direction is the unit. The dot product is alignment-as-a-number. Norms are choosable rulers. Operations vectorize. From here, matrices are just stacks of vectors that act in concert — and that's the next track.

Code

The five patterns in 15 lines·python

import numpy as np

a = np.array([1, 2, 3, 4])
b = np.array([10, 20, 30, 40])

# Element-wise
print(a + b)                         # [11 22 33 44]
print(a * b)                         # [10 40 90 160]

# Reduction
print(a.sum(), a.mean(), a.max())    # 10  2.5  4

# Broadcasting
M = np.array([[1, 2, 3, 4],
              [5, 6, 7, 8]])
print(M + a)                         # vector broadcast over rows
# [[ 2  4  6  8]
#  [ 6  8 10 12]]

# Fancy indexing
mask = a > 2
print(a[mask])                       # [3 4]

MLX flavor — Apple Silicon, NumPy-shaped·python

import mlx.core as mx

# Same patterns — runs on Apple Silicon GPU via Metal, no .to(device) needed.
a = mx.array([1, 2, 3, 4])
b = mx.array([10, 20, 30, 40])

print(a + b)                         # [11, 22, 33, 44]
print((a * b).sum().item())          # 1000

M = mx.array([[1, 2, 3, 4],
              [5, 6, 7, 8]])
print(M + a)                         # broadcasts (4,) vector across rows

# MLX is lazy — the array prints when actually evaluated. For tight loops
# call mx.eval(...) after a step to force materialization.

Exercise

Without writing any explicit for loops: take a (5, 3) matrix of random floats, subtract its column means, divide by its column stds. (That's z-score standardization, applied per column.) Verify that the resulting columns have mean ≈ 0 and std ≈ 1.

Hint

(M - M.mean(axis=0)) / M.std(axis=0). The axis=0 collapses across rows, leaving you with a (3,) vector that broadcasts back across rows.

Vector Operations in Code: The Patterns You'll Reuse Forever

The Five Patterns

Broadcasting in 30 Seconds

Track Reward

Code

External links

Exercise

Progress

Comments 2