Transpose: Flipping Across the Diagonal

The Operation

The transpose of a matrix $A$ , written $A^{T}$ , swaps rows and columns. Element at $(i, j)$ moves to $(j, i)$ . A 2×3 matrix becomes 3×2; a square matrix stays square but reflects across its diagonal.

$A = (142536), A^{T} = 123456$

Why You'll Use It Every Day

Shape-fixing for matmul. Need to multiply a (32, 128) batch by a (256, 128) weight matrix? Transpose the weight to (128, 256) first. Half of "make the shapes work" debugging is figuring out where to insert .T.
Defining symmetry. A matrix is symmetric iff $A = A^{T}$ . Lots of important matrices in ML (covariance, Gram matrices, kernels) are symmetric by construction.
Long ↔ wide reshaping in data analysis. Pandas DataFrames pivot all the time; under the hood it's transpose flavor.
Backprop bookkeeping. The chain rule produces transposed weight matrices when gradients flow backward through linear layers.

Transpose costs nothing in NumPy/PyTorch. Most implementations don't actually move memory — they return a view with strides reversed. Use it freely for shape-fixing; the runtime won't punish you.

Code

Plain transpose + symmetry check·python

import numpy as np

A = np.array([[1, 2, 3],
              [4, 5, 6]])
print(A.T)
# [[1 4]
#  [2 5]
#  [3 6]]

# Symmetric matrix — equal to its transpose
S = np.array([[1, 2, 3],
              [2, 4, 5],
              [3, 5, 6]])
print(np.allclose(S, S.T))   # True

The most common shape fix in deep learning·python

import torch

# Shape mismatch fix in one line
batch = torch.randn(32, 128)
weight = torch.randn(256, 128)

# This fails: shapes (32, 128) and (256, 128) don't align for matmul
# out = batch @ weight   # ✗

# Transpose the weight; now shapes (32, 128) @ (128, 256) work
out = batch @ weight.T
print(out.shape)             # torch.Size([32, 256])

Transpose: Flipping Across the Diagonal

The Operation

Why You'll Use It Every Day

Code

Exercise

Progress

Comments 4