PyTorch ↔ NumPy (and the GPU caveat)

Free conversion, sometimes free memory

PyTorch and NumPy interoperate trivially. The only thing to internalize is when the conversion shares memory and when it copies:

torch.from_numpy(arr) — shares memory. Mutating one mutates the other.
tensor.numpy() — shares memory if the tensor is CPU and contiguous. Errors if it's on GPU.
torch.tensor(arr) — always copies. Use this when you want independence.
torch.as_tensor(arr) — shares memory when possible, copies otherwise. Useful when you don't care which.

The GPU rule

You can only get a NumPy array out of a CPU tensor. If your tensor is on GPU, move it back first (.cpu()). And if it has gradients, call .detach() first to get rid of the graph attachment — otherwise NumPy will refuse to convert it.

The full incantation for "give me numpy from a model output": output.detach().cpu().numpy(). Memorize that chain — you'll write it constantly when computing metrics or plotting.

Code

Sharing memory both directions·python

import torch
import numpy as np

# numpy → tensor (shares memory)
arr = np.array([1.0, 2.0, 3.0])
t = torch.from_numpy(arr)
arr[0] = 99
print(t)   # tensor([99.,  2.,  3.]) — yes, changed

# tensor → numpy (shares memory if CPU)
t2 = torch.tensor([4.0, 5.0, 6.0])
arr2 = t2.numpy()
t2[0] = 88
print(arr2)  # [88. 5. 6.] — yes, changed

Independence — the .clone() / .copy() escape·python

import torch
import numpy as np

# I want PyTorch ownership without NumPy peeking in
arr = np.array([1.0, 2.0, 3.0])
t = torch.from_numpy(arr).clone()
arr[0] = 99
print(t)   # tensor([1., 2., 3.]) — independent

# Or in NumPy land
t = torch.tensor([1.0, 2.0, 3.0])
arr = t.numpy().copy()

The GPU + autograd dance·python

import torch

# A model output: on GPU, has gradient tracking
output = torch.randn(4, 10, device="cuda", requires_grad=True)

# Wrong — fails because of grad
# arr = output.numpy()
# RuntimeError: Can't call numpy() on Tensor that requires grad.

# Wrong — fails because of device
# arr = output.detach().numpy()
# TypeError: can't convert cuda:0 device type tensor to numpy.

# Right — detach first, then move to CPU, then numpy
arr = output.detach().cpu().numpy()
print(type(arr), arr.shape)  # <class 'numpy.ndarray'> (4, 10)

# Memorize this chain. You'll use it every time you plot or compute a metric.

Exercise

Generate predictions from any nn.Linear model on a small batch. Convert the output to NumPy and verify the chain step-by-step: comment out detach() and read the error, then comment out .cpu() (move the tensor to GPU first) and read THAT error. Knowing both messages by heart saves real debugging time.