Creating Tensors

The factory functions you'll actually use

You'll create tensors three ways: from Python data, from a shape, or from another tensor. The factory zoo is wide; in practice, six functions cover ~95% of real code.

Function	What it gives you	Typical use
`torch.tensor`	Tensor from Python list / scalar	Quick test data, labels
`torch.zeros` / `ones`	Filled with 0 or 1	Bias init, masks, accumulators
`torch.randn`	Normal(0, 1)	Weight init, dummy inputs
`torch.rand`	Uniform [0, 1)	Probabilities, dropout-style noise
`torch.arange`	Integer / float range	Positional encodings, indices
`torch.full`	Filled with constant	Padding values, masks

The like family — torch.zeros_like(x), torch.randn_like(x), torch.empty_like(x) — copies x's shape, dtype, AND device, which is exactly what you want when allocating a sibling tensor for a result. It's the small comfort that prevents 90% of "expected cuda:0 got cpu" errors.

Two stylistic rules to commit to

Prefer torch.randn(N) over torch.tensor(np.random.randn(N)). Detour through NumPy is unnecessary and breaks dtype/device contracts silently.
Pass device=... at creation time when you can. Allocating on CPU then .to('cuda') doubles allocation traffic for no reason.

Code

From Python data and from shape·python

import torch

# From Python lists / scalars
t1 = torch.tensor([1, 2, 3, 4])           # int64 by default for ints
t2 = torch.tensor([[1.0, 2.0], [3.0, 4.0]])  # float32 for floats
t3 = torch.tensor(3.14)                    # 0D scalar tensor

# Force a dtype
t4 = torch.tensor([1, 2, 3], dtype=torch.float32)
t5 = torch.tensor([0, 1, 1, 0], dtype=torch.bool)

# From shape — these all return a (3, 4) tensor
zeros = torch.zeros(3, 4)
ones = torch.ones(3, 4)
sevens = torch.full((3, 4), 7.0)
identity = torch.eye(4)

Random tensors and sequences·python

import torch

# Standard normal (mean 0, std 1) — the canonical weight init starting point
w = torch.randn(128, 64)

# Uniform [0, 1)
p = torch.rand(32)

# Random integers (low inclusive, high exclusive)
labels = torch.randint(0, 10, (32,))     # 32 labels in [0, 10)

# Sequences
torch.arange(0, 10, 2)         # tensor([0, 2, 4, 6, 8])
torch.linspace(0.0, 1.0, 5)    # tensor([0.0, 0.25, 0.5, 0.75, 1.0])
torch.logspace(0, 3, 4)        # tensor([1., 10., 100., 1000.])

The *_like family — copy the friend's metadata·python

import torch

x = torch.randn(8, 3, 224, 224, device="cpu", dtype=torch.float32)

# Allocate a sibling tensor — same shape, dtype, AND device
buffer = torch.zeros_like(x)
noise = torch.randn_like(x)

# Without _like you'd have to repeat all three:
# buffer = torch.zeros(8, 3, 224, 224, dtype=x.dtype, device=x.device)
# That's the bug-magnet you avoid by using _like.

Exercise

Write a function init_xavier(in_features, out_features) that returns a weight tensor of shape (out_features, in_features) initialized from a normal distribution with std = sqrt(2 / (in_features + out_features)). Compare your output's std to nn.Linear(in, out).weight.std() — they should be in the same ballpark.

The factory functions you'll actually use

Two stylistic rules to commit to

Code

External links

Exercise

Progress

Comments 0