PyTorch Quest

Tracks

1. 01🔷Tensors — The Foundation

   0/11 lessons

   Shape, dtype, device, stride. The four numbers you'll check ten thousand times.

   Everything in PyTorch is a tensor — model weights, input batches, gradients, loss values, even your learning rate schedule. Master tensors and the rest of the framework starts to feel like syntax sugar over them.

   Lesson list (11)

   • 01
     What Tensors Actually Are~12 min · tensor, ndarray, shape
   • 02
     Creating Tensors~15 min · factory, zeros, randn, arange
   • 03
     Shape, Dtype, Device — The Three Numbers~12 min · dtype, device, shape, metadata
   • 04
     Indexing and Slicing~14 min · indexing, slicing, fancy, boolean
   • 05
     Reshape, View, Permute, and the Contiguous Trap~14 min · reshape, view, permute, contiguous, stride
   • 06
     Math: Element-wise, Matmul, and Broadcasting~15 min · matmul, broadcasting, reduction
   • 07
     In-place Operations and the Trailing Underscore~10 min · inplace, memory, autograd
   • 08
     PyTorch ↔ NumPy (and the GPU caveat)~10 min · numpy, interop, memory-sharing
   • 09
     CPU, CUDA, MPS — Devices and Movement~14 min · device, cuda, mps, apple-silicon
   • 10
     requires_grad — The Gateway to Autograd~10 min · autograd, requires_grad, graph
   • 11
     Storage, Stride, and Reading Memory Maps~12 min · memory, stride, storage, contiguous

   ✓Quiz · 7 questions→

2. 02🧮Autograd — Automatic Differentiation

   0/10 lessons

   The thing that makes neural networks trainable. Dynamic graphs, backward passes, and the gotchas you'll hit.

   Autograd is the engine that turns a forward pass into trainable parameters. This track unpacks the dynamic computational graph, the .backward() call, gradient accumulation, and the modern AMP / clip / detect_anomaly toolset you'll need to debug training that goes sideways.

   Lesson list (10)

   • 01
     What Autograd Actually Does~12 min · autograd, graph, backward
   • 02
     The Computational Graph and grad_fn~12 min · graph, grad_fn, leaf
   • 03
     .backward() and the .grad Buffer~12 min · backward, grad, scalar
   • 04
     no_grad, inference_mode, and detach~10 min · no_grad, inference, detach
   • 05
     Leaf vs Intermediate Tensors and .retain_grad()~10 min · leaf, intermediate, retain_grad
   • 06
     Custom Autograd Functions~12 min · custom, Function, advanced
   • 07
     Gradient Clipping — Taming Exploding Gradients~10 min · clipping, stability, norm
   • 08
     Mixed Precision: autocast and GradScaler~14 min · amp, autocast, fp16, bf16
   • 09
     torch.func — Per-Sample Gradients and Vmap~12 min · torch.func, vmap, grad, jacobian
   • 10
     Debugging Gradients That Misbehave~14 min · debug, anomaly, nan, inf, diagnostics

   ✓Quiz · 8 questions→

3. 03🧠Neural Networks — torch.nn

   0/8 lessons

   Modules, layers, parameters. Subclassing nn.Module is half the framework.

   torch.nn gives you the layer primitives and the nn.Module pattern that wraps them. This track teaches the conventions PyTorch code follows so that any model — yours or someone else's — reads the same way.

   Lesson list (8)

   • 01
     nn.Module — The Subclass Everything Inherits From~14 min · nn.Module, subclass, forward
   • 02
     Linear, Bias, and the MLP~12 min · linear, mlp, bias
   • 03
     Activations: ReLU, GELU, SiLU, and Friends~10 min · activation, relu, gelu, silu
   • 04
     nn.Sequential and Composition Patterns~10 min · sequential, container, composition
   • 05
     Parameters, Buffers, and the State Dict~12 min · parameter, buffer, state_dict
   • 06
     Convolutional Layers — Conv2d and Friends~14 min · conv2d, cnn, channels, padding
   • 07
     Normalization (BatchNorm, LayerNorm, GroupNorm) and Pooling~12 min · batchnorm, layernorm, groupnorm, pool
   • 08
     Dropout, Embedding, and Putting It Together~12 min · dropout, embedding, regularization

   ✓Quiz · 8 questions→

4. 04🔄The Training Loop

   0/8 lessons

   Five steps. The same five steps. Forever. Master them and the rest is engineering.

   PyTorch is a kit for writing your own training loop, not a black-box trainer. This track covers the five-step rhythm, loss functions, optimizers, schedulers, DataLoader plumbing, evaluation, checkpointing, and the production-ready GPU loop.

   Lesson list (8)

   • 01
     The Five-Step Anatomy~12 min · loop, anatomy, rhythm
   • 02
     Loss Functions and What They Expect~12 min · loss, cross_entropy, mse, binary
   • 03
     Optimizers — SGD, Adam, AdamW, and Per-Group Settings~14 min · optimizer, sgd, adam, adamw, weight_decay
   • 04
     Learning Rate Schedulers and Warmup~12 min · scheduler, lr, warmup, cosine
   • 05
     Dataset, DataLoader, and the Worker Plumbing~14 min · dataset, dataloader, num_workers, collate
   • 06
     Metrics and Evaluation~12 min · metrics, accuracy, precision, recall, f1
   • 07
     Checkpointing — Save Everything You'd Need to Resume~12 min · checkpoint, save, load, resume
   • 08
     Putting It Together — A Production-Grade Loop~14 min · production, gpu, logging, tqdm

   ✓Quiz · 8 questions→

5. 05📊Data Handling

   0/7 lessons

   Datasets, transforms, custom collate, HuggingFace, imbalanced data, and the GPU-feeding pipeline.

   The model is the easy part. The pipeline that gets data into the GPU at full throughput — that's where most production engineering happens. This track covers torchvision v2 transforms, custom Datasets, HuggingFace datasets, imbalance handling, and the loader knobs that turn 30%-utilized GPUs into 95%-utilized ones.

   Lesson list (7)

   • 01
     torchvision Datasets and the v2 Transforms API~12 min · torchvision, v2, transforms
   • 02
     Image Augmentation — Random Crops, Flips, Color, MixUp, CutMix~14 min · augmentation, mixup, cutmix, regularization
   • 03
     Custom Datasets for Real Data~14 min · dataset, imagefolder, csv
   • 04
     Custom Collate Functions and IterableDataset~12 min · collate, iterable, stream, padding
   • 05
     Hugging Face Datasets~12 min · huggingface, datasets, arrow, streaming
   • 06
     Imbalanced Data — Sampler, Class Weights, Focal Loss~12 min · imbalanced, sampler, class-weight, focal-loss
   • 07
     Pipeline Performance — Pin Memory, Workers, Prefetch~12 min · performance, pin_memory, num_workers, prefetch

   ✓Quiz · 8 questions→

6. 06🎓Transfer Learning & Hugging Face

   0/6 lessons

   Pretrained weights, frozen layers, LoRA. Fine-tuning is now a tool, not a ritual.

   Almost no one trains from scratch in 2026. Pretrained models — ResNet/ViT for vision, BERT/Llama-family for text — are the starting point. This track covers the three transfer strategies (feature extraction, full fine-tuning, parameter-efficient fine-tuning), the modern torchvision weights API, the HuggingFace Trainer, and LoRA.

   Lesson list (6)

   • 01
     What Transfer Learning Actually Is~12 min · transfer, feature-extraction, fine-tuning
   • 02
     Vision Models — ResNet, EfficientNet, ConvNeXt, ViT~12 min · resnet, efficientnet, convnext, vit
   • 03
     Hugging Face Transformers — AutoModel and Pipelines~14 min · transformers, huggingface, automodel, pipeline
   • 04
     Fine-tuning with the Hugging Face Trainer~14 min · trainer, fine-tune, training-args
   • 05
     PEFT and LoRA — Parameter-Efficient Fine-Tuning~14 min · peft, lora, adapter, fine-tune
   • 06
     Inference Optimization — torch.compile, Quantization, Batching~12 min · inference, compile, quantization, batch

   ✓Quiz · 8 questions→

7. 07⚡Modern PyTorch Patterns

   0/6 lessons

   torch.compile, DDP, FSDP, AMP, gradient checkpointing, profiling. PyTorch 2.x feels native.

   PyTorch 2.x is a different framework from PyTorch 1.x. torch.compile gives you graph-mode speed without giving up eager dev. DDP/FSDP unlock multi-GPU training. AMP and gradient checkpointing unlock larger models. This track covers the modern toolkit that makes large-scale training practical.

   Lesson list (6)

   • 01
     torch.compile() — Graph Mode Without Giving Up Eager~14 min · compile, torchdynamo, inductor
   • 02
     DistributedDataParallel — Multi-GPU Training~15 min · ddp, distributed, multi-gpu
   • 03
     FSDP — Training Models That Don't Fit on One GPU~14 min · fsdp, shard, scale
   • 04
     Mixed Precision and Gradient Checkpointing~12 min · amp, bf16, checkpoint, memory
   • 05
     torch.export — The Modern Export System~12 min · export, torchscript, deploy
   • 06
     Profiling — Finding the Real Bottleneck~12 min · profiler, perf, memory

   ✓Quiz · 8 questions→

8. 08🚀Deployment & Production

   0/6 lessons

   Serialization, ONNX, quantization, serving, edge, MLOps. Get the model off your laptop.

   A trained model is half a product. This track covers the path from .pth file to running production: serialization, torch.export → ONNX / ExecuTorch / CoreML, quantization with torchao, serving via FastAPI / TorchServe / vLLM, edge deployment, and the MLOps practices that keep things alive in the wild.

   Lesson list (6)

   • 01
     Serialization — From state_dict to .pt2~12 min · serialization, save, torchscript, torch.export
   • 02
     Quantization — From fp32 to int8 (and int4)~14 min · quantization, int8, int4, torchao
   • 03
     Serving — FastAPI, TorchServe, vLLM~14 min · serving, fastapi, torchserve, vllm
   • 04
     Edge Deployment — ExecuTorch, CoreML, MLX~14 min · executorch, coreml, mlx, edge
   • 05
     Pruning, Knowledge Distillation, and Benchmarking~12 min · pruning, distillation, benchmark
   • 06
     MLOps — Experiment Tracking, Versioning, Monitoring~12 min · mlops, wandb, mlflow, monitoring

   ✓Quiz · 8 questions→