Deep Learning with PyTorch
A hands-on course in modern PyTorch: tensors and autograd, nn.Module design, data loading, a training loop you understand line by line, GPU and mixed-precision training, torch.compile, and exporting a model for deployment.
For research and ML engineers who need to plan and run a language-model pretraining run end to end: data, tokenizer, architecture, optimizer, stability, evaluation and handoff to post-training.
The curriculum is being prepared and reviewed. Enrollment is not open yet.
Pretraining is where a model's knowledge and most of its capability come from, and where most of its compute is spent. This course follows a pretraining project from decision to handoff. You start with the questions that decide whether to do it at all: pretraining from scratch versus continued pretraining of an open base model, the token and compute budget, and what open recipes such as AI2's OLMo and Hugging Face's FineWeb and SmolLM work have shown publicly.
The data module is the largest: extraction from Common Crawl, language identification, heuristic and model-based quality filtering, MinHash and exact deduplication, PII and toxicity handling, decontamination, and mixing with code, math and multilingual sources using datatrove. You then train a byte-level BPE tokenizer and make architecture and hyperparameter choices using small proxy models and maximal-update parametrization (muP) for transfer. The training modules cover AdamW and newer optimizers such as Muon, warmup-stable-decay schedules, loss spikes and their mitigations (z-loss, QK-norm, gradient clipping, rollback), annealing on high-quality data, and context-length extension. Parallelism, cluster operations and scaling-law compute planning are covered in trn-310, trn-330 and trn-320; this course uses them rather than re-teaching them. Facts are current as of October 2026.
6 modules · 20 lessons · about 24 hours
The exam unlocks once you have completed every lesson. It draws 25 questions from a larger bank, spread across the modules, with the answer options shuffled for each attempt. You have 60 minutes and need 80% to pass. After a failed attempt you can try again after 24 hours, up to 3 attempts in total. Feedback is given per module, never as the answer key.
Passing earns a certificate of completion and assessment from the Wirewalk Institute of AI with a unique ID that anyone can check at institute.wirewalk.com/verify. It is not an accredited qualification or a vendor certification. How certification works.
The Wirewalk Institute of AI is independent. This course teaches publicly documented products and is not affiliated with or endorsed by PyTorch (torchtitan), NVIDIA (Megatron-Core, NeMo), Hugging Face (datatrove, nanotron, FineWeb), Allen Institute for AI (OLMo, Dolma), EleutherAI (lm-evaluation-harness), or Weights & Biases. Product and company names are trademarks of their respective owners.
A hands-on course in modern PyTorch: tensors and autograd, nn.Module design, data loading, a training loop you understand line by line, GPU and mixed-precision training, torch.compile, and exporting a model for deployment.
Build a decoder-only transformer piece by piece: tokenization, embeddings, scaled dot-product attention, multi-head and grouped-query attention, positional schemes such as RoPE, and the training and sampling loop, then map it onto real open-weight model architectures.
Gradient-boosted trees and linear models still win most tabular problems. This course covers leakage-proof pipelines, honest validation, tuning, calibration and explainability, then packaging, serving and monitoring for drift.