trn-201
Intermediate
Waitlist
A hands-on introduction to CUDA C++: the execution model, the memory hierarchy, the classic parallel patterns, streams and CUDA Graphs, and profiling with Nsight. It finishes with the kernels deep learning actually needs, written in CUDA, Triton and the tile-based cuTile Python DSL and plugged into PyTorch.
trn-210
Intermediate
Waitlist
The single-node foundations of fast training: what FP32, TF32, BF16, FP16, FP8, MXFP8 and NVFP4 actually trade away, how to use autocast and FP8 recipes safely, where every gigabyte of GPU memory goes, and the techniques (checkpointing, accumulation, compiled and fused kernels) that make a step fit and run faster.
trn-310
Expert
Available
A practitioner's course on how large-model training is actually distributed: data parallelism and sharding (DDP, ZeRO, FSDP2), tensor, pipeline, context and expert parallelism, how to combine them, and how to launch, checkpoint, measure and debug the result on a real cluster.
trn-320
Expert
Waitlist
How to turn the scaling-laws literature into a defensible compute plan: counting training FLOPs, what Kaplan, Chinchilla and their replications actually found, when to deliberately train past compute-optimal, how to fit your own laws from small runs, and how to convert a model and token budget into GPUs, weeks, risk buffers and cost.
trn-330
Expert
Waitlist
An engineering course on keeping long training runs productive: measuring goodput, choosing checkpoint intervals, capturing complete and reshardable training state, saving fast with asynchronous and in-memory checkpoints, detecting hangs, bad nodes and silent corruption, and recovering automatically through job restarts, in-process restarts and elastic training.
trn-340
Expert
Waitlist
A methodical approach to training performance: define the right metrics, classify a step as compute-, memory-, launch-, input- or communication-bound, capture and read traces with the PyTorch Profiler, Nsight Systems, Nsight Compute and Holistic Trace Analysis, then fix the bottleneck on the host, the GPU or the network and prove the gain is real.
trn-350
Expert
Waitlist
A deep course on how collectives move data across GPUs: the algorithms and cost models, NCCL's internals (channels, algorithms, protocols and topology search), NVLink and rack-scale NVLink domains, InfiniBand and RoCE scale-out networks, rigorous measurement with nccl-tests and NCCL observability, and newer device-initiated and all-to-all patterns used by mixture-of-experts training.