Deep Learning with PyTorch
A hands-on course in modern PyTorch: tensors and autograd, nn.Module design, data loading, a training loop you understand line by line, GPU and mixed-precision training, torch.compile, and exporting a model for deployment.
For ML engineers and evaluation leads who must measure what a model can actually do, compare models fairly, and defend the numbers to skeptics.
Benchmark numbers drive model selection, funding decisions and regulatory filings, yet small changes in prompt format, few-shot examples, answer extraction or sampling settings can move a score by several points. This course teaches evaluation as measurement science. You learn how lm-evaluation-harness, lighteval and Inspect differ, trace why the same model gets different results in each (with an optional hands-on exercise that runs one small model through two harnesses on a GPU), and learn to report scores with confidence intervals and paired significance tests rather than single numbers.
The course covers the major benchmark families (knowledge, reasoning, math, code, long context, multilingual and agentic tasks), how they saturate and how contamination is detected, and why once-standard benchmarks get retired, as with Hugging Face's Open LLM Leaderboard in 2025 and OpenAI's decision in February 2026 to stop reporting SWE-bench Verified. You then build a private, versioned evaluation suite for your own domain, including LLM-as-judge with calibration against human labels, and pairwise human preference methods such as Elo and Bradley-Terry arenas. The final module covers dangerous-capability and safety evaluations and how evaluation evidence feeds governance, including the EU AI Act obligations for general-purpose models with systemic risk. This course evaluates models; evaluating whole LLM applications is covered in bld-230. Facts are current as of October 2026.
6 modules · 20 lessons · about 18 hours
The exam unlocks once you have completed every lesson. It draws 25 questions from a larger bank, spread across the modules, with the answer options shuffled for each attempt. You have 60 minutes and need 80% to pass. After a failed attempt you can try again after 24 hours, up to 3 attempts in total. Feedback is given per module, never as the answer key.
Passing earns a certificate of completion and assessment from the Wirewalk Institute of AI with a unique ID that anyone can check at institute.wirewalk.com/verify. It is not an accredited qualification or a vendor certification. How certification works.
The Wirewalk Institute of AI is independent. This course teaches publicly documented products and is not affiliated with or endorsed by EleutherAI (lm-evaluation-harness), Inspect (open-source evaluation framework), Stanford CRFM (HELM), Hugging Face (lighteval), LMArena, SWE-bench, or vLLM. Product and company names are trademarks of their respective owners.
A hands-on course in modern PyTorch: tensors and autograd, nn.Module design, data loading, a training loop you understand line by line, GPU and mixed-precision training, torch.compile, and exporting a model for deployment.
Build a decoder-only transformer piece by piece: tokenization, embeddings, scaled dot-product attention, multi-head and grouped-query attention, positional schemes such as RoPE, and the training and sampling loop, then map it onto real open-weight model architectures.
Gradient-boosted trees and linear models still win most tabular problems. This course covers leakage-proof pipelines, honest validation, tuning, calibration and explainability, then packaging, serving and monitoring for drift.