For ML and performance engineers who must make a model smaller and faster without losing the quality that matters, and prove it with measurements.
The curriculum is being prepared and reviewed. Enrollment is not open yet.
About this course
Quantization is the single biggest lever on inference cost, and also the easiest way to ship a model that is quietly worse. This course starts with the numerics — integer and floating-point formats, scales and zero points, per-tensor, per-channel and group-wise granularity, and the microscaling block formats (MXFP4, MXFP8, NVFP4) supported by recent NVIDIA and AMD accelerators — and explains why outlier activations make LLMs hard to quantize.
You then apply the main post-training methods (round-to-nearest, GPTQ, AWQ, SmoothQuant, rotation-based methods) with LLM Compressor, NVIDIA Model Optimizer (formerly TensorRT Model Optimizer) and torchao, compare them with quantization-aware training, and produce checkpoints in the formats engines load: compressed-tensors, Hugging Face, GGUF and ONNX. The second half covers optimization beyond quantization — graph compilation, kernel fusion, FlashAttention-style kernels, CUDA graphs and custom kernels in the Triton language — and closes with an evaluation protocol that catches regressions perplexity misses. Engine-level serving (batching, scheduling, speculative decoding) is covered in srv-310, and pruning and distillation as model-design techniques in mod-340. Format and tool support is stated as of October 2026.
What you will be able to do
Explain how scale, zero point, granularity and format (INT8, INT4, FP8 E4M3/E5M2, MXFP4, NVFP4) determine quantization error and hardware support
Choose between weight-only, weight-and-activation and KV cache quantization for a memory-bound or compute-bound workload
Apply GPTQ, AWQ and SmoothQuant with a calibration set and explain why each method handles outliers differently
Decide when quantization-aware training is worth its cost over post-training quantization
Produce quantized checkpoints in the format a target engine loads and verify that it loads the intended kernels
Use graph compilation, kernel fusion and CUDA graphs to remove overhead, and identify the bottleneck with a profiler before optimizing
Design an evaluation protocol that detects task-level quality loss from quantization, not only perplexity change
Syllabus
6 modules · 20 lessons · about 20 hours
1
Module 1: Numerics of Quantization
Number formats, mapping functions and granularity, and why hardware support decides which formats are worth using.3 lessons
Integer and Floating-Point Formats: INT8, INT4, FP8 E4M3/E5M2 and BF16
Scales, Zero Points and Granularity: Per-Tensor, Per-Channel and Group-Wise
Microscaling Block Formats: MXFP8, MXFP4 and NVFP4 on Current Accelerators
2
Module 2: What to Quantize and Why It Is Hard
Deciding between weights, activations and KV cache, and understanding the outliers that break naive methods.3 lessons
Memory-Bound versus Compute-Bound: Choosing Weight-Only or W8A8
Activation Outliers and Sensitive Layers in Transformers
KV Cache Quantization and Its Effect on Long-Context Quality
3
Module 3: Post-Training Quantization Algorithms
The main PTQ methods, how they use calibration data, and how to run them with current tooling.4 lessons
Round-to-Nearest, Calibration Sets and Error Measurement
GPTQ and AWQ: Second-Order and Activation-Aware Weight Quantization
SmoothQuant and Rotation-Based Methods for W8A8 and W4A4
Running PTQ with LLM Compressor, NVIDIA Model Optimizer and torchao
4
Module 4: Training-Aware Compression and Model Formats
When to train for quantization, how sparsity fits in, and the checkpoint formats each runtime expects.3 lessons
Quantization-Aware Training and Fake Quantization
2:4 Structured Sparsity: Hardware Support, Tooling Gaps and When to Skip It
Checkpoint Formats: compressed-tensors, GGUF K-Quants, ONNX and TensorRT Engines
5
Module 5: Compilers, Kernels and Runtime Overheads
Turning fewer bits into real speed: profiling, compilation, fusion and custom kernels.4 lessons
Profiling Inference with PyTorch Profiler and Nsight Systems
torch.compile, TensorRT and ONNX Runtime Graph Optimizations
Fused Attention Kernels, CUDA Graphs and Launch Overhead
Writing a Fused Dequantize-and-Matmul Kernel in Triton
6
Module 6: Proving the Optimization Was Worth It
Measuring quality loss and speed gain honestly, and deciding what to ship.3 lessons
Quality Evaluation Beyond Perplexity: Task Evals, Long Context and Tool Calling
Microbenchmarks versus End-to-End Benchmarks
A Decision Framework for Shipping a Quantized Model
Exam and certificate
The exam unlocks once you have completed every lesson. It draws 25 questions from a larger bank, spread across the modules, with the answer options shuffled for each attempt. You have 60 minutes and need 80% to pass. After a failed attempt you can try again after 24 hours, up to 3 attempts in total. Feedback is given per module, never as the answer key.
Passing earns a certificate of completion and assessment from the Wirewalk Institute of AI with a unique ID that anyone can check at institute.wirewalk.com/verify. It is not an accredited qualification or a vendor certification. How certification works.
The Wirewalk Institute of AI is independent. This course teaches publicly documented products and is not affiliated with or endorsed by NVIDIA, TensorRT, NVIDIA Model Optimizer, vLLM, LLM Compressor, PyTorch, torchao, Hugging Face, bitsandbytes, llama.cpp, ONNX Runtime, Triton (triton-lang), or AMD. Product and company names are trademarks of their respective owners.
The working practices that make machine learning reproducible and operable: versioning data, code and models together, tracking experiments, building pipelines, gating releases with CI/CD, and watching models for drift after they ship. Covers both classical ML and the newer LLM-application lifecycle.
A practical path from a model file to a production service: designing the inference API, packaging it in a lean GPU-ready container, running it on Kubernetes with KServe, autoscaling it on the right signal, and rolling out new versions safely. Focused on general ML models; LLM-specific engines are covered in srv-310.
The engineering of LLM inference from first principles to production: why decode is memory-bound, how to size the KV cache, how continuous batching, PagedAttention, prefix caching, quantization and speculative decoding change the economics, and how to run vLLM, SGLang, TensorRT-LLM and Dynamo at scale. Ends with load testing, Kubernetes autoscaling and a defensible cost per million tokens.