GPU Hardware Deep Dive: Architecture, Memory and Interconnects
For hardware and infrastructure engineers who need to reason about GPU microarchitecture, number formats, HBM, interconnects, power and reliability well enough to evaluate, size and operate NVIDIA, AMD and Intel accelerators.
Level
Intermediate
Study time
14 hours
Lessons
21 in 6 modules
Exam
20 questions
About this course
Spec sheets for AI accelerators are dense with numbers that are easy to quote and easy to misread. This course gives hardware and infrastructure engineers the architectural model needed to read them correctly. It starts inside the chip: streaming multiprocessors and compute units, the SIMT execution model of warps and wavefronts, tensor and matrix cores, register files, shared memory and caches. It then covers number formats from FP64 down to block-scaled FP4 (MXFP4 and NVFP4), structured sparsity, and why the headline throughput figure on a datasheet is rarely the one your workload will see.
The second half follows data outward: HBM generations from HBM2e to HBM4 and the capacity-versus-bandwidth trade-off, roofline analysis with worked arithmetic, and the accelerator generations that matter in 2026 (NVIDIA Hopper, Blackwell and Vera Rubin; AMD CDNA 3, CDNA 4 and the MI400 series; Intel Gaudi 3 and Intel's roadmap). It covers PCIe Gen5 and Gen6, NVLink and NVSwitch generations, Infinity Fabric, UALink and CXL, and rack-scale designs such as NVL72 and Helios. It closes with power and thermal design, liquid cooling, ECC, row remapping and error telemetry, and a worked datasheet comparison. Product figures are stated as of October 2026 and labeled where vendors still mark them preliminary. The course ends with an automated exam and a Wirewalk Institute certificate of completion and assessment; it is not a vendor certification.
What you will be able to do
Explain how SMs and CUs, warps and wavefronts, schedulers, tensor and matrix cores, register files and caches cooperate to execute GPU work, and predict how occupancy and data reuse affect performance
Compare FP64, FP32, TF32, BF16, FP16, FP8, FP6 and FP4 formats, including block-scaled MXFP and NVFP4 formats, and interpret dense versus sparse throughput figures correctly
Describe HBM generations and calculate per-stack and per-GPU bandwidth and capacity, then size weights, KV cache and activations against a GPU's memory
Apply roofline analysis to determine whether a kernel or inference workload is compute-bound or bandwidth-bound on a given accelerator
Compare NVIDIA Hopper, Blackwell and Vera Rubin, AMD MI300, MI350 and MI400 series and Intel Gaudi 3 on the specifications that matter for a given workload
Distinguish PCIe, NVLink and NVSwitch, Infinity Fabric, UALink and CXL by role, bandwidth and topology, and explain the design of rack-scale scale-up domains
Plan for the power, cooling and reliability characteristics of dense GPU systems, including liquid cooling, ECC, row remapping and error telemetry
Read a GPU datasheet critically, normalizing for sparsity, precision, form factor and preliminary status before comparing products
Syllabus
6 modules · 21 lessons · about 14 hours
1
Module 1: Inside the GPU: Microarchitecture
How SMs and CUs, warps and wavefronts, schedulers, tensor and matrix cores and the on-chip memory hierarchy execute work.4 lessons
SMs and CUs: How a Data Center GPU Is Organized
Warps, Wavefronts and Schedulers: The SIMT Execution Model
Tensor Cores and Matrix Cores: How the Matrix Engines Work
Registers, Shared Memory and Caches: The On-Chip Memory Hierarchy
2
Module 2: Number Formats and Throughput
What each precision format trades away, how block scaling makes FP8 and FP4 usable, and how to read throughput tables honestly.3 lessons
Floating-Point Anatomy: FP64, FP32, TF32, BF16 and FP16
FP8, FP6 and FP4: Block-Scaled Formats Including MXFP4 and NVFP4
Structured Sparsity and Dense Versus Sparse Throughput
3
Module 3: HBM and the Memory Wall
HBM generations, capacity versus bandwidth, and roofline analysis with worked examples.3 lessons
HBM Generations: HBM2e, HBM3, HBM3E and HBM4
Capacity Versus Bandwidth: Sizing Weights, KV Cache and Activations
Roofline Analysis: Worked Examples on Real Accelerators
4
Module 4: Accelerator Generations in 2026
NVIDIA Hopper, Blackwell and Vera Rubin, AMD CDNA from MI300 to MI400, and the status of Intel Gaudi.4 lessons
NVIDIA Hopper and Blackwell: H100 Through B300
NVIDIA Vera Rubin: HBM4, NVLink 6 and the Rack as the Product
AMD CDNA: MI300X, MI325X, MI350 Series and MI400 Series
Intel Gaudi 3 and Intel's Data Center AI Roadmap
5
Module 5: Interconnects and Rack-Scale Design
PCIe, NVLink and NVSwitch, Infinity Fabric, UALink and CXL, and how they combine into rack-scale scale-up domains.4 lessons
PCIe Gen5 and Gen6: Lanes, Switches and Host Topology
NVLink and NVSwitch Generations
Infinity Fabric, UALink and CXL
Rack-Scale Systems: NVL72, Helios and Scale-Up Domains
6
Module 6: Power, Reliability and Reading Datasheets
Power and thermal design, liquid cooling, memory reliability features, and a worked datasheet comparison.3 lessons
Power and Thermal Design: TDP, TBP and Liquid Cooling
Reliability: ECC, Row Remapping and Error Telemetry
Reading a GPU Datasheet: A Worked Comparison
Exam and certificate
The exam unlocks once you have completed every lesson. It draws 20 questions from a larger bank, spread across the modules, with the answer options shuffled for each attempt. You have 45 minutes and need 80% to pass. After a failed attempt you can try again after 24 hours, up to 3 attempts in total. Feedback is given per module, never as the answer key.
Passing earns a certificate of completion and assessment from the Wirewalk Institute of AI with a unique ID that anyone can check at institute.wirewalk.com/verify. It is not an accredited qualification or a vendor certification. How certification works.
The Wirewalk Institute of AI is independent. This course teaches publicly documented products and is not affiliated with or endorsed by NVIDIA, AMD, Intel, JEDEC, PCI-SIG, UALink Consortium, or CXL Consortium. Product and company names are trademarks of their respective owners.
A plain-English, one-hour walk through an AI data center: what a model asks of hardware, what is inside a GPU server, how GPUs talk over NVLink, InfiniBand and Ethernet, where the data lives, and why AI racks need so much power and cooling.
A plain-language tour of the hardware that runs AI: why AI is limited by compute, memory bandwidth and data movement; what GPUs, HBM and precision formats are; how servers, networks, storage, power and cooling fit together; and how to size a simple inference or training system and weigh cloud against buying.
How server CPUs are built and where they matter in AI systems: cores, chiplets, caches, memory channels and PCIe lanes; vector and matrix extensions from AVX-512 and AMX to SVE2 and SME; the 2026 Xeon 6, EPYC 9005 and 9006, Grace and Vera, and cloud Arm lineups; and how to pick and tune a host CPU so GPUs are never starved. Reviewed against vendor sources as of October 2026.