Wirewalk Instituteof AI
hw-201 Intermediate AvailableFlagship

GPU Hardware Deep Dive: Architecture, Memory and Interconnects

For hardware and infrastructure engineers who need to reason about GPU microarchitecture, number formats, HBM, interconnects, power and reliability well enough to evaluate, size and operate NVIDIA, AMD and Intel accelerators.

Level
Intermediate
Study time
14 hours
Lessons
21 in 6 modules
Exam
20 questions

About this course

Spec sheets for AI accelerators are dense with numbers that are easy to quote and easy to misread. This course gives hardware and infrastructure engineers the architectural model needed to read them correctly. It starts inside the chip: streaming multiprocessors and compute units, the SIMT execution model of warps and wavefronts, tensor and matrix cores, register files, shared memory and caches. It then covers number formats from FP64 down to block-scaled FP4 (MXFP4 and NVFP4), structured sparsity, and why the headline throughput figure on a datasheet is rarely the one your workload will see.

The second half follows data outward: HBM generations from HBM2e to HBM4 and the capacity-versus-bandwidth trade-off, roofline analysis with worked arithmetic, and the accelerator generations that matter in 2026 (NVIDIA Hopper, Blackwell and Vera Rubin; AMD CDNA 3, CDNA 4 and the MI400 series; Intel Gaudi 3 and Intel's roadmap). It covers PCIe Gen5 and Gen6, NVLink and NVSwitch generations, Infinity Fabric, UALink and CXL, and rack-scale designs such as NVL72 and Helios. It closes with power and thermal design, liquid cooling, ECC, row remapping and error telemetry, and a worked datasheet comparison. Product figures are stated as of October 2026 and labeled where vendors still mark them preliminary. The course ends with an automated exam and a Wirewalk Institute certificate of completion and assessment; it is not a vendor certification.

What you will be able to do

  • Explain how SMs and CUs, warps and wavefronts, schedulers, tensor and matrix cores, register files and caches cooperate to execute GPU work, and predict how occupancy and data reuse affect performance
  • Compare FP64, FP32, TF32, BF16, FP16, FP8, FP6 and FP4 formats, including block-scaled MXFP and NVFP4 formats, and interpret dense versus sparse throughput figures correctly
  • Describe HBM generations and calculate per-stack and per-GPU bandwidth and capacity, then size weights, KV cache and activations against a GPU's memory
  • Apply roofline analysis to determine whether a kernel or inference workload is compute-bound or bandwidth-bound on a given accelerator
  • Compare NVIDIA Hopper, Blackwell and Vera Rubin, AMD MI300, MI350 and MI400 series and Intel Gaudi 3 on the specifications that matter for a given workload
  • Distinguish PCIe, NVLink and NVSwitch, Infinity Fabric, UALink and CXL by role, bandwidth and topology, and explain the design of rack-scale scale-up domains
  • Plan for the power, cooling and reliability characteristics of dense GPU systems, including liquid cooling, ECC, row remapping and error telemetry
  • Read a GPU datasheet critically, normalizing for sparsity, precision, form factor and preliminary status before comparing products

Syllabus

6 modules · 21 lessons · about 14 hours

Module 1: Inside the GPU: Microarchitecture

How SMs and CUs, warps and wavefronts, schedulers, tensor and matrix cores and the on-chip memory hierarchy execute work.
4 lessons
  1. SMs and CUs: How a Data Center GPU Is Organized
  2. Warps, Wavefronts and Schedulers: The SIMT Execution Model
  3. Tensor Cores and Matrix Cores: How the Matrix Engines Work
  4. Registers, Shared Memory and Caches: The On-Chip Memory Hierarchy

Module 2: Number Formats and Throughput

What each precision format trades away, how block scaling makes FP8 and FP4 usable, and how to read throughput tables honestly.
3 lessons
  1. Floating-Point Anatomy: FP64, FP32, TF32, BF16 and FP16
  2. FP8, FP6 and FP4: Block-Scaled Formats Including MXFP4 and NVFP4
  3. Structured Sparsity and Dense Versus Sparse Throughput

Module 3: HBM and the Memory Wall

HBM generations, capacity versus bandwidth, and roofline analysis with worked examples.
3 lessons
  1. HBM Generations: HBM2e, HBM3, HBM3E and HBM4
  2. Capacity Versus Bandwidth: Sizing Weights, KV Cache and Activations
  3. Roofline Analysis: Worked Examples on Real Accelerators

Module 4: Accelerator Generations in 2026

NVIDIA Hopper, Blackwell and Vera Rubin, AMD CDNA from MI300 to MI400, and the status of Intel Gaudi.
4 lessons
  1. NVIDIA Hopper and Blackwell: H100 Through B300
  2. NVIDIA Vera Rubin: HBM4, NVLink 6 and the Rack as the Product
  3. AMD CDNA: MI300X, MI325X, MI350 Series and MI400 Series
  4. Intel Gaudi 3 and Intel's Data Center AI Roadmap

Module 5: Interconnects and Rack-Scale Design

PCIe, NVLink and NVSwitch, Infinity Fabric, UALink and CXL, and how they combine into rack-scale scale-up domains.
4 lessons
  1. PCIe Gen5 and Gen6: Lanes, Switches and Host Topology
  2. NVLink and NVSwitch Generations
  3. Infinity Fabric, UALink and CXL
  4. Rack-Scale Systems: NVL72, Helios and Scale-Up Domains

Module 6: Power, Reliability and Reading Datasheets

Power and thermal design, liquid cooling, memory reliability features, and a worked datasheet comparison.
3 lessons
  1. Power and Thermal Design: TDP, TBP and Liquid Cooling
  2. Reliability: ECC, Row Remapping and Error Telemetry
  3. Reading a GPU Datasheet: A Worked Comparison

Exam and certificate

The exam unlocks once you have completed every lesson. It draws 20 questions from a larger bank, spread across the modules, with the answer options shuffled for each attempt. You have 45 minutes and need 80% to pass. After a failed attempt you can try again after 24 hours, up to 3 attempts in total. Feedback is given per module, never as the answer key.

Passing earns a certificate of completion and assessment from the Wirewalk Institute of AI with a unique ID that anyone can check at institute.wirewalk.com/verify. It is not an accredited qualification or a vendor certification. How certification works.

The Wirewalk Institute of AI is independent. This course teaches publicly documented products and is not affiliated with or endorsed by NVIDIA, AMD, Intel, JEDEC, PCI-SIG, UALink Consortium, or CXL Consortium. Product and company names are trademarks of their respective owners.

hw-001 Free intro Available

Inside an AI Data Center: GPUs, Networks and Storage in One Hour

A plain-English, one-hour walk through an AI data center: what a model asks of hardware, what is inside a GPU server, how GPUs talk over NVLink, InfiniBand and Ethernet, where the data lives, and why AI racks need so much power and cooling.

1 h · 6 lessons Free intro
hw-101 Basic Available

AI Hardware Fundamentals: GPUs, CPUs, Memory, Networking and Storage

A plain-language tour of the hardware that runs AI: why AI is limited by compute, memory bandwidth and data movement; what GPUs, HBM and precision formats are; how servers, networks, storage, power and cooling fit together; and how to size a simple inference or training system and weigh cloud against buying.

8 h · 20 lessons See price at checkout
hw-202 Intermediate Waitlist

CPUs for AI: x86, Arm, AVX-512/AMX and Grace/Graviton

How server CPUs are built and where they matter in AI systems: cores, chiplets, caches, memory channels and PCIe lanes; vector and matrix extensions from AVX-512 and AMX to SVE2 and SME; the 2026 Xeon 6, EPYC 9005 and 9006, Grace and Vera, and cloud Arm lineups; and how to pick and tune a host CPU so GPUs are never starved. Reviewed against vendor sources as of October 2026.

12 h · 21 lessons Join the waitlist