Wirewalk Instituteof AI
hw-202 Intermediate Waitlist

CPUs for AI: x86, Arm, AVX-512/AMX and Grace/Graviton

For infrastructure engineers and architects who need to choose, size and tune the host CPU in GPU servers and the CPUs that run inference, data preparation and agent workloads on their own.

The curriculum is being prepared and reviewed. Enrollment is not open yet.

About this course

In an AI server the CPU rarely gets the headlines, yet it decides how many GPUs and NICs a node can feed, how fast data is decoded and tokenized, how much memory bandwidth is available to offload and preprocessing, and whether small-model or batch inference can run without a GPU at all. This course gives engineers a working model of the modern server CPU: core microarchitecture, chiplet and tile packaging, cache hierarchy, NUMA, memory channels and PCIe and CXL lanes, and how each of those shows up in AI workloads.

The middle of the course covers the instruction-set features that matter for AI on CPUs, including AVX-512, AVX10, AMX with BF16 and FP16 tiles, and Arm SVE2 and SME, and how libraries such as oneDNN, ZenDNN, Arm Compute Library and llama.cpp use them. It then surveys the platforms current in October 2026: Intel Xeon 6 P-core and E-core and Xeon 6+ (Clearwater Forest), AMD EPYC 9005 (Turin) and the Zen 6 EPYC 9006 (Venice), NVIDIA Grace and the Vera CPU in Vera Rubin systems, and cloud Arm CPUs such as AWS Graviton4 and Graviton5, Google Axion and Microsoft Cobalt. It closes with selection and tuning: lanes-per-GPU arithmetic, NUMA pinning, BIOS and power settings, and a worked host-CPU sizing exercise. Vendor figures are stated as of October 2026 and marked as vendor claims where independent results are not yet available. The course ends with an automated exam and a Wirewalk Institute certificate of completion and assessment; it is not a vendor certification.

What you will be able to do

  • Explain how core microarchitecture, chiplet or tile packaging, cache hierarchy and NUMA topology shape CPU performance for AI pipelines
  • Calculate memory bandwidth and PCIe lane budgets for a CPU platform and determine how many GPUs, NICs and NVMe drives it can feed
  • Describe AVX-512, AVX10, AMX, SVE2 and SME and identify which data types and operations each accelerates
  • Estimate whether a given model and batch size can be served acceptably on CPUs, and choose the libraries that exploit the target ISA
  • Compare Intel Xeon 6 and 6+, AMD EPYC 9005 and 9006, NVIDIA Grace and Vera, and cloud Arm CPUs on the specifications that matter for a stated AI role
  • Configure NUMA affinity, BIOS power and memory settings, and process pinning so the host CPU does not bottleneck GPU training or inference
  • Produce a justified host-CPU recommendation for a GPU server or CPU inference fleet, including core count, memory configuration and lane plan

Syllabus

6 modules · 21 lessons · about 12 hours

Module 1: Server CPU Architecture for AI Engineers

Cores, packaging, caches and NUMA, and the jobs the CPU actually performs in an AI system.
4 lessons
  1. What the CPU Does in an AI Server: Orchestration, Data Loading, Preprocessing and Offload
  2. Cores, SMT and Microarchitecture: P-Cores, E-Cores, Zen 5c and Neoverse
  3. Chiplets, Tiles and the Cache Hierarchy
  4. NUMA, Sub-NUMA Clustering and Why Topology Matters

Module 2: Memory and I/O: Feeding the Cores and the GPUs

Memory channels, DDR5 and MRDIMM bandwidth, PCIe and CXL lanes, and how to budget them per GPU.
3 lessons
  1. Memory Channels, DDR5 RDIMM and MRDIMM: Bandwidth per Core
  2. PCIe Gen5 and Gen6 Lanes, Root Complexes and Switches
  3. Lane Budgets: GPUs, NICs and NVMe per Socket

Module 3: Vector and Matrix Extensions

The ISA features that make CPUs useful for AI math and the software that uses them.
4 lessons
  1. AVX-512 and AVX10: Vector Width, Frequency Effects and Data Types
  2. Intel AMX: Tiles, BF16 and FP16 Matrix Math on Xeon
  3. Arm SVE2 and SME: Scalable Vectors and Matrix Engines
  4. Libraries That Use Them: oneDNN, ZenDNN, Arm Compute Library and llama.cpp

Module 4: The 2026 Server CPU Landscape

Current x86 and Arm server platforms, what each is optimized for, and how to read their claims.
4 lessons
  1. Intel Xeon 6 and Xeon 6+: Granite Rapids, Sierra Forest and Clearwater Forest
  2. AMD EPYC 9005 Turin and EPYC 9006 Venice
  3. NVIDIA Grace and Vera: CPUs Built for Coherent GPU Attachment
  4. Cloud Arm CPUs: Graviton4 and Graviton5, Axion and Cobalt

Module 5: CPU Inference and CPU-Side Bottlenecks

When CPUs can serve models on their own, and how to find and remove CPU bottlenecks in GPU jobs.
3 lessons
  1. When CPU Inference Makes Sense: Model Size, Batch and Latency Arithmetic
  2. Data Loader and Tokenizer Bottlenecks: Measuring GPU Starvation
  3. Benchmarking CPUs for AI Without Fooling Yourself

Module 6: Selecting and Tuning the Host CPU

BIOS, power and affinity settings, and a worked selection for a GPU server and a CPU inference fleet.
3 lessons
  1. BIOS, Power and Memory Settings That Matter for AI Nodes
  2. NUMA Pinning and Affinity for GPUs and NICs
  3. Worked Selection: Host CPU for an 8-GPU Node and a CPU Inference Fleet

Exam and certificate

The exam unlocks once you have completed every lesson. It draws 20 questions from a larger bank, spread across the modules, with the answer options shuffled for each attempt. You have 40 minutes and need 80% to pass. After a failed attempt you can try again after 24 hours, up to 3 attempts in total. Feedback is given per module, never as the answer key.

Passing earns a certificate of completion and assessment from the Wirewalk Institute of AI with a unique ID that anyone can check at institute.wirewalk.com/verify. It is not an accredited qualification or a vendor certification. How certification works.

The Wirewalk Institute of AI is independent. This course teaches publicly documented products and is not affiliated with or endorsed by Intel, AMD, NVIDIA, Arm, AWS, Google Cloud, Microsoft Azure, or Ampere. Product and company names are trademarks of their respective owners.

hw-001 Free intro Available

Inside an AI Data Center: GPUs, Networks and Storage in One Hour

A plain-English, one-hour walk through an AI data center: what a model asks of hardware, what is inside a GPU server, how GPUs talk over NVLink, InfiniBand and Ethernet, where the data lives, and why AI racks need so much power and cooling.

1 h · 6 lessons Free intro
hw-101 Basic Available

AI Hardware Fundamentals: GPUs, CPUs, Memory, Networking and Storage

A plain-language tour of the hardware that runs AI: why AI is limited by compute, memory bandwidth and data movement; what GPUs, HBM and precision formats are; how servers, networks, storage, power and cooling fit together; and how to size a simple inference or training system and weigh cloud against buying.

8 h · 20 lessons See price at checkout
hw-201 Intermediate Available

GPU Hardware Deep Dive: Architecture, Memory and Interconnects

An engineer's tour of the modern data center GPU: how SMs and CUs, warps and wavefronts, tensor and matrix cores and the on-chip memory hierarchy work; what FP64 through FP4 and sparsity really buy you; how HBM, PCIe, NVLink, Infinity Fabric, UALink and CXL move data; and how power, cooling and reliability features shape real deployments. Reviewed against vendor sources as of October 2026.

14 h · 21 lessons See price at checkout