Wirewalk Instituteof AI
inf-201 Intermediate AvailableFlagship

GPU Infrastructure for AI

For system administrators and infrastructure engineers who are taking ownership of GPU servers and leave able to choose, install, monitor, share, size and power them with confidence.

Level
Intermediate
Study time
12 hours
Lessons
16 in 6 modules
Exam
20 questions

About this course

Most organizations now own or rent GPUs, and the people who keep them running are usually Linux administrators who were never taught what makes a GPU server different. This course gives you the working model and the commands. It starts with why AI work runs on GPUs and the handful of architectural ideas an operator actually needs, then surveys the current data-center lineup (NVIDIA Hopper, Blackwell and Rubin; AMD Instinct MI300 through MI400 series) and how those chips are packaged into PCIe cards, HGX and OAM baseboards and rack-scale NVLink systems.

The middle of the course is hands-on operations: the driver, CUDA and Fabric Manager compatibility rules that cause most failed upgrades; containers with the NVIDIA Container Toolkit and CDI; the AMD ROCm equivalent; reading nvidia-smi and amd-smi properly; fleet telemetry and diagnostics with DCGM; and a triage runbook for ECC, row remapping, XID errors and thermal throttling. The final modules cover GPU sharing (MIG, time-slicing, MPS), memory sizing for inference and training with worked examples, power and cooling arithmetic, and a structured cloud-versus-on-premises comparison. Chip microarchitecture in depth is left to hw-201; this course is about running the hardware. Product facts are stated as of October 2026.

What you will be able to do

  • Explain why AI training and inference run on GPUs and identify whether a workload is compute-bound or memory-bandwidth-bound
  • Compare current NVIDIA and AMD data-center GPUs and choose between PCIe, HGX/OAM baseboard and rack-scale NVLink systems for a stated workload
  • Plan driver, CUDA, Fabric Manager and container toolkit versions that are mutually compatible, and run GPU containers with Docker, Podman or Kubernetes
  • Monitor GPUs with nvidia-smi, amd-smi and DCGM, and run the right diagnostic level for a given situation
  • Triage ECC, row-remapping, XID and thermal events and decide between reset, drain and RMA
  • Choose between MIG, time-slicing and MPS for sharing a GPU among users or services
  • Estimate GPU memory for inference (weights plus KV cache) and for full and parameter-efficient fine-tuning
  • Calculate node and rack power and compare the total cost of on-premises and cloud GPU capacity

Syllabus

6 modules · 16 lessons · about 12 hours

Module 1: Why GPUs, and What Is Inside One

The workload characteristics that make GPUs the default AI processor and the few architectural ideas every operator needs.
2 lessons
  1. Why AI Workloads Run on GPUs
  2. GPU Architecture Essentials for Operators

Module 2: Data-Center GPU Generations and System Form Factors

The current NVIDIA and AMD lineup, and how GPUs are packaged into cards, baseboards and rack-scale systems.
3 lessons
  1. The Current Data-Center GPU Lineup
  2. PCIe Cards vs SXM and OAM Baseboards
  3. NVLink, NVSwitch and Rack-Scale Systems

Module 3: The GPU Software Stack

Drivers, CUDA, libraries and containers on NVIDIA, and the equivalent ROCm stack on AMD, with the compatibility rules that break upgrades.
3 lessons
  1. Driver, CUDA and the Compatibility Matrix
  2. Libraries and Containers: cuDNN, NCCL and the NVIDIA Container Toolkit
  3. The AMD ROCm Stack for Operators

Module 4: Monitoring and Health

Reading the vendor tools, collecting fleet telemetry with DCGM, and a triage runbook for the errors that take GPUs out of service.
3 lessons
  1. Reading nvidia-smi and amd-smi Like an Operator
  2. DCGM: Fleet Telemetry and Diagnostics
  3. ECC, Row Remapping, XID Errors and Thermals

Module 5: Sharing GPUs and Sizing Memory

How to split a GPU between workloads and how to estimate the memory a model needs for inference and for training.
3 lessons
  1. MIG, Time-Slicing and MPS
  2. Sizing GPU Memory for Inference
  3. Sizing GPU Memory for Training and Fine-Tuning

Module 6: Facilities and Sourcing

Power and cooling arithmetic for GPU servers, and a structured way to decide between cloud and on-premises capacity.
2 lessons
  1. Power and Cooling Basics for GPU Servers
  2. Cloud vs On-Prem GPU Capacity

Exam and certificate

The exam unlocks once you have completed every lesson. It draws 20 questions from a larger bank, spread across the modules, with the answer options shuffled for each attempt. You have 40 minutes and need 80% to pass. After a failed attempt you can try again after 24 hours, up to 3 attempts in total. Feedback is given per module, never as the answer key.

Passing earns a certificate of completion and assessment from the Wirewalk Institute of AI with a unique ID that anyone can check at institute.wirewalk.com/verify. It is not an accredited qualification or a vendor certification. How certification works.

The Wirewalk Institute of AI is independent. This course teaches publicly documented products and is not affiliated with or endorsed by NVIDIA, AMD, CUDA, ROCm, DCGM, NCCL, Docker, Podman, Kubernetes, or Prometheus. Product and company names are trademarks of their respective owners.

More in AI Infrastructure and HPC

All 14 courses in this track
inf-101 Basic Waitlist

AI Infrastructure Explained

A plain-language tour of the stack that runs AI: what a GPU server is, why clusters need special networks and storage, how jobs are scheduled, what an AI data center needs in power and cooling, and how cloud and on-premises options differ. No prior infrastructure background required.

6 h · 15 lessons Join the waitlist
inf-210 Intermediate Waitlist

Linux Administration for AI Clusters

The Linux skills that differ on an AI cluster: choosing and imaging the OS, packaging GPU and RDMA drivers that survive kernel updates, NUMA and PCIe topology tuning, cgroup v2 isolation, shared identity and file systems, scientific software environments with Lmod, Spack and containers, and patching a fleet without breaking jobs.

14 h · 19 lessons Join the waitlist
inf-220 Intermediate Waitlist

Kubernetes for AI Workloads

How to make Kubernetes a good home for GPU work: the GPU and Network Operators, device plugins versus Dynamic Resource Allocation, GPU sharing, RDMA with Multus, batch queueing and gang scheduling with Kueue, JobSet and KAI Scheduler, distributed training and multi-node inference patterns, and the observability and cost controls that keep a shared GPU platform fair.

14 h · 19 lessons Join the waitlist