Wirewalk Instituteof AI
inf-220 Intermediate Waitlist

Kubernetes for AI Workloads

For platform engineers who already run Kubernetes and leave able to operate GPU clusters for training, batch and inference with fair sharing, gang scheduling and RDMA networking.

The curriculum is being prepared and reviewed. Enrollment is not open yet.

About this course

Kubernetes was designed for stateless services that scale one pod at a time. AI work is different: a training job needs 64 GPUs to start together or not at all, an inference replica may span two nodes, GPUs are too expensive to leave reserved but idle, and the fast network is RDMA that the default pod network knows nothing about. This course teaches the components and patterns that close that gap.

You begin with the GPU node: the NVIDIA GPU Operator and the AMD GPU Operator, the device plugin, and Dynamic Resource Allocation, whose core APIs became generally available in Kubernetes 1.34. You then share GPUs with MIG and time-slicing, attach InfiniBand or RoCE interfaces with the Network Operator and Multus, and schedule batch work with Kueue quotas, JobSet, the Kubeflow Trainer and NVIDIA's open-source KAI Scheduler or Volcano. Later modules cover multi-node inference with LeaderWorkerSet, KServe and the Gateway API Inference Extension, Ray on KubeRay, storage through CSI drivers, GPU telemetry with the DCGM exporter, and how managed EKS, GKE and AKS differ. Facts are stated as of October 2026.

What you will be able to do

  • Install and upgrade the GPU Operator and Network Operator, and explain each component they deploy
  • Choose between the device plugin and Dynamic Resource Allocation for exposing GPUs, and write ResourceClaims for a given request
  • Configure MIG and time-slicing on Kubernetes and justify the choice for a stated mix of workloads
  • Attach RDMA-capable secondary networks to pods and verify GPUDirect RDMA from inside a pod
  • Implement team quotas, borrowing and gang scheduling with Kueue, JobSet and a GPU-aware scheduler
  • Deploy distributed training and multi-node inference with Kubeflow Trainer, LeaderWorkerSet and KServe
  • Monitor GPU utilization per namespace with the DCGM exporter and Prometheus and act on idle allocations

Syllabus

6 modules · 19 lessons · about 14 hours

Module 1: Making a Node GPU-Ready

The operators, drivers and device APIs that turn a Kubernetes node into a schedulable GPU node.
3 lessons
  1. Inside the NVIDIA GPU Operator: Driver, Toolkit, Device Plugin and Validators
  2. Device Plugins Versus Dynamic Resource Allocation: ResourceClaims and DeviceClasses
  3. AMD GPUs on Kubernetes: The AMD GPU Operator and Device Plugin

Module 2: Sharing GPUs and Wiring the Fast Network

Splitting GPUs among pods and giving pods direct access to the RDMA fabric.
3 lessons
  1. MIG and Time-Slicing Strategies on Kubernetes
  2. The Network Operator, Multus and SR-IOV for InfiniBand and RoCE
  3. Verifying GPUDirect RDMA and NCCL Performance from Inside a Pod

Module 3: Batch Scheduling and Fair Sharing

Queues, quotas and all-or-nothing scheduling for jobs that need many GPUs at once.
4 lessons
  1. Why the Default Scheduler Fails AI Jobs: Partial Starts and Deadlocks
  2. Kueue: ClusterQueues, Cohorts, Borrowing and Preemption
  3. Gang Scheduling with JobSet, KAI Scheduler and Volcano
  4. Topology-Aware Placement for NVLink Domains and Network Locality

Module 4: Training on Kubernetes

Running distributed PyTorch and Ray workloads with checkpoints and failure recovery.
3 lessons
  1. Distributed PyTorch with the Kubeflow Trainer and torchrun
  2. Ray on Kubernetes with KubeRay
  3. Storage for Jobs: CSI Drivers, Parallel File Systems and Checkpoint Volumes

Module 5: Inference on Kubernetes

Serving models that span GPUs and nodes, and routing requests to them intelligently.
3 lessons
  1. Multi-Node Inference with LeaderWorkerSet
  2. KServe and the Gateway API Inference Extension
  3. Autoscaling GPU Inference: HPA, KEDA and Cluster Autoscaling Limits

Module 6: Operating a Shared GPU Platform

Telemetry, cost attribution and the differences between managed Kubernetes offerings.
3 lessons
  1. GPU Observability: DCGM Exporter, Prometheus and Per-Namespace Utilization
  2. Managed GPU Kubernetes: EKS, GKE and AKS Compared
  3. Upgrades, Node Health and Draining GPU Nodes Safely

Exam and certificate

The exam unlocks once you have completed every lesson. It draws 20 questions from a larger bank, spread across the modules, with the answer options shuffled for each attempt. You have 40 minutes and need 80% to pass. After a failed attempt you can try again after 24 hours, up to 3 attempts in total. Feedback is given per module, never as the answer key.

Passing earns a certificate of completion and assessment from the Wirewalk Institute of AI with a unique ID that anyone can check at institute.wirewalk.com/verify. It is not an accredited qualification or a vendor certification. How certification works.

The Wirewalk Institute of AI is independent. This course teaches publicly documented products and is not affiliated with or endorsed by Kubernetes, NVIDIA, AMD, Kueue, Kubeflow, Volcano, Ray, KServe, Amazon EKS, Google Kubernetes Engine, Azure Kubernetes Service, or Prometheus. Product and company names are trademarks of their respective owners.

More in AI Infrastructure and HPC

All 14 courses in this track
inf-101 Basic Waitlist

AI Infrastructure Explained

A plain-language tour of the stack that runs AI: what a GPU server is, why clusters need special networks and storage, how jobs are scheduled, what an AI data center needs in power and cooling, and how cloud and on-premises options differ. No prior infrastructure background required.

6 h · 15 lessons Join the waitlist
inf-201 Intermediate Available

GPU Infrastructure for AI

The operator's view of data-center GPUs: what the current NVIDIA and AMD systems are, how they are packaged and connected, how the driver, CUDA/ROCm and container stack fit together, how to monitor and triage failing GPUs, how to share and size GPU memory, and what power, cooling and sourcing decisions follow.

12 h · 16 lessons See price at checkout
inf-210 Intermediate Waitlist

Linux Administration for AI Clusters

The Linux skills that differ on an AI cluster: choosing and imaging the OS, packaging GPU and RDMA drivers that survive kernel updates, NUMA and PCIe topology tuning, cgroup v2 isolation, shared identity and file systems, scientific software environments with Lmod, Spack and containers, and patching a fleet without breaking jobs.

14 h · 19 lessons Join the waitlist