For cluster administrators and power users who leave able to configure, tune and troubleshoot Slurm for GPU-heavy AI and HPC workloads.
The curriculum is being prepared and reviewed. Enrollment is not open yet.
About this course
Slurm schedules a majority of the world's largest AI and HPC systems, and since NVIDIA acquired SchedMD in December 2025 it has remained open source on a six-month major release cadence (25.11 and 26.05 are the current lines as of October 2026). Yet many clusters still run defaults that waste GPUs: jobs that can see GPUs they were not allocated, eight-GPU jobs spread across switches, priority rules nobody can explain, and node health problems discovered by users rather than by the scheduler.
This course works through a Slurm deployment as an administrator would. You configure slurmctld, slurmd, slurmdbd and slurmrestd; describe GPUs with gres.conf and NVML autodetection; confine jobs with cgroup v2; and describe the network with the tree and block topology plugins so jobs land inside a switch or an NVLink domain. You then design fair-share, QOS and preemption policies, run containers with Pyxis and Enroot or Apptainer, launch multi-node PyTorch with srun and PMIx, and use prolog, epilog and HealthCheckProgram hooks. The course closes with accounting and reporting with sacct and sreport, troubleshooting stuck jobs and drained nodes, rolling upgrades, and where Slurm and Kubernetes meet, including Slinky. Facts are stated as of October 2026.
What you will be able to do
Explain the roles of slurmctld, slurmd, slurmdbd and slurmrestd and design a highly available control plane
Configure GPU GRES, cgroup v2 confinement and select/cons_tres so jobs receive exactly the GPUs, CPUs and memory they request
Describe network and NVLink topology with the tree and block plugins and verify placement of multi-node jobs
Design partitions, QOS, fair-share and preemption policies for a stated mix of research and production users
Run containerized multi-node training with Pyxis, Enroot and PMIx and diagnose launch failures
Use prolog, epilog and HealthCheckProgram to keep unhealthy GPU nodes out of service automatically
Report utilization with sacct and sreport and plan a rolling Slurm upgrade
Syllabus
6 modules · 19 lessons · about 14 hours
1
Module 1: Slurm Architecture and First Configuration
The daemons, the configuration files and a control plane that survives failure.3 lessons
Daemons and Data Flow: slurmctld, slurmd, slurmdbd and slurmrestd
Writing slurm.conf: Nodes, Partitions and select/cons_tres
Control Plane High Availability, Munge and the Accounting Database
2
Module 2: GPUs, Confinement and Topology
Making Slurm allocate GPUs precisely and place jobs where the network is fastest.3 lessons
gres.conf, NVML Autodetection and GPU Binding Options
cgroup v2 Confinement: ConstrainDevices, Cores and Memory
Topology-Aware Scheduling: topology/tree and topology/block for NVLink Racks
3
Module 3: Policy: Who Runs Next
Priority, fairness and preemption policies that users can understand and administrators can defend.3 lessons
The Multifactor Priority Plugin and Fair-Share Trees
QOS, Limits and TRES Accounting
Backfill and Preemption: Keeping GPUs Busy Without Starving Big Jobs
4
Module 4: Running AI Jobs Well
Job scripts, containers and multi-node launch for distributed training.3 lessons
sbatch, srun and salloc for GPU Jobs: Arrays, Dependencies and Heterogeneous Jobs
Containers with Pyxis and Enroot, and with Apptainer
Multi-Node PyTorch: srun, PMIx, torchrun and NCCL Environment
5
Module 5: Automation and Health
Hooks and checks that keep bad nodes away from jobs, and APIs that let other systems drive Slurm.3 lessons
Prolog, Epilog and HealthCheckProgram with Node Health Check
Job Submit Plugins and Lua Policy Enforcement
Automating with slurmrestd and the REST API
6
Module 6: Day-2 Administration
Reporting, troubleshooting, upgrades and living alongside Kubernetes.4 lessons
Accounting and Reporting with sacct, sreport and sshare
Troubleshooting Pending Jobs, Drained Nodes and Launch Failures
Rolling Upgrades and Version Compatibility Rules
Slurm and Kubernetes Together: Slinky and Hybrid Patterns
Exam and certificate
The exam unlocks once you have completed every lesson. It draws 20 questions from a larger bank, spread across the modules, with the answer options shuffled for each attempt. You have 40 minutes and need 80% to pass. After a failed attempt you can try again after 24 hours, up to 3 attempts in total. Feedback is given per module, never as the answer key.
Passing earns a certificate of completion and assessment from the Wirewalk Institute of AI with a unique ID that anyone can check at institute.wirewalk.com/verify. It is not an accredited qualification or a vendor certification. How certification works.
The Wirewalk Institute of AI is independent. This course teaches publicly documented products and is not affiliated with or endorsed by SchedMD, NVIDIA, Slurm, PMIx, Open MPI, Enroot, Pyxis, Apptainer, or Kubernetes. Product and company names are trademarks of their respective owners.
A plain-language tour of the stack that runs AI: what a GPU server is, why clusters need special networks and storage, how jobs are scheduled, what an AI data center needs in power and cooling, and how cloud and on-premises options differ. No prior infrastructure background required.
The operator's view of data-center GPUs: what the current NVIDIA and AMD systems are, how they are packaged and connected, how the driver, CUDA/ROCm and container stack fit together, how to monitor and triage failing GPUs, how to share and size GPU memory, and what power, cooling and sourcing decisions follow.
The Linux skills that differ on an AI cluster: choosing and imaging the OS, packaging GPU and RDMA drivers that survive kernel updates, NUMA and PCIe topology tuning, cgroup v2 isolation, shared identity and file systems, scientific software environments with Lmod, Spack and containers, and patching a fleet without breaking jobs.