Wirewalk Instituteof AI
mod-206 Intermediate Waitlist

Reinforcement Learning Fundamentals

For engineers and data scientists who want to frame problems as sequential decisions, implement the core RL algorithms in PyTorch, and train and debug agents with Gymnasium and Stable-Baselines3.

The curriculum is being prepared and reviewed. Enrollment is not open yet.

About this course

Reinforcement learning is the branch of machine learning where an agent learns from the consequences of its own actions rather than from labeled examples. It powers game-playing systems, robot controllers and, increasingly, the post-training of large language models. It is also notoriously easy to get wrong: results swing with random seeds, reward functions get exploited, and a bug can look exactly like a hard problem.

This course builds the field from the ground up. You formalize problems as Markov decision processes, solve small ones exactly with value and policy iteration, then move to learning from experience with Monte Carlo, SARSA and Q-learning. The second half goes deep: DQN and its stabilizers, REINFORCE, actor-critic methods with generalized advantage estimation, and PPO, the algorithm that also underpins much of LLM alignment. You implement the core algorithms in PyTorch in the CleanRL single-file style, train them on Gymnasium and MuJoCo environments, compare against Stable-Baselines3, and learn to report results across seeds with proper confidence intervals. The final module connects these foundations to reward design and to how RL is used for language models, which mod-320 and mod-370 cover in depth. Library details are current as of October 2026.

What you will be able to do

  • Formalize a sequential decision problem as an MDP with states, actions, rewards, discount and termination
  • Solve small MDPs with value iteration and policy iteration and explain the Bellman equations they rely on
  • Implement tabular Q-learning and SARSA and explain the on-policy versus off-policy distinction
  • Train a DQN agent with replay and a target network and diagnose overestimation and instability
  • Derive the policy-gradient theorem and implement REINFORCE and an actor-critic with GAE
  • Train and tune PPO with Stable-Baselines3 and interpret its clipping, entropy and KL diagnostics
  • Evaluate RL agents across multiple seeds with confidence intervals and identify reward hacking

Syllabus

6 modules · 22 lessons · about 14 hours

Module 1: Decisions, rewards and the MDP

How RL differs from supervised learning and how to formalize a problem as a Markov decision process.
4 lessons
  1. Agents, environments and the reward hypothesis
  2. Markov decision processes: states, actions, transitions and discounting
  3. Bandits and the exploration-exploitation trade-off: epsilon-greedy, UCB and Thompson sampling
  4. The Gymnasium API: reset, step, spaces, wrappers and vectorized environments

Module 2: Planning with a known model

Bellman equations and the dynamic-programming algorithms that solve small MDPs exactly.
3 lessons
  1. Value functions and the Bellman expectation and optimality equations
  2. Policy evaluation, policy iteration and value iteration on FrozenLake
  3. Generalized policy iteration: the pattern behind every RL algorithm

Module 3: Learning from experience

Model-free tabular methods that learn values from sampled episodes and transitions.
4 lessons
  1. Monte Carlo prediction and control
  2. Temporal-difference learning, TD(0) and n-step returns
  3. SARSA versus Q-learning on CliffWalking: on-policy and off-policy in practice
  4. Eligibility traces and TD(lambda) in brief

Module 4: Deep value-based RL

Function approximation with neural networks and the tricks that make DQN train.
3 lessons
  1. Why function approximation breaks tabular guarantees: the deadly triad
  2. DQN: experience replay, target networks and frame stacking on Atari
  3. Double DQN, dueling networks and prioritized replay

Module 5: Policy gradients, actor-critic and PPO

Optimizing policies directly, from REINFORCE to the clipped objective used across modern RL.
4 lessons
  1. The policy-gradient theorem and REINFORCE with baselines
  2. Actor-critic methods and generalized advantage estimation
  3. PPO: the clipped surrogate objective, entropy bonus and implementation details that matter
  4. Continuous control with SAC and PPO on MuJoCo tasks

Module 6: Running RL experiments and where RL goes next

The experimental discipline RL needs, reward design, and the bridge to RL for language models.
4 lessons
  1. Seeds, variance and honest reporting: interquartile mean and bootstrap confidence intervals
  2. Debugging agents with TensorBoard and Weights & Biases: returns, entropy, value loss and KL
  3. Reward shaping, sparse rewards and specification gaming
  4. From games to language models: offline RL, RLHF and verifiable rewards

Exam and certificate

The exam unlocks once you have completed every lesson. It draws 20 questions from a larger bank, spread across the modules, with the answer options shuffled for each attempt. You have 40 minutes and need 80% to pass. After a failed attempt you can try again after 24 hours, up to 3 attempts in total. Feedback is given per module, never as the answer key.

Passing earns a certificate of completion and assessment from the Wirewalk Institute of AI with a unique ID that anyone can check at institute.wirewalk.com/verify. It is not an accredited qualification or a vendor certification. How certification works.

The Wirewalk Institute of AI is independent. This course teaches publicly documented products and is not affiliated with or endorsed by Farama Foundation (Gymnasium), Stable-Baselines3, CleanRL, PyTorch, or Google DeepMind (MuJoCo). Product and company names are trademarks of their respective owners.

More in Model Development

All 16 courses in this track
mod-201 Intermediate Available

Deep Learning with PyTorch

A hands-on course in modern PyTorch: tensors and autograd, nn.Module design, data loading, a training loop you understand line by line, GPU and mixed-precision training, torch.compile, and exporting a model for deployment.

14 h · 19 lessons See price at checkout
mod-202 Intermediate Available

Transformers from the Ground Up

Build a decoder-only transformer piece by piece: tokenization, embeddings, scaled dot-product attention, multi-head and grouped-query attention, positional schemes such as RoPE, and the training and sampling loop, then map it onto real open-weight model architectures.

16 h · 19 lessons See price at checkout
mod-203 Intermediate Waitlist

Classical Machine Learning in Production

Gradient-boosted trees and linear models still win most tabular problems. This course covers leakage-proof pipelines, honest validation, tuning, calibration and explainability, then packaging, serving and monitoring for drift.

14 h · 19 lessons Join the waitlist