Wirewalk Instituteof AI
mod-390 Expert Waitlist

Robotics and Embodied AI: Foundation Models for Action

For ML engineers moving into robotics who need to collect demonstration data, train and fine-tune imitation and vision-language-action policies, use simulation and world models, and deploy policies on real robots safely.

The curriculum is being prepared and reviewed. Enrollment is not open yet.

About this course

Robotics is going through the same shift language went through: from task-specific models trained from scratch to large pretrained models that are adapted with a modest amount of task data. Vision-language-action (VLA) models take camera images and an instruction and output robot actions; open models and toolchains now let a small team fine-tune one on its own robot. But robots are not chatbots. Data is expensive to collect, actions must run at control rates on embedded compute, evaluation happens in the physical world, and mistakes break hardware or hurt people.

This course covers that whole loop. You start with how robots are represented for learning: observation and action spaces, control frequencies, embodiments and the LeRobot dataset format. You collect demonstrations by teleoperation and train behavior-cloning policies, including Action Chunking with Transformers and diffusion policies. You then study VLA architectures (OpenVLA's tokenized actions, flow-matching action experts in pi0, pi0.5 and Isaac GR00T, SmolVLA for low-cost arms) and fine-tune an open VLA with LeRobot or openpi. Simulation and world models get their own module: Isaac Lab and MuJoCo Playground for RL and data generation, domain randomization for sim-to-real, and video world models such as NVIDIA Cosmos for synthetic data. The course ends with real-robot evaluation protocols, latency and on-robot inference, and safety engineering for learned controllers. Hardware selection is covered in hw-207. Model and toolkit facts are current as of October 2026.

What you will be able to do

  • Specify observation spaces, action representations and control rates for a learned robot policy
  • Plan and run a teleoperated demonstration-collection campaign and store it in LeRobot format
  • Train and compare behavior-cloning policies including ACT and diffusion policy
  • Explain how VLA models map images and language to actions and compare tokenized and flow-matching action heads
  • Fine-tune an open VLA on a new embodiment and task and diagnose common failure modes
  • Use Isaac Lab or MuJoCo for RL and synthetic data and apply domain randomization for sim-to-real transfer
  • Design a real-robot evaluation protocol with success criteria, trial counts and reset procedures
  • Deploy a policy with latency budgets, safety limits and fallback behaviors on real hardware

Syllabus

6 modules · 22 lessons · about 22 hours

Module 1: Robots as a learning problem

What makes embodied learning different and how robot data is represented.
3 lessons
  1. From classical control to learned policies: where learning helps and where it does not
  2. Observations, action spaces and control rates: joint, end-effector and action chunks
  3. Embodiments and cross-embodiment data: Open X-Embodiment, DROID and the LeRobot dataset format

Module 2: Imitation learning

Learning policies from human demonstrations.
4 lessons
  1. Teleoperation and data collection: leader-follower arms, VR and data quality
  2. Behavior cloning and compounding error: why naive imitation fails
  3. Action Chunking with Transformers (ACT) on low-cost arms
  4. Diffusion policy and flow-matching policies for multimodal action distributions

Module 3: Vision-language-action models

Pretrained generalist policies and how to adapt them.
4 lessons
  1. VLA architectures: VLM backbones, action tokenization and action experts
  2. OpenVLA, pi0 and pi0.5, and SmolVLA: design choices and what each is good for
  3. Humanoid foundation models: Isaac GR00T and dual-system designs
  4. Fine-tuning an open VLA with LeRobot or openpi on a new task

Module 4: Simulation, sim-to-real and world models

Generating experience and data without breaking hardware.
4 lessons
  1. Isaac Sim, Isaac Lab and MuJoCo Playground: GPU-parallel simulation for RL
  2. Sim-to-real transfer: domain randomization, system identification and the reality gap
  3. World foundation models and synthetic data with NVIDIA Cosmos
  4. RL fine-tuning of pretrained policies in simulation and on real robots

Module 5: Evaluating policies in the physical world

Measuring robot performance honestly.
3 lessons
  1. Real-world evaluation protocols: success criteria, trial counts and confidence intervals
  2. Simulation benchmarks and their limits: LIBERO, RoboCasa and SIMPLER-style proxies
  3. Generalization tests: new objects, scenes, instructions and embodiments

Module 6: Deployment and safety

Running learned policies on real robots under real constraints.
4 lessons
  1. On-robot inference: latency budgets, quantization and asynchronous action execution
  2. Integrating learned policies with ROS 2 and low-level controllers
  3. Safety for learned controllers: limits, monitors, e-stops and the role of ISO 10218
  4. A deployment readiness review for a learned manipulation policy

Exam and certificate

The exam unlocks once you have completed every lesson. It draws 25 questions from a larger bank, spread across the modules, with the answer options shuffled for each attempt. You have 60 minutes and need 80% to pass. After a failed attempt you can try again after 24 hours, up to 3 attempts in total. Feedback is given per module, never as the answer key.

Passing earns a certificate of completion and assessment from the Wirewalk Institute of AI with a unique ID that anyone can check at institute.wirewalk.com/verify. It is not an accredited qualification or a vendor certification. How certification works.

The Wirewalk Institute of AI is independent. This course teaches publicly documented products and is not affiliated with or endorsed by Hugging Face (LeRobot, SmolVLA), Physical Intelligence (openpi, pi0, pi0.5), NVIDIA (Isaac Sim, Isaac Lab, Isaac GR00T, Cosmos), Google DeepMind (Gemini Robotics, MuJoCo, MuJoCo Playground), OpenVLA, Open X-Embodiment, or ROS 2. Product and company names are trademarks of their respective owners.

More in Model Development

All 16 courses in this track
mod-201 Intermediate Available

Deep Learning with PyTorch

A hands-on course in modern PyTorch: tensors and autograd, nn.Module design, data loading, a training loop you understand line by line, GPU and mixed-precision training, torch.compile, and exporting a model for deployment.

14 h · 19 lessons See price at checkout
mod-202 Intermediate Available

Transformers from the Ground Up

Build a decoder-only transformer piece by piece: tokenization, embeddings, scaled dot-product attention, multi-head and grouped-query attention, positional schemes such as RoPE, and the training and sampling loop, then map it onto real open-weight model architectures.

16 h · 19 lessons See price at checkout
mod-203 Intermediate Waitlist

Classical Machine Learning in Production

Gradient-boosted trees and linear models still win most tabular problems. This course covers leakage-proof pipelines, honest validation, tuning, calibration and explainability, then packaging, serving and monitoring for drift.

14 h · 19 lessons Join the waitlist