Wirewalk Instituteof AI
dat-310 Expert Waitlist

Synthetic Data Generation

For ML engineers and data scientists who need more, safer or more targeted data than they can collect: generate synthetic text, tabular and instruction data with LLMs and statistical models, verify it, and know when it will hurt rather than help.

The curriculum is being prepared and reviewed. Enrollment is not open yet.

About this course

Synthetic data now sits inside most post-training pipelines and many privacy programs, but generated data inherits the generator's blind spots and can quietly degrade a model. This course teaches generation and verification together. For language data you will work through seed-based expansion (Self-Instruct, Evol-Instruct), persona- and taxonomy-driven diversity, multi-turn and tool-use trajectory synthesis, rejection sampling with verifiers and reward models, and distillation from larger teacher models, using open tooling such as NVIDIA NeMo Data Designer and the community-maintained distilabel alongside hosted APIs. You will also build synthetic evaluation sets and learn why they need human audit.

For structured data you will compare statistical and deep generative synthesizers in the Synthetic Data Vault (Gaussian copula, CTGAN, TVAE), add differential privacy, and measure fidelity, downstream utility (train-synthetic-test-real) and privacy leakage through distance-to-closest-record and membership-inference tests. The last modules cover risks and governance: diversity collapse and model collapse under recursive training, benchmark contamination, bias amplification, provider terms that restrict using outputs to train competing models, open-weight license conditions, and documenting synthetic provenance. Labeling of real data is covered in dat-220. The course ends in an automated exam; passing earns a Wirewalk Institute of AI certificate of completion and assessment.

What you will be able to do

  • Decide when synthetic data is appropriate for a goal (coverage, privacy, cost, evaluation) and when collection or labeling is the better investment
  • Build an LLM generation pipeline that controls diversity through seeds, personas and taxonomies and removes near-duplicates
  • Filter generated instruction, reasoning and tool-use data with verifiers, executable checks and reward models
  • Distil a capable teacher into training data for a smaller model while respecting the teacher's license and terms of use
  • Generate tabular synthetic data with SDV and differential privacy and evaluate fidelity, utility and privacy leakage
  • Detect and mitigate model collapse, contamination and bias amplification in synthetic pipelines
  • Document synthetic data provenance, generator versions and validation results for audit

Syllabus

6 modules · 19 lessons · about 18 hours

Module 1: When Synthetic Data Helps

Goals, failure modes and the decision to synthesize.
3 lessons
  1. Coverage, Privacy, Cost and Evaluation: The Four Reasons to Synthesize
  2. How Synthetic Data Fails: Narrow Distributions, Hidden Errors and Generator Bias
  3. Licenses and Terms: Using Model Outputs to Train Other Models

Module 2: Generating Instruction and Dialogue Data

Producing varied, useful text data with LLMs.
4 lessons
  1. Self-Instruct and Evol-Instruct: Seed Expansion and Complexity Evolution
  2. Persona- and Taxonomy-Driven Generation for Diversity
  3. Multi-Turn Conversations and Tool-Use Trajectories
  4. Declarative Generation Pipelines with NeMo Data Designer and distilabel

Module 3: Verification, Filtering and Distillation

Keeping only the generated data that is correct and useful.
4 lessons
  1. Verifiers: Executable Tests, Math Checkers and Schema Validation
  2. Rejection Sampling and Reward-Model Scoring
  3. Distillation from Teacher Models, Including Reasoning Traces
  4. Deduplication and Decontamination of Generated Sets

Module 4: Synthetic Evaluation Data

Using generation to test models without fooling yourself.
2 lessons
  1. Generating Adversarial and Edge-Case Test Sets
  2. Human Audit of Synthetic Evals and Avoiding Self-Grading Loops

Module 5: Tabular and Privacy-Preserving Synthesis

Statistical and deep synthesizers, privacy guarantees and how to measure them.
3 lessons
  1. SDV Synthesizers: Gaussian Copula, CTGAN and TVAE
  2. Differential Privacy for Synthetic Data: Budgets and Trade-offs
  3. Measuring Fidelity, Train-Synthetic-Test-Real Utility and Privacy Leakage

Module 6: Risks at Scale and Governance

Recursive training, bias and the paper trail.
3 lessons
  1. Model Collapse: Evidence, Mechanisms and Mixing with Real Data
  2. Bias Amplification and Representational Harms
  3. Provenance Records for Synthetic Datasets

Exam and certificate

The exam unlocks once you have completed every lesson. It draws 25 questions from a larger bank, spread across the modules, with the answer options shuffled for each attempt. You have 60 minutes and need 80% to pass. After a failed attempt you can try again after 24 hours, up to 3 attempts in total. Feedback is given per module, never as the answer key.

Passing earns a certificate of completion and assessment from the Wirewalk Institute of AI with a unique ID that anyone can check at institute.wirewalk.com/verify. It is not an accredited qualification or a vendor certification. How certification works.

The Wirewalk Institute of AI is independent. This course teaches publicly documented products and is not affiliated with or endorsed by NVIDIA, Hugging Face, DataCebo SDV, OpenAI, Anthropic, or Meta. Product and company names are trademarks of their respective owners.

dat-201 Intermediate Waitlist

Data Engineering for AI

How data pipelines for AI differ from analytics pipelines, and how to build them: lakehouse storage with Parquet and Iceberg or Delta, distributed processing with Spark, Ray Data and DuckDB, text cleaning, filtering and MinHash deduplication at corpus scale, dataset versioning, orchestration with Airflow or Dagster, and feeding GPUs efficiently at training time.

15 h · 18 lessons Join the waitlist
dat-210 Intermediate Waitlist

Document Processing and Unstructured Data Pipelines

Build production pipelines for unstructured documents: format detection and text extraction, layout analysis and reading order, OCR and vision-language parsers, table and form extraction, schema-driven field extraction with LLMs, human review loops, and running it all at scale with measured accuracy and cost.

14 h · 19 lessons Join the waitlist
dat-220 Intermediate Waitlist

Data Labeling and Annotation Operations

Run annotation as an operation with measurable quality: task design and guidelines, labeling tools such as Label Studio, CVAT and Argilla, in-house versus vendor workforces, inter-annotator agreement, gold questions and adjudication, model-assisted and active-learning workflows, and the human preference and rubric labeling used for LLM post-training and evaluation.

12 h · 18 lessons Join the waitlist