Wirewalk Instituteof AI
dat-320 Expert Waitlist

Data Quality, Lineage and Governance for AI

For data leaders and senior engineers accountable for the data behind AI systems: enforce quality with contracts and tests, trace lineage from source to model, document datasets, and meet EU AI Act, GDPR and copyright-driven data obligations.

The curriculum is being prepared and reviewed. Enrollment is not open yet.

About this course

When a model misbehaves, the first questions are about data: what it was trained or grounded on, where that came from, who approved it and whether it was fit for purpose. This course builds the capabilities to answer them. It starts with quality dimensions that matter for AI beyond classic completeness and validity: representativeness, label quality, freshness for retrieval, leakage and contamination. You will encode them as data contracts and automated checks with Great Expectations, Soda and dbt tests, wire them into pipelines as blocking gates, and monitor drift in production data.

The lineage modules use OpenLineage events and catalogs such as DataHub, Unity Catalog and Apache Atlas to trace data from source systems through feature and embedding pipelines to model versions, so you can answer impact and deletion questions. Documentation modules cover datasheets for datasets, data cards and MLCommons Croissant metadata. The regulatory module covers EU AI Act Article 10 data governance for high-risk systems, the Commission's template for the public summary of training content for general-purpose AI models, GDPR lawful basis and erasure requests against trained models and vector stores, and text-and-data-mining opt-outs, treated as EU rules that reach US companies selling AI systems into the EU or processing EU residents' data. Regulatory content is educational, not legal advice. The course ends in an automated exam; passing earns a Wirewalk Institute of AI certificate of completion and assessment.

What you will be able to do

  • Define AI-specific data quality dimensions for a use case, including representativeness, label quality, freshness and contamination
  • Write data contracts and automated quality checks with Great Expectations, Soda or dbt and enforce them as pipeline gates
  • Capture dataset and column-level lineage with OpenLineage and query it in a catalog to answer impact and provenance questions
  • Trace a model version back to the exact datasets, versions and transformations that produced it
  • Produce datasheets, data cards and Croissant metadata that meet internal and external documentation needs
  • Map EU AI Act Article 10, the GPAI training-content summary, GDPR and copyright opt-outs to concrete data controls
  • Design a process to honor deletion and objection requests across training sets, fine-tuned models and vector stores

Syllabus

6 modules · 20 lessons · about 18 hours

Module 1: Data Quality for AI

Quality dimensions that predict model behavior, not just pipeline health.
3 lessons
  1. Beyond Completeness and Validity: Representativeness, Label Quality and Freshness
  2. Leakage, Contamination and Train-Test Hygiene as Quality Rules
  3. Bias and Coverage Audits of Training and Evaluation Data

Module 2: Contracts, Tests and Monitoring

Turning quality expectations into code that blocks bad data.
3 lessons
  1. Data Contracts Between Producers and AI Teams
  2. Great Expectations, Soda and dbt Tests as Pipeline Gates
  3. Monitoring Drift and Anomalies in Production Data and Embeddings

Module 3: Lineage from Source to Model

Knowing where every model input came from.
4 lessons
  1. OpenLineage: Runs, Jobs, Datasets and Facets
  2. Catalogs Compared: DataHub, Unity Catalog and Apache Atlas
  3. Connecting Data Lineage to Model Registries and Experiment Tracking
  4. Impact Analysis: What Breaks, and Which Models Are Affected

Module 4: Documenting Datasets

Documentation that serves engineers, auditors and regulators.
3 lessons
  1. Datasheets for Datasets and Data Cards
  2. MLCommons Croissant Metadata and Machine-Readable Documentation
  3. Provenance and License Records for Third-Party and Scraped Data

Module 5: Regulatory Data Obligations

What the law requires of AI data, and the controls that satisfy it.
4 lessons
  1. EU AI Act Article 10: Data Governance for High-Risk Systems
  2. General-Purpose AI: The Training-Content Summary Template and Copyright Policy
  3. GDPR for AI Data: Lawful Basis, Minimization and Erasure Across Models and Vector Stores
  4. Text-and-Data-Mining Opt-Outs and Honoring Them in Pipelines

Module 6: Operating a Data Governance Program for AI

Roles, workflows and metrics that keep governance working.
3 lessons
  1. Ownership, Stewardship and Approval Workflows for AI Datasets
  2. Access Control, Retention and Audit Trails
  3. Governance Metrics and Evidence Packs for Internal Audit

Exam and certificate

The exam unlocks once you have completed every lesson. It draws 25 questions from a larger bank, spread across the modules, with the answer options shuffled for each attempt. You have 60 minutes and need 80% to pass. After a failed attempt you can try again after 24 hours, up to 3 attempts in total. Feedback is given per module, never as the answer key.

Passing earns a certificate of completion and assessment from the Wirewalk Institute of AI with a unique ID that anyone can check at institute.wirewalk.com/verify. It is not an accredited qualification or a vendor certification. How certification works.

The Wirewalk Institute of AI is independent. This course teaches publicly documented products and is not affiliated with or endorsed by Great Expectations, Soda, dbt, OpenLineage, DataHub, Databricks Unity Catalog, Apache Atlas, or MLCommons. Product and company names are trademarks of their respective owners.

dat-201 Intermediate Waitlist

Data Engineering for AI

How data pipelines for AI differ from analytics pipelines, and how to build them: lakehouse storage with Parquet and Iceberg or Delta, distributed processing with Spark, Ray Data and DuckDB, text cleaning, filtering and MinHash deduplication at corpus scale, dataset versioning, orchestration with Airflow or Dagster, and feeding GPUs efficiently at training time.

15 h · 18 lessons Join the waitlist
dat-210 Intermediate Waitlist

Document Processing and Unstructured Data Pipelines

Build production pipelines for unstructured documents: format detection and text extraction, layout analysis and reading order, OCR and vision-language parsers, table and form extraction, schema-driven field extraction with LLMs, human review loops, and running it all at scale with measured accuracy and cost.

14 h · 19 lessons Join the waitlist
dat-220 Intermediate Waitlist

Data Labeling and Annotation Operations

Run annotation as an operation with measurable quality: task design and guidelines, labeling tools such as Label Studio, CVAT and Argilla, in-house versus vendor workforces, inter-annotator agreement, gold questions and adjudication, model-assisted and active-learning workflows, and the human preference and rubric labeling used for LLM post-training and evaluation.

12 h · 18 lessons Join the waitlist