Wirewalk Instituteof AI
dat-210 Intermediate Waitlist

Document Processing and Unstructured Data Pipelines

For engineers turning PDFs, scans, Office files, email and web pages into clean, structured, searchable data: choose parsers and OCR, extract tables and fields reliably, and run the pipeline at volume.

The curriculum is being prepared and reviewed. Enrollment is not open yet.

About this course

Most enterprise knowledge lives in PDFs, scans, slide decks, spreadsheets and email threads, and most AI projects stall at turning it into clean data. This course teaches the pipeline end to end. You will start with format detection and native text extraction using tools such as PyMuPDF and Apache Tika, then move to layout-aware parsers (Docling, Unstructured, LlamaIndex's LlamaParse) that recover reading order, headings, tables and figures, and to OCR choices ranging from Tesseract and PaddleOCR to Mistral OCR and the cloud services Azure Document Intelligence (now in Microsoft Foundry Tools), Amazon Textract and Google Document AI.

The middle modules focus on getting structure out: tables that survive extraction, key-value fields from forms and invoices, schema-driven extraction with LLMs and validation, classification and splitting of mixed document batches, and confidence-based routing to human review. The final modules cover operating the pipeline: building ground-truth sets and measuring field-level accuracy, handling email, Office and HTML sources, PII redaction, idempotent re-processing when a parser improves, and cost and throughput at millions of pages. How RAG systems chunk and retrieve parsed content is covered in bld-210. The course ends in an automated exam; passing earns a Wirewalk Institute of AI certificate of completion and assessment.

What you will be able to do

  • Classify incoming documents by format and quality and route each to the cheapest extraction method that works
  • Compare layout-aware parsers such as Docling, Unstructured and LlamaParse on reading order, tables and figure handling for a document set
  • Choose between open-source OCR, vision-language OCR models and cloud document services on accuracy, cost and data residency
  • Extract tables and form fields into validated structured records using schema-driven LLM extraction
  • Design confidence thresholds and a human-in-the-loop review queue for low-confidence fields
  • Measure field-level precision and recall against a ground-truth set and track it across parser changes
  • Operate an idempotent, re-runnable document pipeline at volume with cost, throughput and PII controls

Syllabus

6 modules · 19 lessons · about 14 hours

Module 1: The Unstructured Data Problem

What is inside a document file and why extraction fails.
3 lessons
  1. How PDFs Store Text, Fonts and Layout and Why Copy-Paste Lies
  2. Triage: Born-Digital vs Scanned vs Hybrid Documents
  3. Native Extraction with PyMuPDF, pdfplumber and Apache Tika

Module 2: Layout-Aware Parsing

Recovering reading order, structure and figures.
3 lessons
  1. Layout Analysis and Reading Order
  2. Docling, Unstructured and LlamaParse Compared on One Document Set
  3. Output Formats: Markdown, JSON Document Trees and Element Metadata

Module 3: OCR and Vision-Based Extraction

Reading images of text, from classic engines to vision-language models.
3 lessons
  1. Classic OCR: Tesseract, PaddleOCR and Pre-Processing Scans
  2. Vision-Language OCR: Mistral OCR and Open VLM Parsers
  3. Cloud Services: Azure Document Intelligence, Amazon Textract and Google Document AI

Module 4: Tables, Forms and Field Extraction

Turning documents into records that downstream systems can trust.
4 lessons
  1. Table Extraction: Merged Cells, Multi-Page Tables and Validation
  2. Key-Value and Invoice Extraction with Prebuilt and Custom Models
  3. Schema-Driven LLM Extraction with Validation and Cross-Field Checks
  4. Classifying and Splitting Mixed Document Batches

Module 5: Beyond PDFs

Email, Office files, web pages and the sensitive data inside them.
3 lessons
  1. Email Threads, Attachments and Office Formats
  2. HTML and Web Content: Boilerplate Removal and Structure
  3. PII Detection and Redaction in Documents

Module 6: Accuracy and Operations

Measuring, reviewing and running the pipeline at volume.
3 lessons
  1. Ground-Truth Sets and Field-Level Accuracy Metrics
  2. Confidence Routing and Human Review Queues
  3. Idempotent Re-Processing, Throughput and Cost per Thousand Pages

Exam and certificate

The exam unlocks once you have completed every lesson. It draws 20 questions from a larger bank, spread across the modules, with the answer options shuffled for each attempt. You have 40 minutes and need 80% to pass. After a failed attempt you can try again after 24 hours, up to 3 attempts in total. Feedback is given per module, never as the answer key.

Passing earns a certificate of completion and assessment from the Wirewalk Institute of AI with a unique ID that anyone can check at institute.wirewalk.com/verify. It is not an accredited qualification or a vendor certification. How certification works.

The Wirewalk Institute of AI is independent. This course teaches publicly documented products and is not affiliated with or endorsed by Docling, Unstructured, Apache Tika, PyMuPDF, Microsoft, Amazon Web Services, Google, Mistral AI, or LlamaIndex. Product and company names are trademarks of their respective owners.

dat-201 Intermediate Waitlist

Data Engineering for AI

How data pipelines for AI differ from analytics pipelines, and how to build them: lakehouse storage with Parquet and Iceberg or Delta, distributed processing with Spark, Ray Data and DuckDB, text cleaning, filtering and MinHash deduplication at corpus scale, dataset versioning, orchestration with Airflow or Dagster, and feeding GPUs efficiently at training time.

15 h · 18 lessons Join the waitlist
dat-220 Intermediate Waitlist

Data Labeling and Annotation Operations

Run annotation as an operation with measurable quality: task design and guidelines, labeling tools such as Label Studio, CVAT and Argilla, in-house versus vendor workforces, inter-annotator agreement, gold questions and adjudication, model-assisted and active-learning workflows, and the human preference and rubric labeling used for LLM post-training and evaluation.

12 h · 18 lessons Join the waitlist
dat-310 Expert Waitlist

Synthetic Data Generation

An expert course on generating and validating synthetic data: LLM-generated instruction, reasoning and tool-use data (Self-Instruct, Evol-Instruct, persona-driven and verifier-filtered generation), distillation, synthetic evaluation sets, tabular synthesis with SDV and differential privacy, fidelity, utility and privacy metrics, model-collapse risk, and the license terms that govern using model outputs.

18 h · 19 lessons Join the waitlist