Wirewalk Instituteof AI
mod-205 Intermediate Waitlist

Speech and Audio Models

For engineers building transcription, voice and audio-understanding features who need to choose, evaluate, adapt and run open and hosted speech models.

The curriculum is being prepared and reviewed. Enrollment is not open yet.

About this course

Speech is one of the most mature areas of applied AI and one of the easiest to get wrong in production. Word error rate on a public benchmark says little about accented call-center audio, overlapping speakers or domain vocabulary. This course starts with the signal itself, sampling rates, log-mel spectrograms and neural audio codecs, and then explains the three main ASR architectures: CTC, transducers (RNN-T and TDT, as used by NVIDIA Parakeet) and encoder-decoder models such as Whisper and NVIDIA Canary.

You then build the surrounding pipeline: voice activity detection, chunking for long audio, speaker diarization with pyannote, timestamps, and streaming. The course covers text-to-speech, audio language models such as Mistral's Voxtral family that answer questions about audio, and fine-tuning ASR on domain data. It closes with evaluation (WER, CER, real-time factor, latency), deployment, and the consent and voice-cloning risks of synthesis. Model names and rankings are current as of October 2026 and change often; the course teaches how to re-check them on the Hugging Face Open ASR Leaderboard.

What you will be able to do

  • Explain how audio is represented as waveforms, spectrograms and discrete codec tokens
  • Compare CTC, transducer and encoder-decoder ASR architectures and their latency and accuracy trade-offs
  • Build a long-form transcription pipeline with VAD, chunking, timestamps and speaker diarization
  • Measure WER, CER, real-time factor and latency on your own audio, with correct text normalization
  • Fine-tune an ASR model on domain audio and verify the improvement on a held-out set
  • Select and evaluate text-to-speech and audio language models for a product requirement
  • Identify consent, privacy and voice-cloning risks and the controls that address them

Syllabus

6 modules · 18 lessons · about 13 hours

Module 1: Audio as data

What a model actually sees when it hears.
3 lessons
  1. Waveforms, sample rates and log-mel spectrograms
  2. Self-supervised speech encoders and neural audio codecs
  3. Audio datasets, licensing and recording conditions

Module 2: Speech recognition architectures

The main ASR model families and how they decode.
4 lessons
  1. CTC models and greedy versus beam decoding
  2. Transducers: RNN-T, TDT and NVIDIA Parakeet
  3. Encoder-decoder ASR: Whisper and Canary
  4. Reading the Open ASR Leaderboard critically

Module 3: Building a transcription pipeline

Everything around the model that turns hours of audio into usable transcripts.
3 lessons
  1. VAD, chunking and long-form transcription with faster-whisper
  2. Speaker diarization with pyannote.audio and aligning speaker turns to word timestamps
  3. Streaming ASR for live captions and voice agents

Module 4: Measuring and improving accuracy

Honest evaluation and domain adaptation.
3 lessons
  1. WER, CER, normalization and building a domain test set
  2. Fine-tuning ASR with NeMo or Transformers on domain audio
  3. Hotwords, prompting and vocabulary biasing

Module 5: Speech generation and audio understanding

Producing voices and reasoning over audio.
3 lessons
  1. Modern text-to-speech: codec language models and flow-based TTS
  2. Audio language models: question answering and summarization over audio with Voxtral
  3. Speech-to-speech and full-duplex voice models

Module 6: Shipping speech systems responsibly

Running speech models at scale without creating new risks.
2 lessons
  1. Deployment: batching, GPU throughput and real-time factor
  2. Consent, voice cloning, watermarking and privacy of recordings

Exam and certificate

The exam unlocks once you have completed every lesson. It draws 20 questions from a larger bank, spread across the modules, with the answer options shuffled for each attempt. You have 45 minutes and need 80% to pass. After a failed attempt you can try again after 24 hours, up to 3 attempts in total. Feedback is given per module, never as the answer key.

Passing earns a certificate of completion and assessment from the Wirewalk Institute of AI with a unique ID that anyone can check at institute.wirewalk.com/verify. It is not an accredited qualification or a vendor certification. How certification works.

The Wirewalk Institute of AI is independent. This course teaches publicly documented products and is not affiliated with or endorsed by OpenAI (Whisper), NVIDIA (NeMo, Parakeet, Canary), Mistral (Voxtral), pyannote.audio, Hugging Face (Transformers, Open ASR Leaderboard), torchaudio, or faster-whisper (CTranslate2). Product and company names are trademarks of their respective owners.

More in Model Development

All 16 courses in this track
mod-201 Intermediate Available

Deep Learning with PyTorch

A hands-on course in modern PyTorch: tensors and autograd, nn.Module design, data loading, a training loop you understand line by line, GPU and mixed-precision training, torch.compile, and exporting a model for deployment.

14 h · 19 lessons See price at checkout
mod-202 Intermediate Available

Transformers from the Ground Up

Build a decoder-only transformer piece by piece: tokenization, embeddings, scaled dot-product attention, multi-head and grouped-query attention, positional schemes such as RoPE, and the training and sampling loop, then map it onto real open-weight model architectures.

16 h · 19 lessons See price at checkout
mod-203 Intermediate Waitlist

Classical Machine Learning in Production

Gradient-boosted trees and linear models still win most tabular problems. This course covers leakage-proof pipelines, honest validation, tuning, calibration and explainability, then packaging, serving and monitoring for drift.

14 h · 19 lessons Join the waitlist