Wirewalk Instituteof AI
bld-240 Intermediate Waitlist

Conversational and Voice Agents

For engineers building chat and phone agents that talk to customers in real time: design the conversation, choose between speech-to-speech and cascaded pipelines, and ship a voice agent that handles interruptions, telephony and handoff.

The curriculum is being prepared and reviewed. Enrollment is not open yet.

About this course

Voice is the least forgiving interface for an LLM: users interrupt, talk over the agent, mumble account numbers and hang up after a long pause. This course starts with conversation design that works in both chat and voice, including dialogue state, slot filling with tool calls, confirmation strategies and escalation to a human. It then covers the two main voice architectures: native speech-to-speech models (OpenAI's Realtime API with the gpt-realtime-2.1 models, Google's Gemini Live API, Amazon Nova 2 Sonic on Bedrock) and cascaded pipelines that combine speech-to-text such as Deepgram, an LLM, and text-to-speech such as ElevenLabs or Cartesia.

You will learn voice activity detection and semantic turn detection, barge-in handling, a millisecond latency budget from microphone to first audio, transport over WebSockets, WebRTC and SIP, and how open-source frameworks such as LiveKit Agents and Pipecat orchestrate it all. The final modules cover telephony integration with Twilio and SIP trunks, call-recording and AI-disclosure obligations, simulated-caller testing and production monitoring. Regulatory notes are educational, not legal advice. The course ends in an automated exam; passing earns a Wirewalk Institute of AI certificate of completion and assessment.

What you will be able to do

  • Design a conversation flow with explicit dialogue state, slot filling, confirmation and human escalation for both chat and voice
  • Choose between a speech-to-speech model and a cascaded STT-LLM-TTS pipeline for a use case, justifying latency, control, cost and voice quality
  • Implement turn detection and barge-in so the agent stops speaking and recovers context when interrupted
  • Build a voice agent with LiveKit Agents or Pipecat that calls tools during a live conversation
  • Connect a voice agent to the phone network over SIP or Twilio and transfer calls to a human with context
  • Construct and measure a latency budget from end of user speech to first agent audio
  • Test voice agents with scripted simulated callers and monitor production calls for failure patterns

Syllabus

6 modules · 18 lessons · about 14 hours

Module 1: Conversation Design for Chat and Voice

What makes a dialogue succeed before any audio is involved.
3 lessons
  1. Writing for the Ear: How Voice Dialogue Differs from Chat
  2. Dialogue State, Slot Filling and Tool Calls Mid-Conversation
  3. Confirmation, Error Recovery and Escalation to a Human

Module 2: Voice Architectures

Speech-to-speech models versus cascaded pipelines, and what each costs you.
3 lessons
  1. Speech-to-Speech: OpenAI Realtime API, Gemini Live API and Amazon Nova 2 Sonic
  2. Cascaded Pipelines: Choosing STT, LLM and TTS Components
  3. The Latency Budget: From End of Speech to First Audio

Module 3: Real-Time Audio Engineering

Turn-taking, interruptions and getting audio across the network.
3 lessons
  1. Voice Activity Detection and Semantic Turn Detection
  2. Barge-In: Canceling Speech and Truncating Context
  3. Transports: WebSockets, WebRTC and Audio Codecs

Module 4: Building with Voice Agent Frameworks

Orchestrating sessions, tools and media with open-source frameworks.
3 lessons
  1. LiveKit Agents: Sessions, Workers and Tool Calls
  2. Pipecat Pipelines: Frames, Processors and Services
  3. Framework or Direct API: A Decision Guide

Module 5: Telephony and Contact-Center Integration

Putting an agent on a phone number and fitting it into existing operations.
3 lessons
  1. SIP Trunks, Twilio and the Realtime API's SIP Endpoint
  2. Warm Transfer, DTMF and Voicemail Detection
  3. Recording, Consent and AI Disclosure on Calls

Module 6: Testing and Operating Conversational Agents

Finding failures before callers do and catching them when they still happen.
3 lessons
  1. Simulated Callers and Scripted Conversation Tests
  2. Measuring Task Completion, Containment and Transfer Quality
  3. Production Monitoring: Latency, Interruptions and Call Review

Exam and certificate

The exam unlocks once you have completed every lesson. It draws 20 questions from a larger bank, spread across the modules, with the answer options shuffled for each attempt. You have 40 minutes and need 80% to pass. After a failed attempt you can try again after 24 hours, up to 3 attempts in total. Feedback is given per module, never as the answer key.

Passing earns a certificate of completion and assessment from the Wirewalk Institute of AI with a unique ID that anyone can check at institute.wirewalk.com/verify. It is not an accredited qualification or a vendor certification. How certification works.

The Wirewalk Institute of AI is independent. This course teaches publicly documented products and is not affiliated with or endorsed by OpenAI, Google, Amazon Web Services, LiveKit, Pipecat, Twilio, Deepgram, ElevenLabs, or Cartesia. Product and company names are trademarks of their respective owners.

More in Building AI Applications

All 14 courses in this track
bld-101 Basic Waitlist

No-Code AI Assistants: Custom GPTs, Projects, Gems and Copilot Studio

A hands-on course on building no-code assistants with the builders already in your workplace tools: ChatGPT projects, custom GPTs and their successor plugins, Claude Projects and Skills, Gemini Gems, and Microsoft 365 Copilot Agent Builder and Copilot Studio. You leave with a tested, documented assistant and a plan to share it safely.

7 h · 17 lessons Join the waitlist
bld-201 Intermediate Available

Building with LLM APIs Across Vendors

A hands-on, code-first course on the three major LLM APIs side by side: messages, streaming, structured outputs, tool calling, vision, cost, caching, batch, retries, abstraction layers, cloud routes and testing. You finish by building and hardening a small multi-vendor service.

14 h · 18 lessons See price at checkout
bld-210 Intermediate Available

Retrieval-Augmented Generation Systems

A practitioner's course on retrieval-augmented generation: when to use it, how to ingest, chunk, embed and index documents, how to combine BM25 and vector search with reranking, and how to measure, debug and operate the result. Every module includes working Python sketches.

14 h · 16 lessons See price at checkout