For engineers building chat and phone agents that talk to customers in real time: design the conversation, choose between speech-to-speech and cascaded pipelines, and ship a voice agent that handles interruptions, telephony and handoff.
The curriculum is being prepared and reviewed. Enrollment is not open yet.
About this course
Voice is the least forgiving interface for an LLM: users interrupt, talk over the agent, mumble account numbers and hang up after a long pause. This course starts with conversation design that works in both chat and voice, including dialogue state, slot filling with tool calls, confirmation strategies and escalation to a human. It then covers the two main voice architectures: native speech-to-speech models (OpenAI's Realtime API with the gpt-realtime-2.1 models, Google's Gemini Live API, Amazon Nova 2 Sonic on Bedrock) and cascaded pipelines that combine speech-to-text such as Deepgram, an LLM, and text-to-speech such as ElevenLabs or Cartesia.
You will learn voice activity detection and semantic turn detection, barge-in handling, a millisecond latency budget from microphone to first audio, transport over WebSockets, WebRTC and SIP, and how open-source frameworks such as LiveKit Agents and Pipecat orchestrate it all. The final modules cover telephony integration with Twilio and SIP trunks, call-recording and AI-disclosure obligations, simulated-caller testing and production monitoring. Regulatory notes are educational, not legal advice. The course ends in an automated exam; passing earns a Wirewalk Institute of AI certificate of completion and assessment.
What you will be able to do
Design a conversation flow with explicit dialogue state, slot filling, confirmation and human escalation for both chat and voice
Choose between a speech-to-speech model and a cascaded STT-LLM-TTS pipeline for a use case, justifying latency, control, cost and voice quality
Implement turn detection and barge-in so the agent stops speaking and recovers context when interrupted
Build a voice agent with LiveKit Agents or Pipecat that calls tools during a live conversation
Connect a voice agent to the phone network over SIP or Twilio and transfer calls to a human with context
Construct and measure a latency budget from end of user speech to first agent audio
Test voice agents with scripted simulated callers and monitor production calls for failure patterns
Syllabus
6 modules · 18 lessons · about 14 hours
1
Module 1: Conversation Design for Chat and Voice
What makes a dialogue succeed before any audio is involved.3 lessons
Writing for the Ear: How Voice Dialogue Differs from Chat
Dialogue State, Slot Filling and Tool Calls Mid-Conversation
Confirmation, Error Recovery and Escalation to a Human
2
Module 2: Voice Architectures
Speech-to-speech models versus cascaded pipelines, and what each costs you.3 lessons
Speech-to-Speech: OpenAI Realtime API, Gemini Live API and Amazon Nova 2 Sonic
Cascaded Pipelines: Choosing STT, LLM and TTS Components
The Latency Budget: From End of Speech to First Audio
3
Module 3: Real-Time Audio Engineering
Turn-taking, interruptions and getting audio across the network.3 lessons
Voice Activity Detection and Semantic Turn Detection
Barge-In: Canceling Speech and Truncating Context
Transports: WebSockets, WebRTC and Audio Codecs
4
Module 4: Building with Voice Agent Frameworks
Orchestrating sessions, tools and media with open-source frameworks.3 lessons
LiveKit Agents: Sessions, Workers and Tool Calls
Pipecat Pipelines: Frames, Processors and Services
Framework or Direct API: A Decision Guide
5
Module 5: Telephony and Contact-Center Integration
Putting an agent on a phone number and fitting it into existing operations.3 lessons
SIP Trunks, Twilio and the Realtime API's SIP Endpoint
Warm Transfer, DTMF and Voicemail Detection
Recording, Consent and AI Disclosure on Calls
6
Module 6: Testing and Operating Conversational Agents
Finding failures before callers do and catching them when they still happen.3 lessons
Simulated Callers and Scripted Conversation Tests
Measuring Task Completion, Containment and Transfer Quality
Production Monitoring: Latency, Interruptions and Call Review
Exam and certificate
The exam unlocks once you have completed every lesson. It draws 20 questions from a larger bank, spread across the modules, with the answer options shuffled for each attempt. You have 40 minutes and need 80% to pass. After a failed attempt you can try again after 24 hours, up to 3 attempts in total. Feedback is given per module, never as the answer key.
Passing earns a certificate of completion and assessment from the Wirewalk Institute of AI with a unique ID that anyone can check at institute.wirewalk.com/verify. It is not an accredited qualification or a vendor certification. How certification works.
The Wirewalk Institute of AI is independent. This course teaches publicly documented products and is not affiliated with or endorsed by OpenAI, Google, Amazon Web Services, LiveKit, Pipecat, Twilio, Deepgram, ElevenLabs, or Cartesia. Product and company names are trademarks of their respective owners.
A hands-on course on building no-code assistants with the builders already in your workplace tools: ChatGPT projects, custom GPTs and their successor plugins, Claude Projects and Skills, Gemini Gems, and Microsoft 365 Copilot Agent Builder and Copilot Studio. You leave with a tested, documented assistant and a plan to share it safely.
A hands-on, code-first course on the three major LLM APIs side by side: messages, streaming, structured outputs, tool calling, vision, cost, caching, batch, retries, abstraction layers, cloud routes and testing. You finish by building and hardening a small multi-vendor service.
A practitioner's course on retrieval-augmented generation: when to use it, how to ingest, chunk, embed and index documents, how to combine BM25 and vector search with reranking, and how to measure, debug and operate the result. Every module includes working Python sketches.