Wirewalk Instituteof AI
bld-270 Intermediate Waitlist

LLM Gateways: Routing, Keys, Quotas and Audit

For platform and application engineers: put a gateway between your organization and every model provider to centralize keys, route and fail over traffic, enforce budgets and quotas, and keep an audit trail.

The curriculum is being prepared and reviewed. Enrollment is not open yet.

About this course

Once more than one team calls more than one model provider, scattered API keys, unbounded spend and missing audit logs become real problems. An LLM gateway solves them in one place: applications call a single OpenAI-compatible endpoint, and the gateway handles credentials, routing, limits, caching, policy and logging. This course builds that layer. You start with the architecture and the build-or-adopt decision, then deploy LiteLLM Proxy as a reference implementation and compare it with Envoy AI Gateway, Kong AI Gateway, Azure API Management (llm-token-limit and related policies), Cloudflare AI Gateway, Portkey (now part of Palo Alto Networks) and OpenRouter.

Modules cover identity and virtual keys tied to SSO and teams; routing across OpenAI, Anthropic, Gemini, Amazon Bedrock, Microsoft Foundry and self-hosted vLLM, with fallbacks and load balancing; token-based rate limits, budgets and chargeback; exact and semantic caching; guardrail and data-loss-prevention hooks; and request logging with OpenTelemetry GenAI conventions for audit and cost reporting. A dedicated lesson covers the gateway itself as a high-value target, including the March 2026 compromise of LiteLLM releases on PyPI. Resilience engineering in depth is covered in bld-330. Facts are stated as of October 2026. The course ends in an automated exam; passing earns a Wirewalk Institute of AI certificate of completion and assessment.

What you will be able to do

  • Explain the LLM gateway pattern and decide whether to adopt an open-source, managed or API-management-based gateway
  • Deploy LiteLLM Proxy with a model list spanning hosted APIs, cloud platforms and a self-hosted vLLM endpoint
  • Issue virtual keys scoped to teams and applications and integrate them with SSO and secret management
  • Configure routing, load balancing and fallbacks across providers and regions
  • Enforce token-based rate limits, budgets and chargeback reporting per team
  • Add caching, guardrail and DLP hooks without breaking streaming or tool calling
  • Produce audit logs and cost reports from gateway telemetry, and harden the gateway as a supply-chain and credential target

Syllabus

6 modules · 19 lessons · about 12 hours

Module 1: Why Gateways and Which One

The problems a gateway solves and the main product families.
3 lessons
  1. The Gateway Pattern: One Endpoint, Many Providers
  2. Landscape: LiteLLM, Envoy AI Gateway, Kong, Azure API Management, Cloudflare, Portkey and OpenRouter
  3. Build, Adopt or Extend Your API Management Platform

Module 2: Deploying a Reference Gateway

A working LiteLLM Proxy in front of several providers.
3 lessons
  1. LiteLLM Proxy: Model List, Database and Admin UI
  2. Normalizing APIs: OpenAI-Compatible Chat, Responses, Embeddings and Tool Calls
  3. Running the Gateway in Production: Kubernetes, Redis, Postgres and High Availability

Module 3: Identity, Keys and Access

Who may call which model, with which credentials.
3 lessons
  1. Virtual Keys, Teams and Projects
  2. SSO, Service Identities and Model Access Policies
  3. Provider Credentials: Vaults, Rotation and Cloud Workload Identity

Module 4: Routing and Resilience

Sending each request to the right model, and somewhere else when that fails.
3 lessons
  1. Routing Strategies: Latency, Cost, Usage-Based and Tag-Based
  2. Fallbacks, Retries and Cooldowns Across Providers and Regions
  3. Model Aliases and Safe Model Migrations Behind the Gateway

Module 5: Quotas, Budgets and Caching

Controlling spend and load centrally.
3 lessons
  1. Token and Request Rate Limits: Azure llm-token-limit, Envoy AI Gateway, Kong and LiteLLM Compared
  2. Budgets, Spend Alerts and Chargeback by Team
  3. Exact and Semantic Caching at the Gateway: Gains and Correctness Risks

Module 6: Policy, Audit and Gateway Security

Guardrails, logging and protecting the gateway itself.
4 lessons
  1. Guardrail and DLP Hooks: PII Redaction, Prompt-Injection Screening and Content Policy
  2. Audit Logging with OpenTelemetry GenAI Conventions: What to Log and What Not To
  3. The Gateway as a Target: The March 2026 LiteLLM PyPI Compromise and Supply-Chain Hardening
  4. Gateways for MCP and Agent Traffic

Exam and certificate

The exam unlocks once you have completed every lesson. It draws 20 questions from a larger bank, spread across the modules, with the answer options shuffled for each attempt. You have 40 minutes and need 80% to pass. After a failed attempt you can try again after 24 hours, up to 3 attempts in total. Feedback is given per module, never as the answer key.

Passing earns a certificate of completion and assessment from the Wirewalk Institute of AI with a unique ID that anyone can check at institute.wirewalk.com/verify. It is not an accredited qualification or a vendor certification. How certification works.

The Wirewalk Institute of AI is independent. This course teaches publicly documented products and is not affiliated with or endorsed by LiteLLM, Kong, Envoy, Microsoft, Amazon Web Services, Cloudflare, Portkey, Palo Alto Networks, or OpenRouter. Product and company names are trademarks of their respective owners.

More in Building AI Applications

All 14 courses in this track
bld-101 Basic Waitlist

No-Code AI Assistants: Custom GPTs, Projects, Gems and Copilot Studio

A hands-on course on building no-code assistants with the builders already in your workplace tools: ChatGPT projects, custom GPTs and their successor plugins, Claude Projects and Skills, Gemini Gems, and Microsoft 365 Copilot Agent Builder and Copilot Studio. You leave with a tested, documented assistant and a plan to share it safely.

7 h · 17 lessons Join the waitlist
bld-201 Intermediate Available

Building with LLM APIs Across Vendors

A hands-on, code-first course on the three major LLM APIs side by side: messages, streaming, structured outputs, tool calling, vision, cost, caching, batch, retries, abstraction layers, cloud routes and testing. You finish by building and hardening a small multi-vendor service.

14 h · 18 lessons See price at checkout
bld-210 Intermediate Available

Retrieval-Augmented Generation Systems

A practitioner's course on retrieval-augmented generation: when to use it, how to ingest, chunk, embed and index documents, how to combine BM25 and vector search with reranking, and how to measure, debug and operate the result. Every module includes working Python sketches.

14 h · 16 lessons See price at checkout