srv-201
Intermediate
Waitlist
The working practices that make machine learning reproducible and operable: versioning data, code and models together, tracking experiments, building pipelines, gating releases with CI/CD, and watching models for drift after they ship. Covers both classical ML and the newer LLM-application lifecycle.
srv-210
Intermediate
Waitlist
A practical path from a model file to a production service: designing the inference API, packaging it in a lean GPU-ready container, running it on Kubernetes with KServe, autoscaling it on the right signal, and rolling out new versions safely. Focused on general ML models; LLM-specific engines are covered in srv-310.
srv-310
Expert
Available
The engineering of LLM inference from first principles to production: why decode is memory-bound, how to size the KV cache, how continuous batching, PagedAttention, prefix caching, quantization and speculative decoding change the economics, and how to run vLLM, SGLang, TensorRT-LLM and Dynamo at scale. Ends with load testing, Kubernetes autoscaling and a defensible cost per million tokens.
srv-320
Expert
Waitlist
The numerics and engineering of making models cheaper to run: number formats from FP16 to FP4, post-training quantization algorithms such as GPTQ, AWQ and SmoothQuant, quantization-aware training, KV cache quantization, and the compiler and kernel work (torch.compile, TensorRT, fused attention, CUDA graphs) that turns smaller numbers into real speed. Every technique is paired with a way to measure what it costs in quality.
srv-330
Expert
Waitlist
How to instrument, monitor and debug LLM systems end to end: distributed traces across prompts, retrieval, tool calls and agent steps; token, latency and cost metrics; online evaluation of live traffic; and the alerting, privacy controls and incident practice that make it operable. Built on OpenTelemetry so the skills transfer across vendors.
srv-340
Expert
Waitlist
The software side of edge AI: deciding what should run on the device, fitting models into tight memory and power budgets, converting them for ExecuTorch, LiteRT, Core ML, ONNX Runtime and OpenVINO, targeting NPUs and GPUs, running small language models locally, and operating a fleet of devices with secure model updates. Hardware selection is covered in hw-206.