Wirewalk Instituteof AI
dev-302 Expert Waitlist

C and AI: Embedding Inference with llama.cpp and the ONNX Runtime C API

For C and embedded systems engineers who need to run models inside native programs, firmware-adjacent services and constrained hosts, with predictable memory and no Python at runtime.

The curriculum is being prepared and reviewed. Enrollment is not open yet.

About this course

C remains the lingua franca of native integration. Daemons, appliances, plugins and embedded Linux products often cannot take a Python runtime, a container or a network dependency, yet they increasingly need speech, vision or language features. The two most practical routes are llama.cpp (and its ggml tensor library) for LLM and speech workloads and ONNX Runtime's C API for everything that can be exported to ONNX.

This course works at the API level. You build llama.cpp as a library with the right backends, then drive the libllama C API directly: loading GGUF models, creating contexts, tokenizing, batching with llama_batch, managing the KV cache and sampling with sampler chains, plus whisper.cpp for speech. You run ONNX models through the OrtApi function table: environments, sessions, tensors, allocators, execution providers and error-status handling. You then go into memory and performance: memory-mapping weights, arena and custom allocators, threading models, NUMA and cache effects, quantization formats, and calling CUDA from C. The final module covers what makes native AI code shippable: an ABI-stable wrapper library, sanitizers, fuzzing GGUF and ONNX parsers, defending against malicious model files, and build and packaging practice. The llama.cpp C API evolves frequently; function names are taught as of October 2026 together with a method for tracking header changes.

What you will be able to do

  • Build llama.cpp and ONNX Runtime as libraries with the correct CPU and GPU backends and link them into a C program with a reproducible build
  • Implement a complete text-generation loop on the libllama C API, including batching, KV-cache management, sampler chains and correct resource teardown
  • Run ONNX models through the ONNX Runtime C API with explicit allocators, execution providers and status-checked error handling
  • Control memory and threading for native inference: mmap'd weights, arenas, thread pools and NUMA placement, with measured footprints
  • Call CUDA from C to manage device memory and streams around an inference library without leaks or implicit synchronizations
  • Harden native inference code with sanitizers, fuzzing of model-file parsers and treatment of model files as untrusted input
  • Design an ABI-stable C wrapper that other languages can bind to safely

Syllabus

5 modules · 18 lessons · about 20 hours

Module 1: Native Inference Architecture in C

Where in-process inference fits and how the two main libraries are structured.
3 lessons
  1. In-Process vs Sidecar vs Remote: Choosing an Integration Shape for a C Program
  2. llama.cpp and ggml Internals: Backends, Graphs and the GGUF Format
  3. Building and Linking: CMake Options, Backends and Reproducible Builds

Module 2: The llama.cpp C API in Depth

Driving LLM inference directly from C.
4 lessons
  1. Models and Contexts: Loading GGUF, Context Parameters and Memory Mapping
  2. Tokenize, Batch, Decode: Writing the Generation Loop with llama_batch
  3. Samplers, Grammars and Structured Output
  4. KV-Cache Management, Multi-Sequence Batching and whisper.cpp for Speech

Module 3: The ONNX Runtime C API

Running exported models through the OrtApi function table.
3 lessons
  1. OrtApi Basics: Environment, Session Options, Sessions and Status Handling
  2. Tensors, Allocators and I/O Binding Without Extra Copies
  3. Execution Providers from C: CPU, CUDA, TensorRT and OpenVINO

Module 4: Memory, Threads and Performance

Owning the footprint and the latency.
4 lessons
  1. Memory Budgets: mmap, Arenas, Custom Allocators and Measuring RSS
  2. Threading Models, Affinity and NUMA for CPU Inference
  3. Quantization Formats and Their Effect on Speed, Size and Accuracy
  4. CUDA from C: Device Memory, Streams and Avoiding Hidden Synchronization

Module 5: Shipping Native AI Code Safely

Security, testing and API design for C inference libraries.
4 lessons
  1. Model Files Are Untrusted Input: Parser Risks and Validation
  2. Sanitizers, Valgrind and Fuzzing with libFuzzer and AFL++
  3. Designing an ABI-Stable C Wrapper for Other Languages
  4. Tracking Upstream API Changes and Upgrading Without Regressions

Exam and certificate

The exam unlocks once you have completed every lesson. It draws 25 questions from a larger bank, spread across the modules, with the answer options shuffled for each attempt. You have 60 minutes and need 80% to pass. After a failed attempt you can try again after 24 hours, up to 3 attempts in total. Feedback is given per module, never as the answer key.

Passing earns a certificate of completion and assessment from the Wirewalk Institute of AI with a unique ID that anyone can check at institute.wirewalk.com/verify. It is not an accredited qualification or a vendor certification. How certification works.

The Wirewalk Institute of AI is independent. This course teaches publicly documented products and is not affiliated with or endorsed by ggml-org (llama.cpp, ggml, GGUF, whisper.cpp), Microsoft (ONNX Runtime C API), NVIDIA (CUDA runtime and driver APIs), LLVM and GCC (sanitizers, libFuzzer), AFL++, or Valgrind. Product and company names are trademarks of their respective owners.

More in AI in Software Development

All 19 courses in this track
dev-001 Free intro Available

Your First AI API Call: A One-Hour Hands-On Start

A free, one-hour, hands-on start built on one realistic job: sorting a property manager's 1,150 monthly maintenance requests. You get an API key the safe way, make real calls with curl and an official Python SDK, run them over forty work orders, read stop reasons, usage and errors, estimate the monthly cost, and decide whether the script is ready. Exercises run on your own provider account for well under a dollar.

1 h · 7 lessons Free intro
dev-201 Intermediate Available

Python AI Integration: SDKs, Async, Pydantic, Packaging and Testing

A code-first course on integrating LLMs into Python the way production code needs it: uv projects and secrets, the OpenAI, Anthropic and Google GenAI SDKs side by side, async concurrency and streaming, Pydantic structured outputs and tools, retries, local models, embeddings and RAG, FastAPI and pandas integration, testing and evals with pytest, observability, cost tracking and deployment.

15 h · 19 lessons See price at checkout
dev-202 Intermediate Waitlist

Perl and AI: Modernizing Scripts and Text Pipelines with LLM APIs

Call OpenAI, Anthropic, Gemini and local Ollama models from Perl with real CPAN modules, add streaming, concurrency and validated JSON output to text pipelines, and use AI coding assistants to modernize legacy Perl behind characterization tests.

12 h · 18 lessons Join the waitlist