Wirewalk Instituteof AI
sec-202 Intermediate Waitlist

Threat Modeling and Secure Design for LLM Applications

For security architects, AppSec engineers and tech leads who need to threat model LLM features before they ship and turn the results into concrete design decisions.

The curriculum is being prepared and reviewed. Enrollment is not open yet.

About this course

Most LLM security failures are design failures: the model was given more data, more authority or more reach than any input-handling control could safely contain. This course teaches you to find those problems on a whiteboard, before code review. You will model chat, retrieval-augmented and agent architectures as data flow diagrams, mark trust boundaries that account for retrieved content and tool results, and enumerate threats using STRIDE alongside AI-specific catalogs: MITRE ATLAS (v2026.09 at review), the NIST AI 100-2 E2025 adversarial machine learning taxonomy and the OWASP Top 10 lists for LLM and agentic applications.

You will then learn how to rate likelihood and impact when outputs are non-deterministic, how to analyze blast radius, and how to choose between mitigating, accepting and transferring risk. The design module covers privilege separation patterns (dual-LLM, plan-then-execute, quarantined context), deterministic controls around the model, identity propagation, and where guardrail classifiers help. You will turn threat models into testable security requirements, align them to NIST SP 800-218A, the joint Guidelines for Secure AI System Development co-sealed by CISA, and the European standard ETSI EN 304 223, and work through two complete hypothetical threat models. This course goes deeper on design than sec-201, which surveys the OWASP risks one by one. The Wirewalk Institute certificate attests to completion and assessment of this course; it is not endorsed by OWASP, MITRE, NIST or any vendor named here.

What you will be able to do

  • Draw data flow diagrams for chat, RAG and agent features with trust boundaries that include retrieved content and tool output
  • Enumerate threats using STRIDE together with MITRE ATLAS, NIST AI 100-2 and the OWASP LLM and agentic lists
  • Rate risks for probabilistic components using blast-radius analysis rather than assumed model reliability
  • Select privilege separation and deterministic control patterns that limit damage when the model is manipulated
  • Write security requirements and acceptance tests that can be verified in CI
  • Align a threat model with NIST SP 800-218A, the CISA-co-sealed secure AI development guidelines and ETSI EN 304 223
  • Maintain a threat model as models, prompts, tools and data sources change after launch

Syllabus

6 modules · 19 lessons · about 12 hours

Module 1: Scoping an AI Threat Model

What changes when a model is in the loop and how to diagram it honestly.
3 lessons
  1. What Changes When a Model Is in the Loop
  2. Assets, Actors and Trust Boundaries for LLM Features
  3. Data Flow Diagrams for Chat, RAG and Agent Architectures

Module 2: Enumerating Threats

Combining classic STRIDE with AI-specific threat catalogs and abuse cases.
3 lessons
  1. STRIDE Applied to Models, Retrieval, Tools and Memory
  2. Using MITRE ATLAS, NIST AI 100-2 and the OWASP Lists as Threat Catalogs
  3. Abuse Cases and Misuse Stories for Generative Features

Module 3: Rating and Prioritizing

Rating risk when behavior is probabilistic and deciding what to fix first.
3 lessons
  1. Likelihood and Impact When Outputs Are Probabilistic
  2. Blast-Radius Analysis: What the Model Can Read, Write and Reach
  3. Mitigate, Accept or Transfer: Making and Recording the Call

Module 4: Secure Design Patterns

Architectures that stay safe even when the model follows an attacker's instructions.
4 lessons
  1. Privilege Separation: Dual-LLM, Plan-Then-Execute and Quarantined Context
  2. Deterministic Controls Around Non-Deterministic Components
  3. Identity Propagation and Authorization Outside the Model
  4. Guardrail Layers: Where Classifiers Help and Where They Do Not

Module 5: From Threat Model to Requirements

Testable requirements, standards alignment and review gates in the SDLC.
3 lessons
  1. Writing Security Requirements and Acceptance Tests for LLM Features
  2. Aligning with CISA/NCSC Guidelines, ETSI EN 304 223 and NIST SP 800-218A
  3. Design Review Gates and Threat Model Ownership

Module 6: Worked Threat Models

Two complete hypothetical threat models and how to keep them current.
3 lessons
  1. Worked Example: A Customer Support RAG Assistant
  2. Worked Example: A Coding Agent with Repository and CI Access
  3. Keeping the Threat Model Alive After Launch

Exam and certificate

The exam unlocks once you have completed every lesson. It draws 20 questions from a larger bank, spread across the modules, with the answer options shuffled for each attempt. You have 45 minutes and need 80% to pass. After a failed attempt you can try again after 24 hours, up to 3 attempts in total. Feedback is given per module, never as the answer key.

Passing earns a certificate of completion and assessment from the Wirewalk Institute of AI with a unique ID that anyone can check at institute.wirewalk.com/verify. It is not an accredited qualification or a vendor certification. How certification works.

The Wirewalk Institute of AI is independent. This course teaches publicly documented products and is not affiliated with or endorsed by OWASP, MITRE ATLAS, NIST, or ETSI. Product and company names are trademarks of their respective owners.

More in AI Security: Securing AI Systems

All 13 courses in this track
sec-001 Free intro Available

Using AI Safely at Work: The Essentials

Follow one fictional US distributor through four everyday AI mistakes (a guarantor's credit file uploaded to a personal account, a customer letter promising a fee waiver that does not exist, a vendor email with hidden instructions for the assistant, and a note-taker that sent a customer the internal margin floor) and practice the decisions that prevent or contain each one in three short Decision Labs.

1 h · 7 lessons Free intro
sec-101 Basic Available

Using AI Safely: Data Handling for Every Employee

A plain-language course on handling data safely with AI tools: what happens to what you type or upload, how to classify before you paste, how connected apps and agents change the picture, and how to report a mistake quickly.

6 h · 15 lessons See price at checkout
sec-201 Intermediate Available

AI Security Fundamentals: The OWASP Top 10 for LLMs

A practitioner course on securing applications that use large language models, organized around the 2026 OWASP Top 10 for LLM Applications. Every risk comes with a hypothetical scenario, the root cause, and concrete defenses in code and configuration.

14 h · 18 lessons See price at checkout