Wirewalk Instituteof AI
sec-360 Expert Waitlist

Detecting and Responding to Attacks on AI Systems

For detection engineers, incident responders and AI platform owners who must spot attacks on LLM applications, agents and models in production and respond when one succeeds.

The curriculum is being prepared and reviewed. Enrollment is not open yet.

About this course

Attacks on AI systems leave evidence, but usually not in the places a SOC already watches. An indirect prompt injection shows up as a retrieved document followed by an unusual tool call; model extraction looks like a patient, high-volume query pattern; a poisoned dataset may surface weeks later as a behavior change. This course teaches you to instrument AI systems, detect those patterns and respond. It is about defending AI systems; using AI to run security operations is covered in cyb-201.

You will map MITRE ATLAS techniques and the OWASP LLM and agentic risks to observable events, design telemetry for prompts, completions, retrieval, tool calls and infrastructure (including OpenTelemetry's generative AI semantic conventions), and balance forensic value against the privacy risk of logging conversations. You will write and test detections for injection and jailbreak attempts, anomalous agent tool use, scraping, extraction and runaway consumption. Playbooks cover injection-led exfiltration, compromised models, datasets or MCP servers, and agents taking unauthorized actions. You will reconstruct incidents from traces, handle non-deterministic reproduction, preserve model and log evidence, eradicate by rotating credentials and rolling back models, and handle notification duties such as the SEC's Form 8-K cybersecurity incident disclosure, HIPAA and state breach notification, and, for systems placed on the EU market, serious incident reporting under the EU AI Act. The course ends with incident sharing, tabletop exercises and maturity metrics. Regulatory content is educational and is not legal advice. The Wirewalk Institute certificate attests to completion and assessment of this course; it is not endorsed by OWASP, MITRE, NIST or any vendor named here.

What you will be able to do

  • Map MITRE ATLAS techniques and OWASP LLM and agentic risks to telemetry an AI system can emit
  • Design logging for prompts, retrieval, tool calls and infrastructure that supports forensics while limiting privacy exposure
  • Write and test detections for prompt injection, agent misbehavior, extraction and abuse
  • Execute playbooks for injection-led exfiltration, compromised AI artifacts and unauthorized agent actions
  • Reconstruct an AI incident from traces and preserve model and log evidence
  • Plan eradication, recovery and notification, including AI-specific reporting duties
  • Run tabletop exercises and measure AI detection and response maturity

Syllabus

6 modules · 20 lessons · about 18 hours

Module 1: What to Detect and Where to Look

From AI attack techniques to the telemetry that reveals them.
3 lessons
  1. Mapping ATLAS Techniques and OWASP Risks to Observable Events
  2. Telemetry Sources: Prompts, Completions, Retrieval, Tool Calls and Infrastructure
  3. Logging Design That Balances Forensics and Privacy

Module 2: Detection Engineering for AI Systems

Detections for the attacks AI systems actually face.
4 lessons
  1. Detecting Prompt Injection and Jailbreak Attempts in Production
  2. Detecting Agent Misbehavior: Anomalous Tool Use and Data Access
  3. Detecting Scraping, Model Extraction and Unbounded Consumption
  4. Writing and Testing Detections Against AI Telemetry

Module 3: Incident Response Playbooks

Preparation and playbooks for the most likely AI incidents.
4 lessons
  1. Preparation: AI Asset Inventory, Owners and Containment Levers
  2. Playbook: Prompt Injection Leading to Data Exfiltration
  3. Playbook: Compromised Model, Dataset or MCP Server
  4. Playbook: An Agent Taking Unauthorized Actions

Module 4: Investigation and Forensics

Reconstructing what happened when the system is non-deterministic.
3 lessons
  1. Reconstructing an AI Incident from Traces
  2. Non-Determinism and Reproduction: Replaying Model Behavior
  3. Evidence Handling for Model Artifacts and Conversation Logs

Module 5: Eradication, Recovery and Disclosure

Cleaning up, restoring service and meeting reporting duties.
3 lessons
  1. Eradication: Rotating Credentials, Purging Poisoned Data and Rolling Back Models
  2. Notification, Regulatory Reporting and Coordinated Disclosure
  3. Post-Incident Review and Updating the Threat Model

Module 6: Building the Capability

Learning from others, practicing and measuring.
3 lessons
  1. AI Incident Sharing: ATLAS Case Studies, CVEs and Incident Databases
  2. Tabletop Exercises for AI Incidents
  3. Metrics and Maturity for AI Detection and Response

Exam and certificate

The exam unlocks once you have completed every lesson. It draws 25 questions from a larger bank, spread across the modules, with the answer options shuffled for each attempt. You have 60 minutes and need 80% to pass. After a failed attempt you can try again after 24 hours, up to 3 attempts in total. Feedback is given per module, never as the answer key.

Passing earns a certificate of completion and assessment from the Wirewalk Institute of AI with a unique ID that anyone can check at institute.wirewalk.com/verify. It is not an accredited qualification or a vendor certification. How certification works.

The Wirewalk Institute of AI is independent. This course teaches publicly documented products and is not affiliated with or endorsed by MITRE ATLAS, OWASP, NIST, or OpenTelemetry. Product and company names are trademarks of their respective owners.

More in AI Security: Securing AI Systems

All 13 courses in this track
sec-001 Free intro Available

Using AI Safely at Work: The Essentials

Follow one fictional US distributor through four everyday AI mistakes (a guarantor's credit file uploaded to a personal account, a customer letter promising a fee waiver that does not exist, a vendor email with hidden instructions for the assistant, and a note-taker that sent a customer the internal margin floor) and practice the decisions that prevent or contain each one in three short Decision Labs.

1 h · 7 lessons Free intro
sec-101 Basic Available

Using AI Safely: Data Handling for Every Employee

A plain-language course on handling data safely with AI tools: what happens to what you type or upload, how to classify before you paste, how connected apps and agents change the picture, and how to report a mistake quickly.

6 h · 15 lessons See price at checkout
sec-201 Intermediate Available

AI Security Fundamentals: The OWASP Top 10 for LLMs

A practitioner course on securing applications that use large language models, organized around the 2026 OWASP Top 10 for LLM Applications. Every risk comes with a hypothetical scenario, the root cause, and concrete defenses in code and configuration.

14 h · 18 lessons See price at checkout