Sign in Explore the program

The AI Engineering Program: Module 9 of 13

Evals, Guardrails & AI Security

Measure whether an AI system is actually good, catch regressions before users do, and defend it against prompt injection and data leaks.

24guided hours
11lessons
Onlineself-paced
Enrol in the courseTry lesson 1 free

Lessons

Every lesson: deep notes, curated lectures, an animated explainer, a hands-on lab, practice questions and a timed exam.

  1. Why AI apps fail silently
  2. Error analysis on real traces
  3. Golden datasets and synthetic test data
  4. Code-based checks and assertions
  5. LLM-as-judge, calibrated against human labels
  6. Evals in CI: blocking a bad prompt change
  7. Tracing and observability with open-source tools
  8. OWASP Top 10 for LLM apps: prompt injection, data leakage, excessive agency
  9. Guardrails and personal-data redaction
  10. Red-teaming your own app
  11. Module project

Tools introduced: Eval suites, GitHub Actions

Module project

A published evaluation suite and red-team report for the Module 8 assistant, wired into CI so a regression fails the build.

PreviousModule 8: RAG EngineeringNextModule 10: Agents & MCP