Safe & Secure Agentic AI: Governance, Identity, and Red-Teaming Training Course

Foundations: Threat Modeling for Agentic AI

• Categorizing agentic threats: including misuse, privilege escalation, data leakage, and supply-chain vulnerabilities
• Profiling adversaries and understanding attacker capabilities specific to autonomous agents
• Mapping assets, defining trust boundaries, and identifying critical control points for agents

Governance, Policy, and Risk Management

• Establishing governance frameworks for agentic systems, covering roles, responsibilities, and approval gates
• Crafting policies for acceptable use, escalation rules, data handling, and auditability
• Navigating compliance requirements and gathering evidence for audits

Non-Human Identity and Authentication for Agents

• Defining agent identities using service accounts, JWTs, and short-lived credentials
• Implementing least-privilege access patterns and just-in-time credentialing
• Managing the identity lifecycle, including rotation, delegation, and revocation strategies

Access Controls, Secrets, and Data Protection

• Utilizing fine-grained access control models and capability-based patterns for agents
• Managing secrets, ensuring encryption in transit and at rest, and practicing data minimization
• Safeguarding sensitive knowledge sources and PII from unauthorized agent access

Observability, Auditing, and Incident Response

• Designing telemetry for agent behavior, including intent tracing, command logs, and provenance
• Integrating with SIEM, setting alerting thresholds, and ensuring forensic readiness
• Developing runbooks and playbooks for handling agent-related incidents and containment

Red-Teaming Agentic Systems

• Planning red-team exercises by defining scope, rules of engagement, and safe failover mechanisms
• Applying adversarial techniques such as prompt injection, tool misuse, chain-of-thought manipulation, and API abuse
• Executing controlled attacks to measure exposure and assess impact

Hardening and Mitigations

• Implementing engineering controls like response throttles, capability gating, and sandboxing
• Applying policy and orchestration controls, including approval flows, human-in-the-loop mechanisms, and governance hooks
• Deploying model and prompt-level defenses such as input validation, canonicalization, and output filters

Operationalizing Safe Agent Deployments

• Adopting deployment patterns such as staging, canary, and progressive rollouts for agents
• Enforcing change control, testing pipelines, and pre-deployment safety checks
• Fostering cross-functional governance with playbooks involving security, legal, product, and operations teams

Capstone: Red-Team vs Blue-Team Exercise

• Performing a simulated red-team attack against a sandboxed agent environment
• Acting as the blue team to defend, detect, and remediate using established controls and telemetry
• Presenting findings, outlining a remediation plan, and proposing policy updates

Summary and Next Steps