Agentic AI security testing is an end-to-end assessment of what an AI agent can see, decide and do. It tests the model, planner, tools, identity, retrieval data, memory, approval gates and downstream systems as one attack path. The goal is not merely to make the model say something unsafe; it is to prove whether an attacker can steer the agent into an unauthorized action, data disclosure or persistent compromise.
Why agentic AI needs a different security test
A conventional web application receives a request, runs predictable code and returns a response. An agent can interpret an ambiguous goal, create a plan, select a tool, retrieve external content, call another service, save memory and continue acting. Each step may be individually permitted while the combined chain produces an outcome no single control anticipated.
- Tests routes, parameters, sessions and business logic
- Assumes mostly deterministic application behaviour
- Validates authorization at known endpoints
- Finds vulnerabilities within a defined technical boundary
- Tests goals, plans, context, tools, memory and handoffs
- Challenges non-deterministic action sequences
- Rechecks authorization at every consequential action
- Follows trust across models, data sources and downstream systems
A web, API or cloud pentest is still necessary because agents sit on ordinary infrastructure. But it does not answer the agent-specific question: can untrusted content change the system's plan and turn legitimate capabilities into an attack chain? That requires adversarial testing at the level where the model chooses and combines actions.
Map the complete agent attack surface first
Before testing prompts, build an action-oriented system map. Record every source of instructions, every identity the agent can use, every tool it can invoke, every store it can read or change, and every point where a human is expected to approve an action. Missing one hidden integration can invalidate the rest of the assessment.
| Layer | What to inventory | Primary security question |
|---|---|---|
| User and session | Users, tenants, roles, delegated identities | Whose authority is the agent actually using? |
| Model and planner | System instructions, routing, planning and retry logic | Can untrusted content override the intended goal? |
| Tools and functions | APIs, plugins, MCP servers, code execution and automation | Can a permitted tool be abused outside its intended purpose? |
| Knowledge and RAG | Documents, websites, vector stores and connectors | Can poisoned content become an instruction or leak another user's data? |
| Memory | Conversation, long-term, profile and task memory | Can an attacker persist instructions or corrupt future decisions? |
| Approval gates | Human review, confirmation screens and policy checks | Does the reviewer see the real action, target and impact? |
| Downstream systems | SaaS, cloud, databases, email, source control and endpoints | What is the maximum blast radius of one agent identity? |
| Operations | Logs, budgets, rate limits, alerts, pause and revoke controls | Can operators detect, explain and stop harmful behaviour? |
Minimum attack-surface inventory for an AI agent
The 10-point agentic AI security testing checklist
1. Direct and indirect prompt injection
Test hostile instructions entered directly by a user and instructions hidden in content the agent retrieves: documents, support tickets, web pages, emails, code comments and tool responses. The test passes only when untrusted data remains data. Refusing one obvious jailbreak is not enough; assess whether the same payload can influence tool selection, arguments, memory or later steps in the plan.
2. Goal hijacking and instruction priority
Challenge conflicts between system policy, developer instructions, user goals, retrieved text and tool output. Try to make the agent reinterpret a safe objective into a dangerous subtask. Controls should be tied to actions and policy, not to the model remembering which sentence had the highest priority.
3. Tool misuse and function-call abuse
Fuzz tool names, parameters, encodings and multi-step combinations. Look for command injection, server-side request forgery, path traversal, unsafe file operations and business-logic abuse behind a valid function call. The model is not a security boundary: every tool must validate its input and authorization independently. Use the dedicated MCP server security guide when MCP is part of the tool layer.
4. Identity, tenant isolation and confused-deputy paths
Verify that the requesting user's identity and tenant context survive every handoff. An agent with a shared service credential can become a confused deputy: the user is allowed to ask, the agent is allowed to act, but the user should not inherit the agent's full authority. Test cross-tenant object references, cached context, delegated tokens, background jobs and tools that trust agent-supplied identity fields.
5. Excessive agency and privilege
Calculate the maximum damage available to one compromised session. Can the agent approve payments, change access, deploy code, delete data or send messages without a second control? Replace broad credentials with task-scoped, time-limited grants. Require a fresh authorization check for every consequential action, even when an earlier planning step was approved.
6. Secret and sensitive-data exfiltration
Seed the test environment with synthetic markers and determine whether the agent can expose them through chat, logs, tool arguments, URLs, generated files or outbound messages. Include system prompts, connection strings, retrieved records and other users' context. Egress controls should prevent an agent from choosing an unapproved destination simply because the model considers it useful.
7. Memory poisoning and persistence
Test whether a low-trust interaction can plant a durable instruction, preference or false fact that changes future sessions. Verify memory provenance, tenant boundaries, expiry, review and deletion. A security control that blocks the first attempt but stores the attack for later is not a successful control.
8. Tool and model supply-chain manipulation
Review who can change system prompts, tool descriptions, packages, model versions, adapters, connectors and retrieval sources. Test whether a newly added tool can impersonate another tool or influence the planner through its metadata. Releases should be pinned, reviewed and reversible, with an inventory that links every component to an owner and update path.
9. Multi-agent handoffs and trust propagation
When one agent delegates to another, validate the task, data, identity and permission passed across the boundary. A specialist agent must not treat another agent's output as trusted instructions. Test loops, role confusion, spoofed agent identity and a chain in which several low-risk actions combine into a high-impact result.
10. Failure recovery, cost abuse and the kill switch
Force timeouts, malformed tool responses, partial failures and contradictory state. Check whether retries repeat a payment, message or deletion. Test unbounded loops and resource consumption. Operators need hard limits for steps, time, tokens, spend and downstream actions, plus a tested way to pause execution, revoke credentials and preserve an auditable record.
How a useful assessment should run
| Phase | Work performed | Output |
|---|---|---|
| 1. Discovery | Inventory agents, models, prompts, tools, identities, data, memory and environments | System and data-flow map with confirmed scope |
| 2. Threat modelling | Define protected actions, trust boundaries, abuse cases and maximum credible impact | Prioritized attack hypotheses and test plan |
| 3. Adversarial testing | Exercise prompt, tool, identity, data, memory and multi-agent attack chains | Reproducible evidence with affected actions and conditions |
| 4. Control validation | Test prevention, detection, approval, rate limiting, rollback and shutdown | Control-gap matrix tied to business impact |
| 5. Remediation and retest | Review fixes and replay the original chains plus bypass variants | Verified closure record and residual-risk statement |
A practical engagement sequence
Run destructive scenarios in a production-representative test environment using synthetic records and non-production credentials. The important word is representative: a harmless demo agent with no real tools cannot prove that production authorization, approval or monitoring will hold. Where production validation is necessary, agree safe actions, rate limits and rollback conditions before testing begins.
Evidence a buyer should require
A list of jailbreak prompts is not an assessment report. Procurement and engineering teams should ask for evidence that connects a hostile input to an unauthorized outcome, shows the exact control failure, and gives the product team a fix it can verify.
- An architecture and trust-boundary map covering models, data, tools, identities and downstream systems.
- An action inventory that labels read, write, execute, communicate, approve and delete capabilities.
- Reproducible attack chains with prerequisites, prompts or inputs, tool calls and observable impact.
- Severity based on business impact and required attacker access, not on model behaviour alone.
- Developer-ready remediation for code, authorization, prompts, tool schemas and operational controls.
- Coverage notes that state what was not tested and why.
- Retest evidence demonstrating that the original exploit and reasonable variants no longer work.
When should you commission an agentic AI security assessment?
Test before the first production release if the agent can access confidential data or change another system. Repeat the assessment after a material change to its model, system instructions, identity design, tool set, memory, retrieval sources or approval flow. A new high-privilege tool can change the risk more than a complete user-interface redesign.
- The agent can send, publish, purchase, approve, deploy, modify or delete.
- It uses shared or service-level credentials to act for multiple users.
- It reads untrusted external content before choosing an action.
- It stores long-term memory or shares context across users or agents.
- It connects to internal APIs, source control, cloud consoles, support systems or financial workflows.
- A regulator, customer or insurer expects independent security evidence before launch.
How this checklist maps to current guidance
Use this testing plan alongside the OWASP Top 10 for Agentic Applications for 2026, the adversary techniques catalogued in MITRE ATLAS, and the governance lifecycle in the NIST AI Risk Management Framework. Together they provide a useful risk taxonomy, attack-language reference and governance frame. The assessment still needs to translate them into the specific actions, identities and data flows of your system.
The decision rule
If an AI feature only drafts text for a user to review, start with model and data-risk testing. If it can choose a tool or cause a side effect, test it as an agent. The security boundary is not the chat window; it is the furthest system the agent can influence with the strongest identity available to it.
Get an independent assessment of your agent's prompts, tools, identity, data, memory, approval gates and downstream impact, with reproducible findings and a remediation retest.