AI agent security

AI agent security is the practice of keeping an AI agent from being turned against the humans and systems it works for. The main risks are untrusted input, too much authority, data leaving and unclear identity. The defense is layered: screen what the agent reads, limit what it can do, and check what it sends.

Updated

Why is securing an AI agent different?

Securing an AI agent is different because the agent decides what to do from text, and some of that text comes from strangers. A classic program follows code. An agent follows language, and it cannot reliably tell its operator's instructions from instructions inside the content it reads.

A chat model that is fooled gives a wrong answer. An agent that is fooled takes a wrong action: it sends, pays, deletes or shares. The risk grows with what the agent can reach.

Gartner predicted in June 2025 that 33% of enterprise software applications will include agentic AI by 2028, up from less than 1% in 2024. The controls have to arrive with the agents.

What are the main risks?

The main risks when an agent acts on your behalf fall into four groups.

RiskWhat it meansHow it shows up in an inbox
Untrusted inputContent the agent reads can carry instructionsA supplier email with a hidden line telling the agent to forward invoices
Excessive agencyThe agent has more tools, permissions or autonomy than the job needsA summarizing assistant whose connector can also send and delete
Data leavingThe agent is steered to send private data outA reply to a stranger that includes a contract or an API key
IdentityIt is unclear who the agent is, whose authority it uses and who answers for itAn agent that sends from a human's own address with the human's full access

Untrusted input

Untrusted input is the root risk. Prompt injection is LLM01, the top risk, in the OWASP Top 10 for LLM Applications 2026.

NIST calls the agent form agent hijacking: instructions planted in data the agent reads push it toward harmful actions nobody asked for.

Excessive agency

Excessive agency turns a fooled model into real damage. OWASP names three causes: excessive functionality, excessive permissions and excessive autonomy. It is LLM03 in the 2026 list, up from LLM06 in the 2025 list.

Data leaving

Data leaves through whatever the agent can produce: a reply, a link, an image address or a call to an outside tool. See LLM data exfiltration.

Identity

Identity is about authority and accountability. An agent that borrows a human's mailbox and credentials acts with that human's full reach, and its actions are hard to tell apart in a log.

An agent with its own address, its own key and a named human responsible for it can be limited, watched and switched off. See agent identity and email.

When is an agent most exposed?

An agent is most exposed when three conditions hold in one session: it processes input from untrusted sources, it can reach private data or sensitive systems, and it can change state or communicate with the outside.

Meta's security team published this as a design rule in October 2025: allow at most two of the three in one session, and require supervision, such as human approval, when a task needs all three.

An email agent meets all three by default. Mail comes from anyone, the mailbox holds private data, and a reply is outside communication. That is why controls at the inbox carry so much weight.

What does a layered defense look like?

A layered defense assumes each layer will sometimes fail and puts another one behind it. The order below follows the path of an attack.

  1. Screen input before the agent reads it. Stop what can be stopped at the entry. Remove text a human could not see.
  2. Separate content from instructions. Mark outside content as data. This lowers the odds of a successful injection. It does not remove them.
  3. Apply least privilege. Give each agent the fewest tools, the narrowest data and the lowest permission that do the job.
  4. Give the agent its own identity. Its own address and key, limited to its own inbox, with a human responsible.
  5. Require human approval for risky actions. Payments, new recipients and releases of held content wait for a human. See human in the loop email.
  6. Screen output. Check what the agent sends for secrets, sensitive data and unfamiliar recipients.
  7. Keep the agent out of its own controls. An agent must not be able to loosen its protection, approve its own actions or create a stronger key.
  8. Log and test. Record what the agent read and did. Run attack drills on a schedule.

The UK National Cyber Security Centre gives the same advice from another angle: assume the model can be confused, and limit what the system can do with controls that do not depend on a model.

Why is the inbox the widest door?

The inbox is the widest door because it is the one input anyone can reach, at any time, without permission.

A web page must be visited. A file must be shared. An email is pushed to the agent by whoever knows the address.

The inbox is also where value sits: invoices, contracts, sign-in codes and password resets. And it is where an agent's actions leave, as replies and forwards. Entry, prize and exit are in one place.

EchoLeak, CVE-2025-32711, showed the result. Researchers showed that one crafted email could make Microsoft 365 Copilot disclose data with no click.

Microsoft rated it 9.3 (critical) and published its advisory in June 2025, saying the flaw was already mitigated. See zero-click AI attacks.

Where does Trotta Mail fit?

Trotta Mail is a pre-delivery inbox for AI agents and humans. It covers the layers that sit at the mailbox.

  • Input. Each message is screened before delivery, including the text of attached documents. Attacks, known and new, are refused at the door or held outside the inbox for a human.
  • What the agent reads. An agent's key gets the safe view: content a human could see, with instructions aimed at a model replaced by a note.
  • Identity. An agent gets its own inbox and key. Sign-in codes and reset links in a human's inbox are sealed from every API key.
  • Authority. An agent's key cannot read or release held mail, or loosen its own protection. A human can.
  • Output. Outgoing mail is screened for secrets and for replies steered to an address the inbox has never written to.

The full rules are in the security model.

No filter catches every injection. A reworded request with no technical tell can still be delivered. Trotta Mail does not control your agent's other tools, so the layers inside your agent still apply.

Questions

What is AI agent security?

AI agent security is the set of controls that keep an agent from being manipulated into harmful actions. It covers what the agent reads, what it is allowed to do, what it can send out and whose authority it uses.

What is the biggest security risk for AI agents?

Prompt injection through untrusted input. OWASP's 2026 list ranks it LLM01, the top risk for LLM applications, and for an agent with tools it becomes a way to trigger real actions.

How do you secure an AI agent that reads email?

Screen mail before the agent reads it, and give the agent its own inbox with a key limited to it. Keep sign-in codes out of its reach, require human approval for risky sends and screen what it sends.

Can a system prompt keep an agent safe?

Not on its own. A system prompt is text in the same stream an attacker can write into, so its rules can be argued away. Limits that matter are enforced outside the model.

What is the difference between LLM security and agent security?

LLM security is about what a model says. Agent security adds what a model does: tools, permissions, memory and actions with consequences. OWASP publishes a separate Top 10 for Agentic Applications for that reason.

Sources

Give your agent an inbox of its own. Free for two inboxes. No card.