Your AI Assistant Can Be Talked Into Leaking Your Data

The Threat Most Businesses Have Backwards

When a business owner asks me whether AI is "safe" to connect to their systems, they're almost always picturing the wrong threat. They imagine a hacker breaking into the model, or a competitor stealing a fine-tuned system. Real concerns, but not the ones that show up in the incident reports I read.

The failure that's actually happening right now is quieter and stranger. An AI assistant works exactly as designed. It reads a document or an email it was supposed to read. And it quietly does what an attacker hidden inside that content told it to do.

No malware. No stolen password. No firewall breached. Just text.

If you're planning to connect an AI model to your CRM, your inbox, your internal documents, or your customer records, this is the risk that matters most. It's also the one least understood by the people signing off on these projects. So let's walk through what's actually happening, what the real incidents have looked like, and how to think about defense without needing to become a security researcher.

Why "Chatbot" and "Agent" Are Completely Different Risk Profiles

For most of the last two years, business AI meant a chatbot. You typed a question, it answered. If it made something up, you got a wrong answer and moved on. The blast radius was small because the model couldn't do anything.

That era is closing. The AI systems businesses are deploying now read inboxes, write to databases, run queries, trigger workflows, call internal APIs, and increasingly move money. The assistant takes action on your behalf.

Here's the structural fact that changes everything: a system that acts on the text it reads can be manipulated by any text it reads.

A chatbot that gives a wrong answer wastes someone's time. An agent that can be steered by the content it processes is a different kind of exposure entirely, because the consequence isn't a bad answer. It's an action taken against you, using your own access.

The Lethal Trifecta

The clearest way to think about this comes from security researcher Simon Willison, who in mid-2025 named the specific combination that turns a theoretical vulnerability into actual data theft. He calls it the lethal trifecta: an AI system that has access to private data, is exposed to untrusted content, and can communicate externally.

Any one of those on its own is fine. Put all three together and you've built a data-exfiltration machine that an outsider can trigger by planting instructions in something your agent will eventually read.

The attack fits in three lines. An attacker slips an instruction into content the agent processes. The agent follows it. Private data leaves the building.

Now think about how often those three conditions quietly coincide in a real deployment. A support agent reads incoming customer emails, that's untrusted content. It looks up account details to help, that's private data access. It can send replies or call APIs, that's external communication. That single, sensible-sounding automation is the full trifecta. And it describes a large share of what businesses are rushing to build right now.

The reason this catches experienced teams off guard is that none of the three legs looks dangerous in the requirements meeting. Each one is just a feature someone asked for.

What It Looks Like When It Fails

This is not hypothetical, and vague warnings help no one make a decision, so let me be specific.

In June 2025, researchers at Aim Security disclosed a flaw in Microsoft 365 Copilot, later logged as CVE-2025-32711 with a severity score of 9.3 out of 10. They named it EchoLeak. The attack required zero clicks from the victim. Someone sent a single email containing hidden instructions. When Copilot ingested that email during routine summarization, it followed the buried instructions, pulled data from the victim's OneDrive, SharePoint, and Teams, and exfiltrated it through a trusted Microsoft domain. Antivirus and firewalls were irrelevant, because nothing about the attack was code. It was natural language. Microsoft patched it server-side, and there's no public evidence it was exploited in the wild before the fix. It stands as the first well-documented case of prompt injection causing real data theft from a production AI system.

The pattern keeps repeating with different products. Zenity Labs showed that Perplexity's Comet browsing agent could be hijacked by a malicious calendar invite carrying an injection payload, causing it to read local files and exfiltrate their contents once the target simply accepted what looked like a normal meeting invitation.

Then there's the supply-chain angle, which should worry anyone using off-the-shelf AI frameworks. In March 2026, a backdoored version of LiteLLM, a widely used gateway that sits in front of models for frameworks like CrewAI and Microsoft GraphRAG, sat on the PyPI package repository for roughly three hours. Nearly 47,000 downloads happened in that window. Anyone who updated during those three hours pulled an autonomous attack bot in alongside the legitimate package.

The through-line is uncomfortable. These aren't exotic attacks against Fortune 500 targets by nation-states. They exploit the ordinary way AI systems get built and connected. The barrier to entry is low precisely because the attack is written in plain language, not assembly.

Why Your Existing Security Tools Won't Catch This

Here's the part that trips up capable IT teams. Nearly every security control most businesses own, antivirus, firewalls, static code scanning, signature-based detection, was built to catch malicious code. Prompt injection contains no code. It's a plausible sentence embedded in a document, a web page, a support ticket, a PDF, or a code comment. To a scanner, it's just words.

This is why prompt injection now sits at the top of the OWASP Top 10 for LLM applications, and why the honest researchers keep repeating a line nobody wants to hear: there's no perfect fix. You can't fully train a model to always tell "instructions from my owner" apart from "instructions hidden in the content I'm reading," because to the model both arrive as text in the same stream. Some vendors have even declined to patch reported issues, arguing a fix would break functionality. That tells you how fundamental the problem is.

There's a confidence gap worth sitting with. One 2026 survey found 82% of executives believed their existing policies already protected them from unauthorized AI agent actions. A separate survey of a similar population found 88% had experienced a confirmed or suspected AI agent security incident in the past year. Both figures describe roughly the same world. The distance between "we're covered" and "we got hit" is exactly where the current risk lives.

What Actually Defends Against It

If there's no perfect fix, what do you do? You stop trying to make the model behave, and you change the structure around it. The defenses that hold up are architectural, not behavioral. Here's how I'd frame it for a business about to connect AI to something sensitive.

Break the trifecta on purpose. The single most effective move is making sure all three dangerous capabilities are never present in one agent at once. If an agent reads untrusted external content, it shouldn't also hold both sensitive data access and the ability to send data out. Split those jobs across separate systems with separate permissions. It's unglamorous, and it works, because it removes the structural condition instead of hoping the model resists manipulation.

Give each agent the least access that still lets it work. An agent that needs to read three fields from your CRM should not have write access to the whole database. When an injection eventually succeeds, and you should plan as if it will, the damage is bounded by what that agent was allowed to touch. Scope permissions per task, not per convenience.

Put a human in front of anything irreversible. High-consequence actions, moving money, sending external communications at scale, modifying systems, deleting records, should require explicit human approval. The 2025 incidents repeatedly showed auto-approval configurations were the thing attackers exploited. A confirmation step is cheap insurance against an expensive mistake. This is the same governance discipline that separates multi-agent systems that scale from the ones that fail, which I wrote about in [Multi-Agent AI Is Redefining How Enterprises Operate in 2026](https://nexhubai.com/insights/multi-agent-ai-enterprise-2026).

Watch behavior, not keywords. You can't signature-scan for prompt injection, but you can baseline what normal looks like and alert on deviation. A support agent that suddenly reads 5,000 customer records in one session, or makes an API call sequence it's never made before, is a far better signal than trying to spot "bad words" in inputs. Feed that telemetry into whatever monitoring you already run. This is the adversarial cousin of the drift monitoring I covered in [Why Your LLM Works in the Demo and Breaks in Production](https://nexhubai.com/insights/llm-production-failure-lessons): same instinct, watch the system continuously, applied to attacks instead of decay.

Know what you've actually deployed. Shadow AI, staff wiring up unsanctioned agents without review, is now one of the more expensive incident categories precisely because nobody's watching those systems. You can't defend an inventory you don't have. Before hardening anything, find out how many AI agents are already touching your data and who built them.

Where Google's Framework Fits, and Where It Doesn't

Google Cloud recently published a well-circulated piece from its CISO, Chris Betz, on the four lessons behind its AI Threat Defense product: prepare, scan and prioritize, remediate, and monitor. It's a genuinely useful framework and worth reading. But it's important to understand which problem it solves, because it isn't quite the one most businesses have.

Google's framework is about using AI to defend an enormous codebase against AI-accelerated attackers who can now find and weaponize vulnerabilities in hours instead of weeks. It's a vulnerability-management story at a scale almost no reader of this blog operates at, spanning Search, Ads, Android, Chrome, and tens of thousands of packages.

The one lesson that translates cleanly downward is the first one: prepare by reducing your attack surface before anything else, so you have less to defend. That principle maps directly onto breaking the trifecta and scoping permissions. It's the same idea at a different scale.

The rest of Google's toolkit, autonomous patching pipelines with rollback, multi-model scanning harnesses, continuously red-teaming core infrastructure, is what a hyperscaler does to protect billions of users. Borrow the mindset, which is continuous, assume-breach, measure everything. Don't mistake a hyperscaler's machinery for a small-business checklist. Your version of "monitor" is behavioral alerting on a handful of agents, not an agentic security operations center.

The Honest Bottom Line

AI is worth putting into your operations. The productivity gains are real, and retreating out of fear would be the wrong lesson to take from any of this. But there's a specific, verified class of risk that arrives the moment an AI system can both read untrusted content and act on your private data, and most businesses are deploying straight into it without knowing it exists.

The reassuring part is that the defense isn't mainly about buying a product. It's about how you architect the system. Which agent gets which permissions. What stays behind a human approval step. Whether the three dangerous capabilities ever sit together in one place. Get the structure right and you've handled most of the exposure. This is also why grounding an agent's knowledge in a controlled retrieval layer rather than open-ended access matters, a design point I get into in [Why RAG Is the Architecture Every Business AI Actually Needs](https://nexhubai.com/insights/rag-retrieval-augmented-generation-business-guide).

Get the structure wrong and no amount of downstream security tooling will save you, because the attack doesn't look like an attack. It looks like your assistant doing its job.

The businesses that adopt AI well over the next two years won't be the ones that moved fastest, or the ones that stayed scared. They'll be the ones that connected AI to their sensitive systems deliberately, with the trifecta broken by design.

Frequently Asked Questions

What is prompt injection, in plain terms?

It's when someone hides instructions inside content your AI system reads, a document, email, web page, or ticket, and the AI follows those hidden instructions as if they came from you. Because the model can't reliably separate your instructions from instructions buried in the content it's processing, it can be tricked into leaking data or taking actions you never authorized.

Is this a risk if we only use a public chatbot and don't connect it to our data?

Much less so. The serious risk shows up when the AI has private data access and the ability to act, such as sending emails, calling APIs, or writing to databases, and exposure to content from outside your organization. A standalone chatbot that only answers questions carries far less exposure than an agent wired into your systems.

Can't we just train the AI to ignore malicious instructions?

Not reliably, and this is the hard part. To the model, your instructions and the hidden ones arrive as the same kind of text. Training reduces the success rate but doesn't eliminate it, which is why researchers emphasize structural defenses, limiting access and separating capabilities, over trying to make the model perfectly obedient.

Will our firewall or antivirus catch this?

No. Those tools look for malicious code, and prompt injection is ordinary language, so it passes straight through. Defense requires behavioral monitoring to spot unusual agent activity, plus architectural controls that limit what any single agent can do, not traditional signature-based security.

What's the single most important thing to do before connecting AI to sensitive data?

Make sure no single AI agent simultaneously has access to private data, exposure to untrusted external content, and the ability to send information out of your organization. Breaking that combination across separate systems removes most of the exfiltration risk in one decision.