Somewhere in your organization, right now, a product manager is pasting a customer contract into a public chatbot to summarize it. A developer is sending a stack trace with an internal API endpoint to an AI assistant that lives outside your VPN. A finance analyst has installed a browser extension that quietly forwards spreadsheet cells to a model you have never heard of.
None of these people are trying to leak data. They are trying to do their jobs faster. The AI tools they picked up were free, worked well, and did not require an approval process. That is the entire problem.
Every enterprise we work with is dealing with this, and most of them are underestimating how much of it is happening. When we run discovery on a new customer environment, we typically find between four and twelve AI services in active use that the security team was not aware of. In one recent engagement it was twenty three.
Blocking is not the answer. Neither is pretending it is not happening.
What shadow AI actually looks like
Shadow AI is not one thing. In practice we see four distinct patterns, and each one requires a different response.
Direct chatbot use. Employees signing into ChatGPT, Claude, Gemini, or Copilot with personal accounts and pasting work data in. This is the most visible category and the easiest to catch at the network layer, because the destinations are well known.
Browser extensions and IDE plugins. Grammarly, Notion AI, Cursor, Windsurf, Copilot, and dozens of smaller tools that operate silently in the background. Some of them send entire documents or codebases to model providers on every keystroke. Most users have no idea what leaves their machine when they enable one.
SaaS features that quietly added AI. Your existing tools flipped on AI features in the last eighteen months without anyone re-reviewing them. Slack, Zoom, Notion, HubSpot, Salesforce, and Atlassian all route data through model providers now. Some let you disable it, some don't, and the terms of service around what happens to the data vary wildly.
Agents and API integrations built by developers. This is the fastest growing category. Someone on the engineering team wired up an internal tool to call OpenAI's API with a personal key, and it is now processing thousands of requests a day. Nobody wrote a design doc. Nobody threat modeled it. The API key is sitting in a .env file on someone's laptop.
Each of these needs a different control point. That is why "just block OpenAI" is such a bad first move. You will miss most of the actual exposure and push the rest deeper underground.
Why traditional DLP and firewalls miss it
Most enterprises already run DLP, a next-gen firewall, and a CASB. All three struggle with AI traffic for the same reason: the sensitive part of the request is inside an HTTPS POST body going to a legitimate SaaS destination.
Your firewall sees a TLS connection to api.openai.com. That is a well-reputed domain. It matches no threat feed. The category is "artificial intelligence", which your policy probably allows because half the company is using it productively.
Your DLP inspects file uploads and email attachments. It does not read prompt payloads inside an API call to a model provider, because those calls do not look like uploads. They look like normal API traffic.
Your CASB knows about the sanctioned apps you told it about. It does not know that the marketing team just started using a new AI tool last week.
The gap is not that these tools are bad. The gap is that they were built for a world where sensitive data left the network as a file, an email, or a database query. AI changed the shape of the leak. The leak is now a JSON object with a messages field, and it looks identical to every other API call in your environment.
What an AI-aware control plane actually inspects
This is where AI firewall and AI-specific DLP come in. The distinction that matters is what layer of the request they operate on.
An AI firewall inspects the prompt payload itself. Not the destination, not the TLS metadata, the actual content of what the model is being asked. It classifies the request against policy in real time. Is there a credit card number in this prompt? A customer record? A block of source code from a private repository? An unreleased financial figure? If yes, and policy says no, the request is blocked or redacted before it leaves the network.
DLP for AI extends the same idea to responses. If a model returns something sensitive, either because it was trained on data it should not have been, or because it is echoing back something an internal system fed it, that also gets classified. This matters more than most teams think. We have seen models return API keys that were pasted into earlier prompts by the same user.
Shadow AI discovery is the third piece. It is a passive traffic analysis layer that surfaces every AI service in active use across the environment, whether or not you knew about it. It works by identifying the traffic patterns AI APIs produce, which are distinctive enough to catch even novel services. Once you have the list, you can decide which ones to sanction, which ones to replace, and which ones to block.
None of these three replace your existing security stack. They sit alongside it, watching a class of traffic your existing tools were not designed to inspect.
Bringing shadow AI under policy without blocking legitimate work
The mistake we see most often is going straight from "we have no visibility" to "we are blocking everything until we finish a review". This kills productivity, generates a huge backlog of exception requests, and drives the remaining usage onto personal devices and unmanaged networks. You end up with less visibility than you started with.
A better sequence looks like this.
Discover first. Run traffic analysis for two to four weeks with no enforcement. Build the list. Map who is using what, how often, and for what kind of workload. You will find the picture is more nuanced than you expected. Some of the shadow usage is people solving real problems the sanctioned tools do not solve.
Categorize by data sensitivity, not by tool. A public AI tool being used to rewrite a marketing headline is not the same risk as the same tool being used to summarize a customer contract. Your policy should key on the data class, not the destination. This is what a real AI DLP layer lets you do.
Sanction good alternatives before you block bad ones. If people are pasting code into public models, they are doing it because it works. Give them a sanctioned code assistant that runs against an approved model, with prompts logged and controlled. Then you can turn off the shadow tool without creating a productivity fight.
Log everything for the first ninety days. Even after enforcement is on, keep detailed prompt and response logging in a separate store your security team owns. When an incident happens, and one will, this is what you need to actually understand what data went where.
Where the network comes in
The fastest way to make shadow AI go away is to give people a private inference environment that is as easy to use as the public one. That is an infrastructure problem.
We build these for customers who cannot send certain data classes to public model providers, either because of regulatory constraints, contractual obligations, or a straight business decision that their data is not going anywhere they do not control. The architecture is usually a dedicated GPU fabric, VXLAN isolated from the rest of the environment, with high-speed interconnects for the model backends and standard TLS-terminating ingress for the applications. Latency is sub-100ms for most inference workloads. Cost, at steady state, is competitive with equivalent public cloud AI once utilization gets above forty percent.
The security posture is the important part. Traffic never leaves the environment. There is no prompt payload going to a third party. Your AI DLP layer still runs, but the risk profile is different because the whole conversation is contained. For teams working with regulated data, or with source code they cannot expose, this is the release valve that makes strong AI usage policies workable.
Not everyone needs this. If your AI use cases are non-sensitive and the public models are meeting the need, spending on private inference is overkill. The point is that for the class of workload where public AI is not acceptable, the answer is a real network and a real cluster, not a policy document telling people not to use AI.
The one thing most teams get wrong
The biggest mistake we see is treating shadow AI as a compliance problem to be solved with a policy update. Policies do not stop people from using tools that work. They just push the usage into channels you cannot see.
What actually reduces exposure is a combination of three things running at the same time. Real inspection of the prompt payload, so you can see what is actually going out. A sanctioned alternative for every category of legitimate use, so people do not have to pick between doing their job and following the rules. And an internal AI infrastructure for the workloads where public AI is not an option.
If you get all three right, shadow AI does not disappear. It becomes visible AI, running against controls you defined, on infrastructure you operate. That is a much better position to be in.
If you are trying to figure out what AI tools are actually running in your environment, or you want to scope a private inference cluster for a workload that cannot go to public cloud, we run complimentary assessments for both. Our team has done this across finance, healthcare, and high-tech customers over the last two years, and the discovery findings are usually more surprising than the fixes.