Where your data goes when you use AI
A prompt is data. The moment it is sent, whatever was in it has left the building and the ordinary questions apply. Where did it travel. Who holds a copy. For how long. Who can read it.
The answers have almost nothing to do with the model and almost everything to do with the surface: the place in the working day where the AI actually sits. 4 of them turn up inside a normal company, and their defaults are not the same.
What is actually at risk
Section titled “What is actually at risk”The usual fear is that pasted text gets filed inside the model like a record in a database, and that a competitor could later retrieve it by asking the right question.
That is not what training does. Training adjusts the model’s weights, the millions of numbers that decide which word comes next. A conversation used in training is not stored as a conversation anywhere. It nudges numbers, alongside billions of other pieces of text, and then it is over.
3 routes could actually carry pasted text to someone else.
A person at the provider reads it. Safety systems flag a small share of conversations, and flagged material can be looked at by staff or by contractors under confidentiality terms. This is the most ordinary route and the least discussed.
The log store is breached or misused. The copy kept for review is a database like any other database, and it inherits every risk a database has. It can also be reached by legal process.
The model memorises it. Memorisation is real, and it is narrow. Text becomes reproducible when it is both distinctive and repeated many times across the training data, which is why a widely copied passage of a novel is a live memorisation question and a single customer’s renewal terms, seen once, is close to a dead one.
Two of those 3 are about stored copies rather than about training, and the settings are named after that difference.
Almost every provider keeps a short-lived copy of what goes in and what comes out, so that automated checks, and in a small number of cases a person, can look for misuse. That copy is what a retention window refers to, and the practice has a name: abuse monitoring. It runs on tiers where training is switched off, because it is there for a different purpose. “Training is off” and “nothing is kept” are two different claims, and reading the first as the second is the most common mistake in this area.
The window is usually weeks rather than years, and on approved accounts some providers will run without keeping anything at all. But retention is a promise the provider makes, and a promise can be suspended. In a United States copyright case, a court required a major provider to preserve output logs it would otherwise have deleted, then later ordered a large sample of de-identified conversations to be handed over in discovery. Nobody on those tiers was asked. A retention setting describes what the vendor does in the ordinary course, not what a court can require of it.
4 surfaces, 4 default paths
Section titled “4 surfaces, 4 default paths”Path 1: the consumer chatbot
Section titled “Path 1: the consumer chatbot”An employee types into ChatGPT, Claude, Gemini, Copilot or similar.
The prompt leaves the laptop, crosses the internet to the provider’s servers, the model runs, the answer comes back. That round trip is the request lifecycle, and it is the part everyone pictures. What the picture leaves out is everything that happens to the copy afterwards.
This is the surface where the rules move. Consumer defaults on training differ from one provider to the next, and they change: one may treat model improvement as on unless the user turns it off, another may ask each user to choose, a third may keep a long activity history on its own delete schedule. Naming which is which here would be wrong within a year, and that volatility is the durable point. A consumer default changes through an update to the terms of service, announced in an email and a banner, and it lands on every employee account at once without passing through anyone who owns data policy.
The business and enterprise tiers of the same products behave in the opposite way. Training off by default, retention that can usually be tightened, access logs, and a contract that says so. The contract is the difference that matters: it moves the arrangement from a setting the provider can change to a term the provider has agreed.
What an individual account holder controls: the plan, the training toggle, whether conversation history and memory are on. What they do not control, and mostly cannot see: which other companies the provider passes data to, which country the servers are in, and what a review of a flagged conversation looks like from the inside.
Path 2: an internal tool built on an API
Section titled “Path 2: an internal tool built on an API”A team has built a thin app over a model provider’s API, calling the model directly with no chat interface in the middle. Most of these run on OpenAI’s, Anthropic’s or Google’s, or on one of the hosting providers that serve open models.
The route across the network looks the same as Path 1. The defaults do not. Traffic through the major providers’ APIs is not used for training by default, retention is shorter, and the terms attached to an API key are usually stronger than the ones attached to a consumer sign-up.
3 things then change what happens to the data, and all 3 sit inside the company rather than at the provider.
What the app itself keeps. The team’s own code sits between the user and the model, and it may be writing every prompt and every response to its own database, its own monitoring tools, or a third-party analytics product. Those logs are inside the business, which makes them nobody’s headline risk and everybody’s blind spot. They need what any sensitive production data needs: access control, retention, encryption, deletion when someone asks.
Which country. Most major providers let a customer pin processing to a chosen region. Whether that was done when the integration was built is a matter of fact with a checkable answer.
What the model can reach. If the app has tools, retrieval over company documents, web access, a connection into a CRM, then each of those is a data path of its own.
Path 3: a self-hosted model
Section titled “Path 3: a self-hosted model”The team is running an open-weights model on infrastructure the business controls: its own machines, or its own account inside a cloud provider, on its own private network. Both are self-hosted. Only the first is what IT means by on-premise, where the machines are in the building, and the rest of this path applies either way.
The good news is real. The prompt and the response never leave the environment, and the question of exposure to an outside vendor is largely answered.
The harder news is that exposure inside the environment is now owned end to end, and it arrives as a list of things that need an owner. Which machines run the model. Who can log in to them. Whether prompts are being written to a log file, where that file sits, how long it is kept, and who can read it. Whether the software running the model quietly reports usage back to its makers, which at least one widely used option does until it is switched off. Whether the interface wrapped around the model saves conversations to a database, and who has access to that.
None of this is exotic. It is the same list any sensitive internal system gets. It falls through the cracks because the conversation about self-hosting is dominated by “the data doesn’t leave” and stops there.
Path 4: a vendor product where AI is the value
Section titled “Path 4: a vendor product where AI is the value”Third-party software with AI inside it. A note-taker on every meeting. A support assistant. A sales research tool. A coding tool that reads the company’s repository.
This path has the most steps. Company data goes to the vendor’s servers, and most vendors of this shape do not run their own models. They call an upstream provider. So the flow is: company data, to vendor, to model provider, to vendor, back to the company. Two companies hold copies, each with its own retention, its own staff access, and its own list of other companies it passes data to.
Then there are the connections the product needs to be useful. A note-taker that reads the calendar and writes to the CRM. A support assistant that reads the ticket history and the knowledge base. A coding tool that reads a private repository. Each connection is a separate path, and each one is a separate act of trust.
One contract, two paths
Section titled “One contract, two paths”Same document, same question, same model underneath: what differs is who holds the copies and under what agreement.
3 things decide the rest
Section titled “3 things decide the rest”Across all 4 surfaces, the same 3 variables carry most of the weight.
The plan. On a consumer product, the gap between a personal plan and a business plan is the single biggest lever available, because it is the one that swaps a setting for a contract. On an API, the equivalent is whether the account is on standard terms, pinned to one country, or approved for a mode where the provider stores nothing. On a vendor product, it is the gap between the small-business plan and the enterprise plan, which is where two things live: a contract term setting out what the vendor may do with the data, called a data-processing agreement, and a list of the other companies the vendor hands data to, called its sub-processors. That list is where the plain physical fact of which country the data sits in gets settled, which contracts call data residency. Self-hosting has no plan tier, and its closest equivalent is whether the machine running the model can reach the internet at all.
How the model is reached. Chat interface or direct connection is the cleanest version of this. The interface carries features, conversation history, memory, plugins, connections to other systems, and each feature is another path. The direct connection is barer and usually stricter by default. For a self-hosted model the equivalent is whether the model is reachable only from inside the company’s own private network, or from the public internet as well.
Retention and training settings. Within one plan and one way in, the toggles still decide things: training off, history off, memory off, storage set to nothing on the tiers that offer it. For a self-hosted model the equivalent is the company’s own log policy: how long prompts and responses are kept, who can read them, and when they are deleted.
Getting those 3 right is most of the work on any surface.
Where one system hands data to another
Section titled “Where one system hands data to another”Whichever surface the AI lives on, the incidents that actually hurt tend to happen where one system passes data to the next rather than in the model call at the centre.
Companies between the user and the model. A business selling “AI for X” built on top of a major provider is its own company with its own data practices. Its interface is the user’s front door, its database holds the data, and it forwards the request onward, which is the product-on-top-of-a-model arrangement in its usual form. Two companies in the chain, each with its own logs, retention, staff access, and onward suppliers. These are often excellent products. They are also harder to check, because the check is now of two companies in series.
Plugins, tools, and connections. When an AI tool reads your Google Drive, queries your CRM, or fetches a web page on the user’s behalf, each of those touches creates a new path. The model provider’s privacy terms may be spotless while the plugin maker’s are not. A code AI that reads your private repository is, for data purposes, a code-reading vendor, which is a separate question from whether the model trains on the code. Each of those is also a connection with an account behind it, which is a separate question from the data path: what the connection may do, as against where the data travels.
Voice and screen. Voice agents turn speech into text through a transcription service before the model sees anything. Screen-sharing agents capture and process images of the screen. Each of those is another company with its own retention, and it is frequently not the company whose name is on the product.
Search over company documents. Systems that answer from company material (the pattern from tools and memory) first convert every document into a numeric fingerprint the system can search over, called an embedding, and store the fingerprints in a database of their own. Those fingerprints are derived from the original text and carry the original’s sensitivity, and the store holding them is a live data system with its own access rules and its own retention. It is the easiest thing to forget, because on the diagram it is a small box behind the model.
Every outside system in an AI interaction is its own data path. The count is almost always higher than the diagram in the product’s pitch shows.
Self-hosting, when it is actually the answer
Section titled “Self-hosting, when it is actually the answer”If the data must not leave the environment under any circumstances, the model has to come to the data.
Open-weights models run on infrastructure the business controls, and prompts never cross the boundary. Where the country the data sits in is fixed by law, in defence, in parts of finance and healthcare, in some government work, this is the only path that qualifies.
The costs are worth naming plainly. Open-weights models trail the best closed models in quality, the gap has been narrowing, and it is now smaller than the difference in what it takes to run them. Running one well is not a project with an end date. It is a permanently staffed capability: machines that have to be sized and paid for whether or not they are busy, software that has to be upgraded as models change, and people who can tell the difference between a model that is wrong and a deployment that is broken. Under-staffed, it does not fail in a way anyone notices. It degrades: an old model nobody has swapped, a queue that gets slower under load, logs nobody rotates, and a quiet drift back to the consumer chatbot by the people who needed an answer today.
For most businesses an enterprise plan is the cheaper answer, and it is worth being exact about what it buys. It answers the training question and the retention question, because both become contract terms. It does not answer the question of the data sitting on infrastructure the business does not control, under a list of other companies it did not choose, inside someone else’s legal exposure. Self-hosting answers that last question and hands back the internal ones instead.
All 4 at once
Section titled “All 4 at once”What is actually running inside most operations is some combination of all 4 paths. An analyst is in a consumer chatbot. An engineer is shipping a tool on an API. A team has a self-hosted model on an internal cluster or a rented one. A vendor product bought last quarter is reading the calendar every morning.
The gap between those 4 and the person who owns data policy is usually fine while the work is test questions with nothing real in them. It closes the moment real work moves through: customer data, unreported numbers, anything that would matter in a leak. The exposure does not wait for anyone to have answered the question.
Which is why the useful question is not which path the company is on. It is which paths are in use, what each one’s defaults currently are, and which of the 3 variables are set the way somebody actually meant them to be. That is a smaller piece of work than it sounds, and it is the first of the 4 risks worked through in detail.