What to watch for once AI is in production

Four things start moving the day an AI tool goes from a handful of careful users to everyone: what it costs, what data it now carries, who is allowed to act on what it produces, and whether the model answering in November is the model that was tested in May. Nothing has to break on the day it ships for all 4 to be under way.

None of the 4 is a reason not to ship. They are the shapes a production AI system has to be built around, and they are unlike the risks of an ordinary software rollout for one reason: ordinary software holds still once the code stops changing. This does not.

Cost

A server runs whether one person uses it or a hundred. AI reached through a provider works the other way: every request is metered by the amount of text it carries. What that meter counts and how to estimate it is settled before a system ships. What changes in production is that the numbers behind that estimate start moving, and they move one way.

Volume outruns the forecast. A tool that runs 10 times more often costs 10 times more, because a built system is billed per use rather than per seat. A pilot is 2 people being careful on a test extract. Production is everyone, on live records, including the people who were never in the room and the uses nobody scoped.

Prompts grow after launch. The instructions sent with every request get added to one line at a time: a rule after a bad answer, another worked example, another document pushed into the request whole instead of retrieving the part that matters. Every addition is billed on every run from then on. Additions that shift a little each call also miss the discount for repeated text, which needs the repeated part to be identical.

Prices change without a release. The rate is the provider’s to set and the provider can move traffic onto a new default model at a different rate, both without anything changing on the company’s side. Neither arrives as a deployment, so the invoice is where they show up.

Example

Let’s say Northwind’s brief-and-draft tool, the pilot from what makes a good AI project, goes from 2 reps on a test extract to the whole sales team on live account records. It reads 2 years of order lines for an account and writes back a short brief and a draft email, so nearly all of the bill sits on the input side, and it now runs on a model the provider is free to change without telling anyone. Nothing in the code changed. The invoice is where the change shows up first.

The control is a reading and a ceiling. Cost read per use rather than per month, so a rise can be traced to volume or to what each run now carries. And a hard spending limit with an alert set below it, because the first sign of a request loop running away should not be an invoice.

Security

The pilot ran on a test extract. Production runs on the live record: real customer names, real order histories, real prices leaving the building on every call. The code did not change. What flows through it did.

Where your data goes lays out the 4 surfaces and what each one keeps. Three questions turn into facts worth holding in writing once real data is moving through one of them.

• Which door the request goes through. A model reached on the provider’s own servers, one running inside the company’s own cloud account through AWS Bedrock or Azure OpenAI, and one running on hardware the company owns all answer the same question. They answer very differently on where the data sat while they did it.
• What is kept, and for how long. Whether inputs are stored, how long the copy lives, and who inside the provider can open it. This varies by plan and by contract, and the contract is where the answer is actually written down.
• Whether inputs feed training. Business and enterprise plans exclude it by contract. Consumer plans historically have not, and their defaults move through a terms update rather than through a conversation.

Encryption in transit and at rest is standard in every enterprise contract now and is rarely what goes wrong. For health, finance or government data the question that matters is whether this particular setup sits inside the scope of the regime that applies to it: HIPAA, PCI, or the sector’s own framework. A provider holding a certificate is not the same claim as this deployment being covered by it.

The most common failure is that somebody carries on using the free account the prototype was built on. Free tiers are convenient and were never designed to carry operational customer data. Paid tiers usually are, and which protections apply depends on the plan and the contract rather than on the logo staying the same.

Governance

The people in a pilot know how far to trust the tool, because they were there while it was built. Production reaches people who were not, and the output starts landing in places where somebody acts on it. Rewriting an internal note is one thing. The sentence a customer reads, or the reason a candidate gets rejected, is another. With nothing written down, the line between the two gets crossed by whoever is in a hurry.

Four short things cover the minimum.

• Who is approved for what. Which tools are cleared for which work. Default-allow for low-risk work, default-deny for high-risk work, with the boundary written rather than inferred.
• Who owns the verdict. For anything customer-facing or decision-affecting, a named reviewer. “AI handles the draft, the team handles the verdict” only works when the verdict has an owner. (Building trust covers the review patterns at each level of blast radius.)
• What can be reconstructed later. When a customer questions a response or a regulator asks how a decision was made, the system has to be able to say which model, which version, which prompt, against which data. Logging that at build time is cheap. Recovering it afterwards is close to impossible.
• What never goes in. Customer personal details, unreleased financials, salary data, anything under an NDA. Most of it ends up on the do-not-send list, and the list has to be written, because an assumed one is different in every head.

None of this needs to be 40 pages on day one. A one-page note the team has actually read outranks a comprehensive policy sitting in a folder nobody opens.

Drift

A prompt that produced the right brief in May produces a subtly different one in November. Same words going in, different words coming out, and nothing changed on the company’s side.

Drift arrives in 4 shapes.

• Updates under the same name. The provider improves the model behind a version name that does not change. The behaviour moves. Nothing in the company’s code did.
• New defaults. A new model ships and existing traffic gets rolled onto it, usually with a different tone, different refusal patterns and different formatting.
• Retirements. Old versions get switched off. Calls to them start failing, with notice that varies from generous to slight.
• Style shifts. Even inside a stable version, the way answers get phrased and laid out can move through tuning on the provider’s side.

The defence is to stop taking whatever “the latest” points at. Providers publish specific dated versions alongside the moving name, and a production system asks for one of those by name and keeps asking for it until somebody decides otherwise. That is pinning, and what it buys is scheduling: a change that would have arrived on the provider’s timetable now arrives on the company’s.

Pinning buys time rather than safety, because a pinned version is eventually retired too. Two things sit behind it. A second model wired in as a fallback, so a retirement is a switch rather than an outage. And, on any deliberate version change, a re-run of the test set the tool was tested on before launch. That re-run is the only thing standing between a regression and a customer.

Prompts age as well. They get written against one model’s habits and quietly stop fitting the model that answers now. Treating them as living text, re-read on a written cadence rather than fixed at launch, is what stops that accumulating. Prompting is one of the 3 ways to adapt a model, and most drift events call for a small turn of it rather than a rebuild.

What a responsible setup costs

The floor for any AI system that real work depends on is short. Cost measured per use, with a spending ceiling and an alert under it. A data-handling arrangement that matches the sensitivity of what is actually flowing through, checked rather than assumed. A named person who reviews the high-stakes output. A pinned model version, a fallback behind it, and a written cadence for re-checking both. And a one-page usage note the team has read.

None of it is expensive. In a company the size of Northwind it is a spreadsheet, a paragraph in the team handbook, and a line in the code naming a model version. Skipping it is the expensive option, and the bill arrives as a surprise invoice, a leaked record, a decision nobody can reconstruct, or 6 months of quietly worse output that nobody catches until a customer does.