What gets built around the model

A model writes text from pattern, and that is all it does. It does not look anything up, act on any system, remember yesterday, or check its own answer. Every AI product a business uses is that one part plus the parts built around it to cover the rest.

Those surrounding parts are most of the product, and they are the part that varies. The model at the centre is rented from one of a few labs, and a competitor can rent the same one on the same terms. When one AI product is clearly better than another, the difference almost always sits outside the model.

The 5 parts

Standing instructions. Text the product author writes once and the product sends ahead of every message: what the tool is for, what it must not do, what to say when the answer is not in front of it, what format to answer in. The user never sees them and never types them. They set tone, format and refusal behaviour, and they are why one tool says “the policy does not cover this case” where another guesses. Why prompts matter covers what a good instruction carries; at product level the same craft is written once and runs on every request.

Material fetched for the question. A search over the company’s own documents, records or tickets, run at the moment the question arrives, with the passages it finds placed in front of the model alongside the question. This is retrieval, and it decides whether the answer is about this company at all. It also decides how current the answer is, because the documents behind it can be updated the same week. What it does not do is make the model cleverer. It hands the model better material.

Connected tools. Anything the model can call to do what writing cannot: a lookup in the order system, a database query, a calculator, a booking, an update to a record. Tools decide whether the product can only talk or can also act, and the permissions on each tool decide how far an action reaches. A product that can read one order by number and a product that can issue refunds are built out of the same mechanism with very different limits set on it.

Saved memory. Short entries the product keeps between conversations and puts back in front of the model at the start of the next one. It carries preferences and durable facts about the person and the business, and it is why a new chat opens already knowing the account. It is a summary the product chose to keep, so it can be quietly out of date, and it is not access to any system.

Checks before the output counts. What stands between the model’s text and the outside world: a person approving each one, a person reading a sample afterwards, or software testing every answer against a source of truth and holding back what fails. Building trust in AI sets out how to pick, and what belongs here is that the checking is built, like the rest. So are the records of what the AI was given, what it produced and what a person changed.

None of the 5 substitutes for another. Better standing instructions never supply a fact the model was not given. Perfect retrieval does not stop an answer going out wrong when nothing checks it. Memory of what the business does is not permission to touch what the business runs. Products get good by having each part built well, and get embarrassing by having one of them missing.

Why the same model lands differently in 2 places

Traced end to end, a single request runs through about 7 stages and the model is 2 of them. The other 5 are these parts doing their work: assembling what the model will see, fetching, calling tools, holding the conversation. Two products can send the same question to the same model on the same day and receive different answers, because what each one put in front of it was different.

Let’s say 2 retailers wire the same model into their support inbox in the same month.

At the first, the model gets standing instructions to answer only from company sources, quote the clause it used, and say when the policy does not cover a case. A search over the policy pages puts the relevant passages in front of it. A connection to the order system lets it look up one order by number and nothing else. Before any reply leaves, a second system checks every date and refund promise in the text against the order record and the policy page, and holds back anything it cannot find in either.

At the second, the customer’s message is pasted into the model and the reply is sent.

A customer writes on Wednesday: her order was due Tuesday and has not arrived, and a damaged item she bought 40 days ago is still in its box.

The first product looks up the order, sees it left the warehouse on Monday with delivery revised to Thursday, and says Thursday. It quotes the returns clause: 30 days, with a photograph of the damage. 40 days is past 30, so it says the window has closed and passes the case to a person who can make an exception. The checking step finds Thursday in the order record and 30 days on the policy page, and lets the reply go.

The second product has no order record in front of it, so it writes a delivery estimate that reads perfectly and came from nowhere: 3 to 5 working days. It has never seen the returns policy, so it answers from the average of everything it absorbed in training: returns accepted within 90 days, send it back and a refund follows. Both numbers are invented, both are fluent, and nothing sits between them and the customer. The retailer now chooses between honouring a refund its policy does not allow and telling a customer that its own support desk was wrong.

Same model, same month, same question. The first retailer built 4 of the 5 parts. The second built none of them.

What a team is building when it says it is building AI

Almost always, those parts. Connecting the sources and keeping the index current as documents change. Writing the standing instructions and the answer format. Deciding which systems the tools may touch and with what permissions. Deciding what happens when the search finds nothing. Deciding what gets checked, by whom, and what happens to an answer that fails. Keeping the records.

The 3 ways to adapt a model to a business sit on this same picture. Prompting is the standing instructions. Retrieval is the fetched material. Fine-tuning is the only one of the 3 that changes the model itself, and it is the rarest, which is another way of saying that most of the work happens around the model rather than to it.

The same is true of what a vendor sells. Most AI products in a business run on somebody else’s model, so what is being bought is the wrapping: the connections, the instructions written for one job, the checks, the interface. That wrapping can be substantial or it can be a prompt template with a markup on it, and both look similar from the outside.

What is actually being compared

When 2 products name the same model, the model is the part of the comparison that carries the least information. These are the questions that separate them, and each has a factual answer a vendor can give:

Where does an answer come from, and who keeps that source current?
What can it do besides write, on which systems, and with what limits on each?
What does it do when it finds nothing useful? Say so, or answer anyway?
Does it show the source beside the answer, so a person can trace it?
What is checked before an answer goes out, and what happens to one that fails the check?
What is recorded about each run, and who can read the record?
Can the model underneath be changed when a better one ships, without rebuilding the product?

A team that can answer those 7 about its own tool knows what it has. A vendor who cannot answer them is selling the model, which was never theirs.

This is also why changing the model is a weak fix. Between the top few models the differences are small for everyday business work, and a product failing on what it fetches, what it can reach or what it checks will fail the same way on a better model. It will just fail more fluently.