The demo goes well. Someone builds something over a couple of weeks, shows it to a room, and the room agrees it is impressive. Six months later it is still impressive and still not in production. This is the most common shape of an AI project inside an established company, and the reason is almost never the model. A demo and a system people depend on are different artifacts with different requirements, and nobody wrote down the second set.
The eleventh example
A pilot is usually tested against examples the builder chose. They get chosen, mostly unconsciously, because they are clean: the document has the layout the parser expects, the question is phrased the way the prompt anticipated, the record has all its fields populated. Real traffic is not like that. The eleventh example is a scanned fax, a question containing two questions, a record with a null where the process assumed a value, or a customer describing a problem in words nobody at the company uses.
The countermeasure is dull and effective. Before building anything, pull a random sample of real inputs (a hundred is usually enough) and hold it back as an evaluation set. Do not curate it. Do not remove the ones that seem unfair. The distribution of ugly cases in that sample is the distribution the system will face, and a pilot that has not been run against it has not been tested.
- Inputs that are malformed, truncated, or in an entirely different format
- Cases where two plausible answers exist and the business has a rule about which one wins
- Records the process assumes are complete and which are not
- Internal abbreviations, product names, and shorthand that appear nowhere in public text
- Cases that should be refused or escalated rather than answered
No agreed definition of good enough
The second failure is that nobody said what success would look like, so the pilot can never be declared finished. Someone asks whether it is accurate enough and the answer is a shrug, because accurate enough was never defined against anything. In the absence of a stated threshold the default standard becomes perfection, no system clears that, and the pilot expires without ever being rejected.
Agree the threshold before building, and agree it with the people who own the work rather than the people who own the budget. It has two parts: the accuracy level required on the held-out sample, and the failure budget, which is how many wrong outputs per hundred the business can absorb given what happens downstream when one gets through. Those two numbers are the difference between a pilot that ends in a decision and one that ends in a recurring meeting nobody attends.
Nobody owns the workflow the AI sits inside
Pilots are usually run by whoever was curious. That is fine for a demo and fails immediately afterwards, because a production capability generates continuous work. Someone has to review a sample of outputs, decide whether a new failure mode matters, approve prompt and configuration changes, and answer users who think it got something wrong. If that person is not named and given time, quality degrades and nobody notices until a customer does.
The owner is a business role, not an engineering one. Engineering owns the system; the workflow owner owns whether the output is good and what the business does when it is not. Stalled pilots almost always have an engineering owner and no workflow owner, which is why the last mile (approval, rollout, training, enforcement) never gets walked.
If it is not where the work happens, it is not used
A large share of stalled pilots live in a separate window. The user leaves the system they work in, pastes something across, reads a result, and pastes it back. That is tolerable for a fortnight while it is novel, and then people stop. The usage decline gets read as evidence that the AI was not good enough, when what actually happened is that the round trip cost more effort than the output saved.
Integration is where most of the real engineering lives, and it gets underestimated because it is not interesting. The output has to land in the record, the ticket, the order, or the document, wherever the next person looks. It has to respect the same permissions as everything else in that system, so the capability cannot summarize a document the user is not cleared to read. It has to be attributable, so anyone reading the record later can tell that a machine produced that line and when. None of this is model work, and all of it is the difference between a pilot and a product.
What changes on the way to production
It is worth being explicit about the gap, because teams frequently believe they are one approval away from production when they are eight requirements away from it.
| Dimension | Pilot stage | Production stage |
|---|---|---|
| Test data | Ten to thirty examples chosen by the person building it | A held-out sample drawn at random from real traffic, including the malformed cases |
| Success criteria | It looked convincing in the room | A stated accuracy threshold and failure budget, measured and signed off |
| Integration | A standalone window, notebook, or spreadsheet | Inside the system of record, with the result written back and attributed |
| Permissions | Whatever the builder could access | The same access rules as the surrounding system, enforced per user |
| Cost | A few dollars of usage, unexamined | Modeled at peak volume, with a per-unit ceiling and an alert when it is crossed |
| Latency | Whatever it took; nobody was waiting | A budget users tolerate, measured at the slowest realistic input rather than the average |
| Failure behavior | Re-run the demo | A defined path for wrong and for unavailable, with the user told which happened |
| Ownership | Whoever built it | A named workflow owner with allocated time, plus an engineering owner for the system |
Cost per call at real volume
A pilot processes a few hundred items and the usage bill is a rounding error. The same workflow across a year of real volume is a line item somebody has to approve. Do the arithmetic early: cost per call, calls per day at peak, and the ratio of the annual total to the cost of the work being replaced. If it does not clear comfortably, redesign (smaller inputs, a deterministic path for the easy cases, batching where the answer is not needed immediately) or stop. Some tasks are not worth automating at any price, and establishing that in week two is a good outcome.
Latency people will tolerate
Latency is a product requirement that pilots skip because nobody in a demo is under time pressure. A user mid-task will wait a couple of seconds and then switch to something else; an overnight batch can take hours. Decide which one is being built, then measure at the slowest realistic input rather than the average, because the long documents are exactly the ones where people most want help. If the interaction cannot be made fast enough, move it off the critical path: run it ahead of time so the answer is already there when the user arrives.
A fallback for wrong, and a fallback for unavailable
These are two different failures needing two different answers. When the capability is unavailable (an outage, a rate limit, a timeout), the workflow needs a defined path: retry, queue, degrade to the previous process, or tell the user plainly that this step is not available right now. Silent failure is the worst option and the most common one. When the output is confidently wrong, the answer is a review step sized to the consequence: sampled review for low-stakes drafting, mandatory approval before anything reaches a customer, a ledger, or a regulator.
Who is accountable for a wrong output
This is the question that stops pilots at the final gate. It is usually raised by legal, finance, or a cautious executive, and it usually goes unanswered because it was never assigned to anyone. It is not primarily a legal question. It is an operating question with three parts, each of which needs a name written against it.
- Who reviews output before it reaches a customer, a regulator, or an accounting system, and the threshold under which that review can be skipped
- Who is told when something wrong gets through, how they find out, and how quickly
- Who can turn the capability off, how long that takes, and what the business does in the meantime
Write those three down. They fit on one page. Their absence is why a lot of technically working pilots never get approved, and their presence turns a risk objection into a control review, which is a conversation that can actually be closed.
How to restart a stalled pilot
- 1
Write down the decision the pilot was meant to inform and the threshold that would settle it. If nobody can state one, that is the actual blocker, not the technology.
- 2
Assemble a held-out sample of a hundred real, uncurated inputs and measure the existing pilot against it. Expect a worse number than the demo suggested.
- 3
Name the workflow owner, who should be a business role, and get their time allocated in writing rather than assumed.
- 4
Model cost and latency at peak volume before writing more code. If either fails, redesign the workflow or stop.
- 5
Decide the fallback for unavailable and the review step for wrong, each sized to what happens downstream.
- 6
Integrate into the system of record, with permissions and attribution, before expanding scope. One narrow workflow running in production is worth more than three impressive demos.
Almost nothing on that list is about AI. That is the point. The pilots that reach production are not the ones with the cleverest prompt; they are the ones that were treated as software from the second week onward, with an owner, a threshold, a place to live, and a plan for being wrong.
Common questions
- How long should an AI pilot run before we decide?
- Four to eight weeks is usually enough to answer the question, provided the question was written down at the start. If a pilot has run longer than a quarter without a go or no-go decision, the missing piece is a success threshold rather than more engineering. Set the decision date at the same time you set the scope.
- What accuracy is good enough for production?
- It depends entirely on what happens downstream when an output is wrong and who sees it before it matters. A drafting aid that a person reviews can be useful well below the accuracy you would demand of something posting to a ledger unattended. Define the threshold against the consequence and against the current human error rate, not against an abstract number.
- Should we build this or wait for it to appear in software we already own?
- If the capability is generic (summarizing, drafting, or classifying inside a vendor's own data), waiting is often the right call, and building it yourself means maintaining something your vendor will eventually ship. Build when the value depends on your data, your rules, or a workflow that crosses systems no single vendor can see. That test settles most of these decisions quickly.
- Our pilot works but usage dropped after the first month. What happened?
- Usually the capability sits outside the system where the work happens, so using it costs more effort than it saves once the novelty wears off. Sometimes it is a trust failure instead: an early wrong output that nobody acknowledged or corrected. Ask the users which of the two it was before changing anything technical, because the fixes are completely different.
- When is AI the wrong tool?
- When the task has a correct answer that a rule can express, use the rule. It is cheaper, faster, testable, and explains itself. When the input data is missing or untrustworthy, fix that first, because a model reading bad data produces confident bad output. And when nobody can absorb the review workload the output creates, the automation adds work instead of removing it.