Twelve PoCs reviewed in the last two years.
Three went to production. Here’s what separated them.
Every quarter for the last two years, a client has asked us to review an AI proof-of-concept. Sometimes the brief is “can you help us get this into production.” Sometimes it’s “can you tell us why this isn’t moving.” The second framing is more honest.
Twelve PoCs across that period. Three made it to production. The other nine are either parked or quietly dead.
The pattern was clearer than I expected. It wasn’t about model choice. It wasn’t about prompt engineering quality. It wasn’t even about the use case. It was about how much of the production scaffolding was built into the PoC from day one.
The PoC mindset is the problem
A PoC is built to impress someone in a meeting. The way you build something to impress someone in a meeting is to make the demo path bulletproof and ignore everything else. Hardcoded prompts, no error handling, no cost tracking, no PII handling, no fallback if the model is down, no validation on output, no audit trail. The demo works. Leadership is impressed. The team is asked to “productionise” it. The team realises that “productionising” means rewriting all of it. Eighteen months later the project is parked.
This isn’t an AI problem specifically. It’s the same dynamic that kills any “prototype in two weeks, then we’ll scale it.” But AI makes it worse, because the model call is the part everyone focuses on, and the model call is the easy part.
The hard parts
Here are the things that separated the three PoCs that made it from the nine that didn’t. They had nothing to do with the model.
Output validation. Schema validation on every response, with retry-and-repair loops. If the model returns malformed JSON, the system catches it, asks again with a sharper prompt, and falls back to a known-good shape if it still fails. Output that doesn’t meet contract spec doesn’t reach the downstream system. Nine PoCs out of twelve didn’t have this. Three did.
Cost ceiling. Every AI feature has a per-feature token budget, hourly usage caps, and monthly tracking. AI costs scale with traffic and they scale fast. The three production PoCs had cost dashboards on day one. The nine that didn’t, found out in month three when the bill came.
PII redaction at the entry point. Before any text reaches the external model API, sensitive data is identified and redacted. This is not a privacy feature. It is the difference between a feature that can ship in Europe and a feature that cannot. The three that made it had this from the first commit. Some of the nine added it later. None of them added it cleanly.
Provider abstraction. The model behind the feature was reachable through one internal interface. Switching from OpenAI to Anthropic to Azure-hosted open-weight was a configuration change. None of the nine PoCs had this. By the time someone realised it should, the codebase had OpenAI-specific calls in seventy files.
Human-in-the-loop where it mattered. For any decision with real consequences — invoice approval, customer-facing message, anything regulated — the AI was producing a recommendation and a person was approving. Audit trail captured both sides. None of the nine had this either. They were trying to fully automate things that should have stayed semi-automatic.
What we tell clients now
If you have a PoC and you’re trying to decide whether to productionise it, here is the honest test. Open the codebase. Search for the model API call. Look at what’s around it. If there’s no schema validation, no cost cap, no PII filter, no provider abstraction, and no audit log — your PoC is not 80 percent of the way to production. It’s about 15 percent of the way. The other 85 percent is what you skipped to make the demo work.
That doesn’t mean the PoC was wasted. The model selection, the prompt engineering, the user flow — those are real. But the production work is a rewrite of everything that holds the model call. Estimate accordingly.
The three PoCs that made it didn’t impress anyone in their demo. They were slow. They had error messages and admin screens. They had a cost dashboard that the demo team had to apologise for because it made the slide feel less magical.
Those were the ones that shipped.