From prototype to production.
A prototype shows that an AI system can work. Production is where it has to keep working, with real users, real data, and a real budget. This page sets out what that takes, and how to check whether your system is there.
What production-ready means.
An AI system is production-ready when four things are true. It is evaluated against the failures it is known to have, with checks that pass or fail. Every path from the model to a user goes through a guard. Its cost is counted per successful task, not per model call. And every change reaches production through gates that a named engineer signs.
Four gaps between a demo and a launch.
No way to tell a good answer from a bad one. Theenk.AI's first evaluation suite was six numeric "quality" scores that nobody could act on (the Theenk.AI case study).
Nothing between the model and the user. Output reaches a screen, or a memory, unchecked.
Costs counted per call. The model invoice is a fraction of what the system costs once retries, review, and infrastructure are counted (AI Economics).
No road to production. The prototype runs in dev, and nobody has built the environment, the secrets, or the release path it needs.
The engineering, part by part.
Evaluate the failures you have seen.
Binary evaluators, each tied to a failure mode that was actually observed; a pass is evidence about those cases, not a promise about production (Theenk.AI).
Guard every path a user sees.
One component owns the only path from model output to a response or a memory write, and structured outputs pass a blocking check first (the digital health study).
Price the successful task.
Model calls, retries, orchestration, evaluation, human review, and infrastructure, divided by the tasks that produce an accepted result (AI Economics).
Release through gates.
Repo-ready, spec approved, plan approved, verified, production-ready, released and measured (the six gates).
Cut over without breaking anyone.
New paths go live behind flags, with the old service untouched until the new one is verified (the digital health study).
Is your AI system ready for production?
- Every model call runs where your data agreement allows.
- An evaluation suite exists, and each check is tied to a failure you have actually seen.
- Each check passes or fails; no score nobody can act on.
- One component owns the only path from model output to a user or a memory write.
- Structured outputs pass a blocking check before anything downstream reads them.
- A production environment exists, separate from dev and built from code.
- Secrets are listed in a manifest and checked in CI.
- You know the cost per successful task, with retries, review, and infrastructure counted.
- Changes reach production through gates, and a named engineer approves every merge.
- New paths go live behind flags, and the old path stays until the new one is verified.
Fewer than ten? The Production Readiness Review finds which ones are missing, in a week.
Frequently asked questions
What does production-ready AI mean?
An AI system is production-ready when four things are true. It is evaluated against the failures it is known to have, with checks that pass or fail. Every path from the model to a user goes through a guard. Its cost is counted per successful task, not per model call. And every change reaches production through gates that a named engineer signs.
Why do AI prototypes stall before production?
Usually for one of four reasons: no evaluation that tells a good answer from a bad one, no guard between the model and the user, costs counted per call instead of per successful task, and no environment or release path beyond dev.
What is cost per successful task?
The full cost of running an AI workflow (model calls, retries, orchestration, evaluation, human review, and infrastructure) divided by the tasks that produce an accepted result.
How should an AI system be evaluated before release?
With pass/fail checks, each tied to a failure mode that has actually been observed, run against real user turns. A pass is evidence about those cases, not a measure of how the system behaves in production.
How long does it take to get an AI prototype into production?
It depends on the gaps. The Production Readiness Review finds them in one week, and a Velocity Sprint takes one feature to production in 4–6 weeks.