Back to the blogCitește în română

AI systems in production: from an impressive demo to a reliable system

Why many AI projects die after the demo and how you build a system that lasts in production: narrow scope, clean data, continuous evaluation, human in the loop and costs under control.

A bridge from prototype to production rests on four pillars: clear purpose, clean data, evaluation and human in the loop. Error and cost monitoring feeds the evaluation.

Every team has experienced the moment: an AI prototype that responds amazingly to management, and two months later no one uses it. The demo worked because the questions were chosen by someone who knew what to answer well. The production brings the questions that no one chose. In this article, I describe what separates a demo from a reliable AI system, and what you can actually do, whether you’re an entrepreneur ordering an assistant or an engineer building it.

In short

  • A demo shows what the model can do at its best. Production demands acceptable worst-case behavior.
  • AI systems that survive have a narrow goal, a clear definition of the “right answer,” and a set of tests run on every change.
  • Model is a component. Trust comes from what you build around it: data, checks, limits, monitoring.
  • Human remains in the loop where error costs a lot and full automation gradually wins.
  • The cost is planned from the beginning, otherwise the invoice tells you late what you should have known early.

Why AI projects die after the demo

The first reason is trivial: the demo tests a happy road. Someone asks five good questions, the model answers nicely, the room applauds. Real users make mistakes, ask half-assed questions, ask for things the system doesn’t know, and send documents in strange formats. Each of these situations is a test case that you didn’t write.

The second reason is that no one has defined success. “To answer well” is not a criterion. “Answer at least 9 out of 10 questions correctly from a list of 200 chosen together with the support team and say they don’t know the rest” is a criterion. The first form leaves you with an impression, the second with a number that you can track over time.

The third reason is responsibility. A wrong assistant is “no one’s”. When a visible error occurs, someone needs to know why it happened and how to fix it. If the system doesn’t keep track (what it received, what it searched for, what it responded to), you have no way of knowing.

Start with a narrow goal

The most common architectural mistake is ambition. “An assistant that answers everything about the company” sounds good in the meeting and works poorly in practice, because “everything” is not testable. A narrow goal can be: “answer customer questions about returns and delivery”, “prepare a draft offer from briefs received”, “sort support requests by category and urgency”.

A narrow goal gives you three valuable things:

  1. A realistic test set. You can collect in two days a hundred real questions with the correct answer next to them.
  2. A small risk surface. You know what can go wrong and how much it costs to go wrong.
  3. An answer to “is it worth it?”. You can measure the hours saved on a single process, not on a general promise.

My rule of thumb: if I can’t write the goal in one sentence and test it in a week, it’s too big. You chop it up until it’s testable.

Data decides more than the model

When an AI system gives poor answers, the first reaction is to change the model. Most of the time the problem lies elsewhere: conflicting documents, old versions left next to new ones, information kept in people’s heads and not written down anywhere. A more expensive model reads a bad document just as badly.

What works in practice:

  • One owner for each source. Someone is responsible for the up-to-dateness of the price list, the return procedure, the warranty policy.
  • Only one true version. Old documents are archived, not left next to new ones.
  • Structure before volume. Ten well-organized documents beat a thousand piled files.
  • Separate test data. The examples you test the system with must not be the same as the ones it “learns” from.

If your system searches the documents before responding, also read RAG for Entrepreneurs, where I explain how document order affects response quality.

Evaluation is the regression test of the AI era

In classic software, a code change runs through automated tests before reaching users. AI systems need the same reflex, only the check is more nuanced, because the same answer can be formulated in ten correct ways.

A minimal evaluation set looks something like this:

Item What it contains Example
The question The user’s actual text, with all the mistakes “how long does it take to return the order from the weekend”
Reference answer What should the correct answer contain? “14 days from receipt, regardless of day”
The expected source The document from which the information must come “return-policy v3”
Case type Usual, Limit, Trap or “Don’t Know” “trap”

From this table you start three distinct checks. First: Does the answer contain the correct information? Second: Did he cite the right source? Third: when there is no answer in the documents, did he honestly say he didn’t know, instead of making it up? The third is the most important for trust and the most often forgotten.

You run the set at every change: another model, another prompt, other documents, another configuration. If the score drops, you know before customers do. If it goes up, you have a proof, not an impression. A practical thing: it also keeps in the set the cases that gave errors in production. Every real mistake becomes a permanent test and the system does not repeat it.

The man in the loop, at the right level

Not all decisions weigh the same. An assistant that suggests an article title can work alone. One who sends a quote to a customer, approves a refund, or changes an order needs a safety net.

A simple, three-level model:

  • Automatic. Reversible and low-cost actions: classification, summary, internal drafts.
  • With approval. The system prepares, the human confirms with a click: offer, response to the customer, data modification.
  • Human only. Decisions with large legal or financial consequences, where the AI can at most gather the information.

The important thing is that the level is not fixed. You start with approval on almost everything. You measure how often the human changes what the system proposed. When, for a certain category, the human approves without changes for months in a row, it can switch to automatic. Trust is earned on data, category by category.

How AI systems go wrong (and how to catch them)

AI systems don’t fail spectacularly, they degrade insensibly. That’s why you need specific nets:

  • Format check. If you expect a date, amount, or product code, you validate the result with classical rules, not model trust.
  • Citation of the source. An answer without a source is treated as suspect. A sourced answer can be verified in seconds by a human.
  • Confidence Thresholds. When the search finds nothing close enough, the system stops generating and hands the conversation over to a human.
  • Limits on actions. An agent that can call APIs receives only the strictly necessary permissions, with caps (number of calls, maximum amounts).
  • Error behavior. If the model provider doesn’t respond, the user sees a clear message and an alternative, not an endlessly spinning wheel.

About the last point: an AI system depends on external services, so it should be treated like any distributed system. Timeouts, retries with increasing pause, a fallback model for critical cases. Nothing exotic, just the usual engineering discipline applied to a non-deterministic component.

Observability: being able to answer “why did he say that?”

The most valuable thing you can build in the early days is a good journal. For each interaction you keep: the question, the documents found, the final prompt, the answer, the duration, the cost and, if any, the user’s reaction (satisfied or not). You treat personal data with care: we mask or keep them only as long as necessary and in accordance with the GDPR.

From the log you extract three simple reports, enough to run the system:

  1. Questions without a good answer. They are your list of improvements: missing documents, not covered wording.
  2. Most Expensive Interactions. Often a small portion of conversations consumes most of the budget.
  3. User reaction over time. If satisfaction drops after a change, you see it in days, not months.

The cost: Plan it before it goes up

The cost of an AI system has two parts. The first is construction: architecture, data preparation, integrations, tests. The second is the operation: the tokens consumed, the infrastructure, the time of the people who check and update the documents. The second part is the forgotten one, although in the long run it often trumps the first.

Some levers I use:

  • Suitable model for the task. Classification and extraction rarely require the largest model. Reserve the powerful models for passages that really need it.
  • Memorization of repeated answers. If 30% of the questions are the same, there is no point in paying them every time.
  • Brief context. Send only relevant fragments to the model, not entire documents. Cheaper and usually more accurate.
  • Caps and alerts. A daily limit and an alert when exceeded save you from surprises.

The basic formula for the decision remains simple: hours saved per month multiplied by the cost of one hour, minus the monthly cost of operation. If the result is positive by a comfortable margin, move on. If not, you found that out cheaply before you built the whole thing.

A four-step plan

If I were to summarize the method in a route that you can follow:

  1. Choose a narrow process and write under what conditions you consider it successful.
  2. Collect real cases (100–200) and build the evaluation set before the first line of code.
  3. Release with human approval and full log, to a small group of users.
  4. Measure, correct, expand. Increase autonomy and scope only when the numbers justify it.

Conclusion

The difference between a demo and a trusted system is not a better model, but more engineering around it: clear purpose, neat data, repeatable evaluation, people where it counts, visible logs and costs. Those who treat AI as a serious component of a product get results that last. Those who treat it like magic get nice demos.