Back to the blogCitește în română

RAG for business owners: AI that works with company documents

What is RAG and how does an AI assistant answer from the company's contracts and procedures: suitable documents, GDPR, cost structure and calculation formula.

A librarian searches for relevant excerpts in internal documents to answer a source question. Access is controlled and costs come from indexing and usage.

ChatGPT writes well, but knows nothing about your company’s contracts, procedures and offers. RAG is the method by which an AI assistant first searches your documents and only then responds, showing where it got the information. In this article I explain, without unnecessary technical terms, how it works, what documents are suitable for it, what it means for privacy, what the cost consists of and how you can calculate for yourself if it is worth it.

In short

  • RAG is an assistant that answers from company documents and indicates the source of each answer.
  • The model does not “learn” the documents. He consults them on every question. You change the document, the answer changes too.
  • The quality of the answers depends more on the order in the documents than on the model chosen.
  • Data can stay on a server you control, and access is role-based.
  • It’s worth it when multiple people search the same documents often. Below is a simple calculation formula.

Why “plain” ChatGPT knows nothing about your business

A linguistic model was trained on public texts, collected up to a certain date. Your internal documents are not among them. When you ask him what your return policy is, he has two options: say he doesn’t know, or compose a plausible-sounding answer. The second option is the dangerous one, because it looks just like a correct answer.

The handy solution is to paste the contract into the chat window. It has a few problems:

  • Not sure what happens to the text. Depending on the account type and settings, conversations may be kept by the provider and used to improve the service. Check the terms, don’t assume.
  • The contract is not just yours. It often has a confidentiality clause and other people’s personal data. If you send it to a third party without grounds and without a processing contract, the liability remains with you.
  • No trace remains. Each employee pastes what he sees fit, from his account, and no one knows what has left the company.
  • It doesn’t last long. You paste the same document every time, and with very long texts the details can be lost.

What is RAG, with an analogy

RAG stands for “Retrieval-Augmented Generation”, i.e. Search-Augmented Response Generation.

Consider two new employees. The first answers any question from memory, sure of himself, even when he does not know. The second is a librarian. When you ask him something, he goes to the archives, pulls out the right files, reads the pages that matter, and only then answers: “The payment term is 30 days, according to the framework contract, page 4.” If it doesn’t find anything, it tells you that it didn’t.

RAG turns the first employee into the second. The steps in brief:

  1. Documents are broken up into fragments and entered into an index, a kind of library catalog that searches by meaning, not just exact words.
  2. When someone asks a question, the system searches for the fragments closest to it.
  3. The model receives the question along with the found fragments and the instruction to answer only from them.
  4. The answer comes with the source: the document and the place in it.

Two useful things follow from this. The model is not retrained on your data, so nothing needs to be redone when a procedure changes: you update the document and that’s it. And you can always check the answer by opening the indicated page.

Which documents go well and which don’t

Document type How he behaves What is to be done
Procedures, regulations, manuals written as text Very good Keep only one valid version
Contracts and offers in PDF with selectable text Good Name files clearly and mark what has expired
Frequently asked questions, type answers to emails Very good Collect them in one place
Poor scans, phone pictures, handwriting Weak Rescan or use human-verified text recognition (OCR).
Complicated tables, price lists with merged cells Risky: Numbers can get mixed up It is safer to have the assistant read the file or database directly, not the PDF
Conflicting information between documents The assistant can quote any of the options Decide which one is valid and archive the rest

A simple rule of thumb: If a smart new employee couldn’t handle just that document, the assistant won’t either.

Privacy and GDPR, of course

Four questions clear up almost everything.

Where does the data live? There are two places to watch: where the documents and index are kept, and where the model that formulates the answer runs. Both can be in the European Union. There is also the private hosting option, where the model runs on your server and the texts don’t go anywhere. It comes at a cost: you need a more powerful server, and models that can be run locally can be weaker on complicated queries.

Who sees them? The hosting provider, the model provider (if external) and the system administrator. With each you need a data processing contract, as required by Article 28 of the GDPR, which makes it clear that the data is not used for other purposes, including model training.

Who in the company has access to what? Role-based access means that the assistant only searches the documents that the questioner has the right to see. A sales colleague is not getting answers from payroll. This rule is established from the beginning, it is not added afterwards.

What traces remain? A log shows who asked, what they asked, and from which document the answer came. It is useful for checks. Being a dataset itself, it needs a retention period.

Two clarifications. RAG does not exempt you from the obligations you already had: if the documents contain personal data, you still need documents, information and retention rules. And I’m not a lawyer. For sensitive data, such as medical or lawyer records, speak to your Data Protection Officer before you begin.

How much does it cost and what does the cost consist of?

The cost has four parts. Ask for them separately in any offer.

  1. Commissioning, paid once: connecting documents, roles, rules, testing. For a private RAG server, the duration for me is 5–10 business days. As a benchmark, the custom apps I build start at €1,200. The exact price depends on how many sources there are, in what condition and how many roles are needed.
  2. Hosting, monthly: the server where the documents and index reside. It is smaller if the model is external and larger if the model is running on you.
  3. Consumption, monthly and variable. Models are paid in tokens, i.e. bits of word read and written. It grows with the number of questions and can be capped.
  4. Maintenance, monthly: updating sources, periodic checks, model changes. For me, support starts at €250 per month.

I do not give figures for hosting and consumption. They depend on volume and change often, and a number written here would be either outdated or made up.

If you’re not sure you need your own solution, we’ve compared the options in Pre-configured AI solutions or development from scratch.

How do you measure if it’s worth it?

It tracks three things: hours saved searching, time spent responding to a customer, and errors avoided. The first can be calculated simply.

Ore economisite pe lună = persoane × căutări pe zi × minute câștigate la o căutare × zile lucrătoare ÷ 60
Valoare lunară          = ore economisite × costul unei ore de muncă
Câștig net lunar        = valoare lunară − cost lunar de funcționare
Luni până la recuperare = cost de punere în funcțiune ÷ câștig net lunar

Calculation example (illustrative, not an actual result). A company with 6 people searching documents 5 times a day and earning 5 minutes for each search, in 21 working days: 6 × 5 × 5 × 21 = 3,150 minutes, which is 52.5 hours per month. At an assumed cost of €10 per hour, the value is €525. For the cost I take the starting thresholds above: €250 per month and €1,200 at the start. The net profit is €275 per month and the investment is recovered in just over four months.

Change one digit and the conclusion returns. With 2 searches per day instead of 5, I get 21 hours, that is €210, under the monthly cost. That is precisely why it is worth doing the calculation with your figures, measured for a week, not with those in the offer.

Response time and errors avoided do not easily fall into a formula. Write them down separately: an offer sent with an old price or a missed contract term can weigh more than all the hours saved.

Limitations and common mistakes

Limits, first:

  • Can be wrong. Sometimes finds the wrong fragment or gets it wrong. That’s why the displayed source matters. For important decisions, open the document.
  • Does not do totals. “How many contracts expire this year?” it’s a query for a database, not a text search. It can be solved, but with another tool.
  • Does not replace specialist. Find clause. It doesn’t tell you if it’s okay to sign.

Mistakes you can avoid:

  • Upload all company documents, without sorting. Old and contradictory ones spoil the answers.
  • No one is responsible for the update. After a few months, the assistant cites procedures that no longer apply.
  • You don’t have a set of test questions. Without it, you have no way of knowing whether a change improved or broke something.
  • Give everyone the same access, “to keep it simple.”
  • You get him to talk directly to customers without any other measures. That’s a different project, described in Chatbot AI for Your Business: Costs and Mistakes.

Checklist: Are you ready for RAG?

  • At least three to four people often search the same documents.
  • Important documents exist in digital format with selectable text.
  • For each type of document, the valid version is known.
  • You can appoint a person who is responsible for updating the sources.
  • You know who is allowed to see what.
  • You can write 20–30 real questions with correct answers for testing.
  • You have decided where the data can be hosted: in the cloud, in the European Union, or on the company’s server.
  • You have a monthly budget for operation, not just installation.

If most of them are true, you’re good to go. If not, the first step is to get your documents in order. It helps you anyway, with or without AI.

How I work

RAG Private Server is one of the pre-configured AI solutions that we put into operation for small businesses: the architecture is ready, and the work goes into connecting your documents, roles and testing. I always start with a single use case and a set of real questions. If it turns out from the discussion that your problem can be solved more simply, I will tell you.

If you want to know what it would look like on you, write me for a 20 minute chat. Give two or three examples of questions that your people are looking for answers to in folders today.

Sources