Most AI projects that disappoint were scoped as "add a chatbot". The ones that work are scoped as "answer this specific category of question from this specific set of documents, and show where the answer came from".
That second thing is retrieval-augmented generation, and the mechanics are simpler than the acronym suggests.
The mechanism, in four steps
- Prepare. Your documents are split into passages and indexed, usually as embeddings — numeric representations that let the system find text by meaning rather than by exact keyword.
- Retrieve. A question is embedded the same way, and the closest passages are pulled back. Good systems also run a keyword search and merge both sets, because pure semantic search misses exact terms like product codes.
- Ground. Those passages are given to the model with an instruction to answer from them and to say so when they do not contain the answer.
- Cite. The answer is returned alongside links to the source passages, so a human can check it in one click.
The model supplies fluency. Your documents supply truth. When people say an assistant "hallucinated", what usually happened is that step two returned nothing useful and step three did not insist on refusing.
Where it genuinely helps
- Support and documentation. Answering from manuals, policies and past tickets, with the source attached.
- Internal knowledge. The questions new staff ask in their first month, answered from the documents that already exist.
- Long or structured documents. Contracts, specifications, regulations — where the answer exists but finding it takes twenty minutes.
- Triage. Classifying and routing incoming work, with the relevant context summarised for whoever picks it up.
Where it does not
If the answer is a database lookup, write the query. If the rules are fixed, write the rules. If the content is wrong or out of date, a model will repeat it faithfully — retrieval is a search over what you have, not a fact-checker.
The most common blocker is not technical: the documents are scattered, contradictory or stale, and nobody wants to own them. That is a content problem, and it has to be solved either way.
How these fail, and the controls
| Failure | What it looks like | The control |
|---|---|---|
| Bad retrieval | Confident answer from an unrelated passage | Hybrid search, re-ranking, tuned chunk sizes |
| No refusal | Invents an answer when nothing was found | Instructions plus a relevance threshold, and a real "I don't know" |
| Stale content | Correct-sounding, out of date | Re-indexing on a schedule, visible document dates |
| Leakage across permissions | One user sees another's content | Filter retrieval by the user's permissions, not just the interface |
| Cost drift | The bill grows faster than usage | Caching, smaller models, fewer retrieved passages, monitoring |
Evaluation is the part that gets skipped
Before launch, assemble fifty to a hundred real questions with the answers you expect, and run them against every change. Without that set, "it seems better" is the only available verdict, and every prompt tweak is a coin flip.
It is also what tells you when a cheaper model is good enough — usually the largest single lever on running cost.
What we build
Scoped around one task, grounded in your content, evaluated against real examples, deployed in your own infrastructure with keys in your accounts, and costed so the monthly bill is predictable.
That is AI development here. If the assistant has to live inside an existing product, that is software development with a model in the middle — tell us what the task is and we will tell you whether it needs one at all.
Frequently asked questions
- What is retrieval-augmented generation?
- A pattern where the system searches your own content for passages relevant to a question, then asks a language model to answer using only those passages. The model supplies the language; your documents supply the facts.
- Will it stop the model making things up?
- It reduces it substantially, and does not eliminate it. Grounding in real passages, citing the source next to the answer, and refusing to answer when retrieval returns nothing relevant are the three controls that matter most.
- How much does it cost to run?
- Model usage is billed per token, so cost scales with traffic and with how much context you send. Caching common questions, retrieving fewer and better passages, and using the smallest model that passes evaluation are where the order-of-magnitude differences come from.
- How long does it take to build?
- A focused internal assistant over a defined document set is usually a three-to-six week build including evaluation. The variable is rarely the model — it is the state of the documents.
- ai
- rag
- automation
