Building retrieval systems that survive information governance

“Ask questions of our own documents” is the most requested internal AI capability in almost every organisation we work with, and the one most likely to be quietly abandoned nine months later.

It is rarely abandoned because the technology did not work. It is abandoned because the pilot was built without the constraints that the production system would have to satisfy, and by the time information governance, the DPO or the security team looked at it properly, the fixes were structural.

Here are the eight decisions that determine whether it gets approved.

1. Permissions must be resolved at query time

This is the one that kills the most projects.

If your index contains documents that not every user is allowed to see, and your retrieval layer does not enforce that per user, per query, then you have built a system that will eventually surface an HR investigation to the person it concerns. The model does not know that the document was restricted. It only knows it was relevant.

The wrong approaches, in rough order of how often we see them:

• Indexing everything and hoping the prompt discourages disclosure.
• Filtering results after retrieval, in application code, as a bolt-on.
• Building one index per team, which fragments knowledge and drifts out of sync immediately.

The right approach is to carry the source system’s access control into the index as metadata, and filter before the model ever sees a candidate document, with the user’s identity, not the service account’s. This is meaningfully harder than the demo, and it is the difference between a prototype and a system.

Design it in from day one. Retrofitting per-user authorisation into a system built on a service account is close to a rewrite.

2. Answers must cite, and the citations must be checkable

An answer without a source is not usable in a regulated context, because nobody can verify it and nobody will take responsibility for it.

Every response should cite the specific documents it drew on, with links that open the actual source at the relevant place. This does three things at once: it lets users verify, it makes the failure mode visible rather than silent, and it changes user behaviour: people check a cited answer far more often than an uncited one.

It also gives you the audit trail. When someone asks in six months why a decision was made, “here is the answer and the four documents behind it” is a very different conversation from “the system said so”.

3. The corpus needs an owner and a lifecycle

Retrieval quality is mostly a content problem, not a model problem. The single largest source of confidently wrong answers we see is superseded documents left in the index. The 2019 policy and the 2026 policy are both retrieved, the model has no reliable way to know which governs, and it answers from whichever is more textually similar to the question.

You need someone accountable for what is in the corpus, a rule for what gets removed, and, at minimum, effective dates carried as metadata so recency can be weighted and staleness surfaced.

This is unglamorous and it matters more than the model choice.

4. Decide the data boundary before you choose components

If patient data, privileged material or special category data will reach the system in production, that constrains everything: which embedding model you can use, where the vector store lives, which inference endpoint is permissible, and what your logs may contain.

Choosing components first and discovering the constraint later is the expensive path, because embeddings are not portable between models: changing your embedding model means re-indexing the entire corpus.

Note the part that gets missed: the embedding step sends your document text to a model too. Teams who carefully self-host inference sometimes route the whole corpus through a hosted embedding API without noticing they have just exported everything.

5. Logs are a data protection problem

Prompt and response logs are enormously useful for improving the system, and they are a growing store of exactly the sensitive material you were being careful about.

Decide deliberately: what is retained, for how long, who can read it, is it in scope for subject access, and is it covered by your DPIA and your retention schedule. A logging store that nobody classified is a common and entirely avoidable finding.

6. Evaluate before you launch, and keep evaluating

Build a set of 100-200 real questions with known-correct answers, drawn from your actual corpus and reviewed by people who know the subject. Run it before launch and after every material change: new model, new chunking, new content source.

Without this you have no way to know whether a change helped, and no way to answer “how accurate is it?”, a question you will be asked by someone whose approval you need, and to which “it seems good” is not an acceptable answer.

Include questions the system should refuse: things outside the corpus, things the user is not entitled to see, things where the honest answer is “the documents do not say”. Refusal behaviour is a feature, and it is the first thing to degrade when someone tunes for helpfulness.

7. Design the failure mode

The system will be wrong sometimes. Design for that rather than hoping.

Practical measures: display confidence honestly, make “I could not find this in the documents” a first-class outcome rather than a fallback, put a visible route to report a bad answer next to every response, and make it clear in the interface which decisions must not be made on the system’s output alone.

An assistant that admits uncertainty is trusted more, not less, and it is far easier to defend to an assurance panel than one that never says it does not know.

8. Plan the handover from the first week

If the system only runs because one contractor understands it, you have bought a dependency rather than a capability.

That means: infrastructure as code, a documented re-index procedure, the evaluation set in the repository alongside the code, a runbook for the common failure modes, and named internal owners for the corpus, the infrastructure and the evaluation.

We treat this as a deliverable rather than a courtesy, and it is the part clients tell us later mattered most.

The pattern behind all eight

None of these are model problems. They are systems, governance and content problems, which is exactly why a pilot built to demonstrate capability tells you almost nothing about whether the production system will be approved.

If you are commissioning this work, the useful question is not “can you build a chatbot over our documents”. Almost anyone can, and the demo will be impressive. The useful question is “how will this enforce per-user permissions, and what will you show information governance?”

The answer to that one separates suppliers quickly.

---

This is the design standard we build to in AI implementation, on the infrastructure decided in private and self-hosted AI.