Multi-tenant RAG: relevance is not access control
Vector search ranks by similarity, not permission. Why multi-tenant RAG leaks across tenants by default, and where the authorization check actually has to sit.
Nearest-neighbour search answers one question: which chunks are most similar to this query. It has no opinion about who is asking. In a single-tenant system that distinction never comes up. In a multi-tenant one it is the whole problem, because the top result for a customer's question can be another customer's document — not through a bug, but because the index did exactly what it was built to do.
A May 2026 paper from Red Hat AI gives this a name: the relevance-authorization gap. Their framing is blunt — "a query from one tenant can surface another tenant's confidential data simply because it scores highest". Across their probes, ungated retrieval leaked cross-tenant data in 98–100% of cases; the gated configurations leaked in none.
I spent six years before this building insurance distribution platforms where per-carrier authorization was the product — who may see which producer, which licence, which compensation record. Retrieval systems tend to be built by people who have never had to answer that question, and it shows in the defaults.
Post-filtering the top-k is not access control
The intuitive fix is to retrieve normally and drop what the caller isn't allowed to see. It's one line, it's easy to review, and it's wrong in a way that doesn't announce itself.
Ask for the top 10 and filter afterwards, and you don't get "the top 10 the user may see." You get however many of those 10 survive the filter — sometimes three, sometimes zero, and never with an error. The user asks a question their own documents answer perfectly well and the system says it doesn't know, because the ten nearest chunks belonged to someone else. Retrieval quality quietly becomes a function of how many other tenants share your index.
Then there's what happened before the filter ran: another tenant's text was fetched, sat in your process, and probably went to your logs. If the filter is in application code, the data left the database. That's the difference between a system that doesn't show cross-tenant data and one that never retrieves it.
The check has to constrain the search, not the results.
Where the check actually has to live
Two places, and both are needed.
At ingestion, every chunk carries the tenant it belongs to, mirrored from whatever system already owns that fact. Not inferred from the document, not from the folder it arrived in — copied from the source of truth. A chunk whose tenant is unknown never enters the index.
At query time, the tenant becomes a predicate the vector store evaluates as part of the search — a metadata filter, a namespace, a WHERE clause pushed down alongside the similarity operator. The paper's structure is worth copying here: policy-aware ingestion, retrieval gating, then shared inference, with authorization decided server-side rather than by whatever is calling in.
That last part matters more than it sounds. If the tenant id arrives as a parameter from the client, you have not implemented access control — you have implemented a suggestion. It comes from the authenticated session, server-side, or it isn't a control at all. The same paper measured client-side ungated configurations leaking on 80% of prompt-injection probes; when the model is choosing the filter, the model can be talked out of it.
The part nobody plans for: permissions change
Ingestion-time metadata has an expiry problem. Someone leaves a team, a contract ends, a document is reclassified — and the vectors still carry what was true on the day they were written.
There are two honest answers.
Authorize at query time against live data. The index stores a stable identifier; the permission check happens against the system that currently owns the answer. Always correct, and it puts a lookup on the hot path.
Re-index on permission change. Fast reads, and now you own an invalidation pipeline — every ACL change anywhere becomes a re-embed job somewhere, and the failure mode is silent and stale in the direction you least want.
For coarse boundaries that rarely move — a tenant, a customer account — metadata on the chunk is fine, and re-indexing is rare enough to be boring. For anything finer, or anything with genuine revocation semantics, check it live. The mistake is picking the cheap option for a boundary that turns out to be fine-grained after all, and only finding out at an audit.
The related choice — namespace per tenant or one shared index with a filter — is mostly operational rather than a security decision, provided the predicate is enforced server-side either way. Namespaces give you a harder blast radius and a worse time at a thousand tenants; a shared index scales comfortably and puts all your weight on one predicate being right everywhere. Pick deliberately, then write down which one you picked and why.
Test it like an attacker, not like a user
The failing case looks identical to the working one from the outside: results come back, they're plausible, they're relevant. Nothing throws.
So test it directly. Take a real query from tenant A, run it authenticated as tenant B, and assert the result set is empty — not "assert it looks right," assert it is empty. Do it for a document you know scores highly, because a query that wouldn't have matched anyway proves nothing. Run it in CI on every change to the retrieval path, and again after any change to how chunks are written.
Then check your logs and traces for chunk text, because that's the other place tenant data escapes — retrieved correctly, then written somewhere with a much weaker access model than the database it came from.
If you're standing up retrieval over customer data and this is the part that isn't specified yet, that's the work I do.