TL;DR: Enforce isolation at retrieval, server-side, never in the prompt. Scope every query to the tenant (and the user's document permissions) before the vector search runs, using metadata filters or per-tenant indexes, so unauthorized chunks are never retrieved and therefore never reach the model. Mirror the source system's ACLs, keep them fresh, and audit every access. The LLM should only ever see content the requesting user is already allowed to read.
How to approach it. Frame the threat: in shared RAG, the danger is one user (or tenant) getting an answer grounded in another's documents, or in documents within their company they lack permission for. State the cardinal rule: filter at retrieval, server-side, before generation. Then cover index strategy, permission propagation, and freshness.
A strong answer. The model is downstream of retrieval, so access control must happen at retrieval, not by asking the LLM to "only use authorized content." If an unauthorized chunk is retrieved, it can leak into the answer regardless of instructions. The retrieval service is the security boundary, and the filter is derived from the authenticated session, never from anything the client can set.
- Tenant isolation. Partition by tenant: either separate indexes per tenant (strong isolation, simpler reasoning, more overhead) or a shared index with a mandatory
tenant_idfilter applied server-side on every query (efficient, but the filter must be unbypassable and enforced in code, not the prompt). For sensitive customers, dedicated indexes or deployments. - Document-level permissions. Within a tenant, mirror the source system's ACLs: store each chunk's allowed groups/users as metadata and filter retrieval to what the requesting user can access (a pre-filter on the ANN query, or post-filter with enough over-fetch). The retrieval result must be exactly the set of chunks this user could open directly in the source app.
- Permission freshness. ACLs change (someone leaves a project); stale permissions are a leak. Sync permission metadata from the source on a tight cadence (or check at query time for high-sensitivity data), and handle deletions so revoked access takes effect quickly.
- Defense in depth. Enforce the filter in the retrieval service (not the client), validate the
tenant_idfrom the authenticated session (never from a request parameter the client sets), strip cross-tenant data from caches, and audit-log every retrieval (who queried, what was returned) for compliance and incident response.
The honest framing: treat retrieval as the security boundary. If the filter is correct, the generator physically cannot ground an answer in data the user is not entitled to.
Key takeaways
- Access control lives at retrieval, server-side; a prompt instruction to "respect permissions" is not a control.
- Derive
tenant_idand ACLs from the authenticated session, never from client-supplied parameters. - Prefer pre-filtering the ANN query over post-filtering, which avoids leaking ranking signal and undersized result sets.
- Stale ACLs are a leak: sync on a tight cadence, and check live for the most sensitive data.
What interviewers probe next.
- "Per-tenant index vs shared index with filters?" Per-tenant: stronger isolation, more operational overhead, weaker cross-tenant reuse. Shared + filter: efficient but the filter is now safety-critical and must be unbypassable. Choose by sensitivity and scale.
- "Pre-filter vs post-filter on ANN?" Pre-filter is safer and avoids leaking ranking signal; post-filter needs over-fetch and risks returning too few results. Many vector DBs support metadata pre-filtering.
- "How do you keep ACLs in sync?" Stream permission changes from the source of truth; for the most sensitive data, check live at query time rather than trusting cached metadata.
- "Caching across tenants?" Key caches by tenant/user scope; never serve a cached answer across permission boundaries.
Common mistakes.
- Asking the LLM to respect permissions via the prompt instead of filtering at retrieval.
- Trusting a client-supplied tenant id rather than the authenticated session.
- Ignoring permission freshness, so revoked users still retrieve old documents.
- No per-tenant cache keys, leaking answers across customers.