TL;DR: Treat governance as part of the architecture. Minimize and redact PII before it reaches the model, isolate each tenant's data, never train on customer data without explicit consent, log every access for audit, honor deletion and residency requirements, and map controls to the regime: SOC 2 for security posture, GDPR/DPDP for personal-data rights, the EU AI Act for risk-tiered obligations. Assume you must prove all of it to an auditor.
How to approach it. Frame it as defense and accountability across the data lifecycle (ingest, process, store, serve, delete), not a single feature. State the cardinal rule first: do not train on customer data by default. That is the thing enterprises actually care about in security review. Then walk the controls.
A strong answer. Across the lifecycle:
- Data minimization and redaction. Detect and redact or tokenize PII (names, emails, IDs, PHI) before it enters prompts, logs, or any retrieval index. Pass only what the task needs. PII that never reaches the model cannot leak from it.
- Tenant isolation. Separate each customer's data and retrieval indexes, and enforce tenant scoping server-side so one customer can never be answered from another's data. For sensitive customers, offer dedicated or single-tenant deployments and private networking (VPC, sometimes air-gapped).
- No training on customer data by default. The default contract is that customer inputs and outputs are not used to train models. Any exception is explicit, opt-in, and documented. This is usually the first thing security review asks about.
- Auditability. Log who or what accessed which data, and every model interaction, immutably, so you can answer "who saw this record and when." Audit trails are central to SOC 2 and to incident response.
- Data-subject rights and lifecycle. Support deletion and access requests (GDPR/DPDP): you must be able to find and purge a person's data across caches, logs, and indexes, which is why you track lineage. Set retention limits and honor data residency (keep EU data in-region).
- Guardrails at the boundary. Output filtering to block PII or secret echo, prompt-injection defenses on retrieved content, and consumption controls to prevent abuse.
The honest framing: architect so that if a component is compromised, the exposed data is minimized, scoped, and traceable, and you can prove the controls to an auditor instead of asserting them.
Knowing which regime drives which control, and designing to the strictest one that applies, is what separates an architect from someone reciting acronyms:
| Regime | What it governs | What it forces into the design |
|---|---|---|
| SOC 2 | Your security posture and evidence | Access control, monitoring, change management, audit logs |
| GDPR / DPDP | Individual rights over personal data | Consent, access, erasure, purpose limitation, residency |
| EU AI Act | Risk-tiered obligations by system class | Transparency, human oversight, documentation scaled to risk |
Key takeaways
- Minimization is the strongest control: PII that never reaches the model or the logs cannot leak from either.
- Tenant scoping must be enforced server-side, not by prompt convention, or a retrieval bug becomes a cross-customer breach.
- Erasure is the trap with fine-tuning: you cannot surgically remove one record's influence from weights, which argues for RAG over fine-tuning on personal data.
- Auditors want evidence (logs, lineage, DPAs), not assertions; build the proof trail before they ask.
What interviewers probe next.
- "How do you redact PII reliably?" A detection layer (regex patterns plus ML NER) with tokenization or pseudonymization. Accept it is imperfect, so pair it with access controls and minimization rather than trusting detection alone.
- "Right to be forgotten with a fine-tuned model?" Hard: you cannot surgically delete one record's influence from weights. This is a strong argument for RAG over fine-tuning on personal data, plus deleting from the index and logs and documenting the limitation.
- "Data residency?" Region-pinned storage and in-region inference; pick model providers and regions that support it.
- "How do you prove compliance?" Logs, lineage, documented controls, and DPAs. Auditors want evidence, not claims.
Common mistakes.
- Treating governance as a post-launch checkbox instead of an architectural constraint.
- Logging full prompts and outputs with PII intact, creating a fresh breach surface.
- Fine-tuning on customer PII, then being unable to honor a deletion request.
- No tenant isolation, so a retrieval bug becomes a cross-customer data leak.