Guardrails

TL;DR: Guardrails are checks wrapped around the model at runtime. Input guardrails validate the request before it reaches the model: detect prompt injection and jailbreaks, off-topic or disallowed content, and PII. Output guardrails validate the response before it reaches the user: content safety, schema/format validation, grounding (no unsupported claims), and PII/secret leakage. They are built from rules, classifiers, judge models, and validators, with a defined fail-safe action (block, regenerate, redact, fall back) when one trips. Guardrails are imperfect, so layer them and keep monitoring.

A layer around the model

The model itself is not safe by default, so you wrap it: the request passes input guardrails, the model runs, the response passes output guardrails. This layer is where safety, policy, and format reliability are enforced, independent of the model.

Input vs output checks

• Input guardrails: detect prompt injection/jailbreak attempts, filter off-topic or disallowed requests (a support bot asked for medical advice), strip or flag PII, and reject malformed/oversized input. Treat retrieved and tool content as untrusted.
• Output guardrails: check generated text for unsafe/policy-violating content (a classifier or judge), validate the schema/format (the constrained-decoding discipline), verify grounding (claims supported by sources, to catch hallucination), and ensure no PII or secret leakage (including the system prompt). Apply business rules.

Fail-safe behavior

Decide in advance what happens when a guardrail trips: block with a safe message, regenerate, redact, or fall back to a canned/safe response, never silently pass flagged output. And accept that guardrails are imperfect (false positives over-block, false negatives slip through), so layer them (rules plus classifiers plus a judge plus validators), keep latency in check, and keep red-teaming and monitoring. For agents, guardrails extend to actions (see agent guardrails).

Why interviewers probe this

"Add guardrails" is a common hand-wave, so the test is whether you can name the concrete checks and the fail-safe behavior. A strong answer splits input (injection/topic/PII) from output (content safety, schema, grounding, leakage) guardrails, describes how they are built (rules, classifiers, judges, validators), and defines what happens on a trip, then notes they are imperfect so you layer and monitor. That turns "be safe" into an actual design.

Common misconceptions

• "Add guardrails" is a plan. It needs concrete input/output checks and a defined fail-safe action.
• "Only output needs checking." Input guardrails (injection, topic, PII) matter as much as output checks.
• "Guardrails make the system safe." They are imperfect layers; combine them, red-team, and monitor.
• "The model handles its own safety." Runtime guardrails are a separate, necessary layer around the model.

Key takeaways

• Guardrails are a runtime layer: input checks (injection, topic, PII) before the model, output checks (safety, schema, grounding, leakage) before the user.
• Build them from rules, classifiers, judge models, and validators.
• Define a fail-safe action on a trip (block, regenerate, redact, fall back); never pass flagged output silently.
• They are imperfect, so layer them, manage latency, and keep red-teaming and monitoring; for agents, guard actions too.