Fail-Closed Safety Gates: Enforcing Hard Constraints in Safety-Critical AI

TL;DR — When a wrong recommendation can harm someone (an allergen, an age-restricted item, a drug interaction, a spend over a hard cap), safety is a hard constraint, not a preference. Enforce it with a fail-closed safety gate: one independent function every result passes through before ranking, which removes anything unsafe — and, when it cannot verify an item, excludes it (fail-closed) rather than letting it through (fail-open). Prove it with a contract test that no forbidden item is ever returned, even when the checker errors. This is different from an LLM guardrail: a gate is deterministic and must never be the model's job. It breaks the product's usefulness if the hard rule is wrong or over-broad — so keep the constraint narrow and the failure mode loud.

---

1. Simple explanation

Most recommendation logic optimizes for what a user will like. But some rules are not about liking — they are about harm: do not show a peanut dish to someone with a peanut allergy, do not surface an 18+ item to a minor, do not suggest a drug that interacts with one the patient takes. Getting these wrong is not a worse recommendation; it is a dangerous one.

The safe pattern is a gate: a single, independent step that every candidate must pass before anything else happens. The gate removes items that violate a hard rule. The subtle, important part is what the gate does when it is unsure — when the check errors, times out, or gets ambiguous data. Two choices:

• Fail-open — "not sure? let it through." Convenient, and occasionally fatal.
• Fail-closed — "not sure? drop it." Sometimes annoying (you hid a safe item), but never dangerous.

For safety-critical rules, fail-closed is the only correct default, because the cost of a false include (harm) dwarfs the cost of a false exclude (a missing option).

Analogy — the bouncer at a 21+ door. The bouncer checks IDs at the one door everyone uses. If a scanner breaks or an ID is unreadable, the rule is "you don't get in" — not "eh, probably fine." Being turned away by mistake is annoying; letting an underage person in is the failure that matters. The bouncer is the gate; "unreadable ID → refuse" is fail-closed.

---

2. Diagram

  candidates ──▶ [ rank by preference ] ──▶ show top N        ✗ UNSAFE DESIGN
                 (safety mixed into scoring; an unsafe item can
                  still win if its preference score is high)

  candidates ──▶ [ SAFETY GATE ] ──▶ safe set ──▶ [ rank ] ──▶ show   ✓ SAFE
                    │  one independent choke point, BEFORE ranking
                    │  drops anything unsafe
                    ▼
             unsure / error?
                fail-open  → keep it   (dangerous: unverified item shown)
                fail-closed→ drop it   (safe default)

  CONTRACT: for every result r returned,  is_forbidden(r) == False
            — and this must hold even if the checker threw.

---

3. How it works

3.1 Separate hard constraints from soft preferences

A hard constraint must never be violated (allergen, age gate, legal limit, budget cap). A soft preference is a want you can trade off (taste, price sensitivity, popularity). The whole design rests on keeping these apart: soft preferences are scored and ranked; hard constraints are enforced and are not negotiable. Mixing them — e.g. giving an unsafe item a small penalty instead of removing it — means a high enough preference score can still surface harm.

3.2 One independent choke point, before ranking

Route every candidate through a single gate function, and run it before ranking, pricing, or formatting. "Single" matters: if two code paths each do their own safety filtering, they will drift and eventually disagree, and you will not know which is authoritative. "Before ranking" matters: filtering the safe set first means the ranker can only ever order already-safe items, so no ranking bug can promote an unsafe one. The slogan is "proposers propose, the gate disposes" — retrievers, models, and agents may suggest items, but only the gate decides what is allowed through.

3.3 Fail-closed on uncertainty

The gate will sometimes be unable to decide: the allergen data is missing, an external check times out, an LLM tagger returns garbage. Define this path explicitly and make it exclude. A gate that silently returns an empty list on error is safe but confusing; better is to fail closed and say so ("couldn't verify safety — showing nothing" beats showing something unverified). Never let the "unsure" branch fall through to "include."

3.4 A gate is not an LLM guardrail

An LLM guardrail (prompt-injection filter, toxicity classifier, PII masker) is usually probabilistic — it reduces risk but can be wrong, and it lives around the model's text. A safety gate for hard constraints should be deterministic: structured input (item + its allergen tags + the user's constraints) → a yes/no decision by plain code, not by a model's judgment. Use an LLM to extract structured tags if you must, but the decision must be code you can test. If a model decides safety, you cannot guarantee the invariant.

Boundary condition. A gate is only as good as (a) the correctness of the hard rule and (b) the completeness of the data it checks. If the rule is wrong or over-broad it will hide safe items and gut the product; if an item is missing its allergen tags, fail-closed will (correctly) drop it, which can look like "the system found nothing." Section 8 covers when this pattern is the wrong tool.

---

4. The math (the invariant)

There is little arithmetic — the point is a logical invariant the system must always satisfy. Let F be the safe set the gate returns from candidates C, hard(u) the user's hard constraints, and viol(x, u) true iff item x violates them:

  F = { x in C : verified_safe(x, u) }          # gate output

  verified_safe(x, u) =
        True   if the check ran AND viol(x, u) == False
        False  otherwise            # includes: violation OR check errored/unsure
                                    #           (this "otherwise" IS fail-closed)

  INVARIANT (must always hold):   for all x in F,  viol(x, u) == False
  and it must hold even when the check raised on some items.

Contrast with a fail-open gate, whose "otherwise" branch is True — which lets an unverified item into F, breaking the invariant precisely when the checker was unreliable (the worst moment to relax).

4.x Worked example

Four items with tags; the forbidden tag is nuts. The checker errors on item B (say an external lookup times out).

  A tags=[nuts]           B tags=[dairy] (check ERRORS)   C tags=[]   D tags=[nuts,gluten]

• Fail-open keeps [B, C] — C is genuinely safe, but B was never verified and got in anyway. If B had actually carried nuts, that is a harm leak.
• Fail-closed keeps [C] — it drops A and D (real violations) and drops B (couldn't verify). Safe, at the cost of hiding B.
• The contract test on the fail-closed output passes: every returned item is nuts-free, and it held despite the error on B.

---

5. Real code

# A hard-constraint filter with a fail-CLOSED design.
# Rule: an item must NOT carry any forbidden tag. When the checker is unsure or
# errors, fail-CLOSED = DROP the item; fail-OPEN = keep it (the dangerous default).
items = [
    {"id": "A", "tags": ["nuts"]},
    {"id": "B", "tags": ["dairy"]},        # its check will error below
    {"id": "C", "tags": []},
    {"id": "D", "tags": ["nuts", "gluten"]},
]
forbidden = {"nuts"}

def is_safe(item, forbidden):
    if item["id"] == "B":                  # imagine an external checker timing out
        raise RuntimeError("checker unavailable for B")
    return forbidden.isdisjoint(item["tags"])

def filter_fail_open(items, forbidden):
    out = []
    for it in items:
        try:
            if is_safe(it, forbidden): out.append(it)
        except Exception:
            out.append(it)                 # unsure -> INCLUDE  (danger)
    return out

def filter_fail_closed(items, forbidden):
    out = []
    for it in items:
        try:
            if is_safe(it, forbidden): out.append(it)
        except Exception:
            pass                           # unsure -> EXCLUDE  (safe default)
    return out

print("fail-open  kept:", [it["id"] for it in filter_fail_open(items, forbidden)])
print("fail-closed kept:", [it["id"] for it in filter_fail_closed(items, forbidden)])

# CONTRACT: fail-closed must NEVER return a forbidden item, even under errors.
for it in filter_fail_closed(items, forbidden):
    assert forbidden.isdisjoint(it["tags"]), f"LEAK: {it['id']}"
print("contract holds: no forbidden item returned by the fail-closed gate")

# Output:
#   fail-open  kept: ['B', 'C']
#   fail-closed kept: ['C']
#   contract holds: no forbidden item returned by the fail-closed gate

The fail-open result quietly kept B — an item the checker never verified. That is the entire danger in one line. The fail-closed result kept only what it could prove safe, and the contract assertion guarantees the invariant.

---

6. Real-world example

A team shipped a recommender where allergen filtering was one feature inside the ranking score: unsafe items got a large negative weight. It demoed fine — unsafe items sank to the bottom. Then a popular item with incomplete tag data scored high on every other signal, the negative safety weight never fired (its allergen field was empty, not "contains"), and it surfaced in the top results for a user who had listed that exact allergy. Nothing "errored"; the safety signal was simply outvoted by preference signals, because safety had been modeled as a soft penalty instead of a hard gate.

The fix was structural, not a bigger penalty: pull allergen enforcement out of the ranker into a standalone gate that runs first, make missing tag data fail-closed (unknown allergens → exclude), and add a contract test that feeds the ranker adversarial high-score-but-unsafe items and asserts none are ever returned. The recurring lesson across safety-critical systems: if a hard rule is a term in a score, a high enough score will eventually break it — hard rules belong in a gate, not in the objective.

---

7. Interview questions companies actually ask

Q1. What's the difference between fail-open and fail-closed, and which is the safe default? Fail-open includes an item when the safety check is unsure or errors; fail-closed excludes it. For safety-critical constraints, fail-closed is the only correct default, because a false include can cause harm while a false exclude only hides an option — asymmetric costs demand the conservative branch.

Q2. Why enforce safety in a gate before ranking instead of as a scoring penalty? Because a penalty is a soft signal that a high enough preference score can outweigh, so an unsafe item can still win. A gate removes unsafe items first, so the ranker can only order already-safe candidates and no ranking bug can promote harm. Hard rules must be enforced, not weighted.

Q3. Why "one independent choke point"? If multiple code paths each filter for safety, they drift and eventually disagree, and no one path is authoritative — a change to one silently creates a gap in another. A single function that every result passes through is the only place you can test and reason about the invariant.

Q4. How is a safety gate different from an LLM guardrail? A guardrail is usually probabilistic and wraps model text (toxicity, prompt injection, PII); it lowers risk but can be wrong. A hard-constraint gate must be deterministic — structured input to a coded yes/no decision — because you need a guarantee, not a likelihood. An LLM may extract tags, but it must not decide safety.

Q5. Your gate returns nothing for some queries — is that a bug? Not necessarily; it may be fail-closed working (it couldn't verify anything, so it excluded everything). The bug would be returning something unverified. The right UX is to fail closed and say so ("couldn't verify safety") rather than silently empty or, worse, silently unsafe.

Q6. How do you test a safety gate? With a contract test: assert the invariant ("no returned item violates the constraint") holds across normal inputs and under injected failures (checker throws, missing tags, adversarial high-score-but-unsafe items). This proves safety, which is different from proving accuracy — a system can be accurate on average and still leak on the one case that matters.

Q7. When is fail-closed the wrong choice? When the constraint is not safety-critical and unavailability is itself costly — e.g. a non-critical personalization feature where showing a slightly-off item is fine but showing nothing hurts engagement. Fail-closed trades availability for safety; only pay that price where harm is the dominant cost.

---

8. When to use / tradeoffs

Reach for a fail-closed safety gate when:

• A wrong result can cause real harm (allergen, age gate, medical, legal, spend cap).
• You need a guarantee the constraint holds, provable by a test, not a model's confidence.
• Multiple sources (retrievers, LLMs, agents) can propose items and you need one place that decides what's allowed.

Do NOT use it (or soften it) when:

Honest limits. A gate guarantees the invariant you coded, not real-world safety: if the hard rule is wrong, incomplete, or the tag data is bad, the gate faithfully enforces the wrong thing (and fail-closed will over-hide). It trades availability for safety by design — expect more "nothing to show" moments, and make them explicit rather than silent. It cannot handle genuinely contextual judgments (where a deterministic rule is a poor fit) — those need human review or a probabilistic layer with escalation. And a gate protects only what flows through it: any path that bypasses the single choke point is an unguarded hole, so the architecture must force every result through it.

---

9. Summary + related articles

• Some rules are hard constraints (harm), not preferences — enforce, don't score them.
• A safety gate is one independent function every result passes through before ranking; proposers propose, the gate disposes.
• On uncertainty, fail-closed (exclude) — the cost of a false include (harm) dwarfs a false exclude (a hidden option).
• A gate is deterministic and testable; it is not an LLM guardrail, and a model must never decide safety.
• Prove it with a contract test: no forbidden item is ever returned, even when the checker errors.
• Boundary: it trades availability for safety and only guarantees the rule you coded — keep the rule narrow, the data complete, and force every path through the one gate.

Related:

• Guardrails & Output Validation — the probabilistic layer around model text, complementary to a deterministic gate.
• Hallucination Detection & Grounding — verifying model claims, another "trust but verify" safety layer.
• Fair Aggregation: Balancing Utility and Fairness — soft-preference ranking that should run after the gate has produced the safe set.

Resources

• Saltzer, J., Schroeder, M. (1975). "The Protection of Information in Computer Systems." Proc. IEEE — origin of fail-safe defaults (deny by default), the security principle behind fail-closed. (Venue confirmed; pages not verified here.)
• Nancy Leveson, Engineering a Safer World (MIT Press, 2011) — safety as a system property and the difference between reliability and safety. (Book; chapter not verified here.)
• Beyer, B. et al., Site Reliability Engineering (Google/O'Reilly, 2016) — graceful degradation and "safe" failure modes in production systems.