Hygiene (PII & sensitivity)

factflow-hygiene is a governance layer that sanitizes content before it reaches downstream synthesis. It holds the residency invariant — no Norwegian PII leaves the EU boundary in a prompt — then classifies sensitivity, gates broken-access-control content, and blanks strictly-confidential bodies. Two of its four adapters are deterministic (PII and SC redaction); two are LLM-based (sensitivity classification and the anti-BAC gate), and they all run behind the PII redactor by design.

The four adapters, in order

They run as a fixed sequence in a route, ahead of knowledge_diff — the order is wired in scribe-knowledge-synthesis.yaml:

1. pii_redactor — deterministic, regex/pattern-based detection and redaction of personally identifiable information (Norwegian national IDs, IBANs, …). Runs first so the residency invariant holds before anything is sent to a model.
2. sensitivity_classifier — LLM-based: classifies the already-redacted content with a sensitivity label via an llm_profile, caching findings by content hash + model profile. Runs after pii_redactor so the model never sees raw PII.
3. anti_bac_gate — LLM-based: detects org-wide-invariant (broken-access-control) violations and, when found, replaces the body in place with [REDACTED:BROKEN_ACCESS_CONTROL]. Findings are cached by content hash + model profile.
4. sc_redactor — deterministic: when the resolved label is STRICTLY_CONFIDENTIAL, blanks the body so SC content never reaches synthesis; the label itself survives.

storage_retriever → pii_redactor → sensitivity_classifier → anti_bac_gate → sc_redactor → knowledge_diff

The exact config fields for each adapter live in the Adapter catalog, and the package internals in factflow-hygiene.

The governance model: three orthogonal axes

The DNB Kunnskap corpus rests on one invariant: everything in it is org-wide knowledge — material every employee may rightfully see. Hygiene enforces that invariant along three independent axes, which is why there are separate adapters rather than one classifier:

• PII (pii_redactor) — structured personal tokens: national IDs, account numbers, IBANs. Deterministic, token-level.
• Sensitivity (sensitivity_classifier) — a business-harm tier (public → strictly-confidential). A soft scale that shapes retrieval; plenty of confidential material is still rightfully org-wide.
• Broken access control (anti_bac_gate) — an audience-scope violation: a specific identifiable party (a person, or a corporate customer) tied to disclosure-harm material, inside a corpus that promised to hold none of it. A hard alarm, deliberately never derived from the sensitivity tier.

Anti-BAC exists because the first two axes left a gap. The Avalon v0.1.0 run surfaced SharePoint sites set public instead of private — whole units with broken access control — whose content slipped past both PII redaction (no token to match) and sensitivity tiering (never labelled at all). If such content ever reaches the pipeline, the invariant was already broken upstream, and that contradiction is itself a detectable, high-precision signal. Full rationale: issue #355 design.

Why two are deterministic and two are LLM-based

Redaction and residency are correctness guarantees, not best-effort. pii_redactor and sc_redactor are rule-based and reproducible — the same input always yields the same redaction, a property you cannot get from a generative model in the loop. The judgment calls — is this sensitive?, does this leak access control? — genuinely need a model, so sensitivity_classifier and anti_bac_gate call an LLM. Both run only after PII is stripped, and both cache their verdict by content hash + model profile so a given input maps to a stable outcome across runs.

Findings as governance records

Each axis writes a per-source findings row, keyed by (origin, source_ref) and content-hash cached so weekly re-runs and replay stay cheap and deterministic:

Table	Carries
pipeline_pii_findings	origin, source_ref, finding type (fnr, kontonummer, iban, …)
pipeline_sensitivity_findings	label, confidence, rationale, model profile
pipeline_anti_bac_findings	the violation verdict — party + offending fact

These rows are the audit trail: every gate decision traces back to the upstream document that triggered it. They are projected into the dnb-avalon export so a reviewer with only the exported repo can answer "why was this redacted?" — see Knowledge & Avalon synthesis → Source-level findings.