PII Protection

The pii module provides gateway-level privacy protection that filters outbound messages before they reach LLM API providers. The engine is tuned for precision — false positives directly degrade agent comprehension.

Design Philosophy

1. Precision over recall — False positives harm agent performance; the engine favors accuracy
2. Rule-based detection — Regex patterns for known PII categories, plus user-defined rules
3. Per-category action — Each category independently selects Block / Warn / Off
4. Platform overrides — A single platform can carry an independent rule set that overrides the defaults
5. Fail closed — A failed detector refuses to allow, instead of fail-open

Core Components

PiiEngine

The main filtering engine:

pub struct PiiEngine {
    rules: Vec<Box<dyn PiiRule>>,
    allowlist: PiiAllowlist,
    config: PrivacyConfig,
}

The engine exposes a global singleton through OnceLock (PII_ENGINE), injected at startup via PiiEngine::init; the runtime replaces configuration with PiiEngine::reload. process_outbound and process_inbound hold the engine through a synchronous RwLock<PiiEngine>, with a short critical section that does not cross await.

PiiSeverity

pub enum PiiSeverity {
    Low,
    Medium,
    High,
    Critical,
}

PiiAction

The per-category disposition:

pub enum PiiAction {
    Block,   // replace with placeholder (default)
    Warn,    // pass through but record
    Off,     // do not filter
}

FilterResult

pub struct FilterResult {
    pub text: String,           // filtered text with replacements
    pub blocked_count: usize,   // number of replacements
    pub warned_count: usize,    // number of warnings
}

Detection Rules

Built-in rules cover:

The api_key category also catches general Bearer tokens (see leak_detector.rs and vendor_patterns.rs).

Chinese ID Card Validation

Chinese ID card numbers are validated with:

• Length (18 digits)
• Province code (first 2 digits)
• Date (digits 7–14)
• Weighted checksum (last digit)

String slicing uses .get(..n) or char_indices() to honor UTF-8 boundaries instead of &s[..n].

Overlap De-Dup

The match set is sorted before replacement:

1. Block always wins — a Block hit must beat any Warn hit it overlaps, so a high-severity Warn cannot swallow a low-severity Block (e.g. api_key=warn suppressing phone=block)
2. Within equal block-ness, sort by severity descending
3. Dedup overlapping intervals
4. Sort by start descending and replace tail-first so earlier offsets stay valid

Custom Rules

[[privacy.custom_rules]]
name = "internal_token"
pattern = "IT-[A-Z0-9]{16}"
placeholder = "[INTERNAL_TOKEN]"
severity = "high"
action = "block"

build_rules(&config.custom_rules) compiles user regex at startup; a bad pattern is a fatal configuration error.

Allowlist

pub struct PiiAllowlist {
    entries: Vec<Regex>,
}

Entries are regex patterns matched against detected values before the replacement is applied — a hit skips replacement.

Platform Overrides

PrivacyConfig.platform_policies lets a single platform carry an independent rule set:

pub struct PlatformPiiPolicy {
    pub pii_filtering: Option<bool>,
    pub id_card: Option<PiiAction>,
    pub bank_card: Option<PiiAction>,
    pub phone: Option<PiiAction>,
    pub api_key: Option<PiiAction>,
    pub ssh_key: Option<PiiAction>,
    pub email: Option<PiiAction>,
    pub ip_address: Option<PiiAction>,
    pub exclude_providers: Option<Vec<String>>,
}

exclude_providers lets specific providers skip PII filtering entirely; the remaining fields override the default action inside filter_with_platform. effective_config clones the config and applies overrides, so platform policy never mutates the global configuration.

Fail Closed

A detector that fails internally (regex match / config load / state read) refuses to allow — any call that demands an "allow" verdict without a legitimate detection result is denied. This replaces the earlier fail-open behavior where detection failure would let the message through.

Tool-Call and Context Integration

RuntimeSecurityGuard::process_outbound slots the PII engine into the pipeline as: secret resolution → leak detection → PII filtering → placeholder substitution. Outbound messages pass through secret detection first (so the engine never has to handle freshly decrypted secrets again), then through the PII engine, and finally the placeholders are substituted.

process_inbound runs the same engine a second time before the LLM's response reaches the host, in case the LLM re-introduces sensitive material when echoing back.

Boundary-marker handling: prompt-context inputs (transcript / focus / prior-summary) are escaped before being placed inside boundary markers, so a forged marker cannot bleed into the next session.

ToolAwareChunker

ToolAwareChunker (src/context/compact/tool_aware_chunker.rs) chunks paragraphs for compression and recall:

pub struct ToolAwareChunker {
    pub chunk_token_limit: usize,
    pub token_ratio: f64,
}

A non-positive or non-finite token_ratio falls back to 0.25 instead of panicking — the previous assert! would crash the agent loop on a misconfigured caller (e.g. an MCP or Skill that injects a bad ratio). The chunker never splits a single SemanticUnit; an over-budget unit claims its own chunk.

Safety Properties

• UTF-8 safe — .get(..n) and char_indices() for all string slicing
• Lock recovery — unwrap_or_else(|e| e.into_inner()) for RwLock
• No static mut — OnceLock holds the global regex singleton
• Regex literal compile — expect("valid regex literal") for startup literals; runtime ratio errors fall back
• Conflict de-dup — Block always beats Warn to resolve overlaps

Code Location

• src/pii/mod.rs — module entry
• src/pii/engine.rs — core filtering engine
• src/pii/rules/ — detection rules (email / phone / bank / id / api_key / ssh_key / ip)
• src/pii/allowlist.rs — exemption management

See Also

• Security Primitives — runtime guard integration
• Secret Management — encrypted credential storage
• Configuration — PrivacyConfig and platform policies