Process Supervisor
PTY-based process control for external CLI tools, and the Turn-level verifier chain plus shell stop hooks for the agent loop.
The gateway/pty and verification modules split external process control and agent-loop verification: the gateway side runs an interactive CLI tool inside a pseudo-terminal (embedded terminal), and the verification side equips the agent loop with a Turn-level verifier chain and shell stop hooks. All verdicts remain LLM self-checks plus hook receipts — the code never substitutes its own completion judgment (R7/R10).
Design Philosophy
- PTY-embedded terminal —
gateway/pty/wraps a local shell /cmd.exeviaportable-pty; output is pushed over the WebSocketpty.output/pty.exittopics (no second port). - Turn-level verification —
verification::VerifierChainruns between Think and Act withStopHookVerifier(shell hooks),ToolLoopVerifier(death-loop watchdog),ExtensionStopHookVerifier(HookEvent::Stopextension hooks),ScratchpadGoalVerifier(block stop while the execution list has unchecked items), andMutationEvidenceVerifier(a one-shot nudge when a file mutation has no post-edit evidence). - Read-only shell hook —
ShellStopHookruns an external command; exit 0/2/3 = pass/block/halt; everything else (other exit codes, timeouts, signals, shell metacharacters) maps toError, leaving fail-open vs fail-closed to the caller.
Embedded Terminal (src/gateway/pty/)
pty/
├── mod.rs # entry + public re-exports
├── session.rs # PtySession: portable_pty master/child wrapper, Send + Sync
├── manager.rs # process-global PtyManager: 64-session cap, TTL + FIFO eviction
└── handler (in gateway/handlers/pty.rs)PtySession::spawn(opts) builds a portable_pty::native_pty_system() session, clones the writer / reader / killer halves, and pushes base64 pty.output frames onto the gateway event bus; PtyManager is a process-global singleton (OnceLock), and concurrent pty.spawn calls that overflow the 64-session cap evict the oldest entry. The RPCs are pty.spawn / pty.input / pty.resize / pty.close / pty.list; the streaming pty.output / pty.exit events are delivered via events.subscribe("pty.*").
Verifier Chain (src/verification/)
VerifierChain is an Vec<Arc<dyn TurnVerifier>> + AtomicBool kill-switch built during Orchestrator::build; the verifiers run in order between Think and Act, and the first non-Continue verdict wins:
pub struct VerifierChain {
verifiers: Vec<Arc<dyn TurnVerifier>>,
enabled: AtomicBool,
}Default chain (order = precedence):
| Verifier | Triggered when | Verdict | Failure semantics |
|---|---|---|---|
StopHookVerifier | stop_reason.is_some() | Runs the ShellStopHook set; exit 2 → Veto (retry), exit 3 → Halt (exit loop), other → Error | Harness side fail-open, goal side fail-closed |
ExtensionStopHookVerifier | stop_reason.is_some() and hooks.json configured HookEvent::Stop | block → Veto, halt → Halt, action_failed → Allow (infra failure does not block) | Yields after 5 consecutive vetoes from the same session (so a broken hook cannot wedge the loop) |
ToolLoopVerifier | stop_reason.is_none() (mid-turn) | Same name + args_hash ≥ 5 consecutive → Veto; ≥ 8 → Halt; same name + low distinctness + no narration → Veto | Pure structural check on (name, args_hash) |
ScratchpadGoalVerifier | stop_reason.is_some() and the session has an active execution list with unchecked items | Veto (auto-injects "execution list still has N unchecked items") | Fail-open (fatal is cancellation) |
MutationEvidenceVerifier | stop_reason.is_some() and the most recent file_write / file_edit / apply_patch had no bash / code_exec / code_check evidence | Veto, but only once per session (nudge, not a gate) | warn + soft-block once |
Construction lives in bin/aleph-server/commands/start/orchestrator_init.rs::build_verifier_chain:
let verifier_chain = VerifierChain::builder()
.with(StopHookVerifier::new(hooks)) // user [[stop_hooks]] → ShellStopHook set
.with(ExtensionStopHookVerifier::new()) // HookEvent::Stop (hooks.json / plugins)
.with(ToolLoopVerifier::new()) // death-loop watchdog
.with(ScratchpadGoalVerifier::new()) // execution list check
.with(MutationEvidenceVerifier::default()) // post-edit-evidence nudge
.build();TurnVerifyContext carries per-turn state: iterations, tool_calls_made, final_text, recent_tool_calls: &[ToolCallSummary], stop_reason: Option<&str>, session_id: Option<&str>, and robustness_profile: ModelRobustnessProfile (per-model thresholds handed to specific verifiers).
Shell Stop Hooks
StopHookHandler is a Send + Sync async trait:
pub trait StopHookHandler: Send + Sync {
fn name(&self) -> &str;
async fn evaluate(
&self,
ctx: &StopHookContext,
cancel: &CancellationToken,
) -> StopHookVerdict;
}
pub enum StopHookVerdict {
Allow,
Block { reason: String },
Halt { reason: String }, // exit 3 — claude-code preventContinuation
Error { hook_name: String, message: String },
}ShellStopHook pipes StopHookContext JSON (with final_text, iterations, tool_calls_made, stop_reason) over stdin and maps the exit code to a verdict: 0 = Allow, 2 = Block, 3 = Halt, anything else / signal / timeout / cancellation / spawn failure = Error.
Fail-closed boundary: the goal objective gate (goal_continuation::gate_veto) treats Error as a veto — the hook itself failed = completion cannot be verified = completion cannot be claimed. This is the fail-closed position, the dual of the harness's own StopHookVerifier (fail-open). The split exists because the same ShellStopHook impl is shared; the side-effect is the only thing that differs.
Shell-metacharacter rejection: is_shell_safe(&str) is the gate on the goal tool's LLM-supplied goal_gate_command path — enabled via ShellStopHook::shell_safe(), allow-list SAFE = "abc…ABC…0123…/-._:'=". A gate_command carrying && / ; / $() / redirection routes to Error rather than Allow, preventing "verifier syntax error silently passing as verification".
Configuration
[[stop_hooks]]
name = "lint-check"
command = "cargo clippy -- -D warnings"
timeout_secs = 60build_from_config reads the [stop_hooks] array: each entry constructs a ShellStopHook, wraps it as Arc<ShellStopHook> as Arc<dyn StopHookHandler>, packs it into a Vec<Arc<dyn ...>>, then hands it to StopHookVerifier::new(hooks). An empty array returns None; AgentHarnessRunner::stop_hooks stays None and the hook stage is skipped entirely.
Prompt-Level Verification
Aleph's completion judgment lives in the prompt (src/thinker/layers/agent_role.rs), never in Rust code:
- Agents are instructed to emit a
VERDICT: PASS|FAIL|PARTIALblock summarizing their self-checks before stopping. - Redlines R8 (LLM Sovereignty) and R10 (intelligence lives in the prompt) prohibit introducing
JudgeVerifier/ComputationalVerifierstyle cognitive judges; every verifier in the table above is a structural signal (exit codes, argument hashes, execution-list row counts, post-edit evidence presence) — never a semantic verdict. - The 6a module comment in
src/verification/turn_verifier.rspermanently bans the cognitive shape: "JudgeVerifierandComputationalVerifierare permanently prohibited".
A VerifyStopHook Rust struct (a P0-era LLM-based verifier candidate) was never wired into production and was deleted in P4 (commit b54877d7f is retrievable from git log), on the grounds that the prompt-level mechanism already covers the use case.
Safety Properties
- No unwrap in production — every error path uses
map_err+?;stop_hooks.rs::is_shell_safenever panics. - Timeout and cancellation —
ShellStopHookusestokio::time::timeout(per_attempt)paired withCancellationToken; all three signals (join_allresult,select! { _ = cancel },Stopframe) clean up the child. - 64 KB frame cap on realtime paths —
pty.inputandvoice.stream.audioboth rely onmpsc::error::TrySendError::Fullto silently drop frames — realtime audio congestion policy is "drop the newest frame, do not hang the RPC", not "wait for the consumer". - Allow-listed
is_shell_safe— untrustedgate_commandpaths pass throughShellStopHook::shell_safe; metacharacters; | & $ < > \are all rejected. unwrap_or_else(|e| e.into_inner())— lock-poison recovery (P7); everyMutex/RwLockuses it, never propagating poison upward.
Code Locations
PTY embedded terminal:
src/gateway/pty/mod.rs— entry +PtySession/SpawnOptionsre-exportssrc/gateway/pty/session.rs—PtySession::spawnusesportable_pty::native_pty_system()src/gateway/pty/manager.rs— process-globalPtyManager(64-session cap + 10 min TTL + FIFO eviction)src/gateway/handlers/pty.rs— RPCs:pty.spawn/pty.input/pty.resize/pty.close/pty.list
Verifier chain:
src/verification/mod.rs— module entry, re-exports the five verifier submodulessrc/verification/turn_verifier.rs—TurnVerifiertrait +VerifierChain+TurnVerifyContext+hash_tool_args+TOOL_HISTORY_WINDOW = 8src/verification/stop_hooks.rs—StopHookHandler/ShellStopHook/is_shell_safe/execute_stop_hooks_arcsrc/verification/stop_hook_verifier.rs—StopHookVerifier, the chain's stop-hook adaptersrc/verification/extension_stop_gate.rs—ExtensionStopHookVerifier, hooksHookEvent::Stopsrc/verification/tool_loop_verifier.rs—ToolLoopVerifier, theToolCallSummarydimensionsrc/verification/scratchpad_goal_verifier.rs—ScratchpadGoalVerifier, execution-list checksrc/verification/mutation_evidence_verifier.rs—MutationEvidenceVerifier, post-edit-evidence nudge
26.7.x Addendum
DiminishingReturnsDetector Removed
26.7.20+ (see git log): the DiminishingReturnsDetector hard stop was removed wholesale from src/context/budget/, alongside its StopDiminishing directive, the TurnMetrics type, src/thinker/nudges.rs::GRACE_NUDGE_DIMINISHING, the after_turn consumer in src/harness/agent/think.rs, and the GraceReason::Diminishing grace-turn path.
src/harness/tests/budget.rs:280 is blunt about why: "R10 '5 don'ts' #3 — the loop must make no completion judgement of its own". In practice: ToolLoopVerifier (repeat_threshold = 5 consecutive identical tool calls), max_iterations, the consecutive-failure watchdog, and the model's own judgement are what now decide a stop — never a middleware heuristic. The ratchet sits at 5,055 lines today (CEILING in src/harness/tests/budget.rs:335); the code is the authority — every doc number is just a copy of CEILING.
Turn-Level Verifier Chain
26.7.15+: src/verification/turn_verifier.rs provides the TurnVerifier trait + VerifierChain; the shipping verifiers are all in the table above. Every verifier is a pure async fn verify(&TurnVerifyContext, &CancellationToken) -> VerifierVerdict, with no mutable state, so the chain can hold them as Arc<dyn TurnVerifier>.
Only the model's explicit stop (stop_reason.is_some()) ends a turn; a death-loop mid-turn (stop_reason.is_none()) is caught by ToolLoopVerifier, which injects a feedback message and Continues once on a Veto; only halt_threshold consecutive identical calls produce a Halt and exit the loop. max_iterations remains the hard safety cap.
Context "Never-Break" Guarantee
26.7.1+: turns that previously could end early via FinalReply now always route through compact_to_fit and continue, with a deterministic truncate_to_fit floor as a last resort. The consecutive-failure watchdog in think.rs plus ToolLoopVerifier are the two gates: try compact_to_fit first, then truncate_to_fit, and only as a final fallback let a turn end without erroring.
Dual with the Goal Subsystem
StopHookVerifier is fail-open (a broken script must not wedge the loop); goal_continuation::gate_veto is fail-closed (a broken script cannot certify completion). Both share the same ShellStopHook implementation; the side effect is the only difference. The goal tool uses effective_gate(deps.gate, goal.gate_command) to splice both sources (global hook + per-goal command), and each one runs through is_shell_safe.
See Also
- Harness Architecture — verifier integration
- Architectural Redlines R10 — the five "Don'ts"
- Architecture Overview — where the verifier sits
- Tool-Loop Verifier source — full
ToolLoopVerifier - Shell stop hook source —
ShellStopHook+is_shell_safe