Aleph
Concepts

Prompt System

PromptBuilder, PromptPipeline layers, and the cacheable/stable vs transient/dynamic split.

The prompt system is the canonical way Aleph assembles the system string for each LLM call. It lives in src/thinker/; the only public entry point is thinker::PromptBuilder, which wraps a PromptPipeline of PromptLayer implementations. There is no separate agent_loop::PromptBuilder — that path was retired when the agent loop was dissolved (Phase 7).

Architecture

FlowRequest (harness bridge)


┌──────────────────────────────────────────┐
│           PromptBuilder                  │
│  ┌───────────────────────────────────┐  │
│  │        PromptPipeline             │  │
│  │  priority-ascending registration: │  │
│  │  Stable layers first, then Dynamic│  │
│  └───────────────────────────────────┘  │
└──────────────────────────────────────────┘


Vec<SystemPromptPart>     ◀── stable {cache: true}, dynamic {cache: false}

The Layer Trait

pub trait PromptLayer: Send + Sync {
    fn name(&self) -> &'static str;
    fn priority(&self) -> u32;                      // assembly order (ascending)
    fn paths(&self) -> &'static [AssemblyPath];     // which entry points fire it
    fn supports_mode(&self, mode: PromptMode) -> bool { true }
    fn stability(&self) -> LayerStability { Stable } // cache partition
    fn inject(&self, output: &mut String, input: &LayerInput);
}

Two stabilities and two assembly paths exist — no more. The original set included Hydration / Soul / Context paths that survived only inside debug tooling after their last real caller went away; each was removed for the same reason: a phantom path is a layer that renders nowhere, and the omission is invisible. Today:

pub enum AssemblyPath { Basic, Cached }

pub enum LayerStability { Stable, Dynamic }
PathEntry pointCaller
BasicPromptBuilder::build_system_promptsrc/agents/subagent_spawner — inline sub-agent prompt, one flat string, no cache split
CachedPromptBuilder::build_system_prompt_cached_with_modesrc/orchestrator/harness_bridge/prompt_build.rs — main-loop prompt split into stable + dynamic

A layer omits the path only when the producing caller has dropped the entry point that requests it; the registration block in src/thinker/prompt_pipeline.rs::default_layers() is the authoritative order, locked by tests for layer count, priority monotonicity, and Stable-comes-before-Dynamic.

Cacheable / Transient Boundary

The Anthropic adapter reads cache_control markers on messages; the system block precedes them, so from the second call of a session onward every system byte is a cache READ at 0.1× input. Splitting the system string into a cacheable stable prefix and a transient dynamic suffix is therefore the prompt-side optimisation that pays for itself — bytes that move per request must land in the suffix, or they invalidate the cached prefix.

build_system_prompt_cached_with_mode(tools, mode) (src/thinker/prompt_builder/cache.rs) returns:

vec![
    SystemPromptPart { content: stable,  cache: true  }, // rides the cacheable prefix
    SystemPromptPart { content: dynamic, cache: false }, // per-request suffix
]

The split comes from each layer's stability() declaration, plus the strategy/guardrail weld logic that mirrors StrategyLayer's byte-for-byte wrap in the appropriate zone (full <strategy> body on the stable side, <strategy_reminder> echo on the dynamic side). The Anthropic adapter then places the prompt-cache breakpoint at the stable/dynamic boundary.

Token budget

TokenBudget::fit_dynamic_suffix (src/thinker/prompt_budget.rs) only trims the dynamic side when the budget is exceeded. The stable prefix is the protected floor — persona / tools / security / skills — so the cache stays valid even when the dynamic suffix is replaced with a truncation notice.

Entry Points (and Their Welding)

build_system_prompt (Basic, sub-agent inline)

Used by src/agents/subagent_spawner to assemble an inline flat-string system prompt for a child harness. The Basic path threads no ResolvedContext, so layers that read LayerInput::context (e.g. StrategyLayer's guardrails, SecurityLayer's session_mode line) stay silent there. Two post-pipeline welds mirror those layers byte-for-byte:

  • with_strategy(body) — wraps the body in <strategy>\n{body}\n</strategy>\n\n, identical to StrategyLayer.
  • with_session_mode(mode) — appends SessionMode::subagent_prompt_line(), identical to the line SecurityLayer would have rendered if a ResolvedContext were present.

build_system_prompt_cached_with_mode (Cached, main loop)

Used by src/orchestrator/harness_bridge/prompt_build.rs once per turn. The production entry threads every builder field through (identity files, curated envelope, agent def, MCP instructions, chain context, resolved context, behaviour name, model behaviour delta, iteration cap, session summaries) so the Phase-2-widened layers (SecurityLayer, OperationalGuidelinesLayer, ProtocolTokensLayer, RuntimeContextLayer) all fire on the Cached path. Wiring regressions on this entry point have been fixed multiple times — the RoleLayer / CitationStandardsLayer / AgentRoleLayer / SoulLayer / ProfileLayer tests pin the cacheable layers' Cached participation.

Prompt Modes

PromptMode::{Full, Compact, Minimal} controls which layers are loud on a single assembly. Full is the default and reproduces the legacy prompt byte-for-byte. Compact and Minimal opt out of heavier guidance layers via each layer's supports_mode. The stable/dynamic split — and therefore the prompt-cache breakpoint — is preserved across all three modes.

ModeStatic scaffold (measured)Pragmatic use
FullLargestDefault; identity, rules, environment, special actions all visible
CompactExcludes ~15 heavy layersLatency-sensitive high-throughput calls
MinimalExcludes most layersPure tool-call or token-budget-constrained paths

These are blunt instruments for an operator who has accepted what they remove. Nothing auto-selects a mode from a model's price tier or declared window — that was measured (aleph-server prompt-size --path cached, 2026-07-14) and the saving did not pay against a request whose bytes already ride the provider's cached prefix. The genuine "small window" case is handled by TokenBudget::from_context_window, which sizes a character budget to the model's actual window and trims the dynamic suffix on overflow.

Configuration Surface

The pipeline's behaviour is driven by PromptConfig (src/thinker/prompt_builder/mod.rs):

FieldEffect
languageLanguageLayer (Stable, @1600) pick
runtime_capabilitiesPre-formatted runtimes banner (Python / Node / FFmpeg)
token_budgetTokenBudget for the dynamic-side trim
eligible_skillsSnapshot of v2 SkillSystem scope-aware skills
skill_prompt_budgetCap on the injected <available_skills> index
available_agentsAgentCatalogLayer (Dynamic) entries
mcp_instructionsMCP-server instruction blocks
active_tool_namesUsed to keep Tool-scoped skills on the cached path

Other thread-through fields on the PromptBuilder itself (set via builders): agent_def, identity_files, curated_memory_envelope, chain_context, resolved_context, behaviour_name, model_behaviour_delta, iteration_cap, extra_files, strategy, session_mode, has_session_summaries.

There is no [thinker] block in aleph.toml. The single setting that does read from config is [execution] prompt_mode, deserialised directly into PromptMode (lowercase wire: full / compact / minimal).

The Two Rulers

Pruning prompt bytes is a recurring concern — the system prompt itself is Harness, and a smart model needs less direction, fewer constraints, fewer examples. Two rulers codified into the codebase restrict new bytes:

  1. Runtime fact vs teaching rule. "This is a piece of runtime reality the model cannot know" (time, cwd, tool schema, active goal, identity files, security context, MCP instructions) is scaffolding, keep it. "This is me telling a strong model how to think" is a cage, move it to architecture (error-hint signals, verifier chain, stop hooks) rather than prose.
  2. Pi's mirror — "If a tool owns this sentence, write it into that tool's DESCRIPTION (sent with the schema, only to requests that can call it), not the system prompt." The system prompt only carries cross-tool tradeoffs, runtime facts, and security boundaries — never a tool's own contract.

Both rulers are checked architecturally. The prompt-size breakdown (ALEPH_PROMPT_SIZE_TRACE=1) and the duplicate-sentence guard (no_sentence_is_stated_twice, no_environment_fact_is_stated_twice) keep the scaffold ceiling measured, not hand-edited.

Reaching a Layer (Structural Guard)

A layer that lists only an AssemblyPath no caller ever requests renders nowhere, and the omission is invisible — phantom paths are how layers go silently missing. src/thinker/prompt_contract.rs enforces this with three tests that run as part of cargo test:

  • reachable_layers — every registered layer must speak under at least one paradigm; otherwise it must be listed in CONDITIONALLY_SILENT with a reason that names the session content that wakes it.
  • scaffold_bytes_ratchet — the prompt-size ceiling is measured, not hand-edited; only decreases.
  • no_sentence_is_stated_twice and no_environment_fact_is_stated_twice — sentence- and fact-level duplicate guards (the latter matches on the underlying fact's value, not the sentence, so OS written two different ways still counts as a duplicate).

The same prompt_contract.rs exposes a aleph-server prompt-size debug command that reports the current actual scaffold byte count per paradigm (Worst-of-five).

What's Already Dissolved (avoid re-introducing)

The pipeline registration block has shrunk from ~40 layers to its current set through repeated prune rounds. Past deletions that newcomers sometimes re-introduce:

  • Text-schema tool listings (ToolsLayer, HydratedToolsLayer) — both production paths force native_tools_enabled = true, and the text-envelope parser that would have consumed those listings was deleted on 2026-05-10. Discovery converges on the on-demand tool_search tool (MODEL_PERCEIVABLE_ECOSYSTEM.md, R7 static partition).
  • ToolUsageGrammarLayer — read ToolInfo::usage_hint, which had zero producers repo-wide. Its one live sentence (parallel tool calls) moved to RoleLayer, which always fires.
  • GenerationModelsLayer, ThinkingGuidanceLayer, CustomInstructionsLayer, SkillModeLayer, HeartbeatLayer, ResponseFormatLayer, McpToolIndexLayer, InboundContextLayer, SessionResumeLayer, McpResourceIndexLayer — each had zero production input that could make it speak; identifying which is which is the reachable_layers test's job, not memorised trivia.

Read the registration block (src/thinker/prompt_pipeline.rs::default_layers) as the source of truth.

Code Locations

  • src/thinker/prompt_builder/mod.rsPromptBuilder, PromptConfig, SystemPromptPart
  • src/thinker/prompt_builder/cache.rsbuild_system_prompt_cached_with_mode
  • src/thinker/prompt_pipeline.rsPromptPipeline, default_layers, execute_stable_with_mode, execute_dynamic_with_mode, layer_breakdown
  • src/thinker/prompt_layer.rsPromptLayer trait, AssemblyPath, LayerStability, LayerInput
  • src/thinker/prompt_mode.rsPromptMode::{Full, Compact, Minimal}
  • src/thinker/prompt_budget.rsTokenBudget, fit_dynamic_suffix
  • src/thinker/prompt_contract.rsreachable_layers, scaffold_bytes_ratchet, no_sentence_is_stated_twice
  • src/thinker/layers/ — Layer implementations (soul, agent_role, curated_memory, strategy, chain_context, mcp_instructions, voice_mode, profile, role, runtime_context, environment, runtime_capabilities, tool_runtime_state, agent_catalog, security, protocol_tokens, operational_guidelines, multi_step_conduct, provider_guidance, session_budget, citation_standards, skill_instructions, special_actions, doctor_repair_hint, guidelines, identity_files, extra_files, memory_protocol, session_context_guide, timer_loop, graph_topology, standing_goal, execution_plan, strategy_pointer, operating_envelope, language)
  • src/thinker/nudges.rs — 6 GRACE_NUDGE_* + warning strings; lives in nudges because prompt prose is cognition (R9), not scaffold (R10)
  • src/orchestrator/harness_bridge/prompt_build.rs — main-loop caller

See Also

On this page