Search

The search module provides real-time web search for Aleph, enabling the agent to access current information beyond its training data cutoff. It supports multiple search backends through a unified trait interface.

Design Philosophy

The search system follows three principles:

1. Provider abstraction — All search backends implement the same SearchProvider trait
2. Privacy-first — Self-hosted SearXNG is supported alongside commercial providers
3. Failover routing — The registry tries providers until one succeeds

Core Types

SearchResult

Unified result structure for all providers:

pub struct SearchResult {
    pub title: String,
    pub url: String,
    pub snippet: String,
    pub relevance_score: Option<f32>,   // 0.0–1.0, native for Tavily; computed elsewhere
    pub full_content: Option<String>,   // Tavily deep search only
    pub provider: Option<String>,
}

SearchOptions

Configuration for search behavior:

pub struct SearchOptions {
    pub language: Option<String>,        // ISO 639-1
    pub region: Option<String>,          // ISO 3166-1 alpha-2
    pub date_range: Option<String>,      // "day" / "week" / "month" / "year"
    pub safe_search: bool,               // default true
    pub max_results: usize,              // default 5, capped at 50
    pub timeout_seconds: u64,            // default 10, validated to ≥ 1
    pub include_full_content: bool,      // Tavily only
}

SearchOptions also carries the per-provider canonical mappings (language / region / date_range / safe_search → provider-native parameters); adding a provider only extends this mapping table.

SearchProvider Trait

#[async_trait]
pub trait SearchProvider: Send + Sync {
    async fn search(
        &self,
        query: &str,
        options: &SearchOptions,
    ) -> Result<Vec<SearchResult>>;

    fn name(&self) -> &str;
    fn is_available(&self) -> bool;

    async fn get_quota(&self) -> Result<QuotaInfo> {
        Ok(QuotaInfo::unlimited())
    }
}

SearchRegistry

Manages multiple providers by named default_provider + ordered fallback_providers:

pub struct SearchRegistry {
    providers: HashMap<String, Arc<dyn SearchProvider>>,
    default_provider: String,
    fallback_providers: Vec<String>,
    test_cache: Arc<Mutex<HashMap<String, (ProviderTestResult, Instant)>>>,
    web_fetch_fallback: Option<Arc<WebFetchSerpFallback>>,
}

impl SearchRegistry {
    pub async fn search(
        &self,
        query: &str,
        options: &SearchOptions,
    ) -> Result<Vec<SearchResult>> {
        // 1. Try the default_provider
        // 2. Walk fallback_providers in declared order
        // 3. If everything failed, try the WebFetch SERP fallback (armed by default)
        // 4. Still failing: aggregate the error trail
    }
}

Error classification: classify_search_error() collapses AlephError into auth / rate-limit / timeout / network / cancelled / config / provider / other so ops can grep the log by kind.

Aggregated error trail: when every configured provider fails (including the last-resort WebFetch branch), the final error string is prefixed with All search providers failed: and lists each step's kind + message — so the operator can see the fallback was actually attempted.

Privacy note: the user query text is not included in registry failure messages; per-provider failures are still logged in structured provider=... kind=... form for ops grep.

Supported Providers

The SearchProviderType enum is the single source of truth; from_config uses the ProviderFactoryRegistry to instantiate each configured backend, skipping unknown / missing-credential entries with a WARN.

Provider Testing

Test configuration without saving credentials:

pub struct SearchProviderTestConfig {
    pub provider_type: SearchProviderType,
    pub api_key: Option<String>,
    pub base_url: Option<String>,      // Required for SearXNG
    pub engine_id: Option<String>,     // Required for Google CSE
}

Returns ProviderTestResult with latency, error type, and success status; results are cached for 5 minutes, with failed results never cached.

Usage Example

use alephcore::search::{SearchProvider, SearchOptions};
use alephcore::search::providers::TavilyProvider;

let provider = TavilyProvider::new("tvly-xxx".to_string())?;
let options = SearchOptions::default();

let results = provider.search("Rust programming language", &options).await?;

for result in results {
    println!("Title: {}", result.title);
    println!("URL: {}", result.url);
    println!("Snippet: {}\n", result.snippet);
}

Safety Properties

• Aggregated errors carry a stable kind label — classify_search_error collapses AlephError into a stable kind string so logs can be grepped by kind without parsing free-form messages (never includes the user query text)
• Safe truncation — latency_ms = elapsed.min(u32::MAX as u128) as u32 saturates latency to u32
• No lock poisoning — every lock() call uses unwrap_or_else(|e| e.into_inner())
• max_results cap — validated_max_results() clamps max_results to [1, 50]
• timeout validation — validated_timeout() floors timeout_seconds at 1
• Test cache only stores successes — failed results are not cached and are retried on the next call

Code Location

• src/search/mod.rs — Module entry point and re-exports
• src/search/provider.rs — SearchProvider trait
• src/search/registry.rs — SearchRegistry with failover
• src/search/options.rs — SearchOptions and QuotaInfo
• src/search/result.rs — SearchResult type
• src/search/providers/ — Provider implementations

See Also

• Builtin Tools — Search tool exposed to agents
• Configuration — Search provider configuration

26.7.x Addendum

Memory memory.retrieve_with_trace

26.7.x: memory.retrieve_with_trace exposes a per-stage scoring trace (implemented at src/gateway/handlers/memory_config.rs::handle_retrieve_with_trace), consumed by the debug panel and the governance views.

Knowledge Graph

26.7.15+ note_graph_query (implemented under src/memory/notes/):

• Bidirectional BFS path finding
• Typed relation edges
• [[wikilink]] supersession that force-surfaces a correcting note
• CJK trigram full-text search
• Crash-safe index
• Community detection (hand-rolled Louvain, no external crate)
• Associative memory: 4-signal community-aware recall on the primary retrieval path, plus graph snapshot / cache / insights tables and graph-health insights (isolated / sparse / bridge / surprising) exposed to the LLM

Web Fetch

26.6.29+ a category of crawl4ai / firecrawl fetch providers (URL → markdown), vault-stored keys; automatically falls back to the built-in fetch on failure. The Firecrawl search provider reuses its /v2/search configuration (date-range mapping + Test Connection).

Live WebFetch Fallback

26.7.x: SearchRegistry may arm a WebFetchSerpFallback (DDG mirror scrape) after the default + fallback chain is fully exhausted, controlled by [search].web_fetch_fallback (default true). from_config skips backends with missing credentials or unknown provider_type instead of aborting the load.

See Also

• Memory System — full retrieval architecture
• Gateway RPC memory.* — protocol