Rust Runtime
for AI-Protocol.
High-performance Rust runtime for AI-Protocol. Manifest-driven
Pipeline + AiClient, E/P workspace
(ai-lib-core + ai-lib-contact), 13 V2 error codes,
and feature-gated capability modules.
ai-lib-rust = "1.2.0" Key Features
V2 Error Codes & Feature Flags
13 V2 standard error codes for consistent error handling across providers. Feature-gated modules: 7 optional features plus a full meta-feature for minimal or complete builds.
Operator-Based Pipeline
Streaming responses flow through composable operators: Decoder → Selector → Accumulator → FanOut → EventMapper. Each stage is protocol-configured.
Protocol + Pipeline
AiClient loads manifests and runs the operator pipeline
(Decoder → Selector → Accumulator → EventMapper). V1 and V2 manifest paths supported.
E/P Workspace
ai-lib-core (execution) and ai-lib-contact (policy) ship as
separate crates; ai-lib-rust re-exports a stable facade.
Resilience
Built-in max_inflight backpressure on AiClient. Retry,
rate limit, and circuit breaker live in ai_lib_rust::resilience opt-in.
Embeddings & Vectors
EmbeddingClient with vector operations ?cosine similarity, Euclidean distance, dot product. Build semantic search and RAG applications natively.
Cache & Batch
Response caching with TTL (memory backend). Batch execution with configurable concurrency, timeout, and multiple processing strategies.
Plugin System
Extensible plugin architecture with hooks and middleware chain. Add custom behavior without modifying core code. Guardrails for content filtering and PII detection.
Simple, Unified API
The same code works across all 37 providers. Just change the model identifier ?the protocol manifest handles everything else: endpoint, auth, parameter mapping, streaming format.
The builder pattern provides a fluent API for request construction. Stream results
arrive as unified StreamingEvent types regardless of the underlying provider.
use ai_lib_rust::{AiClient, Message, StreamingEvent};
use futures::StreamExt;
// Works with ANY provider ?protocol-driven
let client = AiClient::new(
"anthropic/claude-3-5-sonnet"
).await?;
// Builder pattern for chat requests
let mut stream = client.chat()
.messages(vec![Message::user("Hello")])
.temperature(0.7)
.max_tokens(1000)
.stream()
.execute_stream()
.await?;
// Unified streaming events
while let Some(event) = stream.next().await {
match event? {
StreamingEvent::PartialContentDelta { content, .. }
=> print!("{content}"),
StreamingEvent::StreamEnd { stats, .. }
=> println!("\nTokens: {}",
stats.total_tokens),
_ => {} // ToolCall, Metadata, etc.
}
} Internal Architecture
Five layers from user-facing API to HTTP transport. The streaming pipeline is the heart of the system.
Module Overview
client/ + error_code/
AiClient, AiClientBuilder, ChatRequestBuilder, execution logic, policy engine, preflight checks, error classification, CallStats, CancelHandle.
protocol/
ProtocolLoader (local/URL/GitHub), JSON Schema validator, ProtocolManifest structure, UnifiedRequest compilation, config types.
pipeline/
Decoder (SSE, JSON Lines), Selector (JSONPath), Accumulator (tool calls), FanOut (multi-candidate), EventMapper (unified events), Retry and Fallback operators.
transport/
HttpTransport (reqwest), API key resolution (keyring + env vars), proxy/timeout configuration, middleware support.
resilience/
Circuit breaker (open/half-open/closed), token bucket rate limiter, max-inflight semaphore backpressure.
embeddings/
EmbeddingClient, EmbeddingClientBuilder, vector operations (cosine similarity, Euclidean distance, dot product).
cache/ + batch/
CacheManager with TTL (MemoryCache, NullCache). BatchCollector and BatchExecutor with concurrency control and multiple strategies.
plugins/ + guardrails/ + structured/
Plugin trait, PluginRegistry, HookManager, middleware chain. Guardrails with keyword/pattern filters and PII detection.