Concepts

One page for the mental model. AuditTrail has two halves — explain (why did the agent do that?) and operate (govern and act on live agents) — built on one tracing substrate. Everything below links to its feature page.

The tracing substrate

Trace                       one agent run, end to end
└── Span                    one unit of work inside the run
    ├── llm span            a model call (model, tokens in/out, cost)
    ├── tool span           a tool invocation (+ per-call ToolCall rows)
    ├── agent span          an orchestrator / sub-agent boundary
    └── internal span       everything else (parsing, routing, …)

• A trace is one run of one agent: it carries agent_name, environment, status, duration, and aggregate cost/tokens.
• Spans nest via parent_span_id — that hierarchy IS the decision DAG you see on /traces/[id]. Spans stream live over WebSocket as the agent runs.
• Spans arrive from any integration: SDK clients, wrappers, callback handlers, the gateway proxy, or plain OTLP. Once ingested they're identical.
• Cost resolves per llm span through four layers: agent-reported > operator overrides (rules/pricing.yaml) > bundled community catalog (~3k models, auto-refreshed daily) > default rate.

Governance: the governor and YAML rules

Constitutional rules live in rules/*.yaml — never in code. Each rule declares a policy group (safety / privacy / cost / quality), a metric or pattern, and two thresholds. The governor evaluates every span at ingest under a strict severity contract:

The buckets are disjoint: pass + amber + red = total evaluations, which is exactly how the /compliance donut, group tiles and rules table count. Rules can be scoped per-user with bindings, and red violations feed alerts and the control plane.

Explainability: the WHY pipeline

The pipeline answers "why did the agent pick that tool?" in escalating depth — each stage upgrades the previous, nothing blocks on an LLM key:

1. Surrogate model — an XGBoost classifier trained on the instance's most recent tool spans learns "which features predict tool choice". Its honest quality signal (F1) is shown wherever it's used.
2. Ablation — opt-in and cached, never automatic: re-runs the decision with prompt segments removed to measure real causal impact. A cost estimate is shown before anything spends.
3. SHAP attribution — per-feature contributions from the surrogate, rendered as the Sankey on /sankey (prompt phrases → reasoning → tools, flow width = attribution).
4. Counterfactuals — "what's the smallest prompt change that flips the decision?", computed from the surrogate (no LLM call), with honest statuses when the model is untrained or the decision is genuinely stable.
5. NL explanations — a readable "why this tool?" paragraph; template-based with zero configuration, upgraded by an LLM key when present.
6. SAE mechanistic XAI — self-host feature-level introspection for open-weight models.

Cold start (time-to-WHY). The surrogate needs at least 5 tool spans with two distinct tools before it can train. It trains at API startup and — since v3.31.0 — automatically in the background once a fresh install crosses that threshold, so you never need a restart to get real attributions. Until then every WHY surface degrades honestly: template NL explanations still work, counterfactuals state that the model is untrained, and heuristic SHAP estimates are labelled as heuristics rather than presented as learned values.

Fleet-level WHY. The same surrogate's global feature importances (mean |SHAP| over its training set) are on /analytics → Fleet WHY: what drives tool selection across the whole deployment, with the training-set tool mix and the model's F1 alongside.

Operations: the control plane

Acting on what you learned, with a human in the loop:

• Deployment actions — proposed operations (kill, throttle, switch model) in three approval tiers; even Tier 1 is never autonomous. Approved actions become runtime controls that the gateway/executor actually enforce, with TTLs and a reaper (run status).
• Prompt canary — deterministic traffic splitting between prompt versions with auto-rollback on violation signals.
• Pause / resume — SDK checkpoints where a human inspects (optionally edits, behind a flag) agent state before the run continues.
• Local runner daemon — dispatch allowlisted agent templates to your own machines; AuditTrail orchestrates but never hosts execution.
• Gateway multi-provider routing — one deployment routes by model id to OpenAI (default), Anthropic, and any OpenAI-compatible provider (Ollama, Gemini, Qwen, MiniMax, vLLM) via environment variables — no bespoke provider SDKs. When a provider key is configured, an unrecognised model id is refused with a 400 rather than silently faked, keeping the observability record honest.

Where things live in the UI