Getting Started with AuditTrail

Welcome to AuditTrail — open-source explainable observability for AI agent systems. This guide will walk you through your first steps after logging in.

1. Explore the Overview Dashboard

After logging in, you land on the Overview page. Here you'll find:

• Metric Cards — Total traces, average latency, token usage, cost, and error rate at a glance
• Recent Traces — Your latest agent executions with status, duration, and cost
• Constitutional Evaluations — Recent rule checks showing pass/fail/warning status

Use the time range filter at the top to adjust the window (1h, 24h, 7d, 30d).

2. View a Trace

Click on any trace in the Traces page to see its full detail:

• Summary tab — High-level overview with status, duration, tokens, cost, and model info
• Spans tab — Every step the agent took, with input/output data for each span
• DAG tab — Interactive decision-chain graph showing how the agent moved through steps
• Sankey tab — Causal attribution diagram showing WHY the agent made each decision
• Timeline tab — Waterfall view showing sequential and parallel execution timing

Quick Actions from Trace List

Use the kebab menu (⋮) on any trace row for quick access:

• Detail — Jump to the full trace detail page
• View DAG — Open the standalone DAG visualization
• View Sankey — Open the causal attribution Sankey diagram
• View Timeline — Open the timeline waterfall view

3. Understand Agent Decisions with the DAG

The DAG (Directed Acyclic Graph) view shows the agent's decision chain as an interactive graph:

• Click a node to see its details in the right panel (type, status, duration, tokens, cost, input/output)
• Click a node to highlight its path through the graph — connected edges turn bright blue with arrows
• Use the minimap (bottom-right) to navigate large graphs
• Export as PNG using the download button in the header

Node Types

4. Causal Attribution with Sankey Diagrams

The Sankey view answers "WHY did the agent do that?" by mapping:

Prompt phrases → Reasoning steps → Tool calls

Flow width shows attribution strength — thicker flows mean stronger causal links.

Running Ablation Analysis

1. Open a trace's Sankey page
2. Click "Run Ablation Analysis"
3. Wait for the analysis to complete (this re-runs the agent with masked prompt phrases)
4. The Sankey diagram appears showing causal attribution from your prompt to tool selections
5. SHAP explanations show feature importance for each tool selection decision

Compare tab — View attribution scores and re-run analysis for comparison Export tab — Download results as JSON, CSV, or PNG

Note: Ablation analysis is opt-in and costs additional LLM API calls. Results are cached so subsequent views are instant.

5. Constitutional Governance

AuditTrail monitors your agents against safety rules defined in YAML:

• Green — Rule passed, agent behavior is compliant
• Amber — Agent approached a violation boundary (80% threshold) — a warning worth investigating
• Red — Rule was violated

Viewing Compliance

Navigate to Compliance in the sidebar to see:

• All evaluations grouped by policy (safety, privacy, operational)
• Filter by status (pass/amber/red) and time range
• Click any evaluation to drill into the specific trace and span that triggered it

Adding Custom Rules

Go to Settings → Rules and click "Add Rule" to create a new constitutional rule:

• Define the rule name, description, and policy group
• Set the target span type, condition field, operator, and thresholds
• Rules are stored as YAML and evaluated in real-time

You can also download the YAML template to define rules manually.

6. Analytics & Reports

Analytics Dashboard

The Analytics page shows trends over time:

• Token usage, cost, latency, and error rates
• Tool usage distribution
• Filter by time range and agent

PDF Audit Reports

Navigate to Reports to generate compliance audit reports:

• Select a time range and report type
• Export as PDF for external auditors and compliance teams

7. Keyboard Shortcuts

AuditTrail supports power-user keyboard shortcuts:

8. API Access

AuditTrail exposes a full REST API for programmatic access. See the API Reference for complete documentation.

Key endpoints:

• GET /api/v1/traces — List all traces with filtering
• GET /api/v1/traces/:id — Get trace detail with spans
• POST /api/v1/ablation/run — Run ablation analysis
• GET /api/v1/analytics/overview — Get analytics summary
• GET /api/v1/rules — List constitutional rules

Next Steps

• Read the Architecture Guide to understand how AuditTrail works under the hood
• Check the Deployment Guide for production setup
• Explore the API Reference for integration options
• Review the DAG Styling Reference for customizing node appearance