LlamaIndex integration

There is no bespoke AuditTrail plugin for LlamaIndex, and that's deliberate: the honest contract is OpenTelemetry. Instrument a LlamaIndex app with OpenLLMetry (the traceloop-sdk), point its OTLP exporter at an OpenTelemetry Collector, and let the Collector re-encode to OTLP/JSON and forward to AuditTrail — the same Collector-bridge path documented on the OpenTelemetry page. Your query engines, retrievers, embeddings and LLM calls then show up as traces, with the DAG, cost accounting and constitutional rule evaluation working off the gen_ai.* attributes OpenLLMetry emits.

This page is grounded in a real end-to-end run (a keyless MockLLM + MockEmbedding VectorStoreIndex query, exported through a live Collector into /api/v1/ingest/otlp); the span names and mappings below are what actually landed.

Install

pip install traceloop-sdk llama-index

traceloop-sdk pulls in opentelemetry-instrumentation-llamaindex. A llama-index-core-only install works too, but see the instrumentation note below — auto-instrumentation keys off the umbrella llama-index package.

Instrument the app

OpenLLMetry's LlamaIndex instrumentation is what turns query/retrieve/ synthesize calls into OTel spans. Point its exporter at your local Collector with TRACELOOP_BASE_URL (OpenLLMetry reads this, not the generic OTEL_EXPORTER_OTLP_ENDPOINT; an http:// base selects the HTTP exporter, which posts to <base>/v1/traces):

import os
os.environ["TRACELOOP_BASE_URL"] = "http://localhost:4318"  # the Collector
 
from traceloop.sdk import Traceloop
from opentelemetry.instrumentation.llamaindex import LlamaIndexInstrumentor
 
# disable_batch=True => spans export as they end (handy for scripts).
Traceloop.init(app_name="my-llamaindex-app", disable_batch=True)
LlamaIndexInstrumentor().instrument()
 
# ... build your index and query engine unchanged ...
# index.as_query_engine(llm=llm).query("...")

Why the explicit LlamaIndexInstrumentor().instrument()? Traceloop.init() auto-instruments LlamaIndex only when the umbrella llama-index distribution is importable (its guard is is_package_installed("llama-index")). Modern installs are frequently llama-index-core-only, in which case auto-init silently skips LlamaIndex and you get zero spans. Calling the instrumentor explicitly (or pip install llama-index, the umbrella) fixes it. This was confirmed end-to-end: with llama-index-core alone, Traceloop.init() produced no spans; the explicit call produced the full tree below.

Collector bridge

The Collector re-encodes OpenLLMetry's protobuf OTLP to the OTLP/JSON AuditTrail ingests. This is the same 8-line config as the OpenTelemetry page, co-located with your app on localhost:

# otel-collector.yaml
receivers:
  otlp:
    protocols:
      http:                      # listens on :4318
exporters:
  otlphttp/audittrail:
    traces_endpoint: https://your-audittrail-host/api/v1/ingest/otlp
    encoding: json               # re-encode protobuf -> OTLP/JSON
    headers:
      Authorization: Bearer sk-at-...
service:
  pipelines:
    traces:
      receivers: [otlp]
      exporters: [otlphttp/audittrail]

The otlphttp exporter gzips the request body by default, and AuditTrail's ingest endpoint accepts a Content-Encoding: gzip body — so no compression: override is needed. (Older AuditTrail builds required compression: none.)

What lands

A single LlamaIndex query emits a tree of <Component>.workflow root spans and nested <Component>.task spans. In the E2E run these included RetrieverQueryEngine, VectorIndexRetriever, SentenceSplitter, TokenTextSplitter, MockEmbedding, MockLLM, CompactAndRefine and DefaultRefineProgram.

OpenLLMetry roots each LlamaIndex workflow as its own OTel trace, so one query arrives as several AuditTrail traces — in the test, the index's chunking (SentenceSplitter.workflow), the embedding pass (MockEmbedding.workflow) and the query engine (RetrieverQueryEngine.workflow) each became a separate trace.

How the spans map (via otlp_mapper.py):

LlamaIndex-specific payloads (the retrieved nodes, the formatted prompt, the query string) arrive as traceloop.entity.input / .output attributes and are preserved verbatim under span.attributes. LLM completions land as gen_ai.output.messages.

With Mock models the reported model is literally unknown and no gen_ai.usage.* tokens are emitted, so estimated cost is 0. A real provider (e.g. LlamaIndex's OpenAI LLM through the gateway or directly) reports its real model name and token usage, which drives AuditTrail's cost accounting.

Gotchas

• JSON only. AuditTrail ingests OTLP/JSON; a protobuf body 415s. The Collector's encoding: json is what bridges OpenLLMetry's protobuf export — you cannot point OpenLLMetry straight at /api/v1/ingest/otlp (its Python exporter emits protobuf). See OpenTelemetry for the full rationale.
• Auto-instrumentation needs the umbrella package. A llama-index-core-only install is skipped by Traceloop.init() — call LlamaIndexInstrumentor().instrument() explicitly or install llama-index.
• One query, several traces. Each LlamaIndex workflow is its own OTel trace; expect indexing and query phases as distinct AuditTrail traces rather than one root.
• Batch cap. OTLP requests are capped at 2,000 spans; spans missing span_id/trace_id/start_time are skipped and counted in the response's rejected tally.
• Content capture (prompts/completions) follows OpenLLMetry's own privacy flag (TRACELOOP_TRACE_CONTENT).