AI agents can act. Traccia controls what they're allowed to do.

Traccia is a production-ready Python SDK for observing, evaluating, and enforcing policies on AI agents and LLM applications at runtime.

Built on OpenTelemetry, Traccia provides distributed tracing, token and cost tracking, evaluation, governance evidence, and runtime policy enforcement across AI agents, agentic workflows, and multi-agent systems.

Framework-agnostic and built for OpenAI Agents, LangGraph, CrewAI, and LLM applications in production.

Runtime Control

Runtime Policies: @govern can deny or reshape LLM or tool calls at runtime using policies such as Spend Cap, Model Boundary, and Loop Cap

Guardrail Detection: Passive detection of AI safety controls, provider-native safeguards, and custom guardrails

Security Controls: No secrets in logs and configurable data truncation

Observability & Tracing

Automatic Instrumentation: Auto-patch OpenAI, Anthropic, Gemini (google-genai), requests, and HTTP libraries

Framework Integrations: Support for LangChain, CrewAI, and OpenAI Agents SDK

LLM-Aware Tracing: Track tokens, costs, prompts, completions, and latency automatically

OpenTelemetry Metrics: Emit OTEL-compliant metrics for accurate token and cost tracking independent of sampling

Decorator-Based Tracing: Trace any function with the @observe decorator

Multiple Exporters: OTLP-compatible export to Grafana Tempo, Jaeger, Zipkin, SigNoz, Console, or File

Production-Ready Architecture: Rate limiting, error handling, configuration validation, and reliable flushing

High Performance: Efficient batching, async support, and low-overhead instrumentation

Evaluation & Development

Offline Evaluation: evaluate() runs a task and scorers over a dataset and saves an experiment you can open, compare, and attach on promote

Prompt Management: load_prompt / prefetch_prompts with cache, stale-while-revalidate, fallback, and traccia.prompt.* span identity

Zero-Configuration Setup: Simple init() call with automatic configuration discovery

Type-Safe Configuration: Full Pydantic validation and configuration management

Governance & Compliance

AI Governance and Compliance: EU AI Act and HIPAA-oriented evidence, transparency records, integrity verification, and PII/PHI redaction helpers

🚀 Quick Start

Installation

pip install traccia

Basic Usage

from traccia import init, observe

Initialize (auto-loads from traccia.toml if present)

init()

Trace any function

@observe() def my_function(x, y): return x + y

That's it! Traces are automatically created and exported

result = my_function(2, 3)

With LLM Calls

from traccia import init, observe from openai import OpenAI

init() # Auto-patches OpenAI

client = OpenAI()

@observe(as_type="llm") def generate_text(prompt: str) -> str: response = client.chat.completions.create( model="gpt-4", messages=[{"role": "user", "content": prompt}] ) return response.choices[0].message.content

Automatically tracks: model, tokens, cost, prompt, completion, latency

text = generate_text("Write a haiku about Python")

Gemini (google-genai)

from traccia import init from google import genai

init() # Auto-patches google-genai

client = genai.Client() response = client.interactions.create( model="models/gemini-flash-latest", input="Write a haiku about Python", )

Load a versioned prompt

Fetch prompts from your Traccia prompt registry by name and deploy label. Compiling fills {{variables}} and stamps the active span with prompt identity.

from traccia import init, observe, load_prompt, prefetch_prompts

init(api_key="...", prompt_cache_ttl_s=60) prefetch_prompts(["support-reply"]) # optional warm-up

@observe(as_type="llm") def reply(question: str) -> str: prompt = load_prompt( "support-reply", label="production", fallback={ "type": "chat", "messages": [{"role": "system", "content": "You are a helpful assistant."}], }, ) messages = prompt.compile(question=question)

pass messages to your LLM client

return messages[-1]["content"]

Requires the Traccia platform: create a workspace API key under Settings → API Keys. Tracing and prompt fetch use the same key and default to https://api.traccia.ai. Pass an explicit fallback so agents can still run if a fetch fails. See Prompts in the SDK.

evaluate() runs a task and scorers over a dataset. By default the run is saved as an experiment you can open in the app under Evaluate → Experiments, compare against other runs, and attach when you promote a prompt. Use the same workspace API key as tracing and load_prompt.

from traccia import init, evaluate, load_prompt

init(api_key="...")

prompt = load_prompt("support-reply", label="production")

def task(inp): messages = prompt.compile(**inp) return call_model(messages)

result = evaluate( "support-reply-v3", data="support-golden", # platform dataset name or UUID task=task, scorers=["exact_match"], # builtins, platform names/ids, or callables prompt="support-reply", # labels the experiment cell with this prompt ) print(result.summary()) print(result.url)

Persist is on by default. Set persist=False for a local-only loop (no experiment URL).

Inline rows work as [{"input": {...}, "expected": "..."}]. With persist on, Traccia still saves a full experiment (helper datasets named sdk-eval/... are hidden on Evaluate → Datasets unless you turn on Show SDK-Created).

Builtins run in-process: exact_match, contains, json_valid. Mix in platform scorers (LLM-as-judge, Jev Decision, code) by name or UUID. Pass provider_keys= for judges (typesafe for Jev). A Jev score keeps each question on the experiment.

One throwing row does not abort the run. That cell records error; other items still score. Configuration and API failures raise EvaluateError.

If the first task argument is named row, item, example, or case, Python passes the full row. Any other name (including input) receives only the input dict.

See Evaluate in the SDK and the walkthrough Run Experiments From Code.

traccia eval runs a task script, saves an experiment, and writes summary.json. The process exit code is the gate: 0 pass, 1 quality miss, 2 bad command or baseline, 3 untrustworthy evidence, 4 the experiment did not save, 5 the summary file was not written. With no pass-rate bar, a green check is Execution Only. Add --min-pass-rate or --max-drop for a Quality Gate. --max-drop 0.05 is five percentage points. --max-task-cost-increase 0.2 is 20 percent.

The script defines task and, for a function in the file, a scorers dict passed as --scorer local:name. A local scorer cannot pass --max-drop. Pass --prompt with --prompt-version or --prompt-label when the experiment should record a prompt version. Platform LLM-judge and Jev scorers read OPENAI_API_KEY, ANTHROPIC_API_KEY, GEMINI_API_KEY or GOOGLE_API_KEY, GROQ_API_KEY, and TYPESAFE_API_KEY, or a JSON object in TRACCIA_PROVIDER_KEYS. Those values are not written into the summary. Each Jev question is gated on its own pass rate. Execution Only means no pass-rate gate; cost and latency gates can still apply.

traccia eval experiments/support_agent.py
--output summary.json
--dataset support-golden
--scorer exact_match
--min-pass-rate 0.9
--baseline-prompt support-reply
--max-drop 0.05

See Run Evals In CI.

Create a callback handler and pass it to config={"callbacks": [traccia_handler]}. Install the optional extra: pip install traccia[langchain].

from traccia import init from traccia.integrations.langchain import CallbackHandler # or TracciaCallbackHandler from langchain_openai import ChatOpenAI

init()

Create Traccia handler (no args)

traccia_handler = CallbackHandler()

Use with any LangChain runnable

llm = ChatOpenAI(model="gpt-4o-mini") result = llm.invoke( "Tell me a joke", config={"callbacks": [traccia_handler]} )

Spans for LLM/chat model runs are created automatically with the same attributes as direct OpenAI instrumentation (model, prompt, usage, cost).

Note: pip install traccia[langchain] installs traccia plus langchain-core; you need this extra to use the callback handler. If you already have langchain-core (e.g. from langchain or langchain-openai), base pip install traccia may be enough at runtime, but traccia[langchain] is the supported way to get a compatible dependency.

Traccia automatically detects and instruments the OpenAI Agents SDK when installed. No extra code needed:

from traccia import init from agents import Agent, Runner

init() # Automatically enables Agents SDK tracing

agent = Agent( name="Assistant", instructions="You are a helpful assistant" ) result = Runner.run_sync(agent, "Write a haiku about recursion")

Configuration: Auto-enabled by default when openai-agents is installed. To disable:

init(openai_agents=False) # Explicit parameter

OR set environment variable: TRACCIA_OPENAI_AGENTS=false

OR in traccia.toml under [instrumentation]: openai_agents = false

Compatibility: If you have openai-agents installed but don't use it (e.g., using LangChain or pure OpenAI instead), the integration is registered but never invoked—no overhead or extra spans.

Traccia automatically instruments CrewAI when it is installed in your environment.

from traccia import init from crewai import Agent, Task, Crew, Process

init() # Auto-enables CrewAI tracing when CrewAI is installed

researcher = Agent(role="Research Analyst", goal="Research a topic", llm="gpt-4o-mini") task = Task(description="Research Shawn Michaels", agent=researcher)

crew = Crew(agents=[researcher], tasks=[task], process=Process.sequential, verbose=True) result = crew.kickoff()

Traccia will create spans for the crew (crewai.crew.kickoff), each task (crewai.task.), agents (crewai.agent.), and underlying LLM calls, which nest under the existing OpenAI spans.

Configuration: Auto-enabled by default when crewai is installed. To disable:

init(crewai=False) # Explicit parameter

OR set environment variable: TRACCIA_CREWAI=false

OR in traccia.toml under [instrumentation]: crewai = false

🛡️ Guardrail Detection

Traccia includes a passive guardrail detection engine that runs as a span processor — no runtime enforcement, no changes required to existing agent code. It inspects every span as it ends, classifies guardrail signals into structured findings, and writes results back onto spans so they appear in any configured exporter.

Detection is automatic once traccia.init() is called. The processor runs three tiers against every span's attributes:

At the end of each trace (when the root span ends), the processor evaluates which guardrail categories should exist given the agent's observed capabilities (LLM calls, tool use, user-provided text) and reports which are missing.

Option 1: guardrail_span context manager — recommended for inline checks

from traccia.guardrails import guardrail_span

with guardrail_span("pii_check", category="pii", enforcement_mode="warn") as span: result = run_pii_check(user_input) span.set_attribute("guardrail.triggered", result.found_pii)

Option 2: @observe(as_type="guardrail") decorator — for function-level guardrails

If your guardrail function returns a bool, guardrail.triggered is set automatically:

from traccia import observe

@observe( as_type="guardrail", attributes={ "guardrail.name": "prompt_injection_check", "guardrail.category": "prompt_injection", "guardrail.enforcement_mode": "block", } ) def check_injection(text: str) -> bool: return any(kw in text.lower() for kw in INJECTION_KEYWORDS)

True → triggered (blocked), False → not triggered

For non-bool returns, set guardrail.triggered manually on the current span.

Batch pipelines or internal-only agents often get flagged for prompt_injection / input_validation because they make LLM calls with llm.prompt. Suppress specific categories for a run:

from traccia.guardrails import guardrail_span

Convenience: suppress_missing on guardrail_span

with guardrail_span("root", category="unknown", suppress_missing=["prompt_injection", "input_validation"]): run_ba