Get started
Five minutes, four steps, and the first trace lands: get a key, install, initialize, send. This is the same content the quickstart drawer shows when you create a product - generalized, because here we don't know which product you mean, and the drawer does.
Create a product, pick its environment, copy the key
A product is a name and an environment: you give it a name and pick prod, staging or dev (or a slug of your own), and that pair is the thing telemetry lands under. Register the same name once per environment you deploy to - one key per product per environment. Hit Create key and keep the value: it is hashed at rest and shown exactly once, so a lost key is re-minted, never recovered.
The key is the whole configuration from here on. It decides which tenant, which product and which environment your spans are stamped with - so the code below is the same code on every box you run, and there is nothing in it to get wrong.
Install
pip install "indratrace[fastapi]"
Extras are additive and core stays dependency-clean (OpenTelemetry only). Add the provider you actually call to get model spans with exact, provider-reported token counts: anthropic, openai, gemini, bedrock, claude-agent-sdk - e.g. pip install "indratrace[fastapi,anthropic]". A missing extra is never an error; the SDK skips it silently.
Initialize, once at startup
from fastapi import FastAPI from indratrace import init_observability init_observability(api_key="it_live_…") app = FastAPI() # every request is now a span, automatically
That is the whole integration. No product, no env, no URL - the key carries all three.
The API key is required, and it is the only thing you configure. Telemetry goes to an authenticating ingest gateway that validates every batch and answers 401 - dropping the data - when the key is missing, unknown, or revoked. Once the SDK is running it never raises into your app, so a wrong key looks exactly like “nothing happened”; pass api_key= or set INDRATRACE_API_KEY, and if neither is set, init_observability() says so at startup rather than shipping telemetry into a 401 nobody ever sees.
Send your first trace
# first_trace.py - run it, then open Traces.
import logging
from indratrace import init_observability, trace_agent, trace_tool
init_observability(api_key="it_live_…")
logging.basicConfig(level=logging.INFO) # so your logs ship too
@trace_tool # one span
def lookup(vendor: str) -> int:
logging.getLogger(__name__).info("looking up %s", vendor)
return len(vendor)
@trace_agent("hello-agent") # the parent span
def run(q: str) -> int:
return lookup(q)
print("result:", run("indrasol")) # ← print is fine HERE; it is not telemetryRun it, then open Agents & apps. You should see one run of hello-agent with its tool lookup step inside it. The product card on Products flips to active the moment the first span lands.
Spans are batched and flushed on a short timer, so give it a few seconds. If nothing arrives, the FAQ below starts with the debug banner - it prints exactly where the telemetry was sent and which integrations loaded.
Next: one key per deployment, zero code changes
# dev box / CI export INDRATRACE_API_KEY=it_live_… # your dev key # production - same code, same init line; a different key, and nothing else export INDRATRACE_API_KEY=it_live_… # your prod key
The environment is not in your code, and it is not a variable you set either. You choose it when you register the product, and a deployment picks its environment by which key it holds. So the init line is identical in dev and in prod - there is nothing to edit, and nothing to forget to edit back, on the way to production. A key belongs to exactly one product in exactly one environment, so give every deployment the key that was created for it. That is what keeps dev out of your prod numbers - and it is enforced at the edge, not by convention: a leaked dev key can only ever write into dev.
With INDRATRACE_API_KEY set in the deployment you can drop api_key= from the init call entirely - the SDK reads the variable when the argument is absent. Then the line really is the same everywhere, character for character, and the environment lives where the rest of your deployment config does.
Still waiting for your first span?
You called init_observability(), your app is serving traffic, and the dashboard is empty. There are only three things it can be - work down the list.
- Is
INDRATRACE_API_KEYactually set in the process that is running? Not in your shell - in the process's own environment. Check it the same way you run your app: same shell, same container, same systemd unit. A.envfile your process manager never loads, or a key exported in your terminal but not in the container, is the single most common cause. If the key were missing entirely,init_observability()would have said so at startup rather than starting un-instrumented - so if your app started at all, a key was found somewhere. The question is whether it is the right one. - Is it the key for this product, in this environment? A key belongs to one product in one environment, and your telemetry is arriving under whichever product that key names - so a staging key in a production deploy means the data landed in staging, not nowhere. Check that the environment you are looking at matches the key the process is holding. Keys are shown once at creation; if you cannot tell which one a deploy has, rotate it and set the new one deliberately.
- Set
INDRATRACE_DEBUG=1and read the export lines. Once it is running the SDK never raises, so this is where a silent drop becomes a visible line. Two of them answer almost everything:export okmeans the telemetry left your process and was accepted - you are looking at the wrong product or environment, so go back to check 2.export FAILEDnames the HTTP status the gateway answered with: a 401 means the key was rejected (revoked, or a typo in the value); no HTTP response at all means the gateway was not reachable from where your app runs.
If all three check out, the remaining possibility is placement, not configuration: init ran too late for Django, or after your Flask app object was built. The banner cannot detect either - see the framework caveats above.
Frameworks & extras
What you get out of the box, and what needs an extra. The rule underneath the whole table: logs are automatic, everything else is an extra - because logs need no third-party library and HTTP/model spans need the one you actually use.
| Your app uses | Install | Logs | HTTP spans | Model spans | Agent spans |
|---|---|---|---|---|---|
| logging (stdlib) | - | automatic | - | - | - |
| Loguru | - | automatic (0.6.0+) | - | - | - |
| print() Switch to logging or loguru - print() writes to stdout, which no OTel bridge can see. | - | never captured | - | - | - |
| FastAPI | indratrace[fastapi] | automatic | spans | - | - |
| Django Init must run before get_wsgi_application(). | indratrace[django] | automatic | spans | - | - |
| Flask Add instrument_flask_app(app). | indratrace[flask] | automatic | spans | - | - |
| Anthropic/Claude | indratrace[anthropic] | automatic | - | tokens | - |
| OpenAI | indratrace[openai] | automatic | - | tokens | - |
| Gemini | indratrace[gemini] | automatic | - | tokens | - |
| Bedrock | indratrace[bedrock] | automatic | - | tokens | - |
| Claude Agent SDK | indratrace[claude-agent-sdk] | automatic | - | tokens (per turn) | zero decorators |
| Any other agent/tool code | - | automatic | - | record_llm_usage(…) | @trace_agent / @trace_tool / @trace_step |
The caveats, honestly
Two frameworks have a catch, and both fail the same silent way - you get logs and model spans, no HTTP spans, and no error telling you why. Pick your framework to see the exact snippet:
Model calls and agents
Install the provider extra and every model call becomes a span carrying the provider's own reported token counts - input, output, and cache read/creation. No wrapper, no decorator, no counting on our side:
pip install "indratrace[anthropic]"
Agents and tools that aren't model calls get a span each via @trace_agent / @trace_tool / @trace_step. The Claude Agent SDK needs none of them - indratrace[claude-agent-sdk] traces the whole run (agent → turns → tools → tokens) with zero decorators. For a provider we don't instrument, call record_llm_usage(…) yourself and the tokens land the same way. Full guide in the SDK README ↗
Group a conversation into a session
A trace is one request; a session is the conversation it belongs to. Wrap the turns that go together and they group as one session - with the user, the duration, and any errors along the way:
from indratrace import session
# One conversation = one session id, reused across every turn it contains.
with session(session_id="conversation-42", user_id="u-1001"):
answer = run_my_agent(question) # this trace joins the session
followup = run_my_agent(next_q) # …and so does this oneBoth ids ride along automatically, so every span started inside the block carries them - including work in functions the block calls. user_id is optional: leave it out and the session is anonymous (the tab shows -, never a made-up user). Nothing is stored server-side as a “session”; the tab groups spans by session.id when you ask, so a session lasts exactly as long as its telemetry does.
Wire your feedback form
Tie a 👍/👎 (or a 1–5 rating) back to the run it scores, and it shows up on the Feedback tab, joined to that trace. record_feedback takes a plain (score, comment, trace_id) - so any front-end works: a thumbs widget, a star rating, a survey submitted hours later. Remember the trace id when you answer, pass it back whenever the user reacts:
from indratrace import current_trace_id, record_feedback # When you answer the user, remember which trace produced it: answer = run_my_agent(question) trace_id = current_trace_id() # 32-hex id of the run; store it with the answer # Later - a 👍/👎 click, a star rating, a survey - send the score back to that trace. # score: 1 = up, 0/-1 = down, or any numeric scale (a 1–5 rating works verbatim). record_feedback(1, comment="Nailed it.", trace_id=trace_id)
Score is verbatim (1 up, 0/-1 down, or any scale). The trace_id is optional - omit it inside a live request and the current trace is used; omit it entirely and the score is still recorded, just unlinked. The SDK never drops a score.
Concepts
Six words that mean something specific here. Worth two minutes - most confusion about what a screen is showing turns out to be confusion about one of these.
agent wraps a whole agent run, tool is one tool call, llm is one model call (the one carrying token counts), and turn is one round of a multi-turn agent. A plain HTTP request is a server span. An agent run's steps are coloured by kind, so "the agent spent all its time in one tool" is a shape you see rather than a thing you compute.dev, prod, or whatever you name them - a property of the deployment, not of the code. You choose one when you register the product, and a running deployment is in whichever environment its key belongs to. So the same init line ships everywhere and the separation is enforced at the ingest edge rather than resting on every box setting a variable correctly. Every screen has an environment selector, and it matters more than it looks: a p95 with a developer's laptop blended into your production numbers is not a p95 of anything.FAQ
The questions that actually came up while building this - including the ones whose honest answer is "not yet".
Nothing is showing up. Where do I start?
Turn on debug. The SDK is fail-silent by design - it never raises into your app, never blocks it, and never takes your service down because a collector was unreachable. That is the right trade, and it is also exactly what makes a misconfiguration invisible. debug=True makes it audible:
init_observability(api_key="it_live_…", debug=True) # or: export INDRATRACE_DEBUG=1 # # indratrace [DEBUG] IndraTrace SDK v1.0.0 initialized # indratrace [DEBUG] http[fastapi]: enabled # indratrace [DEBUG] genai[anthropic]: skipped (extra not installed) ← the fix is here # indratrace [DEBUG] export ok ← it left your process
Read the banner top to bottom. Each integration line says enabled or skipped (extra not installed) - a skipped next to the thing you expected to see IS the answer. Then the export lines: export ok means the batch left your process and was accepted, so if the dashboard is still empty you are looking at the wrong product or environment; export FAILED names the status the gateway answered with, and a 401 there means the key was rejected.
Do I need to configure logging?
Usually not. If your app uses Python's standard logging, your logs are already in IndraTrace - don't use print(), which is never captured. Nothing to install and nothing to configure; a line logged inside a span carries that span's trace context. Loguru is automatic too, as of SDK 0.6.0.
That holds as long as your app logs at INFO - the normal case under basicConfig(level=INFO), uvicorn, or gunicorn. The one case that needs a line is an app that never configured logging at all: it sits at the stdlib default of WARNING, so its INFO lines go nowhere, and the SDK deliberately does not raise your root logger's level behind your back. Opt in:
init_observability(api_key="it_live_…", log_level="INFO")
Why don't I see my logs?
Two causes, in order of how often they are the cause:
print()- never captured, and never will be.printwrites bytes to stdout; there is no record, no level, no timestamp and no trace context for a bridge to pick up. Switch tologgingor loguru and the same line ships, with the span it happened inside attached to it.- Your level floor. If your app never called
basicConfig, the stdlib root logger sits at WARNING and your INFO lines are dropped before the SDK ever sees them. Passlog_level="INFO"(above).
Still nothing? Turn on the debug banner - it will tell you whether the SDK is exporting at all, which separates "my logs aren't reaching the SDK" from "the SDK isn't reaching the collector".
How do dev and prod stay separate?
The environment is not in your code, and it is not a variable you set either. You choose it when you register the product, and a deployment picks its environment by which key it holds. So the init line is identical in dev and in prod - there is nothing to edit, and nothing to forget to edit back, on the way to production. A key belongs to exactly one product in exactly one environment, so give every deployment the key that was created for it. That is what keeps dev out of your prod numbers - and it is enforced at the edge, not by convention: a leaked dev key can only ever write into dev. That is the whole mechanism - and note where it is enforced: the gateway stamps the environment from the key's registration and discards whatever the payload claimed, so a deployment cannot mislabel itself even by trying.
Every screen then carries an environment selector, and the numbers are computed inside that scope rather than filtered after the fact - so a prod p95 is a p95 of prod requests, not of everything with the dev ones hidden. A p95 with a developer's laptop blended into it is not a p95 of anything.
What does 'Unregistered' mean on the Products page?
Telemetry exists under a product name and environment that the registry has no row for. It cannot happen to new data: the ingest gateway derives the tenant, the product AND the environment from your API key and overwrites whatever the payload claimed, so everything that arrives lands under a product somebody set up here. A row reads Unregistered for one of three reasons - it is older data, its registration was deleted, or the registry is momentarily unreachable and we cannot tell what was set up from what was not.
It is never another tenant's data. The tenant comes from your key too, so an unregistered row is always something that was sent with one of yourkeys.
Why is a cost showing as '-'?
Because we don't have a price for that model - and "unknown" is not "zero". Cost is derived here from the raw token counts the SDK recorded (the wire never carries a cost), so a model missing from the price table produces no cost, and we render that as - rather than $0.00. Showing $0.00 would mean "this call was free", which is a claim, and a false one.
Spend & budgets names every unpriced model it saw - usually a brand-new model, or a fine-tuned name the price table doesn't know yet. The tokens are still recorded, so the moment a price exists the history prices itself.
Does the SDK work without an API key?
No - the key is the configuration. Since SDK 1.0.0 it is the only thing you pass, and it is the only thing that can say which tenant, product and environment your telemetry belongs to. Call init_observability() with no key (and no INDRATRACE_API_KEY in the environment) and it tells you so at startup, rather than starting up quietly and shipping into a 401 nobody ever sees.
Nothing about your data is locked in, though, and that is a different question: the wire is plain OTLP over HTTP, the SDK is Apache-2.0, and the attributes are the standard OpenTelemetry ones. Instrumenting with IndraTrace is instrumenting with OpenTelemetry.
Does instrumenting cost me latency?
Spans are batched and exported on a background timer, so the export is off your request's critical path. And if the collector is slow, unreachable, or misconfigured, the SDK logs one warning and carries on un-instrumented - your app keeps serving. The decorators hold to the same rule: they run your function even when init_observability() was never called at all.
How it compares
| Typical observability platform | IndraTrace | |
|---|---|---|
| Pricing | Per host, plus a meter for each product | A platform fee plus what you ingest |
| Where an incident lives | A separate product beside the dashboards | The front page, with the on-call assigned automaticallylive |
| AI calls | An add-on, metered in its own credits | The same trace tree as the request, with cost per calllive |
| Languages | A vendor agent per language | OpenTelemetry-native, one keylive |
| Security findings | A different tool and a different queue | Incidents with owners, in the same queuelive |
| AI assistant access | An assistant inside their own screens | An MCP connector with scopes and auditcoming |
| Setup | An agent on every host | One line in the app, or one Collector keylive |