From 5e36f947f30b00f8ae4ee76573bd4799be2f0d1d Mon Sep 17 00:00:00 2001 From: Praveen K B Date: Fri, 31 Jul 2026 11:40:01 +0530 Subject: [PATCH] Updated OpenRouter docs --- .../docs/ingest-data/ai-agents/openrouter.mdx | 219 +++++++++++++----- 1 file changed, 164 insertions(+), 55 deletions(-) diff --git a/content/docs/ingest-data/ai-agents/openrouter.mdx b/content/docs/ingest-data/ai-agents/openrouter.mdx index c37a0f7..4cb8b72 100644 --- a/content/docs/ingest-data/ai-agents/openrouter.mdx +++ b/content/docs/ingest-data/ai-agents/openrouter.mdx @@ -3,83 +3,106 @@ title: OpenRouter description: Send LLM traces to Parseable using OpenRouter Broadcast --- -Send LLM traces to Parseable using OpenRouter's Broadcast feature — no code changes required. +Send LLM traces to Parseable using OpenRouter's Broadcast feature — no application instrumentation required. ## Overview -[OpenRouter](https://openrouter.ai) is a unified API for accessing 100+ LLM models. Its Broadcast feature automatically sends OpenTelemetry traces for every LLM request to your configured destinations. +[OpenRouter](https://openrouter.ai) is a unified API for accessing 100+ LLM models. Its [Broadcast](https://openrouter.ai/docs/guides/features/broadcast) feature sends an OpenTelemetry (OTLP/HTTP, JSON-encoded) trace for every LLM request to destinations you configure in **Settings > Observability**. Parseable accepts OTLP traces natively, so the two integrate directly. Integrate OpenRouter with Parseable to: -- **Zero-Code Observability** - No application instrumentation needed -- **Track All Requests** - Monitor every LLM call automatically -- **Analyze Costs** - Built-in cost tracking per request -- **Debug Issues** - Full trace context for troubleshooting +- **Zero-code observability** - no SDK, no app changes +- **Track every request** - prompt, completion, model, provider, latency +- **Analyze cost** - per-request token and cost breakdown +- **Debug issues** - full trace/span context, including provider retry attempts -## How It Works +There are two ways to wire this up, depending on whether your Parseable endpoint is directly reachable by OpenRouter's servers: -OpenRouter Broadcast automatically sends OpenTelemetry-formatted traces to your configured endpoints. Combined with Parseable's native OTLP ingestion, you get full LLM observability in minutes. +- **Direct** - Broadcast sends OTLP straight to your Parseable OTLP endpoint. Use this whenever Parseable is reachable over HTTPS from the public internet. +- **Via OTel Collector** - Broadcast sends OTLP to a collector you run (locally or in your infra), which forwards to Parseable. Use this when Parseable's endpoint is HTTP-only, on a private network, or you want to transform/route/sample traces before they land in Parseable. OpenRouter's dashboard only accepts `https://` destination URLs, so the collector's public-facing side needs TLS even if Parseable itself doesn't. ``` -Your App → OpenRouter → LLM Provider - ↓ - Broadcast - ↓ - Parseable +Direct: + Your App → OpenRouter → LLM Provider + ↓ + Broadcast (OTLP/HTTPS) + ↓ + Parseable + +Via collector: + Your App → OpenRouter → LLM Provider + ↓ + Broadcast (OTLP/HTTPS) + ↓ + OTel Collector (public HTTPS ingress) + ↓ + Parseable (OTLP/HTTP, private/internal) ``` -## Setup +## Prerequisites -### Step 1: Get Your Parseable Endpoint +- An OpenRouter account with **Organization Admin** access (Broadcast settings are admin-only) +- An OpenRouter API key for sending test requests +- A reachable Parseable instance and an API key (or Basic auth credentials) +- For the collector flow: `otelcol-contrib` (or any OpenTelemetry Collector distro with the OTLP receiver + OTLP/HTTP exporter) and a way to expose it over HTTPS (a reverse proxy, load balancer, or a tunnel such as `cloudflared` for local testing) -Your Parseable OTLP endpoint: +## Flow 1: Direct to Parseable + +Use this when your Parseable instance already has an HTTPS endpoint OpenRouter can reach. + +### Step 1: Parseable OTLP endpoint ``` -https://your-parseable-host:8000/v1/traces +https://:8000/v1/traces ``` -### Step 2: Prepare Authentication Headers - -Encode your Parseable credentials: +### Step 2: Headers -```bash -echo -n "username:password" | base64 -``` +Parseable's OTLP ingestion needs three headers on every request: -Required headers: +| Header | Purpose | +|--------|---------| +| `X-API-Key` | Parseable API key (preferred), **or** `Authorization: Basic ` — never send both | +| `X-P-Stream` | Target dataset (stream) name, e.g. `openrouter-traces` | +| `X-P-Log-Source` | Must be exactly `otel-traces` | ```json { - "Authorization": "Basic ", + "X-API-Key": "", "X-P-Stream": "openrouter-traces", "X-P-Log-Source": "otel-traces" } ``` -| Header | Purpose | -|--------|---------| -| `Authorization` | Basic auth with Parseable credentials | -| `X-P-Stream` | Target dataset name in Parseable | -| `X-P-Log-Source` | Must be `otel-traces` for trace data | - -### Step 3: Configure OpenRouter Broadcast +If using Basic auth instead of an API key: -In OpenRouter dashboard, add a new Broadcast destination: +```bash +echo -n "username:password" | base64 +``` -- **Name**: Parseable Production -- **Endpoint**: `https://.parseable.com/v1/traces` -- **Headers**: ```json { - "Authorization": "Basic ", + "Authorization": "Basic ", "X-P-Stream": "openrouter-traces", "X-P-Log-Source": "otel-traces" } ``` -### Step 4: Configure Sampling (Optional) +### Step 3: Configure Broadcast + +In the OpenRouter dashboard: **Settings > Observability > Enable Broadcast > Add Destination > OpenTelemetry Collector**. -For high-volume applications, sample traces to control costs: +- **Endpoint**: `https://:8000/v1/traces` +- **Headers**: the JSON object from Step 2 +- Click **Test Connection** before saving — OpenRouter sends a synthetic `openrouter-connection-test` span to verify the destination accepts data. + + + The dataset is created automatically on first ingest, with `telemetryType: traces`. If a stream with that name already exists as a plain log stream (created via `/api/v1/ingest` instead of `/v1/traces`), OTLP ingestion will fail — delete and let the OTLP endpoint recreate it, or use a new stream name. + + +### Step 4: Sampling (optional) + +For high-volume apps, sample traces to control ingest volume: | Sampling Rate | Use Case | |---------------|----------| @@ -87,6 +110,78 @@ For high-volume applications, sample traces to control costs: | 0.1 (10%) | Medium-volume production | | 0.01 (1%) | High-volume production | +## Flow 2: Via a local/self-hosted OTel Collector + +Use this when Parseable's ingest endpoint is HTTP-only, sits behind a private network/VPN, or you want a place to transform, filter, or fan out traces before Parseable. OpenRouter's Broadcast destination still needs an `https://` URL, so the collector needs a TLS-terminated front door — a reverse proxy in production, or a quick tunnel for local testing. + +### Step 1: Collector config + +```yaml title="otel-collector-config.yaml" +receivers: + otlp: + protocols: + http: + endpoint: 0.0.0.0:4318 + grpc: + endpoint: 0.0.0.0:4317 + +exporters: + otlphttp/parseable: + traces_endpoint: http://:/v1/traces + encoding: json + headers: + X-API-Key: "" + X-P-Stream: "openrouter-traces" + X-P-Log-Source: "otel-traces" + +service: + pipelines: + traces: + receivers: [otlp] + exporters: [otlphttp/parseable] +``` + +```bash +otelcol-contrib --config otel-collector-config.yaml +``` + +The collector's HTTP receiver listens on `4318`, exposing an OTLP endpoint at `http://localhost:4318/v1/traces` — this is what Broadcast will ultimately hit, once fronted by HTTPS. + +### Step 2: Expose the collector over HTTPS + +For local testing, a quick tunnel is the fastest path: + +```bash +cloudflared tunnel --url http://localhost:4318 +``` + +This prints a public URL like `https://.trycloudflare.com`. In production, put the collector behind a real reverse proxy/load balancer with a certificate instead of a quick tunnel. + + + Quick tunnels (`cloudflared tunnel --url ...` with no named tunnel) have no uptime guarantee and die when the local process stops. Fine for validating the pipeline; not for production traffic. + + +### Step 3: Configure Broadcast + +In the OpenRouter dashboard: **Settings > Observability > Enable Broadcast > Add Destination > OpenTelemetry Collector**. + +- **Endpoint**: `https://.trycloudflare.com/v1/traces` +- **Headers**: leave empty — the collector already injects the Parseable auth/stream/log-source headers in its exporter config +- Click **Test Connection**, then check the collector's logs for an incoming `openrouter-connection-test` span + +### Step 4: Verify end to end + +Send a real request through OpenRouter, then check the collector log and Parseable: + +```bash +curl -X POST https://openrouter.ai/api/v1/chat/completions \ + -H "Authorization: Bearer $OPENROUTER_API_KEY" \ + -H "Content-Type: application/json" \ + -d '{"model":"openai/gpt-4o-mini","messages":[{"role":"user","content":"Say OK"}]}' +``` + +Broadcast delivery is async — traces typically land within 5-15 seconds of the response. + ## Enriching Traces Add context to your API requests for better observability. @@ -137,21 +232,25 @@ response = client.chat.completions.create( ## Trace Schema -OpenRouter traces include these fields: +Parseable stores OTLP trace fields flattened, with dots preserved in column names — quote them in SQL. Verified fields from a live OpenRouter trace: | Field | Description | |-------|-------------| -| `trace_id` | Unique trace identifier | -| `span_id` | Unique span identifier | -| `span_name` | Operation name | -| `span_duration_ns` | Duration in nanoseconds | -| `gen_ai.request.model` | Requested model | -| `gen_ai.response.model` | Actual model used | -| `gen_ai.usage.input_tokens` | Input token count | -| `gen_ai.usage.output_tokens` | Output token count | -| `gen_ai.usage.cost` | Estimated cost | -| `user.id` | User identifier (if provided) | -| `session.id` | Session identifier (if provided) | +| `span_trace_id` / `span_span_id` | OTLP trace/span identifiers | +| `span_name` | e.g. `LLM Generation`, `provider attempt 1: ` | +| `span_duration_ns`, `span_start_time_unix_nano`, `span_end_time_unix_nano` | Timing | +| `span_status_code` | Span status (0 unset, 1 ok, 2 error) | +| `"gen_ai.request.model"` | Requested model slug | +| `"gen_ai.response.model"` | Actual model used | +| `"gen_ai.response.id"` | OpenRouter generation ID (`gen-...`) | +| `"gen_ai.usage.input_tokens"` / `"gen_ai.usage.output_tokens"` / `"gen_ai.usage.total_tokens"` | Token counts | +| `"gen_ai.usage.input_cost"` / `"gen_ai.usage.output_cost"` / `"gen_ai.usage.total_cost"` | Cost in USD | +| `"gen_ai.prompt"` / `"gen_ai.completion"` | Request/response content | +| `"trace.metadata.openrouter.provider_name"` | Upstream provider that served the request (e.g. `Azure`) | +| `"trace.metadata.openrouter.api_key_name"` | Name of the OpenRouter API key used | +| `"trace.metadata.openrouter.finish_reason"` | Completion finish reason | + +If you enrich requests with `user` / `session_id` (see below), OpenRouter maps those to `user.id` and `session.id` span attributes. ## Example Queries @@ -191,7 +290,7 @@ ORDER BY avg_latency_ms DESC; SELECT "user.id" AS user_id, COUNT(*) AS requests, - SUM(CAST("gen_ai.usage.cost" AS DOUBLE)) AS total_cost_usd + SUM(CAST("gen_ai.usage.total_cost" AS DOUBLE)) AS total_cost_usd FROM "openrouter-traces" WHERE p_timestamp > NOW() - INTERVAL '7 days' AND "user.id" IS NOT NULL @@ -237,7 +336,7 @@ ORDER BY error_rate DESC; Monitor when daily LLM spend exceeds budget: - **Stream**: `openrouter-traces` -- **Column**: `gen_ai.usage.cost` +- **Column**: `gen_ai.usage.total_cost` - **Aggregation**: `SUM` - **Threshold**: `> 100` @@ -274,7 +373,7 @@ ORDER BY hour; ```sql SELECT DATE_TRUNC('day', p_timestamp) AS day, - SUM(CAST("gen_ai.usage.cost" AS DOUBLE)) AS daily_cost + SUM(CAST("gen_ai.usage.total_cost" AS DOUBLE)) AS daily_cost FROM "openrouter-traces" WHERE p_timestamp > NOW() - INTERVAL '30 days' GROUP BY day @@ -291,7 +390,17 @@ OpenRouter supports up to 5 destinations. Use this for: | Development | `openrouter-dev` | 100% | `dev-*` keys | | Debugging | `openrouter-debug` | 100% | Specific key | +## Troubleshooting + +- **`Header Error: Log source not found in header [x-p-log-source]`** - Add `X-P-Log-Source: otel-traces` to the destination headers (direct flow) or the collector exporter config (collector flow). Required on every request, not just the first. +- **`Logstream ... already exists, please create a new log stream with unique name`** on first OTLP send - the stream name is already taken by a non-traces stream. Delete it (`DELETE /api/v1/logstream/`) or pick a new name. +- **Test Connection succeeds but no traces from real requests** - Broadcast delivery is async; wait 15-30s. Also confirm the request actually went through OpenRouter (a raw provider API key bypasses OpenRouter entirely and produces no trace). +- **Collector flow: Test Connection fails** - confirm the collector process and tunnel/proxy are both still running, and that the tunnel's public hostname resolves (`dig `). Quick tunnels can take a few seconds after creation before DNS propagates globally. +- **Collector flow: traces reach the collector but not Parseable** - check the collector's own logs for exporter errors; verify `X-API-Key` / `X-P-Stream` / `X-P-Log-Source` are set in the exporter's `headers` block, and that the collector can reach Parseable's ingest host/port. +- **Using a BYOK provider key (e.g. your own OpenAI key added under OpenRouter's BYOK settings)** - traces still generate normally; the request still flows through OpenRouter, so Broadcast still fires. A raw key used *outside* OpenRouter (calling OpenAI directly) does not. + ## Resources -- [OpenRouter Documentation](https://openrouter.ai/docs) +- [OpenRouter Broadcast Documentation](https://openrouter.ai/docs/guides/features/broadcast) +- [OpenRouter OTel Collector Destination Documentation](https://openrouter.ai/docs/guides/features/broadcast/otel-collector) - [Blog Post](/blog/openrouter-broadcast-parseable-llm-observability)