Skip to content

Axiom log shipping is dropped: the onAfterResponse flush empties the buffer before the waitUntil flush #7

Description

@Joker666

On a deployment with AXIOM_API_TOKEN and AXIOM_LOGS_DATASET set, traces arrive in Axiom but logs never do. No error appears anywhere, and the same entries still show up in Cloudflare Workers Logs, so the Worker looks healthy.

The cause is two flush paths that race.

  1. runInvocation flushes correctly. Its finally calls ctx.waitUntil(flush(env)), so the ingest survives the end of the invocation (packages/observability/src/index.ts:230).
  2. apps/api/src/libs/logger.ts:7 also flushes, from an Elysia onAfterResponse hook mounted globally at apps/api/src/modules/index.ts:19. That hook runs after the response is sent and its await is not inside waitUntil.

On every request the second path wins:

  1. The handler logs into invocation.entries.
  2. onAfterResponse fires and calls flush.
  3. flush splices out every entry (index.ts:148) and writes each one to console, which is why Workers Logs has them.
  4. It starts the Axiom fetch, unprotected.
  5. runInvocation's finally runs ctx.waitUntil(flush(env)), but the buffer is empty, so it returns at the guard on index.ts:146 and registers nothing.
  6. The invocation ends and the orphaned fetch is cancelled.

Traces are unaffected because otel-cf-workers registers its own export through waitUntil.

A second defect hides the first. The ingest call at index.ts:161 never checks the response status. fetch rejects only on a transport error, so any 4xx from Axiom is discarded silently and the catch never runs. A log shipper that cannot report its own failure will hide the next one too.

Suggested fix: remove the onAfterResponse hook and let runInvocation own the flush for the fetch path. The Durable Object paths in apps/api/src/actor/subscriber.ts already use this.ctx.waitUntil(this.flush()) and are correct. Separately, check res.ok and log the status and body.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions