Bound OTel traces emitted by long-running subscription handlers - #63
Conversation
There was a problem hiding this comment.
Pull request overview
Removes the long-lived per-lifecycle OTel span (issue #56) so that spans started from L.Context() are roots of independent traces, and propagates lifecycle identity via context-stored attributes that a new NewSpanProcessor stamps onto every span. Adds bounded framework spans for runCleanup and Component.Bootstrap, a process-stable process.nonce, and Context-suffixed variants of Error/Fatal so per-handler spans can be the recording target.
Changes:
- New
tracing.go(tracer,processNonce,WithAttributes/WithForkAttributes,NewSpanProcessor) plus package doc;lifecycle.gono longer opens a lifecycle span and wraps cleanup in a bounded<name>.cleanupspan. Error/Fatal/WithSpan/WithForkSpanNamedeprecated; newErrorContext/FatalContext(+fvariants) record onto caller-supplied span; in-repo callers migrated.loaderback-linksClaim.Footprint, forks components with identity attributes viaclaimIdentity, emits a bounded<component>.bootstrapspan, and warns atLoadwith the process nonce;kafkaloader/fileloaderupdated accordingly.
Reviewed changes
Copilot reviewed 17 out of 17 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| tracing.go | New tracer, process nonce, identity-attribute context plumbing, and NewSpanProcessor. |
| tracing_test.go | Coverage for attribute propagation/replacement, nonce stamping, cleanup span, root spans, and context error/fatal recording. |
| lifecycle.go | Drops lifecycle span, stashes identity attrs on ctx, wraps cleanup in bounded span, deprecates Error/Fatal, adds *Context variants. |
| procedure.go | Removes span field and SpanName, deprecates WithSpan/WithForkSpanName, adds attrs to lifecycleOptions. |
| doc.go | New package doc describing the tracing contract, identity propagation, and operator anchor. |
| loader/loader.go | Back-links Claim.Footprint, logs process nonce, forks with WithForkAttributes, bounded <component>.bootstrap span, FatalfContext for bootstrap errors. |
| loader/example_test.go | Migrates Errorf → ErrorfContext. |
| fileloader/jsonloader.go | Fatalf → FatalfContext. |
| fileloader/yamlloader.go | Fatal → FatalfContext. |
| fileloader/fileloader.go | WithSpan("loader") → WithName("loader"). |
| kafkaloader/kafkaloader.go | WithSpan→WithName, error/fatal migrated to *Context variants, span ctx threaded into handler. |
| examples/direct/{main,ping,pong}.go | Migrate to *Context variants; pong.go adds bounded pong.echo consumer span. |
| examples/embed/{ping,pong,probe}.go | Migrate to ErrorfContext. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
|
This PR is stale because it has been open 14 days with no activity. Remove the stale label or comment to keep it open, otherwise it will be closed in 7 days. |
|
This PR was closed because it has been inactive for 21 days. |
|
This PR is stale because it has been open 14 days with no activity. Remove the stale label or comment to keep it open, otherwise it will be closed in 7 days. |
|
This PR is stale because it has been open 14 days with no activity. Remove the stale label or comment to keep it open, otherwise it will be closed in 7 days. |
|
This PR is stale because it has been open 14 days with no activity. Remove the stale label or comment to keep it open, otherwise it will be closed in 7 days. |
3a63a06 to
1ef2f21
Compare
Long-lived procedures can handle unrelated operations for the lifetime of a process. Keeping their span open made every operation part of one ever-growing trace, exhausting downstream trace limits. Leaving operation ownership with call sites also establishes the boundary the following component-attribution and span-linking work can enrich.
Removing lifecycle parent spans avoids unbounded traces, but would also remove the navigable relationship between an incoming operation and work started by a component. An opt-in processor preserves that relationship without making component own a TracerProvider or its sampling policy. The attribute is applied in time for exporter and Collector policies. Application-side component-aware head sampling remains a separate concern.
With lifecycle spans gone, recording errors through L.Context would be a silent no-op and Fatal would keep termination ownership hidden in the framework. The in-tree callers now own spans for the bounded work they perform and choose explicitly between returning and canceling managed children. The footprint consumer span ends before Load because loaded components may run for the lifetime of the process. Its context crosses the lifecycle boundary only so later operation roots can link back to it.
Call sites now own their bounded spans and error records, so Fatal's runtime.Goexit is no longer needed as the normal error path. ProcE can translate its returned error into lifecycle cancellation and then follow ordinary Go control flow, allowing callers and deferred work to continue predictably. Fatal and Fatalf retain their v1 behavior for compatibility while their deprecations direct new code to choose explicitly between return and Terminate followed by return.
Continue reports graceful-stop state only. Terminate cancels the lifecycle context without closing Stopping, so retrying canceled I/O under a Continue loop can spin forever and prevent the supervisor from completing. Treating context cancellation as the terminal branch lets managed workers release the lifecycle wait for both graceful and abrupt shutdown paths.
|
I ran this disposable program against PR head
|

Long-lived component procedures can process unrelated operations for hours, but the framework kept one OpenTelemetry span open across
Procedure.Exec. Handler spans inherited that lifecycle span throughL.Context, so trace size grew with component uptime and eventually exhausted downstream live-trace limits. This revision makes the lifecycle a control-flow boundary rather than a trace operation: work started by each handler owns a bounded trace that can complete independently.WithContextstill derives cancellation, deadlines, values, and baggage from its input, but detaches any active span before the procedure begins. The opt-inNewSpanProcessorrestores navigability without restoring accidental parentage: it stampscomponent.nameon lifecycle-derived spans and links the detached incoming span only to parentless operation roots. Ordinary child spans retain their normal parent, an explicitly supplied equivalent link is not duplicated, and acomponent.namesupplied when the span starts takes precedence over processor enrichment.The processor intentionally does not own OpenTelemetry Resource or sampling policy. Process identity such as
service.instance.idremains an application-bootstrap concern shared by traces, metrics, and logs. Processor attributes and links are available to exporters and Collector-side policies, but they run after in-process head sampling and therefore cannot influence that decision.Operation owners now start spans where bounded work begins and handle logging, error recording, span status, and return behavior at the same call site.
L.Errorno longer attempts to mutate the removed lifecycle span.L.FatalandL.Fatalfare deprecated while retaining their v1 behavior; new code returns normally for leaf failures and callsL.Terminatebefore returning only when managed children need cancellation.ProcEfollows that supervising form by cancelling with the returned error and then returning normally.This deliberately omits the older branch's broad trace-attribute options, context-bearing Fatal helpers, and generated
process.nonce. It also leaves a possible call-site convenience such ascomponent.Exitfor later evidence rather than hiding logging, tracing, and cancellation policy in the core fix. The earlier detachment-only approach remains available for comparison in #64.The reference loaders and message-loop examples instrument bounded bootstrap, producer, and consumer spans.
Proof: #63 (exported trace topology)
Resolves #56.