feat(agent-observability): dashboard accuracy fixes + custom model pricing overrides - #12
Merged
Conversation
alberthaolihai
force-pushed
the
feat/agent-observability-dashboard-and-pricing
branch
from
September 15, 2026 06:04
1089873 to
e63c41c
Compare
…low runs Adds a Classic-only, opt-in skill that hooks into SessionStart / UserPromptSubmit / PreToolUse / PostToolUse / Stop to record tool calls, token usage, and cost into a local metrics log, plus a script that aggregates it (together with devflow's workflow-state.json) into a static dashboard: per-day cost/tokens, session and tool-call timelines, devflow stage progress, Skill/Rule hit rates, and Auto-Dispatch health. Also includes: - custom per-model pricing overrides for models missing from the built-in price table - accuracy fixes for the dashboard's turn/duration aggregation, dispatch attribution, session counting, and devflow stage classification (medium/large runs no longer misread as SOLO) - a fix for build-classic-hosts.py's generated-bundle check incorrectly flagging __pycache__/.pyc as unmanaged extras - cost-by-model chart fixes: long model names no longer get cut off by their bar, the chart gets a fairer share of its row's width, and the Token composition chart next to it is stretched to fill its row's actual height instead of leaving a gap underneath Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
alberthaolihai
force-pushed
the
feat/agent-observability-dashboard-and-pricing
branch
from
September 15, 2026 09:44
a2b3cbd to
79c1175
Compare
Robert0622
deleted the
feat/agent-observability-dashboard-and-pricing
branch
September 15, 2026 09:48
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
Two commits, both validated against real multi-agent CodeBuddy session data (not just unit tests):
fix: correct SendMessage matcher casing in devflow auto-dispatch hook— the PreToolUse hook fordevflow-auto-dispatch.pywas registered under matchersend_message, but real CodeBuddy CLI tool calls useSendMessage. The casing mismatch meant this hook never fired on any host, soauto_dispatch/fallback_to_mainstats stayed 0 regardless ofdevflow.defaults.yaml'sauto_dispatch.enabled: true. Verified end-to-end on a real medium devflow run after the fix: the Auto-Dispatch health panel showed 5auto_dispatch/ 0fallback_to_main, matching the 5 real*_completedrouting-table transitions in that run.feat: fix dashboard timeline/dispatch accuracy, add custom model pricing overrides:team-lead(a CodeBuddy team-infra alias for main) as if main had dispatched to a real sub-agent..codebuddy/agent-observability/pricing.overrides.json(orAOBS_PRICING_OVERRIDES_PATH), merged field-by-field on top of the built-inconfig/pricing.json— missing/invalid override data silently degrades to the built-in table so a bad config file can never break the hook. The dashboard's "建议关注" panel now flags models seen in usage events that still have zero pricing coverage. Delivered via a real medium-sized devflow run (6-role pipeline: architect/developer/code-reviewer/test-engineer/knowledge-engineer/leader), independently re-verified rather than trusting the team's own self-reported acceptance.raw_responserace condition investigation inquickstart.md: three fix attempts (two bounded-retry budgets, one deferred-retry-queue approach) were tried and reverted after real end-to-end verification exposed problems unit tests didn't catch — including a pre-existing shared byte-offset cursor bug between tool-context lookups and usage-event scanning. Left documented as a known, unfixed, low-impact (Bash-only, failure-detection-only) limitation rather than shipping an unverified fix.Scope
Compatibility
Affects the Classic
agent-observabilityskill (.codebuddy/skills/agent-observability/), synced to the generated.claude/.cursorhost bundles viabuild-classic-hosts.py --write. No migration needed: the pricing-overrides mechanism is opt-in (default behavior is byte-identical when no override file is present, verified via before/after snapshot diff), and the dashboard fixes only correct previously-inaccurate aggregation — no schema/config changes required from existing users.Validation
Safety
🤖 Generated with Claude Code