AIR CLI Integration: fast air list via active run index #5814
Open
riddhibhagwat-db wants to merge 1 commit into
Open
AIR CLI Integration: fast air list via active run index #5814riddhibhagwat-db wants to merge 1 commit into
air list via active run index #5814riddhibhagwat-db wants to merge 1 commit into
Conversation
…nal-run cache Ports the Python AIR CLI's fast `air list`. Three changes, matching Python: - Default to active-only; add `--all-status` (replaces `--active`). Plain `air list` no longer scans every run of every state, which was the main reason it lagged the Python CLI. - For `--all-status` scoped to yourself, fetch cheap (job_run_id, submit_time) pairs from the AiTrainingService index (/api/2.0/ai-training/workflows), order by submit time, keep the newest --limit, and hydrate only those via Jobs runs/get (concurrent, per-run ACL-enforced: 403/404 drops the id, other errors propagate). If the index is unavailable, silently fall back to the Jobs scan. --all-users and other-user filters always use the scan. - Cache hydrated terminal runs on disk (libs/cache, 60-day TTL) so repeat --all-status calls skip runs/get + get-output + MLflow for those ids. The runFetcher now wraps a listStrategy (jobsScanStrategy | indexStrategy) behind the same next(want)/exhausted contract, so the TUI, JSON, and one-shot paths are unchanged. Also converts listModel's fetch helpers to value receivers (fixes a recvcheck lint and a stale-`loading` guard bug). Co-authored-by: Isaac
Contributor
Approval status: pending
|
riddhibhagwat-db
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
riddhibhagwat-db
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
Collaborator
Integration test reportCommit: 7a64fc1
25 interesting tests: 13 SKIP, 9 RECOVERED, 3 flaky
Top 25 slowest tests (at least 2 minutes):
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
Default to active-only; add --all-status (replaces --active). Plain air list previously scanned every run of every state through Jobs runs/list. It now lists only active runs by default;
--all-statusopts into all states.AiTrainingService index fast path for --all-status scoped to yourself. Instead of scanning the Jobs firehose, it fetches cheap (job_run_id, submit_time) pairs from GET /api/2.0/ai-training/workflows, orders by submit time, keeps the newest --limit, and surfaces only those via concurrent Jobs runs/get. If the index is unavailable, it silently falls back to the Jobs scan so the command never hard-fails. --all-users and other-user filters always use the scan (the index is per-user only).
Terminal runs are immutable, so once hydrated, their row is cached; repeat
--all-statuscalls skip runs/get + get-output + MLflow for those ids.The runFetcher now wraps a listStrategy (jobsScanStrategy | indexStrategy) behind the same next(want)/exhausted contract, so the interactive table, JSON, and one-shot output paths are unchanged.
This PR also fixes a pre-existing recvcheck lint failure and a latent stale-loading guard bug in list_tui.go (fetch helpers converted to value receivers). The index path over-fetches (skips the newest-N truncation) when a --filter on task fields is active, so a filtered-out run can't shrink the result below --limit.
Why
air list in the Go CLI was noticeably slower than the Python AIR CLI — in both plain and --limit modes. The Python CLI's speed comes from three architectural choices, not a faster scan; this PR ports all three to reach parity.
Testing