Skip to content

Produce test-output.json even when a test leg fails#5806

Open
pietern wants to merge 1 commit into
mainfrom
fix-testjson-upload
Open

Produce test-output.json even when a test leg fails#5806
pietern wants to merge 1 commit into
mainfrom
fix-testjson-upload

Conversation

@pietern

@pietern pietern commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

We upload test-output.json (the concatenated gotestsum output) as a CI artifact to inspect results and timing after the fact, most importantly when a test fails. But the file is produced by a cat in the test task's cmds, which is skipped when a deps leg fails, so on a failed run the file is never written and the upload finds nothing. This moves the cat (in both test and test-unit) into a defer:, which runs even after a leg fails, so the artifact is always produced. The task still exits non-zero, so CI failures are unaffected.

For the defer to have something to concatenate, the legs now run as sequential cmds rather than parallel deps. An A/B CI run (#5802) showed no timing regression (serial was equal or faster on 5 of 6 matrix cells). Both legs already saturate the runners, so running them in parallel mostly just created core contention.

This pull request and its description were written by Isaac.

We upload test-output.json (the concatenated gotestsum output) as a CI
artifact to inspect results and timing after the fact, most importantly
when a test fails. But the file is produced by a cat in the test task's
cmds, which is skipped when a deps leg fails, so on a failed run the file
is never written and the upload finds nothing. Move the cat (in both
test and test-unit) into a defer, which runs even after a leg fails, so
the artifact is always produced. The task still exits non-zero, so CI
failures are unaffected.

For the defer to have something to concatenate, the legs now run as
sequential cmds rather than parallel deps. An A/B CI run showed no timing
regression; both legs already saturate the runners, so running them in
parallel mostly just created core contention.

Co-authored-by: Isaac
@pietern pietern temporarily deployed to test-trigger-is July 2, 2026 14:07 — with GitHub Actions Inactive
@pietern pietern temporarily deployed to test-trigger-is July 2, 2026 14:07 — with GitHub Actions Inactive
@pietern pietern requested review from denik and janniklasrose and removed request for janniklasrose July 2, 2026 14:08
@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Waiting for approval

Based on git history, these people are best suited to review:

  • @denik -- recent work in ./
  • @janniklasrose -- recent work in ./

Eligible reviewers: @andrewnester, @anton-107, @renaudhartert-db, @shreyas-goenka, @simonfaltum

Suggestions based on git history. See OWNERS for ownership rules.

@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 90aceac

Run: 28596398062

Env ❌​FAIL 🟨​KNOWN 🔄​flaky 💚​RECOVERED 🙈​SKIP ✅​pass 🙈​skip Time
💚​ aws linux 10 13 230 1041 4:36
💚​ aws windows 10 13 232 1039 3:52
💚​ aws-ucws linux 10 13 314 959 4:53
💚​ aws-ucws windows 10 13 316 957 4:51
💚​ azure linux 4 15 230 1040 4:18
🔄​ azure windows 2 2 15 232 1038 5:49
💚​ azure-ucws linux 4 15 316 956 5:02
💚​ azure-ucws windows 4 15 318 954 3:47
❌​ gcp linux 201 2 15 28 1031 1:49
❌​ gcp windows 203 2 15 28 1029 3:02
226 interesting tests: 203 FAIL, 13 SKIP, 6 RECOVERED, 2 KNOWN, 2 flaky
Test Name aws linux aws windows aws-ucws linux aws-ucws windows azure linux azure windows azure-ucws linux azure-ucws windows gcp linux gcp windows
🟨​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 🟨​K 🟨​K
❌​ TestAccept/bundle/deploy/mlops-stacks ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestAccept/bundle/deploy/mlops-stacks/DATABRICKS_BUNDLE_ENGINE=direct ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestAccept/bundle/deploy/mlops-stacks/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
🙈​ TestAccept/bundle/invariant/no_drift 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/permissions 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions 💚​R 💚​R 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=direct 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions 💚​R 💚​R 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=direct 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 💚​R 💚​R 💚​R 💚​R
🙈​ TestAccept/bundle/resources/postgres_branches/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/recreate 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/replace_existing 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/update_protected 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/without_branch_id 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_endpoints/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_projects/update_display_name 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/synced_database_tables/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
❌​ TestAccept/selftest/record_cloud/basic ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestAccept/selftest/record_cloud/basic/DATABRICKS_BUNDLE_ENGINE=direct ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestAccept/selftest/record_cloud/basic/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestAccept/selftest/record_cloud/error ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestAccept/selftest/record_cloud/error/DATABRICKS_BUNDLE_ENGINE=direct ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestAccept/selftest/record_cloud/error/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestAccept/selftest/record_cloud/pipeline-crud ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestAccept/selftest/record_cloud/pipeline-crud/DATABRICKS_BUNDLE_ENGINE=direct ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestAccept/selftest/record_cloud/pipeline-crud/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestAccept/selftest/record_cloud/workspace-file-io ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestAccept/selftest/record_cloud/workspace-file-io/DATABRICKS_BUNDLE_ENGINE=direct ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestAccept/selftest/record_cloud/workspace-file-io/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
🙈​ TestAccept/ssh/connection 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
❌​ TestDashboardAssumptions_WorkspaceImport ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestBundleInitErrorOnUnknownFields ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestBundleInitHelpers ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestAlertsCreateErrWhenNoArguments ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestApiGet ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestAuthDescribeSuccess ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestClustersGet ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestClustersList ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCat ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCat/dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCatDoesNotSupportOutputModeJson ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCatForDbfsInvalidScheme ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCatOnADir ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCatOnADir/dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCatOnNonExistentFile ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCatOnNonExistentFile/dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCompletion ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpDir ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpDir/dbfs_to_dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpDir/dbfs_to_local ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpDir/dbfs_to_uc-volumes 🙈​s 🙈​s ✅​p ✅​p 🙈​s 🙈​s ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpDir/local_to_dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpDir/local_to_local ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpDirToDirFileNotOverwritten ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpDirToDirFileNotOverwritten/dbfs_to_dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpDirToDirFileNotOverwritten/dbfs_to_local ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpDirToDirFileNotOverwritten/dbfs_to_uc-volumes 🙈​s 🙈​s ✅​p ✅​p 🙈​s 🙈​s ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpDirToDirFileNotOverwritten/local_to_dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpDirToDirFileNotOverwritten/local_to_local ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpDirToDirWithOverwriteFlag ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpDirToDirWithOverwriteFlag/dbfs_to_dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpDirToDirWithOverwriteFlag/dbfs_to_local ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpDirToDirWithOverwriteFlag/dbfs_to_uc-volumes 🙈​s 🙈​s ✅​p ✅​p 🙈​s 🙈​s ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpDirToDirWithOverwriteFlag/local_to_dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpDirToDirWithOverwriteFlag/local_to_local ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpErrorsOnInvalidScheme ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpErrorsWhenSourceIsDirWithoutRecursiveFlag ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpErrorsWhenSourceIsDirWithoutRecursiveFlag/dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpFileToDir ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpFileToDir/dbfs_to_dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpFileToDir/dbfs_to_local ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpFileToDir/dbfs_to_uc-volumes 🙈​s 🙈​s ✅​p ✅​p 🙈​s 🙈​s ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpFileToDir/local_to_dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpFileToDir/local_to_local ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpFileToDirFileNotOverwritten ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpFileToDirFileNotOverwritten/dbfs_to_dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpFileToDirFileNotOverwritten/dbfs_to_local ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpFileToDirFileNotOverwritten/dbfs_to_uc-volumes 🙈​s 🙈​s ✅​p ✅​p 🙈​s 🙈​s ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpFileToDirFileNotOverwritten/local_to_dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpFileToDirFileNotOverwritten/local_to_local ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpFileToDirForWindowsPaths 🙈​s ✅​p 🙈​s ✅​p 🙈​s ✅​p 🙈​s ✅​p 🙈​s ❌​F
❌​ TestFsCpFileToDirWithOverwriteFlag ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpFileToDirWithOverwriteFlag/dbfs_to_dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpFileToDirWithOverwriteFlag/dbfs_to_local ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpFileToDirWithOverwriteFlag/dbfs_to_uc-volumes 🙈​s 🙈​s ✅​p ✅​p 🙈​s 🙈​s ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpFileToDirWithOverwriteFlag/local_to_dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpFileToDirWithOverwriteFlag/local_to_local ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpFileToFile ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpFileToFile/dbfs_to_dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpFileToFile/dbfs_to_local ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpFileToFile/dbfs_to_uc-volumes 🙈​s 🙈​s ✅​p ✅​p 🙈​s 🙈​s ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpFileToFile/local_to_dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpFileToFile/local_to_local ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpFileToFileFileNotOverwritten ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpFileToFileFileNotOverwritten/dbfs_to_dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpFileToFileFileNotOverwritten/dbfs_to_local ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpFileToFileFileNotOverwritten/dbfs_to_uc-volumes 🙈​s 🙈​s ✅​p ✅​p 🙈​s 🙈​s ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpFileToFileFileNotOverwritten/local_to_dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpFileToFileFileNotOverwritten/local_to_local ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpFileToFileWithOverwriteFlag ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpFileToFileWithOverwriteFlag/dbfs_to_dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpFileToFileWithOverwriteFlag/dbfs_to_local ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpFileToFileWithOverwriteFlag/dbfs_to_uc-volumes 🙈​s 🙈​s ✅​p ✅​p 🙈​s 🙈​s ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpFileToFileWithOverwriteFlag/local_to_dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpFileToFileWithOverwriteFlag/local_to_local ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpFileToNonExistentDir ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpFileToNonExistentDir/dbfs_to_dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpFileToNonExistentDir/dbfs_to_local ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpFileToNonExistentDir/dbfs_to_uc-volumes 🙈​s 🙈​s ✅​p ✅​p 🙈​s 🙈​s ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpFileToNonExistentDir/local_to_dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpFileToNonExistentDir/local_to_local ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpFileToNonExistentDirWindowsPaths 🙈​s ✅​p 🙈​s ✅​p 🙈​s ✅​p 🙈​s ✅​p 🙈​s ❌​F
❌​ TestFsCpSourceIsDirectoryButTargetIsFile ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpSourceIsDirectoryButTargetIsFile/dbfs_to_dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpSourceIsDirectoryButTargetIsFile/dbfs_to_local ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpSourceIsDirectoryButTargetIsFile/dbfs_to_uc-volumes 🙈​s 🙈​s ✅​p ✅​p 🙈​s 🙈​s ✅​p ✅​p ❌​F ❌​F
❌​ TestFsCpSourceIsDirectoryButTargetIsFile/local_to_dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsLs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsLs/dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsLsForNonexistingDir ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsLsForNonexistingDir/dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsLsOnEmptyDir ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsLsOnEmptyDir/dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsLsOnFile ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsLsOnFile/dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsLsWithAbsolutePaths ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsLsWithAbsolutePaths/dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsLsWithoutScheme ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsMkdir ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsMkdir/dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsMkdirCreatesIntermediateDirectories ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsMkdirCreatesIntermediateDirectories/dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsMkdirWhenDirectoryAlreadyExists ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsMkdirWhenDirectoryAlreadyExists/dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsMkdirWhenFileExistsAtPath ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsMkdirWhenFileExistsAtPath/dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsRmDirRecursively ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsRmDirRecursively/dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsRmEmptyDir ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsRmEmptyDir/dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsRmFile ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsRmFile/dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsRmForNonExistentFile ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsRmForNonExistentFile/dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsRmNonEmptyDirectory ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestFsRmNonEmptyDirectory/dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestSecretsPutSecretBytesValue ✅​p ✅​p 🙈​s 🙈​s ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestSecretsPutSecretStringValue ✅​p ✅​p 🙈​s 🙈​s ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestSyncEnsureRemotePathIsUsableIfRepoDoesntExist ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestSyncEnsureRemotePathIsUsableIfRepoExists ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestSyncEnsureRemotePathIsUsableInWorkspace ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestSyncFullFileSync ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestSyncIncrementalFileOverwritesFolder ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestSyncIncrementalFileSync ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestSyncIncrementalSyncFileToPythonNotebook ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestSyncIncrementalSyncPythonNotebookDelete ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestSyncIncrementalSyncPythonNotebookToFile ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestSyncNestedFolderDoesntFailOnNonEmptyDirectory ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestSyncNestedFolderSync ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestSyncNestedSpacePlusAndHashAreEscapedSync ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestExport ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestExportDir ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestExportDirDoesNotOverwrite ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestExportDirWithOverwriteFlag ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
❌​ TestExportWithFileFlag ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
Top 4 slowest tests (at least 2 minutes):
duration env testname
3:01 aws-ucws windows TestAccept
2:54 azure-ucws windows TestAccept
2:39 azure windows TestAccept
2:38 aws windows TestAccept
(56 table rows omitted to keep the report under 60000 bytes)

Comment thread Taskfile.yml
- task: test-acc
vars:
ACCEPTANCE_TEST_FILTER: "{{.ACCEPTANCE_TEST_FILTER}}"
# Run the legs as sequential cmds (not parallel deps) so the concatenating

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we keep parallel runs though? it saves time

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if not possible, then we can concatenate outside of taskfile

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I measure the diff and it is negligible. Each test run already saturates CPU. See #5802.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants