Skip to content

[MINOR] [CORE] Remove registered executors in soft affinity UT - #13054

Merged
marin-ma merged 1 commit into
apache:mainfrom
marin-ma:remove-executor-softaffinity-suite
Sep 21, 2026
Merged

marin-ma merged 1 commit into
apache:mainfrom
marin-ma:remove-executor-softaffinity-suite

Conversation

@marin-ma

Copy link
Copy Markdown
Contributor

Probably fix the CI failure in #13039

https://opencicd.kyligence.com/job/gluten/job/gluten-ci/21393/

[2026-09-17T16:11:27.803Z] ColumnarShuffleManagerSuite:
  [2026-09-17T16:11:27.803Z] - is a SortShuffleManager so row-based exchanges keep Spark's zero-copy write path
  [2026-09-17T16:11:27.803Z] SoftAffinityWithRDDInfoSuite:
  [2026-09-17T16:11:27.803Z] - Soft Affinity Scheduler with duplicate reading detection *** FAILED ***
  [2026-09-17T16:11:27.803Z]   Expected Set("executor_host-0_0"), but got Set("host-3") (SoftAffinityWithRDDInfoSuite.scala:102)

Copilot AI lite review requested due to automatic review settings September 18, 2026 10:35
@github-actions github-actions Bot added the CORE works for Gluten Core label Sep 18, 2026
@github-actions

Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Cleanup must be guaranteed with lifecycle or finally handling if earlier assertions fail.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds executor cleanup to prevent SoftAffinityManager state leaking between soft-affinity tests.

Changes:

  • Removes all registered executors at test completion.
  • Verifies both executor registries are empty.
File summaries
File Description
gluten-substrait/src/test/scala/org/apache/spark/softaffinity/SoftAffinitySuite.scala Adds executor removal and state assertions.
Review details
  • Files reviewed: 1/1 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +273 to +277
executorsListListener.onExecutorRemoved(removedEvent0)
executorsListListener.onExecutorRemoved(removedEvent1)
executorsListListener.onExecutorRemoved(removedEvent2)
executorsListListener.onExecutorRemoved(removedEvent3)
executorsListListener.onExecutorRemoved(removedEvent4)
@zhouyuan
zhouyuan requested a review from zzcclp September 20, 2026 08:43
@acvictor acvictor self-assigned this Sep 21, 2026
@acvictor acvictor removed their assignment Sep 21, 2026
@marin-ma
marin-ma merged commit 3a0c764 into apache:main Sep 21, 2026
57 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CORE works for Gluten Core

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants