Skip to content

HDDS-15941. Resolve client before block loop and return null from sortDatanodesForWrite - #11235

Open
chihsuan wants to merge 6 commits into
apache:masterfrom
chihsuan:HDDS-15941
Open

chihsuan wants to merge 6 commits into
apache:masterfrom
chihsuan:HDDS-15941

Conversation

@chihsuan

@chihsuan chihsuan commented Sep 13, 2026

Copy link
Copy Markdown
Contributor

What changes were proposed?

Follow-up to #10633.

  • Updates OM write-path datanode sorting to resolve the client once.
  • Let SCM sort if OM cannot find the client's rack.
  • Return null when OM skips sorting because a pipeline datanode is missing from its topology, preserving the pipeline's order.

JIRA

https://issues.apache.org/jira/browse/HDDS-15941

Testing

  • TestOMAllocateBlockRequest and TestOMSortDatanodes passed

https://github.com/chihsuan/ozone/actions/runs/34728833801

Generated-by: Claude Code (Claude Opus 5)

Copilot AI lite review requested due to automatic review settings September 13, 2026 00:17

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No unresolved issues were identified that would block approval.

Pull request overview

Updates OM write-path datanode sorting to resolve the client once, delegate unresolved clients to SCM, and preserve pipeline order when topology data is incomplete.

Changes:

  • Added client resolution before block allocation.
  • Changed skipped write sorting to return null.
  • Expanded unit and integration tests for fallback, caching, and topology cases.
File summaries
File Description
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/key/TestOMAllocateBlockRequest.java Updated as part of this pull request.
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyRequest.java Updated as part of this pull request.
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java Updated as part of this pull request.
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManager.java Updated as part of this pull request.
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/TestOMSortDatanodes.java Updated as part of this pull request.
Review details
  • Files reviewed: 5/5 changed files
  • Comments generated: 0
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@chihsuan
chihsuan marked this pull request as ready for review September 13, 2026 00:57
public Node resolveClientForWrite(String clientMachine, NetworkTopology clusterMap) {
Preconditions.checkArgument(!StringUtils.isEmpty(clientMachine), "clientMachine is empty");
Objects.requireNonNull(clusterMap, "clusterMap is null");
return captureLatencyNs(metrics.getAllocateBlockSortDatanodesLatencyNs(),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use a separate metric for client resolution. ConcurrentMutableRate exports one NumOps/AvgTime pair, so recording the once-per-request lookup here and every uncached pipeline sort into AllocateBlockSortDatanodesLatencyNs makes the count and average represent neither operation. Please add AllocateBlockResolveClientLatencyNs and keep the existing metric for sorting.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants