Skip to content

feat(gax): enforce global timeout for resumable uploads - #14425

Draft
whowes wants to merge 1 commit into
whowes/resumable-upload-recoveryfrom
whowes/resumable-upload-watchdog
Draft

whowes wants to merge 1 commit into
whowes/resumable-upload-recoveryfrom
whowes/resumable-upload-watchdog

Conversation

@whowes

@whowes whowes commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

Enforces ResumableUploadCallSettings.getGlobalTimeout() in ResumableUploadChunkCoordinator across the upload lifecycle. Cancels in-flight RPCs and completes the future with DeadlineExceededException when the deadline is exceeded.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces global timeout handling for resumable uploads in ResumableUploadChunkCoordinator using a scheduled executor to trigger a DeadlineExceededException if the upload exceeds the specified duration. It also includes comprehensive unit tests to verify timeout behavior, clean cancellations, and payload stream closure. The review feedback highlights two key improvements: ensuring thread safety by reading uploadSessionUrl within a synchronized block in onTimeout(), and preventing potential test hangs by specifying timeouts on future.get() calls in the newly added unit tests.

Comment on lines +214 to +225
private void onTimeout() {
synchronized (lock) {
if (done) {
return;
}
}
String message =
uploadSessionUrl != null
? "Resumable upload timed out for session: " + uploadSessionUrl
: "Resumable upload timed out before session initiation completed";
finish(null, new DeadlineExceededException(message, null, TIMEOUT_STATUS_CODE, false));
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The field uploadSessionUrl is read outside of the synchronized (lock) block. If uploadSessionUrl is not declared volatile or is guarded by lock, this can lead to a data race and memory visibility issues across threads (e.g., the timeout thread might see a stale null value even after the session URL has been initialized). To ensure thread safety and consistent visibility, read uploadSessionUrl inside the synchronized (lock) block.

  private void onTimeout() {
    String sessionUrl;
    synchronized (lock) {
      if (done) {
        return;
      }
      sessionUrl = uploadSessionUrl;
    }
    String message =
        sessionUrl != null
            ? "Resumable upload timed out for session: " + sessionUrl
            : "Resumable upload timed out before session initiation completed";
    finish(null, new DeadlineExceededException(message, null, TIMEOUT_STATUS_CODE, false));
  }

ResumableUploadFuture<String> future =
callable.futureCall("resource-path", streamOf("hello"), timeoutSettings);

ExecutionException exception = assertThrows(ExecutionException.class, future::get);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Calling future.get() without a timeout in a timeout-related unit test can cause the test suite to hang indefinitely if the timeout mechanism fails to fire. It is a best practice to always specify a timeout (e.g., 5 seconds) when waiting on futures in tests to prevent blocking the CI/CD pipeline.

Suggested change
ExecutionException exception = assertThrows(ExecutionException.class, future::get);
ExecutionException exception = assertThrows(ExecutionException.class, () -> future.get(5, TimeUnit.SECONDS));

@whowes
whowes force-pushed the whowes/resumable-upload-recovery branch from f7dacda to ea2e309 Compare September 17, 2026 22:08
@whowes
whowes force-pushed the whowes/resumable-upload-watchdog branch from 3011b46 to 3e2fdd4 Compare September 17, 2026 22:08
@whowes
whowes added this pull request to stack #14429 September 17, 2026 22:16
@whowes
whowes force-pushed the whowes/resumable-upload-recovery branch from ea2e309 to e197ee8 Compare September 17, 2026 22:22
@whowes
whowes force-pushed the whowes/resumable-upload-watchdog branch from 3e2fdd4 to 4dfdd22 Compare September 17, 2026 22:22
@whowes
whowes force-pushed the whowes/resumable-upload-recovery branch from e197ee8 to 12cd1cb Compare September 18, 2026 02:23
@whowes
whowes force-pushed the whowes/resumable-upload-watchdog branch from 4dfdd22 to 3794533 Compare September 18, 2026 02:23
@whowes
whowes force-pushed the whowes/resumable-upload-recovery branch from 12cd1cb to dd1aaac Compare September 18, 2026 03:01
@whowes
whowes force-pushed the whowes/resumable-upload-watchdog branch 2 times, most recently from 497d777 to a5ffaf3 Compare September 18, 2026 03:21
@whowes
whowes force-pushed the whowes/resumable-upload-recovery branch 2 times, most recently from 7fb2335 to 4795631 Compare September 18, 2026 15:05
@whowes
whowes force-pushed the whowes/resumable-upload-watchdog branch from a5ffaf3 to fc40846 Compare September 18, 2026 15:05
@whowes
whowes force-pushed the whowes/resumable-upload-recovery branch from 4795631 to 1fe75dc Compare September 19, 2026 01:36
@whowes
whowes force-pushed the whowes/resumable-upload-watchdog branch from fc40846 to 6960f29 Compare September 19, 2026 01:36
@whowes
whowes force-pushed the whowes/resumable-upload-recovery branch from 1fe75dc to 8456db3 Compare September 19, 2026 21:12
@whowes
whowes force-pushed the whowes/resumable-upload-watchdog branch from 6960f29 to 7d98c7b Compare September 19, 2026 21:12
@whowes
whowes force-pushed the whowes/resumable-upload-recovery branch from 8456db3 to 04da457 Compare September 19, 2026 22:57
@whowes
whowes force-pushed the whowes/resumable-upload-watchdog branch from 7d98c7b to a355e4a Compare September 19, 2026 22:57
@whowes
whowes force-pushed the whowes/resumable-upload-recovery branch from 04da457 to f60b76a Compare September 19, 2026 23:17
@whowes
whowes force-pushed the whowes/resumable-upload-watchdog branch from a355e4a to c59321f Compare September 19, 2026 23:17
@whowes
whowes force-pushed the whowes/resumable-upload-recovery branch from f60b76a to d5c48d6 Compare September 20, 2026 00:19
@whowes
whowes force-pushed the whowes/resumable-upload-watchdog branch from c59321f to bf605a2 Compare September 20, 2026 00:19
@whowes
whowes force-pushed the whowes/resumable-upload-recovery branch from d5c48d6 to 599ab38 Compare September 20, 2026 05:24
@whowes
whowes force-pushed the whowes/resumable-upload-watchdog branch from bf605a2 to aacf3b6 Compare September 20, 2026 05:24
@whowes
whowes removed this pull request from stack #14429 September 20, 2026 07:20
@whowes
whowes force-pushed the whowes/resumable-upload-recovery branch from 599ab38 to 292921f Compare September 20, 2026 07:45
@whowes
whowes force-pushed the whowes/resumable-upload-watchdog branch from aacf3b6 to c36f0cd Compare September 20, 2026 07:45
@whowes
whowes force-pushed the whowes/resumable-upload-recovery branch from 292921f to 85c6f69 Compare September 21, 2026 18:33
@whowes
whowes force-pushed the whowes/resumable-upload-watchdog branch from c36f0cd to 17e0368 Compare September 21, 2026 18:33
@whowes
whowes force-pushed the whowes/resumable-upload-recovery branch from 85c6f69 to 467ec8c Compare September 21, 2026 20:26
@whowes
whowes force-pushed the whowes/resumable-upload-watchdog branch from 17e0368 to 9f3a73f Compare September 21, 2026 20:26
@whowes
whowes force-pushed the whowes/resumable-upload-recovery branch 2 times, most recently from 59ea76b to 0d9aef5 Compare September 21, 2026 22:34
@whowes
whowes force-pushed the whowes/resumable-upload-watchdog branch from 9f3a73f to efea040 Compare September 21, 2026 22:34
@whowes
whowes force-pushed the whowes/resumable-upload-recovery branch from 0d9aef5 to cf10342 Compare September 21, 2026 22:59
@whowes
whowes force-pushed the whowes/resumable-upload-watchdog branch from efea040 to 0e50e79 Compare September 21, 2026 22:59
@whowes
whowes force-pushed the whowes/resumable-upload-recovery branch from cf10342 to 26d12d4 Compare September 21, 2026 23:35
@whowes
whowes force-pushed the whowes/resumable-upload-watchdog branch from 0e50e79 to 2eb5a87 Compare September 21, 2026 23:36
@whowes
whowes force-pushed the whowes/resumable-upload-recovery branch 2 times, most recently from 063af1d to 1e2e399 Compare September 22, 2026 06:46
@whowes
whowes force-pushed the whowes/resumable-upload-watchdog branch from 2eb5a87 to c012f3f Compare September 22, 2026 06:46
@whowes
whowes force-pushed the whowes/resumable-upload-recovery branch from 1e2e399 to 7c2f8ff Compare September 22, 2026 07:03
@whowes
whowes force-pushed the whowes/resumable-upload-watchdog branch from c012f3f to de73081 Compare September 22, 2026 07:03
@whowes
whowes force-pushed the whowes/resumable-upload-recovery branch from 7c2f8ff to 56bfa8e Compare September 22, 2026 07:57
@whowes
whowes force-pushed the whowes/resumable-upload-watchdog branch from de73081 to f128749 Compare September 22, 2026 07:57
Enforces ResumableUploadCallSettings.getGlobalTimeout() in ResumableUploadChunkCoordinator across the upload lifecycle. Cancels in-flight RPCs and completes the future with DeadlineExceededException when the deadline is exceeded.
@whowes
whowes force-pushed the whowes/resumable-upload-recovery branch from 56bfa8e to 566ca01 Compare September 22, 2026 08:52
@whowes
whowes force-pushed the whowes/resumable-upload-watchdog branch from f128749 to 439bda6 Compare September 22, 2026 08:52
@sonarqubecloud

Copy link
Copy Markdown

@sonarqubecloud

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed for 'gapic-generator-java-root'

Failed conditions
0.0% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube Cloud

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant