Skip to content

fix(minutes): make media downloads resilient - #2245

Open
liangshuo-1 wants to merge 1 commit into
mainfrom
fix/minutes-resilient-download
Open

fix(minutes): make media downloads resilient#2245
liangshuo-1 wants to merge 1 commit into
mainfrom
fix/minutes-resilient-download

Conversation

@liangshuo-1

@liangshuo-1 liangshuo-1 commented Aug 8, 2026

Copy link
Copy Markdown
Collaborator

Summary

Replaces the single unvalidated GET behind minutes +download with the shared internal/download sequential range reader introduced in #2223, so large recording media survives gateway timeouts and mid-transfer interruptions. Command behavior is unchanged: same flags, filename resolution, batch layout, overwrite protection, and JSON output.

This also tightens one thing in the shared framework itself. Minutes is its second consumer, and wiring it up surfaced a header that misdescribed multipart streams; the fix is included here rather than deferred because Minutes is the first caller that reads those headers to derive a filename.

Changes

Minutes

  • Route media downloads through download.URLdownload.ImmutableSourcedownload.Open instead of a bare client.Do with ad-hoc status handling.
  • Declare Minutes media as an immutable representation. A minute token addresses finalized recording bytes, so parts can be combined without a validator. The media endpoint returns no ETag, so declaring it mutable would silently fall back to a single full response and lose ranged recovery.
  • Use a 128 MiB part size for this domain. Recording media is substantially larger than IM or app files, and smaller parts would multiply requests without benefit. This is a Minutes-level choice and does not change the framework default.
  • Drop the fixed per-request deadline in favor of the framework's 60-second progress-based idle timeout, so slow-but-advancing transfers are no longer cut off while dead connections still fail fast.
  • Remove the local HTTP status and error-body mapping now owned by the download layer.
  • Consume the returned stream directly instead of repacking it into a synthetic http.Response; resolveFilenameFromResponse now takes the http.Header it actually reads.

Shared download framework

  • A multipart stream now reports its own framing: Content-Length is the whole object and the first part's Content-Range is dropped. Previously Stream.Header was a verbatim clone of the opening 206, so a caller reading Content-Length off the header saw the first part's size while Stream.ContentLength held the total. No current caller reads that header field, so this fixes a latent inconsistency rather than a live defect, and it makes the header agree with what the single-response path already produced.

Not included: cross-process checkpointing or resumable downloads. Recovery happens within a single command invocation only.

Test Plan

  • Unit tests pass (make unit-test, race enabled, across cmd, internal, shortcuts, and extension)
  • go vet ./..., go build ./..., and diff-scoped golangci-lint are clean
  • Manual local verification confirms the lark-cli minutes +download flow works as expected

New coverage:

  • TestOpenReportsAssembledStreamHeaders pins the assembled-stream header contract: reverting the change fails it with the first part's length.

Verified against live recording media using a temporary, uncommitted transport observer to capture the exact request sequence:

Media size Part size Requests Result
55 MiB 128 MiB 1 single 206, media decodes cleanly
203 MiB 128 MiB 2 contiguous ranges, no gaps or overlaps
203 MiB 64 MiB 4 contiguous ranges, no gaps or overlaps
203 MiB 8 MiB 26 contiguous ranges, no gaps or overlaps
  • Every response was 206 with an exact Content-Range, and no fallback full re-fetch occurred.
  • All four runs produced byte-identical output, matching an independent range-by-range reconstruction of the same object.
  • Downloading the same token twice produced identical checksums, and the resulting media decodes without errors.
  • With --output omitted, the filename is derived from the assembled stream's headers; verified on both a single-part and a two-part download, confirming Content-Type and Content-Disposition survive the header fix.
  • Peak RSS stayed near 48 MiB while transferring 203 MiB under a 128 MiB part size, confirming parts are streamed rather than buffered.
  • Writing to an existing path without --overwrite returns a typed validation error and leaves the original file untouched.
  • Interrupting a transfer exits without publishing a final file. The partial temporary file is still left behind, which matches current main and is unchanged by this PR.

Related Issues

  • None

Summary by CodeRabbit

  • Bug Fixes
    • Improved media downloads for large files through more reliable streaming and segmented transfers.
    • Added validation to reject truncated or unexpectedly encoded download responses, preventing incomplete output.
    • Improved handling and classification of temporary server errors during downloads.
    • Corrected download metadata for assembled streams so file sizes and content types are reported accurately.

@coderabbitai

coderabbitai Bot commented Aug 8, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

The Minutes media download path now uses the internal download package for immutable-source streaming with 128 MiB parts. Multipart streams report assembled headers. Tests cover ranged responses, invalid bodies, partial-file cleanup, and retryable HTTP 503 errors.

Changes

Minutes media download

Layer / File(s) Summary
Assembled stream headers
internal/download/download.go, internal/download/download_test.go
Multipart streams now report the full Content-Length, omit Content-Range, and preserve Content-Type.
Streaming download integration
shortcuts/minutes/minutes_download.go
downloadMediaFile now uses immutable-source download.Open streaming with 128 MiB parts. Filename resolution reads headers from the download stream, and file saving uses stream metadata.
Download response validation and retry tests
shortcuts/minutes/minutes_download_test.go
Tests require ranged identity-encoded responses, reject truncated or encoded bodies without partial files, and classify HTTP 503 responses as retryable network-server errors.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant downloadMediaFile
  participant download.Open
  participant FilenameResolver
  participant FileSaving
  downloadMediaFile->>download.Open: Open immutable-source stream with 128 MiB parts
  download.Open-->>FilenameResolver: Provide response headers
  download.Open-->>FileSaving: Provide content type, length, and body
Loading

Possibly related PRs

  • larksuite/cli#2223: Introduces the internal download streaming abstraction used by this change.
  • larksuite/cli#2241: Migrates another shortcut download flow to the shared streaming and retry pipeline.
  • larksuite/cli#2176: Adds related typed download errors used by ranged-download validation.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main change: improving resilience for Minutes media downloads.
Description check ✅ Passed The description includes the required summary, changes, test plan, and related issues sections with detailed verification results.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/minutes-resilient-download

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added the size/M Single-domain feat or fix with limited business impact label Aug 8, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@shortcuts/minutes/minutes_download_test.go`:
- Around line 283-332: Update TestDownloadRejectsInvalidResponseBodies in
shortcuts/minutes/minutes_download_test.go:283-332 to inspect errors through
errs.ProblemOf, asserting the network category and expected subtype while
preserving and validating the truncated-response cause; do not use Param on
errs.Problem. Also update the error assertion at
shortcuts/minutes/minutes_download_test.go:889-893 to use errs.ProblemOf, assert
the network category and SubtypeNetworkServer, and retain the existing
retryability assertion.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 455fafad-ffa2-41ad-b2e2-acfd789aaaad

📥 Commits

Reviewing files that changed from the base of the PR and between 7be2476 and 055ae22.

📒 Files selected for processing (2)
  • shortcuts/minutes/minutes_download.go
  • shortcuts/minutes/minutes_download_test.go

Comment on lines +283 to +332
func TestDownloadRejectsInvalidResponseBodies(t *testing.T) {
tests := []struct {
name string
header http.Header
body string
subtype errs.Subtype
}{
{
name: "truncated",
header: http.Header{"Content-Length": []string{"10"}},
body: "short",
subtype: errs.SubtypeNetworkProtocol,
},
{
name: "encoded",
header: http.Header{"Content-Encoding": []string{"gzip"}},
body: "compressed",
subtype: errs.SubtypeNetworkProtocol,
},
}

for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
chdir(t, t.TempDir())
f, _, _, reg := cmdutil.TestFactory(t, defaultConfig())
reg.Register(mediaStub("tok001", "https://example.com/media"))
f.HttpClient = func() (*http.Client, error) {
return &http.Client{Transport: minutesRoundTripFunc(func(req *http.Request) (*http.Response, error) {
return &http.Response{
StatusCode: http.StatusOK,
Header: tt.header,
Body: io.NopCloser(strings.NewReader(tt.body)),
ContentLength: 10,
Request: req,
}, nil
})}, nil
}

err := mountAndRun(t, MinutesDownload, []string{
"+download", "--minute-tokens", "tok001", "--output", "out.media", "--as", "bot",
}, f, nil)
var networkErr *errs.NetworkError
if !errors.As(err, &networkErr) || networkErr.Subtype != tt.subtype {
t.Fatalf("error = %T %v, want network/%s", err, err, tt.subtype)
}
if _, statErr := os.Stat("out.media"); !errors.Is(statErr, os.ErrNotExist) {
t.Fatalf("partial output should not exist: %v", statErr)
}
})
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Assert the complete typed error contract.

Both tests bypass errs.ProblemOf, so neither verifies the error category.

  • shortcuts/minutes/minutes_download_test.go#L283-L332: Assert network category and subtype with errs.ProblemOf. Assert the truncated-response cause is preserved.
  • shortcuts/minutes/minutes_download_test.go#L889-L893: Assert network category and SubtypeNetworkServer with errs.ProblemOf, while retaining the retryability assertion.

As per coding guidelines, error-path tests must assert typed metadata through errs.ProblemOf and preserve applicable causes. Based on learnings, Param is not a field on errs.Problem for these network errors.

📍 Affects 1 file
  • shortcuts/minutes/minutes_download_test.go#L283-L332 (this comment)
  • shortcuts/minutes/minutes_download_test.go#L889-L893
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@shortcuts/minutes/minutes_download_test.go` around lines 283 - 332, Update
TestDownloadRejectsInvalidResponseBodies in
shortcuts/minutes/minutes_download_test.go:283-332 to inspect errors through
errs.ProblemOf, asserting the network category and expected subtype while
preserving and validating the truncated-response cause; do not use Param on
errs.Problem. Also update the error assertion at
shortcuts/minutes/minutes_download_test.go:889-893 to use errs.ProblemOf, assert
the network category and SubtypeNetworkServer, and retain the existing
retryability assertion.

Sources: Coding guidelines, Learnings

@github-actions

github-actions Bot commented Aug 8, 2026

Copy link
Copy Markdown

🚀 PR Preview Install Guide

🧰 CLI update

npm i -g https://pkg.pr.new/larksuite/cli/@larksuite/cli@06185f7feb7071f0d91201e7d71fd83f0dceee7b

🧩 Skill update

npx skills add larksuite/cli#fix/minutes-resilient-download -y -g

@codecov

codecov Bot commented Aug 8, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 76.37%. Comparing base (7be2476) to head (06185f7).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2245   +/-   ##
=======================================
  Coverage   76.36%   76.37%           
=======================================
  Files        1011     1011           
  Lines      111269   111270    +1     
=======================================
+ Hits        84970    84978    +8     
+ Misses      19815    19811    -4     
+ Partials     6484     6481    -3     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Route minute media through the shared sequential range reader, and make the
framework describe what it hands back: a multipart stream now reports the
whole object's length and drops the first part's Content-Range, so
Stream.Header matches the bytes the caller actually receives.
@liangshuo-1
liangshuo-1 force-pushed the fix/minutes-resilient-download branch from 055ae22 to 06185f7 Compare August 10, 2026 03:32

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@internal/download/download_test.go`:
- Around line 388-418: Add coverage in TestOpenReportsAssembledStreamHeaders for
Content-Disposition: set a representative filename header on the ranged response
created by testPartial, then assert the assembled stream preserves the same
value alongside the existing Content-Type assertion.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: c474cff9-4fc0-44fc-b891-155289e36586

📥 Commits

Reviewing files that changed from the base of the PR and between 055ae22 and 06185f7.

📒 Files selected for processing (4)
  • internal/download/download.go
  • internal/download/download_test.go
  • shortcuts/minutes/minutes_download.go
  • shortcuts/minutes/minutes_download_test.go
🚧 Files skipped from review as they are similar to previous changes (2)
  • shortcuts/minutes/minutes_download.go
  • shortcuts/minutes/minutes_download_test.go

Comment on lines +388 to +418
// Callers read Content-Type and Content-Disposition off the assembled stream,
// so those must survive, while the first part's framing must not leak out as a
// length shorter than the bytes actually delivered.
func TestOpenReportsAssembledStreamHeaders(t *testing.T) {
full := []byte("abcdefgh")
stream, err := openTest(context.Background(), func(_ context.Context, req Request) (*http.Response, error) {
start := req.Range.Start
end := min(req.Range.End, int64(len(full))-1)
resp := testPartial(full[start:end+1], start, end, int64(len(full)), "")
resp.Header.Set("Content-Length", fmt.Sprint(end-start+1))
resp.Header.Set("Content-Type", "video/mp4")
return resp, nil
}, testOptions())
if err != nil {
t.Fatalf("Open() error = %v", err)
}
defer stream.Body.Close()

if stream.ContentLength != int64(len(full)) {
t.Fatalf("ContentLength = %d, want %d", stream.ContentLength, len(full))
}
if got, want := stream.Header.Get("Content-Length"), fmt.Sprint(len(full)); got != want {
t.Errorf("Content-Length header = %q, want %q (the whole stream, not the first part)", got, want)
}
if got := stream.Header.Get("Content-Range"); got != "" {
t.Errorf("Content-Range header = %q, want it dropped from the assembled stream", got)
}
if got := stream.Header.Get("Content-Type"); got != "video/mp4" {
t.Errorf("Content-Type header = %q, want it preserved", got)
}
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win

Add a Content-Disposition assertion.

The test comment states that callers depend on Content-Type and Content-Disposition. The fixture and assertions cover only Content-Type. Add a Content-Disposition header to the ranged response and assert that the assembled stream retains it. Otherwise, a regression that drops the filename header can pass this test.

As per coding guidelines, contract tests must assert each changed field or behavior directly.

Proposed test extension
 		resp.Header.Set("Content-Type", "video/mp4")
+		resp.Header.Set("Content-Disposition", `attachment; filename="minute.mp4"`)
 		return resp, nil
@@
 	if got := stream.Header.Get("Content-Type"); got != "video/mp4" {
 		t.Errorf("Content-Type header = %q, want it preserved", got)
 	}
+	if got := stream.Header.Get("Content-Disposition"); got != `attachment; filename="minute.mp4"` {
+		t.Errorf("Content-Disposition header = %q, want it preserved", got)
+	}
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
// Callers read Content-Type and Content-Disposition off the assembled stream,
// so those must survive, while the first part's framing must not leak out as a
// length shorter than the bytes actually delivered.
func TestOpenReportsAssembledStreamHeaders(t *testing.T) {
full := []byte("abcdefgh")
stream, err := openTest(context.Background(), func(_ context.Context, req Request) (*http.Response, error) {
start := req.Range.Start
end := min(req.Range.End, int64(len(full))-1)
resp := testPartial(full[start:end+1], start, end, int64(len(full)), "")
resp.Header.Set("Content-Length", fmt.Sprint(end-start+1))
resp.Header.Set("Content-Type", "video/mp4")
return resp, nil
}, testOptions())
if err != nil {
t.Fatalf("Open() error = %v", err)
}
defer stream.Body.Close()
if stream.ContentLength != int64(len(full)) {
t.Fatalf("ContentLength = %d, want %d", stream.ContentLength, len(full))
}
if got, want := stream.Header.Get("Content-Length"), fmt.Sprint(len(full)); got != want {
t.Errorf("Content-Length header = %q, want %q (the whole stream, not the first part)", got, want)
}
if got := stream.Header.Get("Content-Range"); got != "" {
t.Errorf("Content-Range header = %q, want it dropped from the assembled stream", got)
}
if got := stream.Header.Get("Content-Type"); got != "video/mp4" {
t.Errorf("Content-Type header = %q, want it preserved", got)
}
}
// Callers read Content-Type and Content-Disposition off the assembled stream,
// so those must survive, while the first part's framing must not leak out as a
// length shorter than the bytes actually delivered.
func TestOpenReportsAssembledStreamHeaders(t *testing.T) {
full := []byte("abcdefgh")
stream, err := openTest(context.Background(), func(_ context.Context, req Request) (*http.Response, error) {
start := req.Range.Start
end := min(req.Range.End, int64(len(full))-1)
resp := testPartial(full[start:end+1], start, end, int64(len(full)), "")
resp.Header.Set("Content-Length", fmt.Sprint(end-start+1))
resp.Header.Set("Content-Type", "video/mp4")
resp.Header.Set("Content-Disposition", `attachment; filename="minute.mp4"`)
return resp, nil
}, testOptions())
if err != nil {
t.Fatalf("Open() error = %v", err)
}
defer stream.Body.Close()
if stream.ContentLength != int64(len(full)) {
t.Fatalf("ContentLength = %d, want %d", stream.ContentLength, len(full))
}
if got, want := stream.Header.Get("Content-Length"), fmt.Sprint(len(full)); got != want {
t.Errorf("Content-Length header = %q, want %q (the whole stream, not the first part)", got, want)
}
if got := stream.Header.Get("Content-Range"); got != "" {
t.Errorf("Content-Range header = %q, want it dropped from the assembled stream", got)
}
if got := stream.Header.Get("Content-Type"); got != "video/mp4" {
t.Errorf("Content-Type header = %q, want it preserved", got)
}
if got := stream.Header.Get("Content-Disposition"); got != `attachment; filename="minute.mp4"` {
t.Errorf("Content-Disposition header = %q, want it preserved", got)
}
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/download/download_test.go` around lines 388 - 418, Add coverage in
TestOpenReportsAssembledStreamHeaders for Content-Disposition: set a
representative filename header on the ranged response created by testPartial,
then assert the assembled stream preserves the same value alongside the existing
Content-Type assertion.

Source: Coding guidelines

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/M Single-domain feat or fix with limited business impact

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant