Replace S3 presigned urls in favor of Cloudfront - #29
Conversation
There was a problem hiding this comment.
Pull request overview
This PR introduces CloudFront signed URLs for download (GET) access to library objects (replacing S3 presigned URLs) behind a global configs toggle and a per-user allowlist, while keeping uploads (PUT) on S3 presigned URLs.
Changes:
- Add
CloudFrontServicefor issuing signed CloudFront URLs and wire it intoLibraryServicedownload URL generation with S3 fallback. - Add
ConfigService+ConfigDBto read a newuse_cloudfront_downloadsconfig flag (Redis read-through cached). - Add migration + deployment/env updates, plus unit tests for the new flagging/signing behavior.
Reviewed changes
Copilot reviewed 12 out of 13 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
package.json |
Adds @aws-sdk/cloudfront-signer dependency. |
yarn.lock |
Locks new CloudFront signer + smithy deps. |
src/services/LibraryService.ts |
Routes download URL resolution through CloudFront signer with S3 fallback, plus allowlist/global-flag logic. |
src/services/CloudFrontService.ts |
New service to generate CloudFront signed URLs with configurable TTL. |
src/services/ConfigService.ts |
New config flag reader with Redis read-through caching and safe fallbacks. |
src/services/db/ConfigDB.ts |
New DB access layer for configs table reads. |
src/database/migrations/20260607120000_add_use_cloudfront_downloads_config.ts |
Adds new enum value and seeds use_cloudfront_downloads config row. |
src/config/envs.ts |
Adds CloudFront-related env validation. |
docker/ecs/task-definition.json |
Adds CloudFront env + secrets to ECS task definition. |
development.env.template |
Adds CloudFront env placeholders for local development. |
src/__tests__/services/LibraryServiceDownloadUrl.test.ts |
Tests CloudFront vs S3 decision logic + fallback behavior. |
src/__tests__/services/ConfigService.test.ts |
Tests config caching, type mismatch handling, and fail-safe behavior. |
src/__tests__/services/CloudFrontService.test.ts |
Tests signing output shape, path encoding, and TTL behavior. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| .prop('CLOUDFRONT_URL', S.string().required()) | ||
| .prop('CLOUDFRONT_KEY_PAIR_ID', S.string().required()) | ||
| .prop('CLOUDFRONT_PRIVATE_KEY', S.string().required()) | ||
| .prop('CLOUDFRONT_ALLOWLIST', S.string()) | ||
| .prop('CLOUDFRONT_EXPIRY_SECONDS', S.string()) |
| const useCloudFront = | ||
| this.isCloudFrontAllowlisted(user) || | ||
| (await this._config.getBoolean(ConfigKey.UseCloudFrontDownloads)); | ||
| if (useCloudFront) { |
| { "name": "CLOUDFRONT_URL", "value": "https://library.bookplayer.app" }, | ||
| { "name": "CLOUDFRONT_ALLOWLIST", "value": "29" }, | ||
| { "name": "CLOUDFRONT_EXPIRY_SECONDS", "value": "120" } |
3246615 to
34c0942
Compare
| { "name": "SYNC_AUDIT_ENABLED", "value": "true" }, | ||
| { "name": "CLOUDFRONT_URL", "value": "https://library.bookplayer.app" }, | ||
| { "name": "CLOUDFRONT_ALLOWLIST", "value": "29" }, | ||
| { "name": "CLOUDFRONT_EXPIRY_SECONDS", "value": "120" } |
There was a problem hiding this comment.
🟡 WARN — CLOUDFRONT_EXPIRY_SECONDS=120 in the production task definition overrides CloudFrontService.DEFAULT_EXPIRY_SECONDS (86400) for every signed URL. That directly contradicts the rationale documented in CloudFrontService ("AVURLAsset re-fetches ranges with the same URL, so a shorter expiry would stall playback mid-listen"), and combined with CLOUDFRONT_ALLOWLIST=29 on line 47 it means the canary user gets download/artwork URLs that die after 2 minutes — 403s mid-playback and broken artwork. Note the library-listing path doesn't return expires_in at all, so the client can't even know to refresh.
Fix: drop this env var so the 24h default applies (or set it to the value you actually intend to ship, e.g. 86400). If 120s was only meant for a temporary experiment, land it in a separate, clearly-labelled change.
| ): Promise<{ url: string; expires_in: number }> { | ||
| const useCloudFront = | ||
| this.isCloudFrontAllowlisted(user) || | ||
| (await this._config.getBoolean(ConfigKey.UseCloudFrontDownloads)); |
There was a problem hiding this comment.
🟡 WARN — getBoolean is called once per generated URL, i.e. twice per library item inside the getLibrary loop (lines 214 and 221). For a 500-item library that's up to 1000 extra Redis round-trips per /v1/library/all, and when Redis is unavailable or the key is cold, RedisService.getObject returns null and every one of those calls becomes a configs SELECT — 1000 DB queries for a single listing. The allowlist short-circuit only helps allowlisted users.
Fix: resolve the flag once per request and thread it down, e.g. compute const useCloudFront = this.isCloudFrontAllowlisted(user) || await this._config.getBoolean(ConfigKey.UseCloudFrontDownloads) before the loop and change getDownloadUrl(key, user) to getDownloadUrl(key, useCloudFront). An in-process memo with a short TTL in ConfigService would work too.
| .prop('REVENUECAT_ENTITLEMENT_PLUS', S.string().required()) | ||
| .prop('REVENUECAT_ENTITLEMENT_LITE', S.string().required()) | ||
| .prop('PROXY_FILE_URL', S.string().required()) | ||
| .prop('CLOUDFRONT_URL', S.string().required()) |
There was a problem hiding this comment.
🟡 WARN — The three new CLOUDFRONT_* props are .required(), so envSchema throws at boot if any is missing. That makes the deploy order fragile: the ECS task will crash-loop unless CLOUDFRONT_KEY_PAIR_ID / CLOUDFRONT_PRIVATE_KEY already exist in Secrets Manager, and any other environment (CI, staging, a dev machine on an older .development.env) fails to start — even though the feature ships flag-off and CloudFrontService.getSignedUrl already degrades gracefully to S3 when the config is absent.
Fix: make these optional (S.string() without .required()) and rely on the existing runtime guard + S3 fallback, or gate the requirement on the flag being enabled.
| const cached = (await this._cache.getObject( | ||
| cacheKey, | ||
| )) as ConfigRow | null; | ||
| const row = cached || (await this._configDB.getConfig(key)); |
There was a problem hiding this comment.
🔵 INFO — A missing row is never negatively cached: when getConfig returns null (row absent, or the migration hasn't run in this environment yet) the code returns the fallback without a setObject, so every subsequent call hits Postgres again. With the per-item call pattern in LibraryService.getLibrary that's one query per library item per request.
Fix: cache a sentinel (e.g. { value: 'false', value_type: 'boolean' } or a short-TTL null marker) on a miss so the DB isn't re-queried on every call. Also consider moving the value_type check above the setObject so a malformed row isn't written into the cache.
| this._logger.log({ | ||
| origin: 'CloudFrontService.getSignedUrl', | ||
| message: error.message, | ||
| data: { key }, |
There was a problem hiding this comment.
🔵 INFO — Two small things on this log call:
- PII:
keystarts with the storage prefix, which for legacy users is their email address (your own test coversuser@icloud.com/library/...).sanitizeDataonly redactspassword/token/secret/authorization, so the email lands in the log verbatim. Log the last path segment or a hash of the key instead. - Level: this logs at the default
info, and prod runsLOG_LEVEL=warn, so a signing misconfiguration (bad PEM, wrong key-pair id) is invisible here. Pass'error'as the second argument, consistent with the fallback log inLibraryService.getDownloadUrl.
| key: `${storagePrefix}/${originalFile}`, | ||
| type: StorageAction.GET, | ||
| }); | ||
| const { url } = await this.getDownloadUrl( |
There was a problem hiding this comment.
🔵 INFO — This branch assigns url but never sets libObj.expires_in, so clients on the legacy listing path get no expiry hint. Previously that was mostly harmless (S3 presigns here live 7 days — S3Service.getPresignedUrl uses 3600 * 24 * 7); CloudFront URLs default to 24h (and 120s with the current prod task definition), so a client that caches a listing URL can start getting 403s with no way to know it should refresh. Consider setting libObj.expires_in from getDownloadUrl here as the other two call sites do.
🟡 Claude PR Review —
|
No description provided.