Skip to content

Commit 51bb14d

Browse files
authored
feat(storage): service worker streaming for zip download (#7044)
* feat(storage): service worker streaming zip download Replace the in-memory blob-based zip download with a service worker streaming architecture for StorageBrowser multi-file downloads. Falls back to blob collection when the SW is unavailable. Key changes: - Add download-sw.ts service worker with skipWaiting + clients.claim - Add zipdownload handler with WeakMap-scoped batch state (no singleton) - Stream files sequentially into zip via zip.js (concurrency=1) - MessageChannel handshake for SW stream transfer + keepalive pings - Blob fallback with inline cleanup (no fire-and-forget race) - composedDownloadHandler routes single-file vs multi-file downloads - useServiceWorkerRegistration hook for automatic SW registration - copy-serviceworker bin script for npm consumers - Documentation for SW setup across frameworks - Comprehensive test suite with deterministic async flushing * fix(storage): verify message origin in download SW, raise size limit - Add same-origin verification to the service worker message handler, addressing CodeQL finding 'Missing origin verification in postMessage handler'. Cross-origin messages are now rejected before any stream is stored or acknowledged. - Add a regression test asserting foreign-origin messages are ignored; update existing message mocks to carry the same-origin value. - Raise createStorageBrowser size-limit 130 kB -> 131 kB to accommodate the SW streaming code (was over by 21 B). - Add changeset for the feature. * docs(storage): note download-only progress in changeset Address review feedback: document that per-file progress is download-only by design (reflects S3 read bytes, not zip finalization), since level:0 storage makes the finalization phase near-instant. * fix(storage): address review feedback on SW streaming download - download-sw.ts: decode the request pathname before the pendingStreams lookup so folder names with spaces / URL-unsafe characters resolve (blocker: browser percent-encodes the id in the <a> navigation while the stream is keyed by the unencoded id). - download-sw.ts: use RFC 5987 'filename*=UTF-8''...' Content-Disposition so quotes/backslashes/UTF-8 in S3 keys are handled without escaping issues. - zipdownload.ts: guard the getRegistration().then() callback with a cancelled check so a keepalive interval is never created after the batch was reset (fixes an interval leak when cancel races swReady). - zipdownload.ts: drop fragile substring matching on zip.js internal error messages; flag cancellation from batchAbort.signal.aborted or the standard AbortError name only. - useServiceWorkerRegistration.ts: warn in development when SW registration fails (surfaces the common 'forgot copy-serviceworker' setup mistake). - copy-serviceworker.js: drop redundant existsSync before recursive mkdirSync (also closes a TOCTOU window). - tests: SW round-trip test for unencoded-store/encoded-request + RFC 5987 assertions; genuine early-exit cancelled-path test; space-in-folder test. - Revert an unintended workflow change (reusable-build-system-test-react-native.yml) that a prior rebase resolution pulled in; file now matches main. * fix(storage): stop cancel resurrection + leaked timestamp in SW filename - zipdownload.ts: cancelBatch() no longer calls reset() synchronously. The cancelled flag and the batchMap entry share one key, so deleting the entry let the next queued file (concurrency: 1) miss the cancelled early-exit and build a fresh batch — resurrecting the download (files 2..N still zipped). The entry is now kept (tombstoned, keepalive stopped) until batchDone === batchTotal, so remaining files drain through the early-exit. reset() remains the sole map-deleter, for both the cancel and completion paths. - download-sw.ts / zipdownload.ts: the SW filename is now sent explicitly (`${folder}.zip`) in the postMessage and stored alongside the stream. The Content-Disposition header uses it instead of deriving from downloadId, which embeds Date.now() for map-key uniqueness. Previously the same-origin Content-Disposition (which wins over the anchor download attr) saved e.g. photos-1720080000000.zip on the SW path while the blob fallback saved photos.zip — now both agree and no timestamp leaks. - useServiceWorkerRegistration.ts: drop process.env.NODE_ENV guard (TS2591 'Cannot find name process' in the browser rollup build — no node types) and warn unconditionally; a SW registration failure is a real degradation. - tests: SW explicit-filename precedence; UI-cancel no-resurrection guard (exactly one ZipWriter constructed across a cancelled 2-file batch).
1 parent 8a06872 commit 51bb14d

18 files changed

Lines changed: 1286 additions & 229 deletions

File tree

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
---
2+
'@aws-amplify/ui-react-storage': minor
3+
---
4+
5+
feat(storage): service worker streaming zip download
6+
7+
Replace the in-memory blob-based zip download with a service worker streaming
8+
architecture for `StorageBrowser` multi-file downloads. Files are streamed
9+
sequentially into a zip archive delivered via a service worker, removing the
10+
memory ceiling of the previous blob approach. Falls back to blob collection
11+
when the service worker is unavailable (unsupported browser, missing file, or
12+
insecure context).
13+
14+
Note: per-file progress is now reported download-only by design — it reflects
15+
bytes read from S3, not zip finalization. Because entries are stored without
16+
compression (`level: 0`), the write/finalization phase is near-instant, so
17+
progress transitions directly from downloading to complete without a separate
18+
"zipping"/finishing phase.

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,3 +43,4 @@ coverage/
4343
# Vite
4444
vite.config.js.timestamp-*
4545
vite.config.ts.timestamp-*
46+
*.tgz

docs/src/pages/[platform]/connected-components/storage/storage-browser/react.mdx

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -283,6 +283,79 @@ interface Config {
283283
</ExampleCode>
284284
</Example>
285285

286+
### Download Service Worker
287+
288+
The `StorageBrowser` component uses a [Service Worker](https://developer.mozilla.org/en-US/docs/Web/API/Service_Worker_API) to enable streaming zip downloads for multi-file selections. This allows downloading large batches of files without memory limitations, as the zip is streamed directly to disk rather than being assembled in memory.
289+
290+
#### Setup
291+
292+
The service worker file must be served from your application's public directory. The `@aws-amplify/ui-react-storage` package includes a CLI tool to copy it for you:
293+
294+
<Example>
295+
<ExampleCode>
296+
```bash
297+
npx @aws-amplify/ui-react-storage copy-serviceworker public
298+
```
299+
</ExampleCode>
300+
</Example>
301+
302+
This copies `download-sw.js` into `<target>/amplify-storage-download/download-sw.js`, making it available under the service worker's scope path.
303+
304+
#### Framework setup
305+
306+
**Next.js / Vite / Create React App:**
307+
308+
Run the copy command once after installing the package. The file will be served automatically from the `public/` folder.
309+
310+
**Amplify Hosting (amplify.yml):**
311+
312+
Add the copy step to your build spec so the service worker is available on every deploy:
313+
314+
<Example>
315+
<ExampleCode>
316+
```yaml
317+
frontend:
318+
phases:
319+
preBuild:
320+
commands:
321+
- npx @aws-amplify/ui-react-storage copy-serviceworker public
322+
build:
323+
commands:
324+
- npm run build
325+
```
326+
</ExampleCode>
327+
</Example>
328+
329+
**npm scripts (recommended):**
330+
331+
Add it to your `postinstall` or `prebuild` script so it stays up to date automatically:
332+
333+
<Example>
334+
<ExampleCode>
335+
```json
336+
{
337+
"scripts": {
338+
"prebuild": "npx @aws-amplify/ui-react-storage copy-serviceworker public"
339+
}
340+
}
341+
```
342+
</ExampleCode>
343+
</Example>
344+
345+
#### How it works
346+
347+
When a user downloads multiple files, the `StorageBrowser` component:
348+
349+
1. Registers the service worker at `/amplify-storage-download/download-sw.js` with scope `/amplify-storage-download/`
350+
2. Streams each file from S3 into a zip archive
351+
3. Delivers the zip via the service worker, enabling streaming to disk without size limits
352+
353+
If the service worker is not available (e.g., unsupported browser, missing file, or insecure context), the component automatically falls back to an in-memory blob-based download.
354+
355+
<Message colorTheme="info" variation="filled" heading="Service Worker scope">
356+
The service worker is registered with scope `/amplify-storage-download/`. It only intercepts requests within this scope and does not affect other parts of your application.
357+
</Message>
358+
286359
## Customization
287360

288361
### Pagination
Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
#!/usr/bin/env node
2+
'use strict';
3+
4+
const fs = require('fs');
5+
const path = require('path');
6+
7+
const SW_FILENAME = 'download-sw.js';
8+
const SW_SOURCE = path.resolve(__dirname, '..', 'dist', SW_FILENAME);
9+
10+
const targetDir = process.argv[2];
11+
12+
if (!targetDir) {
13+
console.error(
14+
'Usage: npx @aws-amplify/ui-react-storage copy-serviceworker <target-dir>\n' +
15+
'Example: npx @aws-amplify/ui-react-storage copy-serviceworker public'
16+
);
17+
process.exit(1);
18+
}
19+
20+
if (!fs.existsSync(SW_SOURCE)) {
21+
console.error(
22+
`Error: Service worker source not found at ${SW_SOURCE}\n` +
23+
'Make sure the package is properly installed.'
24+
);
25+
process.exit(1);
26+
}
27+
28+
const resolvedTarget = path.resolve(targetDir, 'amplify-storage-download');
29+
30+
// `recursive: true` is a no-op when the directory already exists, so no
31+
// existence check is needed (and skipping it avoids a TOCTOU window).
32+
fs.mkdirSync(resolvedTarget, { recursive: true });
33+
34+
const dest = path.join(resolvedTarget, SW_FILENAME);
35+
fs.copyFileSync(SW_SOURCE, dest);
36+
console.log(`✅ Copied ${SW_FILENAME}${dest}`);

packages/react-storage/eslint.config.js

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,8 @@ module.exports = defineConfig([
3232
'**/coverage',
3333
'**/dist',
3434
'**/node_modules',
35+
'**/service-worker',
36+
'**/bin',
3537
]),
3638
{
3739
extends: compat.extends('@aws-amplify/amplify-ui/jest'),

packages/react-storage/package.json

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,9 @@
2020
"./styles.css": "./dist/styles.css"
2121
},
2222
"types": "dist/types/index.d.ts",
23+
"bin": {
24+
"copy-serviceworker": "./bin/copy-serviceworker.js"
25+
},
2326
"license": "Apache-2.0",
2427
"repository": {
2528
"type": "git",
@@ -28,6 +31,7 @@
2831
},
2932
"files": [
3033
"dist",
34+
"bin",
3135
"LICENSE",
3236
"browser"
3337
],
@@ -43,7 +47,7 @@
4347
"size": "yarn size-limit",
4448
"test": "jest",
4549
"test:watch": "yarn test --watch",
46-
"typecheck": "tsc --noEmit"
50+
"typecheck": "tsc --noEmit && tsc --project tsconfig.sw.json --noEmit"
4751
},
4852
"dependencies": {
4953
"@aws-amplify/ui": "6.15.4",
@@ -69,7 +73,7 @@
6973
"name": "createStorageBrowser",
7074
"path": "dist/esm/browser.mjs",
7175
"import": "{ createStorageBrowser }",
72-
"limit": "130 kB",
76+
"limit": "131 kB",
7377
"ignore": [
7478
"@aws-amplify/storage"
7579
]

packages/react-storage/rollup.config.mjs

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -62,6 +62,21 @@ const config = defineConfig([
6262
},
6363
plugins: [styles({ mode: ['extract'] })],
6464
},
65+
// Service Worker — standalone IIFE, no externals, self-contained
66+
{
67+
input: 'src/components/StorageBrowser/service-worker/download-sw.ts',
68+
output: {
69+
file: 'dist/download-sw.js',
70+
format: 'iife',
71+
},
72+
plugins: [
73+
typescript({
74+
declaration: false,
75+
sourceMap: false,
76+
tsconfig: 'tsconfig.sw.json',
77+
}),
78+
],
79+
},
6580
]);
6681

6782
export default config;

0 commit comments

Comments
 (0)