Multipart uploads
Upload a large body in parallel parts instead of one request - the robust path past the single-request size limit and for streams of unknown length.
Multipart splits a body into parts and uploads them in parallel, then stitches them back together server-side. It’s the robust path for objects beyond the single-request limit (5 GB on S3) and for ReadableStream bodies of unknown length: a single PUT must either buffer the whole body or know its length up front, while multipart streams part-by-part with bounded memory.
It’s a per-call option on upload — the only method that writes a body, and so the only one to which it applies. Pass multipart: true for sensible defaults, or an object to tune partSize and concurrency:
// Defaults: 5 MiB parts, 4 in flight.
await files.upload("backups/db.tar", stream, { multipart: true });
// Or tune it:
await files.upload("backups/db.tar", stream, {
multipart: { partSize: 16 * 1024 * 1024, concurrency: 8 },
});
In the array form, multipart is a per-item option — set it on each { key, body } that needs it.
partSize?number
Size of each uploaded part, in bytes. Defaults to 5 MiB (`5 * 1024 * 1024`) — also the S3-enforced minimum for every part except the last. Adapters that chunk natively round this to their own valid granularity (OneDrive to a 320-KiB multiple, GCS/Firebase to 256 KiB).
numberconcurrency?number
How many parts upload in parallel. Defaults to `4`. Higher values trade memory (up to `partSize * concurrency` buffered at once) for throughput.
numberWhat each adapter does
The flag maps onto whatever chunking primitive the provider exposes, so the mechanics differ but the contract — a large body uploaded reliably — does not:
- S3 and the S3-compatible adapters (incl. R2 over HTTP with the
"aws-sdk"client) run multipart through the optional@aws-sdk/lib-storagepackage, falling back to a singlePutObjectwhen the body fits in one part. Unknown-length streams use multipart automatically, even without the flag. - OneDrive uploads above 250 MB (and any
multipartrequest) go through a chunked upload session — large files that previously failed now just work. - GCS and Firebase Storage switch to a resumable upload;
partSizemaps to the chunk size. - Azure Blob already splits large bodies into parallel blocks;
multipartonly tunes the block size and concurrency. - Dropbox streams
ReadableStreambodies through its upload session chunk-by-chunk, so a large stream is never buffered whole;partSize(rounded to a 4 MiB multiple) tunes the chunk size. - Other adapters already stream natively or only accept a fully-buffered body, so they ignore the option. The exception is the
fetchS3 engine (s3Fetch(), and R2 / MinIO / RustFS withclient: "fetch"), which has no multipart path and throwsUnsupported.
Adapters that chunk natively round partSize to their own valid granularity — OneDrive to a 320-KiB multiple, GCS and Firebase to 256 KiB — and the S3 adapters raise it to S3’s 5 MiB minimum (which every part but the last must meet).
Sizing the parts
partSize and concurrency trade memory for throughput: up to partSize × concurrency bytes are buffered at once. Both must be positive integers. 0, a negative or fractional number, or NaN throws an Invalid FilesError before any request, on every adapter (including the ones that otherwise ignore multipart) and for resumable uploads. The defaults (5 MiB × 4) keep that footprint small; raise them for fat pipes and large objects, lower them on memory-constrained workers. S3 caps an upload at 10,000 parts, so when the body’s length is known the S3 adapters grow partSize until it fits; an unknown-length stream can’t be fitted, so very large streams need a partSize big enough on their own.
Progress and retries
When onProgress is set, the S3 family reports true byte-level progress across the whole multipart upload through the same @aws-sdk/lib-storage path. Multipart is still a single upload call as far as retries, timeouts, and cancellation are concerned — a failure retries the whole call, not an individual part (and only for a buffered body, since a ReadableStream is never retried), and an aborted signal fails it fast.
To pause and resume an upload — or resume one after a crash from a serializable token, retrying individual parts rather than the whole call — reach for a resumable upload instead.