Resume an S3 multipart upload after a process restart

A Node or Bun uploader that saves its S3 multipart session to disk, survives a crash or Ctrl-C, and finishes from the parts S3 already holds.

Pass an UploadControl to files.upload() on the s3() adapter and the SDK drives S3’s multipart API directly. Write control.toJSON() to disk as soon as the session exists. After a crash, rebuild the control with UploadControl.from(token) and call upload() again with the same key and the same file. The SDK asks S3 which parts it already has (ListParts) and uploads only the rest.

The catch is that S3 keeps parts, not proof of what they contain. Resuming with a different file of the same length completes without an error and stitches two files into one object. Checking that the file is unchanged is your job. Abandoned sessions also keep their parts, and S3 bills you for them, until something aborts them.

Before you start

• An S3 bucket (this guide calls it uploads), AWS_REGION set to its region, and credentials the AWS credential chain can find. The upload needs s3:PutObject, s3:ListMultipartUploadParts, s3:AbortMultipartUpload, and s3:GetObject (the SDK HEADs the object after completing). AWS lists which action covers which call in multipart upload permissions.
• Node 22.18 or later, or Bun. Node runs the .ts files directly because type stripping is on by default from 22.18, and the script opens the file with fs.openAsBlob, which is stable from 22.17.
• Written against files-sdk 3.0, @aws-sdk/client-s3 3.1148, Node 24.14, and Bun 1.4.

npm install files-sdk @aws-sdk/client-s3 @aws-sdk/s3-presigned-post @aws-sdk/s3-request-presigner

pnpm add files-sdk @aws-sdk/client-s3 @aws-sdk/s3-presigned-post @aws-sdk/s3-request-presigner

yarn add files-sdk @aws-sdk/client-s3 @aws-sdk/s3-presigned-post @aws-sdk/s3-request-presigner

bun add files-sdk @aws-sdk/client-s3 @aws-sdk/s3-presigned-post @aws-sdk/s3-request-presigner

nub add files-sdk @aws-sdk/client-s3 @aws-sdk/s3-presigned-post @aws-sdk/s3-request-presigner

aube add files-sdk @aws-sdk/client-s3 @aws-sdk/s3-presigned-post @aws-sdk/s3-request-presigner

files-sdk/s3 imports all three AWS packages. You don’t need @aws-sdk/lib-storage here: resumable uploads call CreateMultipartUpload, UploadPart, and CompleteMultipartUpload themselves rather than going through lib-storage, which never exposes the upload ID.

What gets saved, and what S3 keeps

The token is small. For S3 it holds five fields:

{
  "provider": "s3",
  "bucket": "uploads",
  "key": "backups/db.tar",
  "uploadId": "EXAMPLE-UPLOAD-ID",
  "partSize": 8388608
}

S3 holds everything else: each stored part’s number, size, and ETag. On resume, the SDK:

1. Checks that the token’s bucket and key match this upload, then adopts its upload ID and part size.
2. Calls ListParts and keeps each part whose number and size match how the current file slices at that part size.
3. Uploads the missing parts, retrying each one on its own under the call’s retries policy.
4. Calls CompleteMultipartUpload with every part’s ETag, then HeadObject for the final size and type.

The token doesn’t record part ETags, so a resume completes from what ListParts reports. That’s a trade-off worth knowing: AWS’s multipart overview recommends completing from your own record of part numbers and ETags rather than from a listing.

Write the uploader

import { openAsBlob, writeFileSync } from "node:fs";
import { readFile, rm, stat } from "node:fs/promises";

import {
  Files,
  FilesError,
  UploadControl,
  type ResumableUploadSession,
} from "files-sdk";
import { s3 } from "files-sdk/s3";

const [path, key] = process.argv.slice(2);
if (!path || !key) {
  console.error("Usage: bun upload.ts <file> <key>");
  process.exit(1);
}

const files = new Files({
  adapter: s3({
    bucket: "uploads",
    // Region and credentials come from AWS_REGION and the AWS credential chain.
    // S3_ENDPOINT points the script at MinIO or another S3-compatible server.
    ...(process.env.S3_ENDPOINT && {
      endpoint: process.env.S3_ENDPOINT,
      forcePathStyle: true,
    }),
  }),
});

interface Checkpoint {
  key: string;
  size: number;
  mtimeMs: number;
  session: ResumableUploadSession;
}

const checkpointPath = `${path}.upload.json`;
const { size, mtimeMs } = await stat(path);

const loadCheckpoint = async (): Promise<Checkpoint | undefined> => {
  let saved: Checkpoint;
  try {
    saved = JSON.parse(await readFile(checkpointPath, "utf8"));
  } catch {
    return undefined;
  }
  // Resume only the same key with the same bytes. The SDK can't tell whether
  // the file changed, so this check is yours to make.
  if (saved.key === key && saved.size === size && saved.mtimeMs === mtimeMs) {
    return saved;
  }
  console.warn(
    "Checkpoint belongs to another key or file version; ignoring it."
  );
  return undefined;
};

const saved = await loadCheckpoint();
const control = saved ? UploadControl.from(saved.session) : new UploadControl();

// Ctrl-C cancels through `signal`, which keeps the multipart upload alive.
const stop = new AbortController();
process.once("SIGINT", () => stop.abort());

let checkpointed = Boolean(saved);
const body = await openAsBlob(path); // a lazy, file-backed Blob

try {
  const result = await files.upload(key, body, {
    control,
    contentType: "application/octet-stream",
    multipart: { partSize: 8 * 1024 * 1024, concurrency: 4 },
    signal: stop.signal,
    onProgress: ({ loaded, total = size }) => {
      // The first callback fires once the session exists: persist it now,
      // before any part is in flight.
      const session = control.toJSON();
      if (!checkpointed && session) {
        const checkpoint: Checkpoint = { key, mtimeMs, session, size };
        writeFileSync(checkpointPath, JSON.stringify(checkpoint));
        checkpointed = true;
      }
      console.log(`${((loaded / total) * 100).toFixed(1)}%`);
    },
  });
  await rm(checkpointPath, { force: true });
  console.log(`Uploaded ${result.key} (${result.size} bytes)`);
} catch (error) {
  if (error instanceof FilesError && error.aborted) {
    console.log("Stopped. Run the same command again to resume.");
    process.exit(130);
  }
  if (saved && error instanceof FilesError && error.code === "NotFound") {
    // The multipart upload was aborted, completed, or expired server-side.
    await rm(checkpointPath, { force: true });
    console.error(
      "The saved upload no longer exists. Run again to start over."
    );
    process.exit(1);
  }
  throw error;
}

The parts that matter:

• When the token is written. control.toJSON() is undefined until CreateMultipartUpload returns, and the session is created asynchronously, so reading it right after calling upload() gets nothing. The first onProgress call fires after the session exists and before any part is sent, so the script writes the checkpoint there, synchronously.
• The body. openAsBlob returns a Blob backed by the file, and the SDK slices it lazily, so only the parts in flight sit in memory (part size × concurrency, 32 MiB here). Node’s docs say reads from that Blob fail if the file changes after it was opened, which covers edits during a run. The size and mtimeMs check covers edits between runs.
• Ctrl-C. Aborting through signal cancels in-flight parts and rejects with a FilesError whose aborted is true, but leaves the multipart upload on S3 so the next run can resume it.
• A dead session. If the upload ID no longer exists, ListParts fails and the SDK throws NotFound. The script deletes the stale checkpoint so the next run starts a new upload.

Interrupt it and resume

Run it, kill it partway, and run the same command again:

node upload.ts ./db.tar backups/db.tar    # or: bun upload.ts ./db.tar backups/db.tar
# kill it mid-transfer: Ctrl-C, or kill -9 from another terminal
node upload.ts ./db.tar backups/db.tar

To see what S3 kept between the two runs, ask it directly. files.raw is the adapter’s S3Client, so the escape hatch is one send away:

import { readFile } from "node:fs/promises";

import { ListPartsCommand } from "@aws-sdk/client-s3";
import { Files, type ResumableUploadSession } from "files-sdk";
import { s3 } from "files-sdk/s3";

const files = new Files({ adapter: s3({ bucket: "uploads" }) });

const { session } = JSON.parse(
  await readFile(`${process.argv[2]}.upload.json`, "utf8")
) as { session: ResumableUploadSession };

if (session.provider !== "s3") {
  throw new Error(`Not an S3 session: ${session.provider}`);
}

// One page holds up to 1,000 parts; enough for this check.
const page = await files.raw.send(
  new ListPartsCommand({
    Bucket: session.bucket,
    Key: session.key,
    UploadId: session.uploadId,
  })
);
const parts = page.Parts ?? [];
console.log(
  `${parts.length} parts stored:`,
  parts.map((part) => `${part.PartNumber} (${part.Size} B)`).join(", ")
);

Against a local MinIO server, with a 300 MiB file of random bytes cut into 38 parts of 8 MiB:

• kill -9 at 32%. The script had printed 12 confirmed parts, but ListParts showed 13. One part finished on the server after the last progress line and before the process died. The next run started its progress at 34.7%, which is exactly 13 parts, uploaded the other 25, and completed. The SHA-256 of the downloaded object matched the local file.
• Ctrl-C at 21%. The script exited with code 130 and ListParts showed 8 parts. The parts in flight were cancelled, not finished. The next run started at 21.3% and completed with a matching hash.

MinIO isn’t S3, but the sequence of calls is the same S3 API, and the server’s part list is what the resume trusts in both cases.

Keep the key and the bytes the same

• The key. The token is bound to its bucket and key. Resuming under another key throws Resume token does not match this upload's bucket/key. The key includes the Files instance’s prefix, so changing the prefix between runs counts as a different key.
• The bytes. The SDK checks only that each stored part has the size this file’s slicing expects. It doesn’t fingerprint content. In the local MinIO run, resuming a token with a different 300 MiB file completed without an error, and the resulting object’s SHA-256 matched neither file. If size and modification time aren’t strong enough for your files, store a hash of the first and last few MiB in the checkpoint and compare that.
• The part size. S3 needs every part except the last to be at least 5 MiB, and R2 requires all parts except the last to be the same size. The SDK pins the part size in the token, and on resume the token’s value replaces whatever multipart.partSize you pass. In the local run, resuming an 8 MiB session with partSize: 16 * 1024 * 1024 still sliced at 8 MiB, and the final ETag ended in -38. You can change concurrency between runs; it isn’t pinned.
• The body type. A ReadableStream is rejected before any request: Pausable/resumable uploads require a body with a known length (File, Blob, ArrayBuffer, typed array, or string), not a ReadableStream — a stream can't be re-read to resume. If your data arrives as a stream, write it to a temporary file first, or use plain multipart, which streams but can’t resume.

Stop, cancel, and clean up

There are four ways to stop an upload, and they leave S3 in different states:

To give up on a saved upload, such as one whose source file is gone, use files.abortUpload() with the checkpoint’s key and token:

import { readFile, rm } from "node:fs/promises";

import { Files, type ResumableUploadSession } from "files-sdk";
import { s3 } from "files-sdk/s3";

const files = new Files({ adapter: s3({ bucket: "uploads" }) });

const checkpointPath = `${process.argv[2]}.upload.json`;
const { key, session } = JSON.parse(await readFile(checkpointPath, "utf8")) as {
  key: string;
  session: ResumableUploadSession;
};

// Throws if the token is for another key or bucket; an upload that already
// completed or was aborted resolves without error.
await files.abortUpload(key, session);
await rm(checkpointPath);
console.log(`Discarded the saved upload for ${key}`);

Against the local MinIO server, abortUpload() on a stopped upload’s token made the next ListParts fail with NoSuchUpload. A second call with the same token resolved, and passing the token with another key threw Resume token does not match this upload's bucket/key. before discarding anything.

Don’t reach for UploadControl.from(token).abort() for this. abort() can only discard a session the control is currently driving, and a control rebuilt from a token and never passed to upload() has nothing to discard. In the local MinIO run, calling it set the control’s status to "aborted" and cleared its token, and ListParts still returned every part. On files-sdk 2.6, which has no abortUpload(), send an AbortMultipartUploadCommand with the token’s bucket, key, and uploadId through files.raw.

When you call control.abort() in-process, await it before deleting your checkpoint. The upload() promise can reject before the AbortMultipartUpload request finishes. AWS also notes that part uploads still in progress can succeed after an abort, so an abort issued mid-part may leave a part behind.

Abort abandoned uploads with a lifecycle rule

A crashed uploader that never runs again leaves its parts in the bucket. AWS keeps them with no expiry and bills them as storage until the upload is completed or aborted. Add a rule with the AbortIncompleteMultipartUpload action:

{
  "Rules": [
    {
      "ID": "abort-incomplete-uploads",
      "Status": "Enabled",
      "Filter": { "Prefix": "" },
      "AbortIncompleteMultipartUpload": { "DaysAfterInitiation": 7 }
    }
  ]
}

aws s3api put-bucket-lifecycle-configuration \
  --bucket uploads \
  --lifecycle-configuration file://lifecycle.json

Pick the number of days to be longer than any resume you want to support. Once the rule aborts a session, the next resume of that token fails with NotFound, which the uploader above handles by starting over. The rule never touches completed objects.

R2 already has a default rule: per Cloudflare’s object lifecycle docs, buckets expire multipart uploads seven days after initiation, and you can set a different AbortIncompleteMultipartUpload rule per prefix.

Which adapters can resume after a restart

GCS, Azure, Dropbox, OneDrive, Vercel Blob, and several others resume across processes through their own protocols. Resumable uploads lists every adapter. If you branch at runtime, note that files.capabilities.resumable is true for the in-process-only adapters too, so it tells you whether control works, not whether a token survives a restart.

A process restart isn’t a browser reload

A Node or Bun process restarts with the same file on disk. The checkpoint holds the path’s size and modification time, so the script can reopen the file and confirm it’s the one it was uploading.

A reloaded browser tab has the token, if you stored it, but not the file. A File handle doesn’t survive a reload, so the user has to pick the file again, and you compare its name, size, and lastModified with what you saved before resuming. There’s a second limit: in files-sdk 3.x, the browser bindings (useFiles and createFilesClient) don’t accept an UploadControl. They upload each file with one presigned request or through your gateway. Running s3() itself in a browser would mean shipping AWS credentials to it, so don’t. Resumable uploads with this SDK belong in processes that hold credentials: CLIs, workers, backup jobs, and desktop apps’ main processes.

Troubleshooting

Resume token does not match this upload's bucket/key. The checkpoint belongs to another key, or the Files instance’s prefix or bucket changed. Delete the checkpoint, or run with the original key.

NotFound when resuming. The upload ID is gone: it was completed, aborted, or expired by a lifecycle rule. MinIO’s message reads The specified multipart upload does not exist. The upload ID may be invalid, or the upload may have been aborted or completed. Start a new upload.

This UploadControl has already driven an upload. Each control drives one upload() call. Build a new one with new UploadControl() or UploadControl.from(token) for every attempt.

The object completed but its contents are wrong. The file changed between runs and kept its length. Nothing at the S3 layer catches this. Tighten the checkpoint’s identity check, then delete the object and upload it again.

pause-able/resumable uploads are not supported by this adapter. The adapter has no resumable driver. On R2, MinIO, or RustFS outside Cloudflare Workers, switch to client: "aws-sdk" and install the @aws-sdk/* packages.