tiering

Route operations between a hot and a cold adapter by size, prefix, or age. Uploads land in the right store, reads transparently find them again, and tier() moves objects between stores - body-transparent, no native dependencies.

The built-in tiering() plugin spreads one logical bucket across two adapters - a hot tier for what you reach for often and a cold tier for what you rarely touch. An upload lands in the tier your route function picks; every read transparently finds it again. The hot tier is the instance's own adapter (reached through the rest of the pipeline); the cold tier is a second adapter you pass in.

It's body-transparent: it never buffers or transforms bytes - a cross-tier copy streams straight through - so streaming, range downloads, url(), and signedUploadUrl() all keep working. It has no native dependencies and works on any pair of adapters.

import { createFiles } from "files-sdk";
import { s3 } from "files-sdk/s3";
import { tiering } from "files-sdk/tiering";

const files = createFiles({
  adapter: s3({ bucket: "hot" }), // hot tier
  plugins: [
    tiering({
      cold: s3({ bucket: "cold" }),
      // archives go cold; everything else stays hot
      route: ({ key }) => (key.startsWith("archive/") ? "cold" : "hot"),
    }),
  ],
});

await files.upload("photo.jpg", body); // → hot
await files.upload("archive/2019.zip", zip); // → cold
await files.download("archive/2019.zip"); // transparently read from cold

tierOf() and tier() are contributed by the plugin's extend, so they only appear on the type when you construct with createFiles (identical to new Files() at runtime).

The route function

route is handed { key, size? } and returns "hot" or "cold". It's called once per logical operation:

  • on upload, with size set to the body's declared byte length when it's known up front (a string, Blob, ArrayBuffer, or typed array - a streaming body has no declared length);
  • on every other decision (reads, delete, the destination of copy / move, signedUploadUrl, and locating an object), with size omitted.

Route by prefix and the decision is a pure function of the key, so reads land on the right tier first try with zero overhead:

route: ({ key }) => (key.startsWith("cold/") ? "cold" : "hot");

Route by size and writes go by size, but reads can't recompute it - so turn on fallback and a read that misses the guessed tier checks the other:

tiering({
  cold,
  route: ({ size }) =>
    size !== undefined && size > 5_000_000 ? "cold" : "hot",
  fallback: true,
});

What each verb does

  • upload routes by route({ key, size }) and writes to that tier.
  • download / head / url / exists consult the routed tier.
  • delete removes the routed tier's copy.
  • copy / move locate the source, route the destination by its key, and use a native same-tier op when both keys land in one tier - or stream the bytes across (preserving content type and metadata) when they differ. A move then deletes the source.
  • list merges a page from each tier into one result, keys sorted within the page (see merged listing).
  • signedUploadUrl signs against the tier route({ key }) picks. The resulting direct upload bypasses the plugin, so it can't be size-routed or deduplicated.

fallback

By default routing is deterministic: every operation touches exactly the one tier route names, with no extra round-trip. That's the right mode for prefix / key-based routing.

Set fallback: true to treat an object's tier as discoverable rather than fixed:

  • a read that misses the routed tier retries the other tier;
  • delete removes the key from both tiers;
  • an upload evicts the key from the other tier, so a re-upload that flips tiers leaves exactly one copy.

Turn it on whenever the tier isn't a pure function of the key - size-based routing, or when you move objects between tiers with tier(). The cost is at most one extra round-trip on a cold read.

Moving objects between tiers

Two methods land on the instance via extend:

await files.tierOf("photo.jpg"); // "hot" | "cold" | undefined
await files.tier("photo.jpg", "cold"); // stream it to cold, drop the hot copy

tier(key, target) is the lever for age-based transitions, which can't be a write-time decision (a new object's age is zero). Run it on a schedule: list, check each object's lastModified, and tier down what's gone cold.

for await (const file of files.listAll()) {
  const age = Date.now() - (file.lastModified ?? 0);
  if (age > THIRTY_DAYS) {
    await files.tier(file.key, "cold");
  }
}

Because tier() moves objects to a tier route wouldn't pick from the key alone, pair it with fallback: true so reads still find what you've moved.

Merged listing

list() returns objects from both tiers. It fetches a page from each, deduplicates (hot wins), sorts by key, and paginates the two tiers independently behind one composite cursor - so listAll() walks the whole namespace across both stores.

Two things to know: a page can hold up to the sum of both tiers' page sizes (each tier's limit is applied per tier), and keys are sorted within a page but - because the tiers paginate independently - not globally across pages. For a single fully-ordered enumeration, list an adapter directly.

Ordering and prefixes

  • Place it last (innermost). Body-transforming plugins like encryption() and compression() wrap tiering() and transform the op on the way in, so they apply to both tiers:

    plugins: [encryption(key), tiering({ cold, route })];
  • Address objects by caller-facing keys. The cold adapter does not receive the instance prefix, so configure its own bucket / container and avoid a client prefix on a tiering instance.

Things to keep in mind

  • The cold tier is a real store. Cold reads pay its latency; a hot→cold tier() or cross-tier copy transfers the bytes between adapters.
  • Presigned uploads bypass routing. A signedUploadUrl() upload lands directly in the routed tier and isn't size-routed or deduplicated.
  • Without fallback, routing must be stable per key. If an object can live in a tier the key wouldn't route to (size-based routing, or a tier() move), enable fallback or reads will miss it.

On this page