shared files

[samuelcolvin]

we could reused files when two sites (or the same site) use the same file.

This would required us to hash all files as they're uploaded and store the hash.

There's no way to change the TTL on key, so it would also require some custom logic to duplicate the file (with the new TTL), then delete the old file.

I guess we'd do something like

  const content_key = `site:${random-key}`
  await HIGH_TMP.put(content_key, file_content, {
    expiration: max([site_1_expiration, site_2_expiration]),
    metadata: {content_type, size},
  })
  await HIGH_TMP.put(`site:${public_key_1}:${path_1}`, '1', {
    expiration: site_1_expiration,
    metadata: {content_type, size, content_key},
  })
  await HIGH_TMP.put(`site:${public_key_2}:${path_2}`, '1', {
    expiration: site_2_expiration,
    metadata: {content_type, size, content_key},
  })

Quite a lot of faff, but would save quite a lot of space.

Could we run out of memory when copying the key?