unstorage icon indicating copy to clipboard operation
unstorage copied to clipboard

feat: Add `aws-s3` driver (http)

Open becem-gharbi opened this issue 1 year ago โ€ข 8 comments

๐Ÿ”— Linked issue

#11

โ“ Type of change

  • [ ] ๐Ÿ“– Documentation (updates to the documentation, readme, or JSdoc annotations)
  • [ ] ๐Ÿž Bug fix (a non-breaking change that fixes an issue)
  • [ ] ๐Ÿ‘Œ Enhancement (improving an existing functionality like performance)
  • [x] โœจ New feature (a non-breaking change that adds functionality)
  • [ ] ๐Ÿงน Chore (updates to the build process or auxiliary tools and libraries)
  • [ ] โš ๏ธ Breaking change (fix or feature that would cause existing functionality to change)

๐Ÿ“š Description

There is already an open PR (#96) for an implementation based on AWS sdk.

This PR adds an AWS s3 driver based on HTTP for a lightweight bundle size and edge runtime compatibility.

~This is a WIP, I'm not sure if all driver methods can be implemented readme.~

Progress

  • [x] Implement main methods getItem getItemRaw setItem setItemRaw getKeys getMeta clear removeItem hasItem
  • [x] Refactor & optimize
  • [x] Add respective tests
  • [x] Add respective docs

๐Ÿ“ Checklist

  • [x] I have linked an issue or discussion.
  • [x] I have updated the documentation accordingly.

becem-gharbi avatar Dec 29 '23 18:12 becem-gharbi

Hi @becem-gharbi, thanks for your PR! I would like to address a few points to help get the driver ready for merging:

  • Remove your playground folder from the PR
  • Add the required tests for your driver
  testDriver({
    driver: driver({
      DRIVER_OPTIONS
    }),
  });
  • Lazy load the connection for example with getAwsClient(), you can check some other drivers for similar implementations
  • Can we reevaluate on aws4fetch? It seems unmaintained and the preferred solution from #11 was direct HTTP, which should be possible with ofetch and uncrypto
  • Implement at least getItem, setItem and clear in addition to the existing methods for basic string based kv operations
  • Add respective docs for the driver

itpropro avatar Jan 02 '24 19:01 itpropro

Hi @itpropro, thanks for your help. This solution is a direct HTTP client, aws4fetch is there to sign the request by generating headers for authorization. To be honest I'm not familiar with AWS signature. However the package is stable and works well as expected.

becem-gharbi avatar Jan 03 '24 17:01 becem-gharbi

Live Preview ready!

Name Edit Preview Latest Commit
unstorage Edit on Studio โ†—๏ธŽ View Live Preview 7b24a35cc5ee4ea1154fad20ea76242ed6cadcd0

nuxt-studio[bot] avatar Jan 06 '24 10:01 nuxt-studio[bot]

Hi @itpropro, the driver is now ready to review. Regarding test, I'm not sure if using process.env is the way to pass driver options.

becem-gharbi avatar Jan 06 '24 10:01 becem-gharbi

Hi, I have recently converted this PR to a draft. The reason is because this driver is intended to be used with any S3 compatible service. The problem is that some differences exist in term of their implementations. These differences reflect in the driver usage. I have mentioned them in docs. I'm not sure about the current approach and I'm looking forward to your feedback.

becem-gharbi avatar Jan 20 '24 10:01 becem-gharbi

(@becem-gharbi sorry for delay on my side to give feedback, if you like, we can still keep it open or alternatively i can continue on this. just assuring you it was not intentionally left unattended)

pi0 avatar Mar 15 '24 12:03 pi0

@becem-gharbi Thank you, you've done a good job again. we need to make sure you support all s3 protocols. cloudflare r2, amazon s3 etc.

@pi0 can you prioritize this issue, because this is a need in many areas such as amazon and cloudflare r2. It is an expected feature to use nuxt-og-image together.

productdevbook avatar Mar 16 '24 04:03 productdevbook

Thanks a lot @becem-gharbi for the PR! For now, we have hosted your work here.

Your code passes all the tests with 2 very small issues. See open issues in the repo.

Thank you !

amandesai01 avatar Jul 25 '24 16:07 amandesai01