market icon indicating copy to clipboard operation
market copied to clipboard

People can upload data easily (and we don't store it)

Open trentmc opened this issue 5 years ago • 3 comments

Summary Provide publishers with an easy solution to drag & drop some files during publish flow and in background they are added to decentralized storage. But where we don't store it or pay for storage.

Motivation Status quo: publishers must a provide url for their data, which takes technical savvy. Removing this friction will help adoption.

In Ocean Commons, we had where people could "upload to IPFS" for a smoother flow. They would drag and drop the file, and it would get uploaded. However since IPFS does not do storage, we were actually doing the storage on Pacific nodes. This is not a good idea because then we inadvertently start hosting a ton of files for free.

Target outcome

  • have a GUI / UX similar to what we had in Ocean Commons
  • but store somewhere that we don't have to pay for.

Where to store

  • Recommended: Arweave - pay once, store forever
  • (Backup: Filecoin - pay once, store for a long time)
  • IPFS could still be used as the intermediary ("IPFS pinning"). Data would be encrypted.

trentmc avatar Aug 10 '20 09:08 trentmc

Our team is working on this! As mentioned in our recent issue/PR, overhead is still a challenge. There may be some ways to decrease overhead but will require a bit of engineering + experimentation https://github.com/oceanprotocol/market/issues/1483

https://port.oceanprotocol.com/t/project-coral-core-tech-improvement-desci-oceandao-berlin-hackathon-round-17/1824

hebbianloop avatar Jun 03 '22 14:06 hebbianloop

our recent issue/PR

To be clear, that's an issue. And it's fine as an issue right now, given that there is discussion on the specs. A PR will be needed at some point though.

trentmc avatar Jun 06 '22 09:06 trentmc

As a reminder cause it's mentioned in this issue's description as a goal. This is how the file form field looked in Commons:

Screen Shot 2022-06-06 at 12 49 43

"From URL" revealed the URL text input, "Add to IPFS" opened up a drag & drop field, dropping a file was sending this to our own IPFS node's API and we pinned it there. App then receives back the new IPFS URL and uses that as the final file URL. No centralized 3rd party needed at all. We don't run our own IPFS node anymore, but nice solutions like https://web3.storage came along anyway

kremalicious avatar Jun 06 '22 10:06 kremalicious

We have other issues that handle this in a more direct way

mihaisc avatar Sep 29 '22 13:09 mihaisc