bee
bee copied to clipboard
ethersphere
Store data on file system in dev mode
Summary
Save all the chunks and stamps on the file system instead of keeping them in memory.
Motivation
I'll start by saying that bee in testnet in unusable for various reasons, like slowness of transactions and impossibility of getting your account funded to even begin using it. Right now the only way to test your dapp with speed and efficiency is in dev mode.
The problem i'm facing is that the process in dev mode run out of memory very quickly. The error I get is stdout maxBuffer length exceeded often after uploading a single file in chunks (and not a very large one, few MBs).
Maybe is my system, since I'm on an Apple Silicon Mac, but still, it's unusable in the current state.
I believe that saving the chunks on the file system could solve this issue and it would also enable another nice feature which is the resumability of the previous state, so all the data and stamps created would not be lost after a restart.
Implementation
--
Drawbacks
I know from previous interactions about the dev mode that it is only supposed as a test for the APIs, but I think it should reflect more the actual bee implementation, without having to connect to a chain. Having to develop over swarm using the testnet or mainet is not feasible for when a project is in development, but it certainly is when it's in staging/production.
Hopefully the swarm testnet will improve once it moves to the Sepolia network. In the meantime, you might want to fire up a Sepolia node and start to collect some native coins from faucets. This is just my personal observation and hope, nothing official.
@ldeffenb Thanks, I didn't know about this migrations. Is there a ETA for this?
@mattiaz9 I have not seen a definite date, but the May dev update stated:
Looking ahead, an important improvement lies on the horizon. Swarm developers are preparing to transition from the soon-to-be-obsolete Goerli to the more robust Sepolia test network. This strategic move will provide a significantly improved staging and testing environment and drive further development of the network. As Mark Bliss elaborated, this transition will reduce costs for developers and improve conditions, including multiple faucets for gas fee payments.
https://blog.ethswarm.org/foundation/2023/swarm-community-call-25-may-recap/
Maybe after they get 1.17.* stabilized on mainnet?