ethersphere
Improve Stamp Management By Abstracting Amount and Depth and Adding Auto-Dilution / Top-up
Summary
Management of the amount and depth of stamp batches are a significant hurdle for anyone new to using Swarm. To efficiently manage a batch of stamps requires a decent understanding of what each of these values are and how they relate to the amount which can be stored by the batch and how long the data will stay online.
This requirement places a large burden on new users since understanding how amount and depth work, and when it is best to top up or dilute a batch is a non-trivial task.
Rather than placing this burden on the user, they should simply need to specify how much data they wish to store for how long, and the appropriate amount and depth should be selected for them.
Furthermore, stamp management can be further simplified with the addition of a combined auto-topup and auto-dilution feature. This should be an optional feature which when enabled, when an upload fails due to a batch becoming fully utilized, automatically dilutes the batch and also tops up the batch (the topup must also be done to ensure the TTL does not change), and then continues the upload seamlessly.
Probably this feature should include an option for budget setting. In other words, the user should be allowed to specify a budget of how much xBZZ is allowed to be spent on auto-topup/dilutions.
Motivation
It greatly simplifies the user experience for uploading data to Swarm. They no longer need to understand what amount and depth are, and when to topup or dilute. All they need to understand is that storage prices on Swarm are paid for with postage batches, but that their utilization is non-deterministic, and so they must sometimes be refilled with more xBZZ if they become fully utilized. Then all the user needs to choose is how much to upload for how long, whether they want to auto-refill the postage batch when it becomes fully utilized, and what their budget limit is.
Implementation
I cannot implement it. As for details, I think when purchasing a postage batch amount and depth should be changed to volume and duration which specify the volume of data and the desired TTL. Additional options for auto-refill and budget will also need to be added for purchasing stamps.
Drawbacks
I'm not certain whether there are any advantages to allowing users to manually specify amount and depth? If there are any advantages, then we can still allow them to optionally buy stamps by manually setting these values.
I can't think of any reasons not to include this otherwise.
I would propose both options stay available : either set amount+depth or volume+duration. (Or maybe even other combinations like depth+duration should not be disallowed. Though I see not immediate use case for them.)
I would propose both options stay available : either set amount+depth or volume+duration. (Or maybe even other combinations like depth+duration should not be disallowed. Though I see not immediate use case for them.)
I see no problem with leaving both available. But I do think that setting by "volume+duration" should be the default recommended method, and "depth+amount" should be relegated to an "Advanced" section in the docs.
I see no problem with leaving both available. But I do think that setting by "volume+duration" should be the default recommended method, and "depth+amount" should be relegated to an "Advanced" section in the docs.
I'd actually take it a step further in that all documentation (read: docs.ethswarm.org) doesn't mention depths at all and does "volume+duration", but the depths option gets documented within code, and within the OpenAPI YAML for anyone that chooses to do it (i.e. don't document the use of the depths at all if there is a volume+duration methodology for builders and leave it in the domain of core node developers).
I see no problem with leaving both available. But I do think that setting by "volume+duration" should be the default recommended method, and "depth+amount" should be relegated to an "Advanced" section in the docs.
I'd actually take it a step further in that all documentation (read: docs.ethswarm.org) doesn't mention depths at all and does "volume+duration", but the depths option gets documented within code, and within the OpenAPI YAML for anyone that chooses to do it (i.e. don't document the use of the depths at all if there is a volume+duration methodology for builders and leave it in the domain of core node developers).
Do you think the docs should at all be aimed at people who might want to develop their own clients/contribute to the Bee codebase or only at people building on Swarm? If you DO think it should be aimed at both, I'd suggest just moving those topics to an "Advanced" section or a "Contributors" section, something like that
While converting amount/depth to a volume/duration (or adding new endpoint) is feasible, it's important to note that the resulting BZZ cost per chunk is non-deterministic. This adds complexity to the system. Bee is intended to be a simple as possible infrastructure endpoint, and adding features like auto top-up and dilution not only complicates the code and maintenance but also introduces potential performance issues, unexpected background behaviors, and additional points of failure.
For users who need these advanced capabilities, an "external" solution (such as Swarm Desktop or the latest beekeeper stamper feature with proper setup instructions via docker-compose) is a more appropriate approach. This way, the core Bee client remains lean, while users still have the option to implement enhanced functionality if desired.
I would even say that any automatic funds (BZZ) spending from the wallet is dangerous without very strong notification sistem that will inform a user about the intention to top-up, to give the user a time to stop it and a notification that such action is done. Internet domain auto-renewals at domain name registries is an analogous flow.
A few scenarios come to my mind as problematic and all goes back to the user lack of attention what is the auto top-up should intend to solve. One is that a user forgets that auto top-up is active and it drains the funds from a wallet. Another one is change in storage price, where a user will not be aware of a potential price increase and larger amount will be spent. Even with the budget limit, it is important to notify user that top-up is successful or not.
With that aspect of this feature, I think that it is better to have an external solution, as Ljubisa proposed where these complexities (that are not critical to the swarm functionality) could be solved, just by using bee's api, like beekeeper stamper is doing.
I think that the idea under this issue is fundamentally good, but should be implemented in a different way. Partially in bee, changing the incentive contract, and partially in client.
We have to keep separate the logic and UX layers, in my opinion:
- low level: keeps amount and depth, and has 1:1 relation to implementation mechanics. This direct approach can't be hidden from bee, but could anyway be improved. For example, I believe that my proposal to pre-determinate the expiring block is still valid and could work as a
durationvalue implemented directly in bee: https://github.com/ethersphere/storage-incentives/issues/117 - high level: provide
volumeperspective, and should be implemented into applications layer. To show volume value instead of depth is trivial, and some clients already do it. I don't think that this point-of-view should be implemented directly into Bee, because keeping both depth and volume units would over-complicate the interface. Better to keep a single consistent unit of measure across APIs
Auto-topup and auto-dilute can be implemented in clients, and maybe the client should also verify with actual buckets state if an operation of this kind is required even before to start the upload. With my work I'm following this approach.
I'd actually take it a step further in that all documentation (read: docs.ethswarm.org) doesn't mention depths at all
+1 on simplifying and reducing the massive information overload in the general docs.ethswarm.org
"Advanced" section or a "Contributors" section
maybe a separate developer.ethswarm.org for the advanced developer documentation.
