deltachat-core-rust
deltachat-core-rust copied to clipboard
deltachat
limit unbound grow of state updates in webxdc instances
bots managing a lot of webxdc instances accumulate state updates forever and becomes an storage issue over time
also for users, apps like the Draw app, send a "big" canvas/image blob as state updates, and only the last image is needed, it uses a lot of space to keep all this big blobs forever in long running instances
if we offer a method to remove updates, implementation will use IndexDB or LocalStorage to save the data, unfortunately, this has the disadvantage that this does not go into backups.
to avoid a worsening, maybe give the webxdc the choice to store data inside Delta Chat as well. maybe just offer an LocalStorage API inside the webxdc object.
(we thought about that also earlier, however, for easy adaption of existing code, we voted for existing storages for first implementations) (and that they IndexDB/LocalStorage work is still a good things we should aim for, for many things this is still sufficient)
There was an idea to allow the app to store a summary of updates. A JavaScript call saying "replace all updates up to X with this initial state". Then next time the app is started, it gets the saved state and updates starting from X.
On Mon, May 15, 2023 at 08:49 -0700, link2xt wrote:
There was an idea to allow the app to store a summary of requests. A JavaScript call saying "replace all updates up to X with this initial state". Then next time the app is started, it gets the saved state and updates starting from X.
would the reconciliation/snapshot update be sent to peers or would it only be local?
If the original xdc app sender does a "snapshot" update, removing historic app updates, and then the app gets re-sent, then this would go out instead of the removed historic updates. What happens if other peers also do "snapshot" updates?
Maybe we could add a "snapshot=true" flag to the sendUpdate API which would remove all prior updates but also send out the reconciled app state to other peers?
Maybe we could add a "snapshot=true" flag to the sendUpdate API which would remove all prior updates but also send out the reconciled app state to other peers?
could lead also to problems because you can not be sure to already have updates of all users due to possible delays.
would the reconciliation/snapshot update be sent to peers or would it only be local?
it should be local, you don't need to synchronize your local trimming with other peers, similar to local storage, but it is automatically synchronized among devices because all devices will do the same given the same stream of state updates,
when re-sending, that only affects people that don't have the app yet, so the "snapshot" needs to be sent along with the other state updates and the webxdc file
The issue is a duplicate of https://github.com/deltachat/deltachat-core-rust/issues/4558 It was fixed by https://github.com/deltachat/deltachat-core-rust/pull/4565
As for update snapshotting and other ways to compress updates within a single webxdc app, there are currently no plans for such APIs. If we decide to add it, better open a new issue.