atomic-server
atomic-server copied to clipboard
Improve performance of array append commits
https://github.com/ontola/atomic-data-docs/issues/50
A very common usecase in Atomic Server, is appending an item to an array. For example, when you press enter in a document.
This action is currently relatively expensive for larger arrays. (taking about 15ms on my laptop for an array of 30 items (edit: 5ms)). It scales O(n) where n is the count of array items. It first removes all existing atoms, then adds all the new ones.
Approaches:
Add append to commits
If we introduce an append action, this problem can be removed. Might make the Commit handler a bit more complicated, and I need to think about how this API should look in Commits.
Explore batching in sled
Maybe there is some sort of bottleneck that we can overcome if we stop batching all these inserts?
https://docs.rs/sled/latest/sled/struct.Batch.html
UPDATE: Tried batching, was about 10x as slow. Not good.
UPDATE 2: Most of the bad performance was because of the Tracing itself, as it cloned the resource every call. After that, it sped up about 10x.