atomic-server icon indicating copy to clipboard operation
atomic-server copied to clipboard

Improve performance of array append commits

Open joepio opened this issue 3 years ago • 0 comments

https://github.com/ontola/atomic-data-docs/issues/50

A very common usecase in Atomic Server, is appending an item to an array. For example, when you press enter in a document.

This action is currently relatively expensive for larger arrays. (taking about 15ms on my laptop for an array of 30 items (edit: 5ms)). It scales O(n) where n is the count of array items. It first removes all existing atoms, then adds all the new ones.

Screenshot 2022-01-24 at 21 44 28

Approaches:

Add append to commits

If we introduce an append action, this problem can be removed. Might make the Commit handler a bit more complicated, and I need to think about how this API should look in Commits.

Explore batching in sled

Maybe there is some sort of bottleneck that we can overcome if we stop batching all these inserts?

https://docs.rs/sled/latest/sled/struct.Batch.html

UPDATE: Tried batching, was about 10x as slow. Not good.

UPDATE 2: Most of the bad performance was because of the Tracing itself, as it cloned the resource every call. After that, it sped up about 10x.

joepio avatar Jan 24 '22 20:01 joepio