neofs-node
neofs-node copied to clipboard
Reduce number of transmitted `ObjectService.Search` response messages
Is your feature request related to a problem? Please describe.
object SEARCH
operations are executed additively on all container nodes. Currently, server writes one response message to the stream per response from each individual node. When each node stores a few objects, this leads to a decrease in performance due to the overhead of additional messages when transmitting a fixed payload (list of object IDs for SEARCH
)
Describe the solution you'd like
buffer the results up to the limit volume, and then flush them into the stream. Seems like this component is a good place to do https://github.com/nspcc-dev/neofs-node/blob/9f4bf88ac84156020fef47e14e14b2d1d2ce6cd0/pkg/services/object/transport_splitter.go#L151-L184
Describe alternatives you've considered
leave as it is. One of the possible benefits of the current approach with client POV is faster receipt of primary results from faster nodes. However, if the client understands the internal structure of the cluster so deeply and wants to more flexibly control operations on it, then either:
- it can manage cluster nodes itself - for this all information is available (netmap, containers)
- the protocol can be extended with a buffering bypass service flag upon special feature request (not yet)
Additional context
currently it's hard to detect specifics of such internal container queries. I've noticed current case during deep debug within #2722
This requires both size and time constraints to balance latencies/bandwidth.