Combined writing for FSTree

[roman-khimov]

We can think of replacing peapod with it.

[roman-khimov]

Notice that this scheme also helps with #1951 and #2107 (!) if we're to store meta separately.

[roman-khimov]

So what is the peapod's fate?

TBD

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 54.26357% with 118 lines in your changes missing coverage. Please review.

Project coverage is 23.62%. Comparing base (7baa16c) to head (1b40ec4). Report is 13 commits behind head on master.

Files with missing lines	Patch %	Lines
...ject_storage/blobstor/fstree/fstree_write_linux.go	71.42%	22 Missing and 12 partials :warning:
pkg/local_object_storage/blobstor/fstree/fstree.go	57.33%	27 Missing and 5 partials :warning:
...node/config/engine/shard/blobstor/fstree/config.go	0.00%	18 Missing :warning:
pkg/local_object_storage/blobstor/fstree/option.go	0.00%	12 Missing :warning:
cmd/neofs-lens/internal/storage/root.go	0.00%	9 Missing :warning:
cmd/neofs-node/storage.go	0.00%	5 Missing :warning:
cmd/neofs-node/config.go	0.00%	4 Missing :warning:
...ct_storage/blobstor/fstree/fstree_write_generic.go	40.00%	3 Missing :warning:
cmd/neofs-lens/internal/storage/sanity.go	0.00%	1 Missing :warning:

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2814      +/-   ##
==========================================
+ Coverage   23.51%   23.62%   +0.11%     
==========================================
  Files         776      775       -1     
  Lines       45325    45513     +188     
==========================================
+ Hits        10656    10754      +98     
- Misses      33822    33907      +85     
- Partials      847      852       +5     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[carpawell]

So what is the peapod's fate?

TBD

I am asking cause we have to rework some things if there is no "small" storage anymore.

---

Have you considered GET bench? Isn't it slower to read a combined object for only a part of it if there are a few parallel GET requests? Like reading 8MB for a 4KB object 128 times can be slower than reading 4KB 128 times from a peadod.

[roman-khimov]

Isn't it slower to read

For sure. But I doubt it's noticeable. I'll try to bench it.

[roman-khimov]

Ironically, it doesn't have any conflicts with current master even without a rebase. So my proposal is to merge this as is and then deal with peapod. @cthulhu-rider, @carpawell?

[carpawell]

Have you considered GET bench? Isn't it slower to read a combined object for only a part of it if there are a few parallel GET requests? Like reading 8MB for a 4KB object 128 times can be slower than reading 4KB 128 times from a peadod.

@roman-khimov, my questions are the same. I am afraid there (at least this PR) is not enough info about if it is really a better one than we had before, from any perspective (i consider it a worse solution for GETs). If there is a single slower thing in results, we should discuss it and find out (at least write it somewhere) what is better for us.

[roman-khimov]

But you've seen the test results. GETs are the same, PUTs are ~15% better for the target cases. GETs do incur some additional overhead, but it's totally acceptable, there is no real difference in practice.

[roman-khimov]

Rebased, otherwise lens failed to build after the merge (even though the merge itself is clean).

[roman-khimov]

There is potential for improvement still left, oid-offset mapping can be stored at the end of file minimizing seeks.