neofs-node icon indicating copy to clipboard operation
neofs-node copied to clipboard

Rework shard path configurations

Open roman-khimov opened this issue 1 year ago • 5 comments

Is your feature request related to a problem? Please describe.

I'm always frustrated when I'm looking at the storage node config:

storage:
  shard:
    0:
      blobstor:
      - path: /srv/neofs/data0/blobovnicza
        type: blobovnicza
      - path: /srv/neofs/data0/tree
        type: fstree
      metabase:
        path: /srv/neofs/meta/metabase0.db
      writecache:
        path: /srv/neofs/meta/writecache0
    1:
      blobstor:
      - path: /srv/neofs/data1/blobovnicza
        type: blobovnicza
      - path: /srv/neofs/data1/tree
        type: fstree
      metabase:
        path: /srv/neofs/meta/metabase1.db
      writecache:
        path: /srv/neofs/meta/writecache1
    2:
      blobstor:
      - path: /srv/neofs/data2/blobovnicza
        type: blobovnicza
      - path: /srv/neofs/data2/tree
        type: fstree
      metabase:
        path: /srv/neofs/meta/metabase2.db
      writecache:
        path: /srv/neofs/meta/writecache2
...

What we want to configure here is:

  • three disks that store data
  • one disk that stores metabase(s)
  • one disk that is used for write caches

What we have in the configuration is:

  • separate path to bbcz and fstree
  • three metabases
  • three write caches

It easily leads to duplication and confusion. Of course it gives some additional flexibility, yet at the same time it's never needed.

Describe the solution you'd like

What it may look like:

storage:
  metabase:
    path: /srv/neofs/ssd/meta
  writecache:
    path: /srv/neofs/ssd/writecache
  shard:
    0:
      blobstor:
      - path: /srv/neofs/data0/
        type: bbcz+fstree
    1:
      blobstor:
      - path: /srv/neofs/data1/
        type: bbcz+fstree
    2:
      blobstor:
      - path: /srv/neofs/data2/
        type: bbcz+fstree
...

And never ever MkdirAllX, these directories must exist and then the node takes full control over them.

CC @532910.

roman-khimov avatar Aug 11 '23 09:08 roman-khimov

one disk that stores metabase(s)

storage: metabase: path: /srv/neofs/ssd/meta

Is it ok to lose every metabase at once?

carpawell avatar Aug 11 '23 11:08 carpawell

I've always thought that metabases are OK to lose (everything can be recovered from the actual data stores).

roman-khimov avatar Aug 11 '23 12:08 roman-khimov

How mkdir is related? What the blobstor: word is for? What other types are possible? Can a default one be specified?

532910 avatar Aug 13 '23 12:08 532910

How mkdir is related?

Right now when you have a /srv/neofs/data0/blobovnicza directory configured and only have a real /srv/neofs, /srv/neofs/data0/blobovnicza will be created by the node.

What the blobstor: word is for?

Not the best word, but it's a storage.

What other types are possible?

Right now not a lot of them. fstree and blobovnicza/peapod.

Can a default one be specified?

It has to be some specific type.

roman-khimov avatar Aug 13 '23 19:08 roman-khimov

https://github.com/nspcc-dev/ansible-neofs/blob/bcb7c519dd571529010d654ca3d603ab478808d9/roles/storage/defaults/main.yml#L34-L35

https://github.com/nspcc-dev/ansible-neofs/blob/bcb7c519dd571529010d654ca3d603ab478808d9/roles/storage/defaults/main.yml#L151-L177

532910 avatar Aug 14 '23 09:08 532910