noobaa-core icon indicating copy to clipboard operation
noobaa-core copied to clipboard

Improve list objects performance on top of NS FS

Open nimrod-becker opened this issue 3 years ago • 0 comments

Environment info

  • NooBaa Version: VERSION
  • Platform: Kubernetes 1.14.1 | minikube 1.1.1 | OpenShift 4.1 | other: specify

Actual behavior

  1. All directory entries are brought to memory and ordered (FS doesn't return it in order), saved in cache
  2. If the directory size is larger than the cache size (64MB defaulted). then on each list command we bring everything to mem and organize

Expected behavior

  1. Add a config per account to return un-ordered list object (list v2 only)
  2. Still need to support pagination and delimiter, prefix

Steps to reproduce

More information - Screenshots / Logs / Other output

Should / Can use https://man7.org/linux/man-pages/man3/telldir.3.html https://man7.org/linux/man-pages/man3/seekdir.3.html

nimrod-becker avatar Jun 28 '21 15:06 nimrod-becker