noobaa-core icon indicating copy to clipboard operation
noobaa-core copied to clipboard

NSFS : S3 rm with --recursive option does not delete all the objects

Open naveenpaul1 opened this issue 4 months ago • 0 comments

Explain the changes

  1. Issue is with list_object api, list_object is failing with specific dir strucute and not returning complete items in that bucket.
  2. dir_key and marker_dir comparison will fail when there are dir with structure slimier to "-/migrateDir.popFSDir.3803140/" and "/migrateDir.popFSDir.3803140.OldSet/" because "/" considered greater than "." to fix this all the backslash is replaced with space. This updated marker and dir_key used only for comparison.

Dir structure like this is will fail

/populatefs_dir --/migrateDir.popFSDir.3803140 ----/sparseDir -----lnk.0.qqqqqqqqqqqqqq -----lnk.1.qqqqqqqqqqqqqq -----lnk.2.qqqqqqqqqqqqqq --/migrateDir.popFSDir.3803140.OldSet ----/sparseDir -----lnk.0.qqqqqqqqqqqqqq -----lnk.1.qqqqqqqqqqqqqq -----lnk.2.qqqqqqqqqqqqqq

Issues: Fixed #xxx / Gap #xxx

  1. S3 upload is successful for a data set (mix of all type of objects, sparse files, files with long name, compressed files etc)
  2. When download via "aws-alias s3 rm s3://bucket-10909 --recursive" , this still displays the left over directories and objects in the bucket. I checked from the FS that the dataset still exists there.

https://github.com/noobaa/noobaa-core/issues/8197

closed : PR https://github.com/noobaa/noobaa-core/pull/8317

Testing Instructions:

  1. run NC_CORETEST=true node ./node_modules/.bin/mocha ./src/test/unit_tests/test_namespace_fs.js
  • [ ] Doc added/updated
  • [ ] Tests added

naveenpaul1 avatar Sep 30 '24 06:09 naveenpaul1