NSFS | Regular object key & dir content object - key/ co-existence and overriding

[romayalon]

Environment info

• NooBaa Version: Master
• Platform: NC

key and key/ co-existence -

Actual behavior

In general - On AWS key and key/ are 2 different objects, we can have both simultaneously while in NooBaa we didn't handle this edge case due to file system limitation, when trying to create obj1 and obj1/ directly on the FS we will get the following error -

% echo blbkabkab > obj1
% mkdir obj1/
mkdir: .: Not a directory

Expected behavior

If we do want to support it, it'll require a special design.

Key and Key/ overriding bugs -

Actual behavior

There are 2 faulty scenarios that were found, related to overriding of key and key/ -

1. PUT obj1.txt and then PUT obj1.txt/ - throws an error - NoSuchObject.
2. PUT obj2.txt/ and then PUT obj2.txt - throws an error - InternalError.

Steps to reproduce -

1. Tab 1 - Deploy dev NooBaa NC - sudo node src/cmd/nsfs --debug=5
2. Tab 2 -
   1. Create an account using NooBaa CLI - sudo node src/cmd/manage_nsfs.js account add --name account1 --user root --new_buckets_path=/private/tmp/dir1/ --debug 5
   2. Create a bucket using NooBaa CLI - sudo node src/cmd/manage_nsfs.js bucket add --name bucket1 --owner account1 --path /private/tmp/dir1/
   3. init AWS S3api alias pointing to NooBaa having access keys of account1 - alias s3api='AWS_ACCESS_KEY_ID=<access_key> AWS_SECRET_ACCESS_KEY=<secret_key> aws --endpoint-url=https://localhost:6443 --no-verify-ssl s3api'
   4. From here there are 2 scenarios -
      1. PUT obj1.txt and then PUT obj1.txt/ -
         1. PUT obj1.txt - s3api put-object --bucket=bucket1 --key=obj1.txt --body=obj1.txt
         2. Override obj1.txt with obj1.txt/ - s3api put-object --bucket=bucket1 --key=obj1.txt/ --body=obj1.txt
         3. NooBaa throws an error - NoSuchObject - in the logs it looks like we got ENOTDIR when renaming the new object file to obj1.txt/.folder. [Error: Not a directory] { code: 'ENOTDIR', context: 'Rename _old_path=/private/tmp/dir2/.noobaa-nsfs_66d04430da7de6f67dc274ec/uploads/9b718dce-05b7-49a8-ad06-73cafbf78ad9 _new_path=/private/tmp/dir2/obj1.txt/.folder ' }
      2. PUT obj2.txt/ and then PUT obj2.txt -
         1. PUT ob21.txt/ - s3api put-object --bucket=bucket1 --key=obj2.txt/ --body=obj2.txt
         2. Override obj1.txt with obj1.txt/ - s3api put-object --bucket=bucket1 --key=obj2.txt --body=obj2.txt
         3. NooBaa throws an error - InternalError - in the logs it looks like we got EISDIR when renaming the new object file to obj1.txt/.folder. [Error: Is a directory] { code: 'EISDIR', context: 'Rename _old_path=/private/tmp/dir1/.noobaa-nsfs_66d04430da7de6f67dc274ec/uploads/0aafbda8-6ad0-40d3-815f-24abddacacb1 _new_path=/private/tmp/dir1/obj2.txt ' }

Expected behavior

1. Handle these scenarios correctly.