s3 ceph test cases failing that were passing with earlier builds

[ketankhurana64]

Environment info

• NooBaa Version: master-20210904
• Platform: oc 4.8.5

Actual behavior

1. Some of the Test cases are failing that were passing for build master-20210226

Areas Impacted by failed test cases:

ListBuckets Bucket Listing Meta data operations Bucket Version Validation

Please refer attached sheet3 for complete result

Expected behavior

1. Test cases should pass as previously for sheet3 attached

Steps to reproduce

1. clone s3 ceph test
2. change the access key,secret key, endpoints as suggested
3. Execute the test cases

More information - Screenshots / Logs / Other output

hpo_s3_tests_list.xlsx

[akmithal]

Hi @ketankhurana64 , please define the test areas in the defect section for quick reference.

[nimrod-becker]

test_bucket_list_return_data test_bucket_create_delete test_object_metadata_replaced_on_put test_versioning_bucket_create_suspend test_versioning_obj_plain_null_version_removal test_versioning_obj_plain_null_version_overwrite_suspended test_versioned_concurrent_object_create_and_remove test_object_create_bad_md5_bad

[liranmauda]

Failed tests (Tested on Data bucket NOT nsfs bucket): s3tests_boto3.functional.test_s3.test_bucket_list_return_data.log s3tests_boto3.functional.test_s3.test_bucket_list_return_data_versioning.log s3tests_boto3.functional.test_s3.test_object_metadata_replaced_on_put.log s3tests_boto3.functional.test_s3.test_versioning_obj_plain_null_version_overwrite_suspended.log s3tests_boto3.functional.test_s3.test_versioning_obj_plain_null_version_removal.log

[liranmauda]

Passed tests (Tested on Data bucket NOT nsfs bucket): s3tests_boto3.functional.test_s3.test_bucket_create_delete.log s3tests_boto3.functional.test_s3.test_versioned_concurrent_object_create_and_remove.log s3tests_boto3.functional.test_s3.test_versioning_bucket_create_suspend.log

[liranmauda]

Testes with NooBaa (operator and core) master-20210911 Ceph tests CEPH_TESTS_VERSION=687ab24e7dd36809ee1b4eea667962dce6997b70 and run it using:

for i in $(S3TEST_CONF=src/test/system_tests/ceph_s3_config.conf  ./src/test/system_tests/s3-tests/virtualenv/bin/nosetests -v --collect-only 2>&1 | awk '{print $1}' | grep "test_bucket_list_return_data\|test_bucket_create_delete\|test_object_metadata_replaced_on_put\|test_versioning_bucket_create_suspend\|test_versioning_obj_plain_null_version_removal\|test_versioning_obj_plain_null_version_overwrite_suspended\|test_versioned_concurrent_object_create_and_remove\|test_object_create_bad_md5_bad")
do 
   echo ++++++++++++${i}++++++++++++++++ > ${i}.text
   S3TEST_CONF=./src/test/system_tests/ceph_s3_config.conf ./src/test/system_tests/s3-tests/virtualenv/bin/nosetests -v -s -A 'not_fails_on_rgw' ${i//test_s3./test_s3:} >> ${i}.text 2>&1
   echo "-----------------------------------------------" >> ${i}.text
done

[vh05]

Environment: minikube bucket type: nsfs

Below tests are passing nsfs bucket:

1. S3TEST_CONF=src/test/system_tests/ceph_s3_config.conf ./src/test/system_tests/s3-tests/virtualenv/bin/nosetests s3tests_boto3.functional.test_s3:test_bucket_list_return_data
2. S3TEST_CONF=src/test/system_tests/ceph_s3_config.conf ./src/test/system_tests/s3-tests/virtualenv/bin/nosetests s3tests_boto3.functional.test_s3:test_object_metadata_replaced_on_put

bash-4.4$ S3TEST_CONF=src/test/system_tests/ceph_s3_config.conf ./src/test/system_tests/s3-tests/virtualenv/bin/nosetests s3tests_boto3.functional.test_s3:test_bucket_list_return_data

.
----------------------------------------------------------------------
Ran 1 test in 40.897s

OK
bash-4.4$  S3TEST_CONF=src/test/system_tests/ceph_s3_config.conf ./src/test/system_tests/s3-tests/virtualenv/bin/nosetests s3tests_boto3.functional.test_s3:test_object_metadata_replaced_on_put

.
----------------------------------------------------------------------
Ran 1 test in 15.004s

OK

Earlier the same tests were failing when fsync() call is issued during _create_path() with invalid arg error. I was bit sceptical about the open mode in fsync() path and tried the same call flow in simple snippet but the things were working fine. Later I changed the default minikube driver to --driver = docker and ran the tests and the test got passed.

[vh05]

Environment: nsfs bucket

Out of below tests:

test_bucket_list_return_data
test_bucket_list_return_data_versioning
test_bucket_create_delete
test_object_metadata_replaced_on_put
test_versioning_bucket_create_suspend
test_versioning_obj_plain_null_version_removal
test_versioning_obj_plain_null_version_overwrite_suspended
test_versioned_concurrent_object_create_and_remove

failed test:

test_bucket_list_return_data_versioning