ArcticDB icon indicating copy to clipboard operation
ArcticDB copied to clipboard

Use https for all azurite backed tests by default

Open phoebusm opened this issue 9 months ago • 2 comments

This is for add coverage for our tests as most of users use https connection but we test http connection mostly on our pipeline. Same set of Azurite tests fail on conda linux testing pipeline after https being turned on. https://github.com/man-group/ArcticDB/actions/runs/8872362650/job/24356595901

FAILED tests/integration/arcticdb/test_arctic.py::test_reload_symbol_list[azurite_storage] - arcticdb_ext.exceptions.StorageException: E_UNEXPECTED_AZURE_ERROR Unexpected Error: AzureError#0 :  Error while reading from network socket. CURLE code: 56. Failure when receiving data from the peer
FAILED tests/stress/arcticdb/version_store/test_stress_append.py::test_write_parallel_stress[azure_store_factory] - arcticdb_ext.exceptions.StorageException: E_UNEXPECTED_AZURE_ERROR Unexpected Error: AzureError#0 :  Error while reading from network socket. CURLE code: 56. Failure when receiving data from the peer
ERROR tests/integration/arcticdb/test_arctic_batch.py::test_read_batch_mixed_request_supported[azurite-EncodingVersion.V1] - arcticdb_ext.exceptions.StorageException: E_UNEXPECTED_AZURE_ERROR Unexpected Error: AzureError#0 :  Error while reading from network socket. CURLE code: 56. Failure when receiving data from the peer
ERROR tests/integration/arcticdb/test_arctic_batch.py::test_read_description_batch_empty_nat[azurite-EncodingVersion.V1] - arcticdb_ext.exceptions.StorageException: E_UNEXPECTED_AZURE_ERROR Unexpected Error: AzureError#0 :  Error while reading from network socket. CURLE code: 56. Failure when receiving data from the peer
ERROR tests/integration/arcticdb/test_arctic_batch.py::test_write_batch_dedup[azurite-EncodingVersion.V1] - arcticdb_ext.exceptions.StorageException: E_UNEXPECTED_AZURE_ERROR Unexpected Error: AzureError#0 :  Error while reading from network socket. CURLE code: 56. Failure when receiving data from the peer
ERROR tests/integration/arcticdb/test_arctic_batch.py::test_write_batch_missing_keys_dedup[azurite-EncodingVersion.V1] - arcticdb_ext.exceptions.StorageException: E_UNEXPECTED_AZURE_ERROR Unexpected Error: AzureError#0 :  Error while reading from network socket. CURLE code: 56. Failure when receiving data from the peer
ERROR tests/integration/arcticdb/test_arctic_batch.py::test_append_batch[azurite-EncodingVersion.V1] - arcticdb_ext.exceptions.StorageException: E_UNEXPECTED_AZURE_ERROR Unexpected Error: AzureError#0 :  Error while reading from network socket. CURLE code: 56. Failure when receiving data from the peer
ERROR tests/integration/arcticdb/test_arctic_batch.py::test_append_batch_missing_keys[azurite-EncodingVersion.V1] - arcticdb_ext.exceptions.StorageException: E_UNEXPECTED_AZURE_ERROR Unexpected Error: AzureError#0 :  Error while reading from network socket. CURLE code: 56. Failure when receiving data from the peer
ERROR tests/integration/arcticdb/test_arctic_library_management.py::test_library_creation_deletion[azurite-EncodingVersion.V1] - arcticdb_ext.exceptions.StorageException: E_UNEXPECTED_AZURE_ERROR Unexpected Error: AzureError#0 :  Error while reading from network socket. CURLE code: 56. Failure when receiving data from the peer
= 2 failed, 6278 passed, 1050 skipped, 7 xfailed, 56 xpassed, 65417 warnings, 7 errors in 1427.28s (0:23:47) =

Have tried switching to different libcurl dependency but doesn't help. Lowering no. of worker to 1 doesn't help. It is highly suspicous to the azurite internal design/storage problem. Currently only basic https connection verification tests are added for Azurite

phoebusm avatar Apr 30 '24 00:04 phoebusm

@phoebusm what exactly is this ticket proposing we do?

poodlewars avatar May 01 '24 17:05 poodlewars

@poodlewars The details are outdated. Let me update

phoebusm avatar May 02 '24 08:05 phoebusm