write issue with path style s3-compatible storage ECS

[jsanko9]

S3-compatible storage: DELL EMC ECS S3FS == 2022.3.0 Error message: OSError: [Errno 22] The XML you provided was not well-formed or did not validate against our published schema.

bucket ls and open "rb" are working. only open "wb" is having issue. Which strangely enough is correctly working with old S3FS == 0.4.2, but not anymore with 0.5+

ECS uses path style addresses typically with bucket as a part of path: {bucket}.{server_name}.{domain}, but if I try to put bucket into endpoint URL I get different error: "The specified bucket does not exist"

s3fs-fuse seems to be working OK

I have tried tweaking with:

config_kwargs={ 'signature_version': 's3v4', 's3': {'addressing_style': 'path'}

but made no difference.

I'm sorry but my python knowledge is lacking, but I will try to answer any question.

[martindurant]

Sounds like a rate-limiting issue, see #484

[Adam-D-Lewis]

I'm curious if you found a solution @jsanko9 since I'm hitting something similar trying to upload to a Dell S3 Compatible storage appliance.

[Adam-D-Lewis]

In our case, the solution was to add s3_additional_kwargs={"ACL": "private"}

import s3fs

other_s3fs_kwargs = {<your kwargs here>}

fs = s3fs.S3FileSystem(**other_s3fs_kwargs, s3_additional_kwargs={"ACL": "private"})