s3fs

s3fs copied to clipboard

for dir in cli.scandir("/"):
     print(dir)

<dir 'core'>
<dir 'inbound'>

cli.getinfo('/core')
<dir 'core'>

cli.getinfo('/inbound')
....
Traceback (most recent call last):
  File "/opt/conda/envs/arcql/lib/python3.7/site-packages/fs_s3fs/_s3fs.py", line 173, in s3errors
    yield
  File "/opt/conda/envs/arcql/lib/python3.7/site-packages/fs_s3fs/_s3fs.py", line 346, in _get_object
    obj.load()
  File "/opt/conda/envs/arcql/lib/python3.7/site-packages/boto3/resources/factory.py", line 505, in do_action
    response = action(self, *args, **kwargs)
  File "/opt/conda/envs/arcql/lib/python3.7/site-packages/boto3/resources/action.py", line 83, in __call__
    response = getattr(parent.meta.client, operation_name)(**params)
  File "/opt/conda/envs/arcql/lib/python3.7/site-packages/botocore/client.py", line 357, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/opt/conda/envs/arcql/lib/python3.7/site-packages/botocore/client.py", line 661, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (404) when calling the HeadObject operation: Not Found

i can get to that resource no problem with just boto:

client.get_object(Bucket="my-bucket", Key="inbound/xxx.csv")["Body"]

any hints on how to troubleshoot it would be appreciated :)

I'm guessing that your inbound folder contains files that were uploaded with something other than s3fs, and that core contains at least one file that was uploaded with s3fs. I'm not sure if you directly want to check that /core exists, or if it was just a check that you did for debugging. If you really want /core to exist, then upload an empty file to it with s3fs, then your check should work. More details here