s3fs
s3fs copied to clipboard
PyFilesystem
Amazon S3 filesystem for PyFilesystem2
```python import fs import fs_s3fs from fs import open_fs from fs_s3fs import S3FS def copy_to_latest(): print(f"{fs.__version__=}") print(f"{fs_s3fs.__version__=}") target_fs: S3FS = open_fs(f's3://my-bucket/some/path/') print(f"{target_fs=}") src_path = "2022-06-27/08:05:10/" print(f"{src_path=}") dst_path = "latest" print(f"{dst_path=}")...
Today a new version of aiobotocore was released (1.4.0) The new release breaks s3fs ``` File "/usr/local/lib/python3.7/site-packages/s3fs/core.py", line 378, in set_session self.session = aiobotocore.AioSession(**self.kwargs) AttributeError: module 'aiobotocore' has no attribute...
Been having a strange issue with `fs.open` on various files. The code is basically ``` if not fs.exists(path_in_fs): return None try: with fs.open(path_in_fs, 'rb') as f:
https://github.com/cha0sCat/fs-s3fs-minio This is s3fs implemented using minio sdk. implement the makedir function by creating .pyfs.isdir under the folder. all test passed.
Hi, It looks like the buffer argument that's passed in isn't being passed into the underlying `readinto()` call. This causes a `TypeError` when Python tries to invoke the function: https://github.com/PyFilesystem/s3fs/blob/7f95af69e904e58b068ac4f6b39c2da488d5d918/fs_s3fs/_s3fs.py#L154...
Small PR that will read the credentials from AWS config files (.aws/credentials) given the `profile_name`. Alleviates a bit the issue of having client key/secret in the code.
``` for dir in cli.scandir("/"): print(dir) ``` ``` cli.getinfo('/core') ``` ``` cli.getinfo('/inbound') .... Traceback (most recent call last): File "/opt/conda/envs/arcql/lib/python3.7/site-packages/fs_s3fs/_s3fs.py", line 173, in s3errors yield File "/opt/conda/envs/arcql/lib/python3.7/site-packages/fs_s3fs/_s3fs.py", line 346, in...
fixes #70 By removing check to `client.head_object` in `getinfo` we were able to recover expected behavior where - open works when file exists - open fails with `fs.errors.ResourceNotFound` if file...
Hi this is more a proposal of increment than a issue. I am working on a project where I have to access different buckets from the same code. In a...
