s3fs

s3fs copied to clipboard

I recently downloaded a few hundred files using copy_file. Most of the time it works as expected. However sometimes when the source file does not exist it creates the destination...

simonm3

when using readbytes(path), get ResourceNotFound

4

``` with open_fs('s3://' + self.bucket + '/') as s3fs: print(s3fs.exists(path)) ## true print(s3fs.isfile(path)) ## false return s3fs.readbytes(path) ## ResourceNotFound ``` where the `path` is the full path (minus the bucket)...

jordaniac89

Server-Side Bucket to Bucket

3

Hi, Curious to know if copying between buckets is a server-side affair, or a download-upload process. There are 3 bucket-to-bucket scenarios: 1) Same account, same region 2) Same account, across...

geoffjukes

I suspect s3fs doesn't check integirty during upload/download. This is sometimes causing problems. The normal `aws` CLI does do this automatically, and we can pass an md5hash during the upload....

CMCDragonkai

S3 streaming

10

It allows for streaming directly from / to S3 without creating temporary files. Additionally files opened for reading are seekable (for example it allows for reading contents of zip files...

mrk-its

Be less strict in non-strict mode

7

**Motivation:** S3FS backend do not work well if bucket contains file structure without proper directory markers, even in non-strict mode. This patch skips few directory checks for `strict=False` mode (for...

mrk-its

I can do the following just fine: ```python with file_system.open("/testing/blob", "rb") as f: byte_string = f.read() blob = pickle.loads(byte_string) ``` But if I do ```python with file_system.open("/testing/blob", "rb") as f:...

matschreiner

Space Separated Resource Not Found in scandir(), works good with listdir()

1

Space Separated Resource not found, when accessed through `fs.scandir(resource)`, but working fine with `fs.listdir(resource)`. > Eg: `fs.listdir('/Real Estate/' )` is working but `fs.scandir('/Real Estate/')` is not working. HELP!

charanjit-singh

The current implementation of `fs_s3fs.S3FS.setinfo()` just a call to `self.getinfo()` (presumably to validate the path name). However, if the call to `setinfo()` was due to trying to set the modification...

jwnimmer-tri

Integration tests write to bucket identified by name hard-coded in tests. This patch makes bucket name configurable via environment variable (with fallback to currently used name) cc: @willmcgugan

mrk-its