Michael Penkov comments

Results 254 comments of


                                            Michael Penkov

Using botocore_session parameter of s3.iter_bucket fails

@SootyOwl What is the use case? Why are you attempting to pass a botocore session? Are you sure that object is safe to pass across a subprocess boundary? Under the...

Using botocore_session parameter of s3.iter_bucket fails

As discussed in #749, botocore sessions are not pickle-able, so it is impossible to pass them to a child subprocess, which is what iter_bucket does (it starts a bunch of...

slow http read performance on aws lambda

Are you able to profile the code to work out where the time-consuming part is? It seems that *downloading* is slow, because you're using smart_open for the upload in both...

ssh/scp/sftp: enable pipelining of write operations

Closing as stale

BufferedInputBase pre-reserves data to DEFAULT_BUFFER_SIZE

Closing as stale

Broken documentation links on Gensim's PyPI page

The links on PyPI come from README.md, AFAIK https://github.com/RaRe-Technologies/gensim/blob/develop/README.md

Improve QOL for testing & contributing

Thank you!

Ranged read http header leaves TO bytes blank, causing S3 filesystems to read full file length

Sounds reasonable, but let's use shorter values for the options, e.g. `optimize: [reading|streaming|auto]` @jcushman Can you comment on the defer seek part?

Ranged read http header leaves TO bytes blank, causing S3 filesystems to read full file length

After thinking about this some more, I can think of another alternative. We can specify the range to always read at most N bytes, where N is configurable via transport...

Ranged read http header leaves TO bytes blank, causing S3 filesystems to read full file length

Yes, that makes sense. 1. Seeking should _not_ close the body when the seek position is within the buffer 2. I think we can remove it, and provide documentation for...