Wes McKinney

Results 203 comments of Wes McKinney

I'd be interested in having unions in the Parquet format. It would have to annotate a STRUCT having a first field that is an INT32 indicating which of the subsequent...

I think it would be a good idea to not ship the tests in wheels. If you want users to be able to run the tests against their production installs...

FYI: the Arrow C interface is a perfect match for a libpq bridge that doesn't require taking on any C++ library dependencies http://arrow.apache.org/blog/2020/05/03/introducing-arrow-c-data-interface/ [Some other database engines](https://gerrit.cloudera.org/#/c/15661/) are starting to...

Efficiently decoding data to the exact memory representation that pandas requires is currently somewhat complicated. It uses NumPy arrays as its internal representation, but null-handling adds some complexity - float32/float64...

As a possible added benefit for pandas users, it would be nice to have the option for any string columns to be returned as pandas categorical dtype, which has significant...

Yeah, that would definitely work. Could put this NumPy deserializer in an optional extension so that people can still use asyncpg if they don't have NumPy installed (since you will...

> That is something we need to have before we can start the process (we aren't Pandas users ourselves). It sounds like what you need is a test suite, not...

From a data representation point of view, it may also be worth looking at Apache Arrow as an intermediary to use -- then you don't have to deal with all...

We haven't invested too much in the Cython API for the array builders, so using the C++ API (in Cython) would be the way to go (if Cython is the...

hi @0x0L, great news. I'm interested to see if there is a way to create a small "C middleware library" in Apache Arrow that uses the C data interface to...