vaex icon indicating copy to clipboard operation
vaex copied to clipboard

docs: Add to supported data types table

Open NickCrews opened this issue 1 year ago • 2 comments

I was getting confused as to how my pyarrow timestamps were getting converted, and I think seeing this here would have pointed me in the right direction sooner.

NickCrews avatar Aug 30 '22 00:08 NickCrews

About the timestamp array in arrow.. so in vaex i believe it is stored as arrow, it is not really converted to numpy. However, since many of the datetime and timedelta operations supported come via pandas, it is true that on the fly the data is cast to numpy (well pandas series).

But the data is still stored as arrow.

I don't know how to state this exactly in the short text for the purpose of that table so what you've written probably is fine.. @maartenbreddels ?

JovanVeljanoski avatar Sep 13 '22 22:09 JovanVeljanoski

Indeed, it's being converted on the fly when doing many operations. How did it confused you @NickCrews ?

maartenbreddels avatar Sep 21 '22 10:09 maartenbreddels

I don't remember exactly now 😭, should have been more detailed.

But I think it was constructing a DF from a pyarrow timestamp array, doing some operations on it, and then re-exporting with .values, and I was surprised that it had turned into a numpy array.

NickCrews avatar Oct 03 '22 17:10 NickCrews