Feather.jl
Feather.jl copied to clipboard
JuliaData
Read and write feather files in pure Julia
I was reading the CSV.jl [documentation](https://juliadata.github.io/CSV.jl/latest/#High-level-interface-1) today and I noticed that they give a possible explanation for why `mmap=false` by default on Windows. It actually seems like a surprisingly reasonable...
Here are some results on a 366MB dataframe with mixed types... ``` INFO: Opening feather with python... 3.100129 seconds (65.20 k allocations: 2.547 MB) INFO: Opening feather with julia... 5.888453...
I've been reading up on what happened over on the Python side to deal with arrays with length that excedes `typemax(Int32)`. This is a problem because Arrow specifies that all...
This issue is to discuss how we deal with categorical references that are not `Int32`. This is a violation of the Arrow [standard](https://github.com/apache/arrow/blob/master/format/Layout.md) (this is not the only suspicious thing...
The error occurs [here](https://github.com/JuliaData/Feather.jl/blob/master/src/Feather.jl#L361) on dataframes with sufficiently large columns. The most obvious way to fix this would be to change all of the offsets to Int64, but does the...
I'm getting a few errors when doing `Data.stream!(src, snk)` where `snk = Feather.Sink("filename.feather", Data.schema(src))` and then calling `Data.close!`. There does not seem to be a simple fix. It is really...
The URL of this package does not match that stored in METADATA.jl. cc: @quinnj
I was reading over the source and have 2 questions: 1. Does Feather.jl convert any Data.Source into a DataFrame before writing out to disk? 2. If the input is a...
