arrow-julia
arrow-julia copied to clipboard
apache
Official Julia implementation of Apache Arrow
Last time I checked, `mmap` breaks down for files with compression. This is understandable because the compressed buffers clearly can't be re-interpreted without inflation. But the larger the file is...
Hi @quinnj - could you please add to the release plan of Arrow.jl support for https://github.com/JuliaData/DataAPI.jl/pull/48 for the created Arrow tables. Only read methods need to be implemented for Arrow...
If you try to save a loaded table into the same file, it will lead to an invalid argument error. Seems like it's caused by mmap on windows. See JuliaData/CSV.jl#170....
First of all, thank you for the amazing package! I have noticed unexpected behaviour that I wanted to point out. **Expected behaviour:** rational numbers like 1.0 and 0.1 will be...
I have an `.arrow` file generated with `pyarrow` whose schema is the following: ``` input: struct not null child 0, open: fixed_size_list[512] child 0, item: float child 1, high: fixed_size_list[512]...
```jl julia> using CategoricalArrays, DataFrames, Arrow julia> Arrow.write("/tmp/my.arrow",DataFrame(x=CategoricalArray([1,2,3]))) "/tmp/my.arrow" julia> DataFrame(Arrow.Table("/tmp/my.arrow")).x|>eltype Int64 [69666777] Arrow v2.3.0 ```
Setup: ```julia using Arrow struct A0 end struct A1 x::Int end struct A2 x::Int y::Float64 end ArrowTypes.arrowname(::Type{A0}) = :A0 ArrowTypes.JuliaType(::Val{:A0}) = A0 ArrowTypes.arrowname(::Type{A1}) = :A1 ArrowTypes.JuliaType(::Val{:A1}) = A1 ArrowTypes.arrowname(::Type{A2}) =...
DictEncode signals that a column/array should be dictionary encoded when serialized to the arrow streaming/file format. The current constructor will happily wrap a DictEncode in another DictEncode. https://github.com/apache/arrow-julia/blob/v2.2.1/src/arraytypes/dictencoding.jl#L69 Does it...
A group of Arrow developers are working to create a set of examples showing how to send and receive Arrow data over HTTP APIs. There is a mailing list discussion...
On Arrow v2.7.1 (and Julia v1.10.1): ```julia julia> t = (; a = Arrow.DictEncode([:a])) (a = [:a],) julia> Arrow.write("x.stream.arrow", t, file = false) "x.stream.arrow" julia> Arrow.append("x.stream.arrow", t) ERROR: ArgumentError: Table...
