arrow-julia
arrow-julia copied to clipboard
apache
Official Julia implementation of Apache Arrow
I see #23 closed but only *stream* format got supported. How about files in "arrow file" format, each with multiple record batches inside? With released version I get a single...
It would be very useful to support reading (and potentially writing) partitioned datasets. The implementation could be similar to JuliaIO/Parquet.jl#138/JuliaIO/Parquet.jl#142 and would bring Arrow.jl up to par with [Pyarrow's dataset...
When writing a table with Arrow.jl that contains a nullable column, the Arrow data cannot be read by Pyarrow: ```julia julia> Arrow.write("/tmp/nothing.arrow", (col=Vector{Union{Nothing,Int32}}([1,2,3,nothing]),)) "/tmp/nothing.arrow" julia> Arrow.Table("/tmp/nothing.arrow") Arrow.Table with 4 rows,...
```julia using Dates, Arrow io = Arrow.tobuffer((x=[missing; nothing; Nanosecond(1)], )) ``` ``` ERROR: BoundsError: attempt to access DataType at index [3] Stacktrace: [1] isatypeid(x::Arrow.Duration{Arrow.Flatbuf.TimeUnitModule.NANOSECOND}, #unused#::Type{Nanosecond}, #unused#::Type{Tuple{Union{Missing, Nothing}, Nanosecond}}, i::Int64) (repeats...
It appears that Arrow.jl somehow incorrectly handles columns with nullable types (Union with Nothing) as the schema of the resulting table containing such a column doesn't match what's reported by...
We have a data source (Relations from our database engine at RelationalAI) that have _columnar data_, but without column names. (We represent a Relation as a Set of Tuples, e.g....
In #278, there was some brief discussion around what the new release process looks like now that the repo has been transferred. I'd like to propose the following and we...
We can configure this repository's metadata by `.asf.yaml`. See also: https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features We should change at least notification configurations. All issue changes will be notified to [email protected] with the default configuration.
Not sure if it makes sense, but, would it be possible to allow missing type without copying the underlining arrow vector? As far as I understand, allowing missing only changes...
