flatc support for the FlatBuffer submodule

[jonalm]

(the following is a slightly rewritten version of a slack post at the arrow-dev channel)

I am interested in contributing to getting flatc support for julia.

I have a WIP fork of the official flatbuffers repo which provides some flatc support for the flatbuffer implementation in Arrow.jl. I’ve hijacket the go implementation (I haven’t figured out how to add a new cli option to the flatc command yet). The example of the features which are currently working are shown in this gist (i.e. some serialize-deserialize tests). And the code output is shown here . The functionality is basically inspired by the files at /src/metadata, and are not complete wrt flatbuffer features, as I’m a bit uncertain about how to proceed with, e.g. union types, mutation functionality, getting struct fields from structs etc.

Before I continue this work, I would like to know whether this work would be supported by the main devs of Arrow.jl, and also if would be possible to move the FlatBuffer submodule of Arrow.jl into it's own repo?

[quinnj]

Sorry for the slow response @jonalm; I fully support these efforts! Happy to chat about how to move things forward as well. In terms of the plan of where the code lives, I would be in favor of:

• Fork the current FlatBuffers.jl repo, create a branch, and go to town. It probably makes sense to just delete all existing files, then create new files for copy-pasting the code that currently lives in this repo.
• We could then probably do a setup where the FlatBuffers.jl package provides a binary executable Artifact in the form of your forked flatc compiler that could be used by downstream projects/packages; this is a bit fuzzy in my mind, but I think it could be a worthwhile "stepping stone" while things get ironed out and the package code matures for a minute
• Do a proper PR to the official flatbuffers repo with necessary flatc changes along with donation of the FlatBuffers.jl code; I've experienced this with the official arrow repo and you may have seen that we're now splitting the julia code back out into its own repo; this is due to the non-julia-idiomatic requirements of the official repo release process (new major versions every 6 months, difficult to do minor/patch releases, etc.). Happy to help navigate what makes the most sense as we discover official flatbuffer repo requirements.

[jonalm]

Thanks for the encouragement @quinnj. I made a new FlatBuffer.jl issue here, with links to updated code.