Building on OSX/Windows

[justinclift]

This is just a placeholder/reminder, for figuring out how to get compiling to work on macOS/OSX and Windows.

Some relevant info for reference:

With time and some emotional distance, I'd like to take a stab at cleaning up the Linux build so that it can hook into travis-ci and codecov.

I think once that happened, getting the OS X build to work shouldn't be too difficult. I don't have an OS X machine, though, so I'm not in a great position to test that.

For the OSX part of things, we have an old Mac Mini which we run our automatic nightly builds on. That'd probably work ok for testing the compile side of things out with. :smile:

[0xADD1E]

In an effort to get this working on Fedora/CentOS, I've been looking at using a more platform-agnostic build system such as Meson. It's looking fairly simple to set up, and I think it should take care of Windows/MacOS if you could find Apache Arrow/Parquet devel packages for them. What would your thoughts on that approach be?

[cldellow]

Hi Addie, thanks for your interest!

My thoughts would probably be pretty positive. I've never heard of Meson before, but that probably says more about me than it does about Meson, as I know almost nothing about C++ development. :)

My chief requirements:

• it should be able to automatically build & pass tests in Travis CI's Ubuntu environment
  • this may mean being able to pre-build some of the deps & stash them in S3 to avoid Travis CI timeouts from long-running builds that build the world
• it should build with statically linked libs (if dynamic links are desired, they could be exposed as a flag - the goal is to be able to provide a lib that can just be dropped in w/o requiring admin rights on the box)

Other than that, have at it. Because of my noobishness, I'd have limited ability to help diagnose issues if any crop up, so fair warning that you might be on your own if you hit any roadblocks. :(

[0xADD1E]

In attempting to convert the build system, I've run into an interesting issue, not meson-specific, but more related to the project itself. I'm attempting to use system libraries for ICU, Parquet, and SQLite (speeds up builds, and allows for the possibility of this being integrated by distributions). This however, means version constraints.

In its current state, we rely on SQLite's "Virtual Table Constraint Operator Codes", which were only introduced (in their current state? I haven't investigated that much) in version 3.21.0. If we want to use distribution packages, this would prevent use on Ubuntu older than 18.04 LTS, and CentOS older than 8. Is this an acceptable solution, would we want to refactor that section to work with how VTable plugins used to work, or do we want to continue building SQLite along with the plugin?

[cldellow]

I like the idea of sitting on top of system libs, for the reasons you mention.

Supporting Ubuntu back to 16.04 would be desirable, since it's still in LTS. IIRC, it ships sqlite 3.11.

https://sqlite.org/vtab.html#xbestindex is a useful reference here. Looks like NE, IS, ISNOT, ISNULL, ISNOTNULL would be the operators supported on modern distros but not older ones.

I don't think any of SQLite's structs (eg sqlite3_index_info, sqlite3_index_constraint, sqlite3_index_constraint_usage) grew new fields in those versions, so supporting back to 3.11 is perhaps just a matter of #ifdefing the new operators?

I think on distros with older SQLite, it would mean that SQLite's query planner would never send us the IS/ISNOT/ISNULL/ISNOTNULL entries and so those codepaths would never be exercised, which would be fine. The user's query might be slower, as more filtering would happen inside of SQLite's VM instead of in the virtual table extension, but c'est la vie.