Add unit and/or integration tests

Open corneliusroemer opened this issue 3 years ago • 3 comments

Some basic CI testing would be great to prevent regressions.

Mar 04 '23 20:03 corneliusroemer

Agreed. It probably doesn't make sense to test against exact binary matches but we could run the tools via the cli and check that the generated arrow or parquet parses correctly.

Mar 04 '23 22:03 domoritz

Yeah, arrow/parquet binaries don't need to be identical but we can round trip to a representation that should be invariant via a parquet/arrow -> csv converter, then check the csv is how it should be.

Mar 06 '23 11:03 corneliusroemer

Or we check in a test whether the arrow/parquet has the right values and types without going back to csv in the test.

Mar 06 '23 13:03 domoritz