zui
zui copied to clipboard
Add Parquet as an Export format
At the time of the opening of this issue, Brim is at commit 9d0a742.
Since the backend already supports it, like we did for ZSON with #1486, it should be trivial to add Parquet as an Export format. However, as I was about to impulsively do this, I remembered that the backend still has this limitation:
$ zq -version
Version: v1.2.0-130-g8dd27d98
$ zq -f parquet 'fuse' ./src/test/shared/data/sample.tsv
union type unsupported
Since union types often show up in Zed formats (particularly as a side effect of things like applying fuse
) my inclination is to wait on an improvement in the backend before we add it as an Export option.
For now, I've got a branch https://github.com/brimdata/brim/tree/parquet-export where I've got it working for the happy path. Separately I opened up https://github.com/brimdata/zed/issues/4215 to track what I'm observing in the API interactions that are currently preventing the app from reporting the failure scenarios to the user.