bids-validator icon indicating copy to clipboard operation
bids-validator copied to clipboard

Enh/read parquet files

Open rwblair opened this issue 6 months ago • 1 comments

Use hyparquet library to read read parquet files.Transforms them into a columns map, which may not be desirable. Parquet files have good typing metadata that is thrown away and this implementation, with everything becoming a string like when we read in TSV files. Type checking is then done via regex as per schema.objects.columns/schema.objects.formats entries.

Still need to implement maxRows.

rwblair avatar Apr 14 '25 15:04 rwblair