json2parquet

json2parquet copied to clipboard

ImportError: cannot import name 'load_json' from 'json2parquet' on windows

3

` from json2parquet import load_json, ingest_data, write_parquet, write_parquet_dataset File "directory/file_name.py", line 5, in from json2parquet import load_json, ingest_data, write_parquet, write_parquet_dataset ImportError: cannot import name 'load_json' from 'json2parquet'` I have installed...

mdtaher

Add feature to partition output files by specific column/s in the schema. Initially only by column, no expressions allowed to partition data.

sojovi

Do less copying/passes through the data

3

This will never be perfect in python, but right now its pretty disgusting what we do when we load JSON and convert it to a columnar format.

andrewgross

Check data types match up

1

Check if PyArrow data structures match up with what JSON loads matches up with what Redshift has

andrewgross

Add Helpers for more inputs?

1

Anywhere we can get a valid schema from should work fine

andrewgross

I know its still being worked on with PyArrow, but since it is JSON it probably should be considered.

andrewgross