json2parquet
json2parquet copied to clipboard
INT 96 timestamps giving problems.
in JSON, my timestamp field looms like
block_timestamp":1333413850
after conversion to pq using pa.timestamp('s', tz='UTC') and use_deprecated_int96_timestamps=True my timestamp starts looking like
block_timestamp = zg+IUgAAAACMPSUA
(FYI, I used parquet-tools to check the contents)
also, I get weird dates when I import it into Athena.
What would be the best way to model this? FYI, I am using 0.028 version