dione
dione copied to clipboard
Reproducing strange bug
Summary
playing around with other things I happened to reproduce #67
Detailed Description
How was it tested?
added unit test
@eyala / @shay1bz - if you want to look at it.. we get this error org.apache.avro.AvroRuntimeException: java.io.IOException: Block read partially, the data may be corrupt
when I run the test TestAvroPartialBlock
with N=100000. with N=10000 it doesn't happen..
I am limited to my mobile phone for the next few days :D Will try to debug it when I'm back 👍🏼
I'm going to take all of this PR other than the commit almost support non-partitioned tables and release a 0.5.x version with it - that's where we did some full scale testing on it. Afterwards we can determine whether to merge all of it to main