scio
scio copied to clipboard
The suffix parameter doesn't seem to work when reading using parquetAvroFile
I tried to use suffix to read only parquet files using parquetAvroFile
method. I tried with both suffix = ".parquet"
and suffix = "parquet"
, none seemed to filter out a json file that was in the same directory as my parquet files and caused the job to fail.