[BUG] "don't know how to skip type Set" error

[cetindogu]

i cant view a parquet file. line below throws error -> "don't know how to skip type Set" var parquetReader = await ParquetReader.CreateAsync(parquetFilePath, null, cancellationToken);

The error message "don't know how to skip type Set" in Parquet typically indicates that the Parquet reader or writer you're using does not support the Set data type directly. Parquet, which is a columnar storage format, has a limited set of primitive data types and some complex types (like arrays, maps, and structs), but it doesn't directly support Java specific types like Set. !!!!

[cetindogu]

there are missing compactTypes

namespace Parquet.Meta.Proto { enum CompactType : byte { Stop = 0x00, BooleanTrue = 0x01, BooleanFalse = 0x02, Byte = 0x03, I16 = 0x04, I32 = 0x05, I64 = 0x06, Double = 0x07, Binary = 0x08, List = 0x09, Set = 0x0A, Map = 0x0B, Struct = 0x0C, Uuid = 0x0D } }

1: STOP 2: TRUE 3: FALSE 4: BYTE 5: I16 (16-bit integer) 6: I32 (32-bit integer) 7: I64 (64-bit integer) 8: DOUBLE (64-bit floating point) 9: BINARY (String or binary data) 10: LIST (List collection) 11: SET (Set collection) 12: MAP (Map collection) 13: STRUCT (Structured object) 14: VOID 15: FIXED_LENGTH_BINARY

[miaocb]

The same problem, expect to be resolved.

[mukunku]

Any chance y'all could share a sample file? You can upload it directly here by zipping it.

I also see you opened an issue in the Parquet.net repo as well: https://github.com/aloneguid/parquet-dotnet/issues/544 I'll see if I can recreate the test file you mentioned but no promises.

[cetindogu]

no, i can't. it is a business file.

[mukunku]

This is an issue with the parquet-dotnet library we use. I opened an issue ticket here: https://github.com/aloneguid/parquet-dotnet/issues/583

Going to close this out as a duplicate of #121 . Since there's no sample file in this thread, we can continue discussions there.