ParquetViewer icon indicating copy to clipboard operation
ParquetViewer copied to clipboard

[FEATURE-REQUEST] Support for Map datatype

Open rr-on-gh opened this issue 7 years ago • 10 comments

Currently there is no support for data structures like Map, Lists etc. and an error message is thrown when such files are opened. Can Map datatype be supported?

rr-on-gh avatar Nov 27 '18 14:11 rr-on-gh

Can you please share the error message shown? Even if the utility can't display these types I believe it can be changed so that it at least shows the other non Map/List fields.

mukunku avatar Nov 28 '18 19:11 mukunku

I'll paste the error message in a bit. So the file can still be loaded by not selecting the field that has Map values when the file is first loaded. I was wondering if support to view Map could be added. I guess it doesn't fit well UI-wise in the current tabular format. but even if we could just build a string out of the Map and show that as column, it would be helpful.

rr-on-gh avatar Nov 29 '18 07:11 rr-on-gh

image

I get this message when I try to open a file containing Map type columns.

ShubhamBelwal avatar Jan 29 '19 09:01 ShubhamBelwal

Any update on the support for Map Types?

ShubhamBelwal avatar Feb 06 '19 04:02 ShubhamBelwal

I am also interested in seeing support for logical types. Actually all possible data types...

pdesmarets avatar Aug 25 '19 16:08 pdesmarets

I would also be interested in support for map types, as well as struct types. Do you think you will support this soon?

mith-emsa avatar Oct 29 '19 14:10 mith-emsa

Can you share a sample file? That would be very helpful.

mukunku avatar Oct 30 '19 01:10 mukunku

There are all kinds of sample files of different nature here https://github.com/apache/parquet-testing/tree/master/data

pdesmarets avatar Oct 30 '19 05:10 pdesmarets

Just an update: In the newer versions it should be possible to open a file with unsupported field types but the unsupported fields cannot be selected for display in the UI still, for now.

mukunku avatar Nov 17 '20 20:11 mukunku

List and Array type supported has been added #37. Hopefully will add MAP some time soon as well.

Still no Map support. Sorry 😞

mukunku avatar Jun 13 '21 16:06 mukunku

Non-nested map and list support is available now with v2.7.0

Going to call that a win and close out this issue.

mukunku avatar Apr 23 '23 01:04 mukunku