mongo-arrow
mongo-arrow copied to clipboard
Support for nested ObjectIDs in polars conversion
Hi,
_arrow_to_polars currently has no support to cast extension types for nested fields. This prohibits ObjectIDs to be read in case they are in nested fields.
I could not manage the conversion with the original code,
but I found a way to using arrow_table_without_extensions = arrow_table.cast(schema_without_extensions)
to cast the schema of the whole table in one go.
The schema_without_extensions is created recursively from the old schema. Support for lists is still to be added, should not be that hard, maybe I try tomorrow.
I am not an expert in apache arrow. My world is Pandas and Polars. I have wrote some unit tests locally to test the code, but I do not feel confident that I have not overlooked something, so please review carefully.
#219