mongo-arrow icon indicating copy to clipboard operation
mongo-arrow copied to clipboard

objectID in nested field raises when using aggregate_polars_all

Open sibbiii opened this issue 1 year ago • 1 comments
trafficstars

Hi,

I really appreciate the support for polars, but

collection.insert_one({'obj': {'data_to_test': bson.ObjectId()}})
pymongoarrow.api.aggregate_polars_all(collection, [], 
                                      schema=pymongoarrow.api.Schema({'obj': {'data_to_test': bson.ObjectId}}))

raises with polars.exceptions.ComputeError: cannot create series from Extension("pymongoarrow.objectid", FixedSizeBinary(12), Some("")) as the casting of FixedSizeBinary is not done for nested fields.

Ps.: For not nested fields it works fine. Loading the arrow data frame also works fine. So this is not a showstopper, but it prohibits from using the aggregate_polars_all convenience function.

See #220 for a fix

sibbiii avatar Jun 17 '24 20:06 sibbiii

Should this be closed now that #220 is merged?

ShaneHarvey avatar Aug 29 '24 21:08 ShaneHarvey

@ShaneHarvey yes

lazargugleta avatar Sep 30 '24 14:09 lazargugleta