Ritchie Vink

Not really worth a method IMO: ```python df = pl.DataFrame({ "a": [1, 2, 3, None], "b": [1 ,2, 3, 4] }) df.select([ pl.all().null_count() / pl.count() ]) ``` ``` shape: (1,...

It already is.

> The error still comes from `arrow2`. Can it be the way Polars uses the `arrow2` API? I don't think the fix was already in the polars branch.

> Is there an update on this? I curious on whether something else is required here as this is an important use-cathis. If you can run it in arrow, I...

I can also read the file on latest master: ```python >>> pl.read_parquet("nested_struct_OutOfSpec.snappy.parquet") shape: (2, 1) ┌─────────────────────────────────────┐ │ dim │ │ --- │ │ struct[4] │ ╞═════════════════════════════════════╡ │ {{null,null,null,null,null,null,... │ ├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤...

The issue occurs when appending structs of different chunk sizes. MWE: ```python s = pl.Series([{'_experience': {'aaid': {'id': '7759804769753743647', 'namespace': {'code': '3245164418740504690'}, 'primary': True}, 'mcid': {'id': None, 'namespace': {'code': None}, 'primary':...

#4217 fixes the issue. Note that we still cannot read the file because it contains a `map` datatype, which is not supported by polars.

Polars will not add the map dtype. It's benefit do not outweigh the extra complexity. Maybe we can investigate conversion of maps to struct. But I will have to explore...

With #4226 we can read the entire file. The `map` dtype will be converted to its physical type which is supported by polars.

@andrei-ionescu found another issue, opened it upstream https://github.com/jorgecarleitao/arrow2/issues/1239.