Ritchie Vink comments

Results 1006 comments of


                                            Ritchie Vink

Generated Parquet files are extremely fragmented

I think I will try to hit a row count rather than a row-group size (defaulting to 512^2). Currently there was an issue in Polars that allowed very small splits...

The rest of the queries?

That would be great!

Why is the Spark memory set to 2gb?

@stinodego any clue?

chore: Upgrade ndarray to 0.16

Are you planning to finish this? Otherwise I will close and it will be included in our monthly updates.

fix(python): Handle boolean comparisons in Iceberg predicate pushdown

@dimfeld can you rebase and make CI green?

fix(python): Handle boolean comparisons in Iceberg predicate pushdown

Wow, totally missed this one. Merging! :see_no_evil:

fix: incorrect negative offset in multi-byte string slicing

Any news on this on?

feat(python): supports writing compressed csv.gz files

Iff, we add that functionality, then it should be on the rust side. We shouldn't implement things only for python. Things I am concerned about and want to be improved...

feat(python): supports writing compressed csv.gz files

Well, I am also not a fan of automatically compressing based on the file name. It should be an opt-in keyword argument, (or setting in Rust). So that solves that.

feat(python): supports writing compressed csv.gz files

> It's your choice. To me, pandas approach is intuitive and works decent. I like to have it explicit and it should also work for writing to buffers.