PythonCall.jl icon indicating copy to clipboard operation
PythonCall.jl copied to clipboard

Consider supporting polars

Open PallHaraldsson opened this issue 4 months ago • 1 comments

I see PythonCall has special support for pandas, and I think polars is a replacement for it, so also support it (not either or)?

https://pola.rs/

I hate to be asking for such features, adding to your workload. I see people complaining Julia is much slower than Python (with polars). I'm thinking are there good options that compete, or should users use polars directly, i.e. wrapping the Rust implementation. Or use the Python package/API/binding for it. Its install page gives me the Python download option, not even defaulting to Rust (or JavaScript). I don't know for sure, I think the Python API might be fully featured, or at least more used (as is with some packages like TensorFlow).

Do you think you should not need any support, can just install Python polars package and use it? It's likely the case, and should be used like that, or with a simple Julia package that depends on the Python package, depending on Rust...

I'm just unclear on if that argument didn't also go for pandas, and why was it supported specially in the first place?

PallHaraldsson avatar Aug 18 '25 15:08 PallHaraldsson

PythonCall's special support for pandas is basically:

  • a function to convert a Julia table to a pandas dataframe
  • a wrapper type for a pandas dataframe which satisfies the tables interface.

So it's purely a minimal interoperability layer.

I have no objection to adding polars support too. However I'm working (incredibly slowly) towards PythonCall v1 and there's a question about whether support for specific Python packages is better off in a separate Julia package, such as PythonCall_Pandas, PythonCall_Polars, etc.

cjdoris avatar Aug 18 '25 15:08 cjdoris