How to use polars in an external PyO3 project?

[danielgafni]

Using polars with PyO3

Hi!

I'm currently learning PyO3 and rust. I'm making an external project that would depend on polars. I'm trying to define a #[pymethods] for a #[pyclass] with the following signature:

use polars::prelude::*;

fn fit(&self, df: DataFrame) -> &Self {}

I'm getting this error from PyO3:

#[pymethods]
    | ^^^^^^^^^^^^ the trait `IntoPyCallbackOutput<_>` is not implemented for `polars::prelude::DataFrame`

Should I use something else? Or what's the correct way to do this? As I understand, py-polars isn't currently published.

Thanks a lot!

[ritchie46]

There is an example using the arrow c data interface here: https://github.com/pola-rs/polars/tree/master/examples/python_rust_compiled_function

But now that I think of it. We could actually provide a library that does the conversion for you. I will see I can publish one when I got a bit more time.

[danielgafni]

Thanks!

Seems like the example code is a little broken (or outdated).

My dependencies:

[dependencies]
pyo3 = { version = "0.16.5", features = ["extension-module"] }
polars = "0.22.8"
polars-arrow = "0.22.7"
arrow = "19.0.0"

My IDE is displaying some errors:

I also tried using arrow2. This is causing even more errors.

[kylebarron]

arrow = "19.0.0"

This is your issue. Polars aliases arrow2 to arrow. So every instance of arrow in the polars code base is referring to https://crates.io/crates/arrow2, not https://crates.io/crates/arrow.

If you're curious, we also have an example of using polars in an external pyo3 project here: https://github.com/kylebarron/geopolars/tree/master/py-geopolars

[danielgafni]

Oh, thank you so much! I totally missed the alias.

[alexander-beedie]

Looks like this can be closed? 👍