Nixtla
[Core] Add support for polars
Description
Our codebase primarily relies on pandas for data handling and manipulation tasks. However, we have identified potential performance improvements that could be gained by incorporating support for Polars, a fast DataFrame library implemented in Rust and available in Python.
Polars is designed to outperform pandas in various scenarios and could provide significant speed-ups for our data processing tasks. This can benefit larger datasets and more complex operations, making our toolset more versatile and efficient.
The task would involve reviewing the codebase and integrating the possibility of using Polars as input instead of Pandas. We must ensure the transition is seamless and keeps existing functionalities intact.
This is a substantial task that might require time and careful testing. Any contributors willing to help with this task are welcome. Please feel free to comment below if you'd like to assist or have suggestions on approaching this task.
Use case
No response
This would be useful as currently if you want to calculate dynamic ts features (i.e if the spectral entropy changes over a certain chunk of time) this is painfully slow as I need to fallback to pandas and for loops with multiprocessing.
