rateslib
rateslib copied to clipboard
DEP: change `pandas` to `polars` for package size
pandas is a more fully featured library, but 99% of those features are not used by rateslib.
polars is more efficient for the use cases but suffers from portability.
Until polars has the flexibility of being installed in pyodide this can remain onhold.
0.27 Pyodide has added Polars.
@attack68 Activating this issue. Now that there is a Polars build for Pyodide, would you see a prospect to move onwards with this matter? I'd gladly offer my help.
The hardest part of this are two things:
- maintaining the ability to handle pandas
Seriesinputs (does narwhals allow this?) - replacing the code that sequentially allows the construction of a risk dataframe using keys for a pandas.MultiIndex. I am not familiar enough with polars to know if this is easily doable.
-
Narwhals is a pretty unknown framework for me, but from a quick look, I wouldn’t be surprised if it did. Also, building custom adapters should not be that hard of a task.
-
Assuming you are referring to the Solver module, this migration could get pretty hairy. Polars does not natively support multi-indexing (see https://docs.pola.rs/user-guide/migration/pandas) so that option is off the table. Correct me if I'm wrong, but it looks like multi-indices are predominantly used for formatting results in a readable way and not in any performance critical operations, although it enables quite straight forward retrieval as well. I really don’t have a clear proposal for a more efficient approach that leverages Polars in this regard.
That being said, any performance critical sections currently relying on Pandas (if those exist) could serve as a good starting point for experimenting with how Polars might integrate to Rateslib.
Edit: If the sole purpose of the migration is to shrink the package size, it may not justify the effort.