qlib icon indicating copy to clipboard operation
qlib copied to clipboard

Potential performance issue: concat slow in pandas below 2.1 version

Open TendouArisu opened this issue 1 year ago • 0 comments

Issue Description:

Hello. I have discovered a performance degradation in the .concat function of pandas version below 2.1. And I notice some parts of the repository depend on pandas below 2.1 such as examples/benchmarks/TRA/requirements.txt, examples/benchmarks/Sandwich/requirements.txt. I am not sure whether this performance problem in pandas will affect this repository. I found some discussions on pandas GitHub related to this issue, including #50652 and #52685. I also found that examples/benchmarks/TRA/src/model.py, examples/workflow_by_code.ipynb, and examples/benchmarks/TRA/Reports.ipynb used the influenced api. There may be more files using the influenced api.

Suggestion

I would recommend considering an upgrade to a different version of pandas >= 2.1 or exploring other solutions to optimize the performance of .concat. Any other workarounds or solutions would be greatly appreciated. Thank you!

TendouArisu avatar Mar 01 '24 07:03 TendouArisu