backtest_plot raises error due to Pyfolio package. file: finrl/applications/stock_trading/ensemble_stock_trading.py

[omerfirat]

Hi, it give an error like below in backtest_plot metthod, when i run finrl/applications/stock_trading/ensemble_stock_trading.py file,

I install all environment like setup.py and requirements.txt

i think this error is related with pandas version but , if back to old version get an error for every library Can Aynoone re-run with correct libraries version and fix all libraries.

Traceback (most recent call last): File "C:\Users\Onec\Desktop\ömer\FinRL-master\finrl\applications\stock_trading\ensemble_stock_trading.py", line 195, in main() File "C:\Users\Onec\Desktop\ömer\FinRL-master\finrl\applications\stock_trading\ensemble_stock_trading.py", line 186, in main backtest_plot( File "C:\Python310\lib\site-packages\finrl\plot.py", line 96, in backtest_plot pyfolio.create_full_tear_sheet( File "C:\Python310\lib\site-packages\pyfolio\tears.py", line 201, in create_full_tear_sheet create_returns_tear_sheet( File "C:\Python310\lib\site-packages\pyfolio\plotting.py", line 54, in call_w_context return func(*args, **kwargs) File "C:\Python310\lib\site-packages\pyfolio\tears.py", line 496, in create_returns_tear_sheet plotting.show_perf_stats(returns, benchmark_rets, File "C:\Python310\lib\site-packages\pyfolio\plotting.py", line 648, in show_perf_stats for stat, value in perf_stats[column].iteritems(): File "C:\Python310\lib\site-packages\pandas\core\generic.py", line 5989, in getattr return object.getattribute(self, name) AttributeError: 'Series' object has no attribute 'iteritems'

[notBradPitt]

I'm facing the same issue and I assumed it was just an error on my part, but apparently not.

I think the root of the problem is the Pyfolio library that hasn't been updated for years. The code is a mess, but I got it working

Problem 1

As mentioned here https://github.com/aertslab/pySCENIC/issues/475 and here https://github.com/dusty-nv/jetson-inference/issues/1640, it seems that iteritems() has been deprecated in maybe Pandas 2.0. In line 97 in plotting.py of pyfolio, you can either change iteritems to items or downgrade to older version of Pandas

for name, val in rc_default.items():
    rc.setdefault(name, val)

Problem 2

In my case, another problem arises

AttributeError: 'numpy.int64' object has no attribute 'to_pydatetime'

because in their code, peak is set to something like 2023-03-24 00:00:00+00:00 while valley is set to something like 79. My fix is to add another line valley = underwater.iloc[[valley]].index[0] to the timeseries.py to the get_max_drawdown_underwater function of timeseries.py like so

valley = np.argmin(underwater)  # end of the period
# Find first 0
peak = underwater[:valley][underwater[:valley] == 0].index[-1]
# Find last 0
try:
    recovery = underwater[valley:][underwater[valley:] == 0].index[0]
except IndexError:
    recovery = np.nan  # drawdown not recovered

valley = underwater.iloc[[valley]].index[0]
return peak, valley, recovery

Result

backtest_plot seems to be working again, at least nothing seemed wrong to me and the plots show without an error. I'm not sure how @zhumingpassional or anyone else from FinRL team wants to proceed with this. I don't think I can make a pull request to pyfolio since the library hasn't been updated since like 2019

[AjinkyaBankar]

Thanks! This worked, but I had to modify line# 648 of the plotting.py as for stat, value in perf_stats[column].items():.

Edit: @zhumingpassional and @notBradPitt Apologies for the confusion. I am seeing the following tables instead of plots.

[zhumingpassional]

@AjinkyaBankar could you pls submit a PR?

[notBradPitt]

I'd love to submit a PR to Pyfolio, but the repository hasn't been updated for a while. Should I just make a fork and change the requirements here to reflect the updated code?

[zhumingpassional]

@notBradPitt what do you mean by "change the requirements"? only change the requirements.txt?

[notBradPitt]

@zhumingpassional The fix involves modifying Pyfolio package, which I don't think will be updated or accept PR. So I'm asking if I could just fork the Pyfolio package and change the requirement.txt to use the fork, e.g.

pyfolio git+https://github.com/notbradpitt/pyfolio

instead of

pyfolio

in the requirements.txt

[zhumingpassional]

@notBradPitt

I do not accept the PR since it is caused by the pyfio package.

[zhumingpassional]

I suggest users to revise the pyfio package locally That plot function can be replace by others. I also suggest to use other plot functions in finrl.plot, or write your own functions.

[VishnuRaghavan]

Just install pyfolio-reloaded package from pip.