bta-lib icon indicating copy to clipboard operation
bta-lib copied to clipboard

Memory Usage when calling e.g. btalib.rsi(df) in a loop

Open moritzgun opened this issue 4 years ago • 5 comments

I am running a MultiplexSocket on Binance API to get 5 minute klines of around 200 symbols. On every new completed candle I call a "run_technical_analysis" function. Inside that function I calculate indicators for each symbol like this:

period = 14 value = btalib.ema(candlesticks_df.tail(period + 1), period=period) ## Version A #value = candlesticks_df.Close.tail(14).ewm(span=14).mean() ## Version B

After that I append value.df.tail(1) to a list. On Version A my RAM goes up almost 10 MB every 5 minutes. It seems like the allocated memory for 'value' is not beeing freed. When I use Version B instead there is no such issue.

What can I do to reduce that memory usage?

moritzgun avatar Jan 28 '21 11:01 moritzgun

Same problem on my end!

reedajohns avatar Mar 08 '21 23:03 reedajohns

Same problem on my end as well. Does anybody have a fix for this? Maybe we can create a fork and publish it on pypi ... seems like the developer is not updating this library anymore. :(

drhighliner avatar Mar 30 '21 22:03 drhighliner

with btalib.stochastic(df) same problem, memory leak very fast.

PapoKarlo avatar May 26 '21 13:05 PapoKarlo

https://stackoverflow.com/questions/13784192/creating-an-empty-pandas-dataframe-then-filling-it

This may solve the issue? Never append a row to a dataframe

sam1946 avatar Oct 08 '21 17:10 sam1946

Every indicator with btalib inside a loop is filling memory, even if we use del to free variables returned. This is a big issue.

Davesmacer avatar Nov 18 '21 16:11 Davesmacer