catalyst
catalyst copied to clipboard
scrtlabs
Delisted coins cause crash if have open positions
Hi there,
Currently running latest 0.5.21 version of catalyst and libraries that come with that install (via pip).
- System Version: macOS 10.13.6
- Kernel Version: Darwin 17.7.0
- Python 3.6.6 Packages: Package Version
aiodns 1.1.1 aiohttp 3.4.4 alabaster 0.7.12 alembic 0.9.7 appnope 0.1.0 asn1crypto 0.24.0 async-timeout 3.0.1 attrdict 2.0.0 attrs 18.2.0 autobahn 18.11.1 Automat 0.7.0 Babel 2.6.0 backcall 0.1.0 bcolz 1.2.1 boto3 1.5.27 botocore 1.8.50 Bottleneck 1.2.1 cchardet 2.1.1 ccxt 1.17.94 certifi 2018.1.18 cffi 1.11.5 chardet 3.0.4 click 6.7 ConfigSpace 0.4.7 constantly 15.1.0 contextlib2 0.5.5 cryptography 2.4.1 cycler 0.10.0 cyordereddict 1.0.0 Cython 0.27.3 cytoolz 0.9.0.1 dateparser 0.7.0 decorator 4.3.0 docutils 0.14 empyrical 0.2.1 enigma-catalyst 0.5.21 eth-abi 1.2.2 eth-account 0.2.3 eth-hash 0.2.0 eth-keyfile 0.5.1 eth-keys 0.2.0b3 eth-rlp 0.1.2 eth-typing 1.3.0 eth-utils 1.2.2 hexbytes 0.1.0 hyperlink 18.0.0 idna 2.7 idna-ssl 1.1.0 imagesize 1.1.0 incremental 17.5.0 intervaltree 2.1.0 ipykernel 5.1.0 ipython 7.1.1 ipython-genutils 0.2.0 jedi 0.13.1 Jinja2 2.10 jmespath 0.9.3 joblib 0.13.0 jupyter-client 5.2.3 jupyter-core 4.4.0 kiwisolver 1.0.1 Logbook 0.12.5 lru-dict 1.1.6 lxml 4.2.5 Mako 1.0.7 MarkupSafe 1.0 matplotlib 3.0.2 multidict 4.4.2 multipledispatch 0.4.9 networkx 2.1 numexpr 2.6.4 numpy 1.14.0 packaging 18.0 pandas 0.19.2 pandas-datareader 0.6.0 parsimonious 0.8.1 parso 0.3.1 patsy 0.5.0 pexpect 4.6.0 pickleshare 0.7.5 pip 10.0.1 prompt-toolkit 2.0.7 psutil 5.4.8 ptyprocess 0.6.0 pyasn1 0.4.4 pyasn1-modules 0.2.2 pycares 2.3.0 pycparser 2.19 pycryptodome 3.7.0 Pygments 2.2.0 PyHamcrest 1.9.0 pynisher 0.5.0 pyOpenSSL 18.0.0 pyparsing 2.3.0 pyrfr 0.8.0 python-binance 0.7.0 python-dateutil 2.7.3 python-editor 1.0.3 pytz 2016.4 pyzmq 17.1.2 redo 2.0.1 regex 2018.11.7 requests 2.20.1 requests-file 1.4.3 requests-ftp 0.3.1 requests-toolbelt 0.8.0 rlp 1.0.3 s3transfer 0.1.13 scikit-learn 0.20.0 scipy 1.0.0 service-identity 17.0.0 setuptools 38.5.1 six 1.11.0 smac 0.8.0 snowballstemmer 1.2.1 sortedcontainers 1.5.9 Sphinx 1.8.2 sphinx-rtd-theme 0.4.2 sphinxcontrib-websupport 1.1.0 SQLAlchemy 1.2.2 statsmodels 0.8.0 TA-Lib 0.4.17 tables 3.4.2 toolz 0.9.0 tornado 5.1.1 traitlets 4.3.2 Twisted 18.9.0 txaio 18.8.1 typing 3.6.6 tzlocal 1.5.1 urllib3 1.24.1 wcwidth 0.1.7 web3 4.4.1 websockets 5.0.1 wrapt 1.10.11 yarl 1.1.0 zope.interface 4.6.0
Description of Issue
Currently running a simple momentum backtest (6 day lookback, 3 day refresh interval), on a universe of assets on Binance. Am testing with all available quote currencies (ETH, BTC, BNB, USDT), running each as a separate instance / separate namespace (I am NOT combining all quote pairs (e.g. ada_eth, ada_btc etc) into a single universe).
Everything works fine, but found that holding any assets over the period during which they are delisted from the exchange causes the Portofolio Value to become 'nan'. (E.g. buy an asset on say 2018-10-19 (like icn_, chat_, trig), it becomes delisted while you're holding, then you can't sell it and PV breaks).
I'm guessing it's best that these assets which become delisted are included (and back tested with) to avoid survivorship bias, but how is it best to actually deal with them in the context of a trading / backtesting algorithm? Having PV going to 'nan' (which it seems is further related to 'returns' going to nan as well) breaks all the back testing metrics.
Here's the full error message I get:
RuntimeWarning Traceback (most recent call last)
~/.pyenv/versions/pythonbot3.1/lib/python3.6/site-packages/catalyst/utils/run_algo.py in run_algorithm(initialize, capital_base, start, end, handle_data, before_trading_start, analyze, data_frequency, data, bundle, bundle_timestamp, default_extension, extensions, strict_extensions, environ, live, remote, mail, exchange_name, quote_currency, algo_namespace, live_graph, analyze_live, simulate_orders, auth_aliases, stats_output, output) 641 simulate_orders=simulate_orders, 642 auth_aliases=auth_aliases, --> 643 stats_output=stats_output 644 )
~/.pyenv/versions/pythonbot3.1/lib/python3.6/site-packages/catalyst/utils/run_algo.py in _run(handle_data, initialize, before_trading_start, analyze, algofile, algotext, defines, data_frequency, capital_base, data, bundle, bundle_timestamp, start, end, output, print_algo, local_namespace, environ, live, exchange, algo_namespace, quote_currency, live_graph, analyze_live, simulate_orders, auth_aliases, stats_output) 359 ).run( 360 data, --> 361 overwrite_sim_params=False, 362 ) 363
~/.pyenv/versions/pythonbot3.1/lib/python3.6/site-packages/catalyst/exchange/exchange_algorithm.py in run(self, data, overwrite_sim_params) 401 def run(self, data=None, overwrite_sim_params=True): 402 perf = super(ExchangeTradingAlgorithmBacktest, self).run( --> 403 data, overwrite_sim_params 404 ) 405 # Rebuilding the stats to support minute data
~/.pyenv/versions/pythonbot3.1/lib/python3.6/site-packages/catalyst/exchange/exchange_algorithm.py in run(self, data, overwrite_sim_params) 358 data.attempts = self.attempts 359 return super(ExchangeTradingAlgorithmBase, self).run( --> 360 data, overwrite_sim_params 361 ) 362
~/.pyenv/versions/pythonbot3.1/lib/python3.6/site-packages/catalyst/algorithm.py in run(self, data, overwrite_sim_params) 723 try: 724 perfs = [] --> 725 for perf in self.get_generator(): 726 perfs.append(perf) 727
~/.pyenv/versions/pythonbot3.1/lib/python3.6/site-packages/catalyst/gens/tradesimulation.py in transform(self) 245 yield minute_msg 246 --> 247 risk_message = algo.perf_tracker.handle_simulation_end() 248 yield risk_message 249
~/.pyenv/versions/pythonbot3.1/lib/python3.6/site-packages/catalyst/finance/performance/tracker.py in handle_simulation_end(self) 468 algorithm_leverages=acl, 469 trading_calendar=self.trading_calendar, --> 470 treasury_curves=self.treasury_curves, 471 ) 472
~/.pyenv/versions/pythonbot3.1/lib/python3.6/site-packages/catalyst/finance/risk/report.py in init(self, algorithm_returns, sim_params, trading_calendar, treasury_curves, benchmark_returns, algorithm_leverages) 96 97 self.month_periods = self.periods_in_range( ---> 98 1, start_session, end_session 99 ) 100 self.three_month_periods = self.periods_in_range(
~/.pyenv/versions/pythonbot3.1/lib/python3.6/site-packages/catalyst/finance/risk/report.py in periods_in_range(self, months_per, start_session, end_session) 155 trading_calendar=self.trading_calendar, 156 treasury_curves=self.treasury_curves, --> 157 algorithm_leverages=self.algorithm_leverages, 158 ) 159
~/.pyenv/versions/pythonbot3.1/lib/python3.6/site-packages/catalyst/finance/risk/period.py in init(self, start_session, end_session, returns, trading_calendar, treasury_curves, benchmark_returns, algorithm_leverages) 79 self.algorithm_leverages = algorithm_leverages 80 ---> 81 self.calculate_metrics() 82 83 def calculate_metrics(self):
~/.pyenv/versions/pythonbot3.1/lib/python3.6/site-packages/catalyst/finance/risk/period.py in calculate_metrics(self) 136 self.sharpe = 0.0 137 self.downside_risk = downside_risk( --> 138 self.algorithm_returns.values 139 ) 140
~/.pyenv/versions/pythonbot3.1/lib/python3.6/site-packages/empyrical/stats.py in downside_risk(returns, required_return, period, annualization) 659 660 downside_diff = _adjust_returns(returns, required_return).copy() --> 661 mask = downside_diff > 0 662 downside_diff[mask] = 0.0 663 squares = np.square(downside_diff)
RuntimeWarning: invalid value encountered in greater
---//---
Here's the code I'm using for initialize and handle_data below:
def initialize(context, **kwargs):
context.CANDLE_SIZE = '1T' # T = 'minutely'. Therefore 1T = 1 minute. 15T = 15 minutes.
#context.LOOK_BACK = int(60 * 24 * global_day) # num days. Used to determine look back period as multiple of base CANDLE_SIZE
context.LOOK_BACK = int(global_day) # num days. Used to determine look back period as multiple of base CANDLE_SIZE
context.REFRESH_INTERVAL = context.LOOK_BACK/2 # How often to check for changes in momentum - 1440 = 1 day.60 = 1 hour
context.BUY_MOMENTUM_THRESH = 0 # Fractional Percentage. 1%
context.SELL_MOMENTUM_THRESH = 0
context.active_pairs = 20
context.ORDER_SIZE_METHOD = 'dynamic' # else 'fixed'
context.old_set = set()
context.new_set = None
context.PAIRS = get_universe(global_quote)
context.TOTAL_PAIRS = len(context.PAIRS)
context.counter = 0 # Minute / day counter
context.set_commission(maker=0.001, taker=0.002)
context.set_slippage(slippage=0.001)
def handle_data(context, data):
# Increment minute count to determine when to check momentum
context.counter += 1
# Every refresih interval (60m, 120m, etc), do checks
if context.counter >= context.REFRESH_INTERVAL:
# Reset minute count
context.counter = 0
momentum_df = pd.DataFrame({'pair': [], 'momentum': [], 'position': []})
# Get momentum for pairs
for pair in context.PAIRS:
# Get price data from _x_ lookback ago for each pair in universe
current_asset = symbol(pair)
# Try to get pair. If not active (no historic or current prices),
# or delisted (historic but no current price), will pass as exception, otherwise continue
try:
# Get current price data for each pair in universe
current = data.current(current_asset, fields=['close'])
# Get historic data for pair in universe
historic_close = data.history(current_asset, fields=['close'], bar_count=context.LOOK_BACK, frequency='1T')
# If it's been deslited (either current or historic price returns as none)
#if (current['close'] is not None) or (historic_close.head(1).close.item() is not None):
# Check price momentum (basic positive / not positive) and assign to DataFrame
momentum_df = momentum_df.append({'pair': pair,
'momentum': (current['close'] - historic_close.head(1).close.item()) /
historic_close.head(1).close.item(),
'position': context.portfolio.positions[current_asset].amount},
ignore_index=True)
except Exception as error:
pass
# Get all pairs with positive momentum
top_list = momentum_df.loc[(momentum_df.momentum > context.BUY_MOMENTUM_THRESH)].copy()
# Order the list, descending from highest to lowest momentum
top_list.sort_values(by='momentum', axis=0, inplace=True, ascending=False)
if context.ORDER_SIZE_METHOD == 'dynamic':
# Buy all pairs with positive momentum
buy_list = top_list.copy()
context.active_pairs = len(buy_list)
elif context.ORDER_SIZE_METHOD == 'fixed':
# Buy only pairs with positive momentum to a maximum number of context.active_pairs
buy_list = top_list.head(context.active_pairs)
# Create sell list with pairs not in buy list using set difference
sell_list = momentum_df.drop(labels=buy_list.index)
sell_list = sell_list.loc[sell_list.position > 0].pair
# Keep list: Currently being traded, with position size greater than 0 and momentum above sell threshold still
#keep_list = momentum_df.loc[(momentum_df.position > 0) & (momentum_df.momentum > context.SELL_MOMENTUM_THRESH)].pair
# Buy list: Currently not being traded, but have momentum greater than threshold
#buy_list = momentum_df.loc[(momentum_df.momentum >= context.BUY_MOMENTUM_THRESH) & (momentum_df.position == 0)].pair
#combined_active_list = list(keep_list) + list(buy_list)
combined_active_list = list(buy_list.pair)
# Sell list: Currently being traded, but momentum has dropped below threshold to keep
#sell_list = list(momentum_df.loc[(momentum_df.position > 0) & (momentum_df.momentum <= context.SELL_MOMENTUM_THRESH)].pair)
# Sell everything in sell_list first
if not len(sell_list) == 0:
for sell_pair in sell_list:
# If possible to trade pair
if data.can_trade(symbol(sell_pair)) and not get_open_orders(symbol(sell_pair)):
# Determine pair to sell, convert to usable symbol
sell_target = symbol(sell_pair)
# Set order percentage to 0, equivalent of a sell
order_target_percent(sell_target, 0)
# Get momentum to display for logging purposes
momentum = momentum_df.loc[momentum_df.pair == sell_pair].momentum.item()
# Log sell action
log.info('{}: {}: Sell - momentum: {} - PV: {} - Buy: {} Sell: {}'.format(
data.current_dt,
sell_target,
round(momentum, 5),
round(context.portfolio.portfolio_value, 5),
len(buy_list),
len(sell_list)))
# Sell
# Buy everything (update position size) in combined_active_list second
if not len(combined_active_list) == 0:
for buy_pair in combined_active_list:
# If possible to trade pair
if data.can_trade(symbol(buy_pair)) and not get_open_orders(symbol(buy_pair)):
# Determine pair to sell, convert to usable symbol
buy_target = symbol(buy_pair)
# Take a position of the size ORDER_SIZE for new pair
order_target_percent(buy_target, 1/context.active_pairs)
# Get momentum to display for logging purposes
momentum = momentum_df.loc[momentum_df.pair == buy_pair].momentum.item()
# Log buy action
if not len(buy_list) == 0:
log.info('{}: {}: Buy - momentum: {} - PV: {} - Buy: {} Sell: {}'.format(
data.current_dt,
buy_target,
round(momentum, 5),
round(context.portfolio.portfolio_value, 5),
len(buy_list),
len(sell_list)))
# Buy
record(active_pairs=context.active_pairs, buy_list=buy_list, sell_list=sell_list)
Is this a bug, or is there some best practice way to deal with this?
Cheers, let me know if you need any other info.
For Bitfinex you can realize that with if(pair['info']['expiration'] != 'NA') -> kick pair out of the universe
When you rebalance in short intervals.
But when I remember right, binance does not have a field like this.
