catalyst
catalyst copied to clipboard
scrtlabs
Slippage and Commission Units?
There isn't too much documentation on this, but I'd like clarification on the units for slippage and commission.
context.set_commission(maker=.001, taker=.001)
context.set_slippage(spread=0.0000001)
For example - does the above set a 0.1% commission on both maker and taker orders?
When calculating slippage, is it creating a 10 satoshi price difference from the provided price on the candlestick?
Be extremely careful with that. Setting a commission on my algorithm made it go from a bad algorithm to one that returns 200x on backtest. Lost some money testing it live
@caioaao thank you for reporting this and sorry about this issue. I'm investigating now.
I have not identified this issue yet:
Setting a commission on my algorithm made it go from a bad algorithm to one that returns 200x on backtest
I'm sharing this update to establish a baseline. I'm trying to recreate this issue in an algo using: mean_reversion_simple.py. I've set my commission like this: context.set_commission(maker=0.001, taker=0.002). This is the same as the default commissions for the eth_btc market used in this backtest.
I took the first transaction (buy order) and broke down how the algo calculates the cost basis to validate each calculation individually.
Command: order_target_percent(symbol('eth_btc'), 1)
Date: 2017-10-01 00:00:00+00:00
Order type: Limit Buy
Price: 0.06951499
Limit price: 0.06986256495
Amount: Capital (0.1) / Price (0.06951499) = 1.43853865188
Date next bar: 2017-10-01 00:01:00+00:00
Next bar price: 0.069547
Next bar under limit: Yes
Spread: 0.0001
Execution price: 0.069547 * 1.0001 = 0.0695539547
Cost: Amount (1.43853865188) * Price (0.0695539547) = 0.100056052227
Muptiplier: Taker (0.002), the order executed below limit price so it does not get a maker discount
Fee: Cost (0.100056052227) * Multiplier (0.002) = 0.000200112113959
Cost Basis: Execution price (0.0695539547) + Fee (0.000200112113959) = 0.06969306
Here are the raw stats output for the corresponding algo:
the transactions:
amount commission order_id price sid
dt
2017-10-01 00:01:00+00:00 1.43853865 None 2786256c52b64e5cbf19c259c9f2de85 0.06955395 TradingPair(452516 [eth_btc])
2017-10-02 07:44:00+00:00 -1.43853865 None 7baa00a69d4a4284985a63311f4ef7ae 0.06792321 TradingPair(452516 [eth_btc])
the orders:
amount commission created filled id limit limit_reached reason sid status stop stop_reached
dt
2017-10-01 00:01:00+00:00 1.43853865 0.00020011 2017-10-01 00:00:00+00:00 1.43853865 2786256c52b64e5cbf19c259c9f2de85 0.06986256 True None TradingPair(452516 [eth_btc]) 1 None False
2017-10-02 07:44:00+00:00 -1.43853865 0.00019542 2017-10-02 07:43:00+00:00 -1.43853865 7baa00a69d4a4284985a63311f4ef7ae 0.06759134 True None TradingPair(452516 [eth_btc]) 1 None False
the stats:
symbol amount cost_basis last_sale_price
period_close starting_cash ending_cash portfolio_value pnl long_exposure short_exposure orders transactions
2017-10-01 00:01:00+00:00 0.1 0.10000000 0.10000000 0.00000000 0.00000000 0 0 0 NaN NaN NaN NaN
2017-10-01 00:02:00+00:00 0.1 -0.00025616 0.09978988 -0.00021012 0.10004605 0 1 1 eth_btc 1.43853865 0.06969306 0.06954700
2017-10-01 00:03:00+00:00 0.1 -0.00025616 0.09979564 -0.00020436 0.10005180 0 0 0 eth_btc 1.43853865 0.06969306 0.06955100
2017-10-01 00:04:00+00:00 0.1 -0.00025616 0.09978124 -0.00021876 0.10003740 0 0 0 eth_btc 1.43853865 0.06969306 0.06954099
2017-10-01 00:05:00+00:00 0.1 -0.00025616 0.09972370 -0.00027630 0.09997986 0 0 0 eth_btc 1.43853865 0.06969306 0.06950099
In this one example, the calculations appear accurate as demonstrated here. It arrives to the exact same result whether I set the commission or use the default. I must be missing the condition causing error. I will continue to look for it and add these validation steps to unit tests to ensure that this issue never occurs in the future.
@caioaao I'm still trying to find the error. I can confirm that commissions get calculated correctly whenever Zipline calls: commission.calculate(order, txn) including here: catalyst/finance/blotter.py:371. When I pull the list of transactions in analyze(), I do see that the commission attribute is None. However, I can't see where Zipline applies the commission to the transaction objects like it does to orders in the method referenced above. I'm not sure if there is a consequence to this (need to check with Zipline), both the position cost_basis and pnl calculations use the correct value: tx.price + order.commission.
As far as I can tell, orders get added to the performance tracker here: catalyst/gens/tradesimulation.py:148 and the commission gets calculated correctly. I see not place where it's applying the same commission to transactions nor using it in calculations. So this appears to be a Zipline issue which I will follow-up on. However, this does not impact perf calculations such as pnl, there are being computed correctly here: catalyst.finance.performance.tracker.PerformanceTracker#process_commission.
I'm running more tests to see if I can somehow find an issue under different conditions.
I think I found out the issue, at least with poloniex. In Poloniex you always have to set the limit price, but it doesn't mean it's a limit order. If the price is contained in the order book (eg: buying above market value or selling below market value), it's considered a market taking order. When setting the commissions with set_commission(maker=0.0015, taker=0.0025), the backtester will always choose the market maker commission as all orders must have a limit price, thus improving the backtest results. Setting both to 0.0025 does the trick as it will be the worst case scenario for an algorithm. Probably leaving this as the default for poloniex is a good idea.
Yes, that's how other exchanges work as well. When you make a limit order, you potentially add liquidity to the order book, but you are only rewarded if someone takes your order. If the limit price in not reached and the order still closes, it implicitly means that you took someone else's order. Consequently, your limit order is deleted from the order book.
Unless there is an issue which I could not find, the simulation accounts for this. Specifically, it checks the open order triggers before each frame. In the frame following your order, if the price is lower than your limit, it will close your order and apply the taker fee. If the price reaches your limit, it applies the maker fee. You are correct that this is somewhat biased, it reality someone else may have taken these orders first leaving yours open. Or, the price may have briefly dipped below the limit price, making you pay the taker fees.
A more precise simulation model could evaluate all the order book entries between the frames. This would involve a huge amount of data, difficult to string together with current exchange APIs when even possible, as it would need to retrace all changes to order book and not sure the executed orders. This would still be biased because we can't precisely model the past, the algo would need to assume that you were able to take a particular order before whoever took it in reality. This assumption would be valid only some of the time.
I apologize if I'm stating something which you already know, but I think that this discussion is helpful. We'll write an article about this in our documentation. I definitely agree that assuming the pessimistic scenario is the best way to go. Another way to be pessimistic would be to add more slippage. I'll verify again with a focus on this particular behavior to identify any error.
We appreciate your time looking into this.
@fredfortier reminder to add this to the documentation as per your comment above
I am using Binance backtesting, what is the default maker / taker fees catalyst is using?
Debugging context.blotter.commission_models & context.blotter.slippage_models, it seems that slippage is by default 0.0001, and maker_fee and taker_fee are set to None.
However, adding to my code:
context.set_commission(maker=0.001, taker=0.001)
context.set_slippage(slippage=0.001)
Re-run backtesting and comparing the perf.transactions, I get a lower commission.
The price and amount of the transactions are the same.
