Short only portfolios don't work with size_type='percent'

[dinkovv]

Hi,

I'm having issues with short only portfolios when size_type is set to percent:

>>> close = pd.Series([1, 2, 3, 4, 5])
>>> entries = pd.Series([True, True, True, False, False])
>>> exits = pd.Series([False, False, True, True, True])
>>> portfolio = vbt.Portfolio.from_signals(close, entries, exits, size=0.01, direction='shortonly')
>>> portfolio.share_flow()
0   -0.01
1    0.00
2    0.00
3    0.01
4    0.00
dtype: float64
>>> portfolio = vbt.Portfolio.from_signals(close, entries, exits, size=0.01, size_type='percent', direction='shortonly')
>>> portfolio.share_flow()
0    0.0
1    0.0
2    0.0
3    0.0
4    0.0
dtype: float64

The above code works for long only portfolios:

>>> portfolio = vbt.Portfolio.from_signals(close, entries, exits, size=0.01, size_type='percent', direction='longonly')
>>> portfolio.share_flow()
0    1.0
1    0.0
2    0.0
3   -1.0
4    0.0
dtype: float64

`

[polakowo]

Size type 'percent' has the following logic: buy for a percentage of current cash when going long and sell a percentage of current shares when going short. The percentage here just means a fraction of your cash/shares balance. Since you're not in a position, you are attempting to sell zero shares. Maybe you meant 'targetpercent'?

[dinkovv]

Ok, I see, thank you. But then how can I (short) sell for a percentage of current cash and then close the position?

[dinkovv]

I don't think I can achieve what I want by using 'targetpercent'.

[polakowo]

You can target short to 50% of the current portfolio value (= current cash if not in position) and then long to 0% to close the position. To do so, you need to switch to Portfolio.from_orders:

>>> close = pd.Series([1, 2, 3, 4, 5])
>>> size = pd.Series([0.01, np.nan, np.nan, 0., np.nan])
>>> portfolio = vbt.Portfolio.from_orders(close, size, size_type='targetpercent', direction='shortonly')
>>> portfolio.share_flow()
0   -1.0
1    0.0
2    0.0
3    1.0
4    0.0
dtype: float64

[dinkovv]

On 24 Apr 2021 21:42, Oleg Polakow @.***> wrote: You can target short to 50% of the current portfolio value (= current cash if not in position) and then long to 0% to close the position. To do so, you need to switch to Portfolio.from_orders:

close = pd.Series([1, 2, 3, 4, 5]) size = pd.Series([0.01, np.nan, np.nan, 0., np.nan]) portfolio = vbt.Portfolio.from_orders(close, size, size_type='targetpercent', direction='shortonly') portfolio.share_flow() 0 -1.0 1 0.0 2 0.0 3 1.0 4 0.0 dtype: float64

—You are receiving this because you authored the thread.Reply to this email directly, view it on GitHub, or unsubscribe.

[dinkovv]

Great, thanks. I think the most common scenario in trading is to size positions based on percentage of equity, but it's not very clear from the documentation how one can achieve it. For example, my understanding is that one can not achieve a reversal of position in a single portfolio in this scenario.

[polakowo]

Currently, vectorbt implements all sizers from backtrader, so a percentage of equity is not on the list and cannot be enabled with a single flag but needs some creativity. You can always write a custom order function that implements whatever sizer you want though, but I understand that it's not the easiest approach.

I will look into how to adapt the 'percent' sizer to take into account the current direction and adapt accordingly.

[dinkovv]

Thanks very much!

[polakowo]

Should be fixed in 0.18.0.

It's now symmetric around zero and simply means the percentage of the value that can be moved in both directions (similar to target percentage but without target requirement). It supports position reversal but still cannot work with signals and 'all' direction (long-only and short-only directions are fine though). Example: If you have $50 in cash and $50 in shares, you can sell/short for as much as $150 ($50 from selling and an additional $100 from shorting), so applying the percentage of -50% will sell/short shares for $75.

[kmcentush]

Hi @polakowo,

Related to the fix in 0.18.0, how can I ensure that my short-side (when also taking into account long-side) does not exceed the total portfolio value?

In essence, I have some TargetPercent weights like the following: [0.25, 0.15, 0.2, -0.25, -0.15]. Notice how their absolute values sum to 1. If I have $100 total portfolio value, my goal is for it to be allocated in the following way the next day/trading period: [$25, $15, $20, -$25, -$15].

Based on your example above, I should take the value of cash + other shorts (from the previous trading period), divide that by total portfolio value (so in this case, $40 / $100 = 40%), then apply this scalar to all my short positions, no?

Unrelated, when using TargetPercent with ShortOnly, do the weights all need to be positive by convention in order to backtest correctly? I believe this is what I was seeing when trying locally, but the issue may actually just be related to my misunderstanding of how to make portfolio as described above.

I apologize if some of my language/assumptions are wrong. Hopefully my intention is clear.

[polakowo]

@kmcentush in 0.19.1 (next release, today/tomorrow), vectorbt will further distinguish between cash and free cash. Generally, when you short, cash from the shorting operation gets added to your balance and the asset value becomes negative. This behavior remains unchanged. But now, there will be another variable called free_cash, that does the opposite: it reduces its balance when you short/long, and adds to it if you buy the assets back. It basically signals that some of your cash has collateral that can't be used for further shorting. So, shorting all your $100 will increase your cash to $200 and decrease your free_cash to $0. To make use of free_cash, you can pass lock_cash=True, so it will prevent any operation (long or short) if your free_cash becomes zero. This is already implemented in 0.19.0 (current release) but only for short operations, I must fix one thing for it to also limit longs such that free_cash never turns negative with lock_cash enabled.

I'm not quite sure why you want to perform other operations - using your percentage vector together with size_type='targetpercent' should be more than enough.

A positive value in Direction.ShortOnly gets automatically translated to a negative value in Direction.All. So, shorting 100% means going -100%.

As soon as 0.19.1 is out I will provide you with a small example.

[kmcentush]

Awesome, the free_cash implementation sounds like exactly what I'm looking for. Really appreciate your detailed answer, and looking forward to the next release.

On Fri, Jun 25, 2021, 7:44 AM Oleg Polakow @.***> wrote:

@kmcentush https://github.com/kmcentush in 0.19.1 (next release, today/tomorrow), vectorbt will further distinguish between cash and free cash. Generally, when you short, cash from the shorting operation gets added to your balance and the asset value becomes negative. This behavior remains unchanged. But now, there will be another variable called free_cash, that does the opposite: it reduces its balance when you short/long, and adds to it if you buy the assets back. It basically signals that some of your cash has collateral that can't be used for further shorting. So, shorting all your $100 will increase your cash to $200 and decrease your free_cash to $0. To make use of free_cash, you can pass lock_cash=True, so it will prevent any operation (long or short) if your free_cash becomes zero. This is already implemented in 0.19.1 but only for short operations, I must fix one thing for it to also limit longs such that free_cash never turns negative with lock_cash enabled.

I'm not quite sure why you want to perform other operations - using your percentage vector together with size_type='targetpercent' should be more than enough.

A positive value in Direction.ShortOnly gets automatically translated to a negative value in Direction.All. So, shorting 100% means going -100%.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/polakowo/vectorbt/issues/131#issuecomment-868550623, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE5SCMB7UETLJT5NTCY5BPDTUSI3XANCNFSM43QKHDNQ .

[polakowo]

@kmcentush 0.19.1 is out and everything works as expected now. Here is an example of 50% long and 50% short:

import vectorbt as vbt
import pandas as pd

size = pd.DataFrame([[0.5, -0.5]])
price = pd.Series([1, 2, 3, 2, 1])
pf = vbt.Portfolio.from_orders(
    price, size, size_type='targetpercent', call_seq='auto', lock_cash=True,
    group_by=True, cash_sharing=True)

# portfolio rebalances each tick. In times when `price` grows, both long and short positions have 
# to decrease in size to maintain the correct ratio, thus accumulating `free_cash`. In times when `price` falls, 
# the opposite happens - both positions have to increase in size, thus using `free_cash`. 
print(pf.asset_flow())
           0          1
0  50.000000 -50.000000
1 -25.000000  25.000000
2  -8.333333   8.333333
3   8.333333  -8.333333
4   8.333333 -25.000000

# In the last tick, the strategy runs out of `free_cash` so it cannot maintain the long position of 50% anymore. 
# If you do `lock_cash=False`, the ratio is maintained but now `free_cash` turns negative.
print(pf.asset_value(group_by=False))
           0     1
0  50.000000 -50.0
1  50.000000 -50.0
2  50.000000 -50.0
3  50.000000 -50.0
4  33.333333 -50.0

# portfolio value is the same since there are no costs associated with transactions
print(pf.value())
0    100.0
1    100.0
2    100.0
3    100.0
4    100.0
Name: group, dtype: float64

# Ratios
print(pf.asset_value(group_by=False).vbt / pf.value())
          0    1
0  0.500000 -0.5
1  0.500000 -0.5
2  0.500000 -0.5
3  0.500000 -0.5
4  0.333333 -0.5

# There is always enough cash, but part of it is meant to be collateral (margin)
# `free_cash` prevents from using this collateral
print(pf.cash())
0    100.000000
1    100.000000
2    100.000000
3    100.000000
4    116.666667
Name: group, dtype: float64

# Once free cash is zero, you can't increase any position, only reduce/close out. 
print(pf.cash(free=True))
0     0.000000
1    50.000000
2    66.666667
3    33.333333
4     0.000000
Name: group, dtype: float64

[kmcentush]

Awesome, thanks!

[viper7882]

In case someone wondering why the size wasn't 50% per comment from @polakowo:

You can target short to 50% of the current portfolio value (= current cash if not in position) and then long to 0% to close the position. To do so, you need to switch to Portfolio.from_orders:

>>> close = pd.Series([1, 2, 3, 4, 5])
>>> size = pd.Series([0.01, np.nan, np.nan, 0., np.nan])
>>> portfolio = vbt.Portfolio.from_orders(close, size, size_type='targetpercent', direction='shortonly')
>>> portfolio.share_flow()
0   -1.0
1    0.0
2    0.0
3    1.0
4    0.0
dtype: float64

The reason is that close is an incremental pandas series and if we are going for Short position, the PNL is going to be greater than our initial cash. That's why the size has to be 0.33 or lower, @polakowo put 0.01 in his example above.

IMHO a more appropriate example should be a decremental close pandas series:

close = pd.Series([10, 9, 8, 7, 6])
size = pd.Series([0.5, np.nan, np.nan, 0.0, np.nan])
portfolio = vbt.Portfolio.from_orders(close, size,
                                      size_type=SizeType.TargetPercent,
                                      direction=Direction.ShortOnly,
                                      init_cash=1000.0,
                                      log=True,
                                      )

Sample output:

portfolio.asset_flow():
0   -50.0
1     0.0
2     0.0
3    50.0
4     0.0

The intention is to clarify any confusion in case someone bumped into it like me.