Concatenating portfolios

[trevor-richardson]

Hi Oleg,

I'd like to begin by thanking you for this work. I'm finding your work very useful and quite extensible. There is one issue I'm having when trying to evaluate a strategies performance out-of-sample and that is of putting portfolios together.

Let me set up the context of this problem first. Imagine you have a walk-forward experiment. You do in-sample search over many stop-losses and take profits for the year 2019. You then test out-of-sample for one month. You walk forward 12 times with a distance of a month to give you performance statistics out-of-sample for a year.

I know how to implement this using advstex

def test(ticker: str, entries: pd.DataFrame, exits:pd.DataFrame, data: dict, best_params:dict, start: str, end: str) -> pd.DataFrame:
    advstex = vbt.ADVSTEX.run(
        entries[ticker][start:end], 
        data['open'][ticker][start:end], 
        data['high'][ticker][start:end], 
        data['low'][ticker][start:end], 
        data['close'][ticker][start:end],
        sl_stop=best_params['sl_stop'],
        tp_stop=best_params['tp_stop'],
    )
    exits[ticker][start:end] = advstex.exits | exits[ticker][start:end]
    return exits


#Get all out of sample signals to run through portfolio
combined_exits = exits
for ticker in tickers:
    combined_exits = test(
        ticker,
        entries,
        exits,
        data,
        in_sample_studies[ticker].best_params,
        splits['test']['start'],
        splits['test']['end'],
    )
os_exits = combined_exits[splits['test']['start']: splits['test']['end']]

portfolio = vbt.Portfolio.from_signals(
    data['close'][splits['test']['start']: splits['test']['end']], entries[splits['test']['start']: splits['test']['end']], os_exits, close_first=True
)

However, using a custom strategy with custom orders I need to use from

portfolio = vbt.Portfolio.from_order_func( ...)

In the former case I could just leverage the fact that I use advstex on different subsets of my out-of-sample exits dataframe. But for a custom order func for custom strategies I need a way to put separate out-of-sample periods together.

I'd like to be able to collect 12 portfolios and concatenate them together. Something like the following:

portfolio_final = vbt.concat_portfolio([portfolio_1, portfolio_2 .... portfolio_n])

I've seen you get stats from portfolio's and concatenate them together but never concatenating full portfolios. for example:

for i in tqdm(range(len(exit_types))):
    chunk_mask = exits.columns.get_level_values('exit_type') == exit_types[i]
    chunk_exits = exits.loc[:, chunk_mask]
    chunk_portfolio = vbt.Portfolio.from_signals(ohlcv['Close'], entries, chunk_exits)
    total_returns.append(chunk_portfolio.total_return())
    
    del chunk_portfolio
    gc.collect()
    
total_return = pd.concat(total_returns)

Do you have any insight into how to do this? I've looked through your examples and docs and failed to find something like this.

Again, thank you for your work! Trevor

[polakowo]

Each portfolio object is composed of many pieces of information: array wrapper, order/log records, initial capital, etc. To concatenate multiple portfolios along any axis, you need to combine these pieces together.

There is currently no built-in way to do this because you must make a lot of assumptions to make a chain of multiple independent portfolios being equal to one big portfolio.

One of the bigger problems is initial capital. Assume you allocated $100 to each portfolio, and some of them have rejected orders because of insufficient funds. By concatenating 10 of such portfolios together, you would get a "Frankenstein" portfolio with an initial capital of $1000, but it would differ from a "native" portfolio that would receive $1000 right away because now orders have enough capital to be executed.

Another issue is prolonged events such as trades/positions: if the last position hasn't been closed in one portfolio, the next portfolio can't know about this, but the combination of both will project this open position to the next portfolio automatically and you will experience weird results. This is because trades and positions aren't calculated during the simulation but are building upon order records when the portfolio is already simulated, in a lazy way.

A safer approach in my view would be to concatenate out-of-sample periods together and create such a function for Portfolio.from_order_func that can take multiple parameters, one per period, and just perform one big simulation at once.

I will still mark this issue as a feature request so solutions to your example can be further explored.

[daviddwlee84]

Still looking for this feature. That concatenation here is more like column-wise concatenation but is not meant to put the orders or signals from one Portfolio to another.

The scenario is simple for me:

By default, I can do a parameter search using Indicator and return a "combined" portfolio. Then I can explore each setting's statistics and also average statistics by pf[(param1, param2)].stats() and pf.stats().

I think this should be equivalent to doing the parameter search by myself and concatenating multiple Portfolio objects into a single object. And it would be able to see the overall statistics easily. Furthermore, we can save the Portfolio into a single file, doing parallelism or better memory control, etc.

This can be working like pf_all = vbt.Portfolio.from_portfolios([pf, pf2, pf3]). Just like pd.concat (treated Portfolio object as some kind of DataFrame).