Portfolio handling for multi-venue Accounts

[rsmb7z]

Hi @cjdsellers I managed to reconcile the Open Positions/Orders in efa9bc6 as per your suggestion however there is small problem.

The portfolio will halt the TradingNode because of No Account.

2023-03-18T21:43:57.504536200Z [INF] TESTER-001.TradingNode: Awaiting execution state reconciliation (15.0s timeout)...
2023-03-18T21:43:57.507719600Z [INF] TESTER-001.InteractiveBrokersExecutionClient-002: Generating ExecutionMassStatus for InteractiveBrokers...
2023-03-18T21:43:57.513659500Z [WRN] TESTER-001.InteractiveBrokersExecutionClient-002: Cannot generate `list[TradeReport]`: not yet implemented.
2023-03-18T21:43:57.625185500Z [INF] TESTER-001.ExecEngine: Reconciling ExecutionMassStatus for InteractiveBrokers.
2023-03-18T21:43:57.625216500Z [INF] TESTER-001.ExecEngine: Generating external order ClientOrderId('MESM3.CME')
2023-03-18T21:43:57.625887600Z [WRN] TESTER-001.ExecEngine: Generated inferred OrderFilled(instrument_id=MESM3.CME, client_order_id=MESM3.CME, venue_order_id=MESM3.CME, account_id=InteractiveBrokers-DUXXXXXXX, trade_id=8a31c3df-8b28-41f0-beff-d97829fe3eaf, position_id=MESM3.CME-EXTERNAL, order_side=SELL, order_type=MARKET, last_qty=1, last_px=19956.88 USD, commission=0.00 USD, liquidity_side=TAKER, ts_event=1679175837564181200).
2023-03-18T21:43:57.626133700Z [INF] TESTER-001.Portfolio: MESM3.CME net_position=-1.0
2023-03-18T21:43:57.626155000Z [INF] TESTER-001.ExecEngine: Generating external order ClientOrderId('O-20230317-001-000-5')
2023-03-18T21:43:57.626421200Z [INF] TESTER-001.TradingNode: State reconciled.
2023-03-18T21:43:57.626502100Z [INF] TESTER-001.Portfolio: Initialized 1 open order.
2023-03-18T21:43:57.626512100Z [INF] TESTER-001.Portfolio: Initialized 1 open position.
2023-03-18T21:43:57.626517800Z [INF] TESTER-001.TradingNode: Awaiting portfolio initialization (15.0s timeout)...
2023-03-18T21:43:57.626136900Z [ERR] TESTER-001.Portfolio: Cannot calculate unrealized PnL: no account registered for CME.
2023-03-18T21:43:57.626500200Z [ERR] TESTER-001.Portfolio: Cannot update initial (order) margin: no account registered for CME.
2023-03-18T21:43:57.626509600Z [ERR] TESTER-001.Portfolio: Cannot calculate unrealized PnL: no account registered for CME.
2023-03-18T21:43:57.626511000Z [ERR] TESTER-001.Portfolio: Cannot update maintenance (position) margin: no account registered for CME.

The following two scenarios made it work perfectly.

1. Instrument Venue is IB_VENUE
2. In the Porfolio, find the account using IB_VENUE instead of Instrument.venue. With this the problem was for Backtest however I did this so for now does the job.

2023-03-18T20:02:01.818028500Z [INF] TESTER-001.TradingNode: Awaiting execution state reconciliation (15.0s timeout)...
2023-03-18T20:02:01.821512400Z [INF] TESTER-001.InteractiveBrokersExecutionClient-002: Generating ExecutionMassStatus for InteractiveBrokers...
2023-03-18T20:02:01.827293500Z [WRN] TESTER-001.InteractiveBrokersExecutionClient-002: Cannot generate `list[TradeReport]`: not yet implemented.
2023-03-18T20:02:01.966333000Z [INF] TESTER-001.ExecEngine: Reconciling ExecutionMassStatus for InteractiveBrokers.
2023-03-18T20:02:01.966342100Z [INF] TESTER-001.ExecEngine: Generating external order ClientOrderId('MESM3.CME')
2023-03-18T20:02:01.966731300Z [WRN] TESTER-001.ExecEngine: Generated inferred OrderFilled(instrument_id=MESM3.CME, client_order_id=MESM3.CME, venue_order_id=MESM3.CME, account_id=InteractiveBrokers-DUXXXXXXX, trade_id=da0cd6a3-7f80-48eb-9327-0574eeae707a, position_id=MESM3.CME-EXTERNAL, order_side=SELL, order_type=MARKET, last_qty=1, last_px=19956.88 USD, commission=0.00 USD, liquidity_side=TAKER, ts_event=1679169721891257800).
2023-03-18T20:02:01.966920900Z [INF] TESTER-001.Portfolio: MESM3.CME net_position=-1.0
2023-03-18T20:02:01.966947500Z [INF] TESTER-001.ExecEngine: Generating external order ClientOrderId('O-20230317-001-000-5')
2023-03-18T20:02:01.967197700Z [INF] TESTER-001.TradingNode: State reconciled.
2023-03-18T20:02:01.967320000Z [INF] TESTER-001.Portfolio: MESM3.CME margin_init=0.00 USD
2023-03-18T20:02:01.967338000Z [INF] TESTER-001.Portfolio: Initialized 1 open order.
2023-03-18T20:02:01.967406300Z [INF] TESTER-001.Portfolio: Initialized 1 open position.
2023-03-18T20:02:01.967413100Z [INF] TESTER-001.TradingNode: Awaiting portfolio initialization (15.0s timeout)...
2023-03-18T20:02:01.970221500Z [INF] TESTER-001.TradingNode: Portfolio initialized.

The 1st solution is somewhat related to discussion we had previously about InstrumentId naming convesion. What I was thinking that we have AccountId common and unique in all required objects. This would also solve the problem if we are trading SPY.ARCA at InteractiveBrokers and the same Instrument at TD as Orders and Positions are attached to AccountId. Not only this, it will pave the way for trading two different accounts of same broker. So nutshul we have to somehow integrate Portfolio to adopt based on AccoundId.

I have implemented AccountId query filtering for Cache here but was not successfull in making Portfolio work in my first attempt. Any suggestions or thoughts may help.

[cjdsellers]

This makes sense, and I have actually attempted a multi-account per venue/client extension myself in the past, and remember the challenging part was handling AccountManager and accounting. I was having trouble tracking down some PnL differences in a handful of tests and decided to back out of it to keep things simple - I still think this would be a good change though.

What are the sticking points for you right now?

[rsmb7z]

What are the sticking points for you right now?

The essential thing to achieve this (in single portfolio/account perspective) is allow configuring default Venue for Portfolio as in here. So this will make any adapter including IB or similar to behave all in same fashion.

This makes sense, and I have actually attempted a multi-account per venue/client extension myself in the past, and remember the challenging part was handling AccountManager and accounting. I was having trouble tracking down some PnL differences in a handful of tests and decided to back out of it to keep things simple - I still think this would be a good change though.

This gives me another thought, isolating the Portfolio per Account instead of modify the existing to support multi. Instead of creating Portfolio at NautilusKernel bring it under node_builder.py (or similar) in build_exec_clients where AccoundId is known - or maybe created within NautilusKernel by taking the AccountId from ExecClientConfig, as long serve the purpose. Portfolio will register like endpoint="Portfolio.InteractiveBrokers-DUXXXXXXX.update_account" in this case and ExecClient will be publishing updates to this specific Portfolio only. On Actor/Strategy side Portfolio would be accessed like self.portfolio[AccountId].is_flat() etc. There could be interest of knowing totals per TradingNode - this could be easily achieved, by top-level summary Portfolio class, subscribing to all Portfolios registered for the Node and summing things up. This wouldn't subscribe directly to Venues. What do you think of this?