Computing TWR: corner-cases,complications and considerations

[adept]

I've decided to create this item to document all the corner-cases and complications that affect roi's TWR computations so that we have something to refer back to.

How to compute TWR: Wikipedia has a nice explanation. I am not going to repeat it here in full, but the key points are:

• You look at the (time) series of cash flows going in and out of the investment account
• You compute a value for each interval between cashflows (and possibly two extra intervals [start of reporting period, first cashflow] and [last cashflow, end of reporting period]
• To compute the value for each period, you need to know the current value of the investment either before or after each cashflow (and in some corner cases both before AND after the cashflow)
• You use those values to compute TWR

Historically we had a number of issues opened up against TWR, and I think that most (if not all) of them boil down to: there is no easy way to compute the value of account X in commodity Y right before the transaction Z in a way that would would be robust in the face of seemingly-small changes to the journal and always obvious/non-surprising to the end user.

What works

Let's start with what works: if the user's journal, limited to just the investment account:

• has at most 1 transaction per any given date
• has at most 1 P directive for the commodity that investment is valued in on any given date
• does not have transactions and P directives on the same date
• has P directives for all dates where actual real-life market price has changed
• has no transaction that combine cashflows and balance assertions that actually change the balance of the investment account
• has no @ or @@ pricing directives

then hledger roi [--value=then,<valuation commodity> ...] should always compute the correct TWR

What does not / what are the complications

Lets go through the list of conditions I set out in What works one by one

At most one transactions per date.

• Order in which transactions are recorded in the journal could differ from the order the user expects them to happen. Think about multiple files, include statements, transactions on the same date being separated by 1000s of lines in the same file.

At most 1 P directive per date and no transactions and P directives on the same date

• There is no way to record multiple prices during the day and intersperse them with transactions. As of 1.43 (correct me if I am wrong), the last P directive on any given day wins, and there is no way to say "this applies after transaction TR1, but before transaction TR2"

No transactions that combine cashflow and balance changes

Consider this transaction:

2025-05-05  Hmmm ....
  assets:cash     $-100
  equity:unrealized pnl
  investments:inv1    = $12345

If the balance assertion of investments:inv1 ends up changing the balance (let's say it increases it by $345), then we have two events occurring at the same time that need to be ordered. TWR computation needs to decide whether the balance change of $345 happens before the cash flow of $100, or after. The choice of the order affects the TWR computed.

This choice needs to be deterministic and must match the assumption about the order that the end-user is making. It is easy to see how any choice here will probably line up with the expectations of some users, but will confuse others.

Transaction prices set via @ or @@

Consider the transaction:

2025-05-05 Hmmm..
   assets   -$110
   investments:inv1    10 A @@ $110

TWR needs to compute the value of investments:inv1 either before this transaction, or after, or both. Is the price in @@, combined with --infer-market-prices, should be assumed to also be in effect immediately before this transaction, immediately after, or both? Again, there is a choice to be made here by the TWR algorithm, and whatever that choice is, it might not line up with user assumptions, and they would be able to convincingly argue that their approach also works

Investment being completely sold out

Imagine investment account that is being drained all the way to zero, potentially multiple times, during the reporting interval. This means that for cashflows that reduce the investment to zero we need to carry out TWR computations with pre-transaction balance, as post-transaction balance of zero will cause us to divide by zero. Why is this a problem? Because there is no simple reliable way to compute pre-transaction value.

No simple way to compute pre-transaction value

There is no easy way to compute the pre-transaction value, if we want to take into account prices changing on the same day, inferred prices, multiple transactions on the same day

[simonmichael]

Thanks for these notes. I have applied the WISH label as they seem to be more of that than a current BUG.