Unexpected price used by roi when journal contains multiple prices for the the same day

[aragaer]

trafficstars

Slightly modified snake oil example:

P 2019-09-30 SNKOIL $1.075

2019-12-20 Investing in Snake Oil
 assets:cash
 investment:snake oil   100 SNKOIL @@ $107.5

P 2019-12-23  SNKOIL $1.08

2019-12-23 Fear of missing out
 assets:cash 
 investment:snake oil   83.33 SNKOIL @@ $90 ; $90/$1.08 = 83.33

;P 2019-12-23  SNKOIL $1.09  ; -- this line affect the 2019-12-22 ROI result

; Recording the growth of Snake Oil
P 2019-12-24  SNKOIL $1.1

2019-12-25 check
 investment:snake oil   0 = 183.33 SNKOIL

The issue here is that there is another (commented in the example) price directive for 2019-12-23. When doing a per-day roi I'm expecting that the final price for a date would be the first price of the next day, however the last price is used instead. Adding a price directive to reflect that the price changed after the additional purchase affects the calculated roi of previous day.

Apparently it is not directly related to roi, just the general case of using the next day as "end" of a period and using the last price when multiple directives are given for the same date.

[simonmichael]

Thanks @aragaer, I imagine @adept will have thoughts on this at some point.

[adept]

Roi does not interact with pricing directives directly, it just asks pricing functions to price things. So it would be very hard to change how Roi behaves here without changing the behaviour of the rest of the hledger.

I think this is the case of "multiple pricing directives per day are generally not (very well?) supported in hledger"

[simonmichael]

Ah, yes. I guess that is probably what you're seeing. Valuation (pricing) in hledger uses a single P directive per day, the last one parsed if there is more than one, I am guessing. I found slight mention of it at https://hledger.org/dev/hledger.html#finding-market-price .

[simonmichael]

@aragaer did that explain it ?

[aragaer]

Yes, this is exactly how I understood it. The question is -- should it be fixed? Is it worth implementing a different valuation request which will use first price instead, so that it is used for "end" valuations?

[simonmichael]

I think when there are multiple similar directives, generally the last one wins. Similarly to when there are multiple similar command line options, the last one wins. So it sounds like the current behaviour is at least consistent with hledger's usual way of doing things ?

[aragaer]

When calculating roi for certain period we want the price at the end of period. Current implementation is to use the price for the first day after that period, which means that first directive would be more appropriate. Unfortunately there is currently no way to specify "end price".

[adept]

I dont this that this lines up with how the pricing code works, btw.

But regardless, I think that support of more than one price per day would require a lot of forward planning, cause opportunities for tricky edge-cases would expand significantly.

Nevermind first and last price of the day, what about having 15 price directives for a given day, interspersed with transactions? What if those prices (and transactions) come from different files via include directives, and include graph has "diamonds"? Etc etc

[simonmichael]

I just found this old discussion with @aragaer and @adept. Is it a (UX/doc) bug or not ? @aragaer maybe you could confirm if this is still an issue with current hledger ?

[adept]

Is it a (UX/doc) bug or not ?

This boils down to the following question: if one has 10 "P" directives for given single day, how many of them get used by hledger, when, and is it a bug that you can have so many?

[simonmichael]

Current status:

• all but the last P on a day are ignored, and we don't report an error if there's more than one
• that's standard directive behaviour, so considered not a bug
• it's therefore not useful to have multiple P for one currency pair in a day, but harmless

And I think @aragaer was saying the above doesn't fit well with roi or user's expectations when using it, because it uses a price from the day after report end date, and they have multiple different prices on that day. Or something.

@aragaer, I think in practice there's no reason to record multiple intra-day prices in your journal, right ?

[aragaer]

The original issue was that when calculating roi for day X it uses the price for day X+1 and if there are multiple prices for day X+1 that affects the result for day X. There would be less confusion if there was a clear warning for having multiple price directives. Or if the roi was calculated using the last day of the period, not the first day after it.

When records are split into separate files and you're editing an April file but for some reason the result you get for April depends on what you got in your May file, that doesn't look right.

[simonmichael]

Thanks. Two things -

• you could arrange to have the may 1 P directive with your April data

• why do you have multiple P directives for the same day ?

[aragaer]

It's not about "why data for May is wrong". It's about looking at a correct April file with all the April data in it but getting incorrect result because of issues in a different file. I just don't like this sort of "hidden remote implicit" interaction of data.

Current behavior is not a bug but it isn't obvious. When I learned that only the last price of the first day after the period is used I updated my records and removed extra directives. But it took time to investigate why I was getting an incorrect result.

[simonmichael]

Certainly. Good feedback. I'm just trying to understand the situation fully.

Do you have any suggestions to improve things ?

We could report an error when there are multiple prices on the same day. This would probably break some existing journals, where people have those for some reason (perhaps downloaded prices).

We could add hledger check prices, to check it manually. And possibly include that in -s/--strict mode.

[aragaer]

I think adding it to strict check would be enough.