hledger icon indicating copy to clipboard operation
hledger copied to clipboard

doesn't fully support Ledger's {} price syntax

Open mapreri opened this issue 4 years ago • 21 comments

% cat foo.ledger
2019-04-22 Foo
      ;Program: Bar
      ;Entity: Baz
    Accrued:Accounts Payable:Bar      £-154.83 {=$1.29847} @ $1.29847
    Expenses:Bar:A                    £154.83 {=$1.29847} @ $1.29847
% ledger --file foo.ledger balance
            £-154.83  Accrued:Accounts Payable:Bar
             £154.83  Expenses:Bar:A
--------------------
                   0
% hledger-ui -f foo.ledger                      
hledger-ui: /home/mattia/devel/reproducible/ledger/foo.ledger:4:60:
  |
4 |     Accrued:Accounts Payable:Bar      £-154.83 {=$1.29847} @ $1.29847
  |                                                            ^
unexpected '@'
expecting ';', end of input, or newline
% hledger-ui --version
hledger-ui 1.12.1

Afaik that syntax is common when dealing with invoices in foreign currencies that you want to keep track of without converting them in your ledger entry.

However, it seems hledger doesn't cope with that at all.

mapreri avatar Sep 07 '19 18:09 mapreri

Hi @mapreri.. the {} syntax is used by Ledger, but not hledger. https://hledger.org/journal.html#transaction-prices describes the syntax we currently support, and this seems related at https://hledger.org/faq.html#functional-differences : "Ledger allows amounts to have a fixed lot price (the {} syntax ?) and a regular price in any order (and uses whichever appears first). hledger requires the fixed lot price to come last (and ignores it)."

simonmichael avatar Sep 07 '19 18:09 simonmichael

So, I think you can work around by just putting the {} last:

2019-04-22 Foo
      ;Program: Bar
      ;Entity: Baz
    Accrued:Accounts Payable:Bar      £-154.83 @ $1.29847 {=$1.29847}
    Expenses:Bar:A                    £154.83 @ $1.29847 {=$1.29847}

simonmichael avatar Sep 07 '19 18:09 simonmichael

So, I think you can work around by just putting the {} last:

Mh, that's indeed the case, swapping the two syntaxes does appear work, thank you for the hint!

I'll let you choose the fate of this bug (left opened as a feature request to also support the {} syntax, or close it).

mapreri avatar Sep 08 '19 11:09 mapreri

It's a good question. hledger so far ignores {} because I don't like that syntax - I find it overcomplicated and unnecessary. IIRC it does exactly what hledger's @ does already. (I forget exactly how Ledger's @ is different from ours.) More Ledger compatibility is good to have, but more confusion and complexity in hledger is not. Investigation, opinions welcome.

simonmichael avatar Sep 08 '19 12:09 simonmichael

As far as I remember {} produce additional postings for capital gains and capital loss. I simulate that with next transactions

2018-03-31 Report
    Income:Stocks  $136.57  ; Unrealized capital loss
    Assets:Stocks  -1 AMZN @ $1,583.91  ; corresponds to {=$1,583.91} - original price
    Assets:Stocks  1 AMZN @ $1,447.34

Or

2018-03-31  Sell
    Income:Stocks  $136.57  ; Realized capital loss
    Assets:Stocks  -1 AMZN @ $1,583.91  ; original price
    Assets:Bank:Current  $1,437.34  ; $1,447.34 - $10
    Expenses:Fee  $10

In second case some transaction might not explicitly mention new price. Thus sometimes I kinda merge them and get ugly:

2018-03-31  Sell
    Income:Stocks  $136.57  ; Realized capital loss
    Assets:Stocks  -1 AMZN @ $1,583.91
    Assets:Stocks  1 AMZN @ $1,447.34
    Assets:Stocks  -1 AMZN @ $1,447.34
    Assets:Bank:Current  $1,437.34
    Expenses:Fee  $10

Effectively when you write

    A  -1 AMZN {=$1,447.34} @ $1,583.91

Is roughly equivalent of

    I  $136.57  ; ($1,583.91 - $1,447.34) * 1
    A  -1 AMZN @ $1,583.91
    ; transaction balance = $1,447.34

ony avatar Oct 06 '19 08:10 ony

@ony: thanks, very helpful examples. I will remember: "Ledger's {} is like @ except it also generates capital gain/loss transactions".

Do your manual entries get very tedious, eg if doing a lot of stock trading ?

We may end up having to support {}. But I have always found it really confusing. Is that true, or just me, or just lack of the right docs ?

Can anyone think of a more intuitive syntax for this, if you were designing from scratch ?

simonmichael avatar Oct 08 '19 01:10 simonmichael

PS and what's the difference between {AMT} and {=AMT} ?

simonmichael avatar Oct 08 '19 01:10 simonmichael

If I'm reading https://www.ledger-cli.org/3.0/doc/ledger3.html#Fixing-Lot-Prices right:

  • {AMT} is identical to @ AMT, and does not generate capital gains transactions
  • {=AMT} generates capital gains transactions

See:

This transaction actually introduces a new commodity, ‘GAL {=$2.29}’, whose market value disregards any future changes in the price of gasoline.

If you do not want price fixing, you can specify this same transaction in one of two ways, both equivalent (note the lack of the equal sign compared to the transaction above):

2009/01/01 Shell
    Expenses:Gasoline             11 GAL {$2.299}
    Assets:Checking

2009/01/01 Shell
    Expenses:Gasoline             11 GAL @ $2.299
    Assets:Checking

That reads backwards to me. Going by Ledger's behaviour, I think of the one without the = as fixed for all time, and the one with = as affected by market price changes.

simonmichael avatar Oct 08 '19 01:10 simonmichael

Related to #1029. I think we can consider this a sub-task of that.

simonmichael avatar Oct 08 '19 01:10 simonmichael

But continuing this train of thought:

As I understand it,

Ledger has ~four~six syntaxes for specifying price in a transaction: @, @@, { }, {= }, {{ }}, {{= }}. @@ is another sometimes more convenient form of @. { } is an alternate spelling of @ (I'm not sure why it exists).

So, semantically it has two kinds of transaction price: @ which will not generate capital gains transactions for that amount, and {= } which will.

hledger has one kind of transaction price. I don't yet see that two are needed. I think that market price (P) records are the thing that should cause capital gains transactions.

Ledger's manual says only:

There is no difference in meaning between these two forms. Why do both exist, you ask? To support things like this:

2009/01/01 Shell
    Expenses:Gasoline             11 GAL {=$2.299} @ $2.30

I think this is not a real example, or can someone explain it ? Or describe other situations where two separate prices on a posting are desirable ? Would stock options be one such ?

simonmichael avatar Oct 09 '19 05:10 simonmichael

This is seems interesting and possibly related, from https://www.cointracker.io/blog/new-irs-cryptocurrency-tax-guidance#specific-identification . I've added our terminology in brackets:

To successfully identify a specific unit [lot], information must include:

  • the date and time each unit was acquired,
  • your basis [cost, transaction price] and the fair market value of each unit at the time it was acquired,
  • the date and time each unit was sold, exchanged, or otherwise disposed of, and
  • the fair market value of each unit when sold, exchanged, or disposed of, and the amount of money or the value of property received for each unit

Ie, two amounts are considered with each transaction: the amount exchanged, and the fair market value at that time. hledger currently doesn't expect you to record the market value along with the transaction, assuming that will be figured out from the market prices on that date.

simonmichael avatar Oct 10 '19 19:10 simonmichael

I think that IRS overcomplicates things here (and we should not). Let me try to annotate this piece of text the same way you did, while also considering this example:

2019-01-01 buy BTC
   cash  -$100
   crypto  1 BTC

2019-10-10 sell BTC
    crypto -1 BTC
    cash  $999

To successfully identify a specific unit [lot], information must include:

  • the date and time each unit was acquired [this is date of buy transaction, 2019-01-01],
  • your basis [price at the time of buying, $100] and the fair market value of each unit at the time it was acquired [IRS wants to know this presumably to ensure that you dont cook your books. We don't need this (even though price directive could be in the file and could provide this info)],
  • the date and time each unit was sold, exchanged, or otherwise disposed of [this is date of the sell 2019-10-10], and
  • the fair market value of each unit when sold, exchanged, or disposed of [again, this is IRS checking that you do not misreport the price to lower your taxes], and the amount of money or the value of property received for each unit [this is price at the price of selling, $999]

So I agree that amounts are considered, but I disagree with your description of them. I claim that they are the (qty*buy price) and (qty*sell price). Your capital gains or losses are (qty*sell price) - (qty*buy price), and this is what you pay capital gains tax on.

In the example above, your gains are $999*1 - $100*1 = $899

Specific identification is used not only for crypto, but for shares as well. Helpful example for shares that corroborates what I wrote: https://www.schwab.com/resource-center/insights/content/save-on-taxes-know-your-cost-basis (search for Specific identification method there)

So we need to either have a way to refer back to buy transaction or to include cost basis into the sell transaction. And ledger's syntax `{= cost basis}' could be useful here:

2019-01-01 buy BTC
   cash  -$100
   crypto  1 BTC

2019-10-10 sell BTC
    crypto -1 BTC @@ $999 {= $100}
    cash  $999

Or:

2019-01-01 buy BTC
   cash  -$100
   crypto  1 BTC

2019-10-10 sell BTC
    crypto -1 BTC {= $100}
    cash  $999

If this syntax is allowed, it should be possible to:

  1. automatically create Capital Gains transactions like Ledger does

  2. implement something like --lots from ledger or "account inventory" from beancount, where you compute lots (which are triples of (commodity, price, date)) on each buy and destroy some of them (or their parts) on each sale, and at the end report what is left.

adept avatar Oct 10 '19 21:10 adept

It is worth noting that a way to include cost basis in the sell transaction is most flexible -- you could, for example, record a sell (and capital gains) even if you have no record of buying the thing.

On the other hand, if you have all the buys recorded, then a way to record back to them would be more foolproof:

2019-01-01 buy BTC
   cash  -$1000
   crypto  10 BTC

2019-02-02 buy more BTC
   cash  -$2000
   crypto  10 BTC

2019-10-10 sell BTC
    crypto -10 BTC {2019-01-01}
    crypto -5 BTC {2019-02-02}
    cash  $15000

It would be more foolproof to use dates provided to refer back to buys to figure out that:

  • buy price aka basis on 2019-01-01 was $1000/10 = $100
  • buy price aka basis on 2019-02-02 was $2000/10 = $200
  • sell price on 2019-10-10 is $15000/15 = $1000
  • capital gains could be computed as (15*$1000 - 10*$100 - 5*$200) = $13000

Cause if you record cost basis in the sell transaction directly, you can make a mistake like this:

2019-01-01 buy BTC
   cash  -$1000
   crypto  10 BTC

2019-02-02 buy more BTC
   cash  -$2000
   crypto  10 BTC

2019-10-10 sell BTC
    crypto -15 BTC {= $100}
    cash  $15000

And it would be sad if this would lead to capital gains of $13500 without any cross-checking, even though you never had 15 BTC bought at basis of $100, you had only 10 BTC.

adept avatar Oct 10 '19 21:10 adept

So we need to either have a way to refer back to buy transaction, or to include cost basis into the sell transaction. And ledger's syntax `{= cost basis}' could be useful here.

I’ll respond to the rest later, but first this: wow, I had been considering use of {= } only in the buy transaction. Is it actually intended to be used in the sell transaction ? Only ?

simonmichael avatar Oct 10 '19 22:10 simonmichael

I use the {} syntax to write transactions in this way:

2019-10-10 * Buy ABC
    crypto:abc       1 ABC @ 10 USD
    bank            -10 USD

2019-10-11 * Buy XYZ
    crypto:xyz          10 XYZ @ 0.95 USD ; 1 XYZ = 0.1 ABC
    expenses:loss     0.5 USD
    crypto:abc         -1 ABC {10 USD} [2019-10-10] @ 9.5 USD

Then command ledger -f crypto.journal bal 'crypto:xyz' --flat --lots would say:

10 XYZ {USD0.95} [2019-10-11]

I'm not using the {= 12.34 USD} syntax on any journal

rainbyte avatar Oct 11 '19 06:10 rainbyte

I’ll respond to the rest later, but first this: wow, I had been considering use of {= } only in the buy transaction. Is it actually intended to be used in the sell transaction ? Only ?

I mean, you can use it anywhere (i tried), but if you use it on sell transaction you get a useful side-effect of auto-generated capital gains transaction.

adept avatar Oct 11 '19 09:10 adept

I use the {} syntax to write transactions in this way: .... I'm not using the {= 12.34 USD} syntax on any journal I see that you are generating profit/loss postings manually, which kinda removes the need for {= ... }.

adept avatar Oct 11 '19 09:10 adept

I didn't know that the {= ...} syntax could generate profit/loss postings, it is still a bit confusing for me.

From my pov the important part is being able to select an specific lot using {...} [...] @ ....

rainbyte avatar Oct 12 '19 06:10 rainbyte

It’s just my interpretation of what ony said, don’t take my word for it. It is an extremely confusing UX and learning curve, our job is to clean this up if we can

simonmichael avatar Oct 12 '19 06:10 simonmichael

I think that having multiple obscure ways to do the same thing is not so good.

Maybe manual profit/loss is not so difficult enough to justify a custom automatic syntax.

On the other hand, not having even a single way to select a lot could be a problem.


Maybe hledger can suggest writing profit/loss by hand upon finding the {= ...} syntax.

In that way the user learns the preferred way to represent that transaction.

rainbyte avatar Oct 14 '19 10:10 rainbyte

I track my securities with ledger this way:

commodity CZK
    format 1,000.00 CZK

2019/11/20 * Buy
    Assets:Bank:Securities  479 "CZ5027" {1.0424 CZK} [2019/11/20]
    Expenses:Bank:Excess       0.69   CZK
    Assets:Checking         -500.00   CZK
    Expenses:Bank:Rounding     0.0004 CZK

2021/02/01 * Sell
    Assets:Bank:Securities  -479 "CZ5027" {1.0424 CZK} [2019/11/20] @ 1.1021 CZK
    Income:TY2021:Capital Gains  (-(1.1021 - 1.0424)*479 CZK)
    Assets:Checking          527.91   CZK
    Expenses:Bank:Rounding    -0.0041 CZK

Ledger ignores @ 1.1021 CZK, so it's a comment for me. Sometimes I would put Capital Gains 28.5963 CZK as I don't like to have expressions in my ledger file.

When I need to do taxes, I query income via

$ ledger -f foo.ldg bal TY2021 --unround
        -28.5963 CZK  Income:TY2021:Capital Gains

(--unround is handy for foreign currencies; for tax purposes they should be converted into the domestic currency at the annual exchange, so 0.005 EUR were about 0.13 CZK in 2021)

dmage avatar May 09 '22 11:05 dmage