hledger
hledger copied to clipboard
simonmichael
[Overview issue] Make hledger good for investment tracking
This is an overview/starting point for the project of making hledger good (best-in-class, effective, pleasant) for tracking investments. I'll keep this summary updated.
Related docs
- https://hledger.org/track-investments.html
- https://github.com/adept/full-fledged-hledger/wiki/Investments-easy-approach
- Zoran Zaric: Tracking Investments in Lots with Hledger
Ledger
Ledger Tutorial:
Principles of Accounting with Ledger:
- Currency and Commodities
- Naming Commodities
- Buying and Selling Stock
- Fixing Lot Prices
- Complete control over commodity pricing
Transactions:
- Posting cost
- Explicit posting costs
- Primary and secondary commodities
- Posting cost expressions
- Total posting costs
- Virtual posting costs
- Commodity prices
- Total commodity prices
- Prices versus costs
- Fixated prices and costs
Beancount
- How Inventories Work "how we accumulate commodities and match sales (reductions) against accumulated inventory contents"
- Inventory Booking (proposal) "The problem of inventory booking, that is, selecting which of an inventory’s trade lots to reduce when closing a position, is a tricky one"
- Trading with Beancount "explanation of trading P/L and worked examples of how to deal with various investing and trading scenarios"
- Stock Vesting in Beancount "explains the vesting of restricted stock units in Beancount"
- Exporting Your Portfolio "how to export your portfolio of holdings from Beancount to a Google Finance portfolio"
beans
- https://sboehler.github.io/beans/ Converts to a desired commodity: "derives arbitrary commodity conversions between all commodities, as long as there is at least one path (possibly over several links) from one commodity to another. For example, conversion from SomeStock in CHF is possible when prices of SomeStock in USD and CHF in USD is available. Inverted rates are derived automatically." Generates revaluation transactions (calculates unrealized gains) when reporting in a different commodity: "The journal now not only shows explicit inflows and outflows, but also valuation corrections due to price changes at the specific date they occurred." "When valuating the balance in a certain reference commodity, beans accounts for changes in valuation using the Equity:Valuation account. It reflects gains and losses over time with respect to the reference currency." "applies valuation directly at the transaction level. For example, if you choose to display a balance at market prices, beans will transparently insert valuation transactions to reflect gains and losses due to price changes" "Depending on whether one wants to convert commodities at cost or at market value, beans will add balancing accounts automatically." "offers either no valuation or valuation at market prices. Valuation at cost is yet to be implemented." "applies valuation at the level of transactions, which seems unique amongst plain-text accounting tools" --Transaction-level valuation. 1 Converts all postings to the target commodity using market prices at target valuation date (there are no explicit transaction prices) 2 At each market price change, for each asset/liability account, calculates the balance and adds a revaluation transaction (transfer to/from Equity:Valuation).
Elsewhere
- https://www.mscs.dal.ca/~selinger/accounting/tutorial.html
- https://www.double-entry-bookkeeping.com/marketable-securities/trading-securities/
- https://www.reddit.com/r/plaintextaccounting/comments/1f92hqn/hledger_balancing_transactions_that_track_the/
Related issues
- #82 generate commodity revaluation transactions ? (2013)
- #131 Allow reporting in an arbitrary commodity (Ledger's -X, or something like it) (2013)
- #258 better ledger compatibility (2015)
- #299 Ignore incompatible ledger journal syntax (2015)
- #329 periodic value reports should use the market prices of each period (2016)
- #377 generate revaluation transactions when transaction price changes ? (2016)
- #488 How to realize capital gains or set lot price? (2017)
- #624 Tracking investments in hledger (2017)
- #629 Docs investment guide (2017)
- #648 Balance in only USD (2017)
- #758 Treacherous multi-currency auto-inference (2018)
- #858 Stocks transfer leads to unexpected cost values (2018)
- #1016 Support Ledger-style manual lot matching syntax (2019)
- #1017 Lots: support stock split / repricing correctly (2019)
- #1018 Ledger-style balance --lots report (2019)
- #1019 Ledger-style balance --unrealized report (2019)
- #1020 Average commodity cost report (2019)
- #1021 Ignore unknown syntax in postings (2019)
- #1022 Track & show deposited lots (2019)
- #1023 Support/check lot movements (2019)
- #1029 Report unrealized capital gains/losses (2019)
- #1073 Metadata Syntax Required (2019)
Things we need
docs/examples for common investment-related tasks, using existing features [20%]
- #629 Docs investment guide
- #624 Tracking investments in hledger
- #648 Balance in only USD
market price fetching (latest prices, prices over a period) [10%]
show value at different dates (period end, custom date, maybe transaction date) [50%]
- #329 periodic value reports should use the market prices of each period
show value in specified commodity [10%]
- #131 Allow reporting in an arbitrary commodity (Ledger's -X, or something like it)
show revaluations/unrealized capital gains [10%]
- #82 generate commodity revaluation transactions ?
- #377 generate revaluation transactions when transaction price changes ?
- #1019 Ledger-style balance --unrealized report
- #1029 Report unrealized capital gains/losses
clear price inference & propagation behaviour [40%]
- #758 Treacherous multi-currency auto-inference
track & show lot deposits [10%]
- #1018 Ledger-style balance --lots report
- #1022 Track & show deposited lots
check, execute valid lot movements (FIFO etc.) [0%]
- #858 Stocks transfer leads to unexpected cost values
- #1016 Support Ledger-style manual lot matching syntax
- #1017 Lots: support stock split / repricing correctly
- #1023 Support/check lot movements
show realized capital gains [0%]
- #488 How to realize capital gains or set lot price?
compatibility with ledger/beancount where reasonable, better compatibility docs [40%]
- #258 better ledger compatibility
- #299 Ignore incompatible ledger journal syntax
- #1021 Ignore unknown syntax in postings
- ledger valuation flags
other ?
- #1020 Average commodity cost report
Hi. I'd like to underscore the significant number of important insights contained in this doc (from 2014): https://beancount.github.io/docs/a_proposal_for_an_improvement_on_inventory_booking.html
I think the discussion on the shortcomings with Ledger, the shortcomings with Beancount (up until 2016), and the requirements section are well worth considering. I think the requirements for hledger are largely the same.
This document (from 2016) provides details of the revised inventory system that beancount adopted, which reveals both a few trade-offs, as well as one or two simplifications adopted. But I think the requirements section of the 2014 document is crucial to understand both the mistakes to avoid and what needs to be done to provide the best support for investment tracking.
As an update, here is Beancount's proposed redesign of its inventory (lots) booking system, which involves some simplification of the syntax as well: https://docs.google.com/document/d/1H0UDD1cKenraIMe40PbdMgnqJdeqI6yKv0og51mXk-0/edit#
This 2021 document should ideally be read after the documents from 2014 and 2016 that are linked here: https://github.com/simonmichael/hledger/issues/1015#issuecomment-988118311
I think it would be useful for the hledger, beancount, and ledger devs to discuss this and seek to converge on similar syntax to ensure greater interoperability in the PTA ecosystem.
It would be great if lot tracking syntax could converge for greater interoperability.
However, at least in 2017 when I last looked at Beancount in any depth, it didn't support situations where a lot basis for an asset or commodity purchased with one currency and sold for another could be tracked in either of the two currencies---at least not situations where the cost basis must be tracked (which is unfortunately the case with bitcoin and sometimes with physical commodities)[1]---as Martin correctly says that most people don't think about currency lots when they're spending between currencies.
I need to look at the Selinger multicurrency trading accounts essay again [2]. I think that could work, but if it looses the lot data, it fails to serve the needs of people who need to report gains/losses on bitcoin and other cryptocurrencies. And in that space, people will trivially move between far more than just two currencies. I don't remember if or how Selinger tracks lot data, or if that's simply opaque in his method.
I think Plain Text Accounting (PTA) needs a syntax for tracking lots that can be toggled on or off (as Ledger does) but combined with the ability to programmatically add to and reduce lots according to the user's choice of FIFO, LIFO, average cost, or arbitrary matching. (At least in the USA, my understanding is that if you have the records, specific lot matching is allowed; I think that would allow writing algorithms to minimize tax liability).
What kind of writing and/or examples would be helpful to more experienced programmers than myself to clearly lay out the use cases that would be helpful in PTA regarding lots and cost basis?
jkepler
[1] https://groups.google.com/g/beancount/c/5lLY4nzSNtA/m/6AiU2Z0TCQAJ
[2] https://www.mscs.dal.ca/~selinger/accounting/tutorial.html ------- Original Message ------- On Wednesday, August 17th, 2022 at 9:08 PM, Pranesh Prakash @.***> wrote:
As an update, here is Beancount's proposed redesign of its inventory (lots) booking system, which involves some simplification of the syntax as well: https://docs.google.com/document/d/1H0UDD1cKenraIMe40PbdMgnqJdeqI6yKv0og51mXk-0/edit#
This 2021 document should ideally be read after the documents from 2014 and 2016 that are linked here: #1015 (comment)
I think it would be useful for the hledger, beancount, and ledger devs to discuss this and seek to converge on similar syntax to ensure greater interoperability in the PTA ecosystem.
— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.Message ID: @.***>
On Aug 25, 2022, at 20:20, Joel @.***> wrote: It would be great if lot tracking syntax could converge for greater interoperability.
However, at least in 2017 when I last looked at Beancount in any depth, it didn't support situations where a lot basis for an asset or commodity purchased with one currency and sold for another could be tracked in either of the two currencies---at least not situations where the cost basis must be tracked (which is unfortunately the case with bitcoin and sometimes with physical commodities)[1]---as Martin correctly says that most people don't think about currency lots when they're spending between currencies.
I need to look at the Selinger multicurrency trading accounts essay again [2]. I think that could work, but if it looses the lot data, it fails to serve the needs of people who need to report gains/losses on bitcoin and other cryptocurrencies. And in that space, people will trivially move between far more than just two currencies. I don't remember if or how Selinger tracks lot data, or if that's simply opaque in his method.
I think Plain Text Accounting (PTA) needs a syntax for tracking lots that can be toggled on or off (as Ledger does) but combined with the ability to programmatically add to and reduce lots according to the user's choice of FIFO, LIFO, average cost, or arbitrary matching. (At least in the USA, my understanding is that if you have the records, specific lot matching is allowed; I think that would allow writing algorithms to minimize tax liability).
What kind of writing and/or examples would be helpful to more experienced programmers than myself to clearly lay out the use cases that would be helpful in PTA regarding lots and cost basis?
Hi Joel,
thanks for this, it's useful. I don't follow exactly, could you give a minimal example or two of those situations not supported by Beancount ?