[💰 20$ bounty] Pivoting over multiple tags: Making `--pivot` much more powerful

[nobodyinperson]

Hi everyone,

Tags in combination with pivoting add an incredible amount of depth to an hledger journal. They effectively enable looking at the transactions in an arbitrary amount of different layers of categorization.

I have an idea to push this even further while maintaining backwards-compatibility.

👀 Current --pivot behaviour

As far as I understand (and the docs state it), --pivot=TAG causes each account name to be replaced by the (possibly non-existant) value of the given TAG. So basically pivoting is actually just a string manipulation action at a time where all tags attached to the transaction are known.

Currently, this string manipulation action can only do one thing: replacing the existing account name with the value of a single tag.

✨ Proposed new --pivot behaviour

I propose to make pivoting much more powerful by adding new simple string manipulations. A tag name looks like [[:word:]-]+, so pretty much all special characters are free to have a special meaning in --pivot, for example:

⤵ --pivot=TAG1|TAG2|TAG3 - OR - fallback to other tag value

Replace the account name with value of TAG1, but if this one is empty or doesn't exist, replace it with TAG2's value. If TAG2 is equally empty, try TAG3.

This is useful to fill empty space when pivoting in hledger register over a description/notes tag (be it the built-in desc or another tag one uses for additional info like mynotes): hledger register checking --pivot='mynotes|desc'. A CSV import for example can only really set the desc-tag, which for real-world-accounts will be something like the gibberish transaction details, but I find myself to often add additional info in human-readable form to the transactions for nice presentation in hledger register. This ”or” operator could be used to fall back to certain tag values.

This operator would have the lowest priority, i.e. it would be evaluated last.

➕ --pivot=TAG1:TAG2 - concatenation with colon

Replace the account name with the given tags' values concatenated with a colon :.

This adds the possibility to create nested account structures on-the-fly. Especially if there were an auto-generated tag containing the original account name (is there already?) like acct, you could append/prepend additional nesting levels contained in tags. For example if you tag specific expenses with a concerns: Bob or concerns: Alice tag depending on whom it concerns, you could then quickly get an overview over everyone's concerning balances with a single command hledger balance --pivot=acct:concerns (or hledger balance --pivot=concerns:acct, depending on how you'd want to see it). Currently, you'd have to issue one command per concerned person to get the same information (hledger balance tag:concerns=Alice, hledger balance tag:concerns=Bob, ...).

➕ --pivot=TAG1+TAG2+TAG3 - basic concatenation

Like the above, but the (non-empty) tag values concatenated with a different character. I guess a space would be fine, but I can imagine people wanting to use a custom concatenation string (like , or - or ; , etc...), though I am not yet sure what syntax to use for that (maybe --pivot=TAG1(, )TAG2(, )TAG3?). A custom concatenation string would then also include the concatenation with colon.

This is useful like the fallback “or” above, but if you want to see all of the tags values in hledger register.

This is just a couple of examples, I see a lot of possibilities (like regex substitution, mapping values, etc...).

✏ Syntax

Maybe the simple syntax proposed above is a little too limited for future improvements. It is concise and simple to read though and rather probably simple™️ to implement (should be ”just” string parsing and manipulation). As parentheses are also not allowed in tag names, I could also imagine a ”functional”-style --pivot syntax:

• --pivot=or(TAG1,TAG2,TAG3) fallback
• --pivot=concat(':',TAG1,TAG2,TAG3) colon concatenation
• --pivot=concat(', ',TAG1,TAG2,TAG3) custom concatenation
• --pivot=sub(TAG1,/bob/alice/) regex replacement
• --pivot=or(TAG1,concat(', ',TAG2,TAG3)) nesting calls

This syntax is admittedly more involved but much more future-proof as adding an arbitrary amount of new features will be possible.

⏪ Backwards-Compatibility

As tag names are only allowed to contain something like [a-zA-Z0-9-] and no special characters, I can't imagine someone relying on special characters in --pivot. So, any new syntax with special characters we introduce to --pivot won't break backwards-compatibility.

What do you think? I bet people will come up with amazing ideas to use this new functionality. I for sure would love to use it.

Cheers,

Yann

[Xitian9]

I don't use pivoting, so I can't comment on the proposed implementation, but it sounds pretty cool to me.

[nobodyinperson]

Apparently, IIUC, part of this proposal was silently implemented in #2050 by @glguy. Would you be interested in taking a look at the other ideas to eventually collect the bounty? 🙂