Concealer Issues Thread

[vhyrro]

Hey! This thread exists to basically catch all of the bugs that the new concealer implementation may be causing. Post your problems here! Aand here's the longer version of the above TL;DR:

Good News

Let's start with the great news: we've massively improved the performance of Neorg's concealer.

I wish I were joking but it's gotten to the point where the bottleneck is no longer Neorg but the code that performs treesitter highlighting in neovim 😂. I did a bunch of testing on over 100k+ loc Neorg documents and after doing :TSBufDisable highlight to prevent the lag that TS was causing things still manage to work smoothly on my machine.

The bad news?

It's definitely not bug free.

Now that our concealer does updates incrementally there's definitely a chance that it'll miss a line or two when concealing in certain circumstances. If you have a problem with the concealer, or something doesn't work as intended, please post your issue here!. I'll be trying to fix them as I go.

A feature was dropped...

With the dawn of the more performant concealer we've temporarily dropped the TODO item completion level feature - you know, the one that displayed (0 of 3) [0% complete] on headings when they had a TODO list under them? It's gone for now.

I suppose I owe you an explanation as to why we've decided to do this. Before our update the concealer would reparse the entire buffer every single time you changed some text or left insert mode. As you can imagine this made it really slow. Just to clarify we didn't want to make it this bad-performing in the first place, but there were certain bugs in nvim_buf_attach that prevented us from properly doing incremental updates then.

The thing is now that everything is happening incrementally we cannot properly recalculate TODO item counts without requiring more context than a single line - and requiring more context greatly slows down the concealer again.

We'll be working hard on trying to bring it back, but we really wanted to push these new changes to you guys without having you wait another week.

UPDATE: Forgot to update this thread but that has been reimplemented now as well. If you have problems with the TODO item completion levels then be sure to let me know here too!

Hope you understand!

[vhyrro]

First bug I found myself: markup concealing was not autotriggering on files lower than 1250 lines, that's been fixed in bfe3056feb91d610c8224e8af1923e7c45676fe7 now :)

[max397574]

Concealing on wrapped links is broken: wrapped:

not wrapped:

[danymat]

The : on https://is concealed:

Without conceal:

[mrossinek]

Concealing on wrapped links is broken:

This is potentially tricky with how things are done currently in the concealer. I will need to take a look at Vhyrro's rewritten code though since there have been quite some changes. The problem will be that we need to iterate the char in the TS range one by one rather than just rely on the length of the node.

[mrossinek]

The : on https:// is concealed:

Yes, I have encountered this too. The problem is that the : get's recognized as a link_modifier. I will try if I can fix the TS parser to disregard single link_modifier nodes (i.e. if they do not occur in pairs). That should fix this.

On another note: those screenshot of yours are still using Markdown-like link syntax!

[mrossinek]

Concealing on wrapped links is broken:

This is potentially tricky with how things are done currently in the concealer. I will need to take a look at Vhyrro's rewritten code though since there have been quite some changes. The problem will be that we need to iterate the char in the TS range one by one rather than just rely on the length of the node.

A followup question to this @max397574: is this soft-wrapped? If so, I don't think there is a way for us to handle this. Neovim itself would have to handle wrapping extmarks..

[danymat]

On another note: those screenshot of yours are still using Markdown-like link syntax!

Right, lmao

[max397574]

Concealing on wrapped links is broken:

This is potentially tricky with how things are done currently in the concealer. I will need to take a look at Vhyrro's rewritten code though since there have been quite some changes. The problem will be that we need to iterate the char in the TS range one by one rather than just rely on the length of the node.

A followup question to this @max397574: is this soft-wrapped? If so, I don't think there is a way for us to handle this. Neovim itself would have to handle wrapping extmarks..

Not sure what soft-wrapped is it's just a line longer than the screen is width

[mrossinek]

The : on https:// is concealed:

Yes, I have encountered this too. The problem is that the : get's recognized as a link_modifier. I will try if I can fix the TS parser to disregard single link_modifier nodes (i.e. if they do not occur in pairs). That should fix this.

@danymat This should be fixed now. Can you please update your TS parser to verify?

[danymat]

@danymat This should be fixed now. Can you please update your TS parser to verify?

It is indeed ! thanks !

[mrossinek]

Not sure what soft-wrapped is it's just a line longer than the screen is width

See :help 'wrap'. If you use :set nowrap then a line which is longer than your window width will not get wrapped but instead extends to the right (you then need to scroll horizontally to see the rest).

[vhyrro]

Seems to be a problem with generic line breaks as well.

This is {a
test}[with custom text]

[mrossinek]

Seems to be a problem with generic line breaks as well.

Yep. That's the original problem I discuss here: https://github.com/nvim-neorg/neorg/issues/292#issuecomment-1005205020

[karlc1]

I just started trying neorg today, so I just assume that this is unintended behavior.
When I create a link, the link marker is still being rendered, but its foreground color is set to the normal bg color, so it looks like a bunch of whitespace before the custom text (but i can still yank the text or see it if highlighted in visual mode)

An inline link {| my marker}[with custom text].

looks like this in the editor

An inline link                        with custom text.

[max397574]

I think I know what's the problem could you send your neorg config and a screenshot?

[karlc1]

I cannot take a screenshot at the moment, but it really looks just as described; like a bunch of whitespace of equal length as the marker.

Here is the config https://github.com/karlc1/nvim/blob/main/lua/plugins.lua#L691

[vhyrro]

Just a quick heads up that i've managed to fix some bugs related to the link concealing as discussed earlier above.

Those should now be highlighted properly :)

(also @karlc1 if you haven't already then updating Neorg should fix your problems you've described above) :)

[esquires]

Thanks for the fixes! Here is one more. For the line

Here is {:inbox:}[my link]

with markup_preset set to brave I get the following:

If I then enter insert mode on a line above the link by hitting o (not i and not on the link itself) it seems to get fixed:

However, if I then enter insert mode on the link line it gets reset to the first markup.

[karlc1]

@vhyrro I use neovim nightly and neorg on the latest tag, and i still don't think that the links look right.

This line test: {:some_file:}[link!].

Looks like this: test:       some_file       link! (with the 'some_file word in faded color')

I would expect it to look like this: test:   link!

[max397574]

Check out the markup preset here: https://github.com/nvim-neorg/neorg/wiki/Concealer This is expected

[karlc1]

I see, did not realize this. The 'brave' preset works as I want it to, thanks for the clarification

[esquires]

I see, did not realize this. The 'brave' preset works as I want it to, thanks for the clarification

Interesting it works on your end with the brave setting. Here is how the brave setting works for me for test: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠{:some_file:}[link!].

Can you post your config? Perhaps you have a setting that I don't that is making it work.

[max397574]

this is because stuff breaks with the joiner char that's why we introduced the other presets

[esquires]

this is because stuff breaks with the joiner char that's why we introduced the other presets

Thanks for the clarification. is it fixable or is brave not intended for use?

[karlc1]

@esquires It does look like that for me if I enter insert mode on a line with a link, and then go back to normal mode but stay with the cursor on the same line. However, when I move the cursor to a different line it looks correct again.
I have a pretty basic config i believe https://github.com/karlc1/nvim/blob/main/lua/plugins.lua#L680

[esquires]

Thanks!

[vhyrro]

From what we've found it seems to be very inconsistent across different distros, window managers and configs. We don't really know the source of the problem as to why "brave" doesn't work for some, and because of that it's disabled by default. I use kitty and it breaks there, but mrossinek also uses kitty and it doesn't break for him :sweat_smile:. "brave" uses the word-joiner char to quite literally fake concealing and cause of that things go weird. It's not a Neorg config thing either, we've already tested that :P

[AndOrangutan]

Are there any known working terminals? This isn't the first time I have had issues on Kitty.

[mrossinek]

Are there any known working terminals? This isn't the first time I have had issues on Kitty.

As vhyrro mentioned in the comment before yours, I'm having no issues on Kitty. We ave not been able to track down what differs. My dotfiles are in https://gitlab.com/mrossinek/dotfiles in case you want to go poking around though :+1:

[karlc1]

Link concealing is broken for me today with the brave preset, even though it worked yesterday and I don't think I changed anything.. Are there any possible alternative solutions to using the joiner char for handling this? I could try looking into it myself since this feels like a deal breaker for me