Inconsistent paredit selection between ignore form and other readers

[gaelrech]

Disclaimer: This seems rather similar to #2164, but since I was experimenting a bit with it too I thought on adding to it. If you feel like centralizing there, it's fine by me.

---

Going a bit into the story first, I've tried using the joyride example script for ignore form trigger button, but it had a different approach to what I wanted. What I wanted was something that would get the current form I'm at (not the Up form, which I think is what it does in the example script.

I was able to make it work with joyride, but doing this logic for the ignore form is considerably trickier than doing for a regular clojure reader because the behavior to which the Paredit Expand Selection works on ignore forms is considerably differente than on regular clojure readers.

To show some of the testings:

; Where: 
; - `|` means the current placement of the cursor

; Generic reader
(do foo #foobar b|ar) => #foobar bar
(do foo #foobar |bar) => #foobar bar
(do foo #foo|bar bar) => #foobar bar
(do foo |#foobar bar) => #foobar bar
(+ 1 #foobar |(- 1 3)) => #foobar (- 1 3)
(+ 1 #foo|bar (- 1 3)) => #foobar (- 1 3)
(+ 1 |#foobar (- 1 3)) => #foobar (- 1 3)
; if the reader is glued to the fn call, it remains consistent
(+ 1 #foo|bar(- 1 3))=> #foobar(- 1 3)

; Ignore form
(do foo #_b|ar) => bar
(do foo #_|bar) => bar
(do foo #|_bar) => foo
(do foo |#_bar) => foo
(+ 1 #_|(- 1 3)) => #_
(+ 1 #|_(- 1 3)) => 1
(+ 1 |#_(- 1 3)) => 1

Like I said, I got it to work via a joyride script by treating different cursor placements and moving around according to each situation, so its not blocking me in any way. Regardless, it caught my attention because I'm not sure if this is the expected behavior for the ignore form, and my script became considerably more complex than one for a triggerable generic clojure reader (I have another script for adding/removing a hashp tag which ended up considerably simpler).

So just wanted to share my findings here in case this may be relevant for any debuggings of the ignore form behavior.

[PEZ]

Thanks! Very good with those examples.

I've tried several times to get some kind of consistency into the handling of ignore forms, but it seems I never manage to frame the problem correctly.

[gaelrech]

I don't know if this helps, but the approach I end up taking was to kinda parse the existing word/symbol and afterwards whatever is on its borders to be able to identify the scenarios where the #_ form can happen (this is the script).

The code that does this expansion in Calva is this one?

[PEZ]

The expansion is done there. And it calls here, which may also play in. At the bottom of things there are primitives that treat reader tags special, and that I have tried to adapt so that they also treat ignores, but so far my attempts have failed.

[PEZ]

I now had a convo with Grok about an idea that has been cooking in my mind for a while: https://x.com/i/grok/share/6LgWaQxa9HaBgvRgcRWHsjeXa

WDYT, @gaelrech ? Would it make sense to you that Calva essentially treats ignores same as other reader tags? We would need to be able to treat it specially in certain cases, but anyway.

[PEZ]

I mean, I basically forced Grok to agree with me, so we can't really trust that judgment. 😀

[gaelrech]

Right, so I tried reading through most of grok stuff to capture the idea (grok seems really talkative 😆).

So, my understanding and personal view on it: yes, I think we should handle ignore form as any other reader.

The reason why I see it this way is because my understanding is that readers (specially in the sense of dispatchers and tagged-literals) are behaviors associated to forms, where it does not make sense to consider the form without this reader - or the reader without the form. After all, we are kinda saying "read this form with this specific behavior attached to it".

This is a bit obvious in a case like #inst "2024-01-01T00:00:00", because if we expand on the form without the reader, its just a string. But even though the ignore form has a wildly different behavior, it still sounds to me that this behavior is strongly attached to the form on the right of it, and it does not make sense to read (or at the very least, select) that form without the reader attached to it.

Another good point that could be made is that the ignore form is not only a reader by clojure definition, but a dispatcher. And if we look into how Calva handle other dispatchers, its the same logic as the tagged-literals we were mentioning before, in the sense that it does not separate the dispatcher symbol (#) from the form it is assigned to. E.g. if I expand selection for #(+ % 1)|, it will select everything including the dispatcher; it will not segregate between the dispatcher (#) and the special form itself (the parenthesis).

[gaelrech]

What is your view on how it should be handled? Cause like I said, it really does sound like a case of abstract interpretation 😆, so I'm curious if I'm not missing some other specific scenarios regarding it that might make a good case for having it being parsed differently

[PEZ]

One such case is mentioned in that lengthy convo. (Which gets to the actual point in the last exchange.) When Calva parses out argument vectors, it doesn't make sense to count ignored forms.

And of course the highlighter will need to treat ignored special. But I think we can eat this cake and have it if we tokenize ignores as reader and add some more metadata to the tokens. It's still not a small change, but I think it is manageable.

[gaelrech]

When Calva parses out argument vectors, it doesn't make sense to count ignored forms.

So, this is something that it mentions at some point there in your discussion with grok and that kinda makes sense to me. Like:

• If we are moving to the next/previous sexp, ignore the one with #_ form. Example, the scenario for lists where we have an ignore form
• If we are currently on a sexp with a #_, and we expand selection or something, it then considers the #_ reader attached to its relevant form.

But again, I think I might be biased on the scenario where I'm currently at where all I'm interested at is where the cursor lies.

Anyway I can help out with this task?

[PEZ]

When Calva parses out argument vectors, it doesn't make sense to count ignored forms.

So, this is something that it mentions at some point there in your discussion with grok and that kinda makes sense to me.

Just so we are on the same page. What this means is that this is a case where ignored forms are not like regular reader tags. This is functionality for when Calva shows parameter hints. You can trigger that by a command if you don't have it enabled as you type. Trigger Parameter Hints. If the parameter vector contains ignored forms, they should be ignored.

When moving and selecting it is a different matter. Then probably/hopefully we would not need to distinguish between regular reader tags and ignores.

[PEZ]

If you have Calva running in development (see the wiki) then you can experiment with this. We could have a screen share and I can introduce you to the code a bit. It is not exactly an easy first issue to tackle, but I can assure you that it is fun. 😄

Easiest way to get hold of me for a screen share is on the Clojurians Slack. I'm @pez there and hang around in the #calva channel.