Resolving threading macro inconsistencies

[digikar99]

This is a continuation of https://github.com/cl-library-docs/common-lisp-libraries/issues/3 after the original issue deviated too much.

Relevant discussion from the original thread includes the following:

• comments until and including https://github.com/cl-library-docs/common-lisp-libraries/issues/3#issuecomment-730671646
• https://github.com/cl-library-docs/common-lisp-libraries/issues/3#issuecomment-732013019 and the comment following it
• https://github.com/cl-library-docs/common-lisp-libraries/issues/3#issuecomment-732437773 and https://github.com/cl-library-docs/common-lisp-libraries/issues/3#issuecomment-732455315

I'm refraining from adding threading macros until the inconsistencies are clearly highlighted - in my mind, this should involve the construction of a test suite which the considered (let's restrict to arrows and arrow-macros) threading macros libraries should all pass. Feel free to suggest if there's a better way to do it.

---

As an example, here's an inconsistency that was picked up from arrow-macros-test, as on 10th October 2020:

(->> 1 (+ 2 3) #'1+ 1+ (lambda (x) (+ x 1)) (1+)) ;=> returns 10 with arrow-macros
;; The same expands to the following and fails to run with arrows:
(1+ (lambda (x) (+ x 1) (1+ (function 1+ (+ 2 3 1)))))

A manual evaluation expects the following sequence of operations:

(+ 2 3 1)
(function 1+ (+ 2 3 1))
(1+ (function 1+ (+ 2 3 1)))
(lambda (x)
  (+ x 1)
  (1+ (function 1+ (+ 2 3 1))))
(1+ (lambda (x)
      (+ x 1)
      (1+ (function 1+ (+ 2 3 1)))))

And this is consistent with arrows (and perhaps clojure?). On the other hand, arrow-macros seems to treat functions and lambdas specially.

Is that the only inconsistency? I do not know. I think a test-suite would answer this question best. Until then - or until a better way - I'd rather be willing to wait for a defacto standard to emerge out in a decade or half, and invest my time on other tasks.

[phoe]

On the other hand, arrow-macros seems to treat functions and lambdas specially.

It would be an extension of the Clojure macro syntax. (1+ (lambda ...)) does not make sense in CL, while (1+ (funcall (lambda ...) ...)) does.

On the other hand, this extension is backwards incompatible, as it prevents treating lambda forms as lists. For example, (->> 42 print (lambda ())) will not return the function (lambda () (print 42)) but will instead try to call (lambda ()) on (print 42), which is an argument count error.

Somewhat interesting tidbit: this seems consistent with the feature request that @mfiano mentioned in https://github.com/phoe/binding-arrows/issues/2. @mfiano Could you confirm that's the behavior you want?

[digikar99]

I myself am unopinionated and have no preference for the behavior.

But in the absence of the test of time (= wait for 5-10 years), who gets to decide (and why?) what goes in the defacto standard about this and what does not? At best, I feel these libraries are worth a mention in awesome-cl than here. arrow-macros is listed; is there some case where arrows pass but arrow-macros fail so that it'd be worth a mention there? I'm rather looking forward to either (i) a test suite, or (ii) another way to find the inconsistencies, or (iii) waiting for 5-10 years.

[Zulu-Inuoe]

I think breaking some Clojure compatibility is inevitable, as having the separate function namespace makes some nice things in Clojure's impl be not great in a CL one.

I think it's reasonable to have the macro understand lambda and function, but I'd argue that going with (what I perceive) to be the simplistic intent, we'd 'solve' this problem like:

(-> val
  ((lambda (x) (* x x))) ; Square the value
  (foo)                  ; Call 'foo' on the result
  (funcall bar)          ; Call the value of the variable 'bar' on the result of that
  foo                    ; Call 'foo' on that again
  print)

this preserves the simplicity of the impl & grokking of the reader, while still allowing you to do all the things (I believe) you'd want to do, while still preserving the 'atoms mean single-arg calls'

[phoe]

I'm rather looking forward to either (i) a test suite

The test suite from binding-arrows is free to be adapted as appropriate. The macroexpansion tests will quite obviously need to be removed because they are highly implementation-specific , but the other tests should be good to go.

More tests can also be added to test for e.g. the lambda scenarios mentioned above by @Zulu-Inuoe. (And, honestly, I vote for those - using a list containing a lambda expression solves the contextual issue of whether one should splice things into a lambda form, e.g. for forming closures, or whether the lambda form is already complete and is meant to be funcalled on something.)

[Harleqin]

I fear that if I start to recognize forms that are “meant” to be operators, I won't find an end.

I want the expansion be as simple as possible, and based on a very limited set of principles. One of these principles is that the arrows operate on lists, wherever they come from and whatever is in it (yes, I know, diamonds are already an exception, but they come from within the library). Only if a step is not a list is it wrapped into a list.

I don't know, for example, what would have to be done about syntax quotes if quotes get special treatment.

I agree with letting this simmer for a few years. I guess this is just like with testing frameworks: to each his own.

[phoe]

Note that as of https://github.com/hipeta/arrow-macros/issues/3#event-4133476273 it seems that arrow-macros no longer implicitly evaluates its arguments.