qi
qi copied to clipboard
drym-org
Compact closure syntax and generalizations
What version of Racket are you using?
v8.6
What program did you run?
Welcome to Racket v8.6 [cs].
> (require qi)
> (~> (1 2 3) (clos +) (_ 4))
10
> (~> (1 2 3) (+) (_ 4))
compose: contract violation
expected: procedure?
given: 0
argument position: 2nd
[,bt for context]
>
What should have happened?
Welcome to Racket v8.6 [cs].
> (require qi)
> (~> (1 2 3) (clos +) (_ 4))
10
> (~> (1 2 3) (+) (_ 4))
10
>
I think currying a function directly is similar to using clos, but to my surprise Qi seems to avoid this usage intentionally, is there any special reason?
Analysis
(summarized from comments)
Current
(~> (1 2 3) (clos +) (_ 4)) ;=> 10
(~> ('(1 2 3)) (clos list-ref) (_ 1)) ;=> 2
(~> (5) (clos *) (fanout 3) compose apply) ;=> 125
Proposal
(~> (1 2 3) (+) (_ 4)) ;=> 10
(~> ('(1 2 3)) (list-ref) (_ 1)) ;=> 2
(~> (5) (*) (fanout 3) compose apply) ;=> 125
Benefits
- More succinct
- Feels intuitive to some --- as the signature of the syntax is the introduction of an extra pair of parens, it could be thought of as the contained flow being "packaged for later use". Closures are also in a way "meta" in relation to the main flow. The original flow is a description of how to compute something. A closure seems to be a description of how to compute something that is embedded in and computed by the original description. This increase in meta-level seems to mirror a similar quality of parentheses in human languages, which could be nested, each higher/deeper level of nesting of which qualifies the lower-level parenthetical and ultimately the unparenthesized [or is it just me? - Sid].
- May be preferred by expert users
(clos f)is a native way to generate a flow as a value, instead of a roundtrip to Racket(gen (☯ f)), and arguably,(f)is more suggestive thanclosfor this purpose
Risks
- Confusion with partial application. E.g. starting with
(~> (1 2 3) (+ 4))and then removing the4to get(~> (1 2 3) (+))changes it from a partial application to a closure, and that may be confusing (resolution would be to remove the parens entirely, if the addition is to be performed) - More confusing to beginners
(f)is not a standard symbol for a closure (but Haskell's "section of an infix operator" syntax is one precedent)
As Yet Undetermined
Case where all arguments are supplied
(~> (3) (sqr))
Possible answer: As (f) is always a closure and equivalent to (clos f), this should produce a closure that accepts 0 additional arguments to produce a result:
(~> (3) (sqr) apply) ;=> 9
How to notate clos in identifier form?
(define-flow recurse
(~> (group 1 _ (~> X clos))
feedback))
(recurse 3 5 *)
Possible answer #1: Perhaps we should use () for this after all?
(define-flow recurse
(~> (group 1 _ (~> X ()))
feedback))
(recurse 3 5 *)
Possible answer #2: Do not notate it specially (i.e., just use clos in this case, as before).
Supporting templates
(not needed for initial version, as long as we are convinced that adding it in the future would not lead to any conflicts. This is just to make sure the design is future-proof)
Differentiate closed-over inputs from application inputs using some syntax:
(~> ("a" "c") ((string-append _ ^ _)) (_ "b")) ;=> "abc"
Perhaps support indexes instead of just positional inputs:
(~> ("a" "c") ((string-append _₁ _₃)) (_ "b")) ;=> "abc"
Resolving consistency with other syntax
- Ideally, [the meaning of
(f)] would also give()different behavior from(__)to avoid redundancy. - the currying perspective inherently disallows () (as curry requires at least one argument).
Alternatives to the proposal
None considered yet.
I think (+) is a bit of a special case of a more general rule (writing + here would have the expected outcome).
Consider the following:
> (define ((f a) b) (+ a b))
> (~> (1) (f 2))
3
When the flow is an application like (f 2) or (+), I think Qi tries to immediately resolve if the number of provided arguments is a valid arity for the proc. With (define (f a b) (+ a b)) instead, it does the currying.
Plus can be 0-arity, so it is immediately resolved. Another option (if you want to write parentheses) is (+ __).
It is unclear to me if this behavior comes from Qi or fancy-app, and I haven't looked at the code to see if I'm right, but I recall vaguely a discussion about this.
I seem to have forgotten about the (_ 4) part of your flow: I do not believe Qi will produce flows as (intermediate) results unless directed to (either by partially-applying a curried function, or using clos as you mentioned, etc.). In (~> (1 2 3) + (_ 4)), the + flow consumes the values (1 2 3) and produces 6, which cannot be applied to 4. Can you think of an unambiguous and unsurprising rule that would differentiate between "partially apply this flow" and "fully apply this flow"? Arguably, clos does that in this case.
It is unclear to me if this behavior comes from Qi or fancy-app, and I haven't looked at the code to see if I'm right, but I recall vaguely a discussion about this.
The behavior I expect comes from curry. In my understanding, Qi expands flo in this way:
(~> (1) (f 2))
((curry f 2) 1)
So from this perspective, (op) should be expanded to (curry op):
(~> (1 2 3) (+))
((curry +) 1 2 3)
(~> (1 2 3) (+) (_ 4))
(((curry +) 1 2 3) 4)
In this way Qi naturally gets the function similar to clos. And this change isn't difficult, just change (natex prarg ...+) to (natex prarg ...) in flow/compiler.rkt(https://github.com/countvajhula/qi/blob/main/qi-lib/flow/compiler.rkt#L214) .
But I found that it's forbidden in tests file, so I'm curious if there is any special reason.
Not really a response, more for the record
> (expand #'(~> (1 2 3) (+) (_ 4)))
#<syntax:Library/Racket/8.6/pkgs/qi-lib/on.rkt:24:5 (#%app (#%app compose (lambda (_1) (#%app _1 (quote 4))) (#%app +)) (quote 1) (quote 2) (quote 3))>
> (expand #'(flow (~> (+) (_ 4))))
#<syntax:Library/Racket/8.6/pkgs/qi-lib/flow/compiler.rkt:96:7 (#%app compose (lambda (_2) (#%app _2 (quote 4))) (#%app +))>
> (expand #'(~> (1) (f 2)))
#<syntax:Library/Racket/8.6/pkgs/qi-lib/on.rkt:24:5 (#%app (#%app compose (#%app curryr f (quote 2))) (quote 1))>
https://github.com/countvajhula/qi/blob/main/qi-test/tests/flow.rkt#L573
Ah, right: curry does the hard work of the arity stuff I mentioned earlier. You said this, and I missed it.
And the specific issue you see is from not currying when a function has no arguments provided. Interesting.
The discussion above helps clarify the issue, and I don't actually recall if I had specific reasons in mind for this. But luckily, I seem to have gone through the trouble of documenting the behavior.
There's one explanation here: There's No Escaping esc
The fact that this is about how Qi behaves other than you might expect in this case would seem to be a sign that @NoahStoryM 's suggestion may be more natural here 😄
There's also some related discussion here: Using Racket to Define Flows, but I don't think this case would be affected.
The first link mentions that interpreting this as partial application "would not be useful," but maybe with the (_ ...) application syntax the situation has changed, and your example above does seem like a reasonable use of the currying behavior.
It would be helpful to come up with more examples of cases that would be affected by this change (the docs example could be a good starting point). That would give us more confidence that there aren't any useful cases that would be negatively affected.
As of Qi 4, partial application syntax expects at least 1 argument pre-supplied, so that (+) on its own is currently treated as a syntax error (no rule in the expander would match).
We could have a new "ad hoc" expansion rule (similar to this rule) that rewrites (f) to (clos f) to get the behavior proposed here. (f) seems a pretty simple and intuitive way of indicating a closure...
This might be because I'm used to the old behavior, but every time (flow (+)) signals an error I'm surprised :) That said… it is an "unused syntax." OTOH, inserting clos when we could write close already is the kind of syntax sugar I'm not sure is actually clarifying code.
Are you suggesting that close would be more clear than clos? I think at the time, clos was selected as short for "closure" rather than close/"close over".
I feel now in retrospect that a closure may be such a basic idea that, like partial application, having a name just makes it more complicated and confusing (e.g. (f a b) vs (curry f a b)), and from that perspective, having a compact syntax like (f) to designate it seems appealing to me. Of course, adding (+) syntax needn't mean we get rid of clos or even introduce close.
Sorry, close was a typo. The idea was mostly to avoid a possibly confusing syntactic sugar.