Underscores.jl
Underscores.jl copied to clipboard
Interpolate `do` syntax closures into single _
This allows a single _
to be treated as the slot into which the closure created by do syntax is inserted.
It's somewhat overloaded in that @_ map(_, xs)
normally means the _
is the identity, while @_ map(_, xs) do ...
would mean the same as normal do syntax without the _
, but conceptually we're marking a "slot of map" to receive a closure so it could be acceptably consistent.
Also, it seems likely to be extremely convenient.
A more realistic example would be a function which takes multiple functions as arguments (eg mapreduce(f, _, xs) do ...
) allowing the second argument to be given with the do
block.
CC @tkf @masonprotter and anyone else interested — thoughts? Does this seem acceptably consistent?
Closes #4 by replacing it
Personally, I’m tempted to say this should just be a separate macro from @_
. Its not horribly inconsistent, but it does make it so that you need to understand more context in order to see what @_
is doing. But it’s also not that bad either.
I think it'd be nice to have this if this is compatible with everything else. Otherwise, I think I agree with @MasonProtter that separating it out from @_
may be a wise choice.
In particular, I wonder how it interacts with |>
. I'm guessing that this works?
@_ function ((i, d),)
x = get(d, :a, nothing)
x === nothing && return nothing
y = get(x, :b, nothing)
y === nothing && return nothing
z = tryparse(Float64, y)
z === nothing && return nothing
return (i, z)
end |> mapreduce(__, _, pairs(data); init = nothing) do a, b
a === nothing && return b
b === nothing && return a
a[2] < b[2] ? b : a
end
(I'm expecting function ((i, d),)
to be fed to the first argument of mapreduce
and do a, b
to the second argument.)
I'm guessing that this works?
Yes that should work (but I had a bug, oops. Fixed now).
It's interesting that you can place the closures on both sides :laughing: Certainly confusing to look at IMO, but no reason to disallow it.
julia> @_ function (x)
@show x
x^2
end |> mapreduce(__, _, [1,2,3]) do x,y
x+y
end
x = 1
x = 2
x = 3
14
Overall I think supporting do
is certainly frustrating in Underscores
because it's non-obvious how to do it in a visually straightforward but entirely consistent way. This is the best I've found so far though it borders on being a bit featury.
this should just be a separate macro from
@_
My only objection to that is that macros don't really compose naturally when they act on the same type of syntax for the same reasons. So in some ways it's tempting to build all the closely related behaviors together into one macro.
Don't worry I won't do this in important code :laughing:. It's just that it was the first example I could think of to use __
and _
while only using Base
function.
I guess more practical example is SplitApplyCombine.groupreduce
:
@_ ... |> groupreduce(x -> x.key, _, __) do a, b
a.value < b.value ? b : a
end
BTW, it looks like this does not work?
@_ collect(Filter(_), -3:3) do x
x > 0
end
# => collect(Filter(x -> x > 0), -3:3)
I guess allowing this introduces more inconsistencies, though. I wondered a bit if it makes sense to add do
block to the list of "scope delimiters" [*1] |>,<|,∘
. This way, I guess
@_ collect(Filter(__), -3:3) do x
x > 0
end
would work. But then the groupreduce
example becomes
@_ ... |> itr -> groupreduce(x -> x.key, __, itr) do a, b
a.value < b.value ? b : a
end
which is not nice...
That said, extending the crazy thought, I wonder if it makes sense to use the number of _
to denote how much you escape to the outer scopes:
-
_
: all but one call up to a delimiter -
__
: all calls up to a delimiter -
___
: pass one delimiter, and then all but one call up to an outer delimiter -
____
: pass one delimiter, and then all calls up to an outer delimiter
We then have
@_ ... |> groupreduce(_.key, __, ___) do a, b
a.value < b.value ? b : a
end
Well, too bad that it's hard to count the number of _
s :laughing:
[*1] That's how I interpret them.
Well, too bad that it's hard to count the number of
_
s laughing
Haha! Yes...
I did consider that the closure created by do
could just bind to three underscores ___
BTW, it looks like this does not work?
@_ collect(Filter(_), -3:3) do x x > 0 end # => collect(Filter(x -> x > 0), -3:3)
I guess allowing this introduces more inconsistencies, though.
Yes. Though the fact that you might want this to work kind of suggests to me that this PR isn't really mergeable. It's just too confusing and overloaded, in a similar way that #4 was too confusing.
I feel like the best option may be just to go with an ugly but very clear identifier like _do_
or something.
I think the groupreduce
example is perfect btw — SplitApplyCombine
's grouping operations are one key use cases I had in mind for Underscores.jl. I feel that SplitApplyCombine has a very conceptually nice set of primitives, they only suffer from poor syntax usability in requiring multiple function arguments (CC @andyferris :) )
Yeah agreed. do
isn’t the best when you have lots of function inputs. Keywords are gimped on type constructors (widened to DataType
etc). So it’s a bit tricky to make the interface to groupreduce
readable.
Though the fact that you might want this to work kind of suggests to me that this PR isn't really mergeable.
TBH, I think it's OK to not do everything at AST. Though this is probably mainly because I can tweak Transducers.jl API to add |>
-friendly forms. For example, __
is not a hard requirement for me as long as I'm using Transducers.jl (because transducers are curried by default). Likewise, collect(Filter(_), -3:3)
example can now be written as
@_ -3:3 |> Filter(_ > 0) |> collect
# or even
-3:3 |>
Filter() do x
x > 0
end |>
collect
do
isn’t the best when you have lots of function inputs.
I'm actually experimenting API for challenging this :). With wheninit
etc., I can write
julia> averaging = # transducer version of `OnlineStats.Mean`
function add_average((sum, count), x)
(sum + x, count + 1)
end |>
wheninit() do
(Init(+), 0) # attach "loop header"
end |>
whencomplete() do (sum, count)
sum / count # attach "loop footer"
end |>
whencombine() do (sum1, count1), (sum2, count2)
(sum1 + sum2), (count1 + count2) # for threaded reduce
end;
I think this would be more readable than kwargs-based API especially when you have more complex function bodies
julia> averaging = AdHocRF(
oninit = () -> (Init(+), 0),
complete = (sum, count) -> sum / count,
combine = ((sum1, count1), (sum2, count2)) -> ((sum1 + sum2), (count1 + count2)),
) do (sum, count), x
(sum + x, count + 1)
end;
For groupreduce, maybe we can make something
julia> @_ 1:100 |>
Map((k = gcd(_, 42), v = _)) |>
grouped(_.k, __) .|>
Map(_.v) .|>
maximum
work. Then, since each expression between |>
has only one function, we can use do
block when the body is long enough.