Piping syntax

[sbarzowski]

I don't think there is any rush, I just wanted to get this idea out of my head.

The proposal has two parts:

1. Forward and backward piping operators

a |> b = b(a) // forward piping a <| b = a(b) // backward piping

The arrow points in which direction the value flows. The forward piping allows programming in UNIX-pipes style - allows cleanly doing multi-step processing without defining too many helper functions.

Backward piping is going to be useful with more "functional style" programs - it allows avoiding some parentheses (pretty much like Haskell's $).

2. In Jsonnet curried functions are not that common, so in order to make the use of (1) practical, there needs to be a way to choose an argument for piping. I think I came up with a nice syntax.

I'll start with an example:

["foo", "FOO", "bar"] |> std.map(capitalize, _) |> std.sort | std.uniq

So the idea is that instead of an argument _ can be "passed" - a hole where a value can be put. The _ is interpreted this way only when it is passed as an argument. No complex expressions involving it are allowed. f(2 + _) is not allowed.

So f(a, b, c, _, d) = function(x) f(a, b, c, x, d).

It's unambiguous which expression is wrapped in a function - it's always just the function application.

It would be possible to also do things like this:

arrayOfStrings |> std.filter(std.startsWith(_, "prefix"), _) |> something

The issue is that _ is a valid identifier. It's unfortunate. We can either find a new symbol or interpret it this way only when there is no local variable named that. This can be handled during desugaring, though it's going to be quite annoying. We should also discourage using _ (or anything starting with _ for future proofing) if we go with that - possibly using Linter.

[sparkprime]

I think @hausdorff proposed the same thing before, even with the same syntax

[sbarzowski]

Yeah, I think it was pretty much the same thing. From what I remember the approach to partial application was different (from what I remember it was something like one more argument was implicitly passed, and it couldn't be used without "piping").

I can't find the issue or anything, though (did the discussion take place only on Slack?).

[sbarzowski]

I think that it's better to have just the "forward piping" version (it's better not to provide two competing ways of doing the same thing). I don't see any case where backward piping (a'la Haskell's $) provides a clear advantage.

[sbarzowski]

Hmmm... actually backward piping could be useful for things like std.trace.

[tombrk]

(or anything starting with _ for future proofing) if we go with that - possibly using Linter.

Would that imply object keys like _config are discouraged? These are quite widely used for "internal" hidden fields that are supposed to be mixed into

[sbarzowski]

First, this was just about locals. Object fields are a completely different story. They don't even have to be valid identifiers.

It may still be to strong. One concrete thing I was I was thinking about was having something like _1, _2 meaning first, second argument etc.

[dasch]

Any updates on this? I've found myself wanting this syntax quite a lot. Furthermore, I think there's a detail that could make it even nicer: having the null value have special meaning in a pipeline.

Let's say the pipe function implements piping. I'd love to see this rough implementation:

local pipe(input, f) = if f == null then input else f(input);

Why would that be nice? Consider this:

input
  |> if config.capitalize then capitalize(_)
  |> if config.revert then revert(_)
  ...

I often find myself conditionally applying functions; without this, you'd still need to branch out and use temporary variables to share a common "prefix" of a pipeline. In the above case, you'd need to do something like this:

local a = if config.capitalize then capitalize(input) else input;
local b = if config.revert then revert(a) else a;
...

[sbarzowski]

Yeah, having null == id in this context seems to be quite convenient.

This feature is not strictly required for this flow. You can have:

local id(x) = x; // could also be included in stdlib;

input
  |> if config.capitalize then capitalize else id
  |> if config.revert then revert else id

[sbarzowski]

And there has been no progress. If someone wants to pick it up, it would be a very welcome addition. I'll be happy to help in case of any questions.

[CertainLach]

Implemented feature as described in this thread in jrsonnet (in branch, not in release), but without

So the idea is that instead of an argument _ can be "passed" - a hole where a value can be put. The _ is interpreted this way only when it is passed as an argument. No complex expressions involving it are allowed. f(2 + _) is not allowed.

limitation, as I'm probably not sure, why this limitation is needed. Please point out if I misunderstood proposed semantics :D

I think the proposed semantics is quite vague, I do not like that both value |> mapper and value |> mapper(_) can be used, why not choose one of those syntaxes?

If anyone wants to try, jrsonnet with this feature can be installed using cargo install --git https://github.com/CertainLach/jrsonnet --branch feature/piping, call example: jrsonnet -e '2 |> _ * 2 |> _ + 1'

[sbarzowski]

limitation, as I'm probably not sure, why this limitation is needed.

To avoid ambiguity when nesting. How do you know if f(g(_)) is a partial application of g or of f?

Also, the implementation could be simpler, depending on internal representations.

Generally, I think it's better to err on the side of minimalism in these things – adding is much easier than removing.

[sbarzowski]

I think the proposed semantics is quite vague, I do not like that both value |> mapper and value |> mapper(_) can be used, why not choose one of those syntaxes?

My suggestion was that mapper(_) can be used in any context a function is expected, not just for pipes. So e.g. you could also write something like std.filter(greater_than(3, _), my_arr).

Then |> accepts any single-argument function as the mapper. It's just an operator, without any special syntax on its own.

[CertainLach]

My suggestion was that mapper(_) can be used in any context a function is expected, not just for pipes. So e.g. you could also write something like std.filter(greater_than(3, _), my_arr).

Aha! Makes sense. In this case, something is certainly needed for multiple arguments.

My current implementation only has special handling for |>, desugaring (roughly) a |> b |> c as local _ = (local _ = a; b); c: https://github.com/CertainLach/jrsonnet/blob/8b201a56525fa9c36c32185d9680a9236e31aee0/crates/jrsonnet-evaluator/src/evaluate/mod.rs#L646-L658