Question: How to forward *args?

[gabyx]

When I have

run *args:
   echo "$1, $2,..."

do *args: (run args)

How can I forward arguments properly such that $2 is b in just do a b c?

[casey]

Unfortunately this currently isn't possible. args will be passed to run as a single space-separated argument.

The best way to support this would by some kind of splat syntax:

run *args:
   echo "$1, $2,..."

do *args: (run *args)

[casey]

Whoops, didn't mean to close.

[drmacdon]

A work-around today might be to call back into self.

set positional-arguments

run *args:
   echo "$1, $2,..."

@do *args:
   {{just_executable()}} -f {{justfile()}} run "$@"

[latk]

Unfortunately this currently isn't possible. args will be passed to run as a single space-separated argument.

This currently makes it impossible to use more complicated arguments to pass shell arguments safely to a dependency (i.e., without extra word-splitting or interpolation).

I understand why arguments must be immediately stringified, because the Just data model consists of just String. Changing that would be tedious, but in principle possible. I think the trick to do this in a backwards-compatible way would be to implement something like Perl's "dualvar", where a value is both a string value and possibly another value at the same time.

I'd be interested in prototyping that approach if there will be interest in merging it.

Sketch of such a data structure that could replace String in bindings:

struct Value {
  /// String representation of the value.
  stringy: Box<str>,
  /// Optionally, array representation of this value.
  items: Option<Box[Box<str>]>,
}

impl Value {
  pub fn new(s: String) -> Self {
    Self { stringy: s, items: None }
  }
  pub fn from_vec(items: Vec<Box<str>>) -> Self {
    Self { stringy: items.join(" "), items: Some(items.into()) }
  }
}

impl Deref<str> for Value { ... } // convenience compatibility with String

Just's values are immutable, so the more compact Box<str> can be used instead of String, and Box<[...]> instead of Vec.

Single-value strings s would subsequently be represented with items: None, whereas multi-value variadics could retain the correctly split items, while still being compatible with string-like use.

Once that is done, a dependency splat syntax caller *args: (callee *args) should be implementable. Whereas the current evaluator produces a single value, a list-context evaluator might be necessary. Something like:

pub(crate) fn evaluate_expression(
  &mut self,
  expression: &Expression<'src>,
) -> RunResult<'src, Value> {
  match expression {
    ...
    Expression::Splat => Err(Error::SplatNotAllowedInSingleValueContext),
  }
}

pub(crate) fn evaluate_list_expression(
  &mut self,
  expression: &Expression<'src>,
) -> RunResult<'src, Vec<Value>> {
  match expression {
    Expression::Splat(inner) => match self.evaluate_expression(inner)? {
      Value { items: Some(items), .. } => Ok(items.map(s => s.into()).collect()),
      value => Ok(vec![value]),
    },
    other => Ok(vec![self.evaluate_expression(expr)?]),
  }
}

Then in run_recipe():

         let arguments = arguments
           .iter()
-          .map(|argument| evaluator.evaluate_expression(argument))
+          .flat_map(|argument| evaluator.evaluate_list_expression(argument))
           .collect::<RunResult<Vec<Value>>>()?;

         Self::run_recipe(&arguments, context, ran, recipe, true)?;

The advantage of this approach is that it would be fully backwards-compatible: variadic recipe parameters will continue to be joined, unless an explicit splat expression is used. The downside is that it will complicate the Just data model by introducing strings-that-are-not-just-strings and a list context in which expressions can use different operators.

A more conventional approach would be an enum Value { String(String), Vec(Vec<String>) } and deferring the vec.join(" ") until it is needed in a string-like context. But since most places expect strings, that change would likely require more work.

In any case, such a dependency splat syntax would conflict with this open issue that suggests running a recipe for each splatted argument:

• https://github.com/casey/just/issues/558

[casey]

@latk I definitely agree with the idea and the approach. I think in practice, you could just do:

struct Value {
  items: Vec<String>,
}

And then .join(" ") items whenever you need the actual value, so that you avoid an enum or a more complicated type.

There's also #2458, which would make all values lists of strings, with single strings just being a list with one item, and using this pervasively. I think this is a good idea. just suffers from not being able to represent lists of things, and all the noraml shell headaches. The rc shell had it right that if you have a shell with a single data type, that type should be lists of strings, since that just gets rid of so many shell headaches, like quoting, splitting, and joining.

So yeah, I would love to see this happen, and starting with making variadic arguments nicer would be great. We should think about how to forward args and how to dispatch variadic arguments to multiple dependencies with splats at the same time, and make sure that we have syntax that works for both.

Later, we can think about #2458, which would allow users to actually manipulate lists in more useful ways. All that gets easier once the codebase is converted from using strings to some kind of Value type that can be a list. (I lean towards calling it Val since it's short and this type will be pervasive in the codebase.)

[latk]

Ok. I've read through that context and will take a jab at transitioning the data model to a new Val type over the weekend, and will report back with my findings. My goal is to have that first step be a purely internal refactoring, with zero changes in syntax or semantics.

I agree that using something like a Vec<String> as the sole data type is more elegant than a String, dualvar, or enum. However, I expect the necessary changes for "everything is a Vec" to be more involved, as the Val would not be able to deref to a &str. But implementing Display for Val would give us a to_string() method, which is almost as convenient.

Once we have list-typed values and lossless forwarding of variadic arguments to dependencies, the one feature that I really want is a list-aware quote(*args) function, which would avoid the need for many positional-arguments use cases.

[casey]

Sounds good! I agree a first pass that just introduces the new Val type would be great, even if it doesn't change any behavior or add new features.

[jchook]

Perhaps fundamentally related: it would be nice to have "$@" without positional-arguments, too.

just itself receives the argv list that the shell already split and unescaped.

But {{args}} essentially destroys the boundaries and does not re-escape them.

"{{args}}" has "$*" semantics... and apparently no "$@" semantics exist in just without using positional-arguments and "$@" directly. Is this true?

Imagining something like this:

run *args:
  @printf '%s\n' {{@args}}

just run 1 "two three" ""
1
two three

For example, consider this current version justfile:

set positional-arguments := true

test:
  just def test 1 "two three" ""
  just quo test 1 "two three" ""
  just pos test 1 "two three" ""

def *args="":
  printf '%s\n' {{args}}

quo *args="":
  printf '%s\n' "{{args}}"

pos *args="":
  printf '%s\n' "$@"