command expansions

[QiBaobin]

#3134

[QiBaobin]

only support filename val for now: bc %val{filename}

support simple sh command like:

open %sh{fd\ 16_theme}

[sbromberger]

Just curious: does this quote the filename so that if you pass it to sh it doesn't fall victim to the IFS boundary?

[QiBaobin]

Just curious: does this quote the filename so that if you pass it to sh it doesn't fall victim to the IFS boundary?

I think we can use sh echo "%val{filename}"

[trink]

FYI: This doesn't handle substitution in the keymap arguments.

[keys.normal]
"C-s" = [":sh hx_rcs_write.sh '%val{filename}'", ":w"] # version control the previous and write out the new

[QiBaobin]

"C-s" = [":sh hx_rcs_write.sh '%val{filename}'", ":w"] 

@trink , it shall work now, please take anther look, thanks!

[QiBaobin]

support %val {filename} %val {dirname} and %sh {cmd} now, we can also nest them like %sh { echo %val {dirname} }, also addressed the 'have to escape space' issue as @the-mikedavis suggested.

[danillos]

Thanks, @QiBaobin I really liked this feature, with that I could build:

F8 = ':open %sh { ~/.dotfiles/support/command.sh switch_to_spec %val{filename} }'
F10 = ':sh ~/.dotfiles/support/command.sh run_current_file_spec %val{filename}'

These scripts that I created are not perfect because I had to hard-code how to get the project dirname.

Could we have something like %val{project_dirname} expand to the absolute folder path used in hx . ?

Edit: Another useful variable is something like %val{line_number}, to use in commands like open it in github.

[cammm]

As someone who works with perforce which requires you to checkout files to make them writable before you can save them, this feature (which I am testing out locally) is great for making a shortcut that handles that on save. Thanks for putting this together.

[danillos]

Hey @QiBaobin, I did a change of your code fixing some bugs and adding more variables. It adds %val{root_dirname}, %val{relative_filename} and %val{line_number}

I have a question do you have time to keep working on this feature? I saw that it wasn't rebased for a while.

fn expand_args<'a>(editor: &mut Editor, args: &'a str) -> Cow<'a, str> {
    let (view, doc) = current!(editor);
    let text = doc.text().slice(..);

    let re_val = Regex::new(r"%val\s*?\{+(?P<var_name>\w+)\}").unwrap();
    let re_sh = Regex::new(r"%sh+\s*?\{+(.*+)\}").unwrap();

    let cmd = re_val.replace_all(args, |caps: &regex::Captures| {
        let root_dirname = PathBuf::from(".")
            .canonicalize()
            .unwrap()
            .to_string_lossy()
            .to_string()
            .to_owned();

        let file_name = doc.path().and_then(|p| p.to_str()).unwrap_or("").to_owned();

        let mut pp: String = root_dirname.clone();
        pp.push('/');
        let relative_file_name = file_name.replace(&pp, "");

        let line_number = doc.selection(view.id).primary().cursor_line(text) + 1;

        match caps[1].trim() {
            // ex. /Users/{user}/my_project/src/file.rs
            "filename" => file_name,
            // ex. /Users/{user}/my_project
            "root_dirname" => root_dirname,
            // ex. src/file.rs
            "relative_filename" => relative_file_name,
            // ex. 1
            "line_number" => line_number.to_string().to_owned(),
            // ex. /Users/{user}/my_project/src
            "dirname" => doc
                .path()
                .and_then(|p| p.parent())
                .and_then(|p| p.to_str())
                .unwrap_or("")
                .to_owned(),
            _ => caps[0].to_owned(),
        }
    });

    let cmdd = re_sh.replace_all(&cmd, |caps: &regex::Captures| -> String {
        let shell_command = caps[1].trim();
        let shell = &editor.config().shell;
        if let Ok((output, _)) = shell_impl(shell, shell_command, None) {
            output.trim().into()
        } else {
            "".into()
        }
    });

    Cow::Owned(cmdd.to_string())
}

I'm not a Rust dev, but I'm trying to learn it.

[QiBaobin]

Hey @QiBaobin, I did a change of your code fixing some bugs and adding more variables. It adds %val{root_dirname}, %val{relative_filename} and %val{line_number}

I have a question do you have time to keep working on this feature? I saw that it wasn't rebased for a while.

fn expand_args<'a>(editor: &mut Editor, args: &'a str) -> Cow<'a, str> {
    let (view, doc) = current!(editor);
    let text = doc.text().slice(..);

    let re_val = Regex::new(r"%val\s*?\{+(?P<var_name>\w+)\}").unwrap();
    let re_sh = Regex::new(r"%sh+\s*?\{+(.*+)\}").unwrap();

    let cmd = re_val.replace_all(args, |caps: &regex::Captures| {
        let root_dirname = PathBuf::from(".")
            .canonicalize()
            .unwrap()
            .to_string_lossy()
            .to_string()
            .to_owned();

        let file_name = doc.path().and_then(|p| p.to_str()).unwrap_or("").to_owned();

        let mut pp: String = root_dirname.clone();
        pp.push('/');
        let relative_file_name = file_name.replace(&pp, "");

        let line_number = doc.selection(view.id).primary().cursor_line(text) + 1;

        match caps[1].trim() {
            // ex. /Users/{user}/my_project/src/file.rs
            "filename" => file_name,
            // ex. /Users/{user}/my_project
            "root_dirname" => root_dirname,
            // ex. src/file.rs
            "relative_filename" => relative_file_name,
            // ex. 1
            "line_number" => line_number.to_string().to_owned(),
            // ex. /Users/{user}/my_project/src
            "dirname" => doc
                .path()
                .and_then(|p| p.parent())
                .and_then(|p| p.to_str())
                .unwrap_or("")
                .to_owned(),
            _ => caps[0].to_owned(),
        }
    });

    let cmdd = re_sh.replace_all(&cmd, |caps: &regex::Captures| -> String {
        let shell_command = caps[1].trim();
        let shell = &editor.config().shell;
        if let Ok((output, _)) = shell_impl(shell, shell_command, None) {
            output.trim().into()
        } else {
            "".into()
        }
    });

    Cow::Owned(cmdd.to_string())
}

I'm not a Rust dev, but I'm trying to learn it.

Actually, I don't. It would be great if someone can pick up from here. The reason I use replace instead of replace_all is that I wan to support nested expression, not sure if we shall do that.

[ksdrar]

Hello, I changed the variables and command syntaxes to #{name} for variables and #name [body] for commands. It makes parser easier and I think writing them is easier.

The changes are on my fork: https://github.com/ksdrar/helix

Some simple tests:

Testing: #{filename} -----> filename
Testing: #{filename} #{parent}\new_filename -----> filename parent\new_filename
Testing: #sh [echo #{filename}] -----> sh echo filename
Testing: #sh [echo #{filename},#{filename}] -----> sh echo filename,filename
Testing: #sh [echo #{filename},#{filename},(#sh [echo #{filename}])] -----> sh echo filename,filename,(sh echo filename

Here's a short video of them in action:

https://user-images.githubusercontent.com/22417151/230752179-818ca2aa-1123-4b34-8eb0-05de9020e4c7.mp4

[trink]

Hello, I changed the variables and command syntaxes to #{name} for variables and #name [body] for commands. It makes parser easier and I think writing them is easier.

The intent of the %nnn{} type prefix is to serve as a namespace for things like arguments, registers, configuration options and values to provide a bit more structure, communicate intent, and reduce potential name collisions. The original design is more extensible.

[edrex]

%val{line_number}

If this were done, it would be nice if it could be generalized somehow to support selections and multiple selections. It doesn't do much for a shell command though if the buffer is dirty. Thinking about use-cases, I think the existing shell_pipe etc commands, powered up by command expansions, will serve many. Also, I'm not an LSP expert (yet) but I wonder what could be done with some kind of user-configurable LSP combinator proxy that does things with selections. This is all out of scope for this issue, but relevant to thinking about whether to include line_number.

%val{root_dirname}

I'm for orthogonality. If two vars are equivalent, or one can be derived from the other, I'd prefer only one was included. Functions (either implemented as shell commands, or somehow within helix) from a file path to project path and to project-relative file path seem more generally useful than putting pre-computed derived vals in the list. But maybe I'm not looking at it right.

[danillos]

@edrex about %val{line_number} I'll give some examples on how I'm using it.

1. To git blame the current line. So I can see who did the changed and when.

g = [':sh ~/.dotfiles/commands.sh git_blame %val{relative_filename}:%val{linenumber}']

2. Open current line in Github

G = [':sh ~/.dotfiles/commands.sh open_file_in_github %val{relative_filename}:%val{linenumber}']

3. Run Rspec Test on a specific line number

B = ':sh ~/.dotfiles/commands.sh run_rspec_on_line %val{relative_filename}:%val{linenumber}'

Without it I would not be able to daily drive in Helix.

[edrex]

@danillos are you volunteering to pick this up and take it over the finish line? It seems like @QiBaobin is open to it. If so, you could push up a branch on top of this one (but please preserve existing history for clarity) and open a new PR, and then you'll have an extra expert review in @QiBaobin. Please correct me if I'm misreading intent here.

[edrex]

It doesn't do much for a shell command though if the buffer is dirty

I guess we can trust users to make sure to write out first in their %val{linenumber}-using shell commands

[danillos]

@danillos are you volunteering to pick this up and take it over the finish line? It seems like @QiBaobin is open to it. If so, you could push up a branch on top of this one (but please preserve existing history for clarity) and open a new PR, and then you'll have an extra expert review in @QiBaobin. Please correct me if I'm misreading intent here.

Certainly, I was planning to do it. This pull feature is essential for my development flow.

[ksdrar]

Hey guys, I got the nested commands working with %key{body}

You can try the changes in the branch cmd-expansions of my fork.

The output of the tests:

❯ cargo run --release
   Compiling pattern_matching v0.1.0 (/home/jesus/Development/rust/pattern_matching)
    Finished release [optimized] target(s) in 0.54s
     Running `target/release/pattern_matching`
[TEST] %val {filedir} => filedir
[PASSED 821.124µs] filedir == filedir

[TEST] %val {filedir%val {aasdfasdf}} => filediraasdfasdf
[PASSED 738.772µs] filediraasdfasdf == filediraasdfasdf

[TEST] %sh {rm %val{filedir},%val{filename}} => rm filedir,filename
[PASSED 647.805µs] rm filedir,filename == rm filedir,filename

[TEST] %sh {git blame %val{filename}:%val{linenumber}} => git blame filename:linenumber
[PASSED 666.177µs] git blame filename:linenumber == git blame filename:linenumber

[TEST] :sh (%sh {cat %val{filename} | grep rust}) > %val{filedir}/test.txt => :sh (cat filename | grep rust) > filedir/test.txt
[PASSED 660.74µs] :sh (cat filename | grep rust) > filedir/test.txt == :sh (cat filename | grep rust) > filedir/test.txt

[TEST] :sh bash -c (%sh {(%sh {cat %val{filedir}/test.txt}) > %val{filedir}/../%val{filename}_test.txt}) => :sh bash -c ((cat filedir/test.txt) > filedir/../filename_test.txt)
[PASSED 736.72µs] :sh bash -c ((cat filedir/test.txt) > filedir/../filename_test.txt) == :sh bash -c ((cat filedir/test.txt) > filedir/../filename_test.txt)

The bare-bones code:

fn test(to_test: &str, expected: &str) {
    let start = std::time::Instant::now();
    println!("\x1b[33m[TEST]\x1b[0m {to_test} => {expected}");
    let result = find_and_replace(to_test).unwrap_or_default();
    assert_eq!(result, expected);
    println!(
        "\x1b[32m[PASSED {:2?}]\x1b[0m {result} == {expected}\n",
        start.elapsed()
    );
}

fn main() {
    test("%val {filedir}", "filedir");

    test("%val {filedir%val {aasdfasdf}}", "filediraasdfasdf");

    test(
        "%sh {rm %val{filedir},%val{filename}}",
        "rm filedir,filename",
    );

    test(
        "%sh {git blame %val{filename}:%val{linenumber}}",
        "git blame filename:linenumber",
    );

    test(
        ":sh (%sh {cat %val{filename} | grep rust}) > %val{filedir}/test.txt",
        ":sh (cat filename | grep rust) > filedir/test.txt",
    );

    test(
        ":sh bash -c (%sh {(%sh {cat %val{filedir}/test.txt}) > %val{filedir}/../%val{filename}_test.txt})",
        ":sh bash -c ((cat filedir/test.txt) > filedir/../filename_test.txt)",
    );
}

fn find_and_replace(input: &str) -> anyhow::Result<String> {
    let regexp = regex::Regex::new(r"%(\w+)\s*\{([^{}]*(\{[^{}]*\}[^{}]*)*)\}").unwrap();

    replace_all(&regexp, input, |captures| {
        let keyword = captures.get(1).unwrap().as_str();
        let body = captures.get(2).unwrap().as_str();

        match keyword {
            "val" => Ok(String::from(body)),
            "sh" => Ok(String::from(body)),
            _ => anyhow::bail!("Unknown keyword {keyword}"),
        }
    })
}

fn replace_all(
    regex: &regex::Regex,
    text: &str,
    matcher: impl Fn(&regex::Captures) -> anyhow::Result<String>,
) -> anyhow::Result<String> {
    let mut it = regex.captures_iter(text).peekable();

    if it.peek().is_none() {
        return Ok(String::from(text));
    }

    let mut new = String::with_capacity(text.len());
    let mut last_match = 0;

    for cap in it {
        let m = cap.get(0).unwrap();
        new.push_str(&text[last_match..m.start()]);

        let replace = matcher(&cap)?;

        new.push_str(&replace);

        last_match = m.end();
    }

    new.push_str(&text[last_match..]);

    replace_all(regex, &new, matcher)
}

[danillos]

I’ve tested your branch, @ksdrar, and it worked perfectly. In my opinion, you can submit a Pull Request to replace the current one. I believe @QiBaobin will close this current PR once yours is opened.

Nice work.

[danillos]

@ksdrar I found an issue. I was migrating all keybindings that I did using my version to your version and this one didn't work.

# before
s = ':open %sh { ~/.dotfiles/helix/command.rb switch_to_spec %val{projectdir} %val{relative_filename}}'

# after
s = ':open %sh {~/.dotfiles/helix/command.rb switch_to_spec %sh{pwd} %val{filename}}'

It is not expanding %sh{pwd} and %val{filename}, If I understood, it first expands %sh

[danillos]

I think it should expand the most inner expressions first, and move to the next one only when there is nothing left to expand. For example:

filename = test.rs
line_number = 123

Command: %sh { add_suffix.sh %sh { add_line_number.sh %sh { add_prefix.sh %val{filename} } } }

Expansion Steps:

0. %sh { add_suffix.sh %sh { add_line_number.sh %sh { add_prefix.sh %val{filename} } } }
1. %sh { add_suffix.sh %sh { add_line_number.sh %sh { add_prefix.sh test.rs } } }
2. %sh { add_suffix.sh %sh { add_line_number.sh prefix_test.rs } }
3. %sh { add_suffix.sh prefix_test.rs:%val{line_number} }
4. %sh { add_suffix.sh prefix_test.rs:123 }
5. prefix_test_suffix.rs:123

Scripts:

add_prefix.sh: adds `prefix_{string}` to a string
add_suffix.sh: adds `{string}_suffix` to a string
add_line_number.sh: adds `{string}:%val{line_number}` to a string

[ksdrar]

@danillos Can you check if it's working now with the last commit?

[danillos]

@danillos Can you check if it's working now with the last commit?

Yes, this example is now working.

s = ':open %sh {~/.dotfiles/helix/command.rb switch_to_spec %sh{pwd} %val{filename}}'

[danillos]

@ksdrar could you open your changes as a new PR for replace this one? I could do that too.