treereduce icon indicating copy to clipboard operation
treereduce copied to clipboard

Incorrect instructions when initial test was not interesting

Open bbb651 opened this issue 8 months ago • 2 comments

After treereduce (correctly) detected the initial test was not interesting, it printed the following instructions:

Initial test was not interesting. Try the following:

    tmp="$(mktemp -d)"
    cp main.lua "${tmp}/your-test-case"
    cd "${tmp}"
    ./run.sh < ${tmp}/your-test-case
    echo $?

The last line should print 0 (or any other code passed to `--interesting-exit-code`). See the usage documentation for help: https://langston-barrett.github.io/treereduce/usage.html

After cding into $tmp, ./run.sh won't work because it's not in that directory. Treereduce itself does correctly keep the working directory, and the test case is prepended the temporary path, so I think just removing cd "${tmp}" will correct the instructions and match the behavior.

It would also be nice to have better compatibility for fish. Fish doesn't allow ${var}, only $var, afaik they're should behave exactly the same so it would be nice to change them. $? is also disallowed in favor of $status but I think it necessitates detecting the shell and have variations of the help message which probably isn't worth it.

bbb651 avatar Apr 27 '25 11:04 bbb651

After changing the command ./run.sh @@, I got the another variation of the error:

Initial test was not interesting. Try the following:

    tmp="$(mktemp -d)"
    cp main.lua "/tmp/.tmpJ5XX7S"
    cd "${tmp}"
    ./run.sh "/tmp/.tmpJ5XX7S"
    echo $?

The last line should print 0 (or any other code passed to `--interesting-exit-code`). See the usage documentation for help: https://langston-barrett.github.io/treereduce/usage.html

/tmp/.tmpJ5XX7S doesn't exist which doesn't seem intended? I can reproduce this with:

#!/bin/sh
exit 1

bbb651 avatar Apr 27 '25 11:04 bbb651

Thanks for the report! The problem is that the command from the user is pasted into the above block of shell commands without modifications:

https://github.com/langston-barrett/treereduce/blob/891ff262c185b74e735879c53fed06c7f80a732d/crates/treereduce/src/cli.rs#L320

So if the user provides a command that begins with a relative path like ./, then that relative path will be part of the output.

It would be good to check if the first element of the command is a relative path, and to resolve it to an absolute path before printing this message if so. I'd gladly accept a PR along those lines!

langston-barrett avatar Apr 28 '25 15:04 langston-barrett