script
script copied to clipboard
bitfield
Way to execute from a different directory?
Well I'm looking to execute from a different directory without actually changing my current directory, especially as os.chdir is going cause problems if I do any sort of parallelism/threading.
Yes, I see your point. The exec.Cmd does have a Dir field to control this, but currently script doesn't provide any way to set it for the command executed by Exec. I'm not sure what a good API would look like—any ideas?
A backwards compatible approach would be using variadic arguments to allow passing in optional parameters. This is a pretty common approach and is often used with structs or functions. Unfortunately I don't know if there is a name for that pattern.
I prefer the approach using functions.
type ExecOption func(&exec.Cmd)
func ExecWithDir(dir string) ExecOption {
return func(c &exec.Cmd) {
c.Dir = dir
}
}
func (p *Pipe) Exec(command string, opts ...ExecOption) *Pipe {
return p.Filter(func(r io.Reader, w io.Writer) error {
args, ok := shell.Split(command) // strings.Fields doesn't handle quotes
if !ok {
return fmt.Errorf("unbalanced quotes or backslashes in [%s]", command)
}
cmd := exec.Command(args[0], args[1:]...)
cmd.Stdin = r
cmd.Stdout = w
cmd.Stderr = w
for _, o := range opts {
o(cmd)
}
err := cmd.Start()
if err != nil {
fmt.Fprintln(w, err)
return err
}
return cmd.Wait()
})
}
func main() {
Exec("ls", ExecWithDir("/root")).Stdout()
}
Your example is the functional options API pattern. It's what I use too. It's very good.
@ghostsquad TIL, thank you!
The Blogpost about the talk that might have given it that name: https://dave.cheney.net/2014/10/17/functional-options-for-friendly-apis
That's the one! A really fantastic post. I love the flexibility it provides, and enables you to add functionality to an API without changing the function signatures.
Functional options is definitely a possibility here, though it ends up being a little awkward because the option names need to be prefixed with the package name. A more realistic version of the example given is:
script.Exec("/bin/ls", script.ExecWithDir("/some/directory/path")).Stdout()
...which tends to obscure the otherwise fluent pipeline syntax. You can imagine that if there were more options, this would soon no longer be a one-liner.
That's true. But if you have complex pipelines like that, you could also write them in multiple lines, e.g.
script.Exec("/bin/ls", script.ExecWithDir("/some/directory/path")).
Match("something").
Stdout()
which is still very readable.
The flow-breaking is still there, even if you spread it across multiple lines. And there's a question about what happens if you pipe the output of this Exec into some other Exec, for example: what working directory does that one operate with?
I think if we were going to set the working directory for a pipe, which I must say I'm not yet convinced is necessary, I think it would be more useful to have this be an option for the pipe as a whole:
script.NewPipe().WithWorkingDirectory("/home/me").Exec(...)
If other methods like File() support having a working directory as well, that would also be a good API. I would truncate it to something like WithWorkDir() though, otherwise it makes those one-liners pretty long.
I'm trying to figure out why we can't just run
os.Chdir("home/me")
script.Exec("ls /tmp").Stdout()
Because os.chdir is global. Not thread safe. It's not specific to the command.
You paid too much attention to the os.chdir. It was just to show that no matter what’s the path you are you can pass via args to the cmd itself instead of setting it on the cwd field of exec without changing anything on the package
You paid too much attention to the os.chdir. It was just to show that no matter what’s the path you are you can pass via args to the cmd itself instead of setting it on the cwd field of exec without changing anything on the package
I'm confused by this statement. You just said:
I'm trying to figure out why we can't just run
os.Chdir("home/me") script.Exec("ls /tmp").Stdout()
My answer is because os.Chdir is not specific to script.Exec. Are you suggesting that I run something like this?
workdir := "/foo/bar")
cmd := "ls"
script.Exec(fmt.Sprintf("cd %q && cmd", workdir, cmd)).Stdout()
though I agree this is pretty common in shell scripts, e.g.
(
cd "${workdir}";
ls
)
It just feels really hacky, and there's not a whole lot of room for script to do any sort of validation before hand, such as checking to see if the workdir actually exists yet. It also forces script to treat the command as a shell script, and not a binary, because of the &&. It's not the same.
my idea is even simpler
workdir := "/foo/bar"
cmd := "ls"
script.Exec(fmt.Sprintf("cmd workdir", workdir, cmd)).Stdout()
What to do in the case that the command doesn't accept a path. ls is an over simplified example. Better example may be make.
It's a good suggestion, but I don't think I've seen enough widespread support to convince me that the lack of such a mechanism is a serious problem for a large number of users. Let's close for now and return to this if the issue is raised again by others.
Just coming back to this, and I realized this has been asked for in various ways in several other issues, all of which dismissed in some form.
#79, #109, #48, #8
Just to reiterate, using os.Chdir is not going to work, as it is applies "globally", and is subject to race conditions when executing multiple external binaries or commands in parallel.
I'm just looking for something equivalent to setting the Path field in os/exec#Cmd.
By all means make a specific proposal!
What about this, for example:
script.At("/home/john").Exec("ls")
By all means make a specific proposal!
What about this, for example:
script.At("/home/john").Exec("ls")
I do like that.
Let's see if we can come up with one or two more realistic examples of programs using At, to get a better idea of whether this API works.
What would really help us here is a program that:
- Solves some real user problem (so that it's worth writing the program at all)
- Uses the proposed API (
script.At) - Executes multiple commands in parallel with different working directories (otherwise we don't need
At, becauseos.Chdirwill be fine) - Is better written with
scriptthan the standard library, or some other package
https://gobyexample.com/goroutines
Replace fmt.Println with running a shell command.
I'm not writing a program here to prove this feature is useful. It seems you don't believe that this feature is useful. I'll just close this and I'll go somewhere else.
Thank you for your time.
It's not a matter of belief: we simply don't need features that aren't needed!
In other words, software exists not to provide employment for software engineers, but to solve user problems. If no one has a problem that this feature solves, it's better not to write it. Useless API methods serve only to clutter the program and make it harder to use.
You have several times advanced the idea that there is a real use case requiring this feature, but despite many invitations, you've refused to say what it is. If you or anyone else can provide one, we'll implement this feature like a shot!
If no one has a problem that this feature solves, it's better not to write it.
I agree that features that no one will use should not exist, it's a waste of time. I have not yet met someone who openly admits to posting issues on GitHub for shits and giggles.
You have several times advanced the idea that there is a real use case requiring this feature, but despite many invitations, you've refused to say what it is.
4 months ago I posted this:
Well I'm looking to execute from a different directory without actually changing my current directory, especially as os.chdir is going cause problems if I do any sort of parallelism/threading.
I have a use case for this feature.
Please let me know if I've been unclear in any way.
Yes, I think I see where the confusion is coming from. "Use case" means different things to different people, but in this case I mean a real-life problem that that can only be solved if this feature is present.
For example, maybe you want to build several projects simultaneously, so you need to concurrently run make in a number of separate directories. You'd like to write something like:
for _, p := range projects {
go script.At(p).Exec("make").Stdout()
}
That's the kind of program I was asking for. Sorry that I was unclear—I don't know the right words to use to ask people to supply these examples.