futhark icon indicating copy to clipboard operation
futhark copied to clipboard

Futhark can't get opir output on Windows due to how nimble creates binaries and how `staticExec` works

Open Yardanico opened this issue 2 years ago • 5 comments

On Windows nimble creates bin.cmd files in the ~/.nimble/bin folder instead of symlinks with the contents:

@chcp 65001 > nul
@"%~dp0\..\pkgs\futhark-0.6.1\opir.exe" %*

Futhark executes opir with just opir, and staticExec doesn't work with that approach as it uses startProcess which errors for opir args if there's only opir.cmd in PATH and no opir.exe because that's standard Windows behaviour (it doesn't consider opir.cmd when you run opir).

I can't think of any real solution to this except just using opir.cmd as the executable name on Windows for calling opir, but this makes Futhark dependent on nimble behaviour...

Yardanico avatar May 09 '22 18:05 Yardanico

To reproduce (on Windows):

  1. Create file.cmd with:
@echo "hi!"
  1. Create test.nim with:
import std/[osproc, streams]

proc readOutput(p: Process): (string, int) =
  result[0] = ""
  var output = p.outputStream
  while not output.atEnd:
    result[0].add(output.readLine)
    result[0].add("\n")
  if result[0].len > 0:
    result[0].setLen(result[0].len - "\n".len)
  result[1] = p.waitForExit

proc myStaticExec(cmd: string): (string, int) = 
  var p = startProcess(cmd, options={poEvalCommand, poStdErrToStdOut})
  result = p.readOutput
  p.close()

echo myStaticExec("file.cmd") # with .cmd
echo myStaticExec("file")
  1. Compile and run test.nim, see how it works for file.cmd but fails for file.

Yardanico avatar May 09 '22 18:05 Yardanico

workaround:

let opirRes = gorgeEx
  when defined(windows): 
    "cmd /c " & cmd 
  else:
    cmd

awr1 avatar Sep 04 '22 21:09 awr1

@Yardanico

Futhark executes opir with just opir, and staticExec doesn't work with that approach as it uses startProcess which errors for opir args if there's only opir.cmd in PATH and no opir.exe because that's standard Windows behaviour (it doesn't consider opir.cmd when you run opir).

the problem here is while that this is how CreateProcess() works, yes, this is not how cmd.exe works. citing MSDN:

To run a batch file, you must start the command interpreter; set lpApplicationName to cmd.exe and set lpCommandLine to the following arguments: /c plus the name of the batch file.

and also, as an example, cmd.exe will gladly forego the lack of *.cmd:

C:\Users\me\dev\extra>file
"hi!"

imo this behavior is not really clarified in the context of gorge(), which is assumed to just "act like the shell." this actually feels like a standard library issue with startProcess() specifically, especially when nim specifies a poEvalCommand flag that appropriately passes it to an actual shell on unix-ish systems but doesn't do this on win32.

awr1 avatar Sep 04 '22 22:09 awr1

Please create a PR of this suggestion if it works.

PMunch avatar Sep 05 '22 12:09 PMunch

@PMunch I don't think it's possible (but I might be wrong) without creating another wrapper binary because Futhark calls opir from a macro, so it's in a compile-time context, and there you can't use std/os procs like startProcess

Yardanico avatar Sep 05 '22 12:09 Yardanico