mill icon indicating copy to clipboard operation
mill copied to clipboard

Add some hints to using `os.proc` with Mill

Open lefou opened this issue 3 years ago • 5 comments

Using os.Inherit for output is known to not show up when in client-server-mode. We should mention the use of mill.modules.Jvm.runSubprocess and also give suggestions which settings work best in Mill.

Also, all links to os-lib in the documentation should go through a dedicated Mill-specific page, so that readers following it will have an option to read about those best practices. This could be the existing https://com-lihaoyi.github.io/mill/mill/External_References.html#_os_lib page, or some more dedicated.

Related:

  • #2059

lefou avatar Oct 07 '22 22:10 lefou

I've been doing something like

stdout = os.ProcessOutput.Readlines(line => T.ctx().log.debug("[prefix] " + line))

where prefix is a name for what program I'm running.

nafg avatar Nov 27 '22 01:11 nafg

Although, I guess you'd want to do that for stderr too (except maybe with log.warn)...

Too bad os.proc doesn't have an easy way to reuse groups of parameters, that's something I could use often.

nafg avatar Nov 27 '22 02:11 nafg

Another good place to mention Jvm.runSubprocess is https://com-lihaoyi.github.io/mill/mill/Extending_Mill.html#_custom_targets_commands

which currently says

For subprocess/filesystem operations, you can use the os-lib library that comes bundled with Mill, or even plain java.nio/java.lang.Process. Each target gets its own T.dest folder that you can use to place files without worrying about colliding with other targets.

nafg avatar Nov 27 '22 19:11 nafg

One thing that's worse about Jvm.runSubprocess is that if it exits with a nonzero exit code, it throws a generic Exception that doesn't say the command line that failed. With os.proc I was doing

    catch {
      case e: os.SubprocessException =>
        throw new RuntimeException(
          "rollup failed with exit code " + e.result.exitCode,
          e
        )
    }

etc. Otherwise if a few commands run and their output is not so different-looking from each other, it's hard to tell which one failed. But since runSubprocess throws java.lang.Exception it's not as clean to catch.

A different option would be to just log at the beginning the command line that will be run.

nafg avatar Nov 27 '22 20:11 nafg

I switched to

    catch {
      case e: Exception =>
        throw new RuntimeException("Error running webpack", e)
    }

but then the output is harder to scan:

main.test.fastLinkJSTest java.lang.RuntimeException: Error running webpack
    io.github.nafg.millbundler.ScalaJSWebpackModule.$anonfun$bundle$3(ScalaJSWebpackModule.scala:90)
    io.github.nafg.millbundler.ScalaJSBundleModule$Test.$anonfun$fastLinkJSTest$2(ScalaJSBundleModule.scala:33)
    mill.define.Task$TraverseCtx.evaluate(Task.scala:380)
java.lang.Exception: Interactive Subprocess Failed (exit code 1)
    mill.modules.Jvm$.runSubprocess(Jvm.scala:196)
    io.github.nafg.millbundler.ScalaJSWebpackModule.$anonfun$bundle$3(ScalaJSWebpackModule.scala:86)
    io.github.nafg.millbundler.ScalaJSBundleModule$Test.$anonfun$fastLinkJSTest$2(ScalaJSBundleModule.scala:33)
    mill.define.Task$TraverseCtx.evaluate(Task.scala:380)

nafg avatar Nov 27 '22 20:11 nafg

I think this can be closed since https://github.com/com-lihaoyi/mill/pull/3275 makes os.proc.call/os.Inherit do the right thing by default

lihaoyi avatar Jul 22 '24 12:07 lihaoyi