Scala steward is not updating due to build errors

[tgodzik]

Hi, I just looked at some logs from the Scala Steward repo and it seems currently the job is failing for this repository and not sending updates:

java.io.IOException: 'sbt -Dsbt.color=false -Dsbt.log.noformat=true -Dsbt.supershell=false ;+ stewardDependencies;reload plugins;stewardDependencies' exited with code 1
[info] welcome to sbt 1.7.3 (Eclipse Adoptium Java 11.0.16.1)
[info] loading global plugins from /root/.sbt/1.0/plugins
[info] compiling 1 Scala source to /root/.sbt/1.0/plugins/target/scala-2.12/sbt-1.0/classes ...
[info] done compiling
[info] loading settings for project scala-webapp-template-build from plugins.sbt ...
[info] loading project definition from /opt/workspace/repos/wiringbits/scala-webapp-template/project
[warn] found version conflict(s) in library dependencies; some are suspected to be binary incompatible:
[warn] 	* org.scala-lang.modules:scala-xml_2.12:2.1.0 (early-semver) is selected over {1.2.0, 1.1.1, 1.3.0}
[warn] 	    +- org.scala-lang:scala-compiler:2.12.17              (depends on 2.1.0)
[warn] 	    +- org.scalablytyped.converter:scalajs_2.12:1.0.0-beta39 (depends on 1.3.0)
[warn] 	    +- com.typesafe.sbt:sbt-native-packager:1.5.2 (scalaVersion=2.12, sbtVersion=1.0) (depends on 1.1.1)
[warn] 	    +- com.typesafe.play:twirl-api_2.12:1.5.1             (depends on 1.2.0)
[info] loading settings for project root from build.sbt ...
[info] BuildInfo settings:
[info] (apiUrl,None)
[info] BuildInfo settings:
[info] (apiUrl,None)
[info] set current project to root (in build file:/opt/workspace/repos/wiringbits/scala-webapp-template/)
[warn] there are 8 keys that are not used by any other settings/tasks:
[warn]  
[warn] * admin / Compile / fastOptJS / webpackExtraArgs
[warn]   +- /opt/workspace/repos/wiringbits/scala-webapp-template/build.sbt:199
[warn] * admin / Compile / fullOptJS / webpackExtraArgs
[warn]   +- /opt/workspace/repos/wiringbits/scala-webapp-template/build.sbt:200
[warn] * apiJS / Compile / stMinimize
[warn]   +- /opt/workspace/repos/wiringbits/scala-webapp-template/build.sbt:276
[warn] * commonJS / Compile / stMinimize
[warn]   +- /opt/workspace/repos/wiringbits/scala-webapp-template/build.sbt:250
[warn] * ui / Compile / stMinimize
[warn]   +- /opt/workspace/repos/wiringbits/scala-webapp-template/build.sbt:312
[warn] * web / Compile / fastOptJS / webpackExtraArgs
[warn]   +- /opt/workspace/repos/wiringbits/scala-webapp-template/build.sbt:199
[warn] * web / Compile / fullOptJS / webpackExtraArgs
[warn]   +- /opt/workspace/repos/wiringbits/scala-webapp-template/build.sbt:200
[warn] * web / Compile / stMinimize
[warn]   +- /opt/workspace/repos/wiringbits/scala-webapp-template/build.sbt:392
[warn]  
[warn] note: a setting might still be used by a command; to exclude a key from this `lintUnused` check
[warn] either append it to `Global / excludeLintKeys` or call .withRank(KeyRanks.Invisible) on the key

[AlexITC]

Oh, that explains the lack of PRs (last one being #268), do you have any more logs? the ones attached are just info/warn.

We have a CI workflow that compiles the app which is working https://github.com/wiringbits/scala-webapp-template/blob/master/.github/workflows/pull_request.yml

Thanks for the report.

[tgodzik]

Ach right! I had build failing with that message so I didn't check futher. The problem seems that it tries to run yarn:

[error] java.io.IOException: Cannot run program "yarn" (in directory "/opt/workspace/repos/wiringbits/scala-webapp-template/lib/common/js/target/scala-2.13/scalajs-bundler/main"): error=2, No such file or directory
[error] 	at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1128)
[error] 	at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1071)
[error] 	at scala.sys.process.ProcessBuilderImpl$Simple.run(ProcessBuilderImpl.scala:75)
[error] 	at scalajsbundler.util.Commands$.run(Commands.scala:28)
[error] 	at scalajsbundler.util.Commands$.run(Commands.scala:38)
[error] 	at scalajsbundler.ExternalCommand.run(ExternalCommand.scala:22)
[error] 	at scalajsbundler.ExternalCommand$.$anonfun$install$1(ExternalCommand.scala:90)
[error] 	at scalajsbundler.ExternalCommand$.syncYarnLockfile(ExternalCommand.scala:46)
[error] 	at scalajsbundler.ExternalCommand$.install(ExternalCommand.scala:89)
[error] 	at scalajsbundler.sbtplugin.NpmUpdateTasks$.$anonfun$npmInstallDependencies$1(NpmUpdateTasks.scala:59)
[error] 	at sbt.util.FileFunction$.$anonfun$cached$1(FileFunction.scala:80)
[error] 	at sbt.util.FileFunction$.$anonfun$cached$4(FileFunction.scala:153)
[error] 	at sbt.util.Difference.apply(Tracked.scala:414)
[error] 	at sbt.util.Difference.apply(Tracked.scala:394)
[error] 	at sbt.util.FileFunction$.$anonfun$cached$3(FileFunction.scala:149)
[error] 	at sbt.util.Difference.apply(Tracked.scala:414)
[error] 	at sbt.util.Difference.apply(Tracked.scala:389)
[error] 	at sbt.util.FileFunction$.$anonfun$cached$2(FileFunction.scala:148)
[error] 	at scalajsbundler.sbtplugin.NpmUpdateTasks$.npmInstallDependencies(NpmUpdateTasks.scala:62)
[error] 	at scalajsbundler.sbtplugin.ScalaJSBundlerPlugin$.$anonfun$perConfigSettings$7(ScalaJSBundlerPlugin.scala:650)
[error] 	at scala.Function1.$anonfun$compose$1(Function1.scala:49)
[error] 	at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:62)
[error] 	at sbt.std.Transform$$anon$4.work(Transform.scala:68)
[error] 	at sbt.Execute.$anonfun$submit$2(Execute.scala:282)
[error] 	at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:23)
[error] 	at sbt.Execute.work(Execute.scala:291)
[error] 	at sbt.Execute.$anonfun$submit$1(Execute.scala:282)
[error] 	at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:265)
[error] 	at sbt.CompletionService$$anon$2.call(CompletionService.scala:64)
[error] 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[error] 	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
[error] 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[error] 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[error] 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[error] 	at java.base/java.lang.Thread.run(Thread.java:829)
[error] Caused by: java.io.IOException: error=2, No such file or directory
[error] 	at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
[error] 	at java.base/java.lang.ProcessImpl.<init>(ProcessImpl.java:340)
[error] 	at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:271)
[error] 	at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1107)
[error] 	at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1071)
[error] 	at scala.sys.process.ProcessBuilderImpl$Simple.run(ProcessBuilderImpl.scala:75)
[error] 	at scalajsbundler.util.Commands$.run(Commands.scala:28)
[error] 	at scalajsbundler.util.Commands$.run(Commands.scala:38)
[error] 	at scalajsbundler.ExternalCommand.run(ExternalCommand.scala:22)
[error] 	at scalajsbundler.ExternalCommand$.$anonfun$install$1(ExternalCommand.scala:90)
[error] 	at scalajsbundler.ExternalCommand$.syncYarnLockfile(ExternalCommand.scala:46)
[error] 	at scalajsbundler.ExternalCommand$.install(ExternalCommand.scala:89)
[error] 	at scalajsbundler.sbtplugin.NpmUpdateTasks$.$anonfun$npmInstallDependencies$1(NpmUpdateTasks.scala:59)
[error] 	at sbt.util.FileFunction$.$anonfun$cached$1(FileFunction.scala:80)
[error] 	at sbt.util.FileFunction$.$anonfun$cached$4(FileFunction.scala:153)
[error] 	at sbt.util.Difference.apply(Tracked.scala:414)
[error] 	at sbt.util.Difference.apply(Tracked.scala:394)
[error] 	at sbt.util.FileFunction$.$anonfun$cached$3(FileFunction.scala:149)
[error] 	at sbt.util.Difference.apply(Tracked.scala:414)
[error] 	at sbt.util.Difference.apply(Tracked.scala:389)
[error] 	at sbt.util.FileFunction$.$anonfun$cached$2(FileFunction.scala:148)
[error] 	at scalajsbundler.sbtplugin.NpmUpdateTasks$.npmInstallDependencies(NpmUpdateTasks.scala:62)
[error] 	at scalajsbundler.sbtplugin.ScalaJSBundlerPlugin$.$anonfun$perConfigSettings$7(ScalaJSBundlerPlugin.scala:650)
[error] 	at scala.Function1.$anonfun$compose$1(Function1.scala:49)
[error] 	at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:62)
[error] 	at sbt.std.Transform$$anon$4.work(Transform.scala:68)
[error] 	at sbt.Execute.$anonfun$submit$2(Execute.scala:282)
[error] 	at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:23)
[error] 	at sbt.Execute.work(Execute.scala:291)
[error] 	at sbt.Execute.$anonfun$submit$1(Execute.scala:282)
[error] 	at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:265)
[error] 	at sbt.CompletionService$$anon$2.call(CompletionService.scala:64)
[error] 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[error] 	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
[error] 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[error] 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[error] 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[error] 	at java.base/java.lang.Thread.run(Thread.java:829)

We could try to add it to the image, though I wonder if it will be needed by Scala steward at all? Also, did it ever work previously? We seem to be using the same exact image as previously.

[AlexITC]

Makes sense, yarn is required to install the js dependencies required by the scala.js modules, hence, yarn is needed to compile the app.

I wonder if it will be needed by Scala steward at all?

I bet this project isn't the only one depending on yarn for sjs modules, there is a chance you can find similar logs while building other projects.

Also, did it ever work previously?

Yes! #268 being a proof.

[tgodzik]

Yes! https://github.com/wiringbits/scala-webapp-template/issues/268 being a proof.

I meant the Scala Steward updates, when did they stop showing up?

[AlexITC]

#268 was the last update received, this was our next commit after that, I'm not sure on how we broke scala-steward.

[tgodzik]

Looks like maybe that was with the previous instance of Scala Steward, though not sure how does that happen if we were using the same image :thinking: Will need to check that. Not sure when I will be able to fix it, but you can set up a local Scala steward using github actions if needed.

[AlexITC]

you can set up a local Scala steward using github actions if needed.

Sure, I already host such a workflow for our private repositories, one of those projects was created from this template and it is still receiving updates, it uses scala-steward-org/scala-steward-action@v2

[tgodzik]

Seems like we need to add yarn in https://github.com/scala-steward-org/scala-steward/blob/main/build.sbt#L322-L353

If anyone has a bit of time to do that it would be awesome, otherwise I will try to go back to it at some point.

[AlexITC]

Hmm, I don't see node installed either, thanks for sharing the exact place to apply the fix.

[AlexITC]

@tgodzik I have raised a PR to include node/yarn on the docker image (https://github.com/scala-steward-org/scala-steward/pull/2851), the image builds fine but I haven't had a chance to test whether that will solve the issue while finding updates for this project.

[tgodzik]

Thanks! I think this should get picked up during the next run of Scala Steward. Let me know if you don't get any updates by the next week.

[AlexITC]

@tgodzik looks like my PR did not fixed the problem, a project with similar structure has received a PR (https://github.com/wiringbits/wiringbits-webapp-utils/pull/150).

[tgodzik]

Weirdly enough I can't find anything in the logs now :/ I will try to run it again.

[AlexITC]

@tgodzik I have integrated this repo with my own steward instance, PRs are now flowing https://github.com/wiringbits/scala-webapp-template/pull/312