Some dependencies not updated

[james-s-w-clark]

I didn't see another issue like this one, so I'm opening this.

I saw a few dependencies were outdated in a project despite daily scala-steward runs (on latest image). I confirmed there were updates with mill mill.scalalib.Dependency/showUpdates

Here's some examples:

• log4j api/core/slf4j-impl (2.16.0 -> 2.17.2)
• refined (0.9.14 -> 0.9.28)
• os-lib (0.7.8 -> 0.8.1)
• wiremock-jre8 (2.31.0 -> 2.33.2)

In the case of slf4j, 2.16.0 was Dec 2021 and 2.17.2 is Feb 2022. I manually updated it in late June, so Scala Steward wasn't picking up those updates for about half a year.

Some of these in an dependencies.sc file have the version as part of the value. Some have it interpolated into the ivy dependency. I don't see an obvious pattern for why some things get updates, but others don't.

Posssible causes

• Does scala steward cache failures from fail-fast, so if you have a failed CI check then you won't ever get e.g. 0.0.1 -> 0.0.2 of a dependency? This is a total guess. I don't know how scala-steward works.

[james-s-w-clark]

https://github.com/com-lihaoyi/mill/pull/1954 as example in a live public repo that uses scala-steward

[james-s-w-clark]

Is there anything I could help with here? I had a dig and I can't figure out why this update didn't update, even though there's 9 new verisons of scalafmt-dynamic for example, spread over a few months.

The group/artifact IDs are unchanged, so there is no need for an artifact migration like at release page

There is no pinning to minor version in mill's scala-steward.conf either.

  val scalafmtDynamic = ivy"org.scalameta::scalafmt-dynamic:3.4.3"
  val scalafmtDynamic = ivy"org.scalameta::scalafmt-dynamic:3.5.8"

[james-s-w-clark]

Maybe it's a problem with the public runer - more info here?

[exoego]

Can the reproducible repo be provided?

[james-s-w-clark]

Can the reproducible repo be provided?

See https://github.com/scala-steward-org/scala-steward/issues/2664#issuecomment-1180131005 (mill#1954) for an example.

• https://mvnrepository.com/artifact/org.scalameta/scalafmt-dynamic 3.4.3 -> 3.5.8. All versions between support Scala 2.12/2.13.
• 3.4.3 is from Feb 2022
• Simple one-line dependency upgrade, so nothing more complex like scalafix needed.

---

I haven't scanned through these updates yet, but to illustrate what I did: In the mill repo, one can run mill mill.scalalib.Dependency/showUpdates. ~I didn't check where these come from, or if they're easy updates in dependencies.sc/build.sc yet.~ Versions listed are in build.sc Example output of mill mill.scalalib.Dependency/showUpdates | grep ':' | sort --unique gives:

• ch.epfl.scala:bloop-config_2.13 : 1.5.2 -> 1.5.3
  • mvnrepo 1.5.2 Jun 2022. No new PRs for it search in mill repo
• com.lihaoyi:utest_2.13 : 0.7.11 -> 0.8.0
  • github mvn 25 days ago. steward PR closed, this is OK
• com.typesafe.play:routes-compiler_2.12 : 2.6.25 -> 2.7.0 -> 2.7.1 -> 2.7.2 -> 2.7.3 -> 2.7.4 -> 2.7.5 -> 2.7.6 -> 2.7.7 -> 2.7.8 -> 2.7.9 -> 2.8.0 -> 2.8.1 -> 2.8.2 -> 2.8.3 -> 2.8.4 -> 2.8.5 -> 2.8.6 -> 2.8.7 -> 2.8.8 -> 2.8.9 -> 2.8.10 -> 2.8.11 -> 2.8.12 -> 2.8.13 -> 2.8.14 -> 2.8.15 -> 2.8.16
  • mvn dec 2019. This one is an interpolation, with version hard coded in another line. Probably don't expect auto update for this.
• com.typesafe.play:routes-compiler_2.13 : 2.7.9 -> 2.8.0 -> 2.8.1 -> 2.8.2 -> 2.8.3 -> 2.8.4 -> 2.8.5 -> 2.8.6 -> 2.8.7 -> 2.8.8 -> 2.8.9 -> 2.8.10 -> 2.8.11 -> 2.8.12 -> 2.8.13 -> 2.8.14 -> 2.8.15 -> 2.8.16
  • Same as above, not a problem?
• io.get-coursier:coursier_2.13 : 2.1.0-M6 -> 2.1.0-M6-26-gcec901e9a -> 2.1.0-M6-28-gbad85693f -> 2.1.0-M6-44-gc47ecec07
  • Mayb not updated as it's on a milestone version, and newer versions just have different hash on the end. link to mill. Seems OK. mvn
• org.eclipse.jetty:jetty-server : 8.2.0.v20160908 -> 9.0.0.M0 -> 9.0.0.M1 -> 9.0.0.M2 -> 9.0.0.M3 -> 9.0.0.M4 -> 9.0.0.M5 -> 9.0.0.RC0 -> 9.0.0.RC1 -> 9.0.0.RC2 -> 9.0.0.v20130308 -> 9.0.1.v20130408 -> 9.0.2.v20130417 -> 9.0.3.v20130506 -> 9.0.4.v20130625 -> 9.0.5.v20130815 -> 9.0.6.v20130930 -> 9.0.7.v20131107 -> 9.1.0.M0 -> 9.1.0.RC0 -> 9.1.0.RC1 -> 9.1.0.RC2 -> 9.1.0.v20131115 -> 9.1.1.v20140108 -> 9.1.2.v20140210 -> 9.1.3.v20140225 -> 9.1.4.v20140401 -> 9.1.5.v20140505 -> 9.1.6.v20160112 -> 9.2.0.M0 -> 9.2.0.M1 -> 9.2.0.RC0 -> 9.2.0.v20140526 -> 9.2.1.v20140609 -> 9.2.2.v20140723 -> 9.2.3.v20140905 -> 9.2.4.v20141103 -> 9.2.5.v20141112 -> 9.2.6.v20141205 -> 9.2.7.v20150116 -> 9.2.8.v20150217 -> 9.2.9.v20150224 -> 9.2.10.v20150310 -> 9.2.11.M0 -> 9.2.11.v20150529 -> 9.2.12.M0 -> 9.2.12.v20150709 -> 9.2.13.v20150730 -> 9.2.14.v20151106 -> 9.2.15.v20160210 -> 9.2.16.v20160414 -> 9.2.17.v20160517 -> 9.2.18.v20160721 -> 9.2.19.v20160908 -> 9.2.20.v20161216 -> 9.2.21.v20170120 -> 9.2.22.v20170606 -> 9.2.23.v20171218 -> 9.2.24.v20180105 -> 9.2.25.v20180606 -> 9.2.26.v20180806 -> 9.2.27.v20190403 -> 9.2.28.v20190418 -> 9.2.29.v20191105 -> 9.2.30.v20200428 -> 9.3.0.M0 -> 9.3.0.M1 -> 9.3.0.M2 -> 9.3.0.RC0 -> 9.3.0.RC1 -> 9.3.0.v20150612 -> 9.3.1.v20150714 -> 9.3.2.v20150730 -> 9.3.3.v20150827 -> 9.3.4.RC0 -> 9.3.4.RC1 -> 9.3.4.v20151007 -> 9.3.5.v20151012 -> 9.3.6.v20151106 -> 9.3.7.RC0 -> 9.3.7.RC1 -> 9.3.7.v20160115 -> 9.3.8.RC0 -> 9.3.8.v20160314 -> 9.3.9.M0 -> 9.3.9.M1 -> 9.3.9.v20160517 -> 9.3.10.M0 -> 9.3.10.v20160621 -> 9.3.11.M0 -> 9.3.11.v20160721 -> 9.3.12.v20160915 -> 9.3.13.M0 -> 9.3.13.v20161014 -> 9.3.14.v20161028 -> 9.3.15.v20161220 -> 9.3.16.v20170120 -> 9.3.17.RC0 -> 9.3.17.v20170317 -> 9.3.18.v20170406 -> 9.3.19.v20170502 -> 9.3.20.v20170531 -> 9.3.21.M0 -> 9.3.21.RC0 -> 9.3.21.v20170918 -> 9.3.22.v20171030 -> 9.3.23.v20180228 -> 9.3.24.v20180605 -> 9.3.25.v20180904 -> 9.3.26.v20190403 -> 9.3.27.v20190418 -> 9.3.28.v20191105 -> 9.3.29.v20201019 -> 9.3.30.v20211001 -> 9.4.0.M0 -> 9.4.0.M1 -> 9.4.0.RC0 -> 9.4.0.RC1 -> 9.4.0.RC2 -> 9.4.0.RC3 -> 9.4.0.v20161208 -> 9.4.0.v20180619 -> 9.4.1.v20170120 -> 9.4.1.v20180619 -> 9.4.2.v20170220 -> 9.4.2.v20180619 -> 9.4.3.v20170317 -> 9.4.3.v20180619 -> 9.4.4.v20170414 -> 9.4.4.v20180619 -> 9.4.5.v20170502 -> 9.4.5.v20180619 -> 9.4.6.v20170531 -> 9.4.6.v20180619 -> 9.4.7.RC0 -> 9.4.7.v20170914 -> 9.4.7.v20180619 -> 9.4.8.v20171121 -> 9.4.8.v20180619 -> 9.4.9.v20180320 -> 9.4.10.RC0 -> 9.4.10.RC1 -> 9.4.10.v20180503 -> 9.4.11.v20180605 -> 9.4.12.RC0 -> 9.4.12.RC1 -> 9.4.12.RC2 -> 9.4.12.v20180830 -> 9.4.13.v20181111 -> 9.4.14.v20181114 -> 9.4.15.v20190215 -> 9.4.16.v20190411 -> 9.4.17.v20190418 -> 9.4.18.v20190429 -> 9.4.19.v20190610 -> 9.4.20.v20190813 -> 9.4.21.v20190926 -> 9.4.22.v20191022 -> 9.4.23.v20191118 -> 9.4.24.v20191120 -> 9.4.25.v20191220 -> 9.4.26.v20200117 -> 9.4.27.v20200227 -> 9.4.28.v20200408 -> 9.4.29.v20200521 -> 9.4.30.v20200611 -> 9.4.31.v20200723 -> 9.4.32.v20200930 -> 9.4.33.v20201020 -> 9.4.34.v20201102 -> 9.4.35.v20201120 -> 9.4.36.v20210114 -> 9.4.37.v20210219 -> 9.4.38.v20210224 -> 9.4.39.v20210325 -> 9.4.40.v20210413 -> 9.4.41.v20210516 -> 9.4.42.v20210604 -> 9.4.43.v20210629 -> 9.4.44.v20210927 -> 9.4.45.v20220203 -> 9.4.46.v20220331 -> 9.4.47.v20220610 -> 9.4.48.v20220622 -> 10.0.0-alpha0 -> 10.0.0.alpha1 -> 10.0.0.alpha2 -> 10.0.0.beta0 -> 10.0.0.beta1 -> 10.0.0.beta2 -> 10.0.0.beta3 -> 10.0.0 -> 10.0.1 -> 10.0.2 -> 10.0.3 -> 10.0.4 -> 10.0.5 -> 10.0.6 -> 10.0.7 -> 10.0.8 -> 10.0.9 -> 10.0.10 -> 10.0.11 -> 11.0.0-alpha0 -> 11.0.0.beta1 -> 11.0.0.beta2 -> 11.0.0.beta3 -> 11.0.0 -> 11.0.1 -> 11.0.2 -> 11.0.3 -> 11.0.4 -> 11.0.5 -> 11.0.6 -> 11.0.7 -> 11.0.8 -> 11.0.9 -> 11.0.10 -> 11.0.11
  • Version is hard-coded in a separate line, then interpolated. mill
• org.flywaydb:flyway-core : 8.5.13 -> 9.0.0 -> 9.0.1 -> 9.0.2 -> 9.0.3 -> 9.0.4
  • This one is fine with Steward. It was stuck for a long time because they required you to add a NOTICE-level logging. mill.
• org.jgrapht:jgrapht-core : 1.4.0 -> 1.5.0 -> 1.5.1
  • I guess Scala Steward made a PR that was closed a while ago. This is OK mill code. mill scala steward pr
• org.scala-js:scalajs-js-envs_2.13 : 0.6.33 -> 1.0.0 -> 1.0.1 -> 1.1.0 -> 1.1.1 -> 1.2.0 -> 1.2.1 -> 1.3.0
  • That's fine, it's stuck on 0.6 by choice. mill
• org.scala-js:scalajs-sbt-test-adapter_2.13 : 0.6.33 -> 1.0.0 -> 1.0.1 -> 1.1.0 -> 1.1.1 -> 1.2.0 -> 1.3.0 -> 1.3.1 -> 1.4.0 -> 1.5.0 -> 1.5.1 -> 1.6.0 -> 1.7.0 -> 1.7.1 -> 1.8.0 -> 1.9.0 -> 1.10.0 -> 1.10.1
  • Same as above, no problem
• org.scoverage:scalac-scoverage-plugin_2.13.8 : 1.4.11 -> 2.0.0 -> 2.0.1
  • more complex migration, work is under way. mill
• org.testng:testng : 7.5 -> 7.6.0 -> 7.6.1
  • not upgrading so mill keeps it supported on jdk8 mill pr

I will edit the comment and note if any of the current versions are especially old (i.e. if scala steward should have updated them by now).

[james-s-w-clark]

@exoego I think from all of the above, the only valid cases of Scala Steward missing updates are:

• scalafmt-dynamic
• bloop-config
  • raised a pr to mill to update this manualy

[daddykotex]

it previously opened: https://github.com/com-lihaoyi/mill/pull/1741/files

so it has worked at some point (edit: is:pr is:closed author:scala-steward scalafmt )

[james-s-w-clark]

The problem was with dependency configuration, not Scala Steward itself. Steward could not find a heuristic to do an upgrade with the dependencies we gave it.

If anyone finds themselves with this issue, you can have a play with Scala Steward and try to figure out how you can tweak your dependencies so changes get picked up properly.

• clone Scala Steward
• run sbt command testOnly org.scalasteward.core.edit.UpdateHeuristicTest
• add code like the following to the test:

test("check".only) {
    val original =
      """|object Circe {
         |    private val version = "0.14.1"
         |
         |    val core    = ivy"io.circe::circe-core:$version"
         |  }
         |""".stripMargin
    val expected =
      """|object Circe {
         |    private val version = "0.14.2"
         |
         |    val core    = ivy"io.circe::circe-core:$version"
         |  }
         |""".stripMargin
    val update = ("io.circe".g % Nel.of("circe-core".a) % "0.14.1" %> "0.14.2").group
    assertEquals(update.replaceVersionIn(original), Some(expected) -> UpdateHeuristic.original.name)
  }

This does not find a heuristic to do upgrades with - the test fails. Steward PRs would not be raised for these dependencies.

test("check".only) {
    val original =
      """|object Circe {
         |    private val circeVersion = "0.14.1"
         |
         |    val core    = ivy"io.circe::circe-core:$circeVersion"
         |  }
         |""".stripMargin
    val expected =
      """|object Circe {
         |    private val circeVersion = "0.14.2"
         |
         |    val core    = ivy"io.circe::circe-core:$circeVersion"
         |  }
         |""".stripMargin
    val update = ("io.circe".g % Nel.of("circe-core".a) % "0.14.1" %> "0.14.2").group
    assertEquals(update.replaceVersionIn(original), Some(expected) -> UpdateHeuristic.original.name)
  }

This finds the original heuristic to do upgrades with, and the test passes. Steward would raise PRs for dependencies in this form. Basically, our problem was version needs to be circeVersion for example.

---

I don't see information/examples about heuristics in the docs. I would have found this helpful - it is a little bit odd, right? Do we want a PR for some documentation on this?

[lefou]

One reason for missing updates could be the fact, that the Mill plugin of Scala Steward currently doesn't not support all Mill versions. You can find a more in-depth explanation in issue

• #2704